2

Submatrices and Partitioned Matrices

Two very important (and closely related) concepts are introduced in this chapter:
that of a submatrix and that of a partitioned matrix. These concepts arise very
naturally in statistics (especially in multivariate analysis and linear models) and in
many other disciplines that involve probabilistic ideas. And results on submatrices
and partitioned matrices, which can be found in Chapters 8, 9, 13, and 14 (and
other of the subsequent chapters), have proved to be very useful. In particular,
such results are almost indispensable in work involving the multivariate normal
distributionrefer, for example, to Searle (1971, sec. 2.4f).

2.1 Some Terminology and Basic Results

A submatrix of a matrix A is a matrix that can be obtained by striking out rows
and/or columns of A. For example, if we strike out the second row of the matrix
0
1
2 4 3 6
@ 1 5 7 9A ;

1 0 2 2
we obtain the 2  4 submatrix


2 4 3 6
:

1 0 2 2

Alternatively, if we strike out the first

submatrix
0
4
@5
0

and third columns, we obtain the 3  2

1
6
9A I
2

or, if we strike out the second row and the first and third columns, we obtain the
2  2 submatrix

14

2 Submatrices and Partitioned Matrices


4 6
:
0 2
Note that any matrix is a submatrix of itself; it is the submatrix obtained by striking
out zero rows and zero columns.
Submatrices of a row or column vector, that is, of a matrix having one row or
column, are themselves row or column vectors and are customarily referred to as
subvectors.
Let A represent an r  s submatrix of an m  n matrix A obtained by striking
out the i1 ; : : : ; im
r th rows and j1 ; : : : ; jn
s th columns (of A), and let B represent
the s  r submatrix of A0 obtained by striking out the j1 ; : : : ; jn
s th rows and
i1 ; : : : ; im
r th columns (of A0 ). Then,
B D A0 ;

(1.1)

as is easily verified.
A submatrix of an n  n matrix is called a principal submatrix if it can be
obtained by striking out the same rows as columns (so that the i th row is struck
out whenever the i th column is struck out, and vice versa). The r  r (principal)
submatrix of an n  n matrix obtained by striking out its last n
r rows and
columns is referred to as a leading principal submatrix (r D 1; : : : ; n). A principal
submatrix of a symmetric matrix is symmetric, a principal submatrix of a diagonal
matrix is diagonal, and a principal submatrix of an upper or lower triangular matrix
is respectively upper or lower triangular, as is easily verified.
A matrix can be divided or partitioned into submatrices by drawing horizontal
or vertical lines between various of its rows or columns, in which case the matrix
is called a partitioned matrix and the submatrices are sometimes referred to as
blocks (as in blocks of elements). For example,
1
0
1
0
1
0
2 4 3 6
2 4 3 6
2 4 3 6
@ 1 5 7 9A ;
@ 1 5 7 9A
@ 1 5 7 9A ;

1 0 2 2

1 0 2 2

1 0 2 2
are various partitionings of the same matrix.
In effect, a partitioned m  n matrix is an m  n matrix A D faij g that has
been reexpressed in the general form
1
0
A11 A12 : : : A1c
BA21 A22 : : : A2c C
C
B
B ::
::
:: C :
@ :
:
: A
Ar1 Ar2 : : : Arc
Here, Aij is an mi nj matrix (i D 1; : : : ; r; j D 1; : : : ; c), where m1 , : : : ; mr and
n1 , : : : ; nc are positive integers such that m1 C  Cmr D m and n1 C  Cnc D n.
Or, more explicitly,

2.1 Some Terminology and Basic Results

am1 CCmi
1 C1; n1 CCnj
1 C1 : : :

B ::
Aij D @ :
:::
am1 CCmi ; n1 CCnj
1 C1

15

am1 CCmi
1 C1; n1 CCnj

C
::
A:
:
am1 CCmi ; n1 CCnj

(When i D 1 or j D 1, interpret the degenerate sum m1 C  Cmi
1 or n1 C  C

nj
1 as zero.) Thus, a partitioned matrix can be regarded as an array or matrix
of matrices.
Note that a matrix that has been divided by staggered lines, for example,
1
0
2 4 3 6
@ 1 5 7 9A ;

1 0 2 2
does not satisfy our definition of a partitioned matrix. Thus, if a matrix, say
0
1
A11 A12 : : : A1c
BA21 A22 : : : A2c C
B
C
B ::
::
:: C ;
@ :
:
: A
Ar1 Ar2 : : : Arc
is introduced as a partitioned matrix, there is an implicit assumption that each of
the submatrices Ai1 ; Ai2 ; : : : ; Aic in the i th row of submatrices contains the
same number of rows (i D 1; 2; : : : ; r) and similarly that each of the submatrices
A1j ; A2j ; : : : ; Arj in the j th column of submatrices contains the same number
of columns.
It is customary to identify each of the blocks in a partitioned matrix by referring
to the row of blocks and the column of blocks in which it appears. Thus, the
submatrix Aij is referred to as the ij th block of the partitioned matrix
0
1
A11 A12 : : : A1c
BA21 A22 : : : A2c C
B
C
B ::
::
:: C :
@ :
:
: A
Ar1 Ar2 : : : Arc
In the case of a partitioned m  n matrix A of the form
0
1
A11 A12 : : : A1r
BA21 A22 : : : A2r C
B
C
ADB :
::
:: C
::
@ ::
:
:
: A

(1.2)

Ar1 Ar2 : : : Arr

(for which the number of rows of blocks equals the number of columns of blocks),
the ij th block Aij of A is called a diagonal block if j D i and an off-diagonal
block if j i . If all of the off-diagonal blocks of A are null matrices, that is, if

16

2 Submatrices and Partitioned Matrices

0
A11 0 : : :
B 0 A22
B
ADB :
::
@ ::
:
0

1
0
0 C
C
C;
A
Arr

then A is called a block-diagonal matrix, and sometimes diag.A11 ; A22 ; : : : ; Arr /

is written for A. If Aij D 0 for j < i D 1; : : : ; r, that is, if
0
1
A11 A12 : : : A1r
B 0 A22 : : : A2r C
B
C
ADB :
:: C ;
::
@ ::
:
: A
0

Arr

then A is called an upper block-triangular matrix. Similarly, if Aij D 0 for j >

i D 1; : : : ; r, that is, if
0
1
A11 0 : : : 0
BA21 A22
0 C
B
C
ADB :
C;
:
::
::
@ ::
A
:
Ar1 Ar2

Arr

then A is called a lower block-triangular matrix. To indicate that A is either upper

or lower block-triangular (without being more specific), A is referred to simply as
block-triangular.
Note that a partitioned m  n matrix A of the form (1.2) is block-diagonal if
and only if it is both upper block-triangular and lower block-triangular. Note also
that, if m D n D r (in which case each block of A consists of a single element),
saying that A is block diagonal or upper or lower block triangular is equivalent to
saying that A is diagonal or upper or lower triangular.
Partitioned matrices having one row or one column are customarily referred to
as partitioned (row or column) vectors. Thus, a partitioned m-dimensional column
vector is an m  1 vector a D fa t g that has been reexpressed in the general form
0 1
a1
Ba2 C
B C
B :: C :
@:A
ar
Here, ai is an mi  1 vector with elements am1 CCmi
1 C1 ; : : : ; am1 CCmi
1 Cmi ,
respectively (i D 1; : : : ; r), where m1 ; : : : ; mr are positive integers such that
m1 C    C mr D m. Similarly, a partitioned m-dimensional row vector is a 1  m
vector a0 D fa t g that has been reexpressed in the general form .a01 ; a02 ; : : : ; a0r /.

2.2 Scalar Multiples, Transposes, Sums, and Products of Partitioned Matrices

17

2.2 Scalar Multiples, Transposes, Sums, and Products of

Partitioned Matrices
Let

0
1
A11 A12 : : : A1c
BA21 A22 : : : A2c C
B
C
ADB :
::
:: C
@ ::
:
: A
Ar1 Ar2 : : : Arc

represent a partitioned mn matrix whose ij th block Aij is of dimensions mi nj .

Clearly, for any scalar k,
1
0
kA11 kA12 : : : kA1c
BkA21 kA22 : : : kA2c C
C
B
(2.1)
kA D B :
::
:: C :
@ ::
:
: A
kAr1 kAr2 : : : kArc
In particular,

1

A11
A12 : : :
A1c
B
A21
A22 : : :
A2c C
B
C

A D B :
::
:: C :
@ ::
:
: A

Ar1
Ar2 : : :
Arc

Further, it is a simple exercise to show that

0 0
1
A11 A021 : : : A0r1
BA0 A0 : : : A0 C
22
r2 C
B 12
A0 D B :
::
:: C I
:
@ :
:
: A
A01c A02c : : : A0rc

(2.2)

(2.3)

that is, A0 can be expressed as a partitioned matrix, comprising c rows and r

columns of blocks, the ij th of which is the transpose Aj0 i of the j i th block Aj i of
A.
Now, let
0
1
B11 B12 : : : B1v
BB21 B22 : : : B2v C
B
C
BDB :
::
:: C
@ ::
:
: A
Bu1 Bu2 : : : Buv
represent a partitioned p  q matrix whose ij th block Bij is of dimensions pi  qj .
The matrices A and B are conformal (for addition) provided that p D m and
q D n. If u D r, v D c, pi D mi (i D 1; : : : ; r), and qj D nj (j D 1; : : : ; c),
that is, if (besides A and B being conformal for addition) the rows and columns of
B are partitioned in the same way as those of A, then

18

2 Submatrices and Partitioned Matrices

A11 C B11 A12 C B12 : : : A1c

BA21 C B21 A22 C B22 : : : A2c
B
ACB DB
::
::
@
:
:

1
C B1c
C B2c C
C
C;
::
A
:

(2.4)

Ar1 C Br1 Ar2 C Br2 : : : Arc C Brc

and the partitioning of A and B is said to be conformal (for addition). This result
and terminology extend in an obvious way to the addition of any finite number of
partitioned matrices.
If A and B are conformal (for addition), then
0
1
A11
B11 A12
B12 : : : A1c
B1c
BA21
B21 A22
B22 : : : A2c
B2c C
B
C
(2.5)
A
B DB
C;
::
::
::
@
A
:
:
:
Ar1
Br1 Ar2
Br2 : : : Arc
Brc
as is evident from results (2.4) and (2.2).
The matrix product AB is defined provided that n D p. If c D u and nk D pk
(k D 1; : : : ; c) [in which case all of the products Aik Bkj (i D 1; : : : ; r; j D
1; : : : ; v; k D 1; : : : ; c), as well as the product AB, exist], then
0
1
F11 F12 : : : F1v
BF21 F22 : : : F2v C
B
C
(2.6)
AB D B :
::
:: C ;
@ ::
:
: A
Fr1 Fr2 : : : Frv
P
where Fij D ckD1 Aik Bkj D Ai1 B1j C Ai2 B2j C    C Aic Bcj , and the partitioning of A and B is said to be conformal (for the premultiplication of B by
A).
In the special case where r D c D u D v D 2, that is, where






B11 B12
A11 A12
and B D
;
AD
A21 A22
B21 B22
result (2.6) simplifies to



A11 B11 C A12 B21 A11 B12 C A12 B22
:
AB D
A21 B11 C A22 B21 A21 B12 C A22 B22

(2.7)

If A D .A1 ; A2 ; : : : ; Ac / is an m  n matrix that has been partitioned only by

columns (for emphasis, we sometimes insert commas between the submatrices of
such a partitioned matrix), then
0 01
A1
BA0 C
B 2C
(2.8)
A0 D B : C ;
@ :: A
A0c

2.3 Some Results on the Product of a Matrix and a Column Vector

and further if

19

0 1
B1
BB2 C
B C
BDB : C
@ :: A
Bc

is an n  q partitioned matrix that has been partitioned only by rows (in a way that
is conformal for its premultiplication by A), then
AB D

c
X

Ak Bk D A1 B1 C A2 B2 C    C Ac Bc :

(2.9)

kD1

Similarly, if

1
A1
BA 2 C
B C
ADB : C
@ :: A
Ar

is an m  n matrix that has been partitioned only by rows, then

A0 D .A01 ; A02 ; : : : ; A0r /;

(2.10)

and further if B D .B1 ; B2 ; : : : ; Bv / is an n  q matrix that has been partitioned

only by columns, then
0
1
A1 B1 A1 B2 : : : A1 Bv
BA 2 B 1 A 2 B 2 : : : A 2 B v C
B
C
(2.11)
AB D B :
::
:: C :
@ ::
:
: A
Ar B1 Ar B2 : : : Ar Bv

2.3 Some Results on the Product of a Matrix and a Column

Vector
Let A represent an m  n matrix and x an n  1 vector. Writing A as A D
.a1 ; a2 ; : : : ; an /, where a1 ; a2 ; : : : ; an are the columns of A, and x as x D
.x1 ; x2 ; : : : ; xn /0 , where x1 ; x2 ; : : : ; xn are the elements of x, we find, as a special
case of result (2.9), that
Ax D

n
X

xk ak D x1 a1 C x2 a2 C    C xn an :

(3.1)

kD1

Thus, the effect of postmultiplying a matrix by a column vector is to form a

linear combination of the columns of the matrix, where the coefficients in the
linear combination are the elements of the column vector. Similarly, the effect of

20

2 Submatrices and Partitioned Matrices

premultiplying a matrix by a row vector is to form a linear combination of the rows

of the matrix, where the coefficients in the linear combination are the elements of
the row vector.
Representation (3.1) is helpful in establishing the elementary results expressed
in the following two lemmas.
Lemma 2.3.1. For any column vector y and nonnull column vector x, there
exists a matrix A such that y D Ax.
Proof. Since x is nonnull, one of its elements, say xj , is nonzero. Take A to
be the matrix whose j th column is .1=xj /y and whose other columns are null.
Then, y D Ax, as is evident from result (3.1).
Q.E.D.
Lemma 2.3.2. For any two m  n matrices A and B, A D B if and only if
Ax D Bx for every n  1 vector x.
Proof. It is obvious that, if A D B, then Ax D Bx for every vector x.
To prove the converse, suppose that Ax D Bx for every x. Taking x to be the
n  1 vector whose i th element is 1 and whose other elements are 0, and letting ai
and bi represent the i th columns of A and B, respectively, it is clear from result
(3.1) that
ai D Ax D Bx D bi
(i D 1; : : : ; n). We conclude that A D B.
Q.E.D.
Note that Lemma 2.3.2 implies, in particular, that A D 0 if and only if Ax D 0
for every x.

2.4 Expansion of a Matrix in Terms of Its Rows, Columns, or

Elements
An m  n matrix A D faij g can be expanded in terms of its rows, columns, or
elements by making use of formula (2.9). Denote the i th row of A by ri0 and the i th
column of Im by ei (i D 1; 2; : : : ; m). Then, writing Im as Im D .e1 ; e2 ; : : : ; em /
and A as
0 01
r1
B r0 C
B 2C
ADB : C
@ :: A
0
rm

and applying formula (2.9) to the product Im A, we obtain the expansion

AD

m
X

0
ei ri0 D e1 r10 C e2 r20 C    C em rm
:

(4.1)

iD1

Similarly, denote the j th column of A by aj and the j th row of In by uj0

(j D 1; 2; : : : ; n). Then, writing A as A D .a1 ; a2 ; : : : ; an / and In as

Exercises

21

1
u01
Bu0 C
B 2C
In D B : C
@ :: A
u0n
and applying formula (2.9) to the product AIn , we obtain the alternative expansion
AD

n
X

aj uj0 D a1 u01 C a2 u02 C    C an u0n :

(4.2)

j D1

Moreover, the application of formula (3.1) to the product Im aj gives the expansion
m
X
aij ei :
aj D
iD1

Upon substituting this expansion into expansion (4.2), we obtain the further expansion
m X
n
X
AD
aij Uij ;
(4.3)
iD1 j D1

where Uij D ei uj0 is an m  n matrix whose ij th element equals 1 and whose

remaining mn
1 elements equal 0. In the special case where n D m (i.e., where
A is square), uj D ej and hence Uij D ei ej0 , and in the further special case where
A D Im , expansion (4.3) reduces to
Im D

m
X

ei e0i :

(4.4)

iD1

Note that, as a consequence of result (4.3), we have that

!
n
n
m X
m X
X
X
0
0
0
ei Auj D ei
aks ek us uj D
aks e0i ek u0s uj ;
kD1 sD1

kD1 sD1

which (since e0i ek equals 1, if k D i , and equals 0, if k i , and since u0s uj equals
1, if s D j , and equals 0, if s j ) simplifies to
e0i Auj D aij :

Exercises
Section 2.1

(4.5)

22

2 Submatrices and Partitioned Matrices

1. Verify result (1.1).

2. Verify (a) that a principal submatrix of a symmetric matrix is symmetric, (b)
that a principal submatrix of a diagonal matrix is diagonal, and (c) that a
principal submatrix of an upper triangular matrix is upper triangular.
3. Let

0
1
A11 A12 : : : A1r
B 0 A22 : : : A2r C
B
C
ADB :
:: C
:
:
:
@ :
:
: A
0
0 : : : Arr

represent an n  n upper block-triangular matrix whose ij th block Aij is of

dimensions ni  nj (j  i D 1; : : : ; r). Show that A is upper triangular if
and only if each of its diagonal blocks A11 ; A22 ; : : : ; Arr is upper triangular.
Section 2.2
4. Verify results (2.3) and (2.6).

http://www.springer.com/978-0-387-78356-7