Chapter 6

Matrices
6.1

Introduction

We have made frequent use of matrices in Rnm in the previous chapters: to solve
systems of linear equations, as an example of vector spaces with the operations of
addition of matrices and multiplication by scalars, or as a way to represent linear
transformations from Rm to Rn . In this chapter we will carry out a detailed study of
the set of matrices Rnm , and also of Cnm , and we start by recalling the following
simple notions.
A matrix of size or dimension m n is a rectangular array of numbers (real or
complex), with m rows and n columns.
Transpose of a matrix: Any n m matrix A has a unique transpose matrix At
of size m n; the first row of A is the first column of At ; the second row of A
is the second column of At , and so on. In symbolic terms, if A = (aij ) then
At = (aji ). Note that the transpose of a matrix has no analogous operation
in real numbers, unlike other matrix operations (addition of matrices, scalar
multiplication, multiplication of matrices).
1

 Rank: The rank of a matrix A is the number of non-zero rows in any row-echelon
form (REF) of this matrix. It is sometimes called the row rank. The column rank
is the same concept applied to the columns. However, the row and column
ranks are always the same.
Symmetric matrix: The matrix A is symmetric if A = At . Obviously, symmetric
matrices are square.

Apart from the two operations that provide Rmn and Cmn with the structure of
a vector space, i.e., addition of matrices and multiplication by scalars, the following
operation between elements of Rmn and Cmn , the matrix multiplication, is also
well-known.
Given two matrices A and B of dimensions m n and n p respectively, (where
the number n of columns of A equals the number of rows of B) then the product
C = A B exists and has dimension m p (but B A may or may not exist). This
product is usually written as C = AB without the multiplication sign. If C = (cij )
then it is defined by:
cij =

n
X

aik bkj = ai1 b1j + ai2 b2j + + ain bnj .

k=1

That is, the entry cij of the product is the vector dot product of the i-th row, as a vector,
from A and j-th column, as a vector, from B.

Properties of the matrix multiplication:

1. The matrix multiplication is not commutative in general, so usually AB 6= BA.
2. A(BC) = (AB)C, i.e, when there are two or more multiplications, it does not
matter which pair is multiplied first.
3. A(B + C) = AB + AC (left distributive law).
2

4. (B + C)A = BA + CA (right distributive law).

5. (AB) = (A)B = A(B) (scalar multiplication commutes with the matrix
product).
6. AO1 = O3 , and O2 A = O3 for zero matrices O1 , O2 , O3 of appropriate sizes.
7. Im A = A;

AIn = A for every matrix A of dimension m n, where Im , In are

the m m and n n identity matrices.

8. (AB)t = Bt At (The transpose of a product of two matrices is equal to the product
of the individual matrix transposes, but in reverse order. Hence (AB)t 6= At Bt
(usually) since matrix multiplication is not commutative).

6.2

Inverse of a matrix

A square matrix A of size n n sometimes has rank n, in which case A is called nonsingular. This means that there will be a leading variable in every row (and column)
of any echelon form of A and that the reduced row-echelon form will be the n n
identity matrix. If A is non-singular then it also has a very special associated matrix
called the inverse matrix.
A square matrix A of size nn that is non-singular (has rank n) has an inverse matrix
of size n n denoted A1 satisfying:
AA1 = In ,

and A1 A = In .

A matrix with an inverse is said to be invertible.

The following statements are well known theorems:

Suppose the square matrix A has an inverse. Then A1 is unique.
3

 Suppose A and B are invertible matrices of size n. Then AB is an invertible

matrix and (AB)1 = B1 A1 .
Suppose A is an invertible matrix. Then A1 is invertible and (A1 )1 = A.
Suppose A is an invertible matrix. Then At is invertible and (At )1 = (A1 )t .
Suppose A is an invertible matrix and is a nonzero scalar. Then A is invertible and (A)1 =

1 1
A .

A is non-singular if and only if A is invertible.

Suppose that A and B are square matrices of size n. The product AB is nonsingular if and only if A and B are both nonsingular.
Finally, consider the following definition. Suppose that A is a square matrix of size n
such that At A = In . Then we say A is orthogonal. Obviously, if A is an orthogonal
matrix of size n, then A is nonsingular, and A1 = A.

6.3

Determinants

Every square matrix A has a determinant, denoted |A| or det(A), which is a (real or
complex) number calculated from the entries in the matrix. The definition and numerical value of a determinant is not given here by a formula but by an algorithm (a process). This definition algorithm is recursive, meaning that the determinant of a matrix
of size n n is defined it terms of the determinants of matrices of size (n 1) (n 1)
and those are defined in terms of determinants of matrices of smaller size and so on,
until the matrices of size 2 2 are reached, where the determinant is found by a formula. First, we need the following definition.
For a square matrix A = (aij ) the cofactor of aij , denoted as Cij is defined by:
Cij = (1)i+j det(Aij ) ,
4

where Aij is the matrix A with row i and column j removed.

Algorithm:
For a 2 2 matrix the determinant is computed by multiplying the two entries
on the main diagonal (falling from left to right) and subtracting from this the
product of the two entries on the other diagonal (rising from left to right). That
is:

A=

det(A) = ad bc .

For the 3 3 matrix A = (aij ) the determinant is computed in terms of the

cofactors Cij by:
det(A) = a11 C11 + a12 C12 + a13 C13 .
That is, multiply each entry in the first row by its cofactor and then add these
values.

The determinant can also be computed by following the same process with any
row or column. For example, the evaluation of det(A) by the second row is:
det(A) = a21 C21 + a22 C22 + a23 C23 . The evaluation of det(A) by the third
column is: det(A) = a13 C13 + a23 C23 + a33 C33 .
For larger size matrices the determinant is defined in a similar, but recursive
fashion. For example, if A = (aij ) is a 4 4 matrix then evaluating by the first
row:
det(A) = a11 C11 + a12 C12 + a13 C13 + a14 C14 .
Each of the cofactors C11 , C12 , C13 , C14 is evaluated using the definition of the
determinant of a 3 3 matrix (hence the recursive property).
5

6.3.1

Properties of determinants

For any square matrices A and B of the same size n n:

1. det(A) = det(At ) (a matrix and its transpose have the same determinant).

2. det(In ) = 1 for any identity matrix In .

3. det(A) = n det(A) where is any scalar.

4. det(AB) = det(A)det(B)

5. det(A) = 0 if, and only if, A is singular (any echelon form has at least one row
of zeros). In particular, det(A) = 0 if A has a row/column of zeros, or if two
rows/columns are the same, or if one is a multiple of the other.

6. det(A) 6= 0 if, and only if, A is non-singular or, equivalently, if A1 exists (an
echelon form of A has no zero rows).

7. If A1 exists then det(A1 ) =

1
.
det(A)

1
adj(A), where adj(A) is the adjoint matrix
det(A)
of A formed by replacing each entry in A by its cofactor (with respect to A), then

8. If A1 exists then A1 =

transposing the matrix.

9. If B is formed from A by adding/subtracting a multiple of one row to/from

another row, then det(B) = det(A) (it is unchanged). A similar result applies
for columns.
6

6.4

The four Fundamental Subspaces induced by a matrix

We have learnt in previous chapters that there are two subspaces associated to the
linear transformation T : U V, the kernel ker(T ) U and the range R(T ) V. We
know as well that dim(U) = dim(ker(T )) + dim(R(T )). Also, if U = Rm and Rn ,
there is a unique n m matrix AT associated to the linear transformation satisfying
that T (u) = A u, for every vector u U.

In this section we are going to study four vector spaces associated to any matrix A
(that might represent a linear transformation from Rm to Rn ); two of them are subspaces of Rm and the other two are subspaces of Rn , and our objective is to identify
them, find their dimensions and a basis for them. Obviously, all these results can be
generalized to the matrices in Cnm .

6.4.1

The null space of a matrix

Given an n m matrix A, its nullspace N(A) is the set of all solutions to the linear
system Ax = 0.
The nullspace of a matrix is a subspace of Rm .
Proof: Clearly it is non-empty, for x = 0 is certainly a solution to Ax = 0.
If x and y are both solutions, then A(x + y) = Ax + Ay = 0 + 0 = 0, so the set is
closed under addition. If is a scalar, then A(x) = (Ax) = 0 = 0. So the set
is also closed under scalar multiplication, and is thus a subspace.

Note that solutions to solutions with other right hand sides, Ax = b, do not form
a subspace (they do not contain the zero element).
7

 The dimension of this vector space is m rank(A) (i.e., the number of columns
in A minus its rank).
If AT is the matrix associated to a linear transformation T : Rm Rn , then
N(AT ) = ker(T ).

6.4.2

The column space of a matrix

In previous chapters we have seen that from the graphical interpretation of systems of
linear equations, there is a natural correspondence between solutions to these systems
and linear combinations of the columns of the coefficient matrix. This idea motivates
the following important definition.
Suppose that A is an n m matrix with columns {A1 , A2 , A3 , . . . , Am }. Then the
column space of A, written C(A), is the subset of Rn containing all linear combinations
of the columns of A: C(A) = Span({A1 , A2 , A3 , . . . , Am }).
The column space of a matrix is a subspace of Rn .
Proof: Clearly it is also non-empty: if we take the linear combination
0 A1 + 0 A2 + + 0 Am = 0 ,
the zero is in C(A). And it is closed under addition and scalar multiplication:
(1 A1 + +m Am )+(1 A1 + +m Am ) = (1 +1 )A1+ +(m +m )Am ,
and
k(1 A1 + + m Am ) = (k1 A1 + + km Am ) ,
all of which are linear combinations of the columns of the matrix. Thus C(A) is
a subspace.

 Theorem: b C(A) the linear system Ax = b is consistent.

To find a basis of the column space, we reduce A to row echelon form to identify
columns that form a linearly independent set: those containing a leading variable in the echelon form. Thus, the dimension of this vector space is equal to
r = rank(A).
If AT is the matrix associated to a linear transformation T : Rm Rn , then
C(AT ) = R(T ).

6.4.3

The row space of a matrix

The row space, C(At ), of an n m matrix A is the subspace of Rm spanned by its

rows. The notation is due to the fact that the row space of a matrix is exactly the
column space of its transpose.
Also, to find a basis of the row space, we reduce A to row echelon form; the nonzero rows form a linearly independent set. Thus, the dimension of this vector
space also equals r = rank(A).
The row space is a subspace of Rm .
The vectors in the nullspace of a matrix A and those in its row space are orthogonal.

6.4.4

The left null space of a matrix

The left nullspace of a matrix is simply the nullspace of its transpose, and it is denoted
by N(At ). Solving At x = 0, we could transpose to get xt A = 0t . So the left nullspace
is the set of vectors that could multiply A on the left to give the zero row.
9

 Since there are n columns in the matrix At , the left nullspace must have dimension n r.
We could find a basis for it by working with At , but there is a better alternative.
Perform Gaussian (or better, Gauss-Jordan) elimination on [A|I] to produce [U|J],
and the last n r rows of J form a basis for the left nullspace of A.
The vectors in the left nullspace of a matrix A and those in its column space are
orthogonal.

As a summary, consider the following theorem.

Theorem. Fundamental Theorem of Linear Algebra: The row and column spaces
have the same dimension, r, which equals the rank of the matrix. Bases for them can
be found among the rows and columns of A itself. The nullspaces have dimensions
n r and m r.
Vector Space

Dimension

Subspace of

N(A), nullspace

m rank(A)

Rm

C(A), column space

rank(A)

Rn

C(A)t , row space, nullspace

rank(A)

Rm

N(At ), left nullspace

n rank(A)

Rn

Example: Consider the following matrix

A= 0

11

10

1 .

11

Let us find the four fundamental subspaces associated to it.

10

1) Nullspace: N(A) = {v R4 : Av = 0}. The condition Av = 0 becomes:

v1
3
5 3
6
0

v2

0
= 0 ,
1 4 1

v
3

6 11 10 11
0
v4
which represents a linear system with three equations and four unknowns v1 ,
v2 , v3 and v4 . If we obtain a row echelon form for the corresponding augmented
matrix, we obtain:

1 0

1
0

1 .

Then the nullspace is given by

N(A) = {v = (v1 , v2 , v3 , v4 ) 0 R4 : v1 =

11
17
v3 v4 , v2 = 4v3 + v4 }
3
3

and a basis for it is

B = {(17, 12, 3, 0) 0 , (11, 3, 0, 3) 0 }.
Notice that from this result we find that the rank of A equals 2.
2) Column space: C(A) = {v R3 : v = a1 A1 + a2 A2 + a3 A3 + a4 A4 , ai R} is
the set of vectors that are linear combinations of the columns of the matrix A. To
find a basis, we reduce the matrix A to row echelon form, and get:

3 5 3
6
3 5 3
6

0 1 4 1 0 1 4 1 .

6 11 10 11
0 0 0
0
The leading variables appear in columns 1 and 2, thus a basis of the column
space is
B = {(3, 0, 6) 0 , (5, 1, 11) 0 }.
11

Again, we see that the rank of A equals 2, and so does the dimension of the
column space.
3) Row space: C(At ) is spanned by two rows of the matrix A, the two ones containing the leading variables in the row echelon form above. Thus C(At ) = {v
R4 : v = b1 (3, 5, 3, 6) 0 + b2 (0, 1, 4, 1) 0 , b1 , b2 R}. Obviously, the dimension of
the row space is also 2.
4) Left nullspace: N(At ) is the nullspace of the transpose of A, or equivalently, the
set of vectors of the form v = (v1 , v2 , v3 ) 0 satisfying

(v1 , v2 , v3 ) 0

11

10

1 = (0, 0, 0, 0).

11

If we write the augmented matrix corresponding to this linear system and reduce it to row echelon form, we get:

0 6 0
3

5
1 11 0

3
4 10 0

6 1 11 0

3

0

0


0

.
0

Thus we have
N(At ) = {v = (v1 , v2 , v3 ) 0 : v1 = 2v3 , v2 = v3 } = {v R3 : v = v3 (2, 1, 1) 0 , v3 R}.
Obviously, the dimension of the left nullspace is 3 2 = 1.
An alternative way to obtain the left nullspace is to write the matrix

[A|I] = 0

11

10

12

6 1
1

11 0

and reduce it to row echelon form:

3 5

[U|J] = 0 1

0 0

0 .

The last row of J, (2, 1, 1) 0 gives us a basis for the left nullspace, consistent with
what we already have.

13