MTH6140 Linear Algebra II

Notes 8 16th December 2010

8 Symmetric matrices
We come to one of the most important topics of the course. In simple terms, any real
symmetric matrix is diagonalisable. But there is more to be said!

8.1 Orthogonal projections and orthogonal decompositions

We say that two vectors v, w in an inner product space are orthogonal if v · w = 0.

Definition 8.1 Let V be an inner product space, and U a subspace of V . The ortho-
gonal complement of U is the set of all vectors which are orthogonal to everything in
U:
U ⊥ = {w ∈ V : w · u = 0 for all u ∈ U}.
The symbol U ⊥ is often pronounced ‘U perp’, where ‘perp’ is short for perpendicular,
which is another word for orthogonal.

Proposition 8.1 If V is an inner product space and U a subspace of V , with dim(V ) =

n and dim(U) = r, then U ⊥ is a subspace of V , and dim(U ⊥ ) = n − r. Moreover,
V = U ⊕U ⊥ .

Proof Proving that U ⊥ is a subspace is straightforward from the properties of the

inner product. If w1 , w2 ∈ U ⊥ , then w1 ·u = w2 ·u = 0 for all u ∈ U, so (w1 +w2 )·u = 0
for all u ∈ U, whence w1 + w2 ∈ U ⊥ . The argument for scalar multiples is similar.
Now choose a basis for U and extend it to a basis for V . Then apply the Gram–
Schmidt process to this basis (starting with the elements of the basis for U), to obtain
an orthonormal basis (v1 , . . . , vn ). Since the process only modifies vectors by adding
multiples of earlier vectors, the first r vectors in the resulting basis will form an or-
thonormal basis for U. The last n − r vectors will be orthogonal to U, and so lie in U ⊥ ;
and they are clearly linearly independent. Now suppose that w ∈ U ⊥ and w = ∑i ci vi ,

1
where (v1 , . . . , vn ) is the orthonormal basis we constructed. Then ci = w · vi = 0 for
i = 1, . . . , r; so w is a linear combination of the last n − r basis vectors, which thus
form a basis of U ⊥ . Hence dim(U ⊥ ) = n − r, as required.
Now the last statement of the proposition follows from the proof, since we have a
basis for V which is a disjoint union of bases for U and U ⊥ .
Recall the connection between direct sum decompositions and projections. If we
have projections P1 , . . . , Pr whose sum is the identity and which satisfy Pi Pj = O for
i 6= j, then the space V is the direct sum of their images. This can be refined in an
inner product space as follows.
Definition 8.2 Let V be an inner product space. A linear map P : V → V is an ortho-
gonal projection if
(a) P is a projection, that is, P2 = P;
(b) P is self-adjoint, that is, P∗ = P (where P∗ (v) · w = v · P(w) for all v, w ∈ V ).
Proposition 8.2 If P is an orthogonal projection, then Ker(P) = Im(P)⊥ .
Proof We know, from Proposition 5.1, that V = Ker(P) ⊕ Im(P); we only have to
show that these two subspaces are orthogonal. So take v ∈ Ker(P), so that P(v) = 0,
and w ∈ Im(P), so that w = P(u) for some u ∈ V . Then
v · w = v · P(u) = P∗ (v) · u = P(v) · u = 0,
as required.
Proposition 8.3 Let P1 , . . . , Pr be orthogonal projections on an inner product space V
satisfying P1 + · · · + Pr = I and Pi Pj = O for i 6= j. Let Ui = Im(Pi ) for i = 1, . . . , r.
Then
V = U1 ⊕U2 ⊕ · · · ⊕Ur ,
and if ui ∈ Ui and u j ∈ U j , then ui and u j are orthogonal.
Proof The fact that V is the direct sum of the images of the Pi follows from Proposi-
tion 5.2. We only have to prove the last part. So take ui and u j as in the Proposition,
say ui = Pi v and u j = Pj w. Then
ui · u j = Pi v · Pj w = Pi∗ v · Pj w = v · Pi Pj w = 0,
where the second equality holds since Pi is self-adjoint and the third is the definition
of the adjoint.
A direct sum decomposition satisfying the conditions of Proposition 8.3 is called
an orthogonal decomposition of V .
Conversely, if we are given an orthogonal decomposition of V , then we can find
orthogonal projections satisfying the hypotheses of Proposition 8.3.

2
8.2 The spectral theorem
The main theorem can be stated in two different ways. I emphasise that these two
theorems are the same! Either of them can be referred to as the spectral theorem.

Theorem 8.4 If T is a self-adjoint linear map on a real inner product space V , then
the eigenspaces of T form an orthogonal decomposition of V . Hence there is an or-
thonormal basis of V consisting of eigenvectors of T . Moreover, there exist orthogonal
projections P1 , . . . , Pr satisfying P1 + · · · + Pr = I and Pi Pj = O for i 6= j, such that

T = λ1 P1 + · · · + λr Pr ,

where λ1 , . . . , λr are the distinct eigenvalues of T .

Theorem 8.5 Let A be a real symmetric matrix. Then there exists an orthogonal
matrix P such that P−1 AP (= P> AP) is diagonal.

Proof The second theorem follows from the first, since the transition matrix from
one orthonormal basis to another is an orthogonal matrix. So we concentrate on the
first theorem. It suffices to find an orthonormal basis of eigenvectors, since all the
rest follows from our remarks about projections, together with what we already know
about diagonalisable maps.
The proof will be by induction on n = dim(V ). There is nothing to do if n = 1. So
we assume that the theorem holds for (n − 1)-dimensional spaces.
The first job is to show that T has an eigenvector.
Choose an orthonormal basis; then T is represented by a real symmetric matrix A.
Its characteristic polynomial has a root λ over the complex numbers, by the so-called
“fundamental theorem of algebra”. We temporarily enlarge the field from R to C.
Now we can find a column vector v ∈ Cn such that Av = λ v. Taking the complex
conjugate, remembering that A is real, we have Av = λ v.
If v = [ z1 z2 · · · zn ]> , then we have

λ (|z1 |2 + |z2 |2 + · · · + |zn |2 ) = λ v> v

= (Av)> v
= v> Av
= v> (λ v)
= λ (|z1 |2 + |z2 |2 + · · · + |zn |2 ),

so (λ − λ )(|z1 |2 + |z2 |2 + · · · + |zn |2 ) = 0. Since v is not the zero vector, the second
factor is positive, so we must have λ = λ , that is, λ is real.

3
 Now since T has a real eigenvalue, we can choose a real eigenvector v, and (mul-
tiplying by a scalar if necessary) we can assume that |v| = 1.
Let U be the subspace v⊥ = {u ∈ V : v·u = 0}. This is a subspace of V of dimension
n − 1. We claim that T : U → U. For take u ∈ U. Then

v · T (u) = T ∗ (v) · u = T (v) · u = λ v · u = 0,

where we use the fact that T is self-adjoint. So T (u) ∈ U.

So T is a self-adjoint linear map on the (n − 1)-dimensional inner product space
U. By the inductive hypothesis, U has an orthonormal basis consisting of eigenvectors
of T . They are all orthogonal to the unit vector v; so, adding v to the basis, we get an
orthonormal basis for V , and we are done.

Corollary 8.6 If T is self-adjoint, then eigenvectors of T corresponding to distinct

eigenvalues are orthogonal.

Proof This follows from the theorem, but is easily proved directly. If T (v) = λ v and
T (w) = µw, then

λ v · w = T (v) · w = T ∗ (v) · w = v · T (w) = µv · w,

so, if λ 6= µ, then v · w = 0.

Remark: The above result, combined with the Remark prior to Theorem 7.1, tells
us that the eigenvectors of a self-adjoint map T corresponding to distinct eigenvalues
are linearly independent. But we already knew this, by Lemma 5.8!

Example 8.1 Let  

10 2 2
A = 2 13
 4 .
2 4 13
The characteristic polynomial of A is
 
 x − 10 −2 −2 
−4  = (x − 9)2 (x − 18),

 −2 x − 13

 −2 −4 x − 13 
so the eigenvalues are 9 and 18.
For eigenvalue 18 the eigenvectors satisfy
    
10 2 2 x 18x
 2 13 4   y  =  18y  ,
2 4 13 z 18z

4
so the eigenvectors are multiples of [ 1 2 2 ]> . Normalising, we can choose a unit
eigenvector [ 13 23 32 ]> .
For the eigenvalue 9, the eigenvectors satisfy
    
10 2 2 x 9x
 2 13 4   y  =  9y  ,
2 4 13 z 9z

that is, x + 2y + 2z = 0. (This condition says precisely that the eigenvectors are
orthogonal to the eigenvector for λ = 18, as we know.) Thus the eigenspace is 2-
dimensional. We need to choose an orthonormal basis for it.√ This can√be done
in many√ different
√ ways: for
√ example, we could choose [ 0 1/ 2 −1/ 2 ]> and
[ −4/3 2 1/3 2 1/3 2 ]> . Then we have an orthonormal basis of eigenvectors.
We conclude that, if  √ 
1/3 0√ −4/3√ 2
P =  2/3 1/ √2 1/3√2  ,
2/3 −1/ 2 1/3 2
then P is orthogonal, and  
18 0 0
>
P AP = 0
 9 0.
0 0 9
You might like to check that the orthogonal matrix in Example 7.2 (in the previous
chapter) also diagonalises A.