Sheldon Axler - Linear Algebra Done Right - Second Edition

Linear Algebra
Done Right,
Second Edition
Sheldon Axler
Springer
Contents
Preface to the Instructor
ix
Preface to the Student
xiii
Acknowledgments
xv
Chapter 1
Vector Spaces
Complex Numbers . . . . .
Denition of Vector Space .
Properties of Vector Spaces
Subspaces . . . . . . . . . .
Sums and Direct Sums . . .
Exercises . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
2
4
11
13
14
19
Finite-Dimensional Vector Spaces

Span and Linear Independence .
Bases . . . . . . . . . . . . . . . .
Dimension . . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
21
22
27
31
35
.
.
.
.
.
37
38
41
48
53
59
.
.
.
.
.
.
.
.
.
.
.
.
Chapter 2
Chapter 3
Linear Maps
Denitions and Examples .
Null Spaces and Ranges . .
The Matrix of a Linear Map
Invertibility . . . . . . . . .
Exercises . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
v
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
vi
Contents
Chapter 4
Polynomials
Degree . . . . . . . . .
Complex Coefcients
Real Coefcients . . .
Exercises . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
63
64
67
69
73
Eigenvalues and Eigenvectors

Invariant Subspaces . . . . . . . . . . . . . .
Polynomials Applied to Operators . . . . . .
Upper-Triangular Matrices . . . . . . . . . .
Diagonal Matrices . . . . . . . . . . . . . . . .
Invariant Subspaces on Real Vector Spaces
Exercises . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
75
76
80
81
87
91
94
Inner-Product Spaces
Inner Products . . . . . . . . . . . . . . . . . . . . . . .
Norms . . . . . . . . . . . . . . . . . . . . . . . . . . .
Orthonormal Bases . . . . . . . . . . . . . . . . . . . .
Orthogonal Projections and Minimization Problems
Linear Functionals and Adjoints . . . . . . . . . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
97
98
102
106
111
117
122
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
127
128
132
138
144
147
152
158
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Chapter 5
Chapter 6
Chapter 7
Operators on Inner-Product Spaces

Self-Adjoint and Normal Operators . . . . . . . .
The Spectral Theorem . . . . . . . . . . . . . . . .
Normal Operators on Real Inner-Product Spaces
Positive Operators . . . . . . . . . . . . . . . . . .
Isometries . . . . . . . . . . . . . . . . . . . . . . .
Polar and Singular-Value Decompositions . . . .
Exercises . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Chapter 8
Operators on Complex Vector Spaces

163
Generalized Eigenvectors . . . . . . . . . . . . . . . . . . . . . . 164
The Characteristic Polynomial . . . . . . . . . . . . . . . . . . . 168
Decomposition of an Operator . . . . . . . . . . . . . . . . . . . 173
Contents
Square Roots . . . . . . .
The Minimal Polynomial
Jordan Form . . . . . . .
Exercises . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
vii
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
177
179
183
188
Operators on Real Vector Spaces

Eigenvalues of Square Matrices .
Block Upper-Triangular Matrices
The Characteristic Polynomial .
Exercises . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
193
194
195
198
210
.
.
.
.
.
.
213
214
216
222
225
236
244
Chapter 9
Chapter 10
Trace and Determinant

Change of Basis . . . . . . . .
Trace . . . . . . . . . . . . . .
Determinant of an Operator
Determinant of a Matrix . . .
Volume . . . . . . . . . . . . .
Exercises . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Symbol Index
247
Index
249
You are probably about to teach a course that will give students
their second exposure to linear algebra. During their rst brush with
the subject, your students probably worked with Euclidean spaces and
matrices. In contrast, this course will emphasize abstract vector spaces
and linear maps.
The audacious title of this book deserves an explanation. Almost
all linear algebra books use determinants to prove that every linear operator on a nite-dimensional complex vector space has an eigenvalue.
Determinants are difcult, nonintuitive, and often dened without motivation. To prove the theorem about existence of eigenvalues on complex vector spaces, most books must dene determinants, prove that a
linear map is not invertible if and only if its determinant equals 0, and
then dene the characteristic polynomial. This tortuous (torturous?)
path gives students little feeling for why eigenvalues must exist.
In contrast, the simple determinant-free proofs presented here offer more insight. Once determinants have been banished to the end
of the book, a new route opens to the main goal of linear algebra
understanding the structure of linear operators.
This book starts at the beginning of the subject, with no prerequisites other than the usual demand for suitable mathematical maturity.
Even if your students have already seen some of the material in the
rst few chapters, they may be unaccustomed to working exercises of
the type presented here, most of which require an understanding of
proofs.
Vector spaces are dened in Chapter 1, and their basic properties
are developed.
Linear independence, span, basis, and dimension are dened in
Chapter 2, which presents the basic theory of nite-dimensional
vector spaces.
ix
Linear maps are introduced in Chapter 3. The key result here

is that for a linear map T , the dimension of the null space of T
plus the dimension of the range of T equals the dimension of the
domain of T .
The part of the theory of polynomials that will be needed to understand linear operators is presented in Chapter 4. If you take
class time going through the proofs in this chapter (which contains no linear algebra), then you probably will not have time to
cover some important aspects of linear algebra. Your students
will already be familiar with the theorems about polynomials in
this chapter, so you can ask them to read the statements of the
results but not the proofs. The curious students will read some
of the proofs anyway, which is why they are included in the text.
The idea of studying a linear operator by restricting it to small
subspaces leads in Chapter 5 to eigenvectors. The highlight of the
chapter is a simple proof that on complex vector spaces, eigenvalues always exist. This result is then used to show that each linear
operator on a complex vector space has an upper-triangular matrix with respect to some basis. Similar techniques are used to
show that every linear operator on a real vector space has an invariant subspace of dimension 1 or 2. This result is used to prove
that every linear operator on an odd-dimensional real vector space
has an eigenvalue. All this is done without dening determinants
or characteristic polynomials!
Inner-product spaces are dened in Chapter 6, and their basic
properties are developed along with standard tools such as orthonormal bases, the Gram-Schmidt procedure, and adjoints. This
chapter also shows how orthogonal projections can be used to
solve certain minimization problems.
The spectral theorem, which characterizes the linear operators for
which there exists an orthonormal basis consisting of eigenvectors, is the highlight of Chapter 7. The work in earlier chapters
pays off here with especially simple proofs. This chapter also
deals with positive operators, linear isometries, the polar decomposition, and the singular-value decomposition.
The minimal polynomial, characteristic polynomial, and generalized eigenvectors are introduced in Chapter 8. The main achievement of this chapter is the description of a linear operator on
a complex vector space in terms of its generalized eigenvectors.
This description enables one to prove almost all the results usually proved using Jordan form. For example, these tools are used
to prove that every invertible linear operator on a complex vector
space has a square root. The chapter concludes with a proof that
every linear operator on a complex vector space can be put into
Jordan form.
Linear operators on real vector spaces occupy center stage in
Chapter 9. Here two-dimensional invariant subspaces make up
for the possible lack of eigenvalues, leading to results analogous
to those obtained on complex vector spaces.
The trace and determinant are dened in Chapter 10 in terms
of the characteristic polynomial (dened earlier without determinants). On complex vector spaces, these denitions can be restated: the trace is the sum of the eigenvalues and the determinant is the product of the eigenvalues (both counting multiplicity). These easy-to-remember denitions would not be possible
with the traditional approach to eigenvalues because that method
uses determinants to prove that eigenvalues exist. The standard
theorems about determinants now become much clearer. The polar decomposition and the characterization of self-adjoint operators are used to derive the change of variables formula for multivariable integrals in a fashion that makes the appearance of the
determinant there seem natural.
This book usually develops linear algebra simultaneously for real
and complex vector spaces by letting F denote either the real or the
complex numbers. Abstract elds could be used instead, but to do so
would introduce extra abstraction without leading to any new linear algebra. Another reason for restricting attention to the real and complex
numbers is that polynomials can then be thought of as genuine functions instead of the more formal objects needed for polynomials with
coefcients in nite elds. Finally, even if the beginning part of the theory were developed with arbitrary elds, inner-product spaces would
push consideration back to just real and complex vector spaces.
xi
xii
Even in a book as short as this one, you cannot expect to cover everything. Going through the rst eight chapters is an ambitious goal for a
one-semester course. If you must reach Chapter 10, then I suggest covering Chapters 1, 2, and 4 quickly (students may have seen this material
in earlier courses) and skipping Chapter 9 (in which case you should
discuss trace and determinants only on complex vector spaces).
A goal more important than teaching any particular set of theorems
is to develop in students the ability to understand and manipulate the
objects of linear algebra. Mathematics can be learned only by doing;
fortunately, linear algebra has many good homework problems. When
teaching this course, I usually assign two or three of the exercises each
class, due the next class. Going over the homework might take up a
third or even half of a typical class.
A solutions manual for all the exercises is available (without charge)
only to instructors who are using this book as a textbook. To obtain
the solutions manual, instructors should send an e-mail request to me
(or contact Springer if I am no longer around).
Please check my web site for a list of errata (which I hope will be
empty or almost empty) and other information about this book.
I would greatly appreciate hearing about any errors in this book,
even minor ones. I welcome your suggestions for improvements, even
tiny ones. Please feel free to contact me.
Have fun!
Sheldon Axler
Mathematics Department
San Francisco State University
San Francisco, CA 94132, USA
e-mail: axler@math.sfsu.edu
www home page: http://math.sfsu.edu/axler
Preface to the Student
You are probably about to begin your second exposure to linear algebra. Unlike your rst brush with the subject, which probably emphasized Euclidean spaces and matrices, we will focus on abstract vector
spaces and linear maps. These terms will be dened later, so dont
worry if you dont know what they mean. This book starts from the beginning of the subject, assuming no knowledge of linear algebra. The
key point is that you are about to immerse yourself in serious mathematics, with an emphasis on your attaining a deep understanding of
the denitions, theorems, and proofs.
You cannot expect to read mathematics the way you read a novel. If
you zip through a page in less than an hour, you are probably going too
fast. When you encounter the phrase as you should verify, you should
indeed do the verication, which will usually require some writing on
your part. When steps are left out, you need to supply the missing
pieces. You should ponder and internalize each denition. For each
theorem, you should seek examples to show why each hypothesis is
necessary.
Please check my web site for a list of errata (which I hope will be
empty or almost empty) and other information about this book.
I would greatly appreciate hearing about any errors in this book,
even minor ones. I welcome your suggestions for improvements, even
tiny ones.
Have fun!
Sheldon Axler
Mathematics Department
San Francisco State University
San Francisco, CA 94132, USA
e-mail: axler@math.sfsu.edu
www home page: http://math.sfsu.edu/axler
xiii
Acknowledgments
I owe a huge intellectual debt to the many mathematicians who created linear algebra during the last two centuries. In writing this book I
tried to think about the best way to present linear algebra and to prove
its theorems, without regard to the standard methods and proofs used
in most textbooks. Thus I did not consult other books while writing
this one, though the memory of many books I had studied in the past
surely inuenced me. Most of the results in this book belong to the
common heritage of mathematics. A special case of a theorem may
rst have been proved in antiquity (which for linear algebra means the
nineteenth century), then slowly sharpened and improved over decades
by many mathematicians. Bestowing proper credit on all the contributors would be a difcult task that I have not undertaken. In no case
should the reader assume that any theorem presented here represents
my original contribution.
Many people helped make this a better book. For useful suggestions and corrections, I am grateful to William Arveson (for suggesting
the proof of 5.13), Marilyn Brouwer, William Brown, Robert Burckel,
Paul Cohn, James Dudziak, David Feldman (for suggesting the proof of
8.40), Pamela Gorkin, Aram Harrow, Pan Fong Ho, Dan Kalman, Robert
Kantrowitz, Ramana Kappagantu, Mizan Khan, Mikael Lindstr
om, Jacob Plotkin, Elena Poletaeva, Mihaela Poplicher, Richard Potter, Wade
Ramey, Marian Robbins, Jonathan Rosenberg, Joan Stamm, Thomas
Starbird, Jay Valanju, and Thomas von Foerster.
Finally, I thank Springer for providing me with help when I needed
it and for allowing me the freedom to make the nal decisions about
the content and appearance of this book.
xv
Chapter 1
Vector Spaces
Linear algebra is the study of linear maps on nite-dimensional vector spaces. Eventually we will learn what all these terms mean. In this
chapter we will dene vector spaces and discuss their elementary properties.
In some areas of mathematics, including linear algebra, better theorems and more insight emerge if complex numbers are investigated
along with real numbers. Thus we begin by introducing the complex
numbers and their basic properties.
Chapter 1. Vector Spaces
Complex Numbers
The symbol i was rst
1 by
used to denote
the Swiss
mathematician
Leonhard Euler in 1777.
You should already be familiar with the basic properties of the set R
of real numbers. Complex numbers were invented so that we can take
square roots of negative numbers. The key idea is to assume we have
a square root of 1, denoted i, and manipulate it using the usual rules
of arithmetic. Formally, a complex number is an ordered pair (a, b),
where a, b R, but we will write this as a + bi. The set of all complex
numbers is denoted by C:
C = {a + bi : a, b R}.
If a R, we identify a + 0i with the real number a. Thus we can think
of R as a subset of C.
Addition and multiplication on C are dened by
(a + bi) + (c + di) = (a + c) + (b + d)i,
(a + bi)(c + di) = (ac bd) + (ad + bc)i;
here a, b, c, d R. Using multiplication as dened above, you should
verify that i2 = 1. Do not memorize the formula for the product
of two complex numbers; you can always rederive it by recalling that
i2 = 1 and then using the usual rules of arithmetic.
You should verify, using the familiar properties of the real numbers, that addition and multiplication on C satisfy the following properties:
commutativity
w + z = z + w and wz = zw for all w, z C;
associativity
(z1 + z2 ) + z3 = z1 + (z2 + z3 ) and (z1 z2 )z3 = z1 (z2 z3 ) for all
z1 , z2 , z3 C;
identities
z + 0 = z and z1 = z for all z C;
additive inverse
for every z C, there exists a unique w C such that z + w = 0;
multiplicative inverse
for every z C with z = 0, there exists a unique w C such that
zw = 1;
Complex Numbers
distributive property
(w + z) = w + z for all , w, z C.
For z C, we let z denote the additive inverse of z. Thus z is
the unique complex number such that
z + (z) = 0.
Subtraction on C is dened by
w z = w + (z)
for w, z C.
For z C with z = 0, we let 1/z denote the multiplicative inverse
of z. Thus 1/z is the unique complex number such that
z(1/z) = 1.
Division on C is dened by
w/z = w(1/z)
for w, z C with z = 0.
So that we can conveniently make denitions and prove theorems
that apply to both real and complex numbers, we adopt the following
notation:
Throughout this book,
F stands for either R or C.
The letter F is used

because R and C are
examples of what are
Thus if we prove a theorem involving F, we will know that it holds when

F is replaced with R and when F is replaced with C. Elements of F are
called scalars. The word scalar, which means number, is often used
when we want to emphasize that an object is a number, as opposed to
a vector (vectors will be dened soon).
For z F and m a positive integer, we dene zm to denote the
product of z with itself m times:
z .
zm = z

m times
Clearly (zm )n = zmn and (wz)m = w m zm for all w, z F and all

positive integers m, n.
called elds. In this

book we will not need
to deal with elds other
than R or C. Many of
the denitions,
theorems, and proofs
in linear algebra that
work for both R and C
also work without
change if an arbitrary
eld replaces R or C.
Definition of Vector Space

Before dening what a vector space is, lets look at two important
examples. The vector space R 2 , which you can think of as a plane,
consists of all ordered pairs of real numbers:
R 2 = {(x, y) : x, y R}.
The vector space R 3 , which you can think of as ordinary space, consists
of all ordered triples of real numbers:
R3 = {(x, y, z) : x, y, z R}.
Many mathematicians
To generalize R 2 and R3 to higher dimensions, we rst need to discuss the concept of lists. Suppose n is a nonnegative integer. A list of
length n is an ordered collection of n objects (which might be numbers, other lists, or more abstract entities) separated by commas and
surrounded by parentheses. A list of length n looks like this:
call a list of length n an

n-tuple.
(x1 , . . . , xn ).
Thus a list of length 2 is an ordered pair and a list of length 3 is an
ordered triple. For j {1, . . . , n}, we say that xj is the j th coordinate
of the list above. Thus x1 is called the rst coordinate, x2 is called the
second coordinate, and so on.
Sometimes we will use the word list without specifying its length.
Remember, however, that by denition each list has a nite length that
is a nonnegative integer, so that an object that looks like
(x1 , x2 , . . . ),
which might be said to have innite length, is not a list. A list of length
0 looks like this: (). We consider such an object to be a list so that
some of our theorems will not have trivial exceptions.
Two lists are equal if and only if they have the same length and
the same coordinates in the same order. In other words, (x1 , . . . , xm )
equals (y1 , . . . , yn ) if and only if m = n and x1 = y1 , . . . , xm = ym .
Lists differ from sets in two ways: in lists, order matters and repetitions are allowed, whereas in sets, order and repetitions are irrelevant.
For example, the lists (3, 5) and (5, 3) are not equal, but the sets {3, 5}
and {5, 3} are equal. The lists (4, 4) and (4, 4, 4) are not equal (they
Denition of Vector Space
do not have the same length), though the sets {4, 4} and {4, 4, 4} both
equal the set {4}.
To dene the higher-dimensional analogues of R 2 and R 3 , we will
simply replace R with F (which equals R or C) and replace the 2 or 3
with an arbitrary positive integer. Specically, x a positive integer n
for the rest of this section. We dene Fn to be the set of all lists of
length n consisting of elements of F:
Fn = {(x1 , . . . , xn ) : xj F for j = 1, . . . , n}.
For example, if F = R and n equals 2 or 3, then this denition of Fn
agrees with our previous notions of R 2 and R3 . As another example,
C4 is the set of all lists of four complex numbers:
C4 = {(z1 , z2 , z3 , z4 ) : z1 , z2 , z3 , z4 C}.
If n 4, we cannot easily visualize R n as a physical object. The same
problem arises if we work with complex numbers: C1 can be thought
of as a plane, but for n 2, the human brain cannot provide geometric
models of Cn . However, even if n is large, we can perform algebraic
manipulations in Fn as easily as in R 2 or R 3 . For example, addition is
dened on Fn by adding corresponding coordinates:
For an amusing
account of how R3
would be perceived by
a creature living in R2 ,
read Flatland: A
Romance of Many
Dimensions, by Edwin
1.1
(x1 , . . . , xn ) + (y1 , . . . , yn ) = (x1 + y1 , . . . , xn + yn ).
Often the mathematics of Fn becomes cleaner if we use a single

entity to denote an list of n numbers, without explicitly writing the
coordinates. Thus the commutative property of addition on Fn should
be expressed as
x+y =y +x
for all x, y Fn , rather than the more cumbersome
(x1 , . . . , xn ) + (y1 , . . . , yn ) = (y1 , . . . , yn ) + (x1 , . . . , xn )
for all x1 , . . . , xn , y1 , . . . , yn F (even though the latter formulation
is needed to prove commutativity). If a single letter is used to denote
an element of Fn , then the same letter, with appropriate subscripts,
is often used when coordinates must be displayed. For example, if
x Fn , then letting x equal (x1 , . . . , xn ) is good notation. Even better,
work with just x and avoid explicit coordinates, if possible.
A. Abbott. This novel,

published in 1884, can
help creatures living in
three-dimensional
space, such as
ourselves, imagine a
physical space of four
or more dimensions.
We let 0 denote the list of length n all of whose coordinates are 0:

0 = (0, . . . , 0).
Note that we are using the symbol 0 in two different wayson the
left side of the equation above, 0 denotes a list of length n, whereas
on the right side, each 0 denotes a number. This potentially confusing
practice actually causes no problems because the context always makes
clear what is intended. For example, consider the statement that 0 is
an additive identity for Fn :
x+0=x
for all x Fn . Here 0 must be a list because we have not dened the
sum of an element of Fn (namely, x) and the number 0.
A picture can often aid our intuition. We will draw pictures depicting R2 because we can easily sketch this space on two-dimensional
surfaces such as paper and blackboards. A typical element of R 2 is a
point x = (x1 , x2 ). Sometimes we think of x not as a point but as an
arrow starting at the origin and ending at (x1 , x2 ), as in the picture
below. When we think of x as an arrow, we refer to it as a vector .
x -axis
2
(x , x )
1
x1-axis
Elements of R 2 can be thought of as points or as vectors.

The coordinate axes and the explicit coordinates unnecessarily clutter the picture above, and often you will gain better understanding by
dispensing with them and just thinking of the vector, as in the next
picture.
A vector
Whenever we use pictures in R2 or use the somewhat vague language of points and vectors, remember that these are just aids to our
understanding, not substitutes for the actual mathematics that we will
develop. Though we cannot draw good pictures in high-dimensional
spaces, the elements of these spaces are as rigorously dened as ele
ments of R2 . For example, (2, 3, 17, , 2) is an element of R5 , and we
may casually refer to it as a point in R 5 or a vector in R5 without worrying about whether the geometry of R5 has any physical meaning.
Recall that we dened the sum of two elements of Fn to be the element of Fn obtained by adding corresponding coordinates; see 1.1. In
the special case of R2 , addition has a simple geometric interpretation.
Suppose we have two vectors x and y in R2 that we want to add, as in
the left side of the picture below. Move the vector y parallel to itself so
that its initial point coincides with the end point of the vector x. The
sum x + y then equals the vector whose initial point equals the initial point of x and whose end point equals the end point of the moved
vector y, as in the right side of the picture below.
Mathematical models
of the economy often
have thousands of
variables, say
x1 , . . . , x5000 , which
means that we must
operate in R5000 . Such
a space cannot be dealt
with geometrically, but
the algebraic approach
works well. Thats why
our subject is called

y
x+y
The sum of two vectors

Our treatment of the vector y in the picture above illustrates a standard
philosophy when we think of vectors in R 2 as arrows: we can move an
arrow parallel to itself (not changing its length or direction) and still
think of it as the same vector.
linear algebra.
Having dealt with addition in Fn , we now turn to multiplication. We

could dene a multiplication on Fn in a similar fashion, starting with
two elements of Fn and getting another element of Fn by multiplying
corresponding coordinates. Experience shows that this denition is not
useful for our purposes. Another type of multiplication, called scalar
multiplication, will be central to our subject. Specically, we need to
dene what it means to multiply an element of Fn by an element of F.
We make the obvious denition, performing the multiplication in each
coordinate:
a(x1 , . . . , xn ) = (ax1 , . . . , axn );
In scalar multiplication,
we multiply together a
scalar and a vector,
getting a vector. You
may be familiar with
the dot product in R2
here a F and (x1 , . . . , xn ) Fn .

Scalar multiplication has a nice geometric interpretation in R2 . If
a is a positive number and x is a vector in R2 , then ax is the vector
that points in the same direction as x and whose length is a times the
length of x. In other words, to get ax, we shrink or stretch x by a
factor of a, depending upon whether a < 1 or a > 1. The next picture
illustrates this point.
or R3 , in which we
multiply together two
vectors and obtain a
scalar. Generalizations
of the dot product will
(3/2) x
x
(1/2)x
become important
when we study inner
Multiplication by positive scalars
products in Chapter 6.
You may also be
familiar with the cross
product in R3 , in which
If a is a negative number and x is a vector in R2 , then ax is the vector

that points in the opposite direction as x and whose length is |a| times
the length of x, as illustrated in the next picture.
we multiply together
two vectors and obtain
another vector. No
useful generalization of
this type of
(1/2) x
multiplication exists in
higher dimensions.
(3/2)x
Multiplication by negative scalars
The motivation for the denition of a vector space comes from the
important properties possessed by addition and scalar multiplication
on Fn . Specically, addition on Fn is commutative and associative and
has an identity, namely, 0. Every element has an additive inverse. Scalar
multiplication on Fn is associative, and scalar multiplication by 1 acts
as a multiplicative identity should. Finally, addition and scalar multiplication on Fn are connected by distributive properties.
We will dene a vector space to be a set V along with an addition
and a scalar multiplication on V that satisfy the properties discussed
in the previous paragraph. By an addition on V we mean a function
that assigns an element u + v V to each pair of elements u, v V .
By a scalar multiplication on V we mean a function that assigns an
element av V to each a F and each v V .
Now we are ready to give the formal denition of a vector space.
A vector space is a set V along with an addition on V and a scalar
multiplication on V such that the following properties hold:
commutativity
u + v = v + u for all u, v V ;
associativity
(u + v) + w = u + (v + w) and (ab)v = a(bv) for all u, v, w V
and all a, b F;
additive identity
there exists an element 0 V such that v + 0 = v for all v V ;
additive inverse
for every v V , there exists w V such that v + w = 0;
multiplicative identity
1v = v for all v V ;
distributive properties
a(u + v) = au + av and (a + b)u = au + bu for all a, b F and
all u, v V .
The scalar multiplication in a vector space depends upon F. Thus
when we need to be precise, we will say that V is a vector space over F
instead of saying simply that V is a vector space. For example, R n is
a vector space over R, and Cn is a vector space over C. Frequently, a
vector space over R is called a real vector space and a vector space over
10
The simplest vector

space contains only
one point. In other
words, {0} is a vector
space, though not a
very interesting one.
C is called a complex vector space. Usually the choice of F is either

obvious from the context or irrelevant, and thus we often assume that
F is lurking in the background without specically mentioning it.
Elements of a vector space are called vectors or points. This geometric language sometimes aids our intuition.
Not surprisingly, Fn is a vector space over F, as you should verify.
Of course, this example motivated our denition of vector space.
For another example, consider F , which is dened to be the set of
all sequences of elements of F:
F = {(x1 , x2 , . . . ) : xj F for j = 1, 2, . . . }.
Addition and scalar multiplication on F are dened as expected:
(x1 , x2 , . . . ) + (y1 , y2 , . . . ) = (x1 + y1 , x2 + y2 , . . . ),
a(x1 , x2 , . . . ) = (ax1 , ax2 , . . . ).
With these denitions, F becomes a vector space over F, as you should
verify. The additive identity in this vector space is the sequence consisting of all 0s.
Our next example of a vector space involves polynomials. A function
p : F F is called a polynomial with coefcients in F if there exist
a0 , . . . , am F such that
p(z) = a0 + a1 z + a2 z2 + + am zm
Though Fn is our
crucial example of a
vector space, not all
vector spaces consist
of lists. For example,
the elements of P(F)
consist of functions on
F, not lists. In general,
a vector space is an
abstract entity whose
elements might be lists,
functions, or weird
objects.
for all z F. We dene P(F) to be the set of all polynomials with

coefcients in F. Addition on P(F) is dened as you would expect: if
p, q P(F), then p + q is the polynomial dened by
(p + q)(z) = p(z) + q(z)
for z F. For example, if p is the polynomial dened by p(z) = 2z +z3
and q is the polynomial dened by q(z) = 7 + 4z, then p + q is the
polynomial dened by (p + q)(z) = 7 + 6z + z3 . Scalar multiplication
on P(F) also has the obvious denition: if a F and p P(F), then
ap is the polynomial dened by
(ap)(z) = ap(z)
for z F. With these denitions of addition and scalar multiplication,
P(F) is a vector space, as you should verify. The additive identity in
this vector space is the polynomial all of whose coefcients equal 0.
Soon we will see further examples of vector spaces, but rst we need
to develop some of the elementary properties of vector spaces.
11

The denition of a vector space requires that it have an additive
identity. The proposition below states that this identity is unique.
1.2
Proposition: A vector space has a unique additive identity.
Proof: Suppose 0 and 0 are both additive identities for some vector space V . Then
0 = 0 + 0 = 0,
where the rst equality holds because 0 is an additive identity and the
second equality holds because 0 is an additive identity. Thus 0 = 0,
proving that V has only one additive identity.
The symbol
means
end of the proof.
Each element v in a vector space has an additive inverse, an element

w in the vector space such that v + w = 0. The next proposition shows
that each element in a vector space has only one additive inverse.
1.3
Proposition:
additive inverse.
Every element in a vector space has a unique
Proof: Suppose V is a vector space. Let v V . Suppose that w

and w are additive inverses of v. Then
w = w + 0 = w + (v + w ) = (w + v) + w = 0 + w = w .
Thus w = w , as desired.
Because additive inverses are unique, we can let v denote the additive inverse of a vector v. We dene w v to mean w + (v).
Almost all the results in this book will involve some vector space.
To avoid being distracted by having to restate frequently something
such as Assume that V is a vector space, we now make the necessary
declaration once and for all:
Lets agree that for the rest of the book
V will denote a vector space over F.
12
Because of associativity, we can dispense with parentheses when

dealing with additions involving more than two elements in a vector
space. For example, we can write u+v+w without parentheses because
the two possible interpretations of that expression, namely, (u+v)+w
and u + (v + w), are equal. We rst use this familiar convention of not
using parentheses in the next proof. In the next proposition, 0 denotes
a scalar (the number 0 F) on the left side of the equation and a vector
(the additive identity of V ) on the right side of the equation.
Note that 1.4 and 1.5
assert something about
scalar multiplication
1.4
Proof: For v V , we have
and the additive
0v = (0 + 0)v = 0v + 0v.
identity of V . The only

part of the denition of
a vector space that
connects scalar
multiplication and
vector addition is the
distributive property.
Thus the distributive
property must be used
in the proofs.
Proposition: 0v = 0 for every v V .
Adding the additive inverse of 0v to both sides of the equation above

gives 0 = 0v, as desired.
In the next proposition, 0 denotes the additive identity of V . Though
their proofs are similar, 1.4 and 1.5 are not identical. More precisely,
1.4 states that the product of the scalar 0 and any vector equals the
vector 0, whereas 1.5 states that the product of any scalar and the
vector 0 equals the vector 0.
1.5
Proposition: a0 = 0 for every a F.
Proof: For a F, we have

a0 = a(0 + 0) = a0 + a0.
Adding the additive inverse of a0 to both sides of the equation above
gives 0 = a0, as desired.
Now we show that if an element of V is multiplied by the scalar 1,
then the result is the additive inverse of the element of V .
1.6
Proposition: (1)v = v for every v V .
Proof: For v V , we have

v + (1)v = 1v + (1)v = 1 + (1) v = 0v = 0.
This equation says that (1)v, when added to v, gives 0. Thus (1)v
must be the additive inverse of v, as desired.
Subspaces
13
Subspaces
A subset U of V is called a subspace of V if U is also a vector space
(using the same addition and scalar multiplication as on V ). For example,
{(x1 , x2 , 0) : x1 , x2 F}
Some mathematicians
use the term linear
subspace, which means
the same as subspace.
is a subspace of F3 .
If U is a subset of V , then to check that U is a subspace of V we
need only check that U satises the following:
additive identity
0U
closed under addition
u, v U implies u + v U;
closed under scalar multiplication
a F and u U implies au U.
The rst condition insures that the additive identity of V is in U. The
second condition insures that addition makes sense on U. The third
condition insures that scalar multiplication makes sense on U . To show
that U is a vector space, the other parts of the denition of a vector
space do not need to be checked because they are automatically satised. For example, the associative and commutative properties of addition automatically hold on U because they hold on the larger space V .
As another example, if the third condition above holds and u U , then
u (which equals (1)u by 1.6) is also in U, and hence every element
of U has an additive inverse in U.
The three conditions above usually enable us to determine quickly
whether a given subset of V is a subspace of V . For example, if b F,
then
{(x1 , x2 , x3 , x4 ) F4 : x3 = 5x4 + b}
is a subspace of F4 if and only if b = 0, as you should verify. As another
example, you should verify that
{p P(F) : p(3) = 0}
is a subspace of P(F).
The subspaces of R 2 are precisely {0}, R2 , and all lines in R 2 through
the origin. The subspaces of R 3 are precisely {0}, R 3 , all lines in R3
Clearly {0} is the

smallest subspace of V
and V itself is the
largest subspace of V .
The empty set is not a
subspace of V because
a subspace must be a
vector space and a
vector space must
contain at least one
element, namely, an
additive identity.
14
through the origin, and all planes in R 3 through the origin. To prove
that all these objects are indeed subspaces is easythe hard part is to
show that they are the only subspaces of R2 or R3 . That task will be
easier after we introduce some additional tools in the next chapter.
Sums and Direct Sums
When dealing with

vector spaces, we are
usually interested only
In later chapters, we will nd that the notions of vector space sums

and direct sums are useful. We dene these concepts here.
Suppose U1 , . . . , Um are subspaces of V . The sum of U1 , . . . , Um ,
denoted U1 + + Um , is dened to be the set of all possible sums of
elements of U1 , . . . , Um . More precisely,
in subspaces, as
U1 + + Um = {u1 + + um : u1 U1 , . . . , um Um }.
opposed to arbitrary
subsets. The union of
subspaces is rarely a
subspace (see
Exercise 9 in this
chapter), which is why
we usually work with
sums rather than
You should verify that if U1 , . . . , Um are subspaces of V , then the sum

U1 + + Um is a subspace of V .
Lets look at some examples of sums of subspaces. Suppose U is the
set of all elements of F3 whose second and third coordinates equal 0,
and W is the set of all elements of F3 whose rst and third coordinates
equal 0:
unions.
U = {(x, 0, 0) F3 : x F}
and W = {(0, y, 0) F3 : y F}.
Then
Sums of subspaces in
the theory of vector
spaces are analogous to
unions of subsets in set
theory. Given two
subspaces of a vector
1.7
U + W = {(x, y, 0) : x, y F},
as you should verify.

As another example, suppose U is as above and W is the set of all
elements of F3 whose rst and second coordinates equal each other
and whose third coordinate equals 0:
space, the smallest

subspace containing
W = {(y, y, 0) F3 : y F}.
them is their sum.

Analogously, given two
subsets of a set, the
smallest subset
containing them is
their union.
Then U + W is also given by 1.7, as you should verify.

Suppose U1 , . . . , Um are subspaces of V . Clearly U1 , . . . , Um are all
contained in U1 + + Um (to see this, consider sums u1 + + um
where all except one of the us are 0). Conversely, any subspace of V
containing U1 , . . . , Um must contain U1 + + Um (because subspaces
15
must contain all nite sums of their elements). Thus U1 + + Um is

the smallest subspace of V containing U1 , . . . , Um .
Suppose U1 , . . . , Um are subspaces of V such that V = U1 + +Um .
Thus every element of V can be written in the form
u1 + + um ,
where each uj Uj . We will be especially interested in cases where
each vector in V can be uniquely represented in the form above. This
situation is so important that we give it a special name: direct sum.
Specically, we say that V is the direct sum of subspaces U1 , . . . , Um ,
written V = U1 Um , if each element of V can be written uniquely
as a sum u1 + + um , where each uj Uj .
Lets look at some examples of direct sums. Suppose U is the subspace of F3 consisting of those vectors whose last coordinate equals 0,
and W is the subspace of F3 consisting of those vectors whose rst two
coordinates equal 0:
U = {(x, y, 0) F3 : x, y F}
and W = {(0, 0, z) F3 : z F}.
Then F3 = U W , as you should verify.

As another example, suppose Uj is the subspace of Fn consisting
of those vectors whose coordinates are all 0, except possibly in the j th
slot (for example, U2 = {(0, x, 0, . . . , 0) Fn : x F}). Then
Fn = U1 Un ,
As a nal example, consider the vector space P(F) of all polynomials
with coefcients in F. Let Ue denote the subspace of P(F) consisting
of all polynomials p of the form
p(z) = a0 + a2 z2 + + a2m z2m ,
and let Uo denote the subspace of P(F) consisting of all polynomials p
of the form
p(z) = a1 z + a3 z3 + + a2m+1 z2m+1 ;
here m is a nonnegative integer and a0 , . . . , a2m+1 F (the notations
Ue and Uo should remind you of even and odd powers of z). You should
verify that
The symbol ,
consisting of a plus
sign inside a circle, is
used to denote direct
sums as a reminder
that we are dealing with
a special type of sum of
subspaceseach
element in the direct
sum can be represented
only one way as a sum
of elements from the
specied subspaces.
16
P(F) = Ue Uo .
Sometimes nonexamples add to our understanding as much as examples. Consider the following three subspaces of F3 :
U1 = {(x, y, 0) F3 : x, y F};
U2 = {(0, 0, z) F3 : z F};
U3 = {(0, y, y) F3 : y F}.
Clearly F3 = U1 + U2 + U3 because an arbitrary vector (x, y, z) F3 can
be written as
(x, y, z) = (x, y, 0) + (0, 0, z) + (0, 0, 0),
where the rst vector on the right side is in U1 , the second vector is
in U2 , and the third vector is in U3 . However, F3 does not equal the
direct sum of U1 , U2 , U3 because the vector (0, 0, 0) can be written in
two different ways as a sum u1 +u2 +u3 , with each uj Uj . Specically,
we have
(0, 0, 0) = (0, 1, 0) + (0, 0, 1) + (0, 1, 1)
and, of course,
(0, 0, 0) = (0, 0, 0) + (0, 0, 0) + (0, 0, 0),
where the rst vector on the right side of each equation above is in U1 ,
the second vector is in U2 , and the third vector is in U3 .
In the example above, we showed that something is not a direct sum
by showing that 0 does not have a unique representation as a sum of
appropriate vectors. The denition of direct sum requires that every
vector in the space have a unique representation as an appropriate sum.
Suppose we have a collection of subspaces whose sum equals the whole
space. The next proposition shows that when deciding whether this
collection of subspaces is a direct sum, we need only consider whether
0 can be uniquely written as an appropriate sum.
1.8
Proposition: Suppose that U1 , . . . , Un are subspaces of V . Then
V = U1 Un if and only if both the following conditions hold:
(a)
V = U 1 + + Un ;
(b)
the only way to write 0 as a sum u1 + + un , where each

uj Uj , is by taking all the uj s equal to 0.
17
Proof: First suppose that V = U1 Un . Clearly (a) holds

(because of how sum and direct sum are dened). To prove (b), suppose
that u1 U1 , . . . , un Un and
0 = u1 + + un .
Then each uj must be 0 (this follows from the uniqueness part of the
denition of direct sum because 0 = 0+ +0 and 0 U1 , . . . , 0 Un ),
proving (b).
Now suppose that (a) and (b) hold. Let v V . By (a), we can write
v = u1 + + un
for some u1 U1 , . . . , un Un . To show that this representation is
unique, suppose that we also have
v = v 1 + + vn ,
where v1 U1 , . . . , vn Un . Subtracting these two equations, we have
0 = (u1 v1 ) + + (un vn ).
Clearly u1 v1 U1 , . . . , un vn Un , so the equation above and (b)
imply that each uj vj = 0. Thus u1 = v1 , . . . , un = vn , as desired.
The next proposition gives a simple condition for testing which pairs
of subspaces give a direct sum. Note that this proposition deals only
with the case of two subspaces. When asking about a possible direct
sum with more than two subspaces, it is not enough to test that any
two of the subspaces intersect only at 0. To see this, consider the
nonexample presented just before 1.8. In that nonexample, we had
F3 = U1 + U2 + U3 , but F3 did not equal the direct sum of U1 , U2 , U3 .
However, in that nonexample, we have U1 U2 = U1 U3 = U2 U3 = {0}
(as you should verify). The next proposition shows that with just two
subspaces we get a nice necessary and sufcient condition for a direct
sum.
Sums of subspaces are

analogous to unions of
subsets. Similarly,
direct sums of
subspaces are
analogous to disjoint
unions of subsets. No
two subspaces of a
vector space can be
disjoint because both
must contain 0. So
disjointness is
1.9
Proposition: Suppose that U and W are subspaces of V . Then
V = U W if and only if V = U + W and U W = {0}.
replaced, at least in the

case of two subspaces,
with the requirement
Proof: First suppose that V = U W . Then V = U + W (by the

denition of direct sum). Also, if v U W , then 0 = v + (v), where
that the intersection

equals {0}.
18
v U and v W . By the unique representation of 0 as the sum of a

vector in U and a vector in W , we must have v = 0. Thus U W = {0},
completing the proof in one direction.
To prove the other direction, now suppose that V = U + W and
U W = {0}. To prove that V = U W , suppose that
0 = u + w,
where u U and w W . To complete the proof, we need only show
that u = w = 0 (by 1.8). The equation above implies that u = w W .
Thus u U W , and hence u = 0. This, along with equation above,
implies that w = 0, completing the proof.
Exercises
Exercises
1.
Suppose a and b are real numbers, not both 0. Find real numbers
c and d such that
1/(a + bi) = c + di.
2.
Show that
1 + 3i
2
is a cube root of 1 (meaning that its cube equals 1).

3.
Prove that (v) = v for every v V .
4.
Prove that if a F, v V , and av = 0, then a = 0 or v = 0.
5.
For each of the following subsets of F3 , determine whether it is

a subspace of F3 :
(a)
{(x1 , x2 , x3 ) F3 : x1 + 2x2 + 3x3 = 0};
(b)
{(x1 , x2 , x3 ) F3 : x1 + 2x2 + 3x3 = 4};
(c)
{(x1 , x2 , x3 ) F3 : x1 x2 x3 = 0};
(d)
{(x1 , x2 , x3 ) F3 : x1 = 5x3 }.
6.
Give an example of a nonempty subset U of R2 such that U is

closed under addition and under taking additive inverses (meaning u U whenever u U), but U is not a subspace of R2 .
7.
Give an example of a nonempty subset U of R2 such that U is

closed under scalar multiplication, but U is not a subspace of R2 .
8.
Prove that the intersection of any collection of subspaces of V is

a subspace of V .
9.
Prove that the union of two subspaces of V is a subspace of V if

and only if one of the subspaces is contained in the other.
10.
Suppose that U is a subspace of V . What is U + U ?
11.
Is the operation of addition on the subspaces of V commutative?

Associative? (In other words, if U1 , U2 , U3 are subspaces of V , is
U1 + U2 = U2 + U1 ? Is (U1 + U2 ) + U3 = U1 + (U2 + U3 )?)
19
20
12.
Does the operation of addition on the subspaces of V have an

additive identity? Which subspaces have additive inverses?
13.
Prove or give a counterexample: if U1 , U2 , W are subspaces of V

such that
U1 + W = U2 + W ,
then U1 = U2 .
14.
Suppose U is the subspace of P(F) consisting of all polynomials

p of the form
p(z) = az2 + bz5 ,
where a, b F. Find a subspace W of P(F) such that P(F) =
U W.
15.
Prove or give a counterexample: if U1 , U2 , W are subspaces of V

such that
V = U1 W and V = U2 W ,
then U1 = U2 .
Chapter 2
Finite-Dimensional
Vector Spaces
In the last chapter we learned about vector spaces. Linear algebra

focuses not on arbitrary vector spaces, but on nite-dimensional vector
spaces, which we introduce in this chapter. Here we will deal with the
key concepts associated with these spaces: span, linear independence,
basis, and dimension.
Lets review our standing assumptions:
Recall that F denotes R or C.
Recall also that V is a vector space over F.
21
Chapter 2. Finite-Dimensional Vector Spaces
22
Span and Linear Independence

A linear combination of a list (v1 , . . . , vm ) of vectors in V is a vector
of the form
2.1
Some mathematicians
use the term linear
a1 v1 + + am vm ,
where a1 , . . . , am F. The set of all linear combinations of (v1 , . . . , vm )

is called the span of (v1 , . . . , vm ), denoted span(v1 , . . . , vm ). In other
words,
span, which means the
span(v1 , . . . , vm ) = {a1 v1 + + am vm : a1 , . . . , am F}.
same as span.
As an example of these concepts, suppose V = F3 . The vector

(7, 2, 9) is a linear combination of (2, 1, 3), (1, 0, 1) because
(7, 2, 9) = 2(2, 1, 3) + 3(1, 0, 1).
Recall that by
denition every list has

Thus (7, 2, 9) span (2, 1, 3), (1, 0, 1) .
You should verify that the span of any list of vectors in V is a subspace of V . To be consistent, we declare that the span of the empty list
() equals {0} (recall that the empty set is not a subspace of V ).
If (v1 , . . . , vm ) is a list of vectors in V , then each vj is a linear combination of (v1 , . . . , vm ) (to show this, set aj = 1 and let the other as
in 2.1 equal 0). Thus span(v1 , . . . , vm ) contains each vj . Conversely,
because subspaces are closed under scalar multiplication and addition,
every subspace of V containing each vj must contain span(v1 , . . . , vm ).
Thus the span of a list of vectors in V is the smallest subspace of V
containing all the vectors in the list.
If span(v1 , . . . , vm ) equals V , we say that (v1 , . . . , vm ) spans V . A
vector space is called nite dimensional if some list of vectors in it
spans the space. For example, Fn is nite dimensional because

(1, 0, . . . , 0), (0, 1, 0, . . . , 0), . . . , (0, . . . , 0, 1)
nite length.
spans Fn , as you should verify.

Before giving the next example of a nite-dimensional vector space,
we need to dene the degree of a polynomial. A polynomial p P(F)
is said to have degree m if there exist scalars a0 , a1 , . . . , am F with
am = 0 such that
2.2
p(z) = a0 + a1 z + + am zm
for all z F. The polynomial that is identically 0 is said to have degree .

For m a nonnegative integer, let Pm (F) denote the set of all polynomials with coefcients in F and degree at most m. You should verify that Pm (F) is a subspace of P(F); hence Pm (F) is a vector space.
This vector space is nite dimensional because it is spanned by the list
(1, z, . . . , zm ); here we are slightly abusing notation by letting zk denote
a function (so z is a dummy variable).
A vector space that is not nite dimensional is called innite dimensional. For example, P(F) is innite dimensional. To prove this,
consider any list of elements of P(F). Let m denote the highest degree
of any of the polynomials in the list under consideration (recall that by
denition a list has nite length). Then every polynomial in the span of
this list must have degree at most m. Thus our list cannot span P(F).
Because no list spans P(F), this vector space is innite dimensional.
The vector space F , consisting of all sequences of elements of F,
is also innite dimensional, though this is a bit harder to prove. You
should be able to give a proof by using some of the tools we will soon
develop.
Suppose v1 , . . . , vm V and v span(v1 , . . . , vm ). By the denition
of span, there exist a1 , . . . , am F such that
v = a1 v1 + + am vm .
Consider the question of whether the choice of as in the equation
m is another set of scalars such that
1 , . . . , a
above is unique. Suppose a
m vm .
1 v1 + + a
v=a
Subtracting the last two equations, we have
1 )v1 + + (am a
m )vm .
0 = (a1 a
Thus we have written 0 as a linear combination of (v1 , . . . , vm ). If the
only way to do this is the obvious way (using 0 for all scalars), then
j equals 0, which means that each aj equals a
j (and thus
each aj a
the choice of as was indeed unique). This situation is so important
that we give it a special namelinear independencewhich we now
dene.
A list (v1 , . . . , vm ) of vectors in V is called linearly independent if
the only choice of a1 , . . . , am F that makes a1 v1 + + am vm equal
0 is a1 = = am = 0. For example,
23
Innite-dimensional
vector spaces, which
we will not mention
much anymore, are the
center of attention in
the branch of
mathematics called
functional analysis.
Functional analysis
uses tools from both
analysis and algebra.
24
(1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0)
Most linear algebra

texts dene linearly
independent sets
instead of linearly
independent lists. With
that denition, the set
{(0, 1), (0, 1), (1, 0)} is
linearly independent in
F2 because it equals the
set {(0, 1), (1, 0)}. With
our denition, the list
(0, 1), (0, 1), (1, 0) is
not linearly
independent (because 1
times the rst vector
plus 1 times the
second vector plus 0
times the third vector
equals 0). By dealing
with lists instead of
sets, we will avoid
some problems
associated with the
usual approach.
is linearly independent in F4 , as you should verify. The reasoning in the

previous paragraph shows that (v1 , . . . , vm ) is linearly independent if
and only if each vector in span(v1 , . . . , vm ) has only one representation
as a linear combination of (v1 , . . . , vm ).
For another example of a linearly independent list, x a nonnegative
integer m. Then (1, z, . . . , zm ) is linearly independent in P(F). To verify
this, suppose that a0 , a1 , . . . , am F are such that
2.3
a0 + a1 z + + am zm = 0
for every z F. If at least one of the coefcients a0 , a1 , . . . , am were

nonzero, then 2.3 could be satised by at most m distinct values of z (if
you are unfamiliar with this fact, just believe it for now; we will prove
it in Chapter 4); this contradiction shows that all the coefcients in 2.3
equal 0. Hence (1, z, . . . , zm ) is linearly independent, as claimed.
A list of vectors in V is called linearly dependent if it is not linearly independent. In other words, a list (v1 , . . . , vm ) of vectors in V
is linearly dependent if there exist a1 , . . . , am F, not all 0, such that

a1 v1 + + am vm = 0. For example, (2, 3, 1), (1, 1, 2), (7, 3, 8) is
linearly dependent in F3 because
2(2, 3, 1) + 3(1, 1, 2) + (1)(7, 3, 8) = (0, 0, 0).
As another example, any list of vectors containing the 0 vector is linearly dependent (why?).
You should verify that a list (v) of length 1 is linearly independent if
and only if v = 0. You should also verify that a list of length 2 is linearly
independent if and only if neither vector is a scalar multiple of the
other. Caution: a list of length three or more may be linearly dependent
even though no vector in the list is a scalar multiple of any other vector
in the list, as shown by the example in the previous paragraph.
If some vectors are removed from a linearly independent list, the
remaining list is also linearly independent, as you should verify. To
allow this to remain true even if we remove all the vectors, we declare
the empty list () to be linearly independent.
The lemma below will often be useful. It states that given a linearly
dependent list of vectors, with the rst vector not zero, one of the
vectors is in the span of the previous ones and furthermore we can
throw out that vector without changing the span of the original list.
25
2.4
Linear Dependence Lemma: If (v1 , . . . , vm ) is linearly dependent in V and v1 = 0, then there exists j {2, . . . , m} such that the
following hold:
(a)
vj span(v1 , . . . , vj1 );
(b)
if the j th term is removed from (v1 , . . . , vm ), the span of the

remaining list equals span(v1 , . . . , vm ).
Proof: Suppose (v1 , . . . , vm ) is linearly dependent in V and v1 = 0.

Then there exist a1 , . . . , am F, not all 0, such that
a1 v1 + + am vm = 0.
Not all of a2 , a3 , . . . , am can be 0 (because v1 = 0). Let j be the largest
element of {2, . . . , m} such that aj = 0. Then
2.5
vj =
aj1
a1
v1
vj1 ,
aj
aj
proving (a).
To prove (b), suppose that u span(v1 , . . . , vm ). Then there exist
c1 , . . . , cm F such that
u = c1 v1 + + cm vm .
In the equation above, we can replace vj with the right side of 2.5,
which shows that u is in the span of the list obtained by removing the
j th term from (v1 , . . . , vm ). Thus (b) holds.
Now we come to a key result. It says that linearly independent lists
are never longer than spanning lists.
2.6
Theorem: In a nite-dimensional vector space, the length of
every linearly independent list of vectors is less than or equal to the
length of every spanning list of vectors.
Suppose that for each

positive integer m,
there exists a linearly
independent list of m
Proof: Suppose that (u1 , . . . , um ) is linearly independent in V and

that (w1 , . . . , wn ) spans V . We need to prove that m n. We do so
through the multistep process described below; note that in each step
we add one of the us and remove one of the ws.
vectors in V . Then this

theorem implies that V
is innite dimensional.
26
Step 1
The list (w1 , . . . , wn ) spans V , and thus adjoining any vector to it
produces a linearly dependent list. In particular, the list
(u1 , w1 , . . . , wn )
is linearly dependent. Thus by the linear dependence lemma (2.4),
we can remove one of the ws so that the list B (of length n)
consisting of u1 and the remaining ws spans V .
Step j
The list B (of length n) from step j 1 spans V , and thus adjoining
any vector to it produces a linearly dependent list. In particular,
the list of length (n + 1) obtained by adjoining uj to B, placing it
just after u1 , . . . , uj1 , is linearly dependent. By the linear dependence lemma (2.4), one of the vectors in this list is in the span of
the previous ones, and because (u1 , . . . , uj ) is linearly independent, this vector must be one of the ws, not one of the us. We
can remove that w from B so that the new list B (of length n)
consisting of u1 , . . . , uj and the remaining ws spans V .
After step m, we have added all the us and the process stops. If at
any step we added a u and had no more ws to remove, then we would
have a contradiction. Thus there must be at least as many ws as us.
Our intuition tells us that any vector space contained in a nitedimensional vector space should also be nite dimensional. We now
prove that this intuition is correct.
2.7
Proposition:
Every subspace of a nite-dimensional vector
space is nite dimensional.
Proof: Suppose V is nite dimensional and U is a subspace of V .
We need to prove that U is nite dimensional. We do this through the
following multistep construction.
Step 1
If U = {0}, then U is nite dimensional and we are done. If U =
{0}, then choose a nonzero vector v1 U .
Step j
If U = span(v1 , . . . , vj1 ), then U is nite dimensional and we are
Bases
27
done. If U = span(v1 , . . . , vj1 ), then choose a vector vj U such

that
vj span(v1 , . . . , vj1 ).
After each step, as long as the process continues, we have constructed
a list of vectors such that no vector in this list is in the span of the
previous vectors. Thus after each step we have constructed a linearly
independent list, by the linear dependence lemma (2.4). This linearly
independent list cannot be longer than any spanning list of V (by 2.6),
and thus the process must eventually terminate, which means that U
is nite dimensional.
Bases
A basis of V is a list of vectors in V that is linearly independent and
spans V . For example,

(1, 0, . . . , 0), (0, 1, 0, . . . , 0), . . . , (0, . . . , 0, 1)
is a basis of Fn , called the standard basis of Fn . In addition to the

standard basis, Fn has many other bases. For example, (1, 2), (3, 5)

is a basis of F2 . The list (1, 2) is linearly independent but is not a

basis of F2 because it does not span F2 . The list (1, 2), (3, 5), (4, 7)
spans F2 but is not a basis because it is not linearly independent. As
another example, (1, z, . . . , zm ) is a basis of Pm (F).
The next proposition helps explain why bases are useful.
2.8
Proposition: A list (v1 , . . . , vn ) of vectors in V is a basis of V
if and only if every v V can be written uniquely in the form
2.9
v = a1 v1 + + an vn ,
where a1 , . . . , an F.
Proof: First suppose that (v1 , . . . , vn ) is a basis of V . Let v V .
Because (v1 , . . . , vn ) spans V , there exist a1 , . . . , an F such that 2.9
holds. To show that the representation in 2.9 is unique, suppose that
b1 , . . . , bn are scalars so that we also have
This proof is
essentially a repetition
of the ideas that led us
to the denition of
linear independence.
v = b1 v1 + + bn vn .
28
Subtracting the last equation from 2.9, we get

0 = (a1 b1 )v1 + + (an bn )vn .
This implies that each aj bj = 0 (because (v1 , . . . , vn ) is linearly independent) and hence a1 = b1 , . . . , an = bn . We have the desired uniqueness, completing the proof in one direction.
For the other direction, suppose that every v V can be written
uniquely in the form given by 2.9. Clearly this implies that (v1 , . . . , vn )
spans V . To show that (v1 , . . . , vn ) is linearly independent, suppose
that a1 , . . . , an F are such that
0 = a1 v1 + + an vn .
The uniqueness of the representation 2.9 (with v = 0) implies that
a1 = = an = 0. Thus (v1 , . . . , vn ) is linearly independent and
hence is a basis of V .
A spanning list in a vector space may not be a basis because it is not
linearly independent. Our next result says that given any spanning list,
some of the vectors in it can be discarded so that the remaining list is
linearly independent and still spans the vector space.
2.10 Theorem: Every spanning list in a vector space can be reduced
to a basis of the vector space.
Proof: Suppose (v1 , . . . , vn ) spans V . We want to remove some
of the vectors from (v1 , . . . , vn ) so that the remaining vectors form a
basis of V . We do this through the multistep process described below.
Start with B = (v1 , . . . , vn ).
Step 1
If v1 = 0, delete v1 from B. If v1 = 0, leave B unchanged.
Step j
If vj is in span(v1 , . . . , vj1 ), delete vj from B. If vj is not in
span(v1 , . . . , vj1 ), leave B unchanged.
Stop the process after step n, getting a list B. This list B spans V
because our original list spanned B and we have discarded only vectors
that were already in the span of the previous vectors. The process
Bases
29
insures that no vector in B is in the span of the previous ones. Thus B

is linearly independent, by the linear dependence lemma (2.4). Hence
B is a basis of V .
Consider the list

(1, 2), (3, 6), (4, 7), (5, 9) ,
which spans F2 . To make sure that you understand the last proof, you

should verify that the process in the proof produces (1, 2), (4, 7) , a
basis of F2 , when applied to the list above.
Our next result, an easy corollary of the last theorem, tells us that
every nite-dimensional vector space has a basis.
2.11
Corollary: Every nite-dimensional vector space has a basis.
Proof: By denition, a nite-dimensional vector space has a spanning list. The previous theorem tells us that any spanning list can be
reduced to a basis.
We have crafted our denitions so that the nite-dimensional vector
space {0} is not a counterexample to the corollary above. In particular,
the empty list () is a basis of the vector space {0} because this list has
been dened to be linearly independent and to have span {0}.
Our next theorem is in some sense a dual of 2.10, which said that
every spanning list can be reduced to a basis. Now we show that given
any linearly independent list, we can adjoin some additional vectors so
that the extended list is still linearly independent but also spans the
space.
2.12 Theorem: Every linearly independent list of vectors in a nitedimensional vector space can be extended to a basis of the vector space.
This theorem can be

used to give another
proof of the previous
Proof: Suppose V is nite dimensional and (v1 , . . . , vm ) is linearly

independent in V . We want to extend (v1 , . . . , vm ) to a basis of V . We
do this through the multistep process described below. First we let
(w1 , . . . , wn ) be any list of vectors in V that spans V .
Step 1
If w1 is in the span of (v1 , . . . , vm ), let B = (v1 , . . . , vm ). If w1 is
not in the span of (v1 , . . . , vm ), let B = (v1 , . . . , vm , w1 ).
corollary. Specically,
suppose V is nite
dimensional. This
theorem implies that
the empty list () can be
extended to a basis
of V . In particular, V
has a basis.
30
Step j
If wj is in the span of B, leave B unchanged. If wj is not in the
span of B, extend B by adjoining wj to it.
After each step, B is still linearly independent because otherwise the
linear dependence lemma (2.4) would give a contradiction (recall that
(v1 , . . . , vm ) is linearly independent and any wj that is adjoined to B is
not in the span of the previous vectors in B). After step n, the span of
B includes all the ws. Thus the B obtained after step n spans V and
hence is a basis of V .
As a nice application of the theorem above, we now show that every subspace of a nite-dimensional vector space can be paired with
another subspace to form a direct sum of the whole space.
Using the same basic
ideas but considerably
2.13 Proposition: Suppose V is nite dimensional and U is a subspace of V . Then there is a subspace W of V such that V = U W .
more advanced tools,

this proposition can be
proved without the
hypothesis that V is
nite dimensional.
Proof: Because V is nite dimensional, so is U (see 2.7). Thus

there is a basis (u1 , . . . , um ) of U (see 2.11). Of course (u1 , . . . , um )
is a linearly independent list of vectors in V , and thus it can be extended to a basis (u1 , . . . , um , w1 , . . . , wn ) of V (see 2.12). Let W =
span(w1 , . . . , wn ).
To prove that V = U W , we need to show that
V =U +W
and U W = {0};
see 1.9. To prove the rst equation, suppose that v V . Then,

because the list (u1 , . . . , um , w1 , . . . , wn ) spans V , there exist scalars
a1 , . . . , am , b1 , . . . , bn F such that
v = a1 u1 + + am um + b1 w1 + + bn wn .

u
In other words, we have v = u+w, where u U and w W are dened

as above. Thus v U + W , completing the proof that V = U + W .
To show that U W = {0}, suppose v U W . Then there exist
scalars a1 , . . . , am , b1 , . . . , bn F such that
v = a1 u1 + + am um = b1 w1 + + bn wn .
Thus
Dimension
a1 u1 + + am um b1 w1 bn wn = 0.
Because (u1 , . . . , um , w1 , . . . , wn ) is linearly independent, this implies
that a1 = = am = b1 = = bn = 0. Thus v = 0, completing the
proof that U W = {0}.
Dimension
Though we have been discussing nite-dimensional vector spaces,
we have not yet dened the dimension of such an object. How should
dimension be dened? A reasonable denition should force the dimension of Fn to equal n. Notice that the basis

(1, 0, . . . , 0), (0, 1, 0, . . . , 0), . . . , (0, . . . , 0, 1)
has length n. Thus we are tempted to dene the dimension as the

length of a basis. However, a nite-dimensional vector space in general
has many different bases, and our attempted denition makes sense
only if all bases in a given vector space have the same length. Fortunately that turns out to be the case, as we now show.
2.14 Theorem: Any two bases of a nite-dimensional vector space
have the same length.
Proof: Suppose V is nite dimensional. Let B1 and B2 be any two
bases of V . Then B1 is linearly independent in V and B2 spans V , so the
length of B1 is at most the length of B2 (by 2.6). Interchanging the roles
of B1 and B2 , we also see that the length of B2 is at most the length
of B1 . Thus the length of B1 must equal the length of B2 , as desired.
Now that we know that any two bases of a nite-dimensional vector
space have the same length, we can formally dene the dimension of
such spaces. The dimension of a nite-dimensional vector space is
dened to be the length of any basis of the vector space. The dimension
of V (if V is nite dimensional) is denoted by dim V . As examples, note
that dim Fn = n and dim Pm (F) = m + 1.
Every subspace of a nite-dimensional vector space is nite dimensional (by 2.7) and so has a dimension. The next result gives the expected inequality about the dimension of a subspace.
31
32
2.15 Proposition: If V is nite dimensional and U is a subspace

of V , then dim U dim V .
Proof: Suppose that V is nite dimensional and U is a subspace
of V . Any basis of U is a linearly independent list of vectors in V and
thus can be extended to a basis of V (by 2.12). Hence the length of a
basis of U is less than or equal to the length of a basis of V .
The real vector space
R 2 has dimension 2;
the complex vector
space C has
dimension 1. As sets,
R2 can be identied
with C (and addition is
To check that a list of vectors in V is a basis of V , we must, according

to the denition, show that the list in question satises two properties:
it must be linearly independent and it must span V . The next two
results show that if the list in question has the right length, then we
need only check that it satises one of the required two properties.
We begin by proving that every spanning list with the right length is a
basis.
the same on both

spaces, as is scalar
multiplication by real
2.16 Proposition: If V is nite dimensional, then every spanning

list of vectors in V with length dim V is a basis of V .
numbers). Thus when

we talk about the
dimension of a vector
space, the role played
by the choice of F
cannot be neglected.
Proof: Suppose dim V = n and (v1 , . . . , vn ) spans V . The list

(v1 , . . . , vn ) can be reduced to a basis of V (by 2.10). However, every
basis of V has length n, so in this case the reduction must be the trivial
one, meaning that no elements are deleted from (v1 , . . . , vn ). In other
words, (v1 , . . . , vn ) is a basis of V , as desired.
Now we prove that linear independence alone is enough to ensure
that a list with the right length is a basis.
2.17 Proposition: If V is nite dimensional, then every linearly
independent list of vectors in V with length dim V is a basis of V .
Proof: Suppose dim V = n and (v1 , . . . , vn ) is linearly independent
in V . The list (v1 , . . . , vn ) can be extended to a basis of V (by 2.12). However, every basis of V has length n, so in this case the extension must be
the trivial one, meaning that no elements are adjoined to (v1 , . . . , vn ).
In other words, (v1 , . . . , vn ) is a basis of V , as desired.
As an example of how the last proposition can be applied, consider

the list (5, 7), (4, 3) . This list of two vectors in F2 is obviously linearly
independent (because neither vector is a scalar multiple of the other).
Dimension
33
Because F2 has dimension 2, the last proposition implies that this linearly independent list of length 2 is a basis of F2 (we do not need to
bother checking that it spans F2 ).
The next theorem gives a formula for the dimension of the sum of
two subspaces of a nite-dimensional vector space.
2.18 Theorem: If U1 and U2 are subspaces of a nite-dimensional
vector space, then
This formula for the

dimension of the sum
of two subspaces is
dim(U1 + U2 ) = dim U1 + dim U2 dim(U1 U2 ).
analogous to a familiar
counting formula: the
Proof: Let (u1 , . . . , um ) be a basis of U1 U2 ; thus dim(U1 U2 ) =

m. Because (u1 , . . . , um ) is a basis of U1 U2 , it is linearly independent
in U1 and hence can be extended to a basis (u1 , . . . , um , v1 , . . . , vj ) of U1
(by 2.12). Thus dim U1 = m + j. Also extend (u1 , . . . , um ) to a basis
(u1 , . . . , um , w1 , . . . , wk ) of U2 ; thus dim U2 = m + k.
We will show that (u1 , . . . , um , v1 , . . . , vj , w1 , . . . , wk ) is a basis of
U1 + U2 . This will complete the proof because then we will have
dim(U1 + U2 ) = m + j + k
= (m + j) + (m + k) m
= dim U1 + dim U2 dim(U1 U2 ).
Clearly span(u1 , . . . , um , v1 , . . . , vj , w1 , . . . , wk ) contains U1 and U2
and hence contains U1 + U2 . So to show that this list is a basis of
U1 + U2 we need only show that it is linearly independent. To prove
this, suppose
a1 u1 + + am um + b1 v1 + + bj vj + c1 w1 + + ck wk = 0,
where all the as, bs, and cs are scalars. We need to prove that all the
as, bs, and cs equal 0. The equation above can be rewritten as
c1 w1 + + ck wk = a1 u1 am um b1 v1 bj vj ,
which shows that c1 w1 + + ck wk U1 . All the ws are in U2 , so this
implies that c1 w1 + + ck wk U1 U2 . Because (u1 , . . . , um ) is a
basis of U1 U2 , we can write
c1 w1 + + ck wk = d1 u1 + + dm um
number of elements in
the union of two nite
sets equals the number
of elements in the rst
set, plus the number of
elements in the second
set, minus the number
of elements in the
intersection of the two
sets.
34
for some choice of scalars d1 , . . . , dm . But (u1 , . . . , um , w1 , . . . , wk )

is linearly independent, so the last equation implies that all the cs
(and ds) equal 0. Thus our original equation involving the as, bs, and
cs becomes
a1 u1 + + am um + b1 v1 + + bj vj = 0.
This equation implies that all the as and bs are 0 because the list
(u1 , . . . , um , v1 , . . . , vj ) is linearly independent. We now know that all
the as, bs, and cs equal 0, as desired.
The next proposition shows that dimension meshes well with direct
sums. This result will be useful in later chapters.
Recall that direct sum
is analogous to disjoint
2.19 Proposition: Suppose V is nite dimensional and U1 , . . . , Um

are subspaces of V such that
union. Thus 2.19 is

analogous to the
statement that if a
nite set B is written as
A1 Am and the
sum of the number of
elements in the As
V = U1 + + Um
2.20
and
2.21
dim V = dim U1 + + dim Um .
Then V = U1 Um .
equals the number of

elements in B, then the
union is a disjoint
union.
Proof: Choose a basis for each Uj . Put these bases together in

one list, forming a list that spans V (by 2.20) and has length dim V
(by 2.21). Thus this list is a basis of V (by 2.16), and in particular it is
linearly independent.
Now suppose that u1 U1 , . . . , um Um are such that
0 = u1 + + um .
We can write each uj as a linear combination of the basis vectors (chosen above) of Uj . Substituting these linear combinations into the expression above, we have written 0 as a linear combination of the basis
of V constructed above. Thus all the scalars used in this linear combination must be 0. Thus each uj = 0, which proves that V = U1 Um
(by 1.8).
Exercises
Exercises
1.
Prove that if (v1 , . . . , vn ) spans V , then so does the list

(v1 v2 , v2 v3 , . . . , vn1 vn , vn )
obtained by subtracting from each vector (except the last one)
the following vector.
2.
Prove that if (v1 , . . . , vn ) is linearly independent in V , then so is

the list
(v1 v2 , v2 v3 , . . . , vn1 vn , vn )
obtained by subtracting from each vector (except the last one)
the following vector.
3.
Suppose (v1 , . . . , vn ) is linearly independent in V and w V .

Prove that if (v1 + w, . . . , vn + w) is linearly dependent, then
w span(v1 , . . . , vn ).
4.
Suppose m is a positive integer. Is the set consisting of 0 and all

polynomials with coefcients in F and with degree equal to m a
subspace of P(F)?
5.
Prove that F is innite dimensional.
6.
Prove that the real vector space consisting of all continuous realvalued functions on the interval [0, 1] is innite dimensional.
7.
Prove that V is innite dimensional if and only if there is a sequence v1 , v2 , . . . of vectors in V such that (v1 , . . . , vn ) is linearly
independent for every positive integer n.
8.
Let U be the subspace of R5 dened by

U = {(x1 , x2 , x3 , x4 , x5 ) R5 : x1 = 3x2 and x3 = 7x4 }.
Find a basis of U .
9.
Prove or disprove: there exists a basis (p0 , p1 , p2 , p3 ) of P3 (F)

such that none of the polynomials p0 , p1 , p2 , p3 has degree 2.
10.
Suppose that V is nite dimensional, with dim V = n. Prove that

there exist one-dimensional subspaces U1 , . . . , Un of V such that
V = U1 Un .
35
36
11.
Suppose that V is nite dimensional and U is a subspace of V

such that dim U = dim V . Prove that U = V .
12.
Suppose that p0 , p1 , . . . , pm are polynomials in Pm (F) such that

pj (2) = 0 for each j. Prove that (p0 , p1 , . . . , pm ) is not linearly
independent in Pm (F).
13.
Suppose U and W are subspaces of R 8 such that dim U = 3,

dim W = 5, and U + W = R 8 . Prove that U W = {0}.
14.
Suppose that U and W are both ve-dimensional subspaces of R9 .

Prove that U W = {0}.
15.
You might guess, by analogy with the formula for the number
of elements in the union of three subsets of a nite set, that
if U1 , U2 , U3 are subspaces of a nite-dimensional vector space,
then
dim(U1 + U2 + U3 )
= dim U1 + dim U2 + dim U3
dim(U1 U2 ) dim(U1 U3 ) dim(U2 U3 )
+ dim(U1 U2 U3 ).
Prove this or give a counterexample.
16.
Prove that if V is nite dimensional and U1 , . . . , Um are subspaces

of V , then
dim(U1 + + Um ) dim U1 + + dim Um .
17.
Suppose V is nite dimensional. Prove that if U1 , . . . , Um are

subspaces of V such that V = U1 Um , then
This exercise deepens the analogy between direct sums of subspaces and disjoint unions of subsets. Specically, compare this
exercise to the following obvious statement: if a nite set is written as a disjoint union of subsets, then the number of elements in
the set equals the sum of the number of elements in the disjoint
subsets.
Chapter 3
Linear Maps
So far our attention has focused on vector spaces. No one gets excited about vector spaces. The interesting part of linear algebra is the
subject to which we now turnlinear maps.
Lets review our standing assumptions:
Recall also that V is a vector space over F.
In this chapter we will frequently need another vector space in addition to V . We will call this additional vector space W :
Lets agree that for the rest of this chapter
W will denote a vector space over F.
37
Chapter 3. Linear Maps
38
Definitions and Examples

Some mathematicians
use the term linear
transformation, which
means the same as
linear map.
A linear map from V to W is a function T : V W with the following

properties:
additivity
T (u + v) = T u + T v for all u, v V ;
homogeneity
T (av) = a(T v) for all a F and all v V .
Note that for linear maps we often use the notation T v as well as the
more standard functional notation T (v).
The set of all linear maps from V to W is denoted L(V , W ). Lets
look at some examples of linear maps. Make sure you verify that each
of the functions dened below is indeed a linear map:
zero
In addition to its other uses, we let the symbol 0 denote the function that takes each element of some vector space to the additive
identity of another vector space. To be specic, 0 L(V , W ) is
dened by
0v = 0.
Note that the 0 on the left side of the equation above is a function
from V to W , whereas the 0 on the right side is the additive identity in W . As usual, the context should allow you to distinguish
between the many uses of the symbol 0.
identity
The identity map, denoted I, is the function on some vector space
that takes each element to itself. To be specic, I L(V , V ) is
dened by
Iv = v.
differentiation
Dene T L(P(R), P(R)) by
T p = p .
The assertion that this function is a linear map is another way of
stating a basic result about differentiation: (f + g) = f + g and
(af ) = af whenever f , g are differentiable and a is a constant.
Denitions and Examples
39
integration
Dene T L(P(R), R) by
1
Tp =
p(x) dx.
The assertion that this function is linear is another way of stating

a basic result about integration: the integral of the sum of two
functions equals the sum of the integrals, and the integral of a
constant times a function equals the constant times the integral
of the function.
multiplication by x 2
Dene T L(P(R), P(R)) by
(T p)(x) = x 2 p(x)
Though linear maps are

pervasive throughout
mathematics, they are
not as ubiquitous as
for x R.
imagined by some
backward shift
Recall that F denotes the vector space of all sequences of elements of F. Dene T L(F , F ) by
confused students who

seem to think that cos
is a linear map from R
to R when they write
T (x1 , x2 , x3 , . . . ) = (x2 , x3 , . . . ).
from Fn to Fm
Dene T L(R3 , R2 ) by
T (x, y, z) = (2x y + 3z, 7x + 5y 6z).
More generally, let m and n be positive integers, let aj,k F for
j = 1, . . . , m and k = 1, . . . , n, and dene T L(Fn , Fm ) by
T (x1 , . . . , xn ) = (a1,1 x1 + +a1,n xn , . . . , am,1 x1 + +am,n xn ).
Later we will see that every linear map from Fn to Fm is of this
form.
Suppose (v1 , . . . , vn ) is a basis of V and T : V W is linear. If v V ,
then we can write v in the form
v = a1 v1 + + an vn .
The linearity of T implies that
identities such as
cos 2x = 2 cos x and
cos(x + y) =
cos x + cos y.
40
T v = a 1 T v1 + + a n T vn .
In particular, the values of T v1 , . . . , T vn determine the values of T on
arbitrary vectors in V .
Linear maps can be constructed that take on arbitrary values on a
basis. Specically, given a basis (v1 , . . . , vn ) of V and any choice of
vectors w1 , . . . , wn W , we can construct a linear map T : V W such
that T vj = wj for j = 1, . . . , n. There is no choice of how to do thiswe
must dene T by
T (a1 v1 + + an vn ) = a1 w1 + + an wn ,
where a1 , . . . , an are arbitrary elements of F. Because (v1 , . . . , vn ) is a
basis of V , the equation above does indeed dene a function T from V
to W . You should verify that the function T dened above is linear and
that T vj = wj for j = 1, . . . , n.
Now we will make L(V , W ) into a vector space by dening addition
and scalar multiplication on it. For S, T L(V , W ), dene a function
S + T L(V , W ) in the usual manner of adding functions:
(S + T )v = Sv + T v
for v V . You should verify that S + T is indeed a linear map from V
to W whenever S, T L(V , W ). For a F and T L(V , W ), dene a
function aT L(V , W ) in the usual manner of multiplying a function
by a scalar:
(aT )v = a(T v)
for v V . You should verify that aT is indeed a linear map from V to W
whenever a F and T L(V , W ). With the operations we have just
dened, L(V , W ) becomes a vector space (as you should verify). Note
that the additive identity of L(V , W ) is the zero linear map dened
earlier in this section.
Usually it makes no sense to multiply together two elements of a
vector space, but for some pairs of linear maps a useful product exists.
We will need a third vector space, so suppose U is a vector space over F.
If T L(U, V ) and S L(V , W ), then we dene ST L(U , W ) by
(ST )(v) = S(T v)
for v U. In other words, ST is just the usual composition S T of two
functions, but when both functions are linear, most mathematicians
Null Spaces and Ranges
41
write ST instead of S T . You should verify that ST is indeed a linear

map from U to W whenever T L(U, V ) and S L(V , W ). Note that
ST is dened only when T maps into the domain of S. We often call
ST the product of S and T . You should verify that it has most of the
usual properties expected of a product:
associativity
(T1 T2 )T3 = T1 (T2 T3 ) whenever T1 , T2 , and T3 are linear maps such
that the products make sense (meaning that T3 must map into the
domain of T2 , and T2 must map into the domain of T1 ).
identity
T I = T and IT = T whenever T L(V , W ) (note that in the rst
equation I is the identity map on V , and in the second equation I
is the identity map on W ).
distributive properties
(S1 + S2 )T = S1 T + S2 T and S(T1 + T2 ) = ST1 + ST2 whenever
T , T1 , T2 L(U, V ) and S, S1 , S2 L(V , W ).
Multiplication of linear maps is not commutative. In other words, it
is not necessarily true that ST = T S, even if both sides of the equation
make sense. For example, if T L(P(R), P(R)) is the differentiation
map dened earlier in this section and S L(P(R), P(R)) is the multiplication by x 2 map dened earlier in this section, then
((ST )p)(x) = x 2 p (x) but
((T S)p)(x) = x 2 p (x) + 2xp(x).
In other words, multiplying by x 2 and then differentiating is not the

same as differentiating and then multiplying by x 2 .

For T L(V , W ), the null space of T , denoted null T , is the subset
of V consisting of those vectors that T maps to 0:
Some mathematicians
use the term kernel
instead of null space.
null T = {v V : T v = 0}.
Lets look at a few examples from the previous section. In the differentiation example, we dened T L(P(R), P(R)) by T p = p . The
42
only functions whose derivative equals the zero function are the constant functions, so in this case the null space of T equals the set of
constant functions.
In the multiplication by x 2 example, we dened T L(P(R), P(R))
by (T p)(x) = x 2 p(x). The only polynomial p such that x 2 p(x) = 0
for all x R is the 0 polynomial. Thus in this case we have
null T = {0}.
In the backward shift example, we dened T L(F , F ) by
T (x1 , x2 , x3 , . . . ) = (x2 , x3 , . . . ).
Clearly T (x1 , x2 , x3 , . . . ) equals 0 if and only if x2 , x3 , . . . are all 0. Thus
in this case we have
null T = {(a, 0, 0, . . . ) : a F}.
The next proposition shows that the null space of any linear map is
a subspace of the domain. In particular, 0 is in the null space of every
linear map.
3.1
Proposition: If T L(V , W ), then null T is a subspace of V .
Proof: Suppose T L(V , W ). By additivity, we have

T (0) = T (0 + 0) = T (0) + T (0),
which implies that T (0) = 0. Thus 0 null T .
If u, v null T , then
T (u + v) = T u + T v = 0 + 0 = 0,
and hence u + v null T . Thus null T is closed under addition.
If u null T and a F, then
T (au) = aT u = a0 = 0,
and hence au null T . Thus null T is closed under scalar multiplication.
We have shown that null T contains 0 and is closed under addition
and scalar multiplication. Thus null T is a subspace of V .
A linear map T : V W is called injective if whenever u, v V

and T u = T v, we have u = v. The next proposition says that we
can check whether a linear map is injective by checking whether 0 is
the only vector that gets mapped to 0. As a simple application of this
proposition, we see that of the three linear maps whose null spaces we
computed earlier in this section (differentiation, multiplication by x 2 ,
and backward shift), only multiplication by x 2 is injective.
43
Many mathematicians
use the term
one-to-one, which
means the same as
injective.
3.2
Proposition: Let T L(V , W ). Then T is injective if and only
if null T = {0}.
Proof: First suppose that T is injective. We want to prove that
null T = {0}. We already know that {0} null T (by 3.1). To prove the
inclusion in the other direction, suppose v null T . Then
T (v) = 0 = T (0).
Because T is injective, the equation above implies that v = 0. Thus
null T = {0}, as desired.
To prove the implication in the other direction, now suppose that
null T = {0}. We want to prove that T is injective. To do this, suppose
u, v V and T u = T v. Then
0 = T u T v = T (u v).
Thus u v is in null T , which equals {0}. Hence u v = 0, which
implies that u = v. Hence T is injective, as desired.
For T L(V , W ), the range of T , denoted range T , is the subset of
W consisting of those vectors that are of the form T v for some v V :
range T = {T v : v V }.
For example, if T L(P(R), P(R)) is the differentiation map dened by
T p = p , then range T = P(R) because for every polynomial q P(R)
there exists a polynomial p P(R) such that p = q.
As another example, if T L(P(R), P(R)) is the linear map of
multiplication by x 2 dened by (T p)(x) = x 2 p(x), then the range
of T is the set of polynomials of the form a2 x 2 + + am x m , where
a2 , . . . , am R.
The next proposition shows that the range of any linear map is a
subspace of the target space.
Some mathematicians
use the word image,
which means the same
as range.
44
3.3
Proposition: If T L(V , W ), then range T is a subspace of W .
Proof: Suppose T L(V , W ). Then T (0) = 0 (by 3.1), which implies that 0 range T .
If w1 , w2 range T , then there exist v1 , v2 V such that T v1 = w1
and T v2 = w2 . Thus
T (v1 + v2 ) = T v1 + T v2 = w1 + w2 ,
and hence w1 +w2 range T . Thus range T is closed under addition.
If w range T and a F, then there exists v V such that T v = w.
Thus
T (av) = aT v = aw,
and hence aw range T . Thus range T is closed under scalar multiplication.
We have shown that range T contains 0 and is closed under addition
and scalar multiplication. Thus range T is a subspace of W .
Many mathematicians
use the term onto,
which means the same
as surjective.
A linear map T : V W is called surjective if its range equals W .

For example, the differentiation map T L(P(R), P(R)) dened by
T p = p is surjective because its range equals P(R). As another example, the linear map T L(P(R), P(R)) dened by (T p)(x) = x 2 p(x) is
not surjective because its range does not equal P(R). As a nal example, you should verify that the backward shift T L(F , F ) dened
by
T (x1 , x2 , x3 , . . . ) = (x2 , x3 , . . . )
is surjective.
Whether a linear map is surjective can depend upon what we are
thinking of as the target space. For example, x a positive integer m.
The differentiation map T L(Pm (R), Pm (R)) dened by T p = p
is not surjective because the polynomial x m is not in the range of T .
However, the differentiation map T L(Pm (R), Pm1 (R)) dened by
T p = p is surjective because its range equals Pm1 (R), which is now
the target space.
The next theorem, which is the key result in this chapter, states that
the dimension of the null space plus the dimension of the range of a
linear map on a nite-dimensional vector space equals the dimension
of the domain.
3.4
Theorem: If V is nite dimensional and T L(V , W ), then
range T is a nite-dimensional subspace of W and
dim V = dim null T + dim range T .
Proof: Suppose that V is a nite-dimensional vector space and
T L(V , W ). Let (u1 , . . . , um ) be a basis of null T ; thus dim null T = m.
The linearly independent list (u1 , . . . , um ) can be extended to a basis (u1 , . . . , um , w1 , . . . , wn ) of V (by 2.12). Thus dim V = m + n,
and to complete the proof, we need only show that range T is nite
dimensional and dim range T = n. We will do this by proving that
(T w1 , . . . , T wn ) is a basis of range T .
Let v V . Because (u1 , . . . , um , w1 , . . . , wn ) spans V , we can write
v = a1 u1 + + am um + b1 w1 + + bn wn ,
where the as and bs are in F. Applying T to both sides of this equation,
we get
T v = b 1 T w1 + + b n T wn ,
where the terms of the form T uj disappeared because each uj null T .
The last equation implies that (T w1 , . . . , T wn ) spans range T . In particular, range T is nite dimensional.
To show that (T w1 , . . . , T wn ) is linearly independent, suppose that
c1 , . . . , cn F and
c1 T w1 + + cn T wn = 0.
Then
T (c1 w1 + + cn wn ) = 0,
and hence
c1 w1 + + cn wn null T .
Because (u1 , . . . , um ) spans null T , we can write
c1 w1 + + cn wn = d1 u1 + + dm um ,
where the ds are in F. This equation implies that all the cs (and ds)
are 0 (because (u1 , . . . , um , w1 , . . . , wn ) is linearly independent). Thus
(T w1 , . . . , T wn ) is linearly independent and hence is a basis for range T ,
as desired.
45
46
Now we can show that no linear map from a nite-dimensional vector space to a smaller vector space can be injective, where smaller
is measured by dimension.
3.5
Corollary: If V and W are nite-dimensional vector spaces such
that dim V > dim W , then no linear map from V to W is injective.
Proof: Suppose V and W are nite-dimensional vector spaces such
that dim V > dim W . Let T L(V , W ). Then
dim null T = dim V dim range T
dim V dim W
> 0,
where the equality above comes from 3.4. We have just shown that
dim null T > 0. This means that null T must contain vectors other
than 0. Thus T is not injective (by 3.2).
The next corollary, which is in some sense dual to the previous corollary, shows that no linear map from a nite-dimensional vector space
to a bigger vector space can be surjective, where bigger is measured
by dimension.
3.6
Corollary: If V and W are nite-dimensional vector spaces such
that dim V < dim W , then no linear map from V to W is surjective.
Proof: Suppose V and W are nite-dimensional vector spaces such
that dim V < dim W . Let T L(V , W ). Then
dim range T = dim V dim null T
dim V
< dim W ,
where the equality above comes from 3.4. We have just shown that
dim range T < dim W . This means that range T cannot equal W . Thus
T is not surjective.
The last two corollaries have important consequences in the theory
of linear equations. To see this, x positive integers m and n, and let
aj,k F for j = 1, . . . , m and k = 1, . . . , n. Dene T : Fn Fm by
T (x1 , . . . , xn ) =
n

a1,k xk , . . . ,
k=1
47

am,k xk .
k=1
Now consider the equation T x = 0 (where x Fn and the 0 here is

the additive identity in Fm , namely, the list of length m consisting of
all 0s). Letting x = (x1 , . . . , xn ), we can rewrite the equation T x = 0
as a system of homogeneous equations:
n
context, means that the
a1,k xk = 0
k=1
constant term on the

right side of each
..
.
n
Homogeneous, in this
equation equals 0.
am,k xk = 0.
k=1
We think of the as as known; we are interested in solutions for the

variables x1 , . . . , xn . Thus we have m equations and n variables. Obviously x1 = = xn = 0 is a solution; the key question here is whether
any other solutions exist. In other words, we want to know if null T is
strictly bigger than {0}. This happens precisely when T is not injective
(by 3.2). From 3.5 we see that T is not injective if n > m. Conclusion:
a homogeneous system of linear equations in which there are more
variables than equations must have nonzero solutions.
With T as in the previous paragraph, now consider the equation
T x = c, where c = (c1 , . . . , cm ) Fm . We can rewrite the equation
T x = c as a system of inhomogeneous equations:
n
a1,k xk = c1
k=1
..
.
n
These results about

homogeneous systems
with more variables
than equations and
am,k xk = cm .
k=1
As before, we think of the as as known. The key question here is

whether for every choice of the constant terms c1 , . . . , cm F, there
exists at least one solution for the variables x1 , . . . , xn . In other words,
we want to know whether range T equals Fm . From 3.6 we see that T
is not surjective if n < m. Conclusion: an inhomogeneous system of
linear equations in which there are more equations than variables has
no solution for some choice of the constant terms.
inhomogeneous
systems with more
equations than
variables are often
proved using Gaussian
elimination. The
abstract approach
taken here leads to
cleaner proofs.
48

We have seen that if (v1 , . . . , vn ) is a basis of V and T : V W is
linear, then the values of T v1 , . . . , T vn determine the values of T on
arbitrary vectors in V . In this section we will see how matrices are used
as an efcient method of recording the values of the T vj s in terms of
a basis of W .
Let m and n denote positive integers. An m-by-n matrix is a rectangular array with m rows and n columns that looks like this:
3.7
a1,1
.
.
.
am,1
...
...
a1,n
..
.
.
am,n
Note that the rst index refers to the row number and the second index refers to the column number. Thus a3,2 refers to the entry in the
third row, second column of the matrix above. We will usually consider
matrices whose entries are elements of F.
Let T L(V , W ). Suppose that (v1 , . . . , vn ) is a basis of V and
(w1 , . . . , wm ) is a basis of W . For each k = 1, . . . , n, we can write T vk
uniquely as a linear combination of the ws:
3.8
T vk = a1,k w1 + + am,k wm ,
where aj,k F for j = 1, . . . , m. The scalars aj,k completely determine

the linear map T because a linear map is determined by its values on
a basis. The m-by-n matrix 3.7 formed by the as is called the matrix
of T with respect to the bases (v1 , . . . , vn ) and (w1 , . . . , wm ); we denote
it by

M T , (v1 , . . . , vn ), (w1 , . . . , wm ) .
If the bases (v1 , . . . , vn ) and (w1 , . . . , wm ) are clear from the context
(for example, if only one set of bases is in sight), we write just M(T )

instead of M T , (v1 , . . . , vn ), (w1 , . . . , wm ) .
As an aid to remembering how M(T ) is constructed from T , you
might write the basis vectors v1 , . . . , vn for the domain across the top
and the basis vectors w1 , . . . , wm for the target space along the left, as
follows:
v1
...
w1
..
.
wm
vk
a1,k
..
.
am,k
...
vn
49
Note that in the matrix above only the kth column is displayed (and thus
the second index of each displayed a is k). The kth column of M(T )
consists of the scalars needed to write T vk as a linear combination of
the ws. Thus the picture above should remind you that T vk is retrieved
from the matrix M(T ) by multiplying each entry in the kth column by
the corresponding w from the left column, and then adding up the
resulting vectors.
If T is a linear map from Fn to Fm , then unless stated otherwise you
should assume that the bases in question are the standard ones (where
the kth basis vector is 1 in the kth slot and 0 in all the other slots). If
you think of elements of Fm as columns of m numbers, then you can
think of the kth column of M(T ) as T applied to the kth basis vector.
For example, if T L(F2 , F3 ) is dened by
T (x, y) = (x + 3y, 2x + 5y, 7x + 9y),
then T (1, 0) = (1, 2, 7) and T (0, 1) = (3, 5, 9), so the matrix of T (with
respect to the standard bases) is the 3-by-2 matrix
2
7
5 .
9
Suppose we have bases (v1 , . . . , vn ) of V and (w1 , . . . , wm ) of W .

Thus for each linear map from V to W , we can talk about its matrix
(with respect to these bases, of course). Is the matrix of the sum of two
linear maps equal to the sum of the matrices of the two maps?
Right now this question does not make sense because, though we
have dened the sum of two linear maps, we have not dened the sum
of two matrices. Fortunately the obvious denition of the sum of two
matrices has the right properties. Specically, we dene addition of
matrices of the same size by adding corresponding entries in the matrices:
With respect to any

choice of bases, the
matrix of the 0 linear
map (the linear map
that takes every vector
to 0) consists of all 0s.
50
a1,1
.
.
.
am,1
...
...

b1,1
a1,n
.
..
+ .
.
.
am,n
bm,1
...
...
b1,n
..
bm,n
a1,1 + b1,1
..
=
.
am,1 + bm,1
...
...
a1,n + b1,n
..
.
.
am,n + bm,n
You should verify that with this denition of matrix addition,

3.9
M(T + S) = M(T ) + M(S)
whenever T , S L(V , W ).
Still assuming that we have some bases in mind, is the matrix of a
scalar times a linear map equal to the scalar times the matrix of the
linear map? Again the question does not make sense because we have
not dened scalar multiplication on matrices. Fortunately the obvious
denition again has the right properties. Specically, we dene the
product of a scalar and a matrix by multiplying each entry in the matrix
by the scalar:

ca1,1 . . . ca1,n
a1,1 . . . a1,n
.

..
..
..
=
.
c
.
.
.

..
am,1 . . . am,n
cam,1 . . . cam,n
You should verify that with this denition of scalar multiplication on
matrices,
3.10
M(cT ) = cM(T )
whenever c F and T L(V , W ).

Because addition and scalar multiplication have now been dened
for matrices, you should not be surprised that a vector space is about
to appear. We need only a bit of notation so that this new vector space
has a name. The set of all m-by-n matrices with entries in F is denoted
by Mat(m, n, F). You should verify that with addition and scalar multiplication dened as above, Mat(m, n, F) is a vector space. Note that
the additive identity in Mat(m, n, F) is the m-by-n matrix all of whose
entries equal 0.
Suppose (v1 , . . . , vn ) is a basis of V and (w1 , . . . , wm ) is a basis of W .
Suppose also that we have another vector space U and that (u1 , . . . , up )
51
is a basis of U. Consider linear maps S : U V and T : V W . The

composition T S is a linear map from U to W . How can M(T S) be
computed from M(T ) and M(S)? The nicest solution to this question
would be to have the following pretty relationship:
3.11
M(T S) = M(T )M(S).
So far, however, the right side of this equation does not make sense
because we have not yet dened the product of two matrices. We will
choose a denition of matrix multiplication that forces the equation
above to hold. Lets see how to do this.
Let
b1,1 . . . b1,p
a1,1 . . . a1,n
.
..
..
and M(S) = ..
M(T ) =
.
. .
.
..
am,1 . . . am,n
bn,1 . . . bn,p
For k {1, . . . , p}, we have
n
T Suk = T (
br ,k vr )
r =1
=
=
br ,k T vr
r =1
n
br ,k
r =1
aj,r wj
j=1
(
aj,r br ,k )wj .
j=1 r =1
Thus M(T S) is the m-by-p matrix whose entry in row j, column k

n
equals r =1 aj,r br ,k .
Now its clear how to dene matrix multiplication so that 3.11 holds.
Namely, if A is an m-by-n matrix with entries aj,k and B is an n-by-p
matrix with entries bj,k , then AB is dened to be the m-by-p matrix
whose entry in row j, column k, equals
n
You probably learned

this denition of matrix
multiplication in an
earlier course, although
you may not have seen
aj,r br ,k .
r =1
In other words, the entry in row j, column k, of AB is computed by

taking row j of A and column k of B, multiplying together corresponding entries, and then summing. Note that we dene the product of two
this motivation for it.
52
You should nd an
example to show that
matrix multiplication is
not commutative. In
other words, AB is not
necessarily equal to BA,
even when both are
dened.
matrices only when the number of columns of the rst matrix equals
the number of rows of the second matrix.
As an example of matrix multiplication, here we multiply together
a 3-by-2 matrix and a 2-by-4 matrix, obtaining a 3-by-4 matrix:

1 2
10 7
4 1
6 5 4 3
= 26 19 12 5 .
3 4
2 1 0 1
5 6
42 31 20 9
Suppose (v1 , . . . , vn ) is a basis of V . If v V , then there exist unique
scalars b1 , . . . , bn such that
3.12
v = b1 v1 + + bn vn .
The matrix of v, denoted M(v), is the n-by-1 matrix dened by
b1
.
3.13
M(v) =
.. .
bn
Usually the basis is obvious from the context, but when the basis needs

to be displayed explicitly use the notation M v, (v1 , . . . , vn ) instead
of M(v).
For example, the matrix of a vector x Fn with respect to the standard basis is obtained by writing the coordinates of x as the entries in
an n-by-1 matrix. In other words, if x = (x1 , . . . , xn ) Fn , then
x1
.
M(x) =
.. .
xn
The next proposition shows how the notions of the matrix of a linear
map, the matrix of a vector, and matrix multiplication t together. In
this proposition M(T v) is the matrix of the vector T v with respect to
the basis (w1 , . . . , wm ) and M(v) is the matrix of the vector v with respect to the basis (v1 , . . . , vn ), whereas M(T ) is the matrix of the linear
map T with respect to the bases (v1 , . . . , vn ) and (w1 , . . . , wm ).
3.14 Proposition: Suppose T L(V , W ) and (v1 , . . . , vn ) is a basis
of V and (w1 , . . . , wm ) is a basis of W . Then
M(T v) = M(T )M(v)
for every v V .
Invertibility
53
Proof: Let
M(T ) =
3.15
a1,1
..
.
am,1
...
...
a1,n
..
.
.
am,n
This means, we recall, that

T vk =
3.16
aj,k wj
j=1
for each k. Let v be an arbitrary vector in V , which we can write in the

form 3.12. Thus M(v) is given by 3.13. Now
T v = b 1 T v1 + + b n T v n
m
= b1
j=1
aj,1 wj + + bn
aj,n wj
j=1
(aj,1 b1 + + aj,n bn )wj ,
j=1
where the rst equality comes from 3.12 and the second equality comes
from 3.16. The last equation shows that M(T v), the m-by-1 matrix of
the vector T v with respect to the basis (w1 , . . . , wm ), is given by the
equation
a1,1 b1 + + a1,n bn
..
.
M(T v) =
.
am,1 b1 + + am,n bn
This formula, along with the formulas 3.15 and 3.13 and the denition
of matrix multiplication, shows that M(T v) = M(T )M(v).
Invertibility
A linear map T L(V , W ) is called invertible if there exists a linear
map S L(W , V ) such that ST equals the identity map on V and T S
equals the identity map on W . A linear map S L(W , V ) satisfying
ST = I and T S = I is called an inverse of T (note that the rst I is the
identity map on V and the second I is the identity map on W ).
If S and S are inverses of T , then
54
S = SI = S(T S ) = (ST )S = IS = S ,
so S = S . In other words, if T is invertible, then it has a unique
inverse, which we denote by T 1 . Rephrasing all this once more, if
T L(V , W ) is invertible, then T 1 is the unique element of L(W , V )
such that T 1 T = I and T T 1 = I. The following proposition characterizes the invertible linear maps.
3.17 Proposition: A linear map is invertible if and only if it is injective and surjective.
Proof: Suppose T L(V , W ). We need to show that T is invertible
if and only if it is injective and surjective.
First suppose that T is invertible. To show that T is injective, suppose that u, v V and T u = T v. Then
u = T 1 (T u) = T 1 (T v) = v,
so u = v. Hence T is injective.
We are still assuming that T is invertible. Now we want to prove
that T is surjective. To do this, let w W . Then w = T (T 1 w), which
shows that w is in the range of T . Thus range T = W , and hence T is
surjective, completing this direction of the proof.
Now suppose that T is injective and surjective. We want to prove
that T is invertible. For each w W , dene Sw to be the unique element of V such that T (Sw) = w (the existence and uniqueness of such
an element follow from the surjectivity and injectivity of T ). Clearly
T S equals the identity map on W . To prove that ST equals the identity
map on V , let v V . Then
T (ST v) = (T S)(T v) = I(T v) = T v.
This equation implies that ST v = v (because T is injective), and thus
ST equals the identity map on V . To complete the proof, we need to
show that S is linear. To do this, let w1 , w2 W . Then
T (Sw1 + Sw2 ) = T (Sw1 ) + T (Sw2 ) = w1 + w2 .
Thus Sw1 + Sw2 is the unique element of V that T maps to w1 + w2 . By
the denition of S, this implies that S(w1 + w2 ) = Sw1 + Sw2 . Hence
S satises the additive property required for linearity. The proof of
homogeneity is similar. Specically, if w W and a F, then
Invertibility
55
T (aSw) = aT (Sw) = aw.

Thus aSw is the unique element of V that T maps to aw. By the
denition of S, this implies that S(aw) = aSw. Hence S is linear, as
desired.
Two vector spaces are called isomorphic if there is an invertible
linear map from one vector space onto the other one. As abstract vector
spaces, two isomorphic spaces have the same properties. From this
viewpoint, you can think of an invertible linear map as a relabeling of
the elements of a vector space.
If two vector spaces are isomorphic and one of them is nite dimensional, then so is the other one. To see this, suppose that V and W
are isomorphic and that T L(V , W ) is an invertible linear map. If V
is nite dimensional, then so is W (by 3.4). The same reasoning, with
T replaced with T 1 L(W , V ), shows that if W is nite dimensional,
then so is V . Actually much more is true, as the following theorem
shows.
3.18 Theorem: Two nite-dimensional vector spaces are isomorphic
if and only if they have the same dimension.
Proof: First suppose V and W are isomorphic nite-dimensional
vector spaces. Thus there exists an invertible linear map T from V
onto W . Because T is invertible, we have null T = {0} and range T = W .
Thus dim null T = 0 and dim range T = dim W . The formula
dim V = dim null T + dim range T
(see 3.4) thus becomes the equation dim V = dim W , completing the
proof in one direction.
To prove the other direction, suppose V and W are nite-dimensional vector spaces with the same dimension. Let (v1 , . . . , vn ) be a
basis of V and (w1 , . . . , wn ) be a basis of W . Let T be the linear map
from V to W dened by
T (a1 v1 + + an vn ) = a1 w1 + + an wn .
Then T is surjective because (w1 , . . . , wn ) spans W , and T is injective
because (w1 , . . . , wn ) is linearly independent. Because T is injective and
The Greek word isos

means equal; the Greek
word morph means
shape. Thus
isomorphic literally
means equal shape.
56
surjective, it is invertible (see 3.17), and hence V and W are isomorphic,

as desired.
Because every
nite-dimensional
vector space is
isomorphic to some Fn ,
why bother with
abstract vector spaces?
To answer this
question, note that an
The last theorem implies that every nite-dimensional vector space

is isomorphic to some Fn . Specically, if V is a nite-dimensional vector
space and dim V = n, then V and Fn are isomorphic.
If (v1 , . . . , vn ) is a basis of V and (w1 , . . . , wm ) is a basis of W , then
for each T L(V , W ), we have a matrix M(T ) Mat(m, n, F). In other
words, once bases have been xed for V and W , M becomes a function
from L(V , W ) to Mat(m, n, F). Notice that 3.9 and 3.10 show that M is
a linear map. This linear map is actually invertible, as we now show.
investigation of Fn
would soon lead to
vector spaces that do
not equal Fn . For
3.19 Proposition: Suppose that (v1 , . . . , vn ) is a basis of V and

(w1 , . . . , wm ) is a basis of W . Then M is an invertible linear map between L(V , W ) and Mat(m, n, F).
example, we would
encounter the null
space and range of
linear maps, the set of
matrices Mat(n, n, F),
and the polynomials
Pn (F). Though each of
these vector spaces is
isomorphic to some
Fm , thinking of them
that way often adds
complexity but no new
insight.
Proof: We have already noted that M is linear, so we need only

prove that M is injective and surjective (by 3.17). Both are easy. Lets
begin with injectivity. If T L(V , W ) and M(T ) = 0, then T vk = 0
for k = 1, . . . , n. Because (v1 , . . . , vn ) is a basis of V , this implies that
T = 0. Thus M is injective (by 3.2).
To prove that M is surjective, let
A=
a1,1
..
.
am,1
...
...
a1,n
..
am,n
be a matrix in Mat(m, n, F). Let T be the linear map from V to W such

that
m
T vk =
aj,k wj
j=1
for k = 1, . . . , n. Obviously M(T ) equals A, and so the range of M

equals Mat(m, n, F), as desired.
An obvious basis of Mat(m, n, F) consists of those m-by-n matrices
that have 0 in all entries except for a 1 in one entry. There are mn such
matrices, so the dimension of Mat(m, n, F) equals mn.
Now we can determine the dimension of the vector space of linear
maps from one nite-dimensional vector space to another.
Invertibility
57
3.20 Proposition: If V and W are nite dimensional, then L(V , W )

is nite dimensional and
dim L(V , W ) = (dim V )(dim W ).
Proof: This follows from the equation dim Mat(m, n, F) = mn,
3.18, and 3.19.
A linear map from a vector space to itself is called an operator . If
we want to specify the vector space, we say that a linear map T : V V
is an operator on V . Because we are so often interested in linear maps
from a vector space into itself, we use the notation L(V ) to denote the
set of all operators on V . In other words, L(V ) = L(V , V ).
Recall from 3.17 that a linear map is invertible if it is injective and
surjective. For a linear map of a vector space into itself, you might
wonder whether injectivity alone, or surjectivity alone, is enough to
imply invertibility. On innite-dimensional vector spaces neither condition alone implies invertibility. We can see this from some examples
we have already considered. The multiplication by x 2 operator (from
P(R) to itself) is injective but not surjective. The backward shift (from
F to itself) is surjective but not injective. In view of these examples,
the next theorem is remarkableit states that for maps from a nitedimensional vector space to itself, either injectivity or surjectivity alone
implies the other condition.
3.21 Theorem: Suppose V is nite dimensional. If T L(V ), then
the following are equivalent:
(a)
T is invertible;
(b)
T is injective;
(c)
T is surjective.
Proof: Suppose T L(V ). Clearly (a) implies (b).

Now suppose (b) holds, so that T is injective. Thus null T = {0}
(by 3.2). From 3.4 we have
dim range T = dim V dim null T
= dim V ,
which implies that range T equals V (see Exercise 11 in Chapter 2). Thus
T is surjective. Hence (b) implies (c).
The deepest and most

important parts of
linear algebra, as well
as most of the rest of
this book, deal with
operators.
58
Now suppose (c) holds, so that T is surjective. Thus range T = V .

From 3.4 we have
dim null T = dim V dim range T
= 0,
which implies that null T equals {0}. Thus T is injective (by 3.2), and
so T is invertible (we already knew that T was surjective). Hence (c)
implies (a), completing the proof.
Exercises
59
Exercises
1.
Show that every linear map from a one-dimensional vector space

to itself is multiplication by some scalar. More precisely, prove
that if dim V = 1 and T L(V , V ), then there exists a F such
that T v = av for all v V .
2.
Give an example of a function f : R 2 R such that

f (av) = af (v)
for all a R and all v R 2 but f is not linear.
3.
4.
Exercise 2 shows that

homogeneity alone is
not enough to imply
that a function is a
linear map. Additivity
Suppose that V is nite dimensional. Prove that any linear map

on a subspace of V can be extended to a linear map on V . In
other words, show that if U is a subspace of V and S L(U , W ),
then there exists T L(V , W ) such that T u = Su for all u U.
alone is also not
Suppose that T is a linear map from V to F. Prove that if u V

is not in null T , then
advanced tools that are
enough to imply that a

function is a linear
map, although the
proof of this involves
beyond the scope of
this book.
V = null T {au : a F}.

5.
Suppose that T L(V , W ) is injective and (v1 , . . . , vn ) is linearly

independent in V . Prove that (T v1 , . . . , T vn ) is linearly independent in W .
6.
Prove that if S1 , . . . , Sn are injective linear maps such that S1 . . . Sn

makes sense, then S1 . . . Sn is injective.
7.
Prove that if (v1 , . . . , vn ) spans V and T L(V , W ) is surjective,

then (T v1 , . . . , T vn ) spans W .
8.
Suppose that V is nite dimensional and that T L(V , W ). Prove

that there exists a subspace U of V such that U null T = {0}
and range T = {T u : u U}.
9.
Prove that if T is a linear map from F4 to F2 such that

null T = {(x1 , x2 , x3 , x4 ) F4 : x1 = 5x2 and x3 = 7x4 },
then T is surjective.
60
10.
Prove that there does not exist a linear map from F5 to F2 whose
null space equals
{(x1 , x2 , x3 , x4 , x5 ) F5 : x1 = 3x2 and x3 = x4 = x5 }.
11.
Prove that if there exists a linear map on V whose null space and
range are both nite dimensional, then V is nite dimensional.
12.
Suppose that V and W are both nite dimensional. Prove that

there exists a surjective linear map from V onto W if and only if
dim W dim V .
13.
Suppose that V and W are nite dimensional and that U is a

subspace of V . Prove that there exists T L(V , W ) such that
null T = U if and only if dim U dim V dim W .
14.
Suppose that W is nite dimensional and T L(V , W ). Prove

that T is injective if and only if there exists S L(W , V ) such
that ST is the identity map on V .
15.
Suppose that V is nite dimensional and T L(V , W ). Prove

that T is surjective if and only if there exists S L(W , V ) such
that T S is the identity map on W .
16.
Suppose that U and V are nite-dimensional vector spaces and

that S L(V , W ), T L(U , V ). Prove that
dim null ST dim null S + dim null T .
17.
Prove that the distributive property holds for matrix addition

and matrix multiplication. In other words, suppose A, B, and C
are matrices whose sizes are such that A(B + C) makes sense.
Prove that AB + AC makes sense and that A(B + C) = AB + AC.
18.
Prove that matrix multiplication is associative. In other words,

suppose A, B, and C are matrices whose sizes are such that
(AB)C makes sense. Prove that A(BC) makes sense and that
(AB)C = A(BC).
Exercises
19.
61
Suppose T L(Fn , Fm ) and that
a1,1
.
M(T ) = ..
am,1
This exercise shows
...
...
a1,n
..
,
.
am,n
that T has the form

promised on page 39.
where we are using the standard bases. Prove that

T (x1 , . . . , xn ) = (a1,1 x1 + +a1,n xn , . . . , am,1 x1 + +am,n xn )
for every (x1 , . . . , xn ) Fn .
20.
Suppose (v1 , . . . , vn ) is a basis of V . Prove that the function

T : V Mat(n, 1, F) dened by
T v = M(v)
is an invertible linear map of V onto Mat(n, 1, F); here M(v) is
the matrix of v V with respect to the basis (v1 , . . . , vn ).
21.
Prove that every linear map from Mat(n, 1, F) to Mat(m, 1, F) is

given by a matrix multiplication. In other words, prove that if
T L(Mat(n, 1, F), Mat(m, 1, F)), then there exists an m-by-n
matrix A such that T B = AB for every B Mat(n, 1, F).
22.
Suppose that V is nite dimensional and S, T L(V ). Prove that

ST is invertible if and only if both S and T are invertible.
23.
Suppose that V is nite dimensional and S, T L(V ). Prove that

ST = I if and only if T S = I.
24.
Suppose that V is nite dimensional and T L(V ). Prove that

T is a scalar multiple of the identity if and only if ST = T S for
every S L(V ).
25.
Prove that if V is nite dimensional with dim V > 1, then the set
of noninvertible operators on V is not a subspace of L(V ).
11:45 am, Jan 11, 2005
62
26.
Suppose n is a positive integer and ai,j F for i, j = 1, . . . , n.

Prove that the following are equivalent:
(a)
The trivial solution x1 = = xn = 0 is the only solution

to the homogeneous system of equations
n
a1,k xk = 0
k=1
..
.
n
an,k xk = 0.
k=1
(b)
For every c1 , . . . , cn F, there exists a solution to the system of equations

n
a1,k xk = c1
k=1
..
.
n
an,k xk = cn .
k=1
Note that here we have the same number of equations as variables.
Chapter 4
Polynomials
This short chapter contains no linear algebra. It does contain the

background material on polynomials that we will need in our study
of linear maps from a vector space to itself. Many of the results in
this chapter will already be familiar to you from other courses; they
are included here for completeness. Because this chapter is not about
linear algebra, your instructor may go through it rapidly. You may not
be asked to scrutinize all the proofs. Make sure, however, that you
at least read and understand the statements of all the results in this
chapterthey will be used in the rest of the book.
63
Chapter 4. Polynomials
64
Degree
Recall that a function p : F F is called a polynomial with coefcients in F if there exist a0 , . . . , am F such that
p(z) = a0 + a1 z + a2 z2 + + am zm
When necessary, use

the obvious arithmetic
with . For example,
< m and
+ m = for
every integer m. The 0
polynomial is declared
to have degree so
for all z F. If p can be written in the form above with am = 0, then we

say that p has degree m. If all the coefcients a0 , . . . , am equal 0, then
we say that p has degree . For all we know at this stage, a polynomial
may have more than one degree because we have not yet proved that
the coefcients in the equation above are uniquely determined by the
function p.
Recall that P(F) denotes the vector space of all polynomials with
coefcients in F and that Pm (F) is the subspace of P(F) consisting of
the polynomials with coefcients in F and degree at most m. A number
F is called a root of a polynomial p P(F) if
that exceptions are not
p() = 0.
needed for various

reasonable results. For
example, the degree of
pq equals the degree of
p plus the degree of q
even if p = 0.
Roots play a crucial role in the study of polynomials. We begin by

showing that is a root of p if and only if p is a polynomial multiple
of z .
4.1
Proposition: Suppose p P(F) is a polynomial with degree
m 1. Let F. Then is a root of p if and only if there is a
polynomial q P(F) with degree m 1 such that
4.2
p(z) = (z )q(z)
for all z F.
Proof: One direction is obvious. Namely, suppose there is a polynomial q P(F) such that 4.2 holds. Then
p() = ( )q() = 0,
and hence is a root of p, as desired.
To prove the other direction, suppose that F is a root of p. Let
a0 , . . . , am F be such that am = 0 and
p(z) = a0 + a1 z + a2 z2 + + am zm
Degree
65
for all z F. Because p() = 0, we have

0 = a0 + a1 + a2 2 + + am m .
Subtracting the last two equations, we get
p(z) = a1 (z ) + a2 (z2 2 ) + + am (zm m )
for all z F. For each j = 2, . . . , m, we can write
zj j = (z )qj1 (z)
for all z F, where qj1 is a polynomial with degree j 1 (specically,
take qj1 (z) = zj1 + zj2 + + zj2 + j1 ). Thus
p(z) = (z ) (a1 + a2 q2 (z) + + am qm1 (z))

q(z)
for all z F. Clearly q is a polynomial with degree m 1, as desired.

Now we can prove that polynomials do not have too many roots.
4.3
Corollary: Suppose p P(F) is a polynomial with degree m 0.
Then p has at most m distinct roots in F.
Proof: If m = 0, then p(z) = a0 = 0 and so p has no roots. If
m = 1, then p(z) = a0 + a1 z, with a1 = 0, and p has exactly one
root, namely, a0 /a1 . Now suppose m > 1. We use induction on m,
assuming that every polynomial with degree m 1 has at most m 1
distinct roots. If p has no roots in F, then we are done. If p has a root
F, then by 4.1 there is a polynomial q with degree m 1 such that
p(z) = (z )q(z)
for all z F. The equation above shows that if p(z) = 0, then either
z = or q(z) = 0. In other words, the roots of p consist of and the
roots of q. By our induction hypothesis, q has at most m 1 distinct
roots in F. Thus p has at most m distinct roots in F.
The next result states that if a polynomial is identically 0, then all
its coefcients must be 0.
66
4.4
Corollary: Suppose a0 , . . . , am F. If
a0 + a1 z + a2 z2 + + am zm = 0
for all z F, then a0 = = am = 0.

Proof: Suppose a0 +a1 z+a2 z2 + +am zm equals 0 for all z F.
By 4.3, no nonnegative integer can be the degree of this polynomial.
Thus all the coefcients equal 0.
The corollary above implies that (1, z, . . . , zm ) is linearly independent in P(F) for every nonnegative integer m. We had noted this earlier
(in Chapter 2), but now we have a complete proof. This linear independence implies that each polynomial can be represented in only one way
as a linear combination of functions of the form zj . In particular, the
degree of a polynomial is unique.
If p and q are nonnegative integers, with p = 0, then there exist
nonnegative integers s and r such that
q = sp + r .
and r < p. Think of dividing q by p, getting s with remainder r . Our
next task is to prove an analogous result for polynomials.
Let deg p denote the degree of a polynomial p. The next result is
often called the division algorithm, though as stated here it is not really
an algorithm, just a useful lemma.
Think of 4.6 as giving
the remainder r when
q is divided by p.
4.5
Division Algorithm: Suppose p, q P(F), with p = 0. Then
there exist polynomials s, r P(F) such that
4.6
q = sp + r
and deg r < deg p.

Proof: Choose s P(F) such that q sp has degree as small as
possible. Let r = q sp. Thus 4.6 holds, and all that remains is to
show that deg r < deg p. Suppose that deg r deg p. If c F and j is
a nonnegative integer, then
q (s + czj )p = r czj p.
Choose j and c so that the polynomial on the right side of this equation
has degree less than deg r (specically, take j = deg r deg p and then
Complex Coefcients
67
choose c so that the coefcients of zdeg r in r and in czj p are equal).

This contradicts our choice of s as the polynomial that produces the
smallest degree for expressions of the form q sp, completing the
proof.
Complex Coefficients
So far we have been handling polynomials with complex coefcients
and polynomials with real coefcients simultaneously through our convention that F denotes R or C. Now we will see some differences between these two cases. In this section we treat polynomials with complex coefcients. In the next section we will use our results about polynomials with complex coefcients to prove corresponding results for
polynomials with real coefcients.
Though this chapter contains no linear algebra, the results so far
have nonetheless been proved using algebra. The next result, though
called the fundamental theorem of algebra, requires analysis for its
proof. The short proof presented here uses tools from complex analysis. If you have not had a course in complex analysis, this proof will
almost certainly be meaningless to you. In that case, just accept the
fundamental theorem of algebra as something that we need to use but
whose proof requires more advanced tools that you may learn in later
courses.
4.7
Fundamental Theorem of Algebra: Every nonconstant polynomial with complex coefcients has a root.
This is an existence
theorem. The quadratic
formula gives the roots
Proof: Let p be a nonconstant polynomial with complex coefcients. Suppose that p has no roots. Then 1/p is an analytic function
on C. Furthermore, p(z) as z , which implies that 1/p 0 as
z . Thus 1/p is a bounded analytic function on C. By Liouvilles theorem, any such function must be constant. But if 1/p is constant, then
p is constant, contradicting our assumption that p is nonconstant.
explicitly for
polynomials of
degree 2. Similar but
more complicated
formulas exist for
polynomials of degree
3 and 4. No such
The fundamental theorem of algebra leads to the following factorization result for polynomials with complex coefcients. Note that
in this factorization, the numbers 1 , . . . , m are precisely the roots
of p, for these are the only values of z for which the right side of 4.9
equals 0.
formulas exist for

polynomials of degree
5 and above.
68
4.8
Corollary: If p P(C) is a nonconstant polynomial, then p
has a unique factorization (except for the order of the factors) of the
form
4.9
p(z) = c(z 1 ) . . . (z m ),
where c, 1 , . . . , m C.
Proof: Let p P(C) and let m denote the degree of p. We will use
induction on m. If m = 1, then clearly the desired factorization exists
and is unique. So assume that m > 1 and that the desired factorization
exists and is unique for all polynomials of degree m 1.
First we will show that the desired factorization of p exists. By the
fundamental theorem of algebra (4.7), p has a root . By 4.1, there is a
polynomial q with degree m 1 such that
p(z) = (z )q(z)
for all z C. Our induction hypothesis implies that q has the desired
factorization, which when plugged into the equation above gives the
desired factorization of p.
Now we turn to the question of uniqueness. Clearly c is uniquely
determined by 4.9it must equal the coefcient of zm in p. So we need
only show that except for the order, there is only one way to choose
1 , . . . , m . If
(z 1 ) . . . (z m ) = (z 1 ) . . . (z m )
for all z C, then because the left side of the equation above equals 0
when z = 1 , one of the s on the right side must equal 1 . Relabeling,
we can assume that 1 = 1 . Now for z = 1 , we can divide both sides
of the equation above by z 1 , getting
(z 2 ) . . . (z m ) = (z 2 ) . . . (z m )
for all z C except possibly z = 1 . Actually the equation above
must hold for all z C because otherwise by subtracting the right side
from the left side we would get a nonzero polynomial that has innitely
many roots. The equation above and our induction hypothesis imply
that except for the order, the s are the same as the s, completing
the proof of the uniqueness.
Real Coefcients
69
Real Coefficients
Before discussing polynomials with real coefcients, we need to
learn a bit more about the complex numbers.
Suppose z = a + bi, where a and b are real numbers. Then a is
called the real part of z, denoted Re z, and b is called the imaginary
part of z, denoted Im z. Thus for every complex number z, we have
z = Re z + (Im z)i.
, is dened by
The complex conjugate of z C, denoted z
= Re z (Im z)i.
z
For example, 2 + 3i = 2 3i.
The absolute value of a complex number z, denoted |z|, is dened
by

|z| = (Re z)2 + (Im z)2 .
For example, |1 + 2i| = 5. Note that |z| is always a nonnegative

number.
You should verify that the real and imaginary parts, absolute value,
and complex conjugate have the following properties:
additivity of real part
Re(w + z) = Re w + Re z for all w, z C;
additivity of imaginary part
Im(w + z) = Im w + Im z for all w, z C;
sum of z and z
= 2 Re z for all z C;
z+z
difference of z and z
= 2(Im z)i for all z C;
zz
product of z and z
= |z|2 for all z C;
zz
additivity of complex conjugate
+z
for all w, z C;
w +z =w
multiplicativity of complex conjugate
z
for all w, z C;
wz = w
if and
Note that z = z
only if z is a real
number.
70
conjugate of conjugate
= z for all z C;
z
multiplicativity of absolute value
|wz| = |w| |z| for all w, z C.
In the next result, we need to think of a polynomial with real coefcients as an element of P(C). This makes sense because every real
number is also a complex number.
A polynomial with real
coefcients may have
4.10 Proposition: Suppose p is a polynomial with real coefcients.
If C is a root of p, then so is .
no real roots. For

example, the
Proof: Let
p(z) = a0 + a1 z + + am zm ,
polynomial 1 + x 2 has
no real roots. The
failure of the
where a0 , . . . , am are real numbers. Suppose C is a root of p. Then
fundamental theorem
a0 + a1 + + am m = 0.
of algebra for R
accounts for the
Take the complex conjugate of both sides of this equation, obtaining
differences between
+ + am
m = 0,
a0 + a1
operators on real and

complex vector spaces,
as we will see in later
chapters.
where we have used some of the basic properties of complex conjuga is a root of p.
tion listed earlier. The equation above shows that
We want to prove a factorization theorem for polynomials with real
coefcients. To do this, we begin by characterizing the polynomials
with real coefcients and degree 2 that can be written as the product
of two polynomials with real coefcients and degree 1.
Think about the

connection between the
4.11 Proposition: Let , R. Then there is a polynomial factorization of the form
quadratic formula and

this proposition.
4.12
x 2 + x + = (x 1 )(x 2 ),
with 1 , 2 R, if and only if 2 4.

Proof: Notice that
4.13
x 2 + x + = (x +
2
2
) + (
).
2
4
Real Coefcients
71
First suppose that 2 < 4. Then clearly the right side of the
equation above is positive for every x R, and hence the polynomial
x 2 + x + has no real roots. Thus no factorization of the form 4.12,
with 1 , 2 R, can exist.
Conversely, now suppose that 2 4. Thus there is a real number
2
c such that c 2 = 4 . From 4.13, we have
2
) c2
2
= (x + + c)(x + c),
2
2
x 2 + x + = (x +
which gives the desired factorization.

In the following theorem, each term of the form x 2 + j x + j , with
j < 4j , cannot be factored into the product of two polynomials with
real coefcients and degree 1 (by 4.11). Note that in the factorization
below, the numbers 1 , . . . , m are precisely the real roots of p, for these
are the only real values of x for which the right side of the equation
below equals 0.
2
4.14 Theorem: If p P(R) is a nonconstant polynomial, then p

has a unique factorization (except for the order of the factors) of the
form
p(x) = c(x 1 ) . . . (x m )(x 2 + 1 x + 1 ) . . . (x 2 + M x + M ),
where c, 1 , . . . , m R and (1 , 1 ), . . . , (M , M ) R 2 with j 2 < 4j
for each j.
Proof: Let p P(R) be a nonconstant polynomial. We can think
of p as an element of P(C) (because every real number is a complex
number). The idea of the proof is to use the factorization 4.8 of p as a
polynomial with complex coefcients. Complex but nonreal roots of p
come in pairs; see 4.10. Thus if the factorization of p as an element
of P(C) includes terms of the form (x ) with a nonreal complex
is also a term in the factorization. Combining
number, then (x )
these two terms, we get a quadratic term of the required form.
The idea sketched in the paragraph above almost provides a proof
of the existence of our desired factorization. However, we need to
be careful about one point. Suppose is a nonreal complex number
Here either m or M
may equal 0.
72
and (x ) is a term in the factorization of p as an element of P(C).

also appears as a term in the
We are guaranteed by 4.10 that (x )
factorization, but 4.10 does not state that these two factors appear
the same number of times, as needed to make the idea above work.
However, all is well. We can write
p(x) = (x )(x )q(x)

2
= x 2(Re )x + ||2 q(x)
Here we are not

dividing by 0 because
the roots of
x 2 2(Re )x + ||2
neither of
are and ,
which is real.
for some polynomial q P(C) with degree two less than the degree
of p. If we can prove that q has real coefcients, then, by using induction on the degree of p, we can conclude that (x ) appears in the
factorization of p exactly as many times as (x ).

To prove that q has real coefcients, we solve the equation above
for q, getting
p(x)
q(x) = 2
x 2(Re )x + ||2
for all x R. The equation above implies that q(x) R for all x R.
Writing
q(x) = a0 + a1 x + + an2 x n2 ,
where a0 , . . . , an2 C, we thus have
0 = Im q(x) = (Im a0 ) + (Im a1 )x + + (Im an2 )x n2
for all x R. This implies that Im a0 , . . . , Im an2 all equal 0 (by 4.4).
Thus all the coefcients of q are real, as desired, and hence the desired
factorization exists.
Now we turn to the question of uniqueness of our factorization. A
factor of p of the form x 2 +x + with 2 < 4 can be uniquely written
with C. A moments thought shows that two
as (x )(x )
different factorizations of p as an element of P(R) would lead to two
different factorizations of p as an element of P(C), contradicting 4.8.
Exercises
Exercises
1.
Suppose m and n are positive integers with m n. Prove that

there exists a polynomial p Pn (F) with exactly m distinct
roots.
2.
Suppose that z1 , . . . , zm+1 are distinct elements of F and that

w1 , . . . , wm+1 F. Prove that there exists a unique polynomial
p Pm (F) such that
p(zj ) = wj
for j = 1, . . . , m + 1.
3.
Prove that if p, q P(F), with p = 0, then there exist unique

polynomials s, r P(F) such that
q = sp + r
and deg r < deg p. In other words, add a uniqueness statement
to the division algorithm (4.5).
4.
Suppose p P(C) has degree m. Prove that p has m distinct

roots if and only if p and its derivative p have no roots in common.
5.
Prove that every polynomial with odd degree and real coefcients
has a real root.
73
Chapter 5
Eigenvalues and Eigenvectors
In Chapter 3 we studied linear maps from one vector space to another vector space. Now we begin our investigation of linear maps from
a vector space to itself. Their study constitutes the deepest and most
important part of linear algebra. Most of the key results in this area
do not hold for innite-dimensional vector spaces, so we work only on
nite-dimensional vector spaces. To avoid trivialities we also want to
eliminate the vector space {0} from consideration. Thus we make the
following assumption:
Lets agree that for the rest of the book
V will denote a nite-dimensional, nonzero vector space over F.
75
Chapter 5. Eigenvalues and Eigenvectors
76
Invariant Subspaces
In this chapter we develop the tools that will help us understand the
structure of operators. Recall that an operator is a linear map from a
vector space to itself. Recall also that we denote the set of operators
on V by L(V ); in other words, L(V ) = L(V , V ).
Lets see how we might better understand what an operator looks
like. Suppose T L(V ). If we have a direct sum decomposition
5.1
The most famous

unsolved problem in
functional analysis is
called the invariant
subspace problem. It
deals with invariant
subspaces of operators
on innite-dimensional
vector spaces.
V = U1 U m ,
where each Uj is a proper subspace of V , then to understand the behavior of T , we need only understand the behavior of each T |Uj ; here
T |Uj denotes the restriction of T to the smaller domain Uj . Dealing
with T |Uj should be easier than dealing with T because Uj is a smaller
vector space than V . However, if we intend to apply tools useful in the
study of operators (such as taking powers), then we have a problem:
T |Uj may not map Uj into itself; in other words, T |Uj may not be an
operator on Uj . Thus we are led to consider only decompositions of
the form 5.1 where T maps each Uj into itself.
The notion of a subspace that gets mapped into itself is sufciently
important to deserve a name. Thus, for T L(V ) and U a subspace
of V , we say that U is invariant under T if u U implies T u U.
In other words, U is invariant under T if T |U is an operator on U. For
example, if T is the operator of differentiation on P7 (R), then P4 (R)
(which is a subspace of P7 (R)) is invariant under T because the derivative of any polynomial of degree at most 4 is also a polynomial with
degree at most 4.
Lets look at some easy examples of invariant subspaces. Suppose
T L(V ). Clearly {0} is invariant under T . Also, the whole space V is
obviously invariant under T . Must T have any invariant subspaces other
than {0} and V ? Later we will see that this question has an afrmative
answer for operators on complex vector spaces with dimension greater
than 1 and also for operators on real vector spaces with dimension
greater than 2.
If T L(V ), then null T is invariant under T (proof: if u null T ,
then T u = 0, and hence T u null T ). Also, range T is invariant under T
(proof: if u range T , then T u is also in range T , by the denition of
range). Although null T and range T are invariant under T , they do not
necessarily provide easy answers to the question about the existence
Invariant Subspaces
77
of invariant subspaces other than {0} and V because null T may equal
{0} and range T may equal V (this happens when T is invertible).
We will return later to a deeper study of invariant subspaces. Now
we turn to an investigation of the simplest possible nontrivial invariant
subspacesinvariant subspaces with dimension 1.
How does an operator behave on an invariant subspace of dimension 1? Subspaces of V of dimension 1 are easy to describe. Take any
nonzero vector u V and let U equal the set of all scalar multiples
of u:
5.2
U = {au : a F}.
Then U is a one-dimensional subspace of V , and every one-dimensional

subspace of V is of this form. If u V and the subspace U dened
by 5.2 is invariant under T L(V ), then T u must be in U, and hence
there must be a scalar F such that T u = u. Conversely, if u
is a nonzero vector in V such that T u = u for some F, then the
subspace U dened by 5.2 is a one-dimensional subspace of V invariant
under T .
The equation
5.3
These subspaces are

loosely connected to
the subject of Herbert
Marcuses well-known
book One-Dimensional
Man.
T u = u,
which we have just seen is intimately connected with one-dimensional

invariant subspaces, is important enough that the vectors u and scalars
satisfying it are given special names. Specically, a scalar F
is called an eigenvalue of T L(V ) if there exists a nonzero vector
u V such that T u = u. We must require u to be nonzero because
with u = 0 every scalar F satises 5.3. The comments above show
that T has a one-dimensional invariant subspace if and only if T has
an eigenvalue.
The equation T u = u is equivalent to (T I)u = 0, so is an
eigenvalue of T if and only if T I is not injective. By 3.21, is an
eigenvalue of T if and only if T I is not invertible, and this happens
if and only if T I is not surjective.
Suppose T L(V ) and F is an eigenvalue of T . A vector u V
is called an eigenvector of T (corresponding to ) if T u = u. Because
5.3 is equivalent to (T I)u = 0, we see that the set of eigenvectors
of T corresponding to equals null(T I). In particular, the set of
eigenvectors of T corresponding to is a subspace of V .
The regrettable word

eigenvalue is
half-German,
half-English. The
German adjective eigen
means own in the sense
of characterizing some
intrinsic property.
Some mathematicians
use the term
characteristic value
instead of eigenvalue.
78
Some texts dene
eigenvectors as we
have, except that 0 is
declared not to be an
eigenvector. With the
Lets look at some examples of eigenvalues and eigenvectors. If

a F, then aI has only one eigenvalue, namely, a, and every vector is
an eigenvector for this eigenvalue.
For a more complicated example, consider the operator T L(F2 )
dened by
denition used here,

the set of eigenvectors
corresponding to a
xed eigenvalue is a
subspace.
5.4
T (w, z) = (z, w).
If F = R, then this operator has a nice geometric interpretation: T is

just a counterclockwise rotation by 90 about the origin in R 2 . An
operator has an eigenvalue if and only if there exists a nonzero vector
in its domain that gets sent by the operator to a scalar multiple of itself.
The rotation of a nonzero vector in R 2 obviously never equals a scalar
multiple of itself. Conclusion: if F = R, the operator T dened by 5.4
has no eigenvalues. However, if F = C, the story changes. To nd
eigenvalues of T , we must nd the scalars such that
T (w, z) = (w, z)
has some solution other than w = z = 0. For T dened by 5.4, the
equation above is equivalent to the simultaneous equations
5.5
z = w,
w = z.
Substituting the value for w given by the second equation into the rst
equation gives
z = 2 z.
Now z cannot equal 0 (otherwise 5.5 implies that w = 0; we are looking
for solutions to 5.5 where (w, z) is not the 0 vector), so the equation
above leads to the equation
1 = 2 .
The solutions to this equation are = i or = i. You should be
able to verify easily that i and i are eigenvalues of T . Indeed, the
eigenvectors corresponding to the eigenvalue i are the vectors of the
form (w, wi), with w C, and the eigenvectors corresponding to the
eigenvalue i are the vectors of the form (w, wi), with w C.
Now we show that nonzero eigenvectors corresponding to distinct
eigenvalues are linearly independent.
Invariant Subspaces
5.6
Theorem: Let T L(V ). Suppose 1 , . . . , m are distinct eigenvalues of T and v1 , . . . , vm are corresponding nonzero eigenvectors.
Then (v1 , . . . , vm ) is linearly independent.
Proof: Suppose (v1 , . . . , vm ) is linearly dependent. Let k be the
smallest positive integer such that
5.7
vk span(v1 , . . . , vk1 );
the existence of k with this property follows from the linear dependence
lemma (2.4). Thus there exist a1 , . . . , ak1 F such that
5.8
vk = a1 v1 + + ak1 vk1 .
Apply T to both sides of this equation, getting

k vk = a1 1 v1 + + ak1 k1 vk1 .
Multiply both sides of 5.8 by k and then subtract the equation above,
getting
0 = a1 (k 1 )v1 + + ak1 (k k1 )vk1 .
Because we chose k to be the smallest positive integer satisfying 5.7,
(v1 , . . . , vk1 ) is linearly independent. Thus the equation above implies
that all the as are 0 (recall that k is not equal to any of 1 , . . . , k1 ).
However, this means that vk equals 0 (see 5.8), contradicting our hypothesis that all the vs are nonzero. Therefore our assumption that
(v1 , . . . , vm ) is linearly dependent must have been false.
The corollary below states that an operator cannot have more distinct eigenvalues than the dimension of the vector space on which it
acts.
5.9
Corollary: Each operator on V has at most dim V distinct eigenvalues.
Proof: Let T L(V ). Suppose that 1 , . . . , m are distinct eigenvalues of T . Let v1 , . . . , vm be corresponding nonzero eigenvectors. The
last theorem implies that (v1 , . . . , vm ) is linearly independent. Thus
m dim V (see 2.6), as desired.
79
80
Polynomials Applied to Operators

The main reason that a richer theory exists for operators (which
map a vector space into itself) than for linear maps is that operators
can be raised to powers. In this section we dene that notion and the
key concept of applying a polynomial to an operator.
If T L(V ), then T T makes sense and is also in L(V ). We usually
write T 2 instead of T T . More generally, if m is a positive integer, then
T m is dened by
. . T .
Tm = T
.
m times
0
For convenience we dene T to be the identity operator I on V .

Recall from Chapter 3 that if T is an invertible operator, then the
inverse of T is denoted by T 1 . If m is a positive integer, then we dene
m
T m to be (T 1 ) .
You should verify that if T is an operator, then
T m T n = T m+n
and (T m ) = T mn ,
where m and n are allowed to be arbitrary integers if T is invertible

and nonnegative integers if T is not invertible.
If T L(V ) and p P(F) is a polynomial given by
p(z) = a0 + a1 z + a2 z2 + + am zm
for z F, then p(T ) is the operator dened by
p(T ) = a0 I + a1 T + a2 T 2 + + am T m .
For example, if p is the polynomial dened by p(z) = z2 for z F, then
p(T ) = T 2 . This is a new use of the symbol p because we are applying
it to operators, not just elements of F. If we x an operator T L(V ),
then the function from P(F) to L(V ) given by p p(T ) is linear, as
you should verify.
If p and q are polynomials with coefcients in F, then pq is the
polynomial dened by
(pq)(z) = p(z)q(z)
for z F. You should verify that we have the following nice multiplicative property: if T L(V ), then
Upper-Triangular Matrices
81
(pq)(T ) = p(T )q(T )

for all polynomials p and q with coefcients in F. Note that any two
polynomials in T commute, meaning that p(T )q(T ) = q(T )p(T ), because
p(T )q(T ) = (pq)(T ) = (qp)(T ) = q(T )p(T ).
Now we come to one of the central results about operators on complex vector spaces.
5.10 Theorem: Every operator on a nite-dimensional, nonzero,
complex vector space has an eigenvalue.
Compare the simple

proof of this theorem
given here with the
Proof: Suppose V is a complex vector space with dimension n > 0

and T L(V ). Choose v V with v = 0. Then
(v, T v, T 2 v, . . . , T n v)
cannot be linearly independent because V has dimension n and we have
n + 1 vectors. Thus there exist complex numbers a0 , . . . , an , not all 0,
such that
0 = a0 v + a1 T v + + an T n v.
Let m be the largest index such that am = 0. Because v = 0, the
coefcients a1 , . . . , am cannot all be 0, so 0 < m n. Make the as
the coefcients of a polynomial, which can be written in factored form
(see 4.8) as
a0 + a1 z + + an zn = c(z 1 ) . . . (z m ),
where c is a nonzero complex number, each j C, and the equation
holds for all z C. We then have
0 = a0 v + a1 T v + + an T n v
= (a0 I + a1 T + + an T n )v
= c(T 1 I) . . . (T m I)v,
which means that T j I is not injective for at least one j. In other
words, T has an eigenvalue.
standard proof using

determinants. With the
standard proof, rst
the difcult concept of
determinants must be
dened, then an
operator with 0
determinant must be
shown to be not
invertible, then the
characteristic
polynomial needs to be
dened, and by the
time the proof of this
theorem is reached, no
insight remains about
why it is true.
82
Recall that in Chapter 3 we discussed the matrix of a linear map

from one vector space to another vector space. This matrix depended
on a choice of a basis for each of the two vector spaces. Now that we are
studying operators, which map a vector space to itself, we need only
one basis. In addition, now our matrices will be square arrays, rather
than the more general rectangular arrays that we considered earlier.
Specically, let T L(V ). Suppose (v1 , . . . , vn ) is a basis of V . For
each k = 1, . . . , n, we can write
T vk = a1,k v1 + + an,k vn ,
The kth column of the
matrix is formed from
the coefcients used to
write T vk as a linear
combination of the vs.
We often use to
denote matrix entries
that we do not know
about or that are
irrelevant to the
questions being
discussed.
where aj,k F for j = 1, . . . , n. The n-by-n
a1,1 . . . a1,n
.
..
5.11
.
..
an,1 . . . an,n
matrix
is called the matrix of T with respect to the basis (v1 , . . . , vn ); we de

note it by M T , (v1 , . . . , vn ) or just by M(T ) if the basis (v1 , . . . , vn )
is clear from the context (for example, if only one basis is in sight).
If T is an operator on Fn and no basis is specied, you should assume
that the basis in question is the standard one (where the j th basis vector
is 1 in the j th slot and 0 in all the other slots). You can then think of
the j th column of M(T ) as T applied to the j th basis vector.
A central goal of linear algebra is to show that given an operator
T L(V ), there exists a basis of V with respect to which T has a
reasonably simple matrix. To make this vague formulation (reasonably
simple is not precise language) a bit more concrete, we might try to
make M(T ) have many 0s.
If V is a complex vector space, then we already know enough to
show that there is a basis of V with respect to which the matrix of T
has 0s everywhere in the rst column, except possibly the rst entry.
In other words, there is a basis of V with respect to which the matrix
of T looks like
.
;
..
0
here the denotes the entries in all the columns other than the rst
column. To prove this, let be an eigenvalue of T (one exists by 5.10)
and let v be a corresponding nonzero eigenvector. Extend (v) to a

basis of V . Then the matrix of T with respect to this basis has the form
above. Soon we will see that we can choose a basis of V with respect to
which the matrix of T has even more 0s.
The diagonal of a square matrix consists of the entries along the
straight line from the upper left corner to the bottom right corner.
For example, the diagonal of the matrix 5.11 consists of the entries
a1,1 , a2,2 , . . . , an,n .
A matrix is called upper triangular if all the entries below the diagonal equal 0. For example, the 4-by-4 matrix
6 2 7 5
0 6 1 3
0 0 7 9
0 0 0 8
is upper triangular. Typically we represent an upper-triangular matrix
in the form
..
;
.
0
n
the 0 in the matrix above indicates that all entries below the diagonal
in this n-by-n matrix equal 0. Upper-triangular matrices can be considered reasonably simplefor n large, an n-by-n upper-triangular matrix
has almost half its entries equal to 0.
The following proposition demonstrates a useful connection between upper-triangular matrices and invariant subspaces.
5.12 Proposition: Suppose T L(V ) and (v1 , . . . , vn ) is a basis
of V . Then the following are equivalent:
(a)
the matrix of T with respect to (v1 , . . . , vn ) is upper triangular;
(b)
T vk span(v1 , . . . , vk ) for each k = 1, . . . , n;
(c)
span(v1 , . . . , vk ) is invariant under T for each k = 1, . . . , n.
Proof: The equivalence of (a) and (b) follows easily from the definitions and a moments thought. Obviously (c) implies (b). Thus to
complete the proof, we need only prove that (b) implies (c). So suppose
that (b) holds. Fix k {1, . . . , n}. From (b), we know that
83
84
T v1 span(v1 ) span(v1 , . . . , vk );
T v2 span(v1 , v2 ) span(v1 , . . . , vk );
..
.
T vk span(v1 , . . . , vk ).
Thus if v is a linear combination of (v1 , . . . , vk ), then
T v span(v1 , . . . , vk ).
In other words, span(v1 , . . . , vk ) is invariant under T , completing the
proof.
Now we can show that for each operator on a complex vector space,
there is a basis of the vector space with respect to which the matrix
of the operator has only 0s below the diagonal. In Chapter 8 we will
improve even this result.
This theorem does not
hold on real vector
5.13 Theorem: Suppose V is a complex vector space and T L(V ).

Then T has an upper-triangular matrix with respect to some basis of V .
spaces because the rst

vector in a basis with
respect to which an
operator has an
upper-triangular matrix
must be an eigenvector
of the operator. Thus if
an operator on a real
vector space has no
eigenvalues (we have
seen an example on
Proof: We will use induction on the dimension of V . Clearly the

desired result holds if dim V = 1.
Suppose now that dim V > 1 and the desired result holds for all
complex vector spaces whose dimension is less than the dimension
of V . Let be any eigenvalue of T (5.10 guarantees that T has an
eigenvalue). Let
U = range(T I).
Because T I is not surjective (see 3.21), dim U < dim V . Furthermore,
U is invariant under T . To prove this, suppose u U . Then
R2 ), then there is no
T u = (T I)u + u.
basis with respect to

which the operator has
an upper-triangular
matrix.
Obviously (T I)u U (from the denition of U ) and u U . Thus

the equation above shows that T u U. Hence U is invariant under T ,
as claimed.
Thus T |U is an operator on U. By our induction hypothesis, there
is a basis (u1 , . . . , um ) of U with respect to which T |U has an uppertriangular matrix. Thus for each j we have (using 5.12)
5.14
T uj = (T |U )(uj ) span(u1 , . . . , uj ).
85
Extend (u1 , . . . , um ) to a basis (u1 , . . . , um , v1 , . . . , vn ) of V . For

each k, we have
T vk = (T I)vk + vk .
The denition of U shows that (T I)vk U = span(u1 , . . . , um ).
Thus the equation above shows that
T vk span(u1 , . . . , um , v1 , . . . , vk ).
5.15
From 5.14 and 5.15, we conclude (using 5.12) that T has an uppertriangular matrix with respect to the basis (u1 , . . . , um , v1 , . . . , vn ).
How does one determine from looking at the matrix of an operator
whether the operator is invertible? If we are fortunate enough to have
a basis with respect to which the matrix of the operator is upper triangular, then this problem becomes easy, as the following proposition
shows.
5.16 Proposition: Suppose T L(V ) has an upper-triangular matrix
with respect to some basis of V . Then T is invertible if and only if all
the entries on the diagonal of that upper-triangular matrix are nonzero.
Proof: Suppose (v1 , . . . , vn ) is a basis of V with respect to which
T has an upper-triangular matrix
5.17
M T , (v1 , . . . , vn ) =
1
2
..
0
We need to prove that T is not invertible if and only if one of the k s

equals 0.
First we will prove that if one of the k s equals 0, then T is not
invertible. If 1 = 0, then T v1 = 0 (from 5.17) and hence T is not
invertible, as desired. So suppose that 1 < k n and k = 0. Then,
as can be seen from 5.17, T maps each of the vectors v1 , . . . , vk1 into
span(v1 , . . . , vk1 ). Because k = 0, the matrix representation 5.17 also
implies that T vk span(v1 , . . . , vk1 ). Thus we can dene a linear map
S : span(v1 , . . . , vk ) span(v1 , . . . , vk1 )
86
by Sv = T v for v span(v1 , . . . , vk ). In other words, S is just T

restricted to span(v1 , . . . , vk ).
Note that span(v1 , . . . , vk ) has dimension k and span(v1 , . . . , vk1 )
has dimension k 1 (because (v1 , . . . , vn ) is linearly independent). Because span(v1 , . . . , vk ) has a larger dimension than span(v1 , . . . , vk1 ),
no linear map from span(v1 , . . . , vk ) to span(v1 , . . . , vk1 ) is injective
(see 3.5). Thus there exists a nonzero vector v span(v1 , . . . , vk ) such
that Sv = 0. Hence T v = 0, and thus T is not invertible, as desired.
To prove the other direction, now suppose that T is not invertible.
Thus T is not injective (see 3.21), and hence there exists a nonzero
vector v V such that T v = 0. Because (v1 , . . . , vn ) is a basis of V , we
can write
v = a1 v1 + + ak vk ,
where a1 , . . . , ak F and ak = 0 (represent v as a linear combination
of (v1 , . . . , vn ) and then choose k to be the largest index with a nonzero
coefcient). Thus
0 = Tv
0 = T (a1 v1 + + ak vk )
= (a1 T v1 + + ak1 T vk1 ) + ak T vk .
The last term in parentheses is in span(v1 , . . . , vk1 ) (because of the
upper-triangular form of 5.17). Thus the last equation shows that
ak T vk span(v1 , . . . , vk1 ). Multiplying by 1/ak , which is allowed
because ak = 0, we conclude that T vk span(v1 , . . . , vk1 ). Thus
when T vk is written as a linear combination of the basis (v1 , . . . , vn ),
the coefcient of vk will be 0. In other words, k in 5.17 must be 0,
completing the proof.
Powerful numeric
techniques exist for
nding good
approximations to the
eigenvalues of an
operator from its
Unfortunately no method exists for exactly computing the eigenvalues of a typical operator from its matrix (with respect to an arbitrary
basis). However, if we are fortunate enough to nd a basis with respect to which the matrix of the operator is upper triangular, then the
problem of computing the eigenvalues becomes trivial, as the following
proposition shows.
matrix.
5.18 Proposition: Suppose T L(V ) has an upper-triangular matrix

with respect to some basis of V . Then the eigenvalues of T consist
precisely of the entries on the diagonal of that upper-triangular matrix.
Diagonal Matrices
87

T has an upper-triangular matrix
2

.
M T , (v1 , . . . , vn ) =
.
..
0
n
Let F. Then

M T I, (v1 , . . . , vn ) =
2
..
.
n
Hence T I is not invertible if and only if equals one of the j s

(see 5.16). In other words, is an eigenvalue of T if and only if
equals one of the j s, as desired.
Diagonal Matrices
A diagonal matrix is a square matrix that is 0 everywhere except
possibly along the diagonal. For example,
8 0 0
0 2 0
0 0 5
is a diagonal matrix. Obviously every diagonal matrix is upper triangular, although in general a diagonal matrix has many more 0s than an
upper-triangular matrix.
An operator T L(V ) has a diagonal matrix
0
1
..
0
n
with respect to a basis (v1 , . . . , vn ) of V if and only
T v1 = 1 v1
..
.
T vn = n vn ;
88
this follows immediately from the denition of the matrix of an operator with respect to a basis. Thus an operator T L(V ) has a diagonal
matrix with respect to some basis of V if and only if V has a basis
consisting of eigenvectors of T .
If an operator has a diagonal matrix with respect to some basis,
then the entries along the diagonal are precisely the eigenvalues of the
operator; this follows from 5.18 (or you may want to nd an easier
proof that works only for diagonal matrices).
Unfortunately not every operator has a diagonal matrix with respect
to some basis. This sad state of affairs can arise even on complex vector
spaces. For example, consider T L(C2 ) dened by
5.19
T (w, z) = (z, 0).
As you should verify, 0 is the only eigenvalue of this operator and

the corresponding set of eigenvectors is the one-dimensional subspace
{(w, 0) C2 : w C}. Thus there are not enough linearly independent
eigenvectors of T to form a basis of the two-dimensional space C2 .
Hence T does not have a diagonal matrix with respect to any basis
of C2 .
The next proposition shows that if an operator has as many distinct
eigenvalues as the dimension of its domain, then the operator has a diagonal matrix with respect to some operator. However, some operators
with fewer eigenvalues also have diagonal matrices (in other words, the
converse of the next proposition is not true). For example, the operator
T dened on the three-dimensional space F3 by
T (z1 , z2 , z3 ) = (4z1 , 4z2 , 5z3 )
has only two eigenvalues (4 and 5), but this operator has a diagonal
matrix with respect to the standard basis.
Later we will nd other
conditions that imply
5.20 Proposition: If T L(V ) has dim V distinct eigenvalues, then

T has a diagonal matrix with respect to some basis of V .
that certain operators

have a diagonal matrix
with respect to some
basis (see 7.9 and 7.13).
Proof: Suppose that T L(V ) has dim V distinct eigenvalues

1 , . . . , dim V . For each j, let vj V be a nonzero eigenvector corresponding to the eigenvalue j . Because nonzero eigenvectors corresponding to distinct eigenvalues are linearly independent (see 5.6),
(v1 , . . . , vdim V ) is linearly independent. A linearly independent list of
Diagonal Matrices
89
dim V vectors in V is a basis of V (see 2.17); thus (v1 , . . . , vdim V ) is a

basis of V . With respect to this basis consisting of eigenvectors, T has
a diagonal matrix.
We close this section with a proposition giving several conditions
on an operator that are equivalent to its having a diagonal matrix with
respect to some basis.
5.21 Proposition: Suppose T L(V ). Let 1 , . . . , m denote the
distinct eigenvalues of T . Then the following are equivalent:
(a)
T has a diagonal matrix with respect to some basis of V ;
(b)
V has a basis consisting of eigenvectors of T ;
(c)
there exist one-dimensional subspaces U1 , . . . , Un of V , each invariant under T , such that

V = U 1 Un ;
(d)
V = null(T 1 I) null(T m I);
(e)
dim V = dim null(T 1 I) + + dim null(T m I).
Proof: We have already shown that (a) and (b) are equivalent.
Suppose that (b) holds; thus V has a basis (v1 , . . . , vn ) consisting of
eigenvectors of T . For each j, let Uj = span(vj ). Obviously each Uj
is a one-dimensional subspace of V that is invariant under T (because
each vj is an eigenvector of T ). Because (v1 , . . . , vn ) is a basis of V ,
each vector in V can be written uniquely as a linear combination of
(v1 , . . . , vn ). In other words, each vector in V can be written uniquely
as a sum u1 + + un , where each uj Uj . Thus V = U1 Un .
Hence (b) implies (c).
Suppose now that (c) holds; thus there are one-dimensional subspaces U1 , . . . , Un of V , each invariant under T , such that
V = U1 Un .
For each j, let vj be a nonzero vector in Uj . Then each vj is an eigenvector of T . Because each vector in V can be written uniquely as a sum
u1 + +un , where each uj Uj (so each uj is a scalar multiple of vj ),
we see that (v1 , . . . , vn ) is a basis of V . Thus (c) implies (b).
For complex vector

spaces, we will extend
this list of equivalences
later (see Exercises 16
and 23 in Chapter 8).
90
At this stage of the proof we know that (a), (b), and (c) are all equivalent. We will nish the proof by showing that (b) implies (d), that (d)
implies (e), and that (e) implies (b).
Suppose that (b) holds; thus V has a basis consisting of eigenvectors
of T . Thus every vector in V is a linear combination of eigenvectors
of T . Hence
5.22
V = null(T 1 I) + + null(T m I).
To show that the sum above is a direct sum, suppose that

0 = u1 + + um ,
where each uj null(T j I). Because nonzero eigenvectors corresponding to distinct eigenvalues are linearly independent, this implies
(apply 5.6 to the sum of the nonzero vectors on the right side of the
equation above) that each uj equals 0. This implies (using 1.8) that the
sum in 5.22 is a direct sum, completing the proof that (b) implies (d).
That (d) implies (e) follows immediately from Exercise 17 in Chapter 2.
Finally, suppose that (e) holds; thus
5.23
dim V = dim null(T 1 I) + + dim null(T m I).
Choose a basis of each null(T j I); put all these bases together to
form a list (v1 , . . . , vn ) of eigenvectors of T , where n = dim V (by 5.23).
To show that this list is linearly independent, suppose
a1 v1 + + an vn = 0,
where a1 , . . . , an F. For each j = 1, . . . , m, let uj denote the sum of
all the terms ak vk such that vk null(T j I). Thus each uj is an
eigenvector of T with eigenvalue j , and
u1 + + um = 0.
Because nonzero eigenvectors corresponding to distinct eigenvalues
are linearly independent, this implies (apply 5.6 to the sum of the
nonzero vectors on the left side of the equation above) that each uj
equals 0. Because each uj is a sum of terms ak vk , where the vk s
were chosen to be a basis of null(T j I), this implies that all the ak s
equal 0. Thus (v1 , . . . , vn ) is linearly independent and hence is a basis
of V (by 2.17). Thus (e) implies (b), completing the proof.
91

We know that every operator on a complex vector space has an eigenvalue (see 5.10 for the precise statement). We have also seen an example
showing that the analogous statement is false on real vector spaces. In
other words, an operator on a nonzero real vector space may have no
invariant subspaces of dimension 1. However, we now show that an
invariant subspace of dimension 1 or 2 always exists.
5.24 Theorem: Every operator on a nite-dimensional, nonzero, real
vector space has an invariant subspace of dimension 1 or 2.
Proof: Suppose V is a real vector space with dimension n > 0 and
T L(V ). Choose v V with v = 0. Then
(v, T v, T 2 v, . . . , T n v)
n + 1 vectors. Thus there exist real numbers a0 , . . . , an , not all 0, such
that
0 = a0 v + a1 T v + + an T n v.
Make the as the coefcients of a polynomial, which can be written in
factored form (see 4.14) as
a0 + a1 x + + an x n
= c(x 1 ) . . . (x m )(x 2 + 1 x + 1 ) . . . (x 2 + M x + M ),
where c is a nonzero real number, each j , j , and j is real, m+M 1,
and the equation holds for all x R. We then have
0 = a0 v + a1 T v + + an T n v
= (a0 I + a1 T + + an T n )v
= c(T 1 I) . . . (T m I)(T 2 + 1 T + 1 I) . . . (T 2 + M T + M I)v,
which means that T j I is not injective for at least one j or that
(T 2 + j T + j I) is not injective for at least one j. If T j I is not
injective for at least one j, then T has an eigenvalue and hence a onedimensional invariant subspace. Lets consider the other possibility. In
other words, suppose that (T 2 + j T + j I) is not injective for some j.
Thus there exists a nonzero vector u V such that
Here either m or M
might equal 0.
92
5.25
T 2 u + j T u + j u = 0.
We will complete the proof by showing that span(u, T u), which clearly
has dimension 1 or 2, is invariant under T . To do this, consider a typical
element of span(u, T u) of the form au+bT u, where a, b R. Then
T (au + bT u) = aT u + bT 2 u
= aT u bj T u bj u,
where the last equality comes from solving for T 2 u in 5.25. The equation above shows that T (au + bT u) span(u, T u). Thus span(u, T u)
is invariant under T , as desired.
We will need one new piece of notation for the next proof. Suppose
U and W are subspaces of V with
V = U W.
Each vector v V can be written uniquely in the form
v = u + w,
PU ,W is often called the
projection onto U with
null space W .
where u U and w W . With this representation, dene PU ,W L(V )

by
PU ,W v = u.
You should verify that PU ,W v = v if and only if v U. Interchanging
the roles of U and W in the representation above, we have PW ,U v = w.
Thus v = PU ,W v + PW ,U v for every v V . You should verify that
PU ,W 2 = PU ,W ; furthermore range PU ,W = U and null PU ,W = W .
We have seen an example of an operator on R 2 with no eigenvalues.
The following theorem shows that no such example exists on R3 .
5.26 Theorem: Every operator on an odd-dimensional real vector
space has an eigenvalue.
Proof: Suppose V is a real vector space with odd dimension. We
will prove that every operator on V has an eigenvalue by induction (in
steps of size 2) on the dimension of V . To get started, note that the
desired result obviously holds if dim V = 1.
Now suppose that dim V is an odd number greater than 1. Using
induction, we can assume that the desired result holds for all operators
on all real vector spaces with dimension 2 less than dim V . Suppose
T L(V ). We need to prove that T has an eigenvalue. If it does, we are
done. If not, then by 5.24 there is a two-dimensional subspace U of V
that is invariant under T . Let W be any subspace of V such that
V = U W;
2.13 guarantees that such a W exists.
Because W has dimension 2 less than dim V , we would like to apply
our induction hypothesis to T |W . However, W might not be invariant
under T , meaning that T |W might not be an operator on W . We will
compose with the projection PW ,U to get an operator on W . Specically,
dene S L(W ) by
Sw = PW ,U (T w)
for w W . By our induction hypothesis, S has an eigenvalue . We
will show that this is also an eigenvalue for T .
Let w W be a nonzero eigenvector for S corresponding to the
eigenvalue ; thus (S I)w = 0. We would be done if w were an
eigenvector for T with eigenvalue ; unfortunately that need not be
true. So we will look for an eigenvector of T in U + span(w). To do
that, consider a typical vector u + aw in U + span(w), where u U
and a R. We have
(T I)(u + aw) = T u u + a(T w w)
= T u u + a(PU ,W (T w) + PW ,U (T w) w)
= T u u + a(PU ,W (T w) + Sw w)
= T u u + aPU ,W (T w).
Note that on the right side of the last equation, T u U (because U
is invariant under T ), u U (because u U), and aPU ,W (T w) U
(from the denition of PU ,W ). Thus T I maps U + span(w) into U.
Because U + span(w) has a larger dimension than U , this means that
(T I)|U +span(w) is not injective (see 3.5). In other words, there exists
a nonzero vector v U + span(w) V such that (T I)v = 0. Thus
T has an eigenvalue, as desired.
93
94
Exercises
1.
Suppose T L(V ). Prove that if U1 , . . . , Um are subspaces of V

invariant under T , then U1 + + Um is invariant under T .
2.
Suppose T L(V ). Prove that the intersection of any collection

of subspaces of V invariant under T is invariant under T .
3.
Prove or give a counterexample: if U is a subspace of V that is

invariant under every operator on V , then U = {0} or U = V .
4.
Suppose that S, T L(V ) are such that ST = T S. Prove that

null(T I) is invariant under S for every F.
5.
Dene T L(F2 ) by
T (w, z) = (z, w).
Find all eigenvalues and eigenvectors of T .
6.
Dene T L(F3 ) by
T (z1 , z2 , z3 ) = (2z2 , 0, 5z3 ).
Find all eigenvalues and eigenvectors of T .
7.
Suppose n is a positive integer and T L(Fn ) is dened by

T (x1 , . . . , xn ) = (x1 + + xn , . . . , x1 + + xn );
in other words, T is the operator whose matrix (with respect to
the standard basis) consists of all 1s. Find all eigenvalues and
eigenvectors of T .
8.
Find all eigenvalues and eigenvectors of the backward shift operator T L(F ) dened by
T (z1 , z2 , z3 , . . . ) = (z2 , z3 , . . . ).
9.
Suppose T L(V ) and dim range T = k. Prove that T has at

most k + 1 distinct eigenvalues.
10.
Suppose T L(V ) is invertible and F \ {0}. Prove that is

an eigenvalue of T if and only if 1 is an eigenvalue of T 1 .
Exercises
11.
Suppose S, T L(V ). Prove that ST and T S have the same eigenvalues.
12.
Suppose T L(V ) is such that every vector in V is an eigenvector

of T . Prove that T is a scalar multiple of the identity operator.
13.
Suppose T L(V ) is such that every subspace of V with dimension dim V 1 is invariant under T . Prove that T is a scalar
multiple of the identity operator.
14.
Suppose S, T L(V ) and S is invertible. Prove that if p P(F)

is a polynomial, then
95
p(ST S 1 ) = Sp(T )S 1 .
15.
Suppose F = C, T L(V ), p P(C), and a C. Prove that a is

an eigenvalue of p(T ) if and only if a = p() for some eigenvalue
of T .
16.
Show that the result in the previous exercise does not hold if C
is replaced with R.
17.
Suppose V is a complex vector space and T L(V ). Prove

that T has an invariant subspace of dimension j for each j =
1, . . . , dim V .
18.
Give an example of an operator whose matrix with respect to

some basis contains only 0s on the diagonal, but the operator is
invertible.
19.
Give an example of an operator whose matrix with respect to

some basis contains only nonzero numbers on the diagonal, but
the operator is not invertible.
20.
Suppose that T L(V ) has dim V distinct eigenvalues and that

S L(V ) has the same eigenvectors as T (not necessarily with
the same eigenvalues). Prove that ST = T S.
21.
Suppose P L(V ) and P 2 = P . Prove that V = null P range P .
22.
Suppose V = U W , where U and W are nonzero subspaces of V .

Find all eigenvalues and eigenvectors of PU ,W .
These two exercises

show that 5.16 fails
without the hypothesis
that an uppertriangular matrix is
under consideration.
96
23.
Give an example of an operator T L(R 4 ) such that T has no

(real) eigenvalues.
24.
Suppose V is a real vector space and T L(V ) has no eigenvalues. Prove that every subspace of V invariant under T has even
dimension.
Chapter 6
In making the denition of a vector space, we generalized the linear structure (addition and scalar multiplication) of R 2 and R 3 . We
ignored other important features, such as the notions of length and
angle. These ideas are embedded in the concept we now investigate,
inner products.
Also, V is a nite-dimensional, nonzero vector space over F.
97
Chapter 6. Inner-Product Spaces
98
Inner Products
If we think of vectors
as points instead of
arrows, then x
To motivate the concept of inner product, lets think of vectors in R2

and R3 as arrows with initial point at the origin. The length of a vector x in R 2 or R 3 is called the norm
of x, denoted x. Thus for

x = (x1 , x2 ) R 2 , we have x = x1 2 + x2 2 .
x -axis
2
should be interpreted
as the distance from
the point x to the
(x , x )
1
origin.
x1-axis
x1 2 + x2 2 .

Similarly, for x = (x1 , x2 , x3 ) R3 , we have x = x1 2 + x2 2 + x3 2 .
Even though we cannot draw pictures in higher dimensions, the generalization to R n is obvious: we dene the norm of x = (x1 , . . . , xn ) Rn
by

The length of this vector x is
x = x1 2 + + xn 2 .
The norm is not linear on Rn . To inject linearity into the discussion,

we introduce the dot product. For x, y Rn , the dot product of x
and y, denoted x y, is dened by
x y = x1 y1 + + xn yn ,
where x = (x1 , . . . , xn ) and y = (y1 , . . . , yn ). Note that the dot product
of two vectors in R n is a number, not a vector. Obviously x x = x2
for all x R n . In particular, x x 0 for all x R n , with equality if
and only if x = 0. Also, if y R n is xed, then clearly the map from Rn
to R that sends x Rn to x y is linear. Furthermore, x y = y x
for all x, y R n .
An inner product is a generalization of the dot product. At this
point you should be tempted to guess that an inner product is dened
Inner Products
by abstracting the properties of the dot product discussed in the paragraph above. For real vector spaces, that guess is correct. However,
so that we can make a denition that will be useful for both real and
complex vector spaces, we need to examine the complex case before
making the denition.
Recall that if = a + bi, where a, b R, then the absolute value
of is dened by

|| = a2 + b2 ,
the complex conjugate of is dened by
= a bi,
and the equation
||2 =
connects these two concepts (see page 69 for the denitions and the
basic properties of the absolute value and complex conjugate). For
z = (z1 , . . . , zn ) Cn , we dene the norm of z by

z = |z1 |2 + + |zn |2 .
The absolute values are needed because we want z to be a nonnegative number. Note that
z2 = z1 z1 + + zn zn .
We want to think of z2 as the inner product of z with itself, as we
did in Rn . The equation above thus suggests that the inner product of
w = (w1 , . . . , wn ) Cn with z should equal
w1 z1 + + wn zn .
If the roles of the w and z were interchanged, the expression above
would be replaced with its complex conjugate. In other words, we
should expect that the inner product of w with z equals the complex
conjugate of the inner product of z with w. With that motivation, we
are now ready to dene an inner product on V , which may be a real or
a complex vector space.
An inner product on V is a function that takes each ordered pair
(u, v) of elements of V to a number u, v F and has the following
properties:
99
100
If z is a complex
number, then the
statement z 0 means
that z is real and
nonnegative.
positivity
v, v 0 for all v V ;
deniteness
v, v = 0 if and only if v = 0;
additivity in rst slot
u + v, w = u, w + v, w for all u, v, w V ;
homogeneity in rst slot
av, w = av, w for all a F and all v, w V ;
conjugate symmetry
v, w = w, v for all v, w V .
If we are dealing with
Recall that every real number equals its complex conjugate. Thus
if we are dealing with a real vector space, then in the last condition
above we can dispense with the complex conjugate and simply state
that v, w = w, v for all v, w V .
An inner-product space is a vector space V along with an inner
product on V .
The most important example of an inner-product space is Fn . We
can dene an inner product on Fn by
R rather than Cn , then

again the complex
conjugate can be
ignored.
6.1
(w1 , . . . , wn ), (z1 , . . . , zn ) = w1 z1 + + wn zn ,
as you should verify. This inner product, which provided our motivation for the denition of an inner product, is called the Euclidean inner
product on Fn . When Fn is referred to as an inner-product space, you
should assume that the inner product is the Euclidean inner product
unless explicitly told otherwise.
There are other inner products on Fn in addition to the Euclidean
inner product. For example, if c1 , . . . , cn are positive numbers, then we
can dene an inner product on Fn by
(w1 , . . . , wn ), (z1 , . . . , zn ) = c1 w1 z1 + + cn wn zn ,
as you should verify. Of course, if all the cs equal 1, then we get the
Euclidean inner product.
As another example of an inner-product space, consider the vector
space Pm (F) of all polynomials with coefcients in F and degree at
most m. We can dene an inner product on Pm (F) by
Inner Products
1
6.2
p, q =
p(x)q(x) dx,
as you should verify. Once again, if F = R, then the complex conjugate

is not needed.
Lets agree for the rest of this chapter that
V is a nite-dimensional inner-product space over F.
In the denition of an inner product, the conditions of additivity
and homogeneity in the rst slot can be combined into a requirement
of linearity in the rst slot. More precisely, for each xed w V , the
function that takes v to v, w is a linear map from V to F. Because
every linear map takes 0 to 0, we must have
0, w = 0
for every w V . Thus we also have
w, 0 = 0
for every w V (by the conjugate symmetry property).
In an inner-product space, we have additivity in the second slot as
well as the rst slot. Proof:
u, v + w = v + w, u
= v, u + w, u
= v, u + w, u
= u, v + u, w;
here u, v, w V .
In an inner-product space, we have conjugate homogeneity in the
second slot, meaning that u, av = au,

v for all scalars a F.
Proof:
u, av = av, u
= av, u
u
= av,
= au,
v;
here a F and u, v V . Note that in a real vector space, conjugate
homogeneity is the same as homogeneity.
101
102
Norms
For v V , we dene the norm of v, denoted v, by

v = v, v.
For example, if (z1 , . . . , zn ) Fn (with the Euclidean inner product),
then

(z1 , . . . , zn ) = |z1 |2 + + |zn |2 .
As another example, if p Pm (F) (with inner product given by 6.2),
then

1
|p(x)|2 dx.
p =
0
Note that v = 0 if and only if v = 0 (because v, v = 0 if and only

if v = 0). Another easy property of the norm is that av = |a| v
for all a F and all v V . Heres the proof:
av2 = av, av
= av, av
= aav,
v
= |a|2 v2 ;
Some mathematicians
use the term
perpendicular, which
means the same as
orthogonal.
The word orthogonal
comes from the Greek
taking square roots now gives the desired equality. This proof illustrates a general principle: working with norms squared is usually easier
than working directly with norms.
Two vectors u, v V are said to be orthogonal if u, v = 0. Note
that the order of the vectors does not matter because u, v = 0 if
and only if v, u = 0. Instead of saying that u and v are orthogonal,
sometimes we say that u is orthogonal to v. Clearly 0 is orthogonal
to every vector. Furthermore, 0 is the only vector that is orthogonal to
itself.
For the special case where V = R 2 , the next theorem is over 2,500
years old.
word orthogonios,
which means
6.3
Pythagorean Theorem: If u, v are orthogonal vectors in V , then
right-angled.
6.4
u + v2 = u2 + v2 .
Norms
103
Proof: Suppose that u, v are orthogonal vectors in V . Then
The proof of the

Pythagorean theorem
u + v2 = u + v, u + v
= u2 + v2 + u, v + v, u
= u2 + v2 ,
shows that 6.4 holds if

and only if
u, v + v, u, which
equals 2 Reu, v, is 0.
as desired.
Thus the converse of
Suppose u, v V . We would like to write u as a scalar multiple of v

plus a vector w orthogonal to v, as suggested in the next picture.
u
w
v
v
An orthogonal decomposition
To discover how to write u as a scalar multiple of v plus a vector orthogonal to v, let a F denote a scalar. Then
u = av + (u av).
Thus we need to choose a so that v is orthogonal to (u av). In other
words, we want
0 = u av, v = u, v av2 .
The equation above shows that we should choose a to be u, v/v2
(assume that v = 0 to avoid division by 0). Making this choice of a, we
can write

u, v
u, v
6.5
u=
v
+
u
v
.
v2
v2
As you should verify, if v = 0 then the equation above writes u as a
scalar multiple of v plus a vector orthogonal to v.
The equation above will be used in the proof of the next theorem,
which gives one of the most important inequalities in mathematics.
the Pythagorean
theorem holds in real
inner-product spaces.
104
In 1821 the French
mathematician
Augustin-Louis Cauchy
showed that this
inequality holds for the
inner product dened
by 6.1. In 1886 the
German mathematician
Herman Schwarz
6.6
6.7
Proof: Let u, v V . If v = 0, then both sides of 6.7 equal 0 and

the desired inequality holds. Thus we can assume that v = 0. Consider
the orthogonal decomposition
u=
inequality holds for the

by 6.2.
|u, v| u v.
This inequality is an equality if and only if one of u, v is a scalar multiple of the other.
showed that this

inner product dened
Cauchy-Schwarz Inequality: If u, v V , then
u, v
v + w,
v2
where w is orthogonal to v (here w equals the second term on the right

side of 6.5). By the Pythagorean theorem,

u, v 2
2

v
u2 =
v2 + w
6.8
|u, v|2
+ w2
v2
|u, v|2
.
v2
Multiplying both sides of this inequality by v2 and then taking square
roots gives the Cauchy-Schwarz inequality 6.7.
Looking at the proof of the Cauchy-Schwarz inequality, note that 6.7
is an equality if and only if 6.8 is an equality. Obviously this happens if
and only if w = 0. But w = 0 if and only if u is a multiple of v (see 6.5).
Thus the Cauchy-Schwarz inequality is an equality if and only if u is a
scalar multiple of v or v is a scalar multiple of u (or both; the phrasing
has been chosen to cover cases in which either u or v equals 0).
The next result is called the triangle inequality because of its geometric interpretation that the length of any side of a triangle is less
than the sum of the lengths of the other two sides.
u+v
The triangle inequality
Norms
6.9
Triangle Inequality: If u, v V , then
105
The triangle inequality
can be used to show
6.10
u + v u + v.
that the shortest path

between two points is a
This inequality is an equality if and only if one of u, v is a nonnegative

multiple of the other.
Proof: Let u, v V . Then
u + v2 = u + v, u + v
= u, u + v, v + u, v + v, u
= u, u + v, v + u, v + u, v
= u2 + v2 + 2 Reu, v
6.11
u2 + v2 + 2|u, v|
6.12
u2 + v2 + 2u v

= (u + v)2 ,
where 6.12 follows from the Cauchy-Schwarz inequality (6.6). Taking

square roots of both sides of the inequality above gives the triangle
inequality 6.10.
The proof above shows that the triangle inequality 6.10 is an equality
if and only if we have equality in 6.11 and 6.12. Thus we have equality
in the triangle inequality 6.10 if and only if
6.13
u, v = uv.
If one of u, v is a nonnegative multiple of the other, then 6.13 holds, as

you should verify. Conversely, suppose 6.13 holds. Then the condition
for equality in the Cauchy-Schwarz inequality (6.6) implies that one of
u, v must be a scalar multiple of the other. Clearly 6.13 forces the
scalar in question to be nonnegative, as desired.
The next result is called the parallelogram equality because of its
geometric interpretation: in any parallelogram, the sum of the squares
of the lengths of the diagonals equals the sum of the squares of the
lengths of the four sides.
straight line segment.
106
u
u
v
u+
The parallelogram equality

6.14
Parallelogram Equality: If u, v V , then

u + v2 + u v2 = 2(u2 + v2 ).
Proof: Let u, v V . Then

u + v2 + u v2 = u + v, u + v + u v, u v
= u2 + v2 + u, v + v, u
+ u2 + v2 u, v v, u
= 2(u2 + v2 ),
as desired.
Orthonormal Bases
A list of vectors is called orthonormal if the vectors in it are pairwise orthogonal and each vector has norm 1. In other words, a list
(e1 , . . . , em ) of vectors in V is orthonormal if ej , ek equals 0 when
j = k and equals 1 when j = k (for j, k = 1, . . . , m). For example, the
standard basis in Fn is orthonormal. Orthonormal lists are particularly
easy to work with, as illustrated by the next proposition.
6.15 Proposition:
in V , then
If (e1 , . . . , em ) is an orthonormal list of vectors
a1 e1 + + am em 2 = |a1 |2 + + |am |2

for all a1 , . . . , am F.
Proof: Because each ej has norm 1, this follows easily from repeated applications of the Pythagorean theorem (6.3).
Now we have the following easy but important corollary.
Orthonormal Bases
6.16 Corollary:
pendent.
107
Every orthonormal list of vectors is linearly inde-
Proof: Suppose (e1 , . . . , em ) is an orthonormal list of vectors in V

and a1 , . . . , am F are such that
a1 e1 + + am em = 0.
Then |a1 |2 + + |am |2 = 0 (by 6.15), which means that all the aj s
are 0, as desired.
An orthonormal basis of V is an orthonormal list of vectors in V
that is also a basis of V . For example, the standard basis is an orthonormal basis of Fn . Every orthonormal list of vectors in V with length
dim V is automatically an orthonormal basis of V (proof: by the previous corollary, any such list must be linearly independent; because it
has the right length, it must be a basissee 2.17). To illustrate this
principle, consider the following list of four vectors in R 4 :

1 1 1 1
1 1
1
1
1
1
1 1
1 1
1 1
( 2 , 2 , 2 , 2 ), ( 2 , 2 , 2 , 2 ), ( 2 , 2 , 2 , 2 ), ( 2 , 2 , 2 , 2 ) .
The verication that this list is orthonormal is easy (do it!); because we
have an orthonormal list of length four in a four-dimensional vector
space, it must be an orthonormal basis.
In general, given a basis (e1 , . . . , en ) of V and a vector v V , we
know that there is some choice of scalars a1 , . . . , am such that
v = a 1 e1 + + a n en ,
but nding the aj s can be difcult. The next theorem shows, however,
that this is easy for an orthonormal basis.
6.17 Theorem:
Then
6.18
Suppose (e1 , . . . , en ) is an orthonormal basis of V .
orthonormal bases
v = v, e1 e1 + + v, en en
and
6.19
for every v V .
The importance of
v2 = |v, e1 |2 + + |v, en |2
stems mainly from this

theorem.
108
Proof: Let v V . Because (e1 , . . . , en ) is a basis of V , there exist

scalars a1 , . . . , an such that
v = a1 e1 + + an en .
Take the inner product of both sides of this equation with ej , getting v , ej = aj . Thus 6.18 holds. Clearly 6.19 follows from 6.18
and 6.15.
The Danish
mathematician Jorgen
Gram (18501916) and
the German
mathematician Erhard
Schmidt (18761959)
popularized this
algorithm for
constructing
orthonormal lists.
Now that we understand the usefulness of orthonormal bases, how

do we go about nding them? For example, does Pm (F), with inner
product given by integration on [0, 1] (see 6.2), have an orthonormal
basis? As we will see, the next result will lead to answers to these questions. The algorithm used in the next proof is called the Gram-Schmidt
procedure. It gives a method for turning a linearly independent list into
an orthonormal list with the same span as the original list.
6.20 Gram-Schmidt: If (v1 , . . . , vm ) is a linearly independent list
of vectors in V , then there exists an orthonormal list (e1 , . . . , em ) of
vectors in V such that
span(v1 , . . . , vj ) = span(e1 , . . . , ej )
6.21
for j = 1, . . . , m.
Proof: Suppose (v1 , . . . , vm ) is a linearly independent list of vectors in V . To construct the es, start by setting e1 = v1 /v1 . This
satises 6.21 for j = 1. We will choose e2 , . . . , em inductively, as follows. Suppose j > 1 and an orthornormal list (e1 , . . . , ej1 ) has been
chosen so that
6.22
span(v1 , . . . , vj1 ) = span(e1 , . . . , ej1 ).
Let
6.23
ej =
vj vj , e1 e1 vj , ej1 ej1

.
Note that vj span(v1 , . . . , vj1 ) (because (v1 , . . . , vm ) is linearly independent) and thus vj span(e1 , . . . , ej1 ). Hence we are not dividing
by 0 in the equation above, and so ej is well dened. Dividing a vector
by its norm produces a new vector with norm 1; thus ej = 1.
Orthonormal Bases
109
Let 1 k < j. Then

, ek
ej , ek =
=
vj , ek vj , ek
= 0.
Thus (e1 , . . . , ej ) is an orthonormal list.
From 6.23, we see that vj span(e1 , . . . , ej ). Combining this information with 6.22 shows that
span(v1 , . . . , vj ) span(e1 , . . . , ej ).
Both lists above are linearly independent (the vs by hypothesis, the es
by orthonormality and 6.16). Thus both subspaces above have dimension j, and hence they must be equal, completing the proof.
Now we can settle the question of the existence of orthonormal
bases.
6.24 Corollary: Every nite-dimensional inner-product space has an
orthonormal basis.
Until this corollary,

nothing we had done
with inner-product
Proof: Choose a basis of V . Apply the Gram-Schmidt procedure

(6.20) to it, producing an orthonormal list. This orthonormal list is
linearly independent (by 6.16) and its span equals V . Thus it is an
orthonormal basis of V .
As we will soon see, sometimes we need to know not only that an
orthonormal basis exists, but also that any orthonormal list can be
extended to an orthonormal basis. In the next corollary, the GramSchmidt procedure shows that such an extension is always possible.
6.25 Corollary: Every orthonormal list of vectors in V can be extended to an orthonormal basis of V .
Proof: Suppose (e1 , . . . , em ) is an orthonormal list of vectors in V .
Then (e1 , . . . , em ) is linearly independent (by 6.16), and hence it can be
extended to a basis (e1 , . . . , em , v1 , . . . , vn ) of V (see 2.12). Now apply
spaces required our

standing assumption
that V is nite
dimensional.
110
the Gram-Schmidt procedure (6.20) to (e1 , . . . , em , v1 , . . . , vn ), producing an orthonormal list

6.26
(e1 , . . . , em , f1 , . . . , fn );
here the Gram-Schmidt procedure leaves the rst m vectors unchanged

because they are already orthonormal. Clearly 6.26 is an orthonormal
basis of V because it is linearly independent (by 6.16) and its span
equals V . Hence we have our extension of (e1 , . . . , em ) to an orthonormal basis of V .
Recall that a matrix is called upper triangular if all entries below the
diagonal equal 0. In other words, an upper-triangular matrix looks like
this:
..
.
.
In the last chapter we showed that if V is a complex vector space, then

for each operator on V there is a basis with respect to which the matrix
of the operator is upper triangular (see 5.13). Now that we are dealing
with inner-product spaces, we would like to know when there exists an
orthonormal basis with respect to which we have an upper-triangular
matrix. The next corollary shows that the existence of any basis with
respect to which T has an upper-triangular matrix implies the existence
of an orthonormal basis with this property. This result is true on both
real and complex vector spaces (though on a real vector space, the hypothesis holds only for some operators).
6.27 Corollary: Suppose T L(V ). If T has an upper-triangular
matrix with respect to some basis of V , then T has an upper-triangular
matrix with respect to some orthonormal basis of V .
Proof: Suppose T has an upper-triangular matrix with respect to
some basis (v1 , . . . , vn ) of V . Thus span(v1 , . . . , vj ) is invariant under
T for each j = 1, . . . , n (see 5.12).
Apply the Gram-Schmidt procedure to (v1 , . . . , vn ), producing an
orthonormal basis (e1 , . . . , en ) of V . Because
span(e1 , . . . , ej ) = span(v1 , . . . , vj )
111
for each j (see 6.21), we conclude that span(e1 , . . . , ej ) is invariant under T for each j = 1, . . . , n. Thus, by 5.12, T has an upper-triangular
matrix with respect to the orthonormal basis (e1 , . . . , en ).
The next result is an important application of the corollary above.
6.28 Corollary: Suppose V is a complex vector space and T L(V ).
Then T has an upper-triangular matrix with respect to some orthonormal basis of V .
This result is
sometimes called
Schurs theorem. The
German mathematician
Proof: This follows immediately from 5.13 and 6.27.
Issai Schur published

the rst proof of this
Orthogonal Projections and

Minimization Problems
If U is a subset of V , then the orthogonal complement of U, denoted U , is the set of all vectors in V that are orthogonal to every
vector in U:
U = {v V : v, u = 0 for all u U}.
You should verify that U is always a subspace of V , that V = {0},
and that {0} = V . Also note that if U1 U2 , then U1 U2 .
Recall that if U1 , U2 are subspaces of V , then V is the direct sum of
U1 and U2 (written V = U1 U2 ) if each element of V can be written in
exactly one way as a vector in U1 plus a vector in U2 . The next theorem
shows that every subspace of an inner-product space leads to a natural
direct sum decomposition of the whole space.
6.29
Theorem: If U is a subspace of V , then

V = U U .
Proof: Suppose that U is a subspace of V . First we will show that

6.30
V = U + U .
To do this, suppose v V . Let (e1 , . . . , em ) be an orthonormal basis

of U. Obviously
result in 1909.
112
6.31
v = v, e1 e1 + + v, em em + v v, e1 e1 v, em em .

u
Clearly u U. Because (e1 , . . . , em ) is an orthonormal list, for each j

we have
w, ej = v, ej v, ej
= 0.
Thus w is orthogonal to every vector in span(e1 , . . . , em ). In other
words, w U . Thus we have written v = u + w, where u U
and w U , completing the proof of 6.30.
If v U U , then v (which is in U) is orthogonal to every vector
in U (including v itself), which implies that v, v = 0, which implies
that v = 0. Thus
6.32
U U = {0}.
Now 6.30 and 6.32 imply that V = U U (see 1.9).

The next corollary is an important consequence of the last theorem.
6.33
Corollary: If U is a subspace of V , then

U = (U ) .
Proof: Suppose that U is a subspace of V . First we will show that

6.34
U (U ) .
To do this, suppose that u U . Then u, v = 0 for every v U (by

the denition of U ). Because u is orthogonal to every vector in U ,
we have u (U ) , completing the proof of 6.34.
To prove the inclusion in the other direction, suppose v (U ) .
By 6.29, we can write v = u + w, where u U and w U . We have
v u = w U . Because v (U ) and u (U ) (from 6.34), we
have v u (U ) . Thus v u U (U ) , which implies that v u
is orthogonal to itself, which implies that v u = 0, which implies that
v = u, which implies that v U . Thus (U ) U, which along with
6.34 completes the proof.
113
Suppose U is a subspace of V . The decomposition V = U U given

by 6.29 means that each vector v V can be written uniquely in the
form
v = u + w,
where u U and w U . We use this decomposition to dene an operator on V , denoted PU , called the orthogonal projection of V onto U .
For v V , we dene PU v to be the vector u in the decomposition above.
In the notation introduced in the last chapter, we have PU = PU ,U . You
should verify that PU L(V ) and that it has the following properties:
range PU = U;
null PU = U ;
v PU v U for every v V ;
P U 2 = PU ;
PU v v for every v V .
Furthermore, from the decomposition 6.31 used in the proof of 6.29
we see that if (e1 , . . . , em ) is an orthonormal basis of U , then
6.35
PU v = v, e1 e1 + + v, em em
for every v V .
The following problem often arises: given a subspace U of V and
a point v V , nd a point u U such that v u is as small as
possible. The next proposition shows that this minimization problem
is solved by taking u = PU v.
6.36
Proposition: Suppose U is a subspace of V and v V . Then

v PU v v u
for every u U. Furthermore, if u U and the inequality above is an

equality, then u = PU v.
The remarkable
simplicity of the
solution to this
minimization problem
has led to many
applications of
Proof: Suppose u U. Then

6.37
v PU v2 v PU v2 + PU v u2
6.38
= (v PU v) + (PU v u)2

= v u2 ,
inner-product spaces
outside of pure
mathematics.
114
where 6.38 comes from the Pythagorean theorem (6.3), which applies
because v PU v U and PU v u U . Taking square roots gives the
desired inequality.
Our inequality is an equality if and only if 6.37 is an equality, which
happens if and only if PU v u = 0, which happens if and only if
u = PU v.
v
U
PU v
PU v is the closest point in U to v.

The last proposition is often combined with the formula 6.35 to
compute explicit solutions to minimization problems. As an illustration of this procedure, consider the problem of nding a polynomial u
with real coefcients and degree at most 5 that on the interval [ , ]
approximates sin x as well as possible, in the sense that

| sin x u(x)|2 dx
is as small as possible. To solve this problem, let C[ , ] denote the

real vector space of continuous real-valued functions on [ , ] with
inner product

6.39
f , g =
f (x)g(x) dx.
Let v C[ , ] be the function dened by v(x) = sin x. Let U

denote the subspace of C[ , ] consisting of the polynomials with
real coefcients and degree at most 5. Our problem can now be reformulated as follows: nd u U such that v u is as small as
possible.
To compute the solution to our approximation problem, rst apply
the Gram-Schmidt procedure (using the inner product given by 6.39)
to the basis (1, x, x 2 , x 3 , x 4 , x 5 ) of U, producing an orthonormal basis

(e1 , e2 , e3 , e4 , e5 , e6 ) of U. Then, again using the inner product given
by 6.39, compute PU v using 6.35 (with m = 6). Doing this computation
shows that PU v is the function
6.40
0.987862x 0.155271x 3 + 0.00564312x 5 ,
where the s that appear in the exact answer have been replaced with
a good decimal approximation.
By 6.36, the polynomial above should be about as good an approximation to sin x on [ , ] as is possible using polynomials of degree
at most 5. To see how good this approximation is, the picture below
shows the graphs of both sin x and our approximation 6.40 over the
interval [ , ].
1
0.5
-3
-2
-1
-0.5
-1
Graphs of sin x and its approximation 6.40

Our approximation 6.40 is so accurate that the two graphs are almost
identicalour eyes may see only one graph!
Another well-known approximation to sin x by a polynomial of degree 5 is given by the Taylor polynomial
6.41
x5
x3
+
.
3!
5!
To see how good this approximation is, the next picture shows the
graphs of both sin x and the Taylor polynomial 6.41 over the interval
[ , ].
115
A machine that can

perform integrations is
useful here.
116
0.5
-3
-2
-1
-0.5
-1
Graphs of sin x and the Taylor polynomial 6.41

The Taylor polynomial is an excellent approximation to sin x for x
near 0. But the picture above shows that for |x| > 2, the Taylor polynomial is not so accurate, especially compared to 6.40. For example,
taking x = 3, our approximation 6.40 estimates sin 3 with an error of
about 0.001, but the Taylor series 6.41 estimates sin 3 with an error of
about 0.4. Thus at x = 3, the error in the Taylor series is hundreds of
times larger than the error given by 6.40. Linear algebra has helped us
discover an approximation to sin x that improves upon what we learned
in calculus!
We derived our approximation 6.40 by using 6.35 and 6.36. Our
standing assumption that V is nite dimensional fails when V equals
C[ , ], so we need to justify our use of those results in this case.
First, reread the proof of 6.29, which states that if U is a subspace of V ,
then
6.42
If we allow V to be
innite dimensional
and allow U to be an
innite-dimensional
subspace of V , then
6.42 is not necessarily
true without additional
hypotheses.
V = U U .
Note that the proof uses the nite dimensionality of U (to get a basis
of U) but that it works ne regardless of whether or not V is nite
dimensional. Second, note that the denition and properties of PU (including 6.35) require only 6.29 and thus require only that U (but not
necessarily V ) be nite dimensional. Finally, note that the proof of 6.36
does not require the nite dimensionality of V . Conclusion: for v V
and U a subspace of V , the procedure discussed above for nding the
vector u U that makes v u as small as possible works if U is nite
dimensional, regardless of whether or not V is nite dimensional. In
the example above U was indeed nite dimensional (we had dim U = 6),
so everything works as expected.
Linear Functionals and Adjoints

A linear functional on V is a linear map from V to the scalars F.
For example, the function : F3 F dened by
(z1 , z2 , z3 ) = 2z1 5z2 + z3
6.43
is a linear functional on F3 . As another example, consider the innerproduct space P6 (R) (here the inner product is multiplication followed
by integration on [0, 1]; see 6.2). The function : P6 (R) R dened
by
1
6.44
(p) =
p(x)(cos x) dx
0
is a linear functional on P6 (R).

If v V , then the map that sends u to u, v is a linear functional
on V . The next result shows that every linear functional on V is of this
form. To illustrate this theorem, note that for the linear functional
dened by 6.43, we can take v = (2, 5, 1) F3 . The linear functional
dened by 6.44 better illustrates the power of the theorem below because for this linear functional, there is no obvious candidate for v (the
function cos x is not eligible because it is not an element of P6 (R)).
6.45 Theorem: Suppose is a linear functional on V . Then there is
a unique vector v V such that
(u) = u, v
for every u V .
Proof: First we show that there exists a vector v V such that
(u) = u, v for every u V . Let (e1 , . . . , en ) be an orthonormal
basis of V . Then
(u) = (u, e1 e1 + + u, en en )
= u, e1 (e1 ) + + u, en (en )
= u, (e1 )e1 + + (en )en
for every u V , where the rst equality comes from 6.17. Thus setting
v = (e1 )e1 + + (en )en , we have (u) = u, v for every u V ,
as desired.
117
118
Now we prove that only one vector v V has the desired behavior.
Suppose v1 , v2 V are such that
(u) = u, v1 = u, v2
for every u V . Then
0 = u, v1 u, v2 = u, v1 v2
for every u V . Taking u = v1 v2 shows that v1 v2 = 0. In other
words, v1 = v2 , completing the proof of the uniqueness part of the
theorem.
In addition to V , we need another nite-dimensional inner-product
space.
Lets agree that for the rest of this chapter
W is a nite-dimensional, nonzero, inner-product space over F.
The word adjoint has
another meaning in
linear algebra. We will
not need the second
meaning, related to
inverses, in this book.
Just in case you
encountered the
second meaning for
adjoint elsewhere, be
warned that the two
meanings for adjoint
are unrelated to one
another.
Let T L(V , W ). The adjoint of T , denoted T , is the function from

W to V dened as follows. Fix w W . Consider the linear functional
on V that maps v V to T v, w. Let T w be the unique vector in V
such that this linear functional is given by taking inner products with
T w (6.45 guarantees the existence and uniqueness of a vector in V
with this property). In other words, T w is the unique vector in V
such that
T v, w = v, T w
for all v V .
Lets work out an example of how the adjoint is computed. Dene
T : R 3 R 2 by
T (x1 , x2 , x3 ) = (x2 + 3x3 , 2x1 ).
Thus T will be a function from R2 to R 3 . To compute T , x a point
(y1 , y2 ) R 2 . Then
(x1 , x2 , x3 ), T (y1 , y2 ) = T (x1 , x2 , x3 ), (y1 , y2 )
= (x2 + 3x3 , 2x1 ), (y1 , y2 )
= x2 y1 + 3x3 y1 + 2x1 y2
= (x1 , x2 , x3 ), (2y2 , y1 , 3y1 )
for all (x1 , x2 , x3 ) R 3 . This shows that
119
T (y1 , y2 ) = (2y2 , y1 , 3y1 ).

Note that in the example above, T turned out to be not just a function from R 2 to R3 , but a linear map. That is true in general. Specifically, if T L(V , W ), then T L(W , V ). To prove this, suppose
T L(V , W ). Lets begin by checking additivity. Fix w1 , w2 W .
Then
T v, w1 + w2 = T v, w1 + T v, w2
= v, T w1 + v, T w2
= v, T w1 + T w2 ,
which shows that T w1 +T w2 plays the role required of T (w1 +w2 ).
Because only one vector can behave that way, we must have
T w1 + T w2 = T (w1 + w2 ).
Now lets check the homogeneity of T . If a F, then
v, w
T v, aw = aT
= av,
T w
= v, aT w,
which shows that aT w plays the role required of T (aw). Because
only one vector can behave that way, we must have
aT w = T (aw).
Thus T is a linear map, as claimed.
You should verify that the function T T has the following properties:
additivity
(S + T ) = S + T for all S, T L(V , W );
conjugate homogeneity
for all a F and T L(V , W );
(aT ) = aT
adjoint of adjoint
(T ) = T for all T L(V , W );
identity
I = I, where I is the identity operator on V ;
Adjoints play a crucial

role in the important
results in the next
chapter.
120
products
(ST ) = T S for all T L(V , W ) and S L(W , U) (here U is an
inner-product space over F).
The next result shows the relationship between the null space and
the range of a linear map and its adjoint. The symbol means if and
only if; this symbol could also be read to mean is equivalent to.
6.46
Proposition: Suppose T L(V , W ). Then
(a)
null T = (range T ) ;
(b)
range T = (null T ) ;
(c)
null T = (range T ) ;
(d)
range T = (null T ) .
Proof: Lets begin by proving (a). Let w W . Then
w null T T w = 0
v, T w = 0 for all v V
T v, w = 0 for all v V
w (range T ) .
Thus null T = (range T ) , proving (a).

If we take the orthogonal complement of both sides of (a), we get (d),
where we have used 6.33. Finally, replacing T with T in (a) and (d) gives
(c) and (b).
If F = R, then the
conjugate transpose of
a matrix is the same as
its transpose, which is
the matrix obtained by
interchanging the rows
and columns.
The conjugate transpose of an m-by-n matrix is the n-by-m matrix

obtained by interchanging the rows and columns and then taking the
complex conjugate of each entry. For example, the conjugate transpose
of

2 3 + 4i 7
6
5
8i
is the matrix
3 4i
7
5 .
8i
The next proposition shows how to compute the matrix of T from

the matrix of T . Caution: the proposition below applies only when
121
we are dealing with orthonormal baseswith respect to nonorthonormal bases, the matrix of T does not necessarily equal the conjugate
transpose of the matrix of T .
6.47 Proposition: Suppose T L(V , W ). If (e1 , . . . , en ) is an orthonormal basis of V and (f1 , . . . , fm ) is an orthonormal basis of W ,
then

M T , (f1 , . . . , fm ), (e1 , . . . , en )
is the conjugate transpose of

M T , (e1 , . . . , en ), (f1 , . . . , fm ) .
Proof: Suppose that (e1 , . . . , en ) is an orthonormal basis of V and
(f1 , . . . , fm ) is an orthonormal basis of W . We write M(T ) instead of the

longer expression M T , (e1 , . . . , en ), (f1 , . . . , fm ) ; we also write M(T )

instead of M T , (f1 , . . . , fm ), (e1 , . . . , en ) .
Recall that we obtain the kth column of M(T ) by writing T ek as a linear combination of the fj s; the scalars used in this linear combination
then become the kth column of M(T ). Because (f1 , . . . , fm ) is an orthonormal basis of W , we know how to write T ek as a linear combination
of the fj s (see 6.17):
T ek = T ek , f1 f1 + + T ek , fm fm .
Thus the entry in row j, column k, of M(T ) is T ek , fj . Replacing T
with T and interchanging the roles played by the es and f s, we see
that the entry in row j, column k, of M(T ) is T fk , ej , which equals
fk , T ej , which equals T ej , fk , which equals the complex conjugate
of the entry in row k, column j, of M(T ). In other words, M(T ) equals
the conjugate transpose of M(T ).
The adjoint of a linear

map does not depend
on a choice of basis.
This explains why we
will emphasize adjoints
of linear maps instead
of conjugate
transposes of matrices.
122
Exercises
1.
Prove that if x, y are nonzero vectors in R2 , then

x, y = xy cos ,
where is the angle between x and y (thinking of x and y as
arrows with initial point at the origin). Hint: draw the triangle
formed by x, y, and x y; then use the law of cosines.
2.
Suppose u, v V . Prove that u, v = 0 if and only if

u u + av
for all a F.
3.
Prove that
n

aj b j
2
j=1
n

jaj 2
j=1
n

bj 2
j
j=1
for all real numbers a1 , . . . , an and b1 , . . . , bn .

4.
Suppose u, v V are such that

u = 3,
u + v = 4,
u v = 6.
What number must v equal?

5.
Prove or disprove: there is an inner product on R2 such that the

associated norm is given by
(x1 , x2 ) = |x1 | + |x2 |
for all (x1 , x2 ) R 2 .
6.
Prove that if V is a real inner-product space, then

u, v =
u + v2 u v2
4
for all u, v V .
7.
Prove that if V is a complex inner-product space, then

u, v =
u + v2 u v2 + u + iv2 i u iv2 i

4
for all u, v V .
Exercises
8.
A norm on a vector space U is a function : U [0, ) such

that u = 0 if and only if u = 0, u = ||u for all F
and all u U, and u + v u + v for all u, v U . Prove
that a norm satisfying the parallelogram equality comes from
an inner product (in other words, show that if is a norm
on U satisfying the parallelogram equality, then there is an inner
product , on U such that u = u, u1/2 for all u U ).
9.
Suppose n is a positive integer. Prove that

1
sin nx cos x cos 2x
cos nx
sin x sin 2x
,...,
, ,
,...,
, ,
2
is an orthonormal list of vectors in C[ , ], the vector space of
continuous real-valued functions on [ , ] with inner product

f , g =
f (x)g(x) dx.
10.
On P2 (R), consider the inner product given by

1
p, q =
p(x)q(x) dx.
Apply the Gram-Schmidt procedure to the basis (1, x, x 2 ) to produce an orthonormal basis of P2 (R).
11.
What happens if the Gram-Schmidt procedure is applied to a list

of vectors that is not linearly independent?
12.
Suppose V is a real inner-product space and (v1 , . . . , vm ) is a

linearly independent list of vectors in V . Prove that there exist
exactly 2m orthonormal lists (e1 , . . . , em ) of vectors in V such
that
span(v1 , . . . , vj ) = span(e1 , . . . , ej )
for all j {1, . . . , m}.
13.
Suppose (e1 , . . . , em ) is an orthonormal list of vectors in V . Let

v V . Prove that
v2 = |v, e1 |2 + + |v, em |2
if and only if v span(e1 , . . . , em ).
123
This orthonormal list is

often used for
modeling periodic
phenomena such as
tides.
124
14.
Find an orthonormal basis of P2 (R) (with inner product as in

Exercise 10) such that the differentiation operator (the operator
that takes p to p ) on P2 (R) has an upper-triangular matrix with
respect to this basis.
15.
Suppose U is a subspace of V . Prove that

dim U = dim V dim U .
16.
Suppose U is a subspace of V . Prove that U = {0} if and only if

U = V.
17.
Prove that if P L(V ) is such that P 2 = P and every vector

in null P is orthogonal to every vector in range P , then P is an
orthogonal projection.
18.
Prove that if P L(V ) is such that P 2 = P and

P v v
for every v V , then P is an orthogonal projection.
19.
Suppose T L(V ) and U is a subspace of V . Prove that U is

invariant under T if and only if PU T PU = T PU .
20.
Suppose T L(V ) and U is a subspace of V . Prove that U and

U are both invariant under T if and only if PU T = T PU .
21.
In R 4 , let

U = span (1, 1, 0, 0), (1, 1, 1, 2) .
Find u U such that u (1, 2, 3, 4) is as small as possible.

22.
Find p P3 (R) such that p(0) = 0, p (0) = 0, and

1
|2 + 3x p(x)|2 dx
0
is as small as possible.
23.
Find p P5 (R) that makes

| sin x p(x)|2 dx
as small as possible. (The polynomial 6.40 is an excellent approximation to the answer to this exercise, but here you are asked to
nd the exact solution, which involves powers of . A computer
that can perform symbolic integration will be useful.)
Exercises
24.
125
Find a polynomial q P2 (R) such that

1
1
p( ) =
p(x)q(x) dx
2
0
for every p P2 (R).
25.
Find a polynomial q P2 (R) such that

1
1
p(x)(cos x) dx =
p(x)q(x) dx
0
for every p P2 (R).

26.
Fix a vector v V and dene T L(V , F) by T u = u, v. For

a F, nd a formula for T a.
27.
Suppose n is a positive integer. Dene T L(F n ) by

T (z1 , . . . , zn ) = (0, z1 , . . . , zn1 ).
Find a formula for T (z1 , . . . , zn ).
28.
Suppose T L(V ) and F. Prove that is an eigenvalue of T

is an eigenvalue of T .
if and only if
29.
Suppose T L(V ) and U is a subspace of V . Prove that U is

invariant under T if and only if U is invariant under T .
30.
Suppose T L(V , W ). Prove that
31.
(a)
T is injective if and only if T is surjective;
(b)
T is surjective if and only if T is injective.
Prove that
dim null T = dim null T + dim W dim V
and
dim range T = dim range T
for every T L(V , W ).
32.
Suppose A is an m-by-n matrix of real numbers. Prove that the

dimension of the span of the columns of A (in Rm ) equals the
dimension of the span of the rows of A (in R n ).
Chapter 7
Operators on
The deepest results related to inner-product spaces deal with the

subject to which we now turnoperators on inner-product spaces. By
exploiting properties of the adjoint, we will develop a detailed description of several important classes of operators on inner-product spaces.
Lets agree that for this chapter
V is a nite-dimensional, nonzero, inner-product space over F.
127
Chapter 7. Operators on Inner-Product Spaces
128
Self-Adjoint and Normal Operators

Instead of self-adjoint,
some mathematicians
use the term Hermitian
(in honor of the French
mathematician Charles
Hermite, who in 1873
published the rst
proof that e is not the
root of any polynomial
with integer
coefcients).
If F = R, then by
An operator T L(V ) is called self-adjoint if T = T . For example,

if T is the operator on F2 whose matrix (with respect to the standard
basis) is

2 b
,
3 7
then T is self-adjoint if and only if b = 3 (because M(T ) = M(T ) if and
only if b = 3; recall that M(T ) is the conjugate transpose of M(T )
see 6.47).
You should verify that the sum of two self-adjoint operators is selfadjoint and that the product of a real scalar and a self-adjoint operator
is self-adjoint.
A good analogy to keep in mind (especially when F = C) is that
the adjoint on L(V ) plays a role similar to complex conjugation on C.
; thus a self-adjoint
A complex number z is real if and only if z = z
operator (T = T ) is analogous to a real number. We will see that

this analogy is reected in some important properties of self-adjoint
operators, beginning with eigenvalues.
7.1
Proposition: Every eigenvalue of a self-adjoint operator is real.
denition every
eigenvalue is real, so
this proposition is
interesting only when
F = C.
Proof: Suppose T is a self-adjoint operator on V . Let be an

eigenvalue of T , and let v be a nonzero vector in V such that T v = v.
Then
v2 = v, v
= T v, v
= v, T v
= v, v
2
.
= v
which means that is real, as desired.

Thus = ,
The next proposition is false for real inner-product spaces. As an
example, consider the operator T L(R 2 ) that is a counterclockwise
rotation of 90 around the origin; thus T (x, y) = (y, x). Obviously
T v is orthogonal to v for every v R 2 , even though T is not 0.
129
7.2
Proposition: If V is a complex inner-product space and T is an
operator on V such that
T v, v = 0
for all v V , then T = 0.
Proof: Suppose V is a complex inner-product space and T L(V ).
Then
T u, w =
T (u + w), u + w T (u w), u w
4
T (u + iw), u + iw T (u iw), u iw
+
i
4
for all u, w V , as can be veried by computing the right side. Note

that each term on the right side is of the form T v, v for appropriate
v V . If T v, v = 0 for all v V , then the equation above implies that
T u, w = 0 for all u, w V . This implies that T = 0 (take w = T u).
The following corollary is false for real inner-product spaces, as
shown by considering any operator on a real inner-product space that
is not self-adjoint.
7.3
Corollary: Let V be a complex inner-product space and let
T L(V ). Then T is self-adjoint if and only if
T v, v R
This corollary provides

another example of
how self-adjoint
operators behave like
real numbers.
for every v V .
Proof: Let v V . Then
T v, v T v, v = T v, v v, T v
= T v, v T v, v
= (T T )v, v.
If T v, v R for every v V , then the left side of the equation above
equals 0, so (T T )v, v = 0 for every v V . This implies that
T T = 0 (by 7.2), and hence T is self-adjoint.
Conversely, if T is self-adjoint, then the right side of the equation
above equals 0, so T v, v = T v, v for every v V . This implies that
T v, v R for every v V , as desired.
130
On a real inner-product space V , a nonzero operator T may satisfy

T v, v = 0 for all v V . However, the next proposition shows that
this cannot happen for a self-adjoint operator.
7.4
Proposition: If T is a self-adjoint operator on V such that

T v, v = 0
for all v V , then T = 0.

Proof: We have already proved this (without the hypothesis that
T is self-adjoint) when V is a complex inner-product space (see 7.2).
Thus we can assume that V is a real inner-product space and that T is
a self-adjoint operator on V . For u, w V , we have
7.5
T u, w =
T (u + w), u + w T (u w), u w
;
4
this is proved by computing the right side, using

T w, u = w, T u
= T u, w,
where the rst equality holds because T is self-adjoint and the second
equality holds because we are working on a real inner-product space.
If T v, v = 0 for all v V , then 7.5 implies that T u, w = 0 for all
u, w V . This implies that T = 0 (take w = T u).
An operator on an inner-product space is called normal if it commutes with its adjoint; in other words, T L(V ) is normal if
T T = T T .
Obviously every self-adjoint operator is normal. For an example of a
normal operator that is not self-adjoint, consider the operator on F2
whose matrix (with respect to the standard basis) is

2 3
.
3 2
Clearly this operator is not self-adjoint, but an easy calculation (which
you should do) shows that it is normal.
We will soon see why normal operators are worthy of special attention. The next proposition provides a simple characterization of
normal operators.
7.6
Proposition: An operator T L(V ) is normal if and only if
131
Note that this
proposition implies
T v = T v
that null T = null T

for every normal
for all v V .
operator T .
Proof: Let T L(V ). We will prove both directions of this result

at the same time. Note that
T is normal T T T T = 0
(T T T T )v, v = 0 for all v V
T T v, v = T T v, v for all v V
T v2 = T v2
for all v V ,
where we used 7.4 to establish the second equivalence (note that the
operator T T T T is self-adjoint). The equivalence of the rst and
last conditions above gives the desired result.
Compare the next corollary to Exercise 28 in the previous chapter.
That exercise implies that the eigenvalues of the adjoint of any operator
are equal (as a set) to the complex conjugates of the eigenvalues of the
operator. The exercise says nothing about eigenvectors because an
operator and its adjoint may have different eigenvectors. However, the
next corollary implies that a normal operator and its adjoint have the
same eigenvectors.
7.7
Corollary: Suppose T L(V ) is normal. If v V is an eigenvector of T with eigenvalue F, then v is also an eigenvector of T
with eigenvalue .
Proof: Suppose v V is an eigenvector of T with eigenvalue .
Thus (T I)v = 0. Because T is normal, so is T I, as you should
verify. Using 7.6, we have
0 = (T I)v = (T I) v = (T I)v,

as desired.
and hence v is an eigenvector of T with eigenvalue ,
Because every self-adjoint operator is normal, the next result applies
in particular to self-adjoint operators.
132
7.8
Corollary: If T L(V ) is normal, then eigenvectors of T
corresponding to distinct eigenvalues are orthogonal.
Proof: Suppose T L(V ) is normal and , are distinct eigenvalues of T , with corresponding eigenvectors u, v. Thus T u = u and
Thus
T v = v. From 7.7 we have T v = v.
( )u, v = u, v u, v

= T u, v u, T v
= 0.
Because = , the equation above implies that u, v = 0. Thus u and
v are orthogonal, as desired.
The Spectral Theorem

Recall that a diagonal matrix is a square matrix that is 0 everywhere
except possibly along the diagonal. Recall also that an operator on V
has a diagonal matrix with respect to some basis if and only if there is
a basis of V consisting of eigenvectors of the operator (see 5.21).
The nicest operators on V are those for which there is an orthonormal basis of V with respect to which the operator has a diagonal
matrix. These are precisely the operators T L(V ) such that there is
an orthonormal basis of V consisting of eigenvectors of T . Our goal
in this section is to prove the spectral theorem, which characterizes
these operators as the normal operators when F = C and as the selfadjoint operators when F = R. The spectral theorem is probably the
most useful tool in the study of operators on inner-product spaces.
Because the conclusion of the spectral theorem depends on F, we
will break the spectral theorem into two pieces, called the complex
spectral theorem and the real spectral theorem. As is often the case in
linear algebra, complex vector spaces are easier to deal with than real
vector spaces, so we present the complex spectral theorem rst.
As an illustration of the complex spectral theorem, consider the
normal operator T L(C2 ) whose matrix (with respect to the standard
basis) is

2 3
.
3 2
You should verify that
(i, 1) (i, 1)
,
2
2
133
is an orthonormal basis of C2 consisting of eigenvectors of T and that

with respect to this basis, the matrix of T is the diagonal matrix

2 + 3i
0
.
0
2 3i
7.9
Complex Spectral Theorem: Suppose that V is a complex
inner-product space and T L(V ). Then V has an orthonormal basis
consisting of eigenvectors of T if and only if T is normal.
Because every
self-adjoint operator is
normal, the complex
spectral theorem
Proof: First suppose that V has an orthonormal basis consisting of

eigenvectors of T . With respect to this basis, T has a diagonal matrix.
The matrix of T (with respect to the same basis) is obtained by taking
the conjugate transpose of the matrix of T ; hence T also has a diagonal matrix. Any two diagonal matrices commute; thus T commutes
with T , which means that T must be normal, as desired.
To prove the other direction, now suppose that T is normal. There
is an orthonormal basis (e1 , . . . , en ) of V with respect to which T has
an upper-triangular matrix (by 6.28). Thus we can write
a1,1 . . . a1,n

..
..
7.10
M T , (e1 , . . . , en ) =
.
. .
0
an,n
We will show that this matrix is actually a diagonal matrix, which means
that (e1 , . . . , en ) is an orthonormal basis of V consisting of eigenvectors
of T .
We see from the matrix above that
T e1 2 = |a1,1 |2
and
T e1 2 = |a1,1 |2 + |a1,2 |2 + + |a1,n |2 .
Because T is normal, T e1 = T e1 (see 7.6). Thus the two equations
above imply that all entries in the rst row of the matrix in 7.10, except
possibly the rst entry a1,1 , equal 0.
Now from 7.10 we see that
T e2 2 = |a2,2 |2
implies that every

self-adjoint operator on
a nite-dimensional
complex inner-product
space has a diagonal
matrix with respect to
some orthonormal
basis.
134
(because a1,2 = 0, as we showed in the paragraph above) and

T e2 2 = |a2,2 |2 + |a2,3 |2 + + |a2,n |2 .
Because T is normal, T e2 = T e2 . Thus the two equations above
imply that all entries in the second row of the matrix in 7.10, except
possibly the diagonal entry a2,2 , equal 0.
Continuing in this fashion, we see that all the nondiagonal entries
in the matrix 7.10 equal 0, as desired.
This technique of
We will need two lemmas for our proof of the real spectral theorem. You could guess that the next lemma is true and even discover its
proof by thinking about quadratic polynomials with real coefcients.
Specically, suppose , R and 2 < 4. Let x be a real number.
Then
completing the square

2
2
x 2 + x + = x +
+
2
4
> 0.
can be used to derive

the quadratic formula.
In particular, x 2 + x + is an invertible real number (a convoluted

way of saying that it is not 0). Replacing the real number x with a
self-adjoint operator (recall the analogy between real numbers and selfadjoint operators), we are led to the lemma below.
7.11 Lemma: Suppose T L(V ) is self-adjoint. If , R are such
that 2 < 4, then
T 2 + T + I
is invertible.
Proof: Suppose , R are such that 2 < 4. Let v be a nonzero
vector in V . Then
(T 2 + T + I)v, v = T 2 v, v + T v, v + v, v
= T v, T v + T v, v + v2
T v2 ||T vv + v2

||v 2
2
v2
= T v
+
2
4
> 0,
135
where the rst inequality holds by the Cauchy-Schwarz inequality (6.6).

The last inequality implies that (T 2 + T + I)v = 0. Thus T 2 + T + I
is injective, which implies that it is invertible (see 3.21).
We have proved that every operator, self-adjoint or not, on a nitedimensional complex vector space has an eigenvalue (see 5.10), so the
next lemma tells us something new only for real inner-product spaces.
7.12 Lemma:
eigenvalue.
Suppose T L(V ) is self-adjoint. Then T has an
Proof: As noted above, we can assume that V is a real innerproduct space. Let n = dim V and choose v V with v = 0. Then
(v, T v, T 2 v, . . . , T n v)
n + 1 vectors. Thus there exist real numbers a0 , . . . , an , not all 0, such
that
0 = a0 v + a1 T v + + an T n v.
Make the as the coefcients of a polynomial, which can be written in
factored form (see 4.14) as
a0 + a1 x + + an x n
= c(x 2 + 1 x + 1 ) . . . (x 2 + M x + M )(x 1 ) . . . (x m ),
where c is a nonzero real number, each j , j , and j is real, each
j 2 < 4j , m + M 1, and the equation holds for all real x. We then
have
0 = a0 v + a1 T v + + an T n v
= (a0 I + a1 T + + an T n )v
= c(T 2 + 1 T + 1 I) . . . (T 2 + M T + M I)(T 1 I) . . . (T m I)v.
Each T 2 + j T + j I is invertible because T is self-adjoint and each
j 2 < 4j (see 7.11). Recall also that c = 0. Thus the equation above
implies that
0 = (T 1 I) . . . (T m I)v.
Hence T j I is not injective for at least one j. In other words, T has
an eigenvalue.
Here we are imitating

the proof that T has an
invariant subspace of
dimension 1 or 2
(see 5.24).
136
As an illustration of the real spectral theorem, consider the selfadjoint operator T on R 3 whose matrix (with respect to the standard
basis) is
14 13 8
8 .
13 14
8
8
7
You should verify that

(1, 1, 0) (1, 1, 1) (1, 1, 2)
,
,
3
6
2
is an orthonormal basis of R 3 consisting of eigenvectors of T and that

with respect to this basis, the matrix of T is the diagonal matrix
27
0
0
0
9
0
0 .
15
Combining the complex spectral theorem and the real spectral theorem, we conclude that every self-adjoint operator on V has a diagonal
matrix with respect to some orthonormal basis. This statement, which
is the most useful part of the spectral theorem, holds regardless of
whether F = C or F = R.
7.13 Real Spectral Theorem: Suppose that V is a real inner-product
space and T L(V ). Then V has an orthonormal basis consisting of
eigenvectors of T if and only if T is self-adjoint.
Proof: First suppose that V has an orthonormal basis consisting of
eigenvectors of T . With respect to this basis, T has a diagonal matrix.
This matrix equals its conjugate transpose. Hence T = T and so T is
self-adjoint, as desired.
To prove the other direction, now suppose that T is self-adjoint. We
will prove that V has an orthonormal basis consisting of eigenvectors
of T by induction on the dimension of V . To get started, note that our
desired result clearly holds if dim V = 1. Now assume that dim V > 1
and that the desired result holds on vector spaces of smaller dimension.
The idea of the proof is to take any eigenvector u of T with norm 1,
then adjoin to it an orthonormal basis of eigenvectors of T |{u} . Now
for the details, the most important of which is verifying that T |{u} is
self-adjoint (this allows us to apply our induction hypothesis).
Let be any eigenvalue of T (because T is self-adjoint, we know
from the previous lemma that it has an eigenvalue) and let u V
denote a corresponding eigenvector with u = 1. Let U denote the
one-dimensional subspace of V consisting of all scalar multiples of u.
Note that a vector v V is in U if and only if u, v = 0.
Suppose v U . Then because T is self-adjoint, we have
u, T v = T u, v = u, v = u, v = 0,
and hence T v U . Thus T v U whenever v U . In other words,
U is invariant under T . Thus we can dene an operator S L(U ) by
S = T |U . If v, w U , then
Sv, w = T v, w = v, T w = v, Sw,
which shows that S is self-adjoint (note that in the middle equality
above we used the self-adjointness of T ). Thus, by our induction hypothesis, there is an orthonormal basis of U consisting of eigenvectors of S. Clearly every eigenvector of S is an eigenvector of T (because
Sv = T v for every v U ). Thus adjoining u to an orthonormal basis
of U consisting of eigenvectors of S gives an orthonormal basis of V
consisting of eigenvectors of T , as desired.
For T L(V ) self-adjoint (or, more generally, T L(V ) normal
when F = C), the corollary below provides the nicest possible decomposition of V into subspaces invariant under T . On each null(T j I),
the operator T is just multiplication by j .
7.14 Corollary: Suppose that T L(V ) is self-adjoint (or that F = C
and that T L(V ) is normal). Let 1 , . . . , m denote the distinct eigenvalues of T . Then
V = null(T 1 I) null(T m I).
Furthermore, each vector in each null(T j I) is orthogonal to all vectors in the other subspaces of this decomposition.
Proof: The spectral theorem (7.9 and 7.13) implies that V has a
basis consisting of eigenvectors of T . The desired decomposition of V
now follows from 5.21.
The orthogonality statement follows from 7.8.
137
To get an eigenvector
of norm 1, take any
nonzero eigenvector
and divide it by its
norm.
138
Normal Operators on Real

The complex spectral theorem (7.9) gives a complete description
of normal operators on complex inner-product spaces. In this section
we will give a complete description of normal operators on real innerproduct spaces. Along the way, we will encounter a proposition (7.18)
and a technique (block diagonal matrices) that are useful for both real
and complex inner-product spaces.
We begin with a description of the operators on a two-dimensional
real inner-product space that are normal but not self-adjoint.
7.15 Lemma: Suppose V is a two-dimensional real inner-product
space and T L(V ). Then the following are equivalent:
(a)
T is normal but not self-adjoint;
(b)
the matrix of T with respect to every orthonormal basis of V

has the form

a b
,
b a
with b = 0;
(c)
the matrix of T with respect to some orthonormal basis of V has

the form

a b
,
b a
with b > 0.
Proof: First suppose that (a) holds, so that T is normal but not
self-adjoint. Let (e1 , e2 ) be an orthonormal basis of V . Suppose

a c
.
7.16
M T , (e1 , e2 ) =
b d
Then T e1 2 = a2 + b2 and T e1 2 = a2 + c 2 . Because T is normal,
T e1 = T e1 (see 7.6); thus these equations imply that b2 = c 2 .
Thus c = b or c = b. But c = b because otherwise T would be selfadjoint, as can be seen from the matrix in 7.16. Hence c = b, so

a b
.
7.17
M T , (e1 , e2 ) =
b d
139
Of course, the matrix of T is the transpose of the matrix above. Use

matrix multiplication to compute the matrices of T T and T T (do it
now). Because T is normal, these two matrices must be equal. Equating
the entries in the upper-right corner of the two matrices you computed,
you will discover that bd = ab. Now b = 0 because otherwise T would
be self-adjoint, as can be seen from the matrix in 7.17. Thus d = a,
completing the proof that (a) implies (b).
Now suppose that (b) holds. We want to prove that (c) holds. Choose
any orthonormal basis (e1 , e2 ) of V . We know that the matrix of T with
respect to this basis has the form given by (b), with b = 0. If b > 0,
then (c) holds and we have proved that (b) implies (c). If b < 0, then,
as you should verify, the
of T with respect to the orthonormal
matrix

a b
,
where
b > 0; thus in this case we also
basis (e1 , e2 ) equals b
a
see that (b) implies (c).
Now suppose that (c) holds, so that the matrix of T with respect to
some orthonormal basis has the form given in (c) with b > 0. Clearly
the matrix of T is not equal to its transpose (because b = 0), and hence
T is not self-adjoint. Now use matrix multiplication to verify that the
matrices of T T and T T are equal. We conclude that T T = T T , and
hence T is normal. Thus (c) implies (a), completing the proof.
As an example of the notation we will use to write a matrix as a
matrix of smaller matrices, consider the matrix
1 1 2 2 2
1 1 2 2 2
D=
0 0 3 3 3 .
0 0 3 3 3
0 0 3 3 3
We can write this matrix in the form

A B
D=
,
0 C
Often we can
understand a matrix
better by thinking of it
as composed of smaller
matrices. We will use
where

A=
1
1
1
1

,
B=
2
2
2
2
2
2

,
C= 3
3
and 0 denotes the 3-by-2 matrix consisting of all 0s.
3
3
3
3 ,
3
this technique in the

next proposition and in
later chapters.
140
The next result will play a key role in our characterization of the
normal operators on a real inner-product space.
Without normality, an
easier result also holds:
if T L(V ) and U
7.18 Proposition: Suppose T L(V ) is normal and U is a subspace

of V that is invariant under T . Then
invariant under T , then
(a)
U is invariant under T ;
U is invariant under
(b)
U is invariant under T ;
(c)
(T |U ) = (T )|U ;
(d)
T |U is a normal operator on U;
(e)
T |U is a normal operator on U .
T ; see Exercise 29 in
Chapter 6.
Proof: First we will prove (a). Let (e1 , . . . , em ) be an orthonormal

basis of U . Extend to an orthonormal basis (e1 , . . . , em , f1 , . . . , fn ) of V
(this is possible by 6.25). Because U is invariant under T , each T ej is
a linear combination of (e1 , . . . , em ). Thus the matrix of T with respect
to the basis (e1 , . . . , em , f1 , . . . , fn ) is of the form
M(T ) =
e1
..
.
em
f1
..
.
fn
e1
...
em f1
...
fn
here A denotes an m-by-m matrix, 0 denotes the n-by-m matrix consisting of all 0s, B denotes an m-by-n matrix, C denotes an n-by-n
matrix, and for convenience the basis has been listed along the top and
left sides of the matrix.
For each j {1, . . . , m}, T ej 2 equals the sum of the squares of the
absolute values of the entries in the j th column of A (see 6.17). Hence
7.19
j=1
T ej 2 =
the sum of the squares of the absolute

values of the entries of A.
For each j {1, . . . , m}, T ej 2 equals the sum of the squares of the
absolute values of the entries in the j th rows of A and B. Hence
7.20
T ej 2 =
j=1
the sum of the squares of the absolute

values of the entries of A and B.
Because T is normal, T ej = T ej for each j (see 7.6); thus

m
T ej 2 =
j=1
T ej 2 .
j=1
This equation, along with 7.19 and 7.20, implies that the sum of the
squares of the absolute values of the entries of B must equal 0. In
other words, B must be the matrix consisting of all 0s. Thus
7.21
M(T ) =
e1
..
.
em
f1
..
.
fn
e1
...
em f1
...
fn
This representation shows that T fk is in the span of (f1 , . . . , fn ) for

each k. Because (f1 , . . . , fn ) is a basis of U , this implies that T v U
whenever v U . In other words, U is invariant under T , completing
the proof of (a).
To prove (b), note that M(T ) has a block of 0s in the lower left
corner (because M(T ), as given above, has a block of 0s in the upper
right corner). In other words, each T ej can be written as a linear
combination of (e1 , . . . , em ). Thus U is invariant under T , completing
the proof of (b).
To prove (c), let S = T |U . Fix v U. Then
Su, v = T u, v
= u, T v
for all u U. Because T v U (by (b)), the equation above shows that
S v = T v. In other words, (T |U ) = (T )|U , completing the proof
of (c).
To prove (d), note that T commutes with T (because T is normal)
and that (T |U ) = (T )|U (by (c)). Thus T |U commutes with its adjoint
and hence is normal, completing the proof of (d).
141
142
To prove (e), note that in (d) we showed that the restriction of T to

any invariant subspace is normal. However, U is invariant under T
(by (a)), and hence T |U is normal.
The key step in the

proof of the last
proposition was
showing that M(T ) is
an appropriate block
diagonal matrix;
see 7.21.
In proving 7.18 we thought of a matrix as composed of smaller matrices. Now we need to make additional use of that idea. A block diagonal matrix is a square matrix of the form
0
A1
..
,
.
0
Am
where A1 , . . . , Am are square matrices lying along the diagonal and all
the other entries of the matrix equal 0. For example, the matrix
4 0 0 0 0
0 2 3 0 0
7.22
A=
0 3 2 0 0
0 0 0 1 7
0 0 0 7 1
is a block diagonal matrix with
A1
A=
0
where
7.23
A1 =

,
A2 =
2
3
0
A2
A3
3
2

,
A3 =
1
7
7
1
If A and B are block diagonal matrices of the form
B1
A1
0
0
.
.
..
..
A=
, B =
0
Am
0
Bm

.
where Aj has the same size as Bj for j = 1, . . . , m, then AB is a block

diagonal matrix of the form
A1 B1
0
..
,
7.24
AB =
.
0
Am Bm
as you should verify. In other words, to multiply together two block

diagonal matrices (with the same size blocks), just multiply together the
corresponding entries on the diagonal, as with diagonal matrices.
A diagonal matrix is a special case of a block diagonal matrix where
each block has size 1-by-1. At the other extreme, every square matrix is
a block diagonal matrix because we can take the rst (and only) block
to be the entire matrix. Thus to say that an operator has a block diagonal matrix with respect to some basis tells us nothing unless we
know something about the size of the blocks. The smaller the blocks,
the nicer the operator (in the vague sense that the matrix then contains
more 0s). The nicest situation is to have an orthonormal basis that
gives a diagonal matrix. We have shown that this happens on a complex inner-product space precisely for the normal operators (see 7.9)
and on a real inner-product space precisely for the self-adjoint operators (see 7.13).
Our next result states that each normal operator on a real innerproduct space comes close to having a diagonal matrixspecically,
we get a block diagonal matrix with respect to some orthonormal basis,
with each block having size at most 2-by-2. We cannot expect to do better than that because on a real inner-product space there exist normal
operators that do not have a diagonal matrix with respect to any basis.
For example, the operator T L(R2 ) dened by T (x, y) = (y, x) is
normal (as you should verify) but has no eigenvalues; thus this particular T does not have even an upper-triangular matrix with respect to
any basis of R 2 .
Note that the matrix in 7.22 is the type of matrix promised by the
theorem below. In particular, each block of 7.22 (see 7.23) has size
at most 2-by-2 and each of the 2-by-2 blocks has the required form
(upper left entry equals lower right entry, lower left entry is positive,
and upper right entry equals the negative of lower left entry).
7.25 Theorem: Suppose that V is a real inner-product space and
T L(V ). Then T is normal if and only if there is an orthonormal
basis of V with respect to which T has a block diagonal matrix where
each block is a 1-by-1 matrix or a 2-by-2 matrix of the form

a b
7.26
,
b a
with b > 0.
143
Note that if an operator

T has a block diagonal
matrix with respect to
some basis, then the
entry in any 1-by-1
block on the diagonal
of this matrix must be
an eigenvalue of T .
144
In a real vector space

with dimension 1, there
are precisely two
vectors with norm 1.
Proof: To prove the easy direction, rst suppose that there is an

orthonormal basis of V such that the matrix of T is a block diagonal
matrix where each block is a 1-by-1 matrix or a 2-by-2 matrix of the
form 7.26. With respect to this basis, the matrix of T commutes with
the matrix of T (which is the conjugate of the matrix of T ), as you
should verify (use formula 7.24 for the product of two block diagonal
matrices). Thus T commutes with T , which means that T is normal.
To prove the other direction, now suppose that T is normal. We will
prove our desired result by induction on the dimension of V . To get
started, note that our desired result clearly holds if dim V = 1 (trivially)
or if dim V = 2 (if T is self-adjoint, use the real spectral theorem 7.13;
if T is not self-adjoint, use 7.15).
Now assume that dim V > 2 and that the desired result holds on
vector spaces of smaller dimension. Let U be a subspace of V of dimension 1 that is invariant under T if such a subspace exists (in other
words, if T has a nonzero eigenvector, let U be the span of this eigenvector). If no such subspace exists, let U be a subspace of V of dimension 2 that is invariant under T (an invariant subspace of dimension 1
or 2 always exists by 5.24).
If dim U = 1, choose a vector in U with norm 1; this vector will
be an orthonormal basis of U , and of course the matrix of T |U is a
1-by-1 matrix. If dim U = 2, then T |U is normal (by 7.18) but not selfadjoint (otherwise T |U , and hence T , would have a nonzero eigenvector;
see 7.12), and thus we can choose an orthonormal basis of U with respect to which the matrix of T |U has the form 7.26 (see 7.15).
Now U is invariant under T and T |U is a normal operator on U
(see 7.18). Thus by our induction hypothesis, there is an orthonormal
basis of U with respect to which the matrix of T |U has the desired
form. Adjoining this basis to the basis of U gives an orthonormal basis
of V with respect to which the matrix of T has the desired form.
Positive Operators
Many mathematicians
also use the term
positive semidenite
operator, which means
the same as positive
operator.
An operator T L(V ) is called positive if T is self-adjoint and

T v, v 0
for all v V . Note that if V is a complex vector space, then the
condition that T be self-adjoint can be dropped from this denition
(by 7.3).
Positive Operators
You should verify that every orthogonal projection is positive. For

another set of examples, look at the proof of 7.11, where we showed
that if T L(V ) is self-adjoint and , R are such that 2 < 4,
then T 2 + T + I is positive.
An operator S is called a square root of an operator T if S 2 = T .
For example, if T L(F3 ) is dened by T (z1 , z2 , z3 ) = (z3 , 0, 0), then
the operator S L(F3 ) dened by S(z1 , z2 , z3 ) = (z2 , z3 , 0) is a square
root of T .
The following theorem is the main result about positive operators.
Note that its characterizations of the positive operators correspond to
characterizations of the nonnegative numbers among C. Specically,
a complex number z is nonnegative if and only if it has a nonnegative
square root, corresponding to condition (c) below. Also, z is nonnegative if and only if it has a real square root, corresponding to condition
(d) below. Finally, z is nonnegative if and only if there exists a complex
number w such that z = ww,

corresponding to condition (e) below.
145
The positive operators

correspond, in some
sense, to the numbers
[0, ), so better
terminology would call
these nonnegative
instead of positive.
However, operator
theorists consistently
7.27
Theorem: Let T L(V ). Then the following are equivalent:
call these the positive

operators, so we will
(a)
T is positive;
(b)
T is self-adjoint and all the eigenvalues of T are nonnegative;
(c)
T has a positive square root;
(d)
T has a self-adjoint square root;
(e)
there exists an operator S L(V ) such that T = S S.
follow that custom.
Proof: We will prove that (a) (b) (c) (d) (e) (a).
First suppose that (a) holds, so that T is positive. Obviously T is
self-adjoint (by the denition of a positive operator). To prove the other
condition in (b), suppose that is an eigenvalue of T . Let v be a nonzero
eigenvector of T corresponding to . Then
0T v, v
= v, v
= v, v,
and thus is a nonnegative number. Hence (b) holds.
Now suppose that (b) holds, so that T is self-adjoint and all the eigenvalues of T are nonnegative. By the spectral theorem (7.9 and 7.13),
146
there is an orthonormal basis (e1 , . . . , en ) of V consisting of eigenvectors of T . Let 1 , . . . , n be the eigenvalues of T corresponding to
e1 , . . . , en , so that each j is a nonnegative number. Dene S L(V )
by

Sej = j ej
for j = 1, . . . , n. Then S is a positive operator, as you should verify.

Furthermore, S 2 ej = j ej = T ej for each j, which implies that S 2 = T .
Thus S is a positive square root of T , and hence (c) holds.
Clearly (c) implies (d) (because, by denition, every positive operator
is self-adjoint).
Now suppose that (d) holds, meaning that there exists a self-adjoint
operator S on V such that T = S 2 . Then T = S S (because S = S), and
hence (e) holds.
Finally, suppose that (e) holds. Let S L(V ) be such that T = S S.
Then T = (S S) = S (S ) = S S = T , and hence T is self-adjoint.
To complete the proof that (a) holds, note that
T v, v = S Sv, v
= Sv, Sv
0
for every v V . Thus T is positive.
Each nonnegative number has a unique nonnegative square root.
The next proposition shows that positive operators enjoy a similar
property. Because of this proposition, we can use the notation T

to denote the unique positive square root of a positive operator T , just
as denotes the unique nonnegative square root of a nonnegative

number .
A positive operator can
have innitely many
7.28 Proposition: Every positive operator on V has a unique positive

square root.
square roots (though

only one of them can
be positive). For
example, the identity
operator on V has
innitely many square
roots if dim V > 1.
Proof: Suppose T L(V ) is positive. Let 1 , . . . , m denote the

distinct eigenvalues of T ; because T is positive, all these numbers are
nonnegative (by 7.27). Because T is self-adjoint, we have
7.29
see 7.14.
V = null(T 1 I) null(T m I);
Isometries
147
Now suppose S L(V ) is a positive square root of T . Suppose is

an eigenvalue of S. If v null(S I), then Sv = v, which implies
that
7.30
T v = S 2 v = 2 v,
so v null(T 2 I). Thus 2 is an eigenvalue of

T , which means
2
that must equal some j . In other words, = j for some j.
Furthermore, 7.30 implies that

7.31
null(S j I) null(T j I).
In the paragraph above, we showed that the only possible eigenval

ues for S are 1 , . . . , m . Because S is self-adjoint, this implies that

7.32
V = null(S 1 I) null(S m I);
see 7.14. Now 7.29, 7.32, and 7.31 imply that

null(S j I) = null(T j I)
for each j. In other
words, on null(T j I), the operator S is just
multiplication by j . Thus S, the positive square root of T , is uniquely
determined by T .
Isometries
An operator S L(V ) is called an isometry if
Sv = v
for all v V . In other words, an operator is an isometry if it preserves
norms. For example, I is an isometry whenever F satises || = 1.
More generally, suppose 1 , . . . , n are scalars with absolute value 1 and
S L(V ) satises S(ej ) = j ej for some orthonormal basis (e1 , . . . , en )
of V . Suppose v V . Then
7.33
v = v, e1 e1 + + v, en en
and
7.34
v2 = |v, e1 |2 + + |v, en |2 ,
The Greek word isos

means equal; the Greek
word metron means
measure. Thus
isometry literally
means equal measure.
148
where we have used 6.17. Applying S to both sides of 7.33 gives

Sv = v, e1 Se1 + + v, en Sen
= 1 v, e1 e1 + + n v, en en .
The last equation, along with the equation |j | = 1, shows that
Sv2 = |v, e1 |2 + + |v, en |2 .
7.35
An isometry on a real
inner-product space is
often called an
orthogonal operator.
An isometry on a
complex inner-product
space is often called a
unitary operator. We
will use the term
isometry so that our
results can apply to
both real and complex
inner-product spaces.
Comparing 7.34 and 7.35 shows that v = Sv. In other words, S is
an isometry.
For another example, let R. Then the operator on R 2 of counterclockwise rotation (centered at the origin) by an angle of is an
isometry (you should nd the matrix of this operator with respect to
the standard basis of R 2 ).
If S L(V ) is an isometry, then S is injective (because if Sv = 0,
then v = Sv = 0, and hence v = 0). Thus every isometry is
invertible (by 3.21).
The next theorem provides several conditions that are equivalent
to being an isometry. These equivalences have several important interpretations. In particular, the equivalence of (a) and (b) shows that
an isometry preserves inner products. Because (a) implies (d), we see
that if S is an isometry and (e1 , . . . , en ) is an orthonormal basis of V ,
then the columns of the matrix of S (with respect to this basis) are orthonormal; because (e) implies (a), we see that the converse also holds.
Because (a) is equivalent to conditions (i) and (j), we see that in the last
sentence we can replace columns with rows.
7.36
lent:
Theorem: Suppose S L(V ). Then the following are equiva-
(a)
S is an isometry;
(b)
Su, Sv = u, v for all u, v V ;
(c)
S S = I;
(d)
(Se1 , . . . , Sen ) is orthonormal whenever (e1 , . . . , en ) is an orthonormal list of vectors in V ;
(e)
there exists an orthonormal basis (e1 , . . . , en ) of V such that

(Se1 , . . . , Sen ) is orthonormal;
(f)
S is an isometry;
Isometries
(g)
S u, S v = u, v for all u, v V ;
(h)
SS = I;
(i)
(S e1 , . . . , S en ) is orthonormal whenever (e1 , . . . , en ) is an orthonormal list of vectors in V ;
(j)
there exists an orthonormal basis (e1 , . . . , en ) of V such that

(S e1 , . . . , S en ) is orthonormal.
Proof: First suppose that (a) holds. If V is a real inner-product

space, then for every u, v V we have
Su, Sv = (Su + Sv2 Su Sv2 )/4
= (S(u + v)2 S(u v)2 )/4
= (u + v2 u v2 )/4
= u, v,
where the rst equality comes from Exercise 6 in Chapter 6, the second
equality comes from the linearity of S, the third equality holds because
S is an isometry, and the last equality again comes from Exercise 6 in
Chapter 6. If V is a complex inner-product space, then use Exercise 7
in Chapter 6 instead of Exercise 6 to obtain the same conclusion. In
either case, we see that (a) implies (b).
Now suppose that (b) holds. Then
(S S I)u, v = Su, Sv u, v
=0
for every u, v V . Taking v = (S S I)u, we see that S S I = 0.
Hence S S = I, proving that (b) implies (c).
Now suppose that (c) holds. Suppose (e1 , . . . , en ) is an orthonormal
list of vectors in V . Then
Sej , Sek = S Sej , ek
= ej , ek .
Hence (Se1 , . . . , Sen ) is orthonormal, proving that (c) implies (d).
Obviously (d) implies (e).
Now suppose (e) holds. Let (e1 , . . . , en ) be an orthonormal basis of V
such that (Se1 , . . . , Sen ) is orthonormal. If v V , then
149
150

Sv2 = S v, e1 e1 + + v, en en 2
= v, e1 Se1 + + v, en Sen 2
= |v, e1 |2 + + |v, en |2
= v2 ,
where the rst and last equalities come from 6.17. Taking square roots,
we see that S is an isometry, proving that (e) implies (a).
Having shown that (a) (b) (c) (d) (e) (a), we know at this
stage that (a) through (e) are all equivalent to each other. Replacing S
with S , we see that (f) through (j) are all equivalent to each other. Thus
to complete the proof, we need only show that one of the conditions
in the group (a) through (e) is equivalent to one of the conditions in
the group (f) through (j). The easiest way to connect the two groups of
conditions is to show that (c) is equivalent to (h). In general, of course,
S need not commute with S . However, S S = I if and only if SS = I;
this is a special case of Exercise 23 in Chapter 3. Thus (c) is equivalent
to (h), completing the proof.
The last theorem shows that every isometry is normal (see (a), (c),
and (h) of 7.36). Thus the characterizations of normal operators can
be used to give complete descriptions of isometries. We do this in the
next two theorems.
7.37 Theorem: Suppose V is a complex inner-product space and
S L(V ). Then S is an isometry if and only if there is an orthonormal
basis of V consisting of eigenvectors of S all of whose corresponding
eigenvalues have absolute value 1.
Proof: We already proved (see the rst paragraph of this section)
that if there is an orthonormal basis of V consisting of eigenvectors of S
all of whose eigenvalues have absolute value 1, then S is an isometry.
To prove the other direction, suppose S is an isometry. By the complex spectral theorem (7.9), there is an orthonormal basis (e1 , . . . , en )
of V consisting of eigenvectors of S. For j {1, . . . , n}, let j be the
eigenvalue corresponding to ej . Then
|j | = j ej = Sej = ej = 1.
Thus each eigenvalue of S has absolute value 1, completing the proof.
Isometries
151
If R, then the operator on R 2 of counterclockwise rotation (centered at the origin) by an angle of has matrix 7.39 with respect to
the standard basis, as you should verify. The next result states that every isometry on a real inner-product space is composed of pieces that
look like rotations on two-dimensional subspaces, pieces that equal the
identity operator, and pieces that equal multiplication by 1.
7.38 Theorem: Suppose that V is a real inner-product space and
S L(V ). Then S is an isometry if and only if there is an orthonormal
basis of V with respect to which S has a block diagonal matrix where
each block on the diagonal is a 1-by-1 matrix containing 1 or 1 or a
2-by-2 matrix of the form

cos sin
7.39
,
sin
cos
with (0, ).
Proof: First suppose that S is an isometry. Because S is normal,
there is an orthonormal basis of V such that with respect to this basis
S has a block diagonal matrix, where each block is a 1-by-1 matrix or a
2-by-2 matrix of the form

a b
,
7.40
b a
with b > 0 (see 7.25).
If is an entry in a 1-by-1 along the diagonal of the matrix of S (with
respect to the basis mentioned above), then there is a basis vector ej
such that Sej = ej . Because S is an isometry, this implies that || = 1.
Thus = 1 or = 1 because these are the only real numbers with
absolute value 1.
Now consider a 2-by-2 matrix of the form 7.40 along the diagonal of
the matrix of S. There are basis vectors ej , ej+1 such that
Sej = aej + bej+1 .
Thus
1 = ej 2 = Sej 2 = a2 + b2 .
The equation above, along with the condition b > 0, implies that there
exists a number (0, ) such that a = cos and b = sin . Thus the
This theorem implies

that an isometry on an
odd-dimensional real
inner-product space
must have 1 or 1 as
an eigenvalue.
152
matrix 7.40 has the required form 7.39, completing the proof in this
direction.
Conversely, now suppose that there is an orthonormal basis of V
with respect to which the matrix of S has the form required by the
theorem. Thus there is a direct sum decomposition
V = U1 Um ,
where each Uj is a subspace of V of dimension 1 or 2. Furthermore,
any two vectors belonging to distinct U s are orthogonal, and each S|Uj
is an isometry mapping Uj into Uj . If v V , we can write
v = u1 + + um ,
where each uj Uj . Applying S to the equation above and then taking
norms gives
Sv2 = Su1 + + Sum 2
= Su1 2 + + Sum 2
= u1 2 + + um 2
= v2 .
Thus S is an isometry, as desired.
Polar and Singular-Value Decompositions

Recall our analogy between C and L(V ). Under this analogy, a com corresponds to T .
plex number z corresponds to an operator T , and z
The real numbers correspond to the self-adjoint operators, and the nonnegative numbers correspond to the (badly named) positive operators.
Another distinguished subset of C is the unit circle, which consists of
the complex numbers z such that |z| = 1. The condition |z| = 1 is
z = 1. Under our analogy, this would corequivalent to the condition z
respond to the condition T T = I, which is equivalent to T being an

isometry (see 7.36). In other words, the unit circle in C corresponds to
the isometries.
Continuing with our analogy, note that each complex number z except 0 can be written in the form

z
z
z,
z
|z| =
z=
|z|
|z|
153
where the rst factor, namely, z/|z|, is an element of the unit circle. Our
analogy leads us to guess that any operator T L(V ) can be written
as an isometry times T T . That guess is indeed correct, as we now

prove.
7.41 Polar Decomposition: If T L(V ), then there exists an isometry S L(V ) such that
T = S T T .
If you know a bit of

complex analysis, you
will recognize the
analogy to polar
Proof: Suppose T L(V ). If v V , then

T v2 = T v, T v
= T T v, v
= T T T T v, v
= T T v, T T v
= T T v2 .
complex numbers:
every complex number
can be written in the
form ei r , where
[0, 2 ) and r 0.
Note that ei is in the
unit circle,
corresponding to S
Thus
7.42
coordinates for
T v = T T v
for all v V .
Dene a linear map S1 : range T T range T by
7.43
S1 ( T T v) = T v.
The idea of the proof is to extend S1 to an isometry S L(V ) such that
T = S T T . Now for the details.

First we must check that S1 is well dened. To do this, suppose
v1 , v2 V are such that T T v1 = T T v2 . For the denition given

by 7.43 to make sense, we must show that T v1 = T v2 . However,
T v1 T v2 = T (v1 v2 )
= T T (v1 v2 )
= T T v 1 T T v2
= 0,
where the second equality holds by 7.42. The equation above shows
that T v1 = T v2 , so S1 is indeed well dened. You should verify that S1
is a linear map.
being an isometry, and

r is nonnegative,
corresponding to
T T being a positive
operator.
154
In the rest of the proof
all we are doing is
extending S1 to an
We see from 7.43 that S1 maps range T T onto range T . Clearly
7.42 and 7.43 imply that S1 u = u for all u range T T . In
particular, S1 is injective. Thus from 3.4, applied to S1 , we have
isometry S on all of V .
dim range T T = dim range T .
This implies that dim(range T T ) = dim(range T ) (see Exercise 15
in Chapter 6). Thus orthonormal bases (e1 , . . . , em ) of (range T T )

and (f1 , . . . , fm ) of (range T ) can be chosen; the key point here is that
these two orthonormal bases have the same length. Dene a linear map
S2 : (range T T ) (range T ) by
S2 (a1 e1 + + am em ) = a1 f1 + + am fm .
Obviously S2 w = w for all w (range T T ) .
Now let S be the operator on V that equals S1 on range T T and
equals S2 on (range T T ) . More precisely, recall that each v V

can be written uniquely in the form
v = u + w,
7.44
where u range T T and w (range T T ) (see 6.29). For v V

with decomposition as above, dene Sv by
Sv = S1 u + S2 w.
For each v V we have
S( T T v) = S1 ( T T v) = T v,
so T = S T T , as desired. All that remains is to show that S is an isometry. However, this follows easily from the two uses of the Pythagorean
theorem: if v V has decomposition as in 7.44, then
Sv2 = S1 u + S2 w2
= S1 u2 + S2 w2
= u2 + w2
= v2 ,
where the second equality above holds because S1 u range T and
S2 u (range T ) .
The polar decomposition (7.41) states that each operator on V is the

product of an isometry and a positive operator. Thus we can write each
operator on V as the product of two operators, each of which comes
from a class that we have completely described and that we understand reasonably well. The isometries are described by 7.37 and 7.38;
the positive operators (which are all self-adjoint) are described by the
spectral theorem (7.9 and 7.13).
Specically, suppose T = S T T is the polar decomposition of

T L(V ), where S is an isometry. Then there is an orthonormal basis
of V with respect to which S has a diagonal matrix (if F = C) or a block
diagonal matrix with blocks of size at most 2-by-2 (if F = R), and there
is an orthonormal basis of V with respect to which T T has a diagonal matrix. Warning: there may not exist an orthonormal basis that
simultaneously puts the matrices of both S and T T into these nice

forms (diagonal or block diagonal with small blocks). In other words, S
may require one orthonormal basis and T T may require a different

orthonormal basis.
Suppose T L(V ). The singular values of T are the eigenvalues
of T T , with each eigenvalue repeated dim null( T T I) times.

The singular values of T are all nonnegative because they are the eigen
values of the positive operator T T .
For example, if T L(F4 ) is dened by
7.45
T (z1 , z2 , z3 , z4 ) = (0, 3z1 , 2z2 , 3z4 ),
then T T (z1 , z2 , z3 , z4 ) = (9z1 , 4z2 , 0, 9z4 ), as you should verify. Thus
T T (z1 , z2 , z3 , z4 ) = (3z1 , 2z2 , 0, 3z4 ),
and we see that the eigenvalues of T T are 3, 2, 0. Clearly
dim null( T T 3I) = 2, dim null( T T 2I) = 1, dim null T T = 1.

Hence the singular values of T are 3, 3, 2, 0. In this example 3 and 0
are the only eigenvalues of T , as you should verify.
Each T L(V ) has dim V singular values, as can be seen by applying
the spectral theorem and 5.21 (see especially part (e)) to the positive
(hence self-adjoint) operator T T . For example, the operator T dened by 7.45 on the four-dimensional vector space F4 has four singular
values (they are 3, 3, 2, 0), as we saw in the previous paragraph.
The next result shows that every operator on V has a nice description in terms of its singular values and two orthonormal bases of V .
155
156
7.46 Singular-Value Decomposition: Suppose T L(V ) has singular values s1 , . . . , sn . Then there exist orthonormal bases (e1 , . . . , en )
and (f1 , . . . , fn ) of V such that
T v = s1 v, e1 f1 + + sn v, en fn
7.47
for every v V .
Proof: By the spectral theorem (also see 7.14) applied to T T ,
there is an orthonormal basis (e1 , . . . , en ) of V such that T T ej = sj ej

for j = 1, . . . , n. We have
v = v, e1 e1 + + v, en en
for every v V (see 6.17). Apply T T to both sides of this equation,

getting
T T v = s1 v, e1 e1 + + sn v, en en

This proof illustrates
the usefulness of the
polar decomposition.
for every v V . By the polar decomposition (see 7.41), there is an
isometry S L(V ) such that T = S T T . Apply S to both sides of the

equation above, getting
T v = s1 v, e1 Se1 + + sn v, en Sen
for every v V . For each j, let fj = Sej . Because S is an isometry,
(f1 , . . . , fn ) is an orthonormal basis of V (see 7.36). The equation above
now becomes
T v = s1 v, e1 f1 + + sn v, en fn
for every v V , completing the proof.
When we worked with linear maps from one vector space to a second
vector space, we considered the matrix of a linear map with respect
to a basis for the rst vector space and a basis for the second vector
space. When dealing with operators, which are linear maps from a
vector space to itself, we almost always use only one basis, making it
play both roles.
The singular-value decomposition allows us a rare opportunity to
use two different bases for the matrix of an operator. To do this, suppose T L(V ). Let s1 , . . . , sn denote the singular values of T , and let
(e1 , . . . , en ) and (f1 , . . . , fn ) be orthonormal bases of V such that the
singular-value decomposition 7.47 holds. Then clearly

M T , (e1 , . . . , en ), (f1 , . . . , fn ) =
s1
0
..
157
sn
In other words, every operator on V has a diagonal matrix with respect

to some orthonormal bases of V , provided that we are permitted to
use two different bases rather than a single basis as customary when
working with operators.
Singular values and the singular-value decomposition have many applications (some are given in the exercises), including applications in
computational linear algebra. To compute numeric approximations to
the singular values of an operator T , rst compute T T and then compute approximations to the eigenvalues of T T (good techniques exist
for approximating eigenvalues of positive operators). The nonnegative
square roots of these (approximate) eigenvalues of T T will be the (approximate) singular values of T (as can be seen from the proof of 7.28).
In other words, the singular values of T can be approximated without
computing the square root of T T .
158
Exercises
1.
Make P2 (R) into an inner-product space by dening

1
p(x)q(x) dx.
p, q =
0
Dene T L(P2 (R)) by T (a0 + a1 x + a2 x 2 ) = a1 x.

(a)
Show that T is not self-adjoint.
(b)
The matrix of T with respect to the basis (1, x, x 2 ) is
0 0 0
0 1 0 .
0 0 0
This matrix equals its conjugate transpose, even though T
is not self-adjoint. Explain why this is not a contradiction.
2.
Prove or give a counterexample: the product of any two selfadjoint operators on a nite-dimensional inner-product space is
self-adjoint.
3.
(a)
Show that if V is a real inner-product space, then the set

of self-adjoint operators on V is a subspace of L(V ).
(b)
Show that if V is a complex inner-product space, then the

set of self-adjoint operators on V is not a subspace of
L(V ).
4.
Suppose P L(V ) is such that P 2 = P . Prove that P is an orthogonal projection if and only if P is self-adjoint.
5.
Show that if dim V 2, then the set of normal operators on V is

not a subspace of L(V ).
6.
Prove that if T L(V ) is normal, then

range T = range T .
7.
Prove that if T L(V ) is normal, then

null T k = null T
for every positive integer k.
and
range T k = range T
Exercises
8.
Prove that there does not exist a self-adjoint operator T L(R3 )

such that T (1, 2, 3) = (0, 0, 0) and T (2, 5, 7) = (2, 5, 7).
9.
Prove that a normal operator on a complex inner-product space

is self-adjoint if and only if all its eigenvalues are real.
10.
Suppose V is a complex inner-product space and T L(V ) is a

normal operator such that T 9 = T 8 . Prove that T is self-adjoint
and T 2 = T .
11.
Suppose V is a complex inner-product space. Prove that every

normal operator on V has a square root. (An operator S L(V )
is called a square root of T L(V ) if S 2 = T .)
12.
Give an example of a real inner-product space V and T L(V )

and real numbers , with 2 < 4 such that T 2 + T + I is
not invertible.
159
Exercise 9 strengthens
the analogy (for normal
operators) between
self-adjoint operators
and real numbers.
This exercise shows

that the hypothesis
that T is self-adjoint is
needed in 7.11, even
13.
Prove or give a counterexample: every self-adjoint operator on

V has a cube root. (An operator S L(V ) is called a cube root
of T L(V ) if S 3 = T .)
14.
Suppose T L(V ) is self-adjoint, F, and > 0. Prove that if

there exists v V such that v = 1 and
for real vector spaces.
T v v < ,
then T has an eigenvalue such that | | < .
15.
Suppose U is a nite-dimensional real vector space and T

L(U). Prove that U has a basis consisting of eigenvectors of T if
and only if there is an inner product on U that makes T into a
self-adjoint operator.
16.
Give an example of an operator T on an inner product space such

that T has an invariant subspace whose orthogonal complement
is not invariant under T .
This exercise shows

that 7.18 can fail
without the hypothesis
that T is normal.
17.
Prove that the sum of any two positive operators on V is positive.
18.
Prove that if T L(V ) is positive, then so is T k for every positive

integer k.
160
19.
Suppose that T is a positive operator on V . Prove that T is invertible if and only if

T v, v > 0
for every v V \ {0}.
20.
Prove or disprove: the identity operator on F2 has innitely many

self-adjoint square roots.
21.
Prove or give a counterexample: if S L(V ) and there exists

an orthonormal basis (e1 , . . . , en ) of V such that Sej = 1 for
each ej , then S is an isometry.
22.
Prove that if S L(R 3 ) is an isometry, then there exists a nonzero

vector x R 3 such that S 2 x = x.
23.
Dene T L(F3 ) by
T (z1 , z2 , z3 ) = (z3 , 2z1 , 3z2 ).
Find (explicitly) an isometry S L(F3 ) such that T = S T T .
Exercise 24 shows that
24.
Suppose T L(V ), S L(V ) is an isometry, and R L(V ) is a
positive operator such that T = SR. Prove that R = T T .
25.
Suppose T L(V ). Prove that T is invertible if and only if there
exists a unique isometry S L(V ) such that T = S T T .
26.
Prove that if T L(V ) is self-adjoint, then the singular values

of T equal the absolute values of the eigenvalues of T (repeated
appropriately).
27.
Prove or give a counterexample: if T L(V ), then the singular

values of T 2 equal the squares of the singular values of T .
28.
Suppose T L(V ). Prove that T is invertible if and only if 0 is

not a singular value of T .
29.
Suppose T L(V ). Prove that dim range T equals the number of

nonzero singular values of T .
30.
Suppose S L(V ). Prove that S is an isometry if and only if all

the singular values of S equal 1.
if we write T as the
product of an isometry
and a positive operator
(as in the polar
decomposition), then
the positive operator
must equal T T .
Exercises
31.
Suppose T1 , T2 L(V ). Prove that T1 and T2 have the same

singular values if and only if there exist isometries S1 , S2 L(V )
such that T1 = S1 T2 S2 .
32.
Suppose T L(V ) has singular-value decomposition given by

T v = s1 v, e1 f1 + + sn v, en fn
for every v V , where s1 , . . . , sn are the singular values of T and
(e1 , . . . , en ) and (f1 , . . . , fn ) are orthonormal bases of V .
(a)
Prove that
T v = s1 v, f1 e1 + + sn v, fn en
for every v V .
(b)
Prove that if T is invertible, then

T 1 v =
v, f1 e1
v, fn en
+ +
s1
sn
for every v V .
33.
Suppose T L(V ). Let s denote the smallest singular value of T ,

and let s denote the largest singular value of T . Prove that
sv T v sv
for every v V .
34.
Suppose T , T L(V ). Let s denote the largest singular value

of T , let s denote the largest singular value of T , and let s
denote the largest singular value of T +T . Prove that s s +s .
161
Chapter 8
Operators on
Complex Vector Spaces
In this chapter we delve deeper into the structure of operators on

complex vector spaces. An inner product does not help with this material, so we return to the general setting of a nite-dimensional vector
space (as opposed to the more specialized context of an inner-product
space). Thus our assumptions for this chapter are as follows:
Some of the results in this chapter are valid on real vector spaces,
so we have not assumed that V is a complex vector space. Most of the
results in this chapter that are proved only for complex vector spaces
have analogous results on real vector spaces that are proved in the next
chapter. We deal with complex vector spaces rst because the proofs
on complex vector spaces are often simpler than the analogous proofs
on real vector spaces.
163
Chapter 8. Operators on Complex Vector Spaces
164
Generalized Eigenvectors
Unfortunately some operators do not have enough eigenvectors to
lead to a good description. Thus in this section we introduce the concept of generalized eigenvectors, which will play a major role in our
description of the structure of an operator.
To understand why we need more than eigenvectors, lets examine
the question of describing an operator by decomposing its domain into
invariant subspaces. Fix T L(V ). We seek to describe T by nding a
nice direct sum decomposition
8.1
V = U1 Um ,
where each Uj is a subspace of V invariant under T . The simplest possible nonzero invariant subspaces are one-dimensional. A decomposition 8.1 where each Uj is a one-dimensional subspace of V invariant
under T is possible if and only if V has a basis consisting of eigenvectors
of T (see 5.21). This happens if and only if V has the decomposition
8.2
V = null(T 1 I) null(T m I),
where 1 , . . . , m are the distinct eigenvalues of T (see 5.21).

In the last chapter we showed that a decomposition of the form
8.2 holds for every self-adjoint operator on an inner-product space
(see 7.14). Sadly, a decomposition of the form 8.2 may not hold for
more general operators, even on a complex vector space. An example was given by the operator in 5.19, which does not have enough
eigenvectors for 8.2 to hold. Generalized eigenvectors, which we now
introduce, will remedy this situation. Our main goal in this chapter is
to show that if V is a complex vector space and T L(V ), then
V = null(T 1 I)dim V null(T m I)dim V ,
where 1 , . . . , m are the distinct eigenvalues of T (see 8.23).
Suppose T L(V ) and is an eigenvalue of T . A vector v V is
called a generalized eigenvector of T corresponding to if
8.3
(T I)j v = 0
for some positive integer j. Note that every eigenvector of T is a generalized eigenvector of T (take j = 1 in the equation above), but the
converse is not true. For example, if T L(C3 ) is dened by
165
T (z1 , z2 , z3 ) = (z2 , 0, z3 ),
then T 2 (z1 , z2 , 0) = 0 for all z1 , z2 C. Hence every element of C3
whose last coordinate equals 0 is a generalized eigenvector of T . As
you should verify,
C3 = {(z1 , z2 , 0) : z1 , z2 C} {(0, 0, z3 ) : z3 C},
where the rst subspace on the right equals the set of generalized eigenvectors for this operator corresponding to the eigenvalue 0 and the second subspace on the right equals the set of generalized eigenvectors
corresponding to the eigenvalue 1. Later in this chapter we will prove
that a decomposition using generalized eigenvectors exists for every
operator on a complex vector space (see 8.23).
Though j is allowed to be an arbitrary integer in the denition of a
generalized eigenvector, we will soon see that every generalized eigenvector satises an equation of the form 8.3 with j equal to the dimension of V . To prove this, we now turn to a study of null spaces of
powers of an operator.
Suppose T L(V ) and k is a nonnegative integer. If T k v = 0, then
k+1
v = T (T k v) = T (0) = 0. Thus null T k null T k+1 . In other words,
T
we have
8.4
{0} = null T 0 null T 1 null T k null T k+1 .
The next proposition says that once two consecutive terms in this sequence of subspaces are equal, then all later terms in the sequence are
equal.
8.5
Proposition: If T L(V ) and m is a nonnegative integer such
that null T m = null T m+1 , then
null T 0 null T 1 null T m = null T m+1 = null T m+2 = .
Proof: Suppose T L(V ) and m is a nonnegative integer such
that null T m = null T m+1 . Let k be a positive integer. We want to prove
that
null T m+k = null T m+k+1 .
We already know that null T m+k null T m+k+1 . To prove the inclusion
in the other direction, suppose that v null T m+k+1 . Then
Note that we do not

dene the concept of a
generalized eigenvalue
because this would not
lead to anything new.
Reason: if (T I)j is
not injective for some
positive integer j, then
T I is not injective,
and hence is an
eigenvalue of T .
166
0 = T m+k+1 v = T m+1 (T k v).

Hence
T k v null T m+1 = null T m .
Thus 0 = T m (T k v) = T m+k v, which means that v null T m+k . This
implies that null T m+k+1 null T m+k , completing the proof.
The proposition above raises the question of whether there must exist a nonnegative integer m such that null T m = null T m+1 . The proposition below shows that this equality holds at least when m equals the
dimension of the vector space on which T operates.
8.6
Proposition: If T L(V ), then

null T dim V = null T dim V +1 = null T dim V +2 = .
Proof: Suppose T L(V ). To get our desired conclusion, we need

only prove that null T dim V = null T dim V +1 (by 8.5). Suppose this is not
true. Then, by 8.5, we have
{0} = null T 0 null T 1 null T dim V null T dim V +1 ,
where the symbol means contained in but not equal to. At each of
the strict inclusions in the chain above, the dimension must increase by
at least 1. Thus dim null T dim V +1 dim V + 1, a contradiction because
a subspace of V cannot have a larger dimension than dim V .
Now we have the promised description of generalized eigenvectors.
This corollary implies
that the set of
generalized
eigenvectors of
T L(V )
corresponding to an
eigenvalue is a
subspace of V .
8.7
Corollary: Suppose T L(V ) and is an eigenvalue of T . Then
the set of generalized eigenvectors of T corresponding to equals
null(T I)dim V .
Proof: If v null(T I)dim V , then clearly v is a generalized
eigenvector of T corresponding to (by the denition of generalized
eigenvector).
Conversely, suppose that v V is a generalized eigenvector of T
corresponding to . Thus there is a positive integer j such that
v null(T I)j .
From 8.5 and 8.6 (with T I replacing T ), we get v null(T I)dim V ,
as desired.
An operator is called nilpotent if some power of it equals 0. For

example, the operator N L(F4 ) dened by
N(z1 , z2 , z3 , z4 ) = (z3 , z4 , 0, 0)
2
is nilpotent because N = 0. As another example, the operator of differentiation on Pm (R) is nilpotent because the (m + 1)st derivative of
any polynomial of degree at most m equals 0. Note that on this space of
dimension m + 1, we need to raise the nilpotent operator to the power
m + 1 to get 0. The next corollary shows that we never need to use a
power higher than the dimension of the space.
8.8
167
The Latin word nil
means nothing or zero;
the Latin word potent
means power. Thus
nilpotent literally
means zero power.
Corollary: Suppose N L(V ) is nilpotent. Then N dim V = 0.
Proof: Because N is nilpotent, every vector in V is a generalized

eigenvector corresponding to the eigenvalue 0. Thus from 8.7 we see
that null N dim V = V , as desired.
Having dealt with null spaces of powers of operators, we now turn
our attention to ranges. Suppose T L(V ) and k is a nonnegative
integer. If w range T k+1 , then there exists v V with
w = T k+1 v = T k (T v) range T k .
Thus range T k+1 range T k . In other words, we have
V = range T 0 range T 1 range T k range T k+1 .
The proposition below shows that the inclusions above become equalities once the power reaches the dimension of V .
8.9
Proposition: If T L(V ), then

range T dim V = range T dim V +1 = range T dim V +2 = .
Proof: We could prove this from scratch, but instead lets make use
of the corresponding result already proved for null spaces. Suppose
m > dim V . Then
dim range T m = dim V dim null T m
= dim V dim null T dim V
= dim range T dim V ,
These inclusions go in
the opposite direction
from the corresponding
inclusions for null
spaces (8.4).
168
where the rst and third equalities come from 3.4 and the second equality comes from 8.6. We already know that range T dim V range T m . We
just showed that dim range T dim V = dim range T m , so this implies that
range T dim V = range T m , as desired.
The Characteristic Polynomial
If T happens to have a
diagonal matrix A with
respect to some basis,
then appears on the
diagonal of A precisely
dim null(T I) times,
Suppose V is a complex vector space and T L(V ). We know that

V has a basis with respect to which T has an upper-triangular matrix
(see 5.13). In general, this matrix is not uniqueV may have many
different bases with respect to which T has an upper-triangular matrix,
and with respect to these different bases we may get different uppertriangular matrices. However, the diagonal of any such matrix must
contain precisely the eigenvalues of T (see 5.18). Thus if T has dim V
distinct eigenvalues, then each one must appear exactly once on the
diagonal of any upper-triangular matrix of T .
What if T has fewer than dim V distinct eigenvalues, as can easily
happen? Then each eigenvalue must appear at least once on the diagonal of any upper-triangular matrix of T , but some of them must be
repeated. Could the number of times that a particular eigenvalue is
repeated depend on which basis of V we choose?
You might guess that a number appears on the diagonal of an
upper-triangular matrix of T precisely dim null(T I) times. In general, this is false. For example, consider the operator on C2 whose
matrix with respect to the standard basis is the upper-triangular matrix

5 1
.
0 5
For this operator, dim null(T 5I) = 1 but 5 appears on the diagonal twice. Note, however, that dim null(T 5I)2 = 2 for this operator. This example illustrates the general situationa number appears on the diagonal of an upper-triangular matrix of T precisely
dim null(T I)dim V times, as we will show in the following theorem.
Because null(T I)dim V depends only on T and and not on a choice
of basis, this implies that the number of times an eigenvalue is repeated
on the diagonal of an upper-triangular matrix of T is independent of
which particular basis we choose. This result will be our key tool in
analyzing the structure of an operator on a complex vector space.
169
8.10 Theorem: Let T L(V ) and F . Then for every basis of V

with respect to which T has an upper-triangular matrix, appears on
the diagonal of the matrix of T precisely dim null(T I)dim V times.
Proof: We will assume, without loss of generality, that = 0 (once
the theorem is proved in this case, the general case is obtained by replacing T with T I).
For convenience let n = dim V . We will prove this theorem by induction on n. Clearly the desired result holds if n = 1. Thus we can assume
that n > 1 and that the desired result holds on spaces of dimension
n 1.
Suppose (v1 , . . . , vn ) is a basis of V with respect to which T has an
..
.
8.11
n1
0
n
Let U = span(v1 , . . . , vn1 ). Clearly U is invariant under T (see 5.12),
and the matrix of T |U with respect to the basis (v1 , . . . , vn1 ) is
8.12
1
..
0
n1
Thus, by our induction hypothesis, 0 appears on the diagonal of 8.12

dim null(T |U )n1 times. We know that null(T |U )n1 = null(T |U )n (because U has dimension n 1; see 8.6). Hence
8.13
0 appears on the diagonal of 8.12 dim null(T |U )n times.
The proof breaks into two cases, depending on whether n = 0. First

consider the case where n = 0. We will show that in this case
8.14
null T n U.
Once this has been veried, we will know that null T n = null(T |U )n , and
hence 8.13 will tell us that 0 appears on the diagonal of 8.11 exactly
dim null T n times, completing the proof in the case where n = 0.
Because M(T ) is given by 8.11, we have
Recall that an asterisk

is often used in
matrices to denote
entries that we do not
know or care about.
170
M(T ) = M(T ) =
1 n
..
.
n1 n
n n
This shows that

T n vn = u + n n vn
for some u U. To prove 8.14 (still assuming that n = 0), suppose
v null T n . We can write v in the form
+ avn ,
v=u
U and a F. Thus
where u
+ aT n vn = T n u
+ au + an n vn .
0 = T nv = T nu
and au are in U and vn U , this implies that an n = 0.
Because T n u
U , completing the proof
However, n = 0, so a = 0. Thus v = u
of 8.14.
Now consider the case where n = 0. In this case we will show that
8.15
dim null T n = dim null(T |U )n + 1,
which along with 8.13 will complete the proof when n = 0.

Using the formula for the dimension of the sum of two subspaces
(2.18), we have
dim null T n = dim(U null T n ) + dim(U + null T n ) dim U
= dim null(T |U )n + dim(U + null T n ) (n 1).
Suppose we can prove that null T n contains a vector not in U . Then
n = dim V dim(U + null T n ) > dim U = n 1,
which implies that dim(U + null T n ) = n, which when combined with
the formula above for dim null T n gives 8.15, as desired. Thus to complete the proof, we need only show that null T n contains a vector not
in U .
Lets think about how we might nd a vector in null T n that is not
in U . We might try a vector of the form
u vn ,
171
where u U. At least we are guaranteed that any such vector is not

in U. Can we choose u U such that the vector above is in null T n ?
Lets compute:
T n (u vn ) = T n u T n vn .
To make the above vector equal 0, we must choose (if possible) u U
such that T n u = T n vn . We can do this if T n vn range(T |U )n . Because
8.11 is the matrix of T with respect to (v1 , . . . , vn ), we see that T vn U
(recall that we are considering the case where n = 0). Thus
T n vn = T n1 (T vn ) range(T |U )n1 = range(T |U )n ,
where the last equality comes from 8.9. In other words, we can indeed
choose u U such that u vn null T n , completing the proof.
Suppose T L(V ). The multiplicity of an eigenvalue of T is dened to be the dimension of the subspace of generalized eigenvectors
corresponding to . In other words, the multiplicity of an eigenvalue
of T equals dim null(T I)dim V . If T has an upper-triangular matrix
with respect to some basis of V (as always happens when F = C), then
the multiplicity of is simply the number of times appears on the
diagonal of this matrix (by the last theorem).
As an example of multiplicity, consider the operator T L(F3 ) dened by
Our denition of
multiplicity has a clear
connection with the
geometric behavior
of T . Most texts dene
multiplicity in terms of
the multiplicity of the
roots of a certain
polynomial dened by
determinants. These
8.16
T (z1 , z2 , z3 ) = (0, z1 , 5z3 ).
You should verify that 0 is an eigenvalue of T with multiplicity 2, that

5 is an eigenvalue of T with multiplicity 1, and that T has no additional
eigenvalues. As another example, if T L(F3 ) is the operator whose
matrix is
6 7 7
8.17
0 6 7 ,
0 0 7
then 6 is an eigenvalue of T with multiplicity 2 and 7 is an eigenvalue
of T with multiplicity 1 (this follows from the last theorem).
In each of the examples above, the sum of the multiplicities of the
eigenvalues of T equals 3, which is the dimension of the domain of T .
The next proposition shows that this always happens on a complex
vector space.
two denitions turn

out to be equivalent.
172
8.18 Proposition: If V is a complex vector space and T L(V ), then

the sum of the multiplicities of all the eigenvalues of T equals dim V .
Proof: Suppose V is a complex vector space and T L(V ). Then
there is a basis of V with respect to which the matrix of T is upper
triangular (by 5.13). The multiplicity of equals the number of times
appears on the diagonal of this matrix (from 8.10). Because the diagonal
of this matrix has length dim V , the sum of the multiplicities of all the
eigenvalues of T must equal dim V .
Suppose V is a complex vector space and T L(V ). Let 1 , . . . , m
denote the distinct eigenvalues of T . Let dj denote the multiplicity
of j as an eigenvalue of T . The polynomial
(z 1 )d1 . . . (z m )dm
Most texts dene the
characteristic
polynomial using
determinants. The
approach taken here,
which is considerably
simpler, leads to an
easy proof of the
Cayley-Hamilton
is called the characteristic polynomial of T . Note that the degree of

the characteristic polynomial of T equals dim V (from 8.18). Obviously
the roots of the characteristic polynomial of T equal the eigenvalues
of T . As an example, the characteristic polynomial of the operator
T L(C3 ) dened by 8.16 equals z2 (z 5).
Here is another description of the characteristic polynomial of an
operator on a complex vector space. Suppose V is a complex vector
space and T L(V ). Consider any basis of V with respect to which T
has an upper-triangular matrix of the form
theorem.
8.19
M(T ) =
1
..
0
Then the characteristic polynomial of T is given by

(z 1 ) . . . (z n );
this follows immediately from 8.10. As an example of this procedure,
if T L(C3 ) is the operator whose matrix is given by 8.17, then the
characteristic polynomial of T equals (z 6)2 (z 7).
In the next chapter we will dene the characteristic polynomial of
an operator on a real vector space and prove that the next result also
holds for real vector spaces.
Decomposition of an Operator
8.20 Cayley-Hamilton Theorem: Suppose that V is a complex vector

space and T L(V ). Let q denote the characteristic polynomial of T .
Then q(T ) = 0.
173
The English
mathematician Arthur
Cayley published three
mathematics papers

the matrix of T has the upper-triangular form 8.19. To prove that
q(T ) = 0, we need only show that q(T )vj = 0 for j = 1, . . . , n. To
do this, it sufces to show that
(T 1 I) . . . (T j I)vj = 0
8.21
for j = 1, . . . , n.
We will prove 8.21 by induction on j. To get started, suppose j = 1.

Because M T , (v1 , . . . , vn ) is given by 8.19, we have T v1 = 1 v1 , giving
8.21 when j = 1.
Now suppose that 1 < j n and that
0 = (T 1 I)v1
= (T 1 I)(T 2 I)v2
..
.
= (T 1 I) . . . (T j1 I)vj1 .

Because M T , (v1 , . . . , vn ) is given by 8.19, we see that
(T j I)vj span(v1 , . . . , vj1 ).
Thus, by our induction hypothesis, (T 1 I) . . . (T j1 I) applied to
(T j I)vj gives 0. In other words, 8.21 holds, completing the proof.
We saw earlier that the domain of an operator might not decompose
into invariant subspaces consisting of eigenvectors of the operator,
even on a complex vector space. In this section we will see that every
operator on a complex vector space has enough generalized eigenvectors to provide a decomposition.
We observed earlier that if T L(V ), then null T is invariant under T . Now we show that the null space of any polynomial of T is also
invariant under T .
before he completed
his undergraduate
degree in 1842. The
Irish mathematician
William Hamilton was
made a professor in
1827 when he was 22
years old and still an
undergraduate!
174
8.22 Proposition:
invariant under T .
If T L(V ) and p P(F), then null p(T ) is
Proof: Suppose T L(V ) and p P(F). Let v null p(T ). Then

p(T )v = 0. Thus
(p(T ))(T v) = T (p(T )v) = T (0) = 0,
and hence T v null p(T ). Thus null p(T ) is invariant under T , as
desired.
The following major structure theorem shows that every operator on
a complex vector space can be thought of as composed of pieces, each
of which is a nilpotent operator plus a scalar multiple of the identity.
Actually we have already done all the hard work, so at this point the
proof is easy.
Let 1 , . . . , m be the distinct eigenvalues of T , and let U1 , . . . , Um be
the corresponding subspaces of generalized eigenvectors. Then
(a)
V = U 1 Um ;
(b)
each Uj is invariant under T ;
(c)
each (T j I)|Uj is nilpotent.
Proof: Note that Uj = null(T j I)dim V for each j (by 8.7). From
8.22 (with p(z) = (z j )dim V ), we get (b). Obviously (c) follows from
the denitions.
To prove (a), recall that the multiplicity of j as an eigenvalue of T
is dened to be dim Uj . The sum of these multiplicities equals dim V
(see 8.18); thus
8.24
Let U = U1 + + Um . Clearly U is invariant under T . Thus we can

dene S L(U) by
S = T |U .
Note that S has the same eigenvalues, with the same multiplicities, as T
because all the generalized eigenvectors of T are in U , the domain of S.
Thus applying 8.18 to S, we get
175
dim U = dim U1 + + dim Um .

This equation, along with 8.24, shows that dim V = dim U . Because U
is a subspace of V , this implies that V = U. In other words,
V = U 1 + + Um .
This equation, along with 8.24, allows us to use 2.19 to conclude that
(a) holds, completing the proof.
As we know, an operator on a complex vector space may not have
enough eigenvectors to form a basis for the domain. The next result
shows that on a complex vector space there are enough generalized
eigenvectors to do this.
8.25 Corollary: Suppose V is a complex vector space and T L(V ).
Then there is a basis of V consisting of generalized eigenvectors of T .
Proof: Choose a basis for each Uj in 8.23. Put all these bases
together to form a basis of V consisting of generalized eigenvectors
of T .
Given an operator T on V , we want to nd a basis of V so that the
matrix of T with respect to this basis is as simple as possible, meaning
that the matrix contains many 0s. We begin by showing that if N is
nilpotent, we can choose a basis of V such that the matrix of N with
respect to this basis has more than half of its entries equal to 0.
8.26 Lemma: Suppose N is a nilpotent operator on V . Then there is
a basis of V with respect to which the matrix of N has the form
8.27
0
..
0
If V is complex vector
space, a proof of this
lemma follows easily
from Exercise 6 in this
chapter, 5.13, and 5.18.

But the proof given
here uses simpler ideas
here all entries on and below the diagonal are 0s.
than needed to prove

5.13, and it works for
Proof: First choose a basis of null N. Then extend this to a basis

of null N 2 . Then extend to a basis of null N 3 . Continue in this fashion,
eventually getting a basis of V (because null N m = V for m sufciently
large).
both real and complex

vector spaces.
176
Now lets think about the matrix of N with respect to this basis. The
rst column, and perhaps additional columns at the beginning, consists
of all 0s because the corresponding basis vectors are in null N. The
next set of columns comes from basis vectors in null N 2 . Applying N
to any such vector, we get a vector in null N; in other words, we get a
vector that is a linear combination of the previous basis vectors. Thus
all nonzero entries in these columns must lie above the diagonal. The
next set of columns come from basis vectors in null N 3 . Applying N
to any such vector, we get a vector in null N 2 ; in other words, we get a
vector that is a linear combination of the previous basis vectors. Thus,
once again, all nonzero entries in these columns must lie above the
diagonal. Continue in this fashion to complete the proof.
Note that in the next theorem we get many more zeros in the matrix
of T than are needed to make it upper triangular.
Let 1 , . . . , m be the distinct eigenvalues of T . Then there is a basis
of V with respect to which T has a block diagonal matrix of the form
A1
0
..
Am
where each Aj is an upper-triangular matrix of the form
8.29
Aj =
j
..
0
Proof: For j = 1, . . . , m, let Uj denote the subspace of generalized

eigenvectors of T corresponding to j . Thus (T j I)|Uj is nilpotent
(see 8.23(c)). For each j, choose a basis of Uj such that the matrix of
(T j I)|Uj with respect to this basis is as in 8.26. Thus the matrix of
T |Uj with respect to this basis will look like 8.29. Putting the bases for
the Uj s together gives a basis for V (by 8.23(a)). The matrix of T with
respect to this basis has the desired form.
Square Roots
177
Square Roots
Recall that a square root of an operator T L(V ) is an operator
S L(V ) such that S 2 = T . As an application of the main structure
theorem from the last section, in this section we will show that every
invertible operator on a complex vector space has a square root.
Every complex number has a square root, but not every operator on
a complex vector space has a square root. An example of an operator
on C3 that has no square root is given in Exercise 4 in this chapter.
The noninvertibility of that particular operator is no accident, as we
will soon see. We begin by showing that the identity plus a nilpotent
operator always has a square root.
8.30 Lemma:
square root.
Suppose N L(V ) is nilpotent. Then I + N has a
Proof: Consider the Taylor series for the function

8.31
1 + x = 1 + a1 x + a2 x 2 + .
1 + x:
Because a1 = 1/2, this
formula shows that
We will not nd an explicit formula for all the coefcients or worry

about whether the innite sum converges because we are using this
equation only as motivation, not as a formal part of the proof.
Because N is nilpotent, N m = 0 for some positive integer m. In 8.31,
suppose we replace x with N and 1 with I. Then the innite sum on
the right side becomes a nite sum (because N j = 0 for all j m). In
other words, we guess that there is a square root of I + N of the form
I + a1 N + a2 N 2 + + am1 N m1 .
Having made this guess, we can try to choose a1 , a2 , . . . , am1 so that
the operator above has its square equal to I + N. Now
(I+a1 N + a2 N 2 + a3 N 3 + + am1 N m1 )2
= I + 2a1 N + (2a2 + a1 2 )N 2 + (2a3 + 2a1 a2 )N 3 +
+ (2am1 + terms involving a1 , . . . , am2 )N m1 .
We want the right side of the equation above to equal I + N. Hence
choose a1 so that 2a1 = 1 (thus a1 = 1/2). Next, choose a2 so that
2a2 + a1 2 = 0 (thus a2 = 1/8). Then choose a3 so that the coefcient
of N 3 on the right side of the equation above equals 0 (thus a3 = 1/16).
1 + x/2 is a good
estimate for 1 + x
when x is small.
178
Continue in this fashion for j = 4, . . . , m 1, at each step solving for

aj so that the coefcient of N j on the right side of the equation above
equals 0. Actually we do not care about the explicit formula for the
aj s. We need only know that some choice of the aj s gives a square
root of I + N.
The previous lemma is valid on real and complex vector spaces.
However, the next result holds only on complex vector spaces.
On real vector spaces
there exist invertible
8.32 Theorem: Suppose V is a complex vector space. If T L(V )

is invertible, then T has a square root.
operators that have no

square roots. For
example, the operator
of multiplication by 1
on R has no square
root because no real
number has its square
equal to 1.
Proof: Suppose T L(V ) is invertible. Let 1 , . . . , m be the distinct eigenvalues of T , and let U1 , . . . , Um be the corresponding subspaces of generalized eigenvectors. For each j, there exists a nilpotent
operator Nj L(Uj ) such that T |Uj = j I + Nj (see 8.23(c)). Because T
is invertible, none of the j s equals 0, so we can write

Nj
T | Uj = j I +
j
for each j. Clearly Nj /j is nilpotent, and so I + Nj /j has a square
root (by 8.30). Multiplying a square root of the complex number j by
a square root of I + Nj /j , we obtain a square root Sj of T |Uj .
A typical vector v V can be written uniquely in the form
v = u1 + + um ,
where each uj Uj (see 8.23). Using this decomposition, dene an
operator S L(V ) by
Sv = S1 u1 + + Sm um .
You should verify that this operator S is a square root of T , completing
the proof.
By imitating the techniques in this section, you should be able to
prove that if V is a complex vector space and T L(V ) is invertible,
then T has a kth -root for every positive integer k.
179

As we will soon see, given an operator on a nite-dimensional vector space, there is a unique monic polynomial of smallest degree that
when applied to the operator gives 0. This polynomial is called the
minimal polynomial of the operator and is the focus of attention in
this section.
Suppose T L(V ), where dim V = n. Then
a polynomial whose
highest degree
coefcient equals 1.
For example,
2 + 3z2 + z8 is a monic
(I, T , T 2 , . . . , T n )
polynomial.
2
cannot be linearly independent in L(V ) because L(V ) has dimension n

(see 3.20) and we have n2 + 1 operators. Let m be the smallest positive
integer such that
8.33
A monic polynomial is
(I, T , T 2 , . . . , T m )
is linearly dependent. The linear dependence lemma (2.4) implies that

one of the operators in the list above is a linear combination of the
previous ones. Because m was chosen to be the smallest positive integer such that 8.33 is linearly dependent, we conclude that T m is
a linear combination of (I, T , T 2 , . . . , T m1 ). Thus there exist scalars
a0 , a1 , a2 , . . . , am1 F such that
a0 I + a1 T + a2 T 2 + + am1 T m1 + T m = 0.
The choice of scalars a0 , a1 , a2 , . . . , am1 F above is unique because
two different such choices would contradict our choice of m (subtracting two different equations of the form above, we would have a linearly
dependent list shorter than 8.33). The polynomial
a0 + a1 z + a2 z2 + + am1 zm1 + zm
is called the minimal polynomial of T . It is the monic polynomial
p P(F) of smallest degree such that p(T ) = 0.
For example, the minimal polynomial of the identity operator I is
z 1. Theminimal polynomial of the operator on F2 whose matrix
equals 40 15 is 20 9z + z2 , as you should verify.
Clearly the degree of the minimal polynomial of each operator on V
is at most (dim V )2 . The Cayley-Hamilton theorem (8.20) tells us that
if V is a complex vector space, then the minimal polynomial of each
operator on V has degree at most dim V . This remarkable improvement
also holds on real vector spaces, as we will see in the next chapter.
180
Note that (z )
divides a polynomial q
if and only if is a
root of q. This follows
immediately from 4.1.
A polynomial p P(F) is said to divide a polynomial q P(F) if

there exists a polynomial s P(F) such that q = sp. In other words,
p divides q if we can take the remainder r in 4.6 to be 0. For example, the polynomial (1 + 3z)2 divides 5 + 32z + 57z2 + 18z3 because
5 + 32z + 57z2 + 18z3 = (2z + 5)(1 + 3z)2 . Obviously every nonzero
constant polynomial divides every polynomial.
The next result completely characterizes the polynomials that when
applied to an operator give the 0 operator.
8.34 Theorem: Let T L(V ) and let q P(F). Then q(T ) = 0 if
and only if the minimal polynomial of T divides q.
Proof: Let p denote the minimal polynomial of T .
First we prove the easy direction. Suppose that p divides q. Thus
there exists a polynomial s P(F) such that q = sp. We have
q(T ) = s(T )p(T ) = s(T )0 = 0,
as desired.
To prove the other direction, suppose that q(T ) = 0. By the division
algorithm (4.5), there exist polynomials s, r P(F) such that
q = sp + r
8.35
and deg r < deg p. We have
0 = q(T ) = s(T )p(T ) + r (T ) = r (T ).

Because p is the minimal polynomial of T and deg r < deg p, the equation above implies that r = 0. Thus 8.35 becomes the equation q = sp,
and hence p divides q, as desired.
Now we describe the eigenvalues of an operator in terms of its minimal polynomial.
8.36 Theorem: Let T L(V ). Then the roots of the minimal polynomial of T are precisely the eigenvalues of T .
181
Proof: Let
p(z) = a0 + a1 z + a2 z2 + + am1 zm1 + zm
be the minimal polynomial of T .
First suppose that F is a root of p. Then the minimal polynomial
of T can be written in the form
p(z) = (z )q(z),
where q is a monic polynomial with coefcients in F (see 4.1). Because
p(T ) = 0, we have
0 = (T I)(q(T )v)
for all v V . Because the degree of q is less than the degree of the
minimal polynomial p, there must exist at least one vector v V such
that q(T )v = 0. The equation above thus implies that is an eigenvalue
of T , as desired.
To prove the other direction, now suppose that F is an eigenvalue of T . Let v be a nonzero vector in V such that T v = v. Repeated
applications of T to both sides of this equation show that T j v = j v
for every nonnegative integer j. Thus
0 = p(T )v = (a0 + a1 T + a2 T 2 + + am1 T m1 + T m )v
= (a0 + a1 + a2 2 + + am1 m1 + m )v
= p()v.
Because v = 0, the equation above implies that p() = 0, as desired.
Suppose we are given, in concrete form, the matrix (with respect to
some basis) of some operator T L(V ). To nd the minimal polynomial of T , consider
(M(I), M(T ), M(T )2 , . . . , M(T )m )
for m = 1, 2, . . . until this list is linearly dependent. Then nd the
scalars a0 , a1 , a2 , . . . , am1 F such that
a0 M(I) + a1 M(T ) + a2 M(T )2 + + am1 M(T )m1 + M(T )m = 0.
The scalars a0 , a1 , a2 , . . . , am1 , 1 will then be the coefcients of the
minimal polynomial of T . All this can be computed using a familiar
process such as Gaussian elimination.
You can think of this as

a system of (dim V )2
equations in m
variables
a0 , a1 , . . . , am1 .
182
For example, consider the operator T on C5 whose matrix is given

by
8.37
0
1
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
1
3
6
0
0
0
Because of the large number of 0s in this matrix, Gaussian elimination

is not needed here. Simply compute powers of M(T ) and notice that
there is no linear dependence until the fth power. Do the computations and you will see that the minimal polynomial of T equals
z5 6z + 3.
8.38
Now what about the eigenvalues of this particular operator? From 8.36,
we see that the eigenvalues of T equal the solutions to the equation
z5 6z + 3 = 0.
Unfortunately no solution to this equation can be computed using rational numbers, arbitrary roots of rational numbers, and the usual rules
of arithmetic (a proof of this would take us considerably beyond linear
algebra). Thus we cannot nd an exact expression for any eigenvalues
of T in any familiar form, though numeric techniques can give good approximations for the eigenvalues of T . The numeric techniques, which
we will not discuss here, show that the eigenvalues for this particular
operator are approximately
1.67,
0.51,
1.40,
0.12 + 1.59i,
0.12 1.59i.
Note that the nonreal eigenvalues occur as a pair, with each the complex
conjugate of the other, as expected for the roots of a polynomial with
real coefcients (see 4.10).
Suppose V is a complex vector space and T L(V ). The CayleyHamilton theorem (8.20) and 8.34 imply that the minimal polynomial
of T divides the characteristic polynomial of T . Both these polynomials
are monic. Thus if the minimal polynomial of T has degree dim V , then
it must equal the characteristic polynomial of T . For example, if T is
the operator on C5 whose matrix is given by 8.37, then the characteristic polynomial of T , as well as the minimal polynomial of T , is given
by 8.38.
Jordan Form
183
Jordan Form
We know that if V is a complex vector space, then for every T L(V )
there is a basis of V with respect to which T has a nice upper-triangular
matrix (see 8.28). In this section we will see that we can do even better
there is a basis of V with respect to which the matrix of T contains zeros
everywhere except possibly on the diagonal and the line directly above
the diagonal.
We begin by describing the nilpotent operators. Consider, for example, the nilpotent operator N L(Fn ) dened by
N(z1 , . . . , zn ) = (0, z1 , . . . , zn1 ).
If v = (1, 0, . . . , 0), then clearly (v, Nv, . . . , N n1 v) is a basis of Fn and
(N n1 v) is a basis of null N, which has dimension 1.
As another example, consider the nilpotent operator N L(F5 ) dened by
8.39
N(z1 , z2 , z3 , z4 , z5 ) = (0, z1 , z2 , 0, z4 ).
Unlike the nilpotent operator discussed in the previous paragraph, for

this nilpotent operator there does not exist a vector v F5 such that
(v, Nv, N 2 v, N 3 v, N 4 v) is a basis of F5 . However, if v1 = (1, 0, 0, 0, 0)
and v2 = (0, 0, 0, 1, 0), then (v1 , Nv1 , N 2 v1 , v2 , Nv2 ) is a basis of F5
and (N 2 v1 , Nv2 ) is a basis of null N, which has dimension 2.
Suppose N L(V ) is nilpotent. For each nonzero vector v V , let
m(v) denote the largest nonnegative integer such that N m(v) v = 0. For
example, if N L(F5 ) is dened by 8.39, then m(1, 0, 0, 0, 0) = 2.
The lemma below shows that every nilpotent operator N L(V )
behaves similarly to the example dened by 8.39, in the sense that there
is a nite collection of vectors v1 , . . . , vk V such that the nonzero
vectors of the form N j vr form a basis of V ; here r varies from 1 to k
and j varies from 0 to m(vr ).
8.40 Lemma: If N L(V ) is nilpotent, then there exist vectors
v1 , . . . , vk V such that
(a)
(v1 , Nv1 , . . . , N m(v1 ) v1 , . . . , vk , Nvk , . . . , N m(vk ) vk ) is a basis of V ;
(b)
(N m(v1 ) v1 , . . . , N m(vk ) vk ) is a basis of null N.
Obviously m(v)
depends on N as well
as on v, but the choice
of N will be clear from
the context.
184
Proof: Suppose N is nilpotent. Then N is not injective and thus

dim range N < dim V (see 3.21). By induction on the dimension of V ,
we can assume that the lemma holds on all vector spaces of smaller
dimension. Using range N in place of V and N|range N in place of N, we
thus have vectors u1 , . . . , uj range N such that
The existence of a
(i)
(u1 , Nu1 , . . . , N m(u1 ) u1 , . . . , uj , Nuj , . . . , N m(uj ) uj ) is a basis of

range N;
(ii)
(N m(u1 ) u1 , . . . , N m(uj ) uj ) is a basis of null N range N.
Because each ur range N, we can choose v1 , . . . , vj V such that

Nvr = ur for each r . Note that m(vr ) = m(ur ) + 1 for each r .
Let W be a subspace of null N such that
subspace W with this

property follows from
2.13.
8.41
null N = (null N range N) W
and choose a basis of W , which we will label (vj+1 , . . . , vk ). Because

vj+1 , . . . , vk null N, we have m(vj+1 ) = = m(vk ) = 0.
Having constructed v1 , . . . , vk , we now need to show that (a) and
(b) hold. We begin by showing that the alleged basis in (a) is linearly
independent. To do this, suppose
8.42
0=
k m(v
r )
ar ,s N s (vr ),
r =1 s=0
where each ar ,s F. Applying N to both sides of the equation above,

we get
0=
k m(v
r )
ar ,s N s+1 (vr )
r =1 s=0
j m(ur )
ar ,s N s (ur ).
r =1 s=0
The last equation, along with (i), implies that ar ,s = 0 for 1 r j,

0 s m(vr ) 1. Thus 8.42 reduces to the equation
0 =a1,m(v1 ) N m(v1 ) v1 + + aj,m(vj ) N m(vj ) vj
+ aj+1,0 vj+1 + + ak,0 vk .
Jordan Form
The terms on the rst line on the right are all in null N range N; the
terms on the second line are all in W . Thus the last equation and 8.41
imply that
0 = a1,m(v1 ) N m(v1 ) v1 + + aj,m(vj ) N m(vj ) vj
8.43
= a1,m(v1 ) N m(u1 ) u1 + + aj,m(vj ) N m(uj ) uj
and
0 = aj+1,0 vj+1 + + ak,0 vk .
8.44
Now 8.43 and (ii) imply that a1,m(v1 ) = = aj,m(vj ) = 0. Because

(vj+1 , . . . , vk ) is a basis of W , 8.44 implies that aj+1,0 = = ak,0 = 0.
Thus all the as equal 0, and hence the list of vectors in (a) is linearly
independent.
Clearly (ii) implies that dim(null N range N) = j. Along with 8.41,
this implies that
dim null N = k.
8.45
Clearly (i) implies that
dim range N =
(m(ur ) + 1)
r =0
j
8.46
m(vr ).
r =0
The list of vectors in (a) has length

k
(m(vr ) + 1) = k +
r =0
m(vr )
r =0
= dim null N + dim range N

= dim V ,
where the second equality comes from 8.45 and 8.46, and the third
equality comes from 3.4. The last equation shows that the list of vectors
in (a) has length dim V ; because this list is linearly independent, it is a
basis of V (see 2.17), completing the proof of (a).
Finally, note that
(N m(v1 ) v1 , . . . , N m(vk ) vk ) = (N m(u1 ) u1 , . . . , N m(uj ) uj , vj+1 , . . . , vk ).
185
186
Now (ii) and 8.41 show that the last list above is a basis of null N, completing the proof of (b).
Suppose T L(V ). A basis of V is called a Jordan basis for T if
with respect to this basis T has a block diagonal matrix
0
A1
..
,
0
Am
where each Aj is an upper-triangular matrix of the form
j 1
0
..
..
.
.
.
Aj =
.. 1
0
j
To understand why
each j must be an
eigenvalue of T ,
see 5.18.
The French
mathematician Camille
Jordan rst published a
proof of this theorem
in 1870.
In each Aj , the diagonal is lled with some eigenvalue j of T , the line

directly above the diagonal is lled with 1s, and all other entries are 0
(Aj may be just a 1-by-1 block consisting of just some eigenvalue).
Because there exist operators on real vector spaces that have no
eigenvalues, there exist operators on real vector spaces for which there
is no corresponding Jordan basis. Thus the hypothesis that V is a complex vector space is required for the next result, even though the previous lemma holds on both real and complex vector spaces.
8.47 Theorem: Suppose V is a complex vector space. If T L(V ),
then there is a basis of V that is a Jordan basis for T .
Proof: First consider a nilpotent operator N L(V ) and the vectors v1 , . . . , vk V given by 8.40. For each j, note that N sends the rst
vector in the list (N m(vj ) vj , . . . , Nvj , vj ) to 0 and that N sends each vector in this list other than the rst vector to the previous vector. In other
words, if we reverse the order of the basis given by 8.40(a), then we obtain a basis of V with respect to which N has a block diagonal matrix,
where each matrix on the diagonal has the form
0 1
0
..
..
.
.
.
.. 1
0
0
Jordan Form
Thus the theorem holds for nilpotent operators.

Now suppose T L(V ). Let 1 , . . . , m be the distinct eigenvalues of T , with U1 , . . . , Um the corresponding subspaces of generalized
eigenvectors. We have
V = U1 U m ,
where each (T j I)|Uj is nilpotent (see 8.23). By the previous paragraph, there is a basis of each Uj that is a Jordan basis for (T j I)|Uj .
Putting these bases together gives a basis of V that is a Jordan basis
for T .
187
188
Exercises
1.
Dene T L(C2 ) by
T (w, z) = (z, 0).
Find all generalized eigenvectors of T .
2.
Dene T L(C2 ) by
T (w, z) = (z, w).
Find all generalized eigenvectors of T .
3.
Suppose T L(V ), m is a positive integer, and v V is such

that T m1 v = 0 but T m v = 0. Prove that
(v, T v, T 2 v, . . . , T m1 v)
is linearly independent.
4.
Suppose T L(C3 ) is dened by T (z1 , z2 , z3 ) = (z2 , z3 , 0). Prove

that T has no square root. More precisely, prove that there does
not exist S L(C3 ) such that S 2 = T .
5.
Suppose S, T L(V ). Prove that if ST is nilpotent, then T S is

nilpotent.
6.
Suppose N L(V ) is nilpotent. Prove (without using 8.26) that

0 is the only eigenvalue of N.
7.
Suppose V is an inner-product space. Prove that if N L(V ) is

self-adjoint and nilpotent, then N = 0.
8.
Suppose N L(V ) is such that null N dim V 1 = null N dim V . Prove

that N is nilpotent and that
dim null N j = j
for every integer j with 0 j dim V .
9.
Suppose T L(V ) and m is a nonnegative integer such that

range T m = range T m+1 .
Prove that range T k = range T m for all k > m.
Exercises
10.
189
Prove or give a counterexample: if T L(V ), then

V = null T range T .
11.
Prove that if T L(V ), then

V = null T n range T n ,
where n = dim V .
12.
Suppose V is a complex vector space, N L(V ), and 0 is the only

eigenvalue of N. Prove that N is nilpotent. Give an example to
show that this is not necessarily true on a real vector space.
13.
Suppose that V is a complex vector space with dim V = n and

T L(V ) is such that
null T n2 = null T n1 .
Prove that T has at most two distinct eigenvalues.
14.
Give an example of an operator on C4 whose characteristic polynomial equals (z 7)2 (z 8)2 .
15.
Suppose V is a complex vector space. Suppose T L(V ) is such

that 5 and 6 are eigenvalues of T and that T has no other eigenvalues. Prove that
(T 5I)n1 (T 6I)n1 = 0,
where n = dim V .
16.
Suppose V is a complex vector space and T L(V ). Prove that

V has a basis consisting of eigenvectors of T if and only if every
generalized eigenvector of T is an eigenvector of T .
17.
Suppose V is an inner-product space and N L(V ) is nilpotent.

Prove that there exists an orthonormal basis of V with respect to
which N has an upper-triangular matrix.
18.
Dene N L(F5 ) by
N(x1 , x2 , x3 , x4 , x5 ) = (2x2 , 3x3 , x4 , 4x5 , 0).
Find a square root of I + N.
For complex vector

spaces, this exercise
adds another
equivalence to the list
given by 5.21.
190
For complex vector
19.
Prove that if V is a complex vector space, then every invertible

operator on V has a cube root.
20.
Suppose T L(V ) is invertible. Prove that there exists a polynomial p P(F) such that T 1 = p(T ).
21.
Give an example of an operator on C3 whose minimal polynomial

equals z2 .
22.
Give an example of an operator on C4 whose minimal polynomial

equals z(z 1)2 .
23.

V has a basis consisting of eigenvectors of T if and only if the
minimal polynomial of T has no repeated roots.
24.
Suppose V is an inner-product space. Prove that if T L(V ) is

normal, then the minimal polynomial of T has no repeated roots.
25.
Suppose T L(V ) and v V . Let p be the monic polynomial of

smallest degree such that
spaces, this exercise

adds another
equivalence to the list
given by 5.21.
p(T )v = 0.
Prove that p divides the minimal polynomial of T .
This exercise shows

that every monic
polynomial is the
characteristic
polynomial of some
operator.
26.
Give an example of an operator on C4 whose characteristic and

minimal polynomials both equal z(z 1)2 (z 3).
27.
Give an example of an operator on C4 whose characteristic polynomial equals z(z 1)2 (z 3) and whose minimal polynomial
equals z(z 1)(z 3).
28.
Suppose a0 , . . . , an1 C. Find the minimal and characteristic

polynomials of the operator on Cn whose matrix (with respect to
the standard basis) is
0
a0
1 0
a1
..
.
1
a2
..
..
.
.
0 an2
1 an1
Exercises
29.
Suppose N L(V ) is nilpotent. Prove that the minimal polynomial of N is zm+1 , where m is the length of the longest consecutive string of 1s that appears on the line directly above the
diagonal in the matrix of N with respect to any Jordan basis for N.
30.

there does not exist a direct sum decomposition of V into two
proper subspaces invariant under T if and only if the minimal
polynomial of T is of the form (z )dim V for some C.
31.
Suppose T L(V ) and (v1 , . . . , vn ) is a basis of V that is a Jordan

basis for T . Describe the matrix of T with respect to the basis
(vn , . . . , v1 ) obtained by reversing the order of the vs.
191
Chapter 9
Operators on
Real Vector Spaces
In this chapter we delve deeper into the structure of operators on

real vector spaces. The important results here are somewhat more complex than the analogous results from the last chapter on complex vector
spaces.
Some of the new results in this chapter are valid on complex vector
spaces, so we have not assumed that V is a real vector space.
193
Chapter 9. Operators on Real Vector Spaces
194
Eigenvalues of Square Matrices

We have dened eigenvalues of operators; now we need to extend
that notion to square matrices. Suppose A is an n-by-n matrix with
entries in F. A number F is called an eigenvalue of A if there
exists a nonzero n-by-1 matrix x such that
Ax = x.

For example, 3 is an eigenvalue of 71 85 because

7
1
8
5

2
1

=
6
3

=3
2
1

.

As another example, you should verify that the matrix 01 1
has no
0
eigenvalues if we are thinking of F as the real numbers (by denition,
an eigenvalue must be in F) and has eigenvalues i and i if we are
thinking of F as the complex numbers.
We now have two notions of eigenvalueone for operators and one
for square matrices. As you might expect, these two notions are closely
connected, as we now show.
9.1
Proposition: Suppose T L(V ) and A is the matrix of T with
respect to some basis of V . Then the eigenvalues of T are the same as
the eigenvalues of A.
Proof: Let (v1 , . . . , vn ) be the basis of V with respect to which T
has matrix A. Let F. We need to show that is an eigenvalue of T
if and only if is an eigenvalue of A.
First suppose is an eigenvalue of T . Let v V be a nonzero vector
such that T v = v. We can write
9.2
v = a1 v1 + + an vn ,
where a1 , . . . , an F. Let x be the matrix of the vector v with respect

to the basis (v1 , . . . , vn ). Recall from Chapter 3 that this means
9.3
a1
.
x=
.. .
an
195
We have
Ax = M(T )M(v) = M(T v) = M(v) = M(v) = x,
where the second equality comes from 3.14. The equation above shows
that is an eigenvalue of A, as desired.
To prove the implication in the other direction, now suppose is an
eigenvalue of A. Let x be a nonzero n-by-1 matrix such that Ax = x.
We can write x in the form 9.3 for some scalars a1 , . . . , an F. Dene
v V by 9.2. Then
M(T v) = M(T )M(v) = Ax = x = M(v).
where the rst equality comes from 3.14. The equation above implies
that T v = v, and thus is an eigenvalue of T , completing the proof.
Because every square matrix is the matrix of some operator, the
proposition above allows us to translate results about eigenvalues of
operators into the language of eigenvalues of square matrices. For
example, every square matrix of complex numbers has an eigenvalue
(from 5.10). As another example, every n-by-n matrix has at most n
distinct eigenvalues (from 5.9).

Earlier we proved that each operator on a complex vector space has
an upper-triangular matrix with respect to some basis (see 5.13). In
this section we will see that we can almost do as well on real vector
spaces.
In the last two chapters we used block diagonal matrices, which
extend the notion of diagonal matrices. Now we will need to use the
corresponding extension of upper-triangular matrices. A block uppertriangular matrix is a square matrix of the form
A1
..
,
.
0
Am
where A1 , . . . , Am are square matrices lying along the diagonal, all entries below A1 , . . . , Am equal 0, and the denotes arbitrary entries. For
example, the matrix
As usual, we use an
asterisk to denote
entries of the matrix
that play no important
role in the topics under
consideration.
196
A=
4
0
0
0
0
10
3
3
0
0
11
3
3
0
0
12
14
16
5
5
13
25
17
5
5
is a block upper-triangular matrix with
A=
A1
A2
0
A3
where
A1 =
Every upper-triangular
matrix is also a block

,
A2 =
3
3
3
3

,
A3 =
5
5
5
5

.
Now we prove that for each operator on a real vector space, we can
nd a basis that gives a block upper-triangular matrix with blocks of
size at most 2-by-2 on the diagonal.
with blocks of size

1-by-1 along the
diagonal. At the other
extreme, every square
9.4
Theorem: Suppose V is a real vector space and T L(V ).
Then there is a basis of V with respect to which T has a block uppertriangular matrix
matrix is a block
because we can take
the rst (and only)
block to be the entire
matrix. Smaller blocks
9.5
A1
..
0
Am
where each Aj is a 1-by-1 matrix or a 2-by-2 matrix with no eigenvalues.
are better in the sense

that the matrix then
has more 0s.
Proof: Clearly the desired result holds if dim V = 1.

Next, consider the case where dim V = 2. If T has an eigenvalue ,
then let v1 V be any nonzero eigenvector. Extend (v1 ) to a basis
(v1 , v2 ) of V . With respect to this basis, T has an upper-triangular
matrix of the form

a
.
0 b
In particular, if T has an eigenvalue, then there is a basis of V with
respect to which T has an upper-triangular matrix. If T has no eigenvalues, then choose any basis (v1 , v2 ) of V . With respect to this basis,
197
the matrix of T has no eigenvalues (by 9.1). Thus regardless of whether

T has eigenvalues, we have the desired conclusion when dim V = 2.
Suppose now that dim V > 2 and the desired result holds for all real
vector spaces with smaller dimension. If T has an eigenvalue, let U be a
one-dimensional subspace of V that is invariant under T ; otherwise let
U be a two-dimensional subspace of V that is invariant under T (5.24
guarantees that we can choose U in this fashion). Choose any basis
of U and let A1 denote the matrix of T |U with respect to this basis. If
A1 is a 2-by-2 matrix, then T has no eigenvalues (otherwise we would
have chosen U to be one-dimensional) and thus T |U has no eigenvalues.
Hence if A1 is a 2-by-2 matrix, then A1 has no eigenvalues (see 9.1).
Let W be any subspace of V such that
V = U W;
2.13 guarantees that such a W exists. Because W has dimension less
than the dimension of V , we would like to apply our induction hypothesis to T |W . However, W might not be invariant under T , meaning that
T |W might not be an operator on W . We will compose with the projection PW ,U to get an operator on W . Specically, dene S L(W )
by
Sw = PW ,U (T w)
v = w + u, where
w W and u U,
then PW ,U v = w.
for w W . Note that

T w = PU ,W (T w) + PW ,U (T w)
9.6
Recall that if
= PU ,W (T w) + Sw
for every w W .
By our induction hypothesis, there is a basis of W with respect to
which S has a block upper-triangular matrix of the form
A2
..
,
.
0
Am
Adjoin this basis of W to the basis of U chosen above, getting a basis
of V . A minutes thought should convince you (use 9.6) that the matrix
of T with respect to this basis is a block upper-triangular matrix of the
form 9.5, completing the proof.
198

For operators on complex vector spaces, we dened characteristic
polynomials and developed their properties by making use of uppertriangular matrices. In this section we will carry out a similar procedure
for operators on real vector spaces. Instead of upper-triangular matrices, we will have to use the block upper-triangular matrices furnished
by the last theorem.
In the last chapter, we did not dene the characteristic polynomial
of a square matrix with complex entries because our emphasis is on
operators rather than on matrices. However, to understand operators
on real vector spaces, we will need to dene characteristic polynomials
of 1-by-1 and 2-by-2 matrices with real entries. Then, using block-upper
triangular matrices with blocks of size at most 2-by-2 on the diagonal,
we will be able to dene the characteristic polynomial of an operator
on a real vector space.
To motivate the denition of characteristic polynomials of square
matrices, we would like the following to be true (think about the CayleyHamilton theorem; see 8.20): if T L(V ) has matrix A with respect
to some basis of V and q is the characteristic polynomial of A, then
q(T ) = 0.
Lets begin with the trivial case of 1-by-1 matrices. Suppose V is a
real vector space with dimension 1 and T L(V ). If [] equals the
matrix of T with respect to some basis of V , then T equals I. Thus
if we let q be the degree 1 polynomial dened by q(x) = x , then
q(T ) = 0. Hence we dene the characteristic polynomial of [] to be
x .
Now lets look at 2-by-2 matrices with real entries. Suppose V is a
real vector space with dimension 2 and T L(V ). Suppose

a c
b d
is the matrix of T with respect to some basis (v1 , v2 ) of V . We seek
a monic polynomial q of degree 2 such that q(T ) = 0. If b = 0, then
the matrix above is upper triangular. If in addition we were dealing
with a complex vector space, then we would know that T has characteristic polynomial (z a)(z d). Thus a reasonable candidate might
be (x a)(x d), where we use x instead of z to emphasize that
now we are working on a real vector space. Lets see if the polynomial
199
(x a)(x d), when applied to T , gives 0 even when b = 0. We have

(T aI)(T dI)v1 = (T dI)(T aI)v1 = (T dI)(bv2 ) = bcv1
and
(T aI)(T dI)v2 = (T aI)(cv1 ) = bcv2 .
Thus (T aI)(T dI) is not equal to 0 unless bc = 0. However, the
equations above show that (T aI)(T dI) bcI = 0 (because this
operator equals 0 on a basis, it must equal 0 on V ). Thus if q(x) =
(x a)(x d) bc, then q(T ) = 0.
Motivated by the previous paragraph, we dene the characteristic
c
polynomial of a 2-by-2 matrix a
b d to be (x a)(x d) bc. Here
we are concerned only with matrices with real entries. The next result shows that we have found the only reasonable denition for the
characteristic polynomial of a 2-by-2 matrix.
9.7
Proposition: Suppose V is a real vector space with dimension 2
and T L(V ) has no eigenvalues. Let p P(R) be a monic polynomial
with degree 2. Suppose A is the matrix of T with respect to some basis
of V .
(a)
If p equals the characteristic polynomial of A, then p(T ) = 0.
(b)
If p does not equal the characteristic polynomial of A, then p(T )

is invertible.
Part (b) of this

proposition would be
false without the
hypothesis that T has
no eigenvalues. For
example, dene
T L(R 2 ) by
T (x1 , x2 ) = (0, x2 ).
Take p(x) = x(x 2).
Proof: We already proved (a) in our discussion above. To prove (b),

let q denote the characteristic polynomial of A and suppose that p = q.
We can write p(x) = x 2 + 1 x + 1 and q(x) = x 2 + 2 x + 2 for some
1 , 1 , 2 , 2 R. Now
p(T ) = p(T ) q(T ) = (1 2 )T + (1 2 )I.
If 1 = 2 , then 1 = 2 (otherwise we would have p = q). Thus if
1 = 2 , then p(T ) is a nonzero multiple of the identity and hence is
invertible, as desired. If 1 = 2 , then
p(T ) = (1 2 )(T
2 1
I),
1 2
which is an invertible operator because T has no eigenvalues. Thus (b)

holds.
Then p is not the

characteristic
polynomial of the
matrix of T with
respect to the standard
basis, but p(T ) is not
invertible.
200
Suppose V is a real vector space with dimension 2 and T L(V ) has

no eigenvalues. The last proposition shows that there is precisely one
monic polynomial with degree 2 that when applied to T gives 0. Thus,
though T may have different matrices with respect to different bases,
each of these matrices must have the same characteristic polynomial.
For example, consider T L(R 2 ) dened by
9.8
T (x1 , x2 ) = (3x1 + 5x2 , 2x1 x2 ).
The matrix of T with respect to the standard basis of R 2 is

3
2
5
1

.
The characteristic polynomial of this matrix is (x 3)(x + 1) + 2 5,

which equals x 2 2x + 7. As you should verify, the matrix of T with

respect to the basis (2, 1), (1, 2) equals

1
1
6
1

.
The characteristic polynomial of this matrix is (x 1)(x 1) + 1 6,

which equals x 2 2x + 7, the same result we obtained by using the
standard basis.
When analyzing upper-triangular matrices of an operator T on a
complex vector space V , we found that subspaces of the form
null(T I)dim V
played a key role (see 8.10). Those spaces will also play a role in studying operators on real vector spaces, but because we must now consider
block upper-triangular matrices with 2-by-2 blocks, subspaces of the
form
null(T 2 + T + I)dim V
will also play a key role. To get started, lets look at one- and twodimensional real vector spaces.
First suppose that V is a one-dimensional real vector space and that
T L(V ). If R, then null(T I) equals V if is an eigenvalue
of T and {0} otherwise. If , R with 2 < 4, then
201
null(T 2 + T + I) = {0}.
(Proof: Because V is one-dimensional, there is a constant R such
that T v = v for all v V . Thus (T 2 + T + I)v = (2 + + )v.
However, the inequality 2 < 4 implies that 2 + + = 0, and thus
null(T 2 + T + I) = {0}.)
Now suppose V is a two-dimensional real vector space and T L(V )
has no eigenvalues. If R, then null(T I) equals {0} (because T
has no eigenvalues). If , R with 2 < 4, then null(T 2 + T + I)
equals V if x 2 + x + is the characteristic polynomial of the matrix
of T with respect to some (or equivalently, every) basis of V and equals
{0} otherwise (by 9.7). Note that for this operator, there is no middle
groundthe null space of T 2 + T + I is either {0} or the whole space;
it cannot be one-dimensional.
Now suppose that V is a real vector space of any dimension and
T L(V ). We know that V has a basis with respect to which T has
a block upper-triangular matrix with blocks on the diagonal of size at
most 2-by-2 (see 9.4). In general, this matrix is not uniqueV may
have many different bases with respect to which T has a block uppertriangular matrix of this form, and with respect to these different bases
we may get different block upper-triangular matrices.
We encountered a similar situation when dealing with complex vector spaces and upper-triangular matrices. In that case, though we might
get different upper-triangular matrices with respect to the different
bases, the entries on the diagonal were always the same (though possibly in a different order). Might a similar property hold for real vector
spaces and block upper-triangular matrices? Specically, is the number of times a given 2-by-2 matrix appears on the diagonal of a block
upper-triangular matrix of T independent of which basis is chosen?
Unfortunately this question has a negative answer. For example, the
operator T L(R 2 ) dened by 9.8 has two different 2-by-2 matrices,
as we saw above.
Though the number of times a particular 2-by-2 matrix might appear
on the diagonal of a block upper-triangular matrix of T can depend on
the choice of basis, if we look at characteristic polynomials instead
of the actual matrices, we nd that the number of times a particular
characteristic polynomial appears is independent of the choice of basis.
This is the content of the following theorem, which will be our key tool
in analyzing the structure of an operator on a real vector space.
Recall that 2 < 4

implies that
x 2 + x + has no real
roots; see 4.11.
202
9.9
Theorem: Suppose V is a real vector space and T L(V ).
Suppose that with respect to some basis of V , the matrix of T is
A1
..
,
9.10
.
0
Am
This result implies that

null(T 2 + T + I)dim V
must have even
dimension.
This proof uses the

same ideas as the proof
of the analogous result
on complex vector
spaces (8.10). As usual,
the real case is slightly
more complicated but
requires no new
creativity.
(a)
If R, then precisely dim null(T I)dim V of the matrices

A1 , . . . , Am equal the 1-by-1 matrix [].
(b)
If , R satisfy 2 < 4, then precisely

dim null(T 2 + T + I)dim V
2
of the matrices A1 , . . . , Am have characteristic polynomial equal
to x 2 + x + .
Proof: We will construct one proof that can be used to prove both
(a) and (b). To do this, let , , R with 2 < 4. Dene p P(R)
by
x
if we are trying to prove (a);
p(x) =
2
x + x + if we are trying to prove (b).
Let d denote the degree of p. Thus d = 1 if we are trying to prove (a)
and d = 2 if we are trying to prove (b).
We will prove this theorem by induction on m, the number of blocks
along the diagonal of 9.10. If m = 1, then dim V = 1 or dim V = 2; the
discussion preceding this theorem then implies that the desired result
holds. Thus we can assume that m > 1 and that the desired result
holds when m is replaced with m 1.
For convenience let n = dim V . Consider a basis of V with respect
to which T has the block upper-triangular matrix 9.10. Let Uj denote
the span of the basis vectors corresponding to Aj . Thus dim Uj = 1
if Aj is a 1-by-1 matrix and dim Uj = 2 if Aj is a 2-by-2 matrix. Let
U = U1 + + Um1 . Clearly U is invariant under T and the matrix
of T |U with respect to the obvious basis (obtained from the basis vectors corresponding to A1 , . . . , Am1 ) is
A1
..
.
9.11
.
0
Am1
Thus, by our induction hypothesis,

9.12
precisely (1/d) dim null p(T |U )n of the matrices

A1 , . . . , Am1 have characteristic polynomial p.
Actually the induction hypothesis gives 9.12 with exponent dim U instead of n, but then we can replace dim U with n (by 8.6) to get the
statement above.
Suppose um Um . Let S L(Um ) be the operator whose matrix
(with respect to the basis corresponding to Um ) equals Am . In particular, Sum = PUm ,U T um . Now
T um = PU ,Um T um + PUm ,U T um
= U + Sum ,
where U denotes a vector in U. Note that Sum Um ; thus applying
T to both sides of the equation above gives
T 2 um = U + S 2 um ,
where again U denotes a vector in U, though perhaps a different vector
than the previous usage of U (the notation U is used when we want
to emphasize that we have a vector in U but we do not care which
particular vectoreach time the notation U is used, it may denote a
different vector in U). The last two equations show that
9.13
p(T )um = U + p(S)um
for some U U. Note that p(S)um Um ; thus iterating the last

equation gives
9.14
p(T )n um = U + p(S)n um
for some U U.
The proof now breaks into two cases. First consider the case where
the characteristic polynomial of Am does not equal p. We will show
that in this case
9.15
null p(T )n U.
Once this has been veried, we will know that

null p(T )n = null p(T |U )n ,
203
204
and hence 9.12 will tell us that precisely (1/d) dim null p(T )n of the
matrices A1 , . . . , Am have characteristic polynomial p, completing the
proof in the case where the characteristic polynomial of Am does not
equal p.
To prove 9.15 (still assuming that the characteristic polynomial of
Am does not equal p), suppose v null p(T )n . We can write v in the
form v = u + um , where u U and um Um . Using 9.14, we have
0 = p(T )n v = p(T )n u + p(T )n um = p(T )n u + U + p(S)n um
for some U U. Because the vectors p(T )n u and U are in U and
p(S)n um Um , this implies that p(S)n um = 0. However, p(S) is invertible (see the discussion preceding this theorem about one- and twodimensional subspaces and note that dim Um 2), so um = 0. Thus
v = u U , completing the proof of 9.15.
Now consider the case where the characteristic polynomial of Am
equals p. Note that this implies dim Um = d. We will show that
9.16
dim null p(T )n = dim null p(T |U )n + d,
which along with 9.12 will complete the proof.

Using the formula for the dimension of the sum of two subspaces
(2.18), we have
dim null p(T )n = dim(U null p(T )n ) + dim(U + null p(T )n ) dim U
= dim null p(T |U )n + dim(U + null p(T )n ) (n d).
If U + null p(T )n = V , then dim(U + null p(T )n ) = n, which when combined with the last formula above for dim null p(T )n would give 9.16,
as desired. Thus we will nish by showing that U + null p(T )n = V .
To prove that U + null p(T )n = V , suppose um Um . Because the
characteristic polynomial of the matrix of S (namely, Am ) equals p, we
have p(S) = 0. Thus p(T )um U (from 9.13). Now
p(T )n um = p(T )n1 (p(T )um ) range p(T |U )n1 = range p(T |U )n ,
where the last equality comes from 8.9. Thus we can choose u U
such that p(T )n um = p(T |U )n u. Now
p(T )n (um u) = p(T )n um p(T )n u
= p(T )n um p(T |U )n u
= 0.
205
Thus um u null p(T )n , and hence um , which equals u + (um u),

is in U + null p(T )n . In other words, Um U + null p(T )n . Therefore
V = U +Um U +null p(T )n , and hence U +null p(T )n = V , completing
the proof.
As we saw in the last chapter, the eigenvalues of an operator on a
complex vector space provide the key to analyzing the structure of the
operator. On a real vector space, an operator may have fewer eigenvalues, counting multiplicity, than the dimension of the vector space.
The previous theorem suggests a denition that makes up for this deciency. We will see that the denition given in the next paragraph helps
make operator theory on real vector spaces resemble operator theory
on complex vector spaces.
Suppose V is a real vector space and T L(V ). An ordered pair
(, ) of real numbers is called an eigenpair of T if 2 < 4 and
2
T + T + I
is not injective. The previous theorem shows that T can have only
nitely many eigenpairs because each eigenpair corresponds to the
characteristic polynomial of a 2-by-2 matrix on the diagonal of 9.10
and there is room for only nitely many such matrices along that diagonal. Guided by 9.9, we dene the multiplicity of an eigenpair (, )
of T to be
dim null(T 2 + T + I)dim V
.
2
From 9.9, we see that the multiplicity of (, ) equals the number of
times that x 2 +x+ is the characteristic polynomial of a 2-by-2 matrix
on the diagonal of 9.10.
As an example, consider the operator T L(R 3 ) whose matrix (with
respect to the standard basis) equals
3 1 2
3 2 3 .
1 2
0
You should verify that (4, 13) is an eigenpair of T with multiplicity 1;
note that T 2 4T + 13I is not injective because (1, 0, 1) and (1, 1, 0)
are in its null space. Without doing any calculations, you should verify
that T has no other eigenpairs (use 9.9). You should also verify that 1 is
an eigenvalue of T with multiplicity 1, with corresponding eigenvector
(1, 0, 1), and that T has no other eigenvalues.
Though the word

eigenpair was chosen
to be consistent with
the word eigenvalue,
this terminology is not
in widespread use.
206
In the example above, the sum of the multiplicities of the eigenvalues of T plus twice the multiplicities of the eigenpairs of T equals 3,
which is the dimension of the domain of T . The next proposition shows
that this always happens on a real vector space.
This proposition shows
that though an
operator on a real
9.17 Proposition: If V is a real vector space and T L(V ), then

the sum of the multiplicities of all the eigenvalues of T plus the sum
of twice the multiplicities of all the eigenpairs of T equals dim V .
vector space may have

no eigenvalues, or it
may have no
eigenpairs, it cannot be
lacking in both these
useful objects. It also
shows that an operator
on a real vector space
V can have at most
(dim V )/2 distinct
Proof: Suppose V is a real vector space and T L(V ). Then there

is a basis of V with respect to which the matrix of T is as in 9.9. The
multiplicity of an eigenvalue equals the number of times the 1-by-1
matrix [] appears on the diagonal of this matrix (from 9.9). The multiplicity of an eigenpair (, ) equals the number of times x 2 + x + is
the characteristic polynomial of a 2-by-2 matrix on the diagonal of this
matrix (from 9.9). Because the diagonal of this matrix has length dim V ,
the sum of the multiplicities of all the eigenvalues of T plus the sum of
twice the multiplicities of all the eigenpairs of T must equal dim V .
eigenpairs.
Suppose V is a real vector space and T L(V ). With respect to

some basis of V , T has a block upper-triangular matrix of the form
9.18
A1
..
0
Am
where each Aj is a 1-by-1 matrix or a 2-by-2 matrix with no eigenvalues (see 9.4). We dene the characteristic polynomial of T to be the
product of the characteristic polynomials of A1 , . . . , Am . Explicitly, for
each j, dene qj P(R) by
9.19
Note that the roots of
the characteristic
polynomial of T equal
the eigenvalues of T , as
was true on complex
vector spaces.
qj (x) =
x
(x a)(x d) bc
if Aj equals [];
c
if Aj equals a
b d .
Then the characteristic polynomial of T is

q1 (x) . . . qm (x).
Clearly the characteristic polynomial of T has degree dim V . Furthermore, 9.9 insures that the characteristic polynomial of T depends
only on T and not on the choice of a particular basis.
207
Now we can prove a result that was promised in the last chapter,
where we proved the analogous theorem (8.20) for operators on complex vector spaces.
9.20 Cayley-Hamilton Theorem: Suppose V is a real vector space
and T L(V ). Let q denote the characteristic polynomial of T . Then
q(T ) = 0.
Proof: Choose a basis of V with respect to which T has a block
upper-triangular matrix of the form 9.18, where each Aj is a 1-by-1
matrix or a 2-by-2 matrix with no eigenvalues. Suppose Uj is the one- or
two-dimensional subspace spanned by the basis vectors corresponding
to Aj . Dene qj as in 9.19. To prove that q(T ) = 0, we need only show
that q(T )|Uj = 0 for j = 1, . . . , m. To do this, it sufces to show that
q1 (T ) . . . qj (T )|Uj = 0
9.21
for j = 1, . . . , m.
We will prove 9.21 by induction on j. To get started, suppose that
j = 1. Because M(T ) is given by 9.18, we have q1 (T )|U1 = 0 (obvious if
dim U1 = 1; from 9.7(a) if dim U1 = 2), giving 9.21 when j = 1.
Now suppose that 1 < j n and that
0 = q1 (T )|U1
0 = q1 (T )q2 (T )|U2
..
.
0 = q1 (T ) . . . qj1 (T )|Uj1 .
If v Uj , then from 9.18 we see that
qj (T )v = u + qj (S)v,
where u U1 + + Uj1 and S L(Uj ) has characteristic polynomial qj . Because qj (S) = 0 (obvious if dim Uj = 1; from 9.7(a) if
dim Uj = 2), the equation above shows that
qj (T )v U1 + + Uj1
whenever v Uj . Thus, by our induction hypothesis, q1 (T ) . . . qj1 (T )
applied to qj (T )v gives 0 whenever v Uj . In other words, 9.21 holds,
completing the proof.
This proof uses the

on complex vector
spaces (8.20).
208
Suppose V is a real vector space and T L(V ). Clearly the CayleyHamilton theorem (9.20) implies that the minimal polynomial of T has
degree at most dim V , as was the case on complex vector spaces. If
the degree of the minimal polynomial of T equals dim V , then, as was
also the case on complex vector spaces, the minimal polynomial of T
must equal the characteristic polynomial of T . This follows from the
Cayley-Hamilton theorem (9.20) and 8.34.
Finally, we can now prove a major structure theorem about operators on real vector spaces. The theorem below should be compared
to 8.23, the corresponding result on complex vector spaces.
Either m or M
might be 0.
This proof uses the

on complex vector
spaces (8.23).
9.22 Theorem: Suppose V is a real vector space and T L(V ). Let

1 , . . . , m be the distinct eigenvalues of T , with U1 , . . . , Um the corresponding sets of generalized eigenvectors. Let (1 , 1 ), . . . , (M , M )
be the distinct eigenpairs of T and let Vj = null(T 2 + j T + j I)dim V .
Then
(a)
V = U1 Um V1 VM ;
(b)
each Uj and each Vj is invariant under T ;
(c)
each (T j I)|Uj and each (T 2 + j T + j I)|Vj is nilpotent.
Proof: From 8.22, we get (b). Clearly (c) follows from the denitions.
To prove (a), recall that dim Uj equals the multiplicity of j as an
eigenvalue of T and dim Vj equals twice the multiplicity of (j , j ) as
an eigenpair of T . Thus
9.23
dim V = dim U1 + + dim Um + dim V1 + + VM ;
this follows from 9.17. Let U = U1 + + Um + V1 + + VM . Note

that U is invariant under T . Thus we can dene S L(U ) by
S = T |U .
Note that S has the same eigenvalues, with the same multiplicities, as T
because all the generalized eigenvectors of T are in U , the domain of S.
Similarly, S has the same eigenpairs, with the same multiplicities, as T .
Thus applying 9.17 to S, we get
dim U = dim U1 + + dim Um + dim V1 + + VM .
This equation, along with 9.23, shows that dim V = dim U . Because U
is a subspace of V , this implies that V = U. In other words,
V = U1 + + Um + V1 + + VM .
This equation, along with 9.23, allows us to use 2.19 to conclude that
(a) holds, completing the proof.
209
210
Exercises
1.
Prove that 1 is an eigenvalue of every square matrix with the

property that the sum of the entries in each row equals 1.
2.
Consider a 2-by-2 matrix of real numbers

a c
A=
.
b d
Prove that A has an eigenvalue (in R) if and only if
(a d)2 + 4bc 0.
3.
Suppose A is a block diagonal matrix
A1
0
.
..
A=
0
Am
where each Aj is a square matrix. Prove that the set of eigenvalues of A equals the union of the eigenvalues of A1 , . . . , Am .
Clearly Exercise 4 is a
4.
stronger statement
than Exercise 3. Even
so, you may want to do
Exercise 3 rst because
it is easier than
Suppose A is a block upper-triangular matrix
A1
..
,
A=
.
0
Am
where each Aj is a square matrix. Prove that the set of eigenvalues of A equals the union of the eigenvalues of A1 , . . . , Am .
Exercise 4.
5.
Suppose V is a real vector space and T L(V ). Suppose , R

are such that T 2 + T + I = 0. Prove that T has an eigenvalue
if and only if 2 4.
6.
Suppose V is a real inner-product space and T L(V ). Prove

that there is an orthonormal basis of V with respect to which T
has a block upper-triangular matrix
A1
..
,
.
0
Am
Exercises
7.
Prove that if T L(V ) and j is a positive integer such that

j dim V , then T has an invariant subspace whose dimension
equals j 1 or j.
8.
Prove that there does not exist an operator T L(R 7 ) such that
T 2 + T + I is nilpotent.
9.
Give an example of an operator T L(C7 ) such that T 2 + T + I

is nilpotent.
10.

are such that 2 < 4. Prove that
211
null(T 2 + T + I)k
has even dimension for every positive integer k.
11.

are such that 2 < 4 and T 2 + T + I is nilpotent. Prove that
dim V is even and
(T 2 + T + I)dim V /2 = 0.
12.
Prove that if T L(R3 ) and 5, 7 are eigenvalues of T , then T has

no eigenpairs.
13.
Suppose V is a real vector space with dim V = n and T L(V )

is such that
null T n2 = null T n1 .
Prove that T has at most two distinct eigenvalues and that T has
no eigenpairs.
14.
15.
Suppose V is a vector space with dimension 2 and T L(V ).

Prove that if

a c
b d
You do not need to nd
is the matrix of T with respect to some basis of V , then the characteristic polynomial of T equals (z a)(z d) bc.
specied, here V may
Suppose V is a real inner-product space and S L(V ) is an isometry. Prove that if (, ) is an eigenpair of S, then = 1.
the eigenvalues of T to
do this exercise. As
usual unless otherwise
be a real or complex
vector space.
Chapter 10
Trace and Determinant
Throughout this book our emphasis has been on linear maps and operators rather than on matrices. In this chapter we pay more attention
to matrices as we dene and discuss traces and determinants. Determinants appear only at the end of this book because we replaced their
usual applications in linear algebra (the denition of the characteristic polynomial and the proof that operators on complex vector spaces
have eigenvalues) with more natural techniques. The book concludes
with an explanation of the important role played by determinants in
the theory of volume and integration.
213
Chapter 10. Trace and Determinant
214
Change of Basis
The matrix of an operator T L(V ) depends on a choice of basis
of V . Two different bases of V may give different matrices of T . In this
section we will learn how these matrices are related. This information
will help us nd formulas for the trace and determinant of T later in
this chapter.
With respect to any basis of V , the identity operator I L(V ) has a
diagonal matrix
1
0
..
.
0
1
This matrix is called the identity matrix and is denoted I. Note that we
use the symbol I to denote the identity operator (on all vector spaces)
and the identity matrix (of all possible sizes). You should always be
able to tell from the context which particular meaning of I is intended.
For example, consider the equation
M(I) = I;
Some mathematicians
use the terms
nonsingular, which
means the same as
invertible, and
singular, which means
the same as
noninvertible.
on the left side I denotes the identity operator and on the right side I
denotes the identity matrix.
If A is a square matrix (with entries in F, as usual) with the same
size as I, then AI = IA = A, as you should verify. A square matrix A
is called invertible if there is a square matrix B of the same size such
that AB = BA = I, and we call B an inverse of A. To prove that A has
at most one inverse, suppose B and B are inverses of A. Then
B = BI = B(AB ) = (BA)B = IB = B ,
and hence B = B , as desired. Because an inverse is unique, we can use
the notation A1 to denote the inverse of A (if A is invertible). In other
words, if A is invertible, then A1 is the unique matrix of the same size
such that AA1 = A1 A = I.
Recall that when discussing linear maps from one vector space to
another in Chapter 3, we dened the matrix of a linear map with respect
to two basesone basis for the rst vector space and another basis for
the second vector space. When we study operators, which are linear
maps from a vector space to itself, we almost always use the same basis
Change of Basis
for both vector spaces (after all, the two vector spaces in question are
equal). Thus we usually refer to the matrix of an operator with respect
to a basis, meaning that we are using one basis in two capacities. The
next proposition is one of the rare cases where we need to use two
dierent bases even though we have an operator from a vector space
to itself.
Lets review how matrix multiplication interacts with multiplication
of linear maps. Suppose that along with V we have two other nitedimensional vector spaces, say U and W . Let (u1 , . . . , up ) be a basis
of U , let (v1 , . . . , vn ) be a basis of V , and let (w1 , . . . , wm ) be a basis
of W . If T L(U, V ) and S L(V , W ), then ST L(U, W ) and

M ST , (u1 , . . . , up ), (w1 , . . . , wm ) =

M S, (v1 , . . . , vn ), (w1 , . . . , wm ) M T , (u1 , . . . , up ), (v1 , . . . , vn ) .
10.1
The equation above holds because we dened matrix multiplication to

make it truesee 3.11 and the material following it.
The following proposition deals with the matrix of the identity operator when we use two dierent bases. Note that the kth column of

M I, (u1 , . . . , un ), (v1 , . . . , vn ) consists of the scalars needed to write
uk as a linear combination of the v s. As an example of the proposi

tion below, consider the bases (4, 2), (5, 3) and (1, 0), (0, 1) of F2 .
Obviously

4 5
M I, (4, 2), (5, 3) , (1, 0), (0, 1) =
.
2 3

5/2
, as you should verify. Thus
The inverse of the matrix above is 3/2
1
2
the proposition below implies that

3/2 5/2
M I, (1, 0), (0, 1) , (4, 2), (5, 3) =
.
1
2
10.2 Proposition: If (u1 , . . . , un ) and (v1 , . . . , vn ) are bases of V ,

then M I, (u1 , . . . , un ), (v1 , . . . , vn ) is invertible and
1

M I, (u1 , . . . , un ), (v1 , . . . , vn )
= M I, (v1 , . . . , vn ), (u1 , . . . , un ) .
Proof: In 10.1, replace U and W with V , replace wj with uj , and
replace S and T with I, getting
215
216

I = M I, (v1 , . . . , vn ), (u1 , . . . , un ) M I, (u1 , . . . , un ), (v1 , . . . , vn ) .
Now interchange the roles of the us and v s, getting

I = M I, (u1 , . . . , un ), (v1 , . . . , vn ) M I, (v1 , . . . , vn ), (u1 , . . . , un ) .
These two equations give the desired result.
Now we can see how the matrix of T changes when we change
bases.
10.3 Theorem: Suppose T L(V ). Let (u1 , . . . , un ) and (v1 , . . . , vn )

be bases of V . Let A = M I, (u1 , . . . , un ), (v1 , . . . , vn ) . Then

M T , (u1 , . . . , un ) = A1 M T , (v1 , . . . , vn ) A.
10.4
Proof: In 10.1, replace U and W with V , replace wj with vj , replace

T with I, and replace S with T , getting

M T , (u1 , . . . , un ), (v1 , . . . , vn ) = M T , (v1 , . . . , vn ) A.
10.5
Again use 10.1, this time replacing U and W with V , replacing wj

with uj , and replacing S with I, getting

M T , (u1 , . . . , un ) = A1 M T , (u1 , . . . , un ), (v1 , . . . , vn ) ,
where we have used 10.2. Substituting 10.5 into the equation above
gives 10.4, completing the proof.
Trace
Lets examine the characteristic polynomial more closely than we
did in the last two chapters. If V is an n-dimensional complex vector
space and T L(V ), then the characteristic polynomial of T equals
(z 1 ) . . . (z n ),
where 1 , . . . , n are the eigenvalues of T , repeated according to multiplicity. Expanding the polynomial above, we can write the characteristic
polynomial of T in the form
10.6
zn (1 + + n )zn1 + + (1)n (1 . . . n ).
Trace
217
If V is an n-dimensional real vector space and T L(V ), then the

characteristic polynomial of T equals
(x 1 ) . . . (x m )(x 2 + 1 x + 1 ) . . . (x 2 + M x + M ),
Here m or M might
equal 0.
where 1 , . . . , m are the eigenvalues of T and (1 , 1 ), . . . , (M , M ) are

the eigenpairs of T , each repeated according to multiplicity. Expanding
the polynomial above, we can write the characteristic polynomial of T
in the form
Recall that a pair (, )

of real numbers is an
eigenpair of T if
2 < 4 and
10.7
x n (1 + + m 1 m )x n1 + . . .
m
+ (1) (1 . . . m 1 . . . M ).
In this section we will study the coefcient of zn1 (usually denoted
x
when we are dealing with a real vector space) in the characteristic
polynomial. In the next section we will study the constant term in the
characteristic polynomial.
For T L(V ), the negative of the coefcient of zn1 (or x n1 for real
vector spaces) in the characteristic polynomial of T is called the trace
of T , denoted trace T . If V is a complex vector space, then 10.6 shows
that trace T equals the sum of the eigenvalues of T , counting multiplicity. If V is a real vector space, then 10.7 shows that trace T equals the
sum of the eigenvalues of T minus the sum of the rst coordinates of
the eigenpairs of T , each repeated according to multiplicity.
For example, suppose T L(C3 ) is the operator whose matrix is
3 1 2
10.8
3 2 3 .
1 2
0
T 2 + T + I is not
injective.
n1
Then the eigenvalues of T are 1, 2 + 3i, and 2 3i, each with multiplicity 1, as you can verify. Computing the sum of the eigenvalues, we
have trace T = 1 + (2 + 3i) + (2 3i); in other words, trace T = 5.
As another example, suppose T L(R3 ) is the operator whose matrix is also given by 10.8 (note that in the previous paragraph we were
working on a complex vector space; now we are working on a real vector space). Then 1 is the only eigenvalue of T (it has multiplicity 1)
and (4, 13) is the only eigenpair of T (it has multiplicity 1), as you
should have veried in the last chapter (see page 205). Computing the
sum of the eigenvalues minus the sum of the rst coordinates of the
eigenpairs, we have trace T = 1 (4); in other words, trace T = 5.
Note that trace T

depends only on T and
not on a basis of V
because the
characteristic
polynomial of T does
not depend on a choice
of basis.
218
The reason that the operators in the two previous examples have
the same trace will become clear after we nd a formula (valid on both
complex and real vector spaces) for computing the trace of an operator
from its matrix.
Most of the rest of this section is devoted to discovering how to calculate trace T from the matrix of T (with respect to an arbitrary basis).
Lets start with the easiest situation. Suppose V is a complex vector
space, T L(V ), and we choose a basis of V with respect to which
T has an upper-triangular matrix A. Then the eigenvalues of T are
precisely the diagonal entries of A, repeated according to multiplicity
(see 8.10). Thus trace T equals the sum of the diagonal entries of A.
The same formula works for the operator T L(F3 ) whose matrix is
given by 10.8 and whose trace equals 5. Could such a simple formula
be true in general?
We begin our investigation by considering T L(V ) where V is a
real vector space. Choose a basis of V with respect to which T has a
block upper-triangular matrix M(T ), where each block on the diagonal
is a 1-by-1 matrix containing an eigenvalue of T or a 2-by-2 block with
no eigenvalues (see 9.4 and 9.9). Each entry in a 1-by-1 block on the
diagonal of M(T ) is an eigenvalue of T and thus makes a contribution
to trace T . If M(T ) has any 2-by-2 blocks on the diagonal, consider a
typical one

a c
.
b d
The characteristic polynomial of this 2-by-2 matrix is (xa)(xd)bc,
which equals
x 2 (a + d)x + (ad bc).
You should carefully
review 9.9 to
understand the
relationship between
eigenpairs and
characteristic
polynomials of 2-by-2
blocks.
Thus (a d, ad bc) is an eigenpair of T . The negative of the rst

coordinate of this eigenpair, namely, a + d, is the contribution of this
block to trace T . Note that a + d is the sum of the entries on the diagonal of this block. Thus for any basis of V with respect to which
the matrix of T has the block upper-triangular form required by 9.4
and 9.9, trace T equals the sum of the entries on the diagonal.
At this point you should suspect that trace T equals the sum of
the diagonal entries of the matrix of T with respect to an arbitrary
basis. Remarkably, this turns out to be true. To prove it, lets dene the trace of a square matrix A, denoted trace A, to be the sum
of the diagonal entries. With this notation, we want to prove that
Trace
219

trace T = trace M T , (v1 , . . . , vn ) , where (v1 , . . . , vn ) is an arbitrary
basis of V . We already know this is true if (v1 , . . . , vn ) is a basis with
respect to which T has an upper-triangular matrix (if V is complex) or
an appropriate block upper-triangular matrix (if V is real). We will need
the following proposition to prove our trace formula for an arbitrary
basis.
10.9
then
Proposition: If A and B are square matrices of the same size,

trace(AB) = trace(BA).
Proof: Suppose
a1,1 . . .
.
A = ..
an,1 . . .
a1,n
..
. ,
an,n
b1,1
.
B = ..
bn,1
...
...
b1,n
..
. .
bn,n
The j th term on the diagonal of AB equals

n
aj,k bk,j .
k=1
Thus
trace(AB) =
aj,k bk,j
j=1 k=1
bk,j aj,k
k=1 j=1
kth term on the diagonal of BA
k=1
= trace(BA),
as desired.
Now we can prove that the sum of the diagonal entries of the matrix
of an operator is independent of the basis with respect to which the
matrix is computed.
10.10 Corollary: Suppose T L(V ). If (u1 , . . . , un ) and (v1 , . . . , vn )
are bases of V , then

trace M T , (u1 , . . . , un ) = trace M T , (v1 , . . . , vn ) .
220
The third equality here

depends on the
associative property of
matrix multiplication.
Proof: Suppose (u1 , . . . , un ) and (v1 , . . . , vn ) are bases of V . Let

A = M I, (u1 , . . . , un ), (v1 , . . . , vn ) . Then

trace M T , (u1 , . . . , un ) = trace A1 M T , (v1 , . . . , vn ) A

= trace M T , (v1 , . . . , vn ) A A1

= trace M T , (v1 , . . . , vn ) ,
where the rst equality follows from 10.3 and the second equality follows from 10.9. The third equality completes the proof.
The theorem below states that the trace of an operator equals the
sum of the diagonal entries of the matrix of the operator. This theorem
does not specify a basis because, by the corollary above, the sum of
the diagonal entries of the matrix of an operator is the same for every
choice of basis.
10.11
Theorem: If T L(V ), then trace T = trace M(T ).
Proof: Let T L(V ). As noted above, trace M(T ) is independent

of which basis of V we choose (by 10.10). Thus to show that
trace T = trace M(T )
for every basis of V , we need only show that the equation above holds
for some basis of V . We already did this (on page 218), choosing a basis
of V with respect to which M(T ) is an upper-triangular matrix (if V is a
complex vector space) or an appropriate block upper-triangular matrix
(if V is a real vector space).
If we know the matrix of an operator on a complex vector space, the
theorem above allows us to nd the sum of all the eigenvalues without
nding any of the eigenvalues. For example, consider the operator
on C5 whose matrix is
0 0 0 0 3
1 0 0 0 6
0 1 0 0 0 .
0 0 1 0 0
0 0 0 1 0
Trace
No one knows an exact formula for any of the eigenvalues of this operator. However, we do know that the sum of the eigenvalues equals 0
because the sum of the diagonal entries of the matrix above equals 0.
The theorem above also allows us easily to prove some useful properties about traces of operators by shifting to the language of traces
of matrices, where certain properties have already been proved or are
obvious. We carry out this procedure in the next corollary.
10.12
Corollary: If S, T L(V ), then
trace(ST ) = trace(T S)
and
trace(S + T ) = trace S + trace T .
Proof: Suppose S, T L(V ). Choose any basis of V . Then

trace(ST ) = trace M(ST )

= trace M(S)M(T )

= trace M(T )M(S)
= trace M(T S)
= trace(T S),
where the rst and last equalities come from 10.11 and the middle
equality comes from 10.9. This completes the proof of the rst assertion in the corollary.
To prove the second assertion in the corollary, note that
trace(S + T ) = trace M(S + T )

= trace M(S) + M(T )
= trace M(S) + trace M(T )
= trace S + trace T ,
where again the rst and last equalities come from 10.11; the third
equality is obvious from the denition of the trace of a matrix. This
completes the proof of the second assertion in the corollary.
The techniques we have developed have the following curious corollary. The generalization of this result to innite-dimensional vector
spaces has important consequences in quantum theory.
221
222
The statement of this
corollary does not
10.13 Corollary: There do not exist operators S, T L(V ) such that

ST T S = I.
involve traces, though

the short proof uses
traces. Whenever
something like this
happens in
mathematics, we can be
sure that a good
denition lurks in the
background.
Proof: Suppose S, T L(V ). Then

trace(ST T S) = trace(ST ) trace(T S)
= 0,
where the second equality comes from 10.12. Clearly the trace of I
equals dim V , which is not 0. Because ST T S and I have different
traces, they cannot be equal.
Note that det T
depends only on T and
not on a basis of V
because the
characteristic
polynomial of T does
not depend on a choice
of basis.
For T L(V ), we dene the determinant of T , denoted det T , to

be (1)dim V times the constant term in the characteristic polynomial
of T . The motivation for the factor (1)dim V in this denition comes
from 10.6.
If V is a complex vector space, then det T equals the product of
the eigenvalues of T , counting multiplicity; this follows immediately
from 10.6. Recall that if V is a complex vector space, then there is
a basis of V with respect to which T has an upper-triangular matrix
(see 5.13); thus det T equals the product of the diagonal entries of this
matrix (see 8.10).
If V is a real vector space, then det T equals the product of the
eigenvalues of T times the product of the second coordinates of the
eigenpairs of T , each repeated according to multiplicitythis follows
from 10.7 and the observation that m = dim V 2M (in the notation
of 10.7), and hence (1)m = (1)dim V .
For example, suppose T L(C3 ) is the operator whose matrix is
given by 10.8. As we noted in the last section, the eigenvalues of T are
1, 2 + 3i, and 2 3i, each with multiplicity 1. Computing the product
of the eigenvalues, we have det T = (1)(2 + 3i)(2 3i); in other words,
det T = 13.
As another example, suppose T L(R3 ) is the operator whose matrix is also given by 10.8 (note that in the previous paragraph we were
working on a complex vector space; now we are working on a real vector space). Then, as we noted earlier, 1 is the only eigenvalue of T (it
has multiplicity 1) and (4, 13) is the only eigenpair of T (it has multiplicity 1). Computing the product of the eigenvalues times the product
of the second coordinates of the eigenpairs, we have det T = (1)(13);
in other words, det T = 13.
The reason that the operators in the two previous examples have the
same determinant will become clear after we nd a formula (valid on
both complex and real vector spaces) for computing the determinant
of an operator from its matrix.
In this section, we will prove some simple but important properties
of determinants. In the next section, we will discover how to calculate
det T from the matrix of T (with respect to an arbitrary basis). We begin
with a crucial result that has an easy proof with our approach.
10.14 Proposition: An operator is invertible if and only if its determinant is nonzero.
Proof: First suppose V is a complex vector space and T L(V ).
The operator T is invertible if and only if 0 is not an eigenvalue of T .
Clearly this happens if and only if the product of the eigenvalues of T
is not 0. Thus T is invertible if and only if det T = 0, as desired.
Now suppose V is a real vector space and T L(V ). Again, T is
invertible if and only if 0 is not an eigenvalue of T . Using the notation
of 10.7, we have
10.15
det T = 1 . . . m 1 . . . M ,
where the s are the eigenvalues of T and the s are the second coordinates of the eigenpairs of T , each repeated according to multiplicity.
For each eigenpair (j , j ), we have j 2 < 4j . In particular, each j
is positive. This implies (see 10.15) that 1 . . . m = 0 if and only if
det T = 0. Thus T is invertible if and only if det T = 0, as desired.
If T L(V ) and , z F, then is an eigenvalue of T if and only if
z is an eigenvalue of zI T . This follows from
(T I) = (zI T ) (z )I.
Raising both sides of this equation to the dim V power and then taking
null spaces of both sides shows that the multiplicity of as an eigenvalue of T equals the multiplicity of z as an eigenvalue of zI T .
223
224
The next lemma gives the analogous result for eigenpairs. We will use
this lemma to show that the characteristic polynomial can be expressed
as a certain determinant.
Real vector spaces are
harder to deal with
than complex vector
spaces. The rst time
10.16 Lemma: Suppose V is a real vector space, T L(V ), and

, , x R with 2 < 4. Then (, ) is an eigenpair of T if and only
if (2x , x 2 + x + ) is an eigenpair of xI T . Furthermore, these
eigenpairs have the same multiplicities.
you read this chapter,

you may want to
concentrate on the
basic ideas by
Proof: First we need to check that (2x , x 2 + x + ) satises

the inequality required of an eigenpair. We have
considering only
(2x )2 = 4x 2 + 4x + 2
complex vector spaces
< 4x 2 + 4x + 4
and ignoring the
= 4(x 2 + x + ).
special procedures
needed to deal with
real vector spaces.
Thus (2x , x 2 + x + ) satises the required inequality.

Now
T 2 + T + I = (xI T )2 (2x + )(xI T ) + (x 2 + x + )I,
as you should verify. Thus (, ) is an eigenpair of T if and only if
(2x , x 2 + x + ) is an eigenpair of xI T . Furthermore, raising
both sides of the equation above to the dim V power and then taking
null spaces of both sides shows that the multiplicities are equal.
Most textbooks take the theorem below as the denition of the characteristic polynomial. Texts using that approach must spend considerably more time developing the theory of determinants before they get
to interesting linear algebra.
10.17 Theorem: Suppose T L(V ). Then the characteristic polynomial of T equals det(zI T ).
Proof: First suppose V is a complex vector space. Let 1 , . . . , n
denote the eigenvalues of T , repeated according to multiplicity. Thus
for z C, the eigenvalues of zI T are z 1 , . . . , z n , repeated
according to multiplicity. The determinant of zI T is the product of
these eigenvalues. In other words,
det(zI T ) = (z 1 ) . . . (z n ).
Determinant of a Matrix
The right side of the equation above is, by denition, the characteristic
polynomial of T , completing the proof when V is a complex vector
space.
Now suppose V is a real vector space. Let 1 , . . . , m denote the
eigenvalues of T and let (1 , 1 ), . . . , (M , M ) denote the eigenpairs
of T , each repeated according to multiplicity. Thus for x R, the
eigenvalues of xI T are x 1 , . . . , x m and, by 10.16, the eigenpairs
of xI T are
(2x 1 , x 2 + 1 x + 1 ), . . . , (2x M , x 2 + M x + M ),
each repeated according to multiplicity. Hence
det(xI T ) = (x 1 ) . . . (x m )(x 2 + 1 x + 1 ) . . . (x 2 + M x + M ).
The right side of the equation above is, by denition, the characteristic
polynomial of T , completing the proof when V is a real vector space.
Most of this section is devoted to discovering how to calculate det T
from the matrix of T (with respect to an arbitrary basis). Lets start with
the easiest situation. Suppose V is a complex vector space, T L(V ),
and we choose a basis of V with respect to which T has an uppertriangular matrix. Then, as we noted in the last section, det T equals
the product of the diagonal entries of this matrix. Could such a simple
formula be true in general?
Unfortunately the determinant is more complicated than the trace.
In particular, det T need not equal the product of the diagonal entries
of M(T ) with respect to an arbitrary basis. For example, the operator
on F3 whose matrix equals 10.8 has determinant 13, as we saw in the
last section. However, the product of the diagonal entries of that matrix
equals 0.
For each square matrix A, we want to dene the determinant of A,
denoted det A, in such a way that det T = det M(T ) regardless of which
basis is used to compute M(T ). We begin our search for the correct definition of the determinant of a matrix by calculating the determinants
of some special operators.
Let c1 , . . . , cn F be nonzero scalars and let (v1 , . . . , vn ) be a basis

of V . Consider the operator T L(V ) such that M T , (v1 , . . . , vn )
equals
225
226
10.18
c1
cn
0
c2
0
..
.
..
cn1
here all entries of the matrix are 0 except for the upper-right corner
and along the line just below the diagonal. Lets nd the determinant
of T . Note that
(v1 , T v1 , T 2 v1 , . . . , T n1 v1 ) = (v1 , c1 v2 , c1 c2 v3 , . . . , c1 . . . cn1 vn ).
Recall that if the

minimal polynomial of
an operator T L(V )
Thus (v1 , T v1 , . . . , T n1 v1 ) is linearly independent (the cs are all nonzero). Hence if p is a nonzero polynomial with degree at most n 1,
then p(T )v1 = 0. In other words, the minimal polynomial of T cannot
have degree less than n. As you should verify, T n vj = c1 . . . cn vj for
each j, and hence T n = c1 . . . cn I. Thus zn c1 . . . cn is the minimal
polynomial of T . Because n = dim V , we see that zn c1 . . . cn is also
the characteristic polynomial of T . Multiplying the constant term of
this polynomial by (1)n , we get
has degree dim V , then

the characteristic
polynomial of T equals
the minimal polynomial
of T . Computing the
minimal polynomial is
often an efcient
method of nding the
characteristic
polynomial.
10.19
det T = (1)n1 c1 . . . cn .
If some cj equals 0, then clearly T is not invertible, so det T = 0 and

the same formula holds. Thus in order to have det T = det M(T ), we
will have to make the determinant of 10.18 equal to (1)n1 c1 . . . cn .
However, we do not yet have enough evidence to make a reasonable
guess about the proper denition of the determinant of an arbitrary
square matrix.
To compute the determinants of a more complicated class of operators, we introduce the notion of permutation. A permutation of
(1, . . . , n) is a list (m1 , . . . , mn ) that contains each of the numbers
1, . . . , n exactly once. The set of all permutations of (1, . . . , n) is denoted perm n. For example, (2, 3, . . . , n, 1) perm n. You should think
of an element of perm n as a rearrangement of the rst n integers.
For simplicity we will work with matrices with complex entries (at
this stage we are providing only motivationformal proofs will come
later). Let c1 , . . . , cn C and let (v1 , . . . , vn ) be a basis of V , which
we are assuming is a complex vector space. Consider a permutation
(p1 , . . . , pn ) perm n that can be obtained as follows: break (1, . . . , n)
into lists of consecutive integers and in each list move the rst term to
the end of that list. For example, taking n = 9, the permutation
10.20
(2, 3, 1, 5, 6, 7, 4, 9, 8)
is obtained from (1, 2, 3), (4, 5, 6, 7), (8, 9) by moving the rst term of
each of these lists to the end, producing (2, 3, 1), (5, 6, 7, 4), (9, 8), and
then putting these together to form 10.20. Let T L(V ) be the operator
such that
10.21
T vk = ck vpk
for k = 1, . . . , n. We want to nd a formula for det T . This generalizes

our earlier example because if (p1 , . . . , pn ) happens to be the permutation (2, 3, . . . , n, 1), then the operator T whose matrix equals 10.18 is
the same as the operator T dened by 10.21.
With respect to the basis (v1 , . . . , vn ), the matrix of the operator T
dened by 10.21 is a block diagonal matrix
A=
A1
0
..
AM
where each block is a square matrix of the form 10.18. The eigenvalues
of T equal the union of the eigenvalues of A1 , . . . , AM (see Exercise 3 in
Chapter 9). Recalling that the determinant of an operator on a complex
vector space is the product of the eigenvalues, we see that our denition
of the determinant of a square matrix should force
det A = (det A1 ) . . . (det AM ).
However, we already know how to compute the determinant of each Aj ,
which has the same form as 10.18 (of course with a different value of n).
Putting all this together, we see that we should have
det A = (1)n1 1 . . . (1)nM 1 c1 . . . cn ,
where Aj has size nj -by-nj . The number (1)n1 1 . . . (1)nM 1 is called
the sign of the permutation (p1 , . . . , pn ), denoted sign(p1 , . . . , pn ) (this
is a temporary denition that we will change to an equivalent denition
later, when we dene the sign of an arbitrary permutation).
227
228
To put this into a form that does not depend on the particular permutation (p1 , . . . , pn ), let aj,k denote the entry in row j, column k, of A;
thus
0 if j = pk ;
aj,k =
ck if j = pk .
Then
10.22
det A =

sign(m1 , . . . , mn ) am1 ,1 . . . amn ,n ,
(m1 ,...,mn )perm n
Some texts use the

unnecessarily fancy
term signum, which
means the same
as sign.
because each summand is 0 except the one corresponding to the permutation (p1 , . . . , pn ).
Consider now an arbitrary matrix A with entry aj,k in row j, column k. Using the paragraph above as motivation, we guess that det A
should be dened by 10.22. This will turn out to be correct. We can
now dispense with the motivation and begin the more formal approach.
First we will need to dene the sign of an arbitrary permutation.
The sign of a permutation (m1 , . . . , mn ) is dened to be 1 if the
number of pairs of integers (j, k) with 1 j < k n such that j appears after k in the list (m1 , . . . , mn ) is even and 1 if the number of
such pairs is odd. In other words, the sign of a permutation equals 1 if
the natural order has been changed an even number of times and equals
1 if the natural order has been changed an odd number of times. For
example, in the permutation (2, 3, . . . , n, 1) the only pairs (j, k) with
j < k that appear with changed order are (1, 2), (1, 3), . . . , (1, n); because we have n 1 such pairs, the sign of this permutation equals
(1)n1 (note that the same quantity appeared in 10.19).
The permutation (2, 1, 3, 4), which is obtained from the permutation
(1, 2, 3, 4) by interchanging the rst two entries, has sign 1. The next
lemma shows that interchanging any two entries of any permutation
changes the sign of the permutation.
10.23 Lemma: Interchanging two entries in a permutation multiplies
the sign of the permutation by 1.
Proof: Suppose we have two permutations, where the second permutation is obtained from the rst by interchanging two entries. If the
two entries that we interchanged were in their natural order in the rst
permutation, then they no longer are in the second permutation, and
229
vice versa, for a net change (so far) of 1 or 1 (both odd numbers) in
the number of pairs not in their natural order.
Consider each entry between the two interchanged entries. If an intermediate entry was originally in the natural order with respect to the
rst interchanged entry, then it no longer is, and vice versa. Similarly,
if an intermediate entry was originally in the natural order with respect
to the second interchanged entry, then it no longer is, and vice versa.
Thus the net change for each intermediate entry in the number of pairs
not in their natural order is 2, 0, or 2 (all even numbers).
For all the other entries, there is no change in the number of pairs
not in their natural order. Thus the total net change in the number of
pairs not in their natural order is an odd number. Thus the sign of the
second permutation equals 1 times the sign of the rst permutation.
If A is an n-by-n matrix
10.24
a1,1
.
A = ..
an,1
...
...
a1,n
..
. ,
an,n
then the determinant of A, denoted det A, is dened by

sign(m1 , . . . , mn ) am1 ,1 . . . amn ,n .
10.25
det A =
(m1 ,...,mn )perm n
Our motivation for this

denition comes
from 10.22.
For example, if A is the 1-by-1 matrix [a1,1 ], then det A = a1,1 because perm 1 has only one element, namely, (1), which has sign 1. For
a more interesting example, consider a typical 2-by-2 matrix. Clearly
perm 2 has only two elements, namely, (1, 2), which has sign 1, and
(2, 1), which has sign 1. Thus

a1,1 a1,2
= a1,1 a2,2 a2,1 a1,2 .
10.26
det
a2,1 a2,2
To make sure you understand this process, you should now nd the
formula for the determinant of the 3-by-3 matrix
a1,1 a1,2 a1,3
a2,1 a2,2 a2,3

a3,1 a3,2 a3,3
using just the denition given above (do this even if you already know
the answer).
The set perm 3

contains 6 elements. In
general, perm n
contains n! elements.
Note that n! rapidly
grows large as n
increases.
230
Lets compute the determinant of an upper-triangular matrix
a1,1
..
.
A=
.
0
an,n
The permutation (1, 2, . . . , n) has sign 1 and thus contributes a term
of a1,1 . . . an,n to the sum 10.25 dening det A. Any other permutation
(m1 , . . . , mn ) perm n contains at least one entry mj with mj > j,
which means that amj ,j = 0 (because A is upper triangular). Thus all
the other terms in the sum 10.25 dening det A make no contribution. Hence det A = a1,1 . . . an,n . In other words, the determinant of an
upper-triangular matrix equals the product of the diagonal entries. In
particular, this means that if V is a complex vector space, T L(V ),
and we choose a basis of V with respect to which M(T ) is upper triangular, then det T = det M(T ). Our goal is to prove that this holds for
every basis of V , not just bases that give upper-triangular matrices.
Generalizing the computation from the paragraph above, next we
will show that if A is a block upper-triangular matrix
A1
..
,
A=
.
0
Am
where each Aj is a 1-by-1 or 2-by-2 matrix, then
10.27
det A = (det A1 ) . . . (det Am ).
To prove this, consider an element of perm n. If this permutation

moves an index corresponding to a 1-by-1 block on the diagonal anyplace else, then the permutation makes no contribution to the sum
10.25 dening det A (because A is block upper triangular). For a pair
of indices corresponding to a 2-by-2 block on the diagonal, the permutation must either leave these indices xed or interchange them; otherwise again the permutation makes no contribution to the sum 10.25
dening det A (because A is block upper triangular). These observations, along with the formula 10.26 for the determinant of a 2-by-2 matrix, lead to 10.27. In particular, if V is a real vector space, T L(V ),
and we choose a basis of V with respect to which M(T ) is a block
upper-triangular matrix with 1-by-1 and 2-by-2 blocks on the diagonal
as in 9.9, then det T = det M(T ).
231
Our goal is to prove that det T = det M(T ) for every T L(V ) and
every basis of V . To do this, we will need to develop some properties of determinants of matrices. The lemma below is the rst of the
properties we will need.
An entire book could

be devoted just to
deriving properties of
determinants.
Fortunately we need
10.28 Lemma: Suppose A is a square matrix. If B is the matrix

obtained from A by interchanging two columns, then
det A = det B.
Proof: Suppose A is given by 10.24 and B is obtained from A by
interchanging two columns. Think of the sum 10.25 dening det A and
the corresponding sum dening det B. The same products of as appear
in both sums, though they correspond to different permutations. The
permutation corresponding to a given product of as when computing
det B is obtained by interchanging two entries in the corresponding
permutation when computing det A, thus multiplying the sign of the
permutation by 1 (see 10.23). Hence det A = det B.
If T L(V ) and the matrix of T (with respect to some basis) has two
equal columns, then T is not injective and hence det T = 0. Though
this comment makes the next lemma plausible, it cannot be used in the
proof because we do not yet know that det T = det M(T ).
10.29 Lemma: If A is a square matrix that has two equal columns,
then det A = 0.
Proof: Suppose A is a square matrix that has two equal columns.
Interchanging the two equal columns of A gives the original matrix A.
Thus from 10.28 (with B = A), we have det A = det A, which implies
that det A = 0.
This section is long, so lets pause for a paragraph. The symbols

that appear on the rst page of each chapter are decorations intended
to take up space so that the rst section of the chapter can start on the
next page. Chapter 1 has one of these symbols, Chapter 2 has two of
them, and so on. The symbols get smaller with each chapter. What you
may not have noticed is that the sum of the areas of the symbols at the
beginning of each chapter is the same for all chapters. For example, the
diameter of each symbol at the beginning of Chapter 10 equals 1/ 10

times the diameter of the symbol in Chapter 1.
only a few of the basic

properties.
232
We need to introduce notation that will allow us to represent a matrix in terms of its columns. If A is an n-by-n matrix
a1,1 . . . a1,n
.
..
A=
. ,
..
an,1 . . . an,n
then we can think of the kth column of A as an n-by-1 matrix
a1,k
.
ak =
.. .
an,k
We will write A in the form
[ a1
...
an ],
with the understanding that ak denotes the kth column of A. With this
notation, note that aj,k , with two subscripts, denotes an entry of A,
whereas ak , with one subscript, denotes a column of A.
The next lemma shows that a permutation of the columns of a matrix
changes the determinant by a factor of the sign of the permutation.
Some texts dene the
determinant to be the
function dened on the
square matrices that is
linear as a function of
each column separately
and that satises 10.30
and det I = 1. To prove
that such a function
exists and that it is
unique takes a
nontrivial amount of
work.
10.30 Lemma: Suppose A = [ a1 . . .

If (m1 , . . . , mn ) is a permutation, then
det[ am1
...
an ] is an n-by-n matrix.

amn ] = sign(m1 , . . . , mn ) det A.
Proof: Suppose (m1 , . . . , mn ) perm n. We can transform the

matrix [ am1 . . . amn ] into A through a series of steps. In each
step, we interchange two columns and hence multiply the determinant
by 1 (see 10.28). The number of steps needed equals the number
of steps needed to transform the permutation (m1 , . . . , mn ) into the
permutation (1, . . . , n) by interchanging two entries in each step. The
proof is completed by noting that the number of such steps is even if
(m1 , . . . , mn ) has sign 1, odd if (m1 , . . . , mn ) has sign 1 (this follows
from 10.23, along with the observation that the permutation (1, . . . , n)
has sign 1).
Let A = [ a1 . . . an ]. For 1 k n, think of all columns of A
except the kth column as xed. We have
det A = det[ a1
...
ak
...
233
an ],
and we can think of det A as a function of the kth column ak . This

function, which takes ak to the determinant above, is a linear map
from the vector space of n-by-1 matrices with entries in F to F. The
linearity follows easily from 10.25, where each term in the sum contains
precisely one entry from the kth column of A.
Now we are ready to prove one of the key properties about determinants of square matrices. This property will enable us to connect the
determinant of an operator with the determinant of its matrix. Note
that this proof is considerably more complicated than the proof of the
corresponding result about the trace (see 10.9).
10.31
then
Theorem:
If A and B are square matrices of the same size,
This theorem was rst

proved in 1812 by the
det(AB) = det(BA) = (det A)(det B).
French mathematicians
Jacques Binet and
Proof: Let A = [ a1 . . .
of A. Let
b1,1 . . .
.
B = ..
bn,1 . . .
an ], where each ak is an n-by-1 column
b1,n
..
. = [ b1
bn,n
...
bn ],
where each bk is an n-by-1 column of B. Let ek denote the n-by-1 matrix

that equals 1 in the kth row and 0 elsewhere. Note that Aek = ak and
n
Bek = bk . Furthermore, bk = m=1 bm,k em .
First we will prove that det(AB) = (det A)(det B). A moments
thought about the denition of matrix multiplication shows that AB =
[ Ab1 . . . Abn ]. Thus
det(AB) = det[ Ab1 . . . Abn ]
n
n
= det[ A( m1 =1 bm1 ,1 em1 ) . . . A( mn =1 bmn ,n emn ) ]
n
n
= det[ m1 =1 bm1 ,1 Aem1 . . .
mn =1 bmn ,n Aemn ]
=
m1 =1
bm1 ,1 . . . bmn ,n det[ Aem1
...
Aemn ],
mn =1
where the last equality comes from repeated applications of the linearity of det as a function of one column at a time. In the last sum above,
Augustin-Louis Cauchy.
234
all terms in which mj = mk for some j = k can be ignored because the

determinant of a matrix with two equal columns is 0 (by 10.29). Thus
instead of summing over all m1 , . . . , mn with each mj taking on values
1, . . . , n, we can sum just over the permutations, where the mj s have
distinct values. In other words,
det(AB) =
bm1 ,1 . . . bmn ,n det[ Aem1
...
Aemn ]
(m1 ,...,mn )perm n

bm1 ,1 . . . bmn ,n sign(m1 , . . . , mn ) det A
(m1 ,...,mn )perm n
= (det A)

sign(m1 , . . . , mn ) bm1 ,1 . . . bmn ,n
(m1 ,...,mn )perm n
= (det A)(det B),

where the second equality comes from 10.30.
In the paragraph above, we proved that det(AB) = (det A)(det B).
Interchanging the roles of A and B, we have det(BA) = (det B)(det A).
The last equation can be rewritten as det(BA) = (det A)(det B), completing the proof.
Now we can prove that the determinant of the matrix of an operator is independent of the basis with respect to which the matrix is
computed.
10.32 Corollary: Suppose T L(V ). If (u1 , . . . , un ) and (v1 , . . . , vn )
are bases of V , then

det M T , (u1 , . . . , un ) = det M T , (v1 , . . . , vn ) .
Note the similarity of
this proof to the proof
about the trace
(see 10.10).
Proof: Suppose (u1 , . . . , un ) and (v1 , . . . , vn ) are bases of V . Let

A = M I, (u1 , . . . , un ), (v1 , . . . , vn ) . Then

det M T , (u1 , . . . , un ) = det A1 M T , (v1 , . . . , vn ) A

= det M T , (v1 , . . . , vn ) A A1

= det M T , (v1 , . . . , vn ) ,
where the rst equality follows from 10.3 and the second equality follows from 10.31. The third equality completes the proof.
The theorem below states that the determinant of an operator equals

the determinant of the matrix of the operator. This theorem does not
specify a basis because, by the corollary above, the determinant of the
matrix of an operator is the same for every choice of basis.
10.33
Theorem: If T L(V ), then det T = det M(T ).
Proof: Let T L(V ). As noted above, 10.32 implies that det M(T )
is independent of which basis of V we choose. Thus to show that
det T = det M(T )
for every basis of V , we need only show that the equation above holds
for some basis of V . We already did this (on page 230), choosing a basis
of V with respect to which M(T ) is an upper-triangular matrix (if V is a
complex vector space) or an appropriate block upper-triangular matrix
(if V is a real vector space).
If we know the matrix of an operator on a complex vector space, the
theorem above allows us to nd the product of all the eigenvalues without nding any of the eigenvalues. For example, consider the operator
on C5 whose matrix is
0 0 0 0 3
1 0 0 0 6
0 1 0 0 0 .
0 0 1 0 0
0 0 0 1 0
No one knows an exact formula for any of the eigenvalues of this operator. However, we do know that the product of the eigenvalues equals 3
because the determinant of the matrix above equals 3.
The theorem above also allows us easily to prove some useful properties about determinants of operators by shifting to the language of
determinants of matrices, where certain properties have already been
proved or are obvious. We carry out this procedure in the next corollary.
10.34
Corollary: If S, T L(V ), then

det(ST ) = det(T S) = (det S)(det T ).
235
236
Proof: Suppose S, T L(V ). Choose any basis of V . Then

det(ST ) = det M(ST )

= det M(S)M(T )

= det M(S) det M(T )
= (det S)(det T ),
where the rst and last equalities come from 10.33 and the third equality comes from 10.31.
In the paragraph above, we proved that det(ST ) = (det S)(det T ). Interchanging the roles of S and T , we have det(T S) = (det T )(det S). Because multiplication of elements of F is commutative, the last equation
can be rewritten as det(T S) = (det S)(det T ), completing the proof.
Volume
Most applied
mathematicians agree
that determinants
should rarely be used
in serious numeric
calculations.
We proved the basic results of linear algebra before introducing determinants in this nal chapter. Though determinants have value as a
research tool in more advanced subjects, they play little role in basic
linear algebra (when the subject is done right). Determinants do have
one important application in undergraduate mathematics, namely, in
computing certain volumes and integrals. In this nal section we will
use the linear algebra we have learned to make clear the connection
between determinants and these applications. Thus we will be dealing
with a part of analysis that uses linear algebra.
We begin with some purely linear algebra results that will be useful when investigating volumes. Recall that an isometry on an innerproduct space is an operator that preserves norms. The next result
shows that every isometry has determinant with absolute value 1.
10.35 Proposition: Suppose that V is an inner-product space. If
S L(V ) is an isometry, then |det S| = 1.
Proof: Suppose S L(V ) is an isometry. First consider the case
where V is a complex inner-product space. Then all the eigenvalues of S
have absolute value 1 (by 7.37). Thus the product of the eigenvalues
of S, counting multiplicity, has absolute value one. In other words,
|det S| = 1, as desired.
Volume
Now suppose V is a real inner-product space. Then there is an orthonormal basis of V with respect to which S has a block diagonal matrix,
where each block on the diagonal is a 1-by-1 matrix containing 1 or 1
or a 2-by-2 matrix of the form

cos sin
10.36
,
sin
cos
with (0, ) (see 7.38). Note that the constant term of the characteristic polynomial of each matrix of the form 10.36 equals 1 (because
cos2 + sin2 = 1). Hence the second coordinate of every eigenpair
of S equals 1. Thus the determinant of S is the product of 1s and 1s.
In particular, |det S| = 1, as desired.
Suppose V is a real inner-product space and S L(V ) is an isometry.
By the proposition above, the determinant of S equals 1 or 1. Note
that
{v V : Sv = v}
is the subspace of V consisting of all eigenvectors of S corresponding
to the eigenvalue 1 (or is the subspace {0} if 1 is not an eigenvalue
of S). Thinking geometrically, we could say that this is the subspace
on which S reverses direction. A careful examination of the proof of
the last proposition shows that det S = 1 if this subspace has even
dimension and det S = 1 if this subspace has odd dimension.
A self-adjoint operator on a real inner-product space has no eigenpairs (by 7.11). Thus the determinant of a self-adjoint operator on a
real inner-product space equals the product of its eigenvalues, counting multiplicity (of course, this holds for any operator, self-adjoint or
not, on a complex vector space).
Recall that if V is an inner-product space and T L(V ), then T T
is a positive operator and hence has a unique positive square root, de
noted T T (see 7.27 and 7.28). Because T T is positive, all its eigenvalues are nonnegative (again, see 7.27), and hence its determinant is
nonnegative. Thus in the corollary below, taking the absolute value of
det T T would be superuous.

10.37
then
Corollary: Suppose V is an inner-product space. If T L(V ),
|det T | = det T T .
237
238
Another proof of this
corollary is suggested
Proof: Suppose T L(V ). By the polar decomposition (7.41), there

is an isometry S L(V ) such that
in Exercise 24 in this
T = S T T .
chapter.
Thus
|det T | = |det S| det T T
= det T T ,
where the rst equality follows from 10.34 and the second equality
follows from 10.35.
We are not formally

dening the phrase
reverses direction
because these
comments are meant to
be an intuitive aid to
our understanding, not
rigorous mathematics.
Suppose V is a real inner-product space and T L(V ) is invertible.

The det T is either positive or negative. A careful examination of the
proof of the corollary above can help us attach a geometric meaning
to whichever of these possibilities holds. To see this, rst apply the
real spectral theorem (7.13) to the positive operator T T , getting an
orthonormal basis (e1 , . . . , en ) of V such that T T ej = j ej , where
1 , . . . , n are the eigenvalues of T T , repeated according to multi

plicity. Because each j is positive, T T never reverses direction.
Now consider the polar decomposition
T = S T T ,
where S L(V ) is an isometry. Then det T = (det S)(det T T ). Thus

whether det T is positive or negative depends on whether det S is positive or negative. As we saw earlier, this depends on whether the space
on which S reverses direction has even or odd dimension. Because
T is the product of S and an operator that never reverses direction
(namely, T T ), we can reasonably say that whether det T is positive

or negative depends on whether T reverses vectors an even or an odd
number of times.
Now we turn to the question of volume, where we will consider only
the real inner-product space R n (with its standard inner product). We
would like to assign to each subset of R n its n-dimensional volume,
denoted volume (when n = 2, this is usually called area instead of
volume). We begin with cubes, where we have a good intuitive notion of
volume. The cube in Rn with side length r and vertex (x1 , . . . , xn ) Rn
is the set
Volume
239
{(y1 , . . . , yn ) R n : xj < yj < xj + r for j = 1, . . . , n};

you should verify that when n = 2, this gives a square, and that when
n = 3, it gives a familiar three-dimensional cube. The volume of a cube
in R n with side length r is dened to be r n . To dene the volume of
an arbitrary set R n , the idea is to write as a subset of a union of
many small cubes, then add up the volumes of these small cubes. As
we approximate more accurately by unions (perhaps innite unions)
of small cubes, we get a better estimate of volume .
Rather than take the trouble to make precise this denition of volume, we will work only with an intuitive notion of volume. Our purpose
in this book is to understand linear algebra, whereas notions of volume
belong to analysis (though as we will soon see, volume is intimately connected with determinants). Thus for the rest of this section we will rely
on intuitive notions of volume rather than on a rigorous development,
though we shall maintain our usual rigor in the linear algebra parts
of what follows. Everything said here about volume will be correct
the intuitive reasons given here can be converted into formally correct
proofs using the machinery of analysis.
For T L(V ) and R n , dene T () by
T () = {T x : x }.
Our goal is to nd a formula for the volume of T () in terms of T
and the volume of . First lets consider a simple example. Suppose
1 , . . . , n are positive numbers. Dene T L(Rn ) by T (x1 , . . . , xn ) =
(1 x1 , . . . , n xn ). If is a cube in R n with side length r , then T ()
is a box in R n with sides of length 1 r , . . . , n r . This box has volume
1 . . . n r n , whereas the cube has volume r n . Thus this particular T ,
when applied to a cube, multiplies volumes by a factor of 1 . . . n ,
which happens to equal det T .
As above, assume that 1 , . . . , n are positive numbers. Now suppose that (e1 , . . . , en ) is an orthonormal basis of R n and T is the operator on Rn that satises T ej = j ej for j = 1, . . . , n. In the special
case where (e1 , . . . , en ) is the standard basis of Rn , this operator is the
same one as dened in the paragraph above. Even for an arbitrary orthonormal basis (e1 , . . . , en ), this operator has the same behavior as
the one in the paragraph aboveit multiplies the j th basis vector by
a factor of j . Thus we can reasonably assume that this operator also
multiplies volumes by a factor of 1 . . . n , which again equals det T .
Readers familiar with

outer measure will
recognize that concept
here.
240
We need one more ingredient before getting to the main result in

this section. Suppose S L(R n ) is an isometry. For x, y Rn , we
have
Sx Sy = S(x y)
= x y.
In other words, S does not change the distance between points. As you
can imagine, this means that S does not change volumes. Specically,
if R n , then volume S() = volume .
Now we can give our pseudoproof that an operator T L(R n )
changes volumes by a factor of |det T |.
10.38
Theorem: If T L(R n ), then

volume T () = |det T |(volume )
for R n .
Proof: First consider the case where T L(Rn ) is a positive
operator. Let 1 , . . . , n be the eigenvalues of T , repeated according
to multiplicity. Each of these eigenvalues is a nonnegative number
(see 7.27). By the real spectral theorem (7.13), there is an orthonormal
basis (e1 , . . . , en ) of V such that T ej = j ej for each j. As discussed
above, this implies that T changes volumes by a factor of det T .
Now suppose T L(R n ) is an arbitrary operator. By the polar decomposition (7.41), there is an isometry S L(V ) such that
T = S T T .

T T () . Thus

volume T () = volume S T T ()
= volume T T ()
= (det T T )(volume )
If Rn , then T () = S
= |det T |(volume ),
where the second equality holds because volumes are not changed by
the isometry S (as discussed above), the third equality holds by the
previous paragraph (applied to the positive operator T T ), and the

fourth equality holds by 10.37.
Volume
The theorem above leads to the appearance of determinants in the

formula for change of variables in multivariable integration. To describe this, we will again be vague and intuitive. If R n and f is
a real-valued function (not necessarily linear) on , then the integral
"
"
of f over , denoted f or f (x) dx, is dened by breaking into
pieces small enough so that f is almost constant on each piece. On
each piece, multiply the (almost constant) value of f by the volume of
the piece, then add up these numbers for all the pieces, getting an approximation to the integral that becomes more accurate as we divide
into ner pieces. Actually needs to be a reasonable set (for example, open or measurable) and f needs to be a reasonable function
(for example, continuous or measurable), but we will not worry about
"
those technicalities. Also, notice that the x in f (x) dx is a dummy
variable and could be replaced with any other symbol.
Fix a set Rn and a function (not necessarily linear) : R n .
We will use to make a change of variables in an integral. Before we
can get to that, we need to dene the derivative of , a concept that
uses linear algebra. For x , the derivative of at x is an operator
T L(Rn ) such that
(x + y) (x) T y
lim
= 0.
y0
y
If an operator T L(R n ) exists satisfying the equation above, then
is said to be differentiable at x. If is differentiable at x, then
there is a unique operator T L(Rn ) satisfying the equation above
(we will not prove this). This operator T is denoted (x). Intuitively,
the idea is that for x xed and y small, a good approximation to

(x +y) is (x)+ (x) (y) (note that (x) L(Rn ), so this makes
sense). Note that for x xed the addition of the term (x) does not
change volumes. Thus if is a small subset of containing x, then

volume ( ) is approximately equal to volume (x) ( ).
Because is a function from to R n , we can write

(x) = 1 (x), . . . , n (x) ,
where each j is a function from to R. The partial derivative of j
with respect to the kth coordinate is denoted Dk j . Evaluating this
partial derivative at a point x gives Dk j (x). If is differentiable
at x, then the matrix of (x) with respect to the standard basis of R n
241
If n = 1, then the
derivative in this sense
is the operator on R of
multiplication by the
derivative in the usual
sense of one-variable
calculus.
242
contains Dk j (x) in row j, column k (we will not prove this). In other
words,
D1 1 (x) . . . Dn 1 (x)
..
..
.
10.39
M( (x)) =
.
.
D1 n (x) . . . Dn n (x)
Suppose that is differentiable at each point of and that is
injective on . Let f be a real-valued function dened on (). Let
x and let be a small subset of containing x. As we noted above,

volume ( ) volume (x) ( ),
where the symbol means approximately equal to. Using 10.38, this
becomes
volume ( ) |det (x)|(volume ).
Let y = (x). Multiply the left side of the equation above by f (y) and

the right side by f (x) (because y = (x), these two quantities are
equal), getting
10.40

f (y) volume ( ) f (x) |det (x)|(volume ).
Now divide into many small pieces and add the corresponding versions of 10.40, getting

f (y) dy =
f (x) |det (x)| dx.
10.41
()
If you are not familiar

with polar and
spherical coordinates,
skip the remainder of
this section.
This formula was our goal. It is called a change of variables formula

because you can think of y = (x) as a change of variables.
The key point when making a change of variables is that the factor
of |det (x)| must be included, as in the right side of 10.41. We nish
up by illustrating this point with two important examples. When n = 2,
we can use the change of variables induced by polar coordinates. In this
case is dened by
(r , ) = (r cos , r sin ),
where we have used r , as the coordinates instead of x1 , x2 for reasons
that will be obvious to everyone familiar with polar coordinates (and
will be a mystery to everyone else). For this choice of , the matrix of
partial derivatives corresponding to 10.39 is
Volume
cos
sin
243
r sin
r cos

,
as you should verify. The determinant of the matrix above equals r ,

thus explaining why a factor of r is needed when computing an integral
in polar coordinates.
Finally, when n = 3, we can use the change of variables induced by
spherical coordinates. In this case is dened by
(, , ) = ( sin cos , sin sin , cos ),
where we have used , , as the coordinates instead of x1 , x2 , x3
for reasons that will be obvious to everyone familiar with spherical
coordinates (and will be a mystery to everyone else). For this choice
of , the matrix of partial derivatives corresponding to 10.39 is
sin cos
sin sin
cos
cos cos
cos sin
sin
sin sin
sin cos ,
0
as you should verify. You should also verify that the determinant of the
matrix above equals 2 sin , thus explaining why a factor of 2 sin
is needed when computing an integral in spherical coordinates.
244
Exercises
1.
Suppose T L(V ) and (v1 , . . . , vn ) is a basis of V . Prove that

M T , (v1 , . . . , vn ) is invertible if and only if T is invertible.
2.
Prove that if A and B are square matrices of the same size and
AB = I, then BA = I.
3.
Suppose T L(V ) has the same matrix with respect to every basis of V . Prove that T is a scalar multiple of the identity operator.
4.
Suppose that (u1 , . . . , un ) and (v1 , . . . , vn ) are bases of V . Let

T L(V ) be the operator such that T vk = uk for k = 1, . . . , n.
Prove that

M T , (v1 , . . . , vn ) = M I, (u1 , . . . , un ), (v1 , . . . , vn ) .
5.
Prove that if B is a square matrix with complex entries, then there

exists an invertible square matrix A with complex entries such
that A1 BA is an upper-triangular matrix.
6.
Give an example of a real vector space V and T L(V ) such that

trace(T 2 ) < 0.
7.
Suppose V is a real vector space, T L(V ), and V has a basis

consisting of eigenvectors of T . Prove that trace(T 2 ) 0.
8.
Suppose V is an inner-product space and v, w L(V ). Dene

T L(V ) by T u = u, vw. Find a formula for trace T .
9.
Prove that if P L(V ) satises P 2 = P , then trace P is a nonnegative integer.
10.
Prove that if V is an inner-product space and T L(V ), then

trace T = trace T .
11.
Suppose V is an inner-product space. Prove that if T L(V ) is

a positive operator and trace T = 0, then T = 0.
Exercises
12.
245
Suppose T L(C3 ) is the operator whose matrix is
51 12 21
60 40 28 .
57 68
1
Someone tells you (accurately) that 48 and 24 are eigenvalues
of T . Without using a computer or writing anything down, nd
the third eigenvalue of T .
13.
Prove or give a counterexample: if T L(V ) and c F, then

trace(cT ) = c trace T .
14.
Prove or give a counterexample: if S, T L(V ), then trace(ST ) =

(trace S)(trace T ).
15.
Suppose T L(V ). Prove that if trace(ST ) = 0 for all S L(V ),

then T = 0.
16.
Suppose V is an inner-product space and T L(V ). Prove that

if (e1 , . . . , en ) is an orthonormal basis of V , then
trace(T T ) = T e1 2 + + T en 2 .
Conclude that the right side of the equation above is independent
of which orthonormal basis (e1 , . . . , en ) is chosen for V .
17.
Suppose V is a complex inner-product space and T L(V ). Let

1 , . . . , n be the eigenvalues of T , repeated according to multiplicity. Suppose
a1,1 . . . a1,n
.
..
.
.
.
an,1 . . . an,n
is the matrix of T with respect to some orthonormal basis of V .
Prove that
n
|1 |2 + + |n |2
|aj,k |2 .
k=1 j=1
18.
Suppose V is an inner-product space. Prove that

S, T = trace(ST )
denes an inner product on L(V ).
246
Exercise 19 fails on
19.
innite-dimensional
inner-product spaces,
leading to what are
for every v V , then T is normal.
called hyponormal
operators, which have a

if
T v T v
20.
Prove or give a counterexample: if T L(V ) and c F, then

det(cT ) = c dim V det T .
21.
Prove or give a counterexample: if S, T L(V ), then det(S +T ) =

det S + det T .
22.
Suppose A is a block upper-triangular matrix
well-developed theory.
A=
A1
..
0
Am
where each Aj along the diagonal is a square matrix. Prove that

det A = (det A1 ) . . . (det Am ).
23.
Suppose A is an n-by-n matrix with real entries. Let S L(Cn )

denote the operator on Cn whose matrix equals A, and let T
L(Rn ) denote the operator on R n whose matrix equals A. Prove
that trace S = trace T and det S = det T .
24.

det T = det T .
Use this to prove that |det T | = det T T , giving a different

proof than was given in 10.37.
25.
Let a, b, c be positive numbers. Find the volume of the ellipsoid

$
%
x2
y2
z2
(x, y, z) R 3 : 2 + 2 + 2 < 1
a
b
c
by nding a set R 3 whose volume you know and an operator
T L(R3 ) such that T () equals the ellipsoid above.
Symbol Index
R, 2
Re, 69
C, 2
Im, 69
F, 3
, 69
z
|z|, 69
P(F), 10
T |U , 76

M T , (v1 , . . . , vn ) , 82
, 23
M(T ), 82
Pm (F), 23
PU ,W , 92
dim V , 31
u, v, 99
L(V , W ), 38
v, 102
I, 38
U , 111
null T , 41
PU , 113
range T , 43

M T , (v1 , . . . , vn ), (w1 , . . . , wm ) ,
T , 118
F ,5
F , 10
, 120
T , 146
48
M(T ), 48
, 166
Mat(m, n, F), 50
, 231
M(v), 52

M v, (v1 , . . . , vn ) , 52
T (), 239
"
f , 241
T 1 , 54
Dk , 241
L(V ), 57
, 242
deg p, 66
247
Index
absolute value, 69
addition, 9
adjoint, 118
complex number, 2
complex spectral theorem,
133
complex vector space, 10
conjugate transpose, 120
coordinate, 4
cube, 238
cube root of an operator,
159
basis, 27
block diagonal matrix, 142
block upper-triangular matrix,
195
Cauchy-Schwarz inequality,
104
Cayley-Hamilton theorem for
complex vector
spaces, 173
Cayley-Hamilton theorem for
real vector spaces,
207
change of basis, 216
characteristic polynomial of a
2-by-2 matrix, 199
characteristic polynomial of an
operator on a complex
vector space, 172
characteristic polynomial of an
operator on a real
vector space, 206
characteristic value, 77
closed under addition, 13
closed under scalar
multiplication, 13
complex conjugate, 69
degree, 22
derivative, 241
determinant of a matrix,
229
determinant of an operator,
222
diagonal matrix, 87
diagonal of a matrix, 83
differentiable, 241
dimension, 31
direct sum, 15
divide, 180
division algorithm, 66
dot product, 98
eigenpair, 205
eigenvalue of a matrix, 194
eigenvalue of an operator,
77
eigenvector, 77
249
250
Index
Euclidean inner product,

100
eld, 3
nite dimensional, 22
functional analysis, 23
fundamental theorem of
algebra, 67
generalized eigenvector,
164
Gram-Schmidt procedure,
108
Hermitian, 128
homogeneous, 47
identity map, 38
identity matrix, 214
image, 43
imaginary part, 69
innite dimensional, 23
injective, 43
inner product, 99
inner-product space, 100
integral, 241
invariant, 76
inverse of a linear map, 53
inverse of a matrix, 214
invertible, 53
invertible matrix, 214
isometry, 147
isomorphic, 55
Jordan basis, 186
kernel, 41
length, 4
linear combination, 22
linear dependence lemma,

25
linear functional, 117
linear map, 38
linear span, 22
linear subspace, 13
linear transformation, 38
linearly dependent, 24
linearly independent, 23
list, 4
matrix, 48
matrix of a linear map, 48
matrix of a vector, 52
matrix of an operator, 82
minimal polynomial, 179
monic polynomial, 179
multiplicity of an eigenpair,
205
multiplicity of an eigenvalue,
171
nilpotent, 167
nonsingular matrix, 214
norm, 102
normal, 130
null space, 41
one-to-one, 43
onto, 44
operator, 57
orthogonal, 102
orthogonal complement,
111
orthogonal operator, 148
orthogonal projection, 113
orthonormal, 106
orthonormal basis, 107
parallelogram equality, 106
Index
permutation, 226
perpendicular, 102
point, 10
polynomial, 10
positive operator, 144
positive semidenite operator,
144
product, 41
projection, 92
Pythagorean theorem, 102
range, 43
real part, 69
real spectral theorem, 136
real vector space, 9
root, 64
scalar, 3
scalar multiplication, 9
self-adjoint, 128
sign of a permutation, 228
signum, 228
singular matrix, 214
singular values, 155
251
span, 22
spans, 22
spectral theorem, 133, 136
square root of an operator,
145, 159
standard basis, 27
subspace, 13
sum of subspaces, 14
surjective, 44
trace of a square matrix,
218
trace of an operator, 217
transpose, 120
triangle inequality, 105
tuple, 4
unitary operator, 148
upper triangular, 83
vector, 6, 10
vector space, 9
volume, 238

Sheldon Axler - Linear Algebra Done Right - Second Edition

Uploaded by

Copyright:

Available Formats

Sheldon Axler - Linear Algebra Done Right - Second Edition

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sheldon Axler - Linear Algebra Done Right - Second Edition

Uploaded by

Copyright:

Available Formats

Linear Algebra

Preface to the Student

Finite-Dimensional Vector Spaces

Eigenvalues and Eigenvectors

Operators on Inner-Product Spaces

Operators on Complex Vector Spaces

Operators on Real Vector Spaces

Trace and Determinant

Preface to the Instructor

Preface to the Instructor

Linear maps are introduced in Chapter 3. The key result here

Preface to the Instructor

Preface to the Instructor

Preface to the Student

Chapter 1. Vector Spaces

The symbol i was rst

The letter F is used

Thus if we prove a theorem involving F, we will know that it holds when

Clearly (zm )n = zmn and (wz)m = w m zm for all w, z F and all

called elds. In this

Chapter 1. Vector Spaces

Definition of Vector Space

call a list of length n an

Denition of Vector Space

(x1 , . . . , xn ) + (y1 , . . . , yn ) = (x1 + y1 , . . . , xn + yn ).

Often the mathematics of Fn becomes cleaner if we use a single

A. Abbott. This novel,

Chapter 1. Vector Spaces

We let 0 denote the list of length n all of whose coordinates are 0:

Elements of R 2 can be thought of as points or as vectors.

Denition of Vector Space

our subject is called

The sum of two vectors

Chapter 1. Vector Spaces

Having dealt with addition in Fn , we now turn to multiplication. We

here a F and (x1 , . . . , xn ) Fn .

Multiplication by positive scalars

If a is a negative number and x is a vector in R2 , then ax is the vector

Multiplication by negative scalars

Denition of Vector Space

Chapter 1. Vector Spaces

The simplest vector

C is called a complex vector space. Usually the choice of F is either

for all z F. We dene P(F) to be the set of all polynomials with

Properties of Vector Spaces

Properties of Vector Spaces

Proposition: A vector space has a unique additive identity.

end of the proof.

Each element v in a vector space has an additive inverse, an element

Every element in a vector space has a unique

Proof: Suppose V is a vector space. Let v V . Suppose that w

Chapter 1. Vector Spaces

Because of associativity, we can dispense with parentheses when

Proof: For v V , we have

and the additive

identity of V . The only

Proposition: 0v = 0 for every v V .

Adding the additive inverse of 0v to both sides of the equation above

Proposition: a0 = 0 for every a F.

Proof: For a F, we have