6

Elementary Linear Transform Theory

Whether they are spaces of arrows in space, functions or even matrices, vector spaces quickly
become boring if we dont do things with their elements move them around, differentiate or
integrate them, whatever. And often, what we do with these generalized vectors end up being
linear operations.
6.1 Basic Denitions and Examples
Let us suppose we have two vector spaces V and W (they could be the same vector space). Then
we can certainly have a function L that transforms any vector v from V into a corresponding
vector w = L(v) in W. This function is called a linear transformation (or linear transform or
linear operator) from V into W if and only if
L(
1
v
1
+
2
v
2
) =
1
L(v
1
) +
2
L(v
2
)
whenever
1
and
2
are scalars, and v
1
and v
2
are vectors in V . This expression, of course,
can be expanded to
L
_

k
v
k
_
=

k
L(v
k
)
for any linear combination

k

k
v
k
of vectors in V . (Remember, we are still insisting on linear
combinations having only nitely many terms.)
The domain of the operator is the vector space V , and the range is the set of all vectors in
W given by L(v) where v V . On occasion, we might also call V the input space, and W
the target space. This terminology is not standard, but is descriptive.
Often, V and W will be the same. If this is the case, then we will simply refer to L as a
linear transformation/transform/operator on V .
It is also often true that W is not clearly stated. In such cases we can take W to be any
vector space containing every L(v) for every v V . There is no requirement that every vector
in W can be treated as L(v) for some v V .
Here are a fewexamples of linear transforms on a traditional three-dimensional vector space
V with {i, j, k} being a standard basis. In each case, the operator is dened by explaining what
it does to an arbitrary vector v in V . Also given is a least one possible target space W.
11/28/2011
Elementary Linear Transform Theory Chapter & Page: 62
1. Any constant magnication, say,
M
2
(v) = 2v .
Here, W = V .
2. Projection onto the k vector,

pr
k
(v) .
Here, W is the one-dimensional space subspace of V consisting of all scalar multiples
of k . (Actually, you can also view all of V as W.)
3. Projection onto the plane spanned by i and j ,
P(v
1
i + v
2
j + v
3
k) =

pr
{i,j}
(v) = v
1
i + v
2
j .
Here, W is the plane spanned by i and j . (Again, you can also view all of V as W.)
4. The cross product with some xed vector a , say, a = 1i + 2j + 3k ,
K
a
(v) = a v .
Here, we can take W to be V . Also, since the cross product with a is always perpen-
dicular to a , we can rene our choice of W to being the plane orthogonal to a .
5. The dot product with some xed vector a , say, a = 1i + 2j + 3k ,
D
a
(v) = a v .
Here, it does not really make sense to view V as W since the result is not a vector in
V . Instead, since the result is a scalar (a real scalar, in fact), so W = .
6. The rotation R through some xed angle about some xed vector, say, k .
?

Exercise 6.1: Convince yourself that each of the above transformations are linear transfor-
mations.
We can also have very nontrivial linear transformations on nontraditional vector spaces. For
example, derivatives and the Laplace and Fourier transforms are linear transformations on certain
vector spaces of functions. Later on, we will be studying differential linear transformations of
functions which are associated with partial differential equations.
On the other hand, we can also have operators that are not linear. For example, the norm
is a nonlinear operator from any vector space with an inner product into .
?

Exercise 6.2 a: Verify that the norm is not a linear operator from any traditional vector
space V into .
b: Come up with some other examples of nonlinear operators (i.e, functions from one vector
space into another that are not linear).
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 63
6.2 A Note on Notation
There is something about our notation that should have been noted when we rst saw the ket
notation, |v . Remember, if we have a basis
B = { a
1
, a
2
, . . . , a
N
} ,
then
|v = |v
B
=
_
_
_
_
_
_
_
v
1
v
2
v
3
.
.
.
v
N
_

_
where v =
N

k=1
v
k
a
k
.
Observe:
v =
N

k=1
v
k
a
k
= v
1
a
1
+ v
2
a
2
+ + v
N
a
N
=
_
a
1
a
2
a
N
_
_
_
_
_
_
_
_
v
1
v
2
v
3
.
.
.
v
N
_

_
=
_
a
1
a
2
a
N
_
|v
B
.
So, using our ket notation,
v =
_
a
1
a
2
a
N
_
|v
B
where { a
1
, a
2
, . . . , a
N
} = B .
Note that |v is being multiplied on the left by the row matrix of the vectors making up the basis.
6.3 Matrices for Linear Transforms
Dening the Matrix in General
Suppose L is a linear operator from an N-dimensional vector space V to an M-dimensional
vector space W. Let
B
V
= { a
1
, a
2
, . . . , a
N
} be a basis for V ,
and let
B
W
= { b
1
, b
2
, . . . , b
M
} be a basis for W .
Now, since each L(a
k
) is in W, each L(a
k
) must be a linear combination of the b
j
s .
So, there are constants
j k
such that, for k = 1, 2, . . . , N ,
L(a
k
) =
M

j =1

j k
b
j
for k = 1, 2, . . . N , (6.1a)
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 64
which, we might note, can also be written as
L(a
k
) =
M

j =1
b
j

j k
=
_
b
1
b
2
b
N
_
_
_
_
_
_

1k

2k
.
.
.

Mk
_

_
, (6.1b)
or even as
|L(a
k
)
B
W
=
_
_
_
_
_
_

1k

2k
.
.
.

Mk
_

_
. (6.1c)
Using these
j k
s , we then dene the matrix L for L with respect to these bases to be the
MN matrix given by
[L]
j k
=
j k
for j = 1, 2, . . . , M and k = 1, 2, . . . , N .
That is, the ( j, k)
th
entry in L is the j
th
(scalar) component with respect to B
W
of the k
th
vector
in B
V
,
L(a
k
) =
M

j =1
[L]
j k
b
j
for k = 1, 2, . . . N . (6.2)
Equivalently, we can say that L is the MN matrix such that
_
b
1
b
2
b
M
_
L =
_
L(a
1
) L(a
2
) L(a
N
)
_
, (6.3)
or, perhaps more simply, we can say that
_
_
k
th
column
of L
_
_
= |L(a
k
)
B
W
for k = 1, 2, . . . , N . (6.4)
Any of these can be used to nd L for a given L.
Keep in mind that matrix L depends on the choice of bases B
V
and B
W
. On occasion, it
may be important to explicitly indicate which bases are being used. We will do that by denoting
the matrix for L by
L
B
W
B
V
.
This will be simplied to
L
B
if the input and target spaces are the same and we are just using one basis B (so, in the above,
B
V
= B
W
= B).
!

Example 6.1: Lets consider the cross product with some xed vector a , say, a = 1i +2j ,
K
a
(v) = a v
where V = W is a traditional vector space with standard basis B = {i, j, k} . We already
know K
a
is a linear operator on V . Computing the K
a
for each basis vector, we get
K
a
(i) = a i =
_
1i + 2j
_
i = = 0i + 0j 2k ,
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 65
K
a
(j) = a j =
_
1i + 2j
_
j = = 0i + 0j + 1k
and
K
a
(k) = a k =
_
1i + 2j
_
k = = 2i 1j + 0k .
So,
|K
a
(i) =
_
_
0
0
2
_
_
, |K
a
(j) =
_
_
0
0
1
_
_
, |K
a
(k) =
_
_
2
1
0
_
_
.
But, as noted above (equation (6.4)), these are also the respective columns in the matrix K
for the operator K
a
. Thus,
K =
_
_
0 0 2
0 0 1
2 1 0
_
_
.
So What? (Using the Matrix)
So what? Well, let v be a vector in V , and let w = L(v) . Since v V , it has (scalar)
components v
1
, v
2
, . . . and v
N
with respect to the given basis for V . So we can write
v =
N

k=1
v
k
a
k
and |v = |v
B
V
=
_
_
_
_
_
_
v
1
v
2
.
.
.
v
N
_

_
.
And since w = L(v) is in W, w has (scalar) components w
1
, w
2
, . . . and w
M
with respect
to the given basis for W. So we can write
w =
M

j =1
w
j
b
j
and |w = |w
B
W
=
_
_
_
_
_
_
w
1
w
2
.
.
.
w
M
_

_
.
Using this with the linearity of the operator,

j
w
j
b
j
= w = L(v) = L
_

k
v
k
a
k
_
=

k
v
k
L(a
k
)
=

k
v
k

j k
b
j
=

j k
v
k
b
j
=

j
_

k
[L]
j k
v
k
_
b
j
.
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 66
So

j
w
j
b
j
= w =

j
_

k
[L]
j k
v
k
_
b
j
, (6.5)
which means
w
j
=

k
[L]
j k
v
k
for each j .
But the righthand side of this is simply the formula for the j
th
entry in the matrix product of L
with |v . So,
|L(v) = |w = L|v ,
or, more explicitly,
|L(v)
B
W
= L
B
W
B
V
|v
B
V
.
What we have demonstrated is that:
1. Computationally, any linear transform can be completely described by a corresponding
matrix (provided we have a basis for each of the spaces).
2. This matrix is easily constructed from the components with respect to the basis of the
target space of the transforms of the basis vectors used for the input space.
Just for the heck of it, take another look at equation (6.5), which I will slightly rewrite as
L(v) =
M

j =1
b
j
_
N

k=1
[L]
j k
v
k
_
This is just the matrix product
L(v) =
_
b
1
b
2
b
M
_
L|v ,
or, more explicitly,
L(v) =
_
b
1
b
2
b
M
_
L
B
W
B
V
|v
B
V
.
!

Example 6.2: Again lets consider the cross product operator from example 6.1,
K
a
(v) = a v with a = 1i + 2j .
In that example, we saw that the matrix of this operator with respect to the standard basis
{i, j, k} is
K =
_
_
0 0 2
0 0 1
2 1 0
_
_
.
So, in general,
|a v = |K
a
(v) = K|v .
In particular, if v = 2i + 3j + 5k , then
|a v =
_
_
0 0 2
0 0 1
2 1 0
_
_
_
_
2
3
5
_
_
=
_
_
10
5
1
_
_
.
That is,
a v = 10i 5j 1k .
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 67
Special (but Common) Cases
Using an Orthonormal Basis of the Target Space
Lets now assume that we are being intelligent and using an orthonormal basis at least of the
target space W, and lets emphasize this by renaming the vectors in this basis as
B
W
= { e
1
, e
2
, . . . , e
M
} .
Then equation (6.2) describing the entries of L(a
k
) becomes
L(a
k
) =
M

m=1
[L]
mk
e
m
for k = 1, 2, . . . N .
As weve done at least a couple of times before, well take the inner product of both sides with
one of the e
m
s and use the orthonormallity of this basis along with the linearity of L:
_
e
j

L(a
k
)
_
=
_
e
j

m=1
[L]
mk
e
m
_
=
M

m=1
[L]
mk
_
e
j

e
m
_
=
M

m=1
[L]
mk

j m
= [L]
j k
.
Thus,
If the basis being used for the target space,
B
W
= { e
1
, e
2
, . . . , e
M
} ,
is orthonormal, then the ( j k)
th
entry of the corresponding matrix L for a linear
operator L is
[L]
j k
=
_
e
j

L(a
k
)
_
. (6.6)
Target Space Equals Input Space
In a great many cases, we will be interested in a linear operator L just on a vector space V (i.e.,
L just transforms each vector in V to another vector in V ). Then we might as well use a single
orthonormal basis
B
V
= B
W
= B = { e
1
, e
2
, . . . , e
M
} .
In this case, when we are being explicit about the basis used for the matrix L, we use
L
B
instead of L
B
W
,B
V
.
The above formula for the ( j, k)
th
entry in this matrix reduces a little further to
[L]
j k
=
_
L
B
_
j k
=
_
e
j

L(e
k
)
_
. (6.7)
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 68
Linear Functionals
Alinear functional is simply a linear operator for which the target space is the space of all scalars,
or . The traditional vector operators described in problems A7 and B7, and the integral
operator described in problems C2 and D of Homework Handout V are linear functionals, and an
interesting theorem(the Riesz theorem) relating linear functionals to inner products is developed
in problem K in the same homework set.
Innite Dimensional Spaces
See problem D of Homework Handout V.
6.4 Change of Basis for Transform Matrices
Suppose we have some linear transform L on an N-dimensional vector space V (with V being
both the input and the target space).
1
Let A and B be two orthonormal bases for this space.
Then L has matrices L
A
and L
B
with respect to bases A and B. Our goal now is to gure
out how to nd matrix L
B
from matrix L
A
using the change of basis matrices M
AB
and
M
BA
from the Big Theorem (theorem 5.3).
To do this, let v be any vector in V , and observe that
L
B
|v
B
= |L(v)
B
= M
BA
|L(v)
A
= M
BA
_
L
A
|v
A
_
= M
BA
_
L
A
_
M
AB
|v
B
__
.
After cutting out the middle and dropping the needless brackets, we have
L
B
|v
B
= M
BA
L
A
M
AB
|v
B
for every v V ,
telling us that
L
B
= M
BA
L
A
M
AB
.
This, along with the corresponding formula for computing L
A
from L
B
, is important enough
to be called a theorem (even though it is really just a corollary of Big Theorem 5.3).
Theorem 6.1 (Change of Orthonormal Bases for Linear Transforms)
Assume V is a vector space with orthonormal bases A and B, and let L be a linear operator
on V . Then L
A
and L
B
, the matrices of L with respect to basis A and basis B, respectively,
are related by
L
B
= M
BA
L
A
M
AB
and L
A
= M
AB
L
B
M
BA
(6.8)
where M
AB
and M
BA
are the change of bases matrices described in theorem 5.3.
The formulas
M
BA
L
A
M
AB
and M
AB
L
B
M
BA
1
For fun, you might try redoing this section assuming L is a linear transformfromone vector space V into another
(possibly different) vector space W. Its not that hard.
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 69
are said to dene similarity transforms of matrices L
A
and L
B
respectively. In short, a similarity
transform of a matrix is simply the computations for converting a matrix for some linear operator
with respect to one basis into the corresponding matrix with respect to another basis for the same
operator.
In common practice, the notation M
AB
and M
BA
is not particularly common. Instead, you
might have, say, the matrix we are calling M
BA
denoted by U. Keeping in mind M
AB
and
M
BA
are adjoints of each other, the above similarity transforms would then be written as
UL
A
U

and U

L
B
U ,
and the above theorem would be written as
Theorem 6.1

(Change of Orthonormal Bases for Linear Transforms Alt. Version)

Assume V is a vector space with orthonormal bases
A = { a
1
, a
2
, . . . , a
N
} and B = { b
1
, b
2
, . . . , b
N
} ,
and let L be a linear operator on V . Then L
A
and L
B
, the matrices of L with respect to basis
A and basis B, respectively, are related by
L
B
= UL
A
U

and L
A
= U

L
B
U
where U is the unitary matrix such that
|v
B
= U|v
A
for each v V .
Dont forget that all the above was based on our bases A and B being orthonormal. If
they are not orthonormal, then arguments very similar to that given for theorem 6.1 lead to the
following theorem. Since we plan to limit ourselves to orthonormal bases, we probably wont
need this theorem. It is included here just for completeness, and for those who may be interested.
Theorem 6.2 (Change of General Bases for Linear Transforms)
Assume V is a vector space with bases
A = { a
1
, a
2
, . . . , a
N
} and B = { b
1
, b
2
, . . . , b
N
} ,
and let L be a linear operator on V . Then L
A
and L
B
, the matrices of L with respect to basis
A and basis B, respectively, are related by
L
B
= ML
A
M
1
and L
A
= M
1
L
B
M
where M is the invertible matrix such that
|v
B
= M|b
A
for each v V .
?

Exercise 6.3 (very optional): Derive/prove the above theorem.

version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 610
6.5 Adjoints of Operators
Let L be a linear operator on a vector space V having an inner product | . We dene the
adjoint of L, denoted by L

, to be the linear operator on V satisfying

_
L

(v)

a
_
= v | L(a) for every a, v V
provided it exists. Actually, its not hard to prove it exists when V is nite-dimensional (well
do that in a minute). If V is not nite-dimensional, then L

may or may not exist.

It also is not hard to show that, if it exists, then L

is also a linear operator on V :

?

Exercise 6.4: Let L be a linear operator on a vector space V having inner product
| .
a: Let and denote two arbitrary scalars, and let v , w and c denote three arbitrary
vectors in V . Using the properties of inner products and denition of L

, show that
_
L

(v + w)

c
_
=
_
L

(v) + L

(w)

c
_
.
b: Why does the last exercise verify that L

is a linear operator?
Let us now assume V is nite dimensional and L is the matrix for L with respect to some
orthonormal basis
{ e
1
, e
2
, . . . , e
N
} .
Finding the matrix for L

turns out to be easy just use formula (6.7), the conjugate symmetry
of the inner product and the denition of the adjoint operator:
( j, k)
th
entry in the matrix for L

=
_
e
j

(e
k
)
_
=
_
L

(e
k
)

e
j
_

=
_
e
k

L(e
j
)
_

=
_
[L]
kj
_

= [L

]
j k
.
So the matrix for L

, the adjoint of operator L, is L

, the adjoint of matrix L. What a surprise!

(Side note: The existence of L

actually follows fromthe above computations for its matrix

and the fact that L

certainly exists.)
Unsurprisingly, we refer to a linear operator L as being self adjoint or Hermitian if L

exists and equals L. By the above, an operator (on a nite-dimensional vector space) is self
adjoint if and only if its matrix with respect to any orthonormal basis is self adjoint. We will be
discussing these operators more extensively in a week or so.
?

Exercise 6.5: Take a look at the linear operators dened in Homework Handout Vproblems
A1 (a magnication), A2 (a projection), A3 (a different projection), A5 (a 2-dimensional
rotation), B3 (another projection), B4 (a 3-dimensional rotation) and B8 (a cross product),
and do the following:
a: Find the matrix for the adjoint.
b: Identify those operators that are Hermitian.
c: Verbally describe what the adjoint operator does to inputted vectors.
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 611
Finally, let me throw in one more little exercise, the result of which may be useful later:
?

Exercise 6.6: Let L be a linear operator on V that has an adjoint. Show that
_
a

(v)
_
= L(a) | v for every a, v V .
Hermitian operators are important because they often describe linear processes in which
you expect certain symmetries. Later, we will develop the Sturm-Liouville theory and use it
to nd solutions to partial differential equations. This theory is really a theory of Hermitian
differential operators.
6.6 Inner-Product Preserving Operators and
Unitary/Orthogonal Matrices
Though our main interest with unitary matrices is howthey arise in the change of basis formulas,
their relation with the matrices for inner product preserving linear operators should be noted,
at least in exercise:
?

Exercise 6.7: Let L be a linear operator on a nite-dimensional vector space V with inner
product | , and let L be its matrix with respect to some orthonormal basis for V .
a: Assume L is a unitary matrix. Show that, if
x = L(a) and y = L(b) ,
then
x | y = a | b .
(In other words, L preserves the inner product.)
b: Now do exercise 3.4.26(a) on page 215 of Arfken & Weber, showing that, if L is unitary,
then L preserves the norm. (It may help to rewrite that problem in terms of the notation
we are now using. Dont forget the problem just done.)
c: Now assume that L preserves the inner product. That is,
x | y = a | b whenever x = L(a) and y = L(b) .
Show that L is unitary.
2
2
You can even show that L is unitary if you simply know that it preserves the norm. Thats a bit harder to verify.
version: 11/28/2011
Elementary Linear Transform Theory Chapter & Page: 612
6.7 On Confusing Matrices and Linear Operators A
Brief Rant
You are likely to see many cases where a matrix for an operator and the operator are treated as
the same thing. This is unfortunate and leads to too much confusion, silly computations, and a
lack of understanding as to what the hell is going on. It complicates matters and leads to the idea
that much of mathematical physics is just the blind manipulation of symbols according to arcane
laws. Avoid doing this yourself where possible.
The matrix of an operator depends on the basis. The operator does not. The matrix describes
the operator in terms of some particular basis. If there is only one basis, and everyone knows what
it is, then there is no great harmin confusing the operator and its matrix with respect to that basis.
If no basis is given, however, then the description of the operator as a matrix is vacuous (typically,
though, someone is assuming a standard basis without telling you). Worse yet are the cases
where more than one basis is oating around especially if no one tells you that multiple bases
are involved. For example, you may recall diagonalizing matrices. Did you realize that half
the issue was the nding of a basis for which the matrix of the underlying operator is particularly
simple? Or did you think it was just an exercise to pass the time?
Yes, sometimes everyone knows what everyone really means, and we can use verbal and
written shortcuts. The problem is, of course, that, more often than not, not everyone knows what
everyone else really means.
version: 11/28/2011