221A Lecture Notes

Notes on Tensor Product

1

What is Tensor?

After discussing the tensor product in the class, I received many questions
what it means. Ive also talked to Daniel, and he felt this is a subject he
had learned on the way here and there, never in a course or a book. I myself
dont remember where and when I learned about it. It may be worth solving
this problem once and for all.
Apparently, Tensor is a manufacturer of fancy floor lamps. See
http://growinglifestyle.shop.com/cc.class/cc?pcd=7878065&ccsyn=13549.

For us, the word tensor refers to objects that have multiple indices. In
comparison, a scalar does not have an index, and a vector one index. It
appears in many different contexts, but this point is always the same.

Direct Sum

Before getting into the subject of tensor product, let me first discuss direct
sum. This is a way of getting a new big vector space from two (or more)
smaller vector spaces in the simplest way one can imagine: you just line them
up.

2.1

Space

You start with two vector spaces, V that is n-dimensional, and W that is
m-dimensional. The tensor product of these two vector spaces is n + mdimensional. Here is how it works.
Let {~e1 , ~e2 , , ~en } be the basis system of V , and similarly {f~1 , f~2 , , f~m }
that of W . We now define n + m basis vectors {~e1 , , ~en , f~1 , , f~m }. The
vector space spanned by these basis vectors is V W , and is called the direct
sum of V and W .

2.2

Vectors

Any vector ~v V can be written as a linear combination of the basis vectors,

~v = v1~e1 + v2~e2 + + vn~en . We normally express this fact in the form of a
column vector,

v1

v2

~v = ..
n.
(1)

.
vn

Here, it is emphasized that this is an element of the vector space V , and the
column vector has n components. In particular, the basis vectors themselves
can be written as,

~e1 =

1
0
..
.

~e2 =

0
1
..
.

~en =

0
0
..
.
1

(2)

If the basis system is orthonormal, i.e., ~ei ~ej = ij , then vi = ~ei ~v . Similarly
for any vector w, we can write

w
~=

w1
w2
..
.
wn

m.

(3)

The basis vectors are

f~1 =

1
0
..
.
0

f~2 =

0
1
..
.
0

f~m =

0
0
..
.
1

(4)

The vectors are naturally elements of the direct sum, just by filling zeros

to the unused entries,

~v =

vn
0
..
.
0

v1

..

..

n + m,

0
w
~=
w

1
.
.
.

wm

n + m.

(5)

One can also define a direct sum of two non-zero vectors,

v1
.
..

vn
~v w
~ =
w

1
.
.
.

wm

~v
w
~

!)

n + m,

(6)

where the last expression is used to save space and it is understood that ~v
and w
~ are each column vector.

2.3

Matrices

A matrix is mathematically a linear map from a vector space to another

vector space. Here, we specialize to the maps from a vector space to the
same one because of our interest in applications to quantum mechanics, A :
V V , e.g.,
~v 7 A~v ,
(7)
or

v1
v2
..
.
vn

a11 a12
a22 a22
..
..
.
.
an1 an2

a1n
a2n

. . . ..

.
ann V

v1
v2
..
.
vn

(8)

Similarly, B : W W , and B : w
~ 7 B w.
~
On the direct sum space, the same matrices can still act on the vectors,
so that ~v 7 A~v , and w
~ 7 B w.
~ This matrix is written as A B. This can
3

be achieved by lining up all matrix elements in a block-diagonal form,

A

AB =

0nm
B

0mn

(9)

For instance, if n = 2 and m = 3, and

A=

a11 a12
a21 a22

b11 b12 b13

B = b21 b22 b23 ,

b31 b32 b33 W

,
V

where it is emphasized that A and

sum is

a11

a21

AB =
0

0
0

(10)

B act on different spaces. Their direct

0
0
a12 0
a22 0
0
0
0 b11 b12 b13
0 b21 b22 b23
0 b31 b32 b33

(11)

The reason why the block-diagonal form is appropriate is clear once you
act it on ~v w,
~
(A B)(~v w)
~ =

A
0mn

0nm
B

~v
w
~

A~v
Bw
~

= (A~v ) (B w).
~ (12)

If you have two matrices, their multiplications are done on each vector
space separately,
(A1 B1 )(A2 B2 ) = (A1 A2 ) (B1 B2 ).

(13)

Note that not every matrix on V W can be written as a direct sum of a

matrix on V and another on W . There are (n + m)2 independent matrices on
V W , while there are only n2 and m2 matrices on V and W , respectively.
Remaining (n + m)2 n2 m2 = 2nm matrices cannot be written as a direct
sum.
Other useful formulae are
det(A B) = (detA)(detB),
Tr(A B) = (TrA) + (TrB).

(14)
(15)

Tensor Product

The word tensor product refers to another way of constructing a big vector
space out of two (or more) smaller vector spaces. You can see that the spirit
of the word tensor is there. It is also called Kronecker product or direct
product.

3.1

Space

You start with two vector spaces, V that is n-dimensional, and W that
is m-dimensional. The tensor product of these two vector spaces is nmdimensional. Here is how it works.
Let {~e1 , ~e2 , , ~en } be the basis system of V , and similarly {f~1 , f~2 , , f~m }
that of W . We now define nm basis vectors ~ei f~j , where i = 1, , n and
j = 1, , m. Never mind what means at this moment. We will be more
explicit later. Just regard it a symbol that defines a new set of basis vectors.
This is why the word tensor is used for this: the basis vectors have two
indices.

3.2

Vectors

We use the same notation for the column vectors as in Section 2.2.
The tensor product is bilinear , namely linear in V and also linear in W .
(If there are more than two vector spaces, it is multilinear .) What it implies
P
P
Pn Pm
~
is that ~v w
~ = ( ni vi~ei ) ( m
ei f~j ). In other
j wj fj ) =
i
j vi wj (~
words, it is a vector in V W , spanned by the basis vectors ~ei f~j , and the
coefficients are given by vi wj for each basis vector.
One way to make it very explicit is to literally write it out. For example,
let us take n = 2 and m = 3. The tensor product space is nm = 6 dimensional. The new basis vectors are ~e1 f~1 , ~e1 f~2 , ~e1 f~3 , ~e2 f~1 , ~e2 f~2 ,
and ~e2 f~3 . Let us write them as a six-component column vector,

~
~e1 f1 =

1
0
0
0
0
0

~
~e1 f2 =

0
1
0
0
0
0

~
~e1 f3 =

0
0
1
0
0
0

~
~e2 f1 =

0
0
0
1
0
0

~
~e2 f2 =

0
0
0
0
1
0

~
~e2 f3 =

0
0
0
0
0
1

(16)

Here, I drew a horizontal lines to separate two sets of basis vectors, the first
set that is made of ~e1 , and the second made of ~e2 . For general vectors ~v and
w,
~ the tensor product is

~v w
~ =

3.3

v1 w1
v1 w2
v1 w3
v2 w1
v2 w2
v2 w3

(17)

Matrices

A matrix is mathematically a linear map from a vector space to another

vector space. Here, we specialize to the maps from a vector space to the
same one because of our interest in applications to quantum mechanics, A :
V V , e.g.,
~v 7 A~v ,
(18)
or

v1
v2
..
.
vn

a11 a12
a22 a22
..
..
.
.
an1 an2

a1n

a2n

. . . ..

.
ann

v1
v2
..
.
vn

(19)

On the tensor product space, the same matrix can still act on the vectors,
so that ~v 7 A~v , but w
~ 7 w
~ untouched. This matrix is written as A I,
where I is the identity matrix. In the previous example of n = 2 and m = 3,

the matrix A is two-by-two, while A I is six-by-six,

AI =

a11 0
0 a12 0
0
0 a12 0
0 a11 0
0
0 a11 0
0 a12
a21 0
0 a22 0
0
0 a21 0
0 a22 0
0
0 a21 0
0 a22

(20)

The reason why this expression is appropriate is clear once you act it on
~v w,
~

(A I)(~v w)
~ =

a11 0
0 a12 0
0
0 a12 0
0 a11 0
0
0 a11 0
0 a12
a21 0
0 a22 0
0
0 a21 0
0 a22 0
0 a22
0
0 a21 0
(a11 v1 + a12 v2 )w1
(a11 v1 + a12 v2 )w2
(a11 v1 + a12 v2 )w3
(a21 v1 + a22 v2 )w1
(a21 v1 + a22 v2 )w2
(a21 v1 + a22 v2 )w3

v1 w1
v1 w2
v1 w3
v2 w1
v2 w2
v2 w3

= (A~
v ) w.
~

(21)

Clearly the matrix A acts only on ~v V and leaves w

~ W untouched.
Similarly, the matrix B : W W maps w
~ 7 B w.
~ It can also act on
V W as I B, where

I B =

b11 b12 b13 0

0
0
b21 b22 b23 0
0
0
b31 b32 b33 0
0
0
0
0
0 b11 b12 b13
0
0
0 b21 b22 b23
0
0
0 b31 b32 b33

(22)

It acts on ~v w
~ as

(I B)(~v w)
~ =

b11 b12 b13 0

0
0
b21 b22 b23 0
0
0
b31 b32 b33 0
0
0
0
0
0 b11 b12 b13
0
0
0 b21 b22 b23
0
0
0 b31 b32 b33
v1 (b11 w1 + b12 w2 + b13 w3 )
v1 (b21 w1 + b22 w2 + b23 w3 )
v1 (b31 w1 + b32 w2 + b33 w3 )
v2 (b11 w1 + b12 w2 + b13 w3 )
v2 (b21 w1 + b22 w2 + b23 w3 )
v2 (b31 w1 + b32 w2 + b33 w3 )

v1 w1
v1 w2
v1 w3
v2 w1
v2 w2
v2 w3

=~
v (B w).
~ (23)

In general, (A I)(~v w)
~ = (A~v ) w,
~ and (I B)(~v w)
~ = ~v (B w).
~
If you have two matrices, their multiplications are done on each vector
space separately,
(A1 I)(A2 I) = (A1 A2 ) I,
(I B1 )(I B2 ) = I (B1 B2 ),
(A I)(I B) = (I B)(A I) = (A B).

(24)

The last expression allows us to write out A B explicitly,

AB =

a11 b11
a11 b21
a11 b31
a21 b11
a21 b21
a21 b31

a11 b12
a11 b22
a11 b32
a21 b12
a21 b22
a21 b32

a11 b13
a11 b23
a11 b33
a21 b13
a21 b23
a21 b33

a12 b11
a12 b21
a12 b31
a22 b11
a22 b21
a22 b31

a12 b12
a12 b22
a12 b32
a22 b12
a22 b22
a22 b32

a12 b13
a12 b23
a12 b33
a22 b13
a22 b23
a22 b33

(25)

It is easy to verify that (A B)(~v w)

~ = (A~v ) (B w).
~
Note that not every matrix on V W can be written as a tensor product
of a matrix on V and another on W .
Other useful formulae are
det(A B) = (detA)m (detB)n ,
Tr(A B) = (TrA)(TrB).
8

(26)
(27)

In Mathematica, you can define the tensor product of two square matrices
by
Needs["LinearAlgebraMatrixManipulation"];
KroneckerProduct[a_?SquareMatrixQ, b_?SquareMatrixQ] :=
BlockMatrix[Outer[Times, a, b]]
and then
KroneckerProduct[A,B]

3.4

Tensor Product in Quantum Mechanics

In quantum mechanics, we associate a Hilbert space for each dynamical degree of freedom. For example, a free particle in three dimensions has three
dynamical degrees of freedom, px , py , and pz . Note that we can specify only
px or x, but not both, and hence each dimension gives only one degree of
freedom, unlike in classical mechanics where you have two. Therefore, momentum eigenkets Px |px i = px |px i from a basis system for the Hilbert space
Hx , etc.
The eigenstate of the full Hamiltonian is obtained by the tensor product
of momentum eigenstates in each direction,
|px , py , pz i = |px i |py i |pz i.

(28)

We normally dont bother writing it this way, but it is nothing but a tensor
product.
The operator Px is then Px I I, and acts as
(Px I I)(|px i|py i|pz i) = (Px |px i)(I|py i)(I|pz i) = px |px i|py i|pz i
(29)
and similarly for Py and Pz .
The Hamiltonian is actually
H=


1 
(Px I I)2 + (I Py I)2 + (I I Pz )2 .
2m

(30)

Clearly we dont want to write the tensor products explicitly each time!

3.5

Addition of Angular Momenta

Using the example of n = 2 and m = 3, one can discuss the addition of

s = 1/2 and l = 1. V is two-dimensional, representing the spin 1/2, while
W is three-dimensional, representing the orbital angular momentum one. As
usual, the basis vectors are
1 1
| , i=
2 2

1
0

1 1
| , i =
2 2

0
1

(31)

for s = 1/2, and

|1, 1i = 0 ,
0

|1, 0i = 1 ,
0

|1, 1i = 0 ,
1

(32)

for l = 1. The matrix representations of the angular momentum operators

are
S+ = h

while

L+ = h
0
0

0 1
0 0

S = h

2 0

0
2 ,
0
0

0 0
1 0

Sz =
2

0
0 0

L = h
2 0 0 ,
0
2 0

1 0
0 1

(33)

1 0 0

Lz = h
0 0 0 .
0 0 1
(34)

On the tensor product space V W , they become

2 0 0 0
0
0
0 0 0 1 0 0

0 0 0 0 1 0
0 0
2 0 0
0

0 0
0 0 0 0 0 1
0 0 0
0

S+ I = h

, I L+ = h

0 0
0
0
2
0
0 0 0 0 0 0

0 0 0 0 0 0
0 0 0
2
0 0

0 0 0 0 0 0
0 0
0 0 0
0

S I = h

0
0
0
1
0
0

0
0
0
0
1
0

0
0
0
0
0
1

0
0
0
0
0
0

0
0
0
0
0
0

0
0
0
0
0
0

, I L = h

10

0
0 0 0
0
2 0 0 0
0
0
2 0 0
0
0
0 0 0
0
0
0 0
2 0
0
0 0 0
2

0
0
0
0
0
0

Sz I =
2

1
0
0
0
0
0

0
1
0
0
0
0

0 0
0
0
0 0
0
0
0
0
1 0
0 1 0
0
0 0 1 0
0 0
0 1

1
0
0
0
0
0

, I Lz = h

0 0 0 0 0
0 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 1

(35)
The addition of angular momenta is done simply by adding these matrices,
J,z = S,z I + I L,z ,

0
2 0 1 0
0

0 0
2 0 1
0

0 0
0 0 0
1

J+ = h

,
0 0

0
0
2
0

2
0 0 0
0 0

0 0
0 0 0
0

0 0 0
0
0
2 0 0 0
0
2 0 0
0
0
1
0 0 0
0
0
1 0
2 0
2
0
0 1 0

0
0
0
0
0
0

3
2

Jz = h

0
0
0
0
0

=h

0
1
2

0
0
0
0

0
0
21
0
0
0

0
0
0

0
0
0
0
0
0
1
0
0
2
0 12 0
0 0 23

(36)

It is clear that

3 3
| , i=

2 2

1
0
0
0
0
0

3 3
| , i =

2 2

11

0
0
0
0
0
1

(37)

They satisfy
J | 3 , 3 i = J | 23 , 32 i = 0. By using the general formula
q + 2 2
J |j, mi = j(j + 1) m(m 1)|j, m 1i, we find

3 3
J | , i =

2 2

0
2
0
1
0
0

3 3
J+ | , i =

2 2

3 1

= 3| , i,

2 2

0
0
1
0
2
0

3 1

= 3| , i.

2 2

(38)
We normally write them without using column vectors,
3 3
1 1
J | , i = J | , i |1, 1i
2 2
2 2
1 1
3 1
1 1
= | , i |1, 1i + 2| , i |1, 0i = 3| , i,
2 2
2 2
2 2
3 3
1 1
J+ | , i = J+ | , i |1, 1i
2 2
2 2
1 1
3 1
1 1
= | , i |1, 1i + 2| , i |1, 0i = 3| , i.
2 2
2 2
2 2

(39)

(40)

Their orthogonal linear combinations belong to the j = 1/2 representation,

1
1 1
| , i=

2 2
3

0
1
0

2
0
0

1
1 1
| , i =

2 2
3

0
0
2
0
1
0

(41)

It is easy to verify J+ | 12 , 12 i = J | 21 , 12 i = 0, and J | 12 , 21 i = | 12 , 21 i.

Now we can go to a new basis system by a unitary rotation

1 q0

2
0

U =
0

0
0
1
3

0
0

1
3

1
3

0
0
q

2
3

12

0
0
q
23
0

0
0
q

2
3

0
0

0
1
0

13 0

(42)

This matrix puts transpose of vectors in successive rows so that the vectors
of definite j and m are mapped to simply each component of the vector,

3 3
U| , i =

2 2

1 1
U| , i =

2 2

1
0
0
0
0
0

0
0
0
0
1
0

3
1
, U| , i =

2 2

3 3
, U| , i =

2 2

0
1
0
0
0
0

0
0
0
0
0
1

3
1
, U| , i =

2 2

0
0
1
0
0
0

3
3
, U| , i =

2 2

U Jz U

U J+ U

3
2

= h

0
0
0
0
0

0
0
0
0
0
0

= h

U J U

= h

0
0
0
0
1
2 0
0 32
0
0
0
0

1
2

0
0
0
0

0
3
0
0
0
0

3
0
0
0
0
0

0
0
0
0

0
0
0
0
1
0
2
0 12

0 0 0 0
2 0 0 0
0
3 0 0
0 0 0 0
0 0 0 1
0 0 0 0

0 0 0 0
0 0 0 0
2 0 0 0
0
3 0 0
0 0 0 0
0 0 0 1

0
0
0
0
0
0

(44)

(45)

(46)

Here, the matrices are written in a block-diagonal form. What we see here
is that this space is actually a direct sum of two representations j = 3/2 and
13

(43)

For the generators, we find

0
0
0
1
0
0

j = 1/2. In other words,

V1/2 V1 = V3/2 V1/2 .

(47)

This is what we do with ClebschGordan coefficients. Namely that the

ClebschGordan coefficients define a unitary transformation that allow us
to decompose the tensor product space into a direct sum of irreducible representations of the angular momentum. As a mnemonic, we often write it
as
2 3 = 4 2.
(48)
Here, the numbers refer to the dimensionality of each representation 2j + 1.
If I replace by and by +, my eight-year old daughter can confirm that
this equation is true. The added meaning here is that the right-hand side
shows a specific way how the tensor product space decomposes to a direct
sum.

14