Academia.edu no longer supports Internet Explorer.
To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser.
Linear Algebra with Sage (BigBook, Free e-book, English Version)
Linear Algebra with Sage (BigBook, Free e-book, English Version)
This book, ‘Linear Algebra with Sage’, has two goals. The first goal is to explain Linear Algebra with the help of Sage. Sage is one of the most popular computer algebra system(CAS). Sage is a free and user-friendly software. Whenever the Sage codes are possible, we illustrate examples with Sage codes. The second goal is to make the book accessible to everyone in the world freely. Therefore, the pdf file of this book is free to use in class or in person.
Related Papers
Linear Algebra and its Applications
Review of Linear algebra and its application, volume 1: A first course and Linear algebra and its application, volume 2: More advanced by D.H. Griffel1992 •
Linear Algebra Fourth Edition by Seymour Lipschutz and Marc Lars Lipson
Linear Algebra Fourth Edition2008 •
The fourth edition of this book is a comprehensive guide for students in mathematics, engineering, and computer science. It includes numerous problems and examples directly connected to applications in these fields. The book is updated with new appendices on algebraic systems, polynomials, and matrix applications.
3rd-edition-linear-algebra-and-its-applications-solutions-manual-by-David-C-Lay
V.11
July 7, 2015
Linear Algebra
Translated by
Sang-Gu LEE with Jon-Lark KIM, In-Jae KIM, Namyong LEE,
Ajit KUMAR, Phong VU, Victoria LANG, Jae Hwa LEE
(Based on the book written by Sang-Gu Lee with Jae Hwa Lee, Kyung-Won Kim)
http://matrix.skku.ac.kr/2015-Album/BigBook-LinearAlgebra-2015.pdf
http://sage.skku.edu, http://www.sagemath.org
and http://matrix.skku.ac.kr/LA-Lab/
http://www.bigbook.or.kr/
- 1 -
Linear Algebra with
http://matrix.skku.ac.kr/LA-Sage/
Contents
Chapter 1. Vectors
1.1 Vectors in n-space
1.2 Inner product and Orthogonality
1.3 Vector equations of lines and planes
1.4 Excercise
Chapter 2. Linear system of equations
2.1 Linear system of equations
2.2 Gaussian elimination and Gauss-Jordan elimination
2.3 Exercise
Chapter 3. Matrix and Matrix Algebra
3.1 Matrix operation
3.2 Inverse matrix
3.3 Elementary matrix
3.4 Subsapce and linear independence
3.5 Solution set of a linear system and matrix
3.6 Special matrices
*3.7 LU-decomposition
3.8 Excercise
Chapter 4. Determinant
4.1 Definition and Properties of the Determinants
4.2 Cofactor Expansion and Applications of the Determinants
4.3 Cramer's Rule
*4.4 Application of Determinant
4.5 Eigenvalues and Eigenvectors
4.6 Excercise
Chapter 5. Matrix Model
5.1 Lights out Game
5.2 Power Method
5.3 Linear Model (Google)
Chapter 6. Linear Transformations
6.1 Matrix as a Function (Transformation)
- 2 -
6.2 Geometric Meaning of Linear Transformations
6.3 Kernel and Range
6.4 Composition of Linear Transformations and Invertibility
*6.5 Computer Graphics with Sage
6.6 Exercises
Chapter 7. Dimension and Subspaces
7.1 Properties of bases and dimensions
7.2 Basic spaces of matrix
7.3 Rank-Nullity theorem
7.4 Rank theorem
7.5 Projection theorem
*7.6 Least square solution
7.7 Gram-Schmidt orthonomalization process
7.8 QR-Decomposition; Householder transformations
7.9 Coordinate vectors
7.10 Exercises
Chapter 8. Diagonalization
8.1 Matrix Representation of LT
8.2 Similarity and Diagonalization
8.3 Diagonalization with orthogonal matrix, *Function of matrix
8.4 Quadratic forms
*8.5 Applications of Quadratic forms
8.6 SVD and generalized eigenvectors
8.7 Complex eigenvalues and eigenvectors
8.8 Hermitian, Unitary, Normal Matrices
*8.9 Linear system of differential equations
8.10 Exercises
Chapter 9. General Vector Spaces
9.1 Axioms of Vector Space
9.2 Inner product spaces; *Fourier Series
9.3 Isomorphism
9.4 Exercises
Chapter 10. Jordan Canonical Form
10.1 Finding the Jordan Canonical Form with a Dot Diagram
*10.2 Jordan Canonical Form and Generalized Eigenvectors
10.3 Jordan Canonical Form and CAS
10.4 Exercises
Appendix
- 3 -
Preface
This book, ‘Linear Algebra with Sage’, has two goals. The first
goal is to explain Linear Algebra with the help of Sage. Sage is one
of the most popular computer algebra system(CAS). Sage is a free
and user-friendly software. Whenever the Sage codes are possible,
we illustrate examples with Sage codes. The second goal is to make
the book accessible to everyone in the world freely. Therefore, the pdf file of this
book is free to use in class or in person. For commercial use, please contact us.
Linear
one
Algebra is regarded
of
the
most
as
important
mathematical subjects because it
is used not only in natural sciences and engineering applications but also in
humanities and social sciences. Nowadays, Linear Algebra is studied most actively
in the 21st century.
One of the roles of mathematics in society is to suggest a possible solution by
modeling a practical problem as a mathematical problem, by solving it with the
idea of a system of linear equations, and by interpreting the solution in the setting
of the original problem. The first computer is also based on the linear process.
The study and applications of Linear Algebra grew incredibly in the later part of
the 20th century.
It is interesting to note that Sylvester and Cayley, inventors of
matrices,
and
Babbage,
father
of
the
computer,
were
mathematicians in the 19th century from United Kingdom. Since
then, the study of matrix theory has progressed and contributed to
the development of physics by the appearance of infinite dimensions and tensors.
Matrix theory in the United States of America was neglected from
the European mathematical society before the Second World War.
After that, because the modern computers were built and the
numerical power of matrices became very useful, the matrix
theory was developed well in the United Sates in the 20th century. The United
States has grown as a unique super power in both theories and experiments of
sciences.
- 4 -
How to use Lab
https://www.youtube.com/watch?v=V0xJvW-YjWs
[CAS-Geogebra] http://www.geogebratube.org/student/b121550
- 5 -
[CAS-Sage] http://matrix.skku.ac.kr/knou-knowls/Sag-Ref.htm
- 6 -
1
Chapter
Vectors
1.1 Vectors in n-space
1.2 Inner product and Orthogonality
1.3 Vector equations of lines and planes
1.4 Excercise
Linear
algebra
is
the
branch
of
mathematics
concerning vectors and mappings. Linear algebra is
central
to
both
pure
and
applied
mathematics.
Combined with calculus, linear algebra facilitates the
solution of linear systems of differential equations.
Techniques
from
linear
algebra
analytic
geometry,
engineering,
sciences,
computer
science,
are
also
physics,
computer
used
in
natural
animation,
and
the
social
sciences
(particularly in economics). A geometric quantity described by a magnitude and a
direction is called a vector. In this chapter, we begin with studying basic
properties of vectors starting from 3-dimensional vectors and extending these
properties to -dimensional vectors. We will also discuss the notion of the dot
product (or inner product) of vectors and vector equations of lines and planes.
Introduction : http://youtu.be/Mxp1e2Zzg-A
- 7 -
1.1
*Vectors in -space
Reference video: http://youtu.be/aeLVQoPQMpE http://youtu.be/85kGK6bJLns
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-1-Sec-1-1.html
Among the physical quantities we use and encounter in everyday life,
scalar (e.g. length, area, mass, temperature, etc.) is a quantity that can
be completely described by a single real number. A vector (e.g. force,
velocity, change in position, etc.) is a geometric quantity described by a
magnitude and a direction.
Scalar: length, area, mass, temperature- a one-dimensional physical quantity,
i.e. one that can be described by a single real number.
Vector : velocity, change in position, force - a geometric quantity described by a
magnitude and a direction.
A vector can be sketched as a directed line segment; in 2-and 3-dimensional
space, vectors are often drawn as arrows.
(terminal point)
x
(initial point)
Figure 1
A vector with the same initial and terminal points with magnitude 0 is called the
zero (or null) vector. (Since its magnitude is 0, it does not have a specific
direction).
In physics, vectors provide a useful way to express velocity, acceleration, force,
and the laws of motion. A force vector can be broken down into mutually
perpendicular component vectors. An electric field can be visualized by field
vectors, which indicate both the magnitude and direction of the field at every
point in space. Vectors have a wide variety of applications in the social
sciences, such as population dynamics and economics.
- 8 -
Figure 2
From now on, unless noted otherwise, we will restrict scalars to real numbers that is, if is a scalar, ∈ℝ .
[Vector Addition and Scalar Multiplication]
Definition
For any two vectors x, y, and scalar , the sum of x and y, x y,
and the scalar multiple of x by , x, are defined as follows.
(1) The sum of x and y is found by placing x and y tail-to-tail to form
two
adjacent
sides
of
a
parallelogram.
The
diagonal
of
this
parallelogram is x y. This is called the Parallelogram Law. (See Figure
3.)
(2) The scalar multiple of x by a scalar , is a vector with magntitude
times the magnitude of x and with the same direction as x if
, and is opposite to x if . (See Figure 4.) If is , x is the
zero vector.
y
x
x y
x
Figure 3
x
x
x
x
Figure 4
In the real coordinate plane ℝ ∈ℝ , the initial and terminal
points of every vector determine its the magnitude and direction.
If vectors
have the same magnitude and direction, even if they are in different positions,
we regard these vectors as equivalent.
- 9 -
Definition
An ordered pair of real numbers is called a vector (in ℝ )
and can be written as
x or x .
Here, , are called the components of x.
Definition
Equivalence
Two vectors x y∈ℝ , x and y with , , then
we say that x and y are equivalent (or equal) and we write x y .
[Remark] The case when the initial point is not at the origin.
A directed line from the point to the point is a vector with
the following components:
vector
′ .
The initial point of the
is at the origin and the terminal point is .
′
Figure 5
Example
1
For , , , , , ∈ℝ ,
express the vectors
,
,
in component form.
Solution
,
,
- 10 -
and
are equivalent.
Sage sol.
Copy the following code into http://sage.skku.edu
or
o=vector([0, 0])
http://mathlab.knou.ac.kr:8080/ to practice.
#creates a vector, x=vector([component , component ])
p1=vector([0, -4])
p2=vector([-3, 1])
q=vector([2, 3])
q1=vector([2, -1])
q2=vector([-1, 4])
print "vector OQ=", q-o
# subtract
print "vector P1Q1=", q1-p1
# subtract
print "vector P2Q2=", q2-p2
# subtract
print "vector OQ = vector P1Q1= vector P2Q2"
vector OQ= (2, 3)
vector P1Q1= (2, 3)
vector P2Q2= (2, 3)
vector vector OQ = P1Q1= vector P2Q2
Definition
For any two vectors x , y in ℝ and scalar , the sum of
x and y, x y, and the scalar multiple of x by , x, are defined
component-wise as follows.
(i) x y
(ii) x
In ℝ , the zero vector is a vector where all its components are equal
to 0 (its initial point is taken to be the origin). Then, for an arbitrary
x in , it is clear that
x x, x x .
Here, taking x x, we call x the negative vector or additive
inverse of x.
- 11 -
■
[Remark]
Computer Simulations
[Scalar multiplication] http://matrix.skku.ac.kr/2012-album/2.html
[Vector addition] http://matrix.skku.ac.kr/2012-album/3.html
Example
2
For vectors x , y
in ℝ , find x y, x y, and x.
Ÿ
http://matrix.skku.ac.kr/RPG_English/1-VT-sum-multi.html
Sage sol.
Copy the following code into http://sage.skku.edu
or
x=vector([1, 2])
http://mathlab.knou.ac.kr:8080/ to practice.
#creates a vector, x=vector([component , component ])
y=vector([-2, 4])
print "x+y=", x+y
# adds vectors
print "x-y=", x-y
# subtract vectors
print "-2*x=", -2*x
# multiplies vectors by scalar, (you must
include '*' when multiplying)
x+y=(-1, 6)
x-y=(3, -2)
-2*x=(-2, -4)
■
- 12 -
In ℝ ∈ℝ , we define vectors as follows.
Definition
A 3-tuple of real numbers
is called a vector (in ℝ ) and
can be written as
x
.
×
Here, , , are called the components of x.
Definition
[Equivalence or Equality]
Two vectors x y∈ , x and y with , , ,
are said to be equivalent (or equal) and we write x y .
[Remark]
The case when the initial point is the origin.
A directed line from the pont to the point is a vector
with the following components:
′ .
- 13 -
Figure 6
Example
3
For , , , , ∈ℝ ,
express the vectors
,
,
in component form.
Solution
,
,
and
are equivalent.
Sage sol.
Copy the following code into http://sage.skku.edu
or
o=vector([0, 0, 0])
http://mathlab.knou.ac.kr:8080/ to practice.
#creates a vector
p1=vector([0, -4, 2])
p2=vector([-3, 1, 0])
q=vector([2, 3, 4])
q1=vector([2, -1, 6])
q2=vector([-1, 4, 4])
print "vector OQ=", q-o
# subtract
print "vector P1Q1=", q1-p1
# subtract
print "vector P2Q2=", q2-p2
# subtract
print "vector OQ = vector P1Q1= vector P2Q2"
vector OQ= (2, 3, 4)
vector P1Q1= (2, 3, 4)
vector P2Q2= (2, 3, 4)
vector OQ = vector P1Q1= vector P2Q2
- 14 -
■
Definition
For any two vectors
x , y
in ℝ and scalar , the sum of x and y, x y,
and the scalar
multiple of x by , x, are defined component-wise as follows:
(i) x y
(ii) x .
In ℝ , the zero vector is a vector where all its components are equal
to 0 (its initial point is taken to be the origin). Then, for an arbitrary
x in ℝ , it is clear that
x x, x x .
Here, taking x x, we call x, the negative vector of x.
The Euclidean spaces
Euclidean space ℝ
ℝ
and
ℝ
can be generalized to
-dimensional
as follows:
ℝ … ∈ ℝ …
ℝ is also called -dimensional space and elements of ℝ are called
-dimensional vectors. (We shall formally define vector space later.)
Definition
An ordered n-tuple of real numbers
…
is called a
-dimensional vector and can be written as
x …
⋮
×
Here, real numbers , , … , are called the components of x.
- 15 -
Definition
[Equivalence or Equality]
For vectors x y ∈ℝ ,
x
, y
⋮
⋮
if ( … ) then we say x and y are equivalent (or equal)
and we write x y .
Definition
For any two vectors
x , y
⋮
⋮
in ℝ and scalar , the sum of x and y, x y,
and the scalar
multiple of x by , x, are defined component-wise as follows:
(i) x y
⋮
(ii) x
.
⋮
In ℝ , the zero vector is a vector where all its components are equal
to 0 (its initial point is taken to be the origin). Then, for an arbitrary
x in ℝ , it is clear that
x x, x x .
Here, taking x x, we call x, the negative vector of x.
- 16 -
Example
4
Find x y, x y, x when x
and y
in ℝ .
Ÿ
http://matrix.skku.ac.kr/RPG_English/1-VT-sum-multi-3.html
Solution
x y ,
x y ,
x
Sage
□
Copy the following code into http://sage.skku.edu to practice.
x=vector([1, 2, -3, 4])
y=vector([-2, 4, 1, 0])
print "x+y=", x+y
# adds vectors
print "x-y=", x-y
# subtracts vectors
print "-2*x=", -2*x
x+y=(-1, 6, -2, 4)
x-y=(3, -2, -4, 4)
-2*x=(-2, -4, 6, -8)
Theorem
■
1.1.1
If x, y, z are vectors in ∈ ℝ and and are scalars, then
(1) x y y x
(2) x y z x y z
(3) x x x
(4) x x x x
(5) x y x y
(6) x x x
(7) x x
(8) x x
The proof of above theorem is simple and follows from properties of addition and
multiplication of real numbers.
- 17 -
Theorem
1.1.2
If x is a vector in ∈ ℝ and is a scalar, then
(1) x
(2)
(3) x x
Definition
For vectors v v … v in ℝ and scalars … ,
x v v ⋯ v
is called a linear combination of v v … v .
Example
5
Find x y z , when x
, y
and z
in ℝ .
Ÿ
http://matrix.skku.ac.kr/RPG_English/1-VT-sum-multi-3.html
Solution
x y z
Sage
Copy the following code into http://sage.skku.edu or
http://mathlab.knou.ac.kr:8080/ to practice.
x=vector([1, 2, -3, 4])
y=vector([-2, 4, 1, 0])
z=vector([5, -2, 3, -7])
print "2*x-3*y+z=", 2*x-3*y+z
# linear combination
2*x-3*y+z=(13, -10, -6, 1)
■
- 18 -
Example
6
The above
Example
5
can also be done in Sage as follows. First, we build
the relevant vectors and the command for a linear combination of many
vectors. Then, we can combine all into one line, as follows.
Sage
Copy the following code into http://sage.skku.edu or
http://mathlab.knou.ac.kr:8080/ to practice.
x=vector(QQ,
[1, 2, -3, 4])
y=vector(QQ,
[-2, 4, 1, 0])
z=vector(QQ,
[5, -2, 3, -7])
# computations with quotient numbers in Q
print "2*x-3*y+z=", 2*x-3*y+z
# linear combination
vectors = [x, y, z]
scalars = [2, -3, 1]
multiples = [scalars[i]*vectors[i] for i in range(3)]
print "a*x+b*y+c*z=", sum(multiples)
# linear combination
2*x-3*y+z = (13, -10, -6, 1)
a*x+b*y+c*z = (13, -10, -6, 1)
■
(Comment : We can create an applet to generate a random vectors and scalars
and find the linear combination, as well.)
Rob Beezer's Linear Combination Lab: http://linear.ups.edu/html/section-LC.html
<Sang-Seol LEE, Father of Korean Mathematics education>
http://www.youtube.com/watch?feature=player_embedded&v=NbuRcvLlJOw
- 19 -
Inner product and Orthogonality
1.2
Reference videos: http://youtu.be/g55dfkmlTHE , http://youtu.be/CbfJYPCkbm8
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-1-Sec-1-1.html
In this section, we will discuss the concepts of vector length, distance,
and how to calculate the angle between two vectors, as well as vector
parallelism and orthogonality in ℝ .
Definition
Given a vector x … in ℝ
∥x∥
⋯
is called the norm (or length or magnitude) of x, and is denoted by
the symbol x or x (read as norm x ).
In the above definition, ∥x∥ is the distance from the initial point of
the vector x to its terminal point; equivalently, it is the distance from
the origin to the point … . Therefore, for any two vectors
x … , y … in , ∥x y∥ is the distance
between the two points … and … . That is,
⋯ .
∥x y∥
Example
1
For the vectors
x , y in , we have the
following.
Ÿ
http://matrix.skku.ac.kr/RPG_English/1-B1-norm-distance.html
Solution
∥x∥
∥y∥
- 20 -
∥x y∥
.
Sage
□
Copy the following code into http://sage.skku.edu to practice.
x=vector([2, -1, 3, 2])
y=vector([3, 2, 1, -4])
print x.norm()
# calculate the norm of x
print y.norm()
# calculate the norm of y
print (x-y).norm()
# calculate distance
# sqrt(2) means
3*sqrt(2)
sqrt(30)
5*sqrt(2)
■
Definition
For vectors
x … , y … in ℝ ,
⋯
is called the dot product (or Euclidean inner product) of x and y
and is denoted by x ∙ y. That is,
x ∙ y ⋯
Note that x ∙ x ∥x∥
- 21 -
Example
2
Using the vectors x and y in
Example
1 , calculate x
∙ y.
Solution
x ∙ y · · · · .
Sage
□
Copy the following code into http://sage.skku.edu to practice.
x=vector([2, -1, 3, 2])
y=vector([3, 2, 1, -4])
print x.inner_product(y)
# find the dot product
-1
■
Theorem 1.2.1
If x , y, z are vectors in ℝ and is a scalar, then we have the
following:
(1) x ∙ x ≥ ,
(2) x ∙ x
⇔
x
(3) x ∙ y y ∙ x
(4) x ∙ y x ∙ z y ∙ z
(5) x ∙ y x ∙ y x ∙ y
The proof of all the facts in above theorem are easy and users are encouraged
to complete the same.
Theorem 1.2.2 [The Cauchy–Schwarz inequality]
For any two vectors x , y in ℝ ,
x ∙ y ≤ ∥x∥∥y∥ .
Equality holds if and only if x
and y are scalar multiples of one
another (i.e. x y for some scalar ).
- 22 -
The Cauchy-Schwarz inequality is one of the most important inequalities in
vector spaces. We will give a full details of this proof in section 9.2. This
inequality implies
x ∙ y
x ∙ y
≤ and ≤ ≤ and which gives
∥x∥∥y∥
∥x∥∥y∥
x ∙ y
cos where cos ∈ . This is a more generalized concept of
∥x∥∥y∥
the angle between two vectors, since these vectors can be matrices, polynomials,
functions, etc.
Definition
For vectors
x … , y … in ℝ
x ∙ y ∥x∥∥y∥cos , ≤ ≤ ,
where is called the angle between x and y.
[Remark] Parallelism and Orthogonality
If x ∙ y , then x is orthogonal to y.
If x is a scalar multiple of y (i.e.,
x y for some scalar ), then x is parallel
to y.
Definition
A vector u in with a norm of 1, that is,
∥u∥
is called a unit vector. Additionally, if
x
and y are mutually
orthogonal unit vectors, x and y are called orthonormal vectors.
- 23 -
Figure 7
x
If x is a non zero vector, then u , hence we have ∥u∥
∥x∥
x
∥
∥x∥ ∥
∥x∥
.
∥x∥
Example
3
For
two
vectors
x
and
y
in
ℝ ,
establish
orthogonality.
Ÿ
http://matrix.skku.ac.kr/RPG_English/1-TF-inner-product.html
Solution
x∙ y · · · ·
Sage
□
Copy the following code into http://sage.skku.edu to practice.
x=vector([1, 0, 1, 1])
y=vector([-1, 0, 0, 1])
print x.inner_product(y)
0
#orthogonal
■
- 24 -
Theorem 1.2.3 [Triangle Inequality for Vectors]
For any two vectors x , y in ℝ . we have
∥x y∥ ≤ ∥x∥ ∥y∥
Equality holds if and only if x and y are non-negative scalar multiples
of one another (i.e. x y for some scalar ≥ ).
Figure 8
Geometrically, the sum of any length of any two sides of a triangle is greater than
or equal to the third side. Look at the above figure.
Example
4
Using the vectors x
and y from
Example
1,
verify that the triangle
inequality holds.
Solution
x , y , ∥x∥
,
∥y∥
and
x y . Hence
∥x y∥
.
So, x y
x y .
- 25 -
■
Definition
For an arbitrary, non-zero vector x ≠ ∈ ℝ
u x
∥ x∥
is a unit vector. In
ℝ , unit vectors of the form
e … , e … , … , e …
are called standard unit vectors or coordinate vectors.
If x … is an arbitrary vector in ℝ , using standard unit vectors,
we can express x as follows:
x e e ⋯ e .
In ℝ and ℝ , conventionally, the unit vectors e , e , e along the rectangular
coordinate axes are represented by
i , j , k.
x i j ,
x ∈ ℝ
x i j k,
x ∈ ℝ
Figure 9
<Figure 9 comes from Contemporary Linear algebra (3rd Edition) by Sang-Gu Lee,
ISBN 978-89-6105-195-8, Kyungmoon Books(2009)>
- 26 -
1.3
Equations of Lines and Planes
Reference video: http://youtu.be/4UGACWyWOgA http://youtu.be/YB976T1w0kE
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-1-Sec-1-3.html
In this section, we will derive vector equations of lines and planes in
ℝ , and we will examine shortest distance problems related to these
equations.
Point-Slope (Direction Vector) Equation of a Line
In ℝ , an equation of a line can be uniquely determined when a slope and a
specified point on the line are given. If a line passes through the point
is
and is parallel to a vector v i j k, then the vector
parallel to v, where is any point on the line.
That is, the line is a set of all points that satisfies the following equation:
v ∈ℝ
Suppose
p and
p. Then
p p . Hence we have p p v.
That is, p p v.
Vector equations: p p v, (p
, p
)
Parametric equations: In terms of coordinates, the above equations can be
written as
- 27 -
, ,
( ∞ ∞ ).
Symmetric equations: From the above parametric equations, it is easy to see
that
Example
1
( ≠ ).
Find vector, parametric and symmetric equations of the line that passes
through
the
point
and
is
parallel
to
the
vector
v .
Solution
(1) The vector equation of the line is give by
i j k i j k i j k .
(2) The parametric equation is given by ( ∞ ∞ ).
(3) The symmetric equation is given by .
Example
2
■
Find parametric equations for the line that passes through the points
and .
Solution
Two points and with position vectors r
and r forms a vector
v r r
and the vector equation r r r r can be written
as
, ∈ℝ .
Thus, the parametric equations are:
, , ( ∞ ∞ )
- 28 -
■
Point-Normal Equation of Planes
A plane in ℝ can be uniquely obtained by specifying a point in the
plane and a nonzero vector n that is perpendicular to the plane. The
vector n is called the normal vector to the plane. If is any point in this
plane, then the
r r is orthogonal to n .
Hence
by the property of the dot (inner) product.
n∙
∙
From this, we have
Figure 10
where , and are not all zero.
This is called the point-normal equation of the plane through with
normal
n .
The
above
equation
can
be
simplified
to
.
Vector Equation of Planes
Vector equations: A plane in ℝ can be uniquely obtained by passing
through a point x and two nonzero vectors v and v
in ℝ that
are not scalar multiples of one another.
Let x be any point on , Then x x can be expressed as a
linear combinations of v and v . Look at the Figure 11.
x x v v
or
x x v v
( ∞ ∞ )
where and , called parameters, are in ℝ .
This is called a vector equation of the plane.
Parametric equations: Let
x
Figure 11
be any point in the plane through
x that is parallel to the vectors v and v .
Then, we can express this in component form as
- 29 -
or
∈ ℝ
These are called parametric equations of the plane.
Example
3
Find vector and parametric equations of the plane that passes through
the three points:
Ÿ
, , and .
http://matrix.skku.ac.kr/RPG_English/1-BN-11.html
Solution
Let x , x , x , and x .
Then we have two vectors that parallel to the plane as
x x
, x x
.
Then, from our above definitions, we have
x ,
which is a vector equation of the plane.
If we further simplify the above expression, we have
.
In particular, , , .
is the parametric equations of the plane.
- 30 -
■
[Remark] Computer Simulation (A plane containing three points)
Ÿ
http://matrix.skku.ac.kr/2012-LAwithSage/interact/1/vec8.html
Vector Projection and Components
Consider two vectors x and y with the same initial point , represented by
x
and y
. Let be the foot of the perpendicular from to the line
containing
. Then
is called the vector projection of y onto x and is
denoted by proj x y .
Here, the vector w
is called the component of y along x (or the scalar
projection of y onto x). Therefore, y can be written as
Note that p is parallel to x,
orthogonal to x. Hence
hence p x
for some scalar . Now y p is
x⋅ y p . This implies
the following results:
- 31 -
y p w.
x⋅ y x⋅ x .
This gives
[Remark]
Computer Simulation (Projection)
http://matrix.skku.ac.kr/2012-LAwithSage/interact/1/vec3.html
Ÿ
Theorem 1.3.1 [Projection]
For vectors x ≠ , y in ℝ , we have the following:
y ∙ x
(1) proj x y x x
x∙ x
y∙ x
(2) ∥ proj x y∥ .
∥x∥
Example
4
For vectors x , y , find proj x y (the vector projection
of y onto x) and the component of y along x.
Solution
Since y⋅ x , we have
y⋅ x
proj x y
x
x
w y proj x y
- 32 -
□
Sage
Copy the following code into http://sage.skku.edu to practice.
a=vector([2, -1, 3])
b=vector([4, -1, 2])
ab=a.inner_product(b)
aa=a.inner_product(a)
p=ab/aa*a;w=b-p
print "p=", p
print "w=", w
p= (15/7, -15/14, 45/14)
w= (13/7, 1/14, -17/14)
■
Theorem 1.3.2 [Distance Between a Point and a Plane]
For a point and a plane , the distance
from the point to the plane is given by
.
Figure 12
Note that the distance of the point from the orthogonal projection of the vector
v onto the plane . This distance is same as the
orthogonal projection of the vector onto the normal vector n
to the plane. See the Figure 12. It is as easy exercise to verify that the orthogonal
projection of v onto n is given by the formula above.
- 33 -
Example
5
Find
the
distance
from
the
point
to
the
plane
.
Ÿ
http://matrix.skku.ac.kr/RPG_English/1-B1-point-plane-distance.html
Solution
v⋅ n
p proj n v n n
n⋅ n
Here, n , v , and , so
p proj n v
Sage
□
Copy the following code into http://sage.skku.edu to practice.
n=vector([1, 3, -2])
v=vector([3, -1, 2]);d=-6
vn=v.inner_product(n)
nn=n.norm()
Dist=abs(vn+d)/nn
print Dist
5/7*sqrt(14)
#
■
[2014 Seoul International Congress of Mathematicians] http://www.icm2014.org/kr
- 34 -
Exercises
Chapter 1
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
Problem 2
For points , find the vector
.
What is the initial point of the vector x with terminal point
?
Problem 3
For vectors u , v , and
w , compute the following:
u w v u
Problem 4
Using the same u v w from above, find the vector x that
satisfies the following:
u v x x w
Problem 5
For vectors x y calculate cos , where is
the angle between x and y.
Problem 6
Find
the
distance
between
.
- 35 -
the
two
points
and
Problem 7
For vectors x , y , find the real number
that such that x⋅ y .
Find
Problem 8
a
vector
equation
of
the
line
between
the
two
points
and .
Problem 9
Find a normal vector perpendicular to the plane .
Problem 10
[Projection] For x and y , find the scalar projection and
vector projection of y onto x .
Solution
y⋅ x
proj x y x
x⋅ x
w y proj x y
Sage :
a=vector([2, -1, 3])
b=vector([4, -1, 2])
ab=a.inner_product(b)
aa=a.inner_product(a)
p=ab/aa*a;w=b-p
print "p=", p
print "w=", w
p= (15/7, -15/14, 45/14)
■
w= (13/7, 1/14, -17/14)
Problem P1
[Discussion] Vectors with the same magnitude and direction are
considered to be equivalent. However, in a vector space,
discuss the relationship
between vectors with the same slope but expressed with different equations.
Problem P2
[Discussion]
For vectors v
and v , check
if v and v are orthonormal vectors, and find a third vector v such that
v v v are all orthonormal to one another.
- 36 -
Solution
v
v ∙ v
Let
, v
v
such
v and v are orthonormal.
v
,
that
=>
v ∙ v .
This shows v
=>
v ∙ v ,
, ,
■
[Digital Library of Math Textbooks in 60’s at SKKU]
http://matrix.skku.ac.kr/2012-e-Books/index.htm
<1884~1910 Math books written by Korean authors>
http://www.hpm2012.org/Proceeding/Exhibition/E2.pdf
- 37 -
2
Chapter
Linear system of equations
2.1 Linear system of equations
2.2 Gaussian elimination and Gauss-Jordan elimination
2.3 Exercise
A
system
solution
is
of
linear
one
equations
of
the
most
and
its
important
problems in Linear Algebra. A linear system
with
thousands
of
variables
occurs
in
natural and social sciences, engineering, as
well as traffic problems, weather forecasting,
decision-making,
etc.
Even
differential
equations concerning derivatives such as
velocity and acceleration can be solved by
transforming them into a linear system.
In Linear Algebra, a solution of a linear
system
is
obtained
by
Gauss
elimination
method or with determinants. In Chapter 2,
we consider a geometric meaning of the
solution of a linear system and its solution,
and investigate some applications of a linear
system.
- 38 -
2.1
Linear system of equations
Reference video: http://youtu.be/CiLn1F2pmvY, http://youtu.be/AAUQvdjQ-qk
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-2-Sec-2-1.html
The theory of linear systems is the basis and a fundamental part of
linear algebra, a subject which is used in most parts of modern
mathematics. Computational algorithms for finding the solutions are an
important part of numerical linear algebra, and play a prominent role in
engineering, physics, chemistry, computer science, and economics. In
this section, we study the process of finding solutions of linear system
of equations and its geometric meanings.
Definition
[Linear equations]
Let and … be real numbers. A linear equation with unknowns
… is of the following form:
⋯
In other words, a linear equation consists of variables of degree 1 and a
constant.
Example
1
Equations
,
Definition
,
and
they
can
are
be
written
linear.
as
But
, sin are not linear.
[Linear system of equations]
In general, a set of linear equations with unknowns …
⋯
⋯
⋮
(1)
⋯
is called a system of linear equations. If constants … are all
zeroes, it is called a homogeneous system of linear equations.
- 39 -
Definition
[Solutions of a linear system]
Suppose that unknowns … in a linear system are substituted
by … respectively and each equation is satisfied. Then
… is called a solution of a linear system. For example, given a
linear system
(2)
One can substitute as , respectively, and it satisfies
equation (2). Hence is a solution. In general, if there is a
solution
of a
linear
system,
it
is called
consistent
and
is called
inconsistent otherwise.
The set of all solutions of a linear system is called a solution set. Two linear
systems with the same solution set are called equivalent.
[Remark]
Solution (linear system with two unknowns)
In general, a given linear system satisfies one and only of the following.
(1) a unique solution
(2) infinitely many solutions
(3) no solution
- 40 -
[Remark]
Computer simulation
[Linear system of equations] http://www.geogebratube.org/student/m9704
[Remark] Linear algebra with Geogebra:
http://www.geogebratube.org/student/b121550
Remark: (i) If there is one linear equation in three variables then it has infinitely
many solutions. (ii) If there are two linear equations in three variables then, it
either it has no solution (when the two planes are parallel) or it has infinitely
many solutions which is the points of line of intersection of the two planes. (iii) In
case of three linear equations in three variables, all possibilities can occur.
- 41 -
Example
2
Describe all possible solution sets in of a linear system with three
equations and three unknowns.
Solution
One can show that there are three possibilities by a geometric method.
Let us denote each equation by a plane respectively.
① It has a unique solution.
Three planes meets in a unique point.
[Ex]
② It has infinitely many solutions.
[Ex] (1)
(2)
(3)
③ It has no solution. (It is called 'inconsistent').
[Ex] (1)
(2)
(3)
(4)
■
- 42 -
Example
3
Solve the following linear system.
Solution
Since there are five unknowns and three equations, assign to the
any two unknowns arbitrary real numbers. Rearranging each equation,
we get
Substitute , ( are arbitrary real numbers) to get
Therefore, the solution of a given linear system is
( are arbitrary real numbers).
The solution set is
∈ .
Thus this system has infinitely many solutions.
- 43 -
■
Definition
[Matrix]
An array (or rectangle) consisting of real (or complex) numbers is called
a matrix, and each number is called an entry.
⋮
⋮
The row
⋯
⋯
⋯
⋯
⋮
⋯
⋯
⋯
(2)
⋯ ≤ ≤ of matrix is called the -th
row of , and the column
≤ ≤
⋮
of is called the -th column of . A matrix with rows and
columns is called a size × matrix, and if , it is called a square
matrix of order .
Let denote the th row of , and denote the the column of .
Therefore we can write as follows.
⋯
⋮
The entry of a matrix is also called the entry of , and the entries
… of a matrix of order are called main diagonal entries. Matrix (2)
can be written as the entries as follows.
× or
Example
4
Consider matrices
.
- 44 -
is a × matrix, and . is a × matrix, and
, and
are × × × ×
matrix
respectively. The main diagonal entries of are ,
and is also written as
Definition
[Coefficient
.
■
matrix and augmented matrix of a linear system]
For a linear system with unknowns and linear equations
⋯
⋯
(3)
⋮
⋯ ,
let
⋯
⋯
⋮ ⋮
⋯
⋮
x
⋮
b
⋮
then Equation (3) can be written as
x b.
The matrix is called the coefficient matrix of Equation (3) and the
matrix obtained from and b
⋯
⋯
⋮ b
⋮ ⋮ ⋱
⋯
⋮
⋮
⋮ ⋮ ⋮
⋮
is called the augmented matrix of Equation (3).
Example
5
Find the augmented matrix of the following linear system of equations.
- 45 -
Solution
Let be the coefficient matrix, x the unknown, and b the
constant, then
x
b
Hence we have
x b ⇔
Its augmented matrix is
⋮ b
Sage
⋮
⋮
⋮
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 3, [1,1,2,2,4,-3,3,6,-5])
# 3x3 matrix
b=vector([9,1,0])
# constant vector
print A.augment(b,subdivide=True)
# augmented matrix
[ 1
1
2 |
9 ]
[ 2
4 –3 |
1 ]
[ 3
6 –5 |
0 ]
■
- 46 -
2.2
Gaussian elimination and Gauss-Jordan elimination
Reference video: http://youtu.be/jnC66zvqHJI, http://youtu.be/HSm69YigRr4
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-2-Sec-2-2.html
Gaussian elimination (also known as row reduction) is an algorithm for
solving systems of linear equations. It is usually understood as a
sequence
of
operations
performed
on
the
associated
matrix
of
coefficients. Using row operations to convert a matrix into reduced row
echelon form is sometimes called Gauss–Jordan elimination. Linear
system of equations can be easily solved by using Gauss–Jordan
elimination.
Solving a linear system: using elimination method:
⇒
⇒
⇒
Subtracting the second equation from the first equation,
Dividing the second equation by 7,
Substituting in the first equation,
⇒
⇒ Multiplying 2 on the second equation,
The following operations do not change the solution set.
↔
(1) Exchange two equations.
(2) Multiply a row by a nonzero real number.
→
(3) Add a nonzero multiple of a row to another row.
These are called Elementary Row Operations (ERO).
Example
1
In the following procedure, the left side shows solving a linear system
directly, and the right side shows solving it using an augmented matrix.
⋮
⋮
⋮
Add the multiplication of the first equation by
equation.
- 47 -
to the second
⋮
⋮
⋮
Add the multiplication of the first equation by -3 to the third equation.
⋮
⋮
⋮
Multiply the second equation by to get
⋮
⋮
⋮
Add the multiplication of the second equation by to the third
equation.
⋮
⋮
⋮
Multiply the third equation by .
Thus the system reduces to
⋮
⋮
⋮
Now substituting in the second equation, we get . Substituting
and in the first equation, we get . Hence the solution is
■
- 48 -
Definition
[Row echelon form(REF) and reduced row echelon form(RREF)]
When an × matrix satisfies the following 3 properties, it is called
a row echelon form (REF).
(1) If there is a row consisting of only 0's, it is placed on the bottom
position.
(2) The first nonzero entry appearing in each row is 1. This 1 is called
a leading entry.
(3) If there is a leading entry in both the th row and the row,
the leading entry in the ( )th row is placed on the right of the
leading entry in the th row.
If matrix is a REF and satisfies the following property, is called
a reduced row echelon form (RREF).
(4)
If a column contains the leading entry of some row, then all the
other entries of that column are 0
Example
2
The following are all REF.
Example
3
Consider matrices
Since matrices do not satisfy the above properties (1), (2), (3)
respectively, they are not REF.
- 49 -
Example
4
The following are all RREF.
[Remark]
Below are a general form of a REF and its corresponding RREF(here * is any
number).
Definition
[Elementary Row Operation(ERO)]
Given an × matrix , the following operations are called elementary
row operation (ERO).
E1: Exchange the th row and the th row of .
↔
E2: Multiply the th row of by a nonzero constant .
E3: Add the multiplication of the th row of by to the th row.
EROs transform a given matrix into REF and RREF.
- 50 -
Definition
[Row Equivalent]
If is obtained from a matrix by elementary row operations, and
are row equivalent.
Example
5
The following are equivalent.
Finding REF and RREF
, find REF and RREF by applying ERO's.
For
(
Find a column whose
entries are not all zero and
which is located in
left-most position.
(In this case, it is the first column)
Swap the first row with some
other row below to guarantee
that is not zero.
Swap 1st and 2nd row
(In this case,
became ′ . This is call a pivot.)
Divide the 1st row by 2 to make the
pivot entry = 1.
Multiply
- 51 -
to the 1st row.
■
Eliminate all other entries in the 1st
column by subtracting suitable
multiples of the 1st row from the
other rows.
(Use elementary row operations).
Eliminate
in the 1st column by subtracting -2 multiple of the 1st row from the 3rd row.
Continue steps 1, 2, 3, 4 for the remaining rows except the 1st row .
Find a column whose entries are not all zero and
which is located in the left-most position (excluding the 1st column).
Since the leading entry is not 1, follow step 3.
Eliminate
″ in the 3rd column by subtracting -
5 multiple of the 2nd row from the 3rd row.
Continue steps 1, 2, 3 for the rows except the 1st and 2nd row .
Find a column whose entries are not all zero (excludin
g the 1st and 2nd rows).
- 52 -
Since there is a row whose entries are not all zero, f
ollow step 3.
Therefore we have REF of as follows.
Furthermore, we get the RREF of from the above REF by making nonzer
o ″to be by suitable multiples of each row.
Add the
7
2
multiple of 3rd row to 2nd row.
Add the -6 multiple of 3rd row to 1st row.
Add the 5 multiple of 3rd row to 1st row.
Now we have the RREF of .
□
http://www.math.odu.edu/~bogacki/cgi-bin/lat.cgi?c=rref
- 53 -
Example
6
Find the RREF of .
Solution
http://matrix.skku.ac.kr/RPG_English/2-MA-RREF.html
http://matrix.skku.ac.kr/2014-Album/MC.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 5, [1,1,1,4,4,2,3,4,9,16,-2,0,3,-7,11])
# 3x5 matrix input
print A.rref()
# A's RREF
[ 1
0
0
2 –1]
[ 0
1
0
3
2]
[ 0
0
1 –1
3]
Theorem
■
2.2.1
Two linear systems whose augmented matrices are row equivalent are
equivalent (that is, they have the same solution sets.)
Gauss elimination: This is a method to transform the augmented matrix of a
linear system into REF.
Example
7
Solve the following by the Gauss elimination.
Solution
Its augmented matrix is
⋮
⋮
⋮
and its REF by EROs is
. Therefore, since the corresponding linear system of the
above augmented matrix is
- 54 -
,i.e,
■
The solution is .
Gauss-Jordan elimination: This is a method to transform the augmented matrix
of a linear system into RREF.
Example
8
Solve the following system using the Gauss-Jordan elimination.
Solution
Ÿ
We will use Sage to solve this.
□
http://matrix.skku.ac.kr/RPG_English/2-VT-Gauss-Jordan.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix([[1,3,-2,0,2,0],[2,6,-5,-2,4,-3],[0,0,5,10,0,15], [2,6,0,8,4,18]])
b=vector([0,-1,5,6])
print A.augment(b).rref()
[
1
3
0
4
2
0
0]
[
0
0
1
2
0
0
0]
[
0
0
0
0
0
1 1/3]
[
0
0
0
0
0
0
0]
Its corresponding linear system is
By letting ( are any real), its solution is
- 55 -
From
Example
8,
.
■
we can express a general solution as a vector form.
∈
.
[Remark]
Gauss elimination and Gauss-Jordan elimination
- 56 -
[Remark]
Leading Variable, Free Variable and their Relation to RREF
a free variable: a variable corresponding to the column not containing a
l
leading entry in RREF
a leading (pivot) variable: a variable corresponding to the column containing
l
a leading entry in RREF
Homogeneous linear system
⋯
⋯
(II)
⋮
⋯
It is easy to see that x … is always a solution of a homogeneous
system (II). x is called a trivial solution. Also if x is a solution of (II) then any
scalar multiple x is also a solution of (II). Similarly if x and y are two
solutions of a homogeneous system, so is their sum.
This shows that any
homogeneous system has either a trivial solution or infinitely many solutions.
Example
9
Using the Gauss-Jordan elimination, express the solution of the following
homogeneous equation as a vector form.
- 57 -
Its augmented matrix is
Solution
⋮
⋮
⋮
⋮
.
Thus,
leading
entry
1's
⋮
⋮
⋮
⋮
, and its RREF is
correspond
to
leading
variables , , and the rest variables , , to free variables.
We have the following.
, ,
Now let free variables be , , , then
, , , , , .
.
Therefore
Theorem
2.2.2
[No. of free variables in a homogeneous linear system]
In a homogeneous linear system with unknowns, if the RREF of the
augmented matrix has leading 1's, the solution set has free
variables.
Theorem
2.2.3
The system
■
for ≤ ≤ always has a non-trivial solution if
.
The theorem can be proved using induction on the number on variables.
- 58 -
[Remark]
Computer simulation
[Elementary row operation]
http://www.geogebratube.org/student/b73259#material/28831
[Linear algebra with Sage, Smartphone App]
https://play.google.com/store/apps/details?id=la.sage
- 59 -
Exercises
Chapter 2
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
Answer the questions for the following linear system.
(1) Find the coefficient matrix.
(2) Express the linear system in the form x b.
(3) Find its augmented matrix.
Problem 2
Find a linear system with its augmented matrix.
(Put the unknowns as ⋯ .)
⋮
⋮
⋮
Problem 3
Find the number of leading variables and free variables in the solution
set of the following system.
- 60 -
Problem 4
Which matrices are REF or RREF? If one is not RREF, transform it to
RREF.
, .
Problem 5
Solve the system using Gauss elimination.
.
Problem 6
Solve the system using Gauss-Jordan elimination.
.
Problem 7
Solution
In the following circuit, write a linear system to find current .
Let
, , , .
- 61 -
62
Then x b where
, x
and b
.
■
Problem P1
In general, we are given a linear system with equations and
unknowns.
⋯
⋯
⋮
⋯
If there are free variables, what is the number of leading variables?
From this, think about the relation among the numbers of free variables, leading
variables, and unknowns.
Problem p2
Write a linear system with 4 unknowns and 3 equations whose solution
set is given below.
(here , are any real)
Solution
The linear system
Problem p3
Suppose that three points
is an example.
■
pass through the
parabola . By plugging in these points, obtain three
linear equations. Find coefficients by solving x b.
Problem p4
Write a linear system with four unknowns and four equations satisfying
each condition below.
(a) A solution set with one unknown.
(b) A solution set with two unknowns.
- 62 -
Chapter
3
Matrix and Matrix Algebra
3.1 Matrix operation
3.2 Inverse matrix
3.3 Elementary matrix
3.4 Subsapce and linear independence
3.5 Solution set of a linear system and matrix
3.6 Special matrices
*3.7 LU-decomposition
Matrix is widely used as a tool to transmit digital sounds and images through
internet as well as solving linear systems. We define the addition and product of
two matrices. These operations are tools to solve various linear systems. Matrix
product also becomes an excellent tool in dealing with function composition.
In the previous chapter, we have found the solution set using the Gauss
elimination
method.
In
this
chapter,
we
define
the
addition
and
scalar
multiplication of matrices and introduce algebraic properties of matrix operations.
Then using the Gauss elimination, we show how to find the inverse matrix.
Furthermore, we investigate the concepts such as linearly independence and
subspace which are necessary in understanding the structure of a linear system.
Then we describe the relation between solution set and matrix, and special
matrices.
- 63 -
3.1
Matrix operation
Reference video: http://youtu.be/DmtMvQR7cwA, http://youtu.be/JdNnHGdJBrQ
ractice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-3-Sec-3-1.html
This chapter introduces the definition of the addition and scalar
multiplication of matrices and the algebraic properties of the matrix
operations.
Although many of the properties are identical to those of
the operations on real numbers, some properties are different. Matrix
operation is a generalization of the operation on real numbers.
Definition
[Equality of Matrices]
Two matrices × and × of same size are equal if
for all , and denote it by .
To define equal matrices, the size of two matrices should be the same.
Example
1
For what values of the two matrices
,
are equal?
Solution
For , each entry should be equal. Thus (that is, )
■
, , , .
Definition
[Addition and scalar multiplication of matrix]
Given two matrices × and × and a real number ,
the sum of and , and the scalar multiple of by are
defined by
× , × .
- 64 -
To define addition, the size of two matrices should be the same.
Example
2
For
, what is
, ,
?
,
Solution
·
· ·
· ·
·
· ·
.
· ·
□
Ÿ
http://matrix.skku.ac.kr/RPG_English/3-MA-operation.html
Ÿ
http://matrix.skku.ac.kr/RPG_English/3-MA-operation-1.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ,[[1,2,-4], [-2,1,3]])
B=matrix(QQ,[[0,1,4], [-1,3,1]])
C=matrix(QQ,[[1,1],[2,2]])
print A+B
# matrix addition
print
print 2*A
# scalar multiplication
print
print (-1)*C
# scalar multiplication
[ 1
3
0]
[ 2
4 –8]
[-1 -1]
[-3
4
4]
[-4
2
[-2 –2]
6]
- 65 -
■
Definition
[Matrix product]
Given two matrices × and × , the product of
and is defined below.
× ,
where ⋯
≤ ≤ ≤ ≤ .
For two matrices and to be compatible for multiplication, we require the
number of columns of to be equal to the number of rows of . The resultant
matrix is of size number of rows of by the number of columns of .
[Remark]
[Remark] Meaning of matrix product
Let × × , and denote the th row of by and the th
column of by . Then
⋮
⋯
⋯
⋮
⋮
⋯
×
Thus,
⋯ ⋮ ⋯
Note that the inner product of th row vector of and the th column vector
of is the entry of .
- 66 -
Example
3
Let
,
. Then
□
http://matrix.skku.ac.kr/RPG_English/3-MA-operation-1-multiply.html
Ÿ
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ,[[1,2,-1], [3,1,0]])
B=matrix(QQ,[[-2,1], [0,-3], [2,1]])
print A*B
# Don't forget to include (*)!
[-4 -6]
[-6
0]
■
Using matrix product, one can express a linear system easily. Let us consider
the following linear system
⋯
⋯
⋮
⋯
and let × , x
, b be the coefficient matrix, the unknown
⋮
⋮
vector and the constant vector respectively.
Then we can express the linear
system as
x b
⇔
⋯ b
- 67 -
⋯
⋮
⋮
⋮
⋮
⇔
Theorem
3.1.1
Let be matrices of proper sizes (oeprations are well defined) and
let be scalars. Then the following hold.
(1)
(commutative law of addition)
(2)
(associative law of addition)
(3)
(associative law of multiplication)
(4)
(distributive law)
(5)
(distributive law)
(6)
(7)
(8)
(9)
The proof of the above facts are easy and readers are encouraged to prove them.
Example
4
Check the associative law of the matrix product.
Solution
, we have
Since
Since
, we have
. Hence, .
■
The properties of operations on matrices are similar to those of operations on
real numbers which are well known,
Exception: For matrices , we do not have in general.
- 68 -
Example
5
Suppose that we are given the following matrices .
,
.
Then is defined but is not defined. Similarly is a ×
matrix but is a × matrix, and hence ≠ . Also although
and are × matrices, as we can see below, we have ≠
.
[Remark] Computer simulation
[matrix product] (Commutative law does not hold.)
http://www.geogebratube.org/student/m12831
- 69 -
■
Definition
[Zero matrix]
A zero matrix consists of entries of 0's and denoted by (or × ).
Theorem
⋯
3.1.2
For any matrix and a zero matrix of a proper size, the following hold.
(1)
(2)
(3)
(4)
Note: Although , it is possible to have ≠ , ≠ . Similarly,
although , ≠ , it is possible to have ≠ .
Example
6
Let
. Then
But ≠ and ≠ . Also but ≠ ≠ .
■
We should first define scalar matrices.
Definition
[Identity matrix]
A scalar matrix of order with diagonal entries all 1's is called an
identity matrix of order and is denoted by . That is,
⋮
⋮
⋯
⋯
⋱
⋯
⋮
×
Let be an × matrix and the identity matrix . It is easy to see that
.
- 70 -
Example
7
,
Let
. Then
.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ,[[4,-2,3], [5,0,2]])
I2=identity_matrix(2)
# identity matrix identity_matrix(n), n is the order
I3=identity_matrix(3)
O2=zero_matrix(3, 2)
# zero matrix zero_matrix(m, n), m, n are the order
print I2*A
print
print A*I3
print
print A*O2
[ 4 -2
3]
[ 5
0
2]
[ 4 -2
3]
[ 5
2]
0
[0 0]
[0 0]
■
Definition
Let be a square matrix of order . The th power of is defined
by
⋯ ( times)
Theorem
3.1.3
If is a square matrix and are non negative integers, then
.
- 71 -
Example
8
Let
. Find , , and confirm that .
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ,[[4,-2], [5,0]])
print A^2
# Works only for a square matrix
print
print A^3
# same format as power of real numbers
print
print A^0
# When the exponent is 0, get identity matrix
print
(A^2)^3==A^6
[
6
# check the power rule
-8]
[ 20 -10]
[-16 -12]
[ 30 -40]
[1 0]
[0 1]
True
In
the
■
set
of
real
numbers,
we
have
. However, the commutative law under matrix
product does not work and thus we only have the following.
.
When , we have .
Definition
[Transpose
matrix]
For a matrix × , the transpose of is denoted by and defined by
′ × , ′ ≤ ≤ ≤ ≤ .
The transpose of is obtained by interchanging the rows and columns of
.
- 72 -
Example
9
Find the transpose of the following matrices.
,
Solution
,
.
□
Sage
http://sage.skku.edu 또는 http://mathlab.knou.ac.kr:8080
A=matrix(QQ,[[1,-2,3], [4,5,0]])
C=matrix(QQ,[[5,4], [-3,2], [2,1]])
D=matrix(QQ, [[3,0,1]])
print A.transpose()
# Transpose of a matrix
A.transpose()
print
print C.transpose()
print
print D.transpose()
[ 1
4]
[ 5 –3
2]
[3]
[-2
5]
[ 4
1]
[0]
[ 3
0]
Theorem
2
[1]
3.1.4
Let be matrices of appropriate sizes and a scalar. The following
hold.
(1)
(2)
(3)
(4) .
- 73 -
■
Example
10
Let
. Show that (3) of Theorem 3.1.4 is true.
Since
,
Solution
.
Also,
.
■
Thus .
[Trace]
Definition
The trace of × is defined by tr ⋯
.
3.1.5
Theorem
If , are square matrices of the same size and ∈ , then
(1) tr tr
(2) tr tr , ∈
(3) tr tr tr
(4) tr tr tr
(5) tr tr
Proof
We prove the item (5) only and leave the rest as an exercise.
tr
Example
11
tr .
■
Let
. Show that (5) of Theorem 3.1.5 is true.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ,[[1,-2], [4,5]])
B=matrix(QQ,[[5,4], [-3,2]])
print (A*B).trace()
# trace. A.trace()
print
print (B*A).trace()
37
37
■
- 74 -
3.2
Inverse matrix
Reference video: http://youtu.be/GCKM2VlU7bw, http://youtu.be/yeCUPdRx7Bk
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-3-Sec-3-2.html
In this chapter, we introduce an inverse matrix of a square matrix which
plays like a multiplicative inverse of a real number. We investigate the
properties of an inverse matrix. You will see that some properties holding in
the inverse of a real number are not true in the matrix inverse operation
although most hold in both inverses.
Definition
A square matrix of order is called invertible (or nonsingular) if
there is a square matrix such that
.
This matrix if exists is called the inverse matrix of . If such a
matrix does not exist, is called noninvertible, (or singular).
Example
1
From matrices
, we see that is the inverse
matrix
of
the
,
2
by
Example
following
.
computation.
■
Let . Note that the third row of has all zeroes. Thus for
any matrix
the third row of
is . Therefore there
does not exist such that , that is, is singular.
- 75 -
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ,[[1,4,3],[2,5,6],[0,0,0]])
A.is_invertible()
# check if matrix is invertible A.is_invertible()
False
Theorem
■
3.2.1
If is an invertible square matrix of order , then an inverse of is
unique.
Proof
Suppose that are inverses of . Then as
,
we have
Thus an inverse of is unique.
Example
3
■
A necessary and sufficient condition for
to be invertible is that
≠ . Hence one has
.
It is straightforward to check
.
- 76 -
■
3.2.2
Theorem
If are invertible square matrices of order and is a nonzero
scalar, then the following hold.
(1) is invertible and .
(2) is invertible and .
(3) is invertible and .
(4) is invertible and .
Proof
(1)~(4) Just check that the product of matrices are the identity matrix.
■
3.2.3
Theorem
If is an invertible matrix, then so is and the following holds.
.
Example
4
Let
. Check that .
Solution
Since
,
, we have
. Also since
we have
.
Ÿ
http://matrix.skku.ac.kr/RPG_English/3-SO-MA-inverse.html
- 77 -
■
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(ZZ, 2, 2, [3, 5, 1, 2])
B=matrix(ZZ, 2, 2, [1, 3, 2, 7])
AB=A*B
# AB calculation
print AB.inverse()
# inverse of AB, format A.inverse()
print
print B.inverse()*A.inverse()
# B^(-1)*A^(-1)
[ 17 -44]
[ -5
13]
[ 17 -44]
[ -5
13]
■
<3D printing Object of Conic Section>
http://www.youtube.com/watch?v=q_XPFJjncmQ&feature=youtu.be
- 78 -
3.3
Elementary matrices
Reference video: http://youtu.be/GCKM2VlU7bw, http://youtu.be/oQ2m6SSSquc
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-3-Sec-3-3.html
In the previous section, we defined an inverse of square matrices. In this
section, we shall discuss how to find an inverse of square matrices by using
elementary row operations and elementary matrices.
Definition
An n by n matrix is called an elementary matrix if it can be obtained
from by performing a single elementary row operation (ERO). A
permutation matrix is obtained by exchanging rows of .
Example
1
Listed below are three elementary matrices and the operations that
produce them.
: Interchange the 2nd and the 3th rows.
↔
: Add 2 times the 1st row to the 2nd row. →
: Multiply the 2nd row by 3. →
Sage
http://sage.skku.edu (Warning!! The index of Sage starts
from 0.)
E1=elementary_matrix(4, row1=1, row2=2)
# elementary matrix r2 <--> r3
# elementary_matrix(n, row1=i, row2=j)
exchange of ith row, jth row
E2=elementary_matrix(4, row1=2, scale=-3)
# elemenatry matrix (-3)*r3
# elementary_matrix(n, row1=i, scale=m)
multiply ith row by m
E3=elementary_matrix(4, row1=0, row2=3, scale=7) # row 7*r4 + r1
# elementary_matrix(n, row1=i, row2=j, scale=m)
the ith row.
print E1
- 79 -
add m times jth row to
print E2
print E3
Example
2
[1 0 0 0]
[ 1
0
0
0]
[1 0 0 7]
[0 0 1 0]
[ 0
1
0
0]
[0 1 0 0]
[0 1 0 0]
[ 0
0 -3
0]
[0 0 1 0]
[0 0 0 1]
[ 0
0
1]
[0 0 0 1]
0
■
[Property of elementary matrix] The product of an elementary matrix on
the left
and
any matrix
is the
matrix
that
results when
the
corresponding same row operation is performed on .
↔
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ, 3,3, [1,2,3,1,1,1,0,1,3])
E1=elementary_matrix(3, row1=1, row2=2)
# r2 <--> r3
E2=elementary_matrix(3, row1=1, row2=0, scale=2)
# 2*r1 + r2
E3=elementary_matrix(3, row1=1, scale=3)
#
3*r2
print E1*A
print
print E2*A
print
print E3*A
[1 2 3]
[1 2 3]
[1 2 3]
[0 1 3]
[3 5 7]
[3 3 3]
[1 1 1]
[0 1 3]
[0 1 3]
[Remark] The inverse of an elementary matrix is elementary.
- 80 -
■
Since
,
Since
,
,
Since
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
E1=elementary_matrix(3, row1=1, row2=2)
#
r2 <--> r3
E2=elementary_matrix(3, row1=2, row2=1, scale=4)
#
4*r2 + r3
E3=elementary_matrix(3, row1=1, scale=3)
#
3*r2
print E1.inverse()
print
print E2.inverse()
print
print E3.inverse()
[1 0 0]
[ 1
0
0]
[
1
0
0]
[0 0 1]
[ 0
1
0]
[
0 1/3
0]
[0 1 0]
[ 0 -4
1]
[
0
1]
0
Finding the inverse of an invertible matrix.
We investigate the method to find the inverse of an invertible matrix using
elementary matrices. First consider equivalent statements of an invertible matrix
(its proof will be treated in Chapter 7).
Theorem 3.3.1 [Equivalent statements]
For any × matrix , the followings are equivalent.
(1) is invertible.
(2) is row equivalent to . (i.e. RREF )
(3) can be expressed as a product of elementary matrices.
(4) x has only the trivial solution .
- 81 -
[Remark]
3.3.2 [Computation of an inverse]
Theorem
[Remark] Finding an inverse using the Gauss-Jordan elimination.
[Step 1] For a given , augment on the right side so that we make
a × matrix .
[Step 2] Compute the RREF of ⋮ .
[Step 3] Let ⋮ be the RREF of ⋮ in the step 2. Then, following hold.
(ⅰ) If , then .
(ⅱ) If ≠ , then is not invertible so that does not exist.
Example
3
Find the inverse of
Solution
Consider ⋮ . Then
- 82 -
⋮
⋮ ⋮
⋮
and, its RREF is given as follows.
⋮
⋮
⋮
→
⋮
→ ⋮
⋮
⋮
⋮
⋮ → ⋮
⋮
⋮
⋮
⋮
→
⋮
→ ⋮
⋮
⋮
⋮
Since , .
∴
Example
4
■
Find the inverse of
Solution
→
It follows from a similar way to Example 03,
⋮
⋮
⋮
→
⋮
⋮
⋮
⋮
⋮ ⋮
⋮
■
Since ≠ , does not exist. ≠ .
Example
5
Find the inverse of
Ÿ
http://matrix.skku.ac.kr/RPG_English/3-MA-Inverse_by_RREF.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
- 83 -
A=matrix(QQ, 3, 3, [1, -1, 2, -1, 0, 2, -6, 4, 11])
I=identity_matrix(3)
Aug=A.augment(I).echelon_form() # augmented matrix [A : I] echelon_form
show(Aug)
[
1
0
0
|
8/15 -19/15
2/15]
[
0
1
0
|
1/15 -23/15
4/15]
[
0
0
1
|
4/15 –2/15
1/15]
We can extract inverse of using slicing of the above matrix.
Aug[:, 3:6]
[
8/15 -19/15
2/15]
[
1/15 -23/15
4/15]
[
4/15
1/15]
Thus
‘If
-2/15
.
■
you want to learn about nature, to
appreciate
nature,
it
is
necessary
to
understand the language(Mathematics) that
she speaks in.’
Richard Phillips Feynman (1918–1988) was an American theoretical physicist
known for his work in the path integral formulation of quantum mechanics,
the theory of quantum electrodynamics, and the physics of the superfluidity
of supercooled liquid helium, as well as in particle physics.
- 84 -
Subspaces and Linear Independence
3.4
Reference video: http://youtu.be/HFq_-8B47xM. http://youtu.be/UTTUg6JUFQM
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-4-Sec-3-4.html
In this section, we define a linear combination, a spanning set, a
linear (in)dependence and a subspace of ℝ . We will also learn
how to solve the system of linear equations by using the fact that
solutions for a system of homogeneous linear equations form a
subspace of ℝ .
Note that ℝ with standard addition and scalar multiplication is also called a
vector space over ℝ and its elements are called vectors.
Definition
[Subspace]
Let be a nonempty subset of ℝ . Then is called a subspace of ℝ
if satisfies the following two conditions.
(1) x y∈
⇒
(2) x ∈ ∈ℝ
x y∈
⇒
(closed under the addition)
x ∈ (closed under the scalar
multiplication)
All subspaces of ℝ contain zero vector.
x ∈ ∈ℝ
Example
1
⇒
x ∈
and ℝ are subspaces of ℝ where ⋯ is denoted by
the origin. They are called the trivial subspaces.
Example
2
A subset ∈ℝ of ℝ
■
satisfies two conditions for
subspace. Hence, is a subspace of ℝ . On the other hand, a
subset ∈ℝ of ℝ does not satisfy conditions
for subspace so that is not a subspace of ℝ .
, ∈ but ∉
- 85 -
■
Example
3
All subspaces of ℝ are one of the followings.
1. zero subspace :
2. Lines through the origin.
3. ℝ
All subspaces of ℝ are one of the following.
1. zero subspace :
2. Lines through the origin
3. Planes through the origin
4. ℝ
■
■
Example
4
Show that a subset ∈ℝ is a subspace of
ℝ .
Solution
For
x , y ∈ , ∈ℝ
the following hold.
(ⅰ) x y ∈
(ⅱ) x ∈
Therefore, is a subspace of ℝ .
■
Let × denote the set of all × matrices over ℝ .
Example
5
For ∈ × , show that
x ∈ℝ x
is a subspace of ℝ . (This is called a solution space or null space
of )
Solution
Clearly, so that ∈ , ≠ ∅ . Since for x y∈ , ∈ℝ
x , y ,
we can obtain that
x y x y and
- 86 -
x x .
This implies x y ∈ x ∈ .
■
Therefore, is a subspace of ℝ .
[linear combination]
Definition
If x ∈ ℝ can be expressed in the form
x x x ⋯ x
… ∈ℝ
with x x ⋯ x ⊆ ℝ , then x is called a linear combination of
vectors x x ⋯ x .
Example
6
Let
x x
be
vectors
of
ℝ .
Can
x
be a linear combination of x and x ?
Solution
The answer is depend on whether there exist
in ℝ such
that
x x x .
From this observation, we can obtain
⇒
□
One can easily show that the above system has no solution.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 3, [1, 3, 2, -2, -5, -6, -1, 4, 3])
# augmented matrix
print A.rref()
[1 0 0]
[0 1 0]
[0 0 1]
Since this system of linear equation has no solution, there are no such
scalars exist. Consequently, x is not a linear combination of x x .
- 87 -
■
Example
7
Show that the set of all linear combinations of x x ⋯ x ⊆ ℝ
x x ⋯ x ⋯ ∈ℝ
is a subspace of ℝ .
Solution
Let x y∈ , ∈ℝ . Then there exist ∈ℝ ⋯ such
that
x x x ⋯ x y x x ⋯ x .
Hence
x y x x ⋯ x ,
and x x x ⋯ x .
This implies x y ∈ x ∈ .
■
Hence, is a subspace of ℝ .
In
Example
7 , we saw that for a subset x x ⋯ x ⊆ ℝ , the set of all
linear combinations x x ⋯ x ⋯ ∈ℝ of is a
subspace of ℝ . We say is a subspace of ℝ spanned by . In this case,
we say spans and S is a spanning set of . We denote it
span or .
In particular, if all vectors in ℝ can be expressed a linear combination of ,
then spans ℝ . That is,
ℝ
Example
8
x x ⋯ x … ∈ℝ
(i) Show that is a spanning set of ℝ .
(ii) Show that is a spanning set of ℝ .
- 88 -
Definition
[column space and row space]
Let ∈ × . Then, columns ⋯ of span a
subspace of ℝ . This subspace is called a column space of , denote
by
… or Col( ).
Similarly, a row space of is defined by a subspace of ℝ spanned
by rows ⋯ of , denoted by
… or Row( ).
Example
9
For
x x x
determine whether x x x spans ℝ or not.
Solution
This is a question whether there exist , , such that a given
vector x is written as
x x x x ∈ℝ .
(Using column vectors)
Sage
⇒
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 3, [1, -3, -2, 0, 1, 1, 1, 1, 2])
# coefficient matrix
print A.rref()
[1 0 1]
[0 1 1]
[0 0 0]
This means that one of cannot be determined. Therefore this
linear system has a case that the system cannot determine a unique
solution.
■
- 89 -
Definition
[Linearly
Independent and Linearly Dependent]
If x x ⋯ x ⊆ ℝ satisfies
x x ⋯ x
⇒ ⋯
… ∈ℝ
then x x ⋯ x (or subset ) are called linearly independent.
If x x ⋯ x (or subset ) are not linearly independent, then it is called
linearly dependent.
If is linearly dependent, there exist at least one non-zero scalar
in … such that
x x ⋯ x .
The unit vectors of ℝ
e … e … … e …
are linearly independent. This is because
e e ⋯ e
⇒ ⋯ ⋯ ⋯ ⋯ ⋯
⇒ ⋯ ⋯
⇒ ⋯
Example
10
Show
that
for
.
x x ,
x x
is
independent.
Solution
For any ∈ℝ ,
x x
⇒
⇒
Thus , and is linearly independent.
- 90 -
■
linearly
Example
11
Show that if x x x in ℝ are linearly independent, then
x x x x x x
are also linearly independent.
Solution
For any ∈ ℝ ,
x x x x x x
⇒ x x x .
Since x x x are linearly independent,
⇒
Therefore x x x x x x are linearly independent.
Example
12
■
For
x x x
in ℝ , Show that x x x is linearly dependent.
Solution
For any ∈ℝ , if x x x , then
Sage
⇒
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 3, [1, 1, 0, 0, 1, 1, -1, 0, 1])
# coefficient matrix
print A.rref()
[ 1
0 -1]
[ 0
1
1]
[ 0
0
0]
This means that the above equations can be reduced to two equations
of three variables. Since it has three variables more than the number of
equations so that there are non-trivial solutions. One of them is given
- 91 -
by , , . Therefore there exist non zero scalars ,
■
is linearly dependent.
Theorem
3.4.1
For a set x x ⋯ x ⊆ ℝ , the followings hold.
(1) A set is linearly dependent if and only if some element in can be
expressed as a linear combination of the other elements in .
(2) If contains the zero vector, then is a linearly dependent.
(3) If a subset ′ of is linearly dependent, then is also linearly
dependent.
If is linearly independent, then ′ is also linearly independent.
Proof
(1) ( ⇒ ) If is linearly dependent, then there exist ⋯ such that
x x ⋯ x
where at least one element in ⋯ is a nonzero.
Without loss of generality, if ≠ then,
x x ⋯ x
so that x can be expressed as a linear combination of the other vectors
in
( ⇐ ) Without loss of generality, we can write
x x ⋯ x
so that
x x ⋯ x
Hence, is linearly dependent since ≠ .
Proofs of the rest are left as an exercise.
■
In other words, that set is linearly independent means that any vector in
cannot be written as a linear combination of the other vectors in .
In ℝ , there are at most vectors in a linearly independent set.
- 92 -
Theorem 3.4.2 (For proof, see Theorem 7.1.2)
In ℝ , vectors are always linearly dependent.
Example
13
For x x x
x in ℝ , we
can easily check that x x x x is linearly dependent from
Theorem 3.4.2.
■
[Remark] Lines and plaines (from the viewpoint of subspace)
(1) Note that the span of nonzero vector v in ℝ . v ∈ℝ is a subspace
containing the zero vector. Also
x v ∈ ℝ forms a line through x
and parallel to v. In other words, x x v is translate of x v by x .
(2) In general, if x v v ⋯ v are vectors in ℝ , then x x v ⋯ v
( ∈ ℝ )
is
a
subset
of
ℝ
which
is
x v ⋯ v , through the origin, by x .
- 93 -
the
translation
of
a
subspace
3.5
Solution set and matrices
Reference video: http://youtu.be/daIxHJBHL_g, http://youtu.be/O0TPCpKW_eY
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-4-Sec-3-5.html
In this section, we first state the relationship between invertibility of
matrices and solutions to systems of linear equations, and then consider
homogeneous systems.
Theorem 3.5.1 [Relation between an invertible matrix and its solution]
If an × matrix is invertible and b is a vector in ℝ , the system
x b
has a unique solution x b .
Example
1
The following system can be written as x b.
. It is easy to show that is
where x b
invertible, and
. Thus the solution of the above
system is given by
x b
.
That is .
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 3, [1, 2, 3, 2, 5, 3, 1, 0, 8])
# coefficient matrix
b=vector([1, 3, -1])
Ai=A.inverse()
# inverse matrix calculation
print "x=", Ai*b
print
print "x=", A.solve_right(b)
# solve directly.
- 94 -
x= (-1, 1, 0),
x= (-1, 1, 0)
■
[Remark] The homogeneous linear system
⋯
⋯
⋮
⋯
can be written as x , where
⋯
⋯
⋮ ⋮
⋯
x
⋮
⋮
⋮
The vector x is called a trivial solution, and the solution x≠ is called a
nontrivial solution. Since a homogeneous linear system always has a trivial
solution, there are two cases as follows.
(1) It has only a trivial solution.
(2) It has infinitely many solutions (i.e. it has nontrivial solutions as well.)
Theorem 3.5.2 [Nontrivial solution of a homogeneous system]
A homogeneous system with equations and variables such that
(i.e. the number of variables is greater than that of equations) has
nontrivial solutions.
For a detailed proof for this theorem, see Linear Algebra : A Geometric Approach
by S. Kumaresan, Prentice Hall of India, 2000.
- 95 -
Example
2
The homogeneous linear system
has the following augmented matrix and its RREF.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 5, [1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 2, 1, 0, 0])
# augmented matrix
print "A="
print A
print
print "RREF(A)="
print A.rref()
# RREF
A=
[1 1 1 1 0]
[1 0 0 1 0]
[1 2 1 0 0]
RREF(A)=
[ 1
0
0
1
0]
[ 0
1
0 -1
0]
[ 0
0
1
0]
1
The corresponding system of equations is
Let ( : a real number). Then the solution to (2) is
∈ℝ .
The solution is trivial if , and nontrivial if ≠ .
Definition
Given
[The
a
■
associated homogeneous system of linear equations]
linear
system
x b,
x
is
called
homogeneous system of linear equations of x b.
- 96 -
the
associated
Example
3
Consider a system of linear equations.
The associated homogeneous linear system is as the following:
Solution
Since the matrix size is greater than 2, let us use Sage.
The RREF of the augmented matrix of the above system is as follows :
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(4, 7, [1, 2, -2, 0, 2, 0, 0, 2, 6, -5, -2, 4, -3, -1, 0, 0, 5, 10, 0,
15, 5, 2, 6, 0, 8, 4, 18, 6])
# augmented matrix
print A.rref()
# RREF
[
1
0
0
4
2
0
0]
[
0
1
0
0
0
0
0]
[
0
0
1
2
0
0
0]
[
0
0
0
0
0
1 1/3]
Thus the above system reduces to
, , , .
Note that and are free variables.
Let , . Then we have
, ∈ℝ .
- 97 -
Consider the augmented matrix of RREF of its associated homogeneous
linear system.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
B=matrix(4, 7, [1, 2, -2, 0, 2, 0, 0, 2, 6, -5, -2, 4, -3, 0, 0, 0, 5, 10, 0,
15, 0, 2, 6, 0, 8, 4, 18, 0])
# augmented matrix
print B.rref()
# RREF
[1 0 0 4 2 0 0]
[0 1 0 0 0 0 0]
[0 0 1 2 0 0 0]
[0 0 0 0 0 1 0]
It is easy to see that the solution to this system is given by
, ∈ℝ .
■
When compared geometrically the solutions to a system and those of
an associated homogeneous linear system, the solution set for the
associated homogeneous linear system is merely translated by the
vector x below.
x
We call the vector x a particular solution which can be obtained by
substituting .
- 98 -
[Remark] Relation between the solution set of the linear system and that of the
associated homogeneous linear system.
If x and x b , then
x x x x b b.
Thus a system of linear equation x b has solutions. Let be a solution
space to x . If x is a solution to x b, then
x x x x ∈
is a solution set of x b.
A geometric meaning of x which is a solution set of x b is a set of
translation when a particular solution x is added to a solution set of x .
Since x does not contain a zero vector, it is not a subspace of .
Theorem
3.5.3 [Equivalent theorem of an invertible matrix]
For an × matrix , the following are equivalent.
(1) RREF
(2) is a product of elementary matrices.
(3) is invertible.
(4) is the unique solution to x .
(5) x b has a unique solution for any b ∈ℝ .
(6) The columns of are linearly independent.
(7) The rows of are linearly independent.
[Remark]
The vectors of the solution space of x are orthogonal to the rows of .
- 99 -
Let us think of the homogeneous system x with variables. If the
system has linear equations, then the size of matrix is × . It can be
rewritten using inner product. Let , , ⋯ , indicate rows of a matrix
.
⋅ x
⋅ x
x
⋮
⋮
⋮
⋅ x
Thus ⋅ x ( ≤ ≤ ) if x is a solution to x . That is, the
vectors in this solution space of x are all orthogonal to the row vectors
of the matrix .
Example
4
Consider
the
system
of
linear
equations:
,
, . It is easy to check that
v is non-trivial solution of this system.
Let us verify that v is
orthogonal to row vectors of the coefficient matrix
of the above
system.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix([[1,2,1,-3],[2,-1,1,-2],[2,1,1,-3]])
v=vector([1,1,3,2])
R=A.rows()
print v.dot_product(R[0])
print v.dot_product(R[1])
print v.dot_product(R[2])
0
0
0
Thus v is orthogonal to row vectors of the coefficient matrix .
- 100 -
[Remark] Hyperplane
(1) Line of -plane: the solution set of a linear equation ,
≠
(2)
Plane
of
-space:
the
solution
set
of
a
linear
equation
, ≠
(3) Hyperplane of ℝ : the solution set of ⋯ , ∃ ≠
(If , then it is a hyperplane passing through the origin)
a⋅ x , a≠
The set a ⊥ x∈ℝ a⋅ x is called an orthogonal complement of a .
- 101 -
3.6
Special matrices
Reference video: http://youtu.be/daIxHJBHL_g, . http://youtu.be/jLh77sZOaM8
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-4-Sec-3-6.html
We saw various properties of matrix operations. In this section, we
introduce special matrices and consider some of their crucial
properties.
Diagonal matrix: A square matrix with the entries 0 except the main diagonal. A
diagonal matrix with its main diagonal entries ⋯ can be written as
diag …
diag ⋯
⋱
Identity matrix: the matrix with its main diagonal entries all 1’s, denoted by
Scalar matrix:
⋱
Example
1
,
⋱
The following are all diagonal matrices. and are scalar matrices.
and are written as diag and diag .
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
G=diagonal_matrix([2, -1])
# generate diagonal matrix
H=diagonal_matrix([-3, -2, 1])
# diagonal_matrix([a1, a2, a3])
print G
print H
- 102 -
[ 2
0]
[-3
[ 0 -1]
Example
2
0
0]
[ 0 -2
0]
[ 0
1]
0
■
Consider the following matrix.
If and , .
For a general matrix × , is obtained by multiplying each
row of by the corresponding entry of , and is obtained by
multiplying each column of by the corresponding entry of ,
Furthermore, it satisfies the following.
, ,
In other words, the power of a diagonal matrix is the same as the
diagonal matrix with the powers of the entries of the main diagonal.
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
D=diagonal_matrix([1, -3, 2])
# generating a diagonal matrix
print "D^(-1)="
print D^(-1)
print
print "D^5="
print D^5
D^(-1)=
[
1
0
0]
[
0 -1/3
0]
[
0
0
1/2]
D^5=
[
1
0
0]
[
0 -243
0]
[
0
0
32]
■
- 103 -
Definition
If a square matrix satisfies , is called a symmetric matrix.
If , then is called a skew-symmetric matrix.
Example
3
In the following matrices, and are symmetric matrices and is a
skew-symmetric matrix.
Ÿ
http://matrix.skku.ac.kr/RPG_English/3-SO-Symmetric-M.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(3, 3, [1, 2, 3, 2, 4, 5, 3, 5, 6])
B=matrix(3, 3, [0, 1, -2, -1, 0, 3, 2, -3, 0])
print bool(A==A.transpose())
# Check if A symmetric
print bool(-B==B.transpose())
# Check if B anti-symmetric
True
True
Example
4
■
If is a square matrix, prove the following.
(1) is a symmetric matrix.
(2) is a skew-symmetric matrix.
Solution
(1) Since , is
a symmetric matrix.
(2) Since , is
a skew-symmetric matrix.
- 104 -
■
[Remark]
A given matrix can be written uniquely as a sum of a symmetric matrix and a
skew-symmetric matrix.
For any given matrix and is a symmetric
Proof
matrix and is a skew-symmetric matrix.
■
Lower triangular matrix: A square matrix whose entries under the main diagonal
are all zeros
Upper triangular matrix: A square matrix whose entries above the main diagonal
are all zeros
Example
5
In general, × triangular matrices are as follows.
Theorem
3.6.1 [Property of a triangular matrix]
Let and be a lower triangular matrix.
(1) ⋅ is a lower triangular matrix.
(2) If is an invertible matrix, then is a lower triangular matrix.
(3) If for all , then the main diagonal entries of is all 1’s.
Example
6
Let be a square matrix. If there exists an positive integer such that
(
is called nilpotent), is invertible and
⋯ . This is because
⋯
- 105 -
■
Exercises
Chapter 3
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
T/F Problem
Indicate whether the statement is true (T) or false (F). Justify your
answer.
(a) If three nonzero vectors form a linearly independent set, then each vector in
the set can be expressed as a linear combination of the other two.
(b) The set of all linear combinations of two vectors
v and w in ℝ is a plane.
(c) If u cannot be expressed as a linear combination of v and w, then the three
vectors are linearly independent.
(d) A set of vectors in ℝ that contains is linearly dependent.
(e) If {v , v , v } is a linearly independent set, then so is the set { v , v , v } for
every nonzero scalar .
Problem 1
When
, confirm the following.
.
Problem 2
When
, confirm that
and that ≠ .
Problem 3
When
, compute the following.
- 106 -
Problem 4
Show that is the inverse of . And confirm that .
Problem 5
Problem 6
If , show that .
Find a × elementary matrix corresponding to each elementary
operation.
(1) ↔
(2) →
(3) →
Problem 7
Using elementary operations, find the inverse of the following matrix.
(1)
Solution
(2)
⋮
⋮
⋮ = ⋮
⋮
→
⋮
⋮
⋮
⋮
⋮
⋮
⋮
⋮
= ⋮ .
- 107 -
→
.
■
Problem 8
Let
and be any × matrix.
(1) What is and confirm how affects on .
(2) What is and confirm how affects on .
Problem 9
Determine if is a subspace of ℝ .
Problem 10
Determine if is a subspace of ℝ .
Problem 11
Find a vector equation and a parameterized equation of the subspace
spanned by the following vectors.
(a) v , v
(b) v , v
Solution
(a) , ,
where , in ℝ.
(b) , , , , .
Problem 12
■
Give a solution by finding the inverse of the coefficient matrix of the
system.
Problem 13
Determine if the homogeneous system has a nontrivial solutoin.
- 108 -
Problem 14
Check if the following matrix is invertible. If so, find its inverse by
using a property of special matrices.
Problem 15
Find the product by using a property of special matrices.
Problem 16
Determine so that is skew-symmetric matrix.
satisfies ≠ and
, show that can be expressed
P1 If
as follows.
What is the value of ?
Solution
=>
∴
- 109 -
■
P2 Let be a square matrix. Explain why the following hold.
(1) If contains a row or a column consisting of 0's, is not
invertible.
(2) If contains the same rows or columns, is not invertible.
(3) If contains a row or column which is a scalar multiple of
another row or column of .
P3 Let be an × square matrix. Discuss what condition is need to
have ⇒ .
P4 Find × matrices , and explain the relation with ERO.
P5 Decide if the following 4 vectors are linearly independent.
v , v , v , v
P6 If x b and x c have a solution, prove that x b c has a
solution.
P7
Suppose
is an invertible matrix of order . If
v in ℝ
is
perpendicular to every row of , what is v? Justify your answer.
P8 Prove that a necessary and sufficient condition for a diagonal matrix
to be invertible is that there is no zero entry in the main diagonal.
P9 If is invertible and symmetric, so is .
Solution
,
=>
=>
and .
=> is symmetric.
- 110 -
■
4
Chapter
Determinant
4.1 Definition and Properties of the Determinants
4.2 Cofactor Expansion and Applications of the Determinants
4.3 Cramer's Rule
*4.4 Application of Determinant
4.5 Eigenvalues and Eigenvectors
- 111 -
The concept of determinant was introduced 150
years before the use of modern matrix, and we
have
used
the
determinant
to
solve
the
systems of linear equations for over 100 years.
In late 19th century, Sylvester introduced the
concept of matrix and the method for solving
systems
of
equations
by
using
an
inverse
matrix, where the determinant is used to check
if an inverse of a matrix exists or not. Also,
the determinant can be used to find area,
volume,
equations
of
lines
or
planes,
and
exterior product. It also helps in geometric
interpretation of vectors.
In this chapter, we first define the determinant
and review its properties. Then we study how to compute the determinant by
cofactor expansion. We also study Cramer's rule which solves the systems of
linear equations by using the determinant.
One of the most important concepts in linear algebra is
eigenvalues and
eigenvectors. Eigenvalues have almost all important informations by components
from an object with
components. Eigenvalues are not only important in
theoretical perspective but also applicable to almost all areas related to matrix,
such as, finding the solutions of differential equations, computing the power of
given matrix, Google search, and image compression, etc. In the last section of
this chapter, we compute eigenvalues by using the determinant.
- 112 -
4.1
Definition of Determinant
Reference video: http://youtu.be/DM-q2ZuQtI0, http://youtu.be/Vf8LlkKKHgg
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-5-Sec-4-1.html
In this section, we introduce a determinant function which assigns any
square matrix to a real number . In order to define the
determinant function, we first introduce permutation. Then we review
the properties of the determinant function.
Definition
[Permutation]
For a set of natural numbers … , permutation is a one to
one function from to .
We simply denote a permutation as ⋯ ⋯ . As a
permutation is an one to one correspondence, the range … is
simply a rearrangement of … . Hence, there are permutations on
⋯ . We denote the set of all permutations of set by .
…
- 113 -
[Remark]
Inversion
In permutation ⋯ , an inversion is the case when a bigger natural
number placed on the left hand side of a smaller natural number. For example,
in a permutation , is placed on the left hand side of , and hence
is an inversion. Similarly, is an inversion.
Number of inversions for : after ( )-th index, the number of indexes which
is smaller than -th index is called the number of inversions for . In the
above example, the number of inversion for is . Number of inversions for a
permutation ⋯ is the total sum of each number of inversions for ,
⋯ .
Definition
[Even permutation and odd permutation]
If number of inversions for a permutation is even than it is called an
even permutation, If the number is odd than it is called an odd
permutation.
Example
1
Determine whether it is even or odd permutation by computing the
inversion numbers for a permutation in .
Solution
The number of inversions for 5 is 4. The number of inversions
for 1 is 0, for 2 is 0, for 4 is 1, and 3 is the last index. Hence, the total
sum is , and it is an odd permutation.
Ÿ
http://matrix.skku.ac.kr/RPG_English/4-TF-Permutation.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
Permutation([5,1,2,4,3]).inversions()
- 114 -
# inversions
□
[[0, 1], [0, 2], [0, 3], [0, 4], [3, 4]]
# Note!! Index starts from 0
Permutation([5,1,2,4,3]).number_of_inversions()
# Number of inversions
5
Permutation([5,1,2,4,3]).is_even()
# check whether it is even permutation
False
Definition
■
[Signature function]
Signature function sgn → , which assigns each permuta
tion of to either +1 or -1 as follows.
sgn
Example
2
even permutation
odd permutation
Classify the permutations of to either even or odd permutation.
Solution
permuta
number of
tions
inversions
class
sign
even
even
even
odd
odd
odd
■
In permutation, if two numbers switch the location then the signature is changed
- 115 -
4.1.1
Theorem
Let
be a permutation by switching any two numbers from given
permutation . Then
sgn sgn
Definition
[Determinant]
[Leibniz formula]
Let be an × matrix. We denote the determinant of matrix
as det or and define it as follows.
det
sgn
∈
⋯
By definition, × matrix has it's determinant as det .
Each term sgn ⋯ in the determinant is from the matrix , by
choosing a row and a column, without any overlapping, then multiplying them
and assigning a corresponding signature.
Example
3
.
Find the det , where
Solution
As is × matrix, . Since
sgn , sgn , we have
det sgn sgn .
https://en.wikipedia.org/wiki/Rule_of_Sarrus
- 116 -
■
Example
4
Find the det , where
.
Solution
As is × matrix,
.
Since sgn , sgn , sgn , sgn , sgn ,
sgn ,
by substituting them into the definition of the determinant, we have
det sgn sgn
⋯ sgn
■
Example
5
Compute the determinant of the following matrices.
,
.
Solution
.
Ÿ
.
http://matrix.skku.ac.kr/RPG_English/4-B1-Det-matrix.html
- 117 -
□
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
B=matrix(QQ, 3, 3, [1, 2, 3, -4, 5, 6, 7, -8, 9])
print B.det()
# compute the determinant
■
240
[Remark] Sarrus' method cannot be applied to the case of degree 4 or higher.
Hence, the determinant with degree 4 or higher should be computed by the
definition. But in that case, there are too many terms and signs to be
determined. (Indeed, for degree 4 case, there are terms, and for degree
10, there are terms to compute). Therefore, it is more effective
to study the properties of the determinant and find the determinant by using
these properties. (We will skip the proofs but will verify them by examples).
Properties of the determinant
4.1.2
Theorem
A square matrix
and its transpose matrix
have the same
determinant.
http://math.stackexchange.com/questions/123923/a-matrix-and-its-transpose-havethe-same-set-of-eigenvalues
Example
6
In
Example
5 , , and
. Since
- 118 -
we have .
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
B=matrix(QQ, 3, 3, [1, 2, 3, -4, 5, 6, 7, -8, 9])
print B.transpose().det()
■
240
The properties of the determinant regarding to rows also work to columns.
4.1.3
Theorem
Let be a matrix obtained by switching two rows (columns) from a
square matrix then .
Proof
Let be a matrix obtained by replacing th and th row of
, . This means
sgn
∈
sgn
∈
(by definition)
sgn
∈
, and if ≠ .
sgn
∈
(by theorem 4.1.1)
Example
7
Let
■
.
and
Since
and
,
.
Theorem
■
4.1.4
If a square matrix has two identical rows (columns) then .
Example
8
Let which has identical first and third rows. Note
- 119 -
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ, 3, 3, [1, 2, 3, -1, 0, 7, 1, 2, 3])
print A.det()
# compute the determinant
■
0
Theorem
4.1.5
If a square matrix
has a row (column) with identical zeros then
.
Example
9
Let which has identical zeros in the third row. Observe
×× ×× ×× ×× ×× ××
Theorem
■
4.1.6
Let be a matrix obtained by multiplying times a row of a square
matrix . Then
Example
10
.
Let . Note that
- 120 -
■
4.1.7
Theorem
If a square matrix has two proportional rows then .
4.1.8
Theorem
Let be a square matrix and times of one row is added to another
row of and name this new matrix as , then .
Proof
Let
be a new matrix whose second row is obtained by adding
times of the first row of to ∈ .
det
∈
det
∈
=>
det
Example
11
∈
∈
(by Theorem 4.1.4)
■
Let and 2 times of the second row is added to the first
row and name it as matrix . Then
⋅ ⋅
.
Note that .
Theorem
■
4.1.9
If is an × triangular matrix, the determinant of equals
the product of the diagonal elements. That is,
⋯
Example
12
From the previous theorem,
- 121 -
. ■
[Remark] How to compute the determinant
1. Use elementary row operations to make many zeros to a certain row
(column).
2. Multiply the diagonal elements.
※ Note that during the elementary row operations, if you multiply k times a
row (column) or switch two rows (columns), do not forget to multiply 1/k and
-1.
Example
13
Find the determinant of a matrix , where
Solution
↔
Theorem
4.1.10
Let be an × elementary matrix. Then det det det .
- 122 -
■
[Remark] The determinant of an elementary matrix
1. If is obtained by multiplying ≠ to a row of , det
2. If is obtained by switching two rows of , det
3. If is obtained by multiplying times a row and adding it to another row
of , det
4. If is an × matrix and is an elementary matrix,
det det ⋅ det .
Equivalent conditions for invertible matrix
4.1.11
Theorem
is invertible if and only if det ≠ .
4.1.12
Theorem
For any two × matrices and ,
Example
14
.
Verify the above theorem with matrices
and
.
Since
, and
Solution
.
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ, 2, 2, [1, 2, 3, 4])
B=matrix(QQ, 2, 2, [2, -1, 1, 2])
C=A*B
print "det(AB)=", C.det()
print "det(A)*det(B)=", A.det()*B.det()
det(AB)= -10
det(A)*det(B)= -10
■
- 123 -
Theorem
4.1.13
and .
If a square matrix is invertible then
Example
15
Verify the above Theorem with a matrix
.
Solution
is invertible with
. Observe ≠ and
□
.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ, 2, 2, [1, 2, 3, 4])
Ai=A.inverse()
print "det(A)=", A.det()
print "det(A^(-1))=", Ai.det()
det(A)= -2
det(A^(-1))= -1/2
■
[19th International Linear Algebra Conference(Sungkyunkwan University, 2014)]
http://www.ilas2014.org/
Photos and Movie: http://matrix.skku.ac.kr/2014-Album/ILAS-2014/
- 124 -
Cofactor Expansion and Applications
4.2
of the Determinants
Reference Video: http://youtu.be/XPCD0ZYoH5I, http://youtu.be/m6l2my6pSwY
Practice Site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-5-Sec-4-2.html
In this section, we introduce a method which is convenient to compute
the determinant as well as important in theory. Moreover, by applying
this method, we introduce an easier formula to compute the inverse of
a matrix.
Definition
[Minor and cofactor]
We denote a submatrix, by removing the th row and th column of a
given square matrix ,
det
as
minor
as . We call its determinant
of
for
.
We
also
call
as cofactor of for .
Example
1
For given matrix , find the minor and cofactor of for
.
Solution
The minor of for is det
■
cofactor of A for is .
Definition
and the
[Adjoint matrix]
Let be a cofactor of × matrix for . The matrix
is called an adjoint matrix of and is denoted by adj . That is,
⋯
⋯
adj
⋮
⋮
⋯
⋯
⋯
⋮
⋮
⋮
⋯
- 125 -
⋮
Example
2
Find adj of the following matrix.
Solution
Note the cofactor of for each element is as follows.
Therefore,
adj
Ÿ
□
http://matrix.skku.ac.kr/RPG_English/4-MA-adjoint.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ, 3, 3, [3, -2, 1, 5, 6, 2, 1, 0, -3])
print A.adjoint()
[-18
# adjoint matrix
-6 -10]
[ 17 -10
-1]
[ -6
28]
-2
■
Cofactor expansion
The determinant of × matrix can be expanded as follows.
- 126 -
(Expand around the first column)
This is known as (Laplace) cofactor expansion of around the first column.
Cofactor expansion works for any column and any row.
For any × matrix , the following identity holds. That is,
⋅ adj
Which shows
Read:
⋅ .
.
≠
http://nptel.ac.in/courses/122104018/node29.html
For the previous example
Sage
Example
2,
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ, 3, 3, [3, -2, 1, 5, 6, 2, 1, 0, -3])
print "det(A)=", A.det()
print "A*adj(A)="
print A*A.adjoint()
det(A)= -94
A*adj(A)=
[-94
0
0]
[
0 -94
0]
[
0
0 –94]
■
Therefore the following holds.
- 127 -
Theorem 4.2.1 [Cofactor expansion]
Let be a × matrix. For any ( ≤ ≤ ) the following holds.
⋯
(cofactor expansion around th row)
⋯ (cofactor expansion around th column)
When computing the determinant, it is advantageous to use the cofactor
expansion around the row (column) which has many zeros.
Example
3
Find the determinant of a given matrix by using the cofactor expansion.
Solution
Multiply (-2) to the 2nd row and add it to the 3rd row. Multiply (-3) to
the 2nd row and add it to the both 1st and 4th row. Then
.
Now we cofactor expand around the first column,
.
Theorem 4.2.2 [Inverse matrix by using the adjoint matrix]
Let be an × invertible matrix, then the inverse matrix of is
adj .
- 128 -
■
Example
4
From
Example
Sage
2 , find the inverse matrix of .
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(QQ, 3, 3, [3, -2, 1, 5, 6, 2, 1, 0, -3])
dA=A.det()
# compute the determinant
adjA=A.adjoint()
# compute adjoint matrix
print "(1/dA)*adjA="
print (1/dA)*adjA
# compute inverse matrix
print
print "A^(-1)="
print A.inverse()
# compare the results of inverse matrix
(1/dA)*adjA=
[
9/47
3/47
5/47]
[-17/94
5/47
1/94]
[
1/47 -14/47]
3/47
A^(-1)=
[
9/47
3/47
5/47]
[-17/94
5/47
1/94]
[
3/47
1/47 -14/47]
[
3/47
1/47 –14/47]
■
[ATLAST project] http://www1.umassd.edu/SpecialPrograms/Atlast/
- 129 -
4.3
Cramer's Rule
Reference video: http://youtu.be/OImrmmWXuvU, http://youtu.be/m2NkOX7gE50
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-6-Sec-4-3.html
In this section, we introduce Cramer’s rule which is very useful tool for
solving a system of linear equations.
Cramer's rule can be applied to systems of linear equations with the same
number of unknowns and the equations.
Theorem
4.3.1 [Cramer's Rule]
For a system of linear equations,
⋯
⋯
⋮
⋮
⋮
⋮
⋯
b
let be a coefficient matrix, and x
. Then the system
⋮
⋮
of linear equations can be written as x b . If ≠ , the system of
linear equations has a unique solution as follows:
… .
Where
⋯
denotes
the
matrix
with
th
column
replaced by the vector b.
Proof
Since｜ ｜ ≠ , is invertible. Hence the system of linear equations
x b has a unique solution x b . Since adj , we have
x adj b
⋮
⋯
⋯
⋮
⋮
⋯
⋮
⋮
⋯
- 130 -
⋮
⋮
.
⋮
⋯
Observe the th component of x is . Since
⋯ ,
if we denote as a matrix with th column replaced by the vector b,
then we have
■
⋯ .
Example
1
Solve the following system of linear equations by Cramer's rule.
Solution
Let be the coefficient matrix. Then
, and hence
｜ ｜
｜ ｜
｜ ｜
, , .
｜｜
｜｜
｜｜
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A = matrix(3,3,[-2,3,-1,1,2,-1,-2,-1,1]);
A1 = matrix(3,3,[1, 3, -1,4, 2,-1,-3,-1,1]);
A2 = matrix(3,3,[-2,1, -1, 1,4,-1,-2,-3,1]);
A3 = matrix(3,3,[-2,3,1, 1, 2, 4, -2,-1,-3]);
print A.det()
print A1.det()
print A2.det()
print A3.det()
print "x =", A1.det()/A.det()
- 131 -
■
print "y =", A2.det()/A.det()
print "z =", A3.det()/A.det()
-2
-4
-6
-8
x = 2
y = 3
z = 4
Example
2
■
Solve the following system of linear equations by Cramer's rule.
Solution
From
Example
1 , and each matrix
has zeros column,
. Hence, the solution is z
Theorem
■
4.3.2 [Equivalence Theorem for Invertible Matrix]
For an × matrix , the following are equivalent.
(1) RREF
(2) is a product of elementary matrices.
(3) is invertible.
(4) is the unique solution to x .
(5) x b has a unique solution for any b ∈ℝ .
(6) The columns of are linearly independent.
(7) The rows of are linearly independent.
(8)
Note that there are more equivalent statements for the above theorem. For
more equivalent statements, refer Theorem 7.4.9 in section 7.4.
- 132 -
4.4
*Application of Determinant
Reference video: http://youtu.be/OImrmmWXuvU, http://youtu.be/KtkOH5M3_Lc
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-6-Sec-4-4.html
The concept of determinant was first introduced by Japanese Takakazu
Seki-Kowa in 1683. The term determinant originated from the meaning of
determining the existence of roots. It was Cauchy who used the term in
modern concept in 1815. In this section, we introduce some geometric and
algebraic applications among many other applications of the determinant.
By using a determinant, we can easily find areas, volumes, equations of lines,
equations of elliptic curves, or equations of plane. Also, the determinant of
Vandermonde
matrix
connects
between
discrete
data,
which
arise
from
statistical data and experimental labs, etc.
Example
1
Show that the equation of a line, which passes through two distinct points
and , is as follows.
Solution
Note that the above equation is degree 1 for both and . As the
equation holds by substituting and into the
equation, the equation must be the equation of the line which passes
through two given points.
■
- 133 -
[Remark] Computer simulation
[An equation of a line which passes through two distinct points]
http://www.geogebratube.org/student/m9504
Similar to
Example
distinct points
1,
an equation of a plane, which passes through three
and , is as follows:
- 134 -
.
[Remark] Computer simulation
[An equation of a plane which passes through three distinct points]
http://www.geogebratube.org/student/m56430
Consider an arbitrary non-singular square matrix ∈ . Let be the th
column and
≤ ≤ ⋯ .
For the case is a parallelogram, and for the case ≥ is a generalized
parallelepiped.
Parallelogram can be expressed by adding two vectors as follows.
The area of the above parallelogram is which is the same as the
absolute value of
. Similarly, a parallelepiped is generated by three vectors
which do not lie on the same plane. Let matrix 's columns consist by these
three vectors. Then the volume of the parallelepiped is absolute value of
- 135 -
det .
[Remark] Computer simulation
[Volume of parallelepiped]
http://www.geogebratube.org/student/m57553
Theorem
4.4.1
(1) Let be an × matrix. The area of parallelogram generated by
two column vectors is det .
(2) Let be an × matrix. The volume of parallelepiped generated by
three column vectors is det .
(3) The area of parallelogram generated by two vectors u v ∈ , is
det , where u v.
- 136 -
Proof
We will prove only (3).
Note that u⋅ v u v cos .
Also, the area of parallelogram is u vsin . Now, the determinant
u u u v
u v u vvu
det det
v u v v
u v v u v u u v u⋅ v
u v u v cos
u v cos u v sin (square of base times height)
makes the square of the area of the parallelogram generated by u v∈ .
Example
2
■
Show that the area of a triangle generated by three points ,
, is as follows.
det
Solution
As the area is not changing by parallel translation, the area of triangle
generated by three given vectors are the same as half of the area of
parallelogram generated by and . Hence,
by Theorem 4.1.1, we have
det det .
det
■
Vandermonde matrix and the determinants
If there are distinct points in the -plane with mutually distinct
coordinates, then there exist a unique polynomial which passes through all
given points with degree . Let's find the polynomial.
- 137 -
Let ⋯ be distinct points in the -plane with
mutually distinct coordinates. We want to find a polynomial of degree
which passes through all given points. Let
⋯ be such a polynomial.
Since these points satisfy the given polynomial, we have
⋯
⋯
⋮
⋯ .
Moreover, as ⋯ are mutually distinct, the coefficient matrix has
⋯
⋯
≠ .
⋮
⋯
This coefficient matrix is called Vandermonde matrix with degree . Now we
introduce how to compute the determinant of Vandermonde matrix. For the case
,
det
det
∴
≤ ≤
.
Similarly, as we illustrated in the above case, the determinant of Vandermonde
matrix with degree is product of (with ). That is,
- 138 -
det
⋯
⋯
⋮
≤
⋯
≤
.
[Reference] http://www.proofwiki.org/wiki/Vandermonde_Determinant
Example
3
Find a polynomial that passes through the points (39, 34), (99, 47), (38,
58) by using a Vandermonde matrix.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
def Vandermonde_matrix(x_list):
# generate Vandermonde matrix
n=len(x_list)
A=matrix(RDF, n, n, [[z^i for i in range(n)] for z in x_list])
return A
x_list=[39, 99, 38]
# x coordinate
V=Vandermonde_matrix(x_list)
y_list=vector([34, 47, 58])
# y coordinate
print "V="
print V
print
print "x=", V.solve_right(y_list)
V=
[
1.0
39.0 1521.0]
[
1.0
99.0 9801.0]
[
1.0
38.0 1444.0]
x= (1558.34590164, -54.568579235, 0.396994535519)
p=0.396994535519*x^2 -54.568579235*x + 1558.34590164
plot(p, (x, -20, 120))
# plot the graph
- 139 -
■
[Remark] Computer Simulation
[Curve Fitting]
http://www.geogebratube.org/student/m9911
[Area of parallelogram]
http://www.geogebratube.org/student/m113
[Solomon W. Golomb(Mathematics Magazine, March 1985)]
- 140 -
4.5
Eigenvalues and Eigenvectors
Reference video: http://youtu.be/OImrmmWXuvU, http://youtu.be/96Brbkx1cQ4
Practice site: http://matrix.skku.ac.kr/knou-knowls/CLA-Week-6-Sec-4-5.html
For an × matrix and a vector x ∈ , x is a vector in . One
of the important questions in applied problems is “Is there any nonzero
vector x, which makes both x and x parallel?” If such a vector exists,
then it is called an eigenvector and it plays many important roles in
linear transformation. In this section, we introduce eigenvectors and
eigenvalues.
Definition
[Eigenvalues and Eigenvectors]
Let be an × matrix. For nonzero vector x∈ , if there exist a
scalar which satisfies
x x , then is called an eigenvalue of
, and x is called an eigenvector of corresponding to .
Example
1
Let and x . Then x= x. Hence 3 is
an eigenvalue of , and x is an eigenvector corresponding to 3.
■
Example
2
Since x x, for any x ∈ , the only eigenvalue of identity matrix
is . Also, any nonzero vector x ∈
is an eigenvector of
corresponding to 1.
■
If x ∈ is an eigenvector of corresponding to , then x is also an
eigenvector of corresponding to for any nonzero scalar .
x x ⇒ x x x x .
- 141 -
General method to find eigenvalues
Since
x x ⇔ x x ⇔ x
and x≠ , the system of linear equations x should have nonzero
solution.
Therefore,
the
characteristic
equation,
should
hold.
is called the characteristic polynomial.
4.5.1
Theorem
Let be × matrix and is a scalar, then the following statements
are equivalent:
(1) is an eigenvalue of .
(2) is a solution of the characteristic equation det .
(3) System of linear equations x has a nontrivial solution.
Example
3
Find all eigenvalues and corresponding eigenvectors of
.
Solution
If x satisfies x x. Then,
⇔
⇔
(1)
However, as mentioned above, this system of linear equations should
have nontrivial (nonzero) solution. Hence,
⇔ ⇔
∴
① Let’s find an eigenvector corresponding to .
From (1),
⇔
∈ \
∴ x
- 142 -
② Let’s find an eigenvector corresponding to .
From (1),
⇔
∴ x ∈ \
[Remark] Computer simulation
[Visualize the eigenvalues and eigenvectors]
http://www.geogebratube.org/student/b73259#material/11114
Do eigenvalues exist for any square matrix?
Theorem
4.5.2 [Fundamental Theorem of Algebra]
For any real (or complex) coefficient polynomial with degree
⋯
has roots … , that is, for , on the complex
plane.
- 143 -
■
That is, a real square matrix with degree always has eigenvalues in
complex domain. However, in this textbook we have limited the scalar as real
numbers, and hence there is no eigenvalues means there is no real eigenvalues.
Example
4
Find eigenvalues and eigenvectors of a matrix
.
http://matrix.skku.ac.kr/RPG_English/4-BN-char_ploy.html
Ÿ
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
① Characteristic equation of
A=matrix(QQ, 2, 2, [1, -3, -3, 1])
# input A
print A.charpoly()
# characteristic equation of A
x^2 - 2*x – 8
② Hence the eigenvalues are as follows.
solve(x^2 - 2*x - 8==0, x)
[x == -2, x == 4]
③ We can find the eigenvalues directly by using the built in command.
A.eigenvalues()
# eigenvalues of A
[4, -2]
④ In order to find eigenvector for , solve x .
(-2*identity_matrix(2)-A).echelon_form() # consider only coefficient matrix
[ 1 -1]
[ 0
0]
- 144 -
=>
⇒
x ∈ ∖
⑤ In order to find eigenvector for , solve x .
(4*identity_matrix(2)-A).echelon_form()
[ 1
1]
[ 0
0]
Hence,
⇒
# consider only coefficient matrix
∈ ∖
x
⑥ We can find the eigenvectors directly by using the built in command.
A.eigenvectors_right()
[(4, [(1, -1)], 1), (-2, [(1, 1)], 1)]
# [eigenvalues, eigenvectors(it may appear in different form), multiplicity]
■
Example
5
.
Find eigenvalues and eigenvectors of a matrix
Ÿ
http://matrix.skku.ac.kr/RPG_English/4-VT-eigenvalues.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
① Characteristic equation of
A=matrix(QQ, 3, 3, [1, 2, 2, 1, 2, -1, 3, -3, 0])
print A.charpoly()
# input A
# characteristic equation of A
x^3 - 3*x^2 - 9*x + 27
② Hence the eigenvalues are as follows.
- 145 -
solve(x^3 - 3*x^2 - 9*x + 27==0, x)
[x == -3, x == 3]
③ We can find the eigenvalues directly by using the built in command.
A.eigenvalues()
# eigenvalues of A
[-3, 3, 3]
# shows root with multiplicity 2
④ In order to find eigenvector for , solve x .
(-3*identity_matrix(3)-A).echelon_form() # consider only coefficient matrix
[
1
0
2/3]
[
0
1 -1/3]
[
0
0
0]
Hence,
⇒
x
∈ ∖
⑤ In order to find eigenvector for , solve x .
(3*identity_matrix(3)-A).echelon_form()
# consider only coefficient matrix
[ 1 -1 -1]
[ 0
0
0]
[ 0
0
0]
Hence,
⇒
x
( and are real numbers not simultaneously become zero)
⑥ We can find the eigenvectors directly by using the built in command.
A.eigenvectors_right()
[(-3, [(1, -1/2, -3/2)], 1), (3, [(1, 0, 1),(0, 1, -1)], 2)]
#[eigenvalues, eigenvectors(it may appear in different form), multiplicity]
- 146 -
For a triangular matrix with degree , the main diagonal components
of are ( … ). Therefore, the characteristic polynomial of
is det ⋯ , and hence the eigenvalues of the
triangular matrix are its main diagonal components, … .
Example
6
Find the characteristic polynomial and all the eigenvalues of triangular
matrix
.
Solution
As det , 's eigenvalues are .
Definition
■
[Eigenspace]
Let be an eigenvalue of × matrix . Then the solution space of
the system of linear equations x is called eigenspace of
corresponding to .
That is, an eigenspace of corresponding to is the set of all eigenvectors of
corresponding to and the zero vector, which is a subspace of .
Example
7
From
the
given
matrix
in
Example
5,
find
corresponding to each eigenvalue and .
Solution
From the result of
Example
5,
① if , by solving x , we have
⇒
x
- 147 -
∈
eigenspaces
of
∴
② When , by solving x , we have
⇒
x
( , ∈ )
∴
■
http://matrix.skku.ac.kr/2009-Album/SKKU-Math-Card-F/SKKU-Math-Card-F.html
- 148 -
Exercises
Chapter 4
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
Problem 2
Is permutation of even or odd?
Find the following determinants.
(1) det
(2) det
(1)
(2)
(3)
(4)
For given matrices,
Let be × matrix and｜ ｜ , find the followings.
Problem 3
Problem 4
,
(1) show .
(2) show .
- 149 -
(3) show .
Problem 5
For which and , the given matrix is invertible?
Problem 6
For given matrices,
,
(1) show .
(2) show .
(3) show .
Problem 7
Find all cofactors of the following matrices.
(1)
(2)
Solution
Sage :
- 150 -
A=matrix(QQ, 3, 3, [1, 1, 5, 0, 0, 0, 2, 6, 2])
print "adj A="
print A.adjoint()
B=matrix(QQ, 5, 5, [1, 2, 3, 4, 5, 0, 1, 0, 1, 0, -1, 1, -1, 1, -1, 0, 1, 2, 3, 4, -4, 2, -3, 1, 5])
print "adj B="
print B.adjoint()
adj B=
adj A=
[-18
[ 0 28
0]
[ 0 -8
0]
[ 0 -4
0]
Problem 8
18
0]
[ 14 -28
14 -14
0]
[ 22 -44
22 -22
0]
[-14
[ -4
36 -18
28 -14
8
-4
14
0]
4
0]
■
Find the determinant of the matrix by cofactor expansion.
Problem 9
Find the adjoint matrix adj of the matrix from (Problem 8).
Problem 10
Find the inverse matrix of the given matrix by cofactor expansion.
(1)
Solution
(2) Sage :
(2)
adj
A = matrix(QQ, 5, 5, [1, 0, 1, 3, 5, -1, 3, 0, 7, 2, 1, 0, 2, 1, 8, 2, -4, 0, 0, 3, -8, 9, 2, 5, 4])
dA = A.det()
adjA = A.adjoint()
print "(1/dA)*adjA="
print (1/dA)*adjA
- 151 -
(1/dA)*adjA=
[ -18/133
[
23/133
2/7
13/19
1
-30/19
[ 999/133 -412/133
-48/133
-29/133]
-4/19
-4/19]
-27/7 -129/133
80/133]
[ 164/133
-47/133
-5/7
-6/133
13/133]
[-268/133
106/133
8/7
39/133
-18/133]
∴
Problem 11
■
Solve the systems of linear equations by using the Cramer's rule.
(1)
(2)
(3)
(4)
Problem 12
Solve
the
following
problems
by
using
the
determinant
of
Vandermonde matrix.
(1) Find the line equation which passes through the two points and .
- 152 -
(2) Find the coefficients of parabolic equation which passes
through the three points and .
Problem 13
Solve the following problems by using the determinant.
(1) The area of a parallelogram which is generated by two sides connecting the origin
and
each point and .
(2) The volume of parallelepiped which is generated by three vectors,
and .
Problem 14
Find the eigenvalues and eigenvectors of the following matrices.
(1)
Solution
Sage :
①
(2)
det
A = matrix(QQ, 4, 4, [-3, 0, -2, 8, 0, 1, 4, -2, -4, 10, -1, -2, 6, -4, -2, 3])
print "character polynomial of A ="
print A.charpoly()
character polynomial of A =
x^4 - 118*x^2 - 168*x + 1485
② Eigenvalues
solve(x^4 - 118*x^2 - 168*x + 1485==0, x)
x == 11, x == -9, x == -5, x == 3]
④ Let x1, x2, x3 and x4 be the above eigenvalues.
- 153 -
A = matrix(QQ, 4, 4, [-3, 0, -2, 8, 0, 1, 4, -2, -4, 10, -1, -2, 6, -4, -2, 3])
x1 = 11
x2 = -9
x3 = -5
x4 = 3
print (x1*identity_matrix(4)-A).echelon_form()
print
print (x2*identity_matrix(4)-A).echelon_form()
print
print (x3*identity_matrix(4)-A).echelon_form()
print
print (x4*identity_matrix(4)-A).echelon_form()
[
[
[
[
[
[
[
[
1
0
0
0
1
0
0
0
0
1
0
0
[
[
[
[
[
[
[
[
0
1
0
0
0 2]
0 -1]
1 2]
0 0]
1
0
0
0
1
0
0
0
0 -9/13]
0 7/13]
1 11/13]
0
0]
0
1
0
0
0
1
0
0
0 7/5]
0 7/5]
1 -13/5]
0
0]
0 -1]
0 -1]
1 -1]
0 0]
⑤ Finding eigenvectors
A = matrix(QQ, 4, 4, [-3, 0, -2, 8, 0, 1, 4, -2, -4, 10, -1, -2, 6, -4, -2, 3])
print A.eigenvectors_right()
[(11, [(1, -7/9, -11/9, 13/9)], 1), (3, [(1, 1, 1, 1)], 1), (-5, [(1, 1, -13/7, -5/7)], 1), (-9, [(1, -1/2, 1, -1/2)],
1)]
Eigenvectors corresponding to =11, =-9, =-5, =3 are
∴
x
, x
, x
.
Problem P1
x
,
■
Explain why det for the following matrix .
Problem P2
Show that for two square matrices and , if for an invertible
matrix , then .
Solution
- 154 -
■
Problem P3
Simplify the following determinant.
Problem P4
For × matrix , with , show the following identity.
det adj det
Problem P5
Let be a × matrix, and assume that
adj
.
(1) Find detadj . Which relation does this value have with det ?
(2) Find .
Solution
Sage :
adjA=matrix(QQ, 4, 4, [2, 0, 0, 0, 0, 2, 1, 0, 0, 4, 3, 2, 0, -2, -1, 2])
print adjA
print adjA.det()
# |adj A|
B=(adjA.det())^(1/(4-1))
# |A|^(n-1)=|adj A|
print B
C=(1/B)*adjA
# A^(-1)
print C
print C.inverse()
D=matrix(QQ, 4, 4,[1,
print D.adjoint()
# A
0 , 0 , 0, 0 , 4 ,-1 , 1, 0, -6,
# adj A
Answers :
- 155 -
2, -2, 0 , 1 , 0,
1])
adjA=
A=
A^(-1)=
adjA=
[ 2
0
0
0]
det(adjA)=
[
1
0
0
0]
[ 1
0
0
0]
[ 2
0
0
0]
[ 0
2
1
0]
8
[
0
1 1/2
0]
[ 0
4 -1
1]
[ 0
2
1
0]
[ 0
4
3
2]
det(A)=2
[
0
2 3/2
1]
[ 0 -6
2 -2]
[ 0
4
3
2]
[ 0 -2 –1
2]
[
0
-1 –1/2
1]
[ 0
0
[ 0 -2 –1
2]
1
1]
■
By
Problem P6
using
the
Cramer's
rule,
find
the
degree
3
polynomial
which passes through the following four points.
Solution
=>
Sage :
A=matrix(3, 3, [1, 1, 1, 8, 4, 2, 27, 9, 3])
b=vector([-2, -2, 6])
Ai=A.inverse()
print "x=", Ai*b
print
print "x=", A.solve_right(b)
x= (1, -2, -1)
=>
Problem P7
∴
■
Let the characteristic polynomial of matrix be . Find
eigenvalues of matrix .
Problem P8
Find the eigenspaces of
corresponding to each eigenvalue and
show that they are orthogonal to each other in the plane.
Solution
The eigenspace of corresponding to is = .
- 156 -
The eigenspace of corresponding to is = .
Choose any y = x and y = x from and , resp., then
<y , y > =< x , x >
= x x =
× × .
=> y and y are orthogonal.
∴ and are orthogonal to each other in the plane.
Problem P9
■
Find the characteristic polynomial of the following matrix. And find the roots
of the polynomial by using the Sage.
- 157 -
5
Chapter
Matrix Model
5.1 Lights out Game
5.2 Power Method
5.3 Linear Model (Google)
A mathematical model is a description of a system using mathematical
concepts and language. The process of developing a mathematical model
is called mathematical modeling. Mathematical models are used not only
in
the
natural sciences
(such
as
physics, biology, earth
science,
meteorology) and engineering disciplines (e.g. computer science, artificial
intelligence), but also in the social sciences (such as economics, psychology, sociology and political
science).
Physicists,
engineers,
statisticians,
operations
research
analysts
and
economists
use
mathematical models most extensively. A model may help to explain a system and to study the effects
of different components and to make predictions about behaviour.
Mathematical models can take many forms, such as, dynamical systems, statistical models, differential
equations, or game theoretic models. In this chapter, we illustrate linear models, which show how linear
algebra can be used to solve the real world problems, and review the content from previous chapters.
http://matrix.skku.ac.kr/knou-knowls/cla-week-7.pdf
- 158 -
5.1
*Lights Out Game
Reference video: http://youtu.be/_bS33Ifa29s
ractice site: http://matrix.skku.ac.kr/blackwhite2/blackwhite.html
http://matrix.skku.ac.kr/bljava/Test.html http://matrix.skku.ac.kr/Big-LA/Blackout.htm
The Blackout(Lights Out, Merlin's Magic square) Game, introduced
in the official homepage of the popular movie `A Beautiful Mind', is
a one-person strategy game that has recently gained popularity on
handheld
computing
devices.
An
animated
Macromedia
Flash
version of the game can be found from the official website of the
2001 movie `A Beautiful Mind'. In this section, we will introduce the
question-and-answer process by one student that led to further
development of this game, a purely linear algebraic solution and
corresponding software.
Background of The Lights Out Puzzle
In my recent linear algebra class, we discussed the movie 'A Beautiful Mind',
starring Russell Crowe as Nobel Laureate John F. Nash, Jr. (2001) specifically the
scene where Nash was playing the game “Go” with one of his friends. Some of my
students told me that they played 'the Blackout Puzzle' on the Korean official
website of the movie.
http://www.abeautifulmind.com/
Figure 1: Blackout Game
- 159 -
One of my students asked me “Can we find an optimal solution for the game?”
and, further,
“Is there any possibility that we can not win the game if the given
setting is fixed?” After a couple of days, one of my young students approached me
with a potential solution. Together, we constructed a mathematical model of the
Blackout Game and, utilizing this model, we were able to determine a solution to
the original questions. What we found was that we can, in fact, always win the
game, based on basic knowledge of linear algebra. At that time, the references
about this game were limited, so we developed our own methods; it is these
methods and results that will be explored in this section. Later, the following
website was set up to further explain the puzzle and solutions:
(http://link.springer.com/article/10.1007/BF02896407 and
http://matrix.skku.ac.kr/sglee/album/2004-ICME10SPF/ICME-10-July04.htm).
Introduction of blackout puzzle
A Blackout board is a grid of any size. Each square takes one of two colors black
or white. (The diagram on the website as in Figure 1 used blue and red.) The
player takes a turn by choosing any square, and the selected square and all
squares that share an edge with it change their colors. The object of the game is
to get all squares on the grid (tile) to be the same color - Black or White. When
you click on a tile, the highlighted tile icons will change or “flip'' from their
current state to the opposite state. Remember, the goal is to change all of the tile
icons to black (or white).
Figure 2: End of the Game (all squares having the same color)
How to solve any × game?
The following questions naturally come to mind:
[Q 1.]
Is there any possibility that we can not win the game if the given setting is
fixed?
[Q 2.]
Given a winning pair , how many solutions are there? When is the
solution unique?
- 160 -
[Q 3.]
Can we make a program to give us an optimal solution (shortest sequence
of moves)?
Note that here are × ×⋯ × patterns of × blackout grid. Among
these 512 patterns, there are × patterns such that we can win the game
with only one more click as follows. (Twice of the following basic 9 patterns as we
can change all initial colors.)
Adding some of the above to reach
or
(mod 2) is
the goal of the game.
We
checked
several
examples
through trial-and-error to convince
us
of
the
answer
to
the
first
question regarding any given initial
condition.
The figure 3 illustrates the shortest
sequence
of
moves
for
resolving
possible scenarios on a × board.
Our
Figure 3
approach
to
find
a
winning
strategy was to recognize these 18
patterns in Figure 3.
Then, we tried to make a mathematical model of this game that the only actions
we can perform are 9 clicks (since there are only 9 stones on the board).
assumed “the white stone ≡ 1 and black stone ≡ 0”. Then, we
We
classified effects
of each action as an addition of one vector (or × matrix). Any series of our
actions results in a linear combination of these vectors. We used modular 2
arithmetic to make the zero vector or all 1's vector (or matrix, resp.) to finish the
game.
, , ,
, , ,
- 161 -
, ,
Thus, we now have the 9 vectors as shown above (in fact, twice the amount of
them) to consider, which will end the game with just one more click.
Assume the
following initial condition, and the following 3 clicks make the entire board all
white. Suppose we have 5 black(blue) stones and 4 white(red) stones in
the
board as below.
Then, the above condition can be denoted by the following matrix
.
Now, we choose some of 9 positions to act on it. This can be represented by
thus our problem is to find some and such that
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
Final Goal
(2,3)
(3,1)
(3,2)
(3,3)
Initial
⇒
(1,1)
(1,2)
(1,3)
- 162 -
(2,1)
(2,2)
b
(2,3)
(3,1)
(3,2)
(3,3)
Final Goal
Initial
We now consider × matrix
as a × vector , then the above
linar system of equation can be written as
x b ⇒ where
We can use any computational tool such as Sage and obtain
.
Then we have a system of linear equations to find x . x b is
a given (condition) matrix and j is a vector of all 1’s. Then
RREF and
rank . So the columns (rows) are linearly independent, and the system has a
unique solution x . Furthermore. this entire
process
can
be
done
in
Modular
2
arithmetic
and
x
=
. We only need 0 and 1 because clicking times
of one stone is the same as clicking once, and
same as doing nothing.
clickings of one stone is the
So, our answer for x b, which is a real optimal
winning strategy vector (matrix) x ≡ mod , is
- 163 -
.
This shows that if we click on positions , we will get all white
stones on the board with only 4 clicks. With this idea, one of my students made a
computer program in C++ based on this algorithm to determine an optimal winning
strategy. Let
x′ ≡ x mod 2. Then x′ is a real optimal winning strategy
vector
(matrix) which can be deduced from x. Now, entries of x′ are all
0 or 1 as is in
real game situation and we can always find a (0,1) matrix as
a real optimal
winning strategy vector(matrix). We can download this program and run it from
http://matrix.skku.ac.kr/sglee/blackout_win.exe. This software also
verified our
conjecture and showed the proof was valid.
In the following Figure, the command “(Wizard)” tells us “1 3 4 8,” which
indicates which 4 stones we have to click to win. The number ''4'' shows we won
with 4 clicks (MOVE).
Teachers often think of "teaching" as a one-sided process, but this experience
shows that teachers and creative students can work together to solve problems in
a mathematically-stimulating, mutually beneficial way. This process can be adapted
to resolve other real world problems using basic mathematical knowledge.
After answering our posed questions for the Blackout Game, we looked toward
finding a relationship between the Blackout Game and automata theory. We started
to introduce the concept of sigma-game and find the optimal strategy to win the
Blackout game, as well as a condition to determine the irreversibility of this game
in
larger size boards- up to × . We also verify our algorithm within a
program made in C++.
The sigma-game is played on a directed graph . We suppose that the vertices
- 164 -
of can have one of two different states, which are designated as 0 or 1. A
configuration is an assignment of states to all the vertices, and a move in the
game consists of the player's picking a vertex. The Blackout Game emulates the
sigma-game on the nine point directed graph.
We could classify the reversibility as a direct calculation of the × block
tridiagonal matrix of the blackout game of size . In fact, for ≤ , there are
irreversible
cases
when
.
Using
Mathematica
and
our
eigenvalue method, we can easily show the irreversibility. We could find a way to
reach the goal even for some irreversible case if we give a restriction on the
initial condition b. This complete our generalization of the Blackout game from
× board to the full size Go board. Finally, the following Figure from our
software shows our answer is accurate for larger size boards.
[A
software of Blackout game on different
sizes with 3 colors]
We made a mathematical model from the well-known Blackout game. Surprisingly,
it turned out to be a pure linear algebra problem of finding the optimal solution of
the game, and we generalized it to the full size Go board. We gave a mathematical
proof
and
algorithm
to
solve
it
which
can
be
extended
to
the
study
sigma-automata theory.
More details on the blackout game can be obtained from the following links.
- 165 -
of
http://matrix.skku.ac.kr/sglee/blackout_win.exe
http://matrix.skku.ac.kr/sglee/blackout_win.zip
http://matrix.skku.ac.kr/bljava/Test.html
http://matrix.skku.ac.kr/2012-mm/lectures-2012/A3-blackout-paper-ENG.pdf
http://matrix.skku.ac.kr/2009/2009-MathModeling/lectures/week12.pdf
[References]
Duk-Sun Kim, Sang-Gu Lee, Faqir M. Bhatti, Fibonacci sequences and the winning
conditions for the blackout game, International Journal of Contemporary Mathematical
Sciences, Vol. 5, 2010, no. 30, 1471 - 1485.
S.-G. Lee,
I.-P. Kim, J.-B. Park, and J.-M. Yang, Linear algebra algorithm for the optimal
solution in the Blackout game, J. of KSME Ser. A, 43 (2004), 87-96.
H.-S. Park, Go Game With Heuristic Function, Kyongpook Nat. Univ. Elec. Tech Jour. 15,
No.2 (1994),
35-43.
D. H. Pelletier, Merlin's magic square, Amer. Math. Monthly, 94 (1987) 143-150
K. Sutner, The sigma-game and cellular automata, Amer. Math. Monthly, 97 (1990), 24-34.
J. Uhl, W. Davis, Is the mathematics we do the mathematics we teach?, Contemporary
issues in mathematics education, 36 (1999), 67-74, Berkeley: MSRI Publications, Linear
algebraic approach on real sigma-game
JAVA program by Universal Studios and DreamWorks, Movie: A Beautiful Mind,
Blackout puzzle(2001), http://www.abeautifulmind.com/
3D Printing object 2
http://matrix.skku.ac.kr/2014-Album/2014-12-ICT-DIY/index.html
- 166 -
5.2
*Power Method
Reference video: http://youtu.be/CLxjkZuNJXw
Practice site: http://matrix.skku.ac.kr/2012-LAwithSage/interact/
http://math1.skku.ac.kr/home/pub/1516/
http://matrix.skku.ac.kr/SOCW-Math-Modelling.htm
In many matrix models of social behavior, the corresponding
maximum eigenvalue gives adequate information to predict the
model.
Hence, often finding the maximum eigenvalue is enough to
solve the corresponding problem. However, if the size of a matrix
is significantly large, even with a computer, it is difficult to find all
eigenvalues explicitly. Hence for a large scale matrix, we look at a
new method which finds only the maximum eigenvalue instead of
finding all the eigenvalues. This method, which harnesses the
power of the matrix, is called “Power Method”. The first goal of
this section is to explain how we can find the maximum eigenvalue
numerically. The second goal is to show how this can be applied to
the Google search engine.
We know that finding eigenvalues of an × real square matrix amounts to
finding roots of its characteristic polynomial of degree .
However, for large,
finding the roots of -th degree polynomial is not an easy task. Also finding
numerical roots for a large degree polynomial is sensitive to rounding off errors.
In this article,
we discuss
numerical methods to approximate a largest or
dominant eigenvalue of a matrix if exists. The dominant eigenvalues of a matrix
have several
applications in science, engineering and economics. Google uses it
for page ranking the web pages and Twitter uses it to recommend users
“WHO-TO-FOLLOW” (WTF).
- 167 -
Definition
Let … be the eigenvalues of an × real matrix . Then is
called a dominant eigenvalue of if for all … .
The eigenvector corresponding to the dominant eigenvalue is called
the dominant eigenvector.
[Remark] Note that not every matrix possesses a dominant eigenvalue.
For example,
matrices
do not have dominant eigenvalues.
Power Method
Let be an × real matrix.
The power method is a numerical approach to
find the dominant eigenvalue and the corresponding dominant eigenvector. We
assume the following two conditions:
l The dominant eigenvalue is a real number and its absolute value is strictly
greater than all the other eigenvalue.
l is diagonalizable, in particular has linearly independent eigenvectors.
Let have linearly independent eigenvectors x … x and eigenvalues are
orders as
≤ ≤ ⋯ ≤
Now we start with any nonzero vector x ∈ℝ and we continually multiply x by
A which generates a sequence of vectors x x … x , where
x x …
This implies x x ⋯ x .
Since has linearly independent eigenvectors x … x , there exist scalars
… such that
- 168 -
x x x ⋯ x
Multiplying both sides by , we get
x
x x x
x x
Since for all , the ration
. Thus as → ∞ ,
→ . Hence
→ x .
This leads to one of the very important method of finding the dominant eigenvalue
and eigenvector, namely the “Power Method”.
While applying the power method algorithm, we make sure that the largest
component each of x is unity, in this case the component of x x will
have largest component of absolute value of .
Power Method Algorithm
[Step 1] Select the a vector x having largest component as 1.
[Step 2] Set .
[Step 3] Find y x .
[Step 4] Define to be largest component in absolute value in the vector x .
[Step 5] Define x y .
[Step 6] Check if the convergence criteria is met. Otherwise
[Step 7] Set and go the the step 3.
Example
1
Let us find the dominant eigenvalue and the corresponding eigenvector
of
the matrix
, starting with x .
Iteration 1.
We have
x
x y x
.
- 169 -
Iteration 2.
x
y x
.
Iteration 3.
x
.
y x
Iteration 4.
x
y x
.
Iteration 5.
x
y x
.
Iteration 6.
x
.
y x
Continuing this way the 10th iterate is
x
y x
.
Clearly it means the dominant eigenvalue is approaching to 2 and the
corresponding dominant eigenvector is approaching to
.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
from numpy import argmax,argmin
A=matrix([[4,-5],[2,-3]])
x0=vector([0.0,1.0]) # Initial guess of eigenvector
maxit=20 # Maximum number of iterates
dig=8 # number of decimal places to be shown is dig-1
tol=0.0001
# Tolerance limit for difference of two consecutive eigenvectors
err=1 # Initialization of tolerance
i=0
while(i<=maxit and err>=tol):
y0=A*x0
ymod=y0.apply_map(abs)
imax=argmax(ymod)
c1=y0[imax]
x1=y0/c1
err=norm(x0-x1)
i=i+1
- 170 -
x0=x1
print "Iteration Number:", i-1
print
"y"+str(i-1)+"=",y0.n(digits=dig),
"c"+str(i-1)+"=",
c1.n(digits=dig),
"x"+str(i)+"=",x0.n(digits=dig)
print "n"
Iteration Number: 0
y0= (-5.0000000, -3.0000000) c0= -5.0000000 x1= (1.0000000, 0.60000000)
n
Iteration Number: 1
y1= (1.0000000, 0.20000000) c1= 1.0000000 x2= (1.0000000, 0.20000000)
n
Iteration Number: 2
y2= (3.0000000, 1.4000000) c2= 3.0000000 x3= (1.0000000, 0.46666667)
n
Iteration Number: 3
y3= (1.6666667, 0.60000000) c3= 1.6666667 x4= (1.0000000, 0.36000000)
n
Iteration Number: 4
y4= (2.2000000, 0.92000000) c4= 2.2000000 x5= (1.0000000, 0.41818182)
n
Iteration Number: 5
y5= (1.9090909, 0.74545455) c5= 1.9090909 x6= (1.0000000, 0.39047619)
n
Iteration Number: 6
y6= (2.0476190, 0.82857143) c6= 2.0476190 x7= (1.0000000, 0.40465116)
n
Iteration Number: 7
y7= (1.9767442, 0.78604651) c7= 1.9767442 x8= (1.0000000, 0.39764706)
n
Iteration Number: 8
y8= (2.0117647, 0.80705882) c8= 2.0117647 x9= (1.0000000, 0.40116959)
n
Iteration Number: 9
y9= (1.9941520, 0.79649123) c9= 1.9941520 x10= (1.0000000, 0.39941349)
n
Iteration Number: 10
y10= (2.0029326, 0.80175953) c10= 2.0029326 x11= (1.0000000, 0.40029283)
n
Iteration Number: 11
- 171 -
y11= (1.9985359, 0.79912152) c11= 1.9985359 x12= (1.0000000, 0.39985348)
n
Iteration Number: 12
y12= (2.0007326, 0.80043956) c12= 2.0007326 x13= (1.0000000, 0.40007323)
n
Iteration Number: 13
y13= (1.9996338, 0.79978030) c13= 1.9996338 x14= (1.0000000, 0.39996338)
n
Iteration Number: 14
y14= (2.0001831, 0.80010987) c14= 2.0001831 x15= (1.0000000, 0.40001831)
n
Example
2
■
Let us find the dominant eigenvalue and the corresponding eigenvector
of
the matrix
, starting with x .
Iteration 1. We have
x y x x
Iteration 2. y x x y .
Iteration 3. y x x y .
Iteration 4. y x x y .
Continuing this, the 10th iterate is given by
y x x y .
Clearly it means the dominant eigenvalue is approaching to 4 and the
corresponding dominant eigenvector is approaching to
Sage
.
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
from numpy import argmax, argmin
A=matrix([[1,-3,3],[3, -5, 3],[6,-6,4]])
x0=vector([1.0,1.0,1.0]) ## Initial guess
- 172 -
maxit=20 # Maximum number of iterates
dig=8 # number of decimal places to be shown is dig-1
tol=0.00001
# Tolerance limit for difference of two consecutive eigenvectors
err=1 # Initialization of tolerance
i=0
while(i<=n and err>=tol):
y0=A*x0
ymod=y0.apply_map(abs)
imax=argmax(ymod)
c1=y0[imax]
x1=y0/c1
err=norm(x0-x1)
i=i+1
x0=x1
print "Iteration Number:", i-1
print "y"+str(i-1)+"=",y0.n(digits=dig), "
c"+str(i-1)+"=", c1.n(digits=dig)
print "x"+str(i)+"=",x0.n(digits=dig)
print "n"
[Remark]
The rate convergence of the power method is determined by the ration
.
Smaller is the ratio better is the convergence.
Example
3
Consider matrices
.
Starting with arbitrary vector x observe that for convergence is
obtained in fewer iterates, for the convergence requires many more
iterates where as for there is no convergence.
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
Mat=['A','B','C']
from numpy import argmax,argmin
@interact
def
_QRMethod(A1=input_box(default='[[1,2],[3,4]]',
- 173 -
type
=
str,
label
=
'A'),B1=input_box(default='[[1.7,-0.4],[0.15,2.2]]',
'B'),C1=input_box(default='[[1,2],[-3,4]]',
type
type
=
=
str,
str,
label
label
'C'),example=selector(Mat,buttons=True,label='Choose
Matrix'),maxit=slider(1,
500,
1,
default=100,
label="Maximum
=
=
the
no.
of
iterations"),tol=input_box(label="Tolerance",default=0.001),v=
input_box([0.1,1.0])):
if(example=='A'):
A1=sage_eval(A1)
A=matrix(A1)
elif(example=='B'):
B1=sage_eval(B1)
A=matrix(B1)
elif(example=='C'):
C1=sage_eval(C1)
A=matrix(C1)
x0=vector(v)
html('A=%s,~~ x_0=%s'%(latex(A),latex(x0)))
#html('x_0=%s'%latex(x0))
#x0=vector([0.0,1.0])
i=0
err=1
while(i<=maxit and err>=tol):
y0=A*x0
ymod=y0.apply_map(abs)
imax=argmax(ymod)
c1=y0[imax]
x1=y0/c1
err=norm(x0-x1)
print "Iteration Number:", i+1
html('y_i=%s,~~ c_i=%s~~ x_i=%s'%(latex(y0),latex(c1),latex(x0)))
i=i+1
x0=x1
if(i==maxit+1):
print 'Convergence is not achieved'
else:
print 'The number iteration required for tolerance=',tol,'is:',i
[Remark] nonzero smallest eigenvalue
- 174 -
To find the non zero smallest eigenvalue of a matrix , we can find the dominant
eigenvalues of .
[Remark] shifted power method
Note that if is an eigenvalue of then is an eigenvalue of . The
rate of convergence of a power method can be significantly improved by using a
shifted matrix rather than in the power method. This method is called
the shifted power method.
Inverse Power Method
In case, a reasonably “good approximation” of an eigenvalue is known, then we
can use the “inverse power method” to find an eigenvalue and the corresponding
eigenvector.
Let be an approximation to an eigenvalue such that ≪ for all
… That is, is much closer to than to the other eigenvalues. Then we
have the following algorithm
Inverse Power Method Algorithm
[Step 1] Select an initial estimate sufficiently close to .
[Step 2] Select the a vector x whose largest entry is 1.
[Step 3] Set .
[Step 4] Solve y x for y .
[Step 5] Define to be largest component in absolute value in the vector x .
[Step 6] Find .
[Step 7] Define x y .
[Step 8] Check if the convergence criteria is met. Otherwise
[Step 9] Set and go the the step 4.
In the above algorithm converges to and x converges to the corresponding
eigenvector.
The inverse power method is also know as inverse iteration with shift method.
- 175 -
[Remark]
Note that the inverse power method is nothing but the power method applied to
the matrix .
Example
4
This is why the name.
Consider the matrix
. Suppose is an estimate
of an eigenvalue of . Apply the inverse power method to approximate
an eigenvalue of starting with x .
Iteration 1. y x
x y .
Iteration 2. y x –
x y .
Iteration 3. y x
x y .
Continuing this way, we have Iteration 10.
y x
x .
This mean an approximate eigenvalue is 2 and the corresponding
eigenvector converges to
Sage
.
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
from numpy import argmax, argmin
A=matrix([[10,-8,-4],[-8,13,5],[-4,4,4]])
Id=identity_matrix(3)
x0=vector([1.0,1.0,1.0]) ## Initial guess
maxit=20 # Maximum number of iterates
- 176 -
dig=8 # number of decimal places to be shown is dig-1
tol=0.00001
# Tolerance limit for difference of two consecutive eigenvectors
err=1 # Initialization of tolerance
sig=1.9 # Initial Shifting number
i=0
while(i<=n and err>=tol):
y0=(A-sig*Id).inverse()*x0
ymod=y0.apply_map(abs)
imax=argmax(ymod)
c1=y0[imax]
d1=sig+1/c1
x1=y0/c1
print "Iteration Number:", i+1
print "y"+str(i)+"=",y0.n(digits=dig), "d"+str(i)+"=", d1.n(digits=dig)
print "x"+str(i+1)+"=",x0.n(digits=dig)
print "n"
i=i+1
x0=x1
[Remark]
The advantage of the inverse power method with shift is that it can be adopted to
find any eigenvalue of a matrix, instead of the extreme ones. However, in order to
compute a particular eigenvalue, we need to have an initial approximation that of
that eigenvalue.
Rayleigh Quotient
The Rayleigh quotient of a
non zero vectors x with respect of a matrix is
defines as
x x
x
xx
If x is an eigenvector with respect to the eigenvalue , then x .
In general, for an arbitrary x, x is value that minimizes the function
- 177 -
∥ x x∥ over real number .
The inverse power method can be significantly improved if we drop the restriction
that the shift value remains constant in all the iterates.
Each
iteration
in
the
inverse
power
method
gives
and
approximation
of
eigenvector, given an estimation of eigenvalue. On the other hand, the Rayleigh
quotient gives an approximate eigenvalue, given as estimate of an eigenvector.
Combining the two concepts together, we get a new variation in the inverse power
algorithm in which the shift value is updated in each iterate and it becomes the
Rayleigh quotient of the eigenvector estimates. This method is called the Rayleigh
quotient iteration method (RQI).
Rayleigh Quotient Iteration Algorithm
[Step 1] Select the a vector x with ∥ x ∥ .
[Step 2] Define x x x .
[Step 3] Set .
[Step 4] Define x x x .
[Step 5] Solve y x for y .
y
[Step 6] Define x
.
∥ y ∥
[Step 7] Check if the convergence criteria is met. Otherwise
[Step 8] Set and go the the step 4.
One of the main advantage of the RQI is that it converges much faster than
power method and inverse power method. However, a very significant disadvantage
of RQI is that its convergence is not always guaranteed except when the matrix is
symmetric.
Example
5
Apply the Rayleigh quotient iteration method to find an eigenvalue of the
matrix
starting
with
initial
x .
Iteration 1. x and normalize x to get
- 178 -
approximate
vector
x
.
x
∥ x ∥
Now x x x .
Solving y x for y we get, y
.
y
Hence x
.
∥ y ∥
Iteration 2. x x x .
Solving y x for y we get, y .
y
.
Hence x
∥ y ∥
Iteration 3. x x x .
Solving y x for y and then we have
y
x
.
∥ y ∥
Iteration 4. and x
.
Clearly, in 4 iterates we are getting reasonably accurate eigenvalue
and the corresponding eigenvector
.
Other Eigenvalues
We can use different initial vectors x to get a different eigenvalues
and the corresponding eigenvectors.
For example if we use x
, then after 4 iterates we have
, x
- 179 -
.
If we use x , then after 4 iterates we have
,
Sage
x
.
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix([[10,-8,-4],[-8,13,5],[-4,5,4]])
Id=identity_matrix(3)
x0=vector([1.5,-2.5,5])
#x0=vector([1.0,0.0,0.0])
#x0=vector([1.0,-1,-1])
x0=x0/norm(x0)
maxit=20 # Maximum number of iterates
dig=8 # number of decimal places to be shown is dig-1
tol=0.00001
# Tolerance limit for difference of two consecutive eigenvectors
err=1 # Initialization of tolerance
i=0
while(i<=n and err>=tol):
lam0=x0.dot_product(A*x0)
y0=(A-lam0*Id).inverse()*x0
x1=y0/norm(y0)
print "Iteration Number:", i+1
print "y"+str(i)+"=",y0.n(digits=dig), "lambda"+str(i)+"=", lam0.n(digits=dig)
print "x"+str(i+1)+"=",x0.n(digits=dig)
print "n"
i=i+1
x0=x1
QR Method
The QR method for finding eigenvalues and eigenvectors is a simultaneous
iteration method that allows us to find all eigenvalues and eigenvectors of a real,
symmetric full rank matrix at once.
- 180 -
The algorithm is simple:
l We start with .
l Set and find the QR factorization .
l Let .
The sequence has the following properties: for each ,
is orthogonally
equivalent to and hence is orthogonally equivalent to the original matrix .
since .
Similarly,
.
It can be shown that the sequence converges (under certain conditions) to an
upper triangular matrix or quasi-triangular matrix. In particular, the diagonal
entries of are eigenvalues.
If we define
⋯
Then the columns of
converges to unit eigenvectors of .
Similarly we can define
⋯ .
[Remark]
Example
6
(i)
and
Consider
(ii)
.
. The actual eigenvalues of are
13.1804689044, 3.56330346867, 1.25622762694.
Now we apply the QR-method.
Iteration 1. . We have
- 181 -
.
.
.
Iteration 2. . We have
.
.
.
Iteration 3. . We have
.
.
.
Continuing this iterations, in 20th iterate we have .
We have
× ×
×
× .
× ×
.
.
×
×
×
Clearly, diagonal entries of are close to actual eigenvalues of .
- 182 -
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080
A=matrix(RDF, [[10,3,4],[3,5,1],[4,2,3]])
print "The actual eivenvalues of A are"
ev=A.eigenvalues()
show(ev)
n=10
for i in range(n):
Q1,R1=A.QR()
print "Iteration Number", i
print "The matrix Q"+str(i), "is"
show(Q1)
print "The matrix R"+str(i), "is"
show(R1)
A1=R1*Q1
print "The matrix A"+str(i), "is"
show(A1)
A=A1
[Remark]
QR method mentioned above usually is very expensive. This is why usually,
symmetric matrices are first converted to tridiagonal matrix and then we apply QR
method. For non-symmetric matrix, we convert it to an upper Hessenberg matric
and then apply QR method.
http://www.prenhall.com/bretscher1e/html/proj10.html
- 183 -
*Linear Model
5.3
Reference video: http://youtu.be/CLxjkZuNJXw
Practice site: http://matrix.skku.ac.kr/2012-LAwithSage/interact/
http://math1.skku.ac.kr/home/pub/1516/
http://matrix.skku.ac.kr/SOCW-Math-Modelling.htm
(1) Linear Algebra behind Google
http://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf
by Kurt Bryan and Tanya Leise
Google’s success derives in large part from its PageRank algorithm, which ranks
the importance of webpages according to an eigenvector of a weighted link matrix.
Analysis of the PageRank formula provides a wonderful applied topic for a linear
algebra course. Instructors may assign this article as a project to students, or
spend one or two lectures presenting the material with assigned homework from
the exercises. This material also complements the discussion of Markov chains in
matrix algebra. Maple and Mathematica files supporting this material can be found
in http://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf.
A newsletter article '“Linear Algebra and Google Search Engine” - Pagerank
algorithm' in Korean also can be found in
http://matrix.skku.ac.kr/2012-e-Books/KMS-News-LA-Google-SGLee.pdf.
- 184 -
(2) Sage Matrix Calculator for Linear Algebra
In this section, we introduce a matrix calculator. By utilizing a free
open source tool Sage, one can intuitively understand almost all
concepts in linear algebra. Also, one can study with visualization
and large scale computation. Moreover, one can easily change and
expand the size of a matrix.
Sage Matrix Calculator
http://matrix.skku.ac.kr/2014-Album/MC.html
For over 20 years, the issue of using an adequate CAS tool in the teaching and
learning of linear algebra has been raised constantly. A variety of CAS tools were
introduced in many linear algebra textbooks; however, in Korea, due to some
realistic problems, these tools have not been introduced in the class and the
theoretical aspect of linear algebra has been the primary focus in Linear Algebra
courses.
In this section, we suggest Sage as an alternative for CAS tools
to overcome tthe
problems mentioned above. As well, we introduce the extensive linear algebra
content and a matrix calculator that was developed with Sage. Taking advantage of
these novel tools, almost all concepts of linear algebra can be easily covered, and
the size of matrices can be expanded without difficulty.
The Sage Matrix Calculator uses the Sage Cell server. As shown in the following
picture, it can do not only basic operations, such as matrix addition, subtraction,
multiplication, scalar multiplication, but also can find determinant, rank, trace,
nullity,
eigenvalues,
characteristic
equation,
inverse
matrix,
adjoint
matrix,
transpose of matrix, and conjugate transpose of a matrix. Also, unlike most
web-based
open
matrix
calculators,
it
can
QR-decomposition, which are quite essential to
perform
LU,
SVD,
and
a well-rounded linear algebra
education. By selecting the column size as 1, it can perform vector operations,
such as inner product, cross product, and norm. As well, by using the column
vectors of a matrix, it can perform Gram-Schmidt orthogonal process, and as a
result, one can find the basis of a vector space generated by the matrix. As this
matrix calculator can cover complex numbers, while many other matrix calculators
can handle only real or rational numbers, it can solve almost all problems in
linear algebra. In order to use the Sage matrix calculator, one needs only to
connect to the given URL, or simply copy the codes from the given URL and paste
- 185 -
them to other Sage Cell server or a general Sage server's worksheet. Once it
executed, decide the size of the matrix, enter the elements of the matrix, and then
perform the desired matrix operations.
- 186 -
Visualization of Linear Algebra Concepts with GeoGebra
http://www.geogebratube.org/student/b121550
Vector addition
Sclar multiplication
L. S. of Equations
Matrix product
Areas
Equations
Curve Fitting
Linear Transformation
Projection
LT (Shear)
대칭변환
LT(similarity)
Triangles
Projections
Least Square solution
http://www.geogebratube.org/student/m9493
http://www.geogebratube.org/student/m9494
http://www.geogebratube.org/student/m9704
http://www.geogebratube.org/student/m12831
http://www.geogebratube.org/student/m9497
http://www.geogebratube.org/student/m9504
http://www.geogebratube.org/student/m9911
http://www.geogebratube.org/student/m9702
http://www.geogebratube.org/student/m9910
http://www.geogebratube.org/student/m9912
http://www.geogebratube.org/student/m9703
http://www.geogebratube.org/student/m9705
http://www.geogebratube.org/student/11568
http://www.geogebratube.org/student/m9503
http://www.geogebratube.org/student/m12933
http://matrix.skku.ac.kr/2012-Album/CLA-GeoGebra-Dynamic-Visual.htm
[References]
Sang-Gu LEE*, Kyung-Won KIM and Jae Hwa LEE,
Sage matrix calculator and full
Sage contents for linear algebra, Korean J. Math. 20 (2013), No. 4, pp.
- 187 -
503-521.
6
Chapter
Linear Transformations
6.1 Matrix as a Function (Transformation)
6.2 Geometric Meaning of Linear Transformations
6.3 Kernel and Range
6.4 Composition of Linear Transformations and Invertibility
6.5*Computer Graphics with Sage
Exercises
So far, we have considered matrix mainly as a coefficient
matrix from systems of linear equations. Now, we consider
matrix as a function.
We
have
observed
that
the
set
of
vectors and
two
operations reborn as an algebraic structure, namely a vector space. Matrix will be
reborn as a linear transformation, which is a function that preserves the
operations in a vector space. And linear transformations are used for noise
filtering in signal processing and analysis in engineering processes.
We show a linear transformation from -dimensional space ℝ to -dimensional
space ℝ can be expressed as a × matrix . We shall also look at geometric
meaning of linear transformations from ℝ to ℝ and applications in computer
graphics.
- 188 -
6.1
Matrix as a Function (Transformation)
Reference video: http://youtu.be/YF6-ENHfI6E, http://youtu.be/Yr23NRSpSoM
Practice site: http://matrix.skku.ac.kr/knou-knowls/cla-week-8-Sec-6-1.html
Matrix can be considered as a special function with linearity property.
Such a function play an important role in science and various areas in
daily life, such as mathematics, physics, engineering control theory,
image processing, sound signal, and computer graphics.
What is a Transformation?
Definition
A function, whose input and output are both vectors, is called a
transformation. For a transformation ℝ → ℝ , w x is called
an image of x by , and x is called a pre-image of w.
As a special case of transformations, x x, for × matrix
and x∈ℝ ,
ℝ → ℝ is called a matrix transformation.
x′ x
x
- 189 -
[Remark] Computer simulation
[Matrix transformation)
http://www.geogebratube.org/student/b73259#material/22419
Definition
If
a
transformation
ℝ→ ℝ
ℝ
from
to
following two conditions for any vectors u v ∈ℝ
ℝ ,
satisfies
the
and for any scalar
∈ℝ ,
(1) u v u v
(2) u u
then is called a linear transformation from ℝ to ℝ .
a linear transformation from ℝ
to ℝ
ℝ →ℝ
itself,
Especially,
is called a
linear operator on ℝ .
Example
1
Show that is a linear transformation if we define ℝ → ℝ , for any
vector x in ℝ , as follows
x
- 190 -
Solution
For any two vectors u , v in ℝ and for any scalar ∈ℝ ,
(1) u v
u v.
u .
(2) u
Therefore, by definition, is a linear transformation from ℝ to ℝ .
Example
2
Let ℝ → ℝ ,
.
Show
that
T
is
a
■
linear
transformation.
Solution
For any two vectors v and
v in ℝ and for any scalar ∈ℝ ,
(1) v v
v v .
(2) v
v
Therefore, T is a linear transformation.
■
This type of linear transformation is called orthogonal projection on -plane.
Example
3
If we define ℝ → ℝ
as follows, show that is not a linear
transformation.
- 191 -
.
Solution
For any two vectors, v , v in ℝ ,
v v
However, v v
.
.
Hence v v ≠ v v .
Therefore, we conclude that is not a linear transformation.
■
[Remark] Special Linear Transformations
zero transformation: For any v∈ℝ , if we define ℝ → ℝ as v ,
then is a linear transformation. This is called a zero transformation.
identity transformation: For any v∈ℝ , if we define ℝ → ℝ as v v,
then is a linear transformation. This is called an identity transformation.
matrix transformation: For any × matrix and for any vector x in ℝ ,
if we define x x, then is a linear transformation from ℝ to ℝ .
This is called a matrix transformation.
Example
4
Let ℝ → ℝ is defined as follows. Show is a linear transformation.
Solution
in ℝ and for any scalar ∈ℝ ,
u
v
For any two vectors
- 192 -
u v
(1)
u v
u
u
(2)
■
and hence, is a linear transformation.
Example
5
A linear transformation from
Example
1
,
is
.
is a matrix transformation for a matrix
Theorem
■
6.1.1 [Properties of Linear Transformation]
If ℝ → ℝ
is a linear transformation, then it satisfies the
following conditions:
(1) .
(2) v v.
(3) u v u v .
Proof (1) Since ∀ v∈ v , v v .
(2) v v v v
(3) u v u v u v u v
Each linear transformation
■
from ℝ to ℝ can be expressed as a matrix
transformation.
Let ℝ → ℝ be any linear transformation. For elementary unit vectors,
- 193 -
e e … e of ℝ and for any x∈ℝ , we have
x e e ⋯ e
⋮
and as e , e , … , e are × matrix, we can write them as
e e … e .
⋮
⋮
⋮
Therefore any linear transformation ℝ → ℝ can be expressed as
x e e ⋯ e
⋯
⋯
⋯
.
⋮
⋮
⋮
⋮
⋯
(1)
Now let be an × matrix which has e , e , … , e as it's columns.
⋯
⋯
e e ⋯ e
⋮ ⋮ ⋯ ⋮
⋯
Then,
⋯
⋯
x
⋮ ⋮ ⋮
⋯
x.
⋮
⋮
The above matrix × is called the standard matrix of and is denoted
by . Hence, the standard matrix of the linear transformation given by (1) can
be found easily from the column vectors by substituting the elementary unit
vectors to in that order.
- 194 -
Theorem
6.1.2 [Standard Matrix]
If ℝ → ℝ is a linear transformation, then the standard matrix
of T has the following relation for x∈ℝ .
∀ x∈ℝ
x x,
where e e ⋯ e .
Example
6
ℝ → ℝ ,
For a linear transformation
, by
using the standard matrix of , rewrite it as x x.
Solution
Let
, which columns are e , then x x as
x
Ÿ
x.
□
http://matrix.skku.ac.kr/RPG_English/6-MA-standard-matrix.html
Sage
http://sage.skku.edu
x, y, z = var('x y z')
h(x, y, z) = [x+2*y, -x-y, z, x+z]
T = linear_transformation(QQ^3, QQ^4, h)
# define linear transformation,
# here scalar is rational numbers
C = T.matrix(side='right')
# standard matrix
x0 = vector(QQ, [2, -3, 3])
print C
print T.domain()
# domain
print T.codomain()
# codomain
print T(x0)
# image
print C*x0
# product of standard matrix and a vector
- 195 -
[ 1
2
0]
[-1 -1
0]
[ 0
0
1]
[ 1
0
1]
Vector space of dimension 3 over Rational Field
Vector space of dimension 4 over Rational Field
(-4, 1, 3, 5)
(-4, 1, 3, 5)
■
http://en.wikiquote.org/wiki/Georg_Cantor
Georg Ferdinand Ludwig Philipp Cantor (1845–1918)
“The essence of mathematics lies
entirely in its freedom.”
most famous as the creator of set
theory, and of
Cantor's
theorem
which implies the existence of an
"infinity of infinities."
- 196 -
6.2
Geometric Meaning of Linear Transformations
Reference video: http://http://youtu.be/cgySDj-OVlM, http://youtu.be/12WP-cb6Ymc
Practice site: http://matrix.skku.ac.kr/knou-knowls/cla-week-8-Sec-6-2.html
In this section, we study the geometric meaning of linear transformations.
For a given image, continuous showing of series of images with small
variations makes a motion picture. Linear transformation can be applied to
computer graphics and numerical algorithms, and it is an essential tool
for many areas such as animation.
Linear Transformation from ℝ to ℝ
A linear transformation ℝ → ℝ defined by
moves a
vector
to an another vector
.
Example
1
[rotation, symmetry, orthogonal projection] We illustrate a few linear
transformations on ℝ .
(1) ℝ → ℝ is a linear transformation which rotates a vector in
ℝ counterclockwise by around the origin.
cos
sin
sin
cos
(2) An orthogonal projection ℝ → ℝ on -axis is a linear
transformation.
x
- 197 -
(3) A symmetric movement ℝ → ℝ around -axis is a linear
transformation.
■
x
Ÿ
Example
2
http://matrix.skku.ac.kr/sglee/LT/11.swf
Find the standard matrix for a linear transformation which moves a
point in ℝ to a symmetric image around the given line.
(2) line
(1) -axis
Solution
Symmetric (linear) transformation around -axis and the line are
given in the following figures.
,
,
Ÿ
http://matrix.skku.ac.kr/sglee/LT/22.swf
Ÿ
http://matrix.skku.ac.kr/sglee/LT/44.swf
- 198 -
.
■
[Remark] Simulation
[linear transformation]
http://www.geogebratube.org/student/m9703
[symmetric transformations and orthogonal projection transformations]
http://www.geogebratube.org/student/m9910
[rotation]
Example
3
http://www.geogebratube.org/student/m9702
Linear transformation ℝ → ℝ which moves any vector x
in ℝ to a symmetric image around a line, which passes through the
origin with angle between the -axis and the line, can be expressed
by the following matrix presentation e e .
e e
In
Example
Example
4
3 , if ,
cos
sin
cos
cos
sin
sin cos
sin
■
, i.e. .
As shown in the picture, let us define an orthogonal projection as a
linear transformation (linear operator) ℝ → ℝ
vector x in ℝ
which maps any
to the orthogonal projection on a line, which passes
- 199 -
through the origin with angle between the -axis and the line.
Let us denote the standard matrix correspond to is .
x x x x
(the same direction with a half length)
x x x x x x
cos
sin
cos
sincos
■
sincos
sin
cos
sin
In
Example
4 , if ,
is a projection onto the x-axis.
[Remark] shear transformations (computer simulation)
(1) →
: shear transformation along the -axis with scale
(2) →
: shear transformation along the -axis with scale
Ÿ
http://www.geogebratube.org/student/m9912
- 200 -
Definition
,
A linear transformation ℝ → ℝ , which preserve the magnitude
(or length of a vector), x x , is called Euclidean isometry.
Theorem
For
a
6.2.1
linear
operator
ℝ→ ℝ ,
the
following
statements
are
equivalent:
(1) x x , x∈ℝ (isometry).
(2) x ⋅ y x⋅ y, x y∈ℝ (preserve the inner product).
Definition
For a square matrix , if then is called orthogonal matrix.
Example
5
cos sin
For any real number ,
is orthogonal matrix, and
sin
cos
cos sin
.
sin cos
Example
6
■
Verify the following matrices are orthogonal matrix.
,
Solution
Verify , by using the Sage.
- 201 -
□
Ÿ
http://matrix.skku.ac.kr/RPG_English/6-TF-orthogonal-matrix.html
Sage
http://sage.skku.edu
A=matrix(QQ, 2, 2, [3/5, -4/5, 4/5, 3/5])
B=matrix(3, 3, [1/sqrt(3), 1/sqrt(2), 1/sqrt(6),
1/sqrt(3), -1/sqrt(2), 1/sqrt(6),
1/sqrt(3), 0, -2/sqrt(6)])
print A.transpose()*A
# confirm the orthogonal matrix
print
print B.transpose()*B
[1 0]
[1 0 0]
[0 1]
[0 1 0]
[0 0 1]
Theorem
■
6.2.2
For any × matrix , the following statements are hold:
(1) The transpose of an orthogonal matrix is an orthogonal matrix.
(2) The inverse of an orthogonal matrix is an orthogonal matrix.
(3) The product of orthogonal matrices is an orthogonal matrix.
(4) If is an orthogonal matrix, then det or .
Proof (1) and (2) are left as an exercise to the reader.
(3) If and , then
and hence is an orthogonal matrix.
(4) Observe that det det det det det
∴ det or .
- 202 -
■
Theorem
6.2.3
For any × matrix , the following statements are equivalent:
(1) is an orthogonal matrix.
(2) x x, x∈ℝ .
(3) x⋅ y x⋅ y, x y∈ℝ .
(4) The row vectors of are orthonormal.
(5) The column vectors of are orthonormal.
Proof (1) ⇒ (2):
x x⋅ x x x x x x x
x x x x x x x⋅ x x
(2) ⇒ (3):
∥ x y∥ ∥ x y∥ ∥ x∥ x∙ y ∥ y∥
∥x∥ x∙ y ∥y∥
and
∥ x y∥ ∥x y∥ ∥x∥ x∙ y ∥y∥ .
Hence x⋅ y x⋅ y .
(3) ⇒ (1):
∀ , e e e⋅ e e e e e
⇒
≠
∴
We skip the detailed proof of (4) and (5) as we can get the result easily from
the definition of the orthogonal matrix, , and (1).
[The headquarter of American Mathematical Society, Providence, RI, USA]
http://www.ams.org
- 203 -
■
Kernel and Range
6.3
Reference video: http://youtu.be/9YciT9Bb2B0, http://youtu.be/H-P4lDgruCc
Practice site: http://matrix.skku.ac.kr/knou-knowls/cla-week-8-Sec-6-3.html
We will show that the subset of a domain ℝ , which maps to zero
vector by a linear transformation, becomes a subspace. We will also
show the set of images under any linear transformation forms a
subspace in the co-domain. Finally, we introduce the concept of
isomorphism.
Definition
Let ℝ → ℝ is a linear transformation. The set of all vectors in
ℝ , whose image becomes by , is called kernel of and is
denoted by ker . That is, ker v∈ℝ v .
Example
1
Find
the
ker
for
a
linear
transformation
ℝ → ℝ where
.
Solution
ker ∈ℝ ｜ ∈ℝ ｜ .
Example
2
Find
the
ker
for
a
linear
.
Solution
For any x ∈ℝ ,
- 204 -
transformation
ℝ→ ℝ ,
■
where
⇔
,
and hence, ker ∈ℝ .
■
Definition
For a transformation ℝ → ℝ , if u v ⇒
u v, then it
is called one-to-one (injective).
Definition
For a transformation ℝ → ℝ , if there exist v∈ℝ for any
given w∈ℝ , such that v w,
Theorem
Let
then it is called onto (surjective).
6.3.1
ℝ and ℝ
are
vector
spaces
and
ℝ→ ℝ
is
a
linear
transformation. Then is one-to-one if and only if ker .
Proof ⇒ As ∀ v∈ker , v and is one-to-one,
⇒
v
∴
ker
⇐ v v ⇒ v v v v
⇒ v v ∈ ker ⇒ v v
∴
Example
3
■
is one-to-one.
Let us define a linear transformation ℝ → ℝ as . Is
an one-to-one?
Solution
As
ker x∈ℝ x ,
the
only
element in this set is . Hence ker , and is
one-to-one.
■
- 205 -
Example
4
Is a linear transformation ℝ → ℝ one-to-one if it is defined as
?
Solution
Since ⇔
⇔
,
the system of linear equations has infinitely many solutions. Hence,
ker ≠ and by theorem 6.3.1, is not one-to-one.
Sage
□
http://sage.skku.edu
① verify linear transformations' one-to-one
U = QQ^3
# vector space
x, y, z = var('x, y, z')
h(x, y, z) = [x+2*y-z, y+z, x+y-2*z]
T = linear_transformation(U, U, h)
# generate a linear transformation
print T.is_injective()
# check the one-to-one
False
② Find Kernel of linear transformation
T.kernel()
# verify by finding kernel
Vector space of degree 3 and dimension 1 over Rational Field
Basis matrix:
[
1 –1/3
1/3]
# kernel = span( [1, -1/3, 1/3] ).
■
Let be an × matrix. If we define a linear transformation ℝ → ℝ as
x x, then ker is a solution space of the system of linear equations
x .
- 206 -
Theorem
Let
6.3.2
ℝ ℝ
are
vector
spaces
and
ℝ→ ℝ
is
a
linear
transformation. Then ker is a subspace of ℝ . Hence ker is called
kernel (subspace).
Example
5
Find the kernel of a
.
Sage
http://sage.skku.edu
A = matrix(2, 2, [1, 1, 1, -1])
print A.right_kernel()
# kernel of A
Free module of degree 2 and rank 0 over Integer Ring
Echelon basis matrix:
[]
Definition
# kernel has only 0.
■
[Isomorphism]
For a linear transformation ℝ → ℝ , the set of all v for
v ∈ℝ , is called range of and is denoted by Im .
That is,
Im v∈ℝ v∈ℝ ⊂ ℝ .
Especially, if Im ℝ then is called surjective or onto. If a linear
transformation is one-to-one and onto, then and is called
an isomorphism from ℝ to ℝ .
Example
6
Find the range of the linear transformation .
Solution
Im ∈ℝ ∈ℝ ∈ .
Note that, Im ≠ ℝ ⇒ is not surjective.
- 207 -
■
is not isomorphism as it is not surjective.
7
Example
Let ∈ℝ and ∈ℝ . It
is easy to see that both and are subspaces of . If we define
→ as following linear transformation,
then is both one-to-one and onto, and hence it is isomorphism.
Theorem
■
6.3.3
For a linear transformation ℝ → ℝ , Im is a subspace of ℝ .
∀ w w ∈ Im , ∃ v v ∈ℝ ∋ v w v w
Proof
⇒ w w v v v v
⇒ ∃ v v ∈ℝ ∋ v v w w ∈ℝ
∴
w w ∈ Im
∀ ∈ , w v v
⇒ ∃ v ∈ ∋ v w ∈ℝ
∴
w ∈ Im
∴ Im is a subspace of ℝ .
Example
8
Let
■
be an × matrix, if we define a linear transformation
ℝ → ℝ as x x, then Im is a column space of .
Solution
Let ⋯ , that is, be an × matrix 's th
column vector. Then for any vector x ⋯ ∈ℝ ,
x ⋯ ⋯ .
⋮
That is, any image can be expressed as a linear combination of column
vectors of .
∴ Im x x ∈ℝ …
- 208 -
■
6.3.4
Theorem
For
a
linear
transformation
ℝ → ℝ
defined
by
a
matrix
× satisfies the following two properties.
(1)
is
one-to-one.
⇔
column
vectors
of
are
linearly
independent.
(2) is onto.
⇔
row vectors of are linearly independent.
Proof (1) in one-to-one ⇔ ker
⇔
⇔
There is a unique x ∈ ℝ which satisfies x .
column vectors of are linearly independent.
(2) is onto ⇔ Im ℝ
⇔
For 's column vectors ,
ℝ
⇔
⇔
⇔
Example
9
Im x x ∈ℝ …
In RREF , the number of leading ones is .
row rank of is .
row vectors of are linearly independent.
■
Verify the following by using the Sage.
(1) Let
Sage
. ℝ → ℝ is one-to-one but not onto.
http://sage.skku.edu
① define a linear transformation
U = QQ^2
V = QQ^3
A = matrix(QQ, [[1, 0], [0, 1], [0, 0]])
T = linear_transformation(U, V, A, side='right')
print T
- 209 -
# linear transformation
Vector space morphism represented by the matrix:
[1 0 0]
[0 1 0]
Domain: Vector space of dimension 2 over Rational Field
Codomain: Vector space of dimension 3 over Rational Field
② check the surjectivity (onto)
print T.image()
# generate the range
print T.is_surjective()
# check the surjectivity (onto)
Vector space of degree 3 and dimension 2 over Rational Field
Basis matrix:
[1 0 0]
[0 1 0]
False
③ check the injectivity (one-to-one)
print T.kernel()
# generate the kernel
print T.is_injective()
# check the injectivity (one-to-one)
Vector space of degree 2 and dimension 0 over Rational Field
Basis matrix:
[]
True
(2) Let
. ℝ → ℝ is onto but not one-to-one.
Sage
http://sage.skku.edu
① define a linear transformation
U = QQ^3
V = QQ^2
A = matrix(QQ, [[1, 0, 0], [0, 1, 0]])
T = linear_transformation(U, V, A, side='right')
print T
- 210 -
# linear transformation
Vector space morphism represented by the matrix:
[1 0]
[0 1]
[0 0]
Domain: Vector space of dimension 3 over Rational Field
odomain: Vector space of dimension 2 over Rational Field
② check the surjectivity (onto)
print T.image()
# generate the range
print T.is_surjective()
# check the surjectivity (onto)
Vector space of degree 2 and dimension 2 over Rational Field
Basis matrix:
[1 0]
[0 1]
True
③ check the injectivity (one-to-one)
print T.kernel()
# generate the kernel
print T.is_injective()
# check the injectivity (one-to-one)
Vector space of degree 3 and dimension 1 over Rational Field
Basis matrix:
[0 0 1]
False
Theorem
Let
■
6.3.5
×
be an
×
matrix.
If
ℝ → ℝ
is a
linear
transformation, is one-to-one if and only if is onto.
Proof
is one-to-one ⇔ ker
⇔ There is a unique x ∈ ℝ which satisfies x .
⇔ In 's RREF, number of leading ones is .
- 211 -
⇔ For 's column vectors ,
Im x x ∈ℝ … ℝ
⇔ Im ℝ ⇔ is onto.
Equivalence Theorem of Invertible Matrix
Theorem
6.3.6 [Equivalence Theorem of Invertible Matrix]
Let be an × matrix, the following statements are all equivalent.
(1) column vectors of are linearly independent.
(2) row vectors of are linearly independent.
(3) x has only trivial solution x .
(4) For any × vector b, x b has a unique solution.
(5) and are column equivalent.
(6) is invertible.
(7) det ≠
(8) is not an eigenvalue of .
(9) is one-to-one.
(10) is onto.
[Ranking of International Math Olympiad 2012]
https://www.imo-official.org/results.aspx
- 212 -
■
6.4
Composition of Linear Transformations and Invertibility
Reference video: http://youtu.be/EOlq4LouGao http://youtu.be/qfAmNsdlPxc
Practice site: http://matrix.skku.ac.kr/knou-knowls/cla-week-8-Sec-6-4.html
In this section, we study the composition of
two or more linear
transformations as continuous product of matrices. We also study the
geometric properties of linear transformation by connecting inverse
functions and inverse matrices.
Theorem
6.4.1 [Composition of Functions]
If both ℝ → ℝ and ℝ → ℝ
are linear transformations,
then the composition
∘ ℝ → ℝ
is also a linear transformation.
Theorem
6.4.2
For linear transformations ℝ → ℝ and ℝ → ℝ ,
(1) ∘ is one-to-one implies is one-to-one.
(2) ∘ is onto implies is onto.
Proof
(1) If v v , for v v ∈ ℝ , then v v .
⇒ ∘ v ∘ v ⇒ v v (∵ ∘ is one-to-one)
∴ is one-to-one.
(2) If ∘ is onto, then for ∀ z∈ℝ , there exist v∈ℝ
∘ v z . That is, there exist v∈ℝ
which satisfy v z . Since
v w∈ℝ , there exist w∈ℝ such that
∴ is onto
- 213 -
such that
w z .
■
For the case of composition of two linear transformations, the corresponding
standard matrix is the product of two standard matrices from each linear
transformation.
That is, let ℝ → ℝ ,
matrix
,
ℝ → ℝ
transformation ∘ ℝ
→ℝ
ℝ → ℝ and
has
standard
ℝ → ℝ has a standard
matrix
.
Then
the
linear
has the standard matrix ∘ .
Let the standard matrix of a linear transformation be . If an inverse
transformation exist, then the standard matrix of is the inverse of the
matrix .
Example
1
Let ℝ → ℝ are linear transformations which rotate and
(counterclockwise) respectively around the origin. The corresponding
standard matrices are as follows.
cos sin
, cos sin
cos
sin
sin cos
As the composition of these two transformations rotates around
the origin, ∘ 's standard matrix is as follows.
cos
sin
sin
.
cos
Also the product of standard matrices of and are as follows.
- 214 -
cos sin cos sin
sin cos sin cos
cos cos sin sin cos sin sin cos
sin cos cos sin sin sin cos cos
cos sin
∘ .
cos
sin
Example
2
■
As shown in the picture, find a matrix transformation which transform a
circle with radius 1 to the given ellipse.
Solution
First we find a transformation which expands 3 times around the -axis,
and expands 2 times around the -axis. Then take a transformation
which rotates
around the origin. The first transformation is
, and hence the standard matrices for and the
rotation transformation are
,
.
Therefore, the standard matrix for the composition is the product of two
standard matrices.
.
- 215 -
■
[Remark] Computer simulation
[Matrix Transformation] http://www.geogebratube.org/student/m57556
Similarly a composition of three or more linear transformations, the standard
matrix of the composition is the product of each standard matrix in that
operation order.
Theorem
6.4.3
A function → is invertible if and only if is one-to-one and
onto
.
Theorem
If
a
6.4.4
linear
transformation
ℝ → ℝ
is
invertible,
then
ℝ → ℝ is also a linear transformation.
Inverse transformation of composition of transformation: ∘ ∘
∘
- 216 -
[Remark] Computer simulation
[shrink transformation and expand transformation]
http://www.geogebratube.org/student/m11366
“All human knowledge begins with intuitions, proceeds from
thence to concepts, and ends with ideas.”
Immanuel Kant (1724-1804)
one
of
the
most
influential
philosophers
in
Western
philosophy.
contributions
the
to
is
history
of
His
metaphysics,
epistemology, ethics, and aesthetics
have
had
a
profound
impact
on
almost every philosophical movement
that followed him.
- 217 -
6.5
*Computer Graphics with Sage
Reference video: http://youtu.be/VV5zzeYipZs
Practice site: http://matrix.skku.ac.kr/Lab-Book/Sage-Lab-Manual-2.htm
http://matrix.skku.ac.kr/Big-LA/LA-Big-Book-CG.htm
Computer graphics plays a key role in automotive design, flight
simulation, and game industry. For example, a 3 dimensional object,
such as automobile, its data (coordinates of points) can be described as
a matrix. If we transform the location of these points, we can redraw
the transformed object from the points which are newly generated. If
this transformation is linear, we can easily obtain the transformed data
by matrix multiplication. In this section, we review several geometric
transformations which are used in computer graphics.
Geometric meaning of Linear Transformation 1
(Linear Transformation of Polygon’s Image)
By using the Sage, draw a triangle with three vertices , , and ,
a triangle expanded twice, a figure by a shear transformation along the -axis
with scale 1, and a triangle which is rotated counterclockwise by .
First of all, in order to define the above linear transformations, we input the
following linear transformations by using matrix.
def matrix_transformation(A, L):
n=matrix(L).nrows()
# list L’s number of elements
L2=[[0,0] for i in range(n)]
# define a new list L2
for i in range(n):
L2[i]=list(A*vector(L[i]))
return L2
# L2=A*L
# return L2
print "The matrix_transformation function is activated"#confirm whether it is applied
Then, we define appropriate standard matrices to fit the problems’ condition.
- 218 -
A=matrix([[2,0], [0,2]])
# Expanding twice of given image
B=matrix([[1,1], [0,1]])
# shear transformation along the -axis with scale 1
C=matrix([[cos(pi/3), -sin(pi/3)], [sin(pi/3), cos(pi/3)]])
# rotate counterclockwise the given image by
Draw a triangle which has three vertices , , by using ploygon.
L1=list( [ [0,0], [0,3], [3,0] ])
# input three vertices
SL1=polygon(L1, alpha=0.3, rgbcolor=(1,0,0))
# draw a polygon which passes
through the given three points
SL1.show(aspect_ratio=1, figsize=3)
Draw a twice expanded triangle from the given triangle.
L2=matrix_transformation(A, L1)
# find new three points by a linear transformation
SL2=polygon(L2, alpha=0.8, rgbcolor=(0,0,1))
# draw a polygon which passes
through the given three points
SL2.show(aspect_ratio=1, figsize=3)
Draw a shear transformed figure along the -axis with scale 1 from the given
triangle.
L3=matrix_transformation(B, L1) # find new three points by a linear transformation
SL3=polygon(L3, alpha=0.8, rgbcolor=(1,0,1))
# draw a figure which passes
through the given three points
SL3.show(aspect_ratio=1, figsize=3)
- 219 -
Draw a figure which is rotated counterclockwise by from the given triangle.
L4=matrix_transformation(C, L1) # find new three points by a linear transformation
SL4=polygon(L4, alpha=0.4, rgbcolor=(0,0,1))
# draw a figure which passes
through the given three points
SL4.show(aspect_ratio=1, figsize=3)
Show the above four figures in the same frame.
(SL1+SL2+SL3+SL4).show(aspect_ratio=1, figsize=3)
- 220 -
Geometric meaning of Linear Transformation 2
(Linear Transformation of Line’s Image)
Draw the alphabet letter S on the plane. Then draw figures which expands the
original figure twice, sheer transforms along the -axis with scale 1, and
rotates counterclockwise by
.
First of all, in order to define the above linear transformations, we input the
following linear transformations by using matrix.
def matrix_transformation(A, L):
n=matrix(L).nrows()
# list L’s number of elements
L2=[[0,0] for i in range(n)]
# define a new list L2
for i in range(n):
L2[i]=list(A*vector(L[i]))
return L2
# L2=A*L
# return L2
print "The matrix_transformation function is activated"#confirm whether it is applied
Then, we define appropriate standard matrices to fit the problems’ condition.
A=matrix([[2,0], [0,2]])
# Expanding twice of given image
B=matrix([[1,1], [0,1]])
# shear transformation along the -axis with scale 1
C=matrix([[cos(pi/3), -sin(pi/3)], [sin(pi/3), cos(pi/3)]])
# rotate counterclockwise the given image by
Draw an alphabet letter S by using the line function.
L1=list( [ [0,0], [4,4], [-3,12], [0,15], [3,12], [4,12], [0,16], [-4,12], [3,4],
[0,1], [-3,4], [-4,4], [0,0] ]) # input the data which compose letter S
SL1=line(L1, color="red")
# draw a figure which passes through the given points
SL1.show(aspect_ratio=1, figsize=5)
- 221 -
Draw a twice expanded letter S from the given figure.
L2=matrix_transformation(A, L1)
# compute new points’ coordinates by a linear
transformation
SL2=line(L2, color="purple") #draw a figure which passes through the given points
SL2.show(aspect_ratio=1, figsize=5)
Draw a sheer transformed figure along the -axis with scale 1 from the given S.
L3=matrix_transformation(B, L1)
# compute new points’ coordinates by a linear
transformation
SL3=line(L3, color="blue") # draw a figure which passes through the given points
SL3.show(aspect_ratio=1, figsize=5)
Draw a figure which is rotated counterclockwise by from the given letter S.
L4=matrix_transformation(C, L1)
# compute new points’ coordinates by a linear
transformation
SL4=line(L4, color="green")
#draw a figure which passes through the given points
SL4.show(aspect_ratio=1, figsize=5)
- 222 -
Show the above four figures in the same frame.
(SL1+SL2+SL3+SL4).show(aspect_ratio=1, figsize=5)
http://modular.math.washington.edu/
[William Stein : The first Sage developer]
[Sage code developers: Linear
[Sage developer group]
Algebra]
- 223 -
Exercises
Chapter 6
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
Verify that ℝ → ℝ , where , is
a linear transformation and find x for x .
Solution
The
map
can
be written
as a
matrix
transformation,
. So it is a linear transformation. (x)= = . ■
Problem 2
Find the standard matrix for by using
the standard basis.
Problem 3
Let a linear transformation ℝ → ℝ satisfy the following conditions:
, .
(1) Evaluate .
(2) Evaluate .
Problem 4
Let ℝ → ℝ moves any x∈ℝ to a symmetric image to a line which
passes through the origin and has angle between the line and the
-axis. Find x for x .
Problem 5
Check whether the given matrix is an orthogonal matrix. If that is the
case, find the inverse matrix.
- 224 -
Solution
∴ is an orthogonal matrix. And
Problem 6
.
■
For each given linear transformation, find the kernel and range. Also
determine whether it is bijective or not.
Problem 7
(1)
(2)
Let and are defined as follows:
,
.
(1) Find the standard matrix for each and .
(2) Find the standard matrix for each ∘ and ∘ .
Problem 8
Let x z ∈ℝ be moved by two linear transformations and , where
, z .
x
Find ∘ x .
Solution
,
=>
Problem 9
∘
.
Answer the following questions.
- 225 -
■
(1) Find the dimension of the null space of the following matrix by using the
Sage.
(2) Let be a linear transformation corresponding to the above matrix .
Determine whether w is in the range of by using the
Sage.
Problem 10
cos sin
sin cos
by using the
. Find
Let
Sage.
Solution
var('t')
var('x0')
var('y0')
A=matrix(3,3,[1, 0, x0, 0, 1, y0, 0, 0, 1]);
B=matrix(3,3,[cos(t), -sin(t), 0, sin(t), cos(t), 0, 0, 0, 1]);
C=matrix(3,3,[1, 0, -x0, 0, 1, -y0, 0, 0, 1]);
D=A*B*C
print D
var('x')
var('y')
E=matrix(3,1,[x, y, 1]);
F=D*E
print F
[x*cos(t) - x0*cos(t) - y*sin(t) + y0*sin(t) + x0]
[x*sin(t) - x0*sin(t) + y*cos(t) - y0*cos(t) + y0]
[
1]
- 226 -
Chapter
7
Dimension and Subspaces
7.1 Properties of bases and dimensions
7.2 Basic spaces of matrix
7.3 Rank-Nullity theorem
7.4 Rank theorem
7.5 Projection theorem
*7.6 Lleast square solution
7.7 Gram-Schmidt orthonomalization process
7.8 QR-Decomposition; Householder transformations
7.9 Coordinate vectors
Exercises
The vector space ℝ has a basis, and it is a key concept to understand the
vector space.
In particular, a basis provides a tool to compare sizes of different
vector spaces with infinitely many elements. By understanding the size and
structure of a vector space, one can visualize the space and efficiently use the
data sitting contained within it.
In this chapter, we discuss bases and dimensions of vector spaces and then study
their properties. We also study fundamental vector spaces associated with a matrix
such as row space, column space, and nullspace, along with their properties. We
then derive the Dimension Theorem describing the relationship between the
dimensions of those spaces. In addition, the orthogonal projection of vectors in ℝ
- 227 -
will be generalized to vectors in ℝ , and we will study a standard matrix
associated with an orthogonal projection which is a linear transformation. This
matrix
representation
of
an
orthogonal
projection
will
be
used
to
study
Gram-Schmidt Orthogonalization and QR-Factorization.
It will be shown that there are many different bases for ℝ , but the number of
elements in every basis for ℝ is always . We also show that every nontrivial
subspace of ℝ has a basis, and study how to compute an orthogonal basis from
the basis. Furthermore, we show how to represent a vector as a coordinate vector
relative to a basis, which is not necessarily a standard basis, and find a matrix
that maps a coordinate vector relative to a basis to a coordinator vector relative
to another basis.
[Mathematicians in a Dish]
- 228 -
7.1
Properties of bases and dimensions
Lecture Movie : http://youtu.be/or9c97J3Uk0, http://youtu.be/172stJmormk
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-9-sec-7-1.html
Having learned about standard bases, we will now discuss the concept of
dimension of a vector space. Previously, we learned that an axis
representing time can be added to the 3-dimensional physical space. We
will now study the mathematical meaning of dimension. In this section,
we define a basis and dimension of ℝ
using the concept of linear
independence and study their properties.
Basis of a vector space
Definition
[Basis]
v v … v of ℝ satisfies the following two
conditions, then is called a basis for ℝ :
If
a
subset
(1) is linearly independent; and
(2) span ℝ .
Example
1
(1) If is the subset of ℝ consisting of all the points on a line going
through the origin, then any nonzero vector in forms a basis for .
(2) If a subset of ℝ represents a plane going through the origin,
then any two nonzero vectors in that are not a scalar multiple of the
other form a basis for .
■
- 229 -
2
Example
Let
e e
e e .
where
Since
is
linearly
independent and spans ℝ , is a basis for ℝ .
■
In general e e … e is a basis for ℝ , and it is called the standard
basis for ℝ .
How to show linear independence of vectors in ℝ ?
Set of vectors x … x in ℝ is linear independent if
x x ⋯ x
⇒
⋯
Let x x ⋯ x where x 's are column vectors and c ⋯ . If
the homogeneous linear system c has the unique solution c , then the
columns of the matrix
are linearly independent. In particular, for ,
det ≠ implies the linear independence of the columns of
Theorem
.
7.1.1
The following vectors in ℝ
x … … x …
are linearly independent if and only if
Proof
⋯
⋮ ⋱
⋮
≠ .
⋯
For … ∈ ℝ ,
x ⋯ x
- 230 -
⋯
⋯
.
⇒ ⋯
⋮
⋮
⋮
⋮
⋯
This gives us the following linear system
⋮
This
linear
system
⋮
has
⋯
⋯
⋱
⋯
the
.
⋮
⋮
⋮
trivial
solution
… ,
i.e.,
… if and only if ≠ . Therefore x … x are linearly
independent if and only if ≠ .
■
Example
3
By Theorem 7.1.1,
the following three vectors in ℝ
x x x
are linearly independent because
≠ .□
http://matrix.skku.ac.kr/RPG_English/7-TF-linearly-independent.html
Ÿ
Sage풀이
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
x1=vector([1, 2, 3])
x2=vector([-1, 0, 2])
x3=vector([3, 1, 1])
A=column_matrix([x1, x2, x3])
# Generating the matrix with x1, x2,
# x3 as its columns in that order
print A.det()
9
■
We can also use the inbuilt function of Sage to check if a set of vectors
- 231 -
are linearly independent.
V=RR^3;x1=vector([1, 2, 3]);x2=vector([-1, 0, 2]);x3=vector([3, 1, 1])
S=[x1, x2, x3]
V.linear_dependence(S)
[]
Example
4
Show that x x x with x x x is
a basis for ℝ .
Solution
To show that x x x is a basis for ℝ , we need to show that
is linearly independent and it spans ℝ .
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
A=matrix(QQ, 3, 3, [1, 1, 1, 0, 1, 1, 0, 0, 1])
print A.det()
1
■
Since the computed determinant above is not zero, x x x is linearly
independent. We now show that
spans ℝ . Let x be a vector in
ℝ . Consider a linear system x x x x in . Note that if this
linear system has a solution, then x is spanned by . The linear
system can be written as
∈ℝ
,
more explicitly, we have a linear system in ,
(1)
- 232 -
Hence we need to show that the linear system (1) has a solution to show that
of the linear system (1) is
spans . Indeed, the coefficient matrix
invertible, so the linear system (1) has a solution.
Theorem
7.1.2
Let x x … x be a basis for ℝ . For , any subset
y y … y of ℝ is linearly dependent. Therefore, if is
linearly independent, then
Proof
must be less than or equal to .
http://matrix.skku.ac.kr/CLAMC/chap7/Page6.htm
Since is a basis for ℝ , each vector in y y … y can be written
as a linear combination of x x … x . That is, there are ∈ℝ such
that
x ,
y x x ⋯ x
…
(2)
We now consider a formal equation with … ∈ℝ :
y
y y ⋯ y .
Then, from (2), we get,
x
x
Since x x … x are linearly independent,
∀ …
Hence we get the following linear system
⋯
⋯
⋮
(3)
⋮
⋯
The homogeneous linear system (3) has unknowns, ⋯ , and
linear equations. Since , the linear system (3) must have a nontrivial
solution. Therefore, is linearly dependent.
- 233 -
■
Theorem
7.1.3
If x x … x and y y … y are bases for ℝ , then
.
The proof of this theorem follows the theorem 7.1.2.
There are infinitely many bases for ℝ . However, all the bases have the same
number of vectors.
Definition
[Dimension]
If is a basis for ℝ , then the number of vectors in is called the
dimension of ℝ and is denoted by dim ℝ .
Note that dimℝ . If its subspace is the trivial subspace, , then
dim .
Theorem
7.1.4
For x x … x ⊆ ℝ , the following holds:
(1) If is linearly independent, then is a basis for ℝ .
(2) If spans ℝ (i.e., ℝ ), then is a basis for ℝ .
Example
5
The
determinant
of
the
matrix
having
the
vectors
x x x in ℝ as its column vectors is
≠ .
Hence x x x is linearly independent.
By Theorem 7.1.4, is a basis for ℝ .
- 234 -
■
7.1.5
Theorem
If v v … v is a basis for a subspace of ℝ , then every
vector v in can be written as a unique linear combination of the
vectors in .
Proof
Since spans , a vector v in can be written as a linear combination
of the vectors in . Suppose
v v v ⋯ v and
v ′ v ′ v ⋯ ′ v .
By subtracting the second equation from the first one, we get
′ v ′ v ⋯ ′ v .
Since is linearly independent, ′ ′ ⋯ ′ .
■
Therefore v v v ⋯ v is unique.
[Remark] Many a times a basis of ℝ is defined to a set which satisfies conditions
of theorem 7.1.5.
Example
6
Let v v v v . Then
v v v v v .
However, the vector v can also be written as follows:
v
and
v .
This is possible because is not a basis for ℝ .
- 235 -
■
7.2
Basic spaces of matrix
Lecture Movie : http://youtu.be/KDM0-kBjRoM, http://youtu.be/8P7cd-Eh328
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-9-sec-7-2.html
Associated with an × matrix , there are four important vector
spaces: row space, column space, nullspace, and eigenspace. These
vector spaces are crucial to study the algebraic and geometric
properties of the matrix as well as the solution space of a linear
system having as its coefficient matrix. In this section, we study the
relationship between the column space and the row space of and
how to find a basis for the nullspace of .
Eigenspace and null space
Definition
[Solution space, Null space]
The eigenspace x∈ x x of an × matrix associated to
an eigenvalue is a subspace of ℝ . The solution space of the
homogeneous linear system x is also a subspace of ℝ . This is
also called the null space of and denoted by Null .
- 236 -
Basis and dimension of a solution space
Let
be
an
×
matrix.
For
given
augmented
matrix
⋮
of
a
homogeneous linear system with x , by the Gauss-Jordan Elimination, we can
get its RREF ⋮ . Suppose that matrix has ≤ ≤ nonzero rows.
(1) If , then the only solution to x is x . Hence the dimension of
the solution space is zero.
(2) If , then with permitting column exchanges, we can transform ⋮
as
⋮
⋮
⋮
⋮
⋮
⋮
⋮
⋯
⋯
⋮
⋯
⋯
⋮
⋯
⋯
⋯
⋮
⋮
⋮
⋯
⋯
⋮
⋮
⋮
⋯
⋮ ⋮
⋮ ⋮
⋮
⋮
.
Then the linear system is equivalent to
⋯
⋯
⋮
⋯
Here, … are free variables. Hence, for any real numbers
… , setting … , any solution can be written as a
linear combination of vectors as follows:
⋮
⋮
⋮
⋮
⋯
x
⋮
⋮
⋮
⋮
Since … are arbitrary,
- 237 -
⋮
v
⋮
⋮
⋮
⋯ v
v
⋮
⋮
are also solutions to the linear system. Hence, the previous linear combination
of the vectors can be written as
x v v ⋯ v .
This implies that v v … v spans the solution space of x . In
addition, it can be shown that is linearly independent. Therefore is a basis
for the null space x∈ℝ x of and the dimension of the null space is
.
[Dimension of Null space]
Definition
For an × matrix , the dimension of the solution space of x
is called the nullity of and denoted by nullity( ). That is, dim Null
( ) nullity( ).
Example
1
For the following matrix , find a basis for the null space of and the
nullity of .
Solution
The RREF of the augmented matrix ⋮ for x is
- 238 -
.
Hence the general solution is
∈ ℝ .
x
Therefore a basis and the dimension of the null space of is
, nullity( )
Example
2
2.
■
Find a basis for the solution space of the following homogeneous linear
system and its dimension.
Solution
Using Sage we can find the RREF of the coefficient matrix :
A=matrix(ZZ, 3, 4, [4, 12, -7, 6, 1, 3, -2, 1, 3, 9, -2, 11])
print A.echelon_form()
[1 3 0 5]
[0 0 1 2]
[0 0 0 0]
Hence the linear system is equivalent to
Since and are free variables, letting for real numbers
, the solution can be written
- 239 -
x
.
Hence we get the following basis and nullity:
, nullity( )
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
① Finding a basis for a null space
A=matrix(ZZ, 3, 4, [4, 12, -7, 6, 1, 3, -2, 1, 3, 9, -2, 11])
A.right_kernel()
Free module of degree 4 and rank 2 over Integer Ring
Echelon basis matrix:
[ 1
3
4 -2]
[ 0
5
6 –3]
② Computation of nullity
A.right_nullity()
■
2
Column space and row space
Definition
For given × matrix
⋮
⋯
⋯
⋮
⋯
, the vectors obtained
⋮
from the rows of
⋯ ⋯
- 240 -
… ⋯
are called row vectors and the vectors obtained from the columns of
…
⋮
⋮
⋮
are called column vectors. The subspace of ℝ spanned by the row
vectors … , that is,
…
is called the row space of and denoted by Row . The subspace
of ℝ spanned by the column vectors … , that is,
…
is called the column space of , and denoted by Col . The
dimension of the row space of is called the row rank of , and the
dimension of the column space of is called the column rank of .
The dimensions are denoted by
and , respectively, that is,
dim Row , dim Col
Theorem
7.2.1
If two matrices are row equivalent. then they have the same row
space.
Proof
http://www.millersville.edu/~bikenaga/linear-algebra/matrix-subspaces/matrix-subsp
aces.html
Note that the nonzero rows in the RREF of form a basis for the row space of
. The same result can be applied to the column space of .
- 241 -
Example
3
For the following set , find a basis for which is a subspace
of ℝ :
Solution
Note that the subspace is equal to the row space of the following
matrix
.
By Theorem 7.2.1, it is also equal to the row space of the RREF of
.
Therefore the collection of nonzero row vectors of
is a basis for Row .
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
A=matrix(4, 5, [1, 2, 1, 3, 2, 3, 4, 9, 0, 7, 2, 3, 5, 1, 8, 2, 2, 8, -3, 5])
A.row_space()
Free module of degree 5 and rank 3 over Integer Ring
Echelon basis matrix:
Example
4
[
1
0
7
0 -39]
[
0
1
-3
0
31]
[
0
0
0
1
-7]
■
Find a basis for the column space of :
- 242 -
Solution
The column space of is equal to the row space of
.
By Theorem 7.2.1, it is also equal to the row space of the RREF of :
Therefore
Sage
.
is a basis for the column space of .
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
A=matrix(4, 5, [1, 2, 1, 3, 2, 3, 4, 9, 0, 7, 2, 3, 5, 1, 8, 2, 2, 8, -3, 5])
A.column_space()
Free module of degree 4 and rank 3 over Integer Ring
Echelon basis matrix:
[ 1
0
0 -1]
[ 0
1
0
1]
[ 0
0
1
0]
Theorem
■
7.2.2
For × , the column rank and the row rank of are equal.
For the proof of theorem 7.2.2, see http://mtts.org.in//expository-articles
The same number for the column rank and the row rank of is called the
rank of , and denoted by
- 243 -
rank
[Remark] Relationship between vector spaces associated with a matrix
Ÿ
Row Col( ), Col( )=Row( ,
Ÿ
Row( ) ⊥
Ÿ
Null( ), Null( ) ⊥ Row( ),
Col( ) ⊥ Null( ), Null( ) ⊥ Col( )
http://linear.ups.edu/html/section-CRS.html
Example
5
For
a ≠ ∈ℝ ,
a ⊥ … ∈ℝ ⋯
hyperplane of ℝ . It is easy to see that a
Example
6
⊥
is a subspace of ℝ .
is
a
■
(1) If a ∈ℝ . Then
a ⊥ ∈ℝ ∈ℝ
is a line in the plane passing through the origin perpendicular to the
vector .
(2) Let a ∈ℝ . Then
a ⊥ ∈ℝ
is the plane in ℝ passing through the origin and perpendicular to the
vector . ■
[International Linear Algebra Society]
- 244 -
http://www.ilasic.org/
7.3
Dimension theorem
(Rank-Nullity Theorem)
Lecture Movie: http://youtu.be/ez7_JYRGsb4, http://youtu.be/bM-Pze0suqo
Lab: http://matrix.skku.ac.kr/knou-knowls/cla-week-9-sec-7-3.html
In Section 7.2, we have studied the vector spaces associated to a matrix
. In this section, we study the relationship between the size of matrix
and the dimensions of the associated vector spaces.
Rank
Definition
[rank]
The rank of a matrix is defined to be the column rank (or the row
rank) and denoted by rank .
Let be an × matrix. If RREF , then can be written as the
following:
Hence rank( ) and nullity( ) .
Theorem
7.3.1 [Rank-Nullity theorem]
For any × , we have
rank( ) nullity( )
For the proof of theorem 7.3.1, see http://linear.ups.edu/html/section-IVLT.html
The Rank-Nullity Theorem can be written as follows in terms of a linear
transformation: If ∈ × is the standard matrix for a linear transformation
- 245 -
ℝ → ℝ , then
dim Im rank , dim ker nullity .
Hence
dim Im dim ker dim .
Example
1
The RREF of
is
. Hence rank( )
. Since , the dimension of the solution space for x is
equal to nullity( ) .
■
Example
2
Compute the rank and nullity of the matrix , where
.
Solution
The RREF of can be computed as follows
A = matrix(ZZ, 4, 5, [1, -2, 1, 1, 2, -1, 3, 0, 2, -1, 0, 1, 1, 3, 4, 1, 2, 5,
13, 5])
A.echelon_form()
[1 0 3 7 0]
[0 1 1 3 0]
[0 0 0 0 1]
[0 0 0 0 0]
Hence
rank( ) ,
and
nullity rank .
- 246 -
by
■
Theorem
7.3.1,
http://matrix.skku.ac.kr/RPG_English/7-B2-rank-nullity.html
Ÿ
Sage
http://sage.skku.edu
print A.rank()
# rank computation
print A.right_nullity()
# nullity computation
3
2
■
Theorem
7.3.2
A linear system x b has a solution if and only if
rank rank ⋮
b .
Proof
Let × , x … , b … . Then the linear
system x b can be written as
⋯
.
⋮
⋮
⋮
⋮
(1)
Hence we have the following:
x b has a solution. ⇔ There exist … satisfying the linear system (1).
⇔ b is a linear combination of the columns of .
⇔ b ∈ Col
⇔ rank rank ⋮ b .
- 247 -
■
3
Example
The linear system
has its matrix-vector representation
.
A = matrix(ZZ, 3, 3, [1, -2, 2, 1, 4, 3, 2, 2, 5])
b = vector([1, 2, 3])
print A.rank()
# rank(A)
print A.augment(b).rank()
# rank[A : b]
2
2
Since rank rank ⋮ b , Theorem 7.3.2 implies that the linear
■
system has a solution.
Definition
[Hyperplane]
Let a∈ℝ be a nonzero vector. Then
a ⊥ x∈ a⋅ x is called the orthogonal complement of a .
(This can be understood as the solution space of a⋅ x x a .) The
orthogonal complement of a is a hyperplane of ℝ .
Note that dim a ⊥ nullity(a ) .
Theorem
7.3.3
Let be a dimensional subspace of ℝ . Then a⊥ for some
nonzero vector a∈ .
Proof
Since dim , by the Rank-Nullity Theorem, dim ⊥ . Thus
⊥ span a for a nonzero vector a. Therefore
⊥ ⊥ span a ⊥ a ⊥ .
- 248 -
■
7.4
Rank theorem
Lecture Movie : http://youtu.be/8P7cd-Eh328 http://youtu.be/bM-Pze0suqo
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-9-sec-7-4.html
In this section, we study the relationship between the rank of a matrix
and the theorems that is related to the dimension of subspaces
associated to .
Theorem
7.4.1 [Rank theorem]
For any × , dim Row( ) dim Col( ).
Proof
http://ocw.mit.edu/courses/mathematics/18-701-algebra-i-fall-2010/study-materials
■
/MIT18_701F10_rrk_crk.pdf
Theorem
7.4.2
For any × , rank( ) ≤ min { }.
Proof
Since dim Row( ) ≤ , dim Col( ) ≤ , and rank( )=dim Row( ) dim Col
( ), it follows that rank( ) ≤ min { }
Theorem
■
7.4.3 [Rank theorem]
Given × , the followings hold:
(1) dim Row( ) dim Null( ) the number of columns of (that is,
rank( ) nullity( ) ).
(2) dim Col( ) dim Null( ) the number of rows of (that is, rank( )
nullity( ) ).
Proof
(1) follows from Theorem 7.3.1,
(2) follows from the fact that Row Col and rank rank
- 249 -
■
along with replacing in (1) by .
7.4.4
Theorem
For a square matrix of order , is invertible if and only if
rank( ) .
Proof
If is invertible, then x has the trivial solution only and hence
Null( ) , giving nullity( ) . By the Rank-Nullity Theorem, we have
■
rank . This can be reversed.
Example
1
Find the rank and nullity of the following matrix:
Solution
Using Gaussian Elimination,
REF( ).
Hence rank and the Rank-Nullity Theorem gives rank
nullity .
Sage
http://sage.skku.edu
A=matrix(3, 4, [1, 3, 1, 7, 2, 3, -1, 9, -1, -2, 0, -5])
print A.rank()
# rank computation
- 250 -
print A.right_nullity()
# nullity computation
3
1
Theorem
■
7.4.5
For matrices , with multiplication defined, the followings hold:
(1) Null( )⊆ Null( ).
(2) Null( )⊆ Null( ).
(3) Col( )⊆ Col( ).
(4) Row( )⊆ Row( ).
Proof We prove only (1) here. For
x∈Null ⇒ x ⇒ x x .
∴ x ∈ Null
Theorem
7.4.6
rank( ) ≤ min{rank( ), rank( )}.
Follows from theorem 7.4.5.
Theorem
7.4.7
Multiplying a matrix by an invertible matrix × does not
change the rank of
. That is, if ≠ , then
rank( ) rank( ) rank( ).
Follows from theorem 7.4.6.
- 251 -
■
Theorem
7.4.8
Suppose × has rank( ) . Then
(1) Every submatrix of satisfies rank( ) ≤ .
(2) must have at least one × submatrix whose rank is equal to .
Proof (1) Suppose the submatrix is obtained by taking rows of (we let
be this matrix consisting of the rows of ) and taking columns of
. Since Row ⊆ Row and Col ⊆ Col , the result follows.
(2) Since the rank of is , there are linearly independent rows of .
Then the matrix consisting of the linearly independent rows has the
rank equal to
. We now form a matrix
by taking
linearly
independent columns of . Then is an × submatrix of whose
■
rank is equal to .
Main Theorem of Inverse Matrices
Theorem
7.4.9 [Equivalent statements of invertible matrices]
For an × matrix , the following are equivalent:
(1) is invertible.
(2) det ≠ .
(3) is equivalent to .
(4) is a product of elementary matrices.
*(5)
has a
unique
-factorization. That is, there exists a
permutation matrix such that where is a lower
triangular matrix with all the diagonal entries equal to 1, is an
invertible diagonal matrix, and is an upper triangular matrix
whose main diagonal entries are all equal to 1.
(6) For any × vector b, x b has a unique solution.
(7) x has the unique solution x .
(8) The column vectors of are linearly independent.
(9) The column vectors of span .
*(10) has a left inverse. That is, there exists a matrix of order
such that .
- 252 -
(11) rank .
(12) The row vectors of are linearly independent.
(13) The row vectors of span .
*(14) has a right inverse. That is, there exists a matrix of order
satisfying .
(15) is one-to-one.
(16) is onto.
(17) is not an eigenvalue of .
(18) nullity .
Proof
We first prove the following equivalence:
① (10) ⇒ (7) ⇒ (8) ⇒ (11) ⇒ (10)
(10) ⇒ (7): Suppose has a left inverse such that
. If x satisfies x , then gives
x x x x .
Hence x has the unique solution x .
(7) ⇒
(8): Suppose x has only the trivial solution. If v denotes the th
column vector of and x ⋯ , then
v v ⋯ v ⇔ x ⇒ x ⇔ ≤ ≤
Hence the set
(8) ⇒
v v ⋯ v of the column vectors of is linearly independent.
(11): Suppose the column vectors of are linearly independent. Then
rank , which is equal to the maximum number of linearly independent columns
of , is equal to .
(11) ⇒ (10): Suppose rank . Then the rows of are linearly independent.
Let e be the th standard basis vector. Then the following linear systems
x e ,
≤ ≤
- 253 -
are consistent for all , since rank( ) rank e . Letting x be a
x
solution to the linear systems,
x
is a left inverse of .
⋮
x
② (1) ⇒ (6) ⇒ (14) ⇒ (2) ⇒ (1)
(1) ⇒ (6): Suppose is invertible. Then, for any × vector b,
b b b b.
Hence x b has a solution x b. For the uniqueness of the solution,
suppose x is another solution. Then
x x x x b x .
Therefore x b has a unique solution.
(6) ⇒ (14): Suppose that for each × b, the linear system x b has a unique
solution. If we take b to be e , the th standard basis vector, then the following
linear system
x e ,
≤ ≤
also has a unique solution. If x is the solution to the linear system, then the
matrix x x ⋯ x is a right inverse of .
(14) ⇒ (2): Suppose has a right inverse such that . Then
det det det det .
Hence det ≠ .
(2) ⇒ (1): Suppose det ≠ . If we let adj , then it can be shown
det
that
.
Hence is invertible.
■
- 254 -
7.5
Projection Theorem
Lecture Movie : http://youtu.be/GlcA4l8SmlM, http://youtu.be/Rv1rd3u-oYg
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-10-sec-7-5.html
In Chapter 1, we have studied the orthogonal project in ℝ
where the
vectors and their projecttions can be visualized. In this section, we
generalize the concept of project in ℝ . We also show that the
projection is a linear transformation and find its standard matrix, which
will be crucial to study the Gram-Schmidt Orthogonalization and the
QR-Decomposition.
Orthogonal Projection in ℝ
x∙ a
p a a proj a x
∥a∥
w x p
Projection (in 1-Dimension subspace) on ℝ
Theorem
7.5.1 [Projection]
For any nonzero vector a in ℝ , every vector x∈ℝ can be expressed
as follows:
x proj a x w a w p w,
where p
is a scalar multiple of a and w is perpendicular to a.
Furthermore, the vectors p w can be written as follows:
x⋅ a
p a
a,
a
- 255 -
w x p.
The proof of the above theorem is similar to that in case of orthogonal projection
in the ℝ and ℝ .
In the above theorem, the vector p is called the orthogonal projection of x onto
x⋅ a
span a and denoted by proj a x
a . The vector w is called the orthogonal
a
complement of the vector a.
Definition
[Orthogonal projection on ℝ]
The transformation ℝ → ℝ defined below
x⋅ a
a
x proj a x
a
is called the orthogonal projection of ℝ onto span a.
It can be shown that the orthogonal projection x proj a x is a linear
transformation.
(http://www.math.lsa.umich.edu/~speyer/417/OrthoProj.pdf)
Theorem
7.5.2
Let a be a nonzero column vector in ℝ . Then the standard matrix of
x proj a x x
is
a a
aa .
Note that is a symmetric matrix and rank .
For the proof of this theorem, see the website:
http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/least-squ
ares-determinants-and-eigenvalues/projections-onto-subspaces/MIT18_06SCF11_Ses
2.2sum.pdf
- 256 -
Example
1
Using the above theorem, find the standard matrix of the orthogonal
projection in ℝ onto the line tan passing through the origin.
Solution
This is a problem of finding the orthogonal projection of a vector x
onto the subspace spanned by a vector a . Hence we take a as a unit
vector
u
tan . Since the slope of the line is
on the line
sin
tan ,
cos
cos
u
sin
and
u .
Therefore,
by
the
previous
theorem,
cos
cos
uu uu
cos sin
uu
sin
sincos
u
u u
Example
2
sincos
sin
■
Find the standard matrix for the orthogonal projection in ℝ onto
the subspace spanned by the vector a .
Solution
a a , aa
Hence,
aa
aa
http://en.wikipedia.org/wiki/Fischer_projection
- 257 -
■
Projection of x on subspace
Theorem
in ℝ
7.5.3
Let be a subspace of ℝ . Then every vector x in ℝ can be
uniquely expressed as follows:
x x x
where x ∈ and x ∈ ⊥ .
In this case x is called the orthogonal projection of x onto and is
denoted by proj x.
x proj x , x x x proj ⊥ x
http://www.math.lsa.umich.edu/~speyer/417/OrthoProj.pdf
Theorem
7.5.4
Let be a subspace of ℝ . If is a matrix whose columns are the
vectors in a basis for , then for each vector x ∈ℝ
proj x x.
Proof
http://www.math.lsa.umich.edu/~speyer/417/OrthoProj.pdf
- 258 -
Example
3
Find the standard matrix for the orthogonal projection in ℝ
onto the
plane .
Solution
The general solution to is
( ∈ℝ ).
Thus is a basis of the plane .
Hence, by taking , the standard matrix is .
and
Since
,
Sage
http://sage.skku.edu
M=matrix(3, 2, [4, -2, 1, 0, 0, 1])
print M*(M.transpose()*M).inverse()*M.transpose()
The
[20/21
4/21 -2/21]
[ 4/21
5/21
[-2/21
8/21 17/21]
standard
8/21]
matrix
■
for
an
orthogonal
idempotent ( ).
- 259 -
projection
is
symmetric
and
[Remark]
Ÿ
Simulation of the projection of two vectors
http://www.geogebratube.org/student/m9503
http://matrix.skku.ac.kr/mathLib/main.html
- 260 -
7.6
* Least square solutions
Lecture Movie : https://youtu.be/BC9qeR0JWis
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-10-sec-7-6.html
Previously, we have studied how to find solve the linear system x b
when the linear system has a solution. In this section, we study how to
find an optimal solution using projection when the linear system does
not have any solution.
Details can be found in the following websites:
http://www.seas.ucla.edu/~vandenbe/103/lectures/ls.pdf
Least square solutions with GeoGebra
<Simulations> http://www.geogebratube.org/student/m12933
Least square solutions with Sage
<Simulations> http://matrix.skku.ac.kr/2012-album/11.html
- 261 -
Gram-Schmidt
7.7
Orthonomalization process
Lecture Movie: http://youtu.be/gt4-EuXvx1Y, http://youtu.be/EBCi1nR7EuE
Lab: http://matrix.skku.ac.kr/knou-knowls/cla-week-10-sec-7-7.html
Every basis of ℝ
has elements, but all the bases are distinct.
In this section, we show that every nontrivial subspace of ℝ
has
a basis and how to find an orthonormal basis from a given basis.
[Remark]
The subspaces and ℝ of ℝ are called trivial subspaces.
There are many different bases for ℝ , but all the bases have elements and
the number is called the dimension of ℝ .
Orthogonal set and orthonormal set
Definition
For vectors x x … x in ℝ , let
x x … x .
If every pair of vectors in
is orthogonal, then
is called an
orthogonal set. Furthermore, if every vector in the orthogonal set is
a unit vector, then is called an orthonormal set.
The above definition can be summarized as follows:
is an orthogonal set.
⇔
x ⋅ x
- 262 -
≠
is an orthonormal set.
1
Example
(1) The standard basis
⇔
x ⋅ x
≠
( Kronecker delta)
e e e for is orthonormal.
(2) In let x x x . Then
x x x
is orthogonal, but not orthonormal.
y . Then
(3) In let y y
the set y y y is orthonormal.
(4) If
x … x is an orthogonal set, then
∥
x
x
x ∥ …
∥x ∥
orthonormal set. ■
Orthogonality and Linear independance
Theorem
7.7.1
Let x x … x k be a set of nonzero vectors in ℝ . If is
orthogonal, then is linearly independent.
Proof
For … ∈ℝ , suppose
x x ⋯ x .
Then, for each ( , , … , ),
x x ⋯ x ⋅ x ⋅ x .
That is,
x ⋅ x x ⋅ x ⋯ x ⋅ x ⋅ x
Since, for ≠ , x ⋅ x , we have
- 263 -
is an
x ⋅ x x
… .
Since x ≠ implies x … ,
we have
⋯ .
■
Therefore, is linearly independent.
Orthogonal Basis and Orthonormal Basis
Definition
[Orthonormal basis]
Let be a basis for ℝ . If is orthogonal, then is called an
orthogonal basis. If is orthonormal, then is called an orthonormal
basis.
2
Example
Sets in (1) and (3) of
Example
1
are orthonormal bases of ℝ and the
set in (2) is an orthogonal basis of ℝ .
Theorem
7.7.2
Let x x … x n be a basis for ℝ .
(1) If is orthonormal, then each vector x in ℝ can be expressed as
x x x ⋯ x ,
where x⋅ x ⋯ .
x⋅ x
(2) If is orthogonal, then
.
x
Proof
We prove (1) only. Since is a basis for ℝ , each vector x ∈ℝ can be
expressed as a linear combination of vectors in as follows:
- 264 -
x x x ⋯ x
∈ℝ .
For each … , we have
x⋅ x x x ⋯ x ⋅ x
x ⋅ x x ⋅ x ⋯ x ⋅ x .
Since is orthonormal, x ⋅ x
x⋅ x
Example
3
≠
. Hence
■
… .
Write y as a linear combination of the vectors in
y
y y
that is the orthonormal basis for ℝ in (3)
Solution
Let y y c y c y . Then, by Theorem 7.7.2, y⋅ y
.
Hence
, y⋅ y
.
y⋅ y , y⋅ y
∴ y y c y c y y
y
y .
Theorem
7.7.3
(1) Suppose ′ x x … x is an orthonormal basis for ℝ . Then,
since x , the orthogonal projection y∈ℝ onto the subspace
x x … x in ℝ is
y proj W k y y⋅ x x y⋅ x x ⋯ y⋅ x k x k .
(2) If ′ x x … x is an orthogonal basis, but not an orthonormal
basis for ℝ , then y proj y can be written as
- 265 -
■
y⋅ x
y⋅ x
y⋅ x
x
x ⋯
x .
y proj y
x
x
x
4
Example
Let
be
a
subspace
of
x x
ℝ
spanned
by
the
two
vectors
in an orthonormal set x x . Find
the orthogonal projection of y onto and the orthogonal
component of y perpendicular to .
Solution
y proj y y⋅ x x y⋅ x x
.
The orthogonal component of y perpendicular to is
y y proj y .
■
Gram-Schmidt orthonormal process
Theorem
7.7.4
Let x x … x be a basis for ℝ . Then we can obtain an
orthonormal basis from .
Proof
[Gram-Schmidt Orthonormalization]
We first derive an orthogonal basis y y … yn for ℝ from the basis as
follows:
- 266 -
[Step 1] Take y x .
[Step 2] Let be a subspace spanned by y and let
x ⋅ y
y x proj x x
y .
y
[Step 3] Let be a subspace spanned by y and y and let
x ⋅ y
x ⋅ y
y
y
y x proj x x
y
y .
[Step 4] Repeat the same procedure to get
x ⋅ y
x ⋅ y
x ⋅ y
y x proj x x
y
y ⋯
y … ,
y
y
y
where x x … x k .
It is clear that y y … y is orthogonal. By taking
y
z … ,
y
we get an orthonormal basis
z z … z for .
■
The above process of producing and orthonormal basis from a given basis is
called the Gram-Schmidt Orthogonalization process.
[Remark]
Ÿ
Simulation for Gram-Schmidt Orthonormalization
http://www.geogebratube.org/student/m58812
- 267 -
Example
5
Use the Gram-Schmidt Orthonormalization to find an orthonormal basis
z z for ℝ from the two linearly independent vectors x ,
x .
Solution
We first find orthogonal vectors y , y as follows:
[Step 1] y x
x ⋅ y
y
[Step 2] y x proj x x
y
y
y
, z
z
y
y
Example
6
Let
x x x .
■
Use
the
Gram-Schmidt
Orthonormalization to find an orthonormal basis z z z for ℝ
using the basis x x x for ℝ .
Solution
We first find orthogonal vectors y y y :
[Step 1] Take y x .
[Step 2]
x ⋅ y
y x proj x x
y
y
x ⋅ y
x ⋅ y
y
y
[Step 3] y x proj x x
y
y
By normalizing y y y , we get
y
z
y
z
y
y
y
z .
y
- 268 -
Therefore,
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
① Computation for an orthogonal basis
x1=vector([1,1,0])
x2=vector([0,1,2])
x3=vector([1,2,1])
A=matrix([x1,x2,x3])
# generate a matrix with x1, x2, x3
[G,mu]=A.gram_schmidt()
# find an orthogonal basis. A==mu*G
print G
[
1
1
0]
[-1/2
1/2
2]
[-2/9
2/9 –1/9]
② Normalization
B=matrix([G.row(i) / G.row(i).norm() for i in range(0, 3)]); B
# The rows of matrix B are orthonormal
[
1/2*sqrt(2)
1/2*sqrt(2)
0]
[-1/3*sqrt(1/2)
1/3*sqrt(1/2)
4/3*sqrt(1/2)]
2/3
-1/3]
[
-2/3
Therefore, we get an orthonormal basis
.
We can verify if is orthonormal as follows:
③ Checking for orthonormality
print B*B.transpose()
# Checking if B is an orthogonal matrix.
print
print B.transpose()*B
[1 0 0]
[1 0 0]
- 269 -
□
Example
[0 1 0]
[0 1 0]
[0 0 1]
[0 0 1]
■
7
Let
x x .
Use
the
Orthonormalization to find an orthonormal basis
Gram-Schmidt
z z for a
subspace of ℝ for which x x is a basis.
Solution
y x
x ⋅ y
y x proj x x
y
y
y
y
z
∴ z
y
y
"Good,
he
did
not
have
enough
imagination
mathematician".
David Hilbert (1862–1943)
http://en.wikipedia.org/wiki/David_Hilbert
Hilbert is known as one of the founders of proof
theory and mathematical logic,
- 270 -
to
■
become
a
7.8
QR-Decomposition;
Householder Transformations
Lecture Movie : http://www.youtube.com/watch?v=crMXPi2lgGs
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-10-sec-7-8.html
If an × matrix has linearly independent columns, then the
Gram-Schmidt Orthogonalization can be used to decompose the matrix
in the form of where the columns of are the
orthonormal
vectors
obtained
by
applying
the
Gram-Schmidt
Orthognalization to the columns of and is an upper triangular
matrix. The -decomposition is widely used to compute numerical
solutions to linear systems, least-squares problems, and eigenvalue and
eigenvector problems. In this section, we briefly introduce the
-decomposition.
Details can be found in the following websites:
Ÿ
http://www.math.ucla.edu/~yanovsky/Teaching/Math151B/handouts/GramSchmidt
.pdf
Ÿ
https://inst.eecs.berkeley.edu/~ee127a/book/login/l_mats_qr.html
Ÿ
http://www.ugcs.caltech.edu/~chandran/cs20/qr.html
- 271 -
7.9
Coordinate vectors
Lecture Movie : http://youtu.be/M4peLF7Xur0, http://youtu.be/tdd7gbtCCRg
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-10-sec-7-9.html
In a finite-dimensional vector space, a basis is closely related to a
coordinate system. We have so far used the coordinate system
associated to the standard basis of
ℝ . In this section, we
introduce coordinate systems based on non-standard bases. We
also study the relationship between coordinate systems associated
to different bases.
If x x … x is an ordered basis for ℝ , then any vector x in ℝ is
uniquely expressed as a linear combination of the vectors in as follows:
x x x ⋯ x
Then …
basis .
Definition
… ∈ ℝ
are called coordinates of the vector x
(1)
relative to the
[Coordinate vectors]
The scalars … in (1) are called the coordinates of x relative
to the ordered basis . Furthermore, the column vector in ℝ
⋮
is called the coordinate vector of x relative to the ordered basis
and denoted by x .
Example
1
The vector x in ℝ can be expressed as follows relative to
the standard basis e e e for ℝ :
x e e e .
- 272 -
Therefore
x .
Example
2
■
Let x x x . For x find the
coordinate vector x relative to the basis x x x for ℝ .
Solution
From x x x x ∈ℝ
,
we get the linear system
.
By solving this linear system, we get .
∴ x
.
■
As described above, finding the coordinate vector relative to a basis is
equivalent to solving a linear system.
Theorem
7.9.1
Let be a basis for ℝ . For vectors x y in ℝ and a scalar ∈ℝ ,
the following holds:
(1) x y x y .
(2) x x .
In general we have
y y ⋯ y y y ⋯ y .
- 273 -
Change of Basis
Let x x … x and y y … y be two different ordered bases for
ℝ . In the following, we consider a relationship between x and x .
∈ℝ , the coordinate vector of x ∈ℝ
Letting x y y ⋯ y
relative to is
x
,
⋮
and the coordinate vector x of x ∈ℝ relative to can be expressed as
x y y ⋯ y y y ⋯ y .
Let
y
be the coordiate vector of y relative to and matrix be
⋮
y y ⋯ y
⋮
⋯
⋯
.
⋮
⋮
⋯
Then we have
x
⋯
⋮
⋮
⋮
⋯
⋯
⋮ ⋮
⋯
x ,
⋮
⋮
that is, x x .
(2)
In the equation (2) matrix transforms the coordinate vector x to another
coordinate vector x . Hence the matrix
y y ⋯ y is called a
transition matrix from ordered basis to ordered basis and denoted by
. Therefore, x x x .
- 274 -
This transformation is called change of basis. Note that the change of basis
does not modify the nature of a vector, but it changes coordinate vectors. The
following example illustrates this.
Example
3
Let e e be the standard basis for ℝ and y
y
.
For the two different ordered bases , y y :
(1) Find the transition matrix from basis to basis .
(2) Suppose x . Find the coordinate vector x .
(3) For x , show that equation (2) holds.
Solution
(1) Since y y , we need to compute the coordinate vectors
for y y relative to . Since
y e e
y e e
y
,
y
. Hence
.
(2) x x
(3) Since x e e and also x y y ,
x
x .
x
It can be easily checked that x
- 275 -
■
Example
4
For x x x and
y , y , y ,
let x x x and y y y , both of which are bases for ℝ .
Find .
Solution
Since y y y , we first find the coordinate vectors for
y y y relative to . Letting
x x x y ∈ℝ
x x x y ∈ℝ
x x x y ∈ℝ ,
we get the following three linear systems:
Note that all of the above linear systems have
as their
coefficient matrix. Hence we can solve the linear systems simultaneously
using the RREF of the coefficient matrix. That is, by converting the
augmented matrix
⋮ y ⋮ y ⋮ y in its RREF, we can find the values
of , , at the same time:
⋮
⋮
⋮
⋮
⋮ ⋮ ⋮
⋮
⋮
has the RREF
⋮ ⋮ ⋮
⋮ ⋮ .
⋮
⋮ ⋮
⋮
Therefore, the transition matrix from to is
.
- 276 -
□
http://matrix.skku.ac.kr/RPG_English/7-MA-transition-matrix.html
Ÿ
Sage
http://sage.skku.edu
x1=vector([1,2,0]);x2=vector([1,1,1]);x3=vector([2,0,1])
A=column_matrix([x1, x2, x3])
y1=vector([4, -1, 3]);y2=vector([5, 5, 2]);y3=vector([6, 3, 3])
B=column_matrix([y1, y2, y3]) # Creating the matrix with columns y1, y2,
# y3
aug=A.augment(B, subdivide=True)
aug.rref()
[ 1
0
0|-1
2
1]
[ 0
1
0| 1
1
1]
[ 0
0
1| 2
1
2]
Theorem
■
7.9.2
Suppose and are two different ordered bases for ℝ and
be
the transition matrix from to . Then is invertible and its inverse
is the transisiton matrix from to , that is, .
Example
5
For the two bases for ℝ in
Example
4 , compute the following:
(1) The transition matrix from basis to basis .
(2) The coordinate vector x relative to basis for given x
.
Solution
(1) Since the transition matrix from to is
7.9.2, we have
- 277 -
, by Theorem
.
(2) x
x
.
Sage
http://sage.skku.edu
x1=vector([1,2,0]);x2=vector([1,1,1]);x3=vector([2,0,1])
x0=vector([1,5,2])
A=column_matrix([x1, x2, x3])
y1=vector([4, -1, 3]);y2=vector([5, 5, 2]);y3=vector([6, 3, 3])
B=column_matrix([y1, y2, y3])
aug=B.augment(A, subdivide=True)
Q=aug.rref()
print Q
[
1
0
0|-1/2
[
0
1
0|
[
0
0
1| 1/2 -5/2
0
3/2 -1/2]
2
-1]
3/2]
[Bookmarks] http://blog.daum.net/with-learn/5432044
- 278 -
■
Chapter 7
Exercises
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
Use determinant to check if the following vectors are linearly independent:
v , v , v
Problem 2
Determine if the given set is a basis for ℝ .
(1)
(2)
(Hint: http://math3.skku.ac.kr/spla/CLA-7.1-Exercise-2)
Problem 3
Find two different bases for the subspace of ℝ described by the equation
.
Problem 4
Given a homogeneous linear system, find a basis and the dimension of its
corresponding solution space.
(1)
(Hint: http://math1.skku.ac.kr/home/matrix1/261/)
(2)
(Hint: http://math1.skku.ac.kr/home/pub/548/)
Problem 5
Given matrix , find a basis for its null space and nullity( ).
- 279 -
.
Solution
Sage: Find RREF of
⋮
A=matrix(ZZ,6,5,[2,2,-1,0,1,-1,-1,2,-3,1,1,1,-2,0,-1,0,0,1,1,1,0,0,0,1,1,0,0,1,1,0])
A.echelon_form()
A.right_kernel()
∈
x
.
forms
a
set
of
solutions.
and nullity( ) = 1.
Problem 6
■
For the following matrix , find a basis for its column space Col and
compute the column rank .
.
Problem 7
For given matrix compute its rank and nullity. Verify if the rank and
nullity of satisfy the Rank-Nullity Theorem.
(1)
(2)
.
Solution
⓵
RREF( )
A=matrix(QQ, 3, 6, [2, 5, 7, 9, 10, 11, 2, 3, 1, 2, 4, 8, 8, 6, 2, 1, 2, 3])
print A.echelon_form()
- 280 -
[
1
0
0
-3/4
[
0
1
0
7/8
-3/2 -13/4]
9/4
21/4]
[
0
0
1
7/8
1/4
-5/4]
=> rank( )=3 and nulllity ( )= 6-3 =3.
② Sage:
A=matrix(QQ, 3, 6, [2, 5, 7, 9, 10, 11, 2, 3, 1, 2, 4, 8, 8, 6, 2, 1, 2, 3])
print A.rank()
print A.right_nullity()
∴ rank( )=3 and nullity( )=3.
Problem 8
Check if rank rank .
Problem 9
■
.
Using the table below compute Row Null Col Null for
matrix :
Problem 10
(a)
(b)
(c)
(d)
(e)
size of
×
×
×
×
×
rank
3
2
1
3
2
For ∈ × , if rank , we say that has full row ran, and if
rank , is said to have full column rank. Determine if has
full row rank and/or full column rank:
.
(Hint: http://math1.skku.ac.kr/home/pub/565/)
Problem 11
For a , find a basis and the dimension of the hyperplane
a⊥ x a⋅ x .
- 281 -
Solution
For any x in a⊥ , ∙ .
=>
=> dim a ⊥ (since a∈ ).
A set {(1, 0, 0, -1), (0, 1, 0 ,-2), (0, 0, 1, 1)} forms a basis for hyperplane a⊥ .
Problem 12
For
x
a ,
and
find
the
standard
■
matrix
for
x proj a x.
Problem 13
For x and a , using proj a x, find x and x such
that x ∈ a , x ∈ a ⊥ and x x x .
Solution
x proj a x x and its standard matrix is
aa
a a
.
Since x ∈ a and x x , x x
∴ x x x
Problem 14
.
.
For given x , express x as x x x for which x is in the
direction of a a and x is perpendicular to a .
Problem 15
For the following and b, find the least squares solution to x b:
(1) , b .
■
- 282 -
, b
.
(2)
Problem 16
Find the least squares curve passing through
the five points .
Determine
Problem 17
the
values
of
which
make
the
set
orthogonal.
Problem 18
Find the orthonormal set relative to the following orthogonal vectors:
v v v .
Solution
Sage:
x1=vector([1,2,1])
x2=vector([1,0,1])
x3=vector([3,1,0])
A=matrix([x1,x2,x3])
[G,mu]=A.gram_schmidt()
B=matrix([G.row(i) / G.row(i).norm() for i in range(0, 3)]); B
[1/6*sqrt(6) 1/3*sqrt(6) 1/6*sqrt(6)]
[ sqrt(1/3) -sqrt(1/3)
sqrt(1/3)]
[ sqrt(1/2)
0 -sqrt(1/2)] .
Problem 19
■
Show that each of the following sets of vectors ℝ is linearly independent,
and find its corresponding orthonormal set:
(1) v v v .
(2) v v v .
- 283 -
For given plane and vector v , find the
Problem 20
following (Note that the inner product is defined to be 〈uv〉 u⋅ v.):
(1) A basis for the 2-dimensional vector space represented by the plane and its
corresponding orthonormal basis
(2) proj v
Problem 21
For the ordered basis for
ℝ :
(1) For x , find its coordinate vector x relative to .
(2) For y , find its coordinate vector y relative to .
(3) Find the coordinate vector x y of x y relative to .
(4) For the above x and y, find x and y .
Solution
(1)
(2)
.
Sage :
x1=vector([0,1,1,1]) ; x2=vector([1,0,-1,1]); x3=vector([1,2,0,2]); x4=vector([3,-2,2,0])
P=column_matrix([x1,x2,x3,x4])
A1=matrix(4,1,[7, -7, 5, 4]); A2=matrix(4,1,[1, -4, 4, 3])
P1=P.inverse()
print P1*A1; print; print P1*A2
■
Problem 22
For
u u v v ,
v v which are bases for ℝ .
(1) Find the transition matrix .
(2) Find the transition matrix .
- 284 -
let
u u ,
(3) Suppose w . Find w using the transition matrix .
(4) Suppose w . Find w using the transition matrix .
Problem P1
If the size of matrix is × , what is the value of
rank nullity ?
Problem P2
(Select one) If one replaces a matrix with its transpose, then
A. The image may change, but the kernel, rank, and nullity do not change.
B. The image, kernel, rank, and nullity may all change.
C. The image, rank, and kernel may change, but the nullity does not change.
D. The image, kernel, rank, and nullity all do not change.
E. The image, kernel, and nullity may change, but the rank does not change.
F. The kernel may change, but the image, rank, and nullity do not change.
G. The image and kernel may change, but the rank and nullity do not change.
Problem P3
(Select one) Let ℝ → ℝ be a linear transformation. Then
A. is invertible if and only if the rank is five.
B. is one-to-one if and only if the rank is three; is never onto.
C. is onto if and only if the rank is two; is never one-to-one.
D. is one-to-one if and only if the rank is two; is never onto.
E. is onto if and only if the rank is three; is never one-to-one.
F. is onto if and only if the rank is five; is never one-to-one.
G. is one-to-one if and only if the rank is five; is never onto.
Problem P4
(Select one) If a linear transformation ℝ → ℝ is onto, then
A. The rank is three and the nullity is zero.
B. The rank and nullity can be any pair of non-negative numbers that add up to three.
C. The rank is three and the nullity is two.
D. The rank is two and the nullity is three.
E. The situation is impossible.
F. The rank and nullity can be any pair of non-negative numbers that add up to five.
G. The rank is five and the nullity is two.
- 285 -
Problem P5
(Select one) If a linear transformation ℝ → ℝ is one-to-one, then
A. The rank is five and the nullity is two.
B. The situation is impossible.
C. The rank and nullity can be any pair of non-negative numbers that add up to five.
D. The rank is two and the nullity is three.
E. The rank is three and the nullity is zero.
F. The rank is three and the nullity is two.
G. The rank and nullity can be any pair of non-negative numbers that add up to three.
Problem P6
(1) If the homogeneous linear system x has linear equations and unknowns,
what is the maximum possible value for the dimension of the solution space?
(2) What is the dimension of a hyperplane in ℝ perpendicular to a vector a in ℝ ?
(3) List all of the possible dimensions of subspaces of ℝ ?
(4) What is the dimension of the subspace of ℝ spanned by the three vectors
v , v , v ?
Problem P7
Suppose x … x is a basis for ℝ . If is an invertible matrix of
order , show that the set x … x is also a basis for ℝ .
Problem P8
Determine if the following matrix and have the same null space and
row space:
Problem P9
What happens if the Gram-Schmidt Orthonormalization Procedure is applied
to linearly dependent vectors?
Problem P10
Suppose the columns of are orthonormal. What is a relationship between
the column spaces of and ?
- 286 -
Problem P11
Problem P12
Show that the set spans ℝ .
What are the possible ranks of according to the varying values of :
.
[2014 ICM Seoul, Korea] http://www.icm2014.org/
- 287 -
Chapter
8
Diagonalization
8.1 Matrix Representation of Linear Transformation
8.2 Similarity and Diagonalization
8.3 Diagonalization with orthogonal matrix, *Function of matrix
8.4 Quadratic forms
*8.5 Applications of Quadratic forms
8.6 SVD and generalized eigenvectors
8.7 Complex eigenvalues and eigenvectors
8.8 Hermitian, Unitary, Normal Matrices
*8.9 Linear system of differential equations
Exercises
In Chapter 6, we have studied how to represent a linear transformation from ℝ
into ℝ as a matrix using its corresponding standard matrix. We were able to
compute the standard matrix of the linear transformation based on the fact that
every vector in ℝ or ℝ can be expressed as a linear combination of the
standard basis vectors.
In this chapter, we study how to represent a linear transformation from ℝ to
ℝ with respect to arbitrary ordered bases for ℝ and ℝ . In addition, we study
relationship between different matrix representations of a linear transformation
from ℝ to itself using transition matrices. We also study matrix diagonalization.
- 288 -
Further we study spectral properties of symmetric matrices and show that every
symmetric matrix is orthogonally diagonalizable.
* A quadratic form is a quadratic equation which we come across in mathematics,
physics, economics, statistics, and image processing, etc. Symmetric matrices play
a significant role in the study of quadratic forms. In particular, we will learn how
orthogonal diagonalization of symmetric matrices is used in the study of quadratic
forms.
We introduce one of the most important concept in matrix theory called the
singular value decomposition (SVD) which find many applications in science and
engineering.
We will generalize matrix diagonalization of × matrices and study least squares
solutions and a pseudoinverse.
We introduce complex matrices having complex eigenvalues and eigenvectors. We
also
introduce
counterparts
Hermitian
corresponding
matrices
to
and
unitary
symmetric
matrices
matrices
and
that
orthogonal
respectively. Lastly, we study diagonalization of complex matrices.
- 289 -
are
complex
matrices,
8.1
Matrix Representation
Lecture Movie : http://youtu.be/jfMcPoso6g4
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-1.html
In
Chapter
6,
we
have
transfromation from ℝ
bases for
ℝ
and
studied
into ℝ
ℝ .
how
to
represent
a
linear
as a matrix using the standard
In this section, we find a matrix
representation of a linear transformation from ℝ
respect to arbitrary ordered bases for ℝ
into ℝ
and ℝ .
Matrix Representation Relative to the Standard Bases
- 290 -
with
Matrix Representation In Some Ordered Bases for ℝ
and ℝ
Theorem
8.1.1
Let ℝ → ℝ be a linear transformation, and let
x … x y … y
be ordered bases for and , respectively. Let y x . Then
y ′ x x ,
where the matrix representation ′ of in the ordered bases
and is
′ x x ⋯ x .
Note that the matrix
is called the matrix associated with the linear
transformation with respect to the bases and .
Proof
Recall that any vector x∈ℝ can be uniquely represented as a linear
- 291 -
combination of vectors in x … x , say
x x x ⋯ x .
Then the coordinate vector for x relative to the basis is
x .
⋮
By the linearity of , we have y x x x ⋯ x .
Since y is a vector in ℝ , the coordinate vector of y relative to satisfies
y x x ⋯ x
x x ⋯ x ′ x
⋮
■
Thus the matrix is the matrix whose th column is the coordinate vector
x of x with respect to the basis .
[Remarks]
(1)
By
Theorem
8.1.1
we
can
compute
x
by
a
matrix-vector
multiplication, that is,
y x x ′ x .
(2) The matrix ′ varies according the ordered bases . For example,
if we change the order of the vectors in the ordered base , then the columns
of ′ change as well.
(3) The matrices and ′ are distinct, but they have the
following relationship:
′ ≅
(4) If and , then ′ is denoted by and is called
the matrix representation of relative to the ordered basis .
- 292 -
Example
1
Define a linear transformation ℝ → ℝ via
and let
x x x , y y
be ordered bases for ℝ and ℝ , respectively. Compute ′ .
Solution
Since
′ x x x × ,
we
first
compute
x x x :
, x
, x
x
We now find the coordinate vectors of the above vectors relative to the
ordered basis . Since
x y y
x y y
x y y ,
we need to solve the corresponding linear systems with the same
coefficient matrix
. The augmented matrix for all of the three
⋮
⋮ ⋮
linear systems is
. By converting this into its
⋮ ⋮
⋮
RREF, we get
⋮ ⋮
.
⋮ ⋮
⋮
⋮
Therefore,
,
,
hence
′ x x x ×
.
- 293 -
and
(Note that
.)
Example
2
■
and
Let ℝ → ℝ be defined via
x x y
y y
be ordered bases for ℝ and ℝ , respectively. Find ′ .
Solution
Since x , x , we have
y y
y ⋮ x ⋮ x
⋮
⋮
⋮ ⋮ .
⋮ ⋮
We can get its RREF as follows:
⋮
⋮
⋮
⋮
⋮ ⋮ ⋮ ⋮
⋮ ⋮
⋮
⋮
⋮
.
⋮ ⋮
⋮
⋮
⋮
Therefore, we get ′ as follows:
′
Sage
.
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
① Write y y
y ⋮ x ⋮ x .
- 294 -
□
x, y = var('x, y')
h(x, y) = [x+y, x-3*y, -2*x+y]
T = linear_transformation(QQ^2, QQ^3, h)
x1=vector([1, 1])
x2=vector([2, 1])
y1=vector([1, 0, -1])
y2=vector([-1, 2, 1])
y3=vector([0, 1, 1])
B=column_matrix([y1, y2, y3, T(x1), T(x2)])
# Matrix whose columns are
# the vectors defined above
print B
[ 1 -1
0
2
3]
[ 0
2
1 -2 -1]
[-1
1
1 -1 -3]
② RREF y y
y ⋮ x ⋮ x
C=B.echelon_form()
print C
[
1
0
0
1/2
[
0
1
0 -3/2 -1/2]
[
0
0
1
1
5/2]
0]
③ Finding
A=C.submatrix(0, 3, 3, 2)
# C.submatrix(a, b, c, d)
# submatrix with c consecutive rows of C starting from row a+1 and d
# consecutive columns of C starting from column b+1
print A
[ 1/2
5/2]
[-3/2 -1/2]
[
1
0]
We shall include calculation using the inbuilt function. Following are the
codes.
var('x,y')
- 295 -
h(x,y)=[x+y,x-3*y,-2*x+y]
V=QQ^2;W=QQ^3
T=linear_transformation(V,W,h)
y1=vector(QQ,[1,1]);y2=vector(QQ,[2,1])
x1=vector(QQ, [1,0,-1]);x2=vector(QQ, [-1,2,1]);x3=vector(QQ,[0,1,1]);
alpha=[y1,y2]; beta=[x1,x2,x3]
V1=V.subspace_with_basis(alpha); W1=W.subspace_with_basis(beta)
T1=(T.restrict_domain(V1)).restrict_codomain(W1)
T1.matrix(side='right')
[ 1/2
5/2]
[-3/2 -1/2]
[
1
0]
■
- 296 -
Example
3
Let ℝ → ℝ be a linear transformation defined as
and consider the ordered bases e e , e e e for ℝ and
ℝ , respectively. Answer the following questions:
(1) Find ′ .
(2) Compute
using
′ in (1).
(3) Using the definition of , find the standard matrix
and
, where and are the standard bases of ℝ and ℝ
repectively.
Solution
(1) Since e , e , we get
. Hence
′ .
(2) Since
, we have
′
e e e
e e e .)
(Note that
⋅
.
(3) ,
⋅
- 297 -
■
[Remark]
For given three vector spaces with different ordered bases, we
can consider two linear transformations and , and their
corresponding matrix representations and relative to the
given ordered bases.
Composition of Linear Transformations
Let be a linear transformation from a vector space ℝ with an ordered
basis into a vector space ℝ with an ordered basis , and be a linear
transformation from a vector sapce ℝ with an ordered basis into a vector
space ℝ with an ordered basis . Suppose these linear transformations have
their corresponding matrix representations and , respectively.
We can consider the composition ∘ . Then its matrix representation is
∘ ,
That is, the product of the two matrix representations of and .
[Remark]
Transition Matrix
as
As we have discussed earlier, x x x , the matrix is called
the transition matrix from ordered basis to ordered basis . We can
consider the transition matrix as linear transformation x x.
- 298 -
Example
4
Let ℝ → ℝ be linear transformations defined as
and
respectively,
Consider
for
the
ℝ .
ordered
Find
the
bases
e e ,
matrix
e e ,
representation
of
composition ∘ with respect to the ordered bases and .
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
x, y = var('x, y')
ht(x, y) = [2*x+y, x-y];hs(x, y) = [-x+5*y, 2*x+3*y]
T = linear_transformation(QQ^2, QQ^2, ht)
S = linear_transformation(QQ^2, QQ^2, hs)
x1=vector([1, 0]);x2=vector([0, 1]);x3=vector([1, 1])
B=column_matrix([x2, x1, T(x1), T(x2)])
C=B.echelon_form()
MT=C.submatrix(0, 2, 2, 2)
print "Matrix of T="
print MT
D=column_matrix([x1, x3, S(x2), S(x1)])
E=D.echelon_form()
MS=E.submatrix(0, 2, 2, 2)
print "Matrix of S="
print MS
print "MS*MT="
print MS*MT
Matrix of T=
[ 1 -1]
[ 2
1]
Matrix of S=
[ 2 -3]
[ 3
2]
MS*MT=
[-4 -5]
[ 7 -1]
F=column_matrix([x1, x3, S(T(x1)), S(T(x2))])
G=F.echelon_form()
MST=G.submatrix(0, 2, 2, 2)
- 299 -
the
print "Matrix of S*T="
print MST
Matrix of S*T=
[-4 -5]
[ 7 -1]
■
3D Printing object 1
http://matrix.skku.ac.kr/2014-Album/2014-12-ICT-DIY/index.html
http://youtu.be/FgAzOkqq7Sg
- 300 -
8.2
Similarity and Diagonalization
Lecture Movie : http://youtu.be/xirjNZ40kRk, http://youtu.be/MnfLcBZsV-I
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-2.html
In this section, we present various matrix representations of a
linear transformation from ℝ
to itself in terms of transition
matrix. We also study when the transition matrix becomes a
diagonal matrix.
[Remark]
Relationship between matrix representations and
.
Theorem
8.2.1
Let ℝ → ℝ be a linear transformation and and be ordered
bases for ℝ . If ′ , then we have
′ ,
where is the transition matrix from to .
′ .
http://www.math.tamu.edu/~yvorobet/MATH304-503/Lect2-12web.pdf
- 301 -
Example
1
Let ℝ → ℝ be a linear transfromation defined by
.
If is the standard basis for ℝ and y y
is a
basis for ℝ , find ′ using the transition matrix .
Solution
Let be the standard matrix relative to the standard basis for
linear transformation . Then we can find
. If
, then
.
y y
Therefore,
and by Theorem 8.2.1 we get ′as follows:
.
′
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
x, y = var('x, y')
h(x, y) = [2*x-y, x+3*y]
T = linear_transformation(QQ^2, QQ^2, h)
x1=vector([1, 0]);x2=vector([0, 1])
y1=vector([0, 1]);y2=vector([-1, 1])
B=column_matrix([x1, x2, y1, y2])
C=B.echelon_form()
P=C.submatrix(0, 2, 2, 2)
print "Transition Matrix="
print P
A = T.matrix(side='right')
print "A="
print A
print "P.inverse()*A*P"
print P.inverse()*A*P
D=column_matrix([y1, y2, T(y1), T(y2)])
E=D.echelon_form()
print "Matrix of A wrt beta="
print E.submatrix(0, 2, 2, 2)
- 302 -
Transition Matrix=
[ 0 -1]
[ 1
1]
A=
[ 2 -1]
[ 1
3]
P.inverse()*A*P
[ 2 -1]
[ 1
3]
Matrix of A wrt beta=
[ 2 -1]
[ 1
3]
■
Similarity
Definition
[Similarity]
For square matrices of the same order, if there exists an
invertible matrix such that
,
then we say that is similar to . We use ∼ for similar matrices
.
Example
2
For
, , , it can be shown that
. Hence is similar to , which is denoted by ∼ .
- 303 -
■
8.2.2
Theorem
For square matrices of the same order, the following hold:
(1) ∼
(2) ∼ ⇒ ∼
(3) ∼ ∼ ⇒ ∼
Therefore, the similarity relation is an equivalence relation.
Theorem
8.2.3
For square matrices of the same order, if are similar to
each other, then we have the following:
(1) det det .
(2) tr tr .
Proof
Since ∼ , there exists an invertible matrix such that .
(1) By the multiplicative property of determinant,
det det
⇒
det det det det ( ∵ det det det )
⇒
det det det det
⇒
det det ∵ det det det
(2) tr tr tr
( ∵ tr tr )
tr tr
■
Since similar matrices have the same determinant, it follows that they have the
same characteristic equation and hence the same eigenvalues. For a square
matrix, in ordered to solve problems of determinant and/or eigenvalues, we
can use its similar matrices which make the problems simpler.
- 304 -
Diagonalizable Matrices
[Diagonalizable Matrices]
Definition
Suppose a square matrix is simlar to a diagonal matrix, that is,
there exists an invertible matrix such that is a diagonal
matrix. Then is called diagonalizable and the invertible matrix is
called a diagonalizing matrix for .
If , then and hence we have
⋯ ( multiplications of )
⋯
This implies that if a matrix is diagonalizable, then its powers can be very easily
computed.
Example
3
For invertible matrix
and matrix
, we have
.
Hence is diagonalizable.
Ÿ
□
http://matrix.skku.ac.kr/RPG_English/8-TF-diagonalizable.html
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
A=matrix(QQ, [[1, 1], [-2, 4]])
print A.is_diagonalizable()
# Checking if diagonalizable
True
Example
4
■
Since every diagonal matrix satisfies
diagonalizable.
, it is
■
- 305 -
5
Example
Show that
is not diagonalizable.
Solution
Suppose to the contrary that is diagonalizable, that is, there exist an
invertible matrix and a diagonal matrix with
, ≠ ,
,
such that . Since , we have
,
which gives
. Hence .
If ≠ , then and (≠ ). Hence . Similarly, we can
show that . The conditions and give a contradiction to
≠ . Therefore, is not diagonalizable.
■
Equivalent Condition for Diagonalizability
Theorem
8.2.4 [Equivalent Condition]
Let be a square matrix of order . Then is diagonalizable if and
only if has linearly independent eigenvectors.
Furthermore, if
is diagonalizable, then is similar to diagonal matrix whose main
diagonal entries are equal to the eigenvalues … of , and the
th column of a diagonalizing matrix is an eigenvector of
corresponding to eigenvalue .
Proof
⇒ If is diagonalizable, then there exists an invertible matrix
p p ⋯ p
- 306 -
such that where diag ⋯ . Since , we get
p p p p … p p .
…
Hence
are
eigenvalues of and . Note that p p … p are eigenvectors
corresponding to … , respectively. Since
is
invertible,
it
follows
that
its
columns
p p … p
are
linearly
independent.
⇐ Suppose
has eigenvalues …
and their corresponding
eigenvectors p p … p that are linearly independent. Then we can
construct a matrix as follows:
p p … p .
Then
p p … p p p ⋯ p
p p
⋯
⋯
⋯ p
⋮ ⋮ ⋱
⋯
.
⋮
Since the columns of are linearly independent, the matrix is invertible,
giving . Therefore is diagonalizable.
[Remark]
■
Procedure for diagonalizing a matrix
Ÿ
Step 1: Find linearly independent eigenvectors p p … p of .
Step 2: Construct a matrix whose columns are p p … p in this
order.
Ÿ
Step 3: The matrix diagonalizes and is a diagonal matrix whose
Ÿ
main diagonal entries are eigenvalues … of
diag … .
Example
6
It
can
be
shown
and
that
the
their
matrix
corresponding
- 307 -
has
eigenvalues
eigenvectors
are
x x , respectively.
independent,
by
Theorem
Since these eigenvectors are linearly
8.2.4,
is
diagonalizable.
If
x x
, then we have
Example
7
■
Show that
is diagonalizable and find the diagonalizing
matrix of .
Sage
http://sage.skku.edu or
http://mathlab.knou.ac.kr:8080/
A=matrix([[0, 0, -2], [1, 2, 1], [1, 0, 3]])
print A.eigenvalues()
# Eigenvalue Computation
[1, 2, 2]
has eigenvalues . We now compute linearly independent
eigenvectors of .
For , we solve x x (that is, x ) for x.
E=identity_matrix(3)
print (E-A).echelon_form()
[ 1
0
2]
[ 0
1 -1]
[ 0
0
0]
Since x
∈ , we get x
; For , we solve
x x (that is, x ) for x.
print (2*E-A).echelon_form()
- 308 -
[1 0 1]
[0 0 0]
[0 0 0]
x
This gives
x
∈ and hence
x .
x1=vector([-2, 1, 1])
x2=vector([-1, 0, 1])
x3=vector([0, 1, 0])
P=column_matrix([x1, x2, x3])
print P
print
print P.det()
[-2 -1
0]
[ 1
0
1]
[ 1
1
0]
1
Since the above computation shows that the determinant of is not
zero,
is
invertible.
Hence
its
columns
x x x
are
linearly
independent. Therefore, by Theorem 8.2.4, is diagonalizable.
print P^-1*A*P # Computing diagonal matrix whose main diagonal entries
# are eigenvalues of A.
[1 0 0]
[0 2 0]
[0 0 2]
Theorem
■
8.2.5
If x x … x are eigenvectors of × corresponding to distinct
eigenvalues … , then the set x x … x is linearly
independent.
Proof
(Exercise) Hint. This can be proved by the mathematical induction .
- 309 -
8.2.6
Theorem
If a square matrix of order has distinct eigenvalues, then is
diagonalizable.
Proof
Let x x … x be eigenvectors of corresponding to distinct eigenvalues
… . Then, by Theorem 8.2.5, the eigenvectors are linearly
independent. Therefore, Theorem 8.2.4 implies that is diagonalizable.
■
Example
8
The matrix
in Example 6 has two distinct eigenvalues. Thus,
by Theorem 8.2.6, is diagonalizable.
■
Note that a diagonal matrix can have a repeated eigenvalue. Therefore, the
converse of Theorem 8.2.6 is not necessarily true.
Algebraic Multiplicity and Geometric Multiplicity of an Eigenvalue
Definition
[Algebraic and Geometric Multiplicity]
… be distinct eigenvalues of × . Then the
characteristic polynomial of can be written as
Let
In the above expression the sum of the exponents … is
equal to . The positive integer is called the algebraic multiplicity
of
and
the
number
of
linearly
independent
eigenvectors
corresponding to the eigenvalue is called the geometric multiplicity
of .
- 310 -
8.2.7 [Equivalent Condition for Diagonalizability]
Theorem
Let be a square matrix of order . Then is diagonalizable if and
only if the sum of the geometric multiplicities of eigenvalues of is
equal to .
Proof
By Theorem 8.2.4 an equivalent condition for a square matrix of order
to diagonalizable is to have linearly independent eigenvectors. Since
the sum of the geometric multiplicities of eigenvalues of is equal to the
number of linearly independent eigenvectors of and it is equal to , the
result follows.
Theorem
■
8.2.8
Let be a square matrix and be an eigenvalue of . Then the
algebraic multiplicity of is greater than or equal to the geometric
multiplicity of .
Proof
Let be the geometric multiplicity of an eigenvalue of , and let be
the × matrix whose columns are the linearly independent eigenvectors of
corresponding to eigenvalue . We can construct an invertible matrix by adding
linearly independent columns to . Let be the resulting invertible
matrix and let be the inverse of . Then . Note
that and have same characteristic polynomials. Since has first
columns have in its diagonal, the characteristic polynomial of has a
factor of at least . Hence, the algebraic multiplicity of is greater than or
equal to the geometric multiplicity of .
Theorem
■
8.2.9 [Equivalent Condition for Diagonalizability]
Let be a square matrix of order . Then is diagonalizable if and
only if each eigenvalue of has the same algebraic and geometric
multiplicity.
Proof If
is diagonalizable, then there exists an invertible matrix and a
diagonal matrix such that , or equivalently . This implies
that times column of is equal to scalar multiple of the column of .
- 311 -
Hence, all the columns of are eigenvectors of , which implies that each
eigenvalue of has the same algebraic and geometric multiplicity. The converse
is also true by Theorem 8.2.5.
Example
9
For
■
,
its
characteristic
equation
is
. Hence the eigenvalues of are and
has algebraic multiplicity 2.
The following two vectors are linearly
independent eigenvectors of
x , x .
However, matrix cannot have three linearly independent eigenvectors
and hence Theorem 8.2.4 implies that is not diagonalizable.
Example
10
■
It can be shown that
has eigenvalues 3 and with
algebraic multicity 1 and 2 respectively, We can further show that
geometric multiplicity of 3 and 2 are 1 and 2 respectively. Hence is
diagonalizable. It can be verified
diagonalizes
and
, where . Let us further compute .
- 312 -
■
8.3
Diagonalization with orthogonal matrix, *Function of matrix
Lecture Movie : http://youtu.be/jimlkBGAZfQ, http://youtu.be/B--ABwoKAN4
ab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-3.html
Symmetric matrices appear in many applications. In this section,
we study useful properties of symmetric matrices and show that
every
symmetric
Furthermore,
we
matrix
study
is
matrix
orthogonally
functions
diagonalizable.
using
diagonalization.
Orthogonal Matrix
Definition
[Orthogonal Matrix]
For real square matrix , if is invertible and , then is
called an orthogonal matrix.
Theorem
8.3.1
If is an orthogonal matrix of order , then the following hold:
(1) The rows of are unit vectors and they are perpendicular to each
other.
(2) The columns of are unit vectors and they are perpendicular to
each other.
(3) is invertible.
(4) x x for any × vector x (Norm Preserving Property).
Proof Similar to the proof of Theorem 6.2.3.
Example
1
For , we have . Since
- 313 -
matrix
, is an orthogonal matrix.
■
The inverse of an orthogonal matrix can be obtained by taking transposition of
the orthogonal matrix.
Definition
[Orthogonal Similarity]
Let and be square matrices of the same order. If there exists an
orthogonal matrix such that , then is said to be
orthogonally similar to .
Definition
For
a
[Orthogonally Diagonalizable]
square
matrix
,
if
there
exists
an
orthogonal
matrix
diagonalizing , then is called orthogonally diagonalizable and is
called a matrix orthogonally diagonalizing .
What matrices are orthogonally diagonalizable?
Theorem
(Symmetric Matrices)
8.3.2
Every eigenvalue of a real symmtric matrix is a real number.
Proof
Example
(Exercise) http://www.quandt.com/papers/basicmatrixtheorems.pdf
2
The
symmetric
matrix
and
- 314 -
has
hence
characteristic
its
equation
eigenvalues
are
that are all real numbers.
Theorem
■
8.3.3
If a square matrix is symmetric, then eigenvectors of corresponding
to distinct eigenvalues are perperdicular to each other.
Proof
(Exercise) http://www.quandt.com/papers/basicmatrixtheorems.pdf
Theorem
8.3.4
Let be a square matrix. Then is orthogonally diagonalizable if and
only if the matrix is symmetric.
Proof
(⇒ ) Suppose
is orthogonally diagonalizable. Then there exist an
orthogonal matrix and a diagonal matrix such that . Since
, we have
.
Hence
⇔
⇔
⇔
Therefore, is symmetric.
(⇐ ) : Exercise
Theorem
■
8.3.5
If is a symmetric matrix of order , then has eigenvectors
forming an orthonormal set.
Proof
Since is symmetric, by Theorem 8.3.4, is orthogonally diagonalizabl,
that is, there exist an orthogonal matrix and a diagonal matrix such
that . Hence the main diagonal entries of are the eigenvalues
of and the columns of are eigenvectors of . Since the columns of
- 315 -
the orthogonal matrix form an orthognormal set, the eigenvectors of
are orthonormal.
Theorem
■
8.3.6
For a square matrix of order , the following are equivalent:
(1) is orthogonally diagonalizable.
(2) has eigenvectors that are orthonormal.
(3) is symmetric.
How to find an orthogonal matrix diagonalizing a given symmetric matrix ?
Example
3
For symmetric matrix
, find an orthogonal matrix
diagonalizing .
Solution
Since the characteristic equation of is , the
eigenvalues of are , . Note that all the eigenvalues
are distinct. Hence there exist eigenvectors of that are orthogonal:
x x x .
By normalizing x x x , we get an orthogonal matrix diagonalizing
:
∴
- 316 -
■
Example
4
It can be shown that the matrix has eigenvalues
(algebraic multiplicity 2) and . Hence we need to check if
has two linearly independent eigenvectors. After eigenvector computation,
we get
x
x
that are linearly independent eigenvectors corresponding to eigenvalue
-3. Using the Gram-Schmidt Orthonormalization, we get
x ⋅ y
y
y x , y x
y
y
y
z
, z
.
y
y
We can find an eigenvector x
corresponding to the eigenvalue
and normalization gives
z
.
Therefore, the orthogonal matrix
z z z
.
- 317 -
diagonalizing
is given by
■
[Remark]
Ÿ
* Function of Matrices
There are several techniques for lifting a real function to a square matrix
function such that interesting properties are maintained. You can read the
details in the following:
Ÿ
https://en.wikipedia.org/wiki/Matrix_function
Ÿ
http://youtu.be/B--ABwoKAN4
[Automobiles with polygonal wheels and the roads
customized to the polygonal wheels]
- 318 -
8.4
Quadratic forms
Lecture Movie : http://youtu.be/vWzHWEhAd-k, http://youtu.be/lznsULrqJ_0
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-12-sec-8-4.html
A quadratic form is a polynomial each of whose terms is quadratic.
Quadratic
forms
appear
in
many
scientific
areas
including
mathematics, physics, economics, statistics, and image processing.
Symmetric matrices play an important role in analyzing quadratic
forms. In this section, we study how diagonalization of symmetric
matrices can be applied to analyse quadratic forms.
Definition
An implicit equation in variables , for a quadratic curve can
expressed as
.
(1)
.
This can be rewritten in matrix-vector form as follows:
.
[Remark]
(2)
Graph for a quadratic curve (conic section)
The following are the types of conic sections:
① Non-degenerate conic sections: Circle, Ellipse, Parabola, Hyperbola. See
Figure 1.
② Imaginary conic section: There are no points ∈ satisfying (1)
③ Degenerate conic section: The graph of the equation (1) is one point, one
line, a pair of lines, or having no points.
Figure 1
- 319 -
[Remark]
Conic Sections in the Standard Position
(Ellipse)
Ÿ
(3)
Ÿ
(Hyperbola)
or
(4)
Ÿ
or (Parabola, )
(5)
Example
1
(Non-degenerate conic section)
Since the equation can be written as , the
graph of this equation is an ellipse. The equation has
the standard form and hence its graph is a hyperbola. Since
the equation
can be put into
, its graph is a
parabola.
Sage
http://sage.skku.edu or
- 320 -
http://mathlab.knou.ac.kr:8080/
var('x,y')
implicit_plot(9*x^2+4*y^2-144==0,(x,-8,8),(y,-8,8),axes=True,figsize=5)
■
Example
2
(Degenerate conic section)
The graph of the equation is the -axis. The graph of
consists
of
the
two
horizontal
lines
,
.
The
graph
of
consists of the two lines and . The graph of
consists of one point . The graph of has
no points.
■
The graph of a quadratic equations with both and terms or both and
terms is a translation of a conic section in the standard position.
Example
3
Let us plot the graph of
. By completing
squares in , we get
.
(6)
Hence by using the substitutions ′ ′ , we get
′
′
- 321 -
in the ′ ′-coordinate plane. This equation gives a hyperbola of the
standard position in the ′ ′-coordinate plane. Hence the graph of the
equation (6) is obtained by translating the hyperbola in the standard
position 3 units along the -axis and 1 unit along the -axis.
Sage
http://sage.skku.edu or
http://mathlab.knou.ac.kr:8080/
var('x,y')
c1=implicit_plot(x^2/3-y^2/3-1==0,(x,-8,8),(y,-8,8),axes=True,figsize=5,color
='red')
c2=implicit_plot((x-3)^2/3-(y-1)^2/3-1==0,(x,-8,8),(y,-8,8),axes=True,figsize=
5,color='blue')
c1+c2
■
Quadratic Form
Definition
[Quadratic Form]
(7)
is called the quadratic form of the quadratic equation (1).
Example
4
The quadratic equations are quadratic forms, but
the quadratic equation has a linear term and
- 322 -
constant term 1 and hence it is not a quadratic form.
■
A quadratic form can be written in the form of x x. For example,
or
.
This means that the matrix above is not unique.
We will use a symmetric matrix to write a quadratic form:
.
We use symmetric matrices to represent quadratic forms because symmetric
matrices are orthogonally diagonalizable.
Definition
Let be a symmetric matrix of order and x
for
⋮
real values … . Then x 〈 x x〉 x x
is
called a quadratic form in ℝ .
For a quadratic form in and , the -term is called a cross-product term.
Using orthogonal diagonalization, we can eliminate the cross-product term.
- 323 -
For a quadratic form
x x x ,
the matrix
is symmetric, we can find orthonormal eigenvectors v v
corresponding to the eigenvalues
orthogonal and
,
of
. The matrix v v is
. Since
orthogonally diagonalizes , that is,
we can switch the roles of v and v by switching the roles of and ,
without loss of generality, we can assume det .
cos sin
Therefore, we can consider as the rotation matrix
in . Let
sin cos
′
x x ′ for some x′ . Then
′
x x x x′ x′ x′ x′
′
′ ′ ′ ′
′
and hence
is a quadratic form without any cross-product term in the ′ ′
-coordinate system. Therefore, we get the following theorem.
Theorem
8.4.1 [Diagonalization of a Quadratic Form]
Suppose a symmetric matrix × has as its eigenvalues.
Then, by rotating the coordinate axes, the quadratic form
x x x
can be written as follows in the ′ ′-coordinate system
x ′ ′ .
(8)
If the determinant of is 1 and diagonalizes , then the rotation
can be obtained by x x′ or x x′.
Example
5
Using diagonalization of a quadratic form, determine which conic section
the following quadratic equation describes.
- 324 -
.
(9)
Solution
The quadratic equation can be written as
.
x x
Since
the
characteristic
equation
of
,
the
the
symmetric
eigenvalues
matrix
of
is
are
. By Theorem 8.4.1, x x x ′ ′ . Hence, in
the new coordinate system, the quadratic equation becomes
′ ′ .
Since eigenvectors corresponding to are
v
v
,
respectively,
the
orthogonal
matrix
diagonalizing is
cos ° sin °
sin ° cos ° .
Therefore ′′-coordinate axes are obtained by rotating the -axis 45°
clockwise and the equation (9) is an ellipse in the standard position
relative to the ′′-coordinate system.
Example
6
■
Sketch the graph of the following quadratic equation
.
Solution
- 325 -
(10)
Letting
x ,
we
can
rewrite
the
equation (10) as follows:
x x x .
(11)
Using rotation we first eliminate the cross-product terms. Since the
characteristic equation of is
,
the eigenvalues of
are ,
orthonormal eigenvectors are
and their corresponding
v , v
, respectively.
Hence we can take v ∶v
.
Using axis rotation x x′, we get x x ′ ′
and x
x′ ′ and hence from (11) we obtain
′ ′ ′ .
We
now
use
horizontal
translation
(12)
to
remove
′-term
in
(12).
By
completing the squares in (12) we get
′ ′ ′ .
That is, ′ ′ . Therefore, the equation (12) repesents
an ellipse in the ″ ″ -coordinate system
″
″
where the ″ ″ -coordinate
system is obtained by
horizontally translating the ′ ′
-coordinate system 1 unit along
the ′ -axis.
- 326 -
■
[Remark]
Ÿ
Simulation for quadratic forms
http://www.geogebratube.org/student/m121534
Surface in 3-dimensional space
Let
(13)
Then, after diagonalization, we get
′ ′
(14)
in the rotated ′′ -coordinate system. This enables us to identify the graph of the
equation (13) in ℝ .
In equation (14), if both , are positive, then the graph of equation (14) is a
paraboloid opening upward (see figure (a) below). If both and are negative,
then the graph is a paraboloid opening downward (see figure (b) below). Since the
horizontal cross-section of each paraboloid is an ellipse, the above graphs are
called elliptic paraboloids.
- 327 -
Elliptic Paraboids
In (14) if both of and are nonzero but have different signs, then
graphs looks like a saddle in (a) and is called a
the
hyperbolic paraboloid. If one of
and is zero, then the graph is parabolic cylinder in (b).
Example
7
Show that the graph of the following equation is an elliptic paraboloid
and sketch its cross-section at .
(15)
Solution
The quadratic form in (15) can be written as
. We
first find an orthogonal matrix
. It can be shown that
diagonalizing the symmetric matrix
, and hence using
x x′, we can transform (15) into the following:
′ ′
The
equation
(16)
(16)
represents
an
elliptic
paraboloid
in
the
′ ′
-coordinate system. Note that the ′′-coordinate system is obtained by
- 328 -
rotating the
-coordinate by angle
counterclockwise. Hence, in
x x′, is given by
cos sin
sin cos
and tan . Now we sketch the cross-section of equation (15) at
′
′
. By substituting into (16), we get and hence
the graph looks like the following:
□
Sage
Let use Sage to graph equation (15)
http://sage.skku.edu
① Computing eigenvalues of
A=matrix(2, 2, [34, -12, -12, 41])
print A.eigenvalues()
[50, 25]
② Computing eigenvectors of
print A.eigenvectors_right()
[(50, [(1, -4/3)], 1),
(25, [(1, 3/4)], 1)]
③ Computing diagonalizing
G=matrix([[1, 3/4], [1, -4/3]])
# Constructing a matrix whose columns
- 329 -
# are eigenvectors
P=matrix([1/G.row(j).norm()*G.row(j) for j in range(0,2)])
# Normalizing the row vectors (The orthogonality follows from the fact #
that the eigenvalues are distinct)
P=P.transpose()
# Constructing a matrix whose columns are orthonormal
# eigenvectors
print P
[ 4/5
3/5]
[ 3/5 –4/5]
④ Sketching two ellipses simultaneously
var('u, v')
s=vector([u, v])
B=P.transpose()*A*P
p1=implicit_plot(s*A*s==50, (u, -2, 2), (v, -2, 2), axes='true')
p2=implicit_plot(s*B*s==50, (u, -2, 2), (v, -2, 2), color='red', axes='true')
show(p1+p2)
# Ploting two graphs simultaneously
■
- 330 -
8.5
*Applications of Quadratic forms
Lecture Movie : http://youtu.be/cOW9qT64e0g
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-12-sec-8-5.html
By the theorem of principal axis (theorem 8.4.1), the graph of a 3D
curve is shown in the form of an plane, ellipse or parabola in 2D. The
specific shape is uniquely determined by signs of eigenvalues of the
corresponding quadratic form. In this section, we define the sign of the
quadratic form to identify the type of graph of given quadratic forms,
and learn how to obtain the extrema of multivariable functions using
them.
Given a system of springs and masses, there will be one quadratic form that
represents the kinetic energy of the system, and another which represents the
potential energy of the system in position variables. It can be found in the
following websites:
l Application of Quadratic Forms and Sage:
http://matrix.skku.ac.kr/2014-Album/Quadratic-form/
l http://ocw.mit.edu/ans7870/18/18.013a/textbook/HTML/chapter32/section09.html
- 331 -
SVD and generalized eigenvectors
8.6
Lecture Movie : https://youtu.be/ejCge6Zjf1M, http://youtu.be/7-qG-A8nXmo
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-12-sec-8-6.html
We have learned that symmetric matrices are diagonalizable. We
now extend the concept of diagonalization to × matrices (not
necessarily
square
or
symmetric)
resulting
in
a
matrix
decomposition and study pseudoinverses and least squares solution
using the matrix decomposition.
8.6.1 [Singluar Value Decomposition]
Theorem
Let be an × real matrix. Then there exist orthogonal matrices
of order and of order , and an × matrix such that
,
(1)
where the main diagonal entries of are positive and listed in the
monotonically decreasing order, and is a zero-matrix. That is,
u u ⋯ u u
⋯ u
⋮ ⋮
⋱
⋯
⋯
⋮
⋮
⋮
⋯
⋯
⋯
⋯
⋯
v
⋮
v
,
⋮
v
⋮
where ≥ ≥ ⋯ ≥ .
Definition
Equation (1) is called the singular value decomposition (SVD) of .
The main diagonal entries of the matrix are called the singular
values of . In addition, the columns of are called the left singular
vectors of and the columns of are called the right singular
vectors of .
The following theorem shows that matrices and are orthogonal matrices
diagonalizing and , respectively.
- 332 -
Theorem
8.6.2
Let the decomposition be the singular value decomposition
(SVD) of an × ≥ matrix where are positive diagonal
entries of . Then
(1) diag … … × .
(2) diag … … × .
Since , it follows that . Hence, by considering,
Proof
and , we get
′ diag … … ×
and
diag ⋯ … × ,
respectively.
Example
1
■
Find the SVD of
.
Solution
The eigenvalues of
are
and
hence
the
singular
values
of
are
.
A unit eigenvector of corresponding to is v
, and a
unit eigenvector of corresponding to is v
. We
can also find unit eigenvectors of :
u v
u v
.
- 333 -
Hence we get
v v
u u
.
Therefore, the SVD of is
Sage
⇔
.
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
① Computing the singular values of and eigenvectors of
A=matrix([[sqrt(3), 2], [0, sqrt(3)]])
B=A.transpose()*A
eig=B.eigenvalues()
sv=[sqrt(i) for i in eig]
# Computing singular values
print B.eigenvectors_right()
# Computing eigenvectors of
[(9, [(1, sqrt(3))], 1), (1, [(1, -1/3*sqrt(3))], 1)]
② Computing
G=matrix([[1, sqrt(3)], [1, -1/3*sqrt(3)]])
Vh=matrix([1/G.row(j).norm()*G.row(j) for j in range(0,2)])
Vh=Vh.simplify()
# Transpose of V
print Vh
[
1/2 1/2*sqrt(3)]
[1/2*sqrt(3)
-1/2]
③ Computing eigenvectors of
C=A*A.transpose()
# Computing eigenvectors of
print C.eigenvectors_right()
[(9, [(1, 1/3*sqrt(3))], 1), (1, [(1, -sqrt(3))], 1)]
- 334 -
□
④ Computing
F=matrix([[1, 1/3*sqrt(3)], [1, -sqrt(3)]])
U=matrix([1/F.row(j).norm()*F.row(j) for j in range(0,2)])
U=U.simplify().transpose()
# U
print U
[ 1/2*sqrt(3)
[
1/2]
1/2 –1/2*sqrt(3)]
⑤ Computing diagonal matrix
S=diagonal_matrix(sv); S
[3 0]
[0 1]
⑥ Verifying
U*S*Vh
[sqrt(3)
[
2]
0 sqrt(3)]
■
Equivalent statement of invertible matrix on SVD
Theorem
8.6.3
Let be an × matrix. Then is a nonsingular matrix if and only
if every singular value of is nonzero.
Proof
Since det det , matrix is nonsingular if and only if is
nonsingular. Hence, if is nonsingular, then all the eigenvalues of
are nonzero. By Theorem 8.6.2, the singular values of are the square
roots of the positive eigenvalues of . Hence the singular values of
are nonzero.
■
- 335 -
Theorem
8.6.4
Suppose ≥ ≥ ⋯ ≥ are the singular values of an × matrix .
Then the matrix can be expressed as follows:
u v .
(R)
The equation (R) is called a rank-one decomposition of .
Note that the pseudoinverse of a matrix is important in the study of the least
squares solutions for optimization problems.
We can express an × nonsingular matrix using the SVD
.
(2)
Note that all of , , are × nonsingular matrices and , in particular,
, are orthogonal matrices. Hence the inverse of can be expressed as
.
(3)
If is not a square matrix or is singular, then (3) does not give an inverse
of . However, we can construct a pseudoinverse † of by putting in (2)
into the form (where is nonsingular).
Definition
[Pseudo-Inverse]
For an × matrix the × matrix † ′ is called a
pseudo-inverse of , where , are orthogonal matrices in the SVD
of
and ′ is
′
(where is nonsingular).
We read † as ‘dagger.’ If , then we define † .
Truncated SVD
- 336 -
http://langvillea.people.cofc.edu/DISSECTION-LAB/Emmie'sLSI-SVDModule/p5module.html
What is a truncated SVD?
We learned that singular value decomposition factors any matrix
so that
is a
. Let's take a closer look at the matrix . Remember
diagonal matrix where
⋮ ⋮
⋮
⋯
and ≥ ≥ ⋯ ≥ are the
singular values of the matrix . A full rank decomposition of is usually denoted
by where and are the matrices obtained by taking the first
columns of and , respectively. We can find a -rank approximation (or
truncated SVD) to by taking only the first k largest singular values and the first
k columns of U and V.
Example
2
Find a pseudo-inverse of
.
Solution
We first compute the (truncated) SVD1) of :
http://matrix.skku.ac.kr/2014-Album/MC.html
v
u u
.
v
Then
u
† v v
u
.
■
1) http://langvillea.people.cofc.edu/DISSECTION-LAB/Emmie'sLSI-SVDModule/p5module.html
- 337 -
http://matrix.skku.ac.kr/RPG_English/8-MA-pseudo-inverse.html
Ÿ
Sage
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
A=matrix(RDF,[[1,1],[0,1],[1,0]])
import numpy as np
Pinv=matrix(np.linalg.pinv(A))
Pinv
[ 0.333333333333 -0.333333333333
[ 0.333333333333
0.666666666667]
0.666666666667 -0.333333333333]
If × has rank , then is said to be of the full column rank. If
has full column rank, then is nonsingular. If is nonsingular, then the
pseudo-inverse of is equal to .
Theorem
8.6.5
If an × matrix has full column rank, then the pseudo-inverse of
is
† .
Proof
Let be the SVD of . Then where is nonsingular.
Then
.
Since has full column rank, is nonsingular and matrix is an ×
orthogonal matrix. Hence
and
†.
- 338 -
■
Example
3
Find the pseudo-inverse of using theorem 8.6.5.
.
Solution
Since has full column rank,
is nonsingular and
†
.
Theorem
■
8.6.6
If † is a pseudo-inverse of , then the following hold:
(1) †
(2) † † †
(3) † †
(4) † †
(5) † †
(6) †† .
Proof (Exercise)
[Remark]
A pseudo-inverse provides a tool for solving a least sqaures problem. It is known that
the least squares solution to the linear system x b is the solution to the normal
equation x b. If has full column rank, then the matrix is nonsingular
and hence
x b † b.
This means that if has full column rank, the least squares solution to x b is the
pseudo-inverse † times the vector b.
- 339 -
8.6.7
Theorem
Let be an × matrix and b be a vector in ℝ . Then x † b is
the least squares solution to x b.
Let be the SVD of with ( is nonsingular). Then
Proof
† ′
and hence † b ′ b, Since
† b ′ b ′ b
b b b,
it follows that x † b satisfies x b.
Example
4
Find
the
least
squares
line
passing
■
through
the
four
points
.
Solution
Let be an equation for the line that fits to the points
. Then, by letting x b m T , the given
condition can be written as the linear system x b for which
and b
.
Since has full column rank, we get † which is
†
x † b
.
Hence
. Therefore, the least
squares line is given by .
- 340 -
■
var('x, y')
p1=plot(x + 3/2, x, -1,3, color='blue');
p2 = text("$x+ 3/2 $", (2,2), fontsize=20, color='blue')
show(p1+p2, ymax=4, ymin=-1)
in http://matrix.skku.ac.kr/Cal-Book/part1/CS-Sec-1-3.htm
Team <3D Math> comprises Professor Sang-Gu LEE, and 3 mathematics major students
including Jaeyoon LEE, Victoria LANG, Youngjun LIM, won the prize with ‘DIY Math Tools
with 3D printer’ for the Korea Science and Technology Idea Competition 2014 co-organized
by the Korea Foundation for the Advancement of Science and Creativity, the Ministry of
Science, ICT and Future Planning, the National Museum of Science and YTN.
https://www.facebook.com/skkuscience
- 341 -
8.7
Complex eigenvalues and eigenvectors
Lecture Movie :
http://youtu.be/8_uNVj_OIAk, http://youtu.be/Ma2er-9LC_g
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-13-sec-8-7.html
We have so far focused on real eigenvalues and real eigenvectors.
However, real square matrices can have complex eigenvalues and
eigenvectors. In this section, we introduce complex vector spaces,
complex matrices, complex eigenvalues and complex eigenvectors.
Complex vector spaces
Definition
[Complex Vector Space]
The set of vectors with complex components is denoted by
… ∈ … .
If we define the vector addition and the scalar multiple of a vector in
similar to those for ℝ , then is a vector space over and its
dimension is equal to .
If
e … , e … , … , e … ,
then a vector v in can be expressed as
v e e ⋯ e where ’s
are complex numbers, and the set {e , e … e } is a basis for .
is called the standard basis for .
This basis
For a complex number ,
is called the conjugate of and
is called the modulus of . Furthermore, if we denote a complex
number as cos sin , then and tan . For a complex vector
u … , we define its conjugate as
u
… .
Ÿ
[Example] http://matrix.skku.ac.kr/RPG_English/9-VT-conjugate.html
- 342 -
Inner product
Definition
[Inner Product]
Let u … and v … be vectors in . Then
u⋅ v
⋯
u v
satisfies the following properties:
v u
(1) u v
(2) u v w u w v w
(3) u v u v
(4) v v ≥ in particular, v v ⇔ v
The inner product u⋅ v is called the Euclidean inner product for the
vector space .
Definition
Let u … v … be vectors in . Then,
using the Euclidean inner product u⋅ v, we can define the Euclidean
norm u of u and the Euclidean distance u v between u and v as
the following:
⋯ .
(1) u u⋅ u
⋯ .
(2) u v u v
If u⋅ v , then we say that u and v are orthogonal to each other.
- 343 -
1
Example
For vectors u v , compute the Euclidean
inner product and their Euclidean distance.
Solution
u⋅ v
⋅
u v u v
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
u=vector([2*I, 0, 1+3*I])
# I is the imaginary unit.
v=vector([2-I, 0, 1+3*I])
print v.hermitian_inner_product(u)
# Taking the conjugate for v
# < u, v > = v.hermitian_inner_product(u)
print (u-v).norm()
4*I + 8
sqrt(13)
■
Complex Eigenvalues and Eigenvectors of Real Matrices
We should first define complex eigenvalues and complex eigenvectors along with
example.
Theorem
8.7.1
If is a complex eigenvalue of an × real matrix and x is its
corresponding eigenvector of , then the complex conjugate
of is
also an eigenvalue of and
x is an eigenvector corresponding to
.
Proof
Since an eigenvector is a nonzero vector, x≠ and
x≠ . Since x x
and is real (i.e.,
), it follows that
x
x
x
x.
- 344 -
■
Eigenvalues of Real Symmetric Matrices
8.7.2
Theorem
If is a real symmetric matrix, then all the eigenvalues of are real
numbers.
Proof
Let be an eigenvalue of , that is, there exists a nonzero vector x such
that x x. By multiplying both sides by x x on the left-hand side, we
x x
get x x x x x x x⋅ x x . Hence
. Since x is a
x
nonzero real number, we just need to show that x x is a real number.
Note that
x x x
x x
x
x x
x x x x x x.
Therefore, x x is a real number.
Example
2
■
Show that the eigenvalues of
are ± . In addition show
that if ≠ , then can be decomposed into
cos sin
,
sin
cos
where is the angle between the -axis and the line passing through
the origin and the point .
Solution
Since the characteristic equation of is , the eigenvalues
of
are
± .
If
≠ ,
then
cos ,
sin.
Therefore,
cos sin
.
sin
cos
- 345 -
■
“Pure mathematics is, in its way, the poetry of logical
ideas."
Albert Einstein (1879–1955)
http://en.wikipedia.org/wiki/Albert_E
instein
He developed the general theory of
relativity, one of the two pillars of
modern physics (alongside quantum
mechanics)
- 346 -
8.8
Hermitian, Unitary, Normal Matrices
Lecture Movie : http://youtu.be/8_uNVj_OIAk, http://youtu.be/GLGwj6tzd60
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-13-sec-8-8.html
We used to denote the set of all × real matrices. In this
section, we introduce to denote the set of all
complex matrices.
×
Symmetric matrices and orthogonal matrices in
can be generalized to be Hermitian matrices and unitary
matrices in , We shall further study the diagonalization of
Hermitian and Unitary matrices.
Conjugate Transpose
[Conjugate Transpose]
Definition
For a matrix ∈ × ,
is defined by
∈ × .
The transpose
of the complex conjugate of
conjugate
transpose
× .
and
is
denoted
by
is called the
,
that
is,
[Remark]
Ÿ
The Euclidean inner product in : u⋅ v v u , u u u
Ÿ
If a matrix is real, then .
Example
1
For
matrices
,
conjugate transposes are
- 347 -
,
, their
,
,
.
■
8.8.1 [Properties of Conjugate Transpose]
Theorem
For complex matrices and a complex number , the following
hold:
(1) .
(2) .
(3) .
(4) .
Proof of the above theorem is easy to verify and left as exercises.
Hermitian Matrix
[Hermitian Matrix]
Definition
If a complex square matrix
satisfying ,
is called a
Hermitian matrix.
Example
2
In
Example
1,
≠ and hence is not Hermitian. However, since
■
, is Hermitian.
Theorem
8.8.2 [Properties of Hermitian Matrix]
Suppose ∈ is Hermitian. Then the following hold:
(1) For any vector x ∈ , the product x x is a real number.
(2) Every eigenvalue of is a real number.
(3) Eigenvectors of corresponding to distinct eigenvalues are
orthogonal to each other.
http://people.math.gatech.edu/~meyer/MA6701/module11.pdf
- 348 -
Example
3
Let
. Since , is Hermitian. The characteristic
and hence the
eigenvalues of are 1,
, which confirms that all the
equation
of
is
eigenvalues of a Hermitian matrix are real numbers. Furthermore, it
can be shown that the eigenvectors x, y, and z
x , y
, z
corresponding to ,
, and
, respectively, are
orthogonal to each other.
■
Skew-Hermitian Matrices
[Skew-Hermitian Matrix]
Definition
If a complex square matrix satisfies , then is called a
skew-Hermitian matrix.
Example
4
It
can
be
verified
that
both
matrices
and
below
are
skew-Hermitian:
and
■
Each matrix ∈ can be expressed as , where is Hermitian
and is skew-Hermitian. In particular, since is Hermitian and is
skew-Hermitian, every complex square matrix can be rewritten as
.
- 349 -
Unitary Matrices
Definition
[Unitary Matrix]
If matrix ∈ satisfies , then is called a unitary
matrix. If is unitary, then
. In addition, if the th column
vector of is denoted by u , then
u ⋅ u u u u u
.
≠
Therefore, is a unitary matrix if and only if the columns of form
an orthonormal set in .
Example
5
Show that the following matrix is unitary:
Solution
Since
,
the
product
. Hence a a is a
unitary matrix. We can also show that
a i ⋅ a j a j a i
i j
.
i ≠ j
.
For example, a ⋅ a a a
- 350 -
■
8.8.3 [Properties of a Unitary Matrix]
Theorem
Suppose has the Euclidean inner product and
is a unitary
matrix. Then the following hold:
(1) For x y ∈ , x ⋅ y x⋅ y , which implies x x .
(2) If is an eigenvalue of , then .
(3)
Eigenvectors
of
corresponding
to
distinct
eigenvalues
are
orthogonal to each other.
The property x x of a unitary matrix shows that a unitary matrix is an
isometry, preserving the norm.
Unitary Similarity and Unitarily Diagonalizable Matrices
[Unitary Similarity and Unitary Diagonalization]
Definition
For matrices ∈ , if there exists a unitary matrix such
that , then we say that and are unitarily similar to
each other. Furthermore, if
∈ is unitarily similar to a
diagonal matrix, then is called unitarily diagonalizable.
Example
6
Let
and
is
a
unitary
matrix
and
. Then it can be checked that
.
Therefore, is unitarily diagonalizable.
■
If ∈ is unitarily diagonalizable, then there exists a unitary matrix
such
that
diag …
⋯
and
hence
.
, we get,
⋯ ⋯ .
- 351 -
Letting
This implies that the column of the unitary matrix is a unit eigenvector
of corresponding to the eigenvalue .
Example
7
Find a unitary matrix diagonalizing matrix
.
Solution
The
eigenvalues
of
are
and
their
corresponding
eigenvectors are
,
⇒ x
Letting
.
⇒ x
x
x
, u
u
x
x
and u u
, it follows that
,
where is a unitary matrix.
■
Schur’s Theorem
Transforming a complex square matrix into an upper triangular matrix
Theorem
8.8.4 [Schur’s Theorem]
A square matrix is unitarily similar to an upper triangular matrix
whose main diagonal entries are the eigenvalues of . That is, there
exists a unitary matrix and an upper triangular matrix such that
∈ , ( ),
where ’s are eigenvalues of .
Proof Let
… be the eigenvalues of . We prove this by mathematical
induction. First, if , then the statement holds because . We now
assume that the statement is true for any square matrix of order less than or
equal to .
- 352 -
① Let x be an eigenvector corresponding to eigenvalue .
② By the Gram-Schmidt Orthonormalization, there exists an orthonormal basis for
including
x , say x z … z .
③ Since is orthonormal, the matrix ≡ x z ⋯ z is a unitary matrix. In
addition, since x x , the first column of is x . Hence is of
the following form:
.
＊
,
where ∈ . Since , the eigenvalues of are
⋯ .
④ By the induction hypothesis, there exists a unitary matrix
∈ such
that
.
⋮
⋱⋮
⋯
⑤ Letting ≡
⋮
⋯
∈ , we get
⋮
.
⋮ ⋱ ⋮
⋯
Since ≡ is a unitary matrix, the result follows.
■
[Lecture on this proof] http://youtu.be/lL0VdTStJDM
Not every square matrix is unitarily diagonalizable. (see Chapter 10)
[Issai Schur(1875-1941, Germany)] http://en.wikipedia.org/wiki/Issai_Schur
- 353 -
Normal matrix
Definition
[Normal Matrix]
If matrix ∈ satisfies
,
then is called normal matrix.
Example
8
It can be shown that the following matrices and are normal:
■
Example
9
A Hermitian matrix satisfies and hence . This
implies that any Hermitian matrix is normal. In addition, since a unitary
matrix satisfies , it is a normal matrix.
■
Equivalent Conditions for a Matrix to be Normal
Theorem
8.8.5
For matrix ∈ , the following are equivalent:
(1) is unitarily diagonalizable.
(2) is a normal matrix.
(3) has orthonormal eigenvectors.
Example
10
Let
and
.
Show that is a normal matrix and the columns of are orthonormal
eigenvectors of .
Solution
- 354 -
, and hence is a normal matrix.
Since
Letting
≡
u u , we get
u ,
u
u .
u
Thus u and u are eigenvectors of . In addition, since u u
and u ⋅ u u u , u and u are orthonormal eigenvectors of .
Example
11
■
Unitarily diagonalize
.
Solution
Note that matrix is Hermitian and its eigenvalues are .
An eigenvector corresponding to is x . By normalizing it, we
get
x
u
.
x
Similarly,
we
can
get
a
unit
eigenvector
u
corresponding to .
i
Taking u u
, we get
.
■
[Remark]
Although not every matrix is diagonalizable, using the Schur’s Theorem, we
can obtain an upper triangular matrix (close to a diagonal matrix) similar to
. The upper triangular matrix is called the Jordan canonical form of .
The Jordan canonical form will be discussed in Chapter 10.
- 355 -
8.9
*Linear system of differential equations
Lecture Movie : http://www.youtube.com/watch?v=c0y5DcNQ8gs
Lab : http://matrix.skku.ac.kr/knou-knowls/cla-week-11-sec-8-1.html
Many problems in science and engneering can be written as a
mathematical
problem
of
solving
linear
system
of
differential
equations. In this section, we learn how to solve linear system of
differential equations by using a matrix diagonalization.
Details can be found in the following websites:
http://www.math.psu.edu/tseng/class/Math251/Notes-LinearSystems.pdf
http://matrix.skku.ac.kr/CLAMC/chap8/Page83.htm
http://matrix.skku.ac.kr/CLAMC/chap8/Page84.htm
http://matrix.skku.ac.kr/CLAMC/chap8/Page85.htm
http://matrix.skku.ac.kr/CLAMC/chap8/Page86.htm
http://matrix.skku.ac.kr/CLAMC/chap8/Page87.htm
“It is through science that we prove, but through intuition that we
discover. ”
Jules Henri Poincaré (1854 – 1912)
http://en.wikipedia.org/wiki/Henri_Poincar%C3%A9
He is often described as a polymath, and in mathematics as
The Last Universalist by Eric Temple Bell,[3] since he excelled in all fields
of the discipline as it existed during his lifetime.
- 356 -
Exercises
Chapter 8
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
Suppose a linear transformation
and
is defined by
ℝ . Find the matrix
is an ordered basis for
representation of relative to the ordered basis .
Problem 2
and let v v ,
Let ℝ → ℝ be defined by
v ′ v ′ v ′ be ordered bases for , , respectively, where
v ,
v ′ ,
v ,
v ′ ,
v ′ .
Find
the
matrix
representation of with respect to the ordered bases and .
Problem 3
Suppose a linear transformation ℝ → ℝ is defined by
and
,
are ordered bases for
.
(1) Find the matrix representation of relative to the ordered basis .
(2) Find the transition matrix from to .
(3) Compute .
Solution
(1)
⇒
⇒
- 357 -
(2)
⇒ ,
,
⇒
(3)
Determine
Problem 4
if
the
given
diagonalizable, find matrix
■
matrix
is
diagonalizing
diagonal matrix such that
diagonalizable.
.
(2) .
Solution
Sage:
-----------------------------------------------A=matrix([[2,1,1],[1,2,1],[1,1,2]])
print A.eigenvectors_right()
-----------------------------------------------[(4, [(1, 1, 1)], 1), (1, [(1, 0, -1),(0, 1, -1)], 2)]
-----------------------------------------------x1=vector([1, 1, 1])
x2=vector([1, 0, -1])
x3=vector([0, 1, -1])
P=column_matrix([x1,x2,x3])
print P
print P.det()
-----------------------------------------------[ 1
1
0]
[ 1
0
1]
[ 1 -1 -1]
3
■
- 358 -
is
and the associated
(1)
.
If
Problem 5
Find the algebraic and geometric multiplicity of each eigenvalue of :
.
Find matrix orthogonally diagonalizing matrix and the diagonal
Problem 6
matrix such that , using Sage.
.
Solution
Sage:
A=matrix(QQ,3,3,[1,2,2,2,1,-2,2,-2,1])
A.eigenvalues()
[-3, 3, 3]
A=matrix(QQ,3,3,[1,2,2,2,1,-2,2,-2,1])
A.eigenvevtors_right()
[(-3, [(1, -1, -1)], 1), (3, [(1, 0, 1),(0, 1, -1)], 2)]
C=matrix(3,3,[0,2/sqrt(6),1/sqrt(3),1/sqrt(2),1/sqrt(6),-1/sqrt(3),-1/sqrt(2),1/s
qrt(6),-1/sqrt(3)])
C.transpose()*C
[1 0 0]
[0 1 0]
[0 0 1]
Problem 7
■
In each of the following matrix and set of linear independent
eigenvectors of are given. Find an orthogonal matrix and a
diagonal matrix such that .
, .
(1)
(2)
, .
- 359 -
Problem 8
Compute x x x when
x
Solution
,
x x = = .
Problem 9
■
Write the following expression as a quadratic form x x :
.
Solution
x x = .
Sage :
_________________________________________
var('x','y','z')
a=2;
b=3;
c=1;
d=1;
e=-2;
f=3
A=matrix(3,3,[a,d/2,e/2,d/2,b,f/2,e/2,f/2,c])
X=matrix(3,1,[x,y,z])
print expand(X.transpose() * A * X)
__________________________________________
[2*x^2 + x*y - 2*x*z + 3*y^2 + 3*y*z + z^2].
Problem 10
Eliminate the cross-product term from the following:
.
Problem 11
Sketch the graph of the following equation:
Solution
x x x where x .
v
=> => v
- 360 -
■
=>
.
=>
x x′ and ′
′
′ .
′
′
Sage :
_________________________________________
var('x y')
f=x^2+4*x*y+4*y^2+6*x+2*y-25
implicit_plot(f==0, (x,-10,10), (y,-10,10))
__________________________________________
■
Problem 12
Eliminate
the
cross-product
terms
from
the
quadratic
by properly rotating the axes.
Problem 13
Compute the singular values of matrix :
.
Problem 14
Find the SVD of :
.
Solution
,
Sage :
_________________________________________
aat = matrix(QQ,2,2,[2,1,1,2])
ata = matrix(QQ,3,3,[1,1,0,1,2,1,0,1,1])
print aat.right_eigenvectors()
print ata.right_eigenvectors()
__________________________________________
[(3, [(1, 1)], 1), (1, [(1, -1)], 1)]
[(3, [(1, 2, 1)], 1), (1, [(1, 0, -1)], 1), (0, [(1, -1, 1)], 1)]
- 361 -
surface
,
=>
Problem 15
,
,
.
■
In the below the SVD of is given. Find † .
.
Problem 16
The following matrix has full column rank. Find its pseudo-inverse.
.
Solution
Sage :
_________________________________________
A=matrix(QQ,[[1,1],[0,2],[3,7]])
print A.rank()
B=A.transpose() * A
Pseudo=B^-1*A.transpose()
print Pseudo
__________________________________________
2
[
4/7 -11/14
[ -3/14
5/14
1/7]
1/14]
∴ †
Problem 17
.
■
For given vectors u v , compute Euclidean
inner products u⋅ v and v⋅ u .
- 362 -
Problem 18
Let u
, v
be vectors in with Euclidean inner
product defined. Compute the norms u and v , and u v .
Problem 19
Find the eigenvalues of
and a basis for the eigenspace
associated with each eigenvalue.
Problem 20
Find any invertible matrix diagonalizing a given matrix which
has complex eigenvalues?
.
Solution
Sage :
_________________________________________
A=matrix(QQ,[[6,-4],[8,-2]])
print A.eigenvalues()
print A.eigenvectors_right()
__________________________________________
[2 - 4*I, 2 + 4*I][(2 - 4*I, [(1, 1 + 1*I)], 1), (2 + 4*I, [(1, 1 - 1*I)], 1)].
∴
Problem 21
Find the conjugate transpose of the following matrix :
.
Problem 22
Determine which matrices in the below are Hermitian.
(a)
(b)
- 363 -
■
(c)
(d)
(e)
Problem 23
×
(b)
×
Show that the following matrix is unitary, and find its inverse .
Problem P1
Replace each × by a complex number to make matrix Hermitian.
×
Problem 25
Determine if each matrix in the below is unitary.
(a)
Problem 24
(f)
Let v and v and suppose → is a linear
transformation. If v v and v v , what is the standard matrix of
? In addition, if v v , what is the matrix representation of
relative to the ordered basis ?
Problem P2
Suppose the following polynomial is the characteristic polynomial
- 364 -
of a square matrix .
.
(1) What is the order of ?
(2) If the number of linear independent eigenvectors of cannot exceed 3, is
the matrix diagonalizable?
(3) What is the dimension of each eigenspace of ?
(4) Suppose is diagonalizable. Discuss about a relationship between the
algebraic multiplicity of each eigenvalue and the dimension of the solution
space to the homogeneous linear system x .
Problem P3
(1) Suppose the following are the eigenvalues of a × symmetric matrix
and their corresponding eigenvectors:
v v v .
Find the matrix .
(2) Determine if there exists a × matrix whose eigenvalues and their
corresponding eigenvectors are given in the below:
v v v
Problem P4
Show
has non-real eigenvalues.
Problem P5
Show that if ∈ ( ) is skew-Hermitian, then every eigenvalue of is a
pure imaginary number.
Problem P6
Use properties (1) and (3) of inner product in Section 8.7 to
show that u v u v .
- 365 -
9
Chapter
Vector Space
9.1 Axioms of a Vector Space
9.2 Inner product; *Fourier series
9.3 Isomorphism
9.4 Exercises
The operations used in vector addition and scalar
multiple are not limited to the theory but can be
applied
to
all
areas
in
society.
For
example,
consider objects around you as vectors and make a
set of vectors, then create two proper operations
(vector
addition
and
scalar
multiple)
from
the
relations between
the objects. If these two operations satisfy the two basic laws and 8 operation
properties, the set becomes a mathematical vector space (or linear space). Thus
we can use all properties of a vector space and can analyze the set theoretically
and apply them to real problems.
In this chapter, we give a definition of a vector space and a general theory of a
vector space.
- 366 -
Axioms of a Vector Space
9.1
Ref site : http://youtu.be/m9ru-F7EvNg, http://youtu.be/beXWYXYtAaI
Lab site: http://matrix.skku.ac.kr/knou-knowls/cla-week-14-sec-9-1.html
The concept of vectors has been extended to -tuples in ℝ
from
the arrows in the 2-dimensional or 3-dimensional space. In Chapter
1, we defined the addition and the scalar multiple in the
-dimensional space ℝ . In this section, we extend the concept of
the -dimensional space ℝ
to an -dimensional vector space.
Vector Spaces
Definition
If
a set
[Vector space]
≠ has two well-defined binary operations, vector
addition (A) ‘ ’ and scalar multiplication (SM) ‘⋅ ’, and for any
x y z ∈ and ∈ℝ , two basic laws
A. x y∈
⇒
x y∈ .
SM. x ∈ ∈ℝ
⇒
x ∈ .
and the following eight laws hold, then we say that the set forms a
vector space over ℝ with the given two operations, and we denote it
by ⋅ (simply if there is no confusion). Elements of are
called vectors.
A1. x y y x.
A2. x y z x y z .
A3. For any x ∈ , there exists a unique element in such that
x x.
A4. For each element x of , there exists a unique x such that
x x .
SM1. x y x y.
SM2. x x x.
SM3. x x x .
SM4. x x.
- 367 -
The vector satisfying A3 is called a zero vector, and the vector x
satisfying A4 is called a negative vector of x.
In general, the two operations defining a vector space are important. Therefore,
it is better to write ⋅ instead of just .
Example
1
For vectors x y in ℝ and a scalar ∈ℝ , the
vector sum x y and a scalar multiple x by ∈ℝ are defined as
(1) x y .
(2) x .
The set ℝ ⋅ together with the above operations forms a vector
space over the set ℝ of real numbers.
■
Example
2
For vectors in ℝ
x ,
⋮
y
⋮
and a scalar ∈ℝ , the sum of two vectors x y and the scalar
multiple of x by is defined by
(1) x y
⋮
and
(2) x
.
⋮
The set ℝ form a vector space together with the above two operations.
■
- 368 -
9.1.1
Theorem
Let be a vector space. Let x ∈ and ∈ℝ . Then the following hold.
(1) x .
(2) .
(3) x x .
(4) x
⇔
or x .
Zeo Vector Space
Definition
Let . For a scalar ∈ℝ , if the addition and scalar multiple
are defined as
, then
forms a vector space. This vector space is called a zero vector
space.
Example
3
Let × be the set of all × matrices with real entries. That is,
× × ∈ℝ ≤ ≤ ≤ ≤ .
When , we denote × by .
If
×
is
equipped
with
the
matrix
addition
and
the
scalar
multiplication, then × form a vector space × ⋅ over ℝ .
Then
the
zero
vector
is
the
zero
matrix
and
for
each
∈ × , the negative vector is . Note that each vector
means an × matrix with real entries.
- 369 -
■
Example
4
Let ℰ ℝ be the set of all continuous functions from ℝ to ℝ . That is,
ℰ ℝ ∣ ℝ → ℝ is continuous}
Let ∈ ℰ ℝ and a scalar ∈ℝ , define the addition and the scalar
multiple as
, .
Then ℰ ℝ forms a vector space ℰ ℝ ⋅ over ℝ .
Now the zero vector is and for each ∈ ℰ ℝ , is defined
as .
Vectors in ℰ ℝ mean continuous functions from ℝ to ℝ .
Example
5
■
Let be the set of all polynomials of degree at most with real
coefficients. In other words,
⋯ … ∈ℝ
Let
⋯ ⋯ ∈
and
a
scalar
∈ℝ . The addition and the scalar multiplication are defined as
⋯
⋯ .
Then forms a vector space ⋅ over ℝ . Now the zero vector
is ⋯ and each ∈ has the negative vector
defined as
⋯ .
Vectors in
means polynomials of degree at most
coefficients.
with real
■
- 370 -
Subspace
Definition
Let be a vector space and ≠ be a subset of . If forms a
vector space with the operations defined in , then
is called a
subspace of .
Example
6
If ⋅ is a vector space, and itself are subspaces of .
■
In fact, the only subspaces of are , , and lines passing through the
origin. (see section 3.4
Example
3 ).
In ℝ , only subspaces are (i) Null Spaces, (ii) ℝ , (iii) lines passing through
origin and (iv) planes passing through origins.
How to determine a subspace? (the 2-step subspace test)
Theorem
9.1.2 [the 2-step subspace test]
Let a set ⋅ be a vector space and ≠ ∅ be a subset of .
A necessary and sufficient condition for to be a subspace of is
(1) x y∈ ⇒
x y∈ (closed under vector addition )
(2) x ∈ ∈ℝ ⇒ x ∈ (closed under scalar multiple⋅ )
- 371 -
Example
7
∈ℝ
Show that
is a subspace of the vector space
× ⋅ .
Solution
Note that × is a vector space under the matrix addition and the
scalar multiplication. Let
x
y ∈ ∈ℝ .
The following two conditions are satisfied.
(1) x y
∈
∈ .
(2) x
Hence by Theorem 9.1.2, ⋅ is a subspace of × ⋅ . ■
Example
8
The set of invertible matrices of order is not a subspace of the vector
space .
Solution
One can make a non-invertible matrix by adding two invertible matrices.
For example,
.
Example
9
■
Let be a vector space and x x … x ⊆ . Show that the set
x x ⋯ x … ∈ℝ
is a subspace of . Note that , linear span of the set .
Solution
Suppose that x y∈ , ∈ℝ . Then for ∈ℝ … ,
- 372 -
x x x ⋯ x y x x ⋯ x .
Thus
x y x x ⋯ x ,
x x x ⋯ x .
∴
x y ∈ x ∈
Therefore is a subspace of .
■
Liinear independence and linear dependence
[Linear independence and linear dependence]
Definition
If a subset
v v … v of a vector space satisfies the following
condition, it is called linearly independent.
v v ⋯ v
and
if
the
set
is not
⇒
⋯
linearly independent,
it
is called
linearly
dependent. Hence being linearly independent means that there exist
…
v v ⋯ v .
some
scalars
not
- 373 -
all
zero
such
that
Remark
Ÿ
Example
Linear combination in 2-dimensional space - linear dependence
(computer simulation)
http://www.geogebratube.org/student/m57551
10
Let
,
,
,
. Since
⇒
is a linearly independent set of .
Example
Example
Example
11
12
13
■
Let
,
,
. Since , is
a linearly dependent set of .
■
The subset … of is linearly independent.
■
Let be a subset of . Then since
,
the set is linearly dependent.
- 374 -
■
Basis
[basis and dimension]
Definition
If a subset
(≠ ) of a vector space
satisfies the following
conditions, is a basis of .
(1) span .
(2) is linearly independent.
In this case, the number of elements of the basis , , is called the
dimension of , denoted by dim .
Example
14
The set in
Example
10
consisting of , , ,
is a basis of . Thus dim × . On the other hand,
the
set
in
Example
12
…
is
a
basis
of
.
Thus
dim . These bases play a role similar to the standard basis of
, hence and are called standard bases.
Example
15
Show that is a basis of .
Solution
⇔
⇔
Since , is linearly independent.
Next, given ∈ , the existence of such that
is guaranteed since the coefficient matrix of the linear system
- 375 -
■
that is,
is invertible. Thus spans . Hence is a basis of .
■
Linear independence of continuous function: Wronskian
Theorem
9.1.3 [Wronski's Test]
If … are times differentiable on the interval
∞ ∞ and there exists ∈ ∞ ∞ such that Wronskian
defined below is not zero, then these functions are linearly independent.
⋯
′ ⋯ ′
≠
⋮
⋮
⋮
⋯
Conversely if for every in ∞ ∞ , then … are
linearly dependent.
Example
16
Show by Theorem 9.1.3 that , , are linearly
independent.
Solution
For some (in fact, any) ,
≠ . Thus these
functions are linearly independent.
Sage
□
http://sage.skku.edu or http://mathlab.knou.ac.kr:8080/
var('x')
W=wronskian(1, e^x, e^(2*x))
# wronskian(f1(x), f2(x), f3(x))
print W
- 376 -
2*e^(3*x)
Example
17
Let
,
■
sin.
Show
that
these
functions
are
linearly
independent.
Solution
Since
sin
cos sin ≠
cos
for
some
,
functions are linearly independent.
Example
18
these
■
Show that , are linearly dependent.
Solution
Since for any ,
, these functions are linearly
dependent.
■
http://matrix.skku.ac.kr/kiosk/
- 377 -
9.2
Inner product; *Fourier series
ref movie: http://youtu.be/m9ru-F7EvNg,
http://youtu.be/nIkYF-uvFdA
demo site: http://matrix.skku.ac.kr/knou-knowls/cla-week-14-sec-9-2.html
In this section, we generalize the Euclidean inner product on ℝ
(dot product) to introduce the concepts of length, distance, and
orthogonality in a general vector space.
Inner product and inner product space
Definition
[Inner product and inner product space]
The inner product on a real vector space is a function assigning a
pair of vectors u , v to a scalar u v satisfying the following
conditions. (that is, the function
× → ℝ
satisfies the
following conditions.)
(1) u v v u for every u v in . f
(2) u v w u w v w for every u v w in .
(3) u v u v for every u v in and in ℝ .
(4) u u ≥ u u ⇔ u for every u in .
The inner product space is a vector space with an inner product
u v defined on .
Example
1
The Euclidean inner product, that is, the dot product is an example of
an inner product on ℝ . Let us ask how other inner products on are
possible. For this, consider ∈ ℝ . Let u and v be the column
vectors of . Define u v by u v v u . Then let us find the
condition on so that this function becomes an inner product.
Solution
In order for u v v u to be an inner product, the four conditions
(1)~(4) should be satisfied. First consider conditions (2) and (3).
- 378 -
u v w w u v
w u w v u w v w ,
u v v u v u u v .
Let us check when condition (1) holds. Since v u is a × matrix
(hence a real number), we have
v u v u
That is, to satisfy u v v u v u u v u v v u
we have , in other words, is a symmetric matrix.
Thus the function
u v v u
satisfy condition (1) if
is a
symmetric matrix.
Finally check condition (4). An × symmetric matrix should satisfy
u u for any nonzero vector u . This condition means that is
positive
definite.
In
other
words,
if
is
positive
definite,
u v v u satisfies condition (4).
Therefore, to wrap up, if is an × symmetric and positive definite
matrix, then u v v u defines an inner product on ℝ . The well
known Euclidean inner product u⋅ v v u v u can be obtained as a
special case when (symmetric and positive definite).
■
For any nonzero vector u , if the eigenvalues of are positive, then u u
(the converse also holds.)
Example
2
Let
be a × symmetric matrix and u v in ℝ .
Then
u v v u
satisfies conditions (1), (2), (3) of an inner product on ℝ . Now let us
show that is a positive definite. Let w . Then
- 379 -
w w w w . Thus w w ≥ and
w w ⇔ ⇔ .
Hence the symmetric matrix is positive definite and defines an inner
product on of the form u v v u .
If u and v , then u⋅ v . On the other hand,
u v
Hence the inner product u v v u on ℝ is different from the
Euclidean inner product.
■
Norm and angle
Definition
[norm and angle]
Let be a vector space with an inner product u v . The norm (or
length) of a vector u with respect to the inner product is defined by
u
u u .
The angle between two nonzero vectors u and v is defined by
u v
cos
u v
( ≤ ≤ ).
In particular, if two vectors u and v satisfy u v , then they are
said to be orthogonal.
For example, the norm of u with respect to the inner product given in
Example
2
is
u u u u u
.
- 380 -
Thus u
. On the other hand, the norm u with respect to the
Euclidean inner product is
u
u⋅ u u u
For any inner prodcut space, the triangle inequality u v ≤ u v holds.
Using
the
Gram-Schmidt
v v … v of
u u … u .
a
inner
orthogonality
product
process,
space
we
into
can
an
make
a
basis
orthonormal
basis
Inner product on complex vector space
Definition
Let be a complex vector space. Let u v w be any vectors in and
∈ be any scalar. The function from
× to is called an
inner product (or Hermitian inner product) if the following hold.
(1) u v
v u .
(2) u v w u w v w .
(3) u v u v .
(4) v v ≥ v v ⇔ v
A complex vector space with an inner product is called a complex inner product
space or a unitary space.
If u v for any two nonzero vectors u v, then
we say that u and v are orthogonal.
- 381 -
Let be a complex vector space. By the definition of an inner product on ,
we obtain the following properties.
(1) v v .
(2) u v w u v u w .
(3) u v u v ∈
Example
3
Let u … and v … be vectors in . The
Euclidean inner product u⋅ v
⋯
satisfies the
conditions
Example
4
(∵ u v v u v u u v ).
(1)~(4) for the inner product.
■
Let ℰ be the set of continuous functions from the interval
to the complex set . Let ∈ ℰ . If the addition
and
scalar
multiple
of
these
functions
are
defined
below,
then
ℰ is a complex vector space with respect to these operations.
∈ .
In this case, a vector in ℰ is of the form
and are continuous functions from to ℝ . For
∈ ℰ , define the following inner product
.
Then ℰ is a complex inner product space.
We leave readers to check conditions (1)~(3) for an inner product, and
show condition (4) here. Note
and ≥ , hence ≥ . In particular,
- 382 -
⇒
That is, ≤ ≤ , conversely, if is a zero function, then it is
easy to see that .
■
Complex inner product space, norm, distance
[Norm, and distance]
Definition
Let be a complex inner product space. The norm of u and the
distance between u and v are defined as follows:
u u u
Example
5
Find
the
Euclidean
inner
, u v u v .
product
and
the
distance
of
vectors
u v .
Solution
⋅
u⋅ v
.
u v u v
.
Example
6
From
Example
4,
■
we let and . Find the norm of
.
Solution
- 383 -
.
■
Cauchy-Schwarz inequality and the triangle inequality
9.2.1
Theorem
Let
be a complex inner product space. For any u v in , the
following hold.
(1) u v ≤ u v .
(Cauchy-Schwarz inequality)
(2) u v ≤ u v .
(triangle inequality)
Proof
We prove (1) only and leave the proof of (2) as an exercise.
If
u v ∥u∥∥v∥ .
u ,
Hence
(1)
holds.
Let
u≠
and
v u
p proj u v, w v p. Then w p and p
u . Thus we have
∥u∥
the following.
v
w v p
p u proj u v
u
≤ w w w v p w v w p
w v v p v v v p v
v u
∥v∥ u v ∥v∥
u v .
∥u∥
u v u v u v , (1) holds.
Thus, as∥v∥ ∥u∥ ≥
- 384 -
■
Example
7
u v
Let
be
vectors
in
.
Answer
the
following.
(1) Compute the Euclidean inner product u v u v u v .
(2) Show that u and v are linearly independent.
Solution
(1) u v
u
v
u v
(2) If u v for any scalar ∈ , then
⇒ .
So . Thus u and v are linearly independent.
Example
8
■
Let u v be vectors in . Check that the
Cauchy-Schwarz inequality and the triangle inequality hold.
Solution
Since
u v
and
u v
,
the
Cauchy-Schwarz
inequality holds.
Also since u v
≤
u v , the triangle inequality
holds.
Example
9
■
[Cauchy-Schwarz inequality in and ℰ ]
(1) Let be a complex inner product space with the Euclidean inner
product. Let u … , v … be in . Then
- 385 -
u v
≤
u v
Hence the Cauchy-Schwarz inequality holds.
■
Example
(2) Let u v ∈ ℰ . As in
4,
since the inner
product is given by
u v
≤
u v
the Cauchy-Schwarz inequality holds.
Example
10
■
Example
[triangle inequality] Consider the inner products given in
Example
3
and
4.
(1) Let u … v … ∈ . Then the triangle inequality
holds. That is,
u v
≤
u v .
■
(2) Let u v ∈ ℰ . Then the triangle inequality
holds. That is,
u v
≤
u v .
- 386 -
■
9.3
Isomorphism
Reference site: http://youtu.be/frOcceYb2fc,
http://youtu.be/Y2lhCID0XS8
Lab site: http://matrix.skku.ac.kr/knou-knowls/cla-week-14-sec-9-3.html
We generalize the definition of a linear transformation on ℝ
to a
general vector space . A special attention will be given to both
injective and surjective linear transformations.
Definition
Let and be vector spaces over ℝ . → be a map from a
vector space to a vector space . If satisfies the following
conditions, it is called a linear transformation.
(1) u u for every u in and in ℝ .
(2) u v u v for every u v in .
If , then the linear transformation is called a linear operator.
Theorem
9.3.1
If → is a linear transformation, Then we have the following:
(1) .
(2) u u .
(3) u v u v .
Example
1
If → satisfies that v for any v ∈ , then it is a linear
transformation,
called
the zero
transformation.
Also,
if
→
satisfies that v v for any v ∈ , then it is a linear transformation,
called the identity operator.
■
- 387 -
Example
2
Define
→
by
v v
(
a scalar).
Then
is a linear
transformation. The following two properties hold.
(1) u u u u
(2) u v u v u v u v
If
, then is called a contraction and if , then it is
called a dilation.
Example
3
■
Let ℰ ℝ be the vector space of all continuous functions from ℝ to ℝ
and be the subspace of ℰ ℝ consisting of differentiable functions.
Define → by ′. Then is a linear transformation and
called a derivative operator.
Example
4
Let
■
the subspace of ℰℝ consisting of differentiable functions.
Define
→
by
.
Then
is
linear
transformation.
■
Kernel and Range
Definition
[Kernel and Range]
Let → . Define
ker v ∈ v , Im v ∈ v ∈
ker is called the kernel and Im the range.
Example
5
If → is the zero transformation, ker and Im .
- 388 -
■
Example
Example
6
7
If → is the identity operator, ker and Im .
Let be the derivative operator defined by ′ as in
■
Example
3.
ker is “the set of all constant functions defined on ∞ ∞ ” and Im
is
“the
set
of
all
continuous
functions,
that
is,
∞ ∞ ”.
■
Basic properties of kernel and range
Theorem
9.3.2
If → is a linear transformation, ker and Im are subspaces
of and , respectively.
Theorem
9.3.3
If → is a linear transformation, the following statements are
equivalent.
(1) is an injective (or one-to-one) function.
(2) ker .
Isomorphism
Definition
If a linear transformation → is one-to-one and onto, then it
is called an isomorphism. In this case, we say that is isomorphic to
, denoted by ≅ .
- 389 -
Theorem
9.3.4
Any -dimensional real vector space is isomorphic to ℝ .
Any -dimensional real vector space (defined over the real set ℝ ) is isomorphic
to ℝ and any -dimensional complex vector space (defined over the complex
set ) is isomorphic to .
Example
8
We immediately obtain the following result from the above theorem.
≅ ℝ ,
× ≅ ℝ ×
[the 12th International Congress on Mathematical Education]
http://www.icme12.org/
http://matrix.skku.ac.kr/2012-Album/ICME-HPM.html
- 390 -
■
Exercises
Chapter 9
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
When we define the addition and the scalar multiple on ℝ and as
follows. Check if ℝ and are vector spaces.
Problem 2
Problem 3
(1)
.
(2)
.
(3)
, .
(4)
, .
Which one is a subspace of ?
(1)
∈ℝ.
(2)
∈ℝ.
(3)
.
(4)
.
Let
be a vector in
. Write as a linear
combination of , , .
Solution
Let =
= , ∈ℝ .
- 391 -
=>
=>
Therefore .
Problem 4
■
Determine if the below vectors in a given vector space are linearly
independent or linearly dependent.
(1) ℝ x x x
x .
x
x
(2) x
.
(3)
.
Problem 5
Let be the complex inner product space with the Euclidean inner
product. Let u v . Answer the following.
(1) Compute u v .
(2) Compute u v u v .
(3) Confirm the Cauchy-Schwarz inequality.
(4) Confirm the triangle inequality.
Solution
u ×
(1) 〈uv〉 v
(2) ∥u∥
∥v∥
∥u v∥ ∥ ∥
〈u v〉 , ∥u∥∥v∥ implies 〈u v〉 ≤ ∥ u∥∥v∥ .
(4) ∥u∥ ∥v∥ ∥u v∥
=>
≤
Triangle inequality ∥u v∥≤ ∥u∥ ∥v∥ holds.
(3)
- 392 -
■
Problem 6
Define an inner product on ℝ as u v .
Compute the following. (here u , v )
(1) The × symmetric matrix such that u v v u
(2) The norm u of u .
(3) The norm v of v .
u v
(4) such that cos .
u v
Solution
(1) Let and 〈uv〉 .
=> 〈uv〉 v u .
∴ ,
(2) and (3) ∥u∥ 〈uu〉
∥v∥ 〈vv〉
(4)
〈uv〉
cos
∥u∥∥v∥
×
=>
Problem 7
cos
≈
■
Tell which one is a linear transformation or not. If not, give a reason.
(1) → , .
(2) → , .
(3) ℰ → ℝ ,
.
(4) → ℝ , tr .
- 393 -
(5) → , .
(6) → ℝ , x x .
Problem 8
Find the kernel and the range of the following linear transformations.
(1) → , .
(2) ℰ → ,
.
P1 If are subspaces of a vector space , prove that ∩ is a
subspace of .
P2 Let a be a fixed vector and be the set of all vectors orthogonal to a ,
that is, x ∈ a⋅ x . Show that is a subspace of ℝ .
P3
Let
be the vector space with the Euclidean inner product.
Transform
u u u
into
an
orthonormal
basis by using the Gram-Schmidt process.
P4
Find
the
standard
matrix
corresponding
to
the
given
linear
transformation → defined by
.
P5 Define ℝ → ℝ by . Show that it is a linear
transformation and find the kernel and the range of .
- 394 -
10
Chapter
Jordan Canonical Form
10.1 Finding the Jordan Canonical Form with a Dot Diagram
*10.2 Jordan Canonical Form and Generalized Eigenvectors
10.3 Jordan Canonical Form and CAS
10.4 Exercises
If a matrix is diagonalizable, every thing is much easier. But most of matrices are not
diagonalizable. The Jordan canonical form is an upper triangular matrix of a particular form
called
a
Jordan
matrix
(a
simple
block
diagonal
matrix)
representing an operator with respect to some basis. The
diagonal entries of the normal form are the eigenvalues of the
operator, with the number of times each one occurs given by
its algebraic multiplicity.
Any square matrix has a Jordan normal form if the field of
coefficients is extended to one containing all the eigenvalues of
the matrix. Since each matrix has a corresponding Jordan
canonical form which is similar to it, all computations can be
done with this simple upper triangular matrix. The Jordan
normal
form
is
named
after
Camille
Jordan,
a
French
mathematician renowned for his work in various branches of
mathematics. In this chapter, we will study how to find a Jordan matrix which is similar to
any given matrix and how to find generalized eigenvectors.
- 395 -
Finding the Jordan Canonical Form with a Dot Diagram
10.1
Reference video: http://youtu.be/NBLZPcWRHYI, http://youtu.be/NBLZPcWRHYI
Practice site: http://matrix.skku.ac.kr/knou-knowls/cla-week-15-sec-10-1.html
http://matrix.skku.ac.kr/JCF/
If a given matrix is diagonalizable, most computational problems
involving
that
matrix
and
desired
conclusions
can
be
easily
obtained. However, not every matrix is diagonalizable. In this
section,
we
will
introduce
a
method
for
finding
the
Jordan
Canonical Form of a non-diagonzaliable matrix by a similarity
transformation.
Let us review a few concepts of matrix diagonalization.
Diagonalization of a Square Matrix (Review)
1. Let be an × matrix. Then, A is diagonalizable if and only if it has
linearly independent eigenvectors. However not all matrices are diagonalizable.
2. A normal matrix is unitarily diagonalizable (that is, unitarily
similar to a diagonal matrix). However, not all diagonalizable matrices are
normal.
3. If a matrix
eigenspace
with
is diagonalizable, each eigenvalue of
dimension
equal
to
the
algebraic
generates an
multiplicity
of
that
eigenvalue.
For every square matrix (not necessarily diagonalizable), one can obtain a
block-diagonal matrix called the Jordan canonical form matrix that is similar to
.
For example, the matrices
and
are non diagonalizable.
- 396 -
Theorem 10.1.1
Let
be an × matrix with
eigenvectors.
Then, is similar to a matrix
( ≤ ≤ ) linearly independent
⋱
×
where for some unitary matrix . Furthermore, we have
⋱⋱
,
⋱
×
( ⋯ , ≤ ≤ )
where each , called a Jordan block, corresponds to an eigenvalue of
. The block diagonal matrix is called the Jordan canonical form of
and each are called Jordan blocks of .
The Jordan Canonical Form (JCF) of a matrix is a block diagonal matrix
- 397 -
composed of Jordan blocks, each with eigenvalues of A on its respective
diagonal, 1's on its superdiagonal, and 0's elsewhere.
Remark Properties of Jordan blocks
1. For a given eigenvalue of an × matrix , its geometric multiplicity is
the number of linearly independent eigenvectors associated with : hence,
it is the number of Jordan blocks corresponding to .
2. The sum of the sizes (i.e. orders) of all Jordan blocks corresponding to an
eigenvalue is its algebraic multiplicity.
3. If the geometric multiplicity and algebraic multiplicity of every eigenvalue of
are equal, then the size of every Jordan block is × , and
sum of algebraic multiplicities sum of geometric multiplicities size of
In this case, the matrix is diagonalizable. (This type of matrix is called a
simple matrix.)
Example
1
The matrix
has characteristic polynomial det and is the
Jordan Form of some × square matrix .
Notice how the algebraic multiplicities of each eigenvalue determines the
number of times that eigenvalue appears along the principal diagonal of
: 2 appears four times, while 3 and 0 appear two times. Hence the
algebraic multiplicity of 2 is 4.
- 398 -
You can easily verify that the sum of the sizes of all Jordan blocks
corresponding to a single eigenvalue is also equal to its algebraic
multiplicity.
Also, note that the geometric multiplicities of each eigenvalue determine
the number of Jordan blocks corresponding to that eigenvalue.
Thus geometric multiplicities of 2, 3 and 0 are 2, 1, and 1 respectively.■
Example
2
For a × square matrix , if A has only one eigenvalue with one
associated linearly independent eigenvector, the Jordan form of is the
following:
This is due to the fact that the number of linearly independent
eigenvectors of determines the number of Jordan blocks in the Jordan
form of . Thus in this case the geometric multiplicity of the eigenvalue
5 is 1 where as algebraic multiplicity is 5.
■
How to find the Jordan Canonical Form
Suppose for some matrix ∈ with distinct eigenvalues … , the
Jordan canonical form of , , is the following:
⋯
⋮
⋱
⋯
⋮
.
Here, each corresponds to a Jordan block with the eigenvector along its
diagonal. These are called
block submatrices of . Now, for each eigenvalue
, we have a block submatrix
- 399 -
⋮
⋯
⋯
⋮
⋱
and, knowing its structure, we can easily find the Jordan Canonical Form .
The Jordan form is uniquely determined up to the order of the blocks; that is,
the number and size of the Jordan blocks associated with each eigenvalue is
uniquely determined, but the blocks can appear in any order along the main
diagonal.
For each eigenvalue … , consists of ≤ ≤ Jordan blocks;
let us find the size of each ∈ , namely ( ≤ ≤ ). For the set of linearly
independent eigenvectors x x … x corresponding to , for ease of notation,
let us first consider only one eigenvalue. Therefore,
The number of Jordan blocks in
we let be and be .
, , and their corresponding sizes
… is determined by calculating the rank of . Without loss of
generality, we take ≥ ≥ ⋯ ≥ . Now, for the eigenvalue and the
dimension of its corresponding -eigenspace (its geometric multiplicity), using
and , we introduce a sequence of points to easily calculate ; this is called
the dot diagram. The dots in the dot diagram are configured according to the
following rules:
*
Dot Diagram Properties
1. The dot diagram consists of columns.
2. Counting from left to right, the th column consists of the dots that
correspond to the eigenvectors of , starting with the initial vector at the
top and continuing down to the end vector.
Thus, the following is the dot diagram of :
•
•
⋮
x
•
x
•
x
⋯
x
⋯
•
⋮
• x
⋮
- 400 -
x
• x
• x
• x
Here, ｘ ｘ … ｘ are the linearly independent eigenvectors associated to the
eigenvalue . Let denote t the number of dots in the th row of the dot
diagram; then, is the number of Jordan blocks of size at least × , is the
number of Jordan blocks of size at least × , and is the number of Jordan
blocks of size at least × . Thus, ≥ ≥ ⋯ ≥ . Refer to Theorem 10.1.2
and 10.1.3, and consider the example below.
Example
3
For a × matrix , the number of Jordan blocks contained in is
and
the
size
of
the
Jordan
blocks
is
… . To see this,
take and
Then, following the sequence of block sizes,
completely
determined
by
.
is uniquely determined.
To find the dot diagram of , since
,
and
the dot diagram of is:
∙∙∙∙
(Number of Jordan blocks: 4)
∙∙∙
∙∙
- 401 -
■
Theorem 10.1.2
The number of dots in the first rows of the dot diagram for is
equal to the dimension of solution space of x (i.e. the
nullity of ).
nullity = nullity
Theorem 10.1.3
For ∈ , let denote the number of dots in the th row of the
dot diagram of .
Then, the following are true.
(1) rank .
(2) If , rank rank .
Proof
By Theorem 10.1.2,
⋯ nullity rank (provided ≥ )
Also, rank and
⋯ ⋯
rank rank
rank rank .
(The number of dots in each row, , means the number of blocks of size
at least × )
■
From Theorem 10.1.3, let’s see how the dot diagram for each is completely
determined by the matrix .
Example
4
Find the Jordan Canonical Form of .
- 402 -
Solution
A=matrix(4, 4, [2, -1, 0, 1, 0, 3, -1, 0, 0, 1, 1, 0, 0, -1, 0, 3])
print A.charpoly().factor()
print A.eigenvalues()
(x - 3) * (x - 2)^3
[3, 2, 2, 2]
The matrix has characteristic polynomial det ,
so has two distinct eigenvalues , .
Here
has algebraic multiplicity 1, and
has algebraic
multiplicity 3. Thus, the dot diagram for has 1 dot
•
and has one × Jordan block. That is, . As well, the dot
diagram for has 3 dots, and
E=identity_matrix(4)
print (A-2*E).rank()
print ((A-2*E)^2).rank()
2
1
rank rank
,
rank rank .
Thus, the dot diagram for is the following.
: ∙
∙
(number of Jordan blocks: 2)
: ∙
has one × Jordan block and one × Jordan block. That is,
- 403 -
Hence, the Jordan Canonical form of is
∴
Sage
□
http://sage.skku.edu and http://mathlab.knou.ac.kr:8080/
A=matrix(4, 4, [2, -1, 0, 1, 0, 3, -1, 0, 0, 1, 1, 0, 0, -1, 0, 3])
J=A.jordan_form()
# Jordan Canonical Form
print J
[3|0 0|0]
[-+--+-]
[0|2 1|0]
[0|0 2|0]
[-+--+-]
[0|0 0|2]
Example
5
■
Find the Jordan Canonical Form of .
Solution
The matrix has characteristic polynomial det ,
so there are two distinct eigenvalues of
algebraic multiplicity 2.
, 와 , each with
For ,
rank
Therefore, the dot diagram for is the following.
- 404 -
:
•
:
•
(Number of Jordan blocks: )
So,
.
For , rank . is 0(∵ The number of dots
is ). Therefore, the dot diagram for
:
• •
is the following:
(Number of Jordan blocks = 2)
So,
Thus, the Jordan Canonical Form of is
Sage
.
□
http://sage.skku.edu and http://mathlab.knou.ac.kr:8080/
A=matrix(4, 4, [2, -2, -2, -2, -4, 0, -2, -6, 2, 1, 3, 3, 2, 3, 3, 7])
J=A.jordan_form()
# Jordan Canonical Form
print J
[4 1|0|0]
[0 4|0|0]
[--+-+-]
[0 0|2|0]
[--+-+-]
[0 0|0|2]
■
[Remark] Jordan Canonical Form Learning Materials
Ÿ
http://matrix.skku.ac.kr/2012-mobile/E-CLA/10-1.html
Ÿ
http://matrix.skku.ac.kr/2012-mobile/E-CLA/10-1-ex.html
http://matrix.skku.ac.kr/JCF/index.htm
- 405 -
Jordan Canonical Form and
Generalized Eigenvectors
10.2
Reference video: http://www.youtube.com/watch?v=yJ7n0icjtNA
Practice site: http://matrix.skku.ac.kr/knou-knowls/cla-week-15-sec-10-2.html
http://matrix.skku.ac.kr/sglee/03-Note/GeneralizedEV-f.pdf
http://matrix.skku.ac.kr/MT-04/chp8/3p.html
In Section 10.1, for any × matrix , we discussed the theory
and method for finding a matrix , called the Jordan Canonical
form, such that
. In this section, we will examine a
method for finding the matrix
in the above equation. This
method utilizes the concept of generalized eigenvectors.
The following matrix was referred from the wiki
http://en.wikipedia.org/wiki/Jordan_normal_form.
Let
.
Consider the matrix . The Jordan normal form is obtained by some similarity
transformation , i.e. .
Let have column vectors p , …
p p p p p p p p
, then
p p
p p p .
We see that
p
p
p
p p .
For ,
we
have
p ∈ Ker ,
i.e. p is
an
eigenvector
of
corresponding to the eigenvalue . For , multiplying both sides by
gives
p p .
But
p ,
- 406 -
so
p .
Thus,
p ∈ Ker . Vectors like p
are called generalized eigenvectors of . Thus,
given an eigenvalue , its corresponding Jordan block gives rise to a Jordan chain.
The generator, or lead vector (say, p ) of the chain is a generalized eigenvector
such that p , where is the size of the Jordan block. The vector
p p
is an eigenvector corresponding to . In general, p is the
preimage of p under . So the lead vector generates the chain via
multiplication by . Therefore, the statement that every square matrix can
be put in Jordan normal form is equivalent to the claim that there exists a basis
consisting only of eigenvectors and generalized eigenvectors of .
PS: More details about the Jordan Canonical Form can be found at
http://www.uio.no/studier/emner/matnat/math/MAT2440/v11/undervisningsmateriale/genvectors.pdf.
- 407 -
10.3
Jordan Canonical Form and CAS
Reference video: http://youtu.be/LxY6RcNTEE0,
http://youtu.be/LxY6RcNTEE0
Practice site: http://matrix.skku.ac.kr/knou-knowls/cla-week-15-sec-10-3.html
In practice, in order to find the Jordan Canonical Form of a 10 × 10
matrix, you need to find the roots of a characteristic polynomial of
degree 10 – the factorization and rigorous calculation of these
roots is impossible.
Moreover, a 10 × 10 matrix requires us to
calculate many exponents and coefficients. In order to calculate
these
coefficients,
the
Gaussian
elimination
and
related
computations can be performed by various computer programs –
e.g.
HLINPRAC,
MATHEMATICA,
MATLAB,
developed open-source program, Sage.
computationally
complex
and
the
recently
The use of software for
mathematics
is
necessary
in
an
increasingly technological society.
The following links provide more information about the Jordan Canonical Form and
tools that allow you to explicitly find the Jordan Canoncial Form for a given matrix
without arduous calculations by hand.
1. Theory and tools : http://matrix.skku.ac.kr/JCF/index.htm
2. Jordan Canonical Form; an algorithmic approach:
http://matrix.skku.ac.kr/JCF/JCF-algorithm.html
3. Jordan Canonical Form (step by step) tool:
http://matrix.skku.ac.kr/JCF/JordanCanonicalForm-SKKU.html
4.CAS Tool : http://matrix.skku.ac.kr/2014-Album/MC-2.html
"The man ignorant of mathematics will be increasingly
limited in his grasp of the main forces of civilization."
- John George Kemeny (1926 –1992)
A Jewish-Hungarian American mathematician, computer scientist, and educator
best known for co-developing the BASIC programming language in 1964 and
pioneered the use of computers in college education.
- 408 -
Exercises
Chapter 10
Ÿ
http://matrix.skku.ac.kr/LA-Lab/index.htm
Ÿ
http://matrix.skku.ac.kr/knou-knowls/cla-sage-reference.htm
Problem 1
Let be a × matrix with the only one eigenvalue with algebraic
multiplicity of 5. Find all possible types of
Jordan Canonical forms of
when the number of linearly independent eigenvectors corresponding is 2.
Problem 2
For the given Jordan Canonical form , calculate the following:
(1)
(2)
(3)
(4)
–
[Problem 3 8] Find the Jordan Canonical Form of the given matrix.
Problem 3
Solution
Sage :
_________________________________________
A=matrix(QQ,5,5,[5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5])
J=A.jordan_form()
print J
__________________________________________
- 409 -
[25| 0| 0| 0| 0]
[ 0| 0| 0| 0| 0]
[ 0| 0| 0| 0| 0]
[ 0| 0| 0| 0| 0]
[ 0| 0| 0| 0| 0]
Problem 4
Problem 5
Problem 6
Problem 7
Problem 8
Solution
■
Sage :
_________________________________________
A=matrix(QQ,[[2,-4,2,2],[-2,0,1,3],[-2,-2,3,3],[-2,-6,3,7]])
print A.eigenvalues()
E=identity_matrix(4);
print (A-2*E).rank(); print (A-4*E).rank();print ((A-4*E)^2).rank()
__________________________________________
[4, 4, 2, 2]
- 410 -
2
3
2
rank( )=3 => rank = .
∙ (Number of Jordan block : 1)
∙
rank => rank .
∙ ∙ (Number of Jordan block : 2)
∴
Problem 9
■
( ⊕ means the matrix direct sum of matrices constructs a block
diagonal matrix, http://mathworld.wolfram.com/MatrixDirectSum.html, from a set of
square matrices.)
Problem 10
=
⊕
⊕
- 411 -
⊕
⊕
- Quotes by Great Mathematicians:
http://prezi.com/z0hgrw8a6wql/define-math/
ILAS
ILAS
ILAS
ILAS
ILAS
ILAS
2014
2014
2014
2014
2014
2014
Official Photo : http://matrix.skku.ac.kr/2014-Album/ILAS-2014/
Movie A – Registration and Presentations : http://youtu.be/asJfRFYWPrk
Movie B – Tour
: http://youtu.be/bidJNagmRXQ
Movie C – Banquet
: http://youtu.be/10fDqWA-vVA
Movie D – Group Photo : http://youtu.be/6IlS8U6i_8E
Movie E – Conference Preparations : http://youtu.be/UMwLCtSGByI
ICM 2014, COEX, Seoul, Fields Medalists:
http://matrix.skku.ac.kr/2014-Album/2014-ICM-SGLee/
https://www.facebook.com/SEOULICM2014
http://www.icm2014.org/en/vod/videos
http://www.icm2014.org/en/vod/public
- 412 -
Appendix
http://matrix.skku.ac.kr/Cal-Book/Appnd/index.htm
References
H. Anton, R. Busby, Contemporary Linear Algebra, Anton Textbooks Inc., 2003.
Duk-Sun Kim, Sang-Gu Lee, Greg Markowsky, Mobile Sage-Math for Linear Algebra and
its Application, Electronic Journal of Mathematics & Technology 4 (2010), No. 3, 1-13.
Kyung-Won Kim, Sang-Gu Lee, Shaowei Sun, Modeling of Mobile Sage and Graphing
Calculator, Journal of Modern Education Review 3 (2013), No. 12, 918-925.
Jin Ho Kwak, Sungpyo Hong, Linear Algebra, Birkhäuser, 1997.
Sang-Gu Lee, Contemporary Linear algebra (3rd Edition), Kyungmoon Books, 2009.
Sang-Gu Lee, Kyung-Won Kim, Jae Hwa Lee, Sage matrix calculator and full Sage
contents for linear algebra, Korean J. Math. 20 (2013), No. 4, 503-521.
Steven J. Leon, Eugene Herman, Richard Faulkenberry, ATLAST: Computer Exercise for
Linear Algebra, Prentice Hall Inc., 1996.
Steven J. Leon, Linear Algebra with Applications, Prentice Hall Inc., 2002.
Gilbert Strang, Linear Algebra and its Application, Thomson Learning Inc., 1988.
[National Museum of Mathematics, NYC] http://momath.org/
- 413 -
Sample Exam
Reference video: http://youtu.be/CLxjkZuNJXw
Practice site:
http://matrix.skku.ac.kr/CLA-Exams-Sol.pdf,
http://matrix.skku.ac.kr/2015-album/2015-LA-S-Exam-All-Sol.pdf
http://matrix.skku.ac.kr/2012-album/2012-LA-Lectures.htm
* Provides basic commands if you use Sage in your test.
<Sage Linear Algebra partial commands list>
var('a,b,c,d')
# define variables
eq1=3*a+3*b==12
# define equation1
eq2=5*a+2*b==13
# define equation2
solve([eq1, eq2], a,b)
# solve system of
equations
A=matrix(CDF, 3, 3, [3, 0, 0, 0, 0, 2, 0, 3, 4]);
# define matrix
A.echelon_form()
# RREF
A.inverse()
# inverse matrix
A.det()
# determinant
A.adjoint()
# adjoint matrix
A.eigenvalues()
# eigenvalues
A.eigenvectors_right()
# eigenvectors
A.charpoly()
P,L,U=A.LU()
# LU decomposition
(P: Permutation matrix / L,U: triangle matrix)
vector([3, 1, 2])
# define vector
var('x, y')
# define variables
plot3d(y^2+1-x^3-x, (x, -pi, pi), (y, -pi, pi))
# 3D Plot
implicit_plot3d(n.inner_product(p_0-p)==0,
-10, 10 , (y, -10, 10), (z, -10, 10))
# 3D Hyperplane Plot
var('t')
# define variable (parametric equations)
x=2+2*t
y=-3*t-2
parametric_plot((x,y), (t, -10, 10), rgbcolor='red')
# characteristic equation
# line Plot
I. (3pt x 6= 18pt) True(T) or False(F). Let ∈× and u v ∈ .
1. (
) Every square matrix can be expressed as products of elementary matrices.
2. (
) Let × matrix has all integer components. If the determinant of is 1, then the
components of are all integers.
3. (
) Let ∈ , then det det
4. (
) proj u v u
u⋅ u
5. (
v⋅ u
) One can compute the solution of a system of linear equations with unknowns and equations
by Cramer’s rule.
6. (
) × real matrix satisfies tr det .
II. (3pt x 4 = 12pt) State or Define
1. Choose 4 items from the list given in the box and describe them clearly and concisely.
- 414 -
(x,
normal vector of a plane
,
linearly independent and linearly dependent, condition for subspace,
Cramer's Rule, eigenvalue, eigenvector, linear transformations,
transformation’s →
orthogonal matrix, for linear
range, surjective or onto, injective or 1-1, isomorphism
[Subspace] A nonempty subset of satisfying the following two properties,
w w w∈∈
is called a subspace of .
(where, w w w ∈ , ∈ )
...
[Standard Matrix] For a linear transformation, → the range is defined as
Im v∈ v∈ ⊂ .
If → is a linear transformation and
x∈ , x x ,
∀ x ∈R n
A = [ T]
is the standard matrix of , then for
where e e ⋯ e .
...
III. (3pt x 10 = 30pt) Find or Explain:
...
2.
Find the equation of a plane passing through a point and generated by two
vectors a and b in a vector equation form.
Ans
x p a b
∈
.
■
...
5. Suppose you got a job in a research lab and your boss asked you to find the eigenvalues, the
eigenvectors, and the characteristic polynomial of a matrix
. Explain how to find them
Sol
with a step by step description. You may use Sage.
- 415 -
1) Step 1: Open the webpage http://math1.skku.ac.kr .
2) Step 2: Log in to the webpage with ID= skku, PW = ***
.
3) Step 3: Press the button of “New Worksheet”
4) Step 4: In the first cell, define matrix in CDF format.
A=matrix(CDF,4,4,[4,1,0,2,0,-1,2,0,0,0,1,0,0,4,0,3])
5) Step 5: In the second cell, enter the command to find eigenvalues
A.eigenvalues() and execute.
6) Step 6: In the third cell, enter the command to find eigenvectors
A.eigenvectors_right() and execute.
7) Step 7: In the fourth cell, enter the command to find characteristic polynomial
A.charpoly() and execute.
[4.0, 3.0, -1.0, 1.0]
[(4.0,
[(1.0, 0,
0,
0)],
1),
(3.0,
[(0.894427191, 0,
0,
-0.4472135955)],
1),
(-1.0,
[(0.140028008403, 0.700140042014, 0, -0.700140042014)], 1),
(1.0, [(-0.377964473009, -0.377964473009, -0.377964473009, 0.755928946018)], 1)]
x^4 - 7.0*x^3 + 11.0*x^2 + 7.0*x – 12.0
(Online Sage solution)
■
.....
8. For a given matrix , describe step by step process to find the inverse matrix by using
Sol
the Sage.
1) Step 1: (example) Open the webpage http://math1.skku.ac.kr .
2) Step 2: Log in to the webpage with ID= skku, PW = ***
.
3) Step 3: Press the button of “New Worksheet”
4) Step 4: In the first cell, define matrix in CC format.
A=matrix(CC, 3, 3, [1,0,1,-0,3,0,1,0,2])
5) Step 5: In the second cell, enter the command A.inverse() to find the inverse.
[ 2.00
0
[ 0
0.33
[-1.00
0
-1.0]
0]
■
1.00]
...
9. ...
- 416 -
IV. (5pt x 5 = 25pt) Find or Explain:
1. Let a linear transformation → transforms any vector x ∈ to a symmetric
point to the line which passing through the origin with slope . Find the transformation matrix
e e with the aid of following pictures.
Picture: The image of the standard basis by a symmetric transformation to the line
with slope .
(Sol) e e
cos
sin
cos
cos sin
. ■
sin cos
sin
2. Linear transformation (Linear operator): Let's define → as a projective transformation,
which transforms any vector x in to projection on a line which passes through the origin
and has an angle with -axis. For the given transformation , let's define
as a
corresponding standard matrix. As shown by the right hand side picture, x x x x
<same direction with half length>. Now by using the matrix representation of symmetric
cos sin
, find the standard matrix for .
sin cos
transformation
Picture:
Projective transformation to the
line with slope
(Sol)
x x x x
The relationship between symmetric transformation
and projective transformation to the line with slope
=> x x x x x x
- 417 -
=>
cos
sin
cos sincos
sincos sin
cos
sin
■
3. For invertible matrices and , explain why adj adj ⋅ adj .
...
4. For a degree square matrix , explain why its eigenspace is a subspace of .
Ans
For a given square matrix , let be the eigenspace corresponding to an eigenvalue .
x∈ x x ⊆ , ∈ , ≠ ∅
∀ x y∈ , x y∈R and x∈R .
1) [Show the space is closed under the addition, that is, show x y∈ ]
(Proof) x x y y
x y x y x y x y
(2 pt)
∴ x y∈
2) [Show the space is closed under the scalar multiplication, that is, show x∈ ] (2 pt)
(Proof) x x x x
∴ x∈
∴ is a subspace of as it fulfilled the above two conditions. (1 pt)
■
5. For a linear transformation → , explain why Im is a subspace of .
(Proof) .......
■
[Math Genealogy Tree]
http://genealogy.math.ndsu.nodak.edu
http://genealogy.math.ndsu.nodak.edu/id.ph
p?id=45061
- 418 -
This book was translated by the following authors
from a free Big Book in http://www.bigbook.or.kr/ which is an interactive smart
Linear Algebra textbook for everyone.
[ Authors ]
http://matrix.skku.ac.kr/sglee/vita/LeeSG.htm
Sang-Gu LEE
is a Professor of Mathematics at Sungkyunkwan
University and Vice president of Korean Mathematical Society. He
was a director of BK 21 Math Modeling Division, LOC member of
ICM
2014,
ILAS
2014
and
ICME
12.
He
has
held
visiting
appointments at the College of William and Mary, University of
Northern Iowa and Utah State University. He is a member of the International
Linear Algebra Society's Education Committee. His areas of interest include Matrix
Theory and Mathematical Modeling, Computational Mathematics. He has authored
about 20 books and published 140 scientific papers. sglee@skku.edu
Jon-Lark Kim
is an Associate Professor of Mathematics at Sogang Unviersity,
South Korea. He received the Ph.D. degree in mathematics from the University of
Illinois at Chicago, in 2002. From 2005 to 2012 he was with the Department of
Mathematics at the University of Louisville. He awarded a 2004 Kirkman Medal of
the Institute of Combinatorics and its Applications. He is a member of the Editorial
Board of both “Designs, Codes, and Cryptography” (2011-current) and “Journal of
Algebra, Combinatorics, Discrete Structures and Applications” (2014-current). His
areas
of
interest
include
Coding
Theory
and
its
interaction
with
Algebra,
Combinatorics, Number Theory, Cryptography, and Industrial Mathematics.
In-Jae KIM
is a Professor of Mathematics at Minnesota State University,
Mankato. His research interests include linear algebra & matrix theory and their
applications in machine learning and big data. He was an assistant managing
editor of Electronic Journal of Linear Algebra from Fall 2010 to Spring 2013, and
has been a reviewer for mathematics journals such as Linear Algebra and its
Applications, Linear and Multilinear Algebra, and Electronic Journal of Linear
Algebra.
Namyong Lee
is a Professor of Mathematics at Minnesota State University,
Mankato. He received his Ph. D. degree in Mathematics from University of
Minnesota.
His other academic related appointments includes, research director of
- 419 -
Minnesota Modeling and Simulation Center, secretary of Mathematical Association
of America, NCS, and associate editor of Frontiers in Systems Biology.
His areas
of interest include, Mathematical Biology, Industrial Mathematics, and Partial
Differential Equations.
Ajit Kumar :
Dr. Ajit Kumar is a faculty at the Department of Mathematics,
Institute of Chemical Technology, Mumbai, India since 2004. He received his Ph.D.
degree from the University of Mumbai in 2002. His main interests are differential
geometry,
optimization
techniques
and
the
use
of
technology
in
teaching
mathematics. He has initiated a lot of mathematicians into the use of open source
mathematics software. He has authored two books and several book chapters. For
the last several years, Dr. Ajit Kumar have been associated with the Mathematics
Training and Talent Search Programme, India, aimed at undergraduates.
Victoria Lang
is
a
graduate
student
of
Mathematics
at
Sungkyunkwan
University in South Korea. Originally from Maine, USA, she received her Bachelor’s
of Science in Mathematics and a minor in Korean Language & Culture from the
University of Pittsburgh.
Her most recent research interests are mathematics with
3-D printers, integrating 3-D printers into the curriculum of undergraduate-level
courses, and mathematical modelling. She currently is the webmaster of the
American International Student Association in Korea.
Phong Vu
is a Professor of Mathematics at Ohio University who is active in
research on Operator and Matrix Analysis. He got his Dr.Sc., degree from Kharkov
State University, Russia. Recently, Sang-Gu and Phong wrote several interesting
papers together.
Jae Hwa Lee
received his Ph.D. from Chinese Academy of Sciences in 2009,
And he was a Postdoctoral Researcher at BK21 Math. Modeling HRD. Div.,
Sungkyunkwan University from 2010-2012. Now he is a Lecturer in the Department
of Mathematics at Hallym University. His research interests include Computational
Mathematics, Mathematics Education and History of Korean Mathematics, etc.
< The End >
- 420 -
http://sage.skku.edu http://www.sagemath.org
http://matrix.skku.ac.kr/LA-Lab/
"Headquarter for sharing free textbooks"
was made to share
knowledges and
university textbook for educational equality
and a better future of our society.
http://bigbook.or.kr/
- 421 -
RELATED PAPERS
Ad Vivum? Visual Materials and the Vocabulary of Lifelikeness in Europe before 1800
Introduction: From Living Presence to Lively Likeness -the Lives of ad vivumSlavia Meridionalis
The Bulgarian-Romanian language boundary: anthroponymical data2015 •
In Winds of Change Environment and Society in Anatolia, edited by C.H. Roosevelt and J. Haldon. Chicago and Koç University Press.
Of Wetlands and Reclamation Regimes: Climate Change, Social Upheaval, and Political Practice in Western Anatolia in the Long Nineteenth Century2022 •
Journal of Social History
“Yerba Mate: The Drink that Shaped a Nation. By Julia J.S. Sarreal. Guaraná: How Brazil Embraced the World's Most Caffeine-Rich Plant. By Seth Garfield”2023 •
Cultural Anthropology
Alien Ocean: Anthropological Voyages In Microbial Seas by Stefan Helmreich2010 •
Fervent: A Woman's Battle Plan to Serious, Specific and Strategic Prayer
Fervent A Womans Battle Plan to Serious Specific and Strategic PrayerLa congiuntura del primo Trecento in Lombardia
«PACE, PACE, MORTE AI DAZI E ALLE GABELLE!» IL PESO DELLA GUERRA NELLA LOMBARDIA DEL PRIMO TRECENTO2019 •
International Review of Economics & Finance
Option Valuation Under Stochastic Volatility with Mathematica Code2002 •
Στο Ηλίας Ε. Κουρκούτας, (Επιμ.). Σχολική συμβουλευτική και παρεμβάσεις για παιδιά με διαταραχές. Αθήνα: Πεδίο, σσ. 463 -502.
Ένα πρόγραμμα συμβουλευτικής παρέμβασης και εικαστικών τεχνικών (Art therapy) για την ενίσχυση της ψυχικής ανθεκτικότητας παιδιών με ΕΕΑ/ διαταραχές. Η. Κουρκούτας, M. Bat Or, Κ. Χρηστίδης, Π. Σταύρου, Μ. Σμυρνάκη & Α. Παπαδάκη2017 •
EDUCARE: Journal of Primary Education
Using Poster Comment Method to Improve English Writing Ability of V-A Class at MIN 4 JemberBioelectromagnetics
Analysis of three-dimensional SAR distributions emitted by mobile phones in an epidemiological perspective2011 •
2012 •
The Neuroscience Journal of Shefaye Khatam
P80: The Effects of Progesterone Receptors' Antagonist RU-486 on BrainEdema, Intracranial Pressure and Neurological Outcomes after Traumatic Brain Injury2018 •
The European Physical Journal Special Topics
Isomeric decay studies around 204Pt and 148Tb2007 •
Journal of Computing in Higher Education
Key factors to instructors’ satisfaction of learning management systems in blended learning2011 •
2011 •
- Find new research papers in:
- Physics
- Chemistry
- Biology
- Health Sciences
- Ecology
- Earth Sciences
- Cognitive Science
- Mathematics
- Computer Science