MATH 304 Linear Algebra Orthogonal Bases. The Gram-Schmidt Orthogonalization Process

MATH 304
Linear Algebra
Lecture 28:
Orthogonal bases.
The Gram-Schmidt orthogonalization process.
Orthogonal sets
Let V be an inner product space with an inner
p
product h·, ·i and the induced norm kvk = hv, vi.
Definition. Nonzero vectors v1 , v2 , . . . , vk ∈ V
form an orthogonal set if they are orthogonal to
each other: hvi , vj i = 0 for i 6= j.
If, in addition, all vectors are of unit norm,
kvi k = 1, then v1 , v2 , . . . , vk is called an
orthonormal set.
Theorem Any orthogonal set is linearly

independent.
Orthonormal bases
Let v1 , v2 , . . . , vn be an orthonormal basis for an
inner product space V .
Theorem Let x = x1 v1 + x2 v2 + · · · + xn vn and
y = y1 v1 + y2 v2 + · · · + yn vn , where xi , yj ∈ R. Then
(i) hx, yi = x1 y1 + x2 y2 + · · · + xn yn ,
p
(ii) kxk = x12 + x22 + · · · + xn2 .
Proof: (ii) follows from (i) when y = x.
* n n
+ n
* n
+
X X X X
hx, yi = xi vi , yj vj = xi vi , yj vj
i=1 j=1 i=1 j=1
n
XXn n
X
= xi yj hvi , vj i = xi yi .
i=1 j=1 i=1
Orthogonal projection
Theorem Let V be an inner product space and V0

be a finite-dimensional subspace of V . Then any
vector x ∈ V is uniquely represented as x = p + o,
where p ∈ V0 and o ⊥ V0 .
The component p is the orthogonal projection of
the vector x onto the subspace V0 . We have
kok = kx − pk = min kx − vk.
v∈V0
That is, the distance from x to the subspace V0 is
kok.
x
o
V0
Let V be an inner product space. Let p be the
orthogonal projection of a vector x ∈ V onto a
finite-dimensional subspace V0 .
If V0 is a one-dimensional subspace spanned by a
hx, vi
vector v then p = v.
hv, vi
If v1 , v2 , . . . , vn is an orthogonal basis for V0 then
hx, v1 i hx, v2 i hx, vn i
p= v1 + v2 + · · · + vn .
hv1 , v1 i hv2 , v2 i hvn , vn i
n
X hx, vj i hx, vi i
Indeed, hp, vi i = hvj , vi i = hvi , vi i = hx, vi i
j=1
hvj , vj i hvi , vi i
=⇒ hx−p, vi i = 0 =⇒ x−p ⊥ vi =⇒ x−p ⊥ V0 .

The Gram-Schmidt orthogonalization process
Let V be a vector space with an inner product.

Suppose x1 , x2 , . . . , xn is a basis for V . Let
v1 = x1 ,
hx2 , v1 i
v2 = x2 − v1 ,
hv1 , v1 i
hx3 , v1 i hx3 , v2 i
v3 = x3 − v1 − v2 ,
hv1 , v1 i hv2 , v2 i
.................................................
hxn , v1 i hxn , vn−1 i
vn = xn − v1 − · · · − vn−1 .
hv1 , v1 i hvn−1 , vn−1 i
Then v1 , v2 , . . . , vn is an orthogonal basis for V .
x3
v3
p3
Span(v1 , v2 ) = Span(x1 , x2 )
Any basis Orthogonal basis
−→
x1 , x2 , . . . , xn v1 , v2 , . . . , vn
Properties of the Gram-Schmidt process:

• vk = xk − (α1 x1 + · · · + αk−1 xk−1 ), 1 ≤ k ≤ n;
• the span of v1 , . . . , vk is the same as the span
of x1 , . . . , xk ;
• vk is orthogonal to x1 , . . . , xk−1 ;
• vk = xk − pk , where pk is the orthogonal
projection of the vector xk on the subspace spanned
by x1 , . . . , xk−1 ;
• kvk k is the distance from xk to the subspace
spanned by x1 , . . . , xk−1 .
Normalization
Let V be a vector space with an inner product.

Suppose v1 , v2 , . . . , vn is an orthogonal basis for V .
v1 v2 vn
Let w1 = , w2 = ,. . . , wn = .
kv1 k kv2 k kvn k
Then w1 , w2 , . . . , wn is an orthonormal basis for V .
Theorem Any finite-dimensional vector space with

an inner product has an orthonormal basis.
Remark. An infinite-dimensional vector space with
an inner product may or may not have an
orthonormal basis.
Orthogonalization / Normalization
An alternative form of the Gram-Schmidt process combines
orthogonalization with normalization.
Suppose x1 , x2 , . . . , xn is a basis for an inner
product space V . Let
v1
v1 = x1 , w1 = kv1 k ,
v2
v2 = x2 − hx2 , w1 iw1 , w2 = kv2 k ,
v3
v3 = x3 − hx3 , w1 iw1 − hx3 , w2 iw2 , w3 = kv3 k ,
.................................................
vn = xn − hxn , w1 iw1 − · · · − hxn , wn−1 iwn−1 ,
wn = kvvnn k .
Then w1 , w2 , . . . , wn is an orthonormal basis for V .
Problem. Let V0 be a subspace of dimension k in
Rn . Let x1 , x2 , . . . , xk be a basis for V0 .
(i) Find an orthogonal basis for V0 .
(ii) Extend it to an orthogonal basis for Rn .
Approach 1. Extend x1 , . . . , xk to a basis x1 , x2 , . . . , xn for
Rn . Then apply the Gram-Schmidt process to the extended
basis. We shall obtain an orthogonal basis v1 , . . . , vn for Rn .
By construction, Span(v1 , . . . , vk ) = Span(x1 , . . . , xk ) = V0 .
It follows that v1 , . . . , vk is a basis for V0 . Clearly, it is
orthogonal.
Approach 2. First apply the Gram-Schmidt process to
x1 , . . . , xk and obtain an orthogonal basis v1 , . . . , vk for V0 .
Secondly, find a basis y1 , . . . , ym for the orthogonal
complement V0⊥ and apply the Gram-Schmidt process to it
obtaining an orthogonal basis u1 , . . . , um for V0⊥ . Then
v1 , . . . , vk , u1 , . . . , um is an orthogonal basis for Rn .
Problem. Let Π be the plane in R3 spanned by
vectors x1 = (1, 2, 2) and x2 = (−1, 0, 2).
(i) Find an orthonormal basis for Π.
(ii) Extend it to an orthonormal basis for R3 .
x1 , x2 is a basis for the plane Π. We can extend it
to a basis for R3 by adding one vector from the
standard basis. For instance, vectors x1 , x2 , and
x3 = (0, 0, 1) form a basis for R3 because

1 2 2
1 2
−1 0 2 =
−1 0 = 2 6= 0.

0 0 1
Using the Gram-Schmidt process, we orthogonalize
the basis x1 = (1, 2, 2), x2 = (−1, 0, 2), x3 = (0, 0, 1):
v1 = x1 = (1, 2, 2),
hx2 , v1 i 3
v2 = x2 − v1 = (−1, 0, 2) − (1, 2, 2)
hv1 , v1 i 9
= (−4/3, −2/3, 4/3),
hx3 , v1 i hx3 , v2 i
v3 = x3 − v1 − v2
hv1 , v1 i hv2 , v2 i
2 4/3
= (0, 0, 1) − (1, 2, 2) − (−4/3, −2/3, 4/3)
9 4
= (2/9, −2/9, 1/9).
Now v1 = (1, 2, 2), v2 = (−4/3, −2/3, 4/3),
v3 = (2/9, −2/9, 1/9) is an orthogonal basis for R3
while v1 , v2 is an orthogonal basis for Π. It remains
to normalize these vectors.
hv1 , v1 i = 9 =⇒ kv1 k = 3
hv2 , v2 i = 4 =⇒ kv2 k = 2
hv3 , v3 i = 1/9 =⇒ kv3 k = 1/3
w1 = v1 /kv1 k = (1/3, 2/3, 2/3) = 13 (1, 2, 2),
w2 = v2 /kv2 k = (−2/3, −1/3, 2/3) = 31 (−2, −1, 2),
w3 = v3 /kv3 k = (2/3, −2/3, 1/3) = 31 (2, −2, 1).
w1 , w2 is an orthonormal basis for Π.
w1 , w2 , w3 is an orthonormal basis for R3 .

MATH 304 Linear Algebra Orthogonal Bases. The Gram-Schmidt Orthogonalization Process

Uploaded by

Copyright:

Available Formats

MATH 304 Linear Algebra Orthogonal Bases. The Gram-Schmidt Orthogonalization Process

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MATH 304 Linear Algebra Orthogonal Bases. The Gram-Schmidt Orthogonalization Process

Uploaded by

Copyright:

Available Formats

MATH 304

Theorem Any orthogonal set is linearly

Theorem Let V be an inner product space and V0

=⇒ hx−p, vi i = 0 =⇒ x−p ⊥ vi =⇒ x−p ⊥ V0 .

Let V be a vector space with an inner product.

Properties of the Gram-Schmidt process:

Let V be a vector space with an inner product.

Theorem Any finite-dimensional vector space with

You might also like