UNIT 2 INNER PRODUCT SPACE

UNIT 2
INNER PRODUCT SPACE

2.1 Inner Product Space

2.2 Orthogonal and Orthonormal sets
2.3 Orthogonal Complement and Orthogonal Projection
2.4 Least Square Approximation

Introduction
We know Euclidean inner product on the space  2 and  3 . In this chapter, we extended the
concept to a vector space V over  or  . In this vector space we place an additional
structure to obtain an inner product space, and in this context we are going to define length,
distance and angle.

In linear Algebra I we see the orthogonal projection of a single vector onto another vector.
Here we will extend the notion of orthogonal projection to any vector subspace and consider
these fully.

Objectives
At the end of the lesson, students will be able to
 define inner product spaces.
 identify inner product spaces.
 evaluate length, distance and angle in any inner product space.
 use the Gram-Schmidt orthogonalization process to find orthogonal basis.
 find orthogonal projection operator from a vector subspace to a vector space.
 find an orthogonal complement of a subspace.
 use orthogonal projection to solve real world problems.

BDU,DEPARTEMENT OF MATHEMATIC | 43
 Math 2042 LINEAR ALGEBRA II

2.1 INNER PRODUCT SPACE

Definition 2.1.1 An inner product on a vector space V over field F , where F   or

 , is a function that maps each pair of vectors u, v in V to a unique number u, v
such that for all vectors u, v and w in V and any scalar c in F .
A1) Conjugate axiom: u, v  v, u in  or u, v  v, u in  .
A2) Positivity axiom: u, u  0 and u, u  0 if and only if u  0 .
A3) Homogeneity axiom: cu, v  c u, v .
A4) Additive axiom: u  w, v  u, v  w, v .

The vector space V over field F which satisfy the above four axioms is an inner product
space. If F   we say V is a real inner product space or Euclidean space.
If F   we say V is a complex inner product space or unitary space.

Examples of inner product space

Example 2.1.1 V   n Euclidean inner product space with u, v  u1v1  u2 v2    un vn
called the standard inner product in  n .
Example 2.1.2 V   n unitary space with u, v  u1 v1  u2 v2    un vn called the
standard inner product in  n .
Example 2.1.3 V  C  a, b the space of continuous real valued functions on  a, b with

f , g   f ( x) g ( x)dx is called the usual inner product space on C  a, b .

b

Let us show the above four axioms. Let f ( x), g ( x) and h( x) in C  a, b , that is,
f ( x), g ( x) and h( x) are continuous real valued functions on  a, b .
b b
A1) f , g   f ( x) g ( x)dx   g ( x) f ( x)dx  g , f , since F  
a a

 f ( x)   f ( x) 
b b
A2) f , f   dx  0 and f , f   dx  0if and only if f ( x)  0
2 2
a a
b b
A3) cf , g   cf ( x) g ( x)dx  c  f ( x) g ( x)dx  c f , g
a a

 f ( x)  h( x)  g ( x)dx  a
b b b
A4) f  h, g   f ( x) g ( x)dx   h( x) g ( x)dx  f , g  h, g
a a

44 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

Example 2.1.4 Let V be the vector space of complex continuous functions on the real
b
interval a  t  b , then f , g   f ( x) g ( x)dx is the usual inner product space.
a

Example 2.1.5 V   n
Weighted Euclidean inner product space with weight w1 , w2 ,, wn
where wi  0, for i  1, 2,, n with u, v  w1u1v1  w2 u2 v2    wn un vn
Example 2.1.6 V  Pn the vector space of the set of all polynomials of degree at most n ,
where p  p0  p1 x  p2 x 2    pn x n and q  q0  q1 x  q2 x 2    qn x n in Pn
with p, q  p0 q0  p1q1  p2 q2    pn qn .
Example 2.1.7 V  M nxn the vector space of the set of nxn matrix with A, B  tr ( B* A)
A and B in M nxn , where A* called the adjoint of A defined by A*  At and tr ( A) is the sum
of the diagonal elements of A is called Frobenius inner product space.
Example 2.1.8 Let V be the vector space of all infinite sequences of real numbers
 a1 , a2 , a3 , , satisfying i1 ai 2  a12  a22  a32     , that is, the sum converges


Addition and scalar multiplication are defined in V component wise, that is,
If u   a1 , a2 , a3 , , and v   b1 , b2 , b3 , then
u  v   a1  b1 , a2  b2 , a3  b3 , , and ku   ka1 , ka2 , ka3 ,
An inner product is defined in V by u, v  a1b1  a2 b2  a3b3 
The above sum converges absolutely for any pair of points in V . Hence the inner product is
well defined. This inner product space is called Hilbert space or  2  space .

Theorem 2.1.1 Properties of inner product space

If u, v and w are vectors in an inner product space V and for any c in F .
1) 0, v  v,0  0
2) u, cv  c u, v
3) u, v  w  u, v  u , w
4) u  w, v  u, v  w, v
5) u, v  w  u, v  u, w

Proof 1) 0, v  0 x0, v  0 0, v  0

2) u, cv  cv, u  c v, u  c v, u  c u, v .

BDU,DEPARTEMENT OF MATHEMATIC | 45
 Math 2042 LINEAR ALGEBRA II

EXERCISES 2.1_________________________________________________
1. Compute the inner product u, v with the inner product defined in
a) example 2.1.5 with w1  2, w2  3, w3 1, where u  (1, 1, 2) and v  (3, 2, 1)
 3 0 0 1 0 2
   
b) example 2.1.7 where u   2 4 1  and v   2 7 0 
 4 3 0  4 2 1
   
2. Determine whether the following are inner product spaces or not.
a) V   2 with u, v  u1v1  u2 v1  u1v2  u2 v2
b) V   3 with u, v  3u1v1  2u2 v2  4u3 v3
c) V  M 2 x 2 with A, B  a11b11  a12 b12  a21b21  a22 b22
d) V  P2 let p  p( x) and q  q( x) be polynomials in P2 with
p, q  p(1)q(1)  p(0)q(0)  p(1)q(1)
e) V  M nxn the vector space of the set of nxn real matrix with A, B  tr ( Bt A)

3. a) Let u   z1 , z2  and v   w1 , w2  be in  2 . Verify that

u, v  z1 w1  (1  i) z1 w2  (1  i) z2 w1  3z2 w2 is an inner product space on  2 .

b) Let u   z1 , z2  and v   w1 , w2  be in  2 . For what value of a, b, c, d in 

u, v  a z1 w1  bz1 w2  cz2 w1  dz2 w2 is an inner product space on  2 .

4. Let u , v and w be vectors in an inner product space V and a , b be scalars in F then

a) verify the relation u, av  bw  a u, v  b u, w .
b) u  0 if and only if u, v  0 , for any v in V .
c) u  v if and only if u, w  v, w , for any w in V .
5. In theorem 2.1.1 prove part 3, 4 and 5.

6. Suppose that u, v an inner product space in V . Show that

a) u, v is an inner product space in V , define u, v  r u, v and r  0 .
b) the sum of two inner product spaces in V is an inner product space in V .
7. Let   v1 , v2 ,, vn  be a basis in an inner product space V and let u and v in V .
Show that a) if v, vi  0 for each i  1, 2,, n then v  0 .
b) if u, vi  v, vi for each i  1, 2,, n then u  v .
8. Let A be an n  n invertible matrix. For u and v in  n , define u, v   Au  .  Av  .
Prove that this rule defines an inner product on  n .

46 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

Length and Distance in an inner product space.

Definition 2.1.2 Let u and v be vectors in an inner product space V then

a) the length or norm of u denoted by u , defined by u  u, u
b) the distance between two vectors u and v is denoted by d (u , v) defined by
d (u , v)  u  v
c) If u  1 then u is called a unit vector.

Example 2.1.9 Length and Distance in the above inner product spaces.
1. In V   Euclidean inner product space, if u and v are in  n , then
n

u  u, u  u12  u22    un2

d (u , v)  u  v  (u  v),(u  v)

In particular if V   2 , with u  ( x, y) and u  u, u  x 2  y 2 .

The set C  u   2 : u  1 is a unit circle center at the origin.
2. In V   Weighted Euclidean inner product space with u, v  3u1v1  2u2 v2
2

u  u, u  3u12  2u22 . In particular if u  (1, 0) then u  3 .

The set C  u   2 : u  1 is a unit circle center at the origin.
3. In V  Pn with p, q  p0 q0  p1q1  p2 q2    pn qn , where p  p( x)  Pn ,

p  p, p  p0 2  p12    pn 2

In P3 the set C   p  P : p 
3 
p0 2  p12  p32  1 is a unit sphere.

4. In V  C  a, b with f , g   f ( x) g ( x)dx
b

  f ( x) 
b
f  f, f 
2
dx
a

In particular if  a, b  0,1 and f ( x)  3x  5 then

0  f ( x) 
1 1
f  f, f  dx   (9 x  30 x  25)dx  13 .
2 2
0

Theorem 2.1.2 (Cauchy Schwarz inequality)

If u and v are vectors in an inner product space V then u, v  u v .

BDU,DEPARTEMENT OF MATHEMATIC | 47
 Math 2042 LINEAR ALGEBRA II

Proof. The proof is in the real inner product space.

(See activity 2.2 problem 4 (b) for the complex inner product space)
If u  0 , the statement is trivial.
Assume u  0 by the property of inner product for any scalar t ,
0  tu  v  tu  v, tu  v  t 2 u, u  2t u, v  v, v
2

 t 2 u  2t u, v  v
2 2
a quadratic polynomial of t .
0  t 2 u  2t u, v  v
2 2
implies that quadratic polynomial of t has no real root or has
repeated root. There fore its discriminate must satisfy the inequality
4 u  0 . Hence u, v  u v
2 2 2
4 u, v v
Example 2.1.10 Consider any real numbers a1 , a2 , a3 ,, an , b1 , b2 , b3 ,, bn using Cauchy
Schwarz inequality in Euclidean inner product space,
(a1b1  a2 b2  a3b3    an bn )2   a12  a2 2  a32    an 2 b12  b22  b32    bn 2 
Where u  (a1 , a2 , a3 ,, an ) and v  (b1 , b2 , b3 ,, bn )

Theorem 2.1.3 (Properties of length)

Let u and v be vectors in an inner product space V and c is any scalar in F then
1) u  0.
2) u  0 if and only if u o.
3) cu  c u .
4) Triangle inequality: u  v  u  v .

We shall prove part (4) of theorem 2.1.3 and leave the remaining part of the theorem as an
exercise.
Proof. Proof of part 4.
By definition u  v  u  v, u  v
2

 u, u  u, v  v, u  v, v
 u  u, v  v, u  v
2 2

But u  u, v  v, u  v  u  u, v  v, u  v
2 2 2 2

Since u, v  u v by Cauchy Schwarz inequality

 u  v 
2
u  u, v  v, u  v  u  2 u v  v
2 2 2 2

Thus u  v  u  v

48 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

Definition 2.1.3 Let u and v be two non-zero vectors in an inner product space V .
The angle  between u and v defined by
u, v
cos    , where 0    
u v

From Cauchy Schwarz inequality it follows that 1  cos    1

Thus  is well defined.
The above definition gives the formula u, v  u v cos  
Example 2.1.11 Verify Cauchy Schwarz inequality.
Determine the cosine of the angle between u and v
a) V   3 weighted inner product space with u, v  3u1v1  2u2 v2  u3 v3 and
u  (1, 1,1) and v  (1, 2,3)

Solution. u ,v  3  1 1  2  1 2  1 3  4  4

u  3  1 1  2  1 1  1 1  3  2  1  6

v  3  1 1  2  2  2  3  3  3  8  9  20

Thus u ,v  4  6 20  u v

u, v 4  4 
cos     ,   cos 1  .
u v 120  120 

b) V  P2 with p, q  p0 q0  p1q1  p2 q2 ,

where p  1  5  2 2 and q  2  4  9 2

Solution. p ,q  2  20  18  0

, p  p02  p12  p22  30, q  q02  q12  q22  101

p, q  0  30 101  p q

p, q
cos    0
p q

BDU,DEPARTEMENT OF MATHEMATIC | 49
 Math 2042 LINEAR ALGEBRA II

Theorem 2.1.4 (Properties of distance)

If u, v and w are vectors in an inner product space V then
1) d (u, v)  0.
2) d (u, v)  0 if and only if u  v.
3) d (u, v)  d (v, u).
4) Triangle inequality: d (u, v)  d (u, v)  d (w, v).

Exercsises 2.2_________________________________________________
1. Let V   3 be an Euclidean inner product space with u  (1,1, 1) and v  (6,7, 15) .
a) If cu  v  13 , what is the value of c ?
b) Find d (u, v).
c) Find the cosine of the angle between u and v.
2. Let V   3 be a complex inner product space with u  (1, 2i,1  i) and v  (i, 2  i,3) .
Compute
a) u, v b) u c) 3u, 2iv d) u  2v

3. Let u  2, v  3 and u, v  2  i . Compute

a) u  v b) u  v c) u  v, u  iv

2 1  2 1
4. Let V  M 2 x 2 with A, B  tr ( Bt A) . Let A    and B    then
 5 2  1 3 
a) verify Cauchy Schwarz inequality.
b) find d ( A, B).

5. a) Show that Cauchy Schwarz inequality can be expressed in the form of

u, v  u, u v, v in a real inner product space.
2

b) Prove theorem 2.1.2 for the complex inner product space.

c) Prove theorem 2.1.3 part 1, 2 and3.
d) Prove theorem 2.1.4.
6. Let V be an inner product space. Show that u  v if and only if u  v, u  v  0 .

7. Polarization identity: (which shows the inner product can be obtained from the norm
function). Let u and v be vectors in an inner product space V then
1
a) u, v   u  v  u  v  , if V is a real inner product space.
2 2

4 
1
b) u, v   u  v  u  v  i u  iv  i u  iv  , if V is a complex inner
2 2 2 2

4  
product space.

50 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

8. Use Cauchy-Schwarz inequality.

i) Prove that for all real values of a, b and  ,  a cos   b sin    a 2  b2
2

ii) Let f ( x) and g ( x) be continuous real valued functions on  0,1 .

2
Prove that a)   f ( x) g ( x)dx     f 2 ( x)dx    g 2 ( x)dx 
1 1 1

 0   0   0 
1 1 1

b)    f ( x)  g ( x)  dx     f 2 ( x)dx     g 2 ( x)dx  Parallelogram identity:

1 2 2 1 2 1 2

 0   0   0 
9. Let u and v be vectors in an inner product space V then show
that u  v  u  v
2 2

2 u  v
2 2
.
___________________________________________________________

2.2 Orthogonal and Orthonormal sets

Definition 2.2.1 Let V be an inner product space.

1) Let u and v be vectors in V . We say u and v orthogonal or

perpendicular, denoted by u  v if u, v  0 .
2) A set of non zero vectors v1, , v2 ,, vn is said to be an orthogonal set if
vi , v j  0 , for i  j .
3) An orthogonal set of vectors v1, , v2 ,, vn is called an orthonormal set if
0 , if i  j
each vector is a unit vector. That is vi , v j  
1, if i  j

Example 2.2.1 a) 1,0,0 ,  0,1,0 , 0,0,1 is the standard orthonormal basis of  3

 0,1,0 , 1,0,1 , 1,0, 1 is an orthogonal basis of  under the

3

usual inner product.

b) 1  i,1,1  i, i  , 1  5i,6  5i, 7  i,1  6i  ,  2  4i, 4  3i,6  i  is an
orthogonal set in  4 under the usual inner product.

c) 1,sin  ,cos  ,sin 2  ,cos 2  ,,sin m  ,cos m  is an
orthogonal set where m is a positive integer in C   ,   with

f,g   f ( x) g ( x)dx .


BDU,DEPARTEMENT OF MATHEMATIC | 51
 Math 2042 LINEAR ALGEBRA II

Theorem 2.2.1 (Generalized theorem of Pythagoras)

If u and v are orthogonal vectors in an inner product space V then u  v  u  v .
2 2 2

Theorem 2.2.2 Let V be an inner product space. Let S  v1 , v2 ,, vn  the subset of V
be an orthogonal set then S is linearly independent set.

Proof. Suppose x1v1  x2 v2    xx vn  0

Take inner product on both sides with v j , for j  1, 2, , n . More precisely
x1v1  x2 v2    xn vn , v j  0, v j  0 , for j  1, 2, , n .

 x vi , v j  0 . Since vi , v j  0 , for i  j
n
i 1 i

It follows that x j v j , v j  0 . Hence x j  0 since v j , v j  0 , for j  1, 2, , n .

Computational Advantage of orthogonal sets

Projections
Let V be an inner product space and w and v be in V , with v  0 . We seek the
“projection of w along v ”, which, as indicated below, will be the scalar multiple of v , that
is,  v such that w   v is orthogonal to v . w

w  v
v

v v

w, v
The projection of w along v is denoted by Pv w defined by Pv w  v
v, v
Example 2.2.2 Find the projection of w  (3  4i, 2  3i) along v  (5  i, 2i) in  2 .
Solution. w, v   3  4i  5  i    2  3i  2i   19  17i  6  4i  13  13i
v, v  (5  i)(5  i)  (2i)(2i)  25  1  4  30
w, v 13  13i 1
Thus Pv w  v (5  i, 2i)   26  39i,  13  13i 
v, v 30 15

52 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

Theorem 2.2.3 Let V be an inner product space. Let w and v be vectors in V with
w, v
v  0 then w   v is orthogonal to v , where   is called the Fourier
v, v
coefficient of w with respect to v .

Theorem 2.2.4 Let V be an inner product space and w in V . Let v1 , v2 ,, vn  be an
orthogonal basis of V . Suppose  x1 , x2 ,, xn  is the coordinates of w with respect to
w, vi w, vi
the given basis then xi  . More precisely, w   i 1
n
vi .
vi , vi vi , vi

Proof. Suppose w  x1v1  x2 v2    xn vn .

Take inner product on both sides with v j , for each j  1, 2,, n .
w, v j  x1v1  v2 v2    xn vn , v j

w, v j  x j v j , v j since vi , v j  0 , for i  j .
w, v j
Thus x j  , for each j  1, 2,, n .
vj ,vj

Corollary 2.2.5 Let V be an inner product space and w in V . Let v1 , v2 ,, vn  be an
orthonormal basis of V then w  w, v1 v1  w, v2 v2    w, vn vn  i 1 w, vi vi .
n

Example 2.2.3 v1   2, 2, 1 , v2   1, 2, 2  and v3   2, 1, 2  are orthogonal basis of  3 .

Express w  1, 2,3 as a linear combination of v1 , v2 and v3 .
w, v1 w, v2 w, v3 1 2
Solution. w  v1  v2  v3  v1  v2  v3 or
v1 , v1 v2 , v2 v3 , v3 3 3
v1 1 v 1 v 1
  2, 2, 1 , 2   1, 2, 2  and 3   2, 1, 2  are orthonormal basis of  3 .
v1 3 v2 3 v3 3
v1 v1 v v2 v v3 1 2
w  w,  w, 2  w, 3  v1  v2  v3 .
v1 v1 v2 v2 v3 v3 3 3

BDU,DEPARTEMENT OF MATHEMATIC | 53
 Math 2042 LINEAR ALGEBRA II

In the above discussion to find coordinates of a vector in an orthogonal basis one does not
w, v j
need to solve a linear system, the coordinates are determined by the formula x j  .
vj ,vj
This formula is especially simple for orthonormal bases, when vi  1 .
Remark: The importance of orthonormal bases is that if we fix an orthonormal basis in an
inner product space V, we can work with coordinates in this basis the same way we work
with vectors in  n .

Theorem 2.2.6 (The Gram-Schmidt Orthogonalization Process)

Let v1 , v2 ,, vn  be a linearly independent set in an inner product space V .
Then there is an orthonormal set e1 , e2 ,, en  such that
Span of e1 , e2 ,, e j   span of v1 , v2 ,, v j  , for j  1, 2,, n.

Proof. The proof is constructive. The algorithm for construction is as follows.

v
1) Set e1  1 then e1  is orthonormal.
v1
w2
2) w2  v2  v2 , e1 e1 and set e2  , w2 is orthogonal to e1 .
w2
Hence e1 , e2  is an orthonormal set
w3
3) w3  v3  v3 , e1 e1  v3 , e2 e2 and set e3  , w3 is orthogonal to e1 and e2 .
w3
Hence e1 , e2 , e3  is an orthonormal set.
4) In general using the principle of induction
wn
wn  vn  vn , e1 e1  vn , e2 e2    vn , en1 en1 and set en  ,
wn
wn orthogonal to e1 , e2 ,, en1 . Hence e1 , e2 ,, en  is an orthonormal set.
To show span of e1 , e2 ,, e j   span of v1 , v2 ,, v j  , for j  1, 2,, n
The proof is by induction.
1) Span of e1   span of v1 
2) Assume span of e1 , e2 ,, ek   span of v1 , v2 ,, vk  , for 1  k  n
Need to show for k  1 .
 vk 1 , ei ei in span of e1 , e2 ,, ek  . It follows that vk 1  wk 1  i 1 vk 1 , ei ei in
k k
i 1

54 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

span of e1 , e2 ,, ek 1  . In other words vk 1 in span of e1 , e2 ,, ek 1  .

 vk 1 , ei ei in span of v1 , v2 ,, vk 

k
But by induction assumption i 1

wk 1  vk 1  i 1 vk 1 , ei ei in span of v1 , v2 ,, vk 1 

k

wk 1
ek 1  in span of v1 , v2 ,, vk 1  .
wk 1
Finally wk 1  0 otherwise vk 1  i 1 vk 1 , ei ei is in span of v1 , v2 ,, vk 
k

Which contradicts the linear independence of v1 , v2 ,, vk 1 .

Example 2.2.4 Find an orthogonal basis for the subspace W given by x  2 y  3z  0 of  3
Solution. A basis for W is v1  (3,0,1) , v2  (2,1,0) . Verify.
Using Gram-Schmidt orthogonalization process
w1  v1  (3,0,1)
v2 , w1 6  1 3
w2  v2  w1  (2,1, 0)  (3, 0,1)    ,1,  
w1 , w1 10  5 5
 1 3
w1  (3,0,1) and w2    ,1,   are orthogonal basis for W .
 5 5

Corollary 2.2.7 Let V be an inner product space. Let v1, , v2 ,, vn  be any basis of V
Then there exists an orthonormal basis e1, , e2 ,, en  of V . More over any vector w in

V can be expressed as w  i 1 w, ei ei .
n

Example 2.2.5 Find an orthogonal set of the three vectors

w1  1,1  i,1 , w2   i,1,1  i  , w3   0, i, i  in the complex space  3 .
Solution. Verify that w1  1,1  i,1 , w2   i,1,1  i  and w3   0, i, i  are linearly
independent. Using Gram-Schmidt orthogonalization process
v1  w1  1,1  i,1
w2 , v1 1
v2  w2  v1   2  3i,1  i, 2  5i  . We replace v2 by 4v2 , which we rename
v1 , v1 4
v2   2  3i,1  i, 2  5i  , this simplification will not spoil our construction.
1
v3  w3  w3 , v1 v1  w3 , v2 v2  (3  i,1  3i, 1  i) . Similarly replace v3 by 11v3 ,
11
v3  (3  i,1  3i, 1  i)

BDU,DEPARTEMENT OF MATHEMATIC | 55
 Math 2042 LINEAR ALGEBRA II

1,1  i,1 ,  2  3i,1  i, 2  5i  , (3  i,1  3i, 1  i) is an orthogonal set.

Example 2.2.6 Consider the set   1, x, x 2 , x3  with inner product

1
p, q   p( x)q( x)dx . Find the orthogonal basis.
1

Solution. u1 ( x)  1
1

u ( x)  x 
 1
xdx
.1  x
2 1
 1
dx
1 1

u3 ( x)  x 2

 1
x 2 dx
.1 
1
x3 dx
.x  x 2 
1
1 1
 1
dx 1
xdx 3

1 1
x3 ( x 2  )dx 
1 1

u4 ( x )  x 3 1

 1
x3 dx
.x  .1 
 x 4 dx
3 11 3
.( x 2  )  x3  x and so on .
1 1
1 dx 
1 1 3 5
xdx
1 ( x  3)dx
2
1

The polynomials in this sequence are called Legendre polynomials.

Example 2.2.7 Find an orthonormal basis of the column space of

 1 0 1 
 
 1 2 3
A in Euclidean inner product space of  4 .
0 1 2
 
1 3 0
Solution. Let A   w1 , w2 , w3  , where w1   1,1,0,1 , w2   0, 2,1,3 and w3  1,3, 2,0 
are column of matrix A . Verify that w1 , w2 and w3 are linearly independent.
v1  w1   1,1,0,1
w2 , v1 5 1
v2  w2  v1   0, 2,1,3   1,1,0,1  5,1,3, 4 
v1 , v1 3 3
Replace v2 by 3v2 , which we rename v2   5,1,3, 4 
v3  w3  w3 , v1 v1  w3 , v2 v2
2 14
 1,3, 2,0  
 1,1,0,1  5,1,3, 4 
3 51
51v3  15,105,60, 90   15(1,7, 4, 6)
Replace v3 by 51v3 , v3  (1,7, 4, 6) .
v1   1,1,0,1 , v2   5,1,3, 4  , v3  1,7, 4, 6  our orthogonal basis.

56 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

 1 1 1 
It follows that  (1,1, 0,1),  5,1,3, 4  , (1, 7, 4, 6)  is an orthonormal set.
 3 51 102 
Consider the matrix Q whose column vectors are the above orthonormal set.
 1 5 1 
 1 1 1   3 51

102 
 0 
 3 3 3  1 1 7 
 5 1 3 4   3 51

102 
Qt Q     I3
 51 51 51 51   3 4 
0
6  
 1 
7 4 51 102 
 
 102 102 102 102   1 4 6 
 
 3 51 102 

Theorem 2.2.8 The columns of m  n matrix Q form an orthonormal set if and only if
Qt Q  I n

1, i  j
Proof. We need to show that  Qt Q   
ij
0, i  j
Let qi denotes the ith column of Q (and hence the ith row of Q t ). Since the ( i, j ) entry of
Q t Q is the dot product of the ith row of Q t and the jth column of Q , that is,
Q Q t
ij
 qi  q j (1)
Now the columns of Q form an orthonormal set if and only if
1, i  j
qi  q j   (2)
0, i  j
Combining equation (1) and (2)
1, i  j
Qt Q ij  0, i  j  I n

If the matrix Q in theorem 2.2.8 is a square matrix, it has a special name.

Definition 2.2.2 An n  n matrix Q whose columns for an orthonormal set is

called an orthogonal matrix.

The most important fact about orthogonal matrix is given by the next theorem.

Theorem 2.2.9 A square matrix Q is orthogonal if and only if Q1  Qt

BDU,DEPARTEMENT OF MATHEMATIC | 57
 Math 2042 LINEAR ALGEBRA II

0 0 1
 
Example 2.2.8 Q   1 0 0 
0 1 0
 
is a matrix whose columns form an orthonormal set. Verify Q1  Qt .

EXERCISES 2.3_____________________________________________
1. Let u , v and w be vectors in an inner product space V . Show that
a) if u is orthogonal to both v and w then u is orthogonal to  v   w , for any
scalars  and  .
b) if u, v, w  is an orthonormal set then u  v is not is orthogonal to v  w .

2. a) Show that S  1,1,0  , 1, 1,1 ,  1,1, 2   is an orthogonal basis of  3 in standard

inner product. Express w   2,1,3 as a linear combination of basis vectors.

 
b) Show that S  0.6 x 2  0.8,0.8x 2  0.6, x is an orthonormal basis of P2 with the
inner product p, q  p0 q0  p1q1  p2 q2 . Express w  x2  2 x  3 as a linear
combination of basis vectors.
 
c) Show that S  1, x  1, x 2  2 x  1 is an orthogonal basis of P2 with the inner
product p, q  p(1)q(1)  p(0)q(0)  p(1)q(1)  p(2)q(2)  p(3)q(3) .
Express w  x2  2 x  3 as a linear combination of basis vectors.
3. Determine whether the following set of vectors is orthonormal or not.

a) V   3 with standard inner product S  (i,1,0),(0, i,1),(0,0, i)

b) V  P2 with the inner product p, q  p0 q0  p1q1  p2 q2

2 2 1 2 1 2 1 2 2 
S    x  x2 ,  x  x2 ,  x  x2 
3 3 3 3 3 3 3 3 3 
c) V  M 22 with the inner product A, B  tr ( Bt A)
 2 1   3 0   2 1  
S   , , 
 1 3   0 2   1 1 
d) V  M 22 with the inner product A, B  a11b11  a12 b12  a21b21  a22 b22
 1 1   1 1  0 0   0 0  
S   , , ,  .
 0 0   0 0   1 1   1 1 

58 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

4. Find an orthonormal basis for each of the indicated subspaces.


a) W  ( x, y, z )   3 : x  y  z  0 . 
 a b  
b) W    : a, b, c    with inner product A, B  tr ( B A)
t

 0 c  
c) W   A  M 33 : tr ( A)  0 with inner product A, B  tr ( Bt A)
 dP 
d) W   p( x)  P3 : x  P( x)  with the inner product
 dx 
p, q  p0 q0  p1q1  p2 q2  p3 q3 .
e) W   p( x)  P3 : p( x)is even  with the inner product
1
p, q   p( x)q( x)dx
1
5. Let u and v be vectors in an inner product space V . Show that
u  v  u  v , if u and v orthogonal.
2 2 2
a)
b) u  v  u  v if and only if u  0, v  0 or u  cv ,for c  0 .
6. The column vectors of matrix A below form a linearly independent set, use
the Gram-Schmidt method to find an orthonormal basis for the row
vectors of A .
1 2 5 
 
A  0 2 0 
 1 0 1
 
7. Determine whether the given matrix below is orthogonal matrix, if it is, find its inverse.
 1 2 2   2 1 
 3 3 3   3 5
2
   
 cos   sin    2 1 2   4 2 
i) A    ii) A   3  3 3  iii) A   3  5 1 
 sin  cos      
 2 2 1
    1 2 0 
 3 3 3  3 

8. Prove a) theorem 2.2.1 b) theorem 2.2.3

c) theorem 2.2.5 d) theorem 2.2.7
e) theorem 2.2.9
9. If S  v1 , v2 ,, vn  an orthogonal set of vectors in an inner product space V
2 n

   vi
n 2
then show that a) v
i 1 i
i 1

  v  i 1 i vi , where  i ’s are scalars.

n n
b) i 1 i i

BDU,DEPARTEMENT OF MATHEMATIC | 59
 Math 2042 LINEAR ALGEBRA II

10. Let S  v1 , v2 ,, vn  be an orthonormal basis in an inner product space V

Let v   i 1i vi and u   i 1 i vi , where  i ’s and  i ’s are scalars.
n n

 i 1 i
2 n 2
Then show that a) v

b) u, v  i 1 i i
n

11. Let V be a vector space. Let S  v1 , v2 ,, vn  be a basis of V .

Let x   i 1i vi , and y   i 1 i vi define x, y  i 1i i
n n n

Prove that , define an inner product in V .

__________________________________________________________

2.3 Orthogonal Complement and Orthogonal Projection

Definition 2.3.1 Let V be an inner product space and W be a subspace of V then

1) v is orthogonal to the subspace W if v, w  0 , for each w in W .
2) the orthogonal complement of W , denoted by W  is the set of all vectors in
V which are orthogonal to W . More precisely,
W   v  V : v, w  0 , for each w in W .

Example 2.3.1 For any inner product space V . 0 a subspace of V and 0  V .


Besides V   0 .

Theorem 2.3.1 (Properties of orthogonal complements)

If W is a subspace of a finite dimensional inner product space V then
1) W  a subspace V .
2) the only vector common to both W and W  is the zero vector, that is ,
W  W   0 .
3) V is a direct sum of W and W  , that is, V  W  W  .
 

4) the orthogonal complement of W  is W . That is W  W

Proof. We shall proof (1) and (2). Part (3) and (4) are left as an exercise.
1) 0, w  0 , for each w in W . Thus 0  W 

60 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

Suppose u, v  W  , that is, u, w  0 and v, w  0 , for each w in W .

To show u  v  W  and ku  W  , for any scalar k .
u  v, w  u, w  v, w  0 .
Hence u  v  W  .
ku, w  k u, w  k 0  0 . Hence ku  W  .
2) Suppose u  W  W  then u, u  0 which implies u  0 by axiom 2.
Example 2.3.2 W  ( x, y, z ) : 2 x  y  3z  0 is a subspace of  3 under the usual
inner product then find W  .
Solution. Verify   (1, 2,0),(0, 3,1) is a basis of W .
Let u  W  with u  ( x1 , y1 , z1 ) then u,(1, 2,0)  0 and u,(0, 3,1)  0 .
(See activity 2.4 problem 6.)
In a system of equation form
x1  2 y1  0
3 y1  z1  0 .
Solving the system u is of the form t (2,1,3) , where t is any scalar.
Hence W   t (2,1,3) : t    , a line through the origin perpendicular to the plane W .
Example 2.3.3 Let W  span v1 , v2  , where v1  (1,1, 1, 4) and v2  (1, 1,1, 2)
a subspace of  4 under the usual inner product . Find W  .
Solution. Let u  W  with u  ( x1 , x2 , x3 , x4 ) then u, v1  0 and u, v2  0 .
Thus x1  x2  x3  4 x4  0
x1  x2  x3  2 x4  0
Solving the system we have the vector form of the general solution
u   3x4 , x3  x4 , x3 , x4  . In other words W   x3 (0,1,1,0)  x4  3, 1,0,1 : x3 , x4   
(0,1,1,0),  3, 1,0,1 is a basis of W 
.

Theorem 2.3.2 Let W be a finite dimensional subspace of an inner product space V .

Then for any v  V there exists unique vectors w  W and u  W  such that v  w  u .
Further more if v1 , v2 ,, vn  is an orthonormal basis of W then w  i 1 v, vi vi .
n

Proof. Define w  v  u then w  i 1 v, vi vi in W by corollary 2.27.

n

To show u  v  w  W  it suffices to show u is orthogonal to v j , for each j  1, 2,, n

BDU,DEPARTEMENT OF MATHEMATIC | 61
Math 2042 LINEAR ALGEBRA II

u, v j  v  i 1 v, vi vi , v j  v, v j  
n n
i 1
v, vi vi , v j  v, v j  v, v j  0
For the uniqueness part.
Suppose v  w  u  w1  u1 , where w1  W and u1  W 
then w  w1  u1  u  W  W   0 by theorem 2.3.1.

v u
v
v  projwv
w
projwv
Orthogonal projection

Recalling the definition of orthogonal projection of vector v on to w in  2 or  3 ,

v, w
Pwv  w using inner product notation. We are going to define orthogonal projection in
w, w
any vector space using theorem 2.3.2.

Definition 2.3.2 Let W be a subspace of an inner product space V . For any vector v its
orthogonal projection onto the subspace W , denoted by PW  v  is a vector in W such that
the vector v  PW (v) is orthogonal to the subspace W .

The next question is here. How the orthogonal projection operator is defined?

Theorem 2.3.3 Let W be a subspace of an inner product space V . Let v1 , v2 ,, vn  be an
orthogonal basis of W . The orthogonal projection PW  v  is given by the formula
n v, vi
PW (v)   vi .
i 1 vi , vi

The proof is based on theorem 2.3.1 and theorem 2.3.2.

The orthogonal projection PW is a linear operator. (See activity 2.4 problem 7(a).)

62 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

Theorem 2.3.4 (Best approximation theorem)

Let W be a subspace of an inner product space V . The orthogonal projection PW  v 
minimizes the distance from v to the subspace W . More precisely,
v  PW  v   v  x , for all x  W .

If v  PW  v   v  x then PW (v)  x .

 v  PW  v   PW  v   x
2
Proof. v  x
2

  v  PW  v     PW  v   x  since v  PW (v)  W  and PW (v)  x  W

2

 v  PW  v   PW  v   x  v  PW  v 
2 2 2

 v  PW  v   PW  v   x  v  PW  v 
2 2 2
If v  x
2

v
Then PW  v   x
2
 0 . It follows that PW (v)  x .
v p (v) vx
W

pW (v) W

Remark: PW  v  is a vector in W which is closest to v . More specifically, v  PW  v  is

the distance from v to the subspace W .
Example 2.3.4 W  ( x, y, z ) : x  y  2 z  0 is a subspace of  3 then find
a) orthogonal basis of W .
b) the equation of the orthogonal projection operator PW :  3  W .
c) the vector in W that is closest to v  (1,6,1) .
d) the distance from v  (1,6,1) to the subspace W .
Solution. a) u1  (1,1,0), u2  (2,0,1) a basis of W . Then an orthogonal basis for W is
v1  u1  (1,1,0)
u2 , v1 2
v2  u2  (1,1, 0)  (2, 0,1)  (1,1, 0)  (1,1,1)
v1 , v1 2
v1  (1,1,0), v2  (1,1,1) an orthogonal basis of W
b) See activity 2.4 problem 7(a) PW :  3   3 is a linear operator given by

BDU,DEPARTEMENT OF MATHEMATIC | 63
 Math 2042 LINEAR ALGEBRA II

v, v1 v, v2 x  y x yz
PW (v)  v1  v2  (1,1,0)  (1,1,1) , where v  ( x, y, z)
v1 , v1 v2 , v2 2 3
1
PW ( x, y, z )  (5 x  y  2 z,  x  5 y  2 z, x  y  z )
6
c) PW  v  a vector in the subspace W which is closest to vector v .
1
Using (b) PW (1,6,1)  (1,31,16) , is closest to v  (1,6,1) .
6
1
Observe that v  PW  v   (5,5, 10) is orthogonal to W .
6
1 5
d) v  PW  v   (5,5, 10)  .
6 6

EXERCISES 2.4______________________________________________

1. a) Let W be the plane in  3 with equation x  y  z  0 . Find the parametric

equation of W  .
b) Let W be the line in  3 with parametric equation x  2t , y  3t , z  4t ,
where t   . Find an equation of W  .
c) Let W be the intersection of two planes x  y  z  0 and x  y  z  0 in  3 .
Find an equation of W  .
2. Let W be the subspace of  3 given by W  span((i,1, i)) then find an orthonormal
basis of W  .
3. Let W be a subspace of an inner product space V spanned by
S  v1 , v2 ,, vn  and u  V then u is orthogonal W if and only if u, v j  0
for each j  1, 2,, n .
4. If A is an m  n real matrix then show that the null space of A and the row
space of A are orthogonal complements in  n with respect to the Euclidean
inner product.
5. Find the orthogonal complement of the subspace W of an inner product space V .
a) V  P2 with the inner product f , g   f ( x) g ( x)dx and W   f  P2 : f is even .
1

1

 0 a  
b) V  M 22 with the inner product A, B  tr ( Bt A) and W    : a, b    .
 b 0  

64 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

6. a) Let W be a subspace of an inner product space V and let v1 , v2 ,, vn  be an

orthogonal basis of W . Then show that the orthogonal projection PW  v  given by
the formula
n v, vi
PW (v)   vi is a linear operator.
i 1 vi , vi
b) Find the orthogonal projection operator P : W  V
i) V   3 with standard inner product, W  ( x, y, z ) : y  2 x  z .
ii) V  P2 with the inner product p, q  p0 q0  p1q1  p2 q2 , W  ax : a    .

ii) V  P2 with the inner product f , g   f ( x) g ( x)dx , W   f  P2 : f is even .

1

1

7. Let be an inner product space V . Let Wn and Wn 1 be subspaces of V with bases

v1 , v2 ,, vn  and v1 , v2 ,, vn , vn1 respectively.
Then show that a) v  PWn 1  v   v  PWn  v  .
b) interpret problem (a).
8. Let W and U are subspace of a finite dimensional inner product space V .
Show that a) if W  U then U   W  .
b) (U  W )  U   W  .
______________________________________________________________________

2.4 Least square approximation

One of a very important technique to solve real world problems is the least square
approximation.
Consider a linear system AX  B , where A   aij  matrix (1)
mn

If (1) is a consistent linear system, we know how to solve it. What do we do if (1) is
inconsistent, that is, (1) does not have a solution. This seems to be a silly question,
because if there is no solution, then there is no solution. But, if we obtained
the equation from an experiment, in real life it is impossible to avoid errors in
measurements. So it is possible that an equation that in theory should be consistent does not
have a solution. So, what one can do in this situation?
The simplest idea is to write down the error AX  B and try to find X minimizing it.
If we can find X such that the error is 0, the system is consistent and we have exact
solution. Otherwise, we get the so-called least square solution.

BDU,DEPARTEMENT OF MATHEMATIC | 65
 Math 2042 LINEAR ALGEBRA II

To solve the least square problem, let W be the column space of A . For each n  1 matrix
X , the product AX is a linear combination of the column vectors of A .
Thus, as X varies over  n , the vector AX varies over all possible linear combinations of
the column vectors of A ; that is, AX varies over the entire column space W .
Geometrically, solving the least squares problem amounts to finding a vector X in  n such
that AX is the closest vector in W to B.

It follows from the Best Approximation theorem 2.3.4 that the closest vector in W to B is
the orthogonal projection of B on W . Thus, for a vector X to be a least squares solution of
AX  B , this vector must satisfy
AX  PW  B  (2)
One could attempt to find least squares solutions of AX  B by first calculating the vector
PW  B  and then solving (2).
Example 2.4.1 Find the least squares solution of
x y 7
x  y  0
 x  2 y  7
Solution. In a matrix form
 1 1 7
  x   
 1 1   y    0  , is inconsistent. Let W be the column space of A ,
 1 2     7 
   
that is, W  span (1, 1,1), (1,1, 2) . Let us find orthogonal basis of W .
v1  (1, 1, 1)
2 1
v2  (1,1, 2)  (1, 1, 1)  (5,1, 4) . Take v2  (5,1, 4) for the seek of convince.
3 3
(1, 1, 1), (5,1, 4) is an orthogonal basis of W the column space of A.
B, v1 B, v2 1
PW  B   PW  7, 0, 7   v1  v2  (11, 9, 8)
v1 , v1 v2 , v2 2
Solve the linear system AX  PW  B  , that is,
x  y  11
2
 x  y  9
2
 x  2 y  4

66 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

Line fitting. Suppose that we know that two quantities x and y are related by the law
y  a0  a1 x . The coefficients a0 and a1 are unknown, and we would like to find them
from experimental data.
Suppose we run the experiment n times, and we get n pairs  xi , yi  , i  1, 2,, n

Ideally, all the points  xi , yi  should be on a straight line, but because of errors in
measurements, it usually does not happen. The points are usually close to some line, but not
exactly on it. That is where the least square solution helps!
Ideally, the coefficients a0 and a1 should satisfy the equations
yi  a0  a1 xi , i  1, 2,, n
(note that, xi and yi are some fixed numbers, and the unknowns are a0 and a1 .)
If it is possible to find such a0 and a1 we are lucky. If not, the standard thing to do, is to
minimize the total quadratic error
n
E   yi  ( a0  a1 xi )
2

i 1

 1  x1   y1 
     
1  x2  y
In a vector notation E  y   a0 v1  a1v1   and y   2 
2
, where v1  , v2 
     
     
 1  xn   yn 

But, minimizing this error is exactly finding the least square solution of the system

 1 x1   y1 
   
 1 x2   a0   y2 
  
    a1    
   
 1 xn   yn 

BDU,DEPARTEMENT OF MATHEMATIC | 67
 Math 2042 LINEAR ALGEBRA II

Example 2.4.2 Suppose our data  xi , yi  consist of pairs (2, 4),(1, 2),  0, 1 ,  2, 1 and
 3, 1 . Find the least square solution y  a0  a1 x.
 1 2   4
   
 1 1  2
 0  
a
Solution.  1 0      1  solving in a similar way like example 2.4.1 we have
   a1   
 1 2 1
 3  1
 1  
1 1
a0  2, a1   . The best fitting straight line is y  2  x .
2 2
Curve fitting. Suppose we know that the relation between x and y is given by the
quadratic law y  a  bx  cx 2 , so we want to fit a parabola y  a  bx  cx 2 to the data.
Then our unknowns a, b, c should satisfy the equations
yi  a  bxi  cxi 2 , i  1, 2,, n .
In matrix form
 1 x1 x12   y1 
   a   
 1 x2 x2 2     y2 
b 
         
  c   
 1 xn xn 2   yn 
Example 2.4.3 Using data in example 2.4.2 find the least square solution y  a  bx  cx 2
 1 2 4   4
  a  
 1 1 1     2 
 
Solution. In a matrix form  1 0 0   b    1 
    
 1 2 4  c  1
1 3 9 1
   
86 62 43
Which yields a  , b , c .
77 77 154
86 62 43 2
The best fitting parabola is y   x x .
77 77 154

Let V denote the vector space C  a, b of all continuous real valued functions on the closed
interval  a, b , with inner product
b
f , g   f ( x) g ( x)dx
a

Then

68 | BDU,DEPARTEMENT OF MATHEMATIC
 UNIT 2 INNER PRODUCT SPACE

b
 f ( x)  g ( x) dx  f  g , f  g  f  g
2 2
a

It follows that the least squares approximation problem is reduced to one of finding a
suitable polynomial g to minimize the norm f  g .
Now let W  Pk  a, b be the collection of all polynomials g :  a, b   with real
coefficients and of degree at most k . It is easy to show that W  Pk  a, b is a subspace of
V . In view of theorem 2.3.3, we conclude that
g  PW ( f ) (projection of f on to W )
gives the best least squares approximation among polynomials in W .
Example 2.4.4 Find the least squares approximation by a polynomial of degree at most 1 for
the function f ( x)  e x in the interval  0,1 .
Solution. W  P1 0,1 , with basis 1, x . We now apply the Gram-Schmidt
 1
orthogonalization process to this basis to obtain an orthogonal basis 1, x   of W , and
 2
1
ex , x 
e x ,1 2  1
take g ( x)  1  x 
1 1  2
x
2
It follows that g ( x)  (18  6e) x  (4e  10) best approximation of f ( x)  e x .

EXERCISES 2.5____________________________________________
1. An inconsistent system of linear equation AX  B is given.
Find the least squares solution.
x1  x2  x4  1
x y 3 x  2y  z  1
x2  x4  2
a) x  2 y  5 b) x  y  2 z  3 c)
x1  x2  x3  2 x4  1
3x  y  4 2x  y  z  1
 x2  x3 0
2. Find the equation of the least square line for the given data.
a) (1, 2),(2, 4),(3,7)
b) (1,5),(2,6),(3,8),(4,10)
c) (1, 4),(4, 24),(5,30),(8,32)
3. Use the method of least square to find the polynomial of degree at most n that best fits
the given data.
a) n  2 , with data (0, 2),(1, 2),(2, 4),(3,8) .
b) n  2 , with data (0,1),(1, 2),(2,3),(3,5) .
c) n  3 , with data (2, 5),(1, 1),(0, 1),(1,1),(2,10) .

BDU,DEPARTEMENT OF MATHEMATIC | 69
 Math 2042 LINEAR ALGEBRA II

4. Let A be an m  n matrix and let A( F n ) denotes the set of vectors in F m which are of
the form AX for some X  F n . Then show that A( F n ) is a subspace of F m .
5. Let V  C  a, b the set of all continuous real valued functions on the closed interval

 a, b , with inner product f , g  a f ( x) g ( x)dx. Let W  Pk  a, b be the set of all

b

polynomials g :  a, b   with real coefficients and of degree at most k .

Show that W is a subspace of V .
6. a) Find the orthogonal projection of f (t )  t on P2 using the inner product
1
f , g   f (t )g (t )dt .
1

b) Let W  P2 0,1 and V  P the set of polynomials with inner product

1
f , g   f (t )g (t )dt . Find the projection of f (t )  t 3 on W .
0

7. a) In the standard inner product space of  3 and subspace W  ( x, y, z ) : x  y find

the point in W closest to the point (3, 2,7) .

b) In the inner product space C  1,1 with f , g   f (t )g (t )dt find the closest
1

1

polynomial of degree 2 to f ( x)  x . 4

c) Find a quadratic polynomial approximation of the following data.

xi -1 0 1 2 3
yi 0.5 0.1 0.5 1 2

with inner product p, q  p(1)q(1)  p(0)q(0)  p(1)q(1)  p(2)q(2)  p(3)q3) .

8. Show that for any set of data ( x1 , y1 ),( x2 , y2 ),,( xn , yn ) , where the xi ' s are distinct
and n  3 , the vectors v1  (1,1,,1)t , v2  ( x1 , x2 ,, xn )t and v3  ( x12 , x22 ,, xn2 )t form a
linearly independent subset of  n .
______________________________________________________________________

70 | BDU,DEPARTEMENT OF MATHEMATIC