Inner Product Spaces: Ioana Luca
Inner Product Spaces: Ioana Luca
Ioana Luca
UPB - Dept. Metode si Modele Matematice
2011 2012
I. Luca (UPB)
2011 2012
1 / 33
Content
2 Orthonormal bases
I. Luca (UPB)
2011 2012
2 / 33
a) b) c) d)
for all u, v , w V and for all . A vector space equipped with an inner product is an inner product space; a nite dimensional inner product space is a Euclidean vector space.
I. Luca (UPB)
2011 2012
3 / 33
I. Luca (UPB)
2011 2012
4 / 33
R Cn
x (x1 , . . . , xn ), y (y1 , . . . , yn ) x y x1 y1 + . . . + xn yn
R C
m n
mi=1 j =1
Aij Bij
Mm,n ( )
R Rmn
i= 1
ui vi ; Mm,1 ( )
n
R Rm
Mm,n ( ) A B tr AB =
i,j =1
Aij Bij
f (x)g (x) dx
b a
f (x)g (x) dx
2011 2012 5 / 33
R n [X ]
P p0 + p1 X + . . . + pn X n Q q0 + q1 X + . . . + qn X n P Q p 0 q0 + p 1 q1 + . . . + p n qn
R[X ] P Q
1 1
v u
I. Luca (UPB) Inner Product Spaces
[0, ]
2011 2012
6 / 33
( ) (xn )nN
xn
C,
n=0
xn 2 <
n=0
x (xn )nN ,
y (yn )nN
xy
x n yn
Proposition (Properties of an inner product) 1) u (v ) = (u v ), 0 v = 0 2) u v = 0 for all v V u = 0 3) if {e1 , . . . , en } is a basis of V, then u ei = 0 for i = 1, . . . , n u=0
I. Luca (UPB)
2011 2012
7 / 33
Rn Cn
x = x =
2 x2 1 + . . . + xn
x1 2 + . . . + xn 2
Inner Product Spaces 2011 2012 8 / 33
I. Luca (UPB)
R C
A = tr AAT = A = tr AA =
T
A2 ij
2
Aij
f (x)2 dx
f (x) 2 dx
v is a unit vector
Inner Product Spaces 2011 2012 9 / 33
I. Luca (UPB)
equality holds if and only if u and v are linearly dependent vectors. Exercise Prove this proposition by expanding Examples u 2 v (v u)u
2
0.
Rn
C ([a, b])
n i=1
xi yi
b a 2
n i=1
x2 i
n i=1 2
2 yi
f (x)g (x) dx
n i=1
n i=1
b a
f (x)2 dx
b a
g (x)2 dx
Cn
n i= 1
xi yi
xi 2
yi 2
2011 2012 10 / 33
I. Luca (UPB)
with v1 v1 = 0 , . . . , vn vn = 0.
Exercise Note that AT A is the Gram matrix corresponding to the columns of A Mm,n ( ), and deduce that, if the columns of A are linearly independent, then AT A is invertible.
I. Luca (UPB)
2011 2012
12 / 33
2. Orthonormal bases
Denition Let S be a non-empty subset of an inner product space V. 1) S is orthogonal if u v = 0 for all u, v S, u = v . 2) S is orthonormal if it is orthogonal and v = 1, for any v S. 3) An orthonormal basis of V is a basis which is an orthonormal set. Examples The trigonometric system 1, sin x, cos x, sin 2x, cos 2x, . . . , sin nx, cos nx, . . . is an orthogonal set in C ([0, 2 ]), and 1 sin x cos x sin nx cos nx , , ,..., , ,... 2 is orthonormal.
I. Luca (UPB) Inner Product Spaces 2011 2012 13 / 33
2. Orthonormal bases
The canonical basis of The basis {i, j , k} of V3 is orthonormal.
Rn is an orthonormal basis. R
If {e1 , . . . , en , . . .} is an orthonormal set, then ei ej = ij . Exercise Show that Q Mn ( ) is orthogonal, i.e., Q1 = QT , (Q Mn ( ) is unitary, i.e., Q are orthonormal. Proposition 1) If S is orthogonal and 0 S, S is linearly independent. 2) Any orthonormal set is linearly independent. 3) If {e1 , . . . , en } is an orthonormal basis of V R (V C ) and fj = n j = 1, . . . , n, then i=1 Cij ei , {f1 , . . . , fn } is an orthonormal basis the matrix (Cij ) is orthogonal (unitary)
I. Luca (UPB) Inner Product Spaces 2011 2012 14 / 33
2. Orthonormal bases
Proposition If {e1 , . . . , en } is an orthonormal basis of V representations u=
n i=1
ui ei ,
v=
i=1
the components of v with respect to this basis, the inner product u v , and the norm of v are given by vi = v ei , respectively. Exercise Restate and prove the preceding proposition for the case V C.
I. Luca (UPB) Inner Product Spaces 2011 2012 15 / 33
uv =
n i=1
ui vi ,
n i= 1
2 vi ,
2. Orthonormal bases
Example The components of v = (1, 2, 1) orthonormal basis e1 = are as follows: v1 = v e 1 = 1 3, v2 = v e2 = 7 3, v3 = v e3 = 2 3.
1 2 2 3, 3, 3
2 1 e2 = 3 , 2 3, 3 ,
Proposition (Gram-Schmidt orthonormalization process) If {f1 , . . . , fn , . . .} V is a linearly independent set, there exists an orthonormal set {e1 , . . . , en , . . .} V such that Sp [ e1 , . . . , en ] = Sp [ f1 , . . . , fn ] , for each n = 1, 2, . . . . It is deduced as follows:
I. Luca (UPB) Inner Product Spaces 2011 2012 16 / 33
2. Orthonormal bases
Step 1 One determines the orthogonal set {d1 , d2 , . . . , dn , . . .} of non-zero vectors: d1 f1 d2 f2 + d1 , Sp [ d1 , . . . , dn ] = Sp [ f1 , . . . , fn ] Step 2 One normalizes the vectors d1 , d2 , . . . , dn , . . .: e1 1 d1 , d1 e2 1 d2 , d2 ..., en 1 dn , dn ... where satises d2 d1 = 0 d3 d1 = 0, d3 d2 = 0 d3 f3 + 1 d1 + 2 d2 , where 1 , 2 satisfy
I. Luca (UPB)
2011 2012
17 / 33
2. Orthonormal bases
Example In
f1 (1, 2, 2),
we nd an orthonormal basis by using the Gram-Schmidt algorithm. Step 1 d1 f1 , d2 f2 + d1 , d3 f3 + 1 d1 + 2 d2 , where , 1 , 2 are such that d2 d1 = 0, d3 d1 = 0, d3 d2 = 0 f2 d1 1 f3 d1 1 f3 d2 = = , 1 = = , 2 = = 1 . d1 d1 3 d1 d1 3 d2 d2 Therefore,
2 2 1 d2 = f2 + 1 3 d1 = 3 , 3 , 3 , 1 d3 f3 + 3 d1 d2 = (6, 3, 6) .
2 1 2 3, 3, 3
18 / 33
2011 2012
2. Orthonormal bases
Proposition (The QR decomposition) Suppose the columns of A Mm, n ( ) are linearly independent. Then A = QR, where the columns of Q Mm, n ( ) are orthonormal, and R Mn ( ) is an invertible upper triangular matrix.
Proof Mm,1
Rm the columns of A:
v2 (A12 , A22 , . . . , Am2 ) , . . . , vn (A1n , A2n , . . . , Amn )
Since v1 , . . . , vn are linearly independent, by using the Gram-Schmidt algorithm one obtains an orthonormal set {e1 , . . . , en } such that Sp [e1 , . . . , ek ] = Sp [v1 , . . . , vk ] , k = 1, . . . , n . In particular this shows that ek+1 , . . . , en are orthogonal to v1 , . . . , vk . Thus, v1 , . . . , vn Sp [e1 , . . . , en ] have the following representations with respect to the orthonormal basis {e1 , . . . , en } of Sp [e1 , . . . , en ]:
I. Luca (UPB) Inner Product Spaces 2011 2012 19 / 33
2. Orthonormal bases
v1 = v2 = vn =
n i=1 n
(v1 ei )ei = (v1 e1 )e1 , (v2 ei )ei = (v2 e1 )e1 + (v2 e2 )e2 , (vn ei )ei = (vn e1 )e1 + . . . + (vn en )en .
i=1 n
i=1
Q=
Moreover,
I. Luca (UPB) Inner Product Spaces 2011 2012 20 / 33
2. Orthonormal bases
v1 e1 = d1
d1 d2 = d1 = 0 , v2 e2 = (d2 d1 ) = d2 = 0 , . . . d1 d2
proving that the matrix R is invertible. Example 1 2 A 0 1 1 4 The columns v1 (1, 0, 1) , v2 (2, 1, 4) are linearly independent, and hence the factorization A = QR does exist. We start from {v1 , v2 } and use the Gram-Schmidt algorithm: d1 v1 ,
I. Luca (UPB)
d2 v2 + d1 ,
Inner Product Spaces
d2 d1 = 0
2011 2012 21 / 33
2. Orthonormal bases
This gives = implying e1 = 1 d1 = d1
1 2
v2 d1 = 3 d1 d1
d1 = (1, 0, 1) ,
d2 = (1, 1, 1) ,
(1, 0, 1) ,
e2 =
1 d2 = d2
1 3
(1, 1, 1) .
R=
v1 e1 v2 e1 0 v2 e2
2 3 2 = . 3 0
I. Luca (UPB)
2011 2012
22 / 33
R2
U
(1, 2) 90 (2, 1)
y = 2x } =
= { x(1, 2) x
I. Luca (UPB)
R } = Sp [ (1, 2) ]
Inner Product Spaces
x
2011 2012 23 / 33
I. Luca (UPB)
2011 2012
24 / 33
The Gram determinant corresponding to {e1 , . . . , ek } is non-zero unique solution u1 , . . . , uk . If {e1 , . . . , ek } is an orthonormal basis of U, (1) gives u=
I. Luca (UPB)
i=1
(v ei )ei
(2 )
2011 2012 25 / 33
R2:
U
v u
90
If e.g. U [(2, 1)] and v = (5, 6), an orthonormal basis of U is {e}, = Sp e (2 5, 1 5). Thus, using (2), the orthogonal projection u of v onto U emerges as u = (v e)e = (32 5, 16 5). Since u = v u, the decomposition of v along U and U is v = u + u = (32 5, 16 5) + (7 5, 14 5).
I. Luca (UPB) Inner Product Spaces 2011 2012 26 / 33
min v u = v u ,
(3)
and this minimum is attained only by the orthogonal projection of v onto U. Remark v u = d(v , u) Reformulation of (3): among all the elements of U, the closest one to v V is the orthogonal projection u of v on U. Moreover, u is the unique element of U closest to v ; it is also called the element in U of best approximation for v .
I. Luca (UPB) Inner Product Spaces 2011 2012 27 / 33
2 0
f (x)g (x) dx
(4 )
f (x) dx ,
c2k1 =
2 0
f (x) sin kx dx ,
c0 , . . . , c2n
2 1 f (x) cos kx dx , k = 1, . . . , n ; 0 are the Fourier coecients of f . In C ([a, b]) the norm
c2k =
f g
b a
(f (x) g (x))2 dx
is called the quadratic error. So, we have obtained that, among all trigonometric polynomials of at most degree n, P (x) given by (4) is the trigonometric polynomial of which the quadratic error with respect to f is minimal.
1
(ex (a + bx))2 dx
Inner Product Spaces 2011 2012 29 / 33
Usually, n > k , and the overdetermined system y = c1 x11 + . . . + ck x1k 1 (5) y = c x + . . . + c x 1 n1 k nk n is inconsistent (incompatible). Thus, the goal is to nd c1 , . . . , ck
I. Luca (UPB) Inner Product Spaces 2011 2012 30 / 33
(c1 , . . . , ck ) is called a pseudo-solution (in the sense of the least squares) of the linear system (5). Dening the vectors y , e1 , . . . , ek of n by
y (y1 , . . . , yn ) ,
e1 (x11 , . . . , xn1 ),
...,
ek (x1k , . . . , xnk ),
the least squares requirement can be restated as: nd c1 , . . . , ck which minimize y (c1 e1 + . . . + ck ek ) . Equivalently: nd u Sp [e1 , . . . , ek ] U, such that y u = min y u .
u U
Clearly, u is the orthogonal projection of y onto U. If e1 , . . . , ek are linearly independent (which is most likely to happen, due to
I. Luca (UPB) Inner Product Spaces 2011 2012 31 / 33
x i yi
i=1
x2 i
y
y=
cx
x
I. Luca (UPB) Inner Product Spaces 2011 2012 32 / 33
We have shown that, if the columns of A are linearly independent, there is a unique pseudo-solution of (5 ), which is the (unique) solution of (6 ). The system (6 ) is called the normal system associated to (5 ). Exercise Find the pseudo-solution of the linear system
I. Luca (UPB)