Inner Product Spaces Definition : Let V be a vector space over real field R. Suppose to each pair of vectors u,v €V, there is assigned a real number, denoted by (u,v). This function is called a inner product on V if it satisfies the following axioms : I. Linearity Property : (a+ uy + b+tz,v) = a+ (uy,v) + b+ (uz, v). Ip. Symmetric Property : (u,v) = (v,u). Ig. Positive Defi The vector space V with an inner product is called a real inner product space. te Property + (u,u) > 0; and (u,u) = 0 if and only if w= 0. Note. Axiom I, states that an inner product function is linear in the first position. Using I, and the symmetry axiom Iz, we obtain (u,c- v1 +d V2) = (04 +d v2,U) {By Axiom Ip} (u,c+vy +d -v2) = c+ (vy,u) +d > (v2,u) {By Axiom 11} (u,c- vy +d V2) = c+ (u,v4) + d= (u,v2) {By Axiom Ip} That is,an inner product is also linear in its second position.Combining these axioms I and Iz, and using matematical induction yields the following general formula : & ays 2 by -) Dye aj - by - (ui, v4) That is,an inner product of linear combinations of vectors is equal to a linear combination of the inner products of the vectors. Example 1. Let V be a real inner product space and U;,Uz,V4,V2,03 EV. Then, expand (3u — 4ttz, 2v, — 5v2 + 6v3) as a linear combination of the inner products of the vectors. Solution.Norm of aVector By the third axiom I3 of an inner product, (u,u) is non-negative for any vector u. Thus its, positive square root exists. We denote the norm of a vector was [ul] = V(u, u) This non-negative number is called the norm or length of vector u. The relation |lt||? = (uu) will be used frequently. Note. If ||u|| = 1 or equivalently (u,u) = 1, then wis called a unit vector and is said to be normalized. Every non-zero vector v € V can be multiplied by the reciprocal of its length to obtain the unit vector i.e., 1 ail Ill which is a positive multiple of v. This process is called normalizing v.Example 1. Let V be a real inner product space and u,v € V. Then, expand ||2u — 3v||? as a linear combination of the inner products of the vectors. Solution.Example of Inner Product Spaces 1. Euclidean n- Space R" Consider a vector space R” over field R. The dot product of two vectors w= (uy, U2, ..,Un) and v = (V4, V2, ..., Yn), Where u,v € IR” is defined as we vl = (ty ty, MY v% Un) | P| = Uy y+ Uy V2 +--+ Uy Mp Un, This function defines an inner product in Ri (u,v) = We VT = (Ug, Uy, one Un) | P| Un Vy Fg Vy tet Un Une Yn, For example, The inner product of two vectors u = (1,3,—4,2) and v = (4,—2,2, 1) is given by 4 torenwraganan [fs] ateescnecnrsees 1(u,v) =4-6-84+2=-8. 1 Also, Ilu|| = (a, u) = |(,3,-4,2)- :| =V149+4 1644 = 30 2 4 and, wll = V@,0) = |(4,-2,2,1)- | = VI6+444+1=V25 =5. Hh2. Function Space C[a, b] and Polynomial Space P(t) The notation Ca, b] is used to denote the vector space of all real valued continuous functions on closed interval [a,b] i.e.,a < t 0; and |lvll = 0 if and only if v = 0. No. lik vil = [kl -llvll, Wk ER. Ng. |lu+ vl < [lull + [lull The property N3 is called the triangle inequality.Angle Between Vectors For any non-zero vectors wand v in an inner product space V, the angle between wand vis defined to be the angle @ such that 0 < 9 < mand (u,v) [ell - ell” By the Cauchy-Schwartz inequality, -1 < cos @ < 1,and so the angle exists and is unique. cos =Example 1. Let R° be a real inner product space and u = (2,3,5) and v = (1,—4,3) € R®. Find the angle between vectors uand v. Solution.Orthogonality Let V be an inner product space. The vector u,v € V are said to be orthogonal or wis said to be orthogonal to v if (u,v) = 0. The relation is clearly symmetric i.e., if u is orthogonal to v, then v is also orthogonal to wi.e., (v,u) = 0. We note that 0 € V is orthogonal to every v € V, since (0,v) = (0-v,v) = 0-(v,v) =0. Conversely, if wis orthogonal to every v € V, then (u,u) = 0 and hence u = 0 by axiom I of inner product space. Observe that : wand v are orthogonal if and only if cos = 0,where 9 is the angle between vectors wand v, Also, this is true if and only if wand v are perpendicular vectors i.e.,0 = 1/2.Example 1. Consider the vectors u = (1,1,1),v = (1,2,—3) and w = (1,—4,3) in R°. Determine whether or not u,v and w are orthogonal to each other. Solution.Example 2. Find a non-zero vector w that is orthogonal to u = (1,2, 1) and v = (2,5,4) in R°. Solution.Orthogonal and Orthonormal Set Consider a set S = {81,82,...,8-} of non-zero vectors in an inner product space V. 1. Sis called an orthogonal set if each pair of vectors in S are orthogonal i.e., (uj,uj)=0, fori # j. 2. Sis called an orthonormal set if each pair of vectors in S are orthogonal i.e., wwom={ fore): Note. We can tranform any orthogonal set S into an orthonormal set by simply normalizing each vector in S i.e.,multiplying each vector in S by the reciprocal of norm of that vector. Theorem 1. Suppose S = {81,82,..,S,} is an orthogonal set of non-zero vectors. Then,S is linearly independent. Theorem 2. Suppose S = {s1,82, Isi + 82 +--+ Syll? = IIsall? + [lsall? +--+ Ilsrll?. s,} is an orthogonal set of vectors. Then,an Example 1. Consider the vector space C[0, 271] with inner product (f, g) = f FO + g(t) de. 0 Show that the set {cosx ,cos 2x, cos 3x,...,cos nx, ...} € C[0, 271] is orthogonal on interval [0,27]. Solution.Orthogonal and Orthonormal Basis, and Linear Combinations, Fourier Coef ficients Definition : Let S = {s1,82,...,5,} be a subset of n-dimensional inner product space V over a real field R, then S is an orthogonal basis for an inner product space V over areal field Rif its vectors are mutually orthogonal. If the vectors of an orthogonal basis are normalized, the resulting basis is an orthonormal basis. Theorem 1. Let {84,82 R. Then, for any v €V, . Sn} be an orthogonal basis of an inner product space V over areal field (wisi), (ose) (5) uss) * * asa) 27S sas) Wrhere) == lealled Kourter Coef fictentaf ow hatte =z a si) Theorem 2. Suppose W;,W2,...,W, form an orthogonal set of nonzero vectors in an inner product space V over areal field R. Let v be any vector in V. Define (v4) (v,W2) (v,w,) et co wa)" * Gwa,wa) 2 * Gwe) | Then v' is orthogonal to Wy, W2, ..., Wr.Example 1. Let S consist of the following three vectors in R° : 81 =(12,1), s2=(2,1,-4), $3 = (3,-2,1). (® Show that S = {s4, 82,53} is orthogonal and a basis of R°. (ii) Express v = (3,4, 1) in 3 as linear combination of s1,82,53- Solution.Gram- Schmidt Orthogonalization Process Suppose {V;, V2, ...,Un} be a basis of an inner product space V over a real field R. Then, one can use this basis to construct an orthogonal basis {8}, 52, ...,S,} of an inner product space V over areal field Ras follows + Set S=% =i fo, v2), Soka fo2— Gays i (551) (WsuS2) saat foo Gey easy] A wy, — Wm Sad Pw 82) Pw Sunt) 5 ites WA (Sans) ium (s2iS2) ie CE) os where kz, ks, kn € Ris scalars The above construction is known as Gram- Schmidt Orthogonalization Process. Theorem 1. Let S = {S,8z,...,,} is an orthogonal basis for a subspace W of an inner product space V over areal field R. Then one may extend S to an orthogonal basis for V i.e.,one may find vectors S741,5p42, Sn Such that (84,82, ..,Sp,Sr41,Sr42,~-,Sn} iS an orthogonal basis for V.Example 1. Consider the subspace U of an inner product space R* over a real field R spanned by the vectors + my =(41,1,1), u2=(1,1,2,4), ug = (1,2,-4,-3). Apply the Gram-Schmidt orthogonalization process to find an orthogonal basis and then an orthonormal basis of U. Solution.1 Example 2. Consider the vector space P(t) with inner product (f, g) = fro g(t) dt. 3 Apply the Gram- Schmidt orthogonalization process to the set {fo(t) = 1, f(t) = t, fo(t) = t2} to obtain an orthogonal set {So(t), $1(t), 82(t)} with integer coef ficient. Solution.