mywbut.

com

Eigenvalues and Eigenvectors: An Introduction

The eigenvalue problem is a problem of considerable theoretical interest and wide-ranging application. For example, this problem is crucial in solving systems of differential equations, analyzing population growth models, and calculating powers of matrices (in order to define the exponential matrix). Other areas such as physics, sociology, biology, economics and statistics have focused considerable attention on "eigenvalues" and "eigenvectors"-their applications and their computations. Before we give the formal definition, let us introduce these concepts on an example. Example. Consider the matrix

Consider the three column matrices

We have

In other words, we have

Next consider the matrix P for which the columns are C1, C2, and C3, i.e.,

mywbut.com

We have det(P) = 84. So this matrix is invertible. Easy calculations give

Next we evaluate the matrix P-1AP. We leave the details to the reader to check that we have

In other words, we have

Using the matrix multiplication, we obtain

which implies that A is similar to a diagonal matrix. In particular, we have

for of A.

. Note that it is almost impossible to find A75 directly from the original form

mywbut.com

This example is so rich of conclusions that many questions impose themselves in a natural way. For example, given a square matrix A, how do we find column matrices which have similar behaviors as the above ones? In other words, how do we find these column matrices which will help find the invertible matrix P such that P-1AP is a diagonal matrix? From now on, we will call column matrices vectors. So the above column matrices C1, C2, and C3 are now vectors. We have the following definition. Definition. Let A be a square matrix. A non-zero vector C is called an eigenvector of A if and only if there exists a number (real or complex) such that

If such a number exists, it is called an eigenvalue of A. The vector C is called eigenvector associated to the eigenvalue . Remark. The eigenvector C must be non-zero since we have

for any number

Example. Consider the matrix

We have seen that

where

mywbut.com

So C1 is an eigenvector of A associated to the eigenvalue 0. C2 is an eigenvector of A associated to the eigenvalue -4 while C3 is an eigenvector of A associated to the eigenvalue 3.

Computation of Eigen values

For a square matrix A of order n, the number zero vector C such that is an eigenvalue if and only if there exists a non-

Using the matrix multiplication properties, we obtain

This is a linear system for which the matrix coefficient is . We also know that this system has one solution if and only if the matrix coefficient is invertible, i.e. . Since the zero-vector is a solution and C is not the zero vector, then we must have

Example. Consider the matrix

The equation

translates into

mywbut.com

which is equivalent to the quadratic equation

Solving this equation leads to

In other words, the matrix A has only two eigenvalues. In general, for a square matrix A of order n, the equation

will give the eigenvalues of A. This equation is called the characteristic equation or characteristic polynomial of A. It is a polynomial function in of degree n. So we know that this equation will not have more than n roots or solutions. So a square matrix A of order n will not have more than n eigenvalues. Example. Consider the diagonal matrix

Its characteristic polynomial is

So the eigenvalues of D are a, b, c, and d, i.e. the entries on the diagonal.

mywbut.com

This result is valid for any diagonal matrix of any size. So depending on the values you have on the diagonal, you may have one eigenvalue, two eigenvalues, or more. Anything is possible. Remark. It is quite amazing to see that any square matrix A has the same eigenvalues as its transpose AT because

For any square matrix of order 2, A, where

the characteristic polynomial is given by the equation

The number (a+d) is called the trace of A (denoted tr(A)), and clearly the number (ad-bc) is the determinant of A. So the characteristic polynomial of A can be rewritten as

Let us evaluate the matrix B = A2 - tr(A) A + det(A) I2. We have

We leave the details to the reader to check that

mywbut.com

In other word, we have

This equation is known as the Cayley-Hamilton theorem. It is true for any square matrix A of any order, i.e.

where

is the characteristic polynomial of A.

We have some properties of the eigenvalues of a matrix. Theorem. Let A be a square matrix of order n. If 1. is an eigenvalue of Am, for 2. is an eigenvalue of A, then:

If A is invertible, then 3.

is an eigenvalue of A-1. is an eigenvalue of A.

A is not invertible if and only if 4. If 5. is any number, then

is an eigenvalue of

If A and B are similar, then they have the same characteristic polynomial (which implies they also have the same eigenvalues).

mywbut.com

Computation of Eigenvectors
Let A be a square matrix of order n and associated to . We must have one of its eigenvalues. Let X be an eigenvector of A

This is a linear system for which the matrix coefficient is . Since the zero-vector is a solution, the system is consistent. In fact, we will in a different page that the structure of the solution set of this system is very rich. In this page, we will basically discuss how to find the solutions. Remark. It is quite easy to notice that if X is a vector which satisfies , then the vector Y = c X (for any arbitrary number c) satisfies the same equation, i.e. . In other words, if we know that X is an eigenvector, then cX is also an eigenvector associated to the same eigenvalue. Let us start with an example. Example. Consider the matrix

First we look for the eigenvalues of A. These are given by the characteristic equation , i.e.

If we develop this determinant using the third column, we obtain

mywbut.com

Using easy algebraic manipulations, we get

which implies that the eigenvalues of A are 0, -4, and 3. Next we look for the eigenvectors. 1. Case : The associated eigenvectors are given by the linear system

which may be rewritten by

Many ways may be used to solve this system. The third equation is identical to the first. Since, from the second equations, we have y = 6x, the first equation reduces to 13x + z = 0. So this system is equivalent to

So the unknown vector X is given by

Therefore, any eigenvector X of A associated to the eigenvalue 0 is given by

mywbut.com

where c is an arbitrary number. 2. Case : The associated eigenvectors are given by the linear system

which may be rewritten by

In this case, we will use elementary operations to solve it. First we consider the augmented matrix , i.e.

Then we use elementary row operations to reduce it to a upper-triangular form. First we interchange the first row with the first one to get

Next, we use the first row to eliminate the 5 and 6 on the first column. We obtain

If we cancel the 8 and 9 from the second and third row, we obtain

10

mywbut.com

Finally, we subtract the second row from the third to get

Next, we set z = c. From the second row, we get y = 2z = 2c. The first row will imply x = -2y+3z = -c. Hence

Therefore, any eigenvector X of A associated to the eigenvalue -4 is given by

where c is an arbitrary number. 2. Case : The details for this case will be left to the reader. Using similar ideas as the one described above, one may easily show that any eigenvector X of A associated to the eigenvalue 3 is given by

where c is an arbitrary number. Remark. In general, the eigenvalues of a matrix are not all distinct from each other (see the page on the eigenvalues for more details). In the next two examples, we discuss this problem. Example. Consider the matrix

11

mywbut.com

The characteristic equation of A is given by

Hence the eigenvalues of A are -1 and 8. For the eigenvalue 8, it is easy to show that any eigenvector X is given by

where c is an arbitrary number. Let us focus on the eigenvalue -1. The associated eigenvectors are given by the linear system

which may be rewritten by

Clearly, the third equation is identical to the first one which is also a multiple of the second equation. In other words, this system is equivalent to the system reduced to one equation 2x+y + 2z= 0.

To solve it, we need to fix two of the unknowns and deduce the third one. For example, if we set and , we obtain associated to the eigenvalue -1 is given by . Therefore, any eigenvector X of A

12

mywbut.com

In other words, any eigenvector X of A associated to the eigenvalue -1 is a linear combination of the two eigenvectors

Example. Consider the matrix

The characteristic equation is given by

Hence the matrix A has one eigenvalue, i.e. -3. Let us find the associated eigenvectors. These are given by the linear system

which may be rewritten by

This system is equivalent to the one equation-system x - y = 0.

So if we set x = c, then any eigenvector X of A associated to the eigenvalue -3 is given by

13

mywbut.com

Let us summarize what we did in the above examples. Summary: Let A be a square matrix. Assume is an eigenvalue of A. In order to find the associated eigenvectors, we do the following steps: 1. Write down the associated linear system

2. Solve the system. 3. Rewrite the unknown vector X as a linear combination of known vectors. The above examples assume that the eigenvalue is real number. So one may wonder whether any eigenvalue is always real. In general, this is not the case except for symmetric matrices. The proof of this is very complicated. For square matrices of order 2, the proof is quite easy. Let us give it here for the sake of being little complete. Consider the symmetric square matrix

Its characteristic equation is given by

This is a quadratic equation. The nature of its roots (which are the eigenvalues of A) depends on the sign of the discriminant

Using algebraic manipulations, we get

14

mywbut.com

Therefore,

is a positive number which implies that the eigenvalues of A are real numbers.

Remark. Note that the matrix A will have one eigenvalue, i.e. one double root, if and only if . But this is possible only if a=c and b=0. In other words, we have A = a I2.

The Case of Complex Eigenvalues

First let us convince ourselves that there exist matrices with complex eigenvalues. Example. Consider the matrix

The characteristic equation is given by

This quadratic equation has complex roots given by

Therefore the matrix A has only complex eigenvalues.

The trick is to treat the complex eigenvalue as a real one. Meaning we deal with it as a number and do the normal calculations for the eigenvectors. Let us see how it works on the above example.

15

mywbut.com

We will do the calculations for linear system

. The associated eigenvectors are given by the

A X = (1+2i) X

which may be rewritten as

In fact the two equations are identical since (2+2i)(2-2i) = 8. So the system reduces to one equation (1-i)x - y = 0.

Set x=c, then y = (1-i)c. Therefore, we have

where c is an arbitrary number. Remark. It is clear that one should expect to have complex entries in the eigenvectors. We have seen that (1-2i) is also an eigenvalue of the above matrix. Since the entries of the matrix A are real, then one may easily show that if is a complex eigenvalue, then its conjugate is also an eigenvalue. Moreover, if X is an eigenvector of A associated to , then the vector , obtained from X by taking the complex-conjugate of the entries of X, is an eigenvector associated to . So the eigenvectors of the above matrix A associated to the eigenvalue (1-2i) are given by

where c is an arbitrary number.

16

mywbut.com

Let us summarize what we did in the above example. Summary: Let A be a square matrix. Assume is a complex eigenvalue of A. In order to find the associated eigenvectors, we do the following steps: 1. Write down the associated linear system

2. Solve the system. The entries of X will be complex numbers. 3. Rewrite the unknown vector X as a linear combination of known vectors with complex entries. 4. If A has real entries, then the conjugate is also an eigenvalue. The associated eigenvectors are given by the same equation found in 3, except that we should take the conjugate of the entries of the vectors involved in the linear combination.

In general, it is normal to expect that a square matrix with real entries may still have complex eigenvalues. One may wonder if there exists a class of matrices with only real eigenvalues. This is the case for symmetric matrices. The proof is very technical and will be discussed in another page. But for square matrices of order 2, the proof is quite easy. Let us give it here for the sake of being little complete. Consider the symmetric square matrix

Its characteristic equation is given by

17

mywbut.com

This is a quadratic equation. The nature of its roots (which are the eigenvalues of A) depends on the sign of the discriminant

Using algebraic manipulations, we get

Therefore,

is a positive number which implies that the eigenvalues of A are real numbers.

Remark. Note that the matrix A will have one eigenvalue, i.e. one double root, if and only if . But this is possible only if a=c and b=0. In other words, we have A = a I2.

Diagonalization
When we introduced eigenvalues and eigenvectors, we wondered when a square matrix is similarly equivalent to a diagonal matrix? In other words, given a square matrix A, does a diagonal matrix D exist such that ? (i.e. there exists an invertible matrix P such that A = P-1DP) In general, some matrices are not similar to diagonal matrices. For example, consider the matrix

Assume there exists a diagonal matrix D such that A = P-1DP. Then we have

i.e is similar to . So they have the same characteristic equation. Hence A and D have the same eigenvalues. Since the eigenvalues of D of the numbers on the diagonal, and the only eigenvalue of A is 2, then we must have

18

mywbut.com

In this case, we must have A = P-1DP = 2 I2, which is not the case. Therefore, A is not similar to a diagonal matrix. Definition. A matrix is diagonalizable if it is similar to a diagonal matrix. Remark. In a previous page, we have seen that the matrix

has three different eigenvalues. We also showed that A is diagonalizable. In fact, there is a general result along these lines. Theorem. Let A be a square matrix of order n. Assume that A has n distinct eigenvalues. Then A is diagonalizable. Moreover, if P is the matrix with the columns C1, C2, ..., and Cn the n eigenvectors of A, then the matrix P-1AP is a diagonal matrix. In other words, the matrix A is diagonalizable. Problem: What happened to square matrices of order n with less than n eigenvalues? We have a partial answer to this problem. Theorem. Let A be a square matrix of order n. In order to find out whether A is diagonalizable, we do the following steps: 1. Write down the characteristic polynomial

2. Factorize . In this step, we should be able to get

19

mywbut.com

where the

, may be real or complex. For every i, the powers ni is .

called the (algebraic) multiplicity of the eigenvalue 3.

For every eigenvalue, find the associated eigenvectors. For example, for the eigenvalue , the eigenvectors are given by the linear system

Then solve it. We should find the unknown vector X as a linear combination of vectors, i.e.

where

are arbitrary numbers. The integer mi is called the .

geometric multiplicity of 4.

If for every eigenvalue the algebraic multiplicity is equal to the geometric multiplicity, then we have

which implies that if we put the eigenvectors Cj, we obtained in 3. for all the eigenvalues, we get exactly n vectors. Set P to be the square matrix of order n for which the column vectors are the eigenvectors Cj. Then P is invertible and

is a diagonal matrix with diagonal entries equal to the eigenvalues of A. The position of the vectors Cj in P is identical to the position of the associated eigenvalue on the diagonal of D. This identity implies that A is similar to D. Therefore, A is diagonalizable.

Remark. If the algebraic multiplicity ni of the eigenvalue we have mi = 1. In other words, ni = mi. 5.

is equal to 1, then obviously

If for some eigenvalue the algebraic multiplicity is not equal to the geometric multiplicity, then A is not diagonalizable.

20

mywbut.com

Example. Consider the matrix

In order to find out whether A is diagonalizable, lt us follow the steps described above. 1. The polynomial characteristic of A is

So -1 is an eigenvalue with multiplicity 2 and -2 with multiplicity 1. 2. In order to find out whether A is diagonalizable, we only concentrate ur attention on the eigenvalue -1. Indeed, the eigenvectors associated to -1, are given by the system

This system reduces to the equation -y + z = 0. Set

and

, then we have

So the geometric multiplicity of -1 is 2 the same as its algebraic multiplicity. Therefore, the matrix A is diagonalizable. In order to find the matrix P we need to find an eigenvector associated to -2. The associated system is

which reduces to the system

21

mywbut.com

Set

, then we have

Set

Then

But if we set

then

We have seen that if A and B are similar, then An can be expressed easily in terms of Bn. Indeed, if we have A = P-1BP, then we have An = P-1BnP. In particular, if D is a diagonal matrix, Dn is easy to evaluate. This is one application of the diagonalization. In fact, the above procedure may be used to find the square root and cubic root of a matrix. Indeed, consider the matrix above

22

mywbut.com

Set

then

Hence A = P D P-1. Set

Then we have B3 = A.

In other words, B is a cubic root of A.

23