L.

Vandenberghe 2/12/02
EE103

Midterm Solutions
Problem 1. (20 points)

Suppose you are asked to find a quadratic function

    

 p11 p12 u1
 u1
f (u1 , u2 ) = u1 u2 + q1 q2 +r
p12 p22 u2 u2

that satisfies the following six conditions:

f (0, 1) = 6, f (1, 0) = 6, f (1, 1) = 3, f (−1, −1) = 7, f (1, 2) = 2, f (2, 1) = 6.

The variables in the problem are the parameters p11 , p12 , p22 , q1 , q2 and r.
Show that you can determine p11 , p12 , p22 , q1 , q2 , and r by solving a set of linear equations
Ax = b. State clearly what A, x, and b are. You do not have to solve the equations, or show
that they are solvable.

SOLUTION. We can write f (u1 , u2 ) as

f (u1 , u2 ) = u21 p11 + 2u1 u2 p12 + u22 p22 + u1 q1 + u2 q2 + r.

For given u1 and u2 , this is a linear function of p11 , p12 , p22 , q1 , q2 , r. For example, f (0, 1) = 6
means
p22 + q2 + r = 6.
We therefore obtain the following set of equations:
    
0 0 1 0 1 1 p11 6
    
 1 0 0 1 0 1  p12   6 
    

 1 2 1 1 1 1 
 p22  
= 3 
.
 1 2 1 −1 −1 1  q1   7 
    
    
 1 4 4 1 2 1  q2   2 
4 4 1 2 1 1 r 6

1
Problem 2. (20 points)

1. For what values of a1 , a2 , . . . , an is the n × n-matrix

 
a1 1 0 ··· 0 0
 
 a2 0 1 ··· 0 0
 
 .. .... . . .. ..
A=

 a
. . . . . 
 .
 n−2 0 0 ··· 1 0 

 
 an−1 0 0 ··· 0 1 
an 0 0 ··· 0 0

nonsingular?

2. Assuming A is nonsingular, how many floating-point operations (flops) do you need to

solve Ax = b?

3. Assuming A is nonsingular, what is the inverse A−1 ? (In other words, express the
elements of A−1 in terms of a1 , a2 , . . . , an .)

SOLUTION.

1. an = 0. We can derive this from the definition of nonsingular matrices: A is nonsingular

if and only if Ax = 0 implies x = 0.
Ax = 0 means

x2 = −a1 x1
x3 = −a2 x1
..
.
xn = −an−1 x1
an x1 = 0.

If an = 0, then from the last equation, x1 = 0, hence also x2 = x3 = · · · = xn = 0, i.e.,

x = 0, so A is nonsingular.
If an = 0, then we can take x1 = 1, x2 = −a1 , x3 = −a2 , etc., and obtain a nonzero x
with Ax = 0. Therefore A is singular.

2. If we put the last equation first we obtain

an x 1 = b n
a1 x 1 + x 2 = b 1
a2 x 1 + x 3 = b 2
..
.
an−1 x1 + xn = bn−1 .

2
 We can solve these equations by forward substitution:

x1 = bn /an
x 2 = b 1 − a1 x 1
x 3 = b 2 − a2 x 1
..
.
xn = bn−1 − an−1 x1 ,

which takes 2n − 1 flops.

3. We can find A−1 by solving AX = I column by column using the method of part 2:
 
0 0 0 ··· 0 1/an
 
 1 0 0 ··· 0 −a1 /an 
 
 0 1 0 ··· 0 −a2 /an 
A =
−1

 0 0 1 ··· 0 −a3 /an

.

 .. .. .. .. .. 
 ... 
 . . . . . 
0 0 0 · · · 1 −an−1 /an

3
Problem 3. (20 points)

Give the matrix norm A of each of the following matrices A. Explain your answers.
 
1 1
1. A =
1 1
 
1 −1
2. A =
1 1
 
1 1 0
3. A = 
 1 1 0  
0 0 −3/2
 
1 −1 0

4. A =  1 1 0  
0 0 −3/2
SOLUTION.

1. A = 2.
√
We have Ax = (x1 + x2 , x1 + x2 ) and Ax = 2|x1 + x2 |. From the identity aT x =
a x cos  (a, x) with a = (1, 1), we know that
√ 
|x1 + x2 | ≤ 2 x21 + x22

with equality if x1 = x2 . Therefore,

Ax √ |x1 + x2 |
A = max = max 2  = 2.
x=0 x x=0
x21 + x22

√
2. A = 2.
 
Ax = (x1 − x2 , x1 + x2 ) and Ax = (x1 − x2 )2 + (x1 + x2 )2 = 2(x21 + x22 ).
Therefore 
Ax 2(x21 + x22 ) √
A = max = max  = 2.
x=0 x x=0
x21 + x22

3. A = 2.

Ax = (x1 + x2 , x1 + x2 , −(3/2)x3 ) and Ax = 2(x1 + x2 )2 + (9/4)x23 .
From part 1 we know that

2(x1 + x2 )2 + (9/4)x23 ≤ 4x21 + 4x22 + (9/4)x23

4
 with equality if x1 = x2 . This allows us to derive the norm in the same way as for a
diagonal matrix with diagonal elements (2, 2, 3/2):

Ax
A = max
x=0 x

4x21 + 4x22 + (9/4)x23
≤ max 
x=0
x21 + x22 + x23

4x21 + 4x22 + 4x23
≤ max 
x=0
x21 + x22 + x23
= 2.

This shows that A ≤ 2. Moreover we have Ax/x = 2 for x = (1, 1, 0), so
A = 2.

4. A = 3/2.

Ax = (x1 − x2 , x1 + x2 , −(3/2)x3 ) and Ax = 2x21 + 2x22 + (9/4)x23 .
√ √
The derivation is the same as for a diagonal matrix with diagonal elements ( 2, 2, 3/2):

Ax
A = max
x=0 x

2x21 + 2x22 + (9/4)x23
= max 
x=0
x21 + x22 + x23

(9/4)x21 + (9/4)x22 + (9/4)x23
≤ max 
x=0
x21 + x22 + x23
= 3/2.

This shows that A ≤ 3/2. Moreover we have Ax/x = 3/2 for x = (0, 0, 1), so
A = 3/2.

5
Problem 4. (20 points)

You are given a nonsingular n × n-matrix A and an n-vector b. You are asked to evaluate

x = (I + A−1 + A−2 + A−3 ) b

where A−2 = (A2 )−1 and A−3 = (A3 )−1 .

Describe in detail how you would compute x, and give the flop counts of the different steps
in your algorithm. If you know several methods, you should give the most efficient one (least
number of flops for large n).

SOLUTION.

1. LU-factorization A = P LU ((2/3)n3 flops).

2. Calculate y = A−1 b by solving P LU y = b:

(a) Calculate v = P T b (0 flops, because P T b is a permutation of b)

(b) Solve Lw = v by forward substitution (n2 flops)
(c) Solve U y = w by backward substitution (n2 flops)

3. Calculate v = A−2 b = A−1 y by solving P LU v = y (2n2 )

4. Calculate w = A−3 b = A−1 v by solving P LU w = v (2n2 )

5. x = b + y + v + w (3n).

Total: (2/3)n3 + 6n2 + 3n.

6
Problem 5. (20 points)
You are given the Cholesky factorization A = LLT of a positive definite n × n-matrix A, and
a vector u ∈ Rn .
1. What is the Cholesky factorization of the (n + 1) × (n + 1)-matrix
 
A u
B= ?
uT 1
You can assume that B is positive definite.
2. What is the cost of computing the Cholesky factorization of B, if the factorization of
A (i.e., the matrix L) is given?
3. Suppose L−1  ≤ 1. Show that B is positive definite for all u with u < 1.

SOLUTION.
1. We need to find L11 ∈ Rn×n , L21 ∈ R1×n , L22 ∈ R such that
    
A u L11 0 LT11 LT21
B= = .
uT 1 L21 L22 0 L22
Moreover L11 must be lower triangular with positive diagonal elements, and L22 > 0.
This gives the following conditions:
A = L11 LT11 , u = L11 LT21 , 1 = L21 LT21 + L222 .
The first equation states that L11 must be the Cholesky factor of A, which is given:
L11 = L. The second condition allows us to compute L21 :
LT21 = L−1 u.
The last condition gives L22 :
  
L22 = 1− L21 LT21 = 1− uT L−T L−1 u = 1 − uT A−1 u.
Putting everything together, we obtain the Cholesky factorization of B:
  
L √ 0 LT √ L−1 u
B= T −T .
u L 1 − u L−T L−1 u
T 0 1 − uT L−T L−1 u

2. n2 flops.
We are given L, so we only need to compute L21 and L22 . We can compute L−1 u by
u, which takes n2 flops because L is lower triangular.
√ of equations Lx =√
solving the set
Computing 1 − uT L−T L−1 u = 1 − xT x takes roughly 2n flops, so the total cost is
about n2 .
3. Suppose L−1  ≤ 1 and u < 1. It follows from one the properties of the matrix
norm (Ax ≤ A x) that
uT L−T L−1 u = L−1 u2 ≤ L−1 2 u2 < 1.
Therefore the Cholesky factorization of B exists, i.e., B is positive definite.