ENGINEERING MATHEMATICS 4

(BWM 30603)
Lecture Module 3: System of Linear Equations

Waluyo Adi Siswanto

Universiti Tun Hussein Onn Malaysia

This module is licensed under a Creative Commons Attribution 3.0 License.

Topics

Available build in routines in Matematical Tools

(some examples): Excel, LO/OO Calc, Freemat,
SMath

Elimination Methods
Gauss

Elimination
Gauss Elimination with (partial) pivoting
Iteration Method

Gauss Siedel

Lecture Module 3

BWM30603

Topics

LU Factorization (Decomposition) Method

Doolittle
Crout
Thomas

Variant 1

Variant 2

Thomas Algorithm

Cholesky

Lecture Module 3

BWM30603

What is a system of linear equations ?

In engineering applications, engineers deal with a system of linear equations
( or simultaneous linear equations ) that they have to solve.
For example, relationships of stiffness coefficients, displacements and forces

1 u 1+2 u 2 =1
2 u1 +9 u 2+1 u3=6
1 u 2 +9 u3+4 u 4 =2
4 u3 +3u 4 =8
Or in matrix notation, you can write [K ] {u }= { F }

[ ]{ } { }
1
2
0
0

2
9
1
0

0
1
9
4

0
0
4
3

u1
1
u2 = 6
2
u3
8
u4

The problem is to find all displacements

Lecture Module 3

BWM30603

How to find unknown variables

in simultaneous linear equation ?

[ ]{ } { }
1
2
0
0

2
9
1
0

0
1
9
4

0
0
4
3

u1
1
u2 = 6
2
u3
8
u4

Would it be EASY if we use mathematical tools such as

Smath, Mathcad, Freemat, Matlab, Scilab, MS Excel, LO/OO Calc ?

The answer is

Lecture Module 3

YES !
BWM30603

See how to solve in SMath

If you think in matrix

Lecture Module 3

[K ] {u }= { F }

{u }=[ K ]1 { F }

BWM30603

See how to solve in Freemat

You can do interactively

Or you can write m file,

then execute (run) the file

Lecture Module 3

BWM30603

See how to solve in LO/OO Calc

K=

1
2
0
0

Inv(K)=

1.8462
-0.4231
0.1154
-0.1538

U=

-1.6923
1.3462
-2.7308
6.3077

Lecture Module 3

2
9
1
0

0
1
9
4

-0.4231 0.1154
0.2115 -0.0577
-0.0577 0.2885
0.0769 -0.3846

0
0
4
3

F=

1
6
2
8

-0.1538
0.0769
-0.3846
0.8462

Enter the data

> Insert Function MINVERSE

> Select the range of K
> Press ENTER
> Insert Function MMULT
> Select the first range inv(K)
> Select the second range F
> Press ENTER

BWM30603

See how to solve in MS Excel

K=

1
2
0
0

Inv(K)=

1.8462
-0.4231
0.1154
-0.1538

U=

-1.6923
1.3462
-2.7308
6.3077

Lecture Module 3

2
9
1
0

0
1
9
4

-0.4231 0.1154
0.2115 -0.0577
-0.0577 0.2885
0.0769 -0.3846

0
0
4
3

F=

1
6
2
8

Enter the data

> Formulas Insert Function

MINVERSE
> Select the result area
> Select the range
> Press Ctrl+Shift+ENTER

-0.1538
0.0769
-0.3846
0.8462

> Insert Function MMULT

> Select the result area
> Select the first range inv(K)
> Select the second range
> Press Ctrl+Shift+ENTER
BWM30603

Example 3-1
Find the unknown variable of the system of linear equations below,
by using any mathematical tool that you have. (Freemat, SMath, LO Calc, MS Excel)
If you use proprietary software, make sure you have a proper legal license !

1 u 1+2 u 2 =1
3 u1 +4 u 2 +5 u3 =0
6 u2 +7 u3 +8 u 4 =2
10 u3 +11 u 4=3
Answer:

Lecture Module 3

{}{ }
u1
1.1173
u 2 = 0.0586
0.6235
u3
0.8395
u4

BWM30603

10

So.. Why we need to learn

How to solve simultaneous linear equation ?

If you just need to solve the simultaneous equations,

you just need the mathematical tools,enter the data and acquire
the unknown variables.

If your calculation is not using the available mathematical tools,

but you are writing programming code and part of the calculation
involving solving simultaneous equations, you need to code
your solution routines.

You MUST know the routines.

In this module, you will learn

Lecture Module 3

various methods that you can use.

BWM30603

11

Elimination Method
Gauss Elimination
In this method, the basic idea is modify the matrix into upper triangular.
Below is the basic idea in Gauss Elimination:

2 x 1+5 x 2=2
4 x 1 +10 x 2 =5

4 x1 +10 x 2 =4
4 x 1 +10 x 2 =5
20 x 2 =1

( multiply by - (-4/2) )

2 x 1 +5 x 2 =2
20 x 2=1
Then solve

x 2 =0.05

x 1 =1.125
Lecture Module 3

BWM30603

12

Elimination Method
Gauss Elimination

2 x 1+5 x 2=2
4 x 1 +10 x 2 =5

Forward
Eliminating
the second
equation then
replacing with
a new one

4 x1 +10 x 2 =4
4 x 1 +10 x 2 =5
20 x 2 =1

( multiply by - (-4/2) )

2 x 1 +5 x 2 =2
20 x 2=1

Backward

Then solve

x 2 =0.05

x 1 =1.125
Lecture Module 3

BWM30603

13

Elimination Method
Gauss Elimination
See how the matrix has been decomposed

from

5
10

]{ } { }
x1
= 2
5
x2

[ ]{ } { }
2 5 x1
= 2
0 20 x 2
1

to

[ A] { x }={b }

[U ] { x }= { f }

]{ } { } [

a 11 a12 a 1n x 1
b1
a 21 a 22 a 2n x 2
= b2

bn
a m1 a m2 a mn x n

a ii 0

Lecture Module 3

]{ } { }

u11 u12 u1n x1

f1
0 u22 u2n x 2
= f2

fn
0
0 u mn x n

Upper triangular matrix

the solution is very simple, starting
from the last row then BACKWARD
BWM30603

14

Elimination Method

Gauss Elimination forward elimination step

The routine can be concluded in general algorithm below:

Elimination step:
Step k

( 1k n1 )

n : the number of simultaneous equations

(k )

assume a kk 0
(k)

define multiplier

mik =

a ik
a

i=k +1,, n

(k)
kk

eliminate (remove) the unknown x k from equation k +1 through n

(k +1)

(k)

(k )

( k +1)

(k)

(k )

a ij
bij

Lecture Module 3

=mik a kj +aij

=mik bk +bi

BWM30603

i , j=k +1,, n

15

Elimination Method

Gauss Elimination backward substitution step

Substitution step:
Using the last step from the elimination step

[U ]=[ A](n1) { f }= {b }(n1)

fn
x n=
u nn
Backward step k =n1, n2,, 1

1
xk=
g k u kj x j
ukk
j=k +1

Lecture Module 3

BWM30603

]
16

Example 3-2
a) Write the simultaneous equations below in a complete matrix
form and in augmented matrix form

x1 +2 x 2 =1
3 x 1+4 x 2 +5 x 3=0
6 x 2 +7 x 3 +8 x 4 =2
10 x 3 +11 x 4 =3
b) Find the unknown variables of simultaneous equations above
by using Gauss Elimination.
c) Double check your result by using Freemat.

Lecture Module 3

BWM30603

17

x1 +2 x 2 =1
3 x 1+4 x 2 +5 x 3=0
6 x 2 +7 x 3 +8 x 4 =2
10 x 3 +11 x 4 =3

Lecture Module 3

[
[

]{ } { }
]

1
3
0
0

2 0 0 x1
1
4 5 0 x2 = 0
6 7 8 x3
2
0 10 11 x 4
3

1
3
0
0

2 0 0
4 5 0
6 7 8
0 10 11

BWM30603

1
0
2
3

18

1
3
0
0

2 0 0
4 5 0
6 7 8
0 10 11

]
1
0
2
3

Elimination steps:
E1
E2
E3
E4

Step 1 : eliminate variable x 1 from equation (E2)

m21=

a21 3
= =3
a11 1

1(3) 2(3) 0(3) 0(3)

3
4
5
0
0
6
7
8
0
0
10
11

Lecture Module 3

]
1(3)
0
2
3

BWM30603

1 2
0 0
0 2 5 0
0 6
7 8
0 0 10 11

]
1
3
2
3

19

1 2
0 0
0 2 5 0
0 6
7 8
0 0 10 11

]
1
3
2
3

Elimination steps:
E1
E2
E3
E4

Step 2 : eliminate variable x 2 from equation (E3)

m32=

a 32 6
=
=3
a 22 2

1
2
0
0
0(3) 2(3) 5(3) 0(3)
0
6
7
8
0
0
10
11

Lecture Module 3

]
1
3(3)
2
3

BWM30603

1 2
0 0
0 2 5 0
0 0 22 8
0 0 10 11

]
1
3
7
3

20

1 2
0 0
0 2 5 0
0 0 22 8
0 0 10 11

]
1
3
7
3

Elimination steps:
E1
E2
E3
E4

Step 3 : eliminate variable x 3 from equation (E4)

a 43
=10/22
a 33
1
2
0
0
0
2
5
0
0(10/22) 0(10/22) 22(10/22) 8(10/ 22)
0
0
10
11
m43 =

Lecture Module 3

BWM30603

1
3
7(10/22)
3

1 2
0
0
0 2 5
0
0 0 22
8
0 0
0 7.364

]
1
3
7
6.182
21

1 2
0
0
0 2 5
0
0 0 22
8
0 0
0 7.364

Lecture Module 3

Substitution steps:

]{ } { }
x1
1
x2
= 3
7
x3
6.182
x4

x 4=

6.182
=0.839
7.364

x3=

1
78 x 4 ] =0.623
[
22

x2=

1
[3(5 x 3 +0 x 4 )]=0.059
2

x1=

1
[1(2 x 2+0 x 3 +0 x 4)]=1.117
1

BWM30603

22

Using Freemat

x1
x2
x3
x4

Lecture Module 3

BWM30603

23

Example 3-3
Write a simple code (function) of Gauss Elimination method
that can be used in Freemat, Matlab or Scilab. Use Example 3-2 to test the code.
function [x] = gaussElim(A,b)
% solving vector x from matrix equation A x = b
% filename gausselim.m
N = max(size(A));
% Perform Gaussian Elimination
for j=2:N;
for i=j:N;
m = A(i,j-1)/A(j-1,j-1);
A(i,:) = A(i,:) - A(j-1,:)*m;
b(i) = b(i) - m*b(j-1);
end;
End;
% Perform back substitution
x = zeros(N,1);
x(N) = b(N)/A(N,N);
for j=N-1:-1:1;
x(j) = (b(j)-A(j,j+1:N)*x(j+1:N))/A(j,j);
end ;
Lecture Module 3

BWM30603

24

Elimination Method
Gauss Elimination with Pivoting
(k )

Gauss elimination requires a kk 0

In every step, the matrix must be checked
and rearrange if necessary to satisfy that
diagonal element is not 0.
(k)

Another operation

mik =

a ik

(k)

a kk

The pivot element in this operation should

Use the largest element to avoid round-off error.
The matrix should be rearranged.
Rearranging the matrix to use the largest
element and to satisfy non zero is called
PIVOTING
Partial Pivoting
Only rearrange the rows
Complete Pivoting (rarely used)
Rearranging the columns and the rows

Lecture Module 3

BWM30603

25

Elimination Method
Gauss Elimination with Pivoting
m-code to implement Gauss Elimination with partial pivoting

Taken from: Applied Numerical Methods for Engineers and Scientists (3ed,2012, p.244) by Stephen C. Chapra

Lecture Module 3

BWM30603

26

Example 3-4
a) Write the simultaneous equations below in a complete matrix
form and in augmented matrix form

0.02 x 1 +300 x 2+10 x 3 =9

500 x 1 x 2 +3 x 3 =6
4 x1 +7 x 2 + x 3=2

b) Find the unknown variables of simultaneous equations above

by using Gauss Elimination with partial pivoting
c) Double check your result by using LO Calc
(you can also try your functions gausspivot.m and gausselim.m)

Lecture Module 3

BWM30603

27

0.02 x 1 +300 x 2+10 x 3 =9

500 x 1 x 2 +3 x 3 =6
4 x1 +7 x 2 + x 3=2

[
[

0.02
500
4

300
1
7

0.02 300 10
500 1 3
4
7
1

Lecture Module 3

BWM30603

]{ } { }
]
10
3
1

x1
9
x2 = 6
2
x3

9
6
2

28

Elimination steps:

0.02 300 10
500 1 3
4
7
1

]
9
6
2

E1
E2
E3

rearrange

500 1 3
0.02 300 10
4
7
1

]
6
9
2

E1
E2
E3

Step 1 : eliminate variable x 1 from equation (E2)

m21=

a21 0.02
=
=4105
a11 500

500(4 105 ) 1(4 105 ) 3(4 105 )

0.02
300
10
4
7
1

Lecture Module 3

6(4 105 )
9
2

BWM30603

500 1 3
0 300 10
4
7
1

]
6
9
2

29

Elimination steps:

500 1 3
0 300 10
4
7
1

]
6
9
2

E1
E2
E3

Step 2-1 : eliminate variable x 1 from equation (E3)

m31=

a 31
4
=
=0.008
a 11 500

500(0.008) 1(0.008) 3(0.008)

0
300
10
4
7
1

Lecture Module 3

] [

6(0.008)
9
2

BWM30603

500 1
3
0
300
10
0 7.008 0.976

]
6
9
1.952

30

Elimination steps:

500 1
3
0
300
10
0 7.008 0.976

]
6
9
1.952

E1
E2
E3

Step 2-2 : eliminate variable x 2 from equation (E3)

m32=

a 32 7.008
=
=0.0234
a 22 300

500
1
3
0(0.0234) 300 (0.0234) 10(0.0234)
0
7.008
0.976

6
9(0.0234)
1.952

[
Lecture Module 3

BWM30603

500 1
3
0 300
10
0
0 0.742

]
6
9
1.742

31

Substitution steps:

500 1
3
0 300
10
0
0 0.742

Lecture Module 3

]
6
9
1.742

]{ } { }

x1
500 1
3
6
0 300
10
x2 = 9
0
0 0.742 x 3
1.742

x3=

1.742
=2.348
0.742

x2=

1
910 x 3 ] =0.048
[
300

x1=

1
[ 6(1 x 2+3 x 3 )]=0.002
500

BWM30603

32

LO Calc result :

Lecture Module 3

Results from
gausselim and gausspivot

BWM30603

33

LU Factorization Methods
In Engineering, it is commonly said

Decomposition rather than Factorization

LU Decomposition Methods
[ A ] { x }={b }

In linear system

What is L U decomposition :

Lower triangular matrix [ L ]

l 11 0 0
l
l
0
[ L]= 21 22

l m1 l m2 l mn
In linear system
Lecture Module 3

[ A]

[U ]

Upper triangular matrix

] [

u11 u12 u1n

0 u 22 u2n
[U ]=

0
0 u mn

is rectangular m=n
BWM30603

34

LU Decomposition Methods
Solution Steps
[ A]
1. Decomposition step

[U ]

2. Solving steps

{ x } = {b }

[ L]

[L]

{d } = {b }
{d }

[U ]

{ x } = {d }
{x}

Lecture Module 3

2-1. Forward substitution

2-2. Backward substitution

BWM30603

35

LU Decomposition
Doolitlle Method
[ A]
[L]

1 0 0
l
1 0
[ L]= 21

l n1 l n2 1

{ x } = {b }

[U ]

] [

u11 u12 u1n

0 u 22 u 2n
[U ]=

0
0 u nn

All diagonal terms of [L] are 1

Lecture Module 3

BWM30603

36

Example 3-5
a) Write the simultaneous equations below in a complete matrix form

[ A ] { x }={b }

2 x 11 x 22 x 3=1
4 x 1+6 x 2 +3 x 3 =13
4 x 1 2 x 2 +8 x 3 =6

b) Decompose matrix [A] according to Doolittle method

c) Find the unknown variables of simultaneous equations.
d) Double check your result in Freemat or Matlab (need license)
by using the build-in lu function.

Lecture Module 3

BWM30603

37

Decomposition step :

[ A ] { x }= {b }

2 1 2
4 6
3
4 2 8

]{ } { }
x1
1
x 2 = 13
6
x3

[ L ][ U ]=[ A ]

] [ ][ ] [

2 1 2
1 0 0 d
4 6
3 = a 1 0 0
4 2 8
b c 1 0

e
g
0

f
d
h = ad
i
bd

e
f
ae+g
af +h
be+cg bf +h+i

d =2
e=1
f =2
ad =4
ae+ g=6
af +h=3
a=2
g=4
h=1
bd =4 be+cg =2 bf +ch+i=8
b=2
c=1
i=3
Lecture Module 3

BWM30603

38

Decomposition step :

[
[

d =2 e=1
a =2 g=4
b=2 c=1

][

f =2
h=1
i=3

][

2 1 2
1
0 0 2 1 2
4 6
3 = 2 1 0 0 4 1
4 2 8
2 1 1 0 0
3
[ L]

Lecture Module 3

[U ]

BWM30603

39

Solving step (forward substitution) :

[L]

{d } = {b }

]{ } { }

1
0 0 d1
1
2 1 0 d 2 = 13
2 1 1 d 3
6

Lecture Module 3

d 1=

1
=1
1

d 2=

1
13(2 d 1 ) ]=11
[
1

d 3=

1
[6(2 d 11 d 2 ) ]=3
1

BWM30603

40

Solving step (backward substitution) :

[U ]

{ x } = {d }

]{ } { }

2 1 2 x 1
1
0 4 1 x 2 = 11
0 0
3 x3
3
3
x 3 = =1
3

Lecture Module 3

x2=

1
11(1 x 3 ) ] =3
[
4

x 1=

1
[1(1 x 22 x 3)]=2
2

BWM30603

41

Lecture Module 3

BWM30603

42

LU Decomposition
Doolitlle Method
The routine can be concluded here
Decomposition step:

Solution step:

for i=1:m
% Finding L
for k=1:i-1
L(i,k)=A(i,k);
for j=1:k-1
L(i,k)= L(i,k)-L(i,j)*U(j,k);
end
L(i,k) = L(i,k)/U(k,k);
end
% Finding U
for k=i:m
U(i,k) = A(i,k);
for j=1:i-1
U(i,k)= U(i,k)-L(i,j)*U(j,k);
end
end
end
% Assigning unity for all diagonal items of L
for i=1:m
L(i,i)=1;
end
Lecture Module 3

BWM30603

% Forward substitution 'Ld=b' to solve d

d(1)=B(1)/L(1,1);
for i=2:m
d(i)=-L(i,1)*d(1);
for k=2:i-1
d(i)=d(i)-L(i,k)*d(k);
end;
d(i)=(B(i)+d(i))/L(i,i);
end;
% Bacward substitution 'Ux=d' to solve x
x(m)=d(m)/U(m,m);
i=m-1;
q=0;
while (i~= 0)
x(i)=-U(i,m)*x(m);
q=i+1;
while (q~=m)
x(i)=x(i)-U(i,q)*x(q);
q=q+1;
end;
x(i)=(d(i)+x(i))/U(i,i);
i=i-1;
end;
43

Example 3-6
Write matlab m function (doolittle.m) and test it to solve Example 3-5.
function[L,U,d,x]=doolittle(A,B)
% prepared by Waluyo Adi Siswanto
[m n]=size(A);
if (m ~= n )
disp ( 'LR2 error: Matrix must be square' );
return;
end;
L=zeros(m,m);
U=zeros(m,m);
% LU Decomposition (DOOLITTLE METHOD)
for i=1:m
for k=1:i-1
L(i,k)=A(i,k);
for j=1:k-1
L(i,k)= L(i,k)-L(i,j)*U(j,k);
end
L(i,k) = L(i,k)/U(k,k);
end
for k=i:m
U(i,k) = A(i,k);
for j=1:i-1
U(i,k)= U(i,k)-L(i,j)*U(j,k);
end
end
end
for i=1:m
L(i,i)=1;
End
% End of decomposition (DOOLITTLE METHOD)

Lecture Module 3

% Forward substitution 'Ld=b' to solve d

d=zeros(m,1);
d(1)=B(1)/L(1,1);
for i=2:m
d(i)=-L(i,1)*d(1);
for k=2:i-1
d(i)=d(i)-L(i,k)*d(k);
end;
d(i)=(B(i)+d(i))/L(i,i);
end;
% Bacward substitution 'Ux=d' to solve x
x=zeros(m,1);
x(m)=d(m)/U(m,m);
i=m-1;
q=0;
while (i~= 0)
x(i)=-U(i,m)*x(m);
q=i+1;
while (q~=m)
x(i)=x(i)-U(i,q)*x(q);
q=q+1;
end;
x(i)=(d(i)+x(i))/U(i,i);
i=i-1;
end;

BWM30603

44

Lecture Module 3

BWM30603

45

Exercise 3-7
a) Decompose matrix [A] of the simultaneous equations below
according to Doolittle method

[ ]{ } { }
1
5
9
3

2
2
0
4

3
7
3
5

4
8
2
6

x1
3
x2
= 0
7
x3
8
x4

b) Find the vector {d} in Doolittle step

{d }=[3 15 13.75 1.833]T

c) Find the vector {x}

{ x }=[2 3.25 5.5 2.75]T

d) Use your doolittle.m to check the answer

(See next page)

Lecture Module 3

BWM30603

46

Lecture Module 3

BWM30603

47

LU Decomposition
Crout Method
[ A]
[L]

l 11 0 0
l
l
0
[ L]= 21 22

l n1 l n2 l nn

{ x } = {b }

[U ]

] [

1 u12 u1n
0 1 u2n
[U ]=

0 0 1

All diagonal terms of [U] are 1

Lecture Module 3

BWM30603

48

LU Decomposition
Crout Method
The algorithm for decomposition step:

i j , i=1, 2,, n
j1

Lij = Aij L ik U kj
k 1

i< j , j=2, 3,, n

j1

1
U ij = Aij Lik U kj
Lii
k 1
j=1
Lij = Ai1

i=1
A ij
U ij =
A11
Lecture Module 3

% LU Decomposition (CROUT METHOD)

L(1,1)=A(1,1);
for i=2:n
U(1,i)=A(1,i)/L(1,1);
L(i,1)=A(i,1);
end
for i=2:n
for j=2:n
s=0;
if i<=j;
K=i-1;
else;
K=j-1;
end;
for k=1:K;
s=s+L(i,k)*U(k,j);
end;
if j>i;
U(i,j)=(A(i,j)-s)/L(i,i);
else;
L(i,j)=A(i,j)-s;
end
end
end
for i=1:n
U(i,i)=1;
end
% End of Decomposition (CROUT METHOD)

BWM30603

49

Exercise 3-8
Modify your doolittle.m (example 3-6) to implement CROUT method and save as crout.m.
Try to solve example 3.7 by using crout.m. Try to solve by hand calculation according to Crout.
Simply replace the DOOLITTLE part
of Example 3-6 with the CROUT method
written in page 49.

Lecture Module 3

BWM30603

50

[ ]{ } { }
[ ] [ ][ ]
[
x1
3
x2
= 0
7
x3
8
x4

1
5
9
3

2
2
0
4

3
7
3
5

4
8
2
6

1
5
9
3

2
2
0
4

3
7
3
5

4
a
8
b
=
2
d
6
g

0
c
e
h

a=1
b=5
=
d =9
g=3

Lecture Module 3

Decomposition step :

0
0
f
i

0
0
0
j

1
0
0
0

k l m
1 n o
0 1 p
0 0 1

ak =2
al =3
bk+c=2
bl+cn=7
dk+e=0 dl +en+ f =3
gk+h=4 gl+hn+i=5
BWM30603

am=4
bm+co=8
dm+eo+ fp=2
gm+ho+ip+ j=6

]
51

Decomposition step :

a=1

ak =2
al=3
k =2
l=3
b=5 bk +c=2
bl+cn=7
c=8
n=1
d =9 dk +e=0 dl+en+ f =3
e=18
f =6
g=3 gk +h=4 gl +hn+i=5
h=2
i=2

[ ][
1
5
9
3

2
2
0
4

3
7
3
5

4
a=1
0
8
b=5 c=8
=
2
b=9 e=18
6
b=3 h=2

0
0
f =6
i=2

0
0
0
j=2/3

[ L]
Lecture Module 3

am=4
m=4
bm+co=8
o=1.5
dm+eo+ fp=2
p=7/6
gm+ho+ip+ j=6
j=2/3

][

1 k =2 l=3
0
1
n=1
0
0
1
0
0
0

m=4
o=1.5
p=7/6
1

[U ]
BWM30603

52

Solving step (forward substitution) :

[L]

{d } = {b }

]{ } { }

d1
1
0
0
0
3
d2
5 8
0
0
= 0
9 18 6
0
7
d3
3 2 2 2/3 d 4
8

3
d 1 = =3
1

Lecture Module 3

d 2=

1
[ 0(5 d 1 )]=1.875
8

d 3=

1
7(9 d 118 d 2 ) ]=2.292
[
6

d 4=

1
[ 8(3 d 12 d 22 d 3)]=2.751
2 /3
BWM30603

53

Solving step (backward substitution) :

[U ]

1
0
0
0

2
1
0
0

3 4
1 1.5
1 7 /6
0 1

Lecture Module 3

{ x } = {d }

]{ } { }
x1
3
x2
= 1.875
2.292
x3
2.751
x4
x 4=

2.751
=2.751
1

x3=

1
[2.292(7/6 x 4 )]=5.502
1

x2=

1
1.875(1 x 3 +1.5 x 4 ) ]=3.251
[
1

x 1=

1
[ 3(2 x 2+3 x 3+4 x 4) ]=2
1
BWM30603

54

LU Decomposition
Thomas Method Variant 1
Engineering problems often lead to coefficient matrices that are sparsely populated,
meaning that most elements of the matrix are zero. If all the nonzero terms are clustered
about the leading diagonal, then the matrix is said to be banded.

[ A] =

[ L]

][

[U ]

][

b1 c1 0
0
1 0 0

0
1 1 0
a 2 b2 c 2
0
a 2 2 0

0
0 1 2
0 a3
0 = 0 a3
0 + 0 0


bn1 c n1

n1 0
0 0
0 0 an
bn
0 0 a n n

0
0
0

1
0

n1
1

This (the same with that in A)

is NOT requested
in CROUT method
Lecture Module 3

BWM30603

55

Exercise 3-9a
a) Decompose the banded matrix [A] of the simultaneous equations below
according to Thomas method (variant 1)

1
3
0
0

]{ } { }

2 0 0 x1
1
4 5 0 x2 = 0
6 7 8 x3
2
0 10 11 x 4
3

b) Find the vector {x}

d) Use your crout.m to check the answer, and proof that Crout on banded matrix gives
the same result of L and U as defined by Thomas.

Lecture Module 3

BWM30603

56

[
[

]{ } { }
] [ ][

1
3
0
0

2 0 0 x1
1
4 5 0 x2 = 0
6 7 8 x3
2
0 10 11 x 4
3

1
3
0
0

1 0 0 0
2 0 0
4 5 0 = 3 2 0 0
6 7 8
0 6 3 0
0 10 11
0 0 10 4

Decomposition step :

1 1 0 0
0 1 2 0
0 0 1 3
0 0 0 1

1=1
1 1=2
3
31 + 2=4
2 2=5
=
6
62 +3 =7
3 3 =8
10
3+ 4=11
Lecture Module 3

BWM30603

]
57

Decomposition step :

1=1
1 1 =2
1=1
1 =2
3
31 +2 =4
2 2=5
3
2 =2 2 =2.5
6
62 + 3=7
3 3
6
3=22 3 =0.3636
10
3+ 4=11
10
4 =7.3636

1
3
0
0

][

2 0 0
1 0
0
0
4 5 0
3 2 0
0
=
6 7 8
0 6 22
0
0 10 11
0 0 10 7.3636

][

[ L]
Lecture Module 3

1
0
0
0

2
0
0
1 2.5
0
0
1
0.3636
0
0
1

[U ]
BWM30603

58

Solving step (forward substitution) :

[L]

1 0
0
0
3 2 0
0
0 6 22
0
0 0 10 7.3636

{d } = {b }

]{ } {}
d1
1
d2
= 0
2
d3
3
d4

1
d 1 = =1
1

Lecture Module 3

d 2=

1
[ 0(3 d 1 )]=1.5
2

d 3=

1
2(6 d 2 ) ] =0.3182
[
22

d 4=

1
[ 3(10 d 3)]=0.8395
10
BWM30603

59

Solving step (backward substitution) :

[U ]

1
0
0
0

2
0
0
1 2.5
0
0
1
0.3636
0
0
1

Lecture Module 3

{ x } = {d }

]{ } { }
x1
1
x2
1.5
=
0.3182
x3
0.8395
x4
x 4=

0.8395
=0.8395
1

x3=

1
[0.3182(0.3636 x 4)]=0.6234
1

x 2=

1
1.5(2.5 x 3 ) ]=0.0585
[
1

x 1=

1
[1(2 x 2)]=1.117
1
BWM30603

60

Crout method on banded matrix gives the same result of L and U as defined by Thomas .
Lecture Module 3

BWM30603

61

LU Decomposition
Thomas Method Variant 2
Another variation in Thomas method

[ A] =

[ L]

][

d 1 c1 0
0
1 0 0
a 2 d 2 c2
0
2 1 0
0 a3
0 = 0 3

d n1 c n1

1
0 0 n
0 0 an
dn

[U ]

][

1 c 1 0

0
0
0
0 2 c2
0
0 + 0 0
0

n1 c n1
0
1
0 0 0
n

This (the same with that in A)

is NOT requested
in DOOLITTLE method
Lecture Module 3

BWM30603

62

The algorithm for variant 2:

function [beta,alpha,c,d,x] = thomas(A,B)
% thomas method variant 2
beta=diag(A,-1);
alpha=diag(A);
c=diag(A,1);
N = length(B);
for k=2:N
mult = beta(k-1)/alpha(k-1);
alpha(k) = alpha(k)-mult*c(k-1);
beta(k-1)=mult;
B(k) = B(k)-mult*B(k-1);
end
d=B;
B(N) = B(N)/alpha(N);
for k=N-1:-1:1
B(k) = (B(k)-c(k)*B(k+1))/alpha(k);
end;
x=B;;

1 =b1

i=2,3,n
a1
i =
i1
i=ai i c i1
Forward substitution:

d 1 =b1
d i =bi i d i1
Backward substitution:

x n =d n / n
x i =d ici x i+1
Lecture Module 3

BWM30603

63

Exercise 3-9b
Do example 3-9a by using Thomas method variant 2. Try the thomas.m also by hand!

Lecture Module 3

BWM30603

64

[
[

Decomposition step :

]{ } { }
] [ ][

1
3
0
0

2 0 0 x1
1
4 5 0 x2 = 0
6 7 8 x3
2
0 10 11 x 4
3

1
3
0
0

1 0 0
2 0 0
4 5 0 = 1 1 0
6 7 8
0 2 1
0 10 11
0 0 3

0 1 2 0 0
0 0 2 5 0
0 0 0 3 8
1 0 0 0 4

1 =1
2
=3 21 +2 =4
5
= 1 1
2 2 =6
52 + 3=7
8
3 3 =10 83 + 4 =11
Lecture Module 3

BWM30603

]
65

Decomposition step :

1 =1
2
1 =1
1 1 =3 21+ 2 =4
1 =3 2=2
2 2=6
2 =3

1
3
0
0

5
52 +3 =7
8
3 =22
3 3 =10
83+ 4 =11
3 =0.4545 4 =7.3636

][

2 0 0
1 0
0
4 5 0
3 1
0
=
6 7 8
0 3
1
0 10 11
0 0 0.4545

0
0
0
1

][

1 2
0
0
0 2 5
0
0 0 22
8
0 0
0 7.3636

[ L]
Lecture Module 3

[U ]
BWM30603

66

Solving step (forward substitution) :

[L]

1 0
0
3 1
0
0 3
1
0 0 0.4545

0
0
0
1

{d } = {b }

]{ } {}
d1
1
d2
= 0
2
d3
3
d4

1
d 1 = =1
1

Lecture Module 3

d 2=

1
[ 0(3 d 1 )]=3
1

d 3=

1
2(3 d 2 ) ] =7
[
1

d 4=

1
[ 3(0.4545 d 3)]=6.1818
1
BWM30603

67

Solving step (backward substitution) :

[U ]

1 2
0
0
0 2 5
0
0 0 22
8
0 0
0 7.3636

Lecture Module 3

{ x } = {d }

]{ } { }
x1
1
x2
= 3
7
x3
6.1818
x4
x 4=

6.1818
=0.8395
7.3636

x3=

1
[7(8 x 4 )]=0.6234
22

x 2=

1
3(5 x 3 ) ]=0.0585
[
2

x 1=

1
[1(2 x 2)]=1.117
1
BWM30603

68

Thomas Algorithm
The routine for Thomas (Variant 1) has been generalized for a faster calculation by using
a calculation table.

[ A] =

[ L]

][

[U ]

][

d 1 e1 0
0
1 0 0
0
1 1 0 0
c2 d 2 e2
0
c2 2 0
0
0 1 2 0
0 c3
0 = 0 c3
0 + 0 0
0

1 n1

d n1 e n1

n1 0
0 0 0
1
0 0 cn
dn
0 0 cn n

Lecture Module 3

BWM30603

]
69

Thomas algorithm table:

i

...

di
ei

ci
bi

1=d 1

i =d ici i1

ei
i=
i
b1
y 1=
1

bici yi1
yi=
i

x n= y n

x i = y i i x i+1

Lecture Module 3

BWM30603

70

Exercise 3-9c

1
3
0
0

]{ } { }

2 0 0 x1
1
4 5 0 x2 = 0
6 7 8 x3
2
0 10 11 x 4
3

Find the vector {x} by using the Thomas algorithm table.

Lecture Module 3

BWM30603

71

1
3
0
0

]{ } { }

2 0 0 x1
1
4 5 0 x2 = 0
6 7 8 x3
2
0 10 11 x 4
3

di
ei

1=d 1

i =d ici i1

ei
i=
i

10

4-3(2)
=-2

7-6(-2.5)
=22

11-10(0.364)
=7.36

2/1
=2

5/-2
=-2.5

8/22
=0.364

ci
bi

11

b1
y1=
1

bici yi1
yi=
i

1/1
=1

0-3(1)/-2
=1.5

2-6(1.5)/22
=-0.318

3-10(-0.318)/7.36
=0.840

x n= y n

x i = y i i x i +1

1-2(0.06)
=1.12

1.5-(-2.5)(0.624)
=-0.06

-0.3180.364(0.840
=-0.624

0.840

Lecture Module 3

BWM30603

72

LU Decomposition
Cholesky Method
[ A]
[L]

l 11 0 0
l
l
0
[ L]= 21 22

l n1 l n2 l nn

Lecture Module 3

{ x } = {b }

[ U ]=[ L ]

] [

l 11 l 21 l nl
0 l 22 l n2
[U ]=

0 0 l nn

BWM30603

]
73

The decomposition algorithm can be expressed as:

If i= j (diagonal term)

j1

L jj = A jj L 2jk
k =1

must be
positive

j=2, 3,, n
For non diagonal term

L jj =

j 1

A jj Lik L jk
k =1

% LU Decomposition (CHOLESKY METHOD)

for j = 1:n
temp = A(j,j) - dot(A(j,1:j-1),A(j,1:j-1));
if temp < 0.0
error(Matrix is not positive definite)
end
A(j,j) = sqrt(temp);
for i = j+1:n
A(i,j)=(A(i,j) - dot(A(i,1:j-1),A(j,1:j-1)))/A(j,j);
end
end
L = tril(A);
U=transpose(L);
% End of Decomposition (CHOLESKY METHOD)

L jj
j=2, 3,, n1
i= j+1, j+2,, n

Lecture Module 3

BWM30603

74

Exercise 3-10
Modify your doolittle.m (example 3-6) to implement CHOLESKY method
and save as cholesky.m.

Simply replace the DOOLITTLE part

of Example 3-6 with the CHOLESKY method
written in page 74.

Lecture Module 3

BWM30603

75

Exercise 3-11
a) Decompose matrix [A] of the simultaneous equations below
according to Cholesky method

8
2
3
4

2
5
1
0

]{ } { }

3 4 x1
1
1 0 x2 = 0
7 3 x3
0
3 11 x 4
9

b) Find the vector {x}

c) Use your cholesky.m to check the answer

Lecture Module 3

BWM30603

76

Decomposition step :

[ ]{ } { }
[ ] [ ][ ]
8
2
3
4

2
5
1
0

3 4 x1
1
1 0 x2 = 0
7 3 x3
0
3 11 x 4
9

8
2
3
4

2
5
1
0

3 4
a
1 0
b
=
7 3
d
3 11
g

0
c
e
h

0
0
f
i

0
0
0
j

a b
0 c
0 0
0 0

d
e
f
0

g
h
i
j

a 2 =8
ab=2 b2 +c 2 =5
=
2
2
2
ad =3 bd +ce=1 d +e + f =7
ag=4 bg +ch=0 dg+eh+if =3
Lecture Module 3

BWM30603

g +h +i + j =11

]
77

Decomposition step :

a 2 =8
a=2.8284
2
2
ab=2
b +c =5
b=0.7071 c=2.1213
2
2
2
ad =3
bd +ce=1
d +e + f =7
d =1.0607 e=0.1179
f =2.4210
ag=4
bg+ch=0
dg+eh+if =3 g 2 +h 2+i 2 + j 2=11
g=1.4142 h=0.4714 i=0.6425
j=2.8922

8
2
3
4

2
5
1
0

][

][

3 4
2.8284
0
0
0
2.8284 0.7071 1.0607 1.4142
1 0
0.7071 2.1213
0
0
0
2.1213 0.1179 0.4714
=
7 3
1.0607 0.1179 2.4210
0
0
0
2.4210 0.6425
3 11
1.4142 0.4714 0.6425 2.8922
0
0
0
2.8922
[U ]

[ L]
Lecture Module 3

BWM30603

78

Solving step (forward substitution) :

[L]

{d } = {b }

]{ } { }

d1
2.8284
0
0
0
1
d2
0.7071 2.1213
0
0
= 0
1.0607 0.1179 2.4210
0
0
d3
1.4142 0.4714 0.6425 2.8922 d 4
9

Lecture Module 3

d 1=

1
=0.3536
2.8284

d 2=

1
[ 0(0.7071 d 1)]=0.1179
2.1213

d 3=

1
0(1.0607 d 1+0.1179 d 2 ) ]=0.1492
[
2.4210

d 4=

1
[ 9(1.4142 d 10.4714 d 2+0.6425 d 3 )]=2.9528
2.8922
BWM30603

79

Solving step (backward substitution) :

[U ]

{ x } = {d }

2.8284 0.7071 1.0607 1.4142

0
2.1213 0.1179 0.4714
0
0
2.4210 0.6425
0
0
0
2.8922

Lecture Module 3

]{ } { }
x1
0.3536
x2
= 0.1179
0.1492
x3
2.9528
x4

x 4=

2.9528
=1.0210
2.8922

x3=

1
[0.1492(0.6425 x 4 )]=0.3326
2.4210

x2=

1
0.1179(0.1179 x 30.4714 x 4 ) ] =0.1898
[
2.1213

x1=

1
[ 0.3536(0.7071 x 2+1.0607 x 3+1.4142 x 4)]=0.3082
2.8284
BWM30603

80

Lecture Module 3

BWM30603

81

Iteration Method
Gauss-Seidel
In simultaneous equations

a 11 x 1+a12 x 2++a 1n x n=b1

a 21 x 1 +a 22 x 2 ++a 2n x n =b2

a n1 x 1 +a n2 x 2++a nn x n=bn
and using initial guess at k=1, the iteration can be started

b1 a 12 x(k2 )a 13 x (k3 ) a 1n x (kn )

=
a11

(k +1)
1

( k +1)

(k +1)

b2 a 21 x 1

(k +1)

bn a 2n1 x 1

x2

(k )

(k +1)

xn
Lecture Module 3

(k)

+a 22 x 2 a 2n x n
a 22
( k+1)

+an2 x 2
a nn

BWM30603

(k )

a nn x n

82

Iteration Method
Gauss-Seidel: convergence criterion
Requirement for stable converge criterion: [A] must be diagonally dominant
n

aii= aij
j=1
j i

See this example

Rearrange

[
[

1
1
6

7
0
0

4
9
1

6
1
1

0
7
0

1
4
9

]{ } { }
]{ } { }
x1
2
x2 = 3
5
x3
x1
5
x2 = 2
3
x3

not diagonally dominant

6> 0+ 1
7> 1+ 4
9> 1+ 0
diagonally dominant

Lecture Module 3

BWM30603

83

Iteration Method
Gauss-Seidel
In conclusion, for a general n equations
the Gauss-Seidel iterative method is defined as:

k =0,1,, n
i=1,2,, n
x

Lecture Module 3

i1
n
1
(k +1)
=
bi aij x j aij x(kj )
j=1
j=i +1
a ii

(k +1)
i

BWM30603

84

Iteration Method
Gauss-Seidel
function x = GaussSeidel(A,b,es,maxit)
% Gauss-Siedel method
if nargin<4, maxit=50; end
if nargin<3, es=0.00001; end
[m,n] = size(A);
if m~=n, error('Matrix A must be square'); end
C = A;
for i = 1:n
C(i,i) = 0;
x(i) = 0;
end
x = x';
for i = 1:n
for j = 1:n;
H = abs(A(i,j));
Check(i) = abs(A(i,i)) - sum(H);
if Check(i) < 0
fprintf('The matrix is not strictly diagonally dominant
at row %2i\n\n',i))
end
end
end

Lecture Module 3

BWM30603

for i = 1:n
C(i,1:n) = C(i,1:n)/A(i,i);
end
for i = 1:n
d(i) = b(i)/A(i,i);
end
iter = 0;
while (1)
xold = x;
for i = 1:n
x(i) = d(i)-C(i,:)*x;
if x(i) ~= 0
ea(i) = abs((x(i) - xold(i))/x(i)) * 100;
end
end
iter = iter+1;
if max(ea)<=es | iter >= maxit, break, end
end

85

Example 3-12

]{ } { }

0 2 7 3 x 1
3
4
0
1 9 x 2 = 3
3
1
1
0 x3
2
1 5 2
1 x4
5

a) Check the criterion of diagonally dominant

on every row
b) If not satisfied then rearrange and check
the criterion
c) Write down the iteration equation for
every row
d) Do the Gauss-Seidel iteration by using
GaussSeidel.m function

Lecture Module 3

BWM30603

86

]{ } { }

0 2 7 3 x 1
3
4
0
1 9 x 2 = 3
3
1
1
0 x3
2
1 5 2
1 x4
5

0 >2+ 7+ 3
0 >4+ 1+ 9
1> 3+ 1+ 0
1> 1+ 5+ 2

X
X
X
X

Then rearrange the matrix

3
1
1
0
1 5 2
1
0 2 7 3
4
0
1 9

]{ } { }
x1
2
x2
= 5
3
x3
3
x4

3> 1+ 1+ 0
5> 1+ 2+1
7> 0+ 2+ 3
9> 4+0+ 1

OK
OK
OK
OK

Now you can use Gauss-Seidel

Lecture Module 3

BWM30603

87

]{ } { }

3
1
1
0 x1
2
1 5 2
1 x2 = 5
0 2 7 3 x 3
3
4
0
1 9 x 4
3

x(k)
1

Initial values

(k )

(k )

(k )

(k)

(k )

(k )

x2

x3

(k )

x4

(k )

21 x 2 1 x 3 0 1 x 4
2 x 2 x 3
(k +1)
x1 =
=
3
3
(k +1)

(k +1)

x2

5(1) x 1

( k +1)

( k +1)
3

( k +1)
4

3(0) x 1

Lecture Module 3

(k )

(k +1)

(k)

(k +1)

(2) x 2
7

( k +1)

3(4) x1

(k )

2 x 3 1 x 4
5+ x 1
=
5
(3) x 4

(k +1)

(0) x 2
9

BWM30603

( k +1)

(1) x 3

(k )

(k )

2 x 3 x 4
5

(k)

(k +1)

3+2 x 2 +3 x 4
=
7
( k +1)

( k +1)

34 x 1 x 3
=
9

88

You can use a table to make it more organized :

Lecture Module 3

BWM30603

89

Lecture Module 3

BWM30603

90

Lecture Module 3

BWM30603

91