BDA34003

ENGINEERING MATHEMATICS 4

Lecture Module 3: System of Linear Equations

Waluyo Adi Siswanto

waluyo@uthm.edu.my
 Learning Outcomes

➢ Students know the methods to find unknown variables

in a set of linear equations
➢ Students have the knowledge to use Matematical
Tools: Excel, LO/OO Calc, Freemat, Smath to solve
simultaneous equations.
➢ Students will be able to solve engineering problems
involving symultaneous linear equations

BDA34003 Engineering Mathematics IV 2

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Topics

➢ Available build in routines in Matematical Tools

(some examples): Excel, LO/OO Calc, Freemat,
SMath
➢ Elimination Methods
➢ Gauss Elimination
➢ Gauss Elimination with (partial) pivoting

➢ Iteration Method

➢ Gauss Siedel

BDA34003 Engineering Mathematics IV 3

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Topics

➢ LU Factorization (Decomposition) Method

➢ Doolittle

➢ Crout

➢ Thomas
➢ Variant 1
➢ Variant 2
➢ Thomas Algorithm
➢ Cholesky

BDA34003 Engineering Mathematics IV 4

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 What is a system
of linear equations ?

In engineering applications, engineers deal with a system of linear equations

( or simultaneous linear equations ) that they have to solve.

For example, relationships of stiffness coefficients, displacements and forces

1 u 1+ 2 u 2 =1
2 u1 + 9 u 2+ 1 u3=6
1 u 2 + 9 u3+ 4 u 4 =2
4 u3 + 3 u 4 =8
Or in matrix notation, you can write [K ] {u }= { F }

[ ]{ } { }
1 2 0 0 u1 1
2 9 1 0 u2 = 6
0 1 9 4 u3 2
0 0 4 3 u4 8

The problem is to find all displacements

BDA34003 Engineering Mathematics IV 5

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
How to find unknown variables
in simultaneous linear equation ?

[ ]{ } { }
1 2 0 0 u1 1
2 9 1 0 u2 = 6
0 1 9 4 u3 2
0 0 4 3 u4 8

Would it be EASY if we use mathematical tools such as

Smath, Mathcad, Freemat, Matlab, Scilab, MS Excel, LO/OO Calc ?

The answer is YES !

BDA34003 Engineering Mathematics IV 6

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 See how to solve in SMath

If you think in matrix [K ] {u }= { F } {u }=[ K ]−1 { F }

BDA34003 Engineering Mathematics IV 7

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
See how to solve in Freemat

You can do interactively

Or you can write m file,

then execute (run) the file

BDA34003 Engineering Mathematics IV 8

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 See how to solve
in LO/OO Calc
UTHM provides download link for LibreOffice: http://ptm.uthm.edu.my/it-info.html

K= 1 2 0 0 F= 1
2 9 1 0 6
Enter the data
0 1 9 4 2
0 0 4 3 8

Inv(K)= 1.8462 -0.4231 0.1154 -0.1538

> Insert » Function » MINVERSE
-0.4231 0.2115 -0.0577 0.0769
0.1154 -0.0577 0.2885 -0.3846
> Select the range of K
-0.1538 0.0769 -0.3846 0.8462 > Press ENTER

U= -1.6923
1.3462 > Insert » Function » MMULT
-2.7308 > Select the first range inv(K)
6.3077 > Select the second range F
> Press ENTER

BDA34003 Engineering Mathematics IV 9

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 See how to solve
in MS Excel

K= 1 2 0 0 F= 1
2 9 1 0 6
Enter the data
0 1 9 4 2
0 0 4 3 8

Inv(K)= 1.8462 -0.4231 0.1154 -0.1538 > Formulas » Insert Function »

-0.4231 0.2115 -0.0577 0.0769 MINVERSE
0.1154 -0.0577 0.2885 -0.3846 > Select the result area
-0.1538 0.0769 -0.3846 0.8462 > Select the range
> Press Ctrl+Shift+ENTER
U= -1.6923
1.3462
-2.7308
6.3077 > Insert » Function » MMULT
> Select the result area
> Select the first range inv(K)
> Select the second range
> Press Ctrl+Shift+ENTER

BDA34003 Engineering Mathematics IV 10

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Example 3-1

Find the unknown variable of the system of linear equations below,

by using any mathematical tool that you have. (Freemat, SMath, LO Calc, MS Excel)
If you use proprietary software, make sure you have a proper legal license !

1u 1+ 2 u 2 =1
3 u1 + 4 u 2 + 5 u3 =0
6 u2 + 7 u3 + 8 u 4 =2
10 u3 + 11 u 4=3

Answer:

{}{ }
u1 1.1173
u 2 = −0.0586
u3 −0.6235
u4 0.8395

BDA34003 Engineering Mathematics IV 11

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 So.. Why we need to learn
How to solve
simultaneous linear equations ?

➢ If you just need to solve the simultaneous equations,

you can use the mathematical tools,enter the data and acquire the unknown variables.

➢ If your calculation is not using the available mathematical tools,

but you are writing programming code and part of the calculation
involving solving simultaneous equations, you need to code
your solution routines.
You MUST know the routines.
➢
In this module, you will learn various methods that you can use.

BDA34003 Engineering Mathematics IV 12

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination Method
Gauss Elimination

In this method, the basic idea is modify the matrix into upper triangular.

Below is the basic idea in Gauss Elimination:

2 x 1+ 5 x 2=2
−4 x 1 + 10 x 2 =−5
4 x1 + 10 x 2 =4 ( multiply by - (-4/2) )
−4 x 1 + 10 x 2 =−5
20 x 2 =−1 2 x 1 + 5 x 2 =2
20 x 2=−1

Then solve
x 2 =−0.05
x 1 =1.125

BDA34003 Engineering Mathematics IV 13

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination Method
Gauss Elimination

Forward
2 x 1+ 5 x 2=2
−4 x 1 + 10 x 2 =−5

Eliminating 4 x1 + 10 x 2 =4 ( multiply by - (-4/2) )

the second
equation then
−4 x 1 + 10 x 2 =−5
replacing with 20 x 2 =−1
a new one 2 x 1 + 5 x 2 =2 Backward
20 x 2=−1

Then solve
x 2 =−0.05
x 1 =1.125

BDA34003 Engineering Mathematics IV 14

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination Method
Gauss Elimination

See how the matrix has been decomposed

from [−4
2 5
10 ]{ } { }
x1
x2
= 2
5
to [ ]{ } { }
2 5 x1
0 20 x 2
= 2
−1

[ A] { x }= { b } [U ] { x }= { f }

[ ]{ } { } [ ]{ } { }
a 11 a12 ⋯ a 1n x 1 b1 u11 u12 ⋯ u1n x1 f1
a 21 a 22 ⋯ a 2n x 2 0 u22 ⋯ u2n x 2
= b2 = f2
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
a m1 a m2 ⋯ a mn x n bn 0 0 ⋯ u mn x n fn

a ii ≠0
Upper triangular matrix
the solution is very simple, starting
from the last row then BACKWARD

BDA34003 Engineering Mathematics IV 15

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination Method
Gauss Elimination - forward elimination step

The routine can be concluded in general algorithm below:

Elimination step:

Step k ( 1⩽k ⩽n−1 ) n : the number of simultaneous equations

(k )
assume a kk ≠0
(k)
a ik
define multiplier mik = (k)
i=k + 1,⋯, n
a kk

eliminate (remove) the unknown x k from equation k + 1 through n

( k + 1) (k ) (k )
a ij =−mik a kj + aij

(k + 1) (k ) (k )
bij =−mik b k + bi i , j=k + 1,⋯, n

BDA34003 Engineering Mathematics IV 16

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination Method
Gauss Elimination - backward elimination step

Substitution step:

Using the last step from the elimination step [U ]=[ A](n−1) { f }= {b }(n−1)

fn
x n=
u nn

Backward step k =n−1, n−2,⋯, 1

[ ]
n
1
xk= g k − ∑ u kj x j
ukk j=k + 1

BDA34003 Engineering Mathematics IV 17

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Example 3-2

a) Write the simultaneous equations below in a complete matrix

form and in augmented matrix form

x1 + 2 x 2 =1
3 x 1+ 4 x 2 + 5 x 3=0
6 x 2 + 7 x 3 + 8 x 4 =2
10 x 3 + 11 x 4 =3

b) Find the unknown variables of simultaneous equations above

by using Gauss Elimination.

c) Double check your result by using Freemat.

BDA34003 Engineering Mathematics IV 18

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 x1 + 2 x 2 =1
3 x 1+ 4 x 2 + 5 x 3=0
6 x 2 + 7 x 3 + 8 x 4 =2
10 x 3 + 11 x 4 =3

[ ]{ } { }
1 2 0 0 x1 1
3 4 5 0 x2 = 0
0 6 7 8 x3 2
0 0 10 11 x 4 3

[ ∣]
1 2 0 0 1
3 4 5 0 0
0 6 7 8 2
0 0 10 11 3

BDA34003 Engineering Mathematics IV 19

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination steps:

[ ∣]
1 2 0 0 1 E1
3 4 5 0 0 E2
0 6 7 8 2 E3
0 0 10 11 3 E4

Step 1 : eliminate variable x 1 from equation (E2)

a21 3
m21= = =3
a11 1

[ ∣ ] [ ∣]
1(−3) 2(−3) 0(−3) 0(−3) 1(−3) + 1 2 0 0 1
3 4 5 0 0 0 −2 5 0 −3
0 6 7 8 2 0 6 7 8 2
0 0 10 11 3 0 0 10 11 3

BDA34003 Engineering Mathematics IV 20

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination steps:

[ ∣]
1 2 0 0 1 E1
0 −2 5 0 −3 E2
0 6 7 8 2 E3
0 0 10 11 3 E4

Step 2 : eliminate variable x 2 from equation (E3)

a 32 6
m32= = =−3
a 22 −2

[ ∣ ] [ ∣]
1 2 0 0 1 1 2 0 0 1
0(3) −2(3) 5(3) 0(3) −3(3) 0 −2 5 0 −3
+
0 6 7 8 2 0 0 22 8 −7
0 0 10 11 3 0 0 10 11 3

BDA34003 Engineering Mathematics IV 21

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination steps:

[ ∣]
1 2 0 0 1 E1
0 −2 5 0 −3 E2
0 0 22 8 −7 E3
0 0 10 11 3 E4

Step 3 : eliminate variable x 3 from equation (E4)

a 43
m43 = =10/22
a 33

[ ∣ ]
1 2 0 0 1
0 −2 5 0 −3
0(−10/22) 0(−10/22) 22(−10/22) 8(−10/ 22) −7(−10/22)
+
0 0 10 11 3

[ ∣ ]
1 2 0 0 1
0 −2 5 0 −3
0 0 22 8 −7
0 0 0 7.364 6.182
BDA34003 Engineering Mathematics IV 22
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Substitution steps:

[ ]{ } { }
1 2 0 0 x1 1
0 −2 5 0 x2
= −3
0 0 22 8 x3 −7
0 0 0 7.364 x4 6.182

6.182
x 4= =0.839
7.364
1
x3=
22
[ −7−8 x 4 ] =−0.623

1
x2=
−2
[−3−(5 x 3+ 0 x 4)]=−0.059
1
x1=
1
[1−(2 x 2+ 0 x3+ 0 x 4)]=1.117

BDA34003 Engineering Mathematics IV 23

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
Using Freemat

x
x 21
x3
x4

BDA34003 Engineering Mathematics IV 24

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Example 3-3

Write a simple code (function) of Gauss Elimination method

that can be used in Freemat, Matlab or Scilab. Use Example 3-2 to test the code.

function [x] = gaussElim(A,b)

% solving vector x from matrix equation A x = b
% filename gausselim.m

N = max(size(A));

% Perform Gaussian Elimination

for j=2:N;
for i=j:N;
m = A(i,j-1)/A(j-1,j-1);
A(i,:) = A(i,:) - A(j-1,:)*m;
b(i) = b(i) - m*b(j-1);
end;
End;

% Perform back substitution

x = zeros(N,1);
x(N) = b(N)/A(N,N);

for j=N-1:-1:1;
x(j) = (b(j)-A(j,j+1:N)*x(j+1:N))/A(j,j);
end ;

BDA34003 Engineering Mathematics IV 25

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination Method
Gauss Elimination with Pivoting

(k )
Gauss elimination requires a kk ≠0
In every step, the matrix must be checked
and rearrange if necessary to satisfy that
diagonal element is not 0.

(k )
a ik
Another operation mik = (k )
a kk
The pivot element in this operation should
Use the largest element to avoid round-off error.
The matrix should be rearranged.

Rearranging the matrix to use the largest

element and to satisfy non zero is called
PIVOTING
Partial Pivoting
Only rearrange the rows

Complete Pivoting (rarely used)

Rearranging the columns and the rows

BDA34003 Engineering Mathematics IV 26

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
m-code to implement Gauss Elimination with partial pivoting

Taken from: Applied Numerical Methods for Engineers and Scientists (3ed,2012, p.244) by Stephen C. Chapra

BDA34003 Engineering Mathematics IV 27

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Example 3-4

a) Write the simultaneous equations below in a complete matrix

form and in augmented matrix form

0.02 x 1 + 300 x 2+ 10 x 3 =9
500 x 1 − x 2 + 3 x 3 =6
4 x1 + 7 x 2 + x 3=2

b) Find the unknown variables of simultaneous equations above

by using Gauss Elimination with partial pivoting

c) Double check your result by using LO Calc

(you can also try your functions gausspivot.m and gausselim.m)

BDA34003 Engineering Mathematics IV 28

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
0.02 x 1 + 300 x 2+ 10 x 3 =9
500 x 1 − x 2 + 3 x 3 =6
4 x1 + 7 x 2 + x 3=2

[ ]{ } { }
0.02 300 10 x1 9
500 −1 3 x2 = 6
4 7 1 x3 2

[ ∣]
0.02 300 10 9
500 −1 3 6
4 7 1 2

BDA34003 Engineering Mathematics IV 29

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination steps:

[ ∣] [ ∣]
0.02 300 10 9 E1 500 −1 3 6 E1
500 −1 3 6 E2 rearrange 0.02 300 10 9 E2
4 7 1 2 E3 4 7 1 2 E3

Step 1 : eliminate variable x 1 from equation (E2)

a21 0.02
m21= = =410−5
a11 500

[ 500(−4 10−5 ) −1(−4 10−5 ) 3(−4 10−5 )

0.02
4
300
7
10
1 ∣ 6(−4 10−5 )
9
2 ] +
[
500 −1 3
0 300 10
4 7 1 ∣]
6
9
2

BDA34003 Engineering Mathematics IV 30

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination steps:

[ ∣]
500 −1 3 6 E1
0 300 10 9 E2
4 7 1 2 E3

Step 2-1 : eliminate variable x 1 from equation (E3)

a 31 4
m31= = =0.008
a 11 500

[ ∣ ] [ ∣ ]
500(−0.008) −1(−0.008) 3(−0.008) 6(−0.008) 500 −1 3 6
0 300 10 9 + 0 300 10 9
4 7 1 2 0 7.008 0.976 1.952

BDA34003 Engineering Mathematics IV 31

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Elimination steps:

[ ∣ ]
500 −1 3 6 E1
0 300 10 9 E2
0 7.008 0.976 1.952 E3

Step 2-2 : eliminate variable x 2 from equation (E3)

a 32 7.008
m32= = =0.0234
a 22 300

[ ∣ ]
500 −1 3 6
0(−0.0234) 300 (−0.0234) 10(−0.0234) 9(−0.0234)
+
0 7.008 0.976 1.952

[ ∣ ]
500 −1 3 6
0 300 10 9
0 0 0.742 1.742

BDA34003 Engineering Mathematics IV 32

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Substitution steps:

[ ∣ ] [ ]{ } { }
500 −1 3 6 500 −1 3 x1 6
0 300 10 9 0 300 10 x2 = 9
0 0 0.742 1.742 0 0 0.742 x3 1.742

1.742
x3= =2.348
0.742
1
x2=
300
[ 9−10 x 3 ] =−0.048

1
x1=
500
[ 6−(−1 x 2+ 3 x 3)]=−0.002

BDA34003 Engineering Mathematics IV 33

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
LO Calc result : Results from
gausselim and gausspivot

BDA34003 Engineering Mathematics IV 34

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 LU Factorization Methods

In Engineering, it is commonly said Decomposition rather than Factorization

LU Decomposition Methods
What is L U decomposition : In linear system [ A ] { x }= {b }

Lower triangular matrix [ L ] [U ] Upper triangular matrix

[ ] [ ]
l 11 0 ⋯ 0 u11 u12 ⋯ u1n
l l ⋯ 0 0 u 22 ⋯ u2n
[ L]= 21 22 [U ]=
⋮ ⋮ ⋮ ⋮ ⋮ ⋮
l m1 l m2 ⋯ l mn 0 0 ⋯ u mn

In linear system [ A] is rectangular m=n

BDA34003 Engineering Mathematics IV 35

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 LU Decomposition Methods
Solution Steps

[ A] { x } = {b }

1. Decomposition step [U ] [ L]

2. Solving steps
[L] {d } = {b }

{d } 2-1. Forward substitution

[U ] { x } = {d }

{x} 2-2. Backward substitution

BDA34003 Engineering Mathematics IV 36

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 LU Decomposition
Doolitlle Method

[ A] { x } = {b }

[L] [U ]

[ ] [ ]
1 0 ⋯ 0 u11 u12 ⋯ u1n
l 1 ⋯ 0 0 u 22 ⋯ u 2n
[ L]= 21 [U ]=
⋮ ⋮ ⋮ ⋮ ⋮ ⋮
l n1 l n2 ⋯ 1 0 0 ⋯ u nn

All diagonal terms of [L] are 1

BDA34003 Engineering Mathematics IV 37

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Example 3-5

a) Write the simultaneous equations below in a complete matrix form [ A ] { x }= {b }

2 x 1 −1 x 2 −2 x 3=−1
−4 x 1 +6 x 2 +3 x 3=13
−4 x 1 −2 x 2 +8 x 3 =−6

b) Decompose matrix [A] according to Doolittle method

c) Find the unknown variables of simultaneous equations.

d) Double check your result in Freemat or Matlab (need license)

by using the build-in lu function.

BDA34003 Engineering Mathematics IV 38

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :

[ ]{ } { }
2 −1 −2 x1 −1
[ A ] { x }= {b } −4 6 3 x 2 = 13
−4 −2 8 x3 −6

[ L ][ U ]=[ A ]

[ ] [ ][ ] [ ]
2 −1 −2 1 0 0 d e f d e f
−4 6 3 = a 1 0 0 g h = ad ae+ g af + h
−4 −2 8 b c 1 0 0 i bd be+ cg bf + h+ i

d =2 e=−1 f =−2
ad =−4 ae+ g=6 af + h=3
→ a=−2 → g=4 → h=−1
bd =−4 be+ cg=−2 bf + ch+ i=8
→ b=−2 → c=−1 → i=3

BDA34003 Engineering Mathematics IV 39

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :

[ ]
d =2 e=−1 f =−2
a=−2 g=4 h=−1
b=−2 c=−1 i=3

[ ][ ][ ]
2 −1 −2 1 0 0 2 −1 −2
−4 6 3 = −2 1 0 0 4 −1
−4 −2 8 −2 −1 1 0 0 3

[ L] [U ]

BDA34003 Engineering Mathematics IV 40

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (forward substitution) :
[L] {d } = {b }

[ ]{ } { }
1 0 0 d1 −1
−2 1 0 d 2 = 13
−2 −1 1 d 3 −6

−1
d 1= =−1
1
1
d 2=
1
[ 13−(−2 d 1 ) ]=11

1
d 3=
1
[−6−(−2 d 1−1 d 2 ) ]=3

BDA34003 Engineering Mathematics IV 41

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (backward substitution) :
[U ] { x } = {d }

[ ]{ } { }
2 −1 −2 x 1 −1
0 4 −1 x 2 = 11
0 0 3 x3 3

3
x 3 = =1
3
1
x2=
4
[ 11−(−1 x 3 ) ] =3

1
x 1=
2
[−1−(−1 x 2−2 x 3)]=2

BDA34003 Engineering Mathematics IV 42

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
BDA34003 Engineering Mathematics IV 43
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 LU Decomposition
Doolitlle Method
The routine can be concluded here
Decomposition step: Solution step:
% Forward substitution 'Ld=b' to solve d
for i=1:m d(1)=B(1)/L(1,1);
% Finding L for i=2:m
for k=1:i-1 d(i)=-L(i,1)*d(1);
L(i,k)=A(i,k); for k=2:i-1
for j=1:k-1 d(i)=d(i)-L(i,k)*d(k);
L(i,k)= L(i,k)-L(i,j)*U(j,k); end;
end d(i)=(B(i)+d(i))/L(i,i);
L(i,k) = L(i,k)/U(k,k); end;
end % Bacward substitution 'Ux=d' to solve x
% Finding U x(m)=d(m)/U(m,m);
for k=i:m i=m-1;
U(i,k) = A(i,k); q=0;
for j=1:i-1 while (i~= 0)
U(i,k)= U(i,k)-L(i,j)*U(j,k); x(i)=-U(i,m)*x(m);
end q=i+1;
end while (q~=m)
end x(i)=x(i)-U(i,q)*x(q);
% Assigning unity for all diagonal items of L q=q+1;
for i=1:m end;
L(i,i)=1; x(i)=(d(i)+x(i))/U(i,i);
end BDA34003 Engineering Mathematics i=i-1;IV 44
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
end;
 Example 3-6

Write matlab m function (doolittle.m) and test it to solve Example 3-5.

function[L,U,d,x]=doolittle(A,B) % Forward substitution 'Ld=b' to solve d
% prepared by Waluyo Adi Siswanto d=zeros(m,1);
[m n]=size(A); d(1)=B(1)/L(1,1);
if (m ~= n ) for i=2:m
disp ( 'LR2 error: Matrix must be square' ); d(i)=-L(i,1)*d(1);
return; for k=2:i-1
end; d(i)=d(i)-L(i,k)*d(k);
L=zeros(m,m); end;
U=zeros(m,m); d(i)=(B(i)+d(i))/L(i,i);
% LU Decomposition (DOOLITTLE METHOD) end;
for i=1:m
for k=1:i-1 % Bacward substitution 'Ux=d' to solve x
L(i,k)=A(i,k); x=zeros(m,1);
for j=1:k-1 x(m)=d(m)/U(m,m);
L(i,k)= L(i,k)-L(i,j)*U(j,k); i=m-1;
end q=0;
L(i,k) = L(i,k)/U(k,k); while (i~= 0)
end x(i)=-U(i,m)*x(m);
for k=i:m q=i+1;
U(i,k) = A(i,k); while (q~=m)
for j=1:i-1 x(i)=x(i)-U(i,q)*x(q);
U(i,k)= U(i,k)-L(i,j)*U(j,k); q=q+1;
end end;
end x(i)=(d(i)+x(i))/U(i,i);
end i=i-1;
for i=1:m end;
L(i,i)=1; BDA34003 Engineering Mathematics IV 45
End Waluyo Adi Siswanto (waluyo@uthm.edu.my)
BDA34003 Engineering Mathematics IV 46
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Exercise 3-7

a) Decompose matrix [A] of the simultaneous equations below

according to Doolittle method

[ ]{ } { }
1 2 3 4 x1 3
5 2 7 8 x2
= 0
9 0 3 2 x3 7
3 4 5 6 x4 8

b) Find the vector {d} in Doolittle step { d }=[3 −15 13.75 −1.833]T

c) Find the vector {x} { x }=[2 3.25 −5.5 2.75]T

d) Use your doolittle.m to check the answer (See next page)

BDA34003 Engineering Mathematics IV 47

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
BDA34003 Engineering Mathematics IV 48
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
LU Decomposition
Crout Method

[ A] { x } = {b }

[L] [U ]

[ ] [ ]
l 11 0 ⋯ 0 1 u12 ⋯ u1n
l l ⋯ 0 0 1 ⋯ u2n
[ L]= 21 22 [U ]=
⋮ ⋮ ⋮ ⋮ ⋮ ⋮
l n1 l n2 ⋯ l nn 0 0 ⋯ 1

All diagonal terms of [U] are 1

BDA34003 Engineering Mathematics IV 49

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 LU Decomposition
Crout Method
% LU Decomposition (CROUT METHOD)
The algorithm for decomposition step: L(1,1)=A(1,1);
for i=2:n
U(1,i)=A(1,i)/L(1,1);
i⩾ j , i=1, 2,⋯, n L(i,1)=A(i,1);
j−1 end
for i=2:n
Lij = Aij − ∑ L ik U kj for j=2:n
k −1 s=0;
if i<=j;
i< j , j=2, 3,⋯, n K=i-1;
else;
j−1
1 K=j-1;
U ij = Aij − ∑ Lik U kj end;
Lii k −1 for k=1:K;
s=s+L(i,k)*U(k,j);
end;
j=1 if j>i;
U(i,j)=(A(i,j)-s)/L(i,i);
Lij = Ai1 else;
L(i,j)=A(i,j)-s;
i=1 end
end
Aij end
U ij = for i=1:n
A11 U(i,i)=1;
end
% End of Decomposition (CROUT METHOD)
BDA34003 Engineering Mathematics IV 50
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Exercise 3-8

Modify your doolittle.m (example 3-6) to implement CROUT method and save as crout.m.
Try to solve example 3.7 by using crout.m. Try to solve by hand calculation according to Crout.

Simply replace the DOOLITTLE part

of Example 3-6 with the CROUT method
written in page 49.

BDA34003 Engineering Mathematics IV 51

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :

[ ]{ } { }
1 2 3 4 x1 3
5 2 7 8 x2
= 0
9 0 3 2 x3 7
3 4 5 6 x4 8

[ ] [ ][ ]
1 2 3 4 a 0 0 0 1 k l m
5 2 7 8 b c 0 0 0 1 n o
=
9 0 3 2 d e f 0 0 0 1 p
3 4 5 6 g h i j 0 0 0 1

[ ]
a=1 ak =2 al =3 am=4
b=5 bk + c=2 bl+ cn=7 bm+ co=8
=
d =9 dk + e=0 dl + en+ f =3 dm+ eo+ fp=2
g=3 gk + h=4 gl+ hn+ i=5 gm+ ho+ ip+ j=6

BDA34003 Engineering Mathematics IV 52

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :
a=1 ak =2 al=3 am=4
→ k =2 → l=3 → m=4
b=5 bk + c=2 bl+ cn=7 bm+ co=8
→ c=−8 → n=1 → o=1.5
d =9 dk + e=0 dl+ en+ f =3 dm+ eo+ fp=2
→ e=−18 → f =−6 → p=7/6
g=3 gk + h=4 gl + hn+ i=5 gm+ ho+ ip+ j=6
→ h=−2 → i=−2 → j=−2/3

[ ][ ][ ]
1 2 3 4 a=1 0 0 0 1 k =2 l=3 m=4
5 2 7 8 b=5 c=−8 0 0 0 1 n=1 o=1.5
=
9 0 3 2 b=9 e=−18 f =−6 0 0 0 1 p=7/6
3 4 5 6 b=3 h=−2 i=−2 j=−2/3 0 0 0 1

[ L] [U ]

BDA34003 Engineering Mathematics IV 53

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (forward substitution) :
[L] {d } = {b }

[ ]{ } { }
1 0 0 0 d1 3
5 −8 0 0 d2
= 0
9 −18 −6 0 d3 7
3 −2 −2 −2/ 3 d4 8

3
d 1 = =3
1
1
d 2=
−8
[ 0−(5 d 1 )]=1.875
1
d 3=
−6
[ 7−(9 d 1−18 d 2 ) ]=−2.292

1
d 4=
−2/3
[ 8−(3 d 1−2 d 2−2 d 3)]=2.751

BDA34003 Engineering Mathematics IV 54

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (backward substitution) :
[U ] { x } = {d }

[ ]{ } { }
1 2 3 4 x1 3
0 1 1 1.5 x 2 = 1.875
0 0 1 7/6 x 3 −2.292
0 0 0 1 x4 2.751

2.751
x 4= =2.751
1
1
x 3=
1
[−2.292−(7/6 x 4 )]=−5.502
1
x2=
1
[ 1.875−(1 x 3 + 1.5 x 4 ) ]=3.251

1
x 1=
1
[ 3−(2 x 2+ 3 x 3+ 4 x 4)]=2

BDA34003 Engineering Mathematics IV 55

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 LU Decomposition
Thomas Method – Variant 1

Engineering problems often lead to coefficient matrices that are sparsely populated,
meaning that most elements of the matrix are zero. If all the nonzero terms are clustered
about the leading diagonal, then the matrix is said to be banded.

[ A] = [ L] [U ]

[ ][ ][ ]
b1 c1 0 ⋯ 0 α1 0 0 ⋯ 0 1 β1 0 ⋯ 0
a 2 b2 c 2 ⋯ 0 a2 α2 0 ⋯ 0 0 1 β2 ⋯ 0
0 a3 ⋱ 0 = 0 a3 ⋱ 0 + 0 0 ⋱ 0
⋮ ⋮ bn−1 c n−1 ⋮ ⋮ α n−1 0 ⋮ ⋮ 1 βn−1
0 0 ⋯ an bn 0 0 ⋯ a n αn 0 0 ⋯ 0 1

This (the same with that in A)

is NOT requested
in CROUT method

BDA34003 Engineering Mathematics IV 56

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Exercise 3-9a

a) Decompose the banded matrix [A] of the simultaneous equations below

according to Thomas method (variant 1)

[ ]{ } { }
1 2 0 0 x1 1
3 4 5 0 x2 = 0
0 6 7 8 x3 2
0 0 10 11 x 4 3

b) Find the vector {x}

d) Use your crout.m to check the answer, and proof that Crout on banded matrix gives
the same result of L and U as defined by Thomas.

BDA34003 Engineering Mathematics IV 57

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :

[ ]{ } { }
1 2 0 0 x1 1
3 4 5 0 x2 = 0
0 6 7 8 x3 2
0 0 10 11 x 4 3

[ ] [ ][ ]
1 2 0 0 α1 0 0 0 1 β1 0 0
3 4 5 0 = 3 α2 0 0 0 1 β2 0
0 6 7 8 0 6 α3 0 0 0 1 β3
0 0 10 11 0 0 10 α 4 0 0 0 1

[ ]
α 1=1 α1 β1=2
3 3β1 + α 2=4 α 2 β2=5
=
6 6β2 + α3 =7 α3 β3 =8
10 β3+ α 4=11

BDA34003 Engineering Mathematics IV 58

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :
α 1=1 α 1 β1 =2
α 1=1 → β1 =2
3 3β1 + α2 =4 α2 β2=5
3 → α2 =−2 → β2 =−2.5
6 6β2 + α 3=7 α3 β 3
6 → α 3=22 → β3 =0.3636
10 β3+ α 4=11
10 → α 4 =7.3636

[ ][ ][ ]
1 2 0 0 1 0 0 0 1 2 0 0
3 4 5 0 3 −2 0 0 0 1 −2.5 0
=
0 6 7 8 0 6 22 0 0 0 1 0.3636
0 0 10 11 0 0 10 7.3636 0 0 0 1

[ L] [U ]

BDA34003 Engineering Mathematics IV 59

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (forward substitution) :
[L] {d } = {b }

[ ]{ } {}
1 0 0 0 d1 1
3 −2 0 0 d2
= 0
0 6 22 0 d3 2
0 0 10 7.3636 d4 3

1
d 1 = =1
1
1
d 2=
−2
[ 0−(3 d 1)]=1.5
1
d 3=
22
[ 2−(6 d 2 ) ] =−0.3182

1
d 4=
10
[ 3−(10 d 3)]=0.8395

BDA34003 Engineering Mathematics IV 60

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (backward substitution) :
[U ] { x } = {d }

[ ]{ } { }
1 2 0 0 x1 1
0 1 −2.5 0 x2 1.5
=
0 0 1 0.3636 x3 −0.3182
0 0 0 1 x4 0.8395

0.8395
x 4= =0.8395
1
1
x3=
1
[−0.3182−(0.3636 x 4)]=−0.6234
1
x 2=
1
[ 1.5−(−2.5 x 3 ) ]=−0.0585

1
x1=
1
[1−(2 x 2)]=1.117

BDA34003 Engineering Mathematics IV 61

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
Crout method on banded matrix gives the same result of L and U as defined by Thomas .

BDA34003 Engineering Mathematics IV 62

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 LU Decomposition
Thomas Method – Variant 2

Another variation in Thomas method

[ A] = [ L] [U ]

[ ][ ][ ]
d 1 c1 0 ⋯ 0 1 0 0 ⋯ 0 α1 c 1 0 ⋯ 0
a 2 d 2 c2 ⋯ 0 β2 1 0 ⋯ 0 0 α2 c2 ⋯ 0
0 a3 ⋱ 0 = 0 β3 ⋱ 0 + 0 0 ⋱ 0
⋮ ⋮ d n−1 c n−1 ⋮ ⋮ 1 0 ⋮ ⋮ α n−1 c n−1
0 0 ⋯ an dn 0 0 ⋯ βn 1 0 0 ⋯ 0 αn

This (the same with that in A)

is NOT requested
in DOOLITTLE method

BDA34003 Engineering Mathematics IV 63

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
The algorithm for variant 2:

α1 =b1 function [beta,alpha,c,d,x] = thomas(A,B)

% thomas method variant 2
beta=diag(A,-1);
i=2,3,⋯n
alpha=diag(A);
a1 c=diag(A,1);
βi = α N = length(B);
i−1
for k=2:N
α i=ai −βi c i−1 mult = beta(k-1)/alpha(k-1);
alpha(k) = alpha(k)-mult*c(k-1);
beta(k-1)=mult;
Forward substitution: B(k) = B(k)-mult*B(k-1);
d 1 =b1 end
d=B;
d i =bi −βi d i−1 B(N) = B(N)/alpha(N);
Backward substitution: for k=N-1:-1:1
B(k) = (B(k)-c(k)*B(k+1))/alpha(k);
x n =d n /α n end;
x i =d i−ci x i+ 1 x=B;;

BDA34003 Engineering Mathematics IV 64

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Exercise 3-9b

Do example 3-9a by using Thomas method variant 2. Try the thomas.m also by hand!

BDA34003 Engineering Mathematics IV 65

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :

[ ]{ } { }
1 2 0 0 x1 1
3 4 5 0 x2 = 0
0 6 7 8 x3 2
0 0 10 11 x 4 3

[ ] [ ][ ]
1 2 0 0 1 0 0 0 α1 2 0 0
3 4 5 0 = β1 1 0 0 0 α2 5 0
0 6 7 8 0 β2 1 0 0 0 α3 8
0 0 10 11 0 0 β3 1 0 0 0 α4

[ ]
α1 =1 2
α β =3 2β1 + α2 =4 5
= 1 1
α 2 β2 =6 5β2 + α 3=7 8
α3 β3 =10 8β3 + α 4 =11

BDA34003 Engineering Mathematics IV 66

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :
α1 =1 2
α1 =1
α 1 β1 =3 2β1+ α 2 =4 5
→ β1 =3 → α 2=−2
α 2 β 2=6 5β2 + α3 =7 8
→ β2 =−3 → α3 =22
α3 β3 =10 8β3+ α 4 =11
→ β3 =0.4545 → α 4 =7.3636

[ ][ ][ ]
1 2 0 0 1 0 0 0 1 2 0 0
3 4 5 0 3 1 0 0 0 −2 5 0
=
0 6 7 8 0 −3 1 0 0 0 22 8
0 0 10 11 0 0 0.4545 1 0 0 0 7.3636

[ L] [U ]

BDA34003 Engineering Mathematics IV 67

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (forward substitution) :
[L] {d } = {b }

[ ]{ } {}
1 0 0 0 d1 1
3 1 0 0 d2
= 0
0 −3 1 0 d3 2
0 0 0.4545 1 d4 3

1
d 1 = =1
1
1
d 2=
1
[ 0−(3 d 1 )]=−3
1
d 3=
1
[ 2−(−3 d 2 ) ] =−7

1
d 4=
1
[ 3−(0.4545 d 3)]=6.1818

BDA34003 Engineering Mathematics IV 68

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (backward substitution) :
[U ] { x } = {d }

[ ]{ } { }
1 2 0 0 x1 1
0 −2 5 0 x2
= −3
0 0 22 8 x3 −7
0 0 0 7.3636 x4 6.1818

6.1818
x 4= =0.8395
7.3636
1
x3=
22
[−7−(8 x 4 )]=−0.6234
1
x 2=
−2
[ −3−(5 x 3 ) ]=−0.0585

1
x1=
1
[1−(2 x 2)]=1.117

BDA34003 Engineering Mathematics IV 69

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Thomas Algorithm

The routine for Thomas (Variant 1) has been generalized for a faster calculation by using
a calculation table.

[ A] = [ L] [U ]

[ ][ ][ ]
d 1 e1 0 ⋯ 0 α1 0 0 ⋯ 0 1 β1 0 ⋯ 0
c2 d 2 e2 ⋯ 0 c2 α2 0 ⋯ 0 0 1 β2 ⋯ 0
0 c3 ⋱ 0 = 0 c3 ⋱ 0 + 0 0 ⋱ 0
⋮ ⋮ d n−1 e n−1 ⋮ ⋮ α n−1 0 ⋮ ⋮ 1 βn−1
0 0 ⋯ cn dn 0 0 ⋯ cn αn 0 0 ⋯ 0 1

BDA34003 Engineering Mathematics IV 70

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
Thomas algorithm table:

i 1 2 ... n
di
ei
ci
bi
α 1=d 1 α i =d i−ci β i−1
ei
β i=α
i

b1 bi−ci yi−1
y 1= α yi= αi
1

x n= y n x i = y i − β i x i+ 1

BDA34003 Engineering Mathematics IV 71

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Exercise 3-9c

[ ]{ } { }
1 2 0 0 x1 1
3 4 5 0 x2 = 0
0 6 7 8 x3 2
0 0 10 11 x 4 3

Find the vector {x} by using the Thomas algorithm table.

BDA34003 Engineering Mathematics IV 72

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 [ ]{ } { }
1 2 0 0 x1 1
3 4 5 0 x2 = 0
0 6 7 8 x3 2
0 0 10 11 x 4 3

i 1 2 3 4
di 1 4 7 11
ei 2 5 8
ci 3 6 10
bi 1 0 2 3
α 1=d 1 α i =d i−ci β i−1 1 4-3(2) 7-6(-2.5) 11-10(0.364)
=-2 =22 =7.36

ei 2/1 5/-2 8/22 0

β i=α =2 =-2.5 =0.364
i
1/1 0-3(1)/-2 2-6(1.5)/22 3-10(-0.318)/7.36
b1 bi−ci yi−1 =1 =1.5 =-0.318 =0.840
y 1= α yi= αi
1

1-2(- 1.5-(-2.5)(- -0.318- 0.840

xn= y n xi= yi − β i xi + 1 0.06) 0.624) 0.364(0.840
=1.12 =-0.06 =-0.624

BDA34003 Engineering Mathematics IV 73

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
LU Decomposition
Cholesky Method

[ A] { x } = {b }

T
[L] [ U ]=[ L ]

[ ] [ ]
l 11 0 ⋯ 0 l 11 l 21 ⋯ l nl
l l ⋯ 0 0 l 22 ⋯ l n2
[ L]= 21 22 [U ]=
⋮ ⋮ ⋮ ⋮ ⋮ ⋮
l n1 l n2 ⋯ l nn 0 0 ⋯ l nn

BDA34003 Engineering Mathematics IV 74

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
The decomposition algorithm can be expressed as:

If i= j (diagonal term)
% LU Decomposition (CHOLESKY METHOD)

√
j−1 for j = 1:n
L jj = A jj − ∑ L 2jk must be temp = A(j,j) - dot(A(j,1:j-1),A(j,1:j-1));
k =1 positive if temp < 0.0
error(’Matrix is not positive definite’)
j=2, 3,⋯, n end
A(j,j) = sqrt(temp);
For non diagonal term for i = j+1:n
A(i,j)=(A(i,j) - dot(A(i,1:j-1),A(j,1:j-1)))/A(j,j);

( )
j−1 end
A jj −∑ Lik L jk end
k =1 L = tril(A);
L jj = U=transpose(L);
L jj
% End of Decomposition (CHOLESKY METHOD)

j=2, 3,⋯, n−1

i= j+ 1, j+ 2,⋯, n

BDA34003 Engineering Mathematics IV 75

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Exercise 3-10

Modify your doolittle.m (example 3-6) to implement CHOLESKY method

and save as cholesky.m.

Simply replace the DOOLITTLE part

of Example 3-6 with the CHOLESKY method
written in page 75.

BDA34003 Engineering Mathematics IV 76

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Exercise 3-11

a) Decompose matrix [A] of the simultaneous equations below

according to Cholesky method

[ ]{ } { }
8 2 3 4 x1 1
2 5 1 0 x2 = 0
3 1 7 3 x3 0
4 0 3 11 x 4 9

b) Find the vector {x}

c) Use your cholesky.m to check the answer

BDA34003 Engineering Mathematics IV 77

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :

[ ]{ } { }
8 2 3 4 x1 1
2 5 1 0 x2 = 0
3 1 7 3 x3 0
4 0 3 11 x 4 9

[ ] [ ][ ]
8 2 3 4 a 0 0 0 a b d g
2 5 1 0 b c 0 0 0 c e h
=
3 1 7 3 d e f 0 0 0 f i
4 0 3 11 g h i j 0 0 0 j

[ ]
a 2 =8
ab=2 b2 + c 2 =5
= 2 2 2
ad =3 bd + ce=1 d + e + f =7
2 2 2 2
ag=4 bg + ch=0 dg+ eh+ if =3 g + h + i + j =11

BDA34003 Engineering Mathematics IV 78

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Decomposition step :

a 2 =8
→ a=2.8284
2 2
ab=2 b + c =5
→ b=0.7071 → c=2.1213
2 2 2
ad =3 bd + ce=1 d + e + f =7
→ d =1.0607 → e=0.1179 → f =2.4210
ag=4 bg+ ch=0 dg+ eh+ if =3 g 2 + h 2+ i 2 + j 2=11
→ g=1.4142 → h=−0.4714 → i=0.6425 → j=2.8922

[ ][ ][ ]
8 2 3 4 2.8284 0 0 0 2.8284 0.7071 1.0607 1.4142
2 5 1 0 0.7071 2.1213 0 0 0 2.1213 0.1179 −0.4714
=
3 1 7 3 1.0607 0.1179 2.4210 0 0 0 2.4210 0.6425
4 0 3 11 1.4142 −0.4714 0.6425 2.8922 0 0 0 2.8922

[ L] [U ]

BDA34003 Engineering Mathematics IV 79

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (forward substitution) :
[L] {d } = {b }

[ ]{ } { }
2.8284 0 0 0 d1 1
0.7071 2.1213 0 0 d2
= 0
1.0607 0.1179 2.4210 0 d3 0
1.4142 −0.4714 0.6425 2.8922 d 4 9

1
d 1= =0.3536
2.8284
1
d 2=
2.1213
[ 0−(0.7071 d 1)]=−0.1179
1
d 3=
2.4210
[ 0−(1.0607 d 1+ 0.1179 d 2 ) ]=−0.1492

1
d 4=
2.8922
[ 9−(1.4142 d 1−0.4714 d 2+ 0.6425 d 3 ) ]=2.9528

BDA34003 Engineering Mathematics IV 80

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Solving step (backward substitution) :
[U ] { x } = {d }

[ ]{ } { }
2.8284 0.7071 1.0607 1.4142 x1 0.3536
0 2.1213 0.1179 −0.4714 x2
= −0.1179
0 0 2.4210 0.6425 x3 −0.1492
0 0 0 2.8922 x4 2.9528

2.9528
x 4= =1.0210
2.8922
1
x 3=
2.4210
[−0.1492−(0.6425 x 4)]=−0.3326
1
x2=
2.1213
[ −0.1179−(0.1179 x 3−0.4714 x 4 ) ] =0.1898

1
x1=
2.8284
[ 0.3536−(0.7071 x 2+ 1.0607 x 3+ 1.4142 x 4)]=−0.3082

BDA34003 Engineering Mathematics IV 81

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
BDA34003 Engineering Mathematics IV 82
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Iteration Method
Gauss-Seidel

In simultaneous equations
a 11 x 1+ a12 x 2+ ⋯+ a 1n x n=b1
a 21 x 1 + a 22 x 2 + ⋯+ a 2n x n =b2
⋮ ⋮
a n1 x 1 + a n2 x 2+ ⋯+ a nn x n=bn

and using initial guess at k=1, the iteration can be started

(k + 1) b1 −a 12 x(k2 )−a 13 x (k3 )− ⋯ −a 1n x (kn )

x 1 =
a11

( k + 1) (k ) (k)
(k + 1) b2 −a 21 x 1 + a 22 x 2 − ⋯ −a 2n x n
x2 =
a 22
(k + 1) ( k+ 1) (k )
(k + 1) bn −a 2n1 x 1 + an2 x 2 − ⋯ −a nn x n
xn =
a nn

BDA34003 Engineering Mathematics IV 83

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Iteration Method
Gauss-Seidel: convergence criterion

Requirement for stable converge criterion: [A] must be diagonally dominant

n

|a ii |⩾∑ |aij |
j =1
j≠i

[ ]{ } { }
See this example 1 7 4 x1 2
not diagonally dominant
1 0 9 x2 = 3
6 0 1 x3 5

[ ]{ } { }
6 0 1 x1 5 |6|⩾|0|+ |1|
Rearrange 1 7 4 x2 = 2 |7|⩾|1|+ |4|
1 0 9 x3 3 |9|⩾|1|+ |0|

diagonally dominant

BDA34003 Engineering Mathematics IV 84

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Iteration Method
Gauss-Seidel

In conclusion, for a general n equations

the Gauss-Seidel iterative method is defined as:

k =0,1,⋯, n

i=1,2,⋯, n

1
[ ]
i −1 n
( k + 1) (k + 1)
x i = bi − ∑ aij x j − ∑ aij x(kj )
a ii j =1 j=i + 1

BDA34003 Engineering Mathematics IV 85

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Iteration Method
Gauss-Seidel

function x = GaussSeidel(A,b,es,maxit)
% Gauss-Siedel method for i = 1:n
if nargin<4, maxit=50; end C(i,1:n) = C(i,1:n)/A(i,i);
if nargin<3, es=0.00001; end end
[m,n] = size(A); for i = 1:n
if m~=n, error('Matrix A must be square'); end d(i) = b(i)/A(i,i);
C = A; end
for i = 1:n iter = 0;
C(i,i) = 0; while (1)
x(i) = 0; xold = x;
end for i = 1:n
x = x'; x(i) = d(i)-C(i,:)*x;
for i = 1:n if x(i) ~= 0
for j = 1:n; ea(i) = abs((x(i) - xold(i))/x(i)) * 100;
H = abs(A(i,j)); end
Check(i) = abs(A(i,i)) - sum(H); end
if Check(i) < 0 iter = iter+1;
fprintf('The matrix is not strictly diagonally dominant at row if max(ea)<=es | iter >= maxit, break, end
%2i\n\n',i)) end
end
end
end

BDA34003 Engineering Mathematics IV 86

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Example 3-12

[ ]{ } { }
0 −2 −7 −3 x1 3
4 0 1 −9 x2
= 3
3 1 1 0 x3 2 a) Check the criterion of diagonally dominant
−1 −5 2 1 5 on every row
x4
b) If not satisfied then rearrange and check
the criterion

c) Write down the iteration equation for

every row

d) Do the Gauss-Seidel iteration by using

GaussSeidel.m function

BDA34003 Engineering Mathematics IV 87

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
[ ]{ } { }
0 −2 −7 −3 x1 3 ∣0∣ > ∣2∣+ ∣7∣+ ∣3∣ → X
4 0 1 −9 x2 X
= 3
∣0∣ > ∣4∣+ ∣1∣+ ∣9∣ →
3 1 1 0 x3 2 ∣1∣> ∣3∣+ ∣1∣+ ∣0∣ → X
−1 −5 2 1 x4 5 ∣1∣> ∣1∣+ ∣5∣+ ∣2∣ → X

Then rearrange the matrix

[ ]{ } { }
3 1 1 0 x1 2 |3|⩾ |1|+ |1|+ |0| → OK
−1 −5 2 1 x2 OK
= 5
|5|⩾ |1|+ |2|+ |1| →
0 −2 −7 −3 x3 3 |7|⩾ |0|+ |2|+ |3| → OK
4 0 1 −9 x4 3 |9|⩾ |4|+ |0|+ |1| → OK

Now you can use Gauss-Seidel

BDA34003 Engineering Mathematics IV 88

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
[ ]{ } { }
3 1 1 0 x1 2
−1 −5 2 1 x2
= 5
0 −2 −7 −3 x3 3 (k)
x(1k)
(k ) (k )
Initial values x2 x3 x4
4 0 1 −9 x4 3

(k ) (k) (k ) (k ) (k )
(k + 1) 2−1 x 2 −1 x 3 −0 1 x 4 2− x 2 − x 3
x1 = =
3 3
(k + 1) (k ) (k ) (k + 1) (k ) (k)
(k + 1) 5−(−1) x 1 −2 x 3 −1 x 4 5+ x 1 −2 x 3 − x 4
x2 = =
−5 −5
( k + 1) (k + 1) (k ) (k + 1) (k)
( k + 1) 3−(0) x 1 −(−2) x 2 −(−3) x 4 3+ 2 x 2 + 3 x 4
x 3 = =
−7 −7
( k + 1) (k + 1) (k + 1) ( k + 1) ( k + 1)
(k + 1) 3−(4) x1 −(0) x 2 −(1) x 3 3−4 x 1 − x 3
x 4 = =
−9 −9

BDA34003 Engineering Mathematics IV 89

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 You can use a table to make it easier:

BDA34003 Engineering Mathematics IV 90

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
BDA34003 Engineering Mathematics IV 91
Waluyo Adi Siswanto (waluyo@uthm.edu.my)
Student Activity

BDA34003 Engineering Mathematics IV 92

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Problem 3a

Based on your understanding of the methods you have learned in this lecture module,
prepare a summary table explaining the requirement of each method if any.

BDA34003 Engineering Mathematics IV 93

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Problem 3b

Dazzling Florist offers three sizes of flower arrangements containing roses, daisies
and carnations during Mother’s Day. Each small size arrangement contains one rose,
three daisies and three carnations. The medium size arrangement contains two roses,
four daisies and six carnations. Meanwhile, each large size arrangement contains four
roses, eight daisies and six carnations. The owner of the florist noted that she used a
total of 24 roses, 50 daisies and 48 carnations in filling orders for these three different
types of flower arrangement sizes.

a. Write the simultaneous mathematical equations in a matrix form

b. Determine the number of different arrangement sizes, by using

- Doolittle method
- Crout Method
- Thomas method variant 1
- Thomas algorithm
- Gauss Seidel

If you cannot use a particular method you have to explain why.

c) Verify your result using Smath/Speadsheet/Freemat

BDA34003 Engineering Mathematics IV 94

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
 Problem 3c

a) Solve the system of linear equations: x 1 +2 x 2 +4 x 3=360

2 x 1 +6 x 2 +3 x 3=600
4 x 1 +2 x 2 + x 3=600

By using Gauss Seidel Iteration method and verify the results with SMath

b) Solve the system of linear equations: 5 x 1 +2 x 2 =2

2 x 1 +5 x 2 +2 x 3 =6
2 x 2 +5 x 3 =8

By using Cholesky method and verify the results with Spreadsheet

BDA34003 Engineering Mathematics IV 95

Waluyo Adi Siswanto (waluyo@uthm.edu.my)
BDA34003 Engineering Mathematics IV 96
Waluyo Adi Siswanto (waluyo@uthm.edu.my)