Ch.-4 Solution of A System of Linear Equations
Ch.-4 Solution of A System of Linear Equations
Iterative methods:
The direct methods lead to exact solutions in many cases but are subject to errors
due to round off and other factors. In the iterative method, an approximation to the
true solution is assumed initially to start the method. By applying the method
repeatedly, better and better approximations are obtained. For large systems,
iterative methods are faster than direct methods and round-off errors are also
smaller. Any error made at any stage of computation gets automatically corrected
in the subsequent steps.
We will discuss two iterative methods:
(i) Gauss-Jacobi method
(ii) Gauss-Seidel method
. .
. .
. .
If the diagonal elements are non-zero, each equation is rewritten for the
corresponding unknown, that is, the first equation is rewritten with x1 on the left
hand side, the second equation is rewritten with x 2 on the left hand side and so on
as follows
n
c2 a2 j x j
j 1
j2
x2
a 22
.
.
n
c n 1 a
j 1
n 1, j xj
j n 1
x n 1
a n 1,n 1
n
c n a nj x j
j 1
jn
xn
a nn
Now to find xi ’s, one assumes an initial guess for the xi ’s and then uses the
rewritten equations to calculate the new estimates. Remember, one always uses
the most recent estimates to calculate the next estimates, xi .
The above iteration process is continued until two successive approximations are
nearly equal.
Working Rule:
(i) Arrange the equations in such a manner that the leading elements are large in
magnitude in their respective rows satisfying the conditions
(ii) Express the variables having large coefficients in terms of other variables.
( x1 , y1 , z1 ) using equations
c1 a12 x 2 a13 x3 a1n x n
x1
a11
c 2 a 21 x1 a 23 x3 a 2 n x n
x2
a 22
c n 1 a n 1,1 x1 a n 1, 2 x 2 a n 1,n 2 x n 2 a n 1,n x n
x n 1
a n 1,n 1
c n a n1 x1 a n 2 x 2 a n ,n 1 x n 1
xn
1. a nn
(iv) Start the iteration 2 by putting the values of ( x, y, z) as ( x1, y1, z1 ) and obtain
( x2 , y2 , z2 ) .
(v) The above iteration process is continued until two successive approximations
are nearly
equal.
Example: Solve the system of linear equations by Gauss- Jacobi method correct
up to 2-
decimal places.
6 x 2 y z 4 , x 5 y z 3 , 2 x y 4 z 27
Solution:
1
x (4 2 y z )
6
1
y (3 x z ) ......................................(1)
5
1
z (27 2 x y )
4
1
x2 (4 2(0.6) 6.75) 1.59
6
1
y2 (3 0.67 6.75) 0.884
5
1
z2 (27 2(0.67) 0.6) 6.265
4
Working Rule:
(i) Arrange the equations in such a manner that the leading elements are large in
magnitude in their respective rows satisfying the conditions
(ii) Express the variables having large coefficients in terms of other variables.
(v) The above iteration process is continued until two successive approximations
are nearly
equal.
Example: Find the solution to the following system of equations using the Gauss-
Seidel method.
12 x1 3x2 5x3 1
x1 5x2 3x3 28
3x1 7 x2 13x3 76
Solution:
a 22 5 5 a 21 a 23 1 3 4
and the inequality is strictly greater than for at least one row. Hence, the solution
should converge using the Gauss-Seidel method.
28 x1 3x3
x2
5
76 3x1 7 x 2
x3
13
Iteration 1
1 3 0 5 1
x11
12
0.50000
28 0.50000 3 1
x21
5
4.9000
76 3 0.50000 7 4.9000
x31
13
3.0923
Iteration 2
1 3 4.9000 5 3.0923
x12
12
0.14679
28 0.14679 3 3.0923
x2 2
5
3.7153
76 3 0.14679 7 3.7153
x32
13
3.8118
Iteration x1 x2 x3
Examples: Solve
(1) 5x y z 10;2 x 4 y z 14; x y 8z 20 by using Gauss-Seidel method
correct upto three decimal places. ( x 2; y 2; z 2)
(2) 6 x y z 105;4 x 8 y 3z 155;5x 4 y 10 z 65 by using Gauss-Seidel
method correct upto three decimal places. ( x 15; y 10; z 5)
(3) 20 x y 2 z 17;3x 20 y z 18;2 x 3 y 20z 25 by using Gauss-Jacobi
method correct upto three decimal places. ( x 1; y 1; z 1)
(4) 28x 4 y z 32;2 x 17 y 4z 35; x 3 y 10z 24 by using Gauss-Jacobi
method correct upto two decimal places. ( x 0.99; y 1.51; z 1.85)
We all are aware about algebraic equation, so now we will discuss about
transcendental functions and equations.
1) Bisection Method
3) Newton-Raphson Method
x xu
x
x xu x
f(x)
f(x)
xl x
x x xu
xu
Fig.1 Fig.2
Fig.1 At least one root exists between two points if the function is real,
continuous, and changes sign.
Fig.2 If function f (x) does not change sign between two points, roots of
f ( x) 0 may still exist between the two points.
Fig.3 If the function f (x) does not change sign between two points, there may not
be any roots f ( x) 0 between the two points.
f(x)
xu
x
x
Fig .4 If the function f (x) changes sign between two points, more than one root for
f ( x) 0 may exist between the two points.
The bisection method is always convergent. Since the method brackets the
a)
root, the method is guaranteed to converge.
b) As iterations are conducted, the interval gets halved. So one can guarantee
the decrease in the error in the solution of the equation.
Drawbacks of Bisection Method:
it will be unable to find the lower guess, x , and upper guess, x u , such that
f ( x ) f ( xu ) 0
1. Choose x and x u as two guesses for the root such that f ( x ) f ( x u ) 0 , or in other words, f (x)
changes sign between x and x u .
2. Estimate the root, x m of the equation f ( x) 0 as the mid-point between x and x u as
x xu
xm =
2
xmnew - xmold
a = x 100
xmnew
where
5. Compare the absolute relative approximate error a with the pre-specified relative error tolerance s .
If a s , then go to Step 3, else stop the algorithm. Note one should also check whether the number
of iterations is more than the maximum number of iterations allowed. If so, one needs to terminate the
algorithm and notify the user about it.
Example: Find a root of the equation x3 4 x 9 0 using Bisection Method
correct up to 3 decimal places.
Sol: Let f ( x) x3 4x 9
Since f (2) is –ve and f (3) is +ve , a root lies between 2 and 3.
23
Therefore, the first approximation to the root is x1 2.5
2
2.5 3
Thus the second approximation to the root is x2 2.75
2
2.5 2.75
Thus the third approximation to the root is x3 2.625
2
The root lies between x2 and x3 thus the fourth approximation to the root is
2.625 2.75
x4 2.6875
2
Example:
(1)Find the real root of the equation 3x cos x 1, correct up to four decimal
places. ( x 0.6071)
(2) Find the real root of the equation xe x 1, correct up to four decimal places.
( x 0.5671)
f(xu)
xr
xl
x
xu
f(xl)
False-position method.
A shortcoming of the bisection method is that in dividing the interval from xl to xu
into equal halves, no account is taken of the magnitude of f(xi) and f(xu). Indeed, if
f(xi) is close to zero, the root is more close to xl than x0.
The false position method uses this property:
A straight line joins f(xi) and f(xu). The intersection of this line with the x-axis
represents an improvement estimate of the root. This new root can be computed as:
f xl f xu
xr xl xr xu
a 1. Define the first interval (a, b) such that solution exists between them. Check f (a) f (b) 0 .
2. Compute the first estimate of the numerical solution xr using the above equation.
3. Find out whether the actual solution is between a and xr or xr and b .This is accomplished by checking
Pitfalls of the false-position method:
the sign of the product f (a) f ( xr )
Although, the false position method is an improvement of the bisection method. In
- If f (a) f ( xr ) 0 , the solution is between a and xr .
some cases, the bisection method will converge faster and yields to better results
- If f (a) f ( xr ) 0 , the solution is between xr and b .
4. Select the subinterval that contains the solution (a, xr ) or ( xr , b) is the new interval (a, b) and go to
step no.2 and repeat the process. The method of False Position always converges to an answer, provided a root is
initially bracketed in the interval (a, b) .
f(x)
Slow convergence of the false-position method.
Example Using the False Position method, find a root of the function
f ( x) e x 3 x 2 to an accuracy of 5 digits. The root is known to lie between 0.5
and 1.0.
Sol:
We apply the method of False Position with a = 0.5 and b = 1.0. and the equation
is
af (b) bf (a)
xr
f (b) f (a)
The calculations based on the method of False Position are shown in the Table
n A b f ( a) f (b) xr f ( xr )
1 0.5 1 0.89872 -0.28172 0.88067 0.08577
2 0.88067 1 0.08577 -0.28172 0.90852 0.00441
3 0.90852 1 0.00441 -0.28172 0.90993 0.00022
4 0.90993 1 0.00022 -0.28172 0.91000 0.00001
5 0.91000 1 0.00001 -0.28172 0.91001 0
(2) Find a positive root of x log10 x 1.2 correct up to three decimal places.
( x 2.740)
3) Newton-Raphson method:
Newton-Raphson method is based on the principle that if the initial guess of the
root of f(x)=0 is at xi, then if one draws the tangent to the curve at f(xi), the point
xi+1 where the tangent crosses the x-axis is an improved estimate of the root (Figure
3.12).
f(xi ) 0
i)==
f (x
xi xi 1
which gives
f(xi )
xi 1 = xi -
f'(xi )
f(xi)
x i, f xi
f(xi+1)
xi+2 xi+1 xi X
4. Compare the absolute relative approximate error, a with the pre-specified relative error
tolerance, s . If a > s , then go to step 2, else stop the algorithm. Also, check if the
number of iterations has exceeded the maximum number of iterations.
f(xi )
xi 1 xi
f (xi )
Consequently if an iteration value, x i is such that f xi 0 , then one can face
division by zero or a near-zero number. This will give a large magnitude for the
next value, xi+1.
- Root jumping: In some case where the function f(x) is oscillating and has a
number of roots, one may choose an initial guess close to a root. However, the
guesses may jump and converge to some other root.
Sol :
Let f ( x) x3 x 1
Let x0 1
f '( x) 3x2 1
f(xi )
By the Newton-Raphson method , xi 1 xi
f (xi )
f ( x0 ) f (1) 1
f '( x0 ) f '(1) 4
f(x0 ) 1
x1 x0 1 0.75
0)
f (x 4
f ( x1 ) f (0.75) 0.171875
f '( x1 ) f '(0.75) 2.6875
f(x1 ) 0.171875
x2 x1 0.75 0.68605
1)
f (x 2.6875
f ( x2 ) f (0.68605) 0.00894
f '( x2 ) f '(0.68605) 2.41198
f(x2 ) 0.00894
x3 x2 0.68605 0.68234
2)
f (x 2.41198
f ( x3 ) f (0.68234) 0.000028
f '( x3 ) f '(0.68234) 2.39676
f(x3 ) 0.000028
x4 x3 0.68234 0.68233
3)
f (x 2.39676
Since x3 and x4 are same up to four decimal places, the root is 0.6823.
Example:
(1) Find the root of the function x4 x 10 0 correct up to three decimal places.
( x 2.7406)
(2) Find the root of the function e x 3x correct up to three decimal places.
( x 1.512)
We can derive the following useful results from the Newton-Raphson’s Formula
1
1) Iteration formula for :
N
1 1 1
Let x N N 0
N x x
1 1
Also let f ( x) N and f '( x)
x x2
By the Newton-Raphson method ,
1
N
xi 1 xi
f(xi )
xi xi
i)
f (x 1
2
xi
1
xi xi 2 N
xi
xi (2 Nxi )
xi 1 xi
f(xi )
xi
xi 2 N
i)
f (x 2 xi
1 N
xi xi
2 xi
1 N
xi
2 xi
1
3) Iteration formula for :
N
1 1 1
Let x x2 x2 0
N N N
1
Also let f ( x) x 2 and f '( x) 2 x
N
By the Newton-Raphson method ,
2 1
xi
xi
f(xi ) N
xi 1 xi
i)
f (x 2 xi
2 1
2 xi xi N
2
2 xi
1 1
xi
2 xi N
k
4) Iteration formula for N:
Let x k N N x x N 0
k k
xi 1 xi
f(xi )
xi
xi k N
i)
f (x
kxi k 1
( xi )(kxi k 1 ) xi k N
kxi k 1
1 N
(k 1) xi k 1
k xi
1
Sol: (i)
31
Then
x1 x0 (2 31x0 ) 0.03(2 31(0.03)) 0.0321
x2 x1 (2 31x1 ) 0.0321(2 31(0.0321)) 0.032257
x3 x2 (2 31x2 ) 0.032257(2 31(0.032257)) 0.03226
(ii ) 28
1 N
Taking N 24 in the equation xi 1 xi
2 x i
1 28
We have xi 1 xi , Since the value of 25 5 , we take x0 5 .
2 xi
Then
1 28 1 28
x1 x0 5 5.3
2 x0 2 5
1 28 1 28
x2 x1 5.3 5.29151
2 x1 2 5.3
1 28 1 28
x3 x2 5.29151 5.29150
2 x2 2 5.29151
(iii ) 1/ 14
1 1
Taking N 14 in the equation xi 1 xi
2
xN i
1 1
We have xi 1 xi , Since the value of 1/ 16 1/ 4 0.25 , we take x0 0.25 .
2
x 14 i
Then
1 1 1 1
x1 x0 (0.25) 0.26785
2 x014 2 (0.25)14
1 1 1 1
x2 x1 (0.26785) 0.2672618
2 x114 2 (0.26785)14
1 1 1 1
x3 x2 (0.2672618) 0.2672612
2 x214 2 (0.2672618)14
1 24 1
We have xi 1 (2) xi 2 , Since the value of 27 3 3 , we take x0 3 .
3 x i
Then
1 24 1 24
x1 (2) x0 2 (2)(3) 2 2.8889
3 x0 3 (3)
1 24 1 24
x2 (2) x1 2 (2)(2.8889) 2
2.88451
3 x1 3 (2.8889)
1 24 1 24
x3 (2) x2 2 (2)(2.88451) 2
2.8845
3 x2 3 (2.88451)