A

Duhok Polytechnic University

Technical College of Engineering
AR
Petrochemical Engineering
Energy

Subject: Numerical Analysis

Lecturer’s Name: Dr. Abdulaziz RASHID

Academic Year: 2021/2022

(Second Semester)

Core textbook:
1- Kreyszig, E., Kceyszig, H., and Norminton, E.J.,
”Advanced engineering mathematics”2011.
2- Sauer, Timothy., ” Numerical Analysis”2nd edition, 2014.
Websites: https://www.khanacademy.org/math/precalculus/
x9e81a4f98389efdf:matrices

Dr. Abdulaziz Rashid 1

Contents

0.1 Bisection Method for Solving f (x) = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . 3

0.2 Fixed-Point Iteration for Solving f (x) = 0 . . . . . . . . . . . . . . . . . . . . . . . . 6
0.3 Newton’s Method for Solving f (x) = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . 10

A
0.4 Secant Method for Solving f (x) = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
AR

2
Chapter 2
Solution of Equations by Iteration

The function f (x) has a root at x = s if f (s) = 0.

A
To solve f (x) = 0 when there is no formula for the exact solution available,
we can use an approximation method, such as an iteration method. This is
a method in which we start from an initial guess x0 (which may be poor) and
AR
compute step by step (in general better and better) an approximations x1.
x2. . . of an unknown solution of f (x) = 0. We discuss four such methods
that are of particular practical importance.

1. Bisection Method for Solving f (x) = 0

2. Fixed-Point Iteration for Solving f (x) = 0
3. Newton’s Method for Solving f (x) = 0
4. Secant Method for Solving f (x) = 0

0.1 Bisection Method for Solving f (x) = 0

THEOREM: Let f be a continuous function on [a, b], satisfying f (a) f (b) <
0. Then f has a root between a and b, that is, there exists a number s
satisfying a < s < b and f (s) = 0.
In Figure (1), f (0) f (1) = (−1)(1) < 0. There is a root just to the left of
0.7. How can we refine our first guess of the root’s location to more decimal
places?

3
 Y

x
0.5 1

-1

A
Figure 1: A plot of f (x) = x3 + x − 1, the function has a root between 0.6 and 0.7

Bisection Method (Algorithm)

Given initial interval [a, b] such that f (a) f (b) < 0
AR
while (b − a)/2 > TOL
c = (a + b)/2
if f (c) = 0, stop, end
if f (a) f (c) < 0
b=c
else
a=c
end
end
The final interval [a, b] contains a root. The approximate root is
(a + b)/2. Check the value of the function at the midpoint c = (a + b)/2
of the interval. Since f (a) and f (b) have opposite signs, either f (c) = 0
(in which case we have found a root and are done), or the sign of f (c) is
opposite the sign of either f (a) or f (b). If f (c) f (a) < 0, for example, we
are assured a solution in the interval [a, c], whose length is half that of
the original interval [a, b]. If instead f (c) f (b) < 0, we can say the same
Dr. Abdulaziz Rashid 4
of the interval [c, b]. In either case, one step reduces the problem to
finding a root on an interval of one-half the original size. This step can be
repeated to locate the function more and more accurately.
A solution is bracketed by the new interval at each step, reducing the
uncertainty in the location of the solution as the interval becomes smaller.
An entire plot of the function f is not needed. We have reduced the work
of function evaluation to only what is necessary.
EXAMPLE
Find a root of the function f (x) = x3 + x − 1 by using the Bisection

A
Method on the interval [0,1], Tolerance (TOL) = 0.0001.
Solution
Since f (a0) f (b0) = (−1)(1) < 0, so a root exists in the interval.
The interval midpoint is c0 = a0+b = 0.5. The first step consists of
AR
0
2
evaluating f (0.5) = −0.375 < 0 and choosing the new interval
[a1, b1] = [0.5, 1], since f (0.5) f (0) > 0. The second step consists of
evaluating c1 = a1+b 2
1
= 1.5
2 = 0.75, f (c1 ) = f (0.75) = 0.171875 > 0,
leading to the new interval [a2, b2] = [0.5, 0.75] since f (0.75) f (0.5) > 0.
Further steps are given in the following table:
n an f (an ) bn f (bn ) cn f (cn ) Sign f (b − a)/2
0 0 −1 1 1 0.5 −0.375 + 0.5
1 0.5 −0.375 1 1 0.75 0.171875 − 0.25
2 0.5 −0.375 0.75 0.171875 0.625 −0.130859375 + 0.125
3 0.625 −0.130859375 0.75 0.171875 0.6875 0.012451172 − 0.0625
4 0.625 −0.130859375 0.6875 0.012451172 0.65625 −0.061126709 + 0.03125
5 0.65625 −0.061126709 0.6875 0.012451172 0.671875 −0.024829865 + 0.015625
6 0.671875 −0.024829865 0.6875 0.012451172 0.6796875 −0.006313801 + 0.0078125
7 0.6796875 −0.006313801 0.6875 0.012451172 0.68359375 0.003037393 − 0.00390625
8 0.6796875 −0.006313801 0.68359375 0.003037393 0.681640625 −0.001646005 + 0.001953125
9 0.681640625 −0.001646005 0.68359375 0.003037393 0.682617188 0.000693741 − 0.000976563
10 0.681640625 −0.001646005 0.682617188 0.000693741 0.682128906 −0.00047662 + 0.000488281
11 0.682128906 −0.00047662 0.682617188 0.000693741 0.682373047 0.000108439 − 0.000244141
12 0.682128906 −0.00047662 0.682373047 0.000108439 0.682250977 −0.000184121 + 0.00012207
13 0.682250977 −0.000184121 0.682373047 0.000108439 0.682312012 −3.78487E − 05 + 6.10352E − 05

We conclude from the table (Sign f = sign( f ((an)) ∗ f ((cn)))) that the
solution is bracketed between a13 ≈ 0.682250977 and b13 ≈ 0.682373047
The midpoint of that interval c13 ≈ 0.682312012 is our best guess for the
root since b13−a
2
13
= 0.682373047−0.682250977
2 = 0.00006103515625 < T OL.
Dr. Abdulaziz Rashid 5
We will have to be satisfied with an approximation. Of course, the
approximation can be improved, if needed, by completing more steps of
the Bisection Method.
Note: This method always converge, but often slowly.

0.2 Fixed-Point Iteration for Solving f (x) = 0

By some algebraic steps we transform ( f (x) = 0) into the form

A
x = g(x) (1)
Then we choose an x0 and compute x1 = g (x0) , x2 = g (x1) , and in general
xn+1 = g (xn) (n = 0, 1, · · · ). (2)
AR
A solution of (Eq.1) is called a fixed point of g, motivating the name of
the method This is a solution of f (x) = 0, since from x = g(x) we can
return to the original form f (x) = 0. From ( f (x) = 0) we may get several
different forms of (Eq.1). The behavior of corresponding iterative
sequences x0, x1, · · · may differ, in particular, with respect to their speed
of convergence. Indeed, some of them may not converge at all.
Example An Iteration Process (Fixed-Point Iteration)
Set up an iteration process for the equation f (x) = x2 − 3x + 1 = 0.
Since we know
√ the solutions
x = 1.5 ± 1.25 thus 2.618033989 and 0.381966011,
we can watch the behavior of the error as the iteration proceeds.
Tolerance(Tol)=0.0001
Solution
. The equation may be written
1( ) 1( )
x = g1(x) = x2 + 1 , thus xn+1 = xn2 + 1 . (3)
3 3
If we choose x0 = 1,

Dr. Abdulaziz Rashid 6

 ( ) ( )
x1 = 13 (x02 + 1) = 13 (12 + 1 = 0.666666667)
1 2 1 2
x2 = 3 (x1 + 1 ) = 3 ((0.666666667) + 1 ) = 0.481481481
x3 = 31 x22 + 1 = 13 (0.481481481)2 + 1 = 0.410608139
Further steps are given in the following table: (Fig. 2).
( 2 )
n 1
xn xn+1 = 3 xn + 1 |xn+1 − xn|
0 1 0.666666667 0.333333333
1 0.666666667 0.481481481 0.185185185
2 0.481481481 0.410608139 0.070873342
3 0.410608139 0.389533015 0.021075124

A
4 0.389533015 0.38391199 0.005621025
5 0.38391199 0.382462805 0.001449185
6 0.382462805 0.382092599 0.000370206
AR
7 0.382092599 0.381998251 0.0000943477
We conclude from the table that the approximate root to four correct
places is x8 = 0.381998251, which seems to approach the smaller solution
(0.381966011).
If we choose x0 = 2 the steps are shown in the following table.
( )
n xn xn+1 = 13 xn2 + 1 |xn+1 − xn|
0 2 1.666666667 0.333333333
1 1.666666667 1.259259259 0.407407407
2 1.259259259 0.861911294 0.397347965
3 0.861911294 0.580963693 0.280947601
4 0.580963693 0.445839604 0.135124089
5 0.445839604 0.399590984 0.04624862
6 0.399590984 0.386557652 0.013033333
7 0.386557652 0.383142273 0.0034153789
8 0.383142273 0.382266 0.0008762723
9 0.382266 0.382042432 0.0002235687
10 0.382042432 0.381985473 0.0000569585
We conclude from the table that the approximate root to four correct
Dr. Abdulaziz Rashid 7
places is x11 = 0.381985473, which seems to approach the smaller
solution (0.381966011).
If we choose x0 = 3, we obtain the sequence ((Fig.2) upper part). The
steps are shown in the table.
( )
n xn xn+1 = 13 xn2 + 1 |xn+1 − xn|
0 3 3.333333333 0.333333333
1 3.333333333 4.037037037 0.703703704
2 4.037037037 5.765889346 1.728852309

A
3 5.765889346 11.41515998 5.649270638
4 11.41515998 43.76862582 32.35346584
5 43.76862582 638.8975354 595.1289096
6 638.8975354 136063.6869 135424.7894
AR
which diverges.

3
y

0 x
0 0.5 1 1.5 2 2.5 3 3.5 4

Figure 2: Iterations (Eg. 3)

Our equation may also be written (divide by x )

1 1
x = g2(x) = 3 − , thus xn+1 = 3 − , (4)
x xn
and if we choose x0 = 1, ( Fig. 4), the steps are shown in the following
table.

Dr. Abdulaziz Rashid 8

 n xn xn+1 = 3 − x1n |xn+1 − xn|
0 1 2 1
1 2 2.5 0.5
2 2.5 2.6 0.1
3 2.6 2.615384615 0.015384615
4 2.615384615 2.617647059 0.002262443
5 2.617647059 2.617977528 0.000330469
6 2.617977528 2.618025751 0.0000482230

A
We conclude from the table that the approximate root to four correct
places is x7 = 2.618025751, which seems to approach the larger solution
(2.618033989).
AR
Similarly, if we choose x0 = 2, we obtain (Fig. 4)

n xn xn+1 = 3 − x1n |xn+1 − xn|

0 2 2.5 0.5
1 2.5 2.6 0.1
2 2.6 2.615384615 0.015384615
3 2.615384615 2.617647059 0.002262443
4 2.617647059 2.617977528 0.000330469
5 2.617977528 2.618025751 0.00004822
We conclude from the table that the approximate root to four correct
places is x6 = 2.618025751, which seems to approach the larger solution
(2.618033989).
Similarly, if we choose x0 = 3, we obtain the following table (Fig. 4)

Dr. Abdulaziz Rashid 9

 n xn xn+1 = 3 − x1n |xn+1 − xn|
0 3 2.666666667 0.333333333
1 2.666666667 2.625 0.041666667
2 2.625 2.619047619 0.005952381
3 2.619047619 2.618181818 0.000865801
4 2.618181818 2.618055556 0.000126263
5 2.618055556 2.618037135 0.00001842
We conclude from the table that the approximate root is

A
x6 = 2.618037135, which seems to approach the larger solution
(2.618033989).
4
AR
3.5

2.5

2
y

1.5

0.5

0
x
0 0.5 1 1.5 2 2.5 3 3.5 4

Figure 3: Iterations (Eq. 4)

Figure 4

0.3 Newton’s Method for Solving f (x) = 0

Newton’s method. also known as Newton-Raphson’s method, is another

iteration method for solving equations f (x) = 0, where f is assumed to
have a continuous derivative f ′. The method is commonly used because
of its simplicity and great speed.
The underlying idea is that we approximate the graph of f by suitable

Dr. Abdulaziz Rashid 10

 A
Figure 5

tangents. Using an approximate value x0 obtained from the graph of f ,

AR
we let x1 be the point of intersection of the x -axis and the tangent to the
curve of f at x0 (fig: 5). Then
f (x0) f (x0)
f ′ (x0) = , hence x1 = x0 −
x0 − x1 f ′ (x0)
In the second step we compute x2 = x1 − f (x1) / f ′ (x1) , in the third step
x3 from x2 again by the same formula, and so on.

Dr. Abdulaziz Rashid 11

 ALGORITHM NEWTON ( f , f ′, x0, Tol, N)
This algorithm computes a solution of f (x) = 0 given an initial approx-
imation x0 (starting value of the iteration). Here the function f (x) is
continuous and has a continuous derivative f ′(x).
INPUT: f , f ′, initial approximation x0, tolerance Tol > 0, maximum
number of iterations N
OUTPUT: Approximate solution xn(n ≤ N) or message of failure.

A
For n = 0, 1, 2, · · · , N − 1 do:
1. Compute f ′ (xn)
2. If f ′ (xn) = 0 then OUTPUT ”Failure” Stop. [Procedure completed
AR
unsuccessfully]
3. Else compute
f (xn)
xn+1 = xn − (5)
f ′ (xn)
4. If |xn+1 − xn| 5 Tol |xn+1| then OUTPUT xn+1. Stop. [Procedure
completed successfully] end
5. OUTPUT ”Failure”. Stop. [Procedure completed unsuccessfully
after N iterations]
End NEWTON

Example: Square Root

Set up a Newton iteration for computing the square root x of a given
positive number c and apply it to c = 2. Tolerance(Tol) = 0.0001
Solution
√
We have x = c, hence f (x) = x2 − c = 0. f ′(x) = 2x, and (5) takes the

Dr. Abdulaziz Rashid 12

form ( )
xn2 − c 1 c
xn+1 = xn − = xn +
2xn 2 xn
For c = 2, choosing x0 = 1, we obtain

( )
n −2
2
n xn xn+1 = xn − x2x n
= xn + xn |xn+1 − xn| / |xn+1|
1
2
2

0 1 1.5 0.333333333
1 1.5 1.416666667 0.058823529
2 1.416666667 1.414215686 0.001733102

A
3 1.414215686 1.414213562 1.50182E − 06
We conclude from the table that the approximate root is
x4 = 1.414213562.
AR
Example Iteration for a Transcendental Equation
Find the positive solution of 2 sin x = x.
Solution
Setting f (x) = x − 2 sin x, we have f ′(x) = 1 − 2 cos x, and (5) gives
xn − 2 sin xn 2 (sin xn − xn cos xn) Nn
xn+1 = xn − = =
1 − 2 cos xn 1 − 2 cos xn Dn
n xn f (x) = xn − 2 sin xn f ′ (x) = 1 − 2 cos xn xn+1 = xn − ff′(x)
(x)
|xn+1 − xn | / |xn+1 |
0 2 0.181405146 1.832293673 1.900995594 0.052080292
1 1.900996 0.009040087 1.648463075 1.895511645 0.002893123
2 1.895512 2.84668E − 05 1.638077989 1.895494267 9.16815E − 06
3 1.895494 2.86219E − 10 1.638045049 1.895494267 9.21829E − 11

We conclude from the table that the approximate root is

x4 = 1.895494267.
Example Newton’s Method Applied to an Algebraic Equation.
Apply Newton’s method to the equation
f (x) = x3 + x − 1 − 0, tolerance(Tol) = 0.0000001.
Solution
xn3 +xn −1 2xn3 +1
From (5) we have xn+1 = xn − 3x2+1 = 3x2+1 .
n n

Dr. Abdulaziz Rashid 13

Starting from x0 = 1, we obtain

n xn Nn = 2xn3 + 1 Dn = 3xn2 + 1 Nn
xn+1 = D n
|xn+1 − xn | / |xn+1 |
0 1 3 4 0.75 0.33333333
1 0.75 1.84375 2.6875 0.686046512 0.093220339
2 0.686046512 1.64578905 2.411979448 0.682339583 0.005432675
3 0.682339583 1.635377296 2.396761918 0.682327804 1.72625E − 05
4 0.682327804 1.635344393 2.396713696 0.682327804 1.73661E − 10

We conclude from the table that the approximate root is

x5 = 0.682327804.

0.4

A
Secant Method for Solving f (x) = 0
AR
Newton’s method is very powerful but has the disadvantage that the
derivative f ′ may sometimes be a far more difficult expression than f itself
and its evaluation therefore computationally expensive. This situation
suggests the idea of replacing the derivative with the difference quotient
f (xn) − f (xn−1)
f ′ (xn) ≈
xn − xn−1
Then instead of (5) we have the formula of the popular secant method
xn − xn−1
xn+1 = xn − f (xn) (6)
f (xn) − f (xn−1)
Geometrically, we intersect the x-axis at x. with the secant of f (x)
passing through Pn−1 and Pn in (Fig. 6)
CAUTION! It is not good to write (6) as
xn−1 f (xn) − xn f (xn−1)
xn+1 =
f (xn) − f (xn−1)
because this may lead to loss of significant digits if xn and xn−1 are about
equal.
Dr. Abdulaziz Rashid 14
 A Figure 6: Secant Method
AR
Example Find the positive solution of f (x) = x − 2 sin x = 0 by the secant
method. starting from x0 = 2, x1 = 1.9 and tolerance Tol = 0.0001.
Solution. Here. (6) is
(xn − 2 sin xn) (xn − xn−1) Nn
xn+1 = xn − = xn −
xn − xn−1 + 2 (sin xn−1 − sin xn) Dn
Numeric values are:
n xn−1 xn Nn Dn xn+1 |xn+1 − xn | / |xn+1 |
1 2 1.9 −0.000739982 −0.174005322 1.895747357 0.002243254
2 1.9 1.895747357 −0.000001763290 −0.006985191 1.895494925 0.000133175
3 1.895747357 1.895494925 −0.000000000272 −0.000413557 1.895494267 3.46893E − 07

We conclude from the table that the approximate root is

x4 = 1.895494267.
Summary of Methods:
The methods for computing solutions s of f (x) = 0 with given continuous
(or differentiable) f (x) start with an initial approximation x0 of s and
generate a sequence x1, x2, · · · by iteration. Fixed-point methods solve
f (x) = 0 written as x = g(x), so that s is a fixed point of g, that is,
s = g(s). For g(x) = x − f (x)/ f ′(x) this is Newton’s method, which, for
good x0 and simple zeros, converges quadratically (and for multiple zeros
Dr. Abdulaziz Rashid 15
linearly). From Newton’s method the secant method follows by replacing
f ′(x) by a difference quotient. The bisection method always converge, but
often slowly.
Exercises
Bisection Method
1. Use the Bisection Method to find a root of the function
f (x) = cosx − x in the interval [0,1] to within six correct place
(Tolerance (TOL) = 0.000001).

A
2. Use the Bisection Method to find a root to six correct places
(Tolerance (TOL) = 0.000001).
(a) x3 = 9, (b) 3x3 + x2 = x + 5, (c) cos2 x = x,
AR
3. Use the Bisection Method to find a root to eight correct decimal
places (Tolerance (TOL) = 0.00000001).
(a) sin x = 6x + 5, (b) x5 + x = 1, (c) ln x + x2 = 3,
4. solve e−x = ln x and ex + x4 + x = 2by Bisection Method.
5. Use the Bisection Method to locate all solutions of the following
equations. Sketch the function by using MATLAB’s plot command
and identify three intervals of length one that contain a root. Then
find the roots to six correct decimal places.
(a) 2x3 − 6x − 1 = 0 (b) ex−2 + x3 − x = 0 (c) 1 + 5x − 6x3 − e2x = 0
Fixed-Point Iteration
1. Find a solution of f (x) = x3 + x − 1 = 0 by Fixed-Point Iteration to
within six correct place (Tolerance (TOL) = 0.0001).Use
√
(a) x = g(x) = 1+x2 x0 = 1, (b) x = g(x) = 3 1 − x x0 = 0.5,
1

(c) x = g(x) = (1 + 2x3)/(1 + 3x2), x0 = 0.5

2. Use the Bisection Method to find a root to six correct places
(Tolerance (TOL) = 0.000001).
(a) x3 = 9, (b) 3x3 + x2 = x + 5 = 9, (c) cos2 x = x,
Dr. Abdulaziz Rashid 16
 3. Use Fixed-Pint Iteration to find a root of cos x = sin x, use
g(x) = x + cos x − sin x, Tolerance (TOL) = 0.0001.
4. Find all fixed points of the following g(x).
(a) 3x , (b) x2 − 2x + 2, (c) x2 − 4x + 2
√
5. For which of the following g(x) is s = (3) a fixed point?
(a) g(x) = √x3 , (b) g(x) = 2x 1
3 + x,
(c) g(x) = x2 − x, (d) g(x) = 1 + x+1 2

Newton-Raphson’s Method

A
1. Apply two steps of Newton’s Method with initial guess x0 = 0.
(a) x3 + x − 2 = 0, (b)x2 − +1/(x + 1) − 3, (c) 5x − 10 = 0
AR
2. Apply two steps of Newton’s Method with initial guess x0 = 1.
(a) x3 + x2 − 1 = 0, (b)x4 − x2 + x − 1 = 0 (c) x2 − x − 1 = 0
3. Vibrating beam. Find the solution of cos x cosh x = 1 near x = 32 π .
(This determines a frequency of a vibrating beam). Take
(Tolerance (TOL) = 0.0001).
4. Solve x = cos x by Newton Raphson’s method and by Bisection.
Compare.
Secant Method

1. Apply two steps of the Secant Method to the following equations with
initial guesses x0 = 1 and x1 = 2. Take (Tolerance (TOL) = 0.0001)
(a) x3 = 2x + 2 (b) ex + x = 7 (c) ex + sin x = 4
2. Solve, using x0 and x1 as indicated:
(a) e−x − tan x = 0, x0 = 1, x1 = 0.7
(b) x = cos x, x0 = 0.5, x1 = 1

Dr. Abdulaziz Rashid 17