Roots of Equations (Chapters 5 and 6)

Problem: given f (x) = 0, find x.

In general, f (x) can be any function. For some forms of f (x), analytical solutions are available.
However, for other functions, we have to design some methods, or algorithms to find either exact,
or approximate solution for f (x) = 0.

We will focus on f (x)

• with single unknown variable,

• not linear, and
• continuous.

Outline:

• Incremental search methods: bisection method, false position method

• Open methods: Newton-Raphson method, Secant method
• Finding repeated roots

1
Analytical solutions:
√
2 −b± b2 −4ac
Example 1: ax + bx + c = 0, x = 2a
Example 2: aex − bx = 0. No analytical solution.

Straightforward approach: Graphical techniques.

The most straightforward method is to draw

a picture of the function and find where the
function crosses x-axis. f(x)
Graphically find the root: Plot f (x) for differ-
ent values of x and find the intersection with
the x axis.
Graphical methods can be utilized to provide
estimates of roots, which can be used as start-
ing guesses for other methods. 0 1 2
Drawbacks: not precise. Different people x
may have different guesses.

A better method is called incremental search method, which is a general method that most
computer-based methods are based on.

2
1 Incremental search methods:

Key idea: The function f (x) may change signs around the roots.
f(x) f(x)

xl x ’l xu’ xu
x xl xu x

Figure 1: Illustration of incremental search method

Note: f (x) may not change signs around the roots (see figures on the next page). This case will
be dealt with later.

Assume that function f (x) is real and continuous in interval (xl , xu) and f (x) has opposite signs
at xl and xu, i.e.,
f (xl ) · f (xu) < 0
Then, there is at least one root between xl and xu.

3
4
Next, we can locate the root in a smaller interval until reasonable accuracy is achieved or the root
is found.
Based on this, the basic steps in incremental search methods are as follows:
(1). Locate an interval where the function change signs.
(2). Divide the interval into a number of sub-intervals.
(3). Search the sub-interval to accurately locate the sign change.
(4). Repeat until the root (the location of the sign change) is found.
Errors in an iterative approach
Since getting the exact value of the root is almost impossible during iterations, a termination cri-
terion is needed. (When the error is below a certain stopping threshold, terminate the iterations.)
Define approximate percentage relative error as
¯ ¯
¯ xnew − xold ¯
¯ ¯
²a = ¯ r new r ¯ × 100%
¯ xr ¯

where xnew
r is the estimated root for the present iteration, and xold
r is the estimated root from
the previous iteration.
Define true percentage relative error as
¯ ¯
¯ xt − xr ¯
²t = ¯¯ ¯ × 100%
xt ¯
5
where xt is the true solution of f (x) = 0, i.e., f (xt) = 0.
In general, ²t < ²a. That is, if ²a is below the stopping threshold, then ²t is definitely below it as
well.

2 Bisection (or interval halving) method

Bisection method is an incremental search method where sub-interval for the next iteration is
selected by dividing the current interval in half.

2.1 Bisection steps

(1). Select xl and xu such that the function changes signs, i.e.,
f (xl ) · f (xu) < 0

(2). Estimate the root as xr given by

xl + xr
xr =
2
(3). Determine the next interval:
– If f (xl ) · f (xr ) < 0, then the root lies in (xl , xr ), set xu = xr and return to Step (2).
– If f (xu) · f (xr ) < 0, then the root lies in (xr , xu), set xl = xr and return to Step (2).

6
 – If f (xu) · f (xr ) = 0, then the root has been found. Set the solution x = xr and terminate
the computation.
(4). If ²a < ²threshold, x = xr ; else, repeat the above process.

(new xu )
xl xr xu xl xr xu
(new xu)

(new x l)

xl xr xu xl xr xu
(new x l)
Figure 2: Bisection method

If ²a is below a certain threshold (e.g., 1%), stop the iteration. This is, the computation is stopped
when the solution does not change much from one iteration to another.

7
Example: Find the root of f (x) = ex − 3 × x = 0.
Solution:

1. xl = 0, xu = 1, f (0) = 1 > 0, f (1) = −0.2817 < 0, xr = (xl + xu)/2 = 0.5, ² = 100%.

2. f (0.5) = 0.1487 > 0, xl = xr = 0.5, xu = 1, xr = (xl + xu)/2 = 0.75, ²a = 33.33%.
3. f (0.75) = −0.1330 < 0, xl = 0.5, xu = xr = 0.75, xr = 0.625, ²a = 20%.
4. f (0.625) = −0.0068 < 0, xl = 0.5, xu = xr = 0.625, xr = 0.5625, ²a = 11%.
5. f (0.5625) = 0.0676 > 0, xl = xr = 0.5625, xu = xu = 0.625, xr = 0.5938, ²a = 5.3%.
6. f (0.5938) = 0.0295 > 0, xl = xr = 0.5938, xu = xu = 0.625, xr = 0.6094, ²a = 2.6%.
7. . . .

xr : 0.5 → 0.75 → 0.625 → 0.5625 → 0.5938 → 0.6094 → 0.6172 → 0.6211 → 0.6191 →

0.6182 → 0.6187 → 0.6189 → 0.6190 → 0.6191

8
 Bisection Method (between0and1)
1

0.8

0.6

0.4
f(x)

0.2

−0.2

−0.4
0 0.2 0.4 0.6 0.8 1
x
Figure 3: Example of using Bisection method, f (x) = ex − 3 × x = 0

9
2.2 Evaluation of ²a

x old
l
x old
r
x old
u

x new x new x new

l r u

Figure 4: Evaluation of ²a

¯ ¯
¯ xnew − xold ¯
¯ ¯
²a = ¯ r new r ¯
¯ xr ¯
1 new
xnew
r = (xl + xnew
u )
2
xold = xnew
r l

Therefore,
xnew − x old = 1 xnew + 1 xnew − xnew = 1 ¡xnew − xnew¢
r r
2 l 2 u l
2 u l

and ¯ ¡ ¯
¯ 1 xnew − xnew¢ ¯
¯ ¯
²a = ¯ 21 ¡ unew l
¢ ¯ × 100%
¯ 2 xu + xnew
l
¯

10
or
xu − xl
²a = × 100%
xu + xl
That is, the relative error can be found before starting the iteration.

2.3 Finding the number of iterations

Initially (iteration 0), the absolute error is

Ea0 = x0u − x0l = ∆0x
where superscript indicates the iteration number.
After first iteration, the absolute error is halved. That is
1 0 ∆0x
Ea1 = Ea =
2 2
After n iterations,
1 n−1 ∆0x
Ean = Ea = n
2 2
If the desired absolute error is Ea,d, then
Ean ≤ Ea,d
which is equivalent to
∆0x ∆0x
≤ Ea,d ⇒ n ≥ log2
2n Ea,d
11
The minimum n is  
» ¼ log10 E∆x
0
∆0x
=  a,d
log2  
Ea,d  log10 2 

12
Example:
Find the roots of function f (x) = x3 − 2x2 + 0.25x + 0.75.
Solution:
To find the exact roots of f (x), we first factorize f (x) as
f (x) = x3 − 2x2 + 0.25x + 0.75
= (x − 1)(x2 − x − 0.75)
= (x − 1) · (x − 1.5) · (x + 0.5)
Thus, x = 1, x = 1.5 and x = −0.5 are the exact roots of f (x).
1.5
f(x)=x3−2x2+0.25x+0.75
1
• xl = −1, xu = 2, the result con- 0.5
verges to xr = −0.5.
0
• xl = −2, xu = 2, the result con- −0.5
verges to xr = −0.5.
−1
• xl = −1, xu = 8, the result con- −1.5
verges to xr = 1.5.
−2

−2.5
−1 −0.5 0 0.5 1 1.5 2
x

13
3 Newton-Raphson method

3.1 Iterations

The Newton-Raphson method uses the slope (tangent) of the function f (x) at the current iterative
solution (xi) to find the solution (xi+1) in the next iteration.
The slope at (xi, f (xi)) is given by
0 f (xi) − 0
f (xi) =
xi − xi+1
Then xi+1 can be solved as
f (xi)
xi+1 = xi −
f 0 (xi)
which is known as the Newton-Raphson formula.
¯ ¯
¯ xi+1−xi ¯
Relative error: ²a = ¯ xi+1 ¯ × 100%.

Iterations stop if ²a ≤ ²threshold.

14
Figure 5: Newton-Raphson method to find the roots of an equation

15
Example: Find the root of e−x − 3x = 0.

Solution: Newton−Raphson Method (from−1)

5
−x
f (x) = e − 3x
0
4
−x
f (x) = −e −3
3
With these, the Newton-Raphson solu-
tion can be updated as

f(x)
2
−xi
e − 3xi
xi+1 = xi − 1
−e−xi − 3
−1 → 0.2795 → 0.5680 → 0.6172 → 0
0.6191 → 0.6191
Converges much faster than the bisection −1
−1.5 −1 −0.5 0 0.5 1
method. x

3.2 Errors and termination condition

The approximate percentage relative error is

¯ ¯
¯ xi+1 − xi ¯
²a = ¯¯ ¯ × 100%
xi+1 ¯

16
The Newton-Raphson iteration can be terminated when ²a is less than a certain threshold (e.g.,
1%).

3.3 Error analysis of Newton-Raphson method using Taylor series

Let xt be the true solution to f (x) = 0. That is

f (xt) = 0

According to the Taylor’s theorem, we have

00
0 f (α)
f (xt) = f (xi) + f (xi)(xt − xi) + (xt − xi)2
2
where α is an unknown value between xt and xi. Since f (xt) = 0, the above equation becomes
00
0 f (α)
f (xi) + f (xi)(xt − xi) + (xt − xi)2 = 0 (1)
2
In Newton-Raphson method, we use the following iteration:
f (xi)
xi+1 = xi −
f 0 (xi)
That is
0
f (xi) + f (xi)(xi+1 − xi) = 0 (2)

17
Subtracting (2) from (1), we have
00
0 f (α)
f (xi)(xt − xi+1) + (xt − xi)2 = 0
2
Denoting ei = xt − xi, which is the error in the i-th iteration, we have
00
0 f (α) 2
f (xi)ei+1 + (ei) = 0
2
With convergence, xi → xt, and α → xt. Then
00
f (xt) 2
ei+1 =− 0 e
2f (xt) i
and ¯ 00 ¯
¯ f (x ) ¯
¯ t ¯ 2
|ei+1| = ¯ 0 ¯e
¯ 2f (xt) ¯ i
|ei+1| ∝ |ei|2
The error in the current iteration is proportional to the square of the previous error. That is, we
have quadratic convergence with the Newton-Raphson method. The number of correct decimal
places in a Newton-Raphson solution doubles after each iteration.

Drawbacks of the Newton-Raphson method: (see Fig. 6.6 in [Chapra])

• It cannot handle repeated roots

18
 • The solution may diverge near a point of inflection.
• The solution may oscillate near local minima or maxima.
• With near-zero slope, the solution may diverge or reach a different root.
Example:
Find the roots of function f (x) = x3 − 2x2 + 0.25x + 0.75.
Solution:
To find the exact roots of f (x), we first factorize f (x) as
f (x) = x3 − 2x2 + 0.25x + 0.75
= (x − 1)(x2 − x − 0.75)
= (x − 1) · (x − 1.5) · (x + 0.5)
Thus, x = 1, x = 1.5 and x = −0.5 are the exact roots of f (x).
To find the roots using the Newton-Raphson methods, write the iteration formula as
f (xi)
xi+1 = xi − 0
f (xi)
x3 − 2x2 + 0.25x + 0.75
= xi −
3x2 − 4x + 0.25
• x0 = −1, x1 = −0.6552, x2 = −0.5221, x3 = −0.5005, x4 = −0.5000, x5 = −0.5000.
• x0 = 0, x1 = −3, x2 = −1.8535, x3 = −1.1328, x4 = −0.7211, x5 = −0.5410, x6 = −0.5018,
x7 = −0.5000, x8 = −0.5000.
19
Figure 6: Drawbacks of using NR method

20
• x0 = 0.5, x1 = 1, x2 = 1.
• x0 = 1.3, x1 = 1.5434, x2 = 1.5040, x3 = 1.5000, x4 = 1.5000.
• x0 = 1.25, x1 = −0.5000, x2 = −0.5000.

21
4 Secant method
0
In order to implement the Newton-Raphson method, f (x) needs to be found analytically and
evaluated numerically. In some cases, the analytical (or its numerical) evaluation may not be
feasible or desirable.
The Secant method is to evaluate the derivative online using two-point differencing.
f(x) (x i−1,f( x i−1))

(x i , f(x i ))

x
xi+1 x i x i−1
xi+1 x i x i−1
Figure 7: Secant method to find the roots of an equation

As shown in the figure,

0 f (xi−1) − f (xi)
f (x) ≈
xi−1 − xi

22
Then the Newton-Raphson iteration can be modified as
f (xi)
xi+1 = xi −
f 0 (xi)
f (xi)
= xi − f (xi−1 )−f (xi )
xi−1 −xi
f (xi)(xi − xi−1)
= xi −
f (xi) − f (xi−1)
which is the Secant iteration.
The performance of the Secant iteration is typically inferior to that of the Newton-Raphson
method.
Example: f (x) = ex − 3x = 0, find x.
Solution: ¯ ¯
f (x1 )(x1 −x0 ) ¯ x2−x1 ¯
x0 = −1.1, x1 = −1, x2 = x1 − f (x1 )−f (x0 )= 0.2709, ²a = ¯ x2 ¯ × 100% = 469.09%
¯ ¯
f (x2 )(x2 −x1 ) ¯ x3−x2 ¯
x1 = −1, x2 = 0.2709, x3 = x2 − f (x2)−f (x1) = 0.4917, ²a = ¯ x3 ¯ × 100% = 44.90%
¯ ¯
f (x3 )(x3 −x2 ) ¯ x4−x3 ¯
x2 = 0.2709, x3 = 0.4917, x4 = x3 − f (x3)−f (x2) = 0.5961, ²a = ¯ x4 ¯ × 100% = 17.51%
x5 = 0.6170, ²a = 3.4%
x6 = 0.6190, ²a = 0.32%
x7 = 0.6191, ²a = 5.93 × 10−5

23
5 False position method

f(x)

f(xu)

xr f(xu)
xl xl
xt xu x f(x l) xr xu
f(x l)

Figure 8: False position method to find the roots of an equation

Idea: if f (xl ) is closer to zero than f (xn), then the root is more likely to be closer to xl than to
xu. (NOT always true!)
False position steps:

(1). Find xl , xu, xl < xu, f (xl )f (xu) < 0

(2). Estimate xr using similar triangles:
−f (xl ) xr − xl
=
f (xu) xu − xr

24
 Find xr as
[f (xl ) − f (xu)]xu + f (xu)(xu − xl )
xr =
f (xl ) − f (xu)
or
f (xu)(xu − xl )
xr = xu −
f (xu) − f (xl )
(3). Determine next iteration interval
– If f (xl ) · f (xr ) < 0, then the root lies in (xl , xr ), set xu = xr and return to Step (2).
– If f (xu) · f (xr ) < 0, then the root lies in (xr , xu), set xl = xr and return to Step (2).
– If f (xu) · f (xr ) = 0, then the root has been found. Set the solution x = xr and terminate
the computation.
(4). If ²a < ²threshold, x = xr ; else, back to (2).

False position method is one of the incremental search methods.

In general, false position method performs better than bisection method. However, special case
exists (see fig. 5.14 in textbook).

25
Figure 9: Comparison between false position and Secant methods

26
27
6 Handling repeated (multiple) roots

Examples of multiple roots:

• f (x) = x2 − 4x + 3 = (x − 1)(x − 3) has unrepeated roots: x = 1, x = 3

• f (x) = (x − 1)2(x − 3) has 2 repeated roots at x = 1
• f (x) = (x − 1)3(x − 3) has 3 repeated roots at x = 1
• f (x) = (x − 1)4(x − 3) has 4 repeated roots at x = 1

Observations:

• The sign of the function does not change around even multiple roots — bisection and false
position methods do not work
0
• Both f (x) and f (x) go to zero around multiple roots — Newton-Raphson and secant methods
may converge slowly or even diverge.

6.1 Modified Newton-Raphson method

f (x)
Fact: Both f (x) and u(x) = 0
f (x)
have the same roots, but u(x) has better convergence properties.
Idea: to find the roots of u(x) instead of f (x).

28
 3 4

f(x)=(x−3).*(x−1).2
f(x)=(x−3).*(x−1)
2 2

1 0

0 −2

−1 −4
0 1 2 3 4 0 1 2 3 4
x x

4 4
f(x)=(x−3).*(x−1).3

f(x)=(x−3).*(x−1).4
2 2

0 0

−2 −2

−4 −4
0 1 2 3 4 0 1 2 3 4
x x

Figure 10: Repeated roots

29
The Newton-Raphson method iteration for u(x):
u(xi)
xi+1 = xi −
u0 (xi)
h 0
i2 00
0
f (x) − f (x)f (x)
u (x) = £ 0 ¤2
f (x)
h 0 i2
f (xi) f (xi)
xi+1 = xi − £ 0 ¤2 0
f (xi) − f (xi)f 00 (xi) f (xi)
or 0
f (xi)f (xi)
xi+1 = xi − £ 0 ¤2
f (xi) − f (xi)f 00 (xi)
Modified Newton-Raphson method has quadratic convergence even for multiple roots.

xt is an unrepeated root of u(x) = 0

f (x) has n repeated roots, n ≥ 2
f (x) = g(x) · (x − xt)n
where g(xt) 6= 0. Then
0 0
f (x) = g (x)(x − xt)n + g(x)n(x − xt)n−1
h 0 i
n−1
= (x − xt) g (x)(x − xt) + ng(x)

30
and
f (x)
u(x) =
f 0 (x)
g(x) · (x − xt)n
= £ 0 ¤
(x − xt)n−1 g (x)(x − xt) + ng(x)
g(x) · (x − xt)
= 0
g (x)(x − xt) + ng(x)
Therefore, xt is an unrepeated root of u(x) = 0.

31