NUMERICAL INTEGRATION

1
 Computational Science vs. Calculus

• Calculus tells you how to compute precise integrals &

derivatives when you know the equation (analytical form)
for a problem; e.g., for indefinite integral:
−𝑡 3
නሺ−𝑡 2 + 10𝑡 + 24) 𝑑𝑡 = + 5𝑡 2 + 24𝑡 + 𝐶
3

• Computational science provides methods for estimating

integrals and derivatives from actual data.

2
 1. Differentiation
f(x)

f  xi  x

y
f  xi 

x x

Mathematical definition of
y f  xi  x  f  xi  derivative - rate of change of a

x x dependent variable with respect to
an independent variable
3
 2. Integration
• The inverse process of differentiation
• Mathematically, it is the total value or summation
of f(x)dx over a range of x. In fact the integration
symbol is actually a stylized capital S intended to
signify the connection between integration and
summation.

4
f(x)

x
b

I   f  x dx
a
5
 Mathematical Background
d d x
dx
sin x  ?
dx
e ?  udv  ?
 u du  ?
n
d d x
cos x  ? a ?
 a dx  ?
dx dx bx

d d n
tan x  ? x ? dx
dx dx
 x ?
d
ln x  ?
 dx  ?
ax
dx e
if u and v are functions of x
d n d
u ? (uv )  ?
dx dx
6
Urban area River cross-section Windblow on rocket

7
 Example: numerical integration

• Very often, the function f(x) to

differentiate or the integrand to
integrate is too complex to derive
exact analytical solutions.
• In most cases in engineering, the
function f(x) is only available in a
tabulated form with values
known only at discrete points.

Numerical Solution

8
Numerical Integration
Idea is to do integral in small parts, like the way we presented
integration: a summation.
Numerical methods just try to make it faster and more
accurate.
12

10

0
3 5 7 9 11 13 15 9
Basic Numerical Integration
Idea: Weighted sum of function values to approximate integral
n
f ( x )dx   c i f ( x i )
b
a
i 0
f(x)  c 0 f ( x 0 )  c 1 f ( x 1 )    c n f ( x n )

x0 x1 xn-1 xn x
10
Physics Example: Velocity & Acceleration

• Velocity, v(t), is defined as the derivative of position, x(t).

Acceleration, a(t), is the derivative of v(t), and the second
derivative of x(t).
• The integral of acceleration is velocity. The integral of velocity
is position.
• The graph of v(t) against time, t, shows that the area between
two times is the distance traveled.
• On the other hand, the area under a(t) against time shows the
velocity.

11
 Overview
• Newton-Cotes Integration Formulas
– Trapezoidal rule
– Simpson’s Rules
– Unequal Segments
– Open Integration
• Integration of Equations
– Romberg Integration
– Gauss Quadrature
– Improper Integrals
12
 Quadrature & integration
• Quadrature has become a synonym for numerical integration
– Primarily in one dimension, higher dimensional evaluation is
sometimes dubbed cubature
• All quadrature methods rely upon the interpolation of the
integral function f using a class of functions
– e.g. polynomials
– Utilizing the known closed form integral of the interpolated function
allows simple calculation of the weights
• Different functions will require different quadrature
algorithms
– Singular functions may require specialized Gaussian quadrature, or
changes of variables
– Oscillating functions require simpler methods, e.g. trapezoidal rule

13
Basic Numerical Integration
• We want to find integration of functions of various forms of the
equation known as the Newton-Cotes integration formulas.
• Newton-Cotes formula
Assume the value of f(x) defined on [a,b] is known at equally
spaced points xi (i=0,1,...,n), where x0 =a, and xn =b. Then,
b n

 f(x)dx   c f ( x )
a i 1
i i

where xi =h i + x0, with h (the step size) equal to (xn−x0)/n =

(b−a)/n. The ci's are called weights. The xi's are called nodes. The
precision of the approximation depends on n.
The nodes can be equally spaced or unequally spaced. 14
Numerical Integration
• The weights are derived from the Lagrange basis
polynomials L(x). They depend only on the xi's; not on the
function ƒ. Different methods use different polynomials to
get the ci's.
• Newton-Cotes Closed Formulae -- Use both end points
– Trapezoidal Rule : Linear
– Simpson’s 1/3-Rule : Quadratic
– Simpson’s 3/8-Rule : Cubic
– Boole’s Rule : Fourth-order

• Newton-Cotes Open Formulae -- Use only interior points

– midpoint rule
15
Numerical Integration
• Error analysis
The error of the approximation is the difference between the
value of the integral and the numerical result:

error = ε = ∫ f(x) dx - Σi ci f(xi)

The errors are frequently approximated using Taylor series

for f(x). The error analysis gives a strict upper bound on the
error, if the derivatives of f are available.

16
 Newton-Cotes Integration

• Common numerical integration scheme

• Based on the strategy of replacing a complicated function
or tabulated data with some approximating function that
is easy to integrate

b b

I   f  x dx   f n  x dx
a a

f n  x   a0  a1 x .... an x n fn(x) is an n-th order

polynomial

17
 Newton-Cotes Integration Formulas
b b
• Based on the strategy of replacing a
I   f ( x)dx   f n ( x)dx
complicated function or tabulated data
a a
with an approximating function that is
easy to integrate: f n ( x)  a0  a1 x    an x n

Zero order approximation First-order Second-order

18
 a first order polynomial a second order polynomial

5 5

4 4

3 3
f(x)

f(x)
2 2

1 1

0 0
0 5 10 0 5 10
x x

19
 a first order polynomial a second order polynomial

5 5

4 4

3 3
f(x)

f(x)
2 2

1 1

0 0
0 5 10 0 5 10
x x

We can also approximated the integral by using a

series of polynomials applied piece wise.
20
 5

3
f(x)
2

0
0 1 2 3 4 5 6 7 8 9 10
x

An approximation of an integral by the area under

straight line segments.

21
 5

3
f(x)
2

0
0 1 2 3 4 5 6 7 8 9 10
x

An approximation of an integral by the area under

straight line segments.

22
 Newton-Cotes Formulas
• Closed form - data is at the beginning and end
of the limits of integration
• Open form - integration limits extend beyond
the range of data.
5 5

4 4

3 3
f(x)

2 f(x) 2

1 1

0 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
x x

23
 Trapezoidal Rule
• First of the Newton-Cotes closed integration
formulas
• Corresponds to the case where the
polynomial is a first order
b b

I   f  x dx   f 1  x dx
a a

f n  x   a0  a1 x

24
Trapezoidal Rule

b b

I   f  x dx   f 1  x dx
a a

f n  x   a0  a1 x

A straight line can be represented as:

f  b  f  a 
f 1  x  f  a    x  a
ba

25
Trapezoidal Rule
b b

I   f  x dx   f 1  x dx
a a
b
f b  f a 
 f a    x  a dx
a
ba

Integrate this equation. Results in the trapezoidal

rule.
f  a   f  b
I  b  a 
2

26
Trapezoidal Rule
f a   f b 
I  b  a 
2
Recall the formula for computing the area of a
trapezoid:

height x (average of the bases)

base

height

27
base
Trapezoidal Rule
f a   f b 
I  b  a 
2
The concept is the same but the trapezoid is
on its side.

base

height
height

height width

base

28
 Error of the Trapezoidal Rule

1
Et   f ' '  b  a 
3

12
where a    b

This indicates that if the function being integrated is linear, the trapezoidal
rule will be exact.

Otherwise, for section with second and higher order derivatives (that is with
curvature) error can occur.

A reasonable estimate of x is the average value of b and a

29
Example Single Application of the Trapezoidal Rule
f(x) = 0.2 +25x – 200x2 + 675x3 – 900x4 + 400x5

Integrate f(x) from a=0 to b=0.8

b 0.8
True integral value : I   f ( x)dx  1.64053
a 0

Solution: f(a)=f(0) = 0.2 and f(b)=f(0.8) = 0.232

f (a )  f (b)
Trapezoida l Rule : I  (b  a )
2
0.2  0.232
 0.8  0.1728
2
which represents an error of :

E t  1.64053  0.1728  1.46773  t  89.5%

30
Composite (multiple) trapezoidal rule

31
Multiple Application of the Trapezoidal Rule

• Improve the accuracy by dividing the integration interval

into a number of smaller segments
• Apply the method to each segment
• Resulting equations are called multiple-application or
composite integration formulas

32
 Multiple Application of the Trapezoidal Rule

x1 x2 xn

I  f (x )dx   f (x )dx  f (x )dx

x0 x1 xn 1

f  x0   f  x1  f  x1   f  x2  f  xn  1   f  xn 
Ih h  h
2 2 2

where there are n+1 equally spaced base points.

33
Multiple Application of the Trapezoidal Rule

We can group terms to express a general form

x1 x2 xn

I  f ( x)dx   f ( x)dx   f ( x)dx

x0 x1 xn 1

f  x0   f  x1  f  x1   f  x2  f  xn  1   f  xn 
I h h  h
2 2 2
n 1
f  x0   2 f  xi   f  xn 
I  b  a  i 1

2n
}

width average height

34
Multiple Application of the Trapezoidal Rule
n 1
f  x0   2 f  xi   f  xn 
I  b  a  i 1

2n
}
width
}
average height

The average height represents a weighted average

of the function values

Note that the interior points are given twice the weight
of the two end points
b  a  3
Ea   2
f ''
12 n
35
 Example
Evaluate the following integral using the trapezoidal rule and h
= 0.1
1.6 30

I e
x2
dx 25

1 20

f(x)
15
ba
h 10

n 5

0
0 0.3 0.6 0.9 1 .2 1 .5 1 .8
x

n 1
f  x0   2 f  xi   f  xn 
I  b  a  i 1

2n
36
 Simpson’s Rules
• One drawback of the trapezoidal rule is that the error is related to the
second derivative of the function.
• More complicated approximation formulas can improve the accuracy for
curves - these include using (a) 2nd and (b) 3rd order polynomials.
• The formulas that result from taking the integrals under these polynomials
are called Simpson’s rules.

37
 Simpson’s 1/3 Rule

• Corresponds to the case where the function

is a second order polynomial
b b

I   f  x dx   f 2  x dx
a a

f n  x   a0  a1 x  a2 x 2

38
 Simpson’s 1/3 Rule
• Designate a and b as x0 and x2, and estimate f2(x) as a second
order Lagrange polynomial.
• Simpson’s 1/3 rule corresponds to using second-order
polynomials. Using the Lagrange form for a quadratic fit of
three points:
fn x 
x  x1  x  x2  f x  x  x0  x  x2  f x  x  x0  x  x1  f x
 0  1  2
 0 1  0 2 
x  x x  x  1 0  1 2
x  x x  x  2 0   2 1
x  x x  x
Integration over the three points simplifies to:
b b

I   f  x dx   f  x dx
a a
2

  x  x1  x  x2  
x2

  f  x0  .......dx
x 0  0
x  x1  x0  x2  

39
 Simpson’s 1/3 Rule
• After integration and algebraic manipulation, we get
the following equations
h
I   f  x0   4 f  x1   f  x2 
3
f  x0   4 f  x1   f  x2 
 b  a 
6
}

}
width average height

40
 Error

Single application of Trapezoidal Rule.

1
Et   f ' '  b  a 
3

12
where a    b

Single application of Simpson’s 1/3 Rule

1
Et   f (4)
 b  a  5

2880

41
 Composite Simpson’s 1/3 rule
• Simpson’s 1/3 rule can be used on a
set of subintervals in much the same
way the trapezoidal rule was, except
there must be an odd number of
points.
• Because of the heavy weighting of the
internal points, the formula is a little
more complicated than for the
trapezoidal rule:

   
xn x2 x4 xn
I x0
fn x dx  x0
fn x  dx  x2
fn x  dx   x n2
fn x dx
h h h
I
3
 f  0
x  4 f  1   2   f x2  4 f x3  f x4 
x  f x 
3

3
 f xn2   4 f xn1   f xn 

 n1 n2

h  
I  f x0   4  f xi   2  f xi   f xn 
3


i1
i, odd
j2
j, even 

42
Multiple Application of Simpson’s 1/3 Rule

x1 x2 xn

I  f ( x)dx   f ( x)dx   f ( x)dx

x0 x1 xn 1
n 1 n2
f  x0   4  f x   2  f x   f x 
i j n

I  b  a  i 1, 3,5.. j  2 , 4 , 6..

3n

Ea 
b  a
5

f 4
180n 4

43
 n 1 n2
f  x0   4  f x   2  f x   f x 
i j n

I  b  a  i 1, 3,5.. j  2 , 4 , 6..

3n
The odd points represent the middle term for each
application. Hence carry the weight 4.
The even points are common to adjacent
applications and are counted twice.
i=1 (odd) i=2 (even)
weight of 4 weight of 2
f(x)

x 44
 Simpson’s 3/8 Rule
• Corresponds to the case where the function is a
third order polynomial
b b
I   f x dx   f 3 x dx
a a

f n x   a0  a1 x  a2 x  a3 x
2 3

I   f x0   3 f x1   3 f x2   f x3 

3h
8

45
 Composite Simpson’s 3/8 Rule
• We assume that within any three consecutive subintervals of
width h, the interpolating polynomial φ(x) approximating f(x)
is of degree 3.

3h
I  ( y0  yn )  3( y1  y2  y4  y5   yn 1 )  2( y3  y6  yn 3 ) 
8

Simpson’s three-eighths rule can be applied when the range

[a, b] is divided into a number of subintervals, which must
be a multiple of 3.

46
Simpson’s 1/3
and 3/8 rules
applied in
tandem to
handle multiple
applications with
odd numbers of
intervals.

47
48
Integration of Unequal Segments

• Experimental and field study data is often

unevenly spaced
• In previous equations we grouped the term (i.e.
hi) which represented segment width.
n 1
f  x0   2 f  xi   f  xn 
I  b  a  i 1

2n
f  x0   f  x1  f  x1   f  x2  f  xn  1   f  xn 
I h h  h
2 2 2
49
Integration of Unequal Segments
• We should also consider alternately using higher
order equations if we can find data in
consecutively even segments

trapezoidal
rule

50
 Integration of Unequal Segments
• We should also consider alternately using higher
order equations if we can find data in consecutively
even segments

1/3
trapezoidal rule
rule

51
 Integration of Unequal Segments
• We should also consider alternately using higher
order equations if we can find data in consecutively
even segments

1/3
trapezoidal rule
rule 3/8
rule

52
 Integration of Unequal Segments
• We should also consider alternately using higher
order equations if we can find data in consecutively
even segments

trapezoidal
1/3 rule
trapezoidal rule
rule 3/8
rule

53
 Integration with Unequal Segments
Using Trapezoidal Rule
f ( x0 )  f ( x1 ) f ( x1 )  f ( x2 ) f ( xn 1 )  f ( xn )
I  h1  h2    hn
2 2 2
Example
1.309  0.2 1.305  1.309 0.363  3.181 0.232  2.363
I  0.12  0.10    0.06  0.10
2 2 2 2
 0.0905  0.1307    0.12975  1.594
which represents a relative error x f(x) x f(x)
of e = 2.8% 0.0 0.2 0.44 2.842

0.12 1.309 0.54 3.507

Data for 0.22 1.305 0.64 3.181

0.32 1.743 0.70 2.363

f(x)= 0.2+25x-200x2+675x3-900x4+400x5
0.36 2.074 0.80 0.232

0.40 2.456
54
 Example
Integrate the following using the trapezoidal rule, Simpson’s
1/3 Rule, a multiple application of the trapezoidal rule with
n=2 and Simpson’s 3/8 Rule. Compare results with the
analytical solution.
1 4000

1 2000
4 1 0000

 dx
2x
xe f(x) 8000

6000

4000
0
2000

0
0 1 2 3 4
x

55
 Solution
• Exact solution
4

0
xe 2 x dx

4
4  x 2x 1 2x 
 dx   e  e 
2x
xe
0
2 4 0
1
1 2x
 e ( 2 x  1 )  5216.926477
4 0

56
• Trapezoidal Rule: Approximation Error
ε = -(b – a)3/(12) f’’’(η) (η is a number between a and b).
Let’s take η =2. Then
ε = -43/12*[2*exp(2*2)+2*exp(2*2)+4*(2)*exp(2*2)] =-
3494.282

• Trapezoidal Rule
4 0
 f ( 0 )  f ( 4 )  2( 0  4e 8 )  23847.66
4
I xe dx 
2x
0 2
5216.926  23847.66
  357.12%
5216.926

57
 Solution: Simpson’s Rules
4
• Evaluate the Integral
0
xe 2 x dx
• Simpson’s 1/3-Rule
h
 f (0)  4 f (2)  f (4)
4
I   0
xe 2 x dx 
3
2
 
 0  4(2e 4 )  4e8 
  8240.411
3
5216.926  8240.411
   57.96%
5216.926
• Simpson’s 3/8-Rule
4 3h  4 8 
I   xe 2 x dx  f (0)  3 f ( )  3 f ( )  f (4)
0 8  3 3 
3(4/3)
  0  3(19.18922)  3(552.33933)  11923.832  6819.209
8
5216.926  6819.209
  30.71%
5216.926
58
• Simpson’s 1/3-Rule: Approximation Error
ε = -(b – a)5/(2880) f (4)(η) (η is a number between a and b).
Since f (4)(η) >0, the error is negative (overshooting).
Let’s take η=2.5. Then,
ε = -45/(2880) [ exp(2*2.5)*[16*(2.5)+32] =-3799.3769

• Simpson’s 3/8-Rule: Approximation Error

ε = -(b – a)5/(6480) f (4)(η) (η is a number between a and b).

59
Παραδείγματα & Ασκήσεις

60
 Simpson Rule
• Approximating the function to be integrated by a
quadratic polynomial leads to the basic Simpon Rule:

• The approximate integral is given by

61
Simpson Rule – Example

62
Simpson Rule – Example

63
 Exercise
• Determine exact area A for following integral:
2
A   4 x3dx
0
4 x4 4 2
A  x   (2)4  (0) 4  16
4 0

• Determine approximate area A1 with first-order

integration algorithm and step-size of 0.05.
• Determine approximate area A2 with second-
order integration algorithm and step-size of 0.05.

64
 Trapezoidal Rule
• Given the data in the following table use the
trapezoid rule to estimate the integral from x = 1.8 to
x = 3.4. The data in the table are for ex and the true
value is 23.9144.
x f(x) x f(x)
1,6 4,953 2,8 16,445
1,8 6,05 3,0 20,086
2,0 7,389 3,2 24,533
2,2 9,025 3,4 29,964
2,4 11,023 3,6 36,598
2,6 13,464 3,8 44,701

65
 Example
6
Estimate 0 f ( x)dx given f ( x) is defined by

x 0 1 2 3 4 5 6
y -1 -0.5 0 1 3 1 0

Rectangle A  h( y1  y2  .... yn1 )  1(1  0.5  0  1  3  1)  3.5

h
Trapezium A  ( y1  2 y2  2 y3  ...  yn )
2
 0.5(1 2(0.5  0 1 31)  0)  4
h
Simpson A  ( y1  4 y2  2 y3  4 y4  2 y5  ...  4 yn1  yn )
3
1
 (1  4  0.5  2  0  4 1  2  3  4 1  0)  3.67
3
66
Composite Trapezoid Rule
To improve the Trapezoid Rule, first splits the interval of
integration [a,b] into N smaller, uniform subintervals, and then
applies the trapezoidal rule on each of them.
Two segments Three segments
7 7

6 6

5 5

4 4

3 3

2 2

1 1

0 0
3 5 7 9 11 13 15 3 5 7 9 11 13 15

7
Four segments 7
Many segments
6 6

5 5

4 4

3 3

2 2

1 1

0 0
67
3 5 7 9 11 13 15 3 5 7 9 11 13 15
 Composite Trapezoid Rule
• Use the Trapezoid Rule in n intervals. Then, add them together.
b x1 x2 xn
a
f(x)dx  
x0
f(x)dx  
x1
f(x)dx    
xn  1
f(x)dx


h
 f(x 0 )  f(x 1 )  h  f(x 1 )  f(x 2 )    h  f(x n1 )  f(x n )
2 2 2
  f(x 0 )  2 f(x 1 )    2f(x i )    2 f ( x n 1 )  f ( x n )
h
2
f(x)

ba
h
n
x0 h x1 h x2 h x3 h x4 x
68
Composite Trapezoid Rule
4
• Evaluate the integral I   xe 2 x dx
0

n  1, h  4  I 
h
 f ( 0 )  f ( 4 )  23847.66   357.12%
2
n  2 , h  2  I   f ( 0 )  2 f ( 2 )  f ( 4 )  12142.23   132.75%
h
2
n  4 ,h  1  I   f ( 0 )  2 f ( 1 )  2 f ( 2 )
h
2
 2 f ( 3 )  f ( 4 )  7288.79   39.71%
n  8 , h  0.5  I   f ( 0 )  2 f ( 0.5 )  2 f ( 1 )
h
2
 2 f ( 1.5 )  2 f ( 2 )  2 f ( 2.5 )  2 f ( 3 )
 2 f ( 3.5 )  f ( 4 )  5764.76   10.50%
n  16 , h  0.25  I   f ( 0 )  2 f ( 0.25 )  2 f ( 0.5 )  
h
2
 2 f ( 3.5 )  2 f ( 3.75 )  f ( 4 )
 5355.95   2.66% 69
Composite Trapezoid Rule with Unequal Segments
4
• Evaluate the integral
0 
I  xe 2 x dx
Use the following hi‘s: {h1=2, h2=1, h3=0.5, h4=0.5}

2 3 3.5 4
I   f ( x)dx   f ( x)dx   f ( x)dx   f ( x)dx
0 2 3 3.5

  f (0)  f (2)   f (2)  f (3)

h1 h2
2 2
  f (3)  f (3.5)   f (3.5)  f (4)
h3 h4
2 2
2
2
  
1
 0  2e 4  2e 4  3e 6 
2
0.5 6
2

3e  3.5e 7 

0.5
2
 
3.5e 7  4e8  5971.58    14.45%
70
Composite Simpson’s Rule
• Piecewise Quadratic approximations

ba
h
n
f(x)

…...

x0 h x1 h x2 h x3 h x4 xn-2 xn-1 xn x
71
 Composite Simpson’s Rule
4
• Evaluate the integral I   xe 2 x dx
0
• Using n = 2, h = 2

I   f ( 0 )  4 f ( 2 )  f ( 4 )
h
3
2
 
 0  4( 2e 4 )  4 e 8  8240.411    57.96%
3
• Using n = 4, h = 1

I   f ( 0 )  4 f ( 1 )  2 f ( 2 )  4 f ( 3 )  f ( 4 )
h
3
1

 0  4 ( e 2 )  2( 2 e 4 )  4 ( 3e 6 )  4 e 8
3

 5670.975    8.70%
72
Composite Simpson’s Rule with Unequal Segments
4
• Evaluate the integral I   xe 2 x dx
0
Using h1 = 1.5, h2 = 0.5

3 4
I   f ( x)dx   f ( x)dx
0 3

  f (0)  4 f (1.5)  2 f (3)

h1
3
  f (3)  4 f (3.5)  2 f (4)
h2
3

1.5
3

0  4(1.5e )  3e 
3 6
0. 5 6
3

3e  4(3.5e 7 )  4e8 
 5413.23    3.76% 73
 Richardson Extrapolation
• Richard extrapolation methods use two estimates of an integral to
compute a third, more accurate approximation.
• If two O(h2) estimates I(h1) and I(h2) are calculated for an integral using
step sizes of h1 and h2, respectively, an improved O(h4) estimate may be
formed using:
1
I  I(h2 ) I(h2 )  I(h1 )
(h1 / h2 ) 1
2

• For the special case where the interval is halved (h2=h1/2), this becomes:

 4 1
I  I(h2 )  I(h1 )
3 3

74

 Richardson Extrapolation
• Example
– Use Richardson extrapolation to evaluate the integral of
f(x) = 0.2 + 25x – 200x2 + 675x3 – 900x4 + 400x5 from a = 0 to b = 0.8
– Single and composite application of the trapezoidal rule:
Segments h Integral t
1 0.8 0.1728 89.5%
2 0.4 1.0688 34.9%
4 0.2 1.4848 9.5%
– The estimates for one and two segments can be combined
4 1 Et = 1.640533-1.367467=0.273067
I  (1.0688)  (0.1728)  1.367467 ( = 16.6%)
3 3 t

– The estimates for two and four segments can be combined

4 1
I (1.4848 )  (1.0688 )  1.623467 Et = 1.640533-1.623467 = 0.017067
3 3 (t = 1.0%)
75
 Richardson Extrapolation (cont)

• For the cases where there are two O(h4) estimates and the
interval is halved (hm=hl/2), an improved O(h6) estimate may
be formed using:
16 1
I  I m  Il
15 15
• For the cases where there are two O(h6) estimates and the
interval is halved (hm=hl/2), an improved O(h8) estimate may
be formedusing:
64 1
I I m  Il
63 63

76
 The Romberg Integration Algorithm
• Note that the weighting factors for the Richardson
extrapolation add up to 1 and that as accuracy increases, the
approximation using the smaller step size is given greater
weight.
In general, 4 k1 I j1,k1  I j,k1
I j,k 
4 k1 1

where ij+1,k-1 and ij,k-1 are the more and less accurate integrals,
 and i is the new approximation. k is the level
respectively, j,k

of integration and j is used to determine which approximation

is more accurate.
77
 Romberg Integration
• Generalization of Simpson
– Based on numerical analysis to remove more terms in
error series associated with the numerical integral
• Uses trapezoid as building block as does Simpson
– Romberg is adequate for smooth (analytic) integrands,
over intervals with no singularities, where endpoints are
not singular
– Romberg is much faster than Simpson or the elementary
routines. For a sample integral:
• Romberg: 32 iterations
• Simpson: 256 iterations
• Trapezoid: 8192 iterations
78
 Romberg Integration

• Romberg Integration is an extrapolation of two results of the

trapezoid method applied to a regular partition cut into
subintervals of sizes that are consecutive powers of 2.

A0 4 A1  A0
B1 
A1 3 16 B2  B1
4 A2  A1 C1 
B2  15
A2 3
16 B3  B2
4 A3  A2 C2 
A3 B3  15
3

 

79
The values for Romberg’s Method of integration can be arranged in a triangular
matrix.
0 1 2 3 4 5 The best
0 A0 possible
1 A1 B1 estimate in
2 A2 B2 C1 each row is the
right most
3 A3 B3 C2 D1
entry:
4 A4 B4 C3 D2 E1
A0,B1,C1,D1,E1,F
5 A5 B5 C4 D3 E2 F1
1

Since we will run out of letters after 26 rows we express this with R(i,j).
0 1 2 3 4 5 The best
0 R(0,0) possible
1 R(1,0) R(1,1)
estimate in
each row is the
2 R(2,0) R(2,1) R(2,2)
right most
3 R(3,0) R(3,1) R(3,2) R(3,3) entry:
4 R(4,0) R(4,1) R(4,2) R(4,3) R(4,4)
R(i,i)
5 R(5,0) R(5,1) R(5,2) R(5,3) R(5,4) R(5,5)
80
The R(i,j) can be 0 1 2 3 4 5
obtained in the
following way: 0 R(0,0)
1 R(1,0) R(1,1)
R(i,0) is always the
2 R(2,0) R(2,1) R(2,2)
trapezoid method
applied with a partition 3 R(3,0) R(3,1) R(3,2) R(3,3)
size of 2j i. 4 R(4,0) R(4,1) R(4,2) R(4,3) R(4,4)
4 Ri, j  1  Ri  1, j  1
Ri, j   5 R(5,0) R(5,1) R(5,2) R(5,3) R(5,4) R(5,5)
4 1
j

For the rest of them a triangular pattern exists as how you fill in the other entries.
4 R(1,0)  R(0,0)
R(1,1) 
4 1
4 R(2,0)  R(1,0) 42 R(2,1)  R(1,1)
R(2,1)  R(2,2) 
4 1 42  1
4 R(3,0)  R(2,0) 42 R(3,1)  R(2,1) 43 R(3,2)  R(2,2)
R(3,1)  R(3,2)  R(3,3) 
4 1 42  1 43  1
4 R(4,0)  R(3,0) 42 R(4,1)  R(3,1) 43 R(4,2)  R(3,2)
R(4,1)  R(4,2)  R(4,3) 
4 1 42  1 43  1
4 R(5,0)  R(4,0) 4 2 R(5,1)  R(4,1) 43 R(5,2)  R(4,2)
R(5,1)  R(5,2)  R(5,3) 
4 1 42  1 43  1 81
• The chart below shows the process by which lower
level integrations are combined to produce more
accurate estimates:

82
 Example: i xi yi=f(xi)
0 0 0
Apply Romberg Extrapolation to the data to the right to
1 ½ 1
get the best possible estimate for: 2
 0
f ( x) dx 2 1 4

R(0,0)  12  21 0 0  16  16

3 3/2 9
4 2 16

R(1,0)  12  220 0  2  4  16  12

4 (12) 16
R(1,1)  3  32
3

4 1112 4 2  32  323
R(2,0)  1
2
 0  2 1  2  4  2  9  16  11
2 0
4 R(2,1)  3  32
3
R(2,2)  2
3
4 1
 323

The pattern that exists for the Romberg Entries is as follows:

j-1 j 4 j R(i, j  1)  R(i  1, j  1)

i-1 R(i-1,j-1) R(i, j ) 
4 j 1
i R(i,j-1) R(i,j)
Note: This only applies for i>0 and j>0.
83
 Example
2
Estimate the integral to the right with 3 rows of the
Romberg Method of integration. 0
x2
2
sin xdx  ?
R(0,0)  12 2  00  0  0

R(1,0)  12  220 0  2  0  0  0

R(2,0)  12  240 0  2  14  2  0  2  49   4  4  

Building the triangular matrix:

R (0,0)  0
40  0
R (1,0)  0 R(1,1) 
3
0

4  ( )  0  4 16   43   0  64

R (2,0)   R(2,1) 
3

3
R(2,2) 
15

45

84
 Romberg Integration
4 k I j 1 ,k  I j ,k
I j ,k  ; k  1, 2, 3,
4 1
k

k 0 k 1 k 2 k 3 k 4
O(h 2 ) O(h 4 ) O(h 6 ) O(h8 ) O(h10 )
h I 0, 0 I 0,1 I 0, 2 I 0,3 I 0, 4
h/2 I1, 0 I1,1 I1, 2 I1,3
h/4 I 2, 0 I 2,1 I 2, 2
h/8 I 3, 0 I 3,1
h / 16 I 4, 0
4 I j 1, 0  I j , 0 16 I j 1,1  I j ,1 64 I j 1, 2  I j , 2 256 I j 1,3  I j ,3
3 15 63 255
85
 Romberg Integration
4
I   xe dx  5216.926477
2x
0

k 0 k 1 k 2 k 3 k 4
O(h 2 ) O(h 4 ) O(h 6 ) O(h8 ) O(h10 )
h4 23847.7 8240.41 5499.68 5224.84 5216.95
h2 12142.2 5670.98 5229.14 5217.01
h 1 7288.79 5256.75 5217.20
h  0.5 5764.76 5219.68
h  0.25 5355.95
  2.66%  0.0527%  0.0053%  0.00168%  0.00050%

86
 Romberg Integration: preliminaries
• The starting point for Romberg integration is the composite
trapezoid rule which we may write in the following form

h n 1
 h2
f ( x) dx   f (a)  f (b)  2 f ( xi )  [ f ' (a)  f ' (b)]
b
I 
a 2 i 1  12

• Can define a series of trapezoid integrations each with a

successively larger m and thus more sub-divisions.
– Let n=2k-1 : k=1 => 1 interval
– The widths of the intervals are given by hk = (b-a)/2k-1
hk   hk2
k 1
2 1
f ( x) dx   f (a)  f (b)  2  f (a  ihk )  [ f ' (a)  f ' (b)]
b
I 
a 2 i 1  12
87
Define this as Rk,1 (it’s just the comp. trap. rule)
 Romberg Integration: preliminaries
• The Rk,1 describes a family of
progressively more accurate estimates
• Can show that there is a relationship
between successive Rk,1
h2
1 
k 2
2
 
R k ,1   R k 1,1  hk 1  f (a  (2i  1)hk )
2 
R2,1
 i 1 
• Each new Rk,1 adds 2k-2 new interior
points in the evaluation
• Series converges comparatively slow
h3
Re-use old values in
the new calculation! R3,1
88
 
Consider

0
sin x dx  2
k=1 
R1,1  [sin(0)  sin( )]  0
2

k=2 1  
R2,1  [ R1,1   sin( )] 
2 2 2

k=3 1   3
R3,1  [ R2,1  [sin( )  sin( )]]  1.8962
2 2 4 4

k=4 R4,1  1.9742

k=5 R5,1  1.9936

k=6 R6,1  1.9984

Converges really fairly slowly… 89

 Romberg Table
Error:
Using our Rk , j 1  Rk 1, j 1
Rk , j  Rk , j 1  O(hk2j)
previous
definition: 4 j 1  1

R1,1
Obtain from Construct Romberg Table
a single R2,1 R2,2
composite
trapezoidal R3,1 R3,2 R3,3 I=Rn,n + O(hn2n)
integration
R4,1 R4,2 R4,3 R4,4

Rn,1 Rn,2 Rn,3 Rn,4 … Rn,n

I=Rn,1 + O(hn2) 90
 
Consider previous example
0 sin x dx  2

0.00000000

1.57079633 2.09439511

1.89611890 2.00455976 1.99857073

1.97423160 2.00026917 1.99998313 2.00000555

1.99357034 2.00001659 1.99999975 2.00000001 1.99999999

1.99839336 2.00000103 2.00000000 2.00000000 2.00000000 2.00000000

Error in R6,6 is only 6.61026789e-011 – Accurate to

very rapid convergence O(h612)

91
• The table below shows computation of the
following integral:

92
 Gauss Quadrature

• Gauss quadrature describes a class of

techniques for evaluating the area
under a straight line by joining any
two points on a curve rather than
simply choosing the endpoints.
• The key is to choose the line that
balances the positive and negative
errors.

93
 Gaussian Quadratures
• Newton-Cotes Formulae
– use evenly-spaced functional values
• Gaussian Quadratures
– Select functional values at non-uniformly distributed points
to achieve higher accuracy. The values are not
predetermined, but unknowns to be determined.
– Change of variables => the interval of integration is [-1,1].
– Gauss-Legendre formulae
– Rule is stated as
1 n

 f(x)dx   c
1 i 1
i f ( xi )

94
95
 Gauss-Legendre Formulas
• The Gauss-Legendre formulas seem to optimize estimates to
integrals for functions over intervals from -1 to 1.
• Integrals over other intervals require a change in variables to
set the limits from -1 to 1.
• The integral estimates are of the form:
I  c0 f x0   c1 f x1    cn1 f xn1 
where the ci and xi are calculated to ensure that the method
exactly integrates up to (2n-1)th order polynomials over the
interval
 from -1 to 1.

96
 Gaussian Quadratures
• The Gauss-Legendre quadrature formula is stated as
1 n

 f(x)dx   c
1 i 1
i f ( xi )

the ci's are called the weights, the xi's are called the quadrature
nodes. The approximation error term, ε, is called the truncation
error for integration.

For Gauss-Legendre quadrature, the nodes are chosen to be

zeros of certain Legendre polynomials.

97
 Change of Interval for Gaussian Quadrature
• Coordinate transformation from [a,b] to [-1,1]
This can be done by an affine transformation on t and a
change of variables.
ba ba
t x
2 2
ba
dt  dx
2
 x  1  t  a

x  1  t  b
a t1 t2 b
ba ba ba ba n

b 1
a
f (t )dt   f (
1 2
x
2
)(
2
)dx 
2 i 1
ci f ( xi )
98
Gaussian Quadrature on [-1, 1]
• Gauss Quadrature General formulation:
n
f ( x )dx   c i f ( x i )  c 1 f ( x 1 )  c 2 f ( x 2 )    c n f ( x n )
1
1
i 1

1
n  2:  1
f(x)dx
 c1f(x1 )  c 2 f(x 2 )

-1 x1 x2 1
• For n=2, we have four unknowns (c1, c2, x1, x2). These are found by
assuming that the formula gives exact results for integrating a general 3rd
order polynomial. It can also be done by choosing (c1, c2, x1, x2) such that it
yields “exact integral” for f(x) = x0, x1, x2, x3.
99
 Gaussian Quadrature on [-1, 1]
1
Case n  2   1
f(x)dx  c1f(x 1 )  c 2 f(x 2 )

Exact integral for f = x0, x1, x2, x3

– Four equations for four unknowns

f 1
 1   1dx  2  c 1  c 2 c 1  1
 1
c  1
 1
 2
f  x   xdx  0  c 1 x 1  c 2 x 2 1
1 
 12   x1 
f  x   x dx   c 1 x 12  c 2 x 22
2 2
 3
1
 3  1
f 1
 x   x 3 dx  0  c 1 x 13  c 2 x 23
3  x2  3
 1


1 1 1
I   f ( x )dx  f (  ) f ( )
1
3 3 100
 Gaussian Quadrature on [-1, 1]
1
Case n  3 : 1
f ( x)dx  c1 f ( x1 )  c2 f ( x2 )  c3 f ( x3 )

-1 x1 x2 x3 1
• Now, choose (c1, c2, c3, x1, x2, x3) such that the method yields “exact
integral” for f(x) = x0, x1, x2, x3,x4, x5. (Again, (c1, c2, c3, x1, x2, x3) are
calculated by assuming the formula gives exact expressions for
integrating a fifth order polynomial).
101
 Gaussian Quadrature on [-1, 1]
1
f  1   xdx  2  c1  c2  c3
1
1
f  x   xdx  0  c1 x1  c2 x2  c3 x3
1 c1  5 / 9
1 c  8 / 9
2
f  x   x dx   c1 x12  c2 x22  c3 x32
2 2  2
1
3 c3  5 / 9

 x1   3 / 5
1
f  x 3   x 3dx  0  c1 x13  c2 x23  c3 x33
1
 x2  0

 x3  3 / 5
1
2
f  x   x dx   c1 x14  c2 x24  c3 x34
4 4

1
5
1
f  x 5   x 5 dx  0  c1 x15  c2 x25  c3 x35
1 102
 Gaussian Quadrature on [-1, 1]
• Approximation formula for n=3

1 5 3 8 5 3
I f ( x )dx  f (  ) f (0 ) f ( )
1 9 5 9 9 5

103
Example (1): Gaussian Quadrature
4
Evaluate I   te 2 t dt  5216.926477
0

Coordinate transformation
ba ba
t x  2 x  2 ; dt  2dx
2 2
4 1 1
  
4 x4
I te dt 
2t
( 4 x  4 )e dx  f ( x )dx
0 1 1

Two-point formula (n=2)

1
4 4
1 1 4 4 4 4
I   f ( x)dx  f ()  f ( )  (4  )e  (4  )e 3 3
1
3 3 3 3
 9.167657324  3468.376279  3477.543936 (  33.34%)
104
Example (1): Gaussian Quadrature
Three-point formula (n=3)
1 5 8 5
I   f ( x )dx  f (  0.6 )  f ( 0 )  f ( 0.6 )
1 9 9 9
5 8 5
 ( 4  4 0.6 )e 4  0.6  ( 4 )e 4  ( 4  4 0.6 )e 4  0.6
9 9 9
5 8 5
 ( 2.221191545 )  ( 218.3926001 )  ( 8589.142689 )
9 9 9
 4967.106689 (   4.79%)

Four-point formula (n=4)

f ( x)dx  0.34785  f (0.861136 )  f (0.861136 )

1
I 
1
 0.652145  f (0.339981 )  f (0.339981 )
 5197 .54375 (  0.37 %)
105
Example (2): Gaussian Quadrature
x2
1.64 

2 
Evaluate 1
I e 2 dx  .44949742
0

Coordinate transformation
ba ba
t x  .82 x  .82  .82 (1  x); dt  .82 dx
2 2
t2 1
1.64  1  [.82(1 x )]2 1

2  2  2 
1 .82 .82
I e 2 dt  e 2 dx  f ( x)dx
0 1 1
106
 Example (2): Gaussian Quadrature
Two-point formula (n=2)
 1 1 2 1 1 
1 .82   1 1  .82  2  [. 82(1 )]  [.82(1 )]2


.82
I f ( x)dx   f ( )  f ( )   e 3
e 2 3
2 1 2  3 3  
2  
 
 0.32713267 * (0.9417114 7 + 0.43323413 )  .44978962 (  0.065 %)

Three-point formula (n=3)

1 .82  5 

.82 8 5
I f ( x)dx   f ( 0.6 )  f (0)  f ( 0.6 ) 
2 1 2  9 9 9 

.82  5 2 
1
 2

1
 2 1
5 2 [.82(1 0.6 )]
 2 
 e
[. 82(1 0. 6 )] 8 2
 e
[. 82(1 0 )]
 e 

2  9 9 9 

 .32713267 * (0.5461465 9 + 0.63509351 + 0.19271450 )
 0.44946544 (  0.007 %)
107
 Derivation of the Two-Point
Gauss-Legendre Formula
• Example : Two-Point Gauss-Legendre Formula
– Integrate the following function in between 0 and 0.8. The exact value
of integration is 1.640533
f ( x)  0.2  25x  200x 2  675x3  900x 4  400x5
– Change the variables first
x  0.4  0.4 xd dx  0.4dxd
0.8

0
(0.2  25 x  200 x 2  675 x 3  900 x 4  400 x 5 )dx
1
  [0.2  25(0.4  0.4 x d )  200 (0.4  0.4 x d ) 2  675 (0.4  0.4 x d ) 3
-1

 900 (0.4  0.4 x d ) 4  400 (0.4  0.4 x d ) 5 ]0.4dxd

– Applying the Gauss-Legendre formula:

0.516741 + 1.305837 = 1.822578
108
 Adaptive Quadrature

• Methods such as Simpson’s 1/3 rule has a

disadvantage in that it uses equally spaced points - if
a function has regions of abrupt changes, small steps
must be used over the entire domain to achieve a
certain accuracy.
• Adaptive quadrature methods for integrating
functions automatically adjust the step size so that
small steps are taken in regions of sharp variations
and larger steps are taken where the function
changes gradually.

109
 Improper Integrals
• An improper integral is basically an integral that has
infinity as its limits or has a discontinuity within its
limits.
• Examples of improper integrals:

 1
 -1  x 2
x
e dx dx
0

110
• The improper integral of f over the interval [a,)
is defined to be
 b

 f ( x)dx  lim  f ( x)dx

a
b 
a

• In the case where the limit exists, the improper

integral is said to converge, and the limit is defined to
be the value of the integral. In the case where the
limit does not exist, the improper integral is said to
diverge, and is not assigned a value.
111
• Improper integral defined as having integrable singularity or
approaching infinity at limit of integration
– Use extended midpoint rule instead of trapezoid rule to
avoid function evaluations at singularities or infinities
• Must know where singularities or infinities are
– Use change of variables: often replace x with1/t to
convert an infinity to a zero
• Done implicitly in many routines

112