ADVANCED ENGINEERING

MATHEMATICS

CODE: EHAEM4A
Lecturer: Lawrence (R201)
Topic 3: Finite Difference Methods and Interpolation
 Introduction

• Most engineering and science problems are in the form of

differential equations especially nonlinear and involving large
sets of simultaneous equations which do not have analytical
solutions and require solutions by application of numerical
methods.
• Numerical methods such as differentiation, integration and
solution of ODEs and PDEs are based on calculation of finite
differences.
• This chapter presents a relationship between finite differences
and differential operators needed in numerical solution of
ODEs and PDEs.
– The calculation of finite differences could either involve numerically
integrating a differential equation,
– Or conversely using available finite data to differentiate or integrate
using finite differences method
– Derivation of interpolation/extrapolation formulas (interpolating
polynomials) used to present data which has no actual functionality
 Symbolic operators

• In differential calculus, the definition of the derivative is given as:

𝑑𝑓(𝑥) 𝑓(𝑥) − 𝑓(𝑥0 )
ቤ = 𝑓 ′ 𝑥0 = lim
𝑑𝑥 𝑥 𝑥→𝑥0 𝑥 − 𝑥0
0

• 𝑥 − 𝑥0 does not approach zero but remains a finite quantity represented by h:

ℎ = 𝑥 − 𝑥0
• Therefore the derivative can be expressed as:
𝑓 𝑥𝑜 + ℎ − 𝑓(𝑥𝑜 )
𝑓 ′𝑥𝑜 =
ℎ
• If a point ξ is the interval (a, b), for which the derivative can be calculated as above,
finite value theorem can be calculate this point.
• If f(x) is a continuous range between 𝑎 and 𝑏 (𝑎 ≤ 𝑥 ≤ 𝑏) and differentiable in the 𝑎 ≤
𝑥 ≤ 𝑏 range
• then there exist at least one 𝜉, 𝑎 < 𝜉 < 𝑏, which is:
𝑓 𝑏 − 𝑓(𝑎)
𝑓′ 𝜉 =
𝑏−𝑎

• This forms a basis for both differential calculus and finite difference calculus
 Symbolic operators

• A continuous and differentiable function 𝑓(𝑥) in the interval (𝑋𝑜 , 𝑥) can be

represented by a Taylor series:

𝑥 − 𝑥𝑜 2 𝑓 ′′𝑥𝑜 𝑥 − 𝑥𝑜 3 𝑓 ′′′
𝑥𝑜
𝑓(𝑥) = 𝑓(𝑥𝑜 ) + 𝑥 − 𝑥𝑜 𝑓 ′𝑥𝑜 + + +⋯
2! 3!
𝑥 − 𝑥𝑜 2 𝑓 ′′𝑥𝑜
…+ + 𝑅𝑛 (𝑥)
2!

• Where 𝑅𝑛 (𝑥) is the remainder summing up the remaining terms in the infinite
series from (𝑛 + 1) to infinity. This can be represented using mean-value
theorem to give:
𝑥 − 𝑥0 𝑛+1 𝑓 𝑛+1 (𝜉)
𝑅𝑥 𝑥 =
𝑛+1 !
• The value of 𝜉 is an unknown function therefore it is impossible to evaluate
the remainder, or truncation term error.
• The order of the remainder will always be specified.
 Symbolic operators

• The calculus of finite differences is evaluated with Taylor series of discrete values e.g.
𝑦𝑖−3 𝑦𝑖−2 𝑦𝑖−1 𝑦𝑖 𝑦𝑖+1 𝑦𝑖+2 𝑦𝑖+3

• Or as a continuous function:
𝑦(𝑥−3ℎ) 𝑦(𝑥−2ℎ) 𝑦(𝑥−ℎ) 𝑦(𝑥) 𝑦(𝑥+1ℎ) 𝑦(𝑥+2ℎ) 𝑦(𝑥+3ℎ)

• Also as values of a function:

𝑓(𝑥−3ℎ) 𝑓(𝑥−2ℎ) 𝑓 𝑥−ℎ 𝑓(𝑥) 𝑓(𝑥+1ℎ) 𝑓(𝑥+2ℎ) 𝑓(𝑥+3ℎ)

• The values of the dependent variable 𝑦 or 𝑓 are those corresponding to equally

spaced of the independent variable 𝑥.
 Symbolic operators

• Linear symbolic operators are:

𝒟 = 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡𝑖𝑎𝑙 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟
𝑖 = 𝑖𝑛𝑡𝑒𝑔𝑟𝑎𝑙 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟
𝐸 = 𝑠ℎ𝑖𝑓𝑡 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟
∆= 𝑓𝑜𝑟𝑤𝑎𝑟𝑑 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟
∇= 𝑏𝑎𝑐𝑘𝑤𝑎𝑟𝑑 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟
𝛿 = 𝑐𝑒𝑛𝑡𝑟𝑎𝑙 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟
𝜇 = 𝑎𝑣𝑒𝑟𝑎𝑔𝑒𝑟 𝑜𝑝𝑒𝑟𝑎𝑡𝑜𝑟

• The differential operator D has the following effect when applied to a function
𝑦(𝑥):
𝑑𝑦(𝑥)
𝐷𝑦 𝑥 = = 𝑦′
𝑑𝑥
• And the integral operator is:
𝑥+ℎ
𝐼𝑦 𝑥 = න 𝑦 𝑥 𝑑𝑥
𝑥
• The integral operator is equivalent to the inverse of the differential operator
𝐼 = 𝐷−1
 Symbolic operators

• Shift operator causes the function to shift to the next successive value of the
independent variable:
𝐸𝑦 𝑥 = 𝑦(𝑥 + ℎ)
• The inverse of the shift operator, E-1 causes the function to shift in the negative
direction of the independent variable:
𝐸 −1 𝑦 𝑥 = 𝑦(𝑥 − ℎ)
• Higher powers of the shift operator are defined as:
𝐸 𝑛 𝑦 𝑥 = 𝑦(𝑥 + 𝑛ℎ)
• The shift operator can be expressed in terms of the differential operator by
expanding the function 𝑦(𝑥 + ℎ) into a Taylor series about 𝑥:
ℎ ′ ℎ2 ′′ ℎ3 ′′′
𝑦 𝑥 + ℎ = 𝑦 𝑥 + 𝑦 𝑥 + 𝑦 𝑥 + 𝑦 𝑥 +. .
1! 2! 3!
• Using the differential operator D to indicate the derivatives of y, we obtain:
ℎ ℎ2 2 ℎ3 3
𝑦 𝑥 + ℎ = 𝑦 𝑥 + 𝐷𝑦 𝑥 + 𝐷 𝑦 𝑥 + 𝐷 𝑦 𝑥 +. .
1! 2! 3!
 Symbolic operators

• Factoring out the term 𝑦(𝑥) from the RHS

ℎ ℎ2 2 ℎ3 3
𝑦 𝑥 + ℎ = 1 + 𝐷 + 𝐷 + 𝐷 +. . 𝑦(𝑥)
1! 2! 3!
• The term in parenthesis are equivalent to the series expansion
2 3
ℎ ℎ ℎ
𝑒 ℎ𝐷 = 1 + 𝐷 + 𝐷2 + 𝐷3 +. .
1! 2! 3!
• Therefore it becomes:
𝑦 𝑥 + ℎ = 𝑒 ℎ𝐷 𝑦(𝑥)
• It can be concluded that the shift operator can be expressed in terms of
differential operator by relation:
𝐸 = 𝑒 ℎ𝐷
 Symbolic operators

• Similarly, the inverse of the shift operator can be related to the differential
operator by expanding the function 𝑦 𝑥 − ℎ into a Taylor series about 𝑥:
ℎ ′ ℎ2 ′′ ℎ3 ′′′
𝑦 𝑥 − ℎ = 𝑦 𝑥 − 𝑦 𝑥 + 𝑦 𝑥 − 𝑦 𝑥 +. .
1! 2! 3!
• Replacing the derivatives with differential operators and rearranging we obtain:
ℎ ℎ2 2 ℎ3 3
𝑦 𝑥 − ℎ = 1 − 𝐷 + 𝐷 − 𝐷 +. . 𝑦(𝑥)
1! 2! 3!
• The term in parenthesis are equivalent to the series expansion
−ℎ𝐷
ℎ ℎ2 2 ℎ3 3
𝑒 = 1 − 𝐷 + 𝐷 − 𝐷 +. .
1! 2! 3!
• Therefore it becomes:
𝑦 𝑥 − ℎ = 𝑒 −ℎ𝐷 𝑦(𝑥)
• It can be concluded that the shift operator can be expressed in terms of
differential operator by relation:
𝐸 −1 = 𝑒 −ℎ𝐷
Backward finite differences
Backward finite differences
 Backward finite differences

• From earlier definition, ∇ is a backward operator.

• Therefore first backward difference of 𝑦 at 𝑖 or 𝑥 is defined as:
∇𝑦𝑖 = 𝑦𝑖 − 𝑦𝑖−1
𝑂𝑟
∇𝑦 𝑥 = 𝑦 𝑥 − 𝑦(𝑥 − ℎ)
• Second backward difference of 𝑦 at 𝑖 or 𝑥 is defined as:
∇2 𝑦𝑖 = ∇(∇𝑦𝑖 ) = ∇ 𝑦𝑖 − 𝑦𝑖−1 = ∇𝑦𝑖 − ∇𝑦𝑖−1
= (𝑦𝑖 −𝑦𝑖−1 ) − (𝑦𝑖−1 − 𝑦𝑖−2 )
∇2 𝑦𝑖 = 𝑦𝑖 −2𝑦𝑖−1 − 𝑦𝑖−2
𝑜𝑟
∇2 𝑦 𝑥 = 𝑦 𝑥 − 2𝑦 𝑥 − ℎ + 𝑦(𝑥 − 2ℎ)
• Third backward difference operator of 𝑦 at 𝑖 or 𝑥 is defined as:
∇3 𝑦𝑖 = ∇ ∇2 𝑦𝑖 = ∇ 𝑦𝑖 − 2𝑦𝑖−1 + 𝑦𝑖−2 = ∇𝑦𝑖 − 2∇𝑦𝑖−1 + ∇𝑦𝑖−2
= (𝑦𝑖 −𝑦𝑖−1 ) − 2(𝑦𝑖−1 − 𝑦𝑖−2 ) + (𝑦𝑖−2 − 𝑦𝑖−3 )
∇3 𝑦𝑖 = 𝑦𝑖 −3𝑦𝑖−1 + 3𝑦𝑖−2 − 𝑦𝑖−3
• The coefficient of the terms in each of above finite difference corresponds to the
binomial expansion (𝑎 − 𝑏)𝑛 , where n is the order of the finite difference.
Forward finite differences
Forward finite differences
 Forward finite differences

• From earlier definition, ∆ is a backward operator.

• Therefore first forward difference of 𝑦 at 𝑖 or 𝑥 is defined as:
∆𝑦𝑖 = 𝑦𝑖+1 − 𝑦𝑖
𝑂𝑟
∆𝑦 𝑥 = 𝑦 𝑥 + ℎ − 𝑦(𝑥)
• Second forward difference of 𝑦 at 𝑖 or 𝑥 is defined as:
∆2 𝑦𝑖 = ∆(∆𝑦𝑖 ) = ∆ 𝑦𝑖+1 − 𝑦𝑖 = ∆𝑦𝑖+1 − ∆𝑦𝑖
= (𝑦𝑖+2 −𝑦𝑖+1 ) − (𝑦𝑖+1 − 𝑦𝑖 )
∆2 𝑦𝑖 = 𝑦𝑖+2 −2𝑦𝑖+1 − 𝑦𝑖
𝑜𝑟
∆2 𝑦 𝑥 = 𝑦 𝑥 + 2ℎ − 2𝑦 𝑥 + ℎ + 𝑦(𝑥)
• Third forward difference operator of 𝑦 at 𝑖 or 𝑥 is defined as:
∆3 𝑦𝑖 = ∆ ∆2 𝑦𝑖 = ∆ 𝑦𝑖+2 − 2𝑦𝑖+1 + 𝑦𝑖 = ∆𝑦𝑖+2 − 2∆𝑦𝑖+1 + ∆𝑦𝑖
= (𝑦𝑖+3 −𝑦𝑖+2 ) − 2(𝑦𝑖+2 − 𝑦𝑖+1 ) + (𝑦𝑖+1 − 𝑦𝑖 )
∆3 𝑦𝑖 = 𝑦𝑖+3 −3𝑦𝑖+2 + 3𝑦𝑖+1 − 𝑦𝑖
• The coefficient of the terms in each of above finite difference corresponds to the
binomial expansion (𝑎 − 𝑏)𝑛 , where n is the order of the finite difference.
Central finite differences
 Central finite differences

• Alternatively:

• Neglecting the remainder term:

 Central finite differences

• From earlier definition, 𝛿 is a backward operator.

• Therefore first central difference of 𝑦 at 𝑖 or 𝑥 is defined as:
𝛿𝑦𝑖 = 𝑦𝑖+1Τ2 − 𝑦𝑖−1Τ2
𝑂𝑟
𝛿𝑦 𝑥 = 𝑦 𝑥 + 1Τ2 ℎ − 𝑦(𝑥 −1Τ2 ℎ)
• Second central difference of 𝑦 at 𝑖 or 𝑥 is defined as:
𝛿 2 𝑦𝑖 = 𝛿(𝛿𝑦𝑖 ) = 𝛿 𝑦𝑖+1Τ2 − 𝑦𝑖−1Τ2 = 𝛿𝑦𝑖+1Τ2 − 𝛿𝑦𝑖−1Τ2
= (𝑦𝑖+1 −𝑦𝑖 ) − (𝑦𝑖 − 𝑦𝑖−1 )
𝛿 2 𝑦𝑖 = 𝑦𝑖+1 −2𝑦𝑖 − 𝑦𝑖−1
𝑜𝑟
𝛿 2 𝑦 𝑥 = 𝑦 𝑥 + ℎ − 2𝑦 𝑥 + 𝑦(𝑥 − ℎ)
• Third central difference operator of 𝑦 at 𝑖 or 𝑥 is defined as:
𝛿 3 𝑦𝑖 = 𝛿 𝛿 2 𝑦𝑖 = 𝛿 𝑦𝑖+1 − 2𝑦𝑖 + 𝑦𝑖−1 = 𝛿𝑦𝑖+1 − 2𝛿𝑦𝑖 + 𝛿𝑦𝑖−1
= (𝑦𝑖+11ൗ −𝑦𝑖+1ൗ ) − 2(𝑦𝑖+1Τ2 − 𝑦𝑖−1Τ2 ) + (𝑦𝑖−1Τ2 − 𝑦𝑖−11ൗ )
2 2 2
𝛿 3 𝑦𝑖 = 𝑦𝑖+11ൗ −3𝑦𝑖+1ൗ + 3𝑦𝑖−1Τ2 − 𝑦𝑖−11ൗ
2 2 2
• The coefficient of the terms in each of above finite difference corresponds to the
binomial expansion (𝑎 − 𝑏)𝑛 , where n is the order of the finite difference.
 Finite differences - Example

Problem:
• Use forward, backward and central difference approximations to estimate the
derivative of the equation below at x=0.5 using a step size h=0.5.
• Repeat the computation using h=0.25

•
Finite differences - Example
Finite differences - Example
Finite differences – To do
 Interpolating Polynomials

• Engineers and scientists are often faced with the task of

estimating intermediate values between precise data points
or interpreting and correlating experimental observations in
discrete data formats.
• The approach is to integrate or differentiate these data
numerically or graphically using extrapolation or interpolation
formulas.
• Polynomial interpolation is the most common method that
represent experimental data when the actual functionality of
the data is not known.
• The polynomials can also be used to approximate functions
that are difficult to integrate or differentiate.
 Interpolating Polynomials

• Taking the known values of the function f(x) over (n+1) sets of values of the
independent variable x.
x0 f(x0)
x1 f(x1)
X2 f(X2)

xn f(xn)

• The objective is to develop an interpolating polynomial that fits exactly the points of
the function and connects these points with smooth curve.
𝑃𝑛 𝑥 = 𝑎0 + 𝑎0 𝑥 + 𝑎2 𝑥 2 + 𝑎3 𝑥 3 + ⋯ … . +𝑎𝑛 𝑥 𝑛
• This polynomial can then be used to approximate the function at any value of the
independent variable x between the base points.
• For given (n+1) known base points, the polynomial must satisfy the equation:
𝑃𝑛 𝑥𝑖 = 𝑓 𝑥𝑖 𝑖 = 0, 1, 2, … . , 𝑛
• Substituting the known values of (𝑥𝑖 , 𝑓 𝑥𝑖 ) into the previous equation yields a set of
(n+1) simultaneous linear algebraic equations whose unknown values are the
coefficients 𝑎0 , 𝑎0 , … . , 𝑎𝑛 of the polynomial equation.
 Interpolation of equally spaced points

Gregory-Newton Interpolation
• For a set of known values of the function f(x) at equally spaced values of x:

• These points are tabulated for first, second and third forward and backward differences.

• Go through the notes on the textbook (Alkis C. and Navid M) pp 168 – 172.
Interpolation of equally spaced points
 Gregory-Newton interpolation formulas

• Used for interpolating equally spaced data

• Gregory-Newton forward interpolation formula:

X F(x)
0 0
2 4
4 56
6 204
8 496
10 980
 Gregory-Newton interpolation formulas

• Gregory-Newton backward interpolation formula:

• Go through the notes on the textbook (Alkis C. and Navid M) pp 171.
 Gregory-Newton interpolation formulas

Problem:
• Find the interpolating polynomial satisfying the following data using Newton’s
forward interpolation formula
x 0 2 4 6 8 10
F(x) 0 4 56 204 496 980

Solution:
X F(x) (y) ∆𝒚 ∆𝟐 𝒚 ∆𝟑 𝒚 ∆𝟒 𝒚

0 0
4
2 4 48
52 48
4 56 96 0
148 48
6 204 144 0
292 48
8 496 192
484
10 980
 Gregory-Newton interpolation formulas

Solution:
• Newton’s forward interpolation formula
• ℎ=2
𝑥−𝑥𝑜 𝑥−0 𝑥
• 𝛼= = =
ℎ 2 2
• Substituting these values into the polynomial
• 𝑓 𝑥 = 𝑥 3 − 2𝑥
 Newton’s divided-difference interpolation

• General form of Newton’s interpolating polynomial

 Newton’s divided-difference interpolation

• Finite differences

• Interpolating polynomial for Newton’s divided-difference method:

 Interpolation – Examples

Problem:
• Estimate the natural logarithm of 2 given the following: ln 1=0, ln 4=1.386294
and ln 6=1.791759.
• Take note that the true value of ln 2=0.6931472
Solution:
• Using the nth order polynomial:
 Interpolation – Examples

• Fitting second-order polynomial for three points used:

 Interpolation – Examples

Problem:
• Estimate the ln 2 with a parabola using the following sets of data with a third
Newton’s interpolating polynomial
Variable x Ln x
1 0
4 1.386294
6 1.791759
5 1.609438

Solution:
• The third order polynomial with 𝑛 = 3 is:

• Solving for the first divided differences:

 Interpolation – Examples

• Second divided differences:

• Third divided difference:

• The results for 𝑓 𝑥1 , 𝑥𝑜 , 𝑓 𝑥2 , 𝑥1 , 𝑥𝑜 , 𝑎𝑛𝑑𝑓 𝑥3 , 𝑥2 , 𝑥1 , 𝑥𝑜 are the coefficients of

b1, b2 and b3 respectively. While 𝑏𝑜 = 𝑓 𝑥𝑜 = 0.
 Interpolation – Examples

Problem:
• Using the following set of data, evaluate the 𝑓 𝑥 = ln 𝑥 𝑎𝑡 𝑥 = 2.