Numerical Analysis Module 3

Problem Discretization using Approximation Theory

Sachin C. Patwardhan
Dept. of Chemical Engineering,
Indian Institute of Technology, Bombay
Powai, Mumbai, 400 076, Inda.
Email: sachinp@iitb.ac.in

Contents
1 Polynomial Approximation [4] 3

2 Approximation using Truncated Taylor Series Expansion 5

2.1 Local Polynomial expansion using Taylor series [14, 9] . . . . . . . . . . . . . 5
2.2 Necessary and Su¢ cient Conditions for Unconstrained Optimization . . . . . 9
2.2.1 Necessary Condition for Optimality . . . . . . . . . . . . . . . . . . . 9
2.2.2 Su¢ cient Condition for Optimality . . . . . . . . . . . . . . . . . . . 10
2.3 Newton’s Method for Solving Nonlinear Algebraic Equations . . . . . . . . . 12
2.4 Taylor Series Methods for Solving ODE-IVP . . . . . . . . . . . . . . . . . . 14
2.5 Discretization using the Finite Di¤erence Method [3] . . . . . . . . . . . . . 17
2.5.1 Local Approximation of Spatial Derivatives using Taylor Series . . . . 17
2.5.2 Discretization of ODE-BVPs . . . . . . . . . . . . . . . . . . . . . . . 19
2.5.3 Discretization of PDEs [3] . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Discretization using Polynomial Interpolation 34

3.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.1.1 Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.1.2 Interpolation using Linearly Independent Functions . . . . . . . . . . 36
3.2 Method of Orthogonal Collocations [3] . . . . . . . . . . . . . . . . . . . . . 37
3.2.1 Discretization of ODE-BVPs . . . . . . . . . . . . . . . . . . . . . . . 37
3.2.2 Discretization of PDE’s [3] . . . . . . . . . . . . . . . . . . . . . . . . 44

1
 3.3 Piecewise Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . 52
3.3.1 Piecewise Polynomials Interpolation with First Order Continuous Deriv-
atives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.3.2 Cubic Splines [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4 Orthogonal Collocations on Finite Elements (OCFE) . . . . . . . . . . . . . 59
3.5 Solving ODE-IVPs using Polynomial Interpolation . . . . . . . . . . . . . . . 70
3.5.1 Multi-Step Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.5.2 Orthogonal Collocations . . . . . . . . . . . . . . . . . . . . . . . . . 73

4 Least Squares Approximations 74

4.1 ODE-BVP / PDE Discretization using Minimum Residual Methods . . . . . 76
4.1.1 Raleigh-Ritz method [11, 12] . . . . . . . . . . . . . . . . . . . . . . . 77
4.1.2 Method of Least Squares [5] . . . . . . . . . . . . . . . . . . . . . . . 82
4.1.3 Galerkin’s Method[5, 3] . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.2 ODE-BVP / PDE Discretization using the Finite Element Method . . . . . 88
4.2.1 Discretization of ODE-BVPs using Finite Elements [11] . . . . . . . . 89
4.2.2 Discretization of PDEs using the Finite Element Method [12] . . . . 93

5 Errors in Discretization and Computations [5] 96

6 Summary and Conclusions 97

2
 In the …rst module, we have listed and categorized di¤erent types of equations that arise
in a variety of engineering problems. The fundamentals of vector spaces were introduced in
the subsequent module. We then saw that the concept of vector space allows us to develop
a uni…ed representation of seemingly di¤erent problems, which were initially categorized as
algebraic equations, ODE-IVPs, ODE-BVP, PDEs etc., as a transformation of a vector from
one vector space to another. With this background, we are ready to start our journey in
numerical analysis.
When the transformations involved in a problem at hand are non-linear, it is often not
possible to solve the problem analytically. In all such cases, the problem is approximated
and transformed to a computationally tractable form, i.e.,
" # " #
Original Approximation Computationally Tractable
!
Problem Approximation

and we compute an approximate solution using the computable version. Figure 1 presents a
schematic representation of how a numerical solution scheme is formulated for a problem at
hand. It may be noted that the problem is transformed to one of the standard computable
forms and then one or more standard tools are used to construct approximate solution of the
original problem. In some way, a numerical solution scheme can be considered analogous to
a measuring instrument, which generates a reasonable approximation of a measured physical
variable in a transformed domain. The measurements are acceptable as long as the errors in
approximation are small. In this module, we explain the process of problem approximation
using various approaches available in the literature. In the end, we distill out generic equation
forms that frequently arise in the process of the problem approximation.

1 Polynomial Approximation [4]

Given an arbitrary continuous function over an interval, can we approximate it with another
”simple”function with an arbitrary degree of accuracy? This question assumes signi…cant
importance while developing many numerical methods. In fact, this question can be posed
in any general vector space. We often use such simple approximations while performing
computations. The classic examples of such approximations are the use of a rational number
to approximate an irrational number (e.g. 22=7 is used in place of or a …nite series
expansion is used for number e) or polynomial approximation of a continuous function.
This section discusses the rationale behind such approximations.

De…nition 1 (Dense Set) A set D is said to be dense in a normed space X; if for each
element x 2X and every " > 0; there exists an element d 2D such that kx dk < ":

3
 Figure 1: Formulation of Numerical Solution Scheme

Thus, if set D is dense in X, then there are points of D arbitrary close to any element of
X: Given any x 2 X, a sequence can be constructed in D which converges to x: A classic
example of such a dense set is the set of rational numbers in the real line: Another dense
set, which is widely used for approximations, is the set of polynomials. This set is dense
in C[a; b] and any continuous function f (t) 2 C[a; b] can be approximated by a polynomial
function p(t) with an arbitrary degree of accuracy as evident from the following result. This
classical result is stated here without giving proof.

Theorem 2 (Weierstrass Approximation Theorem): Consider space C[a; b], the set
of all continuous functions over interval [a; b], together with the 1 norm de…ned on it as

max
kf (t)k1 = jf (t)j (1)
t 2 [a; b]

Given any " > 0; for every f (t) 2 C[a; b] there exists a polynomial pn (t) such that kf (t) pn (t)k <
":

This fundamental result forms the basis of problem discretization in a majority of

cases. It may be noted that this is only an existence theorem and does not provide any
method of constructing a polynomial approximation. The following three approaches are
mainly used for constructing approximating polynomials:

Taylor series expansion

4
 Polynomial interpolation

Least squares approximation

These approaches and their applications to problem discretization will be discussed in

detail in the sections that follow.

2 Approximation using Truncated Taylor Series Ex-

pansion
Taylor series approximation is a method of constructing a local approximation of a function
in the neighborhood of a point of interest. It is typically used for constructing local approx-
imations of the function or the derivatives of the function. These approximations are then
used for transforming the problem to a computable form.

2.1 Local Polynomial expansion using Taylor series [14, 9]

To begin with let us consider a Taylor series expansion for a real valued scalar function.
Given any scalar function f (x) : R ! R; which is continuously di¤erentiable n + 1 times at
x = x, the Taylor series expansion of this function attempts to construct a local polynomial
approximation of the form

pn (x) = 0 + 1 (x x) + ::::: + n (x x)n (2)

of f (x) in the neighborhood of a point, say x = x; such that

dk pn (x) dk f (x)
= (3)
dxk dxk
for k = 0, 1, 2, ..., n. For k = 0, we have

pn (x) = 0 = f (x)

Similarly, for k = 1, the derivative condition (3) reduces to

dpn (x)
= 1 +2 2 (x x) + ::::: + n n (x x)n 1
x=x
dx
df (x)
) 1 =
dx

5
and, in general for the k’th derivative, we have
" #
dk pn (x) (k!) k + ((k + 1)k:::2) k+1 (x x) + :::::
=
dx k
+ (n(n 1):::(n k)) n (x x)n k
x=x
1 dk f (x)
) k = (4)
k! dxk
Thus, the local polynomial approximation pn (x) can be expressed as
df (x) 1 d2 f (x) 2 1 dn f (x)
pn (x) = f (x) + x+ ( x) + :::: + : ( x)n (5)
dx 2! dx2 n! dxn
where x = x x: The residual or the approximation error, rn (x; x); is de…ned as follows

rn (x; x) = f (x) pn (x) (6)

plays an important role in analysis. The Taylor theorem gives the following analytical
expression for the residual term
1 dn+1 f (x + x)
rn (x; x) = ( x)n+1 where (0 < < 1) (7)
(n + 1)! dxn+1
which is derived by application of the mean value theorem and the Rolle’s theorem on interval
[x; x] [14]. Thus, given a scalar function f (x) : R ! R; which is continuously di¤erentiable
n + 1 times at x = x, the Taylor series expansion of this function can be expressed as follows
df (x) 1 d2 f (x) 2 1 dn f (x)
f (x) = f (x) + x+ 2
( x) + :::: + n
: ( x)n + rn (x; x) (8)
dx 2! dx n! dx
While developing numerical methods, we require a more general, multi-dimensional ver-
sion of the Taylor series expansion. Given function F(x) : Rn ! Rm ; which is continuously
di¤erentiable n + 1 times at x = x, the Taylor series expansion of this function in the
neighborhood the point x = x can be expressed as follows

F(x) = Pn (x) + Rn (x; x) (9)

@F(x) 1 @ 2 F(x) 1 @ n F(x)
Pn (x) = F(x) + x+ ( x; x) + :::: + ( x; x; :::; x) (10)
@x 2! @x2 n! @xn
where x = x x and the residual Rn (x; x) is de…ned as follows
1 @ n+1 F(x + x)
Rn (x; x) = ( x; x; :::; x) where (0 < < 1) (11)
(n + 1)! @xn+1
h i h 2 i
Here, the F(x) 2 Rm ; Jacobian @F(x)
@x
is a matrix of dimension (m n), @ F(x)
@x2 is a
h r i
(m n n) dimensional array and so on. In general, @ @x F(x)
r is an (m n n::: n)

6
dimensional array such that when the vector x operates on it n times, the result is an
m 1 vector. It may be noted that the multi-dimensional polynomial given by equation (10)
satis…es the condition
dk Pn (x) dk F(x)
= (12)
dxk dxk
for i = 1,2,...n. The following two multidimensional cases are used very frequently in the
numerical analysis.

Case A: Scalar Function f (x) : Rn ! R

1 T
f (x) = f (x) + [rf (x)]T x + x r2 f (x) x + R3 (x; x)
2!
T
@f (x) @f @f @f
rf (x) = = ::::::
@x @x1 @x2 @xn x=x
2 3
@2f @2f @2f
6 :::::: 7
6 @x21 @x1 @x2 @x1 @xn 7
6 7
6 7
2 6 @2f 7
2 @ f (x) 6 @2f :::::: @2f 7
r f (x) = = 6 7
@x2 6 @x2 @x1 @x22 @x2 @xn 7
6 7
6 :::::: :::::: :::::: :::::: 7
6 7
4 @2f @2f @2f 5
::::::
@xn @x1 @xn @x2 @x2n x=x

1 X
n X
n X
n
@ 3 f (x + x)
R3 (x; x) = xi xj xk ; (0 < < 1)
3! i=1 j=1 k=1
@xi @xj @xk

Here, rf (x); referred to as the gradient, is an n 1 vector and, [r2 f (x)]; known
as the Hessian, is an n n matrix. It may be noted that the Hessian is always a
symmetric matrix.

Example 3 Consider the function vector f (x) : R2 ! R

f (x) = x21 + x22 + e(x1 +x2 )

h iT
which can be approximated in the neighborhood of x = 1 1 using the Taylor series

7
expansion as
@f1 @f1
f (x) = f (x) + x
@x1 @x2 x=x
2 2 3
@ f @2f
6 @x21 @x1 @x2 7
1 6 7
+ [ x]T 6
6
7
7 x+R3 (x; x) (13)
2 4 5
@2f @2f
@x2 @x1 @x22
"x=x #
h i x1 1
= (2 + e2 ) + (2 + e2 ) (2 + e2 )
x2 1
" #T " #" #
1 x1 1 (2 + e2 ) e2 x1 1
+ + R3 (x; x) (14)
2 x2 1 e2 (2 + e2 ) x2 1
Case B: Function vector F (x) : Rn ! Rn
@F (x)
F (x) = F (x) + x + R2 (x; x) (15)
@x
2 3
@f1 @f1 @f1
6 @x1 ::::::
6 @x2 @xn 7 7
6 7
6 7
@F (x) 6 @f2 @f2 :::::: @f2 7
= 6
6 @x
7
@x 6 1 @x2 @xn 77
6 :::::: :::::: :::::: :::::: 7
6 7
4 @fn @fn @fn 5
::::::
@x1 @x2 @xn x=x
@F (x)
Here, ; referred to as the Jacobian matrix, is an n n matrix.
@x
Consider the function vector F (x) 2 R2
" # " #
2 2
f1 (x) x1 + x2 + 2x1 x2
F (x) = =
f2 (x) x1 x2 e(x1 +x2 )
h iT
which can be approximated in the neighborhood of x = 1 1 using the Taylor series
expansion as follows
2 3
@f1 @f1
" # 6
f1 (x) 6 @x1 @x2 7 7
F (x) = +6 6 7 x + R2 (x; x)
f2 (x) 7
4 @f2 @f2 5
@x1 @x
" # " #" 2 x=x
#
4 4 4 x1 1
= + + R2 (x; x)
e2 2e2 2e2 x2 1

8
2.2 Necessary and Su¢ cient Conditions for Unconstrained Opti-
mization
Consider a real valued scalar function (x) : Rn ! R de…ned for any x 2 Rn : Let us assume
that (x) is at least twice di¤erentiable w.r.t. x:We are interested in …nding optimum
(maximum or minimum value) of (x) in Rn . Since …nding maximum of (x) can be re-cast
as …nding minimum of (x); let us only consider the problem of …nding a minimum of
(x). How do we de…ne a minimum point?

De…nition 4 (Global Minimum): If there exists a point x 2 Rn such that (x ) < (x)
for any x 2 Rn ; then x is called as the global minimum of (x):

De…nition 5 "-neighborhood of a point x be de…ned as the set N (x; ") = fx : kx xk "g

De…nition 6 (Local Minimum) : If there exists an " neighborhood NC (x) round x such
that (x) < (x) for each x 2 Ne (x); then x is called a local minimum.

Now, given that (x) is at least twice di¤erentiable w.r.t. x;can we derive conditions on
(x) that qualify a point, say x = x; to be a local minimum or a maximum (x)? To derive
these conditions, we use the Taylor series based approximation of (x).

2.2.1 Necessary Condition for Optimality

The necessary condition for optimality, which can be used to establish whether a given point
is a stationary (maximum or minimum) point, is given by the following theorem.

Theorem 7 If (x) is continuous and di¤erentiable and has an extreme (or stationary)
point (i.e., maximum or minimum) point at x = x; then
T
@ @ @
r (x) = :::::::::::::: =0 (16)
@x1 @x2 @xN x=x

Proof: Suppose x = x is a minimum point and one of the partial derivatives, say the
th
k one, does not vanish at x =x; then by Taylor’s theorem

X
N
@
(x + x) = (x) + (x) xi + R2 (x; x) (17)
i=1
@xi

@
i:e: (x + x) (x) = xk (x) + R2 (x; x) (18)
@xi

9
Since R2 (x; x) is of order ( xi )2 ; the terms of order xi will dominate over the higher
order terms for su¢ ciently small x: Thus, the sign of (x + x) (x) is decided by the
sign of
@
xk (x)
@xk
Suppose,
@
(x) > 0 (19)
@xk
then, choosing xk < 0 implies

(x + x) (x) < 0 ) (x + x) < (x) (20)

and (x) can be further reduced by reducing xk : This contradicts the assumption that
x = x is a minimum point. Similarly, if
@
(x) < 0 (21)
@xk
then, choosing xk > 0 implies

(x + x) (x) < 0 ) (x + x) < (x) (22)

and (x) can be further reduced by increasing xk : This contradicts the assumption that
x = x is a minimum point. Thus, x = x will be a minimum of (x) only if

@
(x) = 0 F or k = 1; 2; :::; n (23)
@xk
Similar arguments can be made if x = x is a maximum of (x):

2.2.2 Su¢ cient Condition for Optimality

The su¢ cient condition for optimality, which can be used to establish whether a stationary
point is a maximum or a minimum, is given by the following theorem.

Theorem 8 A su¢ cient condition for a stationary point x = x to be an extreme point (i.e.,
@2
maximum or minimum) is that the matrix (Hessian of ) evaluated at x = x is
@xi @xj

1. positive de…nite when x = x is a minimum

2. negative de…nite when x = x is a maximum

10
 Proof: Using the Taylor series expansion, we have
1
(x + x) = (x)+ [r (x)]T x+ xT r2 (x + x) x
2!
XN
@ 1 XX @ 2 (x +
N N
x)
= (x) + (x) x + xi xj (24)
i=1
@xi 2! i=1 j=1 @xi @xj
(0 < < 1) (25)

:Since x = x is a stationary point we have

r (x) = 0 (26)

Thus, the above equation reduces to

1 XX @ 2 (x +
N N
x)
(x + x) (x) = xi xj (27)
2! i=1 j=1 @xi @xj
(0 < < 1)

This implies that the sign of (a + x) (a) at the extreme point x is same as the sign of
@2
the R.H.S. Since the 2’nd partial derivative is continuous in the neighborhood of
@xi @xj
x = x; its value at x = x + x will have same sign as its value at x = x for all su¢ ciently
small x. If the quantity
XN X N
@ 2 (x + x)
xi xj ' ( x)T [r2 (x)] x 0 (28)
i=1 j=1
@xi @xj

for all x, then x = x will be a local minimum. In other words, if the Hessian matrix
,[r2 (x)], is positive semi-de…nite, then x = x will be a local minimum. If the quantity

XN X N
@ 2 (x + x)
xi xj ' ( x)T [r2 (x)] x 0 (29)
i=1 j=1
@x i @x j

for all x, then x = x will be a local maximum. In other words, if the Hessian matrix,
[r2 (x)], is negative semi-de…nite, then x = x will be a local maximum.
It may be noted that the need to de…ne positive de…nite or negative de…nite matrices
naturally arises from the geometric considerations while qualifying a stationary point in
multi-dimensional optimization problems. Whether a matrix is positive (semi) de…nite, neg-
ative (semi) de…nite or inde…nite can be established using algebraic conditions, such as the
sign of the eigenvalues of the matrix. If the eigenvalues of a matrix are all real positive (i.e.
i 0 for all i) then, the matrix is positive semi-de…nite. If the eigenvalues of a matrix are
all real negative (i.e. i 0 for all i) then, the matrix is negative semi-de…nite. When the
eigenvalues have mixed signs, the matrix is inde…nite.

11
2.3 Newton’s Method for Solving Nonlinear Algebraic Equations
One of the prominent application of the Taylor series expansion in the numerical analysis
is the Newton’s method (or the Newton-Raphson method), which is used for solving a set
of simultaneous nonlinear algebraic equations. Here, the local approximation of a nonlinear
function/ function vector is used to develop a numerical solution scheme. If we have a set of
n coupled nonlinear algebraic in n unknowns, then we can seldom solve them analytically.
Using Taylor series based approximation, we transform the problem to solving a sequence of
linear algebraic equations. If the iterations are initialized with a good initial guess, then the
iterations converge to a solution of the original problem.
To begin with, let us consider the problem of solving a nonlinear equation in one variable,
i.e. f (x) = 0 where x 2 R and f (x) is some nonlinear function, which is di¤erentiable at
df (x )
least once. Let us assume that x = x is a solution of such that f (x ) = 0 and 6= 0.
dx
Using the Taylor series expansion in the neighborhood of x = x ; we can write
df (e
x)
f (x ) = f (e
x+x e)
x f (e
x) + (x e)
x (30)
dx
under the assumption that R2 (x ; x) is su¢ ciently small and can be neglected. Here, x e
represents a point in a small neighborhood of x : Now, using the local approximation of the
function,f (x ) fe(x );we solve for
df (x )
fe(x ) = f (e
x) + (x e) = 0
x (31)
dx
which implies
f (e
x)
x x (32)
df (e
x)
dx
This leads us to an iteration scheme of the form
f (x(k) )
x(k+1) = x(k) (33)
df (x(k) )
dx
Then, at x(k) = x , we have
f (x )
x(k+1) = x =x (34)
df (x )
dx
df (x(k) )
e = x(0) , the sequence x(k+1)
If, starting from some x x(k) ! 0, and 6= 0, then
dx
f (x(k) ! 0 i.e. f (x(k) ) ! f (x ).

12
 Now, consider a set of m coupled nonlinear equations of the form

fi (x) = 0 for i = 1; ::::; m (35)

which have to be solved simultaneously, where x 2Rm . Here, each fi (:) : Rm ! R is a

scalar function. De…ning a function vector
h iT
F(x) = f1 (x) f2 (x) ::: fm (x)

the problem at hand is to solve the vector equation

2 3
f1 (x1 ; x2 ; :::; xm )
6 f2 (x1 ; x2 ; :::; xm ) 7
6 7
F(x) = 6 7=0
4 :::: 5
fm (x1 ; x2 ; :::; xm )

Suppose x is a solution such that F (x ) = 0: If each function fi (x) is continuously di¤er-

entiable, then, in the neighborhood of x we can approximate its behavior by Taylor series,
as
@F
F(x ) = F [ex+ (x x e)] = F(e
x) + (x x e) + R2 (x ; x x e) (36)
@x x=ex
where xe represents a guess solution. If the guess solution is su¢ ciently close to the true
solution, then, neglecting terms higher than the …rst order, we can locally approximate the
nonlinear transformation F(x ) as follows

e ) = F(e @F
F(x ) ' F(x x) + e
x
@x x=e
x
xe = x x e

and solve for

e )=0
F(x
The approximated operator equation can be rearranged as follows

@F
e] =
[ x x)
F(e
@x x=ex
(m m) matrix (m 1) vector = (m 1) vector

which corresponds to the standard form Ax = b: Solving the above linear equation yields
e and, if the guess solution x
x e is su¢ ciently close to true solution, then

x e+ x
x e (37)

13
However, we may not reach the true solution in a single iteration. Thus, equation (37) is
eN ew ; as follows
used to generate a new guess solution, say x

eN ew = x
x e+ e
x (38)

This leads us to an iteration scheme

x(k+1) = x(k) + x(k) (39)

@F
x(k) = F(x(k) ) (40)
@x x=x(k)
This process is continued till
e (k+1) ) < "1
F(x (41)
or
x(k+1) x(k)
< "2 (42)
kx(k+1) k
where the tolerances "1 and "2 are some user speci…ed su¢ ciently small numbers. This
derivation indicates that the Newton’s method is likely to converge only when the guess
e) can be
solution is su¢ ciently close to the true solution, x ; and the term R2 (x ; x x
neglected.

2.4 Taylor Series Methods for Solving ODE-IVP

Consider the problem of solving a scalar ODE-IVP
dx
= f (x; t) (43)
dt
where x 2 R; over a large interval 0 t tf starting from initial condition, x(0). Let
fx (t) : 0 t tf g denote the true solution of the ODE-IVP. Let us further assume that
x (t) is di¤erentiable k + 1 times, i.e. x (t) 2 C (k+1) 2 [0; tf ]. In general, for a nonlinear
ODE, it is seldom possible to obtain the true solution analytically. The aim of numerical
methods is to …nd an approximate solution numerically. Let t1 ; t2 ; :::::::; tn be a sequence of
numbers such that
0 < t1 < t2 < ::::::: < tn < :::: < tf (44)
such that fti = ih : i = 0; 1; 2; :::; N g where h represents interval of integration. Instead of
attempting to approximate the function, x (t); which is de…ned for all values of t such that
0 t tf , we attempt to approximate the sequence of vectors fx (tn ) : n = 1; :::::::N g: To

14
simplify the notation, let us de…ne x(n) x(tn ). Thus, in order to integrate over a large
interval 0 t tf ;we solve a sequence of ODE-IVPs subproblems
dx
= f (x; t) ; x(tn ) = x(n) (45)
dt
tn t < tn+1 ; (n = 0; 1; 2; ::::; N 1)

each de…ned over a smaller interval [tn ; tn+1 ] : This generates a sequence of approximate
solution vectors, fx(n) : n = 0; 1; :::; N g: Suppose we know the exact solution, x (tn ) =
x (n) and the integration step size, h; is selected su¢ ciently small, then we can compute,
x (n + 1); using the Taylor series expansion with respect to independent variable, t; as
follows:

x (n + 1) = x (tn + h) (46)
dx (tn ) 1 2 d2 x (tn )
= x (n) + h + h + ::::::: (47)
dt 2! dt2
The various derivatives in the above series can be calculated using the di¤erential equation,
as follows:
dx (tn )
= f [tn ; x (n)] (48)
dt
d2 x (tn ) df [tn ; x (n)]
2
= (49)
dt dt
and so on. Now, the exact di¤erential of f (x; t) can be expressed as
@f @f
df = dx + dt
@x @t
and we can write
d2 x (tn ) @f dx (tn ) @f [x (n); tn ]
= + (50)
dt2 @x (x (n);tn ) dt @t
@f @f [x (n); tn ]
= f [x (n); tn ] + (51)
@x (x (n);tn ) @t

Let us now suppose that, instead of the exact solution, x (n), we have available an ap-
proximation to x (n), denoted as x(n). With this information, we can construct x(n + 1); as
follows
" #
h2 @f @f
x(n + 1) = x(n) + hf (n) + f (n) + + ::::: (52)
2 @x (x (n);tn ) @t (x (n);tn )

where
f (n) f [x (n); tn ] (53)

15
We can now make a further approximation by truncating the in…nite series. If the Taylor
series is truncated after the term involving, hk , then the Taylor’s series method is said to be
of order, k. For example, the …rst two order methods are as follows:

Order 1 (Euler explicit formula)

x(n + 1) = x(n) + hf (n) (54)

Order 2
" #
h2 @f @f
x(n + 1) = x(n) + hf (n) + f (n) + (55)
2 @x (x (n);tn ) @t (x (n);tn )

It may be noted that, when we solve the ODE-IVP using any order Taylor series methods,
the resulting approximate solution is a sequence of numbers

X = fx(0); x(1); :::::; x(N )g 2 RN +1

Thus, the true solution x (t), which belongs to an in…nite dimensional vector space, has
been approximated using a …nite dimensional vector X 2 RN +1 : When improve on the
approximation by choosing smaller and smaller integration interval, h. However, the resulting
approximate solution is still a …nite dimensional vector.
This approach can be easily extended to solve a set of multivariable ODE-IVP of the
form 2 3 2 3
x1 f1 (x1 ; x2 ; :::; xm ; t)
dx d 6 7 6
6 x2 7 6 f2 (x1 ; x2 ; :::; xm ; t) 7
7
= 6 7 6
= 7 F(x;t) (56)
dt dt 4 :::: 5 4 :::: 5
xm fn (x1 ; x2 ; :::; xm ; t)
where x 2 Rn ; over a large interval 0 t tf starting from initial condition, x(0). Then,
the 1’st and 2’nd order Taylor’s series methods for numerically solving the ODE-IVP are as
follows:

Order 1 (Euler explicit formula)

x(n + 1) = x(n) + hF(n) (57)

Order 2
" #
h2 @F @F
x(n + 1) = x(n) + hF(n) + F(n) + (58)
2 @x (x(n);tn ) @t (x(n);tn )

16
 where
F(n) F [x (n); tn ] (59)
Note that @F @x (x(n);tn )
is an n n Jacobian matrix calculated at (x(n); tn ).
Use of di¤erent order Taylor series approximations and further simpli…cation of the deriv-
ative calculations leads to Runge-Kutta class of methods. A detailed exposition to this class
of algorithms is later presented in Module on ODE-IVPs Solvers.

2.5 Discretization using the Finite Di¤erence Method [3]

Another widely used application of Taylor series is constructing approximations of spatial
derivatives of a function. To begin with we present an application of the scalar Taylor
series expansion to discretization of ODE-BVPs and PDEs. Even when the domain of
the function under consideration is multivariate, the Taylor series approximation is applied
locally by considering one variable at a time.

2.5.1 Local Approximation of Spatial Derivatives using Taylor Series

Let function u(z) denote an n-times di¤erentiable function where the independent variable
z 2 [a; b]: Here z represents some spatial coordinate. Consider the problem of developing
a local approximation of the derivatives of u(z) at a point, say z = z; in (a; b): Let z
> 0 represent a small perturbation from z = z such that [z z; z + z] 2 [a; b]: If z is
su¢ ciently small, then, using the Taylor Series expansion, we can write
du(z) 1 d2 u(z) 2 1 d3 u(z)
u(z + z) = u(z) + ( z) + 2
( z) + 3
( z)3 + r4 (z; z) (60)
dz 2! dz 3! dz
Similarly, using the Taylor series expansion , we can express u(z z) as follows

du(z) 1 d2 u(z) 1 d3 u(z)

u(z z) = u(z) ( zi ) + ( z)2 ( z)3 + re4 (z; z) (61)
dz 2! dz 2 3! dz 3
du
From equations (60) and (61) we can arrive at several approximate expressions for at
dz
z = z. Rearranging equation (60) we obtain

du(z) [u(z + z) u(z)] d2 u(z) z

= + ::: (62)
dz z dz 2 2
and, when z is su¢ ciently small, then neglecting the higher order terms we obtain the
forward di¤erence approximation of the local …rst order derivative as follows
du(z) u(z + z) u(z)
'
dz z
17
Similarly, starting from equation (61), we can arrive at backward di¤erence approximation
of the local …rst order derivative, i.e.

du(z) u(z) u(z z)

'
dz z
It may be noted that the errors in the forward and the backward di¤erence approximation
are of the order of z, which is denoted as O( z): Alternatively, subtracting equation (61)
from (60) and rearranging we can arrive at the following expression

du(z) [u(z + z) u(z z)] (3) z2

= ui ( ) + ::: (63)
dz 2( z) 3!

and, for su¢ ciently small z, we obtain the central di¤erence approximation of the local
…rst order derivative by neglecting the terms of order higher than z 2 , i.e.

du(z) [u(z + z) u(z z)]

' (64)
dz 2( z)

The central di¤erence approximation is accurate to O[( z)2 ] and is more commonly used.
To arrive at an approximation for the second order derivatives at z = z, adding equation
(61) with (60) and rearranging, we have

d2 u(z) [u(z + z) 2u(z) + u(z z)] d4 u(z) ( z)2

2
= 2 + ::: (65)
dz ( z)2 dz 4 4!

When z is su¢ ciently small, we obtain the following approximation for the second deriva-
tive
d2 u(z) u(z + z) 2u(z) + u(z z)
2
' (66)
dz ( z)2
Note that errors in the approximations (64) and (66) are of order O[ z)2 ]. This process can
be continued to arrive at approximations of higher order derivatives at z = z:
The approach developed for functions of one independent variable can easily be ex-
tended to arrive at local approximations to partial derivatives of a continuously di¤erential
function in multiple variables. For example, let function u(x; y) denote an n-times di¤er-
entiable function where the independent x 2 (a; b) and z 2 (c; d): Consider the problem of
developing a local approximation of partial derivatives of u(x; y) at a point, say x = x 2 (a; b)
and y = y 2 (c; d): Let x > 0; y > 0 represent small perturbations from x = x; y = y
such that [x x; x + x] 2 [a; b] and [y y; y + y] 2 [c; d]: Then, using similar argu-
ments, we can arrive at the following approximations of the …rst and the second order partial
derivatives
@u(x; y) [u(x + x; y) u(x x; y)]
' (67)
@x 2( x)

18
 @u(x; y) [u(x; y + y) u(x; y y)]
' (68)
@y 2( y)

@ 2 (x; y) u(x + x; y) 2u(x + x; y) + u(x x; y)

2
' (69)
@x ( x)2
and so on.

2.5.2 Discretization of ODE-BVPs

Consider the following general form of the 2nd order ODE-BVP problem frequently encoun-
tered in engineering problems
d2 u du
; ; u; z = 0 f or z 2 (0; 1) (70)
dz 2 dz

du
B:C: 1 (at z = 0) : f1 ; u; 0 = 0 (71)
dz

du
B:C: 2 (at z = 1) : f2 ; u; 1 = 0 (72a)
dz
Let u (z) 2 C (2) [0; 1] denote the exact/true solution to the above ODE-BVP. Depending on
the nature of the operator ; it may or may not be possible to …nd the true solution to the
problem. In the present case, however, we are interested in …nding an approximate numerical
solution, say u(z); to the above ODE-BVP. The basic idea in the …nite di¤erence approach is
to convert the ODE-BVP into a set of coupled linear or nonlinear algebraic equations using
local approximations of the derivatives based on the Taylor series expansion. In order to
achieve this, the domain 0 z 1 is divided into (n + 1) grid points z1 ; ::::; zn ; zn+1 located
such that
z1 = 0 < z2 < z3 ::: < zn+1 = 1
The simplest option is to choose them equidistant, i.e.

zi = (i 1)( z) = (i 1)=(n) for i = 1; 2; :::; n + 1

which is considered for the subsequent development. Let the value of the approximate
solution, u(z); at location zi be denoted as ui = u(zi ): If z is su¢ ciently small, then
the Taylor Series expansion based approximations of the local derivatives presented in the
previous sub-section can be used to discretize the ODE-BVP. The basic idea is to enforce
the approximation of equation (70) at each internal grid point. The remaining equations are
obtained from discretization of the boundary conditions. While discretizing the ODE, it is
preferable to use the approximations having similar accuracies. Thus, the central di¤erence

19
approximation of the …rst derivative is preferred over the forward or the backward di¤erence
approximations as the order of the error in the approximations is O[( z)2 ]; which is similar
to the order of errors in the approximation of the second order derivatives. The steps involved
in the discretization can be summarized as follows:
Step 1 : Force the residual Ri at each internal grid point to zero, i.e.,
(ui+1 2ui + ui 1 ) (ui+1 ui 1 )
Ri = ; ; u i ; zi = 0 (73)
( z)2 2( z)
i = 2; 3:::; n: (74)
This gives rise to (n 1) equations in (n + 1) unknowns fui : i = 1; 2; :::; n + 1g.

Step 2: Use boundary conditions to generate the remaining algebraic equations. This
can be carried out using either of the following two approaches

– Approach 1: Use one-sided derivatives only at the boundary points, i.e.,

(u2 u1 )
f1 ; u1 ; 0 = 0 (75)
z
(un+1 un )
f2 ; un+1 ; 1 = 0 (76)
z
This gives the remaining two equations.
– Approach 2: This approach introduces two more variables u0 and un+2 at two
hypothetical grid points, which are located at
z0 = z1 z= z
zn+2 = zn+1 + z =1+ z
With the introduction of these hypothetical points, the boundary conditions are
evaluated as
(u u0 )
f1 [ 2 ; u1 ; 0] = 0 (77)
(2 z)
un+2 un
f2 [ ; un+1 ; 1] = 0 (78)
( z)
Now we have n + 3 variables and n + 1 algebraic constraints. Two additional
algebraic equations are generated by setting the residual at the boundary points
to zero, i.e., at z1 and zn+1 , i.e.,
(u2 2u1 + u0 ) (u2 u0 )
R1 (z = 0) = ; ; u1 ; 0 = 0
( z)2 2( z)
(un+2 2un+1 + un ) (un+2 un )
Rn+1 (z = 1) = ; ; un+1 ; 1 = 0
( z)2 2( z)
This results in (n + 3) equations in (n + 3) unknowns fui : i = 0; 1; 2; :::; n + 2g.

20
 It may be noted that the local approximations of the derivatives are developed under the
assumption that z is chosen su¢ ciently small. Consequently, it can be expected that the
quality of the approximate solution would improve with the increase in the number of grid
points.

Example 9 Consider the steady state heat transfer/conduction in a slab of thickness L, in

which energy is generated at a constant rate of q W=m3 . The boundary at z = 0 is maintained
at a constant temperature T ,while the boundary at z = L dissipates heat by convection with a
heat transfer coe¢ cient h into the ambient temperature at T1 . The mathematical formulation
of the conduction problem is represented as an ODE-BVP of the form

d2 T
k + q = 0 f or 0 < z < L (79)
dz 2

B:C: at z = 0 : T (0) = T (80)

dT
B:C: at z = L : k = h [T1 T (L)] (81)
dz z=L

Note that this problem can be solved analytically. However, it is used here to introduce the
concepts of discretization using the …nite di¤erence approach. Dividing the region 0 z L
into n equal subregions with z = L=n and setting the residuals zero at the internal grid
points, we have
(Ti+1 2Ti + Ti 1 ) q
+ =0 (82)
( z)2 k
for i = 2; 3; ::::n: Using the boundary condition (80) i.e. (T1 = T ), the residual at z2 reduces
to
q
2T2 + T3 = ( z)2 T (83)
k
Using one sided derivative at z = L, boundary condition (81) reduces to

(Tn+1 Tn )
k = h(T1 Tn+1 ) (84)
( z)
or
h z T1
Tn+1 (1 + ) Tn = h z (85)
k k
Rearranging all the equations in matrix form, we have

e
x=y
Ae
h iT
e=
x T2 T3 ::: Tn+1

21
 h iT
e=
y 2 2
( z) (q=k) T ( z) (q=k) ::: +h( z)T1 =k
2 3
2 1 0 0 :: 0
6 7
6 1 2 1 0 :: 0 7
6 7
6 0 1 2 1 :: 0 7
A =6
6 ::
7
7
6 :: :: :: :: :: 7
6 7
4 :: :: :: :: 2 1 5
0 0 :: :: 1 (1 + h z=k)
Thus, after discretization, the ODE-BVP is reduced to a set of linear algebraic equation and
the transformation operator Tb = A. It may also be noted that we end up with a tridiagonal
matrix A, which is a sparse matrix, i.e., it contains a large number of zero elements.

Example 10 Consider a isothermal tubular reactor with axial mixing (Isothermal TRAM)
in which an irreversible 2nd order reaction is carried out at a constant temperature. The
steady state behavior of this system can be modelled using the following ODE-BVP:

1 d2 C dC
DaC 2 = 0 (0 z 1) (86)
P e dz 2 dz

dC
B:C: at z = 0 : = P e(C 1) at z = 0; (87)
dz
dC
B:C: at z = 1 : =0 at z = 1; (88)
dz
Forcing the residuals at (n-1) internal grid points to zero, we have

1 Ci+1 2Ci + Ci 1 Ci+1 Ci 1

= DaCi2
Pe ( z)2 2 ( z)

i = 2; 3; :::n
De…ning

1 1 2 1 1
= ; = ; = +
( z)2 P e 2 ( z) P e ( z)2 2
( z) P e 2 ( z)

the above set of nonlinear equations can be rearranged as follows

Ci+1 Ci + Ci 1 = DaCi2

i = 2; 3; :::n

22
The two boundary conditions yield two additional equations
C2 C1
= P e(C1 1)
z
Cn+1 Cn
= 0
z
The resulting set of nonlinear algebraic equations can be arranged as follow

Tb (e
x) = Ae x) =0
x G(e (89)

where 2 3 2 3
C1 P e ( z)
6 7 6 7
6 C2 7 6 DaC22 7
6 7 6 7
e=6
x 6 ::: 7
7 x) = 6
; G(e 6 ::::: 7
7
6 7 6 2 7
4 ::: 5 4 DaCn 5
Cn+1 0
2 3
(1 + zP e) 1 0: ::: ::: 0
6 7
6 ::: ::: :: 7
6 7
A= 6
6 :::: ::: :: ::: ::: 0: 7 7 (90)
6 7
4 ::::: ::: :: ::: 5
0 ::: :: ::: 1 1
Thus, the ODE-BVP is reduced to a set of coupled nonlinear algebraic equations after dis-
cretization.
To provide some insights into how the approximate solutions change as a function of
the choice of n, we have carried out simulation studies on the TRAM problem (with Pe
= 6 and Da = 2). The resulting set of coupled nonlinear algebraic equations were solved
simultaneously using Newton’s method. Figure 2 demonstrates how the approximate solutions
behave as a function of the number of grid points. As can be expected, more re…ned solution
is obtained as the number of grid points are increased from 10 to 100. Further increase in
the number of grid points dose not change the solution signi…cantly.

Example 11 Consider a non-isothermal tubular reactor with axial mixing (Non-Isothermal

TRAM) in which an irreversible 2nd order reaction is carried out. The steady state behavior
of this PFR can be modelled using the following set of coupled ODE-BVPs:

1 d2 u du
+ (u; v) = 0 (0 < z < 1) (91)
P eM dz 2 dz
1 d2 v dv
+ (u; v) + (vH v) = 0 (0 < z < 1)
P eT dz 2 dz

23
 0.85

0.8

0.75 Finite Difference (n=10)

Finite Difference (n=100)
0.7
Concentration

0.65

0.6

0.55

0.5

0.45

0.4

0.35
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
z -->

Figure 2: TRAM Problem: Comparison of approximate solutions constructed using the

…nite di¤erence approach with di¤erent number of internal grid points

v
(u; v) = Da(1 u)n exp
1+ v

1 du 1 dv
B:C: at z = 0 : = u(0) and + v(1) = v(0) (92)
PeM dz PeT dz
du dv
B:C: at z = 1 : = 0 and =0 (93)
dz dz
where v = (T T0 )=( T ) represents dimensionless temperature, u = (CA0 CA )=CA0 rep-
resents degree of conversion, represents thermal recycle coe¢ cient, PeM represents mass
Peclet number, PeT represents thermal Peclet number,n represents the order of reaction, Da
represents Damkolher number, = E=RT0 is a dimensionless number related to activation
energy, is a dimensionless number related to adiabatic increase in temperature, is dimen-
sionless heat transfer coe¢ cient and vH is dimensionless temperature of the cooling medium.
Forcing the residuals at (n-1) internal grid points to zero, we have
1 ui+1 2ui + ui 1 ui+1 ui 1
+ (ui ; vi ) = 0
P eM ( z)2 2 ( z)
1 vi+1 2vi + vi 1 vi+1 vi 1
+ (ui ; vi ) + (vH vi ) = 0
P eT ( z)2 2 ( z)

i = 2; 3; :::n

24
This yields 2 (n 1) equations in 2 (n + 1) unknowns. Using the boundary conditions,
we get additional 4 equations
1 u2 u1 1 v2 v1
B:C: at z = 0 : = u1 and + vn+1 = v1 (94)
PeM z PeT z
un+1 un vn+1 vn
B:C: at z = 1 : = 0 and =0 (95)
z z
Thus, we have 2(n + 1) simultaneous nonlinear equations of the form, F(x) = 02 (n+1) ; in
2(n + 1) unknowns where
h iT
x = u1 ::: un+1 v1 ::: vn+1 2 R2(n+1)

The steady state conversion and dimensionless temperature pro…les obtained for the following
choice of parameters are shown in Figures (3) and (4), respectively. As can be seen from
these …gure, the solution obtained using a coarse grid (n = 10; z = 0:1) is signi…cantly
di¤erent from a …ner grid (n = 50; z = 0:0:2). Since smaller the interval size, z; better
is the approximation, it can be argued that the solution obtained using n = 50 is closer to
the true solution.

PeM = 100; PeT = 100; vH = 0; Da = 0:15

= 1:4; = 10; = 2; = 0:3;

2.5.3 Discretization of PDEs [3]

A typical second order PDE that we encounter in engineering problems is of the form
@u
ar2 u + bru + cg(u) = f (x; y; z; t)
@t
xL < x < xH ; y L < y < yH ; zL < z < zH
subject to appropriate boundary conditions and initial conditions. For example, the Lapla-
cian operators r2 and gradient operator r are de…ned in Cartesian coordinates as follows
@u @u @u
ru = + +
@x @y @z
@2u @2u @2u
r2 u = + +
@x2 @y 2 @z 2
In the Cartesian coordinate system, we construct grid lines parallel to the x, y and z axes
and force the residuals to zero at the internal grid points. For example, adopting the notation

ui;j;k = u(xi; yj; zk )

25
 0.7

0.6

0.5
Concversion

0.4

0.3

0.2 Finite Difference (n=10)

Finit Difference (n=50)
0.1 Orth. Colloc. (polynomial fn)
Orth. Colloc. (7 collocation points)
0
0 0.2 0.4 0.6 0.8 1
z -->

Figure 3: Non-Isothermal TRAM: Steady state conversion pro…le

0.4

0.35
Scaled Temperature

0.3

0.25

0.2
Finite Difference (n=10)
0.15
Finite Difference (n=50)
Orth. Colloc. (polynomial fn)
0.1
Orth. Colloc. (7 collocation points)
0.05
0 0.2 0.4 0.6 0.8 1
z -->

Figure 4: Non-Isothermal TRAM: Steady state dimensionless temperature pro…le

26
the partial derivative of the dependent variable u with respect to x at grid point (xi; yj; zk )
can be approximated as follows
@u (ui+1;j;k ui 1;j;k )
=
@x ijk 2( x)
2
@ u (ui+1;j;k 2ui;j;k + ui 1;j;k )
=
@x2 ijk ( x)2
The partial derivatives in the remaining directions can be approximated in an analogous
manner. It may be noted that the partial derivatives are approximated by considering one
variable at a time and is equivalent to the application of Taylor series expansion of a scalar
function.
When the PDE involves only the spatial derivatives, the discretization process yields
either a coupled set of linear/nonlinear algebraic equations or an ODE-BVP. When the
PDEs involve time derivatives, the discretization is carried out only in the spatial coordinates.
As a consequence, the discretization process yields coupled nonlinear ODEs with initial
conditions speci…ed, i.e. an ODE-IVP.
Example 12 Consider the PDE describing the unsteady state condition in an isothermal
tubular reactor with axial mixing (Isothermal TRAM) in which an irreversible 2nd order
reaction is carried out.
@C 1 @ 2 C @C
= 2
DaC 2 in (0 < z < 1) (96)
@t P e @z @z
t = 0 : c(z; 0) = f (z) in (0 < z < 1) (97)

@C(0; t)
B:C: at z = 0 : = P e (C(0; t) 1) for t 0 (98)
@z
@C(1; t)
B:C: at z = 1 : =0 for t 0 (99)
@z
Using the …nite di¤erence method along the spatial coordinate z with n 1 internal grid
points, we have
dCi (t) 1 Ci+1 (t)
2Ci (t) + Ci 1 (t)
= (100)
dt Pe ( z)2
Ci+1 (t) Ci 1 (t)
Da [Ci (t)]2 (101)
2 ( z)
i = 2; 3; ::::n
The boundary conditions yield
C2 (t)
C1 (t)
B:C:1 : = P e (C1 (t) 1)
z
1
1 C2 (t)
) C1 (t) = + Pe + Pe (102)
z z

27
and
Cn+1 (t) Cn (t)
B:C:2 : = 0 ) Cn+1 (t) = Cn (t) (103)
z
These boundary conditions can be used to eliminate variables C1 (t) and Cn+1 (t) from the set
of ODEs (100). This gives rise to a set of (n 1) coupled ODEs together with the initial
conditions
C2 (0) = f (z2 ) ; C3 (0) = f (z3 ) ; ::::: = Cn (0) = f (zn ) (104)
e of concentration values at the internal grid points as
Thus, de…ning vector x
h iT
e(0) =
x C2 (t) C3 (t) ::: Cn (t)

the discretized problem is an ODE-IVP of the form

de
x
Tb (e
x) = F (e
x) =0 (105)
dt
e(0). Needless to say that better approximation is obtained if
subject to the initial condition x
a large number of grid points are selected. Figures (5) and (6) show dynamic behavior of
the solution for the following scenario:

I:C: : Ci 1 (0) = exp( 0:1zi ) for i = 2; :::n where n = 40 (106)

( )
1 for t 2 [0; 2)
C(0; t) = (107)
2 for t 2 [2; 4]
The ODE-IVP has been solved using Explicit Euler method with integration interval, h =
0:001. As can be seen from these …gures, the chosen initial condition is an unsteady state
operating point. During interval t 2 [0; 2), the TRAM reaches the steady state condition that
corresponds to C(0; t) = 1: A step change is introduced in C(0; t) at t = 2. The …gures show
transient response due to this step change during interval t 2 [2; 4]. Figure (6) clearly shows
the time delay in the step response as grid points move from z = 0 to z = 1.

Example 13 Consider operation of the Non-Isothermal TRAM under unsteady state con-
ditions. The dynamic behavior of this system can be modelled using the following PDE:

@u 1 @ 2 u @u
= + (u; v) (0 < z < 1) (108)
@ P eM @z 2 @z
t = 0 : u(z; 0) = g(z) in (0 < z < 1) (109)
@v 1 @ 2 v @v
= + (u; v) + (vH v) (0 < z < 1)
@ P eT @z 2 @z
t = 0 : v(z; 0) = h(z) in (0 < z < 1) (110)

28
Figure 5: Isothermal TRAM : Concentration dynamics for step change in inlet concentration
(Finite Di¤erence method with n = 40)

Isotherm al T RAM : Dynam ic Sim ulati on (Explicit Euler) - Red: z = 0, Blue: z = 1

1.6

1.4

1.2
Concentration

0.8

0.6

0.4

0.2
0 0.5 1 1.5 2 2.5 3 3.5 4
Time

Figure 6: Isothermal TRAM: Concentration pro…les at individual grid points for step change
in inlet concentration. (Finite Di¤erence method with n = 40)

29
 v
(u; v) = Da(1 u)n exp
1+ v

1 du 1 dv
B:C: at z = 0 : = u(0) and + v(1) = v(0) for 0 (111)
PeM dz PeT dz
du dv
B:C: at z = 1 : = 0 and = 0 for 0 (112)
dz dz
where v represents dimensionless temperature and u represents degree of conversion. Dis-
cretizing the PDE at (n-1) internal grid points, we have get the following 2 (n 1) ODEs
in 2 (n + 1) unknowns
dui ( ) 1 ui+1 ( ) 2ui ( ) + ui 1 ( ) ui+1 ( ) ui 1 ( )
= + (ui ( ); vi ( ))
d P eM ( z)2 2 ( z)
dvi ( ) 1 vi+1 ( ) 2vi ( ) + vi 1 ( ) vi+1 ( ) vi 1 ( )
= + (ui ( ); vi ( )) + (vH vi ( ))
d P eT ( z)2 2 ( z)

i = 2; 3; :::n
Using the boundary conditions, we get additional 4 algebraic equations
1 u2 ( ) u1 ( ) 1 v2 ( ) v1 ( )
B:C: at z = 0 : = u1 ( ) and + vn+1 ( ) = v(113)
1( )
PeM z PeT z
un+1 ( ) un ( ) vn+1 ( ) vn ( )
B:C: at z = 1 : = 0 and =0 (114)
z z
Thus, de…ning
h iT
x( ) = u1 ( ) ::: un+1 ( ) v1 ( ) ::: vn+1 ( ) 2 R2(n+1)

we have 2(n + 1) simultaneous nonlinear di¤erential algebraic equations of the form,

dx
= F(x; )
d
04 1 = G(x( ))
together with an initial condition
h iT
x(0)= g(z1 ) ::: g(zn+1 ) h(z1 ) ::: h(zn+1 )

Example 14 The Laplace equation represents a prototype for steady state di¤usion processes.
For example the 2-dimensional Laplace equation
@2T @2T
+ = f (x; y) (115)
@x2 @y 2

30
 0<x<1; 0<y<1
where the T is temperature and x; y are dimensionless space coordinates. Equations similar
to this arise in many problems of ‡uid mechanics, heat transfer and mass transfer. In the
present case, T (x; y) represents the dimensionless temperature distribution in a furnace and
represents the thermal di¤usivity. Three walls of the furnace are maintained at a constant
temperature. Convective heat transfer occurs from the fourth boundary to the atmosphere.
The boundary conditions are as follows:

x = 0 : T =T ; x=1:T =T (116)
y = 0 : T =T (117)
dT (x; 1)
y = 1:k = h [T1 T (x; 1)] (118)
dy

We construct the 2 -dimensional grid with (nx + 1) equispaced grid lines parallel to the y axis
and (ny + 1) equispaced grid lines parallel to the x axis. The temperature T at the (i; j) th
grid point is denoted as Ti;j = T (xi; yj ):We then force the residual to be zero at each internal
grid point to obtain the following set of equations:

(Ti+1;j 2Ti ;j +Ti 1;j ) (Ti;j+1 2Ti ;j +Ti;j 1 )

+ = f (xi ; yj )= (119)
( x)2 ( y)2

for (i = 2; 3; :::; nx ) and (j = 2; 3; ::; ny ): Note that regardless of the size of the system, each
equation contains not more than …ve unknowns, resulting in a sparse linear algebraic system.
Consider the special case when
x= y=
For this case the above equations can be written as

2
Ti 1;j + Ti;j 1 4Ti;j + Ti;j+1 + Ti+1;j = f (xi ; yj ) (120)

for (i = 2; 3; :::; nx ) and (j = 2; 3; :::; ny )

Using the boundary conditions, we have additional equations

T1;j = T ; Tnx+1 ;j = T f or j = 1; 2; :::; ny

Ti;0 = T f or i = 1; 2; ::::nx + 1

Ti;ny+1 Ti;ny
k = h T1 Ti;ny +1
y
1
) Ti;ny +1 = hT1 + (k= y)Ti;ny
(k= y) + h

31
 f or i = 1; 2; ::::; nx + 1
that can be used to eliminate the boundary variables from the set of ODEs. Thus, we obtain
(nx 1) (ny 1) linear algebraic equations in (nx 1) (ny 1) unknowns. De…ning
e as
vector x
e = [T2;2 T2;3 ::::::T2;ny ; ::::; Tnx ;2 ; :::::::Tnx ;ny ]T
x
we can rearrange the resulting set of equations in the form Ae x = b, then A turns out to
be a large sparse matrix. Even for modest choice of 10 internal grid lines in each direction,
we would get a 100 100 sparse matrix associated with 100 variables.

Example 15 Converting a PDE to an ODE-BVP using the method of lines [3]:

Consider the 2-D steady state heat transfer problem in the previous example. Using the
method of lines, we discretize only in one spatial direction. For example, we choose nx 1
internal grid points along the x coordinate and construct nx 1 grid lines parallel to the
y-axis. The temperature T along the ith grid line is denoted as

Ti (y) = T (xi ; y) (121)

Now, we equate the residuals to zero at each internal grid line as

d2 Ti 1
2
= [Ti+1 (y) 2Ti (y) + Ti 1 (y)] + f (xi ; y)= (122)
dy ( x)2
i = 2; 3; ::::nx

The boundary conditions at x = 0 and x = 1 yield

T1 (y) = T ; Tnx +1 (y) = T

which can be used to eliminate the variables in the above set of ODEs that lie on the corre-
sponding edges. The boundary conditions at y = 0 and y = 1 are:

Ti (0) = T (123)
dTi (1)
k = h(T1 Ti (1)) (124)
dy
i = 2; 3; ::::nx
Thus, de…ning
h iT
e=
u T2 (y) T3 (y) ::: Tn (y)
discretization of the PDE using the method of lines yields the OBE-BVP of the form
2
dye
Tb (e
u) = 2 F [e
u] = 0
dy

32
subject to the boundary conditions

e (0) = T
u
de
u(1)
= G [e
u(1)]
dy
Example 16 Consider the 2-dimensional unsteady state heat transfer problem

@T @2T @2T
= [ 2 + ] + f (x; y; t) (125)
@t @x @y 2
t = 0 : T = H(x; y) (126)
x = 0 : T (0; y; t) = T ; x = 1 : T (1; y; t) = T (127)

y = 0 : T (x; 0; t) = T ; (128)
dT (x; 1; t)
y = 1 : k = h(T1 T (x; 1; t)) (129)
dy
where T (x; y; t) is the temperature at locations (x; y) at time t and is the thermal di¤usivity.
By the …nite di¤erence approach, we construct a 2-dimensional grid with nx 1 equispaced
grid lines parallel to the y-axis and ny 1 grid lines parallel to the x-axis. The temperature
T at the (i; j)th grid point is given by

Ti;j (t) = T (xi ; yi ; t) (130)

Now, we force the residual to zero at each internal grid point to generate a set of coupled
ODE-IVP’s as
dTi;j
= [Ti+1;j (t) 2Ti;j (t) + Ti 1;j (t)]
dt ( x)2
+ [Ti;j+1 (t) 2Ti;j (t) + Ti;j 1 (t)] + f (xi ; yj ; t) (131)
( y)2
for i = 2; 3; :::; nx and j = 2; 3; :::; ny
Using the boundary conditions, we have constraints at the four boundaries

T0;j (t) = T ; Tnx +1;j (t) = T f or j = 1; 2; :::ny + 1

Ti;0 (t) = T f or i = 1; 2; ::::nx + 1

Ti;ny +1 (t) Ti;ny (t)

k = h T1 Ti;ny +1 (t)
y
1
) Ti;ny +1 (t) = hT1 + (k= y)Ti;ny (t)
(k= y) + h

33
 f or i = 2; ::::nx
These constraints can be used to eliminate the boundary variables from the set of ODEs 131.
Thus, de…ning a vector
h iT
e(t) =
x T2;2 (t) T2;3 (t) ::::T2;ny (t) ::::Tnx ;2 (t) ::::Tnx ;ny (t)

the PDE after discretization is reduced to a set of coupled ODE-IVPs of the form

de
x
Tb (e
x) = F (e
x; t) =0
dt
e(0)
subject to the initial condition x
h iT
e(0) =
x H(x2 ; y2 ) H(x2 ; y3 ) :::: H(xnx ; y2 ) :::: H(xnx ; yny )

3 Discretization using Polynomial Interpolation

3.1 Interpolation
Consider a function u(z) to be a continuous function de…ned over z 2 [a; b] and let fu1 ; u2; :::un+1 g
represent the values of the function at an arbitrary set of points fz1 ; z2 ; ::::; zn+1 g in the
domain [a; b]; i.e. ui = u(zi ). Another function, say v(z) in C[a; b] that assumes values
fu1 ; u2; :::un+1 g exactly at fz1 ; z2 ; ::::; zn+1 g ; i.e. v(zi ) = ui for all i; is called an interpolation
function. The most popular form of interpolating functions are polynomials. Polynomial
interpolation has many important applications. It is one of the primary tools used in the ap-
proximation of the in…nite dimensional operators and generating computationally tractable
approximate forms. In this section, we examine applications of polynomial interpolation to
discretization. In the development that follows, for the sake of notational convenience, it is
assumed that grid points are de…ned on the domain [a; b] as follows

z1 = a < z2 < z3 < :::: < zn+1 = b (132)

3.1.1 Lagrange Interpolation

In Lagrange interpolation, it is desired to …nd an interpolating polynomial p(z) of the form

n
p(z) = 0 + 1z + ::::: + nz (133)

such that
p(zi ) = ui for i = 1; 2; :::n + 1

34
To …nd the coe¢ cients of the polynomial that passes exactly through fui : i = 1; 2; :::; n + 1g;
consider the (n+1) equations

n
0 + 1 z1 + ::::: + n z1 = u1
n
0 + 1 z2 + ::::: + n z2 = u2
:::: = ::::
n
0 + 1 zn+1 + ::::: + n zn+1 = un+1

which can be rearranged as follows

A =u (134)
where
h iT
= 0 1 ::: n (135)
h iT
u = u1 u2 ::: un+1 (136)
2 3
1 z1 ::: (z1 )n
6 1 z2 ::: (z2 )n 7
6 7
A =6 7
4 ::: ::: ::: ::::: 5
1 zn+1 ::: (zn+1 )n (n+1) (n+1)

Since the matrix A and the vector u are known, the coe¢ cients of the Lagrange interpolation
polynomial can be found by solving for vector : The matrix A is known as Vandermonde
matrix. Except for small values of n; this matrix is known to to ill-conditioned (refer to
Module 4 for details) and the resulting set of equations are di¢ cult to solve. An alternate
approach to construct an nth degree Legrange polynomial without requiring to solve a set of
linear algebraic equations is as follows [14]:

pn (z) = L1 (z)u1 + L2 (z)u2 + ::: + Ln+1 (z)un+1 (137)

where the polynomial Li (z) is chosen such that

Li (z) = 1 when z = zi and Li (z) = 0 when z 6= zi (138)

It is easy to show that [14]

Y
n+1
z zj
Li (z) = (139)
j=1;i6=j
zi zj

This is known as Legrange form of interpolating polynomial. For example, given 3 function
values, say u1 ; u2 ; u3 ; that correspond to points, z1 ; z2 ; z3 ; we can construct a 2nd order

35
Legrange polynomial as follows

(z z2 ) (z z3 ) (z z1 ) (z z3 ) (z z1 ) (z z2 )
p2 (z) = u1 + u2 + u3 (140)
(z1 z2 ) (z1 z3 ) (z2 z1 ) (z2 z3 ) (z3 z1 ) (z3 z2 )
= [c1 (z z2 ) (z z3 )] u1 + [c2 (z z1 ) (z z3 )] u2 + [c3 (z z1 ) (z z2 )] u3(141)
1 1 1
c1 = ; c2 = and c3 =
(z1 z2 ) (z1 z3 ) (z2 z1 ) (z2 z3 ) (z3 z1 ) (z3 z2 )
Now, the question is, will the 2nd polynomial obtained by solving for
2 32 3 2 3
1 z1 (z1 )2 0 u1
6 76 7 6 7
4 1 z2 (z2 )2 5 4 1 5 = 4 u2 5 (142)
1 z3 (z3 )2 2 u3

be di¤erent from (140)? It turns out that both of them are identical polynomials. This
assertion follows from the following theorem.

Theorem 17 [14] Given the values of u(z) at n + 1 distinct points zi : i = 1; 2; :::; n + 1,

there is a unique polynomial of degree atmost n which takes same values as u(z) at these
points.

Proof. Proof can be found in [14].

2
Polynomial (140) turns out to be a rearranged form of polynomial, 0 + 1z + 2z that
would be obtained by solving equation (142).

3.1.2 Interpolation using Linearly Independent Functions

While polynomials are a popular choice as a basis for interpolation, any set of linearly
independent functions de…ned on [a,b] can be used for developing an interpolating function.
Let ff0 (z); f1 (z); :::fn (z)g represent a set of linearly independent functions in C[a; b]: Then,
we can construct an interpolating function, g(z); as follows

g(z) = 1 f1 (z) + ::::::: + n+1 fn+1 (z) (143)

Forcing the interpolating function to have values ui at z = zi leads to the following set of
linear algebraic equations

1 f1 (zi ) + ::::::: + n+1 fn+1 (zi ) = ui (144)

for i = 1; 2; :::; n + 1

36
which can be further rearranged as A = u where
2 3 2 3
f1 (0) f1 (0) :::: fn+1 (0) 1
6 f2 (z2 ) f1 (z2 ) :::: fn+1 (z2 ) 7 6 7
6 7 6 2 7
A=6 7 and =6 7 (145)
4 :::: :::: :::: :::: 5 4 ::: 5
f0 (1) f1 (1) :::: fn+1 (1) (n+1) (n+1) m+1

and vectors and u are de…ned by equations (135) and (136), respectively. Commonly used
interpolating functions are

Shifted Legendre polynomials

Chebyshev polynomials

Trigonometric functions, i.e., sines and cosines

iz
Exponential functions fe : i = 0; 1; :::; ng with 0 :::: n speci…ed

3.2 Method of Orthogonal Collocations [3]

3.2.1 Discretization of ODE-BVPs

One of the important applications of polynomial interpolation is the method of orthogonal

collocations. By this approach, the di¤erential operator over a spatial / temporal domain
is approximated using an interpolation polynomial. Consider the second order ODE-BVP
given by equations (70), (71) and (72a). To see how the problem discretization can be carried
out using Lagrange interpolation, consider a selected set of collocation (grid) points fzi : i =
1; :::; n + 1g in the domain [0; 1] such that z1 = 0 and zn+1 = 1 and fz2 ; z3 ; ::::zn g 2 (0; 1)
such that
z1 = 0 < z2 < z3 < :::: < zn+1 = 1
Let fui = u(zi ) : i = 1; 2; :::; n + 1g represent the values of the dependent variable at these
collocation points. Given these points, we can propose an approximate solution, u(z); of the
form
u(z) = 0 + 1 z + ::::: + n z n (146)
to the ODE-BVP as an interpolation polynomial that passes exactly through fui : i =
1; :::; n + 1g: This requires that the following set of equations hold

n
u(zi ) = 0 + 1 zi + ::::: + n zi = ui (147)
i = 1; 2; :::; n + 1

37
at the collocation points. The unknown polynomial coe¢ cients f i : i = 0; 1; :::ng can be
expressed in terms of the unknowns fui : i = 1; :::; n + 1g as follows

= A 1u (148)

where matrix A is de…ned in equation (134). To approximate the OBE-BVP in (0; 1), we
force the residuals at the collocation points to zero using the approximate solution u(z), i.e.

d2 u(zi ) du(zi )
Ri = ; ; u(zi ); zi = 0 (149)
dz 2 dz

for i = 2; 3; :::n: Thus, we need to compute the …rst and second derivatives of the approximate
solution ue(z) at the collocation points. The …rst derivative at the ith collocation point can
be computed as follows

de
u(zi )
= 0 0 + 1 + 2 2 zi + ::::::: + n n zin 1
(150)
dz h i
n 1
= 0 1 2zi ::: nzi (151)
h i
= 0 1 2zi ::: nzin 1 A 1 u (152)

De…ning vector h i
(i) T 1
s = 0 1 2zi ::: nzin 1 A
we have
de
u(zi ) T
= s(i) u
dz
Similarly, the second derivative can be expressed in terms of the vector u as follows:

d2 u
e(zi )
= 0 0 + 0 1 + 2 2 + ::::::: + n(n 1) n zin 2
(153)
dz 2 h i
n 2
= 0 0 2 ::: n(n 1)zi (154)
h i
= 0 0 2 ::: n(n 1)zin 2 A 1 u (155)

De…ning vector h i
(i) T 1
t = 0 0 2 ::: n(n 1)zin 2 A
we have
d2 u
e(zi ) T
2
= t(i) u
dz
Substituting for the …rst and the second derivatives of ue(zi ) in equations (149), we have
h T T
i
t(i) u; s(i) u; ui ; zi = 0 (156)

38
 Table 1: Roots of Shifted Legendre Polynomials
Order (m) Roots
1 0.5
2 0.21132, 0.78868
3 0.1127, 0.5, 0.8873
4 0.9305,0.6703, 0.3297, 0.0695
5 0.9543, 0.7662, 0.5034, 0.2286, 0.0475
6 0.9698, 0.8221, 0.6262, 0.3792, 0.1681, 0.0346
7 0.9740, 0.8667, 0.7151, 0.4853, 0.3076, 0.1246, 0.0267

for i = 2; 3:::; n: At the boundary points, we have two additional constraints

de
u(0) h T
i
f1 ; u1 ; 0 = f1 s(1) u; u1 ; 0 = 0
dz

de
u(1) h T
i
f2 ; un+1 ; 1 = f2 s(n+1) u; un+1 ; 1 = 0 (157)
dz
Thus, we have (n + 1) algebraic equations to be solved simultaneously in (n + 1) unknowns,
i.e. fui : i = 1; :::; n + 1g.
It may be noted that the collocation points need not be chosen equispaced. It has
been shown that, if these collocation points are chosen at the roots of nth order orthogonal
polynomial, then the error ju (z) u(z)j is evenly distributed in the entire domain of z [3].
For example, one possibility is to choose the orthogonal collocation points at the roots of the
shifted Legendre polynomials (see Table 1). In fact, the name orthogonal collocation can be
attributed to the choice the collocation points at the roots of the orthogonal polynomials.
Discretization using orthogonal collocation technique requires computation of vectors
(i) (i)
{(s ; t ) : i = 1; 2; :::n + 1g; which can be accomplished by solving the following matrix
equations. Let us de…ne matrices S and T such that these vectors form rows of these matrices,
i.e. 2 3 2 3
T T
s(1) t(1)
6 (2) T 7 6 (2) T 7
6 s 7 6 t 7
S=6
6 ::::
7 ; T=6
7 6 ::::
7
7 (158)
4 5 4 5
T T
s(n+1) t(n+1)

39
In addition, let us de…ne matrices C and D as follows
2 3
0 1 :::: (n)(z1 )n 1
6 0 1 :::: (n)(z2 )n 1 7
6 7
C = 6 7
4 :::: :::: :::: :::: 5
0 1 :::: (n)(zn+1 )n 1

2 3
0 0 2 6z1 :: n(n 1) (z1 )n 2

6 0 0 2 6z2 :: n(n 1)(z2 )n 2 7

6 7
D = 6 7
4 :::: :::: :::: :::: :: :::: 5
0 0 2 6zn+1 :: n(n 1) (zn+1 )n 2

It is easy to see that

1 1
S = CA ; T = DA (159)
where matrix A is de…ned by equation (134).

Example 18 [3] Consider the ODE-BVP describing steady state conditions in a tubular
reactor with axial mixing (Isothermal TRAM) in which an irreversible 2nd order reaction is
carried out. Using the method of orthogonal collocation with n = 4 and de…ning vector
h iT
C= C1 C2 ::: C5

at
z1 = 0; z2 = 0:1127; z3 = 0:5; z4 = 0:8873 and z5 = 1
the matrices A; S and T for the selected set of collocation points are as follows
2 3
1 0 0 0 0
6 7
6 1 0:1127 (0:1127)2 (0:1127)3 (0:1127)4 7
6 7
A=6 6 1 0:5 (0:5)2 (0:5)3 (0:5)4 7
7 (160)
6 7
4 1 0:8873 (0:8873)2 (0:8873)3 (0:8873)4 5
1 1 1 1 1
2 T 3 2 3
s(1) 13 14:79 2:67 1:88 1
6 T 7 6 7
6 s(2) 7 6 5:32 3:87 2:07 1:29 0:68 7
6 7 6 7
S=6 7=6 1:5 7
T
6 s(3) 7 6 1:5 3:23 0 3:23 7 (161)
6 T 7 6 7
4 s(4) 5 4 0:68 1:29 2:07 3:87 5:32 5
T
s(5) 1 1:88 2:67 14:79 13

40
 2 T 3 2 3
t(1) 84 122:06 58:67 44:60 24
6 T 7 6 7
6 t(2) 7 6 53:24 73:33 26:67 13:33 6:76 7
6 7 6 7
T=6 7=6 7
T
6 t(3) 7 6 6 16:67 21:33 16:67 6 7 (162)
6 T 7 6 7
4 t(4) 5 4 6:76 13:33 26:67 73:33 53:24 5
T
t(5) 24 44:60 58:67 122:06 84
Forcing the residual to zero at the internal grid points and using the two boundary conditions
we get following set of …ve simultaneous nonlinear algebraic equations:
1 h (i) T i h (i) T i
t C s C DaCi2 = 0
Pe

i = 2; 3; 4
These equations can be expanded as follows
2 3
C1
6 7 2 3 2 3
6 C2 7 C22 0
6 7 6 2 7 6 7
6 7
M 6 C3 7 Da 4 C3 5 = 4 0 5
6 7
4 C4 5 C42 0
C5
2 3
53:24 73:33 26:67 13:33 6:76
Pe
+ 5:32 Pe
3:87 Pe
2:07 Pe
+ 1:29 Pe
0:68
6 6 16:67 21:33 16:67 6 7
M =4 Pe
1:5 Pe
+ 3:23 Pe Pe
3:23 Pe
+ 1:5 5
6:76 13:33 26:67 73:33 53:24
Pe
+ 0:68 Pe
1:29 Pe
+ 2:07 Pe
+ 3:87 Pe
5:32
The remaining two equations are obtained by discretization of the boundary conditions.
h i
(1) T
B:C:1 : s C P e(C1 1) = 0
h T
i
B:C:2 : s(5) C = 0

or in the expanded form, we have

( 13 P e) C1 +14:79C2 2:67C3 +1:88C4 C5 + Pe = 0

C1 1:88C2 +2:67C3 14:79C4 +13C5 = 0

Thus, the discretization yields a set of …ve nonlinear algebraic equations in …ve unknowns,
which have to be solved simultaneously.
To provide some insights into how the approximate solutions change as a function of the
choice number of collocation points, we have carried out studies on the TRAM problem (with

41
Pe = 6 and Da = 2). The resulting set of coupled nonlinear algebraic equations were solved
simultaneously using Newton’s method. The initial guess solution is chosen as
zi 0 0:1127 0:5 0:8873 1
Ci 0:9 0:75 0:6 0:45 0:3
and the concentration pro…le values obtained after solving the coupled nonlinear equations
are as follows
zi 0 0:1127 0:5 0:8873 1
Ci 0:8316 0:7303 0:5080 0:3956 0:3873
Recollect that vector C is only a proxy for the polynomial coe¢ cient vector , which can be
recovered as = A 1 C. Thus, the approximate solution obtained using orthogonal collocation
method is
C(z) = 0:8316 1:0102z + 1:0841z 2 0:9148z 3 + 0:3966z 4 (163)
which is a function de…ned over z 2 [0; 1]. Thus, we can compute C(z) at any value of z.
On the other hand, the …nite di¤erence method generates values of the concentration pro…le
only at the grid point. Also, when we examine the coe¢ cients of the polynomial solution
approximation, the advantages of transforming the unknowns from to C are amply clear.
It would have been very di¢ cult to generate a good initial guess for .
Figure 7 compares the approximate solutions obtained using orthogonal collocations and
…nite di¤erence methods. It may be noted that orthogonal collocation solution only with 3
internal collocation points is able to match accuracy of …nite di¤erence solution with 100 grid
points. This amounts to a huge reduction in the computational e¤orts.

Remark 19 Are the two methods presented above, i.e., …nite di¤erence and collocation
methods, doing something fundamentally di¤erent? Let us compare the following two cases
(a) …nite di¤erence method with 3 internal grid points (b) collocation with 3 internal grid
points on the basis of expressions used for approximating the …rst and second order derivatives
computed at one of the grid points. For the sake of comparison, we have taken equi-spaced
grid points for the collocation method instead of taking them at the roots of the 3rd or-
der orthogonal polynomial. Thus, for both collocation and …nite di¤erence method, the grid
(or collocation) points are at fz1 = 0; z2 = 1=4; z3 = 1=2; z4 = 3=4; z5 = 1g. Let us compare
expressions for approximate derivatives at z = z3 used in both the approaches.
Finite Di¤ erence
du(z3 ) (u4 u2 )
= = 2u4 2u2 ; z = 1=4
dz 2( z)
d2 u(z3 ) (u4 2u3 + u2 )
2
= = 16u4 32u3 + 16u2
dz ( z)2

42
 0.85

0.8
Finite Difference (n=10)
0.75 Finite Difference (n=100)
Orthogonal Collocation (n=4), C values
i
0.7
Orthogonal Collocation (polynomial soln)
Concentration

0.65

0.6

0.55

0.5

0.45

0.4

0.35
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
z -->

Figure 7: TRAM Problem: Comparison of approximate solutions constructed using Finite

Di¤erence and Orthogonal Collocation Method

Collocation
du(z3 )
= 0:33u1 2:67u2 + 2:67u4 0:33u5
dz
d2 u(z3 )
= 1:33u1 + 21:33u2 40u3 + 21:33u4 1:33u5
dz 2

It is clear from the above expressions that the essential di¤erence between the two ap-
proaches is the way the derivatives at any grid (or collocation) point are approximated. The
…nite di¤erence method takes only immediate neighboring points for approximating the deriv-
atives while the collocation method …nds derivatives as a weighted sum of all the collocation
(grid) points. As a consequence, the approximate solutions generated by these approaches
will be di¤erent.

Example 20 Consider the Non-Isothermal TRAM operating under steady state conditions.
Using the method of orthogonal collocation with n = 4 and de…ning vector
h iT h iT
u= u1 u2 ::: u5 and v= v1 v2 ::: v5

at
z1 = 0; z2 = 0:1127; z3 = 0:5; z4 = 0:8873 and z5 = 1

43
we force the residuals at (n-1) internal collocation points to zero, i.e.
1 h (i) T i h (i) T i
t u s u + (ui ; vi ) = 0
P eM
1 h (i) T i h (i) T i
t v s u + (ui ; vi ) + (vH vi ) = 0
P eT

i = 2; 3; 4
This yields 2 (n 1) equations in 2 (n + 1) unknowns. Using the boundary conditions,
we get additional 4 equations
1 h (1) T i 1 h (1) T i
B:C: at z = 0 : s u = u1 and s v + vn+1 = v1 (164)
PeM PeT
h T
i h T
i
B:C: at z = 1 : s(5) u = 0 and s(5) v = 0 (165)

Thus, we have 2(n + 1) simultaneous nonlinear equations of the form, F(x) = 02 (n+1) ; in
2(n + 1) unknowns where
h iT
x = u1 ::: un+1 v1 ::: vn+1 2 R2(n+1)

The steady state conversion and dimensionless temperature pro…les obtained for the following
choice of parameters

PeM = 100; PeT = 100; vH = 0; Da = 0:15

= 1:4; = 10; = 2; = 0:3;

are shown in Figures (3) and (4), respectively. As can be seen from these …gure, the solution
obtained using only 3 internal collocation points is close to the solution generated using …nite
di¤erence method with 50 grid points. This indicates that OC method is capable for generating
a good solution with signi…cant reduction in the computations.

3.2.2 Discretization of PDE’s [3]

Example 21 Consider the PDE describing unsteady state conditions in a tubular reactor
with axial mixing (TRAM) given earlier. Using the method of orthogonal collocation with
n 1 internal collocation points, we get
dCi (t) 1 h (i) T i h
(i) T
i
= t C(t) s C(t) DaCi (t)2
dt Pe

i = 2; 3; :::n

44
where h i
C(t) = C1 (t) C2 (t) ::: Cn+1 (t)

Ci (t) represents the time varying concentration at the ith collocation point, C(zi ; t); and the
T T
vectors t(i) and s(i) represent row vectors of matrices T and S de…ned by equation
(158). The two boundary conditions yield the following algebraic constraints
h i
(1) T
s C(t) = P e(C1 (t) 1)
h T
i
s(n+1) C(t) = 0

Thus, the process of discretization in this case yields a set of di¤ erential algebraic equa-
tions (DAEs) of the form
dxd
= F (xd ; xa )
dt
0 = Md xd +Ma xa b

which has to be solved simultaneously subject to the speci…ed initial conditions on (xd ; xa ):
In the present case, since the algebraic constraints are linear they can be used to eliminate
algebraic variables xa (i.e. C1 (t) and Cn+1 (t)) from the set of ODEs and the set of DAEs
can be expressed as a set of ODEs

xa = (Ma ) 1 (b Md xd )
dxd
= F [xd ; xa ]
dt
For example, when we select 3 internal grid points as discussed in Example 15, the boundary
constraints can be stated as follows

(13 + P e)C1 (t) +14:79C2 (t) 2:67C3 (t) +1:88C4 (t) C5 (t) = P e (166)
C1 (t) 1:88C2 (t) +2:67C3 (t) 14:79C4 (t) +13C5 (t) = 0 (167)
h iT
These equations can be used to eliminate algebraic variables xa (t) = C1 (t) C5 (t) from
the three ODEs and express derivatives in terms of the di¤erential states
h iT
xd (t) = C2 (t) C3 (t) C4 (t)

Rearranging equations (166) - (167), we have

2 3
" #" # " # C2 (t) " #
(13 + P e) 1 C1 (t) 14:79 2:67 1:88 6 7 Pe
= 4 C3 (t) 5
1 13 C5 (t) 1:88 2:67 14:79 0
C4 (t)

45
or
8 2 3 9
" # " # 1 >" # C2 (t) " #>
C1 (t) (13 + P e) 1 < 14:79 +2:67 1:88 Pe =
6 7
= 4 C3 (t) 5
C5 (t) 1 13 >
: 1:88 2:67 +14:79 0 ;>
C4 (t)
(168)
Thus, the DAE reduced to following 3 ODEs
dC2 (t) 1 h (2) T i h T
i
= t C(t) s(2) C(t) Da [C2 (t)]2 (169a)
dt Pe
dC3 (t) 1 h (3) T i h T
i
= t C(t) s(3) C(t) Da [C3 (t)]2 (169b)
dt Pe
dC4 (t) 1 h (4) T i h T
i
= t C(t) s(4) C(t) Da [C4 (t)]2 (169c)
dt Pe
where h iT
C(t) = C1 (t) C2 (t) C3 (t) C4 (t) C5 (t) (170)
Thus, the resulting set of 3 ODEs have to be solved simultaneously together with initial
conditions
C2 (0) = f (z2 ) ; :C3 (0) = f (z3 ) ; C4 (0) = f (z4 ) (171)
The sequence of calculations to evaluate the time derivatives, dCd =dt; at time instant say
tn = nh; is as follows:
Step 1: Given initial state, Cd (tn ) Cd (n); compute Ca (n) using equation (168).
Step 2: Using Cd (n) and Ca (n) construct vector C(n) as in equation (170)
Step 3: Using vector C(n) and set of equations (169) …nd vector dCd =dt at t = nh
The solution to the DAE at time t > 0 is
h iT
C(t) = C1 (t) C2 (t) C3 (t) C4 (t) C5 (t)

which can be used to compute

h iT
(t) = 0 (t) 1 (t) 2 (t) 3 (t) 4 (t) = A 1 C(t)

Thus, approximate spatial concentration pro…le can be constructed as

2 3 4
C(z; t) = 0 (t) + 1 (t)z + 2 (t)z + 3 (t)z + 4 (t)z = Z T A 1 C(t) (172)
h iT
Z = 1 z z2 z3 z 4 (173)

where z 2 [0; 1]. Figures (8) and (9) show dynamic behavior of the solution for the following
scenario:
I:C: : Ci 1 (0) = exp( 0:1zi ) for i = 2; :::n where n = 40 (174)

46
Figure 8: Isothermal TRAM : Concentration dynamics for step change in inlet concentration
(Orthogonal collocation with n=6, polynomial solution)

( )
1 for t 2 [0; 2)
C(0; t) = (175)
2 for t 2 [2; 4]
The ODE-IVP has been solved using Explicit Euler method with integration interval, h =
0:001. If we compare Figure (8) with Figure (5), i.e. the solution obtained using …nite
di¤erence method, then we notice some di¤erences at the edges of the three dimensional
surface.

Example 22 [3] Consider the 2-dimensional Laplace equation given in Example 12. We
consider a scenario where the thermal di¤usivity, ; is a function of temperature. To begin
with, we choose (nx 1) internal collocation points along the x-axis and (ny 1) internal
collocation points along the y-axis. Using nx 1 internal grid lines parallel to the y axis
and ny 1 grid lines parallel to the x-axis, we get (nx 1) (ny 1) internal collocation
points. Corresponding to the chosen collocation points, we can compute matrices (Sx ; Tx )
and (Sy ; Ty ) using equations (159). Using these matrices, the PDE can be transformed to a
set of coupled algebraic equations as follows
h i
(i) T (j) (j) T (i)
(Ti;j ) tx Tx + ty Ty = f (xi ; yj )

i = 2; :::; nx ; j = 2; :::; ny

47
 T RAM : Expli cit Euler Results - Red: z = 0, Blue: z = 1
1.6

1.4

1.2
Concentration

0.8

0.6

0.4

0.2
0 0.5 1 1.5 2 2.5 3 3.5 4
Time

Figure 9: Isothermal TRAM: Concentration pro…les at individual grid points for step change
in inlet concentration. (Finite Di¤erence method with n = 40)

(j) (i)
where vectors Tx and Ty are de…ned as
h i
T(j)
x = T1;j T2;j ::: T nx +1;j
h i
T(i)
y = Ti;1 Ti;2 ::: Ti;ny +1
At the boundaries, we have
T0;j = T ; (j = 1; :::ny + 1)
T1;j = T ; (j = 1; :::ny + 1)
Ti;0 = T ; (i = 1; :::nx + 1)
T
k s(n
y
y +1)
T(i)
x = h(T1 Ti;ny +1 ) for (i = 2; :::nx )
The above discretization procedure yields a set of (nx + 1) (ny + 1) nonlinear algebraic
equations in (nx + 1) (ny + 1) unknowns, which have to be solved simultaneously.To get a
better insight into discretization, let us consider the scenario where we choose two internal
collocation points each along the x and y directions (ref. Figure 10).. This implies that
(Sx = Sy = S) and (Ty = Tx = T) where S and T matrices are given in Example 23. At
internal collocation point (2,2):
h T
i
(2) T
(T2;2 ) t(2) T(2) x + t T (2)
y = f (x2 ; y2 )

48
 Figure 10: Steady state Laplace equation: Example of grid for application of OC method

h iT
T(2)
x = T1;2 T2;2 T3;2 T4;2 (176)
h iT
T(2)
y = T2;1 T2;2 T2;3 T2;4 (177)

n o
(T2;2 )
16:4T1;2 24T2;2 +12T3;2 4:4T4;2
n o
+ (T2;2 ) 16:4T2;1 24T2;2 12T2;3 4:4T2;4
= f (x2 ; y2 ) (178)

At internal collocation point (2,3):

h T T
i
(T2;3 ) t(2) T(3)
x + t
(3)
T(2)
y = f (x2 ; y3 )
h iT
T(3)
x = T1;3 T2;3 T3;3 T4;3 (179)

n o
(T2;3 ) 16:4T1;3 24T2;3 +12T3;3 4:4T4;3
n o
+ (T2;3 ) 4:4T2;1 12T2;2 24T2;3 16:4T2;4 = f (x2 ; y2 ) (180)

49
At internal collocation point (3,2):
h T T
i
(T3;2 ) t(3) T(2)
x + t
(2)
T(3)
y = f (x2 ; y3 ) (181)
h iT
T(3)
y = T3;1 T3;2 T3;3 T3;4 (182)

n o
(T3;2 ) 4:4T1;2 12T2;2 24T3;2 16:4T4;2
n o
+ (T3;2 ) 16:4T3;1 24T3;2 +12T3;3 4:4T3;4 = f (x3 ; y2 ) (183)

At internal collocation point (3,3):

h T T
i
(T3;3 ) t(3) T(3)
x + t
(3)
T(3)
y = f (x3 ; y3 ) (184)

n o
(T3;3 )
16:4T1;3 24T2;3 +12T3;3 4:4T4;3
n o
+ (T3;3 ) 16:4T3;1 24T3;2 12T3;3 4:4T3;4 = f (x3 ; y3 ) (185)

Boundary conditions:

B:C: at y = 0: T1;1 = T2;1 = T3;1 = T4;1 = T (186)

B:C: at x = 0; : T1;2 = T1;3 = T1;4 = T (187)
B:C: at x = 1; : T4;2 = T4;3 = T4;4 = T (188)

B.C. at point(2,4)
T
k s(4)
y T(2)
y = h(T1 T2;4 )
T2;1 +2:2T2;2 8:2T2;3 +7T2;4 = h(T1 T2;4 ) (189)
B.C. at point(3,4)
T
k s(4) T(3)
y = h(T1 T3;4 )
T3;1 +2:2T3;2 8:2T3;3 +7T3;4 = h(T1 T3;4 ) (190)
Thus, we have 16 equations in 16 unknowns, which have to be solved simultaneously. To see
how his can be achieved, consider variables arranged in a matrix form
2 T 3
(1)
2 3 Tx
T1;1 T2;1 T3;1 T4;1 6 7
6 T 7
6 T 7 h i 6 Tx (2) 7
6 1;2 T2;2 T3;2 T4;1 7 (1) (2) (3) (4) 6 7
MT = 6 7 = Ty Ty Ty Ty =6 7
4 T1;3 T2;3 T3;3 T4;3 5 6 T(3) T 7
6 x 7
T1;4 T2;4 T3;4 T4;4 4 T 5
(4)
Tx

50
Now, using boundary conditions (186) to (188), we have
2 3
T T T T
6 T T2;2 T3;2 T 7
6 7
MT = 6 7
4 T T2;3 T3;3 T 5
T T2;4 T3;4 T

Thus, we de…ne a vector Z of unknowns as follows

h iT
Z= T2;2 T3;2 T2;3 T3;3 T2;4 T3;4

and arrange the set of algebraic equations arising from the boundary conditions and from
setting residual to zero at the internal grid points as F(Z) = 0, i.e.
2 h i 3
T T
(2)
(T2;2 ) t(2) Tx + t(2) Ty
(2)
f (x2 ; y2 ) 2 3
6 h i 7 0
6 (3) T (2) (2) T (3) 7 6 7
6 (T 3;2 ) t T x + t T y f (x ;
2 3y ) 7 6 0 7
6 h i 7 6 7
6 (2) T (3) (3) T (2) 7 6 0 7
6 (T2;3 ) t Tx + t Ty f (x3 ; y2 ) 7 6 7
F(Z) = 6 h i 7=6 7
6 T (3) T (3) 7 6 0 7
6 (T3;3 ) t(3) Tx + t(3) Ty f (x3 ; y3 ) 7 6 7
6 7 4 0 5
6 T +2:2T 8:2T +7T h(T T ) 7
4 2;1 2;2 2;3 2;4 1 2;4 5
0
T3;1 +2:2T3;2 8:2T3;3 +7T3;4 h(T1 T3;4 )

The resulting set of equations can be solved using Newton-Raphson method. While carrying
out iterations, for each guess Z(k) generated by Newton-Raphson, it is useful to construct
matrix MT solver as follows
2 3
T T T T
6 T
(k)
Z1
(k)
Z2 T 7
6 7
MT = 6 (k) (k) 7
4 T Z3 Z4 T 5
(k) (k)
T Z5 Z6 T

and then use rows and columns of this matrix to compute F(Z(k) ).
After computing fTi;j : i; j = 1; 2; 3; 4g ; we can use these values construct approximate
polynomial solutions as follows: Consider vector
h iT
T(j)
x = T1;j T2;j T3;j T4;j

which can be used to compute

h iT
(j)
x = j;0 j;1 j;2 j;3 = A 1 T(j)
x

51
and
2 3 (j)
Tj (x) = j;0 + j;1 x + j;2 x + j;3 x = XT x = X T A 1 T(j)
x (191)
h iT
X = 1 x x2 x 3 (192)

for j = 1; 2; 3; 4. Similarly, consider vector

h iT
T(i)
y = T i;1 Ti;2 Ti;3 Ti;4

which can be used to compute

h iT
(i)
y = i;0 i;1 i;2 i;3 = A 1 T(i)
y

and
2 3
Ti (y) = i;0 + i;1 y + i;2 y + i;3 y = Y T A 1 Ty(i) (193)
h iT
Y = 1 y y2 y3 (194)

for i = 1; 2; 3; 4.

3.3 Piecewise Polynomial Interpolation

Matrix A appearing in equation (134) is known as the Vandermond matrix. Larger di-
mensional Vandermond matrices tend to become numerically ill-conditioned (Refer to the
Module on Solving Linear Algebraic Equations). Also, if the number of data points is large,
…tting a high order polynomial can result in a polynomial which exhibits unexpected os-
cillatory behavior. In order to avoid such oscillations and the di¢ culties arising from ill
conditioning of the Vandermond matrix, the data is divided into sub-intervals and a lower
order polynomial approximation is developed on each sub-interval. Let [a,b] be a …nite
interval. We introduce a partition of the interval by placing points

a Z1 < Z2 < Z3 :::: < Zn+1 b

where Zi are called nodes. A function is said to be a piecewise polynomial of degree k on

this partition if in each subinterval Zi z Zi+1 we develop a k th degree polynomial. For
example, a piecewise polynomial of degree one consists of straight line segments. Such an
approximation is only continuous at the nodes but will have discontinuous derivatives. In
some applications it is important to have a smooth approximation with continuous derivatives
at the nodes. Here we consider two cases:

Piecewise Polynomials Interpolation with First Order Continuous Derivatives

Cubic splines with the …rst two continuous derivatives at the nodes

52
3.3.1 Piecewise Polynomials Interpolation with First Order Continuous Deriv-
atives

Consider a data set consisting of 7 data points z1 = a; z2 ; ::::; z7 = b and let fu1 ; u2; :::; u7 g
represent the values of function u(z) at fz1 ; z2 ; ::::; z7 g in the domain [a; b]. If we do not
consider piecewise approximation, then we have to …t a 6’th order Legrange interpolation
polynomial. Instead, let us construct three segments by selecting Z1 = z1 ; Z2 = z3 ; Z3 = z5
and Z4 = z7 , i.e.

Z1 z Z2 fz1 ; z2 ; z3 g
Z2 z Z3 fz3 ; z4 ; z5 g
Z3 z Z4 fz5 ; z6 ; z7 g

and …t a third order polynomial on each segment. Thus, it is desired to …nd three interpo-
lation polynomials of the form

p1 (z) = 0;1 + 1;1 (z z1 ) + 2;1 (z z1 )2 + 3;1 (z z1 )3 (195)

(z1 z z3 )
p2 (z) = 0;1 + 1;2 (z z3 ) + 2;2 (z z3 )2 + 3;2 (z z3 )3 (196)
(z3 z z5 )
p3 (z) = 0;1 + 1;3 (z z5 ) + 2;3 (z z5 )2 + 3;2 (z z3 )3 (197)
(z5 z z7 )

The choice of structures these polynomials implies that

p1 (z1 ) = 0;1 = u1
p2 (z3 ) = 0;2 = u3
p3 (z5 ) = 0;3 = u5

and as a consequences we can write

p1 (z) = u1 + 1;1 (z z1 ) + 2;1 (z z1 )2 + 3;1 (z z1 )3 (198)

(z1 z z3 )
p2 (z) = u3 + 1;2 (z z3 ) + 2;2 (z z3 )2 + 3;2 (z z3 )3 (199)
(z3 z z5 )
p3 (z) = u5 + 1;3 (z z5 ) + 2;3 (z z5 )2 + 3;2 (z z3 )3 (200)
(z5 z z7 )

53
Thus, there are total of 9 coe¢ cients to be determined and we need to generate 9 equations.
The …rst six of them come from the requirement that ith polynomial should pass through all
the points in the ith segment, i.e.

p1 (z2 ) = u1 + 1;1 (z2 z1 ) + 2;1 (z2 z1 )2 + 3;1 (z2 z1 )3 = u2 (201)

p1 (z3 ) = u1 + 1;1 (z3 z1 ) + 2;1 (z3 z1 )2 + 3;1 (z3 z1 )3 = u3 (202)

p1 (z4 ) = u3 + 1;2 (z4 z3 ) + 2;2 (z4 z3 )2 + 3;2 (z5 z3 )3 = u4 (203)

p1 (z5 ) = u3 + 1;2 (z5 z3 ) + 2;2 (z5 z3 )2 + 3;2 (z5 z3 )3 = u5 (204)

p1 (z6 ) = u5 + 1;3 (z6 z5 ) + 2;3 (z6 z5 )2 + 3;3 (z6 z5 )3 = u6 (205)

p1 (z7 ) = u5 + 1;3 (z7 z5 ) + 2;3 (z7 z5 )2 + 3;3 (z7 z5 )3 = u7 (206)

There are two common boundary points between three segments and requirement to con-
struct smooth piecewise interpolation polynomial implies that

dp1 (z3 ) dp2 (z3 )

= (207)
dz dz
) 1;1 +2 2;1 (z3 z1 ) + 3 3;1 (z3 z1 )2 = 1;2 (208)
and
dp2 (z5 ) dp3 (z5 )
= (209)
dz dz
) 1;2 + 2 2;2 (z5 z3 ) + 3 3;2 (z5 z3 )2 = 1;3 (210)
We need one additional condition to make number of equations equal to the number of
unknowns. One possible approach to generate this additional equation is to impose a free
boundary condition at z = z7 , i.e.

d2 p3 (z7 )
=2 2;3 +6 3;3 (z7 z5 ) = 0 (211)
dz 2
or at z = z1 i.e. d2 p1 (z1 )=dz 2 = 0. Now, we have 9 equations in 9 unknowns. These equations
can be arranged in the standard form

Ax = b

54
where
2 3
z2 ( z2 )2 ( z2 )3 0 0 0 0 0 0
6 7
6
6 z3 ( z3 )2 ( z3 )3 0 0 0 0 0 0 7
7
6 0 0 0 z4 ( z4 )2 ( z4 )3 0 0 0 7
6 7
6 7
6 0 0 0 z5 ( z5 )2 ( z5 )3 0 0 0 7
6 7
A =6
6 0 0 0 0 0 0 z6 ( z6 ) ( z6 )3
2 7
7
6 7
6 0 0 0 0 0 0 z7 ( z7 )2 ( z7 )3 7
6 7
6 1 2 z3 3 ( z3 )2 1 0 0 0 0 0 7
6 7
6 0 0 0 1 2 z5 3 ( z5 )2 1 0 0 7
4 5
0 0 0 0 0 0 0 2 6 z7
(9 9)

z2 = (z2 z1 ); z3 = (z3 z1 ); z4 = (z4 z3 )

z5 = (z5 z3 ); z6 = (z6 z5 ); z7 = (z7 z5 )

h iT
x = 1;1 1;2 ::: 3;1
(9 1)
h iT
b = u2 u1 u 3 u1 ::: u7 u5 0 0 0
(9 1)

Let us consider a speci…c example where it is desired to develop piecewise interpolation

polynomial with for a function u(z) = sin(5z) using vales of function at the following 7
points
i! 1 2 3 4 5 6 7
zi 0 1/6 2/6 3/6 4/6 5/6 1
ui 0 0.740 0.995 0.599 -0.191 -0.855 -0.959
using three segments. Three interpolation polynomials obtained for this data are as follows

p1 (z) = 3:739(z z1 ) + 10:679(z z1 )2 38:817(z z1 )3 (212)

(z1 z z3 )
p2 (z) = 0:995 2:08(z z3 ) + 0:814(z z3 )2 15:743(z z3 )3 (213)
(z3 z z5 )
p3 (z) = 0:191 6:785(z z5 ) + 20:161(z z5 )2 20:161(z z3 )3 (214)
(z5 z z7 )

Figure (11) compares the original function and its values at 7 points with the piecewise
interpolation polynomial. It is easy to see that transition from one polynomial to other is

55
 1
u(z)=sin(5z)
0.8

0.6

0.4
p (z)
u(z) and p (z)

2
0.2
i

u(z)=sin(5z)
0
p (z)
1
-0.2
p (z)
3
-0.4

-0.6

-0.8

-1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
z

Figure 11: Fitting of piecewise cubic polynomial with continuous …rst irder derivatives.

continuous and smooth. Moreover, the value of the piecewise interpolation polynomial are
exactly equal to the function values at the seven points in the interval. There is a mismatch
between the polynomial approximation and the function at rest of the points in the interval.
To generate a better approximation, we have to consider the function value at more number
of points. In general, if we have m segments, each containing p internal points, i.e. we have
total of m(p + 1) + 1 data points, then it can be shown that a polynomial of order p + 2 has
to be used for each segment together with one of the boundary conditions to make number
of equations equal to the number of unknown coe¢ cients.

3.3.2 Cubic Splines [3]

A piecewise k th degree polynomial, which has continuous derivatives up to order k 1 is

called a spline of degree k. In particular, the case k = 3; i.e. cubic spline, has been studied
extensively in the literature. In this section, we restrict our discussion to the development
of cubic splines. Thus, given a set of points z1 = a < z2 < z3 < :::: < zn+1 = b; the nodes
are chosen as
Zi = zi for i = 1; 2; :::; n + 1

56
and n cubic splines that …t (n + 1) data points can be expressed as

p1 (z) = 0;1 + 1;1 (z z1 ) + 2;1 (z z1 )2 + 3;1 (z z1 )3 (215)

(z1 z z2 ) (216)
p2 (z) = 0;2 + 1;2 (z z2 ) + 2;2 (z z2 )2 + 3;2 (z z2 )3 (217)
(z2 z z3 ) (218)
::::::: = :::::::::::::::::::::::
pn (z) = 0;n + 1;n (z zn ) + 2;n (z zn )2 + 3;n (z zn )3
(zn z zn+1 ) (219)

There are a total of 4n unknown coe¢ cients f 0;1 ; 1;1 ; :::::::; 3;n g to be determined. In
order to ensure continuity and smoothness of the approximation, the following conditions
are imposed

The initial point of each polynomial

pi (zi ) = ui for i = 1; 2; :::; n (220)

The terminal point of the last polynomial

pn (zn+1 ) = un+1 (221)

Conditions for ensuring continuity between two neighboring polynomials

pi (zi+1 ) = pi+1 (zi+1 ) ; i = 1; 2; ::::n 1 (222)

dpi (zi+1 ) dpi+1 (zi+1 )
= ; i = 1; 2; ::::n 1 (223)
dz dz
d2 pi (zi+1 ) d2 pi+1 (zi+1 )
= ; i = 1; 2; ::::n 1 (224)
dz 2 dz 2
which result in 4n 2 conditions including earlier conditions.

Conventionally, two additional conditions are imposed at the boundary points

d2 p1 (z1 ) d2 pn (zn+1 )
= =0 (225)
dz 2 dz 2
which are referred to as free boundary conditions. This is because two additional
equations are needed to make the number of equations equal to the number
of unknowns. If the …rst derivatives at the boundary points are known,
dp1 (z1 ) dpn (zn+1 )
= d1 ; = dn+1 (226)
dz dz
then we get the clamped boundary conditions.

57
 Using constraints (220-224) and de…ning zi = zi+1 zi , we get the following set of
coupled linear algebraic equations

0;i = ui ; ( i = 1; 2; :::; n) (227)

0;n + 1;n ( zn ) + 2;n ( zn )2 + 3;n ( zn )3 = un+1 (228)

0;i + 1;i ( zi ) + 2;i ( zi )2 + 3;i ( zi )3 = 0;i+1 (229)

1;i +2 2;i ( zi ) + 3 3;i ( zi )2 = 1;i+1 (230)
2;i +3 3;i ( zi ) = 2;i+1 (231)

f or i = 1; 2; :::; n 1
In addition, using the free boundary conditions, we have

2;1 = 0 (232)
2;n +3 3;n ( zn ) = 0 (233)

Eliminating 3;i using equation (231 and 233), we have

2;i+1 2;i
3;i = for i = 1; 2; :::; n 1 (234)
3 ( zi )
2;n
3;n = (235)
3 ( zn )
and eliminating 1;i using equations (228,229), we have
1 zi
1;i =
( 0;i+1 0;i ) (2 2;i + 2;i+1 ) (236)
zi 3
for i = 1; 2; :::; n 1
un+1 0;n
1;n = ( zn ) 2;n 3;n ( zn )2 (237)
zn
Thus, we get only f 2;i : i = 1; :::; ng as unknowns and the resulting set of linear equations
assume the form

2;1 = 0 (238)
( zi 1 ) 2;i 1 + 2( zi + zi 1 ) 2;i + ( zi ) 2;i+1 = bi (239)

for i = 2; :::; n 1
where
3( 0;i+1 0;i ) 3( 0;i 0;i 1 )
bi =
zi zi 1
3(ui+1 ui ) 3(ui ui 1 )
=
zi zi 1

58
for i = 2; :::; n 1:

1 2
( zn 1 ) 2;n 1 + ( zn 1 + zn ) 2;n = bn (240)
3 3
un+1 1 1 un 1
bn = + un +
zn zn zn 1 zn 1
De…ning vector 2 as
h iT
2 = 2;1 2;2 ::::::: 2;n

the above set of n equations can be rearranged as

A 2 =b (241)

where A is an (n n) matrix and b is an (n 1) vector. Elements of A and b can be

obtained from equations (238-240). Note that matrix A will be a near tridiagonal matrix,
i.e. a sparse matrix. Once all the 2;i are obtained, 1;i and 2;i can be easily obtained.

3.4 Orthogonal Collocations on Finite Elements (OCFE)

As indicated in the previous section, the main di¢ culty with the polynomial interpolation
is that the Vandermond matrix becomes ill conditioned when the order of the interpolation
polynomial is large. A remedy to this problem is to sub-divide the region into …nite
elements and assume a lower order polynomial spline solution. The collocation points are
then selected within each …nite element, where the residuals are forced to zero. The continuity
conditions (equal slopes) at the boundaries of the neighboring …nite elements gives rise to
additional constraints. We illustrate this method by taking two speci…c examples.

Example 23 The steady state behavior of an isothermal Langmuir-Hinshelwood reaction

(with one reactant in stochoimetric excess) occurring in a catalysy sphere can be modelled as
the following ODE-BVP
2
d2 y 2 dy y(E + y)
+ = 0 f or 0 < x < 1
dx2 x dx (1 + Ky)2

dy
B:C:1 : At x = 0 : =0
dx
B:C:2 : At x = 1 : y = 1

Consider solving the ODE-BVP using OCFE method with two …nite elements

Finite element 1 [0; 0:5] and Finite element 2 [0:5; 1]

59
and two internal collocation points at (0.21132, 0.78868) in each element. Matrices S and
T that correspond to chosen collocation points are as follows
2 T
3 2 3
s(1) 7 8:2 2:2 1
6 (2) T 7 6
6 s 7 6 2:7 1:7 1:7 0:7 7
7
S = 6 7
6 s(3) T 7 6= 7 ;
4 5 4 0:7 1:7 1:7 2:7 5
T
s(4) 1 2:2 8:2 7
2 T
3 2 3
t(1) 24 37:2 25:2 12
6 (2) T 7 6
6 t 7 6 16:4 24 12 4:4 7
7
T = 6 7 = 6
6 t(3) T 7 4 4:4 7
4 5 12 24 16:4 5
T
t(4) 12 25:2 37:2 24

Normalized independent variables de…ned on each element are

x 0 x 0:5
1 = ; h1 = 0:5 0 = 0:5 and 2 = ; h2 = 1 0:5 = 0:5 (242)
0:5 0 1 0:5
Let the approximate polynomial solutions on each segment be represented as follows

2 3
P1 ( 1 ) = 0 + 1 1 + 2 1 + 3 1
2 3
P2 ( 2 ) = 0 + 1 2 + 2 2 + 3 2

Boundary and Internal collocation points associated with the …rst …nite element are located
at
1;1 = 0; 1;2 = 0:2113; 1;3 = 0:78868; 1;4 = 1

x1 = 0:5 1;1 = 0; x2 = 0:5 1;2 = 0:105; x3 = 1;3 = 0:394; x4 = 0:5 1;4 = 0:5
Boundary and Internal collocation points associated with the second …nite element are located
at
2;1 = 0; 2;2 = 0:2113; 2;3 = 0:78868; 2;4 = 1

x4 = 0:5 + 0:5 2;1 = 0:5; x5 = 0:5 + 0:5 2;2 = 0:605;

x6 = 0:5 + 0:5 2;3 = 0:894; x7 = 0:5 + 0:5 2;4 =1

De…ning yi = y(zi ); the dependent variable vectors associated with the two elements are
h iT h iT
Y (1) = y( 1;1 ) y( 1;2 ) y( 1;3 ) y( 1;4 ) = y1 y2 y3 y4
h iT h iT
(2)
Y = y( 2;1 ) y( 2;2 ) y( 2;3 ) y( 2;4 ) = y4 y5 y6 y7

60
 dy
B. C. 1: At x = 0 : = 0; i.e.
dx
T
s(1) Y (1) = 7y1 + 8:2y2 2:2y3 + y4 = 0 (243)

Element 1 Internal Collocation Points: At x = x2 :

2
1 d2 y 2 dy y2 (E + y2 )
2 + =0
2
h1 d 1 0:5 1;2 h1 d 1 (1 + Ky2 )2
2
1 T 2 T y2 (E + y2 )
t(2) Y (1) + s(2) Y (1) =0
(0:5)2 0:105 0:5 (1 + Ky2 )2

1
[16:4y1 24y2 + 12y3 4:4y4 ] + (244)
(0:5)2
2
2 y2 (E + y2 )
[ 2:7y1 + 1:7y2 + 1:7y3 0:7y4 ] =0 (245)
0:105 0:5 (1 + Ky2 )2

At x = x3 :
2
1 d2 y 2 dy y3 (E + y3 )
2 + =0
2
h1 d 1 0:5 1;3 h1 d 1 (1 + Ky3 )2
2
1 T 2 T y3 (E + y3 )
t(3) Y (1)
+ s(3) Y (1)
=0
(0:5)2 (0:394 0:5) (1 + Ky3 )2

1
[ 4:4y1 + 12y2 24y3 + 16:4y4 ] + (246)
(0:5)2
2
2 y3 (E + y3 )
[0:7y1 1:7y2 1:7y3 + 2:7y4 ] =0 (247)
0:394 0:5 (1 + Ky3 )2

Continuity Condition x4 = 0:5; i.e. at 1;4 = 1 and 2;1 =0

P1 ( 1;4 ) P2 ( 2;1 ) T T
= ) s(4) Y (1) = s(1) Y (2)
d 1 d 2

1y1 + 2:2y2 8:2y3 + 7y4 = 7y4 + 8:2y5 2:2y6 + y7 (248)

Element 2 Internal Collocation Points: At x = x5
2
1 d2 y 2 dy y5 (E + y5 )
2 + =0
2
h2 d 2 0:5 + 0:5 2;2 h2 d 2 (1 + Ky5 )2
2
1 T 2 T y5 (E + y5 )
t(2) Y (2) + s(2) Y (2) =0
(0:5)2 0:605 0:5 (1 + Ky5 )2

61
 1
[16:4y4 24y5 + 12y6 4:4y7 ] (249)
(0:5)2
2
2 y5 (E + y5 )
+ [ 2:7y4 + 1:7y5 + 1:7y6 0:7y7 ] =0 (250)
0:605 0:5 (1 + Ky5 )2

At x = x6 :
2
1 d2 y 2 dy y6 (E + y6 )
2 + =0
2
h2 d 2 0:5 2;3 h2 d 2 (1 + Ky6 )2
2
T 2 T y6 (E + y6 )
t(3) Y (2) + s(3) Y (2) =0
0:894 0:5 (1 + Ky6 )2

1
[ 4:4y4 + 12y5 24y6 + 16:4y7 ]
(0:5)2
2
2 y6 (E + y6 )
+ [0:7y4 1:7y5 1:7y6 + 2:7y7 ] =0
0:894 0:5 (1 + Ky6 )2

B. C. 2: At x = x7 = 1
y7 1=0 (251)
Thus, we have 7 coupled nonlinear equations in 7 unknowns that need to be solved simulta-
neously. The individual polynomial solutions are

2 3
P1 ( 1 ) = 0:5 + 0:1488 1 0:0108 1
2 3
P2 ( 2 ) = 0:6380 + 0:2652 2 + 0:1098 2 + 0:0129 2

.and the resulting concentration pro…le is shown in Figure (12).

Example 24 [3] Consider the ODE-BVP describing steady state conditions in a tubular
reactor with axial mixing (TRAM) in which an irreversible 2nd order reaction is carried out.
It is desired to solve this problem by the OCFE approach.
Step 1: The …rst step is to create …nite elements in the domain. Let us assume that we
create 3 sub-domains. Finite Element 1: 0 z 0:3; Finite Element 2: 0:3 z 0:7;
Finite Element 3: 0:7 z 1: It may be noted that these sub-domains need not be equi-sized.
Step 2: On each …nite element, we de…ne a scaled spacial variable as follows
z Z1 z Z2 z Z3
1 = ; 2 = and 3 =
Z2 Z1 Z3 Z2 Z4 Z3
where Z1 = 0; Z2 = 0:3, Z3 = 0:7 and Z4 = 1 represent the boundary points of the …nite
elements: It is desired to develop a polynomial spline solution such that the polynomial on

62
 1

0.9
Concentration
0.8

0.7

0.6

0.5
0 0.2 0.4 0.6 0.8 1
z -->

Figure 12: Steady state concentration pro…le inside sperical catalyst generated using OCFE
with 2 elements and 2 collocation points inside each element.

each …nite element is of 4th order. Thus, within each element, we select 3 collocation points
at the root of the 3rd order shifted Legandre polynomial, i.e.,

i;1 = 0; i;2 = 0:1127; i;3 = 0:5, i;4 = 0:8873, and i;5 = 1 for i = 1; 2; 3

In other words, the internal collocation points are placed at

Zi + 0:1127(Zi+1 Zi ); Zi + 0:5(Zi+1 Zi ); and Zi + 0:8873(Zi+1 Zi ) for i = 1; 2; 3

in the ith element Zi z Zi+1 . Thus, in the present case, we have total of 9 internal
collocation points. In addition, we have two points where the neighboring polynomials meet,
i.e. at Z1 = 0:3 and Z2 = 0:7: Thus, there are a total of 11 internal points and two
boundary points, i.e. Z1 = 0 and Z4 = 1: In terms of independent variable, z, these points
are as follows

z1 = 0; z2 = 0:03381; z3 = 0:15; z4 = 0:26619; z5 = 0:3

z6 = 0:34508; z7 = 0:5; z8 = 0:65492; z9 = 0:7
z10 = 0:73381; z11 = 0:85; z12 = 0:96619; z13 = 1

63
 Step 3: Let the total set of points created in the previous step be denoted as fz1 ; z1; :::z13 g
and let the corresponding values of the independent variables be denoted as fC1 ; C2 ; :::; C13 g :
Note that variables associate with each of the …nite elements are as follows
h iT
Finite Element 1 C(1) = C1 C2 C3 C4 C5
h iT
Finite Element 2 C(2) = C5 C6 C7 C8 C9
h iT
Finite Element 3 C(3) = C9 C10 C11 C12 C13

Now, we force the residuals to zero at all the internal collocation points within a …nite
element. Let h1 ; h2 and h3 denote the length of the individual …nite elements, i.e.

h1 = Z2 Z1 = 0:3; h2 = Z3 Z2 = 0:4 and h3 = Z4 Z3 = 0:3 (252)

De…ning scaled spatial variables

z Zi z Zi
i = =
Zi+1 Zi hi
for i = 1, 2, 3, the ODE in each …nite element is modi…ed as follows

1 1 d2 C 1 dC
DaC 2 = 0 for Zi z Zi+1 and i = 1; 2; 3 (253)
Pe h2i d 2i hi d i

The main di¤erence here is that only the variables associated within an element are used
while discretizing the derivatives. Thus, at the collocation point i;2 in …nite element 1, the
residual is computed as follows

1 1 T 1 T
R( 1;2 ) = t(2) C(2) s(2) C(1) Da (C2 )2 = 0 (254)
Pe h21 h1
T
t(2) C(1) = (53:24C1 73:33C2 + 26:27C3 13:33C4 + 6:67C5 )
T
s(2) C(1) = ( 5:32C1 + 3:87C2 + 2:07C3 1:29C4 + 0:68C5 )
T T
where vectors s(2) and t(2) are the 2nd rows of matrices (161) and (162), respectively.
Similarly, at the collocation point z = z8 = 0:65492, which corresponds to 2;4 = 0:8873 in
…nite element 2, the residual is computed as follows

1 1 T 1 T
R( 2;4 ) = t(4) C(2) s(4) C(2) Da (C8 )2 = 0 (255)
Pe h22 h2
T
t(4) C(2) = 6:76C5 13:33C6 + 26:67C7 73:33C8 + 53:24C9
(4) T
s C(2) = 0:68C5 + 1:29C6 2:07C7 3:87C8 + 5:32C9

64
 Other equations arising from forcing the residuals to zero are

Finite Element 1: R( 1;3 ) = R( 1;4 ) =0

Finite Element 2: R( 2;2 ) = R( 2;3 ) =0
Finite Element 3 : R( 3;2 ) = R( 3;3 ) = R( 3;4 ) =0

In addition to these 9 equations arising from the residuals at the collocation points, there are
two constraints at the collocation points z4 and z8 ; which ensure smoothness between the the
two neighboring polynomials, i.e.
1 T 1 T
s(5) C(1) = s(1) C(2)
h1 h2

1 T 1 T
s(5) C(2) = s(1) C(3)
h2 h3
The remaining two equations come from discretization of the boundary conditions.
1 h T
i
s(1) C(1) = P e(C1 1)
h1
1 h T
i
s(5) C(3) = 0
h3
Thus, we have 13 equations in 13 unknowns. It may be noted that, when we collect all the
equations together, we get the following form of equation

AC = F (C)
2 3
A1 [0] [0]
6 7
A = 4 [0] A2 [0] 5
[0] [0] A3 13 13
h iT
C= C1 C1 ::: C13
and F (C) is a 13 1 function vector containing all the nonlinear terms. Here, A1 ; A1 and
A3 are each 5 5 matrices and matrix A is a sparse block diagonal matrix.
To provide insights into how the approximate solutions change as a function of the choice
of the number of collocation points and …nite elements, we have carried out studies on
the TRAM problem (with Pe = 6 and Da = 2). The resulting set of coupled nonlinear
algebraic equations were solved simultaneously using Newton’s method. Figure 13 presents
the approximate solution constructed on three segments with three internal collocation points
on each segment. The solutions obtained in each segment and the corresponding polynomial
solutions are as follows:

65
 0.85

0.8 Poly. Soln. in Element 1

Poly. Soln. in Element 2
0.75 Poly. Soln. in Element 3
Orthogonal Collocation points
0.7
Finite Element 2
Concentration

0.65
Finite Element 3
0.6

0.55
Finite Element 1
0.5

0.45

0.4

0.35
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
z-->

Figure 13: TRAM Problem: Solution obtained using orthogonal collocation on …nite ele-
ments.

Segment 1:

1;i 0 0:1127 0:5 0:8873 1

(1)
C ( 1;i ) 0:8313 0:7983 0:7010 0:6234 0:6037
2 3 4
C1 ( 1 ) = 0:8313 0:3037 1 + 0:0989 1 0:0276 1 + 0:0047 1 (256)
Segment 2:
2;i 0 0:1127 0:5 0:8873 1
(2)
C ( 1;i ) 0:6037 0:5792 0:5076 0:4521 0:4386
2 3 4
C2 ( 2 ) = 0:6037 0:2262 2 + 0:0784 2 0:0240 2 + 0:0067 2 (257)
Segment 3:
3;i 0 0:1127 0:5 0:8873 1
C (3) ( 1;i ) 0:4386 0:4293 0:4028 0:3882 0:3873
2 3 4
C3 ( 3 ) = 0:4386 0:0860 3 + 0:0286 3 0:0046 3 + 0:0106 3 (258)

Finally, solutions obtained using the …nite di¤erence (FD), orthogonal collocation (OC)
and OC on …nite elements (OCFE) are compared in Figure 14. This …gure demonstrates that
orthogonal collocation based approach is able to generate an approximate solution, which is
comparable to the FD solution with a large number of grid points, using signi…cantly less
number of collocation points and hence signi…cantly less computational cost.

66
 Figure 14: TRAM Problem: Comparison of FD, OC and OCFE solutions

The method described above can be easily generalized to any number of …nite elements.
Also, the method can be extended to the discretization of PDEs in a similar way. These
extensions are left to the reader as an exercise and are not discussed separately. Note that
block diagonal and sparse matrices naturally arise when we apply this method.

Example 25 [3] Consider the ODE-BVP describing unsteady state conditions in a tubular
reactor with axial mixing (TRAM) in which an irreversible 2nd order reaction is carried out.
It is desired to solve this problem by the OCFE approach.
Continuing from Example 24, let us assume that we create 3 sub-domains. Discretization
in the spatial direction and forcing residual to zero at the internal grid points in each segment
yields the folloing set of DAEs
1 h (1) T (1) i
B.C.1 : s C (t) = P e(C1 (t) 1)
h1

dCi 1 1 T 1 T
= t(i) C(1) (t) s(i) C(1) (t) Da (Ci (t))2 (259)
dt P e h21 h1
i = 2; 3; 4
1 T 1 T
s(5) C(1) (t) = s(1) C(2) (t)
h1 h2

67
 dC4+i 1 1 T 1 T
= t(i) C(2) (t) s(i) C(2) (t) Da (C4+i (t))2 (260)
dt P e h22 h2
i = 2; 3; 4
1 T 1 T
s(5) C(2) (t) = s(1) C(3) (t)
h2 h3

dC8+i 1 1 T 1 T
= t(i) C(3) (t) s(i) C(3) (t) Da (C8+i (t))2 (261)
dt P e h23 h3
i = 2; 3; 4
1 h T
i
B.C.2 : s(5) C(3) (t) = 0
h3
where h iT
(1)
C (t) = C1 (t) C2 (t) C3 (t) C4 (t) C5 (t)
h iT
(2) (262)
C (t) = C5 (t) C6 (t) C7 (t) C8 (t) C9 (t)
h iT
C(3) (t) = C9 (t) C10 (t) C11 (t) C12 (t) C13 (t)
Thus, the process of discretization in this case yields a set of di¤ erential algebraic equa-
tions (DAEs) of the form
dxd
= F (xd ; xa )
dt
0 = Md xd +Ma xa b

which has to be solved simultaneously subject to the speci…ed initial conditions on (xd ; xa ).
Here, the di¤erential states are
h iT
xd (t) = C2 (t) C3 (t) C4 (t) C6 (t) C7 (t) C8 (t) C10 (t) C11 (t) C12 (t)

and the algebraic states are

h iT
xa (t) = C1 (t) C5 (t) C9 (t) C13 (t)

Since the algebraic constraints are linear the set of DAEs can be expressed as a set of ODEs
of the form

xa = (Ma ) 1 (b Md xd )
dxd
= F [xd ; xa ]
dt
To see how this can be achieved, consider algebraic constraints arising from the boundary
conditions and from the continuity equations

13C1 (t) + 14:79C2 (t) 2:67C3 (t) + 1:88C4 (t) C5 (t) = h1 P e(C1 (t) 1) (263)

68
 C1 (t) 1:88C2 (t) + 2:67C3 (t) 14:79C4 (t) + 13C5 (t)
4
= [ 13C5 (t) + 14:79C6 (t) 2:67C7 (t) + 1:88C8 (t) C9 (t)] (264)
3

C5 (t) 1:88C6 (t) + 2:67C7 (t) 14:79C8 (t) + 13C9 (t)

3
= [ 13C9 (t) + 14:79C10 (t) 2:67C11 (t) + 1:88C12 (t) C13 (t)] (265)
4

C9 (t) 1:88C10 (t) + 2:67C11 (t) 14:79C12 (t) + 13C13 (t) = 0 (266)
To convert the set of DAEs to a set of ODEs, we need to express the algebraic states in terms
of the di¤erential states. This can be done by rearranging equations (263)-(266) as follows

Ma xa (t) + Md xd (t) b=0 (267)

2 3 2 3
13 h1 P e 1 0 0 0:4P e
6 1 13 7 4
0 7 6 0 7
6 3 3 7 6 7
Ma = 6 7 3 7
; b =6 7
4 0 1 13 4 4
5 4 0 5
0 0 1 13 0
2 3
14:79 2:67 1:88 0 0 0 0 0 0
6 1:88 2:67 14:79 14:79 4 2:67 4 1:88 4
0 0 0 7
6 3 3 3 7
Md = 6 14:79 3 2:67 3 1:88 3 7
4 0 0 0 1:88 2:67 14:79 4 4 4
5
0 0 0 0 0 0 1:88 +2:67 14:79
i.e.
1
xa (t) = (M1 ) [b M2 xd (t)] (268)
Thus, the sequence of calculations to evaluate the time derivatives, dCd =dt; at time instant
say tn = nh; is as follows:
Step 1: Given initial state, xd (tn ) xd (n); compute
1
xa (n) = (M1 ) [b M2 xd (n)] (269)

Step 2: Using xd (n) and xa (n) construct vectors C(1) (n); C(2) (n) and C(3) (n) as in equa-
tions (262)
Step 3: Using vectors C(1) (n); C(2) (n) and C(3) (n) and equations (259)-(261) …nd vector
dCd =dt at t = nh
Note that M1 and M2 are constant matrices and have to be created only once in the
beginning.

69
3.5 Solving ODE-IVPs using Polynomial Interpolation
3.5.1 Multi-Step Methods

Use of the polynomial interpolation based approximation gives rise to the multi-step meth-
ods for solving ODE-IVPs. Consider the problem of solving scalar ODE IVP (270). We
approximate the solution of the di¤erential equation using an interpolation polynomial in
the independent variable, t. Consider the scalar di¤erential equation
dx
= f (x; t) ; x(tn ) = x(n) (270)
dt
that we want to solve over tn t < tn+1 . Here, tn = nh : n = 0; 1; 2:::; N 1, where h
represents interval of integration, are uniformly spaced integration time points. Let us adopt
a simplifying notation

x(n) x(tn ); x(n 1) x(tn 1 ); :::; x(n p) x(tn p ) (271)

f (n) f [x(tn ); tn ] ; f (n 1) f [x(tn 1 ); tn 1 ] ; :::; f (n p) f [x(tn p ); tn p(272)
]

and so on. At time t = tn ; we have the state and the derivative information in the ’past’, i.e.

fx(n); x(n 1); x(n 2); ::::::; x(0)g (273)

and
ff (n); f (n 1); f (n 2); ::::::; f (0)g (274)
which can be used to construct the interpolation polynomial. We approximate x(t) in the
neighborhood of t = tn by constructing a local polynomial approximation of the type

xn (t) = a0 (n) + [a1 (n)] t + [a2 (n)] t2 + ::::: + [am (n)] tm (275)

and use it to estimate or extrapolate x(n+1): The coe¢ cients of the polynomial are computed
using the state and the derivative information from the past and possibly f (n + 1). It
may be noted that the time index, n; is included in the representation of the polynomial
coe¢ cients to highlight the fact that these coe¢ cients are time varying. In other words, at
each integration step, we compute a separate local interpolation polynomial and use it for
local extrapolation or estimation.
To understand how this is done, consider a simple case where we want to construct a
second order interpolation polynomial of the form

xn (t) = a0 (n) + [a1 (n)] t + [a2 (n)] t2 (276)

70
at instant t = tn : This implies the derivative, f (x; t); at time, t; can be computed as

dxn (t)
f (x; t) = = a1 (n) + 2a2 (n) t (277)
dt
For the sake of computational convenience, we have chosen a shifted time scale as, = t tn :
Thus, it follows that

t = tn ! = 0 ; t = tn+1 ! =h (278)
t = tn 1 ! = h ; t = tn 2 ! = 2h (279)

Thus, in terms of the transformed time, the proosed local polynomial approximation is
2
xn ( ) = a0 (n) + [a1 (n)] + [a2 (n)] (280)

Now, there are several ways we could go about estimating the unknown parameters of the
polynomial.

Explicit algorithm: Let us use only the current and the past information of the
state variable and the derivatives, which will lead to an explicit algorithm:

f (n 1) = a1 (n) 2ha2 (n)

f (n) = a1 (n) (281)

x(n) = a0 (n) (282)

Solving the above equations simultaneously, we get the coe¢ cients as

f (n) f (n 1)
a0 (n) = x(n) ; a1 (n) = f (n) ; a2 (n) =
2h
and the interpolation polynomial with time varying coe¢ cients can be expressed as
follows
f (n) f (n 1) 2
x( ) = x(n) + [f (n)] +
2h
This interpolation polynomial can be used to extrapolate and generate the value of
x(t = tn+1 ) i.e., = h; as

x(n + 1) x( = h)
f (n) f (n 1)
= x(n) + f (n) h + h2
2h
3 1
= x(n) + h f (n) f (n 1)
2 2

71
 Implicit algorithm: Alternatively, we can choose to estimate x(n + 1) based on the
derivative at tn+1 ;i.e.

f (n + 1) = a1 (n) + 2a2 (n) h (283)

f (n) = a1 (n) (284)

x(n) = a0;n (285)

These equations yield the following set of coe¢ cients

f (n + 1) f (n)
a0 (n) = x(n) ; a1 (n) = f (n) ; a2 (n) =
2h
and the interpolation polynomial solution can be expressed as follows

f (n + 1) f (n) 2
x( ) = x(n) + [f (n)] +
2h

The quantity, x(n + 1); can now be estimated as

x(n + 1) x( = h)
f (n + 1) f (n) 2
= x(n) + [f (n)] h + h
2h

The above expression can be rearranged as

h
x(n + 1) = x(n) + [f (n) + f (n + 1)]
2
which is popularly known as the trapezoidal rule or the Crank-Nicholson algorithm.

This approach is extended to solve a set of multivariable ODE-IVP of the form

dx
= F (x;t) (286)
dt
where x 2 Rn ; over a large interval 0 t tf starting from initial condition, x(0) by
simply using the coe¢ cients derived from the scalar case. Thus, the multivariate version of
Crank-Nicholson algorithm is as follows
h
x(n + 1) = x(n) + [F(n) + F(n + 1)] (287)
2
Use of di¤erent order interpolation polynomials and di¤erent ways of constructing them leads
to a wide range of multi-step methods. A detailed exposition to this class of algorithms is
later presented in Module on ODE-IVPs Solvers.

72
3.5.2 Orthogonal Collocations

Another method that makes use of polynomial interpolation for solving ODE-IVP is based
on orthogonal collocations. Consider the scalar ODE-IVP given by equation (270), which
has to be integrated over interval [tn ; tn+1 = tn + h]: De…ning scaled time variable as
t tn
=
h
and a new state variable y( ) the ODE-IVP can be transformed as follows
dy
= h f (y; tn + h ) (288)
d
y(0) = x(n)

It may be noted that y(1) now corresponds to x(tn+1 ). We illustrate here how this trans-
formed ODE-IVP can be solved using three internal collocation points between [0; 1]. Assum-
ing that we have 3 internal collocation points at the roots of the 3rd order shifted Legendre
polynomial, we de…ne …ve point values of as

1 = 0; 2 = 0:1127; 3 = 0:5; 4 = 0:8873 and 5 =1

Let us de…ne a vector, z; such that the ith element of z corresponds to the value of z at
= i
yi = y( i )
Since the initial value is speci…ed, it follows that

y1 = y(0) = x(n) (289)

Now, using S matrix de…ned in Example 15, we can set up the following algebraic constraints
T
s(i) z =h f (yi ; tn + h i ) (290)

for i = 2; 3; 4,5. Equations (289) and (290) can be combined and rearranged as follows
2 32 3 2 3 2 3
3:87 2:07 1:29 0:68 y2 hf (y2 ; tn + h 2 ) 5:32
6 3:23 0 3:23 7 6 7 6
1:5 7 6 y3 7 6 hf (y3 ; tn + h 3 ) 7 6 1:5 7
6 7 6 7
6 76 7=6 7 6 7 x(n) (291)
4 1:29 2:07 3:87 5:32 5 4 y4 5 4 hf (y4 ; tn + h 4 ) 5 4 0:68 5
1:88 2:67 14:79 13 y5 hf (y5 ; tn + h 5 ) 1

The resulting set of nonlinear algebraic equations can be solved using any standard approach
such as Newton Raphson method. The solution yields

x(tn+1 ) = y( = 1) = y5 (292)

73
along with the values of y( ) at the intermediate time points. Now, using the collocation
points, we can construct the interpolation polynomial solution
2 3 4
x(tn + ) = y( ) = 0 + 1 + 2 + 3 + 4

where h iT h iT
1
= 0 1 ::: 4 =A y1 y2 ::: y5
Generalization of this approach to the case with more number of internal collocation points
in the interval [0; 1] is straightforward and not discussed separately. Also, extension of this
method to solve multivariable ODE-IVPs of the form (286) will be discussed later in the
module on methods for solving ODE-IVPs.

4 Least Squares Approximations

While constructing an interpolation polynomial, we require that the interpolating function
passes exactly through the speci…ed set of points (see Figure 16). Alternatively, one can
relax the requirement that the approximating function passes exactly through the desired
set of points. Instead, an approximate function is constructed such that it captures the trend
in variation of the dependent variable in some optimal way (see Figure 15).
Thus, the problem of …nding an approximating polynomial is fundamentally di¤erent from
the interpolation problem. Let us consider a function u(z) 2 C[a; b] and assume that we
have knowledge of the values this function takes at discrete point fzi : i = 1; :::; ng 2 C[a; b].
Let us denote these values as ui = u(zi ). It is desired to construct a low order polynomial
approximation of the form
m
p(z) = 0 + 1z + ::: + mz +e (293)

where m << n and e represents approximation error. Thus, we have n equations of the form
m
ui = 0 + 1 zi + ::: + mz + ei (294)
i = 1; 2; ::::; n

or 2 3 2 32 3 2 3
u1 1 z1 :::: z1m 0 e1
6 u2 7 6 1 z2 :::: z2m 76 7 6 e2 7
6 7 6 76 1 7 6 7
u =6 7=6 76 7+6 7=A +e (295)
4 :::: 5 4 :::: :::: :::: :::: 5 4 :::: 5 4 :::: 5
un 1 zn :::: znm m en
Thus, we have (m + 1) parameters and n errors, ei : i = 1; 2; :::; n; that are unknown
and we only have n equations. Thus, there are in…nite possible solutions to this problem.

74
Figure 15: Approximating function captures the trend but does not necessarily pass through
all the points

Figure 16: Interpolating function passes through all the points

75
As discusses in module on fundamentals of vector spaces, the least square estimate of the
parameter vector can be found as follows
1
bLS = AT A AT u (296)

In more general settings, let ff1 (z); :::fm (z)g represent a set of linearly independent functions
in C[a; b]: Then, we can propose to construct an approximating function, say g(z); of u(z)
as follows
g(z) = 1 f1 (z) + ::::::: + m fm (z) (297)
where m << n; where the unknown coe¢ cients f 1 ; ::: mg are determined from the data set
in some optimal manner. Thus, we have n equations

ui = [ 1 f1 (zi ) + ::::::: + m fm (zi )] + ei (298)

i = 1; 2; :::n

in (m + n + 1) unknowns. De…ning
h iT
= 1 2 ::: m (299)

the best approximation g(z) in the least squares sense can be found using equation (296)
where matrix A in this case is
2 3
f1 (z1 ) f2 (z1 ) :::: fm (z1 )
6 f (z ) f (z ) :::: f (z ) 7
6 1 2 2 2 m 2 7
A=6 7 (300)
4 :::: :::: :::: :::: 5
f1 (zn ) f2 (zn ) :::: fm (zn ) (n m)

4.1 ODE-BVP / PDE Discretization using Minimum Residual

Methods
In interpolation based methods, we force the residuals to zero at a …nite set of collocation
points. Based on the least squares approach, one can think of constructing an approximation
so that the residual becomes small (in some sense) on the entire domain. Thus, given an
ODE-BVP/PDE, we seek an approximate solution as a linear combination of a …nite number
of linearly independent functions. The parameters of this approximation are determined
in such a way that some norm of the residuals is minimized. There are many discretization
methods that belong to this broad class. In this section, we provide a brief introduction to
these discretization approaches.

76
4.1.1 Raleigh-Ritz method [11, 12]

To understand the motivation for developing this approach, …rst consider a linear system of
equations
Ax = b (301)
where A is an n n positive de…nite and symmetric matrix and it is desired to solve for the
vector x. We can pose this as a minimization problem by de…ning an objective function of
the form

(x) = (1=2)xT Ax xT b (302)

= (1=2) hx; Axi hx; bi (303)

If (x) has a minimum at x = x ; then the necessary condition for optimality requires

@ =@x = Ax b=0 (304)

which is precisely the equation we want to solve. Since the Hessian matrix

@ 2 =@x2 = A

is positive de…nite, the solution of x = x of Ax = b is the global minimum of the objective

function (x):
In the above demonstration, we were working in space Rn . Now, let us see if a similar
formulation can be worked out in another space, namely C (2) [0; 1]; i.e. the set of twice
di¤erentiable continuous functions on [0; 1]: Consider the ODE-BVP

d2 u
Lu = = f (z) (305)
dz 2
B:C: 1 : u(0) = 0 (306)
B:C: 2 : u(1) = 0 (307)

Similar to the linear operator (matrix) A, which operates on vector x 2 Rn to produce

another vector b 2Rn ; the linear operator L = [ d2 =dz 2 ]operates on vector u(z) 2 C (2) [0; 1]
to produce f (z) 2 C[0; 1]: Note that the matrix A in our motivating example is symmetric
and positive de…nite, i.e.

hx; Axi > 0 for all x 6= 0

and AT = A

In order to see how the concept of a symmetric matrix can be generalized to operators on
in…nite dimensional spaces, let us …rst de…ne the adjoint of a matrix.

77
De…nition 26 (Adjoint of Matrix): A matrix A is said to be the adjoint of matrix A
if it satis…es hx; Ayi = hA x; yi : Further, the matrix A is called self adjoint if A = A:

When matrix A has all real elements, we have

xT (Ay) = (AT x)T y

and it is easy to see that A = AT ; i.e.

hx; Ayi = AT x; y (308)

The matrix A is called self-adjoint if AT = A: Does operator L de…ned by equations (305-

307) have some similar properties of symmetry and positiveness? Analogous to the concept
of the adjoint of a matrix, we …rst introduce the concept of adjoint of an operator L on any
inner product space.

De…nition 27 (Adjoint of the Operator) An operator L is said to be adjoint of operator

L if it satis…es
hv; Lui = hL v; ui
Further, the operator L is said to be self-adjoint, if L = L, B:C:1 = B:C:1 and
B:C:2 = B:C:2:

To begin with, let us check whether the operator L de…ned by equations (305-307) is
self-adjoint.
Z1
hv; Lui = v(z)( d2 u=dz 2 )dz
0

1 Z1
du dv du
= v(z) + dz
dz 0 dz dz
0

1 1 Z1
du dv d2 v
= v(z) + u(z) + u(z)dz
dz 0 dz 0 dz 2
0

Using the boundary conditions u(0) = u(1) = 0, we have

1
dv dv dv
u(z) = u(1) u(0) = 0
dz 0 dz dz
If we set

B:C:1 : v(0) = 0
B:C:2 : v(1) = 0

78
then
1
du
v(z) =0
dz 0
and we have
Z1
d2 v
hv; Lui = u(z)dz = hL v; ui
dz 2
0

In fact, it is easy to see that the operator L is self adjoint as L = L, BC1 = BC1 and
BC2 = BC2: In addition to the self-adjointness of L; we have

1 Z1 2
du du
hu; Lui = u(z) + dz
dz 0 dz
0
Z1 2
du
= dz > 0 for all u(z)
dz
0

when u(z) is a non-zero vector in C (2) [0; 1]: In other words, solving the ODE-BVP is anal-
ogous to solving Ax = b by the optimization formulation where A is a symmetric and
positive de…nite matrix, i.e.

A$ d2 =dz 2 ; x $ u(z); b $ f (z)

Let u(z) = u (z) represent the true solution of the ODE-BVP. Now, taking motivation from
the optimization formulation for solving Ax = b, we can formulate a minimization problem
to compute the solution

[u(z)] = (1=2) u(z); d2 u=dz 2 hu(z); f (z)i (309)

Z1 Z1
= 1=2 u(z)( d2 u=dz 2 )dz u(z)f (z)dz (310)
0 0

M in
u (z) = [u(z)] (311)
u(z)
M in
= (1=2) hu(z); Lu(z)i hu(z); f (z)i (312)
u(z)

u(z) 2 C (2) [0; 1] (313)

79
 subject to u(0) = u(1) = 0
Thus, solving the ODE-BVP has been converted to solving a minimization problem. Inte-
grating the …rst term in equation (310) by parts, we have

Z1 Z1 2 1
d2 u du du
u(z) dz = dz u (314)
dz 2 dz dz 0
0 0

Now, using the boundary conditions, we have

1
du du du
u = u(0) u(1) =0 (315)
dz 0 dz z=0 dz z=1

This reduces (u) to

2 3 2 1 3
Z1 2 Z
du
(u) = 41=2 dz 5 4 uf (z)dz 5 (316)
dz
0 0

The above equation is similar to an energy function, where the …rst term is analogous to
kinetic energy and the second term is analogous to potential energy. As

Z1 2
du
dz
dz
0

is positive and symmetric, we are guaranteed to …nd the minimum. The main di¢ culty in
performing the search is that the search space is in…nite dimensional as u(z) 2 C (2) [0; 1].
One remedy to alleviate this di¢ culty is to reduce the in…nite dimensional search problem to
a …nite dimensional search space by constructing an approximate solution using n trial func-
tions. Let v (1) (z); :::::; v (n) (z) represent the trial or basis functions. Then, the approximate
solution is constructed as follows

(1) (n)
u(z) = 1v (z) + ::::: + nv (z) (317)

where v (i) (z) represents trial functions. Using this approximation, we convert the in…nite

80
dimensional optimization problem to a …nite dimensional optimization problem as follows
2 3 2 1 3
Z1 2 Z
du
M in b( ) = 41=2 dz 5 4 uf (z)dz 5 (318)
dz
0 0
Z1 2
dv (1) (z) dv (n) (z)
= 1=2 1 + ::::: + n dz
dz dz
0
Z1
(1) (n)
f (z)[ 1v (z) + ::::: + nv (z)]dz (319)
0

The trial functions v (i) (z) are chosen in advance and coe¢ cients 1 ; :::: m are treated as
unknown. Also, let us assume that these functions are selected such that u(0) = u(1) = 0:
Then, using the necessary conditions for optimality, we get
@b
= 0 f or i = 1; 2; ::::; n (320)
@ i
These equations can be rearranged as follows
@b
=A b=0 (321)
@
where h iT
= 1 2 ::: n
2 3
dv (1) dv (1) dv (1) dv (n)
6 ; :::::::: ; 7
6 dz dz dz dz 7
6 7
A=6 :::::::::::::::::: :::::::: ::::::::::::: 7 (322)
6 7
4 dv (n) dv (1) dv (n) dv (n) 5
; :::::::: ;
dz dz dz dz
2 3
v (1) (z); f (z)
6 7
b = 4 :::::::::::: 5 (323)
(n)
v (z); f (z)
Thus, the optimization problem under consideration can be recast as follows
M in b M in T T
( )= (1=2) A b (324)

It is easy to see that matrix A is positive de…nite and symmetric and the global minimum
of the above optimization problem can be found by using necessary condition for optimality
i.e. @ b=@ = A b = 0 or = A 1 b: Note the similarity of the above equation with the
normal equation arising from the projection theorem. Thus, the steps in the Raleigh-Ritz
method can be summarized as follows

81
 1. Choose an approximate solution.

2. Compute matrix A and vector b

3. Solve for A = b

How does one select the basis function? The basis functions need to be selected such that
the boundary conditions are satis…ed. One possible basis is to choose a global polynomial
basis
v (i) (z) = z i (1 z) for i = 1; 2; :::n (325)
Quali…er global is used here because v (i) (z) is de…ned over z 2 [0; 1]. Trouble with this choice
is severe illconditioning of matrix A [1]. Alternatively, the trial functions can be chosen as

v (i) (z) = sin(i z) for i = 1; 2; :::n (326)

These are orthogonal functions with respect to inner product [1]

Z 1
df dg
< f (z); g(z) >= f (a)g(a) +
0 dz dz

i2
For this choice, v (i) (z) = and matrix A is diagonal. Thus, choosing global trigono-
2
metric polynomial is much better choice of basis or trial functions for this problem. It may
be noted that even the global trigonometric polynomial basis can lead to ill conditioning in
some problems [1]. These di¢ culties can be avoided and matrix conditioning can be main-
tained if we select a basis with small support, such as piecewise polynomial approximations.
This leads to the …nite element methods (ref. Subsection 4.2).

4.1.2 Method of Least Squares [5]

This is probably the best known minimum residual method. When used for solving linear
operator equations, this approach does not require self adjointness of the linear operator. To
understand the method, let us …rst consider a linear ODE-BVP

L [u(z)] = f (z) (327)

B:C:1 : u(0) = 0 (328)
B:C:2 : u(1) = 0 (329)

Consider an approximate solution constructed using a linear combination of a set of a

…nite number of linearly independent functions as follows
(1) (n)
u(z) = 1v (z) + ::::: + nv (z) (330)

82
Let us assume that these basis functions are selected such that the two boundary conditions
are satis…ed, i.e. ui (0) = ui (1) = 0: Given this approximate solution, the residual is de…ned
as follows
"(z) = L [u(z)] f (z) where 0 < z < 1
Linearity of the di¤erential operator implies that
X
n
(1) (2) (n)
L [u(z)] = 1L v (z) + 2L v (z) + :::: + nL v (z) = iL v (i) (z)
i=1

and
X
n
"(z) = iL v (i) (z) f (z)
i=1
The idea is to determine h iT
= 1 2 ::: n

such that
M in
[ ( ) = h"(z); "(z)i]
Z 1
h"(z); "(z)i = !(z)"(z)2 dz
0
where !(z) is a positive function on 0 < z < 1: Using linearity of the di¤erential operator,
we can interpret the least squares problem as …nding projection of vector f (z) on the n
dimensional subspace spanned by vectors

L v (1) (z) ; L v (2) (z) ; ::::L v (n) (z)

Solution of this minimization problem leads to a generalized normal form of equation (ref.
subsection on Orthogonal Projections Module 2)
2 32 3 2 3
Lv (1) ; Lv (1) Lv (1) ; Lv (2) :::: Lv (1) ; Lv (n) 1 Lv (1) ;f (z)
6 Lv (2) ; Lv (1) Lv (2) ; Lv (2) :::: Lv (2) ; Lv (n) 7 6 7 6 Lv (2) ;f (z) 7
6 76 2 7 6 7
6 76 7=6 7 (331)
4 ::::: ::::: ::::: ::::: 5 4 :::: 5 4 :::: 5
Lv (n) ; Lv (1) Lv (n) ; Lv (2) ::::: Lv (n) ; Lv (n) m Lv (n) ;f (z)
which can be solved analytically.

Example 28 [5] Use the least squares method to …nd an approximate solution of the equation
@2u
L [u(z)] = u=1 (332)
@z 2
B:C: 1 : u(0) = 0 (333)
B:C: 2 : u(1) = 0 (334)

83
Let us select the function expansion as

b(z) =
u 1 sin ( z) + 2 sin (2 z) (335)

It may be noted that this choice ensures that the boundary conditions are satis…ed. Now,

L v (1) (z) = ( 2
+ 1) sin( z) (336)
L v (2) (z) = (4 2
+ 1) sin(2 z) (337)

With the inner product de…ned as

Z 1
hf; gi = f (z)g(z)dz (338)
0

the normal equation becomes

" 2 #" # " #
( +1)2 2( 2 +1)

2
0 1
(4 2 +1)2 = (339)
0 2 2 0
and the approximate solution is
4
b(z) =
u sin ( z)
2
(340)
( + 1)
which agrees with the exact solution
ez + e1 z
u(z) = 1 (341)
(e + 1)
to within 0.006.

When the boundary conditions are non-homogeneous, it is sometimes possible to trans-

form them to homogeneous conditions. For example consider a modi…ed problem
@2u
L [u(z)] = u=1 (342)
@z 2
B:C: 1 : u(0) = (343)
B:C: 2 : u(1) = (344)

For this problem, consider transformation

v(z) = u(z) + ( )z + (345)

In terms of the transformed variable, the problem becomes [5]

@2v
L [v(z)] = v=1 ( )z (346)
@z 2
B:C: 1 : v(0) = 0 (347)
B:C: 2 : v(1) = 0 (348)

84
Alternatively, the optimization problem is formulated in such a way that the boundary
conditions are satis…ed in the least square sense. Let X C (2) [0; 1] and Y C[0; 1] R R
and de…ne L : X ! Y where [5]
@2u
L [u(z)] = u(z); u(0); u(1) (349)
@z 2
The ODE-BVP can now be written as

L [u(z)] = [1; ; ] (350)

Now, let us de…ne an inner product on Y as follows

Z 1
h(f (z); a; b) ; (g(z); c; d)i = ! 1 (z)f (z)g(z)dz + w2 ac + w3 bd
0

where w1 (z); w2 ; w3 > 0: Now, consider a …nite dimensional approximate solution of the form
(330). Then,
@2u
("(z); "(0); "(1)) = L [u(z)] [1; ; ] = u(z) 1; u(0) ; u(1)
@z 2
and

( ) = h("(z); "(0); "(1)) ; ("(z); "(0); "(1))i

Z 1 2
@2u
= ! 1 (z) u(z) 1 dz + w2 (u(0) )2 + w3 (u(1) )2 (351)
0 @z 2
With this choice of the objective function, neither the equation nor the boundary conditions
are satis…ed exactly. The choice of weights (w1 (z); w2 ; w3 ) will decide the behavior of the
approximate solution.
While this method can be, in principle, extended to the discretization of the general
ODE-BVP of type (70-72a), working with a parameter vector as a minimizing argument
can pose practical di¢ culties as the resulting minimization problem has to be solved numer-
ically. Coming up with an initial guess of to start the iterative algorithms can prove to
be a tricky task. Alternatively, one can work with trial solutions of the form (378) or (394)
to make the problem computationally tractable (ref. Subsection 4.2.2).

4.1.3 Galerkin’s Method[5, 3]

The Gelarkin’s method can be applied for any problem where the di¤erential operator is
not self adjoint or symmetric. Instead of minimizing (u), we solve for

v (i) (z); Lu(z) = v (i) (z); f (z)

i = 1; 2; ::::; n

85
where u(z) is chosen as a …nite dimensional approximation to u (z) (i.e. the true solution)

u(z) = u1 v (1) (z) + :::::: + un v (n) (z) (352)

Rearranging the above equations as

v (i) (z); (Lu(z) f (z)) = 0 for (i = 1; 2; ::::n)

we can observe that the parameters u1 ; ::::; un are computed such that the error or residual
vector
e(z) = (Lu(z) f (z))
is orthogonal to the (n) dimensional subspace spanned by set S de…ned as

S = v (i) (z) : i = 1; 2; ::::; n

This results in a linear algebraic equation of the form

Au = b (353)

where 2 3
v (1) ; L(v (1) ) :::::::: v (1) ; L(v (n) )
6 7
A = 4 ::::::::::::: :::::::: ::::::::::: 5 (354)
v ; L(v ) :::::::: v (n) ; L(v (n) )
(n) (1)

2 3
v (1) (z); f (z)
6 7
b = 4 ::::::::::::: 5
(n)
v (z); f (z)
b results in an approximate solution given by equation (352). When the
Solving for u
operator L is self adjoint, the Galerkin’s method reduces to the Raleigh-Ritz method.

Example 29 Consider ODE-BVP

Lu = @ 2 u=@z 2 @u=@z = f (z) (355)

in (0 < z < 1) (356)
subject to u(0) = 0; u(1) = 0 (357)

It can be shown that

L (= @ 2 =@z 2 + @=@z) 6= (@ 2 =@z 2 @=@z) = L

Thus, the Raleigh-Ritz method cannot be applied to generate an approximate solution to this
problem. However, the Galerkin’s method can be applied.

86
 It may be noted that one need not restrict to linear transformations while applying the
Gelarkin’s method. This approach can be used even when the ODE-BVP or PDE at hand
is a nonlinear transformation. Given a general nonlinear transformation of the form

T [u(z)] = f (z)

we select a set of trial function v (i) (z) : i = 0; 1; :::n and an approximate solution of the
form (352) and solve for

v (i) (z); T [u(z)] = v (i) (z); f (z) for i = 0; 1; 2; ::::n

Example 30 [5] Use the Galerkin’s method to …nd an approximate solution of the equation
@2u
L [u(z)] = u=1 (358)
@z 2
B:C: 1 : u(0) = 0 (359)
B:C: 2 : u(1) = 0 (360)

Let us select the function expansion as follows

b(z) =
u 1 sin ( z) + 2 sin (2 z)

which implies
2 2
L [b
u(z)] = 1( + 1) sin( z) 2 (4 + 1) sin(2 z)
With the inner product de…ned as
Z 1
hf; gi = f (z)g(z)dz
0

the normal equation becomes

" 2 +1)
#" # " #
( 2
2
0 1
(4 2 +1) =
0 2 2 0

and the approximate solution is

4
b(z) =
u sin ( z)
2
( + 1)
which turns out to be identical to the least squares solution.

Example 31 [3] Consider the ODE-BVP describing steady state conditions in a tubular
reactor with axial mixing (TRAM) in which an irreversible 2nd order reaction is carried out.
1 d2 C dC
T (C) = DaC 2 = 0 (0 z 1)
P e dz 2 dz
87
 dC
= P e(C 1) at z = 0;
dz
dC
= 0 at z = 1;
dz
The approximate solution is chosen as follows (ref. Subsection 4.2)

X
n+1
(1) (n+1)
C(z) = C1 v (z) + :::::: + Cn+1 v (z) = Ci v (i) (z) (361)
i=1

and we then evaluate the following set of equations

1 d2 C(z) dC(z)
v (i) (z); DaC(z)2 = v (i) (z); f (z) for i = 2; ::::n
P e dz 2 dz
where the inner product is de…ned as
Z 1
hg(z); h(z)i = g(q)h(q)dq
0

It may be noted that evaluation of integrals, such as

Z 1
(i) 2
Pn 2
v (z); C(z) = v (i) (q) (i)
i=0 Ci v (q) dq
0

will give rise to equations that are nonlinear in terms of unknown coe¢ cients. Two additional
equations arise from enforcing the boundary conditions. i.e.
dC(0)
= P e(C(0) 1)
dz
dC(1)
= 0
dz
Thus, we get (n+1) nonlinear algebraic equations in (n+1) unknowns, which have to be solved
simultaneously to compute the unknown coe¢ cients C1 ; :::; Cn+1 . Details of computing these
integrals and developing piecewise approximating functions on …nite element can be found in
[3].

4.2 ODE-BVP / PDE Discretization using the Finite Element

Method
The …nite element method is a powerful tool for solving PDEs particularly when the system
under consideration has complex geometry. This method is based on the least square approx-
imation. In this section, we provide a very brief introduction to the method discretization
of PDEs and ODE-BVPs using the …nite element method.

88
4.2.1 Discretization of ODE-BVPs using Finite Elements [11]

Similar to …nite di¤erence method, we begin by choosing (n 1) equidistant internal node

(grid) points as follows
zi = i z (i = 0; 1; 2; ::::; n)
and de…ning n …nite elements

zi 1 z zi for i = 1; 2; ::::; n

Then we formulate the approximate solution using piecewise constant polynomials on each
…nite element. The simplest possible choice is a line

bi (z) = ai + bi z
u (362)
zi 1 z zi for i = 1; 2; ::::; n (363)

With this choice, the approximate solution for the ODE-BVP can be expressed as
8 9
>
> a 1 + b 1 z f or z0 z z1 >
>
>
< a +b z >
=
2 2 f or z1 z z2
b(z) =
u (364)
>
> ::::: >
>
>
: >
;
an + b n z f or zn 1 z zn

In principle, we can work with this piecewise polynomial approximation. However, the
resulting optimization problem has coe¢ cients (ai ; bi : i = 1; 2; :::n) as unknowns. If the
optimization problem has to be solved numerically, it is hard to generate initial guesses for
these unknown coe¢ cients. Thus, it is necessary to parameterize the polynomial in terms of
unknowns for which it is relatively easy to generate the initial guess. This can be achieved
bi denote the value of the approximate solution u
as follows. Let u b(z) at z = zi ; i.e.

bi = u
u b(zi ) (365)

Then, at the boundary points of the i’th element, we have

b(zi 1 ) = u
u bi 1 = ai + bi zi 1 (366)
b(zi ) = u
u bi = ai + bi zi (367)

Using these equations, we can express (ai ; bi ) in terms of the unknowns (b bi ) as follows
ui 1 ; u

bi 1 zi
u ubi zi 1 bi
u bi
u 1
ai = ; bi = (368)
z z

89
Thus, the polynomial on the ith segment can be written as

bi 1 zi
u ubi zi 1 bi u
u bi 1
bi (z) =
u + z (369)
z z
zi 1 z zi for i = 1; 2; ::::; n

and the approximate solution can be expressed as follows

8 9
> b0 z1
u b1 u
u b0 >
>
> + z f or z0 z z1 >
>
>
> z z >
>
>
> >
>
>
< b1 z2 u
u b2 z1 b2 u
u b1 >
=
+ z f or z1 z z2
b(z) =
u z z (370)
>
> >
>
>
> :::::::::::::: >
>
>
> >
>
>
> b z
u ubn zn 1 bn u
u bn 1 >
: n 1 n + z f or zn 1 z zn >
;
z z

Thus, now we can work in terms of the unknown values fb b1 ; ::::b

u0; u un g instead of the
parameters ai and bi: . Since the unknowns fb b1 ; ::::b
u0; u un g correspond to some physical vari-
able, it is relatively easy to generate good guesses for these unknowns from a knowledge of
the underlying physics of the problem. The resulting form is still not convenient from the
viewpoint of evaluating integrals involved in the computation of [b u(z)] : A more elegant
and useful form of equation (370) can be found by de…ning shape functions. To arrive at
this representation, consider the rearrangement of the line segment equation on ith element
as follows
bi 1 zi
u ubi zi 1 bi
u bi
u 1
bi (z) =
u + z (371)
z z
zi z z zi 1
= bi
u 1 + bi
u
z z
Let us de…ne two functions, Mi (z) and Ni (z); which are called as shape functions, as follows
zi z zi z 1
Mi (z) = ; Ni (z) =
z z
zi 1 z zi for i = 1; 2; :::n

The graphs of these shape functions are straight lines and they have fundamental properties
( )
1 ; z = zi 1
Mi (z) = (372)
0 ; z = zi
( )
0 ; z = zi 1
Ni (z) = (373)
1 ; z = zi

90
 bi (z) as
This allows us to express u

bi (z) = u
u bi 1 Mi (z) + u
bi Ni (z)
i = 1; 2; ::::; n

bi appears in the polynomials u

Note that the coe¢ cient u bi (z) and u
bi+1 (z); i.e.

bi (z) = u
u bi 1 Mi (z) + u
bi Ni (z)
bi+1 (z) = u
u bi Mi+1 (z) + u
bi+1 Ni+1 (z)

Thus, we can de…ne a continuous trial function by combining Ni (z) and Mi+1 (z) as follows
8 z zi 1 z zi 9
>
> Ni (z) = =1+ ; zi 1 z zi > >
>
> z z >
>
>
< >
=
(i)
v (z) = zi+1 z z zi (374)
>
> M (z) = = 1 ; z z z >
>
>
> i+1
z z
i i+1 >
>
>
: >
;
0 Elsewhere
i = 1; 2; ::::; n

This yields the simplest and most widely used hat function, which is shown in Figure 17.
This is a continuous linear function of z; but, it is not di¤erentiable at zi 1; zi; and zi+1 : Also,
note that at z = zi ; we have
( )
1 if i = j
v (i) (zj ) = (375)
0 otherwise
j = 1; 2; ::::; n

Thus, plot of this function looks like a symmetric triangle. The two functions at the boundary
points are de…ned as ramps
( z )
M 1 (z) = 1 ; 0 z z1
v (0) (z) = z (376)
0 Elsewhere
8 9
< N (z) = 1 + z zn
; zn z zn =
(n) n 1
v (z) = z (377)
: 0 Elsewhere ;

Introduction of these trial functions allows us to express the approximate solution as

b0 v (0) (z) + :::::: + u

b(z) = u
u bn v (n) (z) (378)
0 z 1

91
 h iT
and now we can work with ub = b0 u
u b1 ::: u
bn as unknowns. Now, we have two
boundary conditions, i.e.
b0 = 0 and u
u bn = 0
h iT
b= u
and the set of unknowns is reduced to u b1 u
b2 ::: u
bn 1 . The optimum parameters
b can be computed by solving equation
u

Ab
u b=0 (379)

where
dv (i) dv (j)
(A)ij = ; (380)
dz dz
and ( )
(i)
dv 1= z on interval left of zi
=
dz 1= z on interval right of zi
If intervals do not overlap, then
dv (i) dv (j)
; =0 (381)
dz dz
The intervals overlap when
Zzi zZ
i+1
dv (i) dv (i)
i=j : ; = (1= z)2 dz + ( 1= z)2 dz = 2= z (382)
dz dz
zi 1 zi

or
Zzi
dv (i) dv (i 1)
i = j+1: ; = (1= z):( 1= z)dz = 1= z (383)
dz dz
zi 1
zZ
i+1
dv (i) dv (i+1)
i = j 1: ; = (1= z):( 1= z)dz = 1= z (384)
dz dz
zi

Thus, the matrix A is a tridiagonal matrix

2 3
2 1 :::: :::: 0
1 6 1 2 1 :::: ::: 7
6 7
A= 6 7 (385)
z 4 :::: :::: :::: :::: ::: 5
0 :::: :::: 1 2

92
 Figure 17: (a) Trial functions and (b) Piece-wise linear approximation

which is similar to the matrix obtained using the …nite di¤erence method. The components
of vector b on the R.H.S. is computed as

bi = v (i) ; f (z) (386)

Zzi zZ
i+1
z zi z zi
= f (z) 1 + dz + f (z) 1 dz (387)
z z
zi 1 zi

i = 1; 2; :::; n 1 (388)

which is a weighted average of f (z) over the interval zi 1 z zi+1 : Note that the R.H.S.
is signi…cantly di¤erent from the …nite di¤erence method.
In this sub-section, we have developed an approximate solution using piecewise linear ap-
proximations. It is possible to develop piecewise quadratic or piecewise cubic approximations
and generate better approximations. Readers are referred to Strang [13].

4.2.2 Discretization of PDEs using the Finite Element Method [12]

The Raleigh-Ritz method can be easily applied to discretize PDEs when the operators are
self-adjoint. Consider Laplace/Poisson’s equation

Lu = @ 2 u=@x2 @ 2 u=@y 2 = f (x; y) (389)

93
in open set S and u(x; y) = 0 on the boundary. Let the inner product on the space C (2) [0; 1]
C (2) [0; 1] be de…ned as

Z1 Z1
hf (x; y); g(x; y)i = f (x; y) g(x; y) dxdy (390)
0 0

We formulate an optimization problem

(u) = 1=2 u(x; y); @ 2 u=@x2 @ 2 u=@y 2 hu(x; y); f (x; y)i (391)

Integrating by parts, we can show that

Z Z
(u) = [1=2(@u=@x)2 + 1=2(@u=@y)2 f u]dxdy (392)
= (1=2) h@u=@x; @u=@xi + 1=2 h@u=@y; @u=@yi hf (x; y); u(x; y)i (393)

We begin by choosing (n 1) (n 1) equidistant (with x = y = h) internal node

(grid) points at (xi ; yj ) where

xi = ih (i = 1; 2; ::::n 1)
yj = jh (j = 1; 2; ::::n 1)

In two dimensions, the simplest element divides the region into triangles on which simple
polynomials are …tted. For example, u(x; y) can be approximated as

b(x; y) = a + bx + cy
u

where a; b; c can be expressed in terms of values of u b(x; y) at the triangle vertices. For
example, consider the triangle de…ned by (xi ; yj ); (xi+1 ; yj ) and (xi ; yj+1 ): The value of the
approximate solution at the corner points is denoted by

bi;j = u
u b(xi ; yj ) ; u
bi+1;j = u
b(xi+1 ; yj ) ; u
bi;j+1 = u
b(xi ; yj+1 )

b(x; y) can be written in terms of the shape functions as follows

Then, u

bi+1;j u
u bi;j bi;j+1 u
u bi;j
b(x; y) = u
u bi;j + (x xi;j ) + (y yi;j )
h h
(x xi;j ) (y yi;j )
bi;j 1
= u
h h
(x xi;j ) (y yi;j )
+u bi+1;j bi;j+1
+ u (394)
h h

94
 Figure 18: Trial functions in two dimensions

Now, the coe¢ cient u bi;j appears in the shape functions of four triangular element around
(xi ; yj ). Collecting these shape functions, we can de…ne a two dimensional trial function as
follows
8 9
> (x xi;j ) (y yi;j )
> 1
> ; xi x xi+1 ; yj y yj+1 > >
>
>
> h h >
>
>
> (x x ) (y y ) >
>
>
> i;j i;j >
> 1 + ; x i 1 x x i ; y j y y j+1 >
>
< h h =
(i;j)
v (z) = (x x i;j ) (y y i;j )
>
> 1 + ; xi x xi+1 ; yj 1 y yj > >
>
> h h >
>
>
> (x x i;j ) (y y i;j ) >
>
>
> 1 + + ; x i 1 x x i ; yj 1 y y j >
>
>
> h h >
>
: ;
0 Elsewhere

The shape of this trial function is like a pyramid (see Figure 18). We can de…ne trial functions
at the boundary points in a similar manner. Thus, expressing the approximate solution using
trial functions and using the fact that u b(x; y) = 0 at the boundary points, we get

u b1;1 v (1;1) (x; y) + :::: + u

b(x; y) = u bn 1;n 1 v
(n 1;n 1)
(x; y)

where v (i;j) (z) represents the (i; j)th trial function. For the sake of convenience, let us re-
number these trial functions and coe¢ cients using a new index l = 0; 1; :::; N such that

l = i + (n 1)j
i = 1; :::; n 1 and j = 0; 1; :::; n 1
N = (n 1) (n 1)

The approximate solution can now be expressed as

b(x; y) = u
u bN v N (x; y)
b0 v 0 (x; y) + :::: + u

95
The minimization problem an be reformulated as

M in M in 1 @b
u @bu 1 @b
u @bu
(b
u) = ; + ; b(x; y)i
hf (x; y); u
b
u b
u 2 @x @x 2 @y @y

where h iT
b=
u b0 u
u b2 ::: u
bN
Thus, the above objective function can be reformulated as

M in M in
(b
u) = uT Ab
1=2b u bT b
u (395)
b
u b
u

where
(A)ij = (1=2) @v (i) =@x; @v (j) =@x + (1=2) @v (i) =@y; @v (j) =@y (396)

bi = f (x; y); v (j) (x; y) (397)

Again, the matrix A is symmetric and positive de…nite and this guarantees that stationary
point of (u) is the minimum. At the minimum, we have

@ =@b
u = Ab
u b=0 (398)
The matrix A will also be a sparse matrix. The main limitation of the Raleigh-Ritz method
is that it works only when the operator L is symmetric or self adjoint.

5 Errors in Discretization and Computations [5]

As evident from various examples discussed in this chapter, the process of discretization
converts the original (often intractable) problem typically from an in…nite dimensional space
to a computationally tractable form in …nite dimensions. Obviously, solving the discretized
version of the problem, ye=Tb (e
x); even exactly is not equivalent to solving the original problem
y =T (x). In fact, the discretized problem, though it is computationally tractable, is often
not solved exactly and errors get introduced due to the limitations of the computational
procedure. These two sources of errors together cause errors in the …nal computed solution.
Computational errors arise when the discretized problem y e = Tb (ex) cannot be solved
exactly. For example, discretization of nonlinear ODE-BVP, such as the steady state behavior
of a TRAM, gives rise to coupled nonlinear algebraic equations, which cannot be solved
exactly. When we employ Newton’s method to solve the resulting set of equations, we
end up constructing an approximate solution to the set of nonlinear algebraic equations. This

96
happens as the iterations are terminated after a …nite number based on some termination
criterion. In addition, the fact that the arithmetic operations in a computer can be carried
out only with a …nite precision introduces round-o¤ errors. It may be noted that these
round-o¤ errors occur in every iteration and their cumulative e¤ect on the …nal solution is
di¢ cult to predict.
Discretization errors arise from approximation of an in…nite dimensional transforma-
tion by a …nite dimensional one. Thus, while studying the discretization errors, we have to
understand the behavior of the approximate solution, x e; with reference to the true solu-
tion, x. It is reasonable to expect that a numerical method is capable of yielding arbitrarily
accurate answers by making discretization su¢ ciently …ne. A method that gives a sequence of
approximations converging to the true solution is called convergent approximation method.

6 Summary and Conclusions

In this chapter, we have discussed various polynomial approximation based approaches to
discretization of transformations in in…nite dimensional spaces to computationally tractable
forms in …nite dimensional spaces. If we examine the transformed problems, then we can
arrive at the following three classes of discretized problems

Linear and nonlinear algebraic equations: Either we have to solve

Ax = b

for x 2 Rn given b 2 Rn and A is an n n matrix or solve

F (x) = 0

for x 2Rn where F (x) is an n 1 function vector.

Ordinary Di¤erential Equations - Initial Value Problem (ODE-IVP): We have to inte-

grate a system of coupled ordinary di¤erential equations of the form
dx
= F (x; )
d
given initial condition x(0) and 2 [a; b]:

Optimization problem: Minimize some scalar objective function (x) : Rn ! R with

respect to argument x:

The numerical solution techniques for solving these fundamental problems forms the basic
toolkit of numerical analysis. In the modules that follow, we examine each tool separately
and in greater details.

97
References
[1] Atkinson, K., Han, W. Theoretical Numerical Analysis, Springer, 2009.

[2] Bazara, M.S., Sherali, H. D., Shetty, C. M.; Nonlinear Programming, Wiley, New York,
2015.

[3] Gupta, S. K.; Numerical Methods for Engineers, 3rd Ed., New Age International, New
Delhi, 1995.

[4] Kreyzig, E.; Introduction to Functional Analysis with Applications, Wiley, New York,
1978.

[5] Linz, P.; Theoretical Numerical Analysis, Dover, New York, 1979.

[6] Luenberger, D. G.; Optimization by Vector Space Approach , Wiley, New York, 1969.

[7] Gourdin, A. and M Boumhrat; Applied Numerical Methods, Prentice Hall, New Delhi.

[8] Moursund, D. G., Duris, C. S., Elementary Theory and Application of Numerical Analy-
sis, Dover, New York, 1988.

[9] Rall, L. B.; Computational Solutions of Nonlinear Operator Equations. Wiley, New
York, 1969.

[10] Rao, S. S., Engineering Optimization, 4th Ed., New Age International, New Delhi, 2009.

[11] Strang, G.; Linear Algebra and Its Applications. Harcourt Brace Jevanovich College
Publisher, New York, 1988.

[12] Strang, G.; Introduction to Applied Mathematics. Wellesley Cambridge, Massachusetts,

1986.

[13] Strang, G.; Computational Science and Engineering. Wellesley-Cambridge Press, Massa-
chusetts, 2007.

[14] Philips, G. M.,Taylor, P. J. ; Theory and Applications of Numerical Analysis (2’nd Ed.),
Academic Press, 1996.

98