Introduction to the Numerical Solution of IVP for ODE 41

Introduction to the Numerical Solution of IVP for ODE

Consider the IVP: DE x′ = f (t, x), IC x(a) = xa. For simplicity, we will assume here that
x(t) ∈ Rn (so F = R), and that f (t, x) is continuous in t, x and uniformly Lipschitz in x
(with Lipschitz constant L) on [a, b] ×Rn. So we have global existence and uniqueness for
the IVP on [a, b].
Moreover, the solution of the IVP x′ = f (t, x), x(a) = xa depends continuously on the
initial values xa ∈ Rn. This IVP is an example of a well-posed problem: for each choice of
the “data” (here, the initial values xa), we have:

(1) Existence. There exists a solution of the IVP on [a, b].

(2) Uniqueness. The solution, for each given xa, is unique.

(3) Continuous Dependence. The solution depends continuously on the data.

Here, e.g., the map xa'→ x(t, xa) is continuous from Rn into (C ([a, b]) ,· ∞ ). A well-posed
problem is a reasonable problem to approximate numerically.

Grid Functions
Choo se a mesh width h (with 0 < h ≤ b − a), and let
h h¸x`˛ N = b—h a (greatest integer ≤ (b − a)/h). Let ti = a +
¸x`˛
(i = 0, 1, . .ih. , N) be the grid points in t (note: t0 = a),
t and let xi denote the approximation to x(ti). Note
a t1 t2 · · · tN
t0 that ti and xi depend on h, but we will usually
suppress this dependence in our notation.
b

Explicit One-Step Methods

Form of method: start with x0 (presumably x0 ≈ xa). Recursively compute x1, . . . , xN by

xi+1 = xi + hψ(h, ti, xi), i = 0, . . . , N − 1.

Here, ψ(h, t, x) is a function defined for 0 ≤ h ≤ b− a, a ≤ t ≤ b, x Rn, and ψ is associated

with the given function f (t, x). ∈

Examples.
Euler’s Method. tangent line
x
xi+1 .
xi+1 = xi + hf (ti, .
........
.............
.
xi) ..................
...

solution through (ti,x9)

xi
Here, ψ(h, t, x) = f (t,
x).
t
ti ti+1
 42 Ordinary Differential Equations

Taylor Methods.
Let p be an integer ≥ 1. To see how the Taylor Method of order p is constructed, consider
the Taylor expansion of a Cp+1 solution x(t) of x′ = f (t, x):
p
x(t + h) = x(t) + hx′(t) + · · · h
+ x(p) (t) + ` O(h
p+1
)
˛ ¸ x
p!
rem ain d er term

where the remainder term is O(hp+1) by Taylor’s Theorem with remainder. In the ap-
proximation, we will neglect the remainder term, and use the DE x′ = f (t, x) to replace
x′(t), x′′(t), . . . by expressions involving f and its derivatives:

x′(t) = f (t, x(t))

(n×1) (n×n) (n×1)
′′
d dx
x (t) = (f (t, x(t))) = Dt f + Dx f
dt dt

= (Dtf + (Dxf )f ) (t,x(t)) (for(t,x(t))

n = 1, this is ft + fxf ).
(t,x(t))

For higher derivatives, inductively differentiate the expression for the previous derivative,
and replace any occurrence of dtdx by f (t, x(t)). These expansions lead us to define the Taylor
methods of order p:

p=1: xi+1 = xi + hf (ti, xi) (Euler’s method, ψ(h, t, x) = f (t, x))

p=2: xi+1 = xi + hf (ti, xi) h2 (Dtf + (Dxf )f )

+ 2 (t ,x ) i i

For the case p = 2, we have

h
ψ(h, t, x) = T (h, t, x) ≡ f+ 2 (D f + (D f )f )2 .t x
(t,x)

We will use the notation Tp(h, t, x) to denote the ψ(h, t, x) function for the Taylor method
of order p.

Remark. Taylor methods of order ≥2 are rarely used computationally. They require deriva-
tives of f to be programmed and evaluated. They are, however, of theoretical interest in
determining the order of a method.

Remark. A “one-step method” is actually an association of a function ψ(h, t, x) (defined for

0 ≤ h ≤ b − a, a ≤ t ≤ b, x ∈ Rn) to each function f (t, x) (which is continuous in t, x and
Lipschitz in x on [a, b]×Rn). We study “methods” looking at one function f at a time.
Many methods (e.g., Taylor methods of order p ≥ 2) require more smoothness of f , either
for their
Introduction to the Numerical Solution of IVP for ODE 43

definition, or to guarantee that the solution x(t) is sufficiently smooth. Recall that∈ if f
Cp (in t and x), then the solution x(t) of the IVP x′ = f (t, x), x(a) = xa is in Cp+1([a,
b]). For “higher-order” methods, this smoothness is essential in getting the error to be
higher order in h. We will assume from here on (usually tacitly) that f is sufficiently
smooth when needed.

Examples.
Modified Euler’s Method
xi+1 = xi + hf ti + h , xi + h f (ti, xi)
2 2

2 2
(so ψ(h, t, x) = f t + h , x + h f (t, x) ).

Here ψ(h, t, x) tries to approximate

2 2 2
x′ t + h
= f t + h, x t + h
,

2 2
using the Euler approximation to x t + h ≈ x(t) + h f (t, x(t)) .

Improved Euler’s Method (or Heun’s Method)

xi+1 = xi + h2 (f (ti, xi) + f (ti+1, xi + hf (ti, xi)))

(so ψ(h, t, x) = 12 (f (t, x) + f (t + h, x + hf (t, x)))).

Here again ψ(h, t, x) tries to approximate

h2 2
x′ t + ≈ 1 (x′(t) + x′(t + h)).

Or ψ(h, t, x) can be viewed as an approximation to the trapezoid rule applied to

1 ∫ t+h
(x(t + h) − x(t)) 1 x′ ≈ 1 x′(t) + 1 x′(t + h).
=
2 2
h h t

Modified Euler and Improved Euler are examples of 2nd order two-stage Runge-
Kutta methods. Notice that no derivatives of f need be evaluated, but f needs to be
evaluated twice in each step (from xi to xi+1).
Before stating the convergence theorem, we introduce the concept of accuracy.

Local Truncation Error

Let xi+1 = xi + hψ(h, ti, xi) be a one-step method, and let x(t) be a solution of the DE
x′ = f (t, x). The local truncation error (LTE) for x(t) is defined to be

l(h, t) ≡ x(t + h) − (x(t) + hψ(h, t, x(t))),

44 Ordinary Differential Equations

that is, the local truncation error is the amount by which the true solution of the DE fails to
satisfy the numerical scheme. l(h, t) is defined for those (h, t) for which 0 < h ≤ b − a and
a ≤ t ≤b −h.
Define
l(h, t)
τ (h, t)
= h
and set τi(h) = τ (h, ti). Also,
set
τ (h) = max |τ (h, t)| for 0 < h ≤ b − a.
a≤t≤b− h

Note that
l(h, ti) = x(ti+1) − (x(ti) + hψ(h, ti, x(ti))),
explicitly showing the dependence of l on h, ti, and x(t).
Definition. A one-step method is called [formally] accurate of order p (for a positive integer
p) if for any solution x(t) of the DE x′ = f (t, x) which is Cp+1, we have l(h, t) = O(hp+1).

Definition. A one-step method is called consistent if ψ(0, t, x) = f (t, x). Consistency is

essentially minimal accuracy:

Proposition. A one-step method

xi+1 = xi + hψ(h, ti, xi),
where ψ(h, t, x) is continuous for 0 ≤ h ≤ h0, a ≤ t ≤ b, x ∈ Rn for some h0 ∈ (0, b − a], is
consistent with the DE x′ = f (t, x) if and only if τ (h) → 0 as h → 0+.

Proof. Suppose the method is consistent. Fix a solution x(t). For 0 < h ≤ h0, let
Z(h) = max |ψ(0, s, x(s)) − ψ(h, t, x(t))|.
a≤s,t≤b, |s− t|≤h

By uniform continuity, Z(h) → 0 as h → 0+. Now

l(h, t) = x(t + h) − x(t) − hψ(h, t, x(t))
∫ t+h
= [x′(s) − ψ(h, t, x(t))] ds
t
∫ t+h
= [f (s, x(s)) − ψ(h, t, x(t))] ds
t
∫ t+h
= [ψ(0, s, x(s)) − ψ(h, t, x(t))] ds,
t

so |l(h, t)| ≤ hZ(h). Therefore τ (h) ≤ Z(h) → 0.

Conversely, suppose τ (h) → 0. For any t ∈ [a, b) and any h ∈ (0, b − t],
x(t + h) − x(t)
= ψ(h, t, x(t)) + τ (h,
t). h
Taking the limit as h ↓ 0 gives f (t, x(t)) = x′(t) = ψ(0, t, x(t)).
Introduction to the Numerical Solution of IVP for ODE 45

Convergence Theorem for One-Step Methods

Theorem. Suppose f (t, x) is continuous in t, x and uniformly Lipschitz in x on [a, b]× Rn.
Let x(t) be the solution of the IVP x′ = f (t, x), x(a) = xa on [a, b]. Suppose that the
function ψ(h, t, x) in the one step method satisfies the following two conditions:
1. (Stability) ψ(h, t, x) is continuous in h, t, x and uniformly Lipschitz in x (with
Lipschitz constant K) on 0 ≤ h ≤ h0, a ≤ t ≤ b, x ∈ Rn for some h0 > 0 with h0 ≤ b
− a, and
2. (Consistency) ψ(0, t, x) = f (t, x).
Let ei(h) = x(ti(h)) − xi(h), where xi is obtained from the one-step method xi+1 = xi +
hψ(h, ti, xi). (Note that e0(h) = xa − x0(h) is the error in the initial value x0(h).) Then
K(t (h)−a) eK(t (h)−a) − 1
i

|ei(h)| ≤ e i
|e0(h)| + τ (h) ,
K
so
—
|ei(h)| ≤ eK(b−a) K(b−a)
|e0(h)| + e 1 τ (h).
K
Moreover, τ (h) → 0 as h → 0. Therefore, if e0(h) → 0 as h → 0, then
max
0≤i≤h |ei(h)| → 0 as h → 0,
b−a

that is, the approximations converge uniformly on the grid to the solution.
Proof. Hold h > 0 fixed, and ignore rounding error. Subtracting
xi+1 = xi + hψ(h, ti, xi)
from
x(ti+1) = x(ti) + hψ(h, ti, x(ti)) + hτi,
gives

|ei+1| ≤ |ei| + h|ψ(h, ti, x(ti)) − ψ(h, ti, xi)| + h|τi|

≤ |ei| + hK|ei| + hτ (h)
= (1 + hK)|ei| + hτ (h).
So

|e1| ≤ (1 + hK)|e0| + hτ (h), and

|e2| ≤ (1 + hK)|e1| + hτ (h)
≤ (1 + hK) 2|e0| + hτ (h)(1 + (1 + hK)).
By induction,
|ei| ≤ (1 + hK)i|e0| + hτ (h)(1 + (1 + hK) + (1 + hK)2 + · · · + (1 + hK)i−1)
i (1 + hK)i − 1
= (1 + hK) |e0| + hτ (h)
(1 + hK) − 1
= (1 + hK) |e0| + τ
i
(1 + hK)i −
(h) 1
K
50 Ordinary Differential Equations
1
Since (1 + hK) ↑ eK as h → 0+ (for K > 0), and i = ti−a
, we have
h
h
ti−a
i K(t −a)
(1 + hK) = (1 + h ≤e i
.
Thus hK)
K(t −a)
K(ti−a)
|ei| ≤ e i
|e0| + τ (h) e −1 .
K
The preceding proposition shows τ (h) → 0, and the theorem follows.
If f is sufficiently smooth, then we know that x(t)∈ Cp+1. The theorem thus implies
that if a one-step method is accurate of order p and stable [i.e. ψ is Lipschitz in x], then for
sufficiently smooth f ,
l(h, t) = O(hp+1) and thus τ (h) = O(hp).
If, in addition, e0(h) = O(hp), then
max |ei(h)| = O(hp),
i
th
i.e. we have p order convergence of the numerical approximations to the solution.
Example. The “Taylor method of order p” is accurate of order p. If∈fCp, then x ∈ Cp+1,
and
h ∫
l(h, t) = x(t + h) − x(t) + hx′(t) + · · · + p x(p) t+h
=1 (t + h − s)px(p+1)(s)ds.
(t)
p! p! t
So
(p+1)
hp+1 where Mp+1 = max |x
|l(h, t)| ≤ Mp+1 (t)|.
(p + a≤t≤b

1)!

Fact. A one-step method xi+1 = xi + hψ(h, ti, xi) is accurate of order p if and only if
ψ(h, t, x) = Tp(h, t, x) + O(hp),
where Tp is the “ψ” for the Taylor method of order p.
Proof. Since
x(t + h) − x(t) = hTp(h, t, x(t)) + O(hp+1),
we have for any given one-step method that
l(h, t) = x(t + h) − x(t) − hψ(h, t, x(t))
= hTp(h, t, x(t)) + O(hp+1) − hψ(h, t, x(t))
= h(Tp(h, t, x(t)) − ψ(h, t, x(t))) + O(hp+1).
So l(h, t) = O(hp+1) iff h(Tp − ψ) = O(hp+1) iff ψ = Tp + O(hp).
Remark. The controlled growth of the effect of the local truncation error (LTE) from
previous steps in the proof of the convergence theorem (a consequence of the Lipschitz
continuity of ψ in x) is called stability. The theorem states:
Stability + Consistency (minimal accuracy) ⇒ Convergence.
In fact, here, the converse is also true.
Introduction to the Numerical Solution of IVP for ODE 51

Explicit Runge-Kutta methods

One of the problems with Taylor methods is the need to evaluate higher derivatives of
f . Runge-Kutta (RK) methods replace this with the much more reasonable need to evaluate f
more than once to go from xi to xi+1. An m-stage (explicit) RK method is of the form

xi+1 = xi + hψ(h, ti, xi),

with
Σ
m
ψ(h, t, x) = ajkj(h, t, x),
j=1

where a1, . . . , am are given constants,

k1(h, t, x) = f (t, x)

and for 2 ≤ j ≤
m,
Σ
j−1
kj(h, t, x) = f (t + αjh, x + h βjrkr(h, t, x))
r=1

≤ r < j ≤ m) given constants. We usually choose 0 < α

with α2, . . . , αm and βjr (1 ≤j 1,
and for accuracy reasons we choose

Σ
j−1
(*) αj = βjr (2 ≤ j ≤ m).
r=1

Example. m = 2
xi+1 = xi + h(a1k1(h, ti, xi) + a2k2(h, ti, xi))
where

k1(h, ti, xi) = f (ti, xi)

k2(h, ti, xi) = f (ti + α2h, xi + hβ21k1(h, ti, xi)).

For simplicity, write α for α2 and β for β2. Expanding in h,

k2(h, t, x) = f (t + αh, x + hβf (t, x))

= f (t, x) + αhDtf (t, x) + (Dxf (t, x))(hβf (t, x)) + O(h2)
= [f + h(αDtf + β(Dxf )f )] (t, x) + O(h2).

So
ψ(h, t, x) = (a1 + a2)f + h(a2αDtf + a2β(Dxf )f ) +
Recalling that O(h2).

h
T2 = f + (Dtf + (Dxf )f ),
2
52 Ordinary Differential Equations

and that the method is accurate of order two if and only if

ψ = T2 + O(h2),

we obtain the following necessary and sufficient conditions on a two-stage (explicit) RK

method to be accurate of order two:
a1 + a2 = 1, a2 α = 1 , and a2β = 1 .
2 2

We require α = β as in (*) (we now see why this condition needs to be imposed), whereupon
these conditions become:
a1 + a2 = 1, a2α = 12 .

Therefore, there is a one-parameter family (e.g., parameterized by α) of 2nd order, two-stage

(m = 2) explicit RK methods.

Examples.
(1) Setting α = 2 gives a2 = 1, a1 = 0, which is the Modified Euler method.
1

(2) Choosing α = 1 gives a2 = 1 , a1 = 1 , which is the Improved Euler method, or Heun’s

2 2
method.

Remark. Note that an m-stage explicit RK method requires m function evaluations (i.e.,
evaluations of f ) in each step (xi to xi+1).

Attainable Orders of Accuracy for Explicit RK methods

# of stages (m) highest order attainable

1 1 ← Euler’s method
2 2
3 3
4 4
5 4
6 5
7 6
8 7

Explicit RK methods are always stable: ψ inherits its Lipschitz continuity from f .
Example.
Modified Euler. Let L be the Lipschitz constant for f , and suppose h ≤ h0 (for some
h0 ≤ b − a).

2 2
xi+1 = xi + hf ti + h , xi + h f (ti, xi)
2 2
ψ(t, h, x) = f t + h , x + h f (t, x)
Introduction to the Numerical Solution of IVP for ODE 53

So
2 2
h
≤ L|x − y| + 2 L|f (t, x) − f (t, y)|
h 2
|ψ(h, t, x) − ψ(h, t, y)| ≤ L x + f (t, x) − y + h f (t, y)
h

≤ L|x − y| + 2 L |x − y|
≤ K|x − y|
where K = L + h02L2 is thus the Lipschitz constant for ψ.

Example. A popular 4th order four-stage RK method is

h
xi+1 = xi + (k1 + 2k2 + 2k3 + k4)
6
where

k1 = f (ti, xi)
2 2
k2 = f ti + h , xi + hk1
k3 = f ti + 2h , xi + 2hk2
k4 = f (ti + h, xi +
hk3).
The same argument as above shows this method is stable.

Remark. RK methods require multiple function evaluations per step (going from xi to
xi+1). One-step methods discard information from previous steps (e.g., xi−1 is not used to
get xi+1
— except in its influence on xi). We will next study a class of multi-step methods. But first,
we consider linear difference equations.

Linear Difference Equations (Constant Coefficients)

In this discussion, xi will be a (scalar) sequence defined for i ≥ 0. Consider the linear
difference equation (k-step)

(LDE) xi+k + αk−1xi+k−1 + · · · + α0xi = bi (i ≥ 0).

If bi ≡ 0, the linear difference equation (LDE) is said to be homogeneous, in which case

we will refer to it as (lh). If/bi = 0 for some
≥ i 0, the linear difference equation (LDE) is
said to be inhomogeneous , in which case we refer to it as (li).
Initial Value Problem (IVP): Given xi for i = 0, . . . , k −1, determine xi satisfying (LDE)
for i ≥ 0.

Theorem. There exists a unique solution of (IVP) for (lh) or (li).

Proof. An obvious induction on i. The equation for i = 0 determines xk, etc.

Theorem. The solution set of (lh) is a k-dimensional vector space (a subspace of the set
of all sequences {xi}i≥0).
54 Ordinary Differential Equations

Proof Sketch. Choosing

 
x0

 x1 = e ∈ Rk
  j
xk−1 
 .

for j = 1, 2, . . . k and then solving (lh) gives a basis of the solution space of (lh).
Define the characteristic polynomial of (lh) to be

p(r) = rk + αk−1rk−1 + · · · + α0.

Let us assume that α0 /= 0. (If α0 = 0, (LDE) isn’t really a k-step difference equation
since we can shift indices and treat it˜ as a k-step difference equation˜for a k < k, namely
˜ k=k
−ν, where ν is the smallest index with αν /= 0.) Let r1, . . . , rs be the distinct zeroes of p,
with multiplicities m1, . . . , ms. Note that each rj /= 0 since α0 /= 0, and m1 + · · · + ms =
k. Then a basis of solutions of (lh) is:
}
{ilrij}∞i=0 : 1 ≤ j ≤ s, 0 ≤ l ≤ mj − 1 .

Example. Fibonacci Sequence:

Fi+2 − Fi+1 − Fi = 0, F0 = 0, F1 = 1.

The associated characteristic polynomial r2 − r − 1 = 0 has roots

√ 1±
r 5=
±
(r ≈ 1.6, r ≈ −0.6).
+ −
2
The general solution of (lh) is

√ !i √ !i
Fi = C+ 1 + 5 + C− 1− 5
.
2 2

The initial conditions F0 = 0 and F1 = 1 imply that C+ = and C− = − √1 . Hence

5
√1
 
√ !i √ !i 5

1 1+ 5 1− 5

Fi = √  − .
5 2 2
Since |r−| < 1, we
have √ !i
1− 5
→ 0 as i → ∞.
2
√ i
1 1+ 5
Hence, the Fibonacci sequence behaves asymptotically like the sequence √ 2
.
5
Introduction to the Numerical Solution of IVP for ODE 55

Remark. If α0 = α1 = · · · = αν−1 = 0 and αν /= 0 (i.e., 0 is a root of multiplicity ν),

then x0, x1, . . . , xν−1 are completely independent of xi for i ≥ ν. So xi+k + · · · + ανxi+ν =
bi for i ≥ 0 with xi given for i ≥ ν behaves like a (k − ν)-step difference equation.
 
xi 
Remark.
x Define xi+1 . Then x˜ Ax˜ for i ≥ 0, where
=
˜ =
i 

i+1 i
 .
xi+k−1 

0 1
A= .. .. ,
.
 .

  0 1 
x0 −α0 ... −αk−1

and x˜0 =
 is given by the I.C. So (lh) is equivalent to the one-step vector difference
 x1 
 .
xk−1
equation
x˜i+1 = Ax˜i ,i ≥ 0,
i
whose solution is x˜ i = A˜x . The characteristic polynomial of (lh) is the characteristic
0
polynomial of A. Because A is a companion matrix, each distinct eigenvalue has only
one Jordan block. If A = PJP −1 is the Jordan decomposition of A (J in Jordan form, P
invertible), then i −1
˜xi = PJ P x˜ 0 .
Let Jj be the mj × mj block corresponding to rj (for 1 ≤ j ≤ s), so Jj = rjI + Zj, where Zj
denotes the mj × mj shift matrix:
 
0 1
 ... ..

Z =
j
. 0  .

Then
J i = (rj I + Zj )i = i−l l
Σ i r Z.
i

j j j
l=0
l

i j i of the form
Since l is a polynomial in i of degree l and Zmj = 0, we see entries in
x˜
(constant) ilri
j
for 0 ≤ l ≤ mj − 1.
 56 Ordinary Differential Equations

Remark. (li) becomes

x˜i+1 = Ax˜ i + ˜b i ,i ≥ 0,
Introduction to the Numerical Solution of IVP for ODE 57

where˜bi = [0, . . . , 0, bi]T . This leads to a discrete version of Duhamel’s principle (exercise).

Remark. All solutions {xi}i≥0 of (lh) stay bounded (i.e. are elements of l∞)
⇔ the matrix A is power bounded (i.e., ∃ M so that Ai ≤ M for all i ≥ 0)
⇔ the Jordan blocks J1, . . . , Js are all power bounded

(a) each |rj| ≤ 1 (for 1 ≤ j ≤ s)

and (b) for any j with mj > 1 (multiple roots), |rj| < 1 .
⇔
If (a) and (b) are satisfied, then the map x˜0 '→ {xi}i≥0 is a bounded linear operator from Rk
(or Ck) into l∞ (exercise).

Linear Multistep Methods (LMM)

A LMM is a method of the form
k k
Σ Σ
αjxi+j = h βjfi+j, i ≥ 0
j=0 j=0

for the approximate solution of an ODE IVP

x′ = f (t, x), x(a) = xa .

Here we want to approximate the solution x(t) of this IVP for a ≤ t ≤ b at the points
ti = a + ih (where h is the time step), 0 h ≤ i ≤ b−a . The term xi denotes the
approximation to x(ti). We have set fi+j = f (ti+j, xi+j). We normalize the coefficients so
that αk = 1. The above is called a k-step LMM (if at least one of the coefficients α0 and
β0 is non-zero). The above equation is similar to a difference equation in that one is
solving for xi+k from xi, xi+1, . . . , xi+k−1. We assume as usual that f is continuous in (t, x)
and uniformly Lipschitz in x. For simplicity of notation, we will assume that x(t) is real
and scalar; the analysis that follows applies to x(t) ∈ Rn or x(t) ∈ Cn (viewed as R2n for
differentiability) with minor adjustments.
Example. (Midpoint rule) (explicit)
∫ ti+2
x(ti+2) − x(ti) = t x′(s)ds ≈ 2hx′(ti+1) = 2hf (ti+1, x(ti+1)).
i

This approximate relationship suggests the LMM

Midpoint rule: xi+2 − xi = 2hfi+1 .

Example. (Trapezoid rule) (implicit)

The approximation
∫ ti+1
h
x(ti+1) − x(ti) = x′(s)ds ≈ (x′(ti+1 ) + x′(ti ))
ti 2
 58 Ordinary Differential Equations

suggests the LMM

h
Trapezoid rule: x i+1 — xi = (f i+1 + fi ) .
2

Explicit vs Implicit.
If βk = 0, the LMM is called explicit : once we know xi, xi+1, . . . , xi+k−1, we compute
imme- diately
Σ
k−1
xi+k = (hβjfi+j − αjxi+j ) .
j=0

On the other hand, if βk /= 0, the LMM is called implicit : knowing xi, xi+1, . . . , xi+k−1, we
need to solve
Σ
k−1
xi+k = hβkf (ti+k, xi+k) − (αjxi+j − hβjfi+j)
j=0

for xi+k.
Remark. If h is sufficiently small, implicit LMM methods also have unique solutions given h
and x0, x1, . . . , xk−1. To see this, let L be the Lipschitz constant for f . Given xi, . . . , xi+k−1,
the value for xi+k is obtained by solving the equation

xi+k = hβkf (ti+k, xi+k) + gi,

where
Σ
k−1
gi = (hβjfi+j − αjxi+j)
j=0

is a constant as far as xi+k is concerned. That is, we are looking for a fixed point of

Φ(x) = hβkf (ti+k, x) + gi .

Note that if h|βk|L < 1, then Φ is a contraction:

|Φ(x) − Φ(y)| ≤ h|βk| |f (ti+k, x) − f (ti+k, y)| ≤ h|βk|L|x − y|.

So by the Contraction Mapping Fixed Point Theorem, Φ has a unique fixed point. Any initial
guess for xi+k leads to a sequence converging to the fixed point using functional iteration
(l+1)
x(l) = hβkf (ti+k, ) + gi
x
i+k i+k

which is initiated at some initial point x(0)i+k. In practice, one chooses either

(1) iterate to convergence, or

(2) a fixed number of iterations, using an explicit method to get the initial guess x(0)
i+k.
This pairing is often called a Predictor-Corrector Method.
Introduction to the Numerical Solution of IVP for ODE 59

Function Evaluations. One FE means evaluating f once.

Explicit LMM: 1 FE per step (after initial start)
Implicit LMM: ? FEs per step if iterate to convergence
usually 2 FE per step for a Predictor-Corrector Method.
Initial Values. To start a k-step LMM, we need x0, x1, . . . , xk−1. We usually take x0 = xa,
and approximate x1, . . . , xk−1 using a one-step method (e.g., a Runge-Kutta method).
Local Truncation Error. For a true solution x(t) to x′ = f (t, x), define the LTE to be
Σk k
Σ
l(h, t) = αjx(t + jh) − h βjx′(t + jh).
j=0 j=0

If x ∈ Cp+1, then
′ (jh)p (p) p+1
x(t + jh) = x(t) + jhx (t) + · · · x (t) + ) and
+ O(h
p!
′ ′ 2 ′′
jp−1hp (p) p+1
hx (t + jh) = hx (t) + jh x (t) + · · · + (t) + O(h )
x (p − 1)!
and so
l(h, t) = C0x(t) + C1hx′(t) + · · · + Cphpx(p)(t) + O(hp+1),
where

C0 = α0 + · · · + αk
C1 = (α1 + 2α2 + · · · + kαk) − (β0 + · · · + βk)
.
C = 1 + 2qα +q · · · + )− 1 (β + 2q−1β + · · · + kq−1β ).
(α kα k 1 2 k
q
q!
1 2 (q − 1)!

Definition. A LMM is called accurate of order p if l(h, t) = O(hp+1) for any solution of
x′ = f (t, x) which is Cp+1.

Fact. A LMM is accurate of order at least p iff C0 = C1 = · · · = Cp = 0. (It is called

accurate of order exactly p if also Cp+1 /= 0.)

Remarks.

(i) For the LTE of a method to be o(h) for all f ’s, we must have C0 = C1 = 0. To see
this, for any f which is C1, all solutions x(t) are C 2 , so

l(h, t) = C0x(t) + C1hx′(t) + O(h2) is o(h) iff C0 = C1 = 0 .

(ii) Note that C0, C1, . . . depend only on α0, . . . , αk, β0, . . . , βk and not on f . So
here, “minimal accuracy” is first order.
 51 Ordinary Differential Equations
0
Definition. A LMM is called consistent if C0 = C1 = 0 (i.e., at least first-order accurate).

Remark. If a LMM is consistent, then any solution x(t) for any f (continuous in (t, x),
Lipschitz in x) has l(h, t) = o(h). To see this, note that since x ∈ C1,

x(t + jh) = x(t) + jhx′(t) + o(h) and hx′(t + jh) = hx′(t) + o(h),

so
l(h, t) = C0x(t) + C1hx′(t) + o(h).
Exercise: Verify that the o(h) terms converge to 0 uniformly in t (after dividing by h) as
h → 0: use the uniform continuity of x′(t) on [a, b].

Definition. A k-step LMM Σ Σ

αjxi+j = h βjfi+j

is called convergent if for each IVP x′ = f (t, x), x(a) = xa on [a, b] (f ∈ (C, Lip)) and
for any choice of x0(h), . . . , xk−1(h) for which

lim |x(ti(h)) − xi(h)| = 0 for i = 0, . . . , k − 1,

h→0

we have
lim max |x(ti(h)) − xi(h)| = 0 .
h→0 {i:a≤ti(h)≤b}

Remarks.

(i) This asks for uniform decrease of the error on the grid as h → 0.

(ii) By continuity of x(t), the condition on the initial values is equivalent to xi(h) → xa
for i = 0, 1, . . . , k − 1.

Fact. If a LMM is convergent, then the zeroes of the (first) characteristic polynomial of the
method p(r) = αkrk + · · · + α0 satisfy the Dahlquist root condition:

(a) all zeroes r satisfy |r| ≤ 1, and

(b) multiple zeroes r satisfy |r| < 1.

Examples. Consider the IVP x′ = 0, a ≤ t ≤ b, x(a) = 0. So f ≡ 0. Consider the LMM:

Σ
αjxi+j = 0 .
Introduction to the Numerical Solution of IVP for ODE 61

(1) Let r be any zero of p(r). Then the solution with initial conditions

xi = hri for 0 ≤ i ≤ k − 1
is
b−a
xi = hri for 0 ≤ i ≤ .
h
Suppose h = b−a for some m ∈ Z. If the LMM is convergent, then
m

xm(h) → x(b) = 0
as m → ∞. But
b−a
xm(h) = hr = m r m.
m
So
b−a
|xm (h) − x(b)| = |rm| → 0 as m→∞
m
iff |r| ≤ 1.
(2) Similarly if r is a multiple zero of p(r), taking xi(h) = hiri for 0 ≤ i ≤ k − 1 gives
b−a
x (h) = hiri, 0 ≤ i ≤ .
i
h
b−a
So if h = , then
m b−a
x (h) = mrm = (b — a)rm,
m
m
so xm(h) → 0 as h → 0 iff |r| < 1.

Definition. A LMM is called zero-stable if it satisfies the Dahlquist root condition.

Recall from our discussion of linear difference equations that zero-stability is equivalent to
Σk
requiring that all solutions of (lh) j=0
αjxi+j = 0 for i ≥ 0 stay bounded as i → ∞.
Remark. A consistent one-step LMM (i.e., k = 1) is always zero-stable. Indeed,
consistency implies that C0 = C1 = 0, which in turn implies that p(1) = α0 +α1 = C0 = 0
and so r = 1 is the zero of p(r). Therefore α1 = 1, α0 = −1, so the characteristic
polynomial is p(r) = r − 1, and the LMM is zero-stable.

Exercise: Show that if an LMM is convergent, then it is consistent.

Key Theorem. [LMM Convergence]
A LMM is convergent if and only if it is zero-stable and consistent. Moreover, for zero-stable
methods, we get an error estimate of the form
maxi |l(h, ti(h))|
max |x(ti (h)) − x i(h)| ≤ K 1 ≤max
a≤ti(h)≤b i≤ k− 1 |
x(ti (h)) − xi (h) +K 2 h
` ˛¸ ` ˛¸
0 |
initia l error “growthxof error”
x controlled by
zero-stability
 60 Ordinary Differential Equations

Remark. If x ∈ Cp+1 and the LMM is accurate of order p, then| LTE /h = O(hp). To get
|
pth-order convergence (i.e., LHS = O(hp)), we need

xi(h) = x(ti(h)) + O(hp) for i = 0, . . . , k − 1.

This can be done using k − 1 steps of a RK method of order ≥ p −

1. Lemma. Consider
Σ
k
(li)
αjxi+j = bi for i ≥ 0 (where αk = 1),
j=0

with the initial values x0, . . . , xk−1 given, and suppose that the characteristic polynomial
Σk
p(r) = j=0 αjrj satisfies the Dahlquist root condition. Then there is an M > 0 such that
for i ≥ 0, !
Σ
i
|xi+k| ≤ M
max{|x0|, . . . , |xk−1|} + |bν| .
ν=0

Remark.
i
Recall that the Dahlquist root condition implies that there is an M > 0 for which
A ∞ ≤ M for all i ≥ 0, where
 
0 1
 .. .. 
. .
A= 
 0 1 
−α0 ··· −αk−1

is the companion matrix for p(r), and · ∞ is the operator norm induced by the vector
norm · ∞. The M in the Lemma can be taken to be the same as this M bounding Ai ∞.

Proof. Let x˜i = [xi, xi+1, . . . , xi+k−1]T and bi =˜ [0, . . . , 0, bi]T . Then x˜i+1 = Axi ˜+ bi, so
by induction ˜
Σi
˜ i+1
xi+1 = A x0 + ˜ i−ν
A bν.
˜ ν=0

Thus

|xi+k| ≤ x
˜i+1 ∞
i
x0 ∞ + Σ Ai−ν
i+1
≤ ∞ ˜b ν
˜ ν=0
∞ ∞

≤ M( i
x0 ∞ +
Σ |bν|).
˜ ν=0
Introduction to the Numerical Solution of IVP for ODE 63

Proof of the LMM Convergence Theorem. The fact that convergence implies zero-
stability and consistency has already been discussed. Suppose a LMM is zero-stable
and consistent. Let x(t) be the true solution of the IVP x′ = f (t, x), x(a) = xa on [a, b], let
L be the Lipschitz constant for f , and set
Σ
k
β= |βj|.
j=0

Hold h fixed, and set

ei(h) = x(ti(h)) − xi(h), E = max{|e0|, . . . , |ek−1|},

li(h) = l(h, ti(h)), λ(h) = max| li(h) ,
i∈I
|
b−a
where I = {i ≥ 0 : i + k ≤ h }.
Step 1. The first step is to derive a “difference inequality” | for ei . This difference
| ∈I
inequality is a discrete form of the integral inequality leading to Gronwall’s inequality.
For i , we have
Σk Σk
αjx(ti+j) = h βjf (ti+j, x(ti+j)) + li
j=0 j=0
k
Σ Σk

αjxi+j = h βjfi+j.
j=0 j=0

Subtraction gives
Σ
k
αjei+j = bi,
j=0

where
Σ
k
bi ≡ h βj (f (ti+j, x(ti+j)) − f (ti+j, xi+j)) + li.
j=0

Then
Σ
k
|bi| ≤ h |βj|L|ei+j| + |li|.
j=0

So, by the preceeding Lemma with xi+k replaced by ei+k, we obtain for i ∈ I
" Σ #

|ei+k| ≤ E |bν|
i
M +
ν=0
" #
Σ
i k Σ
≤ M E + hL |βj||eν+j| + |lν|
Σ i

ν=0 j=0 ν=0

" #
i+k−1
Σ Σ
i
≤ E + hL|βk||ei+k| + hLβ |eν| + |lν| .
ν=0 ν=0
 62 Ordinary Differential Equations

From here on, assume h is small enough that

1
MhL|βk| ≤ .
} 2
(Since h ≤ b − a : MhL|βk| ≥ 1 2 is a compact subset of (0, b − a], the estimate in the Key
Theorem is clearly true for those values of h.) Moving MhL|βk||ei+k| to the LHS gives
i+k−1
Σ
|ei+k| ≤ hM1
|eν| + M2E + M3λ/h

ν=0

for i ∈ I, where M1 = 2MLβ, M2 = 2M , and M3 = 2M (b −a). (Note: For explicit methods,

βk = 0, so the restriction MhL|βk| ≤ 1 2is unnecessary, and the factors of 2 in M1, M2, M3
can be dropped.)
Step 2. We now use a discrete “comparison” argument to bound |ei|. Let yi be the solution
of

(∗) yi+k = hM1 i+k−1

Σ
yν + (M2E + M3λ/h) for i ∈ I,
ν=0

with initial values yj = |ej| for 0 ≤ j ≤ k − 1. Then clearly by induction |ei+k| ≤ yi+k for
i ∈ I. Now
yk ≤ hM1kE + (M2E + M3 λ/h) ≤ M4E + M3λ/h,
where M4 = (b − a)M1k + M2. Subtracting (∗) for i from (∗) for i + 1 gives
yi+k+1 − yi+k = hM1yi+k, and so yi+k+1 = (1 + hM1)yi+k.
Therefore, by induction we obtain for i ∈ I:
yi+k = (1 + hM1)iyk
(b−a)/h
≤ (1 + hM1) yk
≤ eM (b−a)yk
1

≤ K1E + K2λ/h,

where K1 = eM1(b−a)M4 and K2 = eM1(b−a)M3. Thus, for i ∈ I,

|ei+k| ≤ K1E + K2λ/h;
since K1 ≥ M4 ≥ M2 ≥ M ≥ 1, also |ej| ≤ E ≤ K1E + K2λ/h for 0 ≤ j ≤ k − 1. Since
consistency implies λ = o(h), we are done.
Remarks.
(1) Note that K1 and K2 depend only on a, b, L, k, the αj’s and βj’s, and M .
(2) The estimate can be refined — we did not try to get the best constants K1, K2. For
example, eM1(b−a) could be replaced by eM1(ti−a) in both K1 and K2, yielding more
precise estimates depending on i, similar to the estimate for one-step methods.