Chapter 2

Finite dierence schemes

The denition of a derivative given in (1.10),
d
dx
f(x
0
) = lim
x0
f(x
0
+ x) f(x
0
)
x
, (2.1)
could also have been given as
d
dx
f(x
0
) = lim
x0
f(x
0
) f(x
0
x)
x
, (2.2)
or as
d
dx
f(x
0
) = lim
x0
f(x
0
+ x) f(x
0
x)
2x
. (2.3)
Actually, all three denitions are equivalent for a continuously derivable function
f. These denitions are the basis for so-called nite-dierence schemes. For
the case that the function f is given on a discrete grid with the nite spatial
increment x, its derivative may be approximated by one of the methods (2.1) -
(2.3) by omitting the lim. These approximations will of course not be equivalent.
In the next sections, methods will be discussed with the aid of which the quality
of the dierent dierencing schemes will be quantied.
2.1 Accuracy and consistency
In order to test the accuracy of a scheme, the nite-dierence approximation has
to be compared to the approximated dierential. In the case of the rst-order
derivative from (2.1) - (2.3), we obtain by means of a Taylor expansion:
f(x
0
x) = f(x
0
) x
x
f(x
0
) +
x
2
2

xx
f(x
0
)
x
3
6

xxx
f(x
0
) +. . . (2.4)
By means of this, the approximation order of the approximation (2.1) may be
found:
f(x
0
+ x) f(x
0
)
x

x
f(x
0
) =
x
2

xx
f(x
0
) +
x
2
6

xxx
f(x
0
) + . . .
(2.5)
18
The right hand side of (2.5) is the so-called truncation error, and the low-
est power of x in the truncation error is the order of accuracy of the nite-
dierence approximation. Thus, the nite-dierence approximation (2.1) is of
rst order in x which may also be denoted saying that the scheme is O(x).
The same is found for method (2.2). For the central dierencing we nd how-
ever:
f(x
0
+ x) f(x
0
x)
2x

x
f(x
0
) =
x
2
6

xxx
f(x
0
) +
x
4
120

xxxxx
f(x
0
) + . . .
(2.6)
such that this scheme is O(x
2
).
Of course, a higher order of approximation does not guarantee a better ap-
proximation in general, since (i) the higher derivatives in the leading truncation
terms (e.g.
xx
f(x
0
) in (2.5)) may be large and (ii) the overall numerical scheme
might be numerically instable, as we have demonstrated in chapter 1.
Approximations of higher than second order may be constructed as well.
After having examined the accuracy of individual derivatives, we will now
analyse the accuracy of approximation for partial dierential equations. As a
basic test case we use the linear advection equation with constant velocity,

t
+ c
x
= 0 (2.7)
with c > 0, which is discretised on a grid equivalent to that described in assign-
ment 1. The advection equation is discretised by means of the so-called upwind
scheme

j+1
i

j
i
t
+ c

j
i

j
i1
x
= 0. (2.8)
The modied equation for this scheme is now formulated in the point (x
0
, t
0
):
(x
0
, t
0
+ t) (x
0
, t
0
)
t
+ c
(x
0
, t
0
) (x
0
x, t
0
)
x
= 0. (2.9)
The dierence between the modied equation and the dierential equation is
then built for checking the consistency of the scheme determining its order of
approximation:
(x
0
, t
0
+ t) (x
0
, t
0
)
t
+ c
(x
0
, t
0
) (x
0
x, t
0
)
x
(
t
(x
0
, t
0
) + c
x
(x
0
, t
0
))
=
t
2

tt
(x
0
, t
0
)
c
2
x
xx
(x
0
, t
0
)
+
t
2
6

ttt
(x
0
, t
0
) +
c
6
x
2

xxx
(x
0
, t
0
) + . . .
= O(t) +O(x).
(2.10)
19
Thus, the nite-dierence scheme (2.8) is consistent with the linear advection
equation (2.7) to rst order in time and rst order in space.
As shown in equations (1.25) and (1.26), a wave equation can be derived
from the advection equation, with the aid of which the modied equation (2.10)
may be reformulated:
(x
0
, t
0
+ t) (x
0
, t
0
)
t
+ c
(x
0
, t
0
) (x
0
x, t
0
)
x

t
(x
0
, t
0
) + c
x
(x
0
, t
0
)
c
2
x
_
1
t
x
c
_

xx

_
=
t
2
6

ttt
(x
0
, t
0
) +
c
6
x
2

xxx
(x
0
, t
0
) + . . . ,
(2.11)
such that the nite dierence approximation (2.8) is a second-order in time and
space approximation to the advection-diusion equation with the diusivity

num
=
c
2
x(1 ) . (2.12)
with the so-called Courant number
=
t
x
c. (2.13)
This diusivity caused by the numerical scheme is referred to as numerical
diusion. Obviously, for > 1, this advection-diusion is mathematically ill-
posed, since the viscosity becomes negative, a clear sign that the scheme will be
unstable for > 1. Interestingly, the numerical diusivity vanishes for = 1,
in which case the advection velocity equals the so-called grid velocity x/t,
such that all discrete values are simply shifted by one grid index per time step.
2.2 Stability and convergence
We have shown in the previous paragraphs how the order of consistency of an
approximation methods to a dierential equation can be proven. However, as
the short numerical calculation in equation (1.6) (where a consistent central
dierence approximation to the advection equation failed) has shown, consis-
tency is not sucient for convergence. Before we start to introduce methods
for checking the convergence of numerical schemes, it is necessary to introduce
quality measures for the numerical approximation. For calculating the absolute
size of a vector

, two norms are typically used, the maximum norm, dened as
_
_
_

_
_
_

= max
1iM
|
i
| , (2.14)
and the Euclidean norm, dened as
_
_
_

_
_
_
2
=
_
M

i=1

2
i
_
1/2
. (2.15)
20
It should be noted that (2.14) and (2.16) are generalisations of the n-norm
_
_
_

_
_
_
n
=
_
M

i=1
|
i
|
n
_
1/n
. (2.16)
A nite-dierence scheme
j
i
is said to be convergent of order (p, q) towards the
solution if in the limit of t, x 0,
_
_
_(jt, ix)
j
i
_
_
_ = O[(t)
p
] +O[(x)
q
] . (2.17)
The Lax equivalence theorem (Lax and Richtmyer [1956]) proves that
if a nite-dierence scheme is linear, stable, and accurate (i.e. consistent)
of order (p, q), then it is convergent of order (p, q).
Thus, in addition to consistency, stability of the scheme is required. An nu-
merical scheme will be classied as unstable, when the numerical solution grows
more rapidly than the physical solution. A stability criterium which satises
the Lax equivalence theorem is that for any time T there exists a constant C
T
such that
_
_

j
_
_
C
T
_
_

0
_
_
for all jt T, (2.18)
and all suciently small values of t and x. The constant C
T
may depend on
T, but not on t or x. However, this relatively weak constraint (2.18) may in
practical applications still fail, since it allows for quite some oscillations of the
numerical solution.
For many conservation laws for example, the norm of the physical solution
is constant with time, such that numerical solutions with this property will also
be stable:
_
_

j
_
_

_
_

0
_
_
for all j. (2.19)
The condition (2.19) is much stronger than (2.18) and is thus not necessary for
stability.
2.2.1 The energy method
One simple but not general method for estimating the stability of a numerical
scheme is the energy method. Here, the sum of all squares of the numerical
solution,

i
_

j
i
_
2
, (2.20)
will be calculated and its tendency in time will be evaluated. It should be noted
that for this stability method, periodic boundary conditions are required.
Let us again consider the rst-order upwind method for the linear advec-
tion equation. With the Courant number from (2.13), this scheme may be
formulated as

j+1
i
= (1 )
j
i
+
j
i1
. (2.21)
21
Squaring on both sides and summing over the spatial index i leads to

i
_

j+1
i
_
2
=

i
_
(1 )
2
_

j
i
_
2
+ 2(1 )
j
i

j
i1
+
2
_

j
i1
_
2
_
.
(2.22)
With

i
_

j
i
_
2
=

i
_

j
i1
_
2
(2.23)
(shifting by one index in a periodic domain) and

j
i

j
i1

_

j
i
_
1/2
_

j
i1
_
1/2
=

i
_

j
i
_
2
, (2.24)
(Schwartz inequality) we obtain

i
_

j+1
i
_
2

i
_
(1 )
2
+ 2(1 ) +
2
_
_

j
i
_
2
=

i
_

j
i
_
2
, (2.25)
such that the total energy of the numerical solution does not increase. The
condition for obtaining (2.25) is that
(1 ) 0. (2.26)
For 0, (2.26) is equivalent to 0 1, which means that the rst-
order upwind scheme is stable for this range. For negative , we obtain a
contradiction to (2.26), such that stability for the downwind scheme cannot be
proven (implying the well-known fact that the downwind scheme is unstable).
Therefore, also the instability of the central dierence scheme is shown, which is
exactly the average of the upwind and the downwind scheme. Thus the stability
criterium for the upwind scheme is
0 c
t
x
1, (2.27)
meaning that the physical velocity c must not exceed the grid velocity x/t.
The upwind scheme is thus called conditionally stable, whereas the downwind
and the central scheme are unconditionally unstable.
2.2.2 Von Neumanns method
One drawback of the energy method is that for each scheme to be considered,
a new strategy has to be found how to calculate the energy of the numerical
solution.
A more generic method for testing the stability of numerical approximation
schemes is the Von Neumann method, which is however also straight-forward
only for periodic boundary conditions. Furthermore it can only be applied to
22
linear schemes with constant coecients. For each time step, the numerical
solution is represented by a nite Fourier series of the form

n
j
=
N

k=N
a
n
k
e
ikjx
. (2.28)
Note that in comparison to this, each periodic function f(x, t) may be repre-
sented by a innite Fourier series of the form
f(x, t) =

k=
a
k
(t)e
ikx
. (2.29)
A key property of each Fourier mode in (2.29) is an eigenfunction of the deriva-
tive operator:
d
x
e
ikx
= ike
ikx
(2.30)
This property is used in the discrete space for studying the stability of numerical
schemes. After one iteration, the solution for a specic discrete Fourier mode

n
j
= e
ikjx
will be of the form:

n+1
j
= A
k
e
ikjx
(2.31)
with the complex so-called amplication factor A
k
. For linear schemes with
constant coecients, the amplication factor will be constant for each time
step, such we can write:
a
n
k
= A
k
a
n1
k
= (A
k
)
n
a
0
k
. (2.32)
For the stability of a numerical scheme, it will be required that each Fourier com-
ponent of (2.28) is bounded. The Von Neumann stability criterium is therefore
formulated as
|A
k
| 1 + t (2.33)
with being independent of k, t and x. (2.33) guarantees that a consistent
scheme converges for x, t 0. For problems with bounded solutions, (2.33)
may be limited to
|A
k
| 1. (2.34)
As an illustration, the linear advection problem with the rst-order upwind
scheme will be tested again. Expressing the solution at each point as a specic
Fourier component,
n
j
= e
ikjx
, and inserting into (2.21), we obtain
A
k
e
ikjx
= (1 )e
ikjx
+ e
ik(j1)x
, (2.35)
which gives after division by e
ikjx
A
k
= (1 ) + e
ikx
, (2.36)
23
and with the absolute value of A
k
being the square root of the product of A
k
with its complex conjugate,
|A
k
|
2
=
_
(1 ) + e
ikx
_ _
(1 ) + e
ikx
_
= 1 2(1 )(1 cos(kx)).
(2.37)
Thus, condition (2.34) is fullled and thus the numerical scheme is stable for
1 2(1 )(1 cos(kx)) 1. (2.38)
Since 1 cos(kx) 0, this reduces to
(1 ) 0, (2.39)
which is identical to (2.26), such that we have proven the same stability criterium
by means of the Von Neumann method.
2.2.3 Courant-Friedrichs-Lewy condition
In temporal-spatial numerical grid, for each point a domain of inuence and
a domain of dependence for a certain numerical scheme may be drawn. The
Courant-Friedrichs-Lewy (CFL) condition says that the discrete solution must
not be independent of data that determine the solution of the associated partial
dierential equation. Thus the physical domain of dependence as dened by the
partial dierential equation must be inside the numerical domain of dependence,
gure 2.1. With 1/c denoting the slope of the physical velocity in the x t
domain, this means
1
c

t
x
= c
t
x
1 (2.40)
which is the CFL condition for the rst-order upwind scheme. It should be noted
that the CFL criterium is a necessary, but not a sucient stability condition.
2.3 Time dierencing
There is a big variety of possible discretisations of the time derivative in partial
dierential equations. Thus, the temporal discretisation is here dealt with sep-
arately. For practical applications, there are of course a number of constraints
to be obeyed.
Higher order scheme are desirable, but they often require big storage
(which may be a problem in three-dimensional models) or require a high
number evaluation of complicated functions. Furthermore, spatial resolu-
tion is usually the reason for low accuracy of complex models (doubling
the spatial resolution usually means the increase of grid points by a factor
of 8 and the reduction of time step by a factor of 2, such that the extra
costs are a factor of 16 !).
24
Figure 2.1: Graph showing the physical and numerical domains of dependence
for the solution of an advection equation. In panel a, the physical domain of
dependence is not covered by the numerical domain of dependence, and thus the
numerical solution in unstable. In panel b, the time step is reduced such that
now the physical domain of dependence fully lies within the numerical domain
of dependence, such that the numerical solution is stable. The gure has been
adapted from Durran [1999].
Low order schemes are often known for strong damping or amplifying a
numerical solution, which may be a signicant problem in situations with
low physical damping.
Implicit time stepping schemes for three-dimensional models have the
problem that huge systems of linear equations have to be solved.
It is thus a matter of compromise between accuracy and computational costs,
which time stepping scheme to use.
Let as again consider the linear advection equation

t
+ c
x
= 0. (2.41)
The representation of the function as a Fourier series would be of the following
form:
(x, t) =

k=
b
k
(t) exp(ikx), (2.42)
such that (2.41) is represented as
d
t
b
k
= ikcb
k
. (2.43)
This equation will be used for examining amplitude and phase errors of temporal
discretisation methods.
25
(2.43) represents an oscillation equation of which the most general form is
d
t
= i, (2.44)
the analytical solution of which is
(t) =
0
exp(it). (2.45)
Integration of (2.44) over one time step yields thus
(t
0
+ t) = (t
0
) exp(it) = A
e
(t
0
), (2.46)
where A
e
is an exact amplication factor which is a complex number with the
absolute value 1. Thus, the solution moves per time step a distance of t
radians on a circle with constant amplitude.
Now, it is desirable that numerical solutions to (2.44) reproduce this be-
haviour of the solution to sucient accuracy. Let us dene a numerical complex
amplication factor A as

n+1
= A
n
. (2.47)
When rewriting A as
A = |A| exp(i) (2.48)
with
|A| =
_
R(A)
2
+I(A)
2
_
1/2
, = arctan
_
I(A)
R(A)
_
(2.49)
then it is clear that |A| is the amplitude error and R = /(t) is the phase
error of the numerical scheme.
For |A| > 1, the scheme would be amplifying, for |A| < 1 damping and for
|A| = 1 neutral. For R > 1, the scheme would be accelerating, and for R < 1,
it would be decelerating.
In the following, we will investigate several schemes for their amplitude and
phase error.
2.3.1 One-stage two level schemes
A general ordinary dierential equation may be written as
d
t
= F(), (2.50)
where and F may be vectors. The general form of one-stage two level schemes
is

n+1

n
t
= (1 )F(
n
) + F(
n+1
), (2.51)
or, alternatively,

n+1
=
n
+ (1 )tF(
n
) + tF(
n+1
), (2.52)
For = 0, we obtain the Euler forward scheme, for = 1 the Euler backward
scheme and for = 0.5 the trapezoidal scheme.
26
For F() = i, the order of approximation of (2.51) is estimated as usual:
(t
0
+ t) (t
0
)
t
(1 )i(t
0
) i(t
0
+ t)

(t
0
) + i(t
0
)
=
t
2
i

(1 2) +O(t
2
),
(2.53)
where

= ik

has been used. Thus, the scheme is of second order for = 0.5
and of rst order else.
Application of (2.52) to (2.44) results in

n+1
(1 ti) =
n
(1 + ti(1 )), (2.54)
such that the amplication factor is expressed as
A =

n+1

n
=
1 + ti(1 )
1 ti
, (2.55)
which results in
|A|
2
=
1 + t
2

2
(1 )
2
1 + t
2

2
= 1 + (1 2)
t
2

2
1 + t
2

2
.
(2.56)
Thus, the Euler forward scheme is amplifying, the Euler backward scheme is
damping and the trapezoidal method is neutral. Although the Euler forward
scheme is amplifying, we have for t 1
|A|
=0
= (1 +
2
t
2
)
1/2
1 +
2
t
2
1 +
2
t, (2.57)
such that the von Neumann stability criterium is fullled and the scheme is
stable.
With (2.48), the phase error may be estimated. When

A denotes the conju-
gate complex of A, then R(A) = 0.5(A +

A) and I(A) = 0.5i(A +

A). With
this, the phase error of (2.51) is
R =
1
t
arctan
_
t
1 (1 )
2
t
2
_
, (2.58)
such that for the Euler forward and the Euler backward we have
R
forward
= R
backward
=
arctant
t
. (2.59)
By expressing the arctan-function with its Taylor series,
arctanx = x
x
3
3
+
x
5
5
+ . . . , (2.60)
27
it is clear that 0 < R
forward
= R
backward
< 1, such that the schemes are both
decelerating. With (2.60), the approximate error is
R
forward
= R
backward
1
(t)
2
3
, (2.61)
such that the phase error is O(t
2
).
For the trapezoidal scheme, the exact phase error is
R
=0.5
=
1
t
arctan
_
t
1 0.25
2
t
2
_
, (2.62)
which is approximated for small t by mean of the Taylor series (2.60) and
1/(1 x) = (1 + x)/(1 x
2
) 1 + x (2.63)
as
R
=0.5

1
t
arctan
_
t
_
1 +

2
t
2
4
__
1

2
t
2
12
. (2.64)
2.3.2 Multi-stage two level schemes
In contrast to the one-stage schemes, the multi-stage schemes compute one or
more intermediate results which are used for increasing the order of approxima-
tion. As an example, two-stages two level schemes are generally of the following
form:

n+
=
n
+ tF (
n
) ,

n+1
=
n
+ tF
_

n+
_
+ (1 )tF (
n
) ,
(2.65)
with the free parameters and , and the intermediate solution

n+
.
Application of (2.65) to the oscillation equation (2.44) results in

i+1
=
_
1
2
t
2
+ it
_
, (2.66)
such that the amplication factor is
1 (t)
2
+ it, (2.67)
which results into
|A|
2
= 1 + (1 2)(t)
2
+
2

2
(t)
4
. (2.68)
With this it is clear, that high-order two-stage two level schemes are obtained
by setting
=
1
2
. (2.69)
28
The class of schemes fullling (2.69) is called the class of Runge-Kutta schemes,
examples of which are the Heun scheme ( = 1, =
1
2
) and the midpoint
method ( =
1
2
, = 1).
It is clear from (2.68) that all Runge-Kutta schemes are unstable (|A|
2
> 1),
but since the order is high, this is often acceptable for simulations of short
duration.
To cure this problem, often a non-Runge-Kutta is used with = 1 and = 1
(Matsuno scheme), for which the stability criterium may be derived from (2.68)
as
|A|
2
= 1 (t)
2
+ (t)
4
. (2.70)
Thus, the Matsuno scheme is stable for 0 t 1.
The relative phase change for the two-stage two level schemes is
R =
1
t
arctan
_
t
1 (t)
2
_
, (2.71)
such that for the Runge-Kutta schemes we obtain
R
RK2
1 +
1
6
(t)
2
, (2.72)
and for the Matsuno scheme
R
Mat
1 +
2
3
(t)
2
. (2.73)
2.3.3 Three level schemes
A further increase in temporal resolution may be obtained by three-level schemes,
for which numerical solutions on three time levels have to be stored. The explicit
form of such schemes is given as:

n+1
=
1

n
+
2

n1
+
1
tF (
n
) +
2
tF
_

n1
_
, (2.74)
with constant coecients
1
,
2
,
1
and
2
. A check for conditions for second-
order accuracy results in

1
= 1
2
,
1
=
1
2
(
2
+ 3),
2
=
1
2
(
2
1), (2.75)
with a free choice for
2
. Signicant simplications of (2.74) are obtained by the
following two choices: Leap-frag scheme for
2
= 1, such that
1
= 0,
1
= 2
and
2
= 0,

n+1

n1
2t
= F (
n
) , (2.76)
and the Adams-Bashforth scheme for
2
= 0, such that
1
= 1,
1
= 1.5 and

2
= 0.5,

n+1

n
t
=
3
2
F (
n
)
1
2
F
_

n1
_
. (2.77)
29
Application of the leap-frog scheme into the oscillation equation (2.44) leads to

n+1
=
n1
+ 2it
n
. (2.78)
Considering the constant amplication factor A with
n
= A
n1
and
n+1
=
A
2

n1
, we obtain teh following quadratic equation for A:
A
2
2it A1 = 0, (2.79)
with the solutions
A

= it
_
1
2
t
2
. (2.80)
For well-resolved wave numbers with t 0, we obtain A
+
1 and A

1, with A
+
denoting the physical mode and A

denoting the computational

mode, representing an articial numerical oscillation. For |t| 1, we obtain
|A
+
| = |A

| = 1, such that both modes are stable. However, for t > 1, we

obtain

A
2
+

i
_
t +
_

2
t
2
1
_

= t +
_

2
t
2
1 > t > 1, (2.81)
such that the physical mode is unstable. For high resolution, the relative phase
error of the physical mode can be approximated by
R
+
1 +
(t)2
6
, (2.82)
which means that the leapfrog scheme is accelerating. The disadvantage of
the leapfrog scheme is the undamped computational mode, which is typically
avoided by some ltering techniques. Another method is to use the Adams-
Bashforth scheme (2.77) instead, for which the amplication factor can be ap-
proximated by
|A
+
| 1 +
1
4
(t)
4
, |A

|
1
2
t, (2.83)
i.e., the physical mode is amplifying and the computational mode is strongly
damped and the relative phase error of the physical mode as
R
+
1 +
5
12
(t). (2.84)
30