To my Father and Mother

who taught me that knowledge is to be revered

Preface
Computer modeling has had a significant impact on the way we
do physics, both in research and in teaching. We are able to
study problems of all scales from atoms to galaxies, and of
complexities that would be impossible without computer
modeling. Computation has even led to an entirely new field of
science, chaos. In many ways, computational modeling has
become the third pillar of physics alongside experimentation
and theory.

In this book, we introduce computational modeling and

visualization of physical systems that are commonly found in
physics and related areas. Our first and foremost goal is to
introduce a framework that integrates model building,
algorithm development, and data visualization for problem
solving via scientific computing. Through carefully selected
problems, methods, and projects, the student is guided to
learning and discovery by actively doing rather than just
knowing physics. By constructing models and algorithms,
programming and testing them, and analyzing the results, we
will gain insight, ask new questions, tweak the model as
necessary, and change the parameters to test what-if scenarios
like turning knobs in a virtual experiment. Another goal is to
broaden the scope and depth of problems that may be studied
with computational modeling. Many fundamental physical
systems, despite their apparent simplicity, are beyond reach
without computer simulation. Take projectile motion with air
resistance and quantum free fall, for example. Though the
problems can be expressed in basic calculus, they are
unsolvable analytically in closed-form solutions. Computer
modeling enables us to not only solve these problems, but
comparatively study the differences and similarities of their
behavior in classical and quantum mechanical realms. We also
aim to integrate applied computational tools and methods with
effective visualization techniques in the simulations, including
in-situ data graphing and real-time animation within standard
and open-source frameworks.

We take a problem-centric approach to the presentation of

the material. For most problems, we include sufficient
background information or essential relations to make them
self-contained as much as possible, as well as their general
importance and relevance. To streamline the text, many of the
details are given in appendices or exercises, allowing readers of
diverse background to skip ahead or study the detail at their
own pace. We have also created a Digital Online Companion
(DOC) for in-depth and advanced topics. Following the
description of a problem, we discuss model building, the
appropriate computational methods and tools, and
visualization techniques to the simulations. Most results are
represented in graphical form directly from the simulation
programs for analysis and discussion.

Graphical representation of data and effective visualization

techniques, including real-time animations of three-
dimensional objects, are standard parts of our simulations
because they are essential to help interpret and analyze the
results, especially in time-dependent studies. They also bring
the simulations alive, making them more dynamic, instructive,
and exciting.

The tight integration of advanced graphics and visualization

into the simulations is made possible with standard, easy-to-
use, open-source packages such as Matplotlib, SciPy, and
VPython. These packages only require us to manipulate an
object's attributes such as the physical position of a particle, so
we can avoid low-level graphics programming and focus on
modeling and proper techniques in using these tools. Except
for schematic illustrations, most graphical representations and
animations are created using techniques in the actual codes
contained in the book. However, many programs are written
such that the integrated graphics can be decoupled (disabled),
replaced with either Matplotlib output or results written to a
file, without affecting the calculations.

Since coding is essential to understanding an algorithm or

to gaining insight to a physical process, and the most effective
way to learn to code is by studying examples, the reader is
guided in the process of building over ninety fully-working
sample programs, with detailed explanations for most of them.
Many of the coded algorithms form part of basic building
blocks in scientific computing. The associated tools and
methods, whether well-known or otherwise still being currently
researched, share common elements found in computational
research of more complex problems. Through building fully
working programs complete with graphical output, the student
will learn not only how to write codes for computation, but
their finished product is suitable for demonstration as well.

We use Python as the default programming language to

show concrete, working examples and to take several
advantages it offers: being easy to learn and use, readable,
flexible and powerful. One can think of Python as the modern
equivalent of BASIC, but with a large and growing body of open
source libraries for common tasks in scientific computing such
as those mentioned above. Even newcomers to Python can
learn from examples and grow quickly, writing functional
programs and being productive in about two to three weeks.
However, the choice of a programming language can be highly
personal. Readers with programming background in other
languages should find most Python codes to be expressive,
pseudocode-like, and readily adaptable.

Sample systems are drawn from across the fundamental

areas of physical science including mechanical,
electromagnetic, quantum, and statistical systems.
Representative problems within a given topic or theme are
organized by chapter. Most chapters start with an animated
simulation related to the central topic and end with a brief
summary. The chapters and topics are organized as follows.
After a brief introduction in Chapter 1 to our approach to
modeling and basic programming elements, we discuss
methods of solving ordinary differential equations (ODE) in
Chapter 2 as it constitutes the beginning of our framework.
Thematic studies begin with the familiar example of projectile
motion with air resistance (Chapter 3), followed by planetary
and few-body motion including exoplanet modeling (Chapter
4), and nonlinear dynamics and classical chaos (Chapter 5). We
study waves and oscillations in Chapter 6 and electromagnetic
fields in Chapter 7. In Chapters 8 and 9, we present simulations
of time-dependent and time-independent quantum mechanics,
respectively. We consider simple random systems in Chapter
10 and statistical and thermal dynamics in Chapter 11. Finally,
we investigate classical and quantum scattering in Chapter 12.

To effectively study these topics, the student should be

familiar with basic calculus and introductory mechanics for
mechanical systems (Chapters 2 through 6). For quantum and
thermal systems (Chapters 8, 9, and 11), some familiarity with
concepts typically covered in a modern physics course such as
the wave function, expectation value, eigenstates, entropy, etc.,
will be helpful. We also assume the student to have some
programming background, as familiarity with Python is a plus
but not required. Past experience has shown that students
unfamiliar with Python can successfully learn from examples
and become productive quickly. By the time Chapter 3 is
finished, most students will have grasped the essential
programming elements needed to carry on.

Most of the introductory topics are aimed at the

undergraduate level (up to Chapter 7). Starting from Chapter 5
and increasingly toward Chapter 12, we include advanced and
more in-depth topics in the DOC content. These topics are
geared at upper undergraduate or graduate levels, including
quantum chaos, particle transport, Bose-Einstein
condensation, quantum transitions, inelastic scattering and
atomic reactions.

Moreover, we put a special emphasis on the simulation of

quantum systems, expanding the coverage over three chapters
(Chapters 8, 9, and 12). Ideally, these topics are better studied
after a first course in quantum mechanics. However, some
sections should be accessible even without a quantum course,
such as Sections 8.1 through 8.3 and Section 9.1 for which a
basic understanding of wave motion should be sufficient. Our
motivation for including these topics is due to the fact that,
unlike classical mechanics, quantum mechanics has far fewer
analytically solvable problems, and is less intuitive and less
visual compared to classical motion. These factors cause
considerable difficulties to students’ understanding in
introductory quantum mechanics. By simulating and
visualizing the behavior of quantum systems such as the
quantum oscillator or free fall, we are in a position to help
effectively address some of the difficulties, pre- or post-
quantum class. We regularly use computer simulations to
augment and enhance traditional physics classes such as
quantum mechanics.

Although some chapters are inter-related, most from

Chapter 3 on can be independently studied. Figure 1 shows the
major dependencies of the chapters where the primary
numerical methods (under the chapter number) are discussed,
which are used in subsequent chapters. As the book progresses,
most chapters are dependent on two previous chapters or less,
except Chapters 9 and 12 which are dependent on five. There
are several methods spanning multiple chapters, including
ODE solvers (discussed in Chapter 2) for most time-dependent
simulations in Chapters 3 through 9 and Chapter 11, the finite
element method introduced in Chapter 6 and extended in
Chapters 7 and 9, and other methods such as root finding
(Chapter 3), etc. In many cases, the reader may skip the
methodology section without much disruption. For instance,
the fast Fourier transform (FFT) discussed in Chapter 5 of DOC
is lightly used in Chapter 6 to analyze fundamental oscillation
frequencies, but heavily used in Chapter 8 for evolving
quantum systems and for extracting momentum distributions.
In both instances, the reader initially unfamiliar with FFT can
still work through the simulations without worrying about the
technical details of FFT, using it as a supplied library. The
reader may come back and review it later for a fuller
understanding if desired.
 Figure 1: Major chapter dependencies at a glance. Open circles
indicate minor or Digital Online Companion only dependencies. The
abbreviations are: FEM – finite element method; FFT – fast Fourier
transform; itg – numerical integration; lft – leapfrog with time-
transformation; ODE – ordinary differential equations; rnd – random
numbers; and rtf – root finding.

The instructor can pick what sections to cover within a

given chapter in most cases. For example, the instructor may
skip some sections on ODE solvers to save time (see
Introduction in Chapter 2). However, this may be limited due
to progression in methodology or conceptual buildup. In such
situations, indicated in the narrative and aided by Figure 1, the
instructor may choose to discuss the prerequisite section lightly
and leave further investigation to a project. Carefully
constructed and tested projects given at the end of each chapter
are an integral resource enabling the students to extend the
models, make predictions, test them, analyze and present the
results in project reports. Many in-depth projects, some found
in the DOC, are designed for two-person teams to encourage
peer learning and collaboration. There are also numerous
exercises for the purpose of analytical derivations and proofs or
simple programming tasks.

Throughout, references to the DOC content are indicated by

the “S:” prefix, including the completely merged index. The
DOC content and other electronic resources such as programs,
data files, solution manual to select exercises, and installation
instruction and files can be found at:

http://www.wiley.com/college/wang

Acknowledgments

I am indebted to Joachim Burgdörfer from whom I learned

much of the craft of the trade during my PhD study. I also
benefited from professional and personal interactions with
Wolfgang Christian (Davidson College), Steven Gottlieb
(Indiana University, Bloomington), Harvey Gould (Clark
University), and Jan Tobochnik (Kalamazoo College), who
have been pioneers in computational physics and education. I
am grateful to Bruce Sherwood, the original developer of
VPython and GlowScript, for being a constant resource and
teacher. I am thankful to Steve Spicklemire (University of
Indianapolis) for reviewing the manuscript, testing the
programs, and making suggestions that substantially changed
the structure of the codes on vector operations.

I owe a debt of deep gratitude to Michael Carlozzi (Writing

and Reading Center, UMass Dartmouth) for his great
generosity in time and effort to offer expert critique of the
manuscript over two semesters that helped to significantly
improve the writing, organization, and presentation of the
material.

Many reviewers have provided critical and constructive

feedback that improved the manuscript. I thank Brad Marston
(Brown University) for extensive comments and valuable
suggestions on not only modeling, but the validation and
interpretation of computational results. I also thank Johnathan
Barnes (Salt Lake Community College) for careful reading and
annotation of part of the manuscript, and Larry Engelhardt
(Francis Marion University) for many useful discussions and
suggestions including code optimization. I sincerely
acknowledge useful input and viewpoints from other reviewers
including Eric Ayers (California State University, Chico),
Simon Billinge (Michigan State University), Alexander
Godunov (Old Dominion University), Rainer Grobe (Illinois
State University), Dawn Hollenbeck (Rochester Institute of
Technology), Lev Kaplan (Tulane University), James
McDonald (University of Delaware), Brianislav Nikolic
(University of Delaware), Michael Roth (University of
Northern Iowa), Steffen Vojta (Missouri University of Science
and Technology), and Haowen Xi (Bowling Green State
University).

In addition, I would like to thank Anne Caraley (State

University of New York at Oswego), Henry Greenside (Duke
University), and Timothy Roach (College of the Holy Cross) for
their generous time and effort providing helpful input out of
personal interest in the manuscript.

I wish to thank all my colleagues in the Department of

Physics for their strong support and encouragement for the
book project, including the granting of a sabbatical. I am very
grateful to Robert Fisher and Gaurav Khanna who gave specific
comments and valuable suggestions, and to Glenn Volkema
who tested and advised on the installation procedures. I also
wish to thank Adam Hausknecht and Alfa Heryudono in the
Department of Mathematics for helpful ideas and feedback. I
thank my own students in the computational physics course
over the years who left their mark on the project.

Furthermore, I would like to express my thanks to the Wiley

team for their able support and guidance: to editors Stuart
Johnson for initiating and developing the project and to Jessica
Fiorillo for expertly and energetically steering it to completion,
to Mary O’Sullivan, Amanda Rillo, Gladys Soto, and Marcus
Van Harpen for production and logistics support.

Finally, I wish to thank my family for their unconditional

love and support. I want to say thanks to my sons, Gregory and
Adam, for giving advice and opinion whenever I asked, and for
trying out some topics and projects in the book. Last but not
least, a heartfelt thank-you to my wife, Huihua, for showing
great understanding and patience throughout the long project,
without which this project would not have been possible.
Contents
1 Introduction

1.1 Computational modeling and visualization

1.2 The science and art of numerics

1.3 Fundamentals of programming and visualization

1.4 Exercises and Projects

1.A Floating point representation

1.B Python installation

1.C The Matplotlib plot function

1.D Basic NumPy array operations

2 Free fall and ordinary differential equations

2.1 Free fall with Euler's method

2.2 The Runge-Kutta methods

2.3 System of first order ODEs

2.4 The leapfrog method

2.5 Exercises and Projects

2.A Area preservation of the leapfrog method

2.B Program listings and descriptions

3 Realistic projectile motion with air resistance

3.1 Visualization of ideal projectile motion

3.2 Modeling air resistance

3.3 Linear air resistance

3.4 The Lambert W function

3.5 Quadratic air resistance and spin

3.6 Physics of ball sports

3.7 Shooting methods

3.8 Exercises and Projects

3.A Bisection and Newton's root finders

3.B Program listings and descriptions

4 Planetary motion and few-body problems

4.1 Motion of a planet

4.2 Properties of planetary motion

4.3 Precession of Mercury

4.4 Star wobbles and exoplanets

4.5 Planar three-body problems

4.6 The restricted three-body problem

4.7 Exercises and Projects

4.A Rotating frames and rate of change of vectors

4.B Rotation matrices

4.C Radial velocity transformation

4.D Program listings and descriptions

5 Nonlinear dynamics and chaos

5.1 A first model: the logistic map

5.2 Chaos

5.3 A nonlinear driven oscillator

5.4 The Lorenz flow

5.5 Power spectrum and Fourier transform

5.6 Fractals

5.7 Exercises and Projects

5.A Program listings and descriptions

6 Oscillations and waves

6.1 A damped harmonic oscillator

6.2 Vibrations of triatomic molecules

6.3 Displacement of a string under a load

6.4 Point source and finite element method

6.5 Waves on a string

6.6 Standing waves

6.7 Waves on a membrane

6.8 A falling tablecloth toward equilibrium

6.9 Exercises and Projects

6.A Program listings and descriptions

7 Electromagnetic fields

7.1 The game of electric field hockey

7.2 Electric potentials and fields

7.3 Laplace equation and finite element method

7.4 Boundary value problems with FEM

7.5 Meshfree methods for potentials and fields

7.6 Visualization of electromagnetic fields

7.7 Exercises and Projects

7.A Program listings and descriptions

8 Time-dependent quantum mechanics

8.1 Time-dependent Schrödinger equation

8.2 Direct simulation

8.3 Free fall, the quantum way

8.4 Two-state systems and Rabi flopping

8.5 Quantum waves in 2D

8.6 Exercises and Projects

8.A Numerical integration

8.B Program listings and descriptions

9 Time-independent quantum mechanics

9.1 Bound states by shooting methods

9.2 Periodic potentials and energy bands

9.3 Eigenenergies by FDM and FEM methods

9.4 Basis expansion method

9.5 Central field potentials

9.6 Quantum dot

9.7 Exercises and Projects

9.A Numerov's method

9.B The linear potential and Airy function

9.C Program listings and descriptions

10 Simple random problems

10.1 Random numbers and radioactive decay

10.2 Random walk

10.3 Brownian motion

10.4 Potential energy by Monte Carlo integration

10.5 Exercises and Projects

10.A Statistical theory of Brownian motion

10.B Nonuniform distributions

10.C Program listings and descriptions

11 Thermal systems

11.1 Thermodynamics of equilibrium

11.2 The Ising model

11.3 Thermal relaxation by simulated annealing

11.4 Molecular dynamics

11.5 Exercises and Projects

11.A Boltzmann factor and entropy

11.B Exact solutions of the 2D Ising model

11.C Program listings and descriptions

12 Classical and quantum scattering

12.1 Scattering and cross sections

12.2 Rainbow and glory scattering

12.3 Quantum scattering amplitude

12.4 Partial waves

12.5 Exercises and Projects

12.A Derivation of the deflection function

12.B Partial wave analysis

12.C Program listings and descriptions

List of programs

Bibliography

Index
Chapter 1
Introduction
In this book, we will be discussing computational modeling of
physical systems ranging from classical to quantum systems.
The primary factor that makes this possible is the speed of
computation offered by digital computers. In this sense,
computer modeling is simply a tool to get the job done, just like
the abacus or the calculator of the past. It is an important tool,
enabling us to simulate a range of systems from the simple
projectile motion but with added realism such as drag and spin
effects to the complex behaviors of many-body problems or
time-dependent quantum systems. Computational modeling is
important and often indispensable in the study of these
systems.

Below, we outline a three-step strategy in computational

modeling. We also briefly discuss sources of error, and
introduce the basic elements of Python programming with our
first program as well as the required packages for computation
and visualization. Installation of these packages and numerical
array operations are discussed in the Appendices.
.1 Computational modeling and
visualization
Before computational modeling can begin, there are several
required steps to set it up. We generally follow a three-step
approach for a given problem:

First, we build a model based on physical laws governing the

problem. We usually aim to keep the model as simple as
possible that contains the essential physics. On the one hand,
key physical concepts are best understood from the simplest of
the models. On the other hand, many problems are sufficiently
complex that, without simplifying approximations, they cannot
be solved even with the most powerful computers available.

Secondly, we construct a simulation method, or an

algorithm, to solve the problem based on the mathematical
description of the physical model. Consideration of the
operational behavior of an algorithm can often lead to physical
insight about how the physical processes unfold. There is
typically a tradeoff in the selection of algorithms. The main
criteria affecting our choice include accuracy, simplicity,
stability, and robustness.

Lastly, we implement the algorithm in a programming

language and present the results accordingly. Here again, we
face several possible choices. We choose Python because,
among other reasons, it is easy to write and read codes, and is
open source. Python is flexible, and is made more attractive
and powerful for scientific computing due to the availability of
external libraries for numerical simulations. For our purpose,
four open-source packages are especially useful: SciPy and
NumPy, an essential core base for numerical and scientific
computing in Python; Matplotlib, a library for data plotting and
graphical presentation in 2D and 3D; and VPython, a 3D
interactive animation and graphics library.

The NumPy library adds advanced multidimensional array

capability that is missing in native Python. The SciPy library
provides many fundamental numerical analysis tools, among
which the linear algebra routines and special functions are
especially useful to us. These routines not only extend the
functionalities of Python, but many of them also allow
vectorized operations (implicit looping) over whole arrays. We
gain the added benefits of execution speeds of compiled codes,
as well as simpler and more elegant programs.

The graphics libraries enable us to seamlessly integrate

visualization into our Python simulations. Data visualization is
becoming an essential part of computer modeling that is
increasingly data-centric. We will encounter simulations which
yield a large amount of spatial and temporal data. With
Matplotlib, we can produce with ease graphical representation
of simulation results as simple line graphs, contour and surface
plots, and vector fields. For many systems, we also use VPython
to animate the process or data sets as dynamic movies.1 All
 these techniques help us to more effectively analyze and
interpret the simulation results. We can do all this with these
packages within four or five dozen lines of code in most cases
(see Section 1.B for installation of required packages).

We make every effort to present the physical systems as self-

contained models. For mechanical systems, the student should
have a good grasp of physics at the introductory level or above.
For quantum and thermal systems, some familiarity with the
relevant concepts such as the wave function, expectation value,
entropy, etc., will be helpful. We also assume the student to
have some programming background. Familiarity with Python
is helpful but not required. Past experience has shown that
students unfamiliar with Python can successfully learn from
the examples and become productive quickly.

.2 The science and art of numerics

Scientific computing naturally involves numerical computation
(number crunching). It uses integers and real values, or
floating point numbers. Integer arithmetic on the computer is
exact, but its range is limited. It is unsuitable for scientific
computing since our physical world encompasses a vast scale
over many orders of magnitude. Floating point numbers have a
much expanded range, but their operations may not be exact in
the mathematical sense, so exactness is sacrificed for enlarged
range. Here we briefly discuss floating point operations so we
are aware of some of the limitations.
1.2.1 FLOATING POINT OPERATIONS
To check floating point operations, consider for
some real value |x| < 1. We will use IPython (other interactive
shells such as IDLE work as well; see Section 1.B on installing
and running programs). At the prompt In [1], enter the
following (in notebook mode, press Shift Enter at the end of
each input cell to get output):

In [1]: %precision 17
x=0.6
a=(1−x*x)**(0.5)
a
Out[1]: 0.80000000000000004
In [2]: 1−a
Out[2]: 0.19999999999999996

The “%precision 17” keyword (also called a magic function)

tells IPython to print numbers to 17 decimal places. If nothing
follows the keyword, default behavior is restored (see below).

Mathematically, we should have exact values a = 0.8 and 1

− a = 0.2. But the numerical values are not exact though very
close, agreeing to 16 digits. We see right away that
mathematical rules are bent a bit in floating point operations.
This is because of the way decimal numbers (floats) are
represented in the computer. A float f is internally represented
by
where S is the sign, M a fractional number between
(except 0), known as the mantissa, and E an integer exponent.
Python provides an easy way to find M and E for a given f as
follows,

In [3]: import math as ma # import math library

ma. frexp (0.2)
Out[3]: (0.80000000000000004, −2)

In the code above, we first import the mathematics library

and rename it to ma, and use the Python function frexp() to
find the mantissa and the exponent of 0.2, which are M = 0.8,
and E = −2. Indeed, we have 0.2 = 0.8/4 = 0.8 × 2−2, according
to Eq. (1.1). Note the mantissa is not exactly 0.8 as seen by the
computer. Why?

The reason is due to the representation of a float by a finite

number of bits (see Appendix Section 1.A). According to Eq.
(1.7), the mantissa is a sum of powers , i = 1, 2, ..., n, similar
to a geometric series. Not all real values can fit exactly in a
finite n-bit range (see Table 1.2). Python uses double precision
floats, which has an effective n = 53 bits. For a fixed exponent,
the smallest difference between a pair of mantissas
representable on the computer is ~ 2−53 ≈ 10−16. For single
precision floats, the difference is larger. Double precision, the
default in Python, is most common in scientific computing,
though higher precisions such as quadruple precision are used
for special purposes. Regardless, a measure of precision is
given by the machine accuracy ∈, defined as the largest ∈ value
that the sum 1 + ∈ is still 1 in floating point arithmetic (not
mathematical arithmetic),

Let us see what the machine thinks of this:

In [4]: 1. + 1.e−15 == 1.
Out[4]: False
In [5]: 1. + 1.e−16 == 1.
Out[5]: True

The operator “ ==” tests whether a condition is true or false.

Evidently, the machine accuracy ∈ is below 10−15 but above
10−16.

1.2.2 NUMERICAL ERROR

When doing numerical calculations, we should be aware that
numerical results are not always mathematically sensical. For
instance, using the same example earlier with x = 10−7, we
obtain the following,
 In [6]: x=1.e−7
a=(1−x*x)**(0.5)
a
Out[6]: 0.99999999999999500
In [7]: %precision
1−a
Out[7]: 4.9960036108132044e−15

While the value a is as expected, the accuracy of 1 − a is

significantly reduced compared to the approximate value
, a loss of more than 10 significant
digits in double precision. Further reducing x = 10−8, the
numerical answers become a = 1 and 1 − a = 0. The latter is far
worse than a loss of accuracy, since there is a qualitative
difference between a zero value and a finite but nonzero value,
albeit a small one (~ 5 × 10−17). This occurs because x2 is less
than the machine accuracy in this case, so that
in floating point operations, and we have a total cancellation
error when evaluating 1 − a.

Because floating point operations are inexact,

mathematically equivalent expressions can yield different
numerical results. This often leads to various tricks (on the
level of art) in the quest for better numerical results. One trick
to remedy the above situation is to rewrite it equivalently: let b
= , so that , and 1 − a = 1 − b −
bx. Numerically, we have
 In [8]: x=1.e−8
b=((1−x)/(1+x))**0.5
1−b−b*x
Out[8]: 3.922529011343002e−17

The above step improves 1 − a somewhat, at least it is nonzero,

and on the same order of magnitude as the mathematical
answer. A better way is to write
, so that continuing from
above, we have

In [9]: x*x/(1+(1−x*x)**0.5)
Out[9]: 5.0000000000000005e−17

There is no cancellation involved. This result is as close as we

can get to the mathematical answer within double precision.

The above tricks are mathematically equivalent, but

numerically different. This simple example illustrates that
numerics involves both science and art. Real-world examples
are not as simple, but the underlying idea is valid: we need
mathematical thinking as well as numerical thinking in
scientific computing.

Round-off error
Round-off error can occur in arithmetic operations of floating
point numbers for several reasons (see Section 1.A). The floats
may not be exactly representable, or some significant bits are
lost during shifting. Even if the floats are exactly representable
or no shifting is involved, the intermediate results could
require more mantissa bits to hold the full accuracy than the
working precision could provide, resulting in rounding off.

The severity of round-off error varies. The above example is

a case of round-off errors due to cancellation of two floats (1 −
a) within close proximity of each other, leading to a severe loss
of accuracy. We cannot eliminate round-off error. But with
proper choice of methods (algorithms), we may be able to
control or minimize its impact as seen above.

Truncation error
To produce a concrete result, we normally use a practical
method, or an algorithm, for the calculation. There is usually a
difference between the obtained result and the exact result.
This is known as the truncation error. The size of the error
should be dependent on the method, and unlike round-off
error, it is entirely under our control by the choice of the
method.
 Figure 1.1: Approximating the circumference of a circle by polygons.

As an illustrative example, let us calculate the

circumference of a unit circle approximated by the perimeter of
a regular polygon inside the circle shown in Figure 1.1. For an
n-sided regular polygon, adding up all the sides yields the
perimeter

Figure 1.1 shows two polygons, a square and an octagon. We

can see error cumulating as the arc length is replaced by
straight line segments. With increasing n, we expect the
approximation to be ever more accurate.

Table 1.1: The ratio of an n-sided polygon perimeter to the

circumference of a unit circle.

Table 1.1 lists the ratio of the polygon perimeter to the exact
circumference of the unit circle for several n values. Roughly
with every quadrupling of n, the numerical result gains one
significant digit. The method is stable and allows systematic
improvement of accuracy. Most algorithms in our simulations
are chosen to minimize or control truncation error.

Stability
Often, stability is a factor in controlling truncation error.
Consider the golden mean . It is a
root to the quadratic equation2

We can multiply both sides of Eq. (1.4) by ϕn−1 to get ϕn+1 = ϕn

+ ϕn−1. This indicates a recursion relation

Recursion relations are very useful since they are simple,

fast, and can generate a series of values on the fly. Suppose we
want to compute the series yn for a range of n values by the
recursion method. We set the seeds y0 = 1 and y1 = ϕ in Eq.
(1.5), and obtain y2, then y3, etc. To check the accuracy against
the exact results, we calculate the relative error |yn − ϕn|/|ϕn|
and plot the results in Figure 1.2. Over the n range shown, the
error for the ϕ series is on the order of ~ 10−15, so small that it
is practically at the machine accuracy in double precision.
Though there is a slight rise in the error with increasing n due
to inevitable round-off, the recursion method works well in this
case.

There is a second root to Eq. (1.4),

−0.618034. Like the golden mean,
the negative root ϕ_ also satisfies the recursion relation (1.5),
so we could follow the same way as above to generate a series of
values for via recursion, with the initial seeds set as y0 = 1
and y1 = ϕ_. Repeating the calculation, we find quite a
surprise: the relative error growing exponentially, and without
bound. This behavior is easy to spot because we plot the results
on the semilog scale, which shows the trend (a straight line)
and, at the same time, clearly displays data differing over many
orders of magnitude. The exponential explosion of the error
cannot be attributed to round-off error. Rather, it must be due
to the inherent ill-behavior of the algorithm (recursion). A
significant digit is lost in roughly every 2 to 3 steps, rendering
recursion unstable and useless for the ϕ_ series.
 n
Figure 1.2: The relative error of ϕ and by recursion.

We may wonder why the method works well for one series
(ϕ) but fails terribly for another (ϕ_). The reason is that both ϕ
and ϕ_ are present (even a small amount caused by round-off
error) in the recursion since both are solutions of Eq. (1.4). The
growing ϕ series completely overwhelms the vanishing ϕ_
series. A small error is quickly amplified and washes out the
latter in a few steps.3 The behavior is not related to, and cannot
be solved by higher float precisions. It is the property of the
recursion (1.5). There is a silver lining, though: if the upward
recursion is unstable, the reverse direction must be stable. We
can use this fact to accurately obtain the ϕ_ series by
downward recursion (Project P1.3). Stability conditions of
some common recursions can be found in Ref. [1].

Through these examples, we demonstrate that proper

choice of algorithms is important, and algorithm-induced error
(truncation error) is entirely under our control.

.3 Fundamentals of programming and

visualization
We are concerned with applying appropriate programming
techniques to computational simulations but not with
fundamental programming itself. Ideally, the reader should
already have some computer programming background.
Knowledge of the Python programming language is preferred
but not required. If you are using Python for the first time, I
recommend that you start with the tutorial at
https://docs.python.org/2/tutorial/. You should find that
Python is a straightforward and forgiving language, and is easy
to learn from examples [54]. Many examples are found in the
included programs in the chapters ahead. Below, we discuss
the basic elements of Python, and introduce the relevant
packages to be used: Matplotlib, NumPy and SciPy, and
VPython. We assume Python 2.7x is installed first, along with
the latest versions of the packages just mentioned (Section 1.B).

1.3.1 PYTHON
Like driving, the only way to learn programming is to just do it.
Let us write a Python program to compute and graph one-
dimensional motion with constant acceleration a. Taking the
initial value to be zero, the position x as a function of time t is
, where v0 is the initial velocity. The program is
given below and results in Figure 1.3 (see Section 1.B on how to
run programs).

Program listing 1.1: Motion with constant acceleration ( motion.py)

import matplotlib.pyplot as plt # get matplotlib plot functions

2

a, v0 = input( ’enter a, v0 (eg 1.0, 0.0) : ’) # read input

4 t, h, n = 0.0, 0.1, 20 # init time, step size, number of steps
ta, xa = [], [] # time and position arrays for plotting
6
 for i in range(n): # loop for n steps
8 ta.append(t) # record time and position
xa.append(v0*t + a*t*t/2.0)
10 t=t+h # update time

12 plt.figure () # start a figure ; no indent−>end of loop

plt.plot(ta, xa, ’-o’) # plot data
14 plt.xlabel( ’t (s)’) # add labels
plt.ylabel( ’x (m)’)
16 plt.show() # show figure

Figure 1.3: The position-time curve from Program 1.1.

As our first full example, Program 1.1 may feel longer than
necessary. It is okay if you do not understand fully what is
going on here. We will explain the program below, but after
working through the rest of this chapter to the middle of
Chapter 2, it will become clear. The program has three basic
blocks: initialization (lines 1 to 5), computation (lines 7 to 10),
and data presentation (lines 12 to end). Most programs
generally follow this template.

Program 1.1 starts by importing from Matplotlib the pyplot

library, which is the main workhorse of 2D plotting, and is
short-named to plt. Any external library must be imported
before its use.4 To keep separate namespaces, we will follow
this convention from now on using short descriptive aliases for
several libraries including VPython as vp, and NumPy as np.
The program then reads (line 3, blank lines are ignored) the
values a and v0, assumed to be in MKS units, from the
keyboard via the input() function which can parse multiple
parameters from user input. In this case, we should enter two
numbers separated by a comma (also see what happens
without the comma). The next line initializes the time, step
size, and the number of steps, the first two being floats, and the
last one being an integer. We can assign values to multiple
variables on a single line, provided the number of variables on
the left is equal to the number of values on the right (called
sequence unpacking). This saves print space without sacrificing
clarity. In Python, variables need not be declared before
assignment. The type of a variable is automatically derived
from the assigned object. The same variable could be assigned
to a different object later, changing its type. The last line before
the loop creates two empty lists to store time and position
values, rounding up the initialization block. For readability, it
is usually a good idea to group similar constants and
parameters together.
 After the initialization statements, the for loop starts at line
7, and terminates just before the blank line. The range()
function specifies the number of iterations. Unlike other
languages where the programmer controls the appearance of
logical blocks for easy identification, Python requires vertical
alignment (indentation) for block control, including while
loops, if conditions, function declarations def(), and other
logical blocks. The next three lines, indented by the same
amount, belong to the for loop. Inside the loop, the current
time and position are appended to the respective lists ta and
xa,5 and time is incremented. Like the append method, there is
a host of methods for manipulating lists, including insert,
remove, pop, etc.

The rest of the program graphs the results (Figure 1.3, a = 1

m/s2, v0 = 0). We first open a figure (line 12, it may be omitted
if there is only one figure in the program), and plot the data
with plot() from the Matplotlib library. The plot function
accepts the required pair of (x, y) data arrays, plus an optional
string specifying color, line style, or symbol types (see Section
1.C). After adding axis labels, the plot is displayed on the
screen, ending the program. We can manipulate the figure,
zoom in, drag around, or save to a file.

Conditional controls
Besides the “for” loop above, Python has a conditional while
loop (see Program 2.1)
 while (condition):
……

The conditional if statement takes the form (also see

Program 2.1)

if (condition):
……
elif (condition):
……
else:
……

Both the elif and else are optional, and there can be
multiple elif statements. There is also an inline if-else
expression that yields a value depending on the condition:

x = a if condition else b

In the above statement, x will be equal to a if the condition is

true, and equal to b otherwise (see Program 5.1 for an
example).

Function definitions and variable scope

We can define functions with the def keyword as (see Program
2.5)

def func(x1, x2=1.0):

……
return y1, y2

The function may accept input parameters with default

values ( x2=1.0). Unlike traditional programming languages,
Python functions can return multiple values, or none at all. All
variables defined inside the function are local and inaccessible
outside. The function can access any global variables defined in
the main program, but cannot modify them. If the latter is
necessary (should be rarely used and very well controlled, see
Program 9.2), the variables to be modified must be declared as
global inside.

Procedural and object-oriented programming

With few exceptions, we choose procedural programming for
its direct approach to problem solving. We can set up a
problem quickly and solve it by building a series of compact
functions. There are cases when it is more convenient to use
object-oriented programming, such as Brownian motion
(Program 10.2), Einstein solids (Program 11.1), and VPython
modules (Program S:6.4).
 Object-oriented programming combines both methods (i.e.,
functions) and data into standalone entities called objects. It
may be more logical and suitable for building models that are
expandable (see Program 11.1 and Project P11.2). For example,
it would be natural to define particle objects with properties
such as position and velocity. When combined with an
integration method (Chapter 2), one could instantiate one
particle object for projectile motion (Chapter 3), or another one
for planetary motion (Chapter 4), all within a unified approach.
One could even extend it to N-body problems. The interested
reader may wish to explore this object-oriented approach.

File handling
Among Python file handlers, the easiest way may be to read
from and write data to files via the pickle module. Structured
data records can be loaded or dumped easily (see Program 9.7
for an example). A quick way to handle text files can be found
in Program 4.4.

Python 2.7x and 3.xx compatibility

The programs presented assume Python version 2.7x, though
they should run without problems in version 3.xx, provided the
required packages are 3.xx specific. For our purpose, there are
two incompatibilities between the two versions that affect us:
print and division operations. First, the print statement in 2.7x
becomes a function in 3.xx (see Program 3.5), and second,
divisions are carried out as floats in 3.xx. To retain forward
compatibility, the first line of a 2.7x program should be

from _future_ import print_function, division

Python 2.7x treats divisions according to operand types, but

3.xx defaults divisions to float operations. For example, in 2.7x,
we have (using IPython)

In [1]: 1/2
Out[1]: 0
In [2]: 2.0**(1/2)
Out[2]: 1.0

The last result was probably unintended even in 2.7x, and a

source of unintentional human error. To always use float
division like in 3.xx, add the compatibility statement as

In [3]: from __future__ import print_function, division

1/2
Out[3]: 0.5
In [4]: 2.0**(1/2)
Out[4]: 1.4142135623730951
 We believe the programmer should be explicit about
whether integer or float divisions are intended. For instance,
the following works in both versions,

In [5]: 1//2 # integer division

Out[5]: 0
In [6]: 2.**(1./2.) # float division
Out[6]: 1.4142135623730951

We follow the above practice in our programs. The included

programs run correctly under 3.xx without relying on the
division compatibility.

Built-in help
Python has an online help system that can provide information
on keywords, modules, etc. It is invoked by

In [7]: help()
help>

Enter terms at the help> prompt, e.g., range or numpy, to get

help on them. Type q to return to the prompt or exit help.

1.3.2 NUMPY AND SCIPY

At its core, the NumPy library offers advanced
multidimensional array creation and manipulation functions.
In concert with Python, they operate at a high and often
abstract level, and execute at the speed of compiled codes. They
form a unique and powerful combination for scientific
computing. Many other libraries require NumPy, including
Matplotlib and VPython. Appendix 1.D introduces the basic
operations on NumPy arrays. We also use elementary
mathematical functions in NumPy, many of which are the so-
called universal functions (ufunc), functions that operate on
scalar and NumPy array inputs transparently (see Program 4.6
for examples).

We will mainly use the SciPy library for solving linear

systems and eigen-value problems. It has highly specialized
routines for linear algebraic problems, including efficient
banded matrix solvers (see Program 8.3). SciPy also offers
many special functions useful to us, such as Legendre and
Hermite polynomials, and spherical Bessel functions, etc. Like
NumPy functions, many SciPy routines are universal functions.

Speed boost with language extensions

Though easy to learn and use, flexible yet powerful, Python by
itself was not designed for sheer number crunching, a core task
in scientific computing. Being an interpretive language, Python
parses and translates each statement it encounters, one at a
time, into machine instructions. Therefore, explicit, nested
looping is slow and expensive, making the use of NumPy array
operations practically indispensable in many simulations.
Programs 6.7 and 8.2 are such examples where the proper use
of NumPy arrays speeds up the code by 50 times or more over
explicit looping (see also Programs 6.8 and 7.2, and Section
8.2.2).

Sometimes the speed problem cannot be solved so easily

with NumPy alone. In most cases, a small part of the code hogs
most of the computing time. Code profiling (Section S:8.B) can
tell us where the bottleneck is. Are there ways to alleviate, even
eliminate, the bottleneck without losing the benefits of Python?
The answer is yes: with language extensions. There are several
such easy-to-use extensions. Numba is the easiest package that
can compile a standard Python function at runtime. It does not
require us to change the code or use a different language. We
just put the so-called decorator symbol @jit in front of a
function, and almost by magic, it runs faster, sometimes by a
lot. So Numba is nearly effortless to use. This is shown in a
simple example in Program 5.7 for fractals. Cython is a package
providing a simple interface so Python can easily call compiled
C/C++ codes. Another package is Weave, part of the SciPy
library, that can turn inline expressions into C/C++ codes. For
Fortran, one can use F2Py, which is part of NumPy. F2Py can
convert Fortran codes into Python callable binary modules.

For example, molecular dynamics (Section 11.4) is a typical

case where the bottleneck is localized in the computation of
forces among the particles (Program 11.7). When we use F2Py
extensions to convert that part of the code to Fortran, we get a
handsome payoff: a 100x speed gain (see Programs S:11.3 and
S:11.4).

When we have a critical speed bottleneck, one of the above

extensions should solve the problem. With just a little effort,
we can have the best of both worlds.

1.3.3 MATPLOTLIB
Matplotlib is a Python library for data plotting and graphical
presentation in 2D and 3D. It is easy to make quality graphs
with Matplotlib, interact with them like zooming, and save to
standard file formats. At the same time, almost every aspect of
a Matplotlib plot is customizable, making it possible to do hard
or odd things, such as twin axes with different scales or even
data animation (see Figure 3.16, Programs 8.1 and 10.2, and
Matplotlib gallery for examples).

Almost all graphs of simulation results in the book are made

in-situ with Matplotlib. Source codes for a graph can be used as
a template for similar types (e.g., Program 4.6 for surface
plots), with the user substituting data only. This saves us time
as we can simply copy and paste that part of the code.
Furthermore, Matplotlib functions should be familiar to Matlab
users.

Matplotlib also works in IPython notebooks, so graphs can

be embedded in an interactive document. This is illustrated in
Figure 1.4. The statement “ %matplotlib inline” in input cell 1
tells IPython to show Matplotlib output within the notebook.
The first line of input cell 2 imports pylab which combines both
pyplot and NumPy into a single entity (namespace), effectively
importing two packages at once. The next line divides the
interval [−10, 10] into 100 points using the NumPy function
linspace, which returns the points in a NumPy array assigned
to x (see Program 4.6). The last line plots the function sin(x)/x,
which is shown immediately below. Here, the plotted function
accepts an array input automatically, an example of a ufunc.
We can also graph functions symbolically with SymPy (see
Section 1.B).

Figure 1.4: Matplotlib and IPython notebook.

It is also possible to load an existing program such as

Program 1.1 in input cell 2. This way, we have a flexible,
interactive environment within which both simulation and
visualization can be done.

1.3.4 VPYTHON
VPython is a graphics programming library for Python. It
makes manipulation and dynamic animation of illuminated
three-dimensional objects easy and accessible to the end user.
It offers a rich and interactive environment to render the
simulation process or data sets as dynamic movies on the
screen. Program 2.1 shows a simple example of a bouncing ball
(Figure 2.1). We use VPython extensively. In nearly all cases,
animation can be turned off (commented out) without affecting
the non-visual aspect of the simulations, or replaced with
either Matplotlib output (see Exercise E2.1) or data files for
separate analysis.

Starting from VPython 6, a frame-limiting function rate()

is required in the animation loop for it to function correctly
across different computer platforms due to technical reasons.
This normally does not pose a problem, because without it,
most animations would run too fast given the speed of today’s
computers. In cases where we need all the speed we can get
(e.g., restricted three-body motion, Program 4.7), we can set
the limit very high.
 Figure 1.5: Spinning cube in the IPython notebook.

VPython programs are most robustly run by themselves

(see Section 1.B on ways to run programs). Very recently,
however, it has become possible to run VPython animation
inline within IPython notebooks, as is the case with inline
Matplotlib. To do so, just replace VPython with a wrapper
called IVisual (see Section 1.B), and create VPython objects as
usual. Figure 1.5 shows a spinning cube in the IPython
notebook.6 The cube is created using the VPython box object
with default attributes since none is specified in the argument,
so it appears as a cube with equal length, height, and width (see
Program 2.1). Inside the while loop, the rate statement limits
the animation to about 20 loops per second. We make the cube
spin using the rotate method, which rotates an object for the
specified angle (radians) about an axis vector (see Program 3.8
for a spinning baseball).

Run the example, and interact with the animation. You can
view the scene from different perspectives by right-button drag
(or Ctrl-drag) to change the camera angle. You can also zoom in
or out by middle-button drag (Alt-drag, or left+right on a two-
button mouse).

VPython also defines vector objects and operations,

including cross and dot products. We make use of these
capabilities (see Programs 3.8, S:6.2, and 7.5).

1.3.5 DEBUGGING AND OPTIMIZATION

It is rare not to make programming errors when writing
functional programs of more than a dozen or so lines.
Debugging can be challenging and time-consuming. Syntax
errors are easy to find, but logical errors can be hard to trace. A
few things can make debugging easier. Write modular and
readable codes, test them often and in isolation if possible, and
comment out, rather than delete, the changed lines (delete only
when finished). Python is forgiving since it will print out the
lines where errors occurred, but the chain-error messages can
be bewildering. Try to find the error from the earliest line in the
error chain, and print the values of the variables in question.
For really hard-to-find bugs, try the integrated debugger built
in the Python shell, IDLE. It allows stepping through the code
and examination of variables.
 Optimization, if necessary, should be done once a working
program is at hand. It should keep the loss of clarity and
readability to a minimum. Before we start optimizing, find the
bottleneck by profiling first (Section S:8.B).

Chapter summary

We outlined the three-step strategy in computational modeling,

namely: model building, algorithm construction, and
implementation. We discussed floating point numerics and its
limitations, sources of error, including round-off and
truncation errors. We very briefly described the basic Python
elements, and the required libraries: Matplotlib, NumPy and
SciPy, and VPython. Familiarity with these libraries comes
from using them, which has plenty of helping in the chapters
ahead.

General references

We list below several general references on computational

physics. A comprehensive bibliography is included at the end of
the book.

P. DeVries and J. Hasbun, A first course in

computational physics, 2nd ed. (Jones & Bartlett, 2010)
[24]. An introductory computational physics text that
emphasized graphical representation of data even in the
early days of the first edition, with the graphics-
challenged Fortran language. This edition uses the
MATLAB environment.

N. Giordano and H. Nakanishi, Computational physics,

2nd ed. (Benjamin Cummings, 2005) [34]. A general
computational physics book that uses both Fortran and
C as the programming languages. Most examples are
available online only.

H. Gould, J. Tobochnik, and W. Christian, An

introduction to computer simulation methods:
Applications to physical systems, 3rd ed. (Addison-
Wesley, 2007) [42]. A classic text since its first edition
using the True Basic language which had integrated
graphical elements. This book contains a wide breadth
of topics. The third edition uses Java and Open Source
Physics library for graphical output.

R. Landau, M. Páez, and C. Bordeianu, Computational

physics: Problem solving with computers, 2nd ed.
(Wiley, 2007) [53]. A comprehensive how-to book on
numerical methods as applied to computational physics
using the Java programming language. The third
edition (2011) eBook uses Python.

M. Newman, Computational physics, (CreateSpace

Independent Publishing) [68]. A computational physics
text using the Python programming language and the
graphics libraries including VPython.
 T. Pang, An introduction to computational physics, 2nd
ed. (Cambridge University Press, 2006) [70]. A text on
traditional computational physics topics as well as
special topics such as generic algorithm. Sample
programs are given in Java.

J. Thijssen, Computational physics, 2nd ed.

(Cambridge University Press, 2007) [89]. A text aimed
at graduate level with advanced topics such as quantum
physics. Examples in Fortran or C are available online
only.

.4 Exercises and Projects

EXERCISES
E1.1 Find the exponent and mantissa of the following floating
point numbers: 0.5, 1.0, 5.0, 1010, and 10−10.
E1.2 Write down the bit patterns of the following fractions in
their fixed-point format (1.6), keeping only the first 4
most-significant bits.

(a) 1/4; (b) 4/5; (c) 5/8; (d) 3/7.

E1.3 Write a program to find the machine accuracy ∈ on the

computer you are using, to within a factor of 2.
E1.4 (a) Use the sample code with struct (Section 1.A) to
find the mantissa bits in double precision (53 bits
 including the phantom bit) for the floats in Table 1.2.
Replace ’f’ and ’i’ by ’d’ and ’q’, respectively.

(b)* Find the exponent of each float. The machine

typically includes a bias to the exponent. What is the
bias?

E1.5 Derive an expression for the area of a regular n-sided

polygon inside a unit circle. Approximate the area of the
unit circle by the area of the polygon, and calculate the
truncation error for the n-values listed in Table 1.1.

PROJECTS
P1.1 Write a program to convert a floating point number to its
bit pattern. The program should take the input from the
user and print out the bit pattern (sign, and the rest in
groups of 4 bits) for the float, assuming single precision.
Test your program with the results shown in Table 1.2.
See if you can break the program into logical modules.
P1.2 Consider the roots of the quadratic equation ax2 + bx + c
= 0,

(a) Write a program to calculate the roots, taking a, b, c

as input from the user, and printing out the answer to the
 screen. Choose any of the equivalent expressions above.
Test it on the following numbers:

Are the results correct for all cases? What kind of

numerical error is responsible?

(b) To control the error, modify your program so that it

correctly computes the roots to within machine precision,
no matter the values of a, b, and c.

P1.3 We showed that the recursion relation (1.5) is unstable in

the upward direction for . Thus, it

should be stable in the downward direction, yn−1 = −yn +

yn+1.

Iterate the downward recursion for N times, say N = 50,

starting with arbitrary seeds, e.g., y50 = 0 and y49 = 1.0.

Normalize your series so because the seeds

are arbitrary. Compare with the exact results . Also

plot the relative error as in Figure 1.2. Discuss why it

works.
 Since the downward recursion is equally valid for the

golden mean , the series obtained above

should also be applicable to ϕn. Calculate and plot the

relative error on the same graph. Why does it not work?
Given that the seeds are the same, how does the
algorithm “know” what series to converge to?

.A Floating point representation

Fixed point numbers
We are familiar with bit patterns representing integers. They
can also be used to represent fractional numbers, if we imagine
a fixed point placed in front and interpret the n-th bit as a
fraction 2−n.

For example, consider the n-bit binary pattern Xbin given by

where bi is either 0 or 1. Interpreted as above, the decimal

fraction number Xdec can be written as
The range of Xdec is 0 ≤ Xdec ≤ 1 − 2−n, and the smallest
increment representable is 2−n. By increasing the number of
bits n, the increment will be reduced, and finer fractional
values, or greater precision, can be achieved. It is also possible
to shift the fixed point to different positions to represent a
mixture of whole numbers plus fractions. In addition, by
adding a sign bit, negative values can be included using two’s
complementary binary representation.

Floating point numbers

Floating point numbers are natural extensions of fixed point
numbers. As Eq. (1.1) shows, a nonzero real number f can be
decomposed as f = M × 2E where the mantissa M is a fraction
between , and the exponent E = ceiling(log2|f|), the
smallest integer above log2|f|. For example, the number 0.5 has
a mantissa M = 0.5 and exponent 0, and 1.0 has the same
mantissa but a different exponent of 1.

Typically, a floating point number is stored in a finite

number of bits. For example, single precision floats use 32 bits
(4 bytes) as

including 1 bit for the sign, 8 bits for the exponent interpreted
as a signed integer between [−128, 127], and 23 bits for the
mantissa as a fraction according to Eq. (1.6). Since the
mantissa has a lower bound , the first bit (most
significant) of the mantissa in Eq. (1.8) is always 1, and by
convention, it is normally not stored explicitly (so-called
phantom bit). The mantissa actually represents the next 23
bits, for an effective 24-bit length. This means that single
precision floats are accurate to ∈ = 2−24 ~ 6 × 10−8, or about 7
digits. The value ∈ is called the machine accuracy in Eq. (1.2).
The range of single precision floats is [2−128, 2127], or roughly
10−38 to 1038.

Double precision floats, the default in Python, use 64 bits (8

bytes): 1 bit for the sign, 11 bits for the exponent, and 52 bits for
the mantissa (53 effective bits). The machine accuracy for
double precision is ∈ ~ 10−16 (15 digits), and the range is about
10−308 to 10308. Results exceeding the range will cause under-
or over-flow.

We can write a program to calculate the mantissa binary

pattern for a given float (Project P1.1). However, like the earlier
case about the exponent and mantissa (Section 1.2.1), there is
an easier way provided by Python via the struct conversion
module. The code below demonstrates this.

In [1]: import struct as st

fpack=st.pack( ’f’, 6.e−8) # pack float to bytes
fint=st.unpack( ’i’, fpack)[0] # unpack to int
bin(fint)[−23:] # mantissa bits
Out[1]: ’00000001101100101011001’
 This code converts a float (6 × 10−8 in the example) to a
byte pattern (string) in memory, which is subsequently
converted to an integer. The mantissa is contained in the last
23 bits of the integer in binary representation, excluding the
phantom bit. Several other examples including the phantom bit
are listed in Table 1.2. The decimal exponent and the mantissa
are obtained from frexp() explained earlier (Section 1.2.1).

Table 1.2: The exponent E and mantissa M of single-precision floats.

When two floats are to be added, the exponent of the

smaller number is adjusted upward so it is equal to the
exponent of the larger number, and the remainders are added.
For example, suppose we wish to add 1 + 6 × 10−8. Using the E
and M values from Table 1.2, we have

In machine operation, the second mantissa in the parenthesis

in Eq. (1.9) will be shifted to the right 24 times,7 with the most
significant bit (left most) replaced by 0 each time. In single
precision, the effective mantissa is zero after the shifts since all
the bits are flushed to the right. Consequently, the result from
Eq. (1.9) is (numerical, not mathematical)
 We see from the example above that, in general, addition of
two floats whose relative magnitudes differ by more than the
machine accuracy will cause the smaller float to be neglected,
and the result is equal to the larger float. Even when the floats
are within the machine accuracy, there is usually a loss of
accuracy due to the flushed bits. This is the round-off error.
Round-off errors also occur in other arithmetic operations even
if no shifting is involved, because the finite mantissa may not
be sufficient to hold the resultant answer. In particular,
cancellation error from the subtraction of two floats within
close proximity of each other could lead to a complete loss of
significant digits.

.B Python installation
In addition to Python 2.7x, we need the following four
packages: NumPy, SciPy, Matplotlib, and VPython. Optional
packages include IPython, SymPy (both recommended), and
IVisual. IPython is a user-friendly Python shell (see below).
SymPy, a symbolic mathematics library for Python, is not used
in our numerical computation, but is handy nonetheless for
many tasks such as algebra, exercises (see Exercise E2.7 and
Exercise E3.10), etc.

Python distributions
The easiest way to get a full installation up and running quickly
is through all-in-one distributions such as:

Anaconda, http://continuum.io/downloads;
Canopy, https://www.enthought.com/products/canopy/;
Python(x,y), https://code.google.com/p/pythonxy/ (Windows
only);
WinPython, http://sourceforge.net/projects/winpython/.

Each distribution contains all the required packages (and

more) listed above, except Anaconda and Canopy which do not
include VPython as of this writing. To add VPython to
Anaconda, enter the following conda command

$ conda install −c mwcraig vpython

This will install VPython built for Anaconda by Matt Craig on

both Mac OS and Windows. If you are using Canopy on the
Mac OS, there is a ready-made binary installer at vpython.org.

The Python(x,y) and WinPython distributions do include

VPython but are for Windows only. WinPython is also portable.

Custom installation
In general, use binary installers if possible, and stick with one
version (32- or 64-bit) consistently. All required packages have
binary installers at their web sites built for the official
python.org Python (Table 1.3). Binaries built for other Python
installations including all-in-one distributions are usually not
compatible. The online resource of the book has updated
installation instructions and the needed binaries for Windows
at:

You can also use the pip tool to install packages from
Python Package Index, https://pypi.python.org/, for any
platform including Linux. The typical command line is $ pip
install xyz for installing package xyz. Note that pip is in the
Scripts directory of a Python installation, so make sure it is in
the command path (C:\Python27\Scripts\ on Windows).

The pip tool can also install packages in the “wheel” format,
e.g., $ pip install xyz.whl. For Windows, many binaries built
by Christoph Gohlke for the official Python including
Matplotlib dependencies can be found at

www.lfd.uci.edu/~gohlke/pythonlibs/ (referred to as UCI).

Table 1.3 lists the recommended order to install only the

needed packages individually (file names are for Windows or
Mac at the time of writing, always use the most current
versions). Check each package’s installation instructions, and
test the installed package before moving on to the next one.
When running into a brick wall, ask the instructor or colleagues
for help.

Table 1.3: Package installation guide (W=Windows, M=Mac OS).

Package Files to download or method

Python
python.org W: python-2.7.8.msi

M: python-2.7.8-macosx10.6.dmg

pip Download get-pip.py from the

pip.pypa.io site, then run it:
$ python get-pip.py (from a
command terminal)
VPython
vpython.org W: VPython-Win-32-Py2.7-6.10.exe

M: VPython-Mac-Py2.7-6.10.dmg

NumPy
numpy.org W: numpy-1.8.2-win32-superpack-
python2.7.exe

M: $ pip install numpy

This can be skipped since VPython

includes NumPy.

SciPy
scipy.org W: scipy-0.14.0-win32-superpack-
python2.7.exe

M: scipy-0.14.0-py2.7-python.org-
macosx10.6.dmg

Matplotlib
matplotlib.org W: matplotlib-1.4.0.win32-
py2.7.exe; requires python-
dateutil, pyparsing and six
(binaries at UCI)

M: matplotlib-1.3.1-py2.7-
python.org-macosx10.6.dmg

IPython
ipython.org W: ipython-2.4.0-py27-none-
any.whl (from UCI)

W or M: $ pip install ipython

SymPy
sympy.org W: sympy-0.7.5.win32.exe

M: $ pip install sympy

IVisual Install from the command terminal

python.org/pypi/IVisual $ pip install ivisual
 Optional. Some VPython codes may
not run properly.

To check the installed packages, issue this from a command

terminal,

$ pip list

or start IDLE and enter the following

>>> help( ’modules’)

This will list the locally installed packages. Use pip to install
missing dependencies (if any) or other packages.

Running programs
If you have installed the required packages successfully,
congratulations! You are about to enter the exciting world of
simulations, and you can now run all the programs discussed in
the book.

The most direct way to run a program is from a command

terminal. For instance, the following will run Program 1.1,
motion.py8
 $ python motion.py

One can also navigate to the folder of source programs, and

click on them. Programs can be prepared with a text editor and
be executed this way.

To run Python interactively, we can use IPython which is an

interactive shell for Python. It can act like an interactive
calculator, and also integrate codes and other media like
graphics in the notebook mode running in a browser (see
Figure 1.4). IPython has many other useful features, including
online help and tab completion. Start IPython (in Scripts path)
in either the standard line mode, or the notebook mode
(without []),

$ ipython [notebook]

Once in IPython, enter statements line by line (in notebook

mode, press Shift Enter at the end of each input cell to execute
the cell). For example, you can test SymPy with

In [1]: %matplotlib inline

In [2]: from sympy import *
x = symbols( ’x’)
plot(sin (x)/x)
 and you should see the same output as Figure 1.4.

We can also load an existing program in IPython

In [1]: load motion

This will load motion.py, Program 1.1, to be edited or executed.

An alternative to IPython is the standard Python integrated

development environment (IDLE), with an editor and a
debugger. From IDLE, we can write a new program, open an
existing one, and run it from a menu-driven system.

VPython also comes with its own shell called VIDLE. It

offers several improvements over IDLE, including more
graceful handling of exceptions when Matplotlib and VPython
functions are used simultaneously. For maximum reliability,
run VPython programs from the command line, or from IDLE
or VIDLE.9

.C The Matplotlib plot function

The plot() function from Matplotlib produces line graphs. It is
invoked by
 plt .plot(x, y, string)

where x and y are the data set, and string is an optional string
specifying the color, line style, or marker type of the curve.
There are also many other optional arguments. The string is
normally a three-character field, ’CMS’, for color, marker, and
line style, respectively. Refer to Table 1.4 for common codes.
The order of the character in the string is unimportant. For
instance, the string ’g-o’ means a green solid line with filled
circle markers.

Table 1.4: Common line properties for the Matplotlib plot()

function.

Axis labels as well as text can be added. For example,

plt.text(xloc, yloc, ’message’, rotation=45)

would put the “message” at (xloc, yloc), and rotated 45°

counterclockwise. Often, the font size looks fine on the screen,
 but may be too small for print after resizing. Such parameters
can be specified via the optional arguments, or via
configuration records. For example, the following code would
change font size, axis and line thickness, frame spacing, etc.

import matplotlib.pyplot as plt

plt. rc( ’lines’,linewidth=2) # change line style, fonts
plt. rc( ’font’,size=16) # one group at a time
plt. rc( ’axes’,linewidth=2)
plt. rc( ’figure.subplot’,right=0.96) # frame spacing
plt. rc( ’figure.subplot’,top=0.96)
plt. rc( ’figure.subplot’,left=0.1)

Many figures in the book are produced with these

parameters. For convenience, the code can be included in a file
(say rcpara.py) and imported before use.

.D Basic NumPy array operations

We give a very brief introduction to the usage of NumPy arrays
(also called ndarrays). There are many other powerful features
that can be understood through use, examples (see programs
mentioned at the end of this section), and the NumPy
documentation.10 Remember, import NumPy before use, and
online help is available in IPython or IDLE on any installed
packages. To get help on np.array for instance, issue
 In [1]: import numpy as np
In [2]: help(np.array)

Array creation, indexing, and slicing

A NumPy array can be created from any array-like objects:

In [3] np.array( [1., 2., 3.] )

Out[3]: array ( [ 1. , 2. , 3. ] )
In [4]: a=np. arange(5)
a
Out[4]: array([0, 1, 2, 3, 4])

The NumPy arange function is just like the Python range

except it returns an ndarray. The data type (e.g., float or
integer) is derived from the assigned objects or specified with
dtype. NumPy has other functions for initializing arrays, e.g.,
zeros(5) and ones(5) will create two, 5-element arrays filled
with zeros and ones, respectively, with float as the default type.
The array a will be continuously updated and used in the
following examples using IPython notebook mode.

Array elements can be accessed by an index, either positive

or negative,
 In [5]: a[2]
Out[5]: 2
In [6]: a[−2]
Out[6]: 3

A positive index counts from the beginning of the array.

Remember that array indices start from zero, thus a[0] is the
first element (0), and a[2] the third element (2). A negative
index is allowed, which counts backward from the end. So
a[-1] refers to the last element (4), a[-2] the second last
element (3), etc.

Slicing is triggered by the colon [:] symbol, and is an

important part of using NumPy arrays. It normally takes the
form a[start:end:step], producing an array from start up to
but not including end, in steps step. If start or end are absent,
the first or the last elements (inclusive), respectively, are
assumed. If step is omitted, it defaults to 1. A negative step
means going in the reverse direction of the array. Some basic
slicing examples are:

In [7]: a [ : ]
Out[7]: array([0, 1, 2, 3, 4])
In [8]: a[1:3]
Out[8]: array([1, 2])
In [9]: a[:3]
Out[9]: array([0, 1, 2])
In [10]: a[2:]
 Out[10]: array([2, 3, 4])
In [11]: a[::−1]
Out[11]: array([4, 3, 2, 1, 0])

Note that a[:] returns the whole array, but a[] would be
illegal.

Array manipulations
Elements can be inserted or removed from ndarrays. The
following example inserts a “9” in front of the original index “1”
of a, and deletes the element at index “2”:

In [12]: np.insert(a,1,9)
Out[12]: array([0, 9, 1, 2, 3, 4])
In [13]: np.delete(a,2)
Out[13]: array([0, 1, 3, 4])

Each time, a new array is created, and the original array a is

not modified.

Operations on an array occur in an element-wise, vectorized

fashion. For example:

In [14]: 2*a
Out[14]: array([0, 2, 4, 6, 8])
 In [15]: a+3
Out[15]: array([3, 4, 5, 6, 7])

When the scalar “3” is added to the array, it is cast into an array
of the same dimension (called broadcasting) internally and
then the arrays are added element by element. Since no explicit
looping is invoked in Python, the speed gain is considerable for
larger arrays. For explicit looping, Python lists are faster.

Elements can also be selectively changed via slicing. The

following assigns two elements to new values:

In [16]: a[1:3]=[−1, −2]

a
Out[16]: array([ 0, −1, −2, 3, 4])

Basic slicing like above returns a “view” of the array, i.e., it

points to the elements within the same array. In contrast,
slicing of a Python list returns a copy.

If we wish to work with a copy of an ndarray, we can use the

copy() function. The following copies part of the array, and
changes the copy without affecting the original array:

In [17]: b=np.copy(a[1:3])
b
 Out[17]: array([−1, −2])
In [18]: b[:]=[−3, −4]
b
Out[18]: array([−3, −4])
In [19]: a
Out[19]: array([ 0, −1, −2, 3, 4])

Advanced indexing
Copies of ndarrays can also be obtained with advanced
indexing. This occurs when the index is not simply an integer.
The following code obtains a copy with a list index which
triggers advanced indexing. Again, the source array is not
affected:

In [20]: c=a[ [ 1, 2 ] ]
c
Out[20]: array([−1, −2])
In [21]: c[:]=[−5, −6]
c
Out[21]: array([−5, −6])
In [22]: a
Out[22]: array([ 0, −1, −2, 3, 4])

It is also possible to select elements with a boolean or truth

array. A truth array can be created by logical relations such as
<, >, ==, etc. For example,
 In [23]: a>0
Out[23]: array ([False, False, False, True, True], dtype=bool)
In [24]: a==0
Out[24]: array ([True, False, False, False, False], dtype=bool)

The statement a>0 tests every element of a against 0, and sets

the corresponding indices to True if they are greater than zero,
and False otherwise.

When a truth array is an index to another array, only the

true elements are taken. This is sometimes called fancy
indexing in NumPy. The following are some examples of truth
array indexing:

In [25]: a[a>0]
Out[25]: array([3, 4])
In [26]: a[a<0]
Out[26]: array([−1, −2])
In [27]: a[a==0]
Out[27]: array([0])

The positive elements are selected in the first instance, negative

ones in the second instance. Note that the third instance has
only a single element, but it is still returned as a one-element
array.
 Like assignment with slicing, we can also selectively assign
elements with a truth array:

In [28]: a[a<0]=0
a
Out[28]: array([0, 0, 0, 3, 4])

All elements of the array less than zero are set to 0 in the above
example.

Advanced indexing can be very powerful but also

convoluted at the same time, making the code harder to
understand. Unless necessary, we prefer the direct approach,
e.g., to obtain array copies with the copy() function.

Multidimensional arrays
Multidimensional arrays can be created and handled similarly.
For instance, zeros((2,5)) will create a 2-row by 5-column
array filled with zeros. Each dimension is called an axis, so axis
0 refers to going down the rows, and axis 1 to going across the
columns.

Below, we create a two-dimensional array, and find its

dimensions via the shape attribute:
 In [29]: d=np. array([[11,12,13],[21,22,23],[31,32,33]])
d
Out[29]: array([ [11, 12, 13],
[21, 22, 23],
[31, 32, 33] ])
In [30]: d. shape
Out[30]: (3, 3)

The array can be accessed with two indices and sliced like
before, but the results may be scalars, one-dimensional arrays
(row or column vectors), or two-dimensional sub-arrays:

In [31]: d[1,2]
Out[31]: 23
In [32]: d[0]
Out[32]: array([11, 12, 13])
In [33]: d[ : , 1 ]
Out[33]: array([12, 22, 32])
In [34]: d[ 1 :, : 2 ]
Out[34]: array([ [21, 22],
[31, 32]])

In this example, d[1,2] refers to the element at the second row

and third column, d[0] the first (entire) row, and d[:,1] the
second column, both 1D arrays. But d[1:,:2] refers to the
second row and above, and all columns up to, but not
including, the third column. The result is a 2D sub-array. See
Program 7.2 for actual uses.
 Rows or columns can be inserted or deleted as well. The
following deletes the second row along axis 0 and the third
column along axis 1, respectively:

In [35]: np. delete(d,1, axis=0)

Out[35]: array([ [11, 12, 13],
[31, 32, 33] ])
In [36]: np. delete(d,2, axis=1)
Out[36]: array([ [11, 12],
[21, 22],
[31, 32]])

Operations on ndarrays with advanced indexing can be very

efficient and elegant. For instance, we can swap the first and
third columns simply by:

In [37]: d[ :,[0,2]]=d[ :, [2,0] ]

d
Out[37]: array([ [13, 12, 11],
[23, 22, 21],
[33, 32, 31]])

The target d[:,[0,2]] refers to the first and third columns of all
rows, and the source d[:,[2,0]] specifies the third and first
columns of all rows. Slicing and advanced indexing work
together to enable us to swap the columns in a single
statement. See Program 9.9 for examples in actual codes.
Universal functions
NumPy arrays of the same shape can be added, subtracted,
multiplied, or divided, all element-wise. Python operators like
+, −, *, /, **, etc., are context-aware. When acting on ndarrays,
operations are carried out element-wise transparently and
automatically, irrespective of the type of operands as long as
they can be broadcast into an ndarrary.

In [38]: x=np. array([1.,2.,3.,4])

x−2
Out[38]: array([−1., 0., 1., 2.])
In [39]: x/2.
Out[39]: array([ 0.5, 1., 1.5, 2. ])
In [40]: x*x
Out[40]: array([ 1., 4., 9., 16.])

In the first two cases, the second operand is broadcast to an

ndarrary [2., 2., 2., 2.] before element-wise
subtraction/division is carried out. In the third case, no
broadcasting is necessary, and the elements are multiplied one
to one, amounting to x**2.

In NumPy the concept of element-wise operations is also

extended to functions, called universal functions (ufunc). They
operate on scalar and ndarray inputs transparently. For
example, with the same x as above,
 In [41]: np. sqrt (x)
Out[41]: array ([ 1. , 1.41421356, 1.73205081, 2. ])
In [42]: np.maximum(x,x[::−1])
Out[42]: array([ 4., 3., 3., 4.])

In the first instance, the square root of every element is taken.

Next, each element of x is compared to the element of the
reversed array, and the greater one is picked. Using ufuncs with
ndarrays involves implicit looping. It it not only simple and
elegant, but also efficient: the actual coding or computation is
often done in compiled languages like CPython or C/C++ for
increased speed.

Like sqrt and maximum, most NumPy functions are ufuncs.

Thus, any function we define using NumPy versions of
mathematical functions such as sqrt is also a universal
function.11 See potential() in Program 4.6 for an example.

We have described very briefly some features of the NumPy

library. There are many other operations and advanced
features on NumPy arrays, including concatenation, stacking,
sorting, and logical operations, etc. Numerous examples are
found and explained in the accompanying programs,
particularly starting from Chapter 4 such as Programs 4.6, 6.8,
7.3, 8.2, 9.7 and S:6.1, to name but a few.
 1
See http://www.faculty.umassd.edu/j.wang/vp/movie.htm or the Digital Online
Companion site for sample movies.

2
Another form for Eq. (1.4) is ϕ/1 = (ϕ+1)/ϕ, meaning that the ratio ϕ to 1 is the same
as ϕ + 1 to ϕ. This “perfect” ratio has inspired artists and architects in history.

3
A quantitative von Neumann stability analysis is discussed later (Section 6.5.4).

4
There seems to be a specialized library for any task. Want to fly? Try import
antigravity.

5
Some may prefer to use more descriptive names. We try to use short names if they do
not lead to confusion. Here, ta and xa designate time and position arrays.

6
If the animation does not show in the notebook, restarting the IPython kernel usually
fixes it.

7
In binary arithmetic, dividing by 2 is equivalent to shifting the bits to the right by one
position.

8
On Windows, the keyword “python” can be omitted. Any program entered at command
line (cmd.exe) with .py extension should automatically invoke Python.

9
One can also run most VPython programs without VPython installation in a web
browser via GlowScript (glowscript.org), or capture output to a movie (see vpython.org).

10
There are also online tutorials, e.g., scipy-lectures.github.io.

11
If necessary, we can also make a non-ufunc to a unfunc by vectorizing it with a NumPy
module named, well, you guessed it: vectorize().
Chapter 2
Free fall and ordinary
differential equations
Many problems in physics and engineering are expressed in the
form of either ordinary or partial differential equations
(denoted ODEs, or PDEs). To a large part, this seems to be due
to the fact that mathematical descriptions of problems of
interest, especially the complex ones, simplify significantly
when we consider only infinitesimal changes of the variables
involved. The result is the emergence of differential equations
of one form or another. Take Newton’s second law F = ma, for
example. Since the acceleration a is the second derivative of
position with respect to time, Newtonian mechanics always
involves differential equations, be it free fall on earth, or star
motion in the sky. Another example is wave motion such as a
vibrating string. To describe waves, we must treat the space
and time variables independently. This leads to wave equations
in the form of PDEs. Therefore, solving differential equations is
a major undertaking in scientific computing.
 Some ODEs, like those for free fall, yield analytic solutions
in closed-form, known functions. Others do not, so numerical
solutions are needed. Because ODEs are among the single,
most common class of mathematical relations used across
science and engineering including this book, we will devote this
chapter to the discussion of methods of numerical solutions
suitable to the problems we study (PDEs will be discussed
separately starting from Chapter 6). The ODE solvers
developed here will be used throughout this book. We will first
study one ODE at a time, and generalize it to a set of coupled
ODEs. The collection of these methods serves as a toolbox that
will be applied to computer simulations in subsequent chapters
where analytic solutions are rarely known.

If you are familiar with ODE solvers or wish to save time,

you can skip Sections 2.1 to 2.3. Also see Program 2.10 (Section
2.B) which serves as a blackbox ODE wrapper for a general
integrator from SciPy.

.1 Free fall with Euler’s method

Let us begin with a very familiar problem, free fall. We will use
it as the main example in this chapter. This is useful since it
allows us to test the methods against exact solutions.

2.1.1 A FIRST EXAMPLE: FREE FALL

To get a feel for our approach to modeling and visualization
first, we start with the example of a bouncing ball: a ball is let
go from a certain height above the floor, and upon reaching the
floor it reverses its velocity and bounces right up without loss
of speed. Thus, between the contacts with the floor, the ball is
in constant free fall motion. The following program simulates
this motion.

Program listing 2.1: A bouncing ball ( bouncing_ball.py)

import visual as vp # get VPython modules for animation

2

# draw the ball at (0,5,0) and a thin floor at (0,−5,0)

4 ball = vp.sphere(pos=(0,5,0), radius=1, color=vp.color.yellow) # ball
floor = vp.box(pos=(0, −5,0), length=8, height=0.2, width=4) # floor
6

h = 0.01 # time step size

8 v = 0.0 # initial velocity
while True: # loop forever
10 vp.rate(400) # limit animation rate to 400 loops/sec
ball.pos.y = ball.pos.y + v*h # update y position
12 if ball.pos.y > floor.y + ball.radius:
v = v − 9.8*h # above floor, update velocity
14 else:
v=−v # below floor, reverse velocity
 Figure 2.1: Snapshots of a bouncing ball from Program 2.1 using
VPython.

If the program has been entered correctly and everything

goes smoothly (see Section 1.B on running programs), we
should see the free fall animation, with the ball bouncing up
and down continuously. See the motion from different angles
by dragging with the right mouse button (camera) or both
buttons (zoom). Several snapshots of the motion are shown in
Figure 2.1.

Program 2.1 consists of about a dozen working lines, with

brief comments after each line. Below, we explain the program
flow. The first line of the program imports the VPython
package. The next two lines (4 and 5) create the ball and the
floor using the “sphere” and “box” objects of VPython,
respectively. Each object accepts a set of parameters (or
attributes) such as the position, radius, color, etc. For instance,
the position vector is specified by a point (x, y, z). The objects
are assigned names, ball and floor in this case, for later
reference. VPython objects can have more attributes than
specified. Those unspecified will be assigned default values
automatically. Once created, the objects can be modified by
changing the attributes. For example, we can “move” the ball to
a new position (a, b, c) via ball.pos=(a,b,c), or ball.pos.x=a if
we only want to move in the x direction.

The rest of the program sets the step size and the initial
velocity (lines 7 and 8), then enters the while loop. The “rate”
function (line 10) inside the loop is another VPython module
that limits the rate of animation so it will not run too fast on
fast machines. The position of the ball is updated by changing
the variable ball.pos.y by an amount v*h each time (line 11).
Since it is an attribute of the sphere object assigned to ball,
VPython, running in the background and keeping track of all
attributes of objects created, will automatically redraw the ball
at the new position, creating the animated effect. The last if-
else statement checks to see if the ball is above the floor: if it is,
the velocity is updated according to free fall with constant
acceleration −9.8 m/s2 (line 13); otherwise, it is reversed (line
15).

After understanding the program flow, you should

experiment a little and play with different parameters to see
how they affect the animation. For example, change the ball to
your favorite color (mine is green), give it a nonzero initial
velocity (positive and negative), a slower or faster frame rate,
or even a different value of g. In the problems ahead, the
programs may grow to more lines (not too many more in most
cases, I assure you), and the structure may be a bit more
complex, but the basic elements and approach will remain the
same.

This object oriented approach of VPython makes it easy for

us to create 3D animation and data visualization effects in our
programs. All we need to do is to change the attributes such as
the positions of the objects in motion. We do not have to worry
about drawing or moving the objects. It allows us to focus on
the task of modeling. We will use this to our fullest advantage
throughout. If you do not use VPython, you can use Matplotlib
output (see Exercise E2.1 and Program 2.2) or save the position
and time data to a file (Section 1.3.1) for separate analysis.

2.1.2 EULER’S METHOD

The free fall example above calculates the changes in position
and velocity of the ball with a method as Δy = vΔt and Δv =
aΔt, respectively, where v = dy/dt is the velocity, a = dv/dt =
−g the acceleration, Δt the step size ( h), and g = −9.8 m/s2 the
gravitational acceleration. It is convenient to describe free fall
in terms of differential equations as

with the analytic solutions given by

where the initial position and velocity are assumed to be y(0) =
0 and v(0) = v0, respectively. Accordingly, the method of
calculating the changes in Program 2.1 is first order in Δt, also
known as Euler’s method.

To better understand Euler’s method, including an

estimation of the error term, we work with a single ordinary
differential equation for now. This simplifies the discussion
without loss of generality. The general form of an ODE is

where y is the solution we seek, and f(y, t) is a well-behaved

but otherwise arbitrary function.

The problem at hand now becomes an initial value problem,

i.e., given the initial condition y(t = 0) = y0, find y at all later
times. More generally, assuming we know y(t), we could obtain
y(t + Δt) from the Taylor series

It is understood that in Eq. (2.4) the derivatives are carried

out at t. Looking at Eq. (2.4), the easiest thing to do is to drop
all orders higher than Δt, to yield a first order solution as
Here is basically the leading-order
error correction that we throw away.1 This is also called the
local truncation error. A method is called order n if it is
accurate to order (Δt)n with an error of order O[(Δt)n+1]. Hence
Eq. (2.5) is order 1, or first order. For linear functions, d2y/dt2
= 0, and the error term vanishes. Therefore we expect this first
order method to be exact, numerically speaking, for linear
solutions.

Given its first order accuracy and dropping the

approximation sign, we can write down the classic Euler’s
method as

Note that we have replaced dy/dt with f(y, t) by virtue of Eq.

(2.3). It is to be evaluated at t. Therefore Euler’s method is said
to be explicit and self-starting, i.e., given y(0), we know y(Δt),
y(2Δt), y(3Δt), etc.

So here is Euler’s algorithm, given y(t = 0) and h = Δt as the

time step:
 How accurate is Euler’s method as applied to the free fall
problem? Since we know the exact solutions, we can readily
answer this question by comparing the numerical and analytic
results for the position y. From Eqs. (2.1) and (2.2), we have

Below is the program that solves the above equation with

Euler’s method (2.7) and plots the results.

Program listing 2.2: Free fall with Euler’s method

( freefall_plot.py)

import matplotlib.pyplot as plt # get matplotlib plot functions

2

g, h = 9.8, 0.02 # g, step size. Multiple assignments in one line

4 y, v0 = 0.0, 5.0 # initial position, velocity
ta, ya = [], [] # time and position arrays for plotting
6 t, yb = 0.0, [] # yb[] holds analytic solution

8 while t<1.0: # loop for one second

ta.append(t) # record time and position
10 ya.append(y)
yb.append(v0*t−g*t*t/2.0) # analytic solution
12 v = v0 − g*t # calc v(t)
y = y + v*h # Euler’s method
14 t=t+h

16 plt.figure () # start a figure

plt.plot(ta,ya, ta,yb,’--’) # draw 2nd curve as dashed
18 plt.xlabel(’ t ( s)’) # add labels
plt.ylabel(’ y ( m)’)
20 plt.show() # show figure
 The program flow is very similar to Program 1.1. It first
imports the Matplotlib library, then sets parameters and initial
conditions. As before, empty lists such as ta, ya, and yb are
created to hold intermediate values for later plotting. Inside the
loop, elements are added to lists via the append method in the
first three statements. The rest of the loop updates the position
with Euler’s method.

When presenting many data points, instead of printing

numbers, graphs are preferred. This is done after the main
loop. We plot two curves with the plot function (Section 1.C):
the first pair ta,ya plots the numerical (Euler) y-t results, and
the second pair ta,yb plots the analytic results. An optional
string argument immediately after the data arrays specifies the
line style, and a dashed line is drawn with ’--’ in this case (see
Table 1.4). The plots are shown in Figure 2.2.
 Figure 2.2: The position-time curves of free fall with Euler’s method
(solid/blue) and the analytic result (dashed/green).

Starting with the same value y = 0 at t = 0, the numerical

and analytic results stay close to each other in the beginning
(Figure 2.2). But as time increases, the difference between
them grows linearly. The absolute error, which measures the
global error at a given t, is doubled from t ~ 0.5 to t ~ 1 s.
Though the exact global error is unknown, we can roughly
estimate it to be the cumulation of local error. As stated earlier,
the local truncation error of Euler’s method is second order,
O(h2). The number of steps to reach t is proportional to 1/h, so
we expect the global error to be ~ O(h2) × h−1 ∝ O(h). This
means that if h is halved, the global error should be reduced by
half. Project P2.3 shows that this is indeed the case.

Aside from the error, the peak position (around t ~ 0.5 s) is

shifted as well, with the numerical peak occurring at a slightly
later time. Since Euler’s method is exact for linear solutions,
error in the numerical result is expected because the analytic
solution (2.2) is quadratic. Nonetheless, Euler’s method does
yield an overall correct parabolic shape for free fall.

In addition to accuracy, another important factor in the

consideration of an algorithm is its efficiency. The efficiency of
an ODE solver is usually measured in terms of the number of
function calls made to f(y, t) per step. The cost of computation
of Euler’s method is one function evaluation per time step. This
 is the minimum number we would expect, so it is apparently
very efficient, 100% in fact.

Euler’s method is an instructive example to introduce us to

numerical solutions of ODEs, but is very limited for practical
use. For instance, to try to reduce the error, we might use a
smaller step size h. This partially works, but is very limited. To
reach a certain time t, we need more steps, using more
computing time and becoming less efficient, and partially
defeating our purpose. Furthermore, there is a fundamental
limit to the lower bound of h due to machine accuracy. More
accurate methods with reasonable efficiency would be needed
for problems whose solutions may not be as smooth as free fall.
We examine those methods next.

.2 The Runge-Kutta (RK) methods

In this section, we introduce a family of ODE solvers known as
the Runge-Kutta methods in order to improve accuracy over
Euler’s method. We will discuss a second order method and a
fourth order one.

2.2.1 SECOND ORDER RK METHOD: RK2

Looking at Euler’s method (2.6), we notice that the use of the
derivative f(y, t) is not quite symmetrical, i.e., at each step, only
information at the beginning of the step is used, not at the end.
This asymmetry is illustrated graphically in Figure 2.3(a). To
arrive at the second step (t1), only the derivative at t0 is used
(similarly for subsequent steps).

Figure 2.3: Graphical illustration of Euler’s method (a), and the

second order Runge-Kutta method (b).

Here is where we could make an improvement: to make a

more symmetrical use of the information at the beginning and
at the end of each step. The idea is shown schematically in
Figure 2.3(b). If we used an average derivative at the middle
point (dash circles), which is half-way between t0 and t1, we
just might get a better approximation of the solution at t1. This
is exactly the rationale behind the second order Runge-Kutta
method, which we will call RK2 from now on.

To see how this works out, let us use the Taylor series (2.4)
again, first at the beginning of the step as (using same notation
as in last section)
and then at the end of the step as

In Eqs. (2.9) and (2.10), , and it is understood that the

derivatives are also evaluated at h/2, the midpoint.

Since all we know from Eq. (2.3) is the first derivative y′ =

f(y, t), we really don’t know the second (y″) or the third (y″′)
order derivatives in Eqs. (2.9) and (2.10) . Fortunately we can
eliminate y″ by subtracting both sides of Eqs. (2.9) and (2.10),

Cancellation helped us to get rid of the second derivative,

but the third (and other odd) derivative is still present. Since
our goal is to obtain second order accuracy, we can safely throw
away the term with the third derivative because it is a third
order correction h3. Doing so, we have

This is the RK2 method. Comparing Eq. (2.6) with (2.12),

we see they are very similar in form, except that for RK2 the
derivative is evaluated at h/2, the midpoint. Thus RK2 has
another name, the midpoint method. It is locally accurate to
second order. The global error is expected to scale as O(h2), so
it would decrease by a factor of 4 if h is halved (Project P2.3).
 Incidentally, Eq. (2.12) leads to the midpoint method for
numerical differentiation,

This is a commonly used method for finding derivatives

between two points.

Now only one question remains: how do we get ? We

cannot use Eq. (2.13) because we do not have y1 yet. According
to Eq. (2.3), we could also calculate it as follows

However, at this point, we only have the value y0 = y(0), not

y(h/2).

Consistent with second order accuracy, we only need to

be accurate to first order because of the additional h in the term
in Eq. (2.12). One way is to obtain y(h/2) from Euler’s
method, Eq. (2.6), and put it in Eq. (2.14) to get as

Combining Eqs. (2.12) and (2.15), we have the algorithm for

the RK2 method, given yn and tn = nh, to advance to tn+1 as
 In each step, the RK2 method makes two function
evaluations, one for k1 and another one for k2. Since it is second
order, two calls are the minimum expected, so it is still very
efficient and self-starting.

As it turns out, the RK2 method as expressed in Eq. (2.16) is

not unique. The RK2 family has an infinite number of ways to
achieve second order accuracy with the same efficiency. For
example, we could approximate as the average of f(y, t) at
the two end points, in which case the RK2 method reads

We leave the proof of Eq. (2.17) and another popular variation

to Exercise E2.2. We will use Eq. (2.16) as the RK2 method of
choice in our study.

2.2.2 FOURTH ORDER RK METHOD: RK4

The RK4 method is of fourth order accuracy, two orders higher
than RK2, but the basic idea is the same, i.e., using more of
available information at the end points of the step, plus two in
the middle. With proper weighting factors, we get higher order
cancellations (see Sec. 3.4 of Ref. [56]).

As with RK2, the RK4 method is not unique. Two common

forms are illustrated in Figure 2.4. In the Runge method
(Figure 2.4(a)), the derivative is evaluated four times in each
step, twice at the end points, and twice in the middle. The
mathematical proof that it is indeed of fourth order accuracy is
given elsewhere [78]. The final results – the Runge formula –
for the RK4 algorithm are

Figure 2.4: Illustration of the fourth order Runge method (a), and the
Kutta method (b).

The notations used in Eq. (2.18) are the same as in Eq. (2.16).
 Alternatively, we can divide the step size h into three
intervals, and evaluate the derivative at 0, h/3, 2h/3, and h
(Figure 2.4(b)). This leads to the Kutta formula

Both formulas work fine. We will stick with the simpler Runge
formula (2.18). The RK4 method is self-starting and efficient.
There are more elaborate methods around, but a basic RK4
with some step size adjustment as necessary will serve us well
in most situations. Except for certain cases such as
Hamiltonian systems to be discussed later, RK4 is the
recommended method as the general ODE solver in our
simulations. Of course, we will develop appropriate methods
for special problems we encounter going forward.

We might well ask, why stop at the fourth order RK4? Can
we not continue to higher orders? As it turns out, higher than
fourth order will require more function calls to f(y, t) than the
order number. For example, a fifth order Runge-Kutta method
would need six function calls (see Runge-Kutta-Felhberg
method, Section 2.3.2). So the RK4 is a natural place to stop.

2.2.3 APPLICATION TO FREE FALL

Now let us apply the Runge-Kutta methods to free fall in the
same manner as Euler’s method in Program 2.2. Starting with
RK2 first and according to Eq. (2.16), we need only to change
two lines of that program inside the while loop, from

v = v0 − g*t # calc v(t)

y = y + v*h # Euler’s method

to

……
k1 = h*(v0 − g*t) # k1, for clarity only, not needed
k2 = h*(v0 − g*(t+h/2.0)) # k2=f(y+k1/2, t+h/2)
y = y + k2 # RK2 method
……

The modified code works, but the way of putting everything

in one chunk is not a good idea for organizing programs in
terms of well-laid structure and clarity, or for reusing certain
parts. Following the modular revolution, we break the code
down into self-contained functions, or callable subroutines. For
RK2, we would like the function to take as input the initial
value y0 at t and advance it one time step to y1 at t + h. We code
it as follows.

Program listing 2.3: RK2 method ( rk2.py)

 1 def RK2(diffeq, y0, t, h):
””” RK2 method for ODEs:
3 Given y0 at t, returns y1 at t+h ”””
k1 = h*diffeq(y0, t) # get dy/dt at t first
5 k2 = h*diffeq(y0+0.5*k1, t + h/2.) # get dy/dt at t+h/2,
return y0 + k2 # calc. y1 = y(t+h)

The function diffeq(y,t) above is a generic name for the

user supplied function that returns the derivative, i.e., the
right-hand side (RHS) of Eq. (2.3), dy/dt = f(y, t). It looks like
an argument, but in Python it is the name of a user-defined
function (in other languages such as C/C++, it might be a
pointer to a function). For free fall, this function is, according
to Eq. (2.8),

def velocity (y, t): # returns the RHS of Eq. (2.8)

v0 = 0 # locally defined init value
v = v0 − 9.8*t
return v

Note that in the function above, the initial value v0 is

defined locally. Of course, one could also define it in a scope
accessible to the function, as a global variable for instance. But
the use of global variables that can be modified by other
functions should not be too casual in larger codes since it is
hard to track down potential bugs caused by them.
 With RK2() and velocity() thus defined, the modularized
code snippet looks like

……
y = RK2(velocity, y, t, h) # call RK2
……

The plots generated from running this program should be

similar to Figure 2.2, but the numerical and analytic results
should show no difference because the RK2 method is exact for
quadratic solutions. We will explore this in Project P2.1.

When RK2() is called with the function name velocity, the

program will match the generic function diffeq() inside to
velocity(), and any reference to diffeq() is redirected to
velocity() instead. This way, the program is much clearer
structurally. This is helpful in understanding what we are
doing, much easier, in fact, than to hard-wire the RK2 method
in the main body of codes. Furthermore, the RK2 method is
totally encapsulated (hidden) from the main code, so it can be
tested separately, and called from anywhere.

In addition, the user-supplied function can be changed to

suit the problem of the day. We could use the same RK2
module to solve any problem that is a single, first order ODE,
by supplying a different function diffeq(y,t).
 In an analogous way, the RK4 method, Eq. (2.18), can be
implemented as a standalone function that uses the same
diffeq(y,t), as given below.

Program listing 2.4: RK4 method ( rk4.py)

1 def RK4(diffeq, y0, t, h):

””” RK4 method for ODEs:
3 Given y0 at t, returns y1 at t+h
For a system of ODEs, diffeq must return a NumPy array”””
5 k1 = h*diffeq(y0, t) # dy/dt at t
k2 = h*diffeq(y0+0.5*k1, t + h/2.) # dy/dt at t+h/2
7 k3 = h*diffeq(y0+0.5*k2, t + h/2.) # dy/dt at t+h/2
k4 = h*diffeq(y0+k3, t + h) # dy/dt at t+h
9 return y0 + (k1+k4)/6.0 + (k2+k3)/3.0

For example, we can solve the nuclear decay problem this

way (Project P2.3). Though the RK2 and RK4 codes are
developed with a single ODE, they can be used for multiple
ODEs without change using vector NumPy arrays described
below.

.3 System of first order ODEs

With what we have now, we can solve any problem expressible
as a single, first order ordinary differential equation with
Euler’s method, RK2, or RK4. What about multiple, or even
higher order differential equations? As the main example, we
have treated free fall as a single ODE so far, assuming the
analytic formula for velocity for simplicity. But free fall actually
consists of two ODEs, Eq. (2.1), which come from this second
order equation

As we shall see, finding solutions to higher order ODEs is

not any more difficult than dealing with a single ODE. There
are just two steps involved, each is straightforward. First,
convert one (or more) second or higher order ODE into a set of
first order ODEs. Second, solve them using the methods we
have already discussed in this chapter.

2.3.1 CONVERSION FROM THE NTH TO THE

FIRST ORDER
Consider an ODE of the general form

Here we assume the highest order being n (do not confuse

derivative order with the order of accuracy of algorithms
discussed earlier), and the RHS function f contains all orders
up to n − 1. This is not too restrictive as most problems can be
formulated this way.
 To convert Eq. (2.21) into a set of first order ODEs, we
define some new variables as

We can express Eq. (2.22) as a recursive relationship

between the first derivative dxi/dt and xi+1 as

Running through the indices fully, we have

Equation (2.24) is what we wanted: a set of n, first-order

ODEs in terms of xi. The RHS of the first n − 1 equations is
trivial, it is just a simple assignment. The last equation is where
the real work is done in function f.

The above can be summarized by the following statement:

Any ODE higher than first order can be converted into a system
of n first-order ODEs, where n is the highest order derivative in
Eq. (2.21).
 The surrogate variables xi in Eqs. (2.22) and (2.23) are used
not just for convenience, they often have real physical meaning
as well. Again using the free fall example (2.20), we associate

and the set of ODEs is

The original Eq. (2.21) is of nth order. The solution will be

uniquely determined given n initial conditions, y, dy/dt, …, etc.
In terms of the converted Eq. (2.24), it means knowing the
values of xi, i = 1, 2, …, n. For free fall, n = 2, and this translates
into x1 for the initial position, and x2 for the initial velocity.

How do we deal with arbitrary n? The answer in one word:

vectors.

2.3.2 SOLUTION OF COUPLED ODES

We know now how to: (a) solve one single, first-order ODE;
and (b) reduce one or more ODEs of second or higher orders to
a set of first-order ones. We just need to find a way to solve (b)
in a way similar to solving (a).
 As it turns out, all the methods we have discussed so far
about (a) could be applied straightforwardly to (b), if we
generalize these methods using vector representation.

First, let us say we have a vector y that has n components

y[1] to y[n]. We denote it by the short-hand {y} which means a
collection of all its components as

Next, we consider a system of n ODEs similar to the set

(2.24) but more general in form as

In each of the n equations (2.28), the left-hand side (LHS)

is the derivative of one component y[i]. But on the RHS, the
function fi is understood to depend on possibly all other
components of y, i.e., with the short-hand notation (2.27),
fi({y}, t) = fi(y[1], y[2], …, y[n], t). Thus, the value of one
component y[i] depends on the values of all other n
components. This interdependency is the reason that Eq.
(2.28) is called a set of coupled equations. The coupled
equations must be solved together, meaning all the
components of {y} need to be updated at every time step, even
though in the end perhaps only a subset of the components are
desired or actually useful.
 If we compare Eq. (2.28) with (2.3), dy/dt = f(y, t), we see
that they are formally equivalent. The only difference is that the
first argument to the RHS function f(y, t) of Eq. (2.3) is
replaced by a vector fi({y}, t) in Eq. (2.28). This is purely
cosmetic, as we know how to deal with that, just plugging in
more arguments. What this means is that all the algorithms we
discussed, be it Euler’s, RK2, or RK4 methods, can be used for
a system of ODEs without modification. We only require that
diffeq return the derivatives in a NumPy (vector) array. In
fact, we can broaden the scope to include an array of vector
objects (2D or multidimensional NumPy arrays) returned by
diffeq (e.g., see Program 3.8, most codes in Chapter 4, and
Program 6.8).

Euler method
To illustrate this point, let us port Euler’s method, Eq. (2.7), for
a single ODE to a system of coupled ODEs, Eq. (2.28). Instead
of using yn, yn+1 for the start and end values in each step, we
adopt a slightly different notation as

Then, using vector notation, Euler’s method is simply

Note we had generalized {f} to mean a function vector.

Expanded out, Eq. (2.30) reads
This is the explicit formula for Euler’s method for a system of
ODEs. It is essentially one Euler’s method repeated n times for
all the components.

As an example that expressly ties it all together, consider

the set of equations (2.26) resulting from free fall (2.20),

We would write a subroutine implementing Eq. (2.31) called

Euler. To make it general, we would want to use a user
supplied subroutine that returns the derivatives fi({y}, t) in Eq.
(2.31). Similar to Program 2.3, we shall call it generically
diffeq. The standalone Euler, the user supplied subroutine
freefall, and the main code go are listed below.

Program listing 2.5: Free fall with modular Euler

( freefall_euler.py)

import numpy as np # get numpy functions

2 import matplotlib.pyplot as plt # get matplotlib plot functions
 g = 9.8 # gravitational constant
4

def Euler(diffeq, y0, t, h): # uses docstring ”””…”””

6 ””” Euler’s method for n ODEs:
Given y0 at t, returns y1 at t+h ”””
8 dydt = diffeq(y0, t) # get {dy/dt} at t
return y0 + h*dydt # Euler method on a vector
10

def freefall (y, t): # returns {dy/dt}, the RHS of ODEs

12 dydt = np.zeros(2) # initialize 2−element numpy array
dydt[0] = y[1] # f1(), Eq. (2.32)
14 dydt[1] = −g # f2(), Eq. (2.33)
return dydt # note: returns whole array dydt []
16

def go(v0): # main program, v0=initial velocity

18 y0 = [0.0, v0] # initial values
t, h = 0.0, 0.02 # init time, step size
20 ta,ya,yb = [],[],[] # declare arrays for plotting
while t<1.0: # loop for one second
22 ta.append(t) # record time and position
ya.append(y0[0])
24 yb.append(v0*t−g*t*t/2.0)
y1 = Euler(freefall, y0, t, h) # Euler’s method
26 for i in range(len(y0)): # reseed y0
y0[i] = y1[i]
28 t=t+h

30 plt.figure() # start a figure

plt.plot(ta,ya, ta,yb, ’--’) # draw 2nd curve as dashed
32 plt.xlabel( ’t (s)’) # add labels
plt.ylabel( ’y (m)’)
34 plt.show() # show figure

36 go (5.0) # run the program

 The program defines the global variable g so it is accessible
to all modules within the program. The vector operations in Eq.
(2.31) are effected by element-wise operations with NumPy
arrays (ndarrays, see Section 1.D) at line 9 in Euler: y0+h*dydt,
with dydt returned from diffeq. The actual form of diffeq for
free fall, Eqs. (2.32) and (2.33), is contained in freeall which
defines the derivatives as an ndarray (line 12). Note that we
made a name switch in the main code from the generic name
diffeq to freefall when calling Euler. As was pointed out
earlier (Section 2.2.3), the advantage of doing things this way is
that we can supply another function to calculate something
entirely different without having to change Euler. But if a
programming language does not support function names as
arguments, you may have to rename freefall to its generic
name, diffeq.

The main code is similar to Program 2.2, but is modularly

contained in the function go(). Here we used another variable,
y1, to store the returned array from Euler. We could have saved
space without it, using y0 instead, by changing to y0=Euler(…).
We introduced y1 mainly for clarity here, but there may be
other reasons. For example, if in-situ analysis requires the data
between time steps (e.g., calculating the difference), it is
necessary to use another variable. The program should produce
plots exactly like Figure 2.2.
 Before going further, it is instructive to walk through the
vector method step by step for one cycle. To do so, let us denote
the position (pos) and velocity (vel) pair by a 2×1 matrix, and
represent the equations of motion (2.32) and (2.33) as

Equation (2.34) means that the derivative function just

transforms the input vector into another vector (where position
is discarded). Then the vectorized Euler’s algorithm, Eq. (2.30),
may be schematically represented as (assuming initial position
and velocity are 0, v0)

With input {y0}, the derivative function generates a vector

which, when multiplied by h and added to {y0}, gives new
values {y1}. As we can see, the new position, v0h is linear in h.
Exactly, it should be quadratic, v0h − gh2/2. So Euler’s method,
accurate to first order, misses the second order correction as
expected. For velocity, it is exact, v0 − gh, also as expected
since it is linear.

RK2 method
Higher order methods like RK2 can be ported over in exactly
the same way. We vectorize the single ODE version (2.16) as
 The actual code given in Program 2.3 works for a single
ODE as well as a system of ODEs, provided the derivatives are
returned as vector ndarrays.

We can use freefall as defined in Program 2.5, and only

change one line in the main code ( go) to use RK2 instead of
Euler,

def go(v0):
……
y1 = RK2(freefall, y0, t, h)
……

It would be helpful if you took time to follow through one

working cycle of RK2 for the free fall problem to really
understand what is going on (see Exercise E2.3). And you will
convince yourself of not only the vectorization process
(especially if you are new to it), but also that RK2 is exact for
free fall, which we know is a second order process (t2).

RK4 method
By now we know the routine pretty well. We can vectorize RK4
(2.18) so it becomes applicable to a system of ODEs. This is
shown below.

Like RK2, Program 2.4 can be used for a system of ODEs as

well, as long as the derivatives are returned in ndarrays.

Non-vectorized RK4 method

The vectorized ODE solvers in Programs 2.3 and 2.4 are simple
and elegant, and should be suitable for most problems.
However, vectorized operations involving ndarrays can be
much slower than Python lists in some cases. This typically
happens if the number of ODEs is small ( 10) and the ODEs
are relatively simple to evaluate. Then, the overhead caused by
creating ndarrays can be a relatively large portion of the load,
yielding significant speed loss.2 The difference in running time
could be minutes instead of seconds.

For problems which calls simple ODEs repeatedly and

where the need for speed is critical, we should use non-
vectorized methods suitable for lists. One could also use the
ODE solvers in the SciPy library scipy.integrate as blackbox
routines (see Program 2.10). We give a non-vectorized RK4
using lists and explicit looping in Program 2.8 (see Section
2.B), called RK4n.

To make it easy to reuse the ODE solvers Euler, RK2,

RK4/n, we will put them in one file, say ode.py. It will be our
standalone library of ODE solvers, and should be placed in the
Python installation folder. To use the solvers, just import the
ODE library as

import ode # get ODE solvers

Step size and characteristic time

We will use RK4 as the general purpose ODE solver in our
simulations. To be accurate, the step size h should be small
compared to the characteristic time scale of the problem. What
is the characteristic time, τ, you may wonder? It depends on the
problem under consideration. For Earth moving around the
Sun, τ is on the order of one year. For tossing a ball across the
room, τ would be the time it takes the ball to hit the wall. So a
step size of one hour would be small for planet motion (see
Section 4.2.4), but it would be too large for a ball toss. In
general, we can estimate τ as τ ~ L/v, where L is the typical size
of the problem, and v the typical speed. To ensure accuracy, we
should set h/τ 1 (say 0.01). But, it is important to always
keep the characteristic time in mind when choosing a step size.
 Sometimes we need to balance between speed and accuracy.
To increase speed, we could implement a variable step size h
between calls to RK4. One simple way to adjust the step size is
to monitor the differences Δ32 = |k3 − k2| relative to Δ21 = |k2 −
k1|. Since k2 and k3 are refinements of the derivative at the
midpoint, Δ32 should be small compared to Δ21. Generally, if
Δ32/Δ21 ~ 0.01, h should be adequate. If Δ32/Δ21 is several
times larger (say ~ 0.05), the step size h may be too large, and
it should be reduced by half and the integration restarted from
the beginning of the step. On the other hand, if Δ32/Δ21 < 0.01,
h may be too small. To avoid overkill, the step size h should be
doubled. With some type of automatic step size control, we
would be able to take baby steps around rough patches, and
giant leaps over smooth terrains.

The Runge-Kutta-Felhberg method

The Runge-Kutta-Felhberg method is a fifth order method that
requires six function calls, two more than RK4. The method
achieves fifth order accuracy by a combination of weighting
factors and proper choice of abscissa points. It has some
advantages over RK4 other than the higher accuracy. By a
different combination of the (same) function values, the
method gives a fourth-order result at the same time, hence it is
also called the embedded Runge-Kutta method. This is useful
for step size adjustment because the difference between the
fourth- and the fifth-order results tells us about the error at a
given step size h. Therefore, adjustment of h can be made to
achieve a specified tolerance.
 We include a fifth-order embedded Runge-Kutta program
in Section 2.B (Program 2.9) named RK45n. The function RK45n
is non-vectorized like RK4n, and can be used in place of the
latter. It should be included in our standalone ODE library.

Implicit methods
All the methods we have discussed so far are explicit: given the
initial conditions, we march forward for solutions at later
times. There are cases where explicit methods are not suitable,
and even fail to produce acceptable solutions. For instance, stiff
differential equations are known to be ill-suited for explicit
methods. To obtain accurate solutions to these equations, the
steps size h needed with explicit methods is often much smaller
than that for non-stiff differential equations. The reason is
usually due to terms varying over very different time scales.
Implicit methods such as the backward Euler method are often
used for such cases. We will encounter one such case in Section
8.3.1.

.4 The leapfrog method

So far we have discussed solutions of ODEs purely from the
mathematical point of view. However, there are physical
systems possessing special properties that should be respected
in the numerical solution. Hamiltonian systems fall into this
category. Take the simple harmonic oscillator as an example.
The motion is oscillatory and time reversible, i.e., if the velocity
at a certain point in its path is reversed, the trajectory will
move backward over the original path, as if time is reversed, t
→ −t. The motion is also area preserving in phase space which
is a multidimensional space consisting of all coordinates and
momenta [31]. Two phase space trajectories for the harmonic
oscillator are shown in Figure 2.5.

Figure 2.5: Phase space plot of the harmonic oscillator for two
energies, E2 > E1.

Their shape is an ellipse, controlled by the energy

For a given energy E, the area within each ellipse is

proportional to E and is constant because energy is conserved.
The Euler and the Runge-Kutta methods do not preserve these
properties. Below we discuss the leapfrog method that does
(also known as the Verlet method [92]).

Let us start with the equations of motion of a one-

dimensional Hamiltonian system,
where a(x) = F(x)/m is the acceleration. The force F(x) depends
on the coordinate x only. For the harmonic oscillator, a(x) =
−ω2x. The important thing is that the RHS of the first equation
(2.39) depends on the velocity v only, and the second equation
(2.40) on the coordinate x only. This separation of coordinates
and velocity is required for the leapfrog method (Appendix
2.A).

Given the values x0 and v0 at t, the leapfrog method

calculates the values x1 and v1 at t + h in three steps,

First, half a step is taken to obtain the position x1/2 at the

midpoint. Then a full step is taken to calculate the new velocity
v1 using acceleration evaluated at x1/2. Finally, the new position
x1 is found by taking another half a step, using the new velocity.
Each step uses the newest values as they become available.

If the leapfrog method looks a lot like the midpoint (RK2)

method, it partially is, at least in the order of accuracy. The
leapfrog method is second order, as is the midpoint method.
But there is a big difference: the leapfrog method is area-
preserving. To understand this property, let us think of the
algorithm as a transformation in phase space that maps the
points (x, v) at t to the new points (q, p) at t+h. Figure 2.6
illustrates such a transformation.

Figure 2.6: Schematic of area transformation. The transformation

maps the points in phase space from [x, v] at t to [q, p] at t + h, e.g.,
[x, v] → [q(x, v), p(x, v)] for the lower-left corner, similarly for other
points. In the leapfrog method, the area is preserved, dA′ = dA.

The corners of the rectangular area element at t are

transformed to new points at t + h. When h is small enough,
the transformed points form an approximate parallelogram,
whose area dA′ is given by the cross product of the vectors
and . We can associate a transformation with each step in
the leapfrog method represented by Eqs. (2.41) to (2.43). Using
these transformations, it can be shown that the area is
preserved, dA′ = dA. Details are given in Section 2.A. The
leapfrog method is also time reversible. We leave its proof to
Exercise E2.4.
 We wish to write a standalone leapfrog subroutine for a
system of ODEs more general than Eqs. (2.39) and (2.40).
Similar to Eq. (2.27), let us introduce the coordinate and
velocity vectors as {r} = r1, r2, …, rn and {v} = v1, v2, …, vn,
respectively. The system of ODEs can be written as two sets,
with i = 1, 2, …, n,

The function fi({v}, t) depends on the velocity {v} only, and

gi({r}, t) on the coordinate {r} only. Again we need the
separation of {r} and {v} for the leapfrog method to be
applicable. We can think of fi({v}, t) as the generalized velocity,
and gi({r}, t) the generalized acceleration. For Hamiltonian
systems, they become the actual velocity and acceleration. For
others, such as time-dependent quantum systems discussed in
Chapter 8, they are pseudo velocity and acceleration.

By vectorizing the leapfrog method (2.41) to (2.43) in a way

analogous to Eq. (2.16), we can write a leapfrog solver for Eqs.
(2.44) and (2.45) as follows.

Program listing 2.6: The leapfrog method ( leapfrog.py)

1 def leapfrog(lfdiffeq, r0, v0, t, h): # vectorized leapfrog

””” vector leapfrog method using numpy arrays.
3 It solves general (r,v) ODEs as:
 dr[i]/dt = f[i](v), and dv[i]/dt = g[i](r).
5 User supplied lfdiffeq (id, r, v, t) returns
f[i](r) if id=0, or g[i](v) if id=1.
7 It must return a numpy array if i>1 ”””
hh = h/2.0
9 r1 = r0 + hh*lfdiffeq(0, r0, v0, t) # 1st: r at h/2 using v0
v1 = v0 + h* lfdiffeq (1, r1, v0, t+hh) # 2nd: v1 using a(r) at h/2
11 r1 = r1 + hh*lfdiffeq(0, r0, v1, t+h) # 3rd: r1 at h using v1
return r1, v1

Like RK2/RK4 requiring diffeq, leapfrog needs a user

supplied function that returns the RHS of Eqs. (2.44) and
(2.45). This function is defined as lfdiffeq(id, r, v, t). Note
that by our convention, the leapfrog method assumes a
different interface in the derivative function ( lfdiffeq) than
the Runge-Kutta methods. See Program 2.7 for an example.

Here, lfdiffeq is basically the same as diffeq except for the

id flag: when id=0, lfdiffeq returns fi({v}, t), and when id=1, it
returns gi({r}, t). We could have split it into two functions, but
it is generally more convenient to have just one function. As
before, we put leapfrog in the file ode.py, which should be
imported before any ODE solver is used.

THE HARMONIC OSCILLATOR WITH THE

LEAPFROG METHOD
As an example, let us apply the leapfrog method to a
Hamiltonian system, the simple harmonic oscillator. The
equations of motion analogous to Eqs. (2.39) and (2.40) are

The solution is a perfect harmonic,

The constants A and φ are the amplitude and initial phase,

respectively. We see that x2/A2 + v2/(ωA)2 = 1, indeed an
ellipse in phase space.

The program to solve Eq. (2.46) numerically is as follows (ω

= 1 rad/s).

Program listing 2.7: Oscillator with leapfrog method ( sho_lf.py)

1 import ode # get ODE solvers

import math as ma # get math functions
3 import matplotlib.pyplot as plt # get matplotlib plot functions

5 def oscillator (id, x, v, t): # return dx/dt or dv/dt

if (id==0): # calc velocity
7 return v
else: # calc acceleration
9 return −x # −ω2x, ω = 1,

11 def go(): # main program

x, v = 1.0, 0.0 # initial values
13 t, h = 0.0, 0.1 # init time, step size
 xa, va = [],[] # declare arrays for plotting
15 while t<4*ma.pi: # loop for two periods
xa.append(x) # record position and velocity
17 va.append(v)
x, v = ode.leapfrog(oscillator, x, v, t, h) # solve it
19 t=t+h

21 plt.figure () # start a figure

plt.plot(xa,va) # plot vel−pos
23 plt.xlabel( ’x (m)’) # add labels
plt.ylabel( ’v (m/s)’)
25 plt.show() # show figure

27 go() # run the program

The program first imports several libraries: our ODE

solvers in ode.py, standard mathematical functions, as well as
Matplotlib. According to the leapfrog scheme (2.44) and (2.45),
we have n = 1 in Eq. (2.46) (with r replaced by x), and f1(v, t) =
v, g1(x, t) = −x. Therefore, the function oscillator returns v if
id=0, and −x otherwise. The main program go() is very similar
to the one in Program 2.5. It sets up the initial values, and
enters the while loop which runs for two periods, storing the
position and velocity at each time step before advancing to the
next step with leapfrog. The velocity-position curve in phase
space is plotted after the loop.

Figure 2.7 (top) shows a sample plot from the program

above. For comparison, results using the Euler and RK2
methods are also shown (generated separately, see Project
P2.4). Starting from the initial point (1, 0), the path by Euler’s
method spirals outward noticeably. The RK2 and leapfrog
methods produce ellipses similar to what is expected from
Figure 2.5. Although hard to see on the scale shown, the path of
the RK2 method is not closed and the area is not preserved. If
we wait long enough, it would spiral outward eventually, but at
a slower pace than Euler’s method. In contrast, the leapfrog
method has a closed path, thus preserving the area.

Figure 2.7: Phase space plot of the harmonic oscillator (top), and the
relative error in energy (bottom) for ω = 1 rad/s. The results are
 calculated with the Euler (Eul), RK2, and the leapfrog (LF) methods.

We can get a clearer picture by considering the relative

error in energy defined as |Enumeric − Eexact|/Eexact.3 The
numerical energy Enumeric can be obtained by slightly
modifying Program 2.7. The results are shown in Figure 2.7
(bottom) as a semilog plot. When plotting results that vary over
orders of magnitude, the logarithmic scale (semilog or log-log)
should be used to show clearly the difference. The energy by
Euler’s method increases with time. The RK2 method shows a
similar trend, albeit with an offset and at a slower rate because
it is second order. The energy in both methods increases
monotonically and shows no upper bound. In the leapfrog
method it fluctuates around the exact energy, and does not
diverge without bound. Being of finite order of accuracy, the
leapfrog method does not yield the exact energy, but it does
preserve the proper structure in phase space of a Hamiltonian
system, including a closed path as required. Accordingly, area-
preserving methods (also known as symplectic methods) such
as the leapfrog method is preferred when integrating systems
whose motion is bounded, including periodic and oscillatory
motion. Later, we discuss a modified leapfrog method with step
size control in Section 4.3.3.4

Chapter summary

We have discussed several methods of numerical solutions,

namely, Euler, RK2, RK4, and leapfrog methods. Using free fall
 as the main example, they have been applied to a single ODE at
first, and generalized to a system of coupled ODEs later. Along
the way we have showed animation and data plotting
techniques using the simple yet powerful and flexible
Matplotlib, NumPy and VPython functions.

We have constructed standalone ODE solvers with these

methods, including Euler, RK2, RK4/n, RK45n, and leapfrog.
They have been included in one file, ode.py, which will be our
toolbox for ODE solvers. To use it, import ode.py first, and
supply an appropriate diffeq or lfdiffeq function for the
problem being solved. In subsequent chapters, except for
specific cases requiring special methods, we will regard RK4 as
the general purpose ODE solver, and leapfrog for Hamiltonian
problems. The vectorized solvers can be applied to a single or a
system of ODEs. For the latter, diffeq must return a NumPy
array. The non-vectorized routines RK4n and RK45n use explicit
looping through lists or NumPy arrays, and are intended for
systems of a handful ODEs where speed is important.

.5 Exercises and Projects

EXERCISES
E2.1 Modify Program 2.1 to replace VPython animation with
Matplotlib output. Use elements from Program 1.1, and
initialize three lists for time, position, and velocity before the
 loop. Terminate the loop after 10s, and plot position and
velocity vs. time, respectively.
E2.2 (a) Show that the alternative RK2 algorithm (2.17) is second
order in accuracy and is equivalent to Eq. (2.16).

(b) Among other possibilities of the RK2 family is t

following

where the second point is chosen at 2h/3. Show that this

equivalent to Eq. (2.16) to second order. Hint: Expand k1 a
k2 around the midpoint. For simplicity, assume f(y, t) = f(
i.e., no explicit y dependence.

E2.3 Follow through one working cycle of the RK2 method for fr
fall similar to Eq. (2.35) of Euler’s method. Show that in RK
(2.16)

proving that RK2 is exact for free fall, no matter what st

size h is.

E2.4 Prove that the leapfrog method, Eqs. (2.41) to (2.43), is time
reversible. Use primed variables to represent reversed motio
 and let . Show that after one step, the

position is restored but the velocity is reversed, i.e.,

.

E2.5 Use an approach analogous to Exercise E2.4, show that

Euler’s method when applied to Eqs. (2.39) and (2.40) is not
time reversible unless a(x) = 0. In other words, Euler’s meth
(as well as RK2 and RK4) is time reversible only for a free
particle (straight-line motion).
E2.6 The Euler-Cromer algorithm is a first-order symplectic
method. It solves Eqs. (2.39) and (2.40) by the following,

which would be the standard Euler’s method except for t

acceleration, a(x1), evaluated at x1 rather than at x0 in E
(2.49). Show that the Euler-Cromer method is time reversib
and area-preserving by their Jacobians.

E2.7 Nuclear decay is a random process. But given sufficient

number of nuclei N, it may be modeled as a deterministic
process,
 where λ is the decay constant.

(a) One interpretation of the lifetime of a single nucleus is t

average time of their existence before decay, i.e., τ = ∫ t(−dN
∫(−dN). Based on this interpretation and given N =
exp(−λt), show analytically that the lifetime of an unstab
nucleus is τ = 1/λ.

b) If you have not used SymPy before, this might be a go

place to start (see Section 1.B). Try the following in IPython

In [1]: from sympy import * # import SymPy

In [2]: t, lamb = symbols( ’t lamb’,positive=True) # declare vars
In [3]: integrate(exp(−lamb*t),(t,0,oo)) # integrate from [0,∞]
Out[3]: 1/lamb

Now compute τ with SymPy.

PROJECTS
P2.1 Simulate free fall using Euler’s method and VPython.

(a) Do this by combining and modifying Programs 2.1

and 2.5. Delete Euler from Program 2.5 since we will be
using ode.py, but keep freefall as is. Optionally, you
may also delete the plotting functions below the “while”
loop. Your program blocks should look like

from visual import *

from ode import *
import numpy as np
# draw ball and floor
# set initial conditions
……
while True:
……
y = Euler (…)
……
# update ball position
# reverse velocity if below floor

Note that the position and velocity of the ball are

computed by the standalone Euler and contained in the
array y, and should not be computed separately as in
Program 2.1. Reversing the ball’s velocity upon hitting
the floor amounts to changing the initial condition (or
seed value for next step). Once the program is running,
what do you see? Notice how the “camera” keeps
zooming out to keep the ball in view. Why? Does the ball
bounce higher and higher? Is it numerical or physical?
Try a few different initial velocities and see if anything
changes.
 (b) Replace Euler’s method with the RK2 method.
Change just one line as

#y = Euler(…)
y = RK2(…)

It is helpful that in testing out different things such as we

are doing here, rather than deleting the original code,
simply comment out the lines we do not need. It makes it
easier to go back to an earlier state by uncommenting
them. You should now see the ball bouncing up and
down, but not gaining height after each bounce.

(c) Compare your observations about the different results

with Euler’s and the RK2 methods. Discuss the reasons.
Refer to Figure 2.2 for hints.

(d) Using energy conservation as a further check, add

two statements in the loop that records time and energy
(kinetic and potential, setting mass to one). Break the
loop when time reaches certain value (say 10s), and plot
energy vs. time. Compare results obtained with Euler’s
and the RK2 methods. What went “wrong” with Euler’s
method?

P2.2 Let us consider free fall in two dimensions, i.e., projectile

motion. The equations of motion are
 (a) Solve these equations with the Euler and RK2 (or
RK4) methods. Your program should be very similar to
Program 2.5. The key part is to write a derivative
function that plays the role of the generic diffeq(). Let
us name it projectile(y,t), where y[] should be a 4-
element array. The first two elements should contain the
positions (x, y) and the last two the velocities (vx, vy) (see
Program 3.5). Assume initial conditions as: x = y = 0,
launching speed and angle at 10 m/s and 30°,
respectively. Follow the motion until the projectile hits
the ground. Plot the x-t and y-t curves, and compare with
analytic results.

(b) Animate projectile motion using VPython. As in

Program 2.1, you only need to update the position of the
projectile after each step. The following line will change
both the x and y positions (assuming they are stored in
y[0],y[1] as suggested above)

ball.pos=(y[0],y [1])

P2.3 (a)Write a program to investigate nuclear decay (Exercise

 E2.7) using several methods: Euler, RK2, RK4, and
−1
RK45n. Assume λ = 0.5 min and N0 = 1000. Choose
your time step h judiciously and integrate to 8 min. Plot
the results as N-t curves for each method on one plot.
Discuss their differences.

Next, plot the absolute errors of each method on a

semilog plot. Compare the magnitudes of the error to
each other. Then halve h and plot the errors again.
Compare the reduction in error for each method. Does
the global error for a given method scale with its order?

(b) Calculate numerically the lifetime τ according to the

interpretation in Exercise E2.7. This can be done by first
computing the cumulative lifetime as ∑ t(−ΔN) in the
main loop of (a) because “−ΔN” is the number of nuclei
that “lived” to time t but decayed within the next time
step, and then averaging over all the nuclei that decayed,
i.e., N0 − N, where N is taken after the loop. Compare the
numerical results for each method with the exact answer.
Discuss the discrepancies, the reason, and how they may
be reduced.

P2.4 (a) Modify Program 2.7 to plot the phase space

trajectories with different methods, including Euler, RK2,
and leapfrog. Of course, you can use the derivative
function oscillator() as is with the leapfrog()
function. However, with the Euler and RK2 methods, you
need a slightly different derivative function, call it
oscillator_rk(y,t). It is similar to freefall() in
Program 2.5 but returns the RHS of the oscillator
equations (2.46) based on the input of the 2-element
 array y[], with y[0] = position and y[1] = velocity. Run
the program with different initial conditions: x0 = 1 and
1.5, and v0 = 0.

(b) Calculate the relative error in the energy of the

harmonic oscillator with the same methods. Plot the
results on a semilog scale as in Figure 2.7 with
plt.semilogy() after the plot() function. All other

plotting commands are the same.

(c) Reduce the step size h by half and repeat (b). Discuss
the changes and trends in the results.

(d) Redo the calculation in part (b) using a higher order

method, RK4 or RK45n. Comment on your results.

P2.5 Consider the motion of an electron (m = 1) in a one-

dimensional Coulomb field V(x) = 1/x (all in atomic units
(a.u.), see Table 8.1).

(a) Using conservation of energy E, obtain the

relationship between velocity and position, v(x), for a
given E.

(b) Convert d2x/dt2 = F/m = −∂V/∂x into two first order

ODEs.

(c) Use RK2 to simulate the motion of the electron with

initial conditions as: position x = 5 a.u. and velocity v =
−1 a.u. You need a proper derivative function based on
part (b). Plot position-time and velocity-time curves from
 the beginning to when it returns to its original position.
Also plot the phase space trajectories, the v-x curves, one
of the numerical results, and another of the analytic
result, part (a). How do they compare?

(d) Repeat (c) with the leapfrog method. You would

need a slightly different derivative function than the one
above.

(e) Do the same using RK4 and RK45n, but with different
initial conditions of x = 5 and v = −4. You can use the
same derivative function as in (c). Discuss and compare
the performance of the two methods relative to each
other.

Observation: As you work on this project, think about the

following questions. What would be the “characteristic”
time scale? How did you choose the step size h relative
to this time scale? Be conservative with h because the
potential is singular at the origin. Since the motion is
unbound (the electron can escape to x = ∞), is there still
any advantage of the leapfrog method compared to the
RK2 method?

P2.6 Let us play a game popular on many campuses: Humans

vs. Zombies! The students join one of two groups,
designated as Humans and Zombies, respectively.
Members of each group are armed with toy dart guns. To
make it interesting, let us assume that the firing rates, or
“kill” rates, between the groups are different. Let H be
the number of Humans with a kill rate λH, and Z the
number of Zombies with a kill rate λZ. In the simplest
model, we can assume that the number of Humans
decreases at a rate proportional to the kill rate and the
number of Zombies, and vice versa. This model can be
written as

The game ends when one side reaches zero, and the other
side is the winner.

(a) Say you are the last one to join, and you are free to
choose either group. The current situation is as follows:
69 Humans with a kill rate 2, and 99 Zombies with a kill
rate 1. There are fewer Humans, but they have twice the
kill rate. If your goal is to win, which group would you
join? Choose wisely (and write it down).

(b) It is war now! Simulate the game by solving the

model with the initial condition as: λH = 2.0, λZ = 1.0; if
you decided to join the Humans, then H = 70, Z = 99;
otherwise H = 69, Z = 100. Plot the H-t and Z-t curves
when the game ends. You may use either RK2/RK4 or
the leapfrog method. Are you victorious? Why?

(c) Switch sides and repeat (b).

 (d)* Run the program with different initial H, say 80,
100, 120, etc., and for each H, find the upper limit Z such
that the Humans barely win. Plot H vs. Z, and you should
discover a scaling law, i.e., H/Z = C. Find the constant C.
As it turns out, the model yields analytic solutions. Find
them, and discuss how C is related to λH and λZ.

.A Area preservation of the leapfrog

method
If we view a numerical method that advances the solution (x, v)
at t to (q, p) at t + h as a transformation in phase space, we
have

The functional forms of q(x, v) and p(x, v) will depend on the

numerical method, or the algorithm. As we will see shortly,
each of the three steps (2.41) to (2.43) in the leapfrog method
represents one transformation.

Let us first consider the transformation of area elements by

Eq. (2.50), schematically shown in Figure 2.6. Specifically, let
us consider small changes in dx and dv, respectively. There are
four points associated with these changes: (x, v), (x + dx, v), (x
+ dx, v + dv), and (x, v + dv). They are among the possible
points in phase space at t.

The corner points of the rectangle with sides and

will be mapped to new locations at t + h according to Eq.
(2.50). The change dx, while holding v constant, will transform
the vector to a new vector (Figure 2.6). Similarly, the
change dv, while holding x constant, will transform the vector
to a new vector . The two vectors and will not be
perpendicular to each other in general. Provided the changes
are small, the transformed points form an approximate
parallelogram with and as the sides.

The vectors and can be found by

The area element of the transformed parallelogram is dA′ =

du dw sin θ, where θ is the angle between and (Figure
2.6). This is just the magnitude of the cross product of the
vectors and , i.e., . Using Eq. (2.51), we find
the area element to be

Introducing the Jacobian J as

we can express the area element as

where dA = dxdv is the area element of the original rectangle.

Now we can construct the three transformations

represented by the three steps in the leapfrog method, Eqs.
(2.41) to (2.43). We may consider the first step (2.41) as a
transformation that changes only the coordinate while keeping
the velocity unchanged. In terms of pre-transformed variables
x0, v0, the transformation q1, p1 is

The associated Jacobian is

Similarly, the transformations and the Jacobians for the

next two steps, respectively, are
and

Note that the pre-transformed variables for the second and the
third steps are (x1/2, v0) and (x1/2, v1), respectively.

The cumulative transformation of the area element is the

product of the three successive transformations, dA′ =
J3J2J1dA. We see from Eqs. (2.54) to (2.56) that each Jacobian
(determinant) is exactly 1, J1 = J2 = J3 = 1, such that

We have shown that one area element enclosed by the four

points at the corners of the rectangle is preserved (Figure 2.6).
To consider the total area of the accessible phase space, we
would divide it into many small (infinitesimal) rectangular
elements. Since each area element is preserved by the
transformation, the total area, representing all possible points
in phase space, must also be preserved.
 Hamiltonian systems preserve the phase space area. This is
known as the Liouville theorem [40]. Area-preserving methods
are said to be symplectic, so the leapfrog method is symplectic.
The leapfrog method (or other symplectic methods) is generally
preferred when integrating Hamiltonian systems. However, if
the force is velocity dependent, the leapfrog method as
presented by Eqs. (2.54) to (2.56) is ambiguous with respect to
velocity, and is not area-preserving, because the Jacobian J2
would not be 1. A modified scheme would be necessary, but we
will not pursue it here. Speaking of modifications, it is also
possible to start the leapfrog method (2.54) to (2.56) with
velocity v1/2, basically swapping x and v. There is no advantage
to this approach because it is equivalent to the original method,
but is less efficient because two calls to a(x) are required per
step, instead of just one.

It should be noted that the leapfrog method, being of finite

order, may not yield the exact area in phase space. But,
however inexact that area may be, it is preserved exactly.

.B Program listings and descriptions

Program listing 2.8: RK4 nonvector method ( rk4n.py)

1 def RK4n(diffeq, y0, t, h): # non−vectorized with lists

””” RK4 method for n ODEs:
3 Given y0 at t, returns y1 at t+h ”””
n, y1 = len(y0), [0.0]* len(y0)
 5 k1 = diffeq(y0, t) # dy/dt at t
for i in range(n): # loop thru n ODEs
7 y1[i] = y0[i] + 0.5*h*k1[i] # prep for k2[]
k2 = diffeq(y1, t + h/2.) # dy/dt at t+h/2
9 for i in range(n):
y1[i] = y0[i] + 0.5*h*k2[i]
11 k3 = diffeq(y1, t + h/2.) # dy/dt at t+h/2
for i in range(n):
13 y1[i] = y0[i] + h*k3[i]
k4 = diffeq(y1, t + h) # dy/dt at t+h
15 for i in range(n):
y1[i] = y0[i] + h*(k1[i]+2*k2[i]+2*k3[i]+k4[i])/6.0
17 return y1

Program 2.8 works with both lists and NumPy arrays

returned by diffeq. It should be applicable for problems fitting
the discussed criteria (Section 2.3.2).

Program listing 2.9: RK45 nonvector method ( rk45n.py)

1 def RK45n(diffeq, y0, t, h): # RK45 method

a2, a3, a4, a5, a6 = 0.2, 0.3, 0.6, 1.0, 0.875
3 b21, b31, b32, b41, b42, b43 = 0.2, 3./40., 9./40., 0.3, −0.9, 1.2
b51, b52, b53, b54 = −11./54., 2.5, −70./27., 35./27.
5 b61, b62, b63, b64, b65 = [1631./55296., 175./512., 575./13824.,
44275./110592., 253./4096.]
7 c1, c3, c4, c6 = 37./378., 250./621., 125./594., 512./1771.

9 n, y1 = len(y0), [0.0]* len(y0)

k1 = diffeq(y0, t)
11 for i in range(n):
y1[i] = y0[i] + h*b21*k1[i]
 13 k2 = diffeq(y1, t + a2*h)
for i in range(n):
15 y1[i] = y0[i] + h*(b31*k1[i] + b32*k2[i])
k3 = diffeq(y1, t + a3*h)
17 for i in range(n):
y1[i] = y0[i] + h*(b41*k1[i] + b42*k2[i] + b43*k3[i])
19 k4 = diffeq(y1, t + a4*h)
for i in range(n):
21 y1[i] = y0[i] + h*(b51*k1[i] + b52*k2[i] + b53*k3[i] + b54*k4[i])
k5 = diffeq(y1, t + a5*h)
23 for i in range(n):
y1[i] = y0[i] + h*(b61*k1[i] + b62*k2[i] + b63*k3[i] + b64*k4[i]
25 + b65*k5[i])
k6 = diffeq(y1, t + a6*h)
27 for i in range(n):
y1[i] = y0[i] + h*(c1*k1[i] + c3*k3[i] + c4*k4[i] + c6*k6[i])
29 return y1

The above program (non-vectorized) is based on the Cash-

Karp coefficients for the embedded Runge-Kutta formula [71].
We omitted calculation of error estimate in the code. It seems
from our experience that for some conservative oscillating
systems, the error estimate substantially underestimates the
true error. However, this does not impact the accuracy of the
method (fifth order).

Program listing 2.10: SciPy ODE wrapper ( odewrap.py)

1 from scipy.integrate import odeint # SciPy integrator

3 def odewrapper(diffeq, y0, t, h): # ode wrapper

 y = odeint(diffeq, y0, [t, t+h])
5 return y[1]

7 Euler = RK2 = RK4 = RK45 = RK4n = RK45n = odewrapper # alias

This is just a wrapper for the general ODE integrator odeint

from the SciPy library scipy.integrate. We call odeint with
two points in time (t, t + h), and return the solutions at t + h. It
is functionally (but not accuracy wise) equivalent to the Runge-
Kutta methods, all assigned to the same alias on the last line.

The wrapper may be useful to those who wish to skip the

discussion of the RK methods, or to use it as a blackbox. To use
the wrapper, import it as

import odewrap as ode # use wrapper as RK substitutes

All programs using the RK family of methods should work

the same way as with our own ODE library. However, the
wrapper is not a substitute for the leapfrog method.

1
We may also turn Eq. (2.5) around to find the derivative by two-point numerical
differentiation, dy/dt ≈ [y(t + Δt) − y(t)]/Δt. The error term is O(Δt). However, we can do
better with two points using the midpoint method, Eq. (2.13).

2 5
Tests show that creating 10 two-element ndarrays is about 30 times slower than
comparable lists.
 3
To validate a program, it is always important to check the results against exact solutions
if available, or against conservation laws such as energy and angular momentum, etc.

4
The use of symplectic methods has completely transformed our understanding of the
stability of the solar system by extending accurate time integrations from millions of years
prior to 1990s to timescales comparable to the age of the solar system (see Ref. [65] and
Chapter 4).
Chapter 3
Realistic projectile motion with
air resistance
One of the most classic and well-known problems in physics is
projectile motion. Tossing a ball in the air, we expect to see it
move in a familiar arched path. Under ideal conditions
assuming constant gravitational acceleration and negligible air
resistance, projectile motion is analytically solvable, i.e., its
solutions are expressible in closed-form, known functions. Its
properties such as the parabolic path are well explained from
introductory physics. Free fall discussed in Chapter 2 is a
special case of one-dimensional ideal projectile motion, while
the general case is three-dimensional.

To describe realistic projectile motion, we need to consider

the effects of air resistance, or drag, which can be significant
and interesting. However, the inclusion of these effects renders
the problem analytically nonsolvable, and no closed-form
solutions are known except under limited conditions.
Numerically, this presents no particular difficulty for us, given
the toolbox and ODE solvers we just developed in Chapter 2. In
 fact, realistic projectile motion is an ideal case study for us to
begin application of these numerical and visualization
techniques to this classic problem in this chapter, for it is
relatively simple, intuitive, and its basic features are already
familiar to us. We will learn to construct models of appropriate
degree of complexity to reflect the effects of drag and spin.
Furthermore, we will also discuss the application of a recently
discovered Lambert W-function to projectile motion, as well as
methods of finding roots.

.1 Visualization of ideal projectile

motion
In his work Two New Sciences in 1638, Galileo theorized that a
free body in motion was “compounded from equable horizontal
and naturally accelerated downward motion, which I call
‘projection’.” Of course we recognize in today's nomenclature
that Galileo was talking about ideal projectile motion which, if
air resistance is ignored, moves with constant horizontal
velocity and constant vertical acceleration, i.e., and
. In 2D, this can be expressed in terms of
differential equations similar to Eq. (2.1) as

where x, y and vx, vy are the components of position and

velocity, respectively, and g the gravitational acceleration as
usual. The analytical solutions to Eq. (3.1) are well known from
introductory physics. For example, the projectile range is given
simply by

where v0 is the initial speed and θ0 the initial projection angle.

It is worth noting that the angle that maximizes the range is
45°, independent of speed in ideal projectile motion. We shall
see that this is no longer true if air resistance is included.

Visualizing projectile motion is also straightforward. The

program below displays perpetual projectile motion of a ball,
captured in Figure 3.1.

Program listing 3.1: Ball toss ( balltoss.py)

1 import visual as vp # get VPython modules for animation

3 vp.display(background=(.2,.5,1)) # make scene, ball, floor, and path

ball = vp.sphere(pos=(−4, −4,0), radius=1, color=vp.color.yellow)
5 floor = vp.box(pos=(0, −5,0), length=12, height=0.2, width=4)
path = vp.points(pos=ball.pos, size=4) # use make_trail in newer VP
7 h, g, vx, vy = 0.01, 9.8, 4.0, 9.8 # Δt, g, and initial velocity
while True:
9 vp.rate(400) # limit animation rate
ball.pos.x += vx*h # update position
11 ball.pos.y += vy*h
path.append(pos=ball.pos, retain=300) # draw path, keep 300 pts
13 if ball.pos.y > floor .y + ball.radius:
vy = vy − g*h # above floor, update vy
15 else: vx, vy = − vx, − vy # below floor, reverse vel.
 Figure 3.1: Snapshots of a ball toss from Program 3.1.

The program is very similar to Program 2.1 in both

structure and flow. Note that the position attributes of objects
such as the sphere include now both x and y components in 2D
motion. We have added the path of the ball made with points,
which is continually appended to in the loop (lines 6 and 12).
The retain=n attribute keeps only the last n points, useful for
ensuring smooth animation where a large number of points are
generated. The position and the vertical velocity vy are updated
according to Eq. (3.1) by first order Euler's method. When the
ball would fall below the floor, its velocity vector is reversed, so
the ball moves back and forth perpetually. The path traces out
the familiar parabolic shape, with front-back symmetry.
Experiment with different parameters such as g, and initial vx,
vy, etc. What happens if only vy is reversed upon collision with
the floor?

.2 Modeling air resistance

Anyone riding a bicycle can feel the rush of oncoming air and
the resistive force that grows with increasing speed. As a fact of
life, air resistance is easy to understand but hard to quantify,
because the underlying physics involves the interactions of a
very large number of particles, and there is no exact form for
that force. Nonetheless, we observe that, among other things,
the force is dependent on the velocity of relative motion . In
particular, it is zero if . This leads us to describe the
resistive force phenomenologically by a power series as

Here is the force due to air resistance (or drag), is the

velocity of the projectile relative to the medium (air), and
is the speed. The constants b1 and b2 are called linear
and quadratic coefficients. The negative sign in front of b1 and
b2 indicates that the drag force is opposite the direction of
velocity.

Linear drag
Physically, the first term in Eq. (3.3) comes from viscosity of
the medium. The linear coefficient b1 can be calculated from
Stoke's law applicable to laminar (smooth) flow. We can
understand qualitatively how b1 comes about from Newtonian
physics.
 Figure 3.2: Viscous fluid flow between two moving plates.

Let us consider a fluid between two large parallel plates

shown in Figure 3.2. The top plate is moving with velocity v to
the right, and the bottom one is stationary. We assume v is
small, so the flow is smooth and separates into thin layers from
top to bottom. The topmost layer moves at speed v, and the
bottom one at rest, since they “stick” to the top and bottom
plates, respectively. In between, we expect the fluid velocity to
decrease uniformly, depicted as shrinking arrows in the figure.

To keep the top plate moving, a force to the right has to be

applied on it to counter the drag from the layer below.
Newton's conjecture is that the force should be proportional to
the velocity v, the area of contact A between the plate and the
fluid, and inversely proportional to the plate separation d, i.e.,
F1 ∝ Av/d. Proportionality to v and A can be understood as
follows: a larger v means a higher rate of momentum change
for the fluid,1 and a larger A means a greater amount of fluid
being pulled, so both factors lead to a greater force. As to the
inverse proportionality to d, we imagine trying to push a piece
of jello from the top, and the taller the jello, the easier to
deform (less shear force).
 Let the proportionality constant be μ, then

The coefficient μ is the viscosity. Its unit is kg/s/m, or Pa·s. The

negative sign is added because the force is exerted on the plate
by the fluid (Newton's third law).

For a sphere moving in a fluid, Eq. (3.4) does not apply

since the velocity gradient (v/d) around it is not constant.
Nevertheless, we expect Eq. (3.4) to give at least a rough
estimate. Replacing A = 4πr2 (surface of sphere) and d = 2r,
with r being the radius of the sphere, we have F1 = −2πμrv. The
exact expression is

This is Stoke's law. Our estimate gives the correct dependence

on the relevant parameters, but the constant factor is wrong (2
instead of 6), which is to be expected given the crudeness of our
model.

If the velocity v is very small, linear drag is the dominant

force. But as it turns out, the speed where the linear term
dominates is usually quite low (see Exercise E3.3). Therefore
for all practical purposes, the second order term is the
important one.

Quadratic drag
The second term in Eq. (3.3) arises when the flow is turbulent,
meaning that it is no longer laminar in nature. The
dimensionless Reynolds number defined as

is the determining factor as to whether the flow is laminar or

turbulent.2 Here, ρ is the fluid density, μ the fluid viscosity
defined in Eq. (3.4), L the characteristic length of the object,
and v the velocity. It is generally noted that the transition from
laminar to turbulent flow for circular objects occurs at Re
2300, i.e., the flow is laminar below 2300, and turbulent above
it. For example, the transition occurs at about 0.5 m/s for a
baseball, so the quadratic term plays the dominant role in
practice.

Figure 3.3: An object of cross section area A moving at speed v. In

time interval Δt, the air mass in its path is displaced and accelerated.

We can obtain the quadratic coefficient from elementary

considerations. Let us assume an object (say a disc) with an
effective cross sectional area A. As it moves through air with
speed v in time interval Δt, it will displace all the air mass in its
path (Figure 3.3). The amount of air displaced is equal to the
cylindrical volume AvΔt swept by the disc, multiplied by air
density ρ, or Δm = ρAvΔt. If, on average, the air mass Δm is
accelerated to the speed v, the net gain of impulse is Δp = Δmv
= ρAv2Δt. The force necessary to exert this impulse is given by
Newton's second law, F = Δp/Δt = ρAv2. By the third law, the
object (disc) experiences an equal magnitude but opposite
force, −ρAv2. By this estimate, the coefficient b2 in Eq. (3.3) is
identified to be b2 ≈ ρA.

To be more precise, we let b2 = CdρA/2 and express the

second term in Eq. (3.3) as

Now Cd is dimensionless, and is called the (quadratic) drag

coefficient. To the extent that the linear drag force is negligible,
then F2 Fd, and F2 is usually referred to as the drag force. It is
the dominant force from air resistance. This expression (3.7) is
by no means fully accurate, but contains the essential physics.

Model building, realism, and complexity

Numerically, the nominal drag coefficient for spherical shapes
is Cd ∼ 0.5. According to our estimation above, it should be
about 2. The difference reflects the crudeness of our model,
assuming uniform acceleration of displaced air mass, flat
surface, etc. Here a word is in order about model building in
computer modeling. A computer simulation is only as accurate
as the ingredients contained in the model. Simple models may
be crude, but they are useful for bringing out the essential
physical features for a better understanding of the problem.
Complex models, though more complete and accurate, may
give results that are difficult to interpret. One can increase the
accuracy by increasing sophistication of the model. There is
clearly a tradeoff. In practice, it is usually a good idea keep the
model as simple as possible within required realism.

For instance, the drag force of the form (3.7) with constant
Cd is adequate for most projectile problems. We could certainly
fine tune the model, e.g., by including some velocity
dependence in the coefficient Cd. In fact, experimental evidence
shows that Cd is velocity-dependent and not a constant over all
the range of v. This dependence is displayed in Figure 3.4.

Figure 3.4 presents a rather interesting feature: a sharp drop

in Cd at the critical speed ∼ 34 m/s for baseball. A similar drop
occurs for smooth balls at a larger speed. Before and after the
sudden drop, the drag coefficient remains relatively flat. The
difference in Cd between smooth and rough balls is very
interesting, and is mainly due to the boundary layer
surrounding the ball. A rough ball (stitches on the baseball)
creates a more turbulent layer than a smooth ball, so the onset
of the drop occurs earlier. It is thought that at the critical
speed, the turbulence produces a smaller wake, thus less air
resistance and the precipitous drop in Cd.

It is convenient to fit the baseball data in Figure 3.4 to a

simple form for use in simulations. We choose the following
empirical formula
 Figure 3.4: The drag coefficient as a function of velocity for smooth
balls and baseball [32]. The data for baseball were derived from
actual games at 1996 Atlanta Olympics [81]. The dashed line is a fit
to the data.

Here, speed v is in m/s, and vc denotes the critical speed. The

second term in Eq. (3.8) mainly causes the steep drop near vc,
and the third term, an asymmetric Gaussian, yields the steep
rise after vc.

From the above discussions on the velocity dependence of

the drag coefficient, we see that the power series (3.3) is
effectively of higher order than quadratic. In other words, an
adequate representation of air resistance should include orders
v3, v4, etc. Then, how meaningful is it to speak of quadratic air
resistance? Because the drag coefficient Cd is rather flat outside
the critical speed region, we take the view that Eq. (3.7)
 captures the essential quadratic nature of the drag force.
However, we should keep in mind that if the velocity-
dependent coefficient (3.8) is used, Eq. (3.7) effectively takes
into account all orders within the specified range shown in
Figure 3.4.

.3 Linear air resistance

Though for practical purposes quadratic air resistance
dominates for all but extremely small speeds, linear air
resistance is nonetheless very insightful from the viewpoint of
modeling because it is simpler and, surprisingly, even admits
analytic solutions in terms of special functions. It is an ideal,
first case study to begin our exploration into simulations.3

In this section, we shall discuss linear air resistance only.

Ignoring the quadratic term in Eq. (3.3), we take the drag force
to be linear

The Newtonian equations of motion with the net force of

gravity plus linear drag are
where m is the mass of the projectile, and x, y, vx, vy are the
same as in Eq. (3.1). These four first-order ODEs (3.10) are
slightly modified versions from Eq. (3.1).

3.3.1 NUMERICAL SIMULATION

The equations of motion (3.10) are tailor-made for a numerical
integrator like Euler's or Runge-Kutta methods discussed in
the Chapter 2. Similar to Program 2.5, we only need to supply a
subroutine diffeq which returns the RHS of Eq. (3.10). So here
it is, named Lin_drag:

Program listing 3.2: Linear drag ( projectile.py)

1 import ode, numpy as np # ODE solvers, numpy

import matplotlib.pyplot as plt # plot functions
3
def Lin_drag(Y, t): # projectile motion with linear drag
5 # returns RHS of Eq. (3.10) : [vx, vy, ax, ay]
return np.array([Y[2], Y[3], −b1*Y[2]/m, −g −
b1*Y[3]/m])
7
def go(vx=5., vy=5.): # default velocity =(5,5)
9 Y = [0., 0., vx, vy] # initial values, [x,y,vx,vy]
t, h, x, y = 0., .01, [], [] # time, step size, temp
arrays
11 while Y[1] >= 0.0: # loop as long as y>=0
x.append(Y[0]), y.append(Y[1]) # record
pos.
13 Y, t = ode.RK4(Lin_drag, Y, t, h), t+h # update Y, t

15 plt.figure () # open fig, plot, label, show

 fig
plt.plot(x, y), plt.xlabel(’ x (m)’), plt.ylabel(’ y (m)’)
17 plt.show()

19 g, b1, m = 9.8, 0.2, 0.5 # g, linear coeff ., mass

go(vx=10.0, vy=10.0)

In Lin_drag, the input array Y[0..3] is assumed to contain

x, y, vx, vy in order. It returns a NumPy array converted from a
list in order to use vectorized RK4. The array just contains the
respective derivatives on the RHS of Eq. (3.10). It also requires
the parameters b1 and m, which are chosen (somewhat
arbitrarily) at the end of the code. The main function go()
initializes the variables, calls RK4 in the main loop until the y-
position becomes negative, and plots the results (similar to
Figure 3.5). Note the front-back asymmetry in the ascending
and descending parts of the trajectory and the sharp falloff
toward the end. This is a signature of air resistance breaking
the symmetry in ideal projectile motion.

3.3.2 ANALYTIC SOLUTIONS

Analytic solutions can be obtained in nearly the same
straightforward manner as for numerical solutions. To make
things a bit tidy, we redefine the linear drag coefficient by
including the mass m of the projectile as
 We begin with the x component of the velocity of Eq. (3.10),
dvx/dt = −bvx, whose solution is

It shows that vx declines exponentially from v0x, the initial x

velocity, when air resistance is included. Integrating vx to find x
gives

where a starting position at the origin is assumed.

For the y component of the velocity, dvy/dt = −bvy−g, and

the solutions are

Note that the terminal y velocity as t → ∞ is −g/b.

Time t may be eliminated from x and y to get the trajectory

as
 Figure 3.5: The trajectories for projectile motion with linear air
−1
resistance at a large initial speed (b = 1 s , v0 = 200 m/s) for several
firing angles.

Several sample trajectories are shown in Figure 3.5. Unlike

ideal projectile motion, the trajectories with air resistance are
no longer front-back symmetric. Note the steep drop-off after
the peak at larger angles. It also shows that the range increases
with decreasing angle. In addition, the maximum range is not
at 45°. It is around 20° in this case (see Exercise E3.9).

Explicitly, the range, R, is defined as the x coordinate where

y = 0, so

The range R is a root to Eq. (3.17), a transcendental

equation. There are no known solutions in terms of elementary
functions, though analytic solutions do exist, albeit in terms of
a special function (see Section 3.4). To find the root without
resorting to special functions, we need a numerical root finder.
We take up this subject next.

3.3.3 ROOT FINDERS

In scientific computing, we frequently encounter situations just
described above where we need to find the roots of equations.
Root finding is an important operation that we will be using in
several chapters ahead. The basic task of root finding may be
stated as follows: Given a function f(x), find a root x* such that
f(x*) = 0. Equation (3.17) is such an example with f(R) equal to
the LHS of the equation. We shall limit our discussion to two
common, practical methods.4

Bisection method
The bisection method is a simple and sure way of finding a
root, provided we know in advance that there is at least one
root within an interval (bracket). It relies on the fact that the
function f(x) changes sign when passing through a root. The
basic idea is shown graphically in Figure 3.6.

Suppose a root is known to lie within an initial bracket [a,

b], that is to say, f(a) × f(b) < 0. The bisection method works
simply as 1-2-3 (see Figure 3.6): 1) cutting the bracket in half;
2) determining which new bracket contains the root; 3)
repeating steps 1-2 until the bracket is sufficiently small.
 Program 3.6 implementing these steps is given in Appendix
3.A. The function has the form bisect(f, a, b, eps). It
requires as input a user-defined function f(x), initial bracket
[a,b], and an optional error tolerance eps.

Figure 3.6: Bisection root finding method. Starting with an initial

bracket [a, b] in which a root lies, the interval containing the root is
successively halved (iterations marked 1 to 4), until the required
accuracy is reached.

Figure 3.7: Newton's method of root finding. The left panel shows
the successive steps of the process. In each step, the x-intercept of the
tangent line is found, which is used as the next estimate for the root.
The process is repeated until the change of the intercept is within
allowed accuracy. The right panel shows the unstable nature of this
method. In this case, the intercept cycles between two points
endlessly.
Newton's method
Compared to the bisection method, Newton's method for root
finding requires not only the function f(x) itself, but its
derivative f′(x) as well. It can be a lot faster and more efficient
when it works, but may also be unstable and less robust.
Nonetheless, there are situations when this method is useful
and appropriate, or even preferred.

The idea for Newton's method is illustrated in Figure 3.7.

Starting with the initial guess x1, Newton's method uses the
local slope to find the x-intercept of the tangent, x2 on the x-
axis. Presumably, the point x2 is closer to the actual root, and it
is used to find the next approximation, x3, and so on. Various
stages are related by

Unless the derivative f′ is readily available, Newton's method is

not suitable, especially if f′ is expensive to evaluate.

The mathematical basis of Newton's method is easy to

understand. Suppose we expand f(x + δ) with our trusted
Taylor series,
If we wish to find the correction δ which hopefully brings us
closer to the root where f(x + δ) ∼ 0, then from Eq. (3.19) we
have

This leads precisely to the correction terms in Eq. (3.18). The

root solver, newton(f, df, x, eps), is given in Program 3.7
(Section 3.A). We supply the function and its derivative f(x)
and df(x), the initial guess x, and the error tolerance eps as
before.

Root finders library

As with the library for ODE solvers we develop, we will put
both Programs 3.6 and 3.7 in a standalone library called
rootfinder.py, and import it before use. Practice the use of
bisect() and newton() with the next example and Exercise
E3.8. The bisection method is robust, i.e., given a root known
to exist within a bracket, it will find the root without fail. As
such, the bisection method should be the first method to
consider in a practical problem.

Numerical solution of range by root-finding

As an example, we solve Eq. (3.17) to obtain the dependence of
range R on launching angle θ0. We use the program below,
which yields the results shown in Figure 3.8.
Program listing 3.3: Range vs. angle ( range.py)

import rootfinder as rtf, math as ma # get root finders, math

2 import matplotlib.pyplot as plt # get matplotlib plot functions

4 def f(R): # range function, Eq. (3.17)

return R*(vy+g/b)/vx + g*ma.log(1.−b*R/vx)/(b*b)
6

g, b, v0 = 9.8, 1.0, 100. # g, linear coeff ., firing speed

8 x, y, dtr = [], [], ma.pi/180. # temp arrays, deg−to−rad conv

10 for theta in range(1,90):

vx, vy = v0*ma.cos(theta*dtr), v0*ma.sin(theta*dtr) # init vel
12 R = rtf.bisect (f, 0.01, vx/b−1.e−5, 1.e−6) # solve
if (R != None): x.append(theta), y.append(R)
14

plt.figure () # open fig, plot, label, show fig

16 plt.plot(x, y), plt.xlabel(’ angle (deg)’), plt.ylabel(’ R (m)’)
plt.show()

The function f(R) represents the range equation (3.17) to be

solved. It uses several global variables defined in the main
program, including vx and vy, components of the initial velocity
that depend on the launching angle θ0. The function is supplied
to the bisection root finder (line 12). Though the root lies
between zero and infinity, the lower end of the initial bracket
should be set to just above zero because it is a trivial solution.
The upper end should be a large number, but not so large that
the argument to the logarithm term in Eq. (3.17) becomes
negative. This yields a maximum of vx/b for the upper end.
Therefore, we set the bracket to [∈1, vx/b−∈2], ∈1,2 > 0, on line
12. The next line checks to make sure a valid root is found (not
None) before storing the results.

The results (Figure 3.8) for two initial speeds show a steep
rise from small angles to the maximum, and a shallower falloff
toward larger angles. The maxima occur at angles much earlier
than 45°, and there is no symmetry as there would be for ideal
projectile motion. Achieving the maximum range requires
optimal balancing between the time of flight and the horizontal
velocity. With air resistance, it is more important to have a
larger horizontal velocity and a smaller time of flight because
the former decreases exponentially (3.12), leading to a smaller
angle θmax that maximizes the range.

Figure 3.8: The range as a function of launching angle for projectile

−1
motion with linear air resistance (b = 1 s , v0 = 50 and 100 m/s).

We note that θmax decreases as the speed increases (∼ 22°

vs. 15°). Qualitatively, this can be understood as the balancing
 act discussed above. However, it begs an interesting question:
How fast does θmax decrease? How does it scale with the initial
speed? We turn to a special function next for answers.

.4 The Lambert W function

As mentioned earlier, Eq. (3.17) does not possess analytic
solutions in terms of elementary functions, so numerical
solutions have traditionally been obtained by means of root
finding. However, it turns out that it does have analytically
exact solutions in terms of a special function, the Lambert W
function. Though discovered (or rediscovered) only recently,
the Lambert W function has been linked to the solutions of a
growing number of problems where only numerical solutions
were thought possible previously. These problems include
projectile motion with drag, Wien's displacement law in
blackbody radiation, and the Dirac δ-molecule (see Section
9.3.2 and Ref. [90, 94] for more examples). Because it is
important to compare numerical and analytic solutions where
possible for testing numerical methods, we will briefly
introduce this special function and its application to projectile
motion with linear drag below.

The Lambert W function is defined as a function satisfying

the following inverse relation for a given z,
This definition is similar to stating that the square root
function f is defined such that f(x) × f(x) = x for a given value of
x ≥ 0. Some special values can be inferred from Eq. (3.21):

Figure 3.9: The Lambert W function for real argument x. The two real
branches are: the principal branch W0 (solid line), and the secondary
−1
branch W−1 (dashed line). The branch point is at x = −e and W =
−1.

The Lambert W function is generally complex-valued (see

Ref. [20, 46] for general properties). For our purpose we are
interested in the real-valued W(x), shown in Figure 3.9. There
are two branches, the principal branch W0, and the secondary
branch W−1.

The power series expansion for W is

 Asymptotically, W(x) behaves largely like the logarithm
function,

Without the prefactor, Eq. (3.21) would be the inverse of an

exponential function, i.e., an exact logarithm.

Evaluation of W
Unlike elementary functions, the Lambert W function has yet
to make its way on a handheld calculator with a dedicated
button. Fortunately, it can be computed efficiently and
accurately using the root finders we have discussed.5 For a
given x, the value of the W function is the root w to

according to Eq. (3.21). Whereas the bisection method would

certainly work, Newton's method is preferred here for its rapid
convergence. Making use of the derivative of f(w) in Eq. (3.25)
as

Newton's method for the W works as follows (see Eq. (3.18)):

Starting with an initial guess w1 and assuming local stability,
successive iterations wn approach rapidly to the true value as,
 It only remains to determine the initial guess w1. We choose
w1 to be

Equation (3.27) with the seed from (3.28) always converges to

the correct branch W0 or W−1.

The actual implementation for the Lambert W function is

given below.

Program listing 3.4: Lambert W function ( lambertw.py)

1 import math as ma # import math library

def lambertw(x, branch = 0): # branch = 0 or −1
3 if (x<−1.0/ma.e or branch<0 and x>=0.0): return None # bad arg
w, nmax = x, 20
5 if (x > 1.5 or branch == −1): w = ma.log(abs(x))

7 for i in range(nmax):
w, v = (w*w + x*ma.exp(−w))/(1.0+w), w # Eq. (3.27)
9 if (abs(w−v) <= 1.e−15*abs(w)): break # to double precision
return w
 The module lambertw() accepts an argument x and an
optional branch switch. If the argument is invalid, a type None
is returned. Otherwise, it computes the value by Eq. (3.27) and
checks for convergence in the loop. The convergence is fast,
usually to machine accuracy in just a few iterations in nearly all
cases.

There is one exception, however. It occurs near x ∼ −1/e for

both branches, where f(w) ∼ −1 and the derivative vanishes, f′
(w) ∼ 0. This causes slow convergence in Newton's method in
the immediate neighborhood around −1/e. If accurate values
are required in this narrow region, we may switch to the
bisection method which does not suffer from the vanishing
derivative problem. Alternatively, we may also use the rational
polynomial approximation (Section S:3.A). Otherwise, Program
3.4 works very well for computing the Lambert W function on
the real branches, W0 and W−1. We will use it next.

Analytic solution of range by Lambert W function

Until recently it was thought that the transcendental range
equation (3.17) admitted only numerical solutions. Having
introduced the Lambert W function, we are now ready to
discuss how it can be used to solve Eq. (3.17) analytically.6 To
begin, Eq. (3.17) may be rewritten in exponential form as
 Our strategy is to cast the above equation into the same
form as the defining equation for the Lambert W function
(3.21), where the multiplicative factor in front of the
exponential is the same as the exponent. This can be
accomplished by multiplying both sides of (3.29) by c
exp(−cv0x/b) to give

All R-dependence has been moved to the LHS of Eq. (3.30),

and the prefactor is the same as the exponent. Comparing Eq.
(3.30) with (3.21), we identify the following,

Expressing R in terms of W(z) and explicitly substituting in

c, we have for the range

This is the analytic solution for the range in terms of the special
Lambert W function. Since the argument z in Eq. (3.32) is
negative, and W(z) is multivalued with two branches (Figure
3.9), how do we decide which one to use, W0 or W−1?
Mathematically, both are correct solutions. Physically, the valid
choice is the primary branch W0, since W−1 would produce a
 negative range. It is understood that W0(z) must be used in Eq.
(3.32).

With the use of the special W function, projectile motion

with linear air resistance is now completely and analytically
solvable. The analytic solutions can help us determine the
accuracy of the numerical methods, and to validate the
program (see Project P3.2). Furthermore, it can also help us
answer several interesting questions, including one posed
earlier about the scaling law, namely: How does the range-
maximizing angle, θmax, scale with the initial speed? It turns
out the scaling law is rather simple but powerful,

It predicts that, for increasing speed v0 (or equivalently drag

b), there exists a small but finite angle that maximizes the
range. Proof of the scaling law and its further exploration are
left to Exercise E3.12.7

.5 Quadratic air resistance and spin

In the preceding sections we had a thorough airing-out of
projectile motion with linear resistance. But in most cases, the
quadratic term in Eq. (3.3) dominates. No known analytic
solutions exist with this type of resistive force, except for one
special case where we restrict the projectile to a straight up-
down motion.
3.5.1 QUADRATIC AIR RESISTANCE
For the general case involving quadratic drag forces, we will
have to rely on an ODE solver to get numerical solutions. The
equations of motion similar to Eq. (3.10) but for quadratic drag
forces are

where is the projectile speed. The appearance of

the square root in Eq. (3.34) makes the problem unsolvable
analytically.

Numerically, we need a diffeq that returns the RHS of Eq.

(3.34) for quadratic drag. We call it Quad_drag given below.

def Quad_drag(Y, t): # returns RHS of Eq. (3.34): [vx, vy,

ax, ay]
v = np.sqrt(Y[2]*Y[2] + Y[3]*Y[3])
return np.array([Y[2], Y[3], −b2*v*Y[2]/m, −g −
b2*v*Y[3]/m])

Being equivalent to Lin_drag in Program 3.2, Quad_drag

assumes the array Y[] holding x, y, vx, vy in that order. We can
plug Quad_drag in an ODE solver just like in Program 3.2 and
obtain the desired results.
 One complication with quadratic air resistance is that the
drag coefficient Cd changes with speed, and has a dramatic
drop near critical speeds (Figure 3.4). The effect can be
simulated with Quad_drag, and sample results are shown in
Figure 3.10 for both constant Cd and changing Cd from Eq.
(3.8).

Figure 3.10: The trajectories for projectile motion (baseball) with

quadratic air resistance with constant Cd (solid), changing Cd
(dashed), and zero Cd (dotted).

With realistic drag coefficient used, we can see significant

effects of air resistance. As expected, a constant Cd = 0.5
produces the shortest range. A changing Cd from Eq. (3.8)
increases the range by about 20%, mainly because of reduced
drag at the critical speed. Of course, both are significantly
smaller than the ideal range without air resistance.

It is interesting to note that greater dynamical influence

happens sooner than the closeness of the trajectories in Figure
3.10 seems to suggest, because the points in the trajectories
correspond to different moments in time. In fact, the greatest
difference in acceleration, particularly the horizontal
component, occurs at the beginning where the speed is largest.
To a significant degree, this initial phase determines
subsequent motion, including the difference in range.

Even with air resistance, projectile motion remains in a

vertical plane. Sideway (transverse) motion can occur if airflow
such as wind is introduced. In that case, the motion becomes
three-dimensional, and 3D simulation is necessary. We can
extend the model because the drag force depends on the
relative velocity, , between the projectile and air (wind). If air
is flowing with a wind velocity , the drag force becomes

The full 3D equations of motion are (see effect in Project

P3.8)

3.5.2 EFFECTS OF SPIN

Transverse motion can also occur if airflow is deflected off the
direction of motion. Many an example can be found in
everyday life, including in sports such as baseball and ping
pong where the ball can move in curving, lateral directions.
 Redirection of airflow can be accomplished by moving
objects having uneven shapes or spin. Figure 3.11 shows a
sketch of airflow around an aerofoil and a spinning ball in a
wind tunnel.

Figure 3.11: Schematic airflow around an aerofoil (left) and

measurement of a suspended ball spinning in a wind tunnel (right).
The ball is spinning counterclockwise (Photo credit: F. N. M. Brown,
Univ. of Notre Dame).

It is clear from the sketch that as air flows past the aerofoil,
it is bent downward. There must be a net downward force
exerted on the airflow by the aerofoil. By Newton's third law,
the aerofoil experiences an equal magnitude but opposite force
from the airflow, i.e., an upward lift.8

Similar effect exists for a spinning ball, as the wind-tunnel

experiment shows. The ball is spinning counterclockwise and
the wind is blowing to the left. We can see a puff of smoke
downstream which comes off the spinning ball, and is deflected
downward. The airflow over the top is “carried” around the ball
more due to rotation. Again, by Newton's third law, this means
a net downward force is acting on the air mass due to the
spinning ball. The ball must experience a reciprocal upward
force. This effect is known as the Magnus effect (force). It is
schematically shown in Figure 3.12.

Figure 3.12: The Magnus effect on a spinning ball. Airflow past the
spinning ball (counterclockwise) is bent downward. The direction of
the force on the ball is given by the vector product ,
upward here.

The magnitude of the Magnus force is found to be

proportional to the angular velocity as well as to the linear
velocity of the ball. Its direction is given by the cross product
. The final result is

where α is a constant with the dimensions of mass. Because the

direction of the Magnus force is perpendicular to the velocity,
the ball will always curve, but will not change speed by this
force alone (no work). Thus, if a ball travels forward spinning
 around its vertical axis, the Magnus force will be lateral,
inducing a sideway motion to either the left or the right.

Analogous to Eq. (3.7), the Magnus force for a sphere is

often written as

where CL is called the lift coefficient. The virtue of writing it

this way is that when , the magnitude is simply Fmagnus =
CLρAv2/2.

Undoubtedly, we expect CL to depend on quite a few things

such as the roughness of the ball. Given the roughness, CL is
experimentally found to be mainly a function of the so-called
spin parameter, S = rω/v. Putting all together, we have

Here, ρ and A are the air density and cross section, respectively
(same as in (3.7)), and r is the radius of the ball. Effects of both
drag and Magnus forces should be included in more realistic
simulations of projectile motion, and are considered next.

.6 Physics of ball sports

Sports games such as baseball, ping pong (table tennis), and
soccer bring out the fans to share the excitement. To a
physicist, the games are fascinating because they bring out the
interesting effects due to drag and spin [22]. At times it seems
the ball is doing nearly impossible tricks as it moves from point
A to point B, be it a curveball, a back chop, or a corner kick. We
have discussed the basic physics needed to understand the
complexity in the ball movement. We are interested in the
combined effects of drag and spin on a ball in flight.

3.6.1 PHYSICS OF BASEBALL

Baseball may be low in scoring (normally), but very high in
physics content [2, 3]. We lead off with baseball in our
discussion.

Lift coefficient for a baseball

Like the drag coefficient, accurate lift coefficient CL in Eq.
(3.39) needs to be determined experimentally. Figure 3.13
shows experimental data of CL as a function of the spin
parameter S for a baseball.
 Figure 3.13: The lift coefficient as a function of spin parameter for a
baseball. The line is a fit to the data [67] shown as symbols.

The data are an aggregate of several measurements using a

variety of techniques in wind tunnels and open labs. The value
of CL increases with S. Within the data scatter, it is
approximately linear in S, though there is noticeable curvature
near small values of S. As we might expect from the
parameterization of Cd (3.7), CL must have a more complex
dependence than on S alone. Experimental evidence shows that
CL is also dependent on the Reynolds number, although it is
relatively weak. Nonetheless, it is useful to have a simple form
similar to Eq. (3.8) for use in simulations. We choose a power
law as

This relationship is displayed in Figure 3.13.

Forces on a baseball
With the drag and lift coefficients determined, we can examine
and compare quantitatively the forces acting on a baseball in
flight. They are shown in Figure 3.14.

The quadratic drag force grows with v2 until around 30 m/s

where it starts to decrease due to the drop-off in the drag
coefficient near the critical speed (Figure 3.4). It rebounds
afterwards, is equal to the weight of the baseball (gravity) at
about 43 m/s (97 mph), and exceeds gravity eventually.
Therefore, quadratic drag has considerable effect on the path
and range of a baseball, as Figure 3.10 depicts.

Figure 3.14: The forces acting on a baseball as a function of speed.

The spin of the baseball is 200 rad/s (1900 rpm).

The Magnus force, on the other hand, grows monotonically

as a power law, slightly faster than linearly in v. Of course, it
would be exactly linear if the exponent in the scaling law (3.40)
were 1, which is reasonably valid within the scatter of Figure
3.13 for larger spin parameters S. In the latter case, it is simpler
to use Eq. (3.37) with a constant α for the Magnus force.

Because the Magnus force is perpendicular to the velocity, it

always deflects the moving body, making it appear to “curve”
and more perceptive to visual detection by the naked eye. This
is especially true in sideway curves where gravity has zero
effect.

The equations of motion, including gravity, drag , and the

Magnus force , are

Here is the position vector, with x referring to the

forward motion, y the vertical motion, and z the lateral
(sideways) motion. This is a set of 6 ODEs to be solved.

Before doing a full scale simulation, it is useful to have an

estimate of the effects. Let us estimate the magnitude of the
Magnus force at speed v = 30 m/s (∼ 70 mph) and angular
velocity ω = 100 rad/s (950 rpm). By scaling from Figure 3.14,
we obtain a Magnus force that is roughly 1/6 of gravity, which
is equivalent to an acceleration of g/6. If the ball is thrown
from the pitcher's mound, it takes about 0.6 seconds for it to
travel to the home plate (a distance of 18.4 m). Over this
period, the Magnus force causes a lateral displacement of 0.3
meters (nearly one foot), a substantial amount of adjustment
for the hitter. For comparison, a ball would drop about 1.8 m in
the same period by gravity.

Visualization of realistic flight of a baseball

We solve Eq. (3.41) and display the motion with VPython in
Program 3.8 listed in Section 3.B. The trajectory is shown in
Figure 3.15.

Figure 3.15: Views of a curveball from the front, behind, and above
(bird's eye). Also shown is the ideal projectile motion (darker/green,
without the ball). The coordinates are such that from the bird's eye, x
axis points to the right, y axis out of page, and z axis down.

The essential physics is contained in the function baseball().

def baseball(Y, t): # Y = [r, v] assumed

v = Y[1] # name v for clarity
 fm = alpha*vp.cross(omega, v) # Magnus force
a = (fm − b2*vp.mag(v)*v)/mass − [0,g,0] # minus g−vec
return np.array([v, a]) # np array

Because the motion becomes three-dimensional when spin is

involved, it is much easier to use vector objects. The program
uses vectors as the basic building blocks instead of individual
vector components like x, vx, etc. Vector operations can be
simulated using NumPy arrays, which can be added or
multiplied like true vectors (see Program 2.5 and Section 1.D).
As stated in Chapter 2, our vectorized ODE solvers like RK4
works transparently on vector objects as on scalar objects
without any change.

The input Y is assumed to be a 2×3 NumPy array containing

the position and velocity vectors as . The output is the
RHS of Eq. (3.41) in a NumPy array consisting of velocity and
acceleration vectors . We calculate the Magnus and drag
forces using VPython's vector functions such as the magnitude
vp.mag() and the cross product vp.cross(). This makes our life
easier and the code more compact. Note that in the calculation
of acceleration, the gravity vector is simply represented by
a list −[0, g, 0], not a NumPy array. This is possible because
when a list is added to a NumPy array, it is automatically
broadcast to an array type before adding.

The actual trajectory can be seen to rise above and curve to

the left relative to ideal projectile motion (Figure 3.15). This is
due to the ball rotating with a back and side spin, i.e., ωz > 0
(back) and ωy > 0 (side).

Quantitative results are shown in Figure 3.16. Both the

lateral shift (z, curving) and the vertical rise (Δy) represent the
difference between two cases: with or without spin (drag is
included in both cases). The side spin – rotation about the
vertical axis – is responsible for curving to the left (negative z),
and the back spin for the rise (lift). The amount of curving and
rise is nearly identical because the angular velocity components
are the same. We note that the lift produced by the back spin
actually increases the range of motion. This fact turns out to be
very important in golf for long drives as we will see shortly.
Conversely, we could reverse it to a top spin (as if rolling in the
forward direction). Then the ball would appear to sink faster.

The program uses constant drag and lift coefficients. It can

be improved by using variable Cd and CL. Then, the simulation
could be considered fully realistic. Even then, we must
remember that there are uncertainties and fluctuations in
actual games, so our results should not be taken as perfectly
accurate. However, the trend and conclusions we draw from
these effects are valid.
 Figure 3.16: The lateral shift (z) and vertical rise (Δy) of a baseball
with spin relative to no spin. The parameters are the same as in
Program 3.8.

A final caveat: in all likelihood, the ball's spin rate (ω) is not
constant, and there is little data at present on how it precisely
changes. If we assume the torque is proportional to angular
velocity, we expect an exponential decay (see Eq. (3.12)) as

where ω0 is the initial angular velocity and τ the characteristic

decay time.

3.6.2 GOLF, PING PONG, AND SOCCER

We briefly discuss key elements of physics involved in golf,
ping pong, and soccer games. We mainly present the results,
and leave the details to be investigated in projects.
Long golf drives
Golf balls have a roughened surface with small depressions, or
dimples. Coupled with its smaller size (diameter 4.3 cm) and
mass (46 g), and high spin rate (up to 10,000 rpm), drag and
Magnus forces can have much larger effects on the range and
direction of golf drives.

The drag coefficient for a golf ball is qualitatively similar to

data for smooth balls shown in Figure 3.4. Quantitatively,
however, the critical speed happens at a much smaller value,
and the dip, present in baseball data, appears to be much
smoothened [32]. As a result, we can approximate the drag
coefficient by dropping the third term from Eq. (3.8) as

with v in m/s.

The lift coefficient is also qualitatively similar to that for

baseball (see Figure 3.13), at least at higher Reynolds numbers
(Re ≥ 105), and can be roughly described similarly to Eq. (3.40)
 Figure 3.17: Trajectories of golf with different amount of spin.

Sample results are shown in Figure 3.17 for a firing angle of

20° and speed of 60 m/s. For simplicity, we assume constant
backspin only. The spin rates are none, normal (∼ 8000 rpm),
and double spin. Compared to the case of no spin, the range is
increased by roughly 1/3 with normal spin, and nearly 1/2 with
double spin. The increase in range is due to lift which primarily
increases the height, and hence the time of flight. The change
in the maximum height is even more dramatic, approximately
doubling in each step. It shows the strong influence of the
Magnus force that can be greater than gravity and drag
combined. We can see this in the double spin case where the
beginning part of the trajectory actually curves upward.
Another notable feature is that the front-back asymmetry is
less pronounced for golf than for baseball when spin is
included. The back end of the trajectory falls off less steeply,
owing to the relatively strong Magnus force that “stretches” the
peak out on both sides.
 If the rotation is not purely backspin, curving can also
occur. This may be used to counter such factors as wind or
other obstacles. Of course, given how strongly spin affects the
trajectory of a golf ball, controlling the flight by imparting a
proper spin on the ball can be tricky.

Ping pong smash

Ping pong games are played with a ball that has a 4.0-cm
diameter and 2.7-g mass. Unlike other ball sports, ping pong
balls have a rather smooth surface. It is played over a table that
is 2.72 m long and 1.53 m wide. In the middle of the table is a
net, 15.3 cm high.

Ping pong is rather unique due to its small mass and the
compact playing area. Of all ball sports, spin is most important
in ping pong games. Top spin rates can easily reach 6000 to
8000 rpm, and top speeds can be more than 40 m/s. These
factors make ping pong one of the fastest sports where quick
reaction and accurate anticipation of ball movement is crucial
[98].

Figure 3.18 shows part of a rally in a singles ping pong game

at the 2012 London Olympics. The plot shows the last two plays
by each of the players (not shown). It starts with the player to
the right hitting the ball from the upper-right corner, and ends
with the player to the left who returns the ball with sufficiently
large side-spin that the ball curves and bounces off the table
near the edge, out of the opponent's reach. The right panel
shows the last play, with a dotted line superimposed to
illustrate the amount of curving.

Because ping pong balls are smooth, we expect the drag

coefficient shown in Figure 3.4 should work well. Furthermore,
since the critical speed for smooth balls occurs at a rather large
value relative to the speeds ping pong is played at, we can safely
set the drag coefficient to 1/2. For the lift coefficient, available
data suggests that a linear function of the spin parameter
should be adequate [83]. Therefore, we can use the following in
our ping pong simulation,

Figure 3.18: Overhead view of trajectories of a ping pong ball in a

game at the 2012 London Olympics. Left: the rally starts from upper-
right corner and ends at the lower-right corner when the ball bounces
off the table. Right: the last return in the rally showing curved path;
the dots are in a straight line to guide the eye.
 Figure 3.19: Trajectories of ping pong with different amount of spin.
Left: side view; Right: vertical view.

Representative results are shown in Figure 3.19. Parameters

are used to mimic the conditions of an overhand smash: an
initial speed of 25 m/s, and a spin of 60 rev/s (3600 rpm) in
the direction (1, 0, −1), i.e., roll-spin (ωx > 0) and topspin (ωz <
0). The coordinates are such that the x-axis runs along the
length of the table, the z-axis along the width, and the y-axis is
vertical as usual. The left plot again shows three cases, no spin,
normal spin, and double spin. Without spin, the ball flies past
the far end of the table, and the player loses a point. But with
normal topspin, the Magnus force is downward (negative lift).
The ball sinks, just clears the net, and lands in play. This is a
good smash. In this case, we also allow the ball to bounce back
up with a loss of energy, i.e., reversing . Too much
spin, the ball accelerates downward too fast, and is blocked by
the net (but allowed to continue in the simulation).

In the overhand smash, we also assume a roll-spin in the

positive x-direction, a clockwise rotation from the attacking
player's perspective. This causes a slight curving motion to the
left (Figure 3.19, right panel). The amount is small because in
the cross product , the z component of the Magnus force is
proportional to ωxvy. The initial value of vy is small, and |vy|
increases due to gravity and the negative lift as the ball travels
across. Hence we see increased curving later on. After the
bounce (normal spin), the vertical velocity is reversed, and that
slows down the rate of curving.

In comparison to baseball, the effect of spin is about 5% to

10% for ping pong judging from Figure 3.19 (left), and 1% to 2%
for baseball from Figure 3.16. Spin plays a more important role
in ping pong games.

Soccer corners
Last but not least, we turn our attention to the most popular
sports the world over, soccer. The soccer ball is considerably
bigger and heavier than the cases discussed so far, but the
effects of drag and spin are no less important [97]. Casual
spectators can easily spot the curving of the ball in long passes.
Curving is such an important art in soccer that set plays such as
direct kicks or corner kicks are major scoring opportunities. In
direct kicks, for example, players form a human wall to block
the direct path of the ball to the goal. However, a skilled player
could curve the ball around the wall and score. Direct scoring is
also possible, albeit more difficult, in corner kicks.

Typical soccer speeds are below 30 m/s, and spin rates

roughly under 160 rad/s (25 rev/s). The drag and lift
coefficients for soccer are shown in Figure 3.20. The drag
coefficient again exhibits the critical speed phenomenon, but
the dip is relatively shallow because of low roughness of soccer
balls. We see weak dependence of the lift coefficient on the
Reynolds number Re: a higher Re leads to a slightly higher CL.

Figure 3.20: Drag and lift coefficients for soccer [5, 36]. The data for
the lift coefficient correspond to different Reynolds numbers from
5
3.3 to 4.5 × 10 .

The parameterized expression similar to Eq. (3.8) for the

drag coefficient can be written as

This relation is shown in Figure 3.20 as the dashed line. The

average lift coefficient is modeled as
Except for the smallest spin parameter (S) values, we may
assume CL as a linear function in S, with appropriate
adjustment of the constant factor. In this case, we can use the
constant α defined in Eq. (3.39).

Using these coefficients, we have the necessary physics to

simulate realistically the flight of soccer balls with a code
similar to Program 3.8. Figure 3.21 displays a corner kick
which, absent a virtual goal keeper, would result in a direct,
curling goal.

Figure 3.21: Trajectories of a soccer ball in a corner kick with

(curving) and without spin, from forward (top) and reverse (bottom)
angles.
.7 Shooting methods
We have dealt with initial value problems, that is, given initial
conditions, we integrate the equations of motion to find out
where the trajectory ends up. There is another scenario in
which we want the ball to end at some specified position, and
we wish to know what initial conditions will lead it there. For
instance, if we want to kick a ball so that it clears a bar (or hits
a spot) at certain distance, what kicking angle should we aim
initially? Problems of this kind are called two-point boundary
value problems.

One way to solve such a problem seems straightforward,

trial and error: pick a value, run the simulation, check the
result, adjust and repeat. As illustrated in Figure 3.22, we want
to find the launching angle leading the projectile to the
boundary point at (xb, yb). The first pass overshoots, the second
one undershoots, and the third try is the correct angle. This is
the basic idea of the shooting method.

Figure 3.22: The shooting method. Trial 3 is the correct solution.

 Manual adjustment can work, but is inefficient if high
accuracy is required. In that case, it is better to cast the
problem into a form suitable for automatic adjustment. Below
we discuss an example using root finders.

We assume a projectile is launched from the origin with a

given speed v0. Our goal is to find the firing angle θ such that
the projectile hits the point at (xb, yb) (Figure 3.22). For a given
θ, let the projectile reach xb in time t, where its vertical position
is y(t). The condition for the correct θ satisfying the boundary
values is

The first equation in (3.48) defines time t, which is an

implicit function of θ. We rewrite Eq. (3.48) as a pair of
functions

We have treated θ explicitly as the independent variable in Eq.

(3.49b). We can solve the two equations by finding the roots to
fx(t) = 0 for t first, then fy(θ) = 0 for θ, the desired solution.

The shooting method can be implemented as follows.

 1. For a given θ, solve (3.49a) for t, by integrating the
equation of motion such as (3.1) or (3.34) to obtain x(t)
and by root-finding as necessary.
2. Find y(t) by integrating the same equations of motion to
t. Return y(t) − yb as fy(θ), (3.49b).
3. Feed fy(θ) to a root finder to obtain the desired θ, using
the bisection method to be safe.

This strategy is used in sample Program 3.5.

Program listing 3.5: Shooting method ( shoot.py)

1 from _future_ import print_function # use print() as

function
import ode, rootfinder as rtf, numpy as np # ode, root solvers,
numpy
3
def proj(Y, t): # ideal projectile motion
5 return np.array([Y[2],Y[3], 0.0, −g]) # [vx,vy,ax,ay]

7 def fy(theta): # return f as a func of theta

Y = [0., 0., v0*np.cos(theta), v0*np.sin(theta)] #
[x,y,vx,vy]
9 t = xb / Y[2] # Step 1: time to xb
h = t / nsteps
11 for i in range(nsteps):
Y = ode.RK2(proj, Y, t, h) # no need to
update t
13
return Y[1] − yb # Step 2: y(θ) −
yb
15
# number of steps, g, init speed, x and y boundary values
17 nsteps, g, v0, xb, yb = 100, 9.8, 22., 40., 3. # para.

19 theta = rtf. bisect (fy, 0.6, 1.2, 1.e−6) # Step 3: shoot for θ
if (theta != None): print(’ theta(deg)=’,theta*180/np.pi) #
result

To be concrete and avoid unnecessary clutter, we neglect air

resistance, i.e., assume ideal projectile motion. This simplifies
the first step (line 9) since we can find the time simply as t =
xb/v0x without integrating the equations of motion or using a
root finder.

In the function fy(theta), y(t) is calculated using the RK2

ODE solver. Though it is unnecessary because analytic
solutions are available, we do it this way so it can be easily
generalized to cases involving air resistance, for example. By
the same token, the value of nsteps does not matter in the ideal
case, because RK2 yields exact solutions for ideal projectile
motion regardless of the step size. After setting the initial speed
and boundary point (line 17), the bisection method is called to
find the solution. The key to be successful lies in choosing the
initial bracket. There are usually two solutions if the initial
speed is sufficiently large. In the present case, the solutions are
33.1° (ascending) and 61.2° (descending).

Chapter summary

The focus of this chapter is on one of the most classic and

celebrated problems in physics, realistic projectile motion. We
have discussed the effects of air resistance and spin. The drag
force is broken down into linear and quadratic powers in speed.
We modeled them separately. In the case of linear drag, we
have presented a new method of solution in terms of the
Lambert W function. We briefly described the key properties of
W and several ways of evaluating it. With the help of the W
function, the closed-form solutions enable us to compare
numerical and analytic solutions and to gain more insight than
possible with only numerical solutions, such as the asymptotic
scaling law for the angle that maximizes the range.

For the quadratic drag which dominates in practice, we

discussed the dependence of the drag coefficient on speed,
models to parameterize it, and its effect on projectile motion.
We also introduced the Magnus force and discussed its effect,
as well as modeling of the lift coefficient. Finally, simulations
combining drag and spin are applied to the study of physics of
sports, including baseball, golf, ping pong, and soccer. Both
qualitative and quantitative results have been presented to help
explain the observed effects.

Numerical methods for root finding have been discussed,

including bisection and Newton's methods. They are included
in one of the libraries that we will use frequently in the
chapters ahead. We introduced the shooting method for two-
point boundary value problems. We have also introduced
visualization techniques using 3D capabilities of VPython and
applied them to illustrate realistic projectile motion effectively
and from unique perspectives.
.8 Exercises and Projects
EXERCISES
E3.1 Modify Program 3.1 to use the RK2 ODE solver.
E3.2 Use dimensional analysis to show that the linear drag
force must be of the form given by Eq. (3.4) without the
explicit factor 6π.
E3.3 Calculate the speed at which the transition from laminar
to turbulent flow occurs for a baseball. The viscosity
and density of air are μ = 1.810−5 kg/m/s and ρ = 1.2
kg/m3 under standard atmospheric conditions. The mass
of a baseball is 0.15 kg, and the diameter is 7.4 cm.
E3.4 Estimate the size of falling rain drops if their speed
ranges typically from 10 to 30 km/h.
E3.5 Find the speed at which the linear and quadratic drag
forces are equal for a baseball. Do the same for a ping
pong ball.
E3.6 Verify that Eqs. (3.13), (3.14) and (3.15) are solutions to
(3.10). Also derive the range (3.16).
E3.7 In the limit b → 0, show that Eqs. (3.16) and (3.17)
reduce to the expressions for ideal projectile motion,
with the former to

and the latter to Eq. (3.2), respectively.

E3.8 (a) Use both the bisection and Newton's methods to find
3
 the roots of f(x) = 5x−x3. Investigate what happens
when the initial guess is at ±1 in Newton's method.

(b) Alternatively, find the roots using SciPy's equation

solver, fsolve, which requires a (vector) function and
an initial guess, as shown below:

In [1] : from scipy.optimize import fsolve

In [2] : def f(x): return x*(5.−x*x)
In [3] : x = fsolve(f, −2.)

E3.9 Solve Eq. (3.17) to find R as a function of θ with both

the bisection and Newton's methods. Take b = 1 s−1, and
do this for several initial speeds: 10, 20, 50, 100, 200,
and 500 m/s. Note the position of the range-maximizing
angle in each case. Compare with the scaling law
(3.33). Be warned that it is challenging to calculate R by
root-finding at large angles and large speeds.
E3.10 (a) Show that W(x ln x) = ln x. (b) Solve xx = a, and
express x in terms of W. Verify your answer for a = 4
and 27. Here you can also use SymPy's solve,
In [1] : from sympy import *
In [2] : x, a = symbols(’ x a’)
In [3] : solve(x**x−a, x) # solve xx − a =
0
Out [3] : [exp(LambertW(log(a)))]

(c)* Solve Eq. (3.29) with SymPy to obtain Eq. (3.32).

(d) Verify the first two terms in the series expansion of
W, Eq. (3.23). Note that W′ = W/[x(1 + W)].
E3.11 Consider projectile motion with linear air resistance. (a)
Find the maximum height. (b) Calculate the time it
takes to reach the top, to fall back to the ground, and the
total time of flight, respectively. (c) Give the numerical
values for each if b = 1 s−1, v0 = 50 m/s, and θ0 = 20°.
E3.12 Prove the scaling law (3.33). Hint: simplify the range
(3.32) with the assumption β sin θ 1 using (3.23),

and solve dR/dθ = 0 in terms of the Lambert W

function. See Project P3.4 for a numerical analysis.

PROJECTS
P3.1 (a) Integrate animation into Program 3.2 so that the
projectile and its trail are displayed during flight as in
Program 3.1. Place the camera to capture the scene from
different angles, e.g., forward , reverse , top

(−ŷ), or other arbitrary angles. Also add a keyboard

check to the main loop so the program can be paused or
resumed with key presses.

(b) Modify the program and add two arrows to the

projectile indicating the vector forces of gravity and
drag, respectively. The lengths of the arrows should be
proportional to the magnitudes of the forces. Scale them
appropriately so they are clearly visible but not
overwhelming the scene. Note that the drag force (both
direction and magnitude) changes with time.
 Refer to Program 3.8 on useful VPython techniques.

P3.2 Numerically investigate projectile motion with linear air

resistance.

(a) Add necessary elements to Program 3.2 so the

position-time and velocity-time curves (x-t, y-t, vx-t and
vy-t) are graphed. In each case, add the corresponding
curve for the analytic results (Eqs. (3.12) to (3.15)) and
for ideal projectile motion.

Do the numerical and analytic results agree? The curves

may be too close to see any difference. Instead, plot the
differences, Δx = xnum − xana and Δy = ynum − yana as a
function of time. Comment on the error.

Now turn off drag (b1 = 0) and run the program again.
Do the results converge to the ideal case?

Note: We have assumed the step size h to be small

compared to the characteristic time scale τs. Estimate τs
in this case. How do we check whether h is adequate?

(b) Calculate and plot the range as a function of the

launching angle like Figure 3.8 for the same parameters.
This is best done automatically with a double loop: the
outer one over the angle (say in 5-degree intervals), and
the inner one over time. The range for a given angle is
the last value of x before y turns negative. Your results
should agree with Figure 3.8 to within the accuracy of
the step size.

(c) On the same graph as above, plot the analytic range

using the Lambert W function. To do so, you should
write a function that accepts as input the launching
speed and angle and returns the range given by Eq.
(3.32). The function can access other parameters such
as b and g as global variables. In the main code, add a
loop iterating through the angles (for a given speed) and
append the results to a list. How do the numerical and
analytic results compare? Are the differences as
expected for the step size h used? Halve h and repeat
the comparison.

As an optional challenge, find the angle θmax for which

the range is maximum, from both the numerical and
analytic results. Study the asymptotic behavior as the
speed (or β in Eq. (3.33)) becomes large. Plot θmax vs. 1/
β for 1 ≤ β ≤ 103. Compare with the prediction of the
scaling law (3.33). Is it easy to extract the ln β
dependence from the graph? Why? See also Project
P3.4.

(d) Plot as a function of the launching angle the time of

flight, τ, and the mean horizontal velocity, 〈vx〉. Compute
the latter as 〈vx〉 = Σvx/N where the sum runs over the
number of time steps N. Discuss your results. Plot 〈vx〉 ×
τ as a function of the launching angle and compare how
they relate to the range calculated earlier.

(e)* When recording the range or time of flight, it is

unlikely that the projectile would hit the ground exactly
at y = 0 because of the finite step size h. More often
than not, we will have the projectile above ground (y >
0) at t and below it (y < 0) at t + h. To pinpoint the
moment more exactly without too small a step size, we
can linearly interpolate between the two points just
above and just below.

Let (x1, y1) and (x2, y2) be the positions of the projectile
at t1 and t2, respectively. Connecting the points by a
straight line, the equation of the line is

Hence, if the projectile crosses y = 0 between t1 and t2,

we can approximate the range and time of flight as

Refine the calculation of R and τ accordingly. Change

the main loop in the program to keep the positions of
the last two iterations. Apply Eq. (3.52) after the loop.
 Compare your results with part (d). Optionally, do the
comparison for a larger step size h and comment on
your observations.

P3.3 Investigate one-dimensional free fall with quadratic air

resistance.

(a) Consider only vertical (y) motion in Eq. (3.34) (i.e.,

set x = 0, vx = 0). Calculate the terminal speed,
assuming b2/m = 10−2 m−1.

(b) Predict how position, velocity, and acceleration will

change with time if motion starts from rest. Sketch
them. With initial conditions y = 200 m and vy = 0,
simulate the motion until it hits the ground (y = 0). Plot
the y-t, vy-t, and ay-t curves. Discuss your results and
predictions. At what time and position does the velocity
reach 95% of terminal velocity?

(c) Animate the motion with VPython. Mark the

position of the projectile at regular time intervals. Make
sure to space out the marks so they are not too close
(say every 1/4 seconds or so). What do you conclude
from the trail of the position marks?

(d) We have used a constant b2. However, air density is

not constant, which means that b2 is a function of y. Let
us model the effect of changing air density. Assuming
constant temperature, thermodynamics predicts that air
density decreases exponentially with height,
 Here, ρ0 is the density at y = 0, and y0 9000 m. This

model is not terribly accurate, but not grossly inaccurate

either.

Repeat parts (a) to (c), assuming b2/m = 10−3 ×

exp(−y/y0) (m−1). You will have to adjust the initial
position so it reaches at least within 5% of terminal
speed before hitting the ground. Analyze your results
and observations.

P3.4 The project concerns numerical exploration of the

scaling law (3.33).

(a) Differentiate Eq. (3.32) with respect to θ, set dR/dθ

= 0, and show that the angle maximizing the range
satisfies

Consult Exercise E3.10 on the chain derivative of W.

Also see Exercise E3.12.

(b) Solve Eq. (3.54) for θmax as a function of β from 10

to 1000 using a root finder. Space out the data points by
increasing β by a constant factor, e.g., doubling it each
 time. Plot your exact results along with the scaling law
(3.33) on a semi-log scale. Discuss your findings.

P3.5 Consider a batted baseball hit near the ground level at

an angle θ = 32° above the horizontal with speed v0 =
150 km/h. Assume air resistance in the form of Eq.
(3.7).

(a) Is the use of quadratic air resistance justified?

Support your answer by computing the Reynolds
number for a typical size and speed of the baseball.
Obtain a reasonable value for the constant b2 (including
units). The relevant parameters for a baseball is given in
Exercise E3.3.

(b) Modify Program 3.8 to simulate the motion of a

baseball. As input, the program should ask the user for
v0 and θ. Verify the program works correctly.

(c) Plot the x-t, y-t, and y-x curves calculated with the
drag force and compare each with the respective curve
in the ideal case. Find the range and maximum height of
the baseball.

(d) Let us assume that when the baseball hits the

ground, it rebounds with 60% of speed (coefficient of
restitution), but keeps the same angle of “incidence”.
Plot the trajectory up to the third bounce. (Fun
exploration: let the bounce continue until it does not
 move appreciably. Find the maximum horizontal
distance traveled.)

(e)* Track the work done by the drag force and verify
that the work-energy theorem is satisfied by testing the
relation Wdrag ≈ E − Ei, where

is the work done up

to time t, E the energy at t (kinetic and potential), and Ei

the initial energy. Plot E, Wdrag, and E − Wdrag as a
function of time. This relation, another form of energy
conservation, may be used to validate the program and
the step size h.

P3.6 Let us continue the investigation of the batted ball with

regard to the angle maximizing the range and a scaling
law.

(a)With the same initial condition as in Project P3.5,

vary the launching angle θ and locate the optimum θmax
to within 0.1° for which the range is maximum. Use
linear interpolation (3.52) for improved accuracy.
Compare with the theoretical maximum range without
air resistance.

(b) How does θmax scale with large v0? First, make a
qualitative prediction. Now, calculate θmax for different
values of v0, say between 10 and 1000 m/s. Double the
speed each time to space out the data points. Plot the
results on a semilog scale for v0. You should automate
 the process of finding θmax. Keep θmax to at least two
significant digits. Does your prediction and the results
agree? Why or why not?

(c)* Assume a power scaling law . Visually

fit it to the data, and extract γ. Comment on how the

results compare with the results of Project P3.4 if
available.

P3.7 Like other ball sports, the American football is also full
of physics [33].9 It is typically thrown with a speed of
80 to 90 km/h. Use the optimum launch angle from
Project P3.6 if available, or else θ = 30°.

(a) Given that the mass of the football is about 400 g,

and the radius at the center of the football is ∼ 9 cm,
estimate the b2 value for quadratic drag (ignore the fact
that the football is prolate, and assume a reasonable
frontal cross section). Write a program, or use one you
developed, to calculate the range of the football. Should
we worry about any forward spin (zip)? Plot the
trajectory and compare with no drag. Are they
reasonable?

(b) Suppose the b2 above is for standard atmospheric

conditions. Let us assume that air density is inversely
proportional to temperature T (kelvin), so that the
temperature dependent coefficient is b2(T) = b2T0/T.
Assuming T0 = 300 K, repeat the range calculation as a
 function of T from T0 to T0−30 K in steps of 2 K. Plot
and discuss the results. What is the difference in range
between T0 and freezing (273 K)? What is the range
reduction per 5 K (∼ 9 F) drop in temperature?

P3.8 Study the effects of constant versus variable drag and

lift coefficients.

(a) Calculate the trajectories depicted in Figure 3.10 of

a baseball with constant Cd = 0.5 and variable Cd given
by Eq. (3.8). See Exercise E3.3 for relevant parameters
of a baseball. Use initial values vx = 30 m/s and vy = 20
m/s, for example.

(b) Add spin to the calculation. Let us give it a side-spin

rad/s. First, assume a constant ratio CL/S = 1.0

so α is a constant in Eq. (3.39). Second, use a variable

CL from (3.40). Compare the sideway motion curves (z-
t), and discuss the difference. Note the net displacement
in z direction at the end.

(c) Examine the equivalency between wind and spin.

Assume cross wind in the direction. Use Eq. (3.36)

in your simulation. Find the wind speed vwind which

gives the same net displacement as above. Convert it
into km/h or mph, and comment on your answer. This
may be done by trial and error, or by the shooting
method. Try the latter for added challenge!
P3.9 Let us investigate the necessary parameters to hit a
home run or a tree-top clearing golf drive.

(a) If the baseball leaves the bat at 35° above the

horizontal and with a backspin of 2000 rpm, what
should be the initial speed to make a 420-foot home
run? Use realistic drag and lift coefficients.

(b) Assume you want to hit a golf ball directly over the
tree top which stands at 10 m high and 120 m away.
With an initial speed of 60 m/s and a backspin of 8000
rpm, find the angle (at least two significant figures) so
your shot just clears the tree top. Compare the cases of
constant and changing drag and lift coefficients.

P3.10 Pick your favorite sport, and simulate an event or play

that interests you, e.g., a corner kick in soccer, or an
overhand smash in ping pong. Show your results both
quantitatively by graphs and visually via animation.
Additionally, study the effect of changing spin rate
during the flight according to Eq. (3.42). You will need
to use a reasonable τ.
.A Bisection and Newton's root finders
In this section we implement the root-finding methods
outlined in Section 3.3.3 and discuss their properties including
applicability and convergence.

Bisection root finder

The key step in the implementation of the bisection method is
to determine which new bracket contains the root (Section
3.3.3). It may be accomplished by first finding the midpoint
xmid and then deciding which end point to replace. To
determine which side the root is on, the function value f(xmid)
at xmid = (b + a)/2 (the first being c in Figure 3.6) is compared
to an end point, say f(a). If they are of the same sign, then a is
replaced by xmid and the new bracket for the root is [xmid, b].
Otherwise, b is replaced, and the bracket is set to [a, xmid]. This
strategy may be expressed as

The code below implements the above strategy.

Program listing 3.6: Bisection root finder ( bisect.py)

 def bisect (f, a, b, eps=1.e−6): # user−defined f(x) and bracket [a,b]
2 fa, fb, gap = f(a), f(b), abs(b−a) # end points and initial gap
if (fa*fb > 0.0): # no root in bracket
4 print(’ Bisection error: no root bracketed’)
return None
6 elif fa == 0.0: return a
elif fb == 0.0: return b
8

while (True):
10 xmid = 0.5*(a+b)
fmid = f(xmid)
12 if (fa*fmid > 0.0): # root in [xmid, b]
a, fa = xmid, fmid # set a=xmid and save a function call
14 else: b=xmid # root in [a, xmid]
if (fmid == 0.0 or abs(b−a) < eps*gap): break # root found
16

return xmid

This code needs a user supplied function f(x), an initial

bracket [a, b], and an optional relative accuracy eps (default:
10−6). It starts by checking easy cases first: if no root is
bracketed, it returns a type None;10 or if either end happens to
be a root, it just returns as such. The main loop computes the
midpoint, decides which end point to replace according to Eq.
(3.55). If the midpoint satisfies f(x) = 0, or the relative accuracy
is satisfied (line 15), a root is presumed found. The program
exits the loop and returns the root.

The bisection method never fails, provided the initial

bracket [a, b] contains at least one root. While for an arbitrary
function it is usually not trivial to locate the initial bracket, it is
relatively easy to do so based on physics, because physical
considerations should lead to a reasonable guess. For example,
for the projectile motion problem (3.17), we know the range
must be between (0, R0], where R0 is given by Eq. (3.2), the
range for ideal projectile motion.

The cost of the bisection method amounts to one function

call per iteration. The rate of convergence is linear, since each
iteration reduces the interval by half. Let δ be the bracket width
after n iterations, then δ ∼ |b − a|/2n, or n ∼ −log2 ∈ with ∈ =
δ/|b − a| being the relative error. For example, if ∈ = 10−6,
roughly 20 iterations are required. It is not the fastest, but fast
enough for most cases. Coupled with its robustness, it should
be the first root finding method to consider for a given
problem.

Newton's root finder

In implementing Newton's method, we need to be careful about
potential pitfalls lurking behind it. Figure 3.7 illustrates one of
those, where Newton's method happily jumps in a never-
ending loop. Other pitfalls might occur such that the algorithm
marches to ±∞ without bounds, unless safeguards are built into
the implementation of the method. We implement Newton's
method (3.18) below.

Program listing 3.7: Newton's method ( newton.py)

 1 def newton(f, df, x, eps=1.e−6): # user−defined f(x),df/dx, init root
nmax, fx = 20, f(x) # max number of iterations
3 if (fx == 0.0): return x

5 for i in range(nmax):
delta = fx/df(x)
7 if (i == 0): gap = abs(delta) # save initial gap
x = x − delta # ’improved’ root
9 fx = f(x) # prep for next round
if (fx == 0.0 or abs(delta) < eps*gap): break # root found
11

return x

To use newton(), the user supplies both the function f(x)

and its derivative df(x), the initial guess x, and the optional
relative error eps. The choice of the starting value can be
critical, it could mean a successful run or a not-so-successful
run into the abyss. We safeguard against the latter by setting a
maximum number of iterations nmax. However, no check is
made on whether the loop exited normally due to accuracy
satisfied (line 10) or just reached the maximum iterations.

The rate of convergence of Newton's method turns out to be

quadratic for well-behaved functions in the neighborhood of a
root. This means that if the initial error is ∈0, the error after
one iteration is . After n iterations, the error roughly scales
as , or n ∼ log2(ln ∈/ln ∈0). For instance, if ∈0 = 10−2 and
∈ = 10−16, then n ∼ 3, i.e., each iteration roughly doubles the
 number of significant digits. It is very efficient and usually
converges to machine accuracy in a few iterations. When we are
certain that Newton's method is locally stable in the proximity
of a root, it can be a powerful tool, and we should use it.

.B Program listings and descriptions

Program listing 3.8: Motion of a baseball ( baseball.py)

import ode, visual as vp, numpy as np # get ODE, VPython, numpy

2

def baseball(Y, t): # Y = [r, v] assumed

4 v = Y[1]
fm = alpha*vp.cross(omega, v) # Magnus force
6 a = (fm − b2*vp.mag(v)*v)/mass − [0,g,0] # minus g−vec
return np.array([v, a]) # np array
8

def set_scene(R): # draw scene, ball, trails, spin, info box

10 scene = vp.display(background=(.2,.5,1), forward=(−1, −.1, −.1),
center=(.5*R,1,0), ambient=.4, fullscreen=1)
12 floor = vp.box(pos=(R/2,0,0), length=1.1*R, height=.1, width=8,
color=vp.color.orange, opacity=0.7) # transparent
14 zone = vp.curve(pos=[(R,0,1),(R,1,1),(R,1, −1),(R,0, −1)], radius=.02)
ball = vp.sphere(pos=(0,0,0), radius=.2, material=vp.materials.rough)
16 trail = vp.curve(pos=(0,0,0), radius=0.04)
ideal = vp.curve(pos=(0,0,0), radius=0.04, color=vp.color.green)
18 spin = vp.arrow(axis=omega,pos=(0,0,0),length=1) # omega dir
info = vp.label(pos=(1.1*R,2, −2),text=’ Any key=repeat’)
20 return scene, ball, trail, ideal, spin

22 def go(x, y, vx, vy): # motion with full drag and spin effects
h, t, Y = 0.01, 0., np.array ([[x, y, 0.], [vx, vy,0.]]) # initialize
 24 while (Y[0,0]<R and Y[0,1]>0.2): # before homeplate&above ground
vp.rate(40)
26 t, Y = t+h, ode.RK4(baseball, Y, t, h) # integrate
ball.pos, spin.pos = Y[0], Y[0]−offset # move ball, arrow
28 spin.rotate(angle=phi), ball.rotate(angle=phi,axis=omega) #spin
trail.append(pos=ball.pos)
30 ideal.append(pos=(x+vx*t, y+vy*t−0.5*g*t*t, 0.)) # ideal case
while (not scene.kb.keys): # check for key press
32 vp.rate(40)
spin.rotate(angle=phi), ball.rotate(angle=phi,axis=omega)
34 scene.kb.getkey() # clear key
trail.append(pos=(0,0,0), retain=0) # reset trails
36 ideal.append(pos=(0,0,0), retain=0)

38 g, b2, alpha, mass = 9.8, .0013, 5e−5, .15 # parameters

R, omega = 18.4, 200.*np.array ([0,1,1]) # range, angular velocity
40 phi, offset = np.pi/16., 0.4*omega/vp.mag(omega)

42 scene, ball, trail, ideal, spin = set_scene(R)

while (1):
44 go(x=0., y=2., vx=30., vy=0.) # initially z=0, vz=0

As discussed in the main text, baseball() returns the

equations of motion, RHS of Eq. (3.41), in a NumPy array in
order to use a vectorized ODE solver. Subroutine set_scene()
does what it says: it orients the camera to the scene containing
various objects: the ground (floor), strike-zone, ball, trails
( vp.curve(), one for the actual motion, and another for ideal
projectile motion), angular velocity arrow, and an
informational text box, vp.label(). The floor is drawn as a
transparent box via the opacity parameter (line 13), so that
even if the camera is below the ground while being swung
around, the baseball remains visible. The lower the opacity
parameter, the higher the transparency.

The function go() takes input for initial conditions, and

integrates the motion until the ball reaches the strike-zone or
hits the ground. The array Y holds the position and velocity
vectors as , which are represented as NumPy arrays. At
each step, the positions of the ball and of the arrow
representing spin are updated. The rotation is effected via the
rotate method of the object (line 28), which rotates a given

angle counterclockwise about the object's axis. For objects that

do not have an axis such as the sphere, we must specify the axis
as a direction vector. After the numerical integration ends, the
second loop keeps rotating the ball and the arrow until a key
has been pressed (so we can admire the scenery, explore it from
different angles, or take a snapshot). This is done by
monitoring whether a key has been pressed (line 31). If so,
scene.kb.keys will be set to True in VPython, and the loop
exits. The function scene.kb.getkey() is called to clear the
keyboard buffer (line 34). Finally, the trails are cleared by
setting retain=0 (line 35) so fresh ones can be drawn in the
next go-around.

The main body of the program sets the necessary

parameters. The angular velocity vector is set (line 39) such
that the ball has a back and side spin. The purpose of the
variable offset, a vector along the direction of , is to center
the spin arrow at the ball (line 27). Once the scene is set, go() is
called repeatedly after each key press until the window is
closed.

1
We should not confuse the velocity dependence with kinetic friction which is
independent of velocity. For kinetic friction, the contact surface does not move with the
body.

2
Laminar and turbulent flows can be observed in a column of rising smoke, say from a
burning incense. At first, the column is smooth as silk, and is laminar. As the column rises, it
spreads and starts to develop twirls and twists, and becomes turbulent.

3
To quote the 5th century BC philosopher Laozi: the longest journey starts with a single
step.

4
SciPy also has a general root solver, scipy.optimize.fsolve. See Exercise E3.8.

5
The Lambert W function is also available in the SciPy special function library,
lambertw, and in SymPy as LambertW.

6
It can also be solved more quickly (and opaquely) with SymPy, see Exercise E3.10.

7
Numerical extraction of the scaling law is rather tricky because of the slow-varying
logarithmic term ln β in Eq. (3.33) (see Project P3.2).

8
Another common explanation invokes Bernoulli's principle. The flow velocity is greater
on the top than on the bottom, resulting in a net upward pressure difference, hence the lift.
But we think the third-law argument is more direct.

9
See the so-called “Deflategate” in which the New England Patriots team was alleged of
deflating or under-inflating the football to gain an unfair advantage in a playoff game, and
the science behind it in The New York Times Sports, “Upon scientific review”, January 30,
2015. The team won that year's Super Bowl. See also Section 11.4.4 on pressure and
temperature.

10
Having no root bracketed is not an uncommon error, so type None should be inspected
by the calling program to make sure a legitimate root is found.
Chapter 4
Planetary motion and few-body
problems
In projectile motion discussed in Chapter 3, gravity is
considered to be constant. This is an approximation, valid only
for distances small compared to the size of Earth. Now imagine
we stood atop Mt. Everest and fired a cannon horizontally. At
low speed, the cannonball would travel just like projectile
motion. With greater firing power and ever increasing speed,
the cannonball would travel further and curve less. At some
critical speed, it would fly all the way around Earth and come
back from behind (ignoring drag, of course). This is exactly
how the Moon moves around Earth, or the planets around the
Sun. In a sense, planetary motion is akin to projectile motion in
that a planet falls continuously toward the Sun. We can no
longer treat gravity as a constant force, however.

The motion of planets, or the heavenly bodies as they were

known, is a fascinating subject. Observations of planetary
motion led to the discovery of Kepler's laws which, strictly
speaking, are valid only for a two-body system (the planet and
 the Sun). The fact that these empirical laws could be
successfully explained by Newton's theory of gravity confirmed
the validity and universality of the classical theory of gravity.
However, beyond two-body systems, no analytic solutions are
known for the general configuration, even though simple three-
body systems are still at the heart of physics at all scales, such
as helium (two electrons plus the nucleus), water (one oxygen
and two hydrogen atoms), or celestial systems. These systems,
part of the so-called few-body problems, prove to be a fertile
ground for numerical investigation.

We discuss several interesting cases in celestial mechanics

ranging from differential precession of Mercury including
correction from Einstein's theory of relativity, to exoplanets
and three-body problems.

.1 Motion of a planet
Planetary motion is by and large governed by Newton's theory
of gravity. The force F obeys the inverse-square law

where r is the radial distance, and the constant k is

Here, M and m are the masses.

 From Newton's second law , the equations of
motion are

Note that the mass of the planet, m, is canceled out, and the
force components have 1/r3 dependence because of the vector
in the numerator. Assuming the motion is confined in the xy
plane, the vectors have only x and y components.

To see what a typical motion is like, run the following

program.

Program listing 4.1: Planetary motion ( earth.py)

1 import ode, numpy as np # get ODE solvers, numpy

import visual as vp # get VPython modules for
animation
3
def earth(id, r, v, t): # return the eqns of motion
5 if (id == 0): return v # velocity, dr/dt
s = vp.mag(r) #s=

7 return −GM*r/(s*s*s) # accel dv/dt, faster than s**3

9 def go():
r = np.array([1.017, 0.0]) # initial x,y position for earth
11 v = np.array([0.0, 6.179]) # initial vx, vy

13 # draw the scene, planet earth/path, sun/sunlight

scene = vp.display(title = ’Planetary motion’, # scene
start
15 background=(.2,.5,1), forward=(0,2, −1))
 planet= vp.sphere(pos=r, radius=0.1, make_trail=True,
17 material=vp.materials.earth, up=(0,0,1))
sun = vp.sphere(pos=(0,0), radius=0.2,
color=vp.color.yellow,
19 material=vp.materials.emissive)
sunlight = vp. local_light (pos=(0,0),
color=vp.color.yellow) #scn end
21
t, h = 0.0, 0.001
23 while True:
vp.rate(200) # limit animation speed
25 r, v = ode.leapfrog(earth, r, v, t, h) # integrate
planet.pos = r # move planet
27 if (scene.kb.keys): scene.kb.getkey(), scene.kb.getkey()
#pause

29 GM = 4*np.pi*np.pi # G*Msun
go()

We should see planet Earth orbiting the Sun as shown in Figure

4.1.

Figure 4.1: Motion of Earth around the Sun.

Do not be too concerned if Program 4.1 seems to have

increased complexity in a hurry, for we have yet to explain key
elements such as the model or the units in it. By the time we
finish Section 4.3, you should understand everything about this
program. We put the program upfront because it serves as a
central problem around which most problems are built
subsequently. We note, however, that the program follows the
basic templates and structure of programs we have seen so far,
especially Programs 2.7, 3.1, and 3.8. For instance, Program 3.1
first sets up the visualization scene, then goes into the “while”
loop for computation. This is basically what the main function
go() does here. The difference is that the former uses Euler's
method within the loop, while the latter calls the leapfrog
method separately. As stated earlier, most programs generally
follow this template, with problem specific differences.

In Program 4.1, the equations of motion (4.3) are computed

by earth() in a form suitable to leapfrog integration (see
Chapter 2, Section 2.4).1 It returns either the velocity or the
acceleration depending on the flag id, which is requested by
the leapfrog integrator. The function earth() is similar to
oscillator() in Program 2.7 but is two-dimensional, so it uses
two-element NumPy arrays to represent the position, velocity,
and acceleration vectors (lines 10, 11, and 7, respectively). As a
result, the acceleration is computed vectorially in identical
expression as is written in Eq. (4.3). Besides simplifying the
computation, it is also conceptually clearer with vector
operations. The advantage of this approach, last used in
Program 3.8, will be seen frequently in subsequent problems
(see Programs 4.3, 4.5, and 4.7).
 In the main code go(), the initial position and velocity
vectors are given as two-component vectors (lines 10 and 11).
These initial values represent the motion of Earth at the point
farthest away from the Sun (aphelion, see Table 4.1). The next
few lines (14 to 20) add visualization effects much like those in
Program 3.1. In order, they set the display window and viewing
direction, draw an Earth-like planet via material.earth
attribute, put a glowing (emissive) Sun at origin, and add a
light source representing sunlight (when attributes such as
position are given as (x, y) pairs, the z value defaults to zero).
Only the line for the “planet” is required, the rest are just bells
and whistles, if you will. The make_trail attribute for the planet
will leave a trail marking the path of VPython objects.

After setting the time and step size, the program enters the
main loop, which is functionally similar to the loop in Program
2.7. Because this is a Hamiltonian system, the leapfrog
integrator is called to advance the solution in time (line 25).
The position of the planet is updated (line 26) after every step,
leaving a trail automatically. The program pauses if a key has
been pressed (line 27). It then calls scene.kb.getkey() twice,
first to read the key and then to wait for the second key press
before continuing.2 Finally, we define a constant GM=4π2 which
sets the unit of mass (to be discussed shortly) before executing
the main function. Earth should move around the Sun
repeatedly over the same path. In other words, the orbit is
closed.
 I encourage you to play around with Program 4.1, e.g.,
change the initial position or velocity slightly, turn off
(commenting out) the sunlight, disable the trail, or add spin to
the planet (see Program 3.8), etc.

.2 Properties of planetary motion

Since planetary motion falls under the broader scope of two-
body central field problems, we will briefly review some helpful
properties of motion with central forces for easy reference [40].

4.2.1 REDUCTION TO AN EFFECTIVE ONE-

BODY PROBLEM
A general two-body problem can be reduced to an equivalent
one-body problem in the center of mass (CM) coordinates.
Figure 4.2 illustrates this.

Let and be the positions of the particles with mass m

and M, respectively. We define the coordinates of the CM, ,
and the relative coordinates, , the usual way

It can be shown (see Exercise E4.1) that in the CM

coordinate system, the equations of motion may be written as
where the force F is given by Eq. (4.1). The center of mass
moves at constant velocity (can be set to zero for simplicity),
and the relative motion is a one-body problem with the reduced
mass, μ. If one mass (say M) is large compared to the other, i.e.,
M/m 1 as in the case of Sun/planet mass ratio, the reduced
mass takes on the value of the smaller mass, μ = m. In this case
the bigger mass (Sun) can be regarded as infinitely heavy and
frozen in space. Only the smaller mass (planet) moves. It is for
this reason that the Earth-Sun system in Program 4.1 is
effectively a one-body problem. Hereafter, we will use μ and m
interchangeably except where confusion might occur.

Figure 4.2: The reduction of a two-body problem to a one-body

problem in the center of mass coordinates. Left: the space-fixed
coordinate system; Right: the center of mass coordinate system. In
either case, the relative coordinate is the same.

4.2.2 ANGULAR MOMENTUM AND

EFFECTIVE POTENTIAL
Let and be the radial and angular velocities, respectively.
The angular momentum will be in the z direction
as
Since the torque is zero for a central force, is a constant of
motion. This is the basis of Kepler's second law which states
that the radial vector sweeps an equal area in equal time.

The second constant of motion is energy, of course. From

Eq. (4.5), we have

We can eliminate in E by substituting into Eq. (4.7)

as

The term L2/2mr2 is the centrifugal potential, and it keeps the

particle from falling into the origin. The effective potential Veff
is the sum of the real potential plus the centrifugal potential.
Equation (4.8) describes the motion as if it is one-dimensional
along the radial direction only in the potential Veff. For bound
motion in an attractive potential, the radius r usually oscillates
between rmin ≤ r ≤ rmax. The values rmin and rmax are called
turning points because at those points, the radial velocity .

4.2.3 KEPLER ORBITS

Of course, the actual motion is two-dimensional, say in radius r
and angle θ. The orbit is most conveniently expressed as r(θ).
For the inverse-square law (4.1), r(θ) is

The constant θ0 determines the orientation of the orbit and can

be chosen for convenience. The other constant e is called the
eccentricity. Geometrically, Eq. (4.9) is the equation of a conic
section (slice of a cone) with the focus at the origin.

The shape of the orbit is determined by the value of e as

follows:

The unbound, hyperbolic motion corresponds to scattering of

particles and will be discussed later (Chapter 12). For bound,
elliptic motion, it is useful to introduce a, the semimajor axis,
and rewrite e and L2/mk as
 To be more definite, let us choose θ0 = π in Eq. (4.9), and
with the equalities (4.12), Eq. (4.9) becomes

With this choice of θ0,3 it follows that r is bounded by

Figure 4.3: An elliptic orbit (e = 0.4) of a particle moving about the

origin O (focus) with semimajor and semiminor axes a and b (b =
). The orbit is oriented such that r = rmax at θ = 0
(aphelion) and r = rmin at θ = π (perihelion). Note rmax + rmin = 2a by
Eq. (4.14).

Figure 4.3 illustrates the parameters of an elliptic orbit. The

particle moves around the center of force at the origin, which is
also one focus of the ellipse. For planetary motion, the force
center is at (very nearly) the Sun. The radial velocity ( ) is zero
at the maximum distance (aphelion) and also at the minimum
distance (perihelion) from the force center. The velocity at
these two positions is perpendicular to the x-axis and may be
obtained from energy conservation (4.7) and (4.12) as

We list in Table 4.1 some properties of Kepler's orbits of the

planets.4

Table 4.1: Properties of Kepler's orbits of the planets.

4.2.4 UNITS FOR PLANETARY MOTION

As in Table 4.1, we will use the astronomical units (AU) for
distance, 1 AU=1.496 ×1011 m. This is convenient for planetary
motion in our solar system. It is equal to the semimajor axis of
Earth's orbit, and represents the average distance between the
Earth and the Sun. The natural unit of time is chosen to be one
year (365.25 days), or 3.156 ×107 s. This is the characteristic
time scale for planet motion. The unit of speed is 1 AU/year =
4740 m/s.

We now have two of the three basic units. The third one
should be about mass. Often we need the ratio k/m = GM with
M being the mass of the Sun, as in Eq. (4.3). If we assume
circular motion such as the Earth around the Sun, then

With a mean radius r = 1 AU, the period is 1 year, and the speed
is v = 2π AU/year (29.77 km/s). Using the reduced mass given
by (4.2), we get

where we have used the condition that m/M 1. In effect, Eq.

(4.17) defines the mass of the Sun as the unit of mass. We
will use GM = 4π2 for planetary motion in our units system, as
in Program 4.1. In absolute units (rarely needed),
kg, and GM = 1.327 × 1020 m3/s2.

Note that Eq. (4.17) is a special case of Kepler's third law

which states that the square of period T is proportional to the

cube of semimajor axis a. For the purpose of setting the value
of GM in our unit system, Eq. (4.18) is equivalent to Eq. (4.17).
 Finally, in the unit system thus defined, everything becomes
dimensionless. Distance and time are to be compared with 1,
and the time step must satisfy h 1 in numerical integration,
as was the case in Program 4.1.

4.2.5 OPEN AND CLOSED ORBITS

We have seen from running Program 4.1 and Eq. (4.13) that
Earth moves in a fixed orbit, retracing itself over the same path
periodically (Figure 4.1). The orbit is said to be closed.

Closed orbits may seem ordinary at first, but they are

extraordinary. They are the exception rather than the rule.
Gravity is one of only two forces supporting them. To wit,
slightly modify Program 4.1 by changing the force to 1/r2+δ
with δ ~ 0.1, set the initial values for Mercury (so the effect is
clearer), run it, and we should see an open orbit as shown in
Figure 4.4. The small δ = 0.1 is such that the force deviates
from the inverse-square law (4.1) only slightly. But significant
differences are seen in the orbit after about two revolutions.
 2+δ
Figure 4.4: Open orbit of Mercury with a presumed force ∝ 1/r ,δ
= 0.1. The initial condition is [x, y] = [0.4667, 0.0], [vx, vy] = [0.0,
8.198].

We can understand qualitatively the nature of open and

closed orbits by considering the stability of circular motion.5
Circular motion is possible for any attractive potential. One just
has to adjust the energy such that Eq. (4.16) holds. But circular
motion may be stable or unstable. If, by increasing the energy
slightly from the condition (4.16), the orbit deviates only a
small amount from the circular orbit, the motion is said to be
stable (with a slightly deformed orbit). Since the full motion
consists of radial r(t) and angular θ(t) motion, we can think of
a deformed orbit as composed of radial oscillations and angular
sweeps, illustrated in Figure 4.5.

The radial distance oscillates between the turning points

rmin ≤ r ≤ rmax, so the motion is restricted to the region
between the concentric circles shown in Figure 4.5 (bottom). If
the radius completes nr integer oscillations and the angle θ
makes nθ integer sweeps (revolutions) in the same time
interval, then the orbit is closed, since nr/nθ is rational
(commensurate). As it turns out, of all the possible forces in the
universe, only 1/r (gravitational and Coulombic) and r2
(harmonic, i.e., Hooke's law) potentials can produce closed
orbits.

Other forms of potentials do not support stable circular

motion, and an integer pair of (nr, nθ) cannot be found, i.e.,
nr/nθ is irrational. The orbit is open, as it does not repeat itself
within a finite period of time. Furthermore, the orbit
orientation rotates a bit with each revolution. This rotation is
known as orbital precession, to be discussed next.
 Figure 4.5: An open orbit for bound motion (E < 0) in an effective
potential.

.3 Precession of Mercury
Astronomers have long known that the orbits of planets precess
with time [25]. Since observations are carried out on Earth
which wobbles like a gyroscope (called precession of equinoxes,
not our concern here), the data must be analyzed to separate
the kinematic effects from true dynamic effects. This is
equivalent to going to the fixed space frame where the Sun is at
rest. Reduction of measurements of the inner-most planet,
Mercury, indicates a precession rate of 574 seconds of arc per
century in the space frame (1 arcsecond, or ″, is 1/3600°). In
other words, the perihelion (or aphelion, Figure 4.3) of
Mercury slowly rotates through an angle of 0.16 degrees per
century.

4.3.1 RECONCILIATION OF 43″

From our earlier discussion, we understand that planetary
orbits should not be exactly closed because they do not move in
a pure 1/r2 force field with a single center. Besides the Sun, the
other planets pull and tug each other all the time, not to
mention their moons [72]. What planet would have the largest
effect on the precession of Mercury? It is probably not a
surprise that Venus does, because it is closest to Mercury, so its
force is largest (see Table 4.2). Farther out radially, the effect
decreases except for Jupiter which has the second largest effect
because of its mass. Beyond Saturn, the effects are negligible.

The problem is (or was), taking into account all the known
perturbations present on Mercury due to these planets (within
Newtonian theory of gravity), and after careful number
crunching, the precession is expected to be about 531″ for
Mercury per century (Table 4.2). That would leave 43″
unaccounted for, or 0.012 degrees/century. This is a very small
amount, but well beyond observational uncertainties to be
ignored.

As it turns out, the discrepancy can be explained only with

Einstein's theory of general relativity. The relativistic
correction beautifully accounts for the difference of 43″ exactly
[16, 85]. It is one of the major triumphs of Einstein's theory of
gravity.

In general relativity, the gravitational field is modified and

can be construed as an effective force given by

The details of the ingredients going into producing Eq. (4.19)

are rather involved, and beyond our scope [40]. Compared to
Eq. (4.1), though, there is an additional 1/r4 term in the
relativistically modified force. This term destroys the perfect
inverse-square law. It causes the orbit to precess.
4.3.2 THE RUNGE-LENZ VECTOR
At first glance, it seems that we can calculate the precession of
Mercury due to the relativistic effect by just tracking the
rotation of the aphelion (Figure 4.5). While this is possible, it is
harder than it sounds. Because the parameter λ in Eq. (4.19) is
very small, it is like searching for a needle in a haystack. If we
were to use the actual value of λ directly in our simulation, we
would have found that, for any moderately practical step size h,
the change of the aphelion after one revolution would be
insignificant compared to the displacement in one time step.
Careful fitting and extrapolation would be needed to find the
exact position of the aphelion. Is there a better way? In this
case, there is: the Runge-Lenz vector [39, 47].

For observational purposes, of course, it might be natural to

want to track the change of the aphelion (or perihelion) in
order to discern any precession. This would be unnecessary in a
simulation if we have another way to measure the precession.
Fortunately, we do. There is a quantity known as the Runge-
Lenz vector that serves our purpose well. The Runge-Lenz
vector is defined as [40]

where is the momentum, the angular momentum as

usual, m the mass and k the force constant given in Eq. (4.1).
 We know energy E and are constants of motion, a fact
very useful to test if our programs are performing correctly (see
Project P4.1). As it turns out, is also a constant of motion in a
pure inverse-square law of force, i.e., (see Exercise
E4.4). Using vector product rules, we find that for
circular orbits. For ellipses, points in the direction from the
origin (center of force) to the perihelion, see Figure 4.6.

When the force is not a pure inverse-square in r, as is the

case in Eq. (4.19), the Runge-Lenz vector is no longer a
constant of motion. Rather it will rotate with the perihelion.
This helps us greatly: unlike finding the aphelion, calculating
the Runge-Lenz vector involves no fitting. We can now just
follow directly in our simulation, and the angle it sweeps
through gives the amount of precession. For example, suppose
we start our simulation with pointing in the negative x-
direction. In time, it will develop a y-component. We can
evaluate (both components Ax and Ay) at any time using Eq.
(4.20). The angle of precession can be found by θ =
tan−1(Ay/Ax).
 Figure 4.6: The Runge-Lenz vector (arrows). It is constant in a closed
2
orbit in a pure 1/r force field (left). For other forces where the orbit
is open, it rotates with the perihelion (right).

4.3.3 THE LEAPFROG METHOD WITH TIME

TRANSFORMATION
If we went ahead and calculated the Runge-Lenz vector with
the leapfrog method, we would be disappointed with the
results: we would see mostly noise for any meaningfully
practical step size h. Again, the problem of the needle-in-a-
haystack bites us. Decreasing h is not a preferred solution since
it causes more round off error. But there is something more.
Because Mercury's relatively large eccentricity (Table 4.1), its
orbit is quite elliptic. Close to the perihelion, the force and
speed are much larger than the averages. This large variation is
what causes the noise. The physical reason, and the insight we
gain, is that the force has a singularity at r = 0 and the leapfrog
integrator somehow “feels” it. In fact, it is possible to induce
artificial precession in a pure 1/r2 force law if the orbit is
elliptic enough and the step size is not small enough. We need
to smooth out, or “regularize”, the singularity.

An effective way to deal with the problem is by a time

transformation. The idea is that the step size is adjusted so that
near the closest approach (perihelion), we take smaller steps.
Let us consider the vector equations of motion equivalent to
Eqs. (2.39) and (2.40)
We introduce a time transformation from the real time t to a
fictitious time s

Now rewrite Eqs. (4.21) and (4.22) in terms of s using the chain
rule ,

Suppose we choose a function Ω(r). We would not be able to

solve the above equations using the leapfrog method because
the position also appears on the RHS of Eq. (4.24), not velocity
alone (see Section 2.4). That means the method would not be
area preserving.

What we need to do is to treat Ω(r) as a generalized

“velocity” [63]. Let us introduce an auxiliary variable, W, as
We will regard W as a “velocity”. Using
and Eq. (4.23), we can add two auxiliary equations to Eqs.
(4.24) and (4.25)

The above equations are in the proper form for the leapfrog
method if we treat and t as the generalized “coordinates”, and
and W as the generalized “velocities”. Equations (4.27a) and
(4.27b) are the exact analogue to Eq. (2.44), and Eqs. (4.27c)
and (4.27d) to Eq. (2.45). We are free to choose any form of
Ω(r). A good choice for planetary motion (and other 1/rn
forces) is

It ensures that when r is small, Ω(r) is large, and dt is small by

Eq. (4.23). Furthermore, it is easy to calculate the gradient,
. With Eq. (4.28), we can write the leapfrog
algorithm with time transformation analogous to Eqs. (2.41) to
(2.43)
As usual, the subscripts 0 and 1 denote values at the beginning
and the end of a time step h, respectively. However, here the
time is the fictitious (transformed) time s, not the actual time t.
The actual time is returned in t1. The difference t1 − t0 is not
constant, giving us the desired effect of smoothing out the
singularity near the origin.

Here is the leapfrog method with time transformation, Eqs.

(4.29a) to (4.29f).

Program listing 4.2: Leapfrog with time transformation

( leapfrog_tt.py)

1 def leapfrog_tt (lfdiffeq, r0, v0, t0, w0, h):

””” vectorized leapfrog_tt with time transformation,
3 Omega=1/r, that solves general (r,v) ODEs as:
 dr[i]/dt = f[i](v), and dv[i]/dt = g[i](r).
5 User supplied lfdiffeq (id, r, v, t) returns
f [i](r) if id=0, or g[i](v) if id=1 ”””
7 # 1st step: calc r at h/2
hw = h/(2.0*w0) # half
h/2w0
9 t1 = t0 + hw
r1 = r0 + hw*lfdiffeq(0, r0, v0, t0) # id=0,
get
11 r2 = np.dot(r1, r1) # get
rˆ2=x∗x+y∗y+z∗z
r12 = np.sqrt(r2) # r1/2
13
# 2nd step: calc v1 using r at h/2
15 v1 = v0 + h∗r12∗lfdiffeq(1, r1, v0, t1) # id=1
for g(r) at h/2
rdotv = np.dot(r1, v0+v1)/2. #
17 w1 = w0 − rdotv∗h/r2 # w0 −
h/r2

19 # 3rd step: calc r by another 1/2 step using v1

hw = h/(2.0∗w1)
21 t1 = t1 + hw
r1 = r1 + hw∗lfdiffeq(0, r1, v1, t1) # get at
t+h
23 return r1, v1, t1, w1

The module leapfrog_tt() uses the same lfdiffeq format

as required by the standard leapfrog (Section 2.4). Except for
additional elements of time transformation and calculations of
dot products with the NumPy function np.dot, it works very
much the same way also. We will use the transformed leapfrog
integrator for sensitive problems such as the precession of
Mercury. As discussed in later chapters, it is also a very useful
method for N-body systems, including molecular dynamics and
atomic reactions (Section S:12.5.1). We include leapfrog_tt in
the library file ode.py with other ODE solvers.

4.3.4 SIMULATION OF DIFFERENTIAL

PRECESSION
Precession due to the relativistic correction
Finally we have all the necessary parts to calculate the
relativistic correction to the precession of Mercury. Putting it
all together, we give the complete simulation in Program 4.3
(Section 4.D).

Run the simulation with animation on, and enter a λ large

enough, say 0.01, to see exaggerated effect of precession. It
should be similar to Figure 4.6. You should also see that the
motion of the Runge-Lenz vector is not uniform: it rocks back
and forth, with the mean motion forward (counterclockwise),
in the same direction as the circulation of Mercury.
 Figure 4.7: Differential precession of Mercury, i.e., the rotation of the
Runge-Lenz vector. A zoom-in of the last two years is shown on the
right.

Now set animation=False, and enter the true value λ = 1.1 ×

10−8 when prompted. After the simulation ends, we should see
a plot showing the angle of precession differential in time over
one century in Figure 4.7. It is seen that right at the century
mark, the precession is exactly 43″, a remarkable success of
Einstein's theory of gravity. Equally remarkable is the precision
of observational astronomy which allows the detection of this
tiny amount. Numerically, the leapfrog method with time
transformation is sufficiently accurate to pick up this amount,
and at the same time, preserves the phase space properties. It
is also quite robust. For example, we could increase the step
size by a factor of 10 and still get reasonably accurate results.

The full curve (Figure 4.7, left) is not as straight as it looks.

It has small-scale oscillations which become visible in the
enlarged figure. Because these oscillations are so small (a
fraction of an arcsecond), we should ask ourselves, are they
real? Could they be artifacts of numerics due to such factors as
limited accuracy or incorrect algorithm, for example?

In numerical work, we always have to be mindful of

potential pitfalls which could produce artificial effects not
present in the actual system. In our case, we can check whether
the effect remains for a larger perturbation. As seen from the
animation with λ ~ 0.01 discussed earlier, the back-and-forth
rocking motion of the Runge-Lenz vector shows that the
oscillations are not restricted to small perturbations. Other
checks including conservation laws and step sizes can be
performed to see if the oscillations are affected. The answer is
no (see Project P4.1). In all likelihood, we can exclude
numerical error (noise) as the cause.

Once we are certain that the oscillations are real, what is the
physical reason for them? We note that there are about four
oscillations per year, or the period is about 1/4 years. This is
just the orbital period of Mercury (Table 4.1). Therefore, the
oscillations might be connected to the orbital period. With
animation turned on and with an artificially large λ ~ 0.01, we
can observe the phenomenon visually. Let us assume that
Mercury starts at the aphelion going counterclockwise, with the
Runge-Lenz vector pointing at the perihelion (Ay = 0). The y-
component, Ay, has two terms from Eq. (4.86), Ay/m2 = −vxL −
GMy/r.

Roughly,6 when vx turns negative, Ay increases, and when y

increases, Ay decreases. Near the aphelion, Mercury is farthest
away from the Sun, and the attractive relativistic term causes vx
to turn negative faster than the increase in y (more precisely
y/r). The net effect is that Ay increases. This causes the
clockwise (retrograde) rotation of . Near the perihelion (past
the x = 0 plane), the opposite occurs, and rotates
counterclockwise. Because the perturbation is greater near the
perihelion, the counterclockwise rotation is larger than the
clockwise rotation, resulting in a net precession in the same
direction as Mercury's circulation (ccw, prograde). It then
follows that if the sign of the perturbation is reversed
(repulsive), the precession will be in the opposite direction.
This can be verified through an exercise.

Central field approximation and precession due to other

planets
Mercury interacts with other planets through pair-wise forces
that are not central relative to the Sun. A general description
would need to take into account all the planets simultaneously
(a many-body problem). But if the effects are small, and if we
are interested in only averaged values like precession over a
period of time, we could approximate each pair-wise force as a
central force and treat it individually. The idea is that Mercury
moves around the origin (Sun) rapidly and the average force
points in the radial direction. In such cases, we speak of central
field (or mean field) approximations which are very useful and
important in many areas of physics.

If we can construct the force between Mercury and a planet

in a central field approximation, we can calculate the
precession of Mercury due to that force with the same
methodology as the relativistic effect discussed above. One way
to obtain the force is as follows. Imagine the mass of a planet,
Mp, is distributed uniformly over a coplanar ring of radius a
(semimajor axis) [72]. By symmetry, the force between
Mercury and the ring is central, i.e., . The potential is
given by an elliptic integral [49] (see Exercise E4.6)
Because Mercury is the inner-most planet and ρ < 1, this
elliptic integral can be expanded in a power series. The first few
terms are

The force can be found via the gradient −∇V as

This force is repulsive (radially outward), and the leading term

behaves like Hooke's law with a negative spring constant. Two
factors determine the magnitude of the force, the mass of the
planet Mp and its average radius a. Accordingly, with
everything being equal, we expect either a larger mass or a
smaller semimajor axis will cause a greater precession. The
actual amount depends on the interplay between the two
parameters.

Using the force in the central field approximation (4.32) in

Program 4.3 with the parameters from Table 4.1, we obtain the
results of Table 4.2.

Table 4.2: Precession of Mercury due to the planets

(arcseconds/century).
 The results vary over three orders of magnitude. Venus, the
closest one, has the largest effect, followed by Jupiter, the most
massive one. In between, the order reflects the balance between
the radius and mass. Note that the results for Earth are
obtained with the combined mass of Earth and the moon.
Neptune, the outer-most planet, gives the smallest
contribution, 0.0437″, because of its large radius and moderate
mass. This is three orders of magnitude smaller than the
relativistic contribution (43″). If 43″ was a needle in a hay
stack, Neptune's contribution might be a drop in a lake. Yet, the
leapfrog method with time transformation is robust enough to
crunch it out accurately and efficiently. Also, although the
central field approximated force (4.32) is positive, the
precession is still counterclockwise, the same as the relativistic
correction even though the force there (4.19) is negative. We
will investigate these questions and related aspects in Project
P4.3.

Scaling law in the central field approximation

As the numbers listed in Table 4.2 jump up and down with
different planets, they are not particularly insightful for
discovering trends or revealing more general properties. As Eq.
(3.33) from Chapter 3 showed, scaling laws are very useful in
this regard. Whenever possible, we should try to find scaled
variables and plot the results in these variables to show the
general trends.

What is a good scaled variable to use in this case? The

amount of precession depends on the combination of the mass
and radius. The prefactor in the potential (4.31) suggests Mp/a,
but the force (4.32) points to Mp/a3. Since the force is directly
involved in the simulation, the latter is more appropriate.
Therefore, for small perturbations, we expect precession to
scale linearly with Mp/a3, i.e., a linear scaling law.7

When we go ahead and calculate the scaled variable Mp/a3

for the planets using Table 4.1, we find that it also varies over
three orders of magnitude similar to the precession results in
Table 4.2. If we were to display precession versus the scaled
variable as linear plots, the data points would be clustered at
two locations, the lower-left and upper-right corners. Thus, to
more evenly spread out the data points in the whole range and
to help visually identify any trends, we should plot the data
using log-log scales as we did with the semilog plot in Figure
2.7, for example. The results are shown in Figure 4.8.
 Figure 4.8: The precession of Mercury due to other planets as a
3
function of the scaled variable Mp/a . The dashed trend line indicates
a linear scaling law expected for small perturbations in the central
field approximation.

Most data points in Figure 4.8 fall on a straight line (trend

line), clearly showing a linear scaling law. Small deviations are
seen for Venus, Earth, and Mars. This is due to their relative
proximity to Mercury, causing the second order term in Eq.
(4.32) to have a non-negligible effect on the deviation.
Nonetheless, Venus and Jupiter, the closest and the most
massive planets, are vastly different and yet nearly identical in
the scaled variable, 6.47 vs. 6.78, respectively (Figure 4.8).
Therefore, they cause similar amount of precession according
to the scaling law. The predicative power and the physical
insight of scaling laws make them very useful in practice and in
our understanding.

Since each planet produces precession in the same

counterclockwise direction, the total precession due to all
 planets is additive, and is equal to 529.4″ (Table 4.2). The
result is in very good agreement with the accepted
observational data of 531″ [72, 85]. The relative error is about
0.3%, with most of it coming from Venus. What are the sources
of error? We have assumed the planets are coplanar with
Mercury, which is only approximate. The inclination angles of
the planets differ from each other by a few degrees. Another
source is the assumption of circular orbits (rings). From the
eccentricities in Table 4.1, except for Pluto which has negligible
effects and Mercury itself, the perturbing planets are nearly,
but not perfectly, circular. Lastly, we have included only a few
terms in the series expansion (4.31). This will affect only the
close planets though, such as Venus and Earth.

.4 Star wobbles and exoplanets

We have seen from Section 4.2 that a two-body problem can be
reduced to an effective one-body problem in the center of mass
frame. The actual motion, of course, still consists of two bodies
pulling on each other (with forces of equal magnitude and
opposite direction) and revolving around the center of mass. In
the case of a star-planet system, the star barely wobbles due to
the large mass ratio.

Nonetheless, the star's wobbling is important to a current

topic of considerable interest: the discovery and detection of
extrasolar planets, or exoplanets, including Earth-like planets
and planets in the habitable zone supporting an atmosphere
and liquid water.8 Unlike planets at the doorstep in our solar
system, direct observation of exoplanets is very difficult
because they are at astronomical distances, small and faint, and
in sharp contrast to the brightness of their host stars. Indirect
methods of detection are necessary, including measuring the
wobbles of stars (radial velocity), the light blocked off when
exoplanets pass in front of the stars (transiting planets), and
light bending and brightening due to gravitational fields
(microlensing), among others.

4.4.1 RADIAL VELOCITY METHOD

We discuss the method of detecting the wobbles, known as the
radial velocity (RV) method. The RV method relies on the fact
that light from stars moving about the center of mass will be
Doppler-shifted. The Doppler shifts depend on the velocity of
the star along the line of sight, i.e., the radial velocity from the
observer's perspective. Light from the star will be blue-shifted
moving toward us, and red-shifted moving away. By measuring
the Doppler profile, one can infer the velocity profile of orbital
motion. From there, we can determine the orbital parameters
of the exoplanets, including the period, eccentricity, and the
lower limit of the mass. The RV method is responsible for a
majority of exoplanets discovered early on [14].9

Referring to the position vectors in Figure 4.2 and Eq. (4.4)

(with the results from Exercise E4.1), the wobbling velocity of
the star, , is related to the orbital velocity as
 Figure 4.9: The wobbling velocity of the Sun due to Jupiter (left) and
of HD 139357 due to its exoplanet (right). The solid curve is a fit
described in text.

For a given star-planet system, we can obtain from

Program 4.1 and therefore . The result for the Sun-Jupiter
system is shown in Figure 4.9. One component of the Sun's
velocity is plotted, in this case, which is equivalent to the
edge-on view of the orbital motion. The velocity is periodic (~
4330 days, Jupiter's period), and nearly sinusoidal because the
orbit is almost circular (e = 0.0484, see Table 4.1). It would be
sinusoidal for circular orbits. The magnitude of the velocity is
roughly 10 m/s, typical of the pull by planets of Jupiter's size
on a star of about one solar mass. For smaller planets like
Earth, the magnitude is about 10 cm/s.

Also shown in Figure 4.9 is an observational RV dataset for

a star in the Henry Draper catalog, HD 139357. It shows an
almost sinusoidal curve, clearly indicating the presence of an
exoplanet moving in a nearly circular orbit around the star,
with a period ~ 1100 days. This exoplanet was discovered in
2009. From the RV dataset, the lower limit of the exoplanet's
mass is determined to be about 9.8 Jupiter mass. The fact that
only a mass limit can be established, and not the mass itself,
has to do with observing geometry and information on the
inclination angle, or lack thereof.

Figure 4.10: Observing geometry for radial velocity measurements.

The geometry is illustrated in Figure 4.10. The space-fixed

coordinates (the sky) are denoted by XYZ, where the XY plane
defines the space plane. We choose the X-axis along the line
where the Kepler orbit of the exoplanet intersects the space
plane (ascending node in astronomy nomenclature). The
inclination angle, i, is the angle between the orbital plane and
the space plane. It is equal to the angle between the Z-axis and
the normal of the orbit (direction of angular momentum ). We
use ω to denote the angle between the aphelion and the X-axis.
Except for the azimuthal angle around Z (not shown), which is
unimportant due to rotational symmetry, i and ω determine the
orientation of the Kepler orbit. As usual, θ represents the angle
between the radial vector of the exoplanet and the aphelion, the
same as in Figure 4.3.

The observer's line of sight to the star runs parallel to the Z-

axis. Therefore, only the Z component of the star's velocity, v*z,
is relevant and measurable by Doppler shift methods. From the
observer's perspective, v*z appears as a radial velocity (away or
toward), thus the name of the RV method.10 We have to
determine the relationship between the wobbling velocity v*z
and the orbital velocity in terms of orbital parameters and the
angles i and ω.

The radial velocity of the star is derived in Section 4.C. For a

star with a single planet, the result from Eq. (4.85) is

where C is a constant representing the velocity of the center of

mass, T the period, and a the semimajor axis. The first term
indicates that v*z oscillates with a period T and an amplitude
|V|.

From fitting a given RV dataset like the one shown in Figure

4.9 according to Eq. (4.34), parameters such as T, e, and V can
be obtained. If the mass of the star M is independently known,
the mass of the exoplanet m can be determined from
We have assumed m/M 1 and used Kepler's third law (4.18)
to eliminate a. Equation (4.35) shows that we can determine
the product m sin i only. Because the inclination angle i is
generally unknown, the RV method yields only a lower limit for
the exoplanet mass (sin i ≤ 1).

If the period T is given in years, M in solar masses, and V in

m/s, the numerical value for the exoplanet mass in Earth
masses is

This formula is useful for a rough estimate of an exoplanet's

mass.

Take for instance the exoplanet of the star HD 139357. The

star is known to be about 400 light-years away and a mass M ~
1.3 . We can estimate the other parameters by inspecting
Figure 4.9 to obtain: T ~ 1100 days = 3 years, and V ~ 160 m/s.
The RV curve is almost sinusoidal, so the orbit is nearly
circular, and e ~ 0. Substituting these numbers into Eq. (4.36),
we obtain m sin i ~ 3000 Earth mass, or ~ 9.5 Jupiter mass.
More careful data fitting gives m sin i = 9.8 Jupiter mass,
which means that the exoplanet is at least 9.8 times more
massive than Jupiter.
4.4.2 MODELING RV DATASETS
To accurately model observational RV datasets according to Eq.
(4.34), we need to know θ as a function of time t, θ(t).
However, this is not as straightforward as one might expect.
Except for circular orbit where θ is linear in t, no simple closed-
form solutions exist. We could obtain θ(t) numerically from
simulation codes such as Program 4.3, but that would be the
hard way. Fortunately, it is a two-way street here, a hard way
and an easy way.

We first write using conservation of energy from Eq. (4.8)

and obtain t as

This equation can be solved by introducing a new variable ψ

through

where ψ is called the eccentric anomaly.11 Substituting Eq.

(4.38) into (4.37), and after some algebraic details left to
Exercise E4.7, we obtain

This is the celebrated Kepler's equation relating t to ψ. We can

solve for θ in terms of ψ by combining Eqs. (4.38) and (4.13) to
get

We see from Eq. (4.40) that ψ has the same range as θ, i.e.,
when θ changes from 0 to 2π, so does ψ. Furthermore, the two
variables are equal at 0, π, and 2π, no matter the eccentricity e.

The pair of equations (4.39) and (4.40) give the relationship

between t and θ. For a given t, Eq. (4.39) can be solved for ψ,
which can be substituted into Eq. (4.40) to obtain θ. The
relationship for several eccentricities is depicted in Figure 4.11.
The dependence is linear for e = 0 (circular orbit) as expected.
For nonzero e, it becomes increasingly nonlinear. The figure
was generated without actually solving Eq. (4.39). The trick is
this: first generate a mesh for ψ in [0, 2π]; second, compute t
from Eq. (4.39) for each mesh point and store it in an array;
third, compute θ from Eq. (4.40) and store it in another array;
finally plot the two arrays. This trick is easy and fast, but the
drawback, of course, is that the t values calculated are not pre-
determined and non-uniform. If we wanted to generate θ for a
given t, we would have to solve Eq. (4.39) using a root solver.
It's not necessary for our purpose here.
 Figure 4.11: Left: The relationship between θ and t for four
eccentricities. Right: an actual RV dataset of Fischer et al. [30] and a
fit.

The effect of eccentricity on radial velocities is also shown in

Figure 4.11. The dataset is for HD 3651 with an exoplanet
reported in 2003 [30]. The curve is not sinusoidal at all. The
sharp dip in the radial velocity can be reproduced only with a
significant value of eccentricity. Physically, the planet moves
much faster near the perihelion than anywhere else in a highly
eccentric orbit. This causes a dip (or spike) in the velocity of the
recoiling star.

A general best fit of Eq. (4.34) to a given RV dataset is not

trivial because of the number of parameters involved. We use
least square fitting, leastsq, from SciPy as a blackbox. It
requires only an error function that returns the difference
between our model and the actual data points. The details are
given in Program 4.4, yielding the results in Figure 4.11.

The blackbox method is easy to use and gives satisfactory

results, but leaves the modeling process a bit opaque. To see
 various factors at play, we outline an iterative fitting process by
simple visual inspection that also works well for our purpose.
Starting with an initial guess of parameters T, V, e, ω, C:

1. generate a mesh grid for ψ

2. adjust the appropriate parameters T, V, e, ω, C for
better visual fit
3. compute t from Eq. (4.39) and θ from Eq. (4.40)
4. calculate v*z from Eq. (4.34) and plot the results
5. inspect the match of the curve and the data, repeat step
2 as necessary

With code segments from Program 4.4, we can begin to

model any datasets and visualize the process. We leave
modeling of the dataset shown in Figure 4.11 to Project P4.6.
Many other RV datasets can be found at the exoplanet archive
[66].

.5 Planar three-body problems

So far we have studied motion of two-body systems reducible to
effective one-body motion. If we add just one more body to a
two-body system, we have a three-body problem, the simplest
of general N-body problems. Even if the pair-wise forces obey
the pure inverse-square law (4.1), general analytic solutions to
the three-body problem are still not possible at present, making
simulations indispensable.
 Since the early development of Newtonian mechanics,
efforts have been made to reduce the intractability of the three-
body problem. There are nine degrees of freedom in total, three
for each body. Going to the center of mass frame reduces that
number by three. Further reduction is possible by invoking
conservation laws (energy, angular momentum, etc.), but the
mathematical difficulties involved are still insurmountable.
Special attention, therefore, has been devoted to finding
solutions of the planar three-body problem: the motion of all
bodies is confined to a two-dimensional plane. This is still an
area of ongoing research, especially in the search of periodic
orbits. We will discuss several numerical examples to get a
glimpse of the planar three-body problem and its intricacies.

4.5.1 EQUATIONS OF MOTION IN THE

THREE-BODY PROBLEM
Qualitatively, the motion of three bodies is no more difficult to
simulate than one-body motion. Let the three bodies be
numbered as i = 1, 2, and 3. Their masses, coordinates, and
velocities are mi, , and , respectively. The equations of
motion are
It is understood that i ≠ j ≠ k, and {ijk} is a permutation of
{123}. The relative coordinates between two bodies are
, and . Note that , a relation
useful in programming. Because the gravitational constant G
always appears in the product Gmi, we can set G = 1 in our
program. It is equivalent to setting the scale of mass.

Equations (4.41) and (4.42) are 12 first order differential

equations of a Hamiltonian system in 2D. Given the initial
condition, we can solve them using the leapfrog method. Of
course, the more elaborate leapfrog method with time
transformation discussed earlier may be extended to the three-
body problem as well.12 However, we are not interested in very
small or sensitive processes like precession of Mercury, so its
use is unnecessary here. In addition, most of the periodic orbits
are physically unstable, and there is little a numerical
algorithm can do to rectify that. What remains is to determine
the initial condition for a given numerical solution.

4.5.2 EULER'S COLLINEAR MOTION

One of the simplest and most interesting systems is Euler's
collinear motion of the three-body problem. It is illustrated in
Figure 4.12.

At any given time, the three bodies are on a straight line,

with one body, say m2, lying in-between the other two bodies,
m1 and m3. Each moves in an ellipse with the same period.
When they reach the aphelion simultaneously on the axis of the
ellipses, the distances among them are the largest, and when
they pass the perihelion, the distances are the closest. At other
times, the distances oscillate between the extrema. The motion
is periodic, but unstable. That is to say, if there is a slight
deviation from the perfect configuration, the system flies apart
from collinear motion.

Figure 4.12: Euler's collinear motion of the three-body problem.

To determine the initial condition, let us assume the bodies

start from the x-axis, thus yi = 0, and only the horizontal
positions xi are needed. To execute Euler's collinear motion,
the distances between the bodies must obey certain
proportions. Referring to Figure 4.12 (top), let a be the distance
between m2 and m3, and λa between m1 and m2. It follows that

The equation of motion for a given body, say m1, is

where we have set G = 1 for convenience. Because of the
oscillating nature of the distances, we seek a solution of the
form d2x1/dt2 = −ω2x1 (simple harmonic oscillator).
Substituting this into the above equation and dividing −m1ω2
on both sides, we obtain

Similarly, we have for the other two bodies

We can combine Eqs. (4.43) to (4.46) to obtain the parameter λ

first,

This is known as Euler's quintic equation. We leave as an

exercise the verification of Eq. (4.47) and the results below,
Eqs. (4.48) to (4.50). Given a set of masses, there is one
positive root. The variable a in terms of λ is
With both λ and a found, we can find the initial positions as

We recognize that the last term means that the center of mass
is the origin. The initial velocities (all along y-axis) are

The procedure to simulate Euler's collinear motion is as

follows. First, solve Eq. (4.47) for λ using a root finder such as
bisection or Newton's methods in Chapter 3. Then compute the
initial condition from Eqs. (4.49) and (4.50). Finally, solve the
equations of motion (4.41) and (4.42) with the leapfrog
method.

The code implementing this strategy is given in Program 4.5

listed in Section 4.D. The program animates the collinear
motion of the three-body problem and snapshots are shown in
Figure 4.13.

Watching the motion, we should see that at first the bodies

move perfectly (to the naked eye) in their own ellipses,
retracing over them periodically (Figure 4.13, left). After a
while, typically three periods or so depending on the energy,
two bodies will be so attracted to each other that they quickly
move off the periodic orbits, forming a binary system. The
binary partners usually perform choreographically beautiful
(albeit mechanical) “dances” (Figure 4.13, right) as they move
along. The third body, attracted to the binary system acting
roughly as a single body, often turns around. If it does and
moves in close, there is a high probability that the dancing
partners switch, forming a new binary system. This may
continue for a while, but eventually, the final outcome is that a
tight binary is formed, and the third body is ejected, moving
indefinitely away from the binary system. Gravitationally
bound classical three-body systems are mostly unstable, and
often so sensitive to the initial conditions that a small
difference there can lead to totally different outcomes. This
behavior is known as classical chaos, a topic we will discuss
next in Chapter 5. Our knowledge at present seems to indicate
that common many-body systems held by gravity are chaotic,
including our solar system [58].

Figure 4.13: Euler's collinear motion of the three-body problem at

different times. The camera in the right figure is tilted to keep the
third body in view.

4.5.3 THREE-BODY CHOREOGRAPHY

There are more elaborate periodic orbits that have been found.
For example, the following initial condition [84] produces a
heart-shaped choreographic orbit shown in Figure 4.14.

The three bodies rotate counterclockwise, so there is a net

angular momentum. The starting position of each body is
approximately at the position of the body immediately behind
it (clockwise) in the left panel of Figure 4.14. The most delicate
part occurs near the center of mass (CM, white dot) where the
speed is the lowest. Referring to the animation (or Figure 4.14,
right), as the red body (closest to the CM) moves in on the
upper part of the inner orbit, it slows down, coming to a
temporary stop right at the tip (the protrusion) of the inner
orbit. At the same time, the other two bodies (light/green and
dark/blue) are almost diametrically positioned on the outer
orbit, so the net force on the red body is very small. But, there
is a slight bend to the diametrical which provides just enough
net force that allows the red body to step back from the tip and
continue on to the lower part of the inner orbit (where the body
immediately ahead (blue) was previously). However, the
motion is unstable. After about two periods, the system breaks
up, suffering the same fate of most three-body systems: a tight
binary plus a runaway body (not shown in Figure 4.14). But, if
we reversed the velocities at any time, the system would run
backward to its original configuration precisely like rewinding
a videotape (see Exercise E4.9). This is because the leapfrog
method preserves time reversibility of the system, which may
be unstable but not indeterministic.

Figure 4.14: Choreographic orbit of the three-body problem with the

initial condition (4.51). The arrows are proportional to the velocities.

Another interesting orbit is a figure-8 orbit. It is shown in

Figure 4.15 produced from the following initial condition [15],
 Figure 4.15: Choreographic orbit of the three-body problem with the
initial condition (4.52). The arrows are proportional to the velocities.

This orbit is symmetric both horizontally and vertically.

Unlike the previous configuration, the bodies do not circulate
uniformly in one direction. The motion is clockwise on the left
half of the orbit (Figure 4.15), and counterclockwise on the
right half. The combination is such that there is no net angular
momentum. When one body is at the origin, the other two are
symmetrically situated on the orbit, all on a straight (slanted)
line. In this sense it is like Euler's collinear configuration
(Figure 4.13), although the three bodies never line up
horizontally. Whenever one body is at the edge of the figure-8
(Figure 4.15, middle), the other two are on the opposite half
lined up vertically, forming an isosceles triangle. At this
 moment, the velocity of the first body is vertical, and the
velocities of the other two bodies are such that their x
components cancel each other and the sum of their y
components cancels the y component of the first body, so that
the center of mass velocity remains zero.

Numerical experiments show a rare but remarkable

property of this orbit: it is stable. If the orbit is perturbed
slightly, the motion remains close to the figure-8 shape. For
example, if the position or the velocity of a body (or even its
mass) is changed by a small amount, the orbit may be slightly
deformed, or it may even precess, but the three-body system
does not disintegrate (Exercise E4.9).

.6 The restricted three-body problem

The planar three-body problem does not have any restrictions
other than that the motion is confined to a plane. We impose
now two additional constraints: the masses of two primary
bodies are much larger than the mass of the third body, and the
two primaries move in a circular orbit about their center of
mass. As a result, the simplified system is called the restricted
three-body problem. It is illustrated in Figure 4.16.
 Figure 4.16: The restricted three-body coordinate system.

Let us call the third body of mass m the test body. Its mass
is considered so small compared to the two primaries, i.e.,
m/M1,2 ~ 0, that it has no effect on the motion of the primaries,
which rotate around the center of mass at constant angular
velocity ω and constant separation. We are only concerned with
the motion of the test body. Though the restricted three-body
problem is an idealized situation and no actual systems behave
exactly like it, it is still a very good approximation for many
systems, including the motion of a satellite (test body) in the
Earth-Moon system, or of the asteroids in the Sun-Jupiter
system. The simplification allows us to gain much insight from
studying such systems.

The motion of the test body may be best understood in the

rotating coordinate system of the primaries. For example, if a
and b are the respective distances of M1 and M2 from the center
of mass (the origin, Figure 4.16), then the primaries appear as
stationary points at (−a, 0) and (b, 0), respectively, in the
rotating coordinates (x-y), though they are rotating in space-
fixed coordinates (X-Y). The test body will move around the
stationary force centers in the rotating coordinates.

4.6.1 EQUATIONS OF MOTION IN THE

ROTATING SYSTEM
The force on the test body is

where (x, y) are the coordinates of the test body in the rotating
system. The calculation of velocity or acceleration must take
the rotating coordinates into account. Details are given in
Appendix 4.A. For example, according to Eq. (4.72) the
acceleration is,

Note that we have dropped the prime (′) according to our

coordinate system in Figure 4.16 for clarity. Of course, all
variables , , and refer to the rotating coordinates.

Using the fact and ,

we can simplify the last term in Eq. (4.54) to
We have used because is perpendicular to the plane
of motion (x-y plane). Substituting these results and Eq. (4.53)
into (4.54), we obtain

The system can be readily integrated with our ODE solvers.

However, before doing that, we would like to examine the rich
structure contained within the effective potential which will
help us better understand the motion.

4.6.2 THE EFFECTIVE POTENTIAL

There are four terms in Eq. (4.55). We recognize that the first
two terms are actual forces due to the primaries M1 and M2, the
third term due to the centrifugal force (or effect), a pseudo-
force, and the last term due to the Coriolis effect. Except for the
velocity-dependent Coriolis term, the other three terms may be
obtained from the gradient of a scalar function of coordinates
only, say V (x, y), such that the acceleration can be written as

We call V the effective potential given by

The effective potential consists of the actual gravitational
potentials, and a centrifugal potential which resembles the
harmonic oscillator potential but with a negative spring
constant (see Eq. (4.32)). The force is therefore repulsive and
pushes radially outward. This is because the motion is being
described in the rotating system. We could imagine that, to
keep the test body stationary in the rotating system, a real force
toward the center, i.e., a centripetal force, would be needed to
balance an extra (unseen) force to keep it in place. That extra
force is called the centrifugal force.

Units for restricted three-body problems

The effective potential (4.57) (and equations of motion) are
valid for any unit system. To further analyze its properties
quantitatively, we should adopt a unit system most convenient
for the restricted three-body (RTB) problem at hand.
Analogous to planetary motion, we choose the RTB units as
shown in Table 4.3.

Table 4.3: RTB unit system for the restricted three-body problem.

Consequently, it follows by definition that ω = 2π/T, and

from Eqs. (4.16) and (4.17) that
This means that in actual calculations employing the RTB unit
system (M = R = T = 1), we can set angular velocity ω = 2π and
GM = 4π2, just as in the planetary unit system.

For the distances, because the center of mass is chosen as

the origin (Figure 4.16), we have

Solving for a and b and introducing the mass parameter α, we

have

It is customarily assumed that M2 ≤ M1, so 0 < α ≤ 0.5. In

terms of α, the primaries’ masses in our unit system can be
expressed as

Consider the Earth-Moon system as an example. The

primaries’ masses are M1 = 5.974 × 1024 kg and M2 = 7.348 ×
1022 kg, respectively. The distance between them is 3.844 × 108
m, and the rotational period is about one month (27.3 days).
The absolute units and the RTB units are listed in Table 4.4.

Table 4.4: RTB units of the Earth-Moon system.

 We note that the RTB units are elastic. In other words,
when the restricted three-body system changes, e.g., to the
Sun-Jupiter system, the units of mass, length, etc., have
different numbers on the absolute scale, even though they may
all be 1 in the RTB units.

The Lagrange points

We can simplify the effective potential (4.57) by rewriting it in
the newly introduced RTB unit system as

One single mass parameter α fully determines the effective

potential. The potential surface for α = 0.121 is shown in Figure
4.17. This α is about ten times larger than that of the actual
Earth-Moon system in order to show some fine features.
 Figure 4.17 is produced with Program 4.6. The program is
explained in detail there. However, the block (lines 21 to 34)
can be used as a template for making surface and contour plots.

The surface plot (Figure 4.17, top) shows several expected

features. The potential has two holes (singularities) at the
primaries, M1 at (−a, 0) and M2 at (b, 0). It also goes to −∞ at
large r due the centrifugal potential −r2/2. In between, there is
a broad C-shaped ridge surrounding the bigger body M1. The
formation of the ridge is due to the balance of the gravitational
and centrifugal potentials. As the ridge nears the other body
M2, a hole is “burned” into it, and a saddle point is created
between M1 and M2.
Figure 4.17: The effective potential of a restricted three-body system
with α = 0.121. The arrows on the contours indicate the force from
the potential.
 The contour plot (Figure 4.17, bottom) shows that there are
finer structures embedded along the ridge. There are five
points where the potential is locally maximum. They are known
as the Lagrange points, labeled L1 to L5.13 Three of them, L1, L2
and L3, are located on the x-axis, and the other two, L4 and L5,
are off the axis but symmetrically located about it, one above
and one below. On closer look, the locations of the primaries
and L4 (or L5) form an equilateral triangle.

The gradient of the potential is graphically illustrated by the

arrows. Since , the arrows are proportional to the
force (hence acceleration) on the test body. We see large
gravitational forces around each of the primaries, as well as
large centrifugal forces at large distances. Near the Lagrange
points, the forces are small. Right at the Lagrange points, the
forces are exactly zero. If we were to place a test body at any
point L1 to L5 exactly and perfectly at rest (in the rotating
system), it would remain there indefinitely. However, the
Lagrange points are unstable, because forces in the
neighborhood point away from them. Any slight deviation (or
perturbation) would most likely lead the test body to move
away. The existence of Lagrange points is a result of balance
between the attractive gravitational and the repulsive
centrifugal forces. Without rotation, all Lagrange points but
one would vanish. Only L2, the saddle point, would remain. The
effect of the potential surface is wholly dynamic.

For small α, all five Lagrange points lie close to the unit
circle r = 1. While the locations of L4 and L5 can be found
analytically, the locations of L1, L2 and L3 need to be solved
numerically. But for α 1, the following series expansion can
be used (Exercise E4.11)

Because the existence of the Lagrange points are due to the

balance between gravity and rotation, it might appear curious
that the locations (4.62) depend on α only, and not on other
parameters such as the angular velocity ω. The reasons are
two-fold. First, the mass parameter α as defined by Eq. (4.59)
depends on the masses M1 and M2. Because ω also depends on
the masses via Eq. (4.16), α is related to ω indirectly. Second, as
discussed earlier, the RTB unit system (Table 4.3) is elastic.
This hides some parameters which would be explicit if we used
absolute units.

4.6.3 MOTION AROUND THE LAGRANGE

POINTS
As discussed above, with the effective potential alone, none of
the Lagrange points are stable. However, we must also consider
the Coriolis effect (4.56), , when the test body is not
stationary. The combination of the force from the effective
potential and the Coriolis effect makes it possible to have stable
periodic orbits in the neighborhood of the Lagrange points.14
 Let us examine that possibility by focusing on the
equipotential island surrounding L4 in Figure 4.17 (bottom). All
around the island, the force from the effective potential (the
arrows) points away from L4. Suppose the test body moves
clockwise around the edges. On the outer edge, it will be
moving to the right. Recall that points out of the page, and
the Coriolis effect, , points inward. It is thus possible
for the Coriolis effect to cancel the outward force, such that the
net acceleration is toward L4. On the inner edge, the velocity is
reversed. So does the Coriolis effect. The net result is the same:
it counter balances the inward force from the effective
potential. Because no energy is added by the Coriolis effect,
, it should be possible for the test body to move
about L4 periodically with the right initial conditions. As it
turns out, stable periodic motion does happen if α < 0.03852
[79].

The Trojan asteroids

Both the Earth-Moon and the Sun-Jupiter systems support
stable periodic orbits. We start with an actual example for the
motion of asteroids in the Sun-Jupiter system. Most asteroids
in our solar system are in the so-called asteroid belt. But there
are two groups of asteroids that revolve around the Sun at
about the same radius and with the same period as Jupiter
(Figure 4.18, left). The two groups are symmetrically located
with respect to the Sun-Jupiter radial vector: one group 60°
ahead and another 60° behind. They are known as the Trojan
asteroids (or simply Trojans) [95]. Because their locations form
equilateral triangles with the Sun and Jupiter just like the
Lagrange points L4 and L5 (Figure 4.18, right), we suspect there
is a connection.

Figure 4.18: The Trojan asteroids and their connection to the

Lagrange points L4 and L5.

Though Jupiter is the most massive planet, the mass

parameter for the Sun-Jupiter system is α = 0.0009542 (see
Table 4.1), still well within the limit for stable orbits. If the
Trojans move in clockwise orbits around L4 and L5 as indicated
in Figure 4.18 (right), stable orbits can be formed.

Program 4.7 (Section 4.D) simulates stable orbits around

Lagrange points. Three orbits are shown in Figure 4.19. The
first orbit (top left), also the smallest, corresponds to the initial
condition as given in the program. It is a stable periodic orbit
with a period of about 13 time units (Jupiter orbital periods), or
about 150 years (using Table 4.1). The second orbit (top right)
is produced with the initial condition
 Figure 4.19: The orbits of Trojan astroids near the Lagrange point L4
(top) and along the C-ridge (bottom).

It is approximately periodic and has a period of about 15 (~ 180

years). Both orbits are within the Trojan group. We have seen
that Jupiter has a significant effect on the precession of
Mercury because it is the most massive planet (Table 4.2). Here
it shows its influence again, only this time its lasting effect is on
the asteroids.

The largest orbit (Figure 4.19, bottom) goes all the way
around the C-shaped ridge. Its initial condition is
, . It appears to be periodic with a
period of 33 (~ 390 years). The motion is very delicate,
navigating along the equipotential lines. It does not seem to be
connected to any asteroid motion at present.

As for the Earth-Moon system, although there is no

evidence like the Trojan asteroids, numerical results show that
stable periodic orbits do exist (see Ref. [73] and Exercise
E4.12). The mass parameter is α = 0.0121, much larger than the
Sun-Jupiter mass parameter but still within the limit for the
existence of stable periodic orbits. Therefore, it is theoretically
possible to put a satellite or space station around L4 or L5.
However, with the Sun being very close and perturbing the
motion, the long term stability of these orbits is questionable.

All three orbits move clockwise ( ) around the potential

ridge. Motion in the restricted three-body system is highly
directional. Reversing the velocity of a stable periodic orbit in
the restricted three-body problem results in unstable and
nonperiodic motion (see Project S:P4.2). In contrast, reversing
the velocity would have no effect on a Kepler orbit.

4.6.4 ORBITAL RESONANCE

As viewed in the space-fixed frame where the Sun is at rest,
Jupiter and the Trojan asteroids have the same orbital period.
We say their periods are commensurate. We can think of
Jupiter's large effect on the Trojan asteroids as a result of the
commensurability, or orbital resonance. If we consider the
ratio of the orbital periods, we will find that orbital resonance
exists if that ratio is a low rational number. In the case of
Trojans and Jupiter, the ratio is 1/1. When orbits are in
resonance, there is a periodic interaction (“kicks”) between the
orbits, which does not get averaged out and can lead to
enhanced effects in the long run.

Trojans are not the only asteroids in resonance with

Jupiter, though they are the most prominent. There is another
group of asteroids known as the Hilda group which has a
period ratio of roughly 2/3 to Jupiter. The Hildas are in three
clusters forming a triangle about the Sun (see Project P4.7).

Here we discuss another interesting case of orbital

resonance between Pluto and Neptune.15 The mass parameter
of the primaries, namely the Sun and Neptune, is very small, α
= 0.0000515. The ratio of Pluto's period to Neptune's is very
close to 3/2 (see Table 4.1). For every three revolutions of
Neptune around the Sun, Pluto makes two revolutions. Like the
Trojan asteroids, the motion of Pluto is mainly determined by
the Sun. However, considering the low commensurability ratio,
we expect significant orbital resonance effect from the
perturbation of Neptune. Also, because of Pluto's relatively
high eccentricity (see Table 4.1), its orbit near the perihelion is
inside Neptune's orbit (Figure 4.20, top). The question arises
as to whether Pluto's orbit is stable, and if an “accidental”
collision may occur between Pluto and Neptune that might
knock Pluto out.
Figure 4.20: Orbits of Pluto in the Sun-Neptune system, α =
0.0000515, overlaid on the equipotential contours. Top: after several
periods. The dotted line is the circular orbit of Neptune (viewed in a
nonrotating frame). Bottom: after one half of libration (60 time units,
or ~ 10,000 years).
 Let us investigate these questions by running our
simulation (see details in Project S:P4.3). We can run Program
4.7 as is, with a different α and initial conditions, of course. But
if we combine it with Program 4.6, we can observe the orbit in
relation to the potential contours and the Lagrange points,
which will be helpful in understanding what is going on. All we
need to do is to copy parts of the code that does the numerical
integration accel() and r3body(), integrate with RK4 as usual
and record the positions x and y, and plot them on the same
figure as the potential contours when the integration ends. The
results are displayed in Figure 4.20. The initial condition used
is [48]

This starting position is close to L5.

The top figure shows the orbit for four Pluto periods.
Starting near L5 and the perihelion, we can see that Pluto's
orbit dips inside Neptune's orbit by a small amount. It is about
0.013 length units when Pluto is closest to the Sun. The length
unit is the distance between Neptune and the Sun, or 30 AU
from Table 4.1. That means Pluto penetrates Neptune's orbit by
about 0.39 AU, or nearly 40% the orbital radius of Earth. After
one Pluto period, Pluto is again closest to the Sun but at the
crossover loop diagonally opposite from the starting position
near L5. It takes two periods for Pluto to return to the starting
position. If we look carefully at the crossover loop near L5, we
find that the orbit is not closed. The loop has rotated to the left.
During this time, Neptune has revolved around the Sun three
times.

Letting the program run for longer times, we obtain the

complete orbit shown in Figure 4.20 (bottom). We see the
continued pattern of the rotation of the orbit, or precession.
The precession occurs over an angle of 76°, and remarkably, it
reverses direction after that, and returns to the starting
position to complete the cycle, which then repeats itself. The
oscillatory back-and-forth precession is also called libration
[17]. Half a libration cycle takes about 60 time units, or 10,000
years, making a full libration cycle approximately 20,000
years.

We can understand the reason of the precession as follows.

When Pluto is at the crossover loop nearer to Neptune, say at
the Lagrange point L5, it is behind Neptune (taking
counterclockwise direction as the forward). The forward pull
from Neptune increases Pluto's energy slightly. As a result, its
period also increases. The next time Pluto comes back (two
revolutions around the Sun), it will be at a later time, and
Neptune will be farther ahead, so the distance between
Neptune and Pluto's perihelion (tip of the crossover loop)
increases. Thus, Pluto appears to precess to the left, or rotate
clockwise. In effect, Neptune repels Pluto, or more accurately
the nearer loop, away. Eventually, Pluto gets ahead of Neptune
at the perihelion because of the commensurate orbital periods
(the other crossover loop near L4). Then the backward pull
from Neptune will decrease Pluto's energy and its period,
which means that Neptune will be farther back, again
effectively repelling the nearer loop. The cycles continues,
giving rise to back and forth oscillations, or libration. The
energy also oscillates with the libration period and an
amplitude of about 0.4% (see Project S:P4.3).

Because of librational motion, at the closest approach to the

Sun, Pluto gets no closer than 52° to Neptune in angular
separation. Of course, they do line up radially when passing
each other, for example, at (1.6, 0) in Figure 4.20 (bottom). But
when that happens, they are radially far apart (~ 0.6 length
units, or 18 AU). As a result, if Pluto and Neptune are close
radially, they are far apart in angular separation, and vice
versa. Therefore, a collision between them would not occur. In
this case, orbital resonance adds dynamic stability to Pluto, and
Pluto will stay in its current orbit for the foreseeable future.

We may then ask, how did Pluto end up in its current orbit?
It is possible that Pluto, at the time it was formed, happened to
be right where it is now, locked in a stable configuration due to
Neptune's perturbation. It is also possible, arguably more
likely, that Pluto evolved into its current orbit over a long
period of time (billions of years). There is evidence which
suggests that Pluto's motion may be chaotic [88]. It may well
be that we are seeing a Pluto that has been marching on its
chaotic path all along.

Chapter summary
Starting with an animated motion of a planet, we briefly
reviewed useful properties of central field motion and Kepler's
orbits. We then discussed precession of Mercury due to
corrections of the general theory of relativity. Because the effect
is small, we introduced the Runge-Lenz vector as a sensitive
indicator of precession. An accurate and area-preserving
method, the leapfrog method with time-transformation, was
implemented to simulate this process. We found an interesting
effect that precession is oscillatory, not monotonic. Precession
of classical origin due to other planets was also discussed
within the framework of central field approximation.

We also described the radial velocity method for the

detection of exoplanets. We presented two ways to model RV
datasets, least square and visual fit, and applied them to
observational datasets.

The planar three-body problem has been discussed,

including Euler's collinear motion and several interesting
choreographic solutions of gravitational systems. We studied
the restricted three-body problem, and analyzed in detail the
structure of the Lagrange points. Motion around the Lagrange
points was simulated. We presented the application to the
Trojan asteroids in the Jupiter-Sun system. We also discussed
orbital resonance and the stability of Pluto in the Neptune-Sun
system.

Throughout this chapter, we have applied several

visualization techniques using the 3D elements of VPython and
 Matplotlib. We have consistently presented the results as
dynamic animations or informative graphs with these
techniques. They are contained in blocks of sample codes that
can be used as basic templates for integrating simple but
effective visualization elements into other programs.

.7 Exercises and Projects

EXERCISES
E4.1 With the coordinate transformation (4.4), show that (a) the
equations of motion for two bodies interacting with the
internal force (4.1) are given by Eq. (4.5); and (b)* the two

body Hamiltonian can be

reduced to , where is

the center of mass motion, and H is the effective one-body

Hamiltonian given by .

E4.2 Write a program to plot the effective potential of the Earth-

Sun system, Eq. (4.8). Use Table 4.1 to determine the
angular momentum L. Use planetary motion units. Adjust
the scales so the structure near the minimum is clearly
displayed.
E4.3 (a) Modify Program 4.1 to simulate the motion of a planet
under the force F = −k/r2+δ. Run the program using the sam
initial condition as in Figure 4.4, and with δ = 0.2 and 0.5.
Compare your results with Figure 4.4. Optionally, add an
 arrow object to the animation of the planet to indicate
instantaneous velocity using the arrow() function of
VPython.

(b) When δ ≠ 0, open orbits are real. However, numeric

problems can cause unphysical artifacts, including ope
orbits that should be closed. Let us use the same program

but set δ = 0 and the initial condition to

(for exaggerated effect). First run it wit

h = 0.001, then with h = 0.004. Discuss your observation

Should we always “trust” the computer results?

E4.4 (a) Prove that the Runge-Lenz vector is a constant, i.e.,

, in the inverse-square law of force (4.1).

(b) Consider an additional perturbation force, , such th

the total force is , where is given by E

(4.1). Show that the Runge-Lenz vector obeys

As usual, and are the momentum and angula

momentum, respectively.
E4.5 Suppose that, instead of Ω(r) = 1/r, we choose Ω(r) = 1/r2 a
the time transformation. Rewrite the corresponding
equations of the leapfrog method to Eqs. (4.29a) to (4.29f).
Discuss the merits and shortcomings of this time
transformation.
E4.6 (a) A total mass M is uniformly distributed over a ring of
radius a. A point particle of mass m is placed at a distance r
from the center and in the plane of the ring. Show that the
potential energy between the particle and the ring is given
by Eq. (4.30).

(b) Expand the potential for r/a < 1 and r/a > 1, keeping fou
terms in each case. Verify Eq. (4.31), and explain why onl
even terms appear. You may wish to use SymPy to expan
the integrand,

In [1]: from sympy import *

In [2]: init_printing (pretty_print=True) # neater output
In [3]: r, x = symbols(’ rho x’)
In [4]: f=series (1/sqrt(1+r*r−2*r*cos(x)), r, 0,7) # expand to 7th

Inspect f, then integrate x ∈ [0, 2π] in SymPy (see Exercis

E2.7).

(c) Compute the value Mp/a3 for each planet in Table 4

using the parameters in Table 4.1. Plot precession vs. Mp/a
on linear scales. Compare with Figure 4.8 and discuss th
differences.
E4.7 (a) Obtain from Eq. (4.35) a relation similar to Eq. (4.36)
but with m sin i given in Jupiter masses, T in days, M in
solar masses, and V in m/s.

(b) Carry out the algebraic details to prove Eq. (4.39) fro
(4.37).

(c) Verify Eq. (4.79), and reduce it to Eq. (4.80) usin

conservation of angular momentum (4.6) and Kepler's thir
law (4.18).

E4.8 (a) Solve Eqs. (4.44) to (4.46) to obtain Euler's quintic

equation (4.47).

(b) Write a program to plot Eq. (4.47) for a few sets o

masses, say [m1, m2, m3] = [1, 1, 1] and [1, 2, 3]. How man
positive roots are there in each case? Numerically solve fo
the positive root(s) in each case using either the bisection o
Newton's methods discussed in Chapter 3.

E4.9 (a) The periodic orbit shown in Figure 4.14 is unstable but
reversible. First find its period. One way to do so is to add a
text box to Program 4.5 using the label() function of
VPython, see Program 4.3 for an example. Output the time
information, and record the period T when the three bodies
return to their initial positions (4.51). To make it easier, you
can pause the program by intercepting key presses with
scene.kb.keys and scene.kb.getkey(). The system
should start to come apart after roughly two periods. Wait
until the bodies are well off their orbits, and reverse the
velocities of all three bodies. This may be done as a
conditional statement based on time or by detecting certain
key strokes. You should observe that the bodies retrace thei
 paths and go back to the initial positions in the same amoun
of time.

(b) The figure-8 orbit shown in Figure 4.15 is stable.

means that if we start the motion from a slightly differen
initial condition, the orbit will remain close to the origin
orbit. Verify that this is true. Run Program 4.5 with th
initial condition (4.52). Next change the initial condition b

a small amount, say , and run the program

again. Finally, repeat with . Explain you

observations.

E4.10 Make a table similar to Table 4.4 showing the units of

various quantities in the restricted three-body units and in
the absolute units for the Neptune-Sun system. You may
need the data from Table 4.1.
E4.11 (a) Set up the equations for the Lagrange points where ∇V
0 from Eq. (4.61). Solve these equations to obtain the lowe
order expansions for the collinear ones L1, L2 and L3 and th
exact expressions for L4 and L5. Show that they agree with
Eq. (4.62).

(b) Write a program to solve the equations you obtained

part (a) for the collinear Lagrange points L1, L2 and L3. Us
the Earth-Moon system as an example. Compare th
numerical results with the series expansions.

E4.12 The Earth-Moon system supports stable periodic orbits

around the Lagrange points L4 and L5. Explore these orbits
with the following initial conditions,
 (a) For each orbit, find the period, and estimate its lengt
along the arc and its width where it is widest. Convert them
to absolute units. From your estimates, find the approxima
area of the smaller orbit. If that area were to be colonize
what would be the population density if the entir
population on Earth (7 billion) were to be moved? Compar
it with the current population density.

(b) Check the stability of the orbits by varying the initi

positions slightly, say to ±10%. You may also play aroun
with the velocities. Refer to [73] for a general strategy
finding initial conditions of periodic orbits.

PROJECTS
P4.1 (a) The artificial precession as observed in Exercise E4.3
above could be eliminated by using a smaller h, which
would lead to slower program speed and more round-off
error. Though speed is not a problem in this case, it could
be crucial in more complex situations. A better choice is
to use a leapfrog method with time transformation.
Repeat part (b) of Exercise E4.3 with leapfrog_tt().
Remember to initialize W0 before entering the loop (see
Program 4.3). Try different step sizes. You should see
that, no matter how large h is, there is no precession.
Explain why.

(b) Perform checks to make sure that the program works

correctly and the oscillations in Figure 4.7 are not
numerical artifacts. First, modify Program 4.3 to
calculate and record the energy and angular momentum
in the main loop. Ignore mass since it is an overall scale
factor (see Eq. (4.86)), and use a large λ ~ 10−2 for
exaggerated effects. It is best to define a separate
function for energy. Run the program for t < 10, and plot
the energy and the z-component (Lz) of angular
momentum as a function of t. Make sure your potential is
correct.

To get a better view of the fluctuations, plot the relative

error |E(t)−E0|/|E0| and |Lz(t) − L0|/L0, where E0 and L0
are the initial values computed before the loop. The
output should be analogous to Figure 2.7. Are they
conserved as expected?

(c) On the precession figure, use the zoom-in feature to

expand the scale so the oscillations are clearly visible.
Measure the amplitudes and periods. Repeat the
simulation by doubling and halving h. Check whether the
amplitudes or the periods change.

Perform the same checks by changing the sign of λ, i.e.,

enter the number −1.1 × 10−8 when prompted. Does the
 direction of precession change? Describe your
observations and conclusions.

P4.2 Though Mercury has the largest precession due to

general relativity, other planets also experience this effect
to a lesser degree. For example, the parameter for Venus
is λ = 2.1 × 10−8 AU2 (see Eq. (4.19)). Change Program
4.3 accordingly to calculate the precession of Venus per
century due to relativistic correction. Refer to Table 4.1
for the initial condition at the aphelion. Plot the results in
a similar fashion to Figure 4.7. Note the amount of
precession and the amplitude and the period of
oscillations. Expand the horizontal scale to observe the
oscillations. They should be sinusoidal, so count the
period carefully (two crests, for example).

For Earth, the parameter is λ = 2.9 × 10−8 AU2. Based on

the above results for Venus and for Mercury (Figure 4.7),
make a prediction about the amount of precession and the
period of oscillations we would expect. Write down your
predictions. Repeat the simulation for Earth. How do the
results compare with your predictions? Briefly explain.

Compare the results for Venus and Earth with Mercury.

The parameters λ for Venus and Earth are larger than for
Mercury by a factor of 2 and 3, respectively. Why are the
results not proportional to λ? Also discuss the differences
in the amplitudes and periods of the oscillations. Explain
why the oscillations are sharper (more sinusoidal) for
Venus and Earth than for Mercury. Offer any other
observations you may have.
P4.3 As we have discussed, the main contribution to the
precession of Mercury is from other planets. This amount
must be subtracted from the total precession in order to
find the contribution from general relativity.

(a) Calculate the precession due to each planet in the

central field approximation using the force (4.32) with
the parameters from Table 4.1. Reproduce the results of
Table 4.2. Make sure your results are converged with
respect to the step size h. One way to check for
convergence is to run the simulation with h = 0.002, for
example, and then reduce it by half to h = 0.001. The
results should agree to with certain error you set, say 1%
for instance.

(b) Observe that the net precession is counterclockwise,

which is in the same direction for the relativistic case.
However, the force (4.32) in the central field
approximation is repulsive, while the relativistic effect
(4.19) is attractive. Investigate why forces of opposite
signs produce precession in the same direction, because
only then can we add them up. Do this by switching on
the animation, and watch carefully how the Runge-Lenz
vector swings in relations to where Mercury is in the
orbit. Explain your observations. The results in Exercise
E4.4 may help.

P4.4 If the force on a planet does not obey the inverse-square

law, even for only a part of the trajectory, the orbit is
open and will precess. Let us investigate this effect by
modeling the Sun as a uniform sphere of radius R instead
of a point source. If the planet is outside the sphere, the
force is the usual gravity. But inside the sphere, the force
will change to a form different from 1/r2.

(a) Show that the gravitational potential and force

between a point mass m and a uniform sphere of radius R
and mass M are given by

(b) Modify the force in mercury() in Program 4.3

according to the expression above. Set R = 0.3, leave
everything else the same and run the program. Does the
Runge-Lenz vector move? Why?

(c) Compute the perihelion distance of Mercury rmin from

Eq. (4.14). Set R just above rmin and run the program.
Does the Runge-Lenz vector move now? Experiment
with R values slightly above rmin and determine the value
that produces the same precession (43″) as the relativistic
effect. Discuss your findings. Does it surprise you?

(d) Let us have fun making a Runge-Lenz clock. With the

same initial conditions, find the value of R such that the
hour hand (Runge-Lenz vector) ticks exactly 12 times in
a full circle. Alternatively, you can achieve the same
effect by varying the eccentricity of the orbit while
keeping R fixed. Set R = 0.2 and vary the initial vertical
 velocity vy. Find the value of vy when the hour hand
moves 12 times in one revolution. What would be the
eccentricity of the corresponding Kepler orbit?

P4.5 Simulate the wobbling motion of a star-planet system

with animation of the two-body problem, and discover a
mystery planet.

(a) With Program 4.1 as a basic template, add the Sun as

a movable object. Do not include the Sun as a dynamic
body in the numerical integration. We still assume an
effective one-body problem. We only need to obtain the

positions of the Sun and the planet ( and in Figure

4.2) in the center of mass (CM) from the relative planet

position stored in r, which is already available in line 26
of Program 4.1.

Assume = 0, and solve Eq. (4.4) for and in the

CM (see Exercise E4.1). Animate the Sun and the planet

by updating their positions in the CM. Make sure the
modified code runs correctly for the Sun-Earth system.

(b) Note that due to the large Sun-Earth mass ratio, the
Sun barely moves. Now change the mass of the planet,
say 0.4 solar mass. Run the code again and observe the
visible wobbles. You may also need to change the initial
conditions depending on the mass ratio.
 (c) Plot the radial velocity of the Sun, v*z, in the actual
Sun-Earth system similar to Figure 4.9, assuming edge-
on view. Discuss the results, such as the shape and
magnitude of the curve.

(d) Make your own mystery planet by assuming certain

mass, say anywhere from 1 to 10,000 Earth mass, and
initial condition (For added challenge, choose your initial
condition such that the orbit has a high eccentricity). Run
your program and generate an RV dataset accordingly.
Exchange your dataset with another classmate (or team),
and try to visually fit the dataset according to Eq. (4.34).
Calculate the planet mass from your fit. Compare your
value with the value your classmate used to generate the
dataset. Do they match? Why or why not?

P4.6 A subset of the radial velocity dataset for HD 3651

plotted in Figure 4.11 is given in the table below.

Table 4.5: Radial velocity (m/s) vs. time (day) for HD

3651.
The star has 0.8 solar masses, and the wobbling period is
assumed to be 62 days. The data points were
observations from Keck Observatory spread over the
years. They are converted (phased) to fall within one
period.

(a) Follow the procedure outlined in Section 4.4.2 to

generate a visual fit similar to the curve shown in Figure
4.11. Use Program 4.4, but turn off least square fit by
commenting out line 38 and set time shift to 0 (line 27).
Vary V, e, ω, and C but keep T and time shift fixed. As
the parameters are being adjusted, watch the effect of
each parameter, and try to isolate them to achieve a better
match. Once the curve takes the shape of the dataset,
adjust the parameters separately and in small steps to
visually fine tune the fit, focus on the shape and ignore
an apparent horizontal shift for now. Describe the effect
of each parameter.

(b) You should see that the fitted curve is shifted from the
data, e.g., the dip should be at a different location, even
though the overall shapes match very well. If you would
just shift your curve left or right, it would look quite like
the fit in Figure 4.11. Explain the physical reason for the
shift. Adjust the time shift to get the best fit. Write down
the parameters, and calculate the lower limit of the
exoplanet's mass, in Earth and in Saturn masses.
Compare with observed values [30].
 (c) Run Program 4.4 in full to obtain a least square fit.
Compare the best results between visual and automatic
fits. Optionally, switch to the reduced dataset of Table 4.5
or the self-generated dataset from Project P4.5, and
discuss any difference in the fit.

P4.7 Similar to the Trojans, there is another group of asteroids

known as the Hildas whose motion around the Sun is
heavily influenced by Jupiter. The Hildas (Figure 4.18)
do not move around the Lagrange points L4 or L5. Rather,
like Pluto, they are influenced by orbital resonance with
Jupiter. The Hildas have a period ratio of roughly 2/3 to
Jupiter.

(a) Explore the motion of the Hilda asteroids with the

following initial condition

Find the period of motion. First plot the orbit in the

rotating frame, then plot it in the nonrotating space frame
where the Sun is at rest. You would need to make a
coordinate transformation from the rotating frame to the
space frame using the rotation matrix (4.74). Compare
the orbits between the frames.

(b) Normally the orbit would also have a librational

motion. The following initial condition will show
substantial libration,
 Run the program for 10 time units. Note the angles of
libration. Modify the initial condition slightly, and
describe how the librational motion changes.

.A Rotating frames and rate of change of

vectors
Consider the coordinate systems of a space-fixed reference
frame and a reference frame rotating at constant angular
velocity ω about the common z axis. Let the coordinates be (x,
y) and (x′, y′) in these reference frames, respectively, as shown
in Figure 4.21. We assume that at t = 0 the axes (e.g., x and x′)
coincide, so the angle of rotation is θ = ωt at later times.

Figure 4.21: The space reference frame and the rotating reference
frame.
 From Figure 4.21, the relationship between the primed unit
vectors in the rotating frame and the unprimed ones in the
space frame can be written as

Because of rotation, the primed unit vectors î′ and ĵ′ change

with time. By differentiating the above equation with the help
of dθ/dt = ω, the rate of change is

Now, let us consider a position vector in the rotating

frame with components x′ and y′. The same vector in the space
frame will be denoted as . Since we are talking about the same
vector, only different notations, we have

To find the rate of change, we differentiate both sides

simultaneously

The left side refers to the rate of change in the space frame. The
first two terms on the right hand side are the rate of change due
to the change of coordinates x′ and y′ as measured in the
rotating frame, and the last two terms are due to the change of
the primed unit vectors. Let us denote this more clearly by
rewriting Eq. (4.65) as

where space is the velocity in the space frame , and

the velocity in the rotating frame
. We have also used Eq. (4.64). We can write the second
term in Eq. (4.66) as a vector product by introducing .
This leads to a more compact form

or equivalently and in a more familiar form,

In deriving Eq. (4.67), we have used the position vector as

an example and restricted the angular velocity in the z
direction. It is actually valid for an arbitrary vector and angular
velocity. The more general rule is

Equation (4.69) should be read this way: the derivative of a

vector in the space frame is equal to the derivative of the
(same) vector in the rotating frame plus the cross product of
the angular velocity and the vector. This is a very useful rule.

For instance, suppose we wish to calculate acceleration

from Eq. (4.68). We have

Applying Eq. (4.69) to each of the two terms on the right hand
side, we have

We have used and . Substituting them

into Eq. (4.70) and using the fact that is the
acceleration in the rotating frame, we obtain the relationship
for and as

Let be the net force on a particle of mass m. The equation

of motion in the space frame (inertial) is . Expanded in
the rotating frame (noninertial), it becomes
 Besides the actual force, this equation has two additional force-
like terms. The velocity-dependent term, , is the
Coriolis effect, and the other term, , is the
centrifugal force.

.B Rotation matrices
Sometimes it is necessary to transform the coordinates from
the rotating frame to the space frame. This can be done using
(x, y) = and Eq. (4.63). The results may be
expressed conveniently in the form of a rotation matrix as

The inverse transformation is

These rotation matrices work on any vectors, not just position

vectors.

Denoting the rotation matrix by A, we can write Eq. (4.74)

as
Note that we are careful with the notation rather than .
The reason is that the transformation has dual interpretations.
Here, we are following the interpretation that the operator A
transforms the components of the vector in the space frame
(unprimed) into the components of the same vector in the
rotating frame (primed). In other words, A is interpreted to act
on the coordinate system, the vector itself is unchanged, and
only the components are transformed from the unprimed
frame to the primed frame which is rotated counterclockwise
by an angle θ.

The second interpretation is that the operator A acts on the

vector , transforming it into a new vector , both in the same
reference frame. The new vector represents a clockwise
rotation of the original vector by an angle θ. We would
represent this operation as . Both interpretations
involve identical mathematical operations. In the current
context, we follow the first interpretation for convenience, i.e.,
the operator A relates the components of the same vector
between rotated coordinates.

In three dimensions, the rotation matrices about the x, y,

and z axes follow straightforwardly from Eq. (4.74),
 The inverse transformation can be obtained by changing the
sign of the angle θ → −θ, because a rotation by an angle θ is
exactly canceled by a reverse rotation by −θ, resulting in no
change (the identity transformation). If represents the
inverse transformation, we can write , and
, i = x, y, or z.

Successive rotations can be represented by a product of

rotation matrices. For example, a rotation of α about the x-axis
followed by a second rotation of β about the z-axis can be
written as

The order is important for finite rotations, and the final matrix
is built from right to left in the order of rotations. The inverse
matrix is .

.C Radial velocity transformation

Our goal is to transform the velocity in the orbital plane from
the center of mass frame to the space-fixed frame where
observation is made. As Figure 4.3 shows, the relative position
in the orbital plane is

Using Eq. (4.13), the velocity components can be obtained as,

With the help of Eqs. (4.6) and (4.12), being careful to replace
m by reduced mass μ, and using the exact form of Kepler's
third law (4.18), we can reduce the orbital (relative) velocity to

where, as before, T is the period, a the semimajor axis, and e

the eccentricity.

Let and denote the velocities of the planet and the star
in the center of mass, respectively. Then
. Solving for , we have

It is identical to Eq. (4.33) except for the primed notation

which will be convenient for the transformation below.
Substituting Eq. (4.80) into (4.81), we obtain the wobbling
velocity of the star in the center of mass frame

Next, we need to transform the wobbling velocity from the

center of mass frame (primed) to the space-fixed frame
(unprimed). This can be done conveniently by a rotation
matrix. As Figure 4.10 illustrates, the orbit orientation is the
result of two successive rotations. Imagine the aphelion (x′-
axis) and the normal (the vector, or z′-axis) of the orbital
plane are initially aligned with X and Z, respectively. The final
orientation of the orbit can be obtained as: a first rotation by an
angle i about the x′-axis, and a second rotation by an angle ω
about the z′-axis. The rotation matrix follows from Eq. (4.77), A
= Az(ω)Ax(i). Recall that A will transform a vector from
unprimed frame (space) to the primed frame (center of mass).
What we need is the inverse transformation from the center of
mass frame to the space frame, i.e.,
. Using the rotation
matrices (4.76), we have the inverse transformation

Transforming the velocity vector in Eq. (4.82) by A−1, we obtain

the velocity of the star in the space-fixed frame

This is the desired result. We pick up a Z component in the

process because the orbit is tilted due to the inclination angle.
The inclination angle i = 90° corresponds to the edge-on view,
and i = 0° the face-on view.
 We can make two extensions to the result. First, if the
center of mass is moving with velocity , we must add it to .
Second, if the star has multiple planets, and if planet-planet
interaction is negligible,16 then the total wobbling velocity is a
vector sum of contributions from all the planets. For example,
for the Z component v*z of a star with N planets, we have from
Eq. (4.84)

where MT = M + Σj mj is the total mass, and C = VCM,z is the

velocity of the center of mass along Z. Equation (4.85) can be
used to fit the Doppler velocity profile to obtain the orbital
parameters.

.D Program listings and descriptions

Program listing 4.3: Precession of Mercury ( mercury.py)

import ode, numpy as np # get ODE solvers,

numpy
2 import visual as vp # get VPython modules
for animation
import matplotlib.pyplot as plt # get matplotlib plot
functions
4
def mercury(id, r, v, t): # eqns of motion for
mercury
6 if (id == 0): return v # velocity, dr/dt
 s = vp.mag(r)
8 return −GM*r*(1.0 + lamb/(s*s))/(s*s*s) # acceleration,
dv/dt

10 def set_scene(r): # r = init position of planet

# draw scene, mercury, sun, info box, Runge−Lenz vector
12 scene = vp.display(title =’ Precession of Mercury’,
center=(.1*0,0), background=(.2,.5,1))
14 planet= vp.sphere(pos=r, color =(.9,.6,.4), make_trail=True,
radius=0.05,
material=vp.materials.diffuse)
16 sun = vp.sphere(pos=(0,0), color=vp.color.yellow,
radius=0.02,
material=vp.materials.emissive)
18 sunlight = vp. local_light (pos=(0,0),
color=vp.color.yellow)
info = vp.label(pos=(.3, −.4), text=’ Angle’) # angle info
20 RLvec = vp.arrow(pos=(0,0), axis=(−1,0,0), length = 0.25)
return planet, info, RLvec
22
def go(animate = True): # default: True
24 r, v = np.array([0.4667, 0.0]), np.array ([0.0, 8.198]) #
init r, v
t, h, ta, angle = 0.0, 0.002, [], []
26 w = 1.0/vp.mag(r) # W0 = Ω(r)

28 if (animate): planet, info, RLvec = set_scene(r)

while t<100: # run for 100
years
30 L = vp.cross(r, v) #
A = vp.cross(v, L) − GM*r/vp.mag(r) # scaled RL
vec, Eq. (4.86)
32 ta.append(t)
angle.append(np.arctan(A.y/A.x)*180*3600/np.pi) #
arcseconds
34 if (animate):
vp.rate(100)
36 planet.pos = r #
move planet
RLvec.axis, RLvec.length = A, .25
# update RL vec
38 info.text=’ Angle“: %8.2f’ %(angle[−1]) #
angle info
r, v, t, w = ode.leapfrog_tt(mercury, r, v, t, w, h)
40
plt.figure () # make plot
42 plt.plot(ta, angle)
plt.xlabel(’ Time (year)’), plt.ylabel(’ Precession
(arcsec)’)
44 plt.show()

46 GM = 4*np.pi*np.pi # G*Msun
# lamb=relativistic correction, global, used in ’mercury()’

48 lamb = input(’ Please enter lambda, eg: 0.01, or 1.1E-8

:> ’)
go(animate = True) # set to False to speed up calc. for
plots

The equations of motion due to the relativistic correction

(4.19) are computed by mercury(). It uses vector concepts and
is nearly identical to earth() in Program 4.1, with the only
difference in the correction factor 1 + λ/r2. Also, the animation
scene is contained in a standalone function set_scene().

The first block in the main program go() initializes the

position and velocity for Mercury according to Table 4.1, as well
as W0. If animate is true, the simulation will set up the
Mercury-Sun system by calling set_scene() which draws the
scene, including an arrow representing the Runge-Lenz vector
plus a text box ( label) that will display the angle of precession.
In each iteration of the main loop, the scaled angular
momentum and Runge-Lenz vectors are computed as

By this scaling, mass drops out. The angle of precession is

found by arctan(Ay/Ax) and converted into arcseconds from
radians. We use full vector operations provided by VPython for
and . Note that the components of a VPython vector such as
can be accessed by index or subscript, e.g., A[0] is the same
as A.x.

Next, animation is effected as necessary. This involves

changing the Mercury position and updating the direction of
the Runge-Lenz vector. The current precession angle
( angle[-1]) is converted to a float string17 (line 38) and
displayed in the text box. Lastly, the integrator leapfrog_tt()
advances the system forward by one step h, in transformed
time s. Upon exiting the main loop, the results are displayed in
a graph.

Program listing 4.4: Radial velocity data modeling ( rvfit.py)

import numpy as np # get

numpy
2 import matplotlib.pyplot as plt # get
plot functions
from scipy.optimize import leastsq #
least square fit
 4
def rv(V, e, omega, C, theta): #
radial velocity
6 return −V*(np.cos(omega + theta) − e*np.cos(omega)) + C

8 def nearest(a, b): # find indices of neareast

differences
diff = np.subtract.outer(a, b) #
find diff : a=[], b=[] or scalar
10 return np.argmin(np.abs(diff), 0) #
nearest index

12 def time_theta(e, tshift, fit = True): #

calc time and theta
psi = np.linspace (0., 2*np.pi, 200) # psi
grid
14 t = (psi+e*np.sin(psi))/(2*np.pi) + tshift #
Kepler eqn
over, under = t>1.0, t<=1.0 #
truth arrays
16 t = np.concatenate((t[over]−1, t[under])) #
wrap around
theta = np.arccos((e+np.cos(psi))/(1+e*np.cos(psi))) #
calc theta
18 theta[psi>np.pi] = 2*np.pi−theta[psi>np.pi] #
remap to [0,2*pi]
theta = np.concatenate((theta[over], theta[under]))
# wrap around
20 idx = nearest(t, time) #
index of data points
return (t[idx], theta[idx]) if fit else (t, theta)
22
def error(p, v, tshift): # error function for leastsq
24 t, theta = time_theta(p[e], tshift)
return v − rv(p[V], p[e], p[omega], p[C], theta)
26
T, tshift = 62.23, 0.38 # T=period (day), expt. with
 time shift
28 time, vel, err = [], [], []
with open(’ hd3651.txt’, ’ r’) as file: # read HD 3651 dataset
30 for line in file :
if line.strip () and line.strip ()[0]!= ’ #’: # not
blank/comment
32 t, v, er = eval(line) # comma separated fields
time.append(t), vel.append(v), err.append(er)
34 time, vel = np.array(time)/T, np.array(vel) # scaled
time

36 p = [10., 0.5, 1., 1.] # initial guess: [V, e, omega, C]

V, e, omega, C = range(4) # para labels [0−3] for easy
ref
38 p, flag = leastsq(error, p, args=(vel, tshift)) # Go fit

40 t, theta = time_theta(p[e], tshift, 0) # calc fitted

results
vfit = rv(p[V], p[e], p[omega], p[C], theta)
42
plt.figure ()

44 plt.errorbar(time, vel, err, fmt=’ o’) # data and fit

plt.plot(t, vfit, lw=2)
46 plt.xlabel(’ $t/T$’), plt.ylabel(’ Radial velocity (m/s)’)
plt.text (0.10, −30, ’ HD 3651’)
48 plt.ylim(−40, 30), plt.show() # set ylim

The purpose of this program is to model radial velocity (RV)

datasets. Starting with a set of parameters, the overall strategy
is: compute the radial velocity over a time grid, match it with
the nearest observation points, find the error at these points,
and input the error to a least square fitting routine which
repeats the process until it hopefully converges to an optimal
set of parameters.
 The main program starts (line 27) by setting the period T
and a time shift tshift (change the value to see its effects, see
below). The period is not treated as a free parameter, and the
time shift is chosen manually. We do so to reduce the
parameter space, which is still considerable (four, see Eq.
(4.34)).

The next block reads in the RV dataset from a file. The file is
opened with with…as (line 29), a handy Python construct that
automatically cleans up after operations within the block (e.g.,
closing the file in this case). For each line in the file, after being
stripped of white space, if it is not blank (zero length) or a
comment (“#” as the first character), eval() is called to parse
the data fields assumed to contain three comma-separated
values, namely time, velocity, and error bar, all added to the
respective lists. After processing the file, the time and velocity
lists are converted to ndarrays, time and vel respectively, for
vector operations throughout. Time is also scaled so it is
between 0 and 1.

As the initial guess, the parameters [V, e, ω, C] from Eq.

(4.34) are assigned to a list and indexed for easy reference (e.g.,
p[e] instead of p[1] to avoid confusion). We are ready now to

enter leastsq, a least square routine from SciPy, which

requires a user-supplied function computing the difference
between the model and the data, an initial guess, plus optional
parameters (such as velocity and time shift). We name this
function error().
 Upon entering error(), the first function called is
time_theta() which accepts as arguments the eccentricity, time
shift, and a boolean variable (default true) indicating fitting or
evaluation. The logic of this function is a bit more subtle than
usual (and it may take a little digging to understand it), but its
purpose is clear: to match a pre-generated time grid with the
nearest data points for error computation later. We use the
NumPy function, linspace(start, end, num), to generate a 1D
grid over ψ. It returns an array of num evenly spaced points
between [start, end] inclusively ([0, 2π] in this case). Using
Kepler's equation (4.39), the time grid t is created from array
psi in element-wise operations. Here, it is necessary to shift
time by a trial amount tshift because the starting time in the
exoplanet orbit is unknown. To reduce the fitting complexity,
we keep this parameter manually adjusted as stated earlier (see
Project P4.6).

Once shifted, some points will be over the maximum time 1,

so they need to be wrapped around. This is done in line 16 via
np.concatenate to combine the two halves of array t. The first
half of the original array is found by t[over]. The condition
over = t>1.0 yields a NumPy truth array whose elements are
true if the elements of t are greater than 1 (hence we subtract 1
before wrapping) and false otherwise. This truth array is then
used to take only true elements with advanced indexing (see
Section 1.D). Similarly, the second half is found by t[under]
with under = t<=1.0.
 Likewise, the theta (θ) grid is created from ψ via Eq. (4.40).
Because arccos is multivalued, θ thus computed will be in the
range [0, π] for 0 ≤ ψ < π, and in [π, 0] for π ≤ ψ < 2π. Since
both θ and ψ should vary over the same range [0, 2π] (see Eq.
(4.40)), we need to remap the second half of θ correctly. We do
this in the same way as for t above, replacing the respective
condition with psi>np.pi, and remapping the second half to [π,
2π] by subtracting itself from 2π. Finally, the θ values are
wrapped around like time t above, so they are synchronized
with each other.

Now, we need to match time t nearest to data points time

so as to compute the difference at approximately the same
(orbital) time. For this task, we define nearest(a,b) which
accepts a as a 1D array and b as either a 1D array or a scalar,
and returns the indices of the elements of a that are nearest to
the elements of b. For example,

In [2]: nearest ([1., 2., 3., 4.], [1.2, 2.7])

Out[2]: array([0, 2])

The pair of numbers in b are closest to the first and the third
elements of a, respectively, so the returned index array is [0,
2].
 The function nearest() uses the NumPy outer method,
which turns most functions to work in an outer-product
manner (see Program 8.4 for the outer function). Here (line 9),
the np.subtract function modified by the outer method
computes the difference between every possible pair of
elements in t and time in such a way that a 2D array is formed
such that diff[m,n]=t[m]-time[n]. On the next line we locate
the indices of the nearest data points using np.argmin which
returns the indices of the minimum absolute differences along
axis 0 (down a column). This basically amounts to a table
lookup. On return from nearest(), the variable idx now
contains the indices of t closest to the data points time. Lastly,
depending on the fitting flag in the inline if-else statement
being true or false, only the indexed or all values of time and θ
are returned. The false condition should be used after a
successful fit to evaluate the full results.

Back to the error() function, we can now calculate the

difference between the velocity data and the model with the
function rv() according to Eq. (4.34) using the fit parameters.
The difference, an array of length equal to the number of data
points, is passed to leastsq for further optimization.

Assuming a successful fit, we calculate the results of our

model using the fitted parameters. The data and the fit are
plotted, with error bars and smooth curves, respectively (see
Figure 4.11). The y-axis limit is set manually (line 48).
 The combination of the table-lookup strategy and the least
square routine leastsq seems to be stable, and should work
well for moderately elliptic orbits. It can become inadequate if
the eccentricity is high. In such cases, it may be better to solve
Kepler's equation (4.39) to obtain ψ at each data point exactly.

Program listing 4.5: Three-body motion ( 3body.py)

import ode, rootfinder as rtf # ode, root solvers

2 import visual as vp, numpy as np # VPython, numpy

4 def threebody(id, r, v, t): # Eqns of motion for

3−body
if (id==0): return v # return velocity
array
6 else: # calc acceleration
r12, r13, r23 = r[0]−r[1], r[0]−r[2], r[1]−r[2]
8 s12, s13, s23 = vp.mag(r12), vp.mag(r13), vp.mag(r23)
a = [−m2*r12/s12**3 − m3*r13/s13**3, #
, Eq. (4.42)
10 m1*r12/s12**3 − m3*r23/s23**3,
m1*r13/s13**3 + m2*r23/s23**3]
12 return np.array(a) # return accel
array

14 def quintic(x): # Euler’s quintic equation, Eq. (4.47)

return −m1−m2 + x*(−3*m1−2*m2 + x*(−3*m1−m2
16 + x*(m2+3*m3 + x*(2*m2+3*m3 + x*(m2+m3)))))

18 def dquintic(x): # derivative

return −3*m1−2*m2 + x*(2*(−3*m1−m2) + x*(3*
(m2+3*m3)
20 + x*(4*(2*m2+3*m3) + x*5*(m2+m3))))
22 def init_cond(scale): # collinear initial condition
r, v = np.zeros ((3,2)), np.zeros ((3,2)) #y=
[ ], same for v
24 x = rtf.newton(quintic, dquintic, 1.,2. e−16) #
solve for λ
a = (m2+m3−m1*(1+x+x)/((x*(1+x))**2))**(1./3.)
26
r[1,0] = (m1/(x*x)−m3)/(a*a) #
non−zero x only
28 r[0,0] = r[1,0]−x*a
r[2,0] = −(m1*r[0,0] + m2*r[1,0])/m3 #
CoM at 0
30 v[0,1], v[1,1] = scale*r [0,0], scale*r[1,0] #
non−zero Vy only
v[2,1] = −(m1*v[0,1] + m2*v[1,1])/m3 #
CoM at rest
32 return r, v

34 def set_scene(R, r): # create bodies, velocity

arrows
vp.display(title =’ Three-body motion’, background=
(1,1,1))
36 body, vel = [], [] # bodies, vel arrows
c = [(1,0,0), (0,1,0), (0,0,1), (0,0,0)] # RGB
colors
38 for i in range(3):

body.append(vp.sphere(pos=r[i],radius=R,color=c[i],make_trail
=1))
40
vel.append(vp.arrow(pos=body[i].pos,shaftwidth=R/2,color=c[i]
))
line, com = vp.curve(color=c[3]), vp.sphere(pos=(0,0),
radius=R/4.)
42 return body, vel, line

44 def run_3body(scale):
 t, h, ic, cycle, R = 0.0, 0.001, 0, 20, 0.1 # anim
cycle, R=obj size
46 r, v = init_cond(scale)
body, vel, line = set_scene(R, r) # create
objects
48 while True:
vp.rate(1000)
50 r, v = ode.leapfrog(threebody, r, v, t, h)
ic = ic + 1
52 if (ic % cycle == 0): # animate once per
’cycle’
for i in range(3): # move bodies, draw
vel, path, lines
54 body[i]. pos = r[i] # bodies
vel [i].pos, vel [i].axis = body[i].pos, v[i]
56 vel [i].length = R*(1+2*vp.mag(v[i])) #
scale vel vector
line.pos = [body[i].pos for i in [0,1,2]] #
lines
58 m1, m2, m3 = 1., 2., 3. # masses, global
60 run_3body(scale = input(’ enter scale, eg 0.7 :> ’))

The heart of the program is threebody() which returns the

equations of motion of the coplanar three-body problem, Eqs.
(4.41) and (4.42). It uses the same vector concept as in earth()
of Program 4.1. There are three bodies, so r and v are 3 × 2
arrays (line 23) holding two-component position and velocity
vectors, respectively, of the three bodies. For instance, the
position vectors are stored as r = [[x1, y1], [x2, y2], [x3, y3]], so
r[0] = [x1, y1] gives the position vector of body 1, and r[2, 1] =
y3 gives the y-component of body 3. This simplifies the
calculation of acceleration, which again takes the algebraically
written form of Eq. (4.42), and at the same time satisfies the
requirement of the leapfrog integrator. We have also used
Newton's third law with .

The next two functions, quintic() and dquintic() return

the quintic equation (4.47) and its derivative. They are used for
finding the root λ by Newton's method, which is particularly
suited for our purpose here because it stably converges to the
only root faster than the bisection method, and does not
require an initial bracket.

The initial condition is calculated in init_cond(). After

initializing the position and velocity vectors as 3 × 2 NumPy
arrays described above, it goes on to find λ, as well as a.
Because the three bodies are initially on the horizontal axis
rotating about the center of mass, all vertical positions and
horizontal velocities are zero. The nonzero components are
calculated from Eqs. (4.49) and (4.50). The parameter ω is
omitted and defaults to 1 without loss of generality. We
introduce a scale factor, scale, which basically controls the
energy and the shape of the ellipse. When scale=1, the orbits
are circular. Too small, the ellipses are very elongated; and too
large, the motion becomes unbound. A useful range is between
0.5 and 1.3. Note that the y velocity component of body 3,
v[2,1], is set more accurately assuming the center of mass at
rest.

The main function run_3body() takes the input scale, and

obtains the initial condition. It calls set_scene() which creates
the three bodies, velocity vectors as arrows, all appended to the
respective lists, and the line connecting the bodies. In the main
loop, leapfrog() is used to integrate the motion. Rather than
animating the motion after each step during which the bodies
move only slightly, the scene is updated every cycle number of
steps, controlled by the counter ic whenever ic modulus cycle
is zero. Animation is updated by changing the current positions
of the bodies (line 54), as well as the position, direction, and
the length (proportional to speed) of the velocity arrows. The
lines connecting the bodies are redrawn by setting the points in
the curve to the bodies’ positions (line 57). Lastly, the masses,
m1 : m2 : m3 = 1 : 2 : 3, are set as global variables because they
are used in several functions.

Program listing 4.6: Plotting the effective potential

( r3body_veff.py)

import numpy as np # get numpy functions

2 import matplotlib.pyplot as plt # get matplotlib plot functions
from mpl_toolkits.mplot3d import Axes3D # for surface plots
4

def potential(x, y): # returns effective potential, without 4piˆ2

6 r1 = np.sqrt((x+a)*(x+a)+y*y) # use np.sqrt for array operations
r2 = np.sqrt((x−b)*(x−b)+y*y)
8 V = −(1.0−a)/r1 − a/r2 − 0.5*(x*x+y*y)
return np.maximum(V, −4.0) # low cutoff value
10

def makeplot():
12 n, s = 201, 5 # grid points and stride
x = np.linspace(−1.5, 1.5, n) # same x,y grids
14 x, y = np.meshgrid(x, x) # make meshgrid, ie, 1D −> 2D
16 V = potential(x, y) # compute V, −grad(V)
Fy, Fx = np.gradient(−V) # swap Fx, Fy since V[i,j]=V[x,y]
18 big = Fx*Fx + Fy*Fy > 0.003 # truth array for A[i, j] >.003
Fx[big], Fy[big] = 0, 0 # cut off large values
20

fig = plt.figure () # add surface plot with strides and color map
22 axis=fig.add_subplot(111, projection=’ 3d’) # add subplot
axis.plot_surface (x, y, V, rstride=3, cstride=3,
24 linewidth=0, cmap=plt.cm.spectral)
axis.set_xlabel (’ x’), axis.set_ylabel (’ y’), axis.set_zlabel (’ V’)
26

plt.figure () # draw contours and arrows

28 plt.subplot(111, aspect=’ equal’) # square picture

plt.contour(x, y, V, 26) # 26 contour lines
30 plt.xticks ([],[]), plt.yticks ([],[]) # omit ticks

32 # draw arrows on strided grid to space them out, via array slicing
plt.quiver(x [:: s ,:: s], y [:: s ,:: s], Fx[:: s ,:: s], Fy[:: s ,:: s],
34 width=0.005, minshaft=2, minlength=0) # arrow shape
txt = [’ $L_1$’, ’ $L_2$’, ’ $L_3$’, ’ $L_4$’, ’ $L_5$’] # Lag. pts label
36 loc = [(−1.1, −.07),(.5, −.07),(1.2, −.07),(.3,.8),(.3, −1)] # label pos
for i in range(len(txt)):
38 plt.text(loc [i][0], loc [i][1], txt[i], fontsize=16)

40 plt.show()

42 alpha = 0.121 # globals

a, b = alpha, 1.0−alpha
44 makeplot()
 This program computes and plots the scalar potential and
vector force fields (Figure 4.17). The function potential()
evaluates the effective potential (4.57). It might seem to take on
only scalar input x and y, but in fact it works equally well on
array input. This is because only NumPy universal functions
sqrt and maximum are used (Section 1.D). Therefore,
potential() is a ufunc, and it is more efficient to generate the
potential on a grid on the fly.

The next routine makeplot() makes the plots. First we use

np.linspace to generate a 1D grid as in Program 4.4. The
function meshgrid() transforms two 1D-arrays into two 2D-
arrays (or mesh). If X,Y=meshgrid(x,y), then X consists of
repeating rows of x and Y repeating columns of y. Therefore, a
grid point (x[i], y[j]) would be given by (X[j,i], Y[j,i])
after the transformation. For example, let x=[1, 2, 3, 4] and y=
[7, 8, 9], then

If we choose i=1, j=2, the grid point is (x[1], y[2])=(2,9),

which is the same as (X[2,1], Y[2,1]).

The actual statement x, y=meshgrid(x, x) (line 14) means

that we have a square grid, stored in 2D arrays. With array
inputs to potential(x, y), the potential is calculated at every
grid point and returned in an array with the same shape as the
input arrays automatically. In effect, each statement in
potential() does implicit looping over two-nested loops. All

this magic occurs because potential() is a universal function.

Having obtained the potential on the grid, we compute its

gradient (line 17) using the NumPy gradient function, which
returns the x and y components. Note that we have to swap the
Fy and Fx components since x runs across columns and y

across rows, which are the second and first dimensions,

respectively, in matrix notation.18

Just as we set a low cutoff value for the potential (line 9) to

avoid singularities when plotting, we also set a high cutoff value
for the gradient to suppresses very big arrows in plotting the
force vectors. Line 18 builds a truth array (boolean) big whose
elements are true if the magnitude exceeds the specified value,
and false otherwise. We then invoke advanced indexing, using
the boolean array as the index (see Section 1.D), to set to zero
the elements of the gradient corresponding to true elements in
big (line 19).

With both the potential and force fields properly prepared,

we proceed to plot them with Matplotlib. The first figure (line
21) adds a subplot with 3d projection, and calls plot_surface()
from the Axes3D package to make the surface plot of the
potential. The rstride and cstride parameters use fewer data
points by skipping rows and columns, making the plot easier to
manipulate and figure saves smaller. The line width linewidth
(or lw) is set to zero to suppress the wire mesh. In the second
figure (line 27), a contour plot of equal aspect ratio is produced
with contour(). In addition, a force vector field is overlaid on
the contours with the quiver() function. Again, points are
skipped via slicing so the arrows are not too crowded. The
paired ’ $’ characters in the text labels (line 35) typesets the text
in math mode. The block (lines 21 to 34) can be used as a
basic template for plotting any data.

Program listing 4.7: Restricted three-body ( r3body.py)

import ode, numpy as np # get ODE solvers, numpy

2 import visual as vp # get VPython modules for animation

4 def r3body(y, t): # equations of motion for restricted 3body

r, v = y[0], y[1]
6 r1, r2 = r − [−a,0], r − [b,0] # rel pos vectors
acc = −GM*(b*r1/vp.mag(r1)**3 + a*r2/vp.mag(r2)**3) # Eq. (4.55)
8 acc += omega**2*r + 2*omega*np.array([v[1], −v[0]]) # Coriolis term
return np.array([v, acc])
10

def set_scene(r): # r = position of test body

12 vp.display(title =’ Restricted 3body’, background=(1,1,1))
body = vp.sphere(pos=r, color=(0,0,1), radius=0.03, make_trail=1)
14 sun = vp.sphere(pos=(−a,0), color=(1,0,0), radius=0.1)
jupiter = vp.sphere(pos=(b, 0), color=(0,1,0), radius=0.05)
16 circle = vp.ring(pos=(0,0), color=(0,0,0), thickness=0.005,
axis=(0,0,1), radius=1) # unit circle
18 return body

20 def restricted_3body(y): # y = [r, v] expected

 testbody = set_scene(y[0])
22 t, h = 0.0, 0.001
while True:
24 vp.rate(2000)
y = ode.RK4(r3body, y, t, h)
26 testbody.pos = y[0]

28 GM, omega = 4*np.pi**2, 2*np.pi # G(M1+M2), omega, RTB units

alpha = 0.0009542 # Sun−Jupiter system
30 a, b = alpha, 1.0−alpha
r, v = [0.509046,0.883346], [0.162719, −0.0937906] # init pos, vel
32 restricted_3body(np.array([r, v]))

The program simulates restricted three-body motion in the

Sun-Jupiter system. It is very similar to Program 4.5. But it
uses the RK4 method rather than the leapfrog integrator
because the motion involves a velocity-dependent force (4.55).
Accordingly, r3body() has the same structure as baseball() in
Program 3.8. It takes the input y containing the position and
velocity vectors in an ndarray, and returns the velocity and
acceleration vectors. The Coriolis term in Eq. (4.55) is
simplified to , because and
. This is used in line 8.

Most of the VPython objects are the same as in Program 4.5.

We use vp.ring() to represent the unit circle.

1 3
In the calculation of s , instead of the power operator s**3, multiplications ( s*s*s) are
used. It is faster this way (up to power ~ 5), and equally readable.
 2
Such keyboard polling is included in the VPython module (VPM) library (Program
S:6.4). The library also defines several classes of objects for use in later chapters, such as the
slinky and mesh surfaces.

3
Another common choice is θ0 = 0. In that case, rmin and rmax would be swapped.

4
Though Pluto was reclassified as a minor planet recently, it does not affect our treatment
below.

5
For readers interested in technical details, see Sec. 3.6 of Ref. [40].

6
A more quantitative explanation of the direction of precession is given in Ref. [61].

7 α
This is a special case of power scaling laws, y = cx , where x is the scaled variable, α
α
the exponent (or power), and c a constant (usually unimportant). If x″ = λx, then y′ = λ y.

8
The latest estimate based on the Kepler mission is that one fifth of Sun-like stars in our
galaxy has an Earth-like planet, totaling to tens of billions habitable exoplanets.

9
As of this writing, 1022 exoplanets have been confirmed, 548 of which were discovered
with the RV method [66].

10
This is not to be confused with the radial velocity in Eq. (4.8) which represents the
orbital velocity of the planet directed radially.

11
Physics is not only interesting but also colorful sometimes, judging by some terms
used.

12
We do this in scattering studies of atomic reactions. See Section S:12.5.

13
There are variations about the labeling of L1, L2 and L3. We sequentially label them
from left to right, agreeing with the convention in Ref. [40].

14
This should not be confused with the ability of a central potential to support closed
orbits discussed in Section 4.2.5. The forces involved here are velocity dependent and not
central, and the orbits are not closed in the space-fixed coordinates.

15
Pluto and Neptune have different inclination angles. We ignore that difference here and
assume they move in the same plane.
 16
This amounts to the independent particle approximation. It is a good approximation,
particularly if we are primarily interested in the wobbling motion of the star.

17
The float-to-string conversion works as ’ format’ % (expressions). Here, format is a
string containing the C-style format specifier %W.PT where W is the field length, P the
precision (decimal places), T the number type. All except % and the number type are
optional. In the present case, ’ %8.2f’ specifies a float of length 8 and 2 decimal places.

18
The same problem arises in electrostatic potentials on a grid, see Program 7.2.
Chapter 5
Nonlinear dynamics and chaos
We have thus far dealt with systems whose time evolution is
described by a set of ODEs. Given initial conditions, we can
integrate the ODEs and obtain numerical solutions at any point
in the future. When the outcome is uniquely determined by the
initial condition, we say the system is deterministic. Newtonian
systems such as projectile and planetary motion are
deterministic. For the most part (except unstable few-body
systems alluded to in Section 4.5), we did not have to worry
about any unpredictable results, in the sense that if we
propagate the system from two nearby initial conditions, we
should expect the final results to be close, at least not
catastrophically different.

Starting with Poincaré's work on celestial mechanics, that

notion of predictability had been challenged. But the field has
rapidly accelerated in the last half century with what Edward
Lorenz thought was a computation error when the computer
generated results showed a totally unexpected outcome. Since
then, numerical computation has led to the discovery of an
entirely new field of science, deterministic chaos, often
considered a glory of computational science. The field of chaos
covers everything from physical systems to biological ones,
from surprisingly simple models to complex ones [35]. When
chaos occurs, the system becomes unpredictable and loses long
term predictability, in the sense that if we start the evolution of
the same system (same ODEs) with slightly different initial
values, the outcomes are totally different. The so-called
butterfly effect, stemming from the seminal work of Lorenz on
hydrodynamic flows, refers to extreme sensitivity to initial
conditions in weather in which a small disturbance may cause a
storm half a world away. It is qualitatively different than a coin
toss where random outcomes are a result of not catastrophic
difference but the lack of precise initial conditions, assuming
negligible air resistance (e.g., in vacuum).

Because these systems require knowledge of ODEs only, we

can apply the well-tested techniques covered so far to study
them before turning attention to systems described by partial
differential equations starting next chapter.

A necessary ingredient for chaos to occur is nonlinearity in

the system dynamics. Broadly speaking, if a system responds to
input in a nonlinear (or disproportionate) fashion, we say it
exhibits nonlinear dynamics. In this chapter, we will study
nonlinear dynamics and chaos in systems from discrete logistic
maps to continuous systems including driven nonlinear
oscillators and the Lorenz model. We will examine how chaos
 develops, how to characterize it, and how to predict the
unpredictable.

.1 A first model: the logistic map

We begin with the simplest system exhibiting chaotic behavior,
the so-called logistic map. We will explore this system in
greater detail, for it helps us introduce several important
concepts with clarity.

The logistic map is defined as

This is a discrete, nonlinear system where given the value xn at

stage n, its next value xn+1 can be calculated (mapped). Here r
is called the control parameter.1 If we introduce the map
function as

then Eq. (5.1) can be written as

 Figure 5.1: The map function of the logistic map.

We speak of this equation as a 1D map, which maps an

existing point into a new one, deterministically. The map
function f(x) is shown in Figure 5.1. Since the maximum of f(x)
is at x = 1/2 with fmax = r, the series will remain bounded
according to Eq. (5.1).

What could the logistic map represent, you may wonder?

One way is to view it as the cycles of an insect population. For
example, we could think of xn as representing the scaled
population (in some unit) of a colony of bugs at time n. The
population in the next cycle, xn+1, would crudely depend on two
things: the number of existing bugs (xn) for reproduction, and
the amount of resources available (e.g., food), 1−xn, where we
assume the resource decreases at a rate proportional to the bug
population. Then r would be a combination of birth rate and
consumption rate. The detail of the model does not matter, but
the nonlinear term in Eq. (5.1) does.
5.1.1 MAP ITERATION
For a given control parameter r and initial value x0, a series of
map iterates may be determined as

We will use the short hand f(n) to denote the nth iterate of the
map as

Figure 5.2: The values xn as a function n of the logistic map for

different values of the control parameter r. The starting value is the
same, x0 = 0.8.

We can explore the map iterates using Program 5.1. The

program first reads the input for x0 and r from the user, then
calculates the map iterates up to a specified number of steps. At
each step, the value xn is also format-printed to the console,
four values per line, controlled by the end variable (line 8). Like
in Program 4.3, the format string “’ %8f’ %x” prints 8 digits of
the float x. The inline if-else statement produces either a
space, or a new line (’ \n’) every four steps. The results are
plotted at the end.

Program listing 5.1: Logistic map ( logisticmap.py)

from _future_ import print_function # use print() as function

2 import matplotlib.pyplot as plt # get matplotlib plot functions

4 x, r = input(’ enter x0, r: ’)

n, xn = 40, []
6 for i in range(n):
xn.append(x) # new line every 4 steps
8 print(’ %8f’ %x, end=’ \n’ if len(xn)%4==0 else ’ ’) # inline if
x = 4*r*x*(1.0−x) # next iteration, logistic map
10

plt.figure()
12 plt.plot(range(n), xn, ’ --o’) # plot dashed line with 'o’ symbol
plt.xlabel(’ $n$’), plt.ylabel(’ $x_n$’)
14 plt .ylim (0,1), plt .text(33, .1, ’ r=’+ repr(r)) # add text
plt.show()

We show in Figure 5.2 representative plots for several

values of the control parameter r. For all values of r except the
last, xn (bugs population) undergoes an initial transient phase,
then settles into a periodic pattern. For the smallest r = 0.2, the
population quickly drops to zero and stays there. Too low a
birth rate or too little resource cause the colony of bugs to go
extinct. Increasing r to 0.7 yields a stable population. In both
cases the system is in a period one cycle.

For the next three larger r values, the population repeats

after two, four, and eight steps, respectively. We speak of
period two, four, and eight cycles. For example, the first 24
points for r = 0.88 and x0 = 0.8 as printed from the program
are

0.800000 0.563200 0.865940 0.408629

0.850613 0.447289 0.870220 0.397539
0.843046 0.465764 0.875874 0.382689
0.831558 0.493043 0.879830 0.372168
0.822479 0.513945 0.879315 0.373542
0.823709 0.511148 0.879563 0.372882

We can see that after initial transients, the series quickly

converges to a cycle of period four, i.e., the same value
appearing after every four steps.

Here interesting dynamics is starting to emerge for

increasing r, as cyclic populations start to appear. These
periodic values are known as fixed points. Without the
nonlinear term in the logistic map (5.1), the only fixed
point would be zero. Increasing r further to 0.95, no apparent
periodic structure is recognizable. As we will soon discover, the
logistic map has entered the chaotic region.
5.1.2 FIXED POINTS AND ATTRACTORS
As we have just seen, only fixed points appear to remain after
transients. Let x* represent the fixed points. For period-one
cycles, the value repeats after one step, so we have

We can rewrite the above relation using the map function (5.2)
and (5.3) to get

Solving the above equation gives us two fixed points for period-
one cycles as

Similarly, period two cycles repeat every two steps,

Correspondingly, the fixed points are given by two successive

applications of the map function like in Eq. (5.4), resulting in a
fourth-order (quartic) equation. The four roots are the fixed
points of the period-two cycle,2
The first two fixed points are also those of the period-one cycle,
because if a value repeats after one step, it automatically
repeats after two steps.

For higher period cycles, it gets progressively messy to find

the fixed points algebraically, and it would be easier to do so
numerically. We can also locate the fixed points graphically, as
illustrated in Figure 5.3 for period one and two (see Exercise
E5.2).

The map functions are drawn as curves for f(x) and f(2)(x),
respectively, for period one and two. Note the double hump in
the period-two curve. The intersections of the diagonal lines
and these curves are the fixed points: 2 for period one, 4 for
period two, both including the point 0.

Figure 5.3 also demonstrates how the fixed points are

attracted to, starting from x0 = 0.1 in both cases. For period
one, the map iterates converge to the larger fixed point ~ 0.64
instead of 0, though the latter is closer to the starting value. For
period two, something more interesting happens: the iteration
marches toward a fixed point (~ 0.7) quickly at first but does
not stay there. Rather, it moves away and ends up at another
fixed point (~ 0.47). This explains the appearance of these two
points in Figure 5.2 for the corresponding r values.
 Figure 5.3: The fixed points and their convergence for period one and
two.

The fixed points that the iterates eventually converge to are

called attractors. All the periodic points observed in Figure 5.2
after the initial transients are attractors. However, not all fixed
points are attractors. It begs the question: What is it that leads
the iterates toward some fixed points but away from others?
What makes a fixed point an attractor? The answer lies in the
stability of fixed points.

5.1.3 STABILITY OF FIXED POINTS

For convenience, let us formally introduce the (compound)
map function of period n as

where f(n) represents n successive operations defined in Eq.

(5.5). It follows that for the fixed points of period n, x*, we have
 Figure 5.4: Deviations around a fixed point. If x is at a small distance
∈ (horizontally) from the fixed point x*, the value xn in the next cycle
of period n can be found from the mapping function gn(x) (the
arrows).

For stability analysis, let us pick a point x from the small

neighborhood surrounding a fixed point x*. As illustrated in
Figure 5.4, if x starts at a distance ∈ away from x*, it will be
mapped in the next step to xn = gn(x) which is at a distance ∈n
from the fixed point. When projected on the x-axis via the
projection line y = x (similar to Figure 5.3), xn will be at a
horizontal distance ∈n from x*.

The distances are related to the local properties of the map

function gn(x). Graphically from Figure 5.4, we see that ∈n = h
= ∈| tan θ|. For small ∈, , and we have

We can get the same result mathematically (Exercise E5.3).

If a small deviation leads back to the fixed point, i.e., if ∈n <

∈, the fixed point will be stable. Conversely, if ∈n > ∈, the fixed
point is unstable, because points in that neighborhood will
move away from it. Therefore, the condition for the stability of
a fixed point depends on as,

If the derivative is zero, , the fixed point is said to

be super-stable because the convergence is much faster than
the exponential rate in normal cases.

The calculation of can be carried out numerically, or

analytically by chain rule with the aid of Eq. (5.12),

For example, the derivatives for period one at the fixed

points (5.8) are

According to Eq. (5.14), the first fixed point (x* = 0) is stable,

i.e., 1, for r < 1/4, and the second one is stable for 1/4 <
r < 3/4. It is for this reason that in Figure 5.2 the population
drops to zero for r = 0.2 and reaches a finite value for r = 0.7
(Figures 5.2 and 5.3).
 By the same token, only the last two fixed points of period
two in Eq. (5.10) are stable (Exercise E5.4), as the first two are
from period one (0 and ~ 0.7 in Figure 5.3, right panel), which
as we have seen are unstable at r > 3/4.

Figure 5.5: Regions of stability of the fixed points of period one and
two.

The regions of stability of the fixed points are summarized

in Figure 5.5 over the full range of the control parameter. At a
given r, one of the fixed points of period one is stable until r =
r1 = 3/4 when both become unstable. Filling the void are two,
new fixed points from period two, which double the number of
stable fixed points.

However, the new stable fixed points themselves become

unstable at r = r2 ~ 0.862 (see Exercise E5.4). What then? Will
period doubling continue? Figure 5.2 suggests yes. We will
discuss how next.
5.1.4 PERIOD DOUBLING AND ROUTE TO
COMPLEXITY
To explore the region beyond r1 in Figure 5.5, we map the
stable fixed points (attractors) as a function of r in small
increments with Program 5.2.

Program listing 5.2: Period doubling ( perioddbl.py)

import matplotlib.pyplot as plt # get matplotlib plot functions

2 ntrans, nsteps = 1000, 200 # num. of transients and to keep
r, rend, dr, x, xa = 0.7, 1.0, 0.001, 0.5, [0.0]* nsteps
4 plt.figure()
while r <= rend:
6 for i in range(ntrans): x = 4*r*x*(1−x) # discard transients
for i in range(nsteps): xa[i ], x = x, 4*r*x*(1−x) # keep rest
8 plt.plot ([r]*nsteps, xa, 'b,’) # blue pixel markers
r = r + dr
10 plt.xlabel('r’), plt.ylabel('x’), plt.show()

For each r, we iterate the map for ntrans steps and discard
these points to get rid of transients. The number of steps it
takes for the transients to pass would depend on r. For
simplicity we use a fixed, large value, as it is adequate for our
purpose. The next nsteps iterates are presumably the
attractors. They are retained and plotted.

The result is a period doubling diagram in Figure 5.6. It

shows one fixed point belonging to period one up to r1 = 0.75,
where it becomes unstable, and is taken over by two stable
fixed points of period two, in accordance with earlier
observations (Figure 5.5). They remain stable to r2 ~ 0.862,
which is the onset of period four. Period doubling, or
bifurcation, continues at closer and closer values of r, e.g., r3 ~
0.886 for period eight, and r4 ~ 0.891 for period sixteen (barely
visible in Figure 5.6).

Further period doubling (16, 32, etc., not noticeable at the

scale shown in Figure 5.6) occurs so close to each other and so
fast that, within a narrow window around r ~ 0.9, the period
becomes infinite. Subsequently, no apparent structure remains
other than whole bands being filled. It shows that period
doubling is the route to complexity. Although our example is
the logistic map, it turns out to be universal of nonlinear
systems. As we will see shortly, the logistic map becomes
chaotic for r > 0.9.
Figure 5.6: Period doubling diagram and self similarity. Vertical lines
indicate the locations of period doubling. The boxed region in the
upper figure is magnified in the lower left figure, whose own box is
magnified in the lower right figure.
 Figure 5.7: 3D view of period doubling.

Figure 5.6 shows the static structure after initial transients

have been discarded. The dynamics of period doubling may be
more fully appreciated in a 3D representation which also
includes temporal changes. Figure 5.7 shows a snapshot from a
VPython animation (Exercise E5.5). With time (n) increasing
out of the page, the initial transients give way to steady “finger”
sheets of bifurcation. At a given r, the series of points bounce
alternately between the sheets but never successively between
adjacent fingers. For instance, if we label the four branches of
the period-4 cycle as 1 to 4 from top to bottom, the sequence is
1 → 4 → 2 → 3…, without direct jumps between adjacent pairs
(1,2) and (3,4). As we will see later (Figure 5.8), branch 3 is
closest to . If we start from there, we will arrive at the
adjacent finger (branch 4) two steps later, i.e., half a cycle. This
is true for higher cycles. This observation is important for our
discussion of universal features including the Feigenbaum
constants next.

5.1.5 UNIVERSAL FEATURES

It turns out that several features in the logistic map are
universal to a large family of nonlinear maps. One universal
constant known as the Feigenbaum δ-number [28] can be
estimated from Figure 5.6. Slightly beyond r4, the logistic map
has reached the limit of period doubling series, i.e., the order of
periodic doubling becomes infinite. Let us denote that limit as
r∞ 0.8924864 (see Project P5.2 and Ch. 18 of Ref. [4]). The rn
value for the onset of period 2n is related to r∞ and δ as

where c is some constant. Eliminating c gives

This says that the Feigenbaum number is the ratio of adjacent

distances in r between successive bifurcations. Using the four
approximate r values stated earlier, we find the ratio to be
roughly δ ~ 4.7. More accurate evaluation of δ requires more
precise determination of the r value. The exact Feigenbaum
number is δ = 4.6692016… (see Ref. [12], Project P5.2).

Self-similar diagrams at different scales can be seen in

Figure 5.6 in the magnification of selected regions. The boxed
region in the original, upper figure is magnified and shown in
the lower-left figure. The magnified region is very similar to the
original diagram. We can enlarged a subregion in the already-
magnified diagram, which again shows strong resemblance to
its predecessor. This phenomenon is known as self similarity,
i.e., the overall structure, including period doubling and onset
of chaos, is replicated on the finer and finer scale.

It seems clear from Figure 5.6 that the successive

bifurcations are copies spawned from prior ones, only shifted
in location and shrunk in scale. Intuitively, we expect to be able
to rescale, or renormalize, one bifurcation and reproduce it
universally. Indeed, self-similarity is a universal phenomenon
in nonlinear dynamics, and can be better understood with
renormalization theory discussed in Section S:5.A. For
example, the approximate rn values and the δ number can be
estimated from renormalization (Project S:P5.1). The beauty of
the logistic map is that it exhibits these universal features while
keeping things simple and accessible [19].

Feigenbaum's α number
Another result of renormalization is that we can estimate the
second Feigenbaum number, the α-number which measures
the scaling in the vertical direction in the period doubling
diagram (Figure 5.6).
 Figure 5.8: Vertical displacements and the Feigenbaum's α number.

The α-number is a constant defined as the limiting ratio

where dn is the vertical displacement from the fixed point x* =

1/2 to the nearest branch of the period-2n cycle in the
bifurcation diagram (see Figure 5.8). The negative sign in Eq.
(5.19) refers to the displacement dn flipping directions between
the cycles.

Let be the value of the control parameter of the period-2n

cycle where one of the fixed points is x* = 1/2. The map
function is an extremum at this point (see Figure 5.1 or 5.3),
and its derivative is zero. Therefore, this fixed point is
superstable, hence the r* designation. For example, the first
 two r values for the superstables are and
(Project P5.2). In general, for the period-2n cycle satisfies

It is understood that f(2n) depends implicitly on .

The vertical displacement dn starts from x* and ends at the

nearest branch point of the bifurcation tree, which occurs
exactly at the halfway point through the cycle per our
discussion of Figure 5.7. In other words, the displacement is
given by

Knowing dn, we can calculate α from Eq. (5.19). There is an

easier, approximate way, from renormalization where we
obtain α = −2.24, compared to the exact value ~ −2.50 in Eq.
(5.19) (see Project S:P5.2).

.2 Chaos
As we have seen above, orderly period doubling ceases to exist
for r ≥ r∞ and the bifurcation diagram looks rather “chaotic”.
How does the abrupt change happen? Can we quantify it?
Earlier we gave a general definition of chaos as the sensitive
dependence on initial conditions and loss of long term
predictability. With the logistic map detailed so far, we are in a
position to demonstrate this, and in the process to give an
operational definition of chaos.

5.2.1 SENSITIVITY TO INITIAL CONDITIONS

Suppose we start the logistic map with two initial values
differing by a small amount, and iterate the two series in
parallel. How do the series compare? Program 5.3 (Appendix
5.A) lets us explore this question.

The heart of the program iterates the two series separately

and simultaneously, and records their differences as follows.

x2 = x1 + 0.01 # initial difference

for i in range(n):
xn1.append(x1), xn2.append(x2), diff.append(abs(x1−x2))
x1, x2 = 4*r*x1*(1.0−x1), 4*r*x2*(1.0−x2) # parallel iterates

Results generated from the program with different control

parameters are shown in Figure 5.9 with a small difference in
the initial values. For the control parameter r = 0.8, the initial
difference is rapidly reduced. The two series overlap each
other, and are practically identical after a couple of steps,
yielding the period-2 cycle as first seen in Figure 5.2.
 Figure 5.9: Logistic map with slightly different initial conditions for
two control parameters. The initial values differ by 0.01.

The situation is quite different for the larger r = 0.9 case,

where the difference between them grows quickly. By around n
= 5, the two series are so far apart that they are no longer
recognizable as coming from two nearby initial values. In other
words, they become unpredictable. We note that the system is
still deterministic, but something happens at this r that makes
it very sensitive to the initial conditions.

We can get a better idea of what is happening here by

studying the difference between the series. It is shown in
Figure 5.10 for a small but arbitrary initial difference, produced
with the same program.

While there are some scatter in the data, the overall trend is
that the initial difference shrinks exponentially (note the
semilog scale) for r = 0.8, but grows exponentially for r = 0.9.
In the former case, the difference becomes insignificant, and
we can predict where the system will be even if the initial
conditions are not known precisely. However, in the latter case,
any difference in the initial conditions, no matter how small,
will lead to such exponentially diverging outcomes that any
long term prediction is meaningless. It is for this reason that
we saw the different behaviors in Figure 5.9. We stress that it is
not due to inaccuracies in numerics, it is a characteristic of
nonlinear systems in the chaotic regime.

Figure 5.10: The difference between series with slightly different

initial conditions for two control parameters. The initial values differ
by 0.001.

5.2.2 A MEASURE OF CHAOS

We are now ready to discuss a more precise definition and a
quantitative measure of deterministic chaos. Chaos is
characterized by extreme sensitivity to initial conditions.
Figure 5.10 suggests that the separation between nearby initial
points will shrink or grow exponentially. Let us denote two
series differing slightly in the initial value by ∈,
To first order, we expect that the separation will behave as

where c is some proportional constant. The factor λ is called the

Lyapunov exponent. Accordingly, systems can be classified as

The meaning of λ refers to the average rate of exponential

convergence or divergence of nearby trajectories. We can use λ
as a quantitative measure of chaoticity, and give an operational
definition of chaos: a system is chaotic if the Lyapunov
exponent is positive. The more positive the λ, the more chaotic,
or disorder. Conversely, the more negative the λ, the more
predictable the system.

Note that the Lyapunov exponent should not depend on the

initial separation in the limit ∈ → 0. In fact, given the values of
Δx1 and Δx2 at t2 and t2, we can eliminate c∈ and obtain λ from
Eq. (5.23),

Accurate determination of λ requires an infinitesimal initial

separation and a very long evolution time.
 In most cases, λ can be determined only numerically. For
example, using the data in Figure 5.10 and Eq. (5.25), we can
estimate that λ ~ (ln 10−13 − ln 10−3)/28 = −0.8 for r = 0.8, and
λ ~ +0.2 for r = 0.9. These estimates are accurate only to their
signs, and their magnitudes are crude because, numerically, we
start with a finite initial separation and end in finite evolution
time. In practice, the separation is limited by the range of the
system, |Δx| < 1 for the logistic map, and cannot grow
indefinitely in Eq. (5.23). As a result, it is important that we
measure the slope (such as in Figure 5.10) before the
separation levels off, and preferably average it over some initial
conditions.

5.2.3 LYAPUNOV EXPONENT OF THE

LOGISTIC MAP
It turns out that the Lyapunov exponent of the logistic map can
also be determined analytically. From Eqs. (5.22), (5.23), and
(5.11), we have the separation of two nearby points after n
iterations

Solving for λ yields

In the limit of small ∈ → 0 and large n 1, the argument of the
logarithm function becomes a derivative at x0, and the
second term is negligible,

Recalling the chain rule (5.15), we have

To use Eq. (5.29), we choose an initial x0, wait some steps

for transients to pass, generate gi recursively from Eq. (5.12)
and calculate f′(gi) with Eq. (5.15), for some suitably large n.
You may be worried that λ obtained this way may depend on
the initial value x0. The dependence is very weak, if we take
care to let the transients pass, and if n is sufficiently large.
Essentially, the summation in Eq. (5.29) will wash out any
trace of x0, in both regular and the chaotic regions.
 Figure 5.11: Lyapunov exponent of the logistic map. Vertical lines
indicate onsets of period doubling, at the same locations as those
marked in Figure 5.6. Period-3 occurs around r ~ 0.96 where λ dips
below zero.

Program 5.4 (Appendix 5.A) calculates and plots λ.

Experiment with ntrans (as well as x0) a little, as sometimes it
takes a while for the transients to pass. When λ is independent
of all the parameters for a given r (say to within 3 or 4 digits),
we know it has converged.

The results as a function of r are shown in Figure 5.11. For

the control parameter up to r ~ 0.892, the Lyapunov exponent
is negative, so the dynamics is regular. Bifurcations occur at the
cusps when λ comes up to zero (but not crossing zero) and then
abruptly falls back down. The locations of the cusps are the
same as the vertical lines in Figure 5.6. Afterwards, λ is largely
positive and increasing, signaling increasing disorder or
chaoticity. But embedded in the land of chaos are narrow
 windows of stable islands where λ dips down to negative
values. One of these contains the stable fixed points of period
three marked by the arrow. There are other regions embedded
as well, including period five, six, seven, etc. (See Project P5.1.)

.3 A nonlinear driven oscillator

The logistic map lets us explore the concept of chaos in a
simple model system. We now discuss a real, continuous
physical system, the simple pendulum, a classic apparatus
(Figure 5.12) made up of a weight connected to a pivot by a thin
rod. It oscillates periodically about the equilibrium, the vertical
line. This system exhibits chaos that can be experimentally
measured.

Figure 5.12: The nonlinear pendulum.

The equation of motion for a damped pendulum is

where ω0 is the natural angular frequency dependent on the
rod length and gravitational acceleration, b the damping
coefficient due to viscous friction or linear drag. The sin θ term
comes from the tangential gravitational force. Without
damping and in the small angle approximation, Eq. (5.30)
reduces to the simple harmonic oscillator (2.47).3 However, the
nonlinearity present in this term is crucial for nonlinear
dynamics.

With damping, the pendulum would stop moving unless an

external driving force is applied. We then speak of a nonlinear,
driven oscillator. Including the driving force, and converting to
a set of first-order ODEs, we have the equations of motion

where Fd is the driving force amplitude, and ωd the driving

angular frequency.

We will assume scaled units for Fd and b according to the

units of time or frequency. If time is in seconds, then Fd will be
in 1/s2, and b in 1/s. Their units will be omitted unless
otherwise noted.

In comparison to the logistic map, Fd plays the role of the

control parameter. We expect that for some Fd values, the
system should exhibit chaotic behavior, as does the logistic
map for certain values of r. To investigate the behavior of the
system, we solve Eq. (5.31) numerically with an ODE solver
next.

5.3.1 REGULAR AND CHAOTIC MOTION

Program 5.5, named “Nonlindro”, integrates the equations of
motion of the nonlinear driven oscillator. The system dynamics
(5.31) is calculated in the function pendulum(),

def pendulum(y, t): # y = [theta, omega], omega_0 = 1

return [y[1], −ma.sin(y[0]) − b*y[1] + fd*ma.cos(omega_d*t)]

The two-element input array y[] contains the angular

displacement and angular velocity as .

The results, θ(t) and ω(t), are shown in Figure 5.13 for two
driving amplitudes. For Fd = 0.7, the angular displacement and
angular velocity both undergo an initial transient period.
During this period, the oscillator loses track of where it came
from due to damping, and adjusts the motion to the driving
force. After the transients, it oscillates regularly and
periodically with the same frequency as the driving frequency.
This is akin to pushing a child on a swing at regular intervals.
No matter what the initial conditions were, the final
oscillations are the same. The system has no memory effect (we
will see this again in a linear driven oscillator, Section 6.1).
 Figure 5.13: The angular displacement (θ) and angular velocity (ω)
as a function of time for two driving force amplitudes, Fd = 0.7 and
−2 2 −1
1.1 s . The other parameters are ω0 = 1 rad/s , b = 0.5 s , and ωd =
0.6 rad/s.

At a larger Fd = 1.1, the motion looks rather irregular. There

is no clearly defined transient period, and the oscillator never
settles into a regular motion. The angle θ(t) shows wild swings
over the top, exceeding the regular range |θ| < π. The same is
true for the angular velocity, which shows no apparent
structure. We suspect that, whilst the motion is regular for Fd =
0.7, it is chaotic for the larger driving force.

To confirm this, we need to calculate the Lyapunov

exponent in each case. We can do this by modifying program
Nonlindro slightly to start two oscillators with a small
difference in the initial conditions, and see how the difference
develops in time. We leave this to Project P5.4 and only focus
on discussing the results.

Figure 5.14: The difference in ω from slightly different initial

conditions for two driving amplitudes. The parameters are the same
as in Figure 5.13. The initial conditions are θ1(0) = θ2(0) = 0.6, and
ω1(0) = 0 and ω2(0) = 0.01.

Figure 5.14 shows how the difference in angular velocity,

Δω = |ω1(t) − ω2(t)|, changes over time from two nearby initial
conditions. For the smaller perturbation of Fd = 0.7, the
difference becomes smaller with increasing time. There are
oscillations in the data due to the oscillating nature of the
system, just as there were scatter in the logistic map. But the
overall trend is clear: the rate of convergence is exponential.
The slope of the trend line, the Lyapunov exponent, is negative.
Thus the motion is regular, nonchaotic, for the smaller Fd.
 However, for the larger driving force Fd = 1.1, the difference
grows exponentially with time. Here, the data fluctuates more
wildly than before because of the stronger perturbation. The
difference also seems to saturate a little, reaching a plateau for
t ~ 70 to 100 s. This usually happens in any system that is
bound physically, like in the logistic map where |Δx| < 1. In this
case, the amount of energy in the system must be finite.
Therefore, the difference cannot grow indefinitely. No matter.
Here, it is the initial trend of the separation before saturation
occurs that is important in measuring the Lyapunov exponent,
or the chaoticity. The trend line drawn between 0 to 60 s
certainly gives a positive Lyapunov exponent.

After all, if the trajectories diverge exponentially, the system

is extremely sensitive to the initial condition, and after a short
time they will “forget” where they came from. This, however, is
different than the no-memory effect mentioned in the regular
motion above. There, every point converged to the same
motion. Here, no two points lead to the same motion. Put
another way, the current path could have come from any point
in the chaotic region. We cannot trace backward to reconstruct
the original path, even though the system is deterministic.

Quantitatively, estimating from both trend lines in Figure

5.14, we find the Lyapunov exponents to be λ ~ −0.25 s−1 for Fd
= 0.7 s−2, and λ ~ +0.1 s−1 for Fd = 1.1 s−2. This means that, over
one period of the driving force (2π/ωd), the separation between
two nearby trajectories is reduced by a factor ~ 0.07 in regular
motion, and increased by a factor ~ 3 in chaotic motion. Both
these numbers confirm our expectations from Figure 5.13.

In the chaotic regime, we can think of τ = 1/λ as the time

scale of trajectory divergence. Predictions of motion beyond τ
would be meaningless.

5.3.2 THE PHASE SPACE AND THE

POINCARÉ MAP
As we will see shortly, another useful way to study nonlinear
dynamics is to plot the phase space trajectories showing θ and
ω together. As mentioned earlier (Section 2.4), phase space is a
concept taken from Hamiltonian mechanics, a
multidimensional space encompassing all coordinates and
momenta of a dynamical system. In the case of the driven
oscillator, it refers to the θ and ω space. Sample trajectories
calculated with a slightly modified Nonlindro are shown in
Figure 5.15.

At Fd = 0.7 the path starts from the initial point and quickly
converges to an ellipse in which the motion repeats
indefinitely. This is not surprising since we know from earlier
discussions that the motion is regular at this driving amplitude.
This ellipse is an attractor, though it is not a single point for a
continuous system. On the other hand, at Fd = 1.1, the path
looks very complex. Here the numerical values of θ have no
limits (see Figure 5.13), although as an observable it should be
within [−π, π]. We thus remap θ whenever it exceeds this range
as

Figure 5.15: Phase space plots for different driving force amplitudes
Fd = 0.7, 1.1, and 1.2. Other parameters are the same as in Figure
5.13.

The path wanders all over the diagram like spaghetti and
there is no stable attractor for it to evolve around. If we
increase Fd to 1.2, the system returns to a simple pattern again,
though it is not obvious from the plot because of the
continuous nature of the variables.
 To make more sense out of continuous changes, a better
idea is not to plot every point. Rather, we look at them at
discrete times. The result is a distribution of points taken at
specific time intervals, also known as a Poincaré map. To
produce a Poincaré map, we plot only the points in
synchronization with the driving force, e.g., when it is
maximum or minimum, after the initial transients pass. The
technique is used in Program 5.6. The results are shown in
Figure 5.16.

For regular motion (Fd = 0.7), the plot is simply

remarkable: only two points, corresponding to two samplings
in each period of the driving force, one at the beginning of the
period, and one at the halfway point. Incidentally, at Fd = 1.2
we again see two points, even though it looks more complex in
Figure 5.15. We can guess that the motion must be regular too.
In these Poincaré maps, regular motion repeats periodically as
discrete points. It is similar to stroboscopic imaging. For
example, a spinning disc may rotate too fast to see the marking
on it. But, if we were to shine light on it at periodic intervals,
then instead of seeing a blur, some patterns could be
recognized. Poincaré maps are indispensable for understanding
complex dynamics, especially in higher dimensions.
 Figure 5.16: A Poincaré map of the nonlinear driven oscillator. The
plots are generated with the same parameters as in Figure 5.15.

For Fd = 1.1, there are many more points in the Poincaré

map, but they trace out relatively simple structures. The points
do not fill up the graph, rather they follow a disjointed line, like
a shoreline. When a point is recorded, we know it will be on the
shoreline, though not exactly where. This shows that even
chaotic motion obeys certain rules. We can predict the
unpredictable, so to speak.
 Figure 5.17: The Poincaré surface of section of the nonlinear
oscillator.

The appearance of shoreline structures is a result of the

trajectory meandering through available phase space. It is a
signature of chaos. To understand how it comes about, it is
helpful to think of the Poincaré map as a cut of a topological
surface, like a twisted donut, in a multidimensional space. We
can imagine the phase space trajectory moving along and
wrapping around the surface. When a cut is made, the cross
section of the surface, the shoreline structure, is exposed, and
that is what we see. It is called the Poincaré surface of section.

Figure 5.17 shows the Poincaré surface of section of the

nonlinear driven oscillator for Fd = 1.1. It represents several
cuts, or layers of the Poincaré map. The first layer (bottom) is
recorded at the start of the period, the second layer one step
after the start, the third one two steps after, and so on. The
bottom layer is the same as the Poincaré map shown Figure
5.16 (half of it to be precise, since only one point per period is
taken).

The shape of this surface of section is relatively simple. In

other chaotic systems it can be quite complex. Generally, the
surface over which the phase space trajectory travels is referred
to as a torus. The Poincaré maps represent cross sections of
these tori. In more than three dimensions of phase space, the
Poincaré surface of section is an abstract multidimensional
surface. For regular motion, the surface of section reduces to a
curve.

5.3.3 BIRFURCATION DIAGRAM

Figure 5.16 tells us that the motion is regular at Fd = 0.7,
chaotic at 1.1, and regular again at 1.2. The fact that there are
regions of regular motion below and above this particular value
of Fd = 1.1 should not surprise us. We have seen similar islands
of regular motion sandwiched between chaotic regions in the
logistic map (see Figure 5.6).

As in the logistic map, a bifurcation diagram can give us an

overview of regions of regular and chaotic motion. We use the
same discrete sampling technique to calculate the bifurcation
diagram for the driven oscillator. For each Fd, we wait for the
transients to pass, and record data points in synchronization
with the driving force. Program 5.6 can be used for this
purpose, only the graphing part needs to be modified.
 The results are shown in Figure 5.18. We see period
doubling structures similar to Figure 5.6. There are period-2
cycles in the beginning, which bifurcate into period-4, period-
8, etc., and quickly becomes period infinite, i.e., chaotic. The
chaotic region is followed by windows of regular motion again
(Fd ~1.05 and 1.2). This all looks very much like what we have
in the logistic map. We again see that the route to complexity is
through period doubling.

The enlarged figure is so strikingly similar to the logistic

map (see Figure 5.6) that we could have mistaken for it without
looking carefully at the graph labels. Two totally different
systems, yet quite similar structures. It shows that
deterministic chaos is not randomness, has universal
properties, and plays by a set of predictable rules.

.4 The Lorenz flow

Simple as they appear, the two examples we have discussed so
far would be far less understood or illuminating as to their
chaotic nature without digital computation. Since Lorenz's
work, computational studies have firmly established chaos as a
new field of science.

In the seminal paper published in 1963 on hydrodynamic

flow [59], Lorenz wrote: “… it is found that nonperiodic
solutions are ordinarily unstable with respect to small
modifications, so that slightly differing initial states can evolve
into considerably different states”. The term chaos was not a
common scientific vocabulary then and the science of chaos
was yet to be advanced. Today, we recognize, unmistakenly,
that Lorenz was speaking about chaos. He is widely credited
with the discovery of chaos.
 Figure 5.18: Period doubling diagram of the nonlinear driven
oscillator with the same parameters as in Figure 5.13. The bottom
figure is an enlargement of the top figure in the window 1.25 ≤ Fd ≤
1.3 and −0.5 ≤ ω ≤ 1.

Lorenz was studying convection problems of hydrodynamic

flow in climatic and atmospheric physics. Given the very
limited computing power at that time by today's standard,4
Lorenz modeled the process using a much simplified version of
the Navier-Stokes equations, which are essentially Newton's
laws expressed in continuous variables like density,
temperature, pressure, etc., to describe fluid dynamics. The
result is the Lorenz model, which consists of three ODEs as

Equations (5.33a) to (5.33c) approximately describe the

convective flow of fluid between two plates kept at a
temperature difference. The variable x represents the flow
intensity, y the temperature difference between the rising and
falling currents, and z the temperature gradient. The
parameters σ, r and b are positive constants.5 Given the
idealization of the Lorenz model, we do not expect it to
accurately describe realistic flow, which surely requires a great
deal more variables and parameters than present in the model.
However, the model does capture one of the most important
features in the actual system: the instability and sensitivity to
initial conditions. For this reason, we will treat the model as a
nonlinear physical system, and not expect it to predict the exact
behavior of actual systems.

Though simplified, the Lorenz model exhibits complex

behaviors typical of nonlinear systems. Note the nonlinear
terms −xz in Eq. (5.33b) and xy in Eq. (5.33c). In σ, r and b, we
have three control parameters. Relative to the previous
examples, this is a rather large parameter space to explore. We
will follow Lorenz and use fixed values σ = 10, b = 8/3, and
vary r as our sole control parameter. We will see that as r is
varied, the motion will transition between regular and chaotic
behaviors.

5.4.1 SOLUTIONS OF THE LORENZ MODEL

To follow the dynamics, we can use the RK4 method to
integrate the Lorenz equations as we did for the nonlinear
oscillator, because the model is dissipative (Eq. (5.33c), for
example). We can follow any of the variables, as they behave
similarly for a given control parameter. We choose to show the
results of y for three values of r in Figure 5.19 (see Project
P5.5).
 Figure 5.19: The variable y of the Lorenz model as a function of time
for three values of r. The initial condition is (0, 1, 0).

For the two smaller r values, y increases rapidly in the

beginning, reaches a peak value, then drops sharply. Because it
represents the temperature difference of rising and falling
currents, convective flow is too far out of equilibrium when the
difference becomes large, causing subsequent instabilities in
the form of rapid rises and falls. After the initial phase, y
oscillates around an equilibrium point with reduced amplitude
(fixed point), and eventually settles to that point like a damped
oscillator.

For r = 28, however, the behavior of y after the initial phase

is very different. It oscillates with increasing amplitude,
reaching a peak at t ~ 17. Thereafter, it falls and rises sharply,
sometimes having two or three peaks of the same sign before
flipping to the other side. It never reaches a steady state. The
motion becomes irregular and chaotic.

At this stage, the system becomes very sensitive to initial

conditions or small perturbations. In fact it is so sensitive that
one must take care to make sure the results are converged. For
the r = 28 results shown in Figure 5.19, calculations with the
RK4 method converge only for step size of ~ 0.005 or smaller.

Incidentally, in his original calculation, Lorenz used a quasi

second-order method with a step size of 0.01. Compared to our
converged results, Lorenz's calculation was accurate only up to
about t ~ 17. There is considerable disagreement between the
two results after that point. However, this does not alter the
conclusions Lorenz drew: long term predictions in this system
are not feasible. The key factor is that when the system is
chaotic, a small numerical error will be amplified exponentially
just like a small separation between two trajectories. In this
sense, numerical accuracy is less critical in assessing long term
trends, as long as the results are statistically representative of
the chaotic nature of the problem (numerical accuracy still
needs to be treated with care, of course). Weather systems,
though much more complicated than the idealized Lorenz
model, behave similarly in the essential point, that is: the
fundamentally chaotic nature of weather prevents reliable long
term forecasts.6
5.4.2 THE ANATOMY OF A STRANGE
ATTRACTOR
The wild oscillations observed above seem to be random by the
variable y alone. But, when viewed in the three-dimensional
phase space, the trajectories are in fact meandering between
two regions known as a strange attractor, tracing out a
butterfly-like figure, illustrated in Figure 5.20.
 Figure 5.20: Strange attractor of the Lorenz model. The plot with
fewer windings starts at (*) and ends at ( ). The initial condition is
(−7, −2, 30) and the parameters are σ = 10, β = 8/3, and r = 28.

Two “eyes” of the attractor (or steady state solutions,

Exercise E5.6) are located roughly at the centers of the
butterfly's wings, on the opposite side of the x-z plane. From
the starting point, the trajectory spirals outward from one eye
for a few rounds, then crosses the x-z plane to the
neighborhood of the other eye, where it swirls around about
two times before crossing the x-z plane again toward the first
eye. Thereafter, the cycle repeats. Because the attractor is in the
chaotic regime, they show sensitive dependence on initial
conditions. This behavior seems abnormal, and is one reason
the attractor has been called a strange attractor.

Figure 5.21: The butterfly trajectory of the Lorenz model, with the
same initial condition and parameters as in Figure 5.20. The camera
 points to the negative z-axis (into the page), with the x-axis to the
right and y-axis up.

After many crossings, a beautiful figure emerges like a

butterfly. We can better visualize the structure of the butterfly
with VPython. Figure 5.21 shows one 3D view where we are
peering down in the negative z direction, with the x-axis to the
right and y-axis up. We can clearly see the spirals about the two
eyes and the crossings through the x-z and y-z planes. The
structure formed by the points is approximately the shape of
the Poincaré surface of section.

Note, however, the trajectory never crosses itself. This is a

general property of phase space for any system. Also note from
Figure 5.21 that there is a “thickness” to the spirals, i.e.,
adjacent spirals are out of plane with each other. So the space
traversed by the trajectory has a “volume” structure. However,
as we will discuss shortly, it has a fractal dimension greater
than 2 but less than 3, meaning it is a volume-like object with
zero volume. This is the other technical reason the attractor is
labeled strange: it has a fractal dimension (Section 5.6).

Because the weather system is nonlinear, and chaotic under

certain conditions, a small difference in its initial conditions
may lead to totally different outcomes. This is sometimes
referred to as the butterfly effect. The flapping of a butterfly's
wings somewhere in the world may cause either nothing, or a
storm somewhere else. The unpredictability is not just due to
the limitation of computer modeling. More powerful computers
and more refined initial conditions can improve weather
forecasts, but will not alter the inherit unpredictability and
limited long-range forecasts of weather systems.

As we learned from the nonlinear driven oscillator, even

chaotic motion is predictable in the Poincaré map (see Figure
5.16). How do we make such a map for the Lorenz model which
does not have a periodic driving force? We do this by making a
cut along a given plane in phase space, say y-z plane, such that
every time the trajectory crosses this plane, we plot a point at
(y, z). When plane-crossing happens, x = 0. This means that
when we integrate the equations of motion (5.33a) to (5.33c),
we keep track of the value of x, and if it changes signs, we plot
the pair of values (y, z).

A Poincaré map for the strange attractor is shown in Figure

5.22. As the trajectory moves between the wings of the strange
attractor, it crosses the y-z plane numerous times (Figure 5.22,
top). We mark the position on the plane at each crossing. In the
end, the marked points form a “V” shaped distribution (Figure
5.22, bottom).

Suppose the trajectory is spiraling around the eye on the

negative side of the x-axis at a given moment. By following the
trajectory, we observe that it will move to the positive side of
the x-axis by crossing on the right branch of the “V”, and
returning to the other side on the left branch. The order is
never violated. Over a long time, there are equal number of
crossings on each branch. However, there is no telling how
many spirals around an eye it will execute before crossing, or
where exactly on the “V” it will cross (other than the branch).

A Poincaré map is just one slice of the Poincaré surface of

section (Figure 5.17). The one shown in Figure 5.22 is a slice of
the Poincaré surface of section for the Lorenz model at x = 0.
We can build up a complete surface of section by making a
series of Poincaré maps on planes parallel to the y-z plane at
regular intervals of x. The finished surface of section should
resemble Figure 5.21 (see Project P5.5).
 Figure 5.22: The making of (top) and the finished Poincaré map of
the Lorenz model. The parameters are the same as in Figure 5.21.

.5 Power spectrum and Fourier

transform
We have discussed ways to characterize chaotic dynamics in
several nonlinear systems. They include the Lyapunov
exponent, the Poincaré map, and the nonperiodicity of the
trajectories. There is another powerful method, the power
spectrum.

The power spectrum method can reveal frequencies in the

system. For regular, periodic motion, we expect that the
spectrum will be dominated by discrete frequencies. If the
system becomes chaotic, what do we expect to see in the
spectrum?

Figure 5.23: The power spectra of the logistic map at different values
of r.
 Power spectra can be obtained via the Fourier transform
defined by

The inverse Fourier transform can be written as

The functions g(ω) and f(t) are uniquely related to each

other by Eqs. (5.34) and (5.35). Furthermore, they satisfy the
following integral relation,7

If the function f(t) represents the amplitude of a wave, we

expect the energy carried by the wave to be proportional to the
square of the amplitude, |f(t)|2. We can therefore interpret
|g(ω)|2 (or simply |g(ω)|) as the energy density in frequency
domain, namely, the power spectrum.

As an example, we show the power spectra of the logistic

map at several control parameters in Figure 5.23. They are
obtained by Fourier transform explained below. At the two
smaller values of r, we see discrete peaks. The number of peaks
corresponds to period-4 and period-16 cycles, respectively. This
agrees with the period of stable fixed points in regular motion.
By far the largest contribution comes from ω = 0 component. It
is a result of non-sinusoidal data.

Increasing r by just a small amount r = 0.892 → 0.893, we

see the spectra change qualitatively. There is a “noisy”
background beneath the peaks. The logistic map is weakly
chaotic since it is at the boundary between regular and chaotic
regions (Figure 5.6). Some quasi-periodic trajectories still
persist. In the case of fully chaotic motion (r = 0.99), the
spectrum is nearly continuous, except the ω = 0 peak which is
always present. There are no periodic trajectories, so the power
spectrum resembles the spectrum of white noise. In general,
when a system becomes chaotic, the power spectrum will be
dominated by white noise whose frequencies extend the full
range.

THE FFT METHOD

The spectra are generated by fast Fourier transform (FFT) of
the map iterates. The data is the same as that used in making
Figure 5.6. For numerical work, a discrete version of the
continuous Fourier transform (5.34) is required. The discrete
Fourier transform (DFT) takes the form

where f(tk) are the N sampled data points at equidistant

intervals tk over a period of T. The discrete angular frequencies
are

The direct approach to DFT as given by Eq. (5.37) is

straightforward to implement. For each component gm, the
summation over k requires N operations (multiplications plus
exponential function evaluations). To compute all N
components, N2 operations are needed. For large N, it is
increasingly inefficient and slow.

The FFT algorithm can compute the Fourier transform in N

ln N operations [18]. With the FFT method, one can achieve
large gains in speed. It is one of the few truly significant
algorithms discovered in digital computing that has had a
tremendous impact on computational physics and nearly every
field of scientific computing. Though FFT is widely available in
nearly every programming language, it is well worth the effort
to understand how it works to not only appreciate its power
and elegance, but also to have a portable and flexible FFT
function, and to be able to modify it if necessary.

The FFT method is described separately in Section S:5.B,

and the Python codes shown in Program S:5.1. From now on we
will include these functions in our FFT library fft.py, and
import fft before use.

Figure 5.23 is produced with the FFT library. Because the

Fourier transforms are generally complex, the power spectrum
 plots the absolute magnitude of the transformed data. The use
of FFT is not limited to calculating the power spectrum. It can
be used to find the onset of period doubling, for example.
Exploration of power spectra of other systems are left to
projects.

.6 Fractals
Earlier we mentioned that the strange attractor of the Lorenz
model apparently has a fractional dimension, i.e., non-integer
dimensions like 1- or 2-dimensional space that we are used to.
For example, the attractor has a dimension ~ 2.05, so it is
strange. How does this happen? A rough explanation is that the
region around the attractor is chaotic in some directions but
regular in others, i.e., some Lyapunov exponents (λ) are
positive and some negative. So the area is stretched (positive λ)
in some directions but compressed in others (negative λ),
making the area irregular and non-measurable in normal
integer dimensions.

To understand fractional dimensions, suppose we wish to

measure the length of a line, L. Typically, we would choose a
measuring unit (scale), say l, and count how many units are
needed to cover the line, say N(l), then we would have the
answer, L = N(l) × l. So normally, we expect N(l) ∝ 1/l1. We say
the dimension of a line is 1. For an area, we could use a unit
square of area σ = l2 as the basic measuring unit, and would
expect the number of unit squares to cover an area to vary as
N(σ) ∝ 1/σ = 1/l2. This is then a 2-dimensional space (surface).

In general we define the dimension of an object as

where l is the scale, and N(l) the number of basic units required
to cover the object. The dimension is df, also known as the
Hausdorff dimension. For regular lines, df = 1 and for surfaces,
df = 2 as discussed above.

Note that Eq. (5.39) is another case of scaling laws, which

we have seen to be very useful in prior cases (see Eq. (3.33) and
Figure 4.8).

To obtain df, we just need to measure an object with two

different scales l and l′, count the numbers of units needed in
each case, N(l) and N(l′), and determine df as

Let us see what happens if we apply Eq. (5.40) to several

self-similar objects. First, let us start with a line of unit length,
cut out the middle third, and form a triangle, as shown in
Figure 5.24. As we repeat the process indefinitely, a self-similar
curve is formed. It is called a Koch curve. To determine the
dimension of the curve, we count the number of units needed
to cover the curve at various unit sizes, and we get Table 5.1.
 According to Eq. (5.40), the dimension of this Koch curve is

Table 5.1: The number of measuring units as a function of unit size

for the Koch curve.

Figure 5.24: The Koch curve. The last level is enlarged by a factor of
two.

So it is not an integer dimension, but a fractional one. It is

between 1 and 2, between a regular line and a regular area. We
can think of it as an object of infinite length but zero area.
Indeed, the length of the Koch curve does go to infinity as we
add more and more triangles. We see that the process produces
a self-similar object which has a fractional dimension, or a
fractal. We could construct other types of fractals such as the
Sierpinski carpet, which consists of taking out the middle third
of a square repeatedly, or the Cantor set, etc., all having
fractional dimensions (Exercise E5.7 and Exercise E5.8).

Fractal dimensions of attractors

How do we define the fractal dimension of a strange attractor
like that shown in Figure 5.21, which does not have a well
defined shape?

It turns out a rather elegant yet simple measure is more

suitable for chaotic trajectories. Consider the trajectory
spiraling around the strange attractor in the Lorenz model
(Figure 5.21). Let us pick a point P on the trajectory and place a
sphere of radius R at that point, like one of those depicted in
the figure. Because the trajectory is chaotic, it will travel over
all the allowed volume in phase space. If we wait long enough,
the trajectory is bound to return to the neighborhood of the
sphere.

We can count the number of times the trajectory enters the

sphere over a given amount of time. Let us denote it by NP (R).
We expect that NP (R) will increase with increasing R. Let us
assume a power-law relation,
where C is a proportional constant, and dc is called the
correlation dimension [86].

Roughly, correlation dimension measures the frequency (or

density) of the same trajectory visiting the neighborhood
around point P. To find dc, vary the radius and record NP (R)
for different values of R. We can then plot NP (R) as a function
of R on a log-log scale. Experiments show that over an
intermediate range of R, the slope is a straight line. This slope
is dc, the correlation dimension.

The value of dc thus obtained will have a weak dependence

on the location of the point P. To get a more accurate value, the
above procedure is repeated at different points, and we take the
average over them. By this method, we can obtain the
correlation dimension of the strange attractor, ~ 2.05, see
Project P5.6. It is thus a fractal with infinite area but zero
volume, like an infinitely crumbled piece of (elastic) paper.

Mandelbrot fractals
There are many kinds of fractals. A common feature among
them is self-similarity. We have seen self-similarity in chaotic
systems as well. Perhaps the most well-known ones are the
Mandelbrot fractals, shown in Figure 5.25. The Mandelbrot
fractals are generated by the equation which defines the
Mandelbrot set,
 Figure 5.25: Successive levels (column major) of the Mandelbrot
fractals.

Here c is a complex number. The number c is in the

Mandelbrot set if for large iterations zn remains bounded.

To produce the Mandelbrot fractals, we pick a pair of

numbers (x, y) and set c = x+iy. We then iterate Eq. (5.42) and
see if |zn| remains finite. It can be shown that if |zn| > 2, then it
will grow with each iteration and tend to infinity, so this c is not
in the set. We record the number of steps it takes until |zn|
reaches the value 2, and color-code that point accordingly
(Program 5.7).

Figure 5.25 shows successive magnifications of the

Mandelbrot fractals starting with the top figure. The smaller
figures are displayed in column-major order. The white box in
each picture is blown up in the next level. We see at each scale,
self-similar structures repeat themselves. By the last level
shown, we have reached the limit of double precision.
Renormalization would be required to go deeper. The level of
complexity is infinitely deep.

Modify Program 5.7 to explore different parts of the

domain. It is not possible to see the complete structure of the
set. No one has, or ever will. Have fun making fractals, your
own, beautiful, unique fractals. Who knew doing science would
be so much fun?
Chapter summary

Through the logistic map, this chapter opened our discussion

on nonlinear dynamics and a window to the rich and
bewildering world of chaotic dynamics in deterministic
systems. We carefully examined the logistic map with a mixture
of numerical and analytical results, which enabled us to
introduce several important concepts in the description of
chaotic dynamics. We described periodic orbits revolving
around stable fixed points. The fixed points become unstable as
the control parameter increases, destroying the old periodic
orbits but creating new ones of higher order in a series of
period doubling. This process continues until at a critical point
when all periodic orbits have been destroyed. The system
becomes chaotic.

We discussed ways to characterize and quantify chaos,

including the Lyapunov exponent, phase space trajectories, the
Poincaré map and surface of section, and the power spectrum
with the introduction of FFT. All these concepts are universal
and demonstrated in nonlinear systems we presented, such as
the driven pendulum and the Lorenz model of flow. Through
the study of these exemplar systems, we see that chaotic
motion is unpredictable in the long term, but it is not
randomness. The evolution toward chaotic motion mostly
follows the same pattern, the destruction of periodic orbits, the
birth of self-similar but more complex periodic orbits, and
 eventually, sometimes abruptly, the onset of chaos. Besides
chaos characterization, there are ways for limited chaos control
(e.g., see [91]) since even chaotic systems follow deterministic
laws.

Self-similarity at infinitely sub-divisible levels leads to

fractals, including the strange attractor of the Lorenz model.
We used a variety of visualization and numerical techniques to
aid our discovery of the interesting world of nonlinear
dynamics and chaos, including unique perspectives in
visualizing the Poincaré maps in 3D and the structure of the
butterfly attractor.

.7 Exercises and Projects

EXERCISES
E5.1 (a) Expand Eq. (5.9) for the fixed points of period 2 and
show explicitly that the quartic equation is

(b) Solve the above equation by using explicit formulas

for the roots of quartic equations, or, via a more
insightful approach, by successively reducing it to a
cubic and then a quadratic equation by re-using the
 period-one fixed points (5.8). Do it by hand, or better yet,
let SymPy work for us (for free),

In [1] : from sympy import *

In [2] : x, r = symbols( 'x r’)
In [3] : f = 4*r*x*(1−x)
In [4] : f2= 4*r*f*(1−f)
In [5] : factor (f2−x) # factor expression

After factoring out these terms, the resulting equation

should look like

where x1 = 0 and x2 = 1 − 1/4r are the fixed points of

period one (5.8), b = −1 − 1/4r, and c = (1+1/4r)/4r.
Solving the remaining quadratic equation yields the
results, Eq. (5.10).

E5.2 (a) Write a program that generates the cobweb diagram

showing the convergence toward attractors as illustrated
in Figure 5.3. The procedure is as follows: plot the map
function for a given cycle, e.g., y = f(x) for period one,
and f(2)(x) for period two; place the initial point on the
projection (diagonal) line, y = x; draw a vertical line to
find its next value on the y(x) curve; draw a horizontal
line to the projection line y = x for the seed value for the
next cycle; and repeat the last two steps.
 (b) Apply your program to other cases such as f(3)(x) and
f(4)(x). Optionally, find the fixed points with SymPy, and
compare with the graphical results.

E5.3 Prove the relation (5.13) mathematically. Start with

Assume x is in the small neighborhood of x*, i.e., x = x*

+ ∈, and expand gn(x) in powers of ∈.

E5.4 Show that the last two fixed points of period-2 (5.10) are

stable for .

E5.5 Create a program to visualize the dynamics of period

doubling using VPython. It should produce output like
the sample Figure 5.7. Generate the time series for a
given r, and vary it from ~ 0.7 to 1. Describe how the
points move between the branches. Annotate the path for
a representative value of r if possible.
E5.6 Find the steady state solutions of the Lorenz model
(5.33a) to (5.33c), i.e., solutions when dx/dt = dy/dt =
dz/dt = 0. Analyze their stabilities.
E5.7 A Cantor set can be formed this way (Figure 5.26): start
with a line, cut out the middle third, remove the middle
thirds of the remaining two segments, and repeat this
process indefinitely. Calculate the fractal dimension of
the set. Discuss its meaning.
 Figure 5.26: The first three levels of the Cantor set.

E5.8 Figure 5.27 shows the Sierpinski carpet. Describe how it

is made, and calculate its dimension. Discuss your result.

Figure 5.27: The first three iterations of the Sierpinski

carpet.

E5.9 Generate a Julia set by solving f(z) = z3 + 1 with

Newton's method as follows. Pick a point (x, y) in the
complex plane as our initial guess and set z1 = x + iy.
Iterate Newton's method (3.18) explicitly as
 The iterations will converge to one of three roots
(attractors), namely [−1, exp(±iπ/3)]. Check which one
after each step within some tolerance. Record the number
of iterations, and put a point at (x, y) using a color
scheme. For instance, color the point in hues of red,
green, or blue, depending on which root it converged to.
Explore the basins of attraction and their boundaries.
Discuss your observations regarding self-similarity and
connectedness of the points.

PROJECTS
P5.1 We have seen from Figure 5.6 that there is a period-3
stable window embedded in the chaotic region of the
logistic map around ~ 0.96. Why does it not appear at
smaller values of r? Explore this and related questions
of the period-3 orbits.

(a) Plot the map function f3(x) at r = 0.959, and the

straight line y = x. Find the intersection of x = f3(x).
How many fixed points do you see? Which ones are
stable?
 (b) Repeat the above for r = 0.95. Are there any stable
fixed points?

(c) Decrease r from 0.959. At each r generate a power

spectrum of the map iterates after discarding transients.
What is the lower limit of r when no period-3 cycles

exist? Compare your results with .

(d)* Investigate the other visible stable windows around

r ~ 0.935, 0.926, and 0.907, corresponding to period-5,
7, and 6, respectively.

P5.2* We take up the task of calculating the r values for the

superstable fixed points in the logistic map. Direct
calculations are useful in understanding the universal
properties, including the Feigenbaum constants and the
rate of convergence.

(a) Show analytically that the first two values for

period-1 and period-2 cycles are and

, respectively.

(b) Write a program to calculate the higher series by

solving Eq. (5.20) with Newton's root finder. The

following suggested strategy should be useful. For a
given n ≥ 2:
 Define , with

iterating according to Eq. (5.5).

Define h(r) = dg(r)/dr, and evaluate the

derivative in a way analogous to the chain rule
(5.15), but with respect to r (not x). Again,
iterate (recursion-like) the intermediate gi ≡ xi
as above.

Set the initial guess as

where δn−1 is the latest δ-number computed

according to (5.18).

Call Newton's root finder with g, h, and the

initial r. Cross fingers and record the returned

value. That is .

Repeat above steps for the next n, etc. See how

far you can push to higher n, or how close to r∞.

(c) Compute the α numbers at each n. Make a table

listing both the δ and the α numbers. Plot the results. Is
the convergence uniform? Where does it break down?
 Why? Discuss these and other observations you have
made.

P5.3 Investigate the properties of the sine map,

(a) Plot the map function and compare it to the logistic

map.

(b) Find the fixed points of period-1 and period-2

cycles. Do this with a root finder. Discuss their stability.

(c) Generate a bifurcation diagram. Note where the

bifurcations occur. Locate the r values for the onset of
period-2n cycles for n = 1 to 5. Do the first two values
agree with your analysis above? Discuss and compare
your results with the logistic map.

(d) Use the FFT technique to find the higher rn values.

What is the onset of chaos? Compute the Feigenbaum δ
number with these values.

(e) Calculate the Lyapunov exponent of the map.

Discuss similarities and differences with the logistic
map.

(f) Pick two representative r values corresponding to a

positive and a negative Lyapunov exponent,
 respectively. Iterate the sine map with a small difference
in the initial conditions, and obtain the Lyapunov
exponent graphically. Compare your direct, graphical
results with the numerical results above.

P5.4 We will carry out further investigation of the nonlinear

driven pendulum in this project. It is assumed that you
will program each part as necessary.

(a) The Lyapunov exponents shown in Figure 5.14 are

from only one initial condition. Compute these values
more accurately by averaging results from several
different initial conditions. Space them out over the
ranges of θ and ω. How do the results change? How
sensitive are they?

(b) For chaotic motion at Fd = 1.1, generate Poincaré

maps from a number of different initial conditions, say
5 or more. Graph the data on the same plot. Are the
shorelines different from Figure 5.16? Do they fill up
the phase space? Explain.

(c) Produce the bifurcation diagram in Figure 5.18.

(d) Calculate the power spectra at Fd = 0.7 and Fd = 1.1.

Use at least 1024 points. What is the difference between
the two? Pick a few interesting regions from the
bifurcation diagram, calculate and compare the spectra.
 (e)* Visualize chaotic motion of two pendula through
animation. Write a program to interactively display their
positions. Introduce a small difference in their initial
conditions, and see them either “attract” each other in
regular motion or diverge from each other in chaotic
motion. Add key detection to change Fd interactively
(e.g., “up” to increase and “down” to decrease).

P5.5 Write a program to explore the Lorenz model and the

strange attractor.

(a) Compare your results with Figure 5.19 for y. What is

the characteristic time scale of the problem? What is
your step size? How do you check convergence?

(b) Once your results are converged, calculate and plot

the results for x and z for the same r values. Also
compute the power spectra and analyze them.

(c)* Make a butterfly diagram and a Poincaré map

(Figure 5.22). Construct the Poincaré surface of section
by producing a series of Poincaré maps at several
surfaces of constant x. Stack them next to each other
using ‘3d’ projection in Matplotlib as

ax = plt.subplot(1, 1, 1, projection= ’3d’)

......
ax.plot ([x]*len(y), y, z, 'o’)
 where y and z are the points for the Poincaré map at x.

(d)* Repeat parts (a) and (b) using RK45n (or if you
used it already, switch to RK4n). Compare their
performance, and the values of step size when the
results are first converged.

P5.6 We quoted 2.05 as the fractal dimension of the strange

attractor in the Lorenz model. Let us confirm it by
computing the correlation dimension from Eq. (5.41) in
this project. We will first try a simple approach to make
things work. Once it is working, we can try a more
complex method.

(a) Start a trajectory in the vicinity of the strange

attractor, say at (−7, −2, 30). You can use the program
developed in Project P5.5 if applicable. Integrate the
Lorenz equations with r = 28. Wait for transients to
pass, 1,000 steps would suffice if the step size is 0.01.

(b) Take the first point after the transients as our center
point P, with position (X, Y, Z). Let us imagine we make
three spheres of different radii Ri (i = 1, 2, and 3)
centered at P. For example, radii of 1, 2, and 3 work
well. Initialize three counters that will keep track of the
numbers the trajectory has entered the spheres.

(c) After every step, check if the current position of the

trajectory is inside any of the three spheres. A point is
inside if
If so, increment the corresponding counter. The total
number of integration steps should be such that the
counter for the smallest sphere should be 100 or so.
Roughly 20,000 to 30,000 steps are enough.

(d) When the integration stops, plot the three counter

values versus the radii on a log-log scale. Calculate the
three slopes between the three points (1 to 2, 2 to 3, and
1 to 3). Take the average, and this is an approximate
correlation dimension at P. It should be 5% to 10%
within the value 2.

(e) Once the above steps are working, try the same
procedure for different center points. Average all the
values, and it should be approaching 2.05.

(f)* Now let us try a dozen or so different-sized spheres.

The radii should differ by some constant factor so that
when plotted on the log-log scale, they spread out
evenly. The factor should be such that the largest radius
should be no more than half the size of the attractor.
Note that if the trajectory is inside a sphere of given
size, then it is also inside the spheres of larger sizes
(outer spheres). Discuss the shape of your curve. What
happens at the two extremes of the radii? Compare your
results with the three-sphere results.
 (g)* Repeat part (f), but instead of spheres, use boxes of
different sizes. Are the results comparable? Comment
on your findings.

P5.7 The double-well oscillator is a particle moving in a

potential with two minima on either side of a local

maxima, . The equation of motion is

where b, Fd, and ωd have the same meaning as those in

the driven nonlinear pendulum (5.31). It is a useful
model for two-state systems.

(a) Sketch the potential, then graph it. Find the natural
frequency for small oscillations without the driving
force.

(b) Predict how the particle will move if Fd is zero,

small, and large. Write down your predictions.

(c) Simulate the double-well oscillator, assuming fixed

values b = 0.25 and ωd = 1 (in some scaled units). Plot
the position and velocity curves (x-t and v-t), for Fd =
0.1, 0.2, and 0.4. Describe the results, and how they
compare with your predictions.
 (d) Compute the power spectrum for each Fd above.
Relate your results above to the power spectra.

(e)* Construct Poincaré maps, one for each Fd. What do

they represent? Are any of them strange attractors?
Quantify your answers.

.A Program listings and descriptions

Program listing 5.3: Initial value dependence of logistic map
( logisdiff.py)

1 import matplotlib.pyplot as plt # get matplotlib plot functions

3 x1, r = input( 'enter x1, r; eg. 4, .7 : ’)

x2 = x1 + 0.01 # initial difference
5 n, xn1, xn2, diff = 20, [], [], [] # buffers
for i in range(n):
7 xn1.append(x1), xn2.append(x2), diff.append(abs(x1−x2))
x1, x2 = 4*r*x1*(1.0−x1), 4*r*x2*(1.0−x2) # parallel
iterates
9
plt.figure() # plot the series
11 plt.plot(range(n), xn1, 's-’, range(n), xn2, 'o-’) # squares &
circles
plt.xlabel(’n’), plt.ylabel(’xn’)
13 plt.text(3, .01+min(xn1),’r =’ + repr(r))

15 plt.figure() # plot the difference

plt.plot(range(n), diff)
17 plt.semilogy() # semilog scale
plt.xlabel('n’), plt.ylabel(’Δx’)
19 plt.show()

This program calculates and plots the difference of the

logistic map from two initial values as shown in Figures 5.9 and
5.10. Note the latter is on a semilog scale.

Program listing 5.4: Lyapunov exponent of logistic map

( lyapunov.py)

1 import matplotlib.pyplot as plt # get matplotlib plot functions

import math as ma # get math functions
3

def lyapunov(r): # compute Lyapunov exponent

5 sum, x, ntrans, nsteps = 0.0, 0.5, 2000, 2000 # try diff num.
for i in range(ntrans): # let transients pass
7 x = 4*r*x*(1−x)
for i in range(nsteps): # sum up next n steps
9 x = 4*r*x*(1−x)
dfdx = 4.0*r*(1.0−x−x) # Eq. (5.15)
11 sum += ma.log(abs(dfdx))
return sum/float(nsteps) # lambda
13

ra, lyap, r, dr = [], [], 0.7, 0.0001

15 while r < 1.0:
r = r + dr
17 ra.append(r), lyap.append(lyapunov(r))

19 plt.figure()
plt.plot(ra, lyap, ’,’) # ’,’=pixels
21 plt.axhline(0.), plt.ylim(−2, 1) # draw horiz.line, set y limits
plt.xlabel(’r’), plt.ylabel(’λ’)
23 plt.show()
 The Lyapunov exponent is computed from the analytic
expression (5.29) in Program 5.4.

Program listing 5.5: Nonlindro: Nonlinear driven oscillator

( nonlindro.py)

import matplotlib.pyplot as plt # get matplotlib plot functions

2 import ode, math as ma # get ODE, math functions

4 def pendulum(y, t): # y = [theta, omega], omega_0 = 1

return [y[1], −ma.sin(y[0]) − b*y[1] + fd*ma.cos(omega_d*t)]
6

def solution(n_periods): # find solutions for n_periods

8 bins = 40 # number of points per period
t, y, h = 0.0, [1.0, 0.0], 2*pi/(omega_d*bins) # init values
10 ta, theta, omega = [], [], []
for i in range(n_periods*bins):
12 ta.append(t), theta.append(y[0]), omega.append(y[1])
t, y = t+h, ode.RK4n(pendulum, y, t, h)
14 return ta, theta, omega

16 b, omega_d = 0.5, 0.6 # damping coeff., driving frequency

subnum, pi = 1, ma.pi # subplot number, pi
18 plt.figure()
for fd in [0.7, 1.1]:
20 ax1 = plt.subplot(2, 2, subnum) # 2x2 subplots
ax2 = plt.subplot(2, 2, subnum+2)
22 ta, theta, omega = solution(n_periods = 5)
ax1.plot(ta, theta), ax2.plot(ta, omega)
24 if (subnum == 1): # subplot specific label
ax1.set_ylabel( ’$\\theta$ (rad)’)
 26 ax2.set_ylabel( ’$\\omega$ (rad/s)’)
subnum, fdtxt = subnum + 1, ’$F_d=$’+repr(fd)
28 ax1.text(17, max(theta), fdtxt), ax2.text(17, max(omega), fdtxt)
plt.xlabel( 't (s)’)
30 plt.show()

Program “Nonlindro” integrates the equations of motion of

the nonlinear driven oscillator (5.31), which are calculated in
pendulum(). The two-element array y contains the angular
displacement and angular velocity as . It returns the
derivatives in a list to be used with non-vectorized ODE solvers,
because the two ODEs are simple and speed is important in
such calculations like the bifurcation diagram (see Project
P5.4). The integration uses the non-vectorized Runge-Kutta
RK4n which is faster than vectorized RK4 for a handful of ODEs.
Since energy is not conserved, there is no need to use
symplectic integrators such as the leapfrog method. The
parameters b, fd, and omega_d are global variables defined in
the main program.

To maintain accuracy, the step size must be small compared

to the characteristic time. In this case, it should be the smaller
of the two time scales: the natural period 2π/ω0, and the period
of the driving force 2π/ωd. In practice, we aim to achieve
consistent accuracy by specifying a fixed number of points per
period (line 8), and calculate the step size accordingly (line 9).
 The results are plotted in four subplots (Figure 5.13) made
up of 2-rows by 2-columns (line 20) arranged as

Each subplot is positioned by a subplot number, subnum, that

ranges from 1 to 4, starting from the top-left, increasing across
and then down.

Program listing 5.6: Poincaré map ( poincare.py)

1 import matplotlib.pyplot as plt # get matplotlib plot functions

import ode, math as ma # get ODE, math functions
3

def remap(x): # remap theta to [−pi,pi]

5 if (abs(x) > pi): x = (x − 2*pi if x>0 else x + 2*pi)
return x
7

def pendulum(y, t): # y = [theta, omega], omega_0 = 1

9 return [y[1], −ma.sin(y[0]) − b*y[1] + fd*ma.cos(omega_d*t)]

11 def poincare(transient, n_periods): # transient periods, n_periods

bins = 40 # number of points per period
13 t, y, h = 0.0, [0.6, 0.0], 2*pi/(omega_d*bins) # init values
theta, omega = [], []
15 for i in range(bins*transient): # discard transients
t, y = t+h, ode.RK4n(pendulum, y, t, h)
17 y[0] = remap(y[0])
for i in range(bins*n_periods):
19 if (i%(bins//2) == 0): # record every half a period
theta.append(y[0]), omega.append(y[1])
 21 t, y = t+h, ode.RK4n(pendulum, y, t, h)
y[0] = remap(y[0])
23

return theta, omega

25

b, omega_d = 0.5, 0.6 # damping coeff., driving frequency

27 subnum, pi = 1, ma.pi # subplot number, pi
plt.figure()
29 for fd in [0.7, 1.1, 1.2]:
theta, omega = poincare(transient = 20, n_periods = 400)
31 ax = plt.subplot(3, 1, subnum) # 3x1 subplots
ax.plot(theta, omega, ’.’), ax.set_xlim(−pi, pi)
33 if (subnum == 2): ax.set_ylabel( ’$\\omega$ (rad/s)’)
subnum, fdtxt = subnum + 1, ’$F_d=$’+repr(fd)
35 ax.text(−3, min(omega), fdtxt)

37 plt.xlabel( ’$\\theta$ (rad)’)

plt.show()

The poincare() function takes as input two parameters: the

first specifies the number of periods to wait for transients, and
the second the number of periods to record data. Data is
recorded only at discrete points in a given period, at the
beginning and in the middle in this case (line 19). Results are
displayed in multiple plots with manual control of x-axis limits.

Program listing 5.7: Mandelbrot fractal ( fractal.py)

1 import numpy as np, matplotlib.pyplot as plt

from numba import jit # comment out if jit not available
3 @jit # just−in−time compiling
 def mandelbrot(c, maxi):
5 z=c
for m in range(maxi): # maxi=max iterations
7 z = z*z + c
if (z.real*z.real + z.imag*z.imag >=4.0): break
9 return m

11 xl, xr, yb, yt = −2.2, 0.8, −1.2, 1.2 # box size

nx, ny, maxi = 600, 481, 100
13 x, y = np.linspace(xl,xr,nx), np.linspace(yb,yt,ny)
fractal = np.zeros((ny, nx, 3)) # fractal RGB image
15 for i in range(nx):
for k in range(ny):
17 m = mandelbrot(x[i] + 1j*y[k], maxi) # point in complex plane
fractal [k, i] = [m, 2*m, 3*m] # RGB color mix
19 plt.imshow(fractal/maxi, extent=[xl,xr,yb,yt]) # plot as image
plt.show()

The program computes the Mandelbrot fractal using

complex arithmetic. In Python, complex numbers are formed
by combining real values with the pure imaginary number
represented by 1j. If z is a complex number, z.real
and z.imag return the real and imaginary parts, respectively.

The box size can be changed to look at different parts of the

domain. At finer scales, the maximum iteration needs to be
increased. The fractal is stored as an RGB image with a color
mixing scheme (line 18, experiment a bit to your liking). The
image is displayed by plt.imshow() on line 19. It is similar to a
contour plot but may be visually more effective in cases like
this.

Mapping the Mandelbrot set is computationally intensive.

The bottleneck is in determining if a point is in the set. It is an
ideal candidate for a speed boost. Here we use Numba for
simplicity. The @jit statement instructs Numba to compile
mandelbrot at runtime. The speedup factor is about six in this
case, quite good considering how effortless it is on our part. If
Numba is not installed, comment out the two lines. There are
other remedies for speed boost (see Section 1.3.2). Another
example is given in Program S:11.4.

1
The factor of 4 in Eq. (5.1) is included to make things tidy. Other sources treat r′ = 4r as
the control parameter (e.g., Ch. 18 of Ref. [4]).

2
Many of the results can be obtained with the aid of SymPy. See Exercise E5.1.

3
Strictly speaking, ω0 is the angular frequency only in this limit. For the nonlinear,
driven oscillator, it is best to view ω0 as parameter, against which the strength of the driving
force is to be compared.

4
Lorenz carried out his calculation on a Royal McBee LGP-30 computer, the size of a
chest freezer weighing 700 pounds, and classified as a “desk” computer. It could perform
one iteration of calculations per second. Its speed was probably around 10 FLOPS. Typical
speed of today's PCs is around 100 GigaFLOPS.

5
The Lorenz model approximates the Rayleigh-Bénard flow. The parameter σ is the
Prandtl number, r the scaled Rayleigh number, and b the scaled dimension of the system.

6
We should take multi-day forecasts such as 10-day planners with a grain of salt,
especially in weather-prone regions like New England.

7
It is a result of the convolution theorem and is also known as the Parseval's relation.
Chapter 6
Oscillations and waves
Periodic motion is ubiquitous in nature, from the obvious like
heart beats, planetary motion, vibrations of a cello string, and
the fluttering of leaves, to the less noticeable such as the
vibrations of a gong or shaking of buildings in the wind.
Oscillations and periodicity are the unifying features in these
problems, including nonlinear systems in Chapter 5. The
motion is back and forth repeating itself over and over, and can
cause waves in a continuous medium.

We will study periodic motion including oscillations and

waves in this chapter. There are two motivating factors for
studying these phenomena at this juncture. Physically, waves
involve independent motion in both space and time. We can
see waves moving along a string (space) and also feel the
vibrations in time at a fixed point. We can better understand
wave motion after understanding particle motion.
Mathematically, waves are described by partial differential
equations (PDE) with at least two independent variables such
 as space and time. We are equipped to tackle PDEs only after
studying ODEs first in the previous chapters.

We begin with the discussion of a single, damped harmonic

oscillator, and work our way to oscillations of small systems
such as triatomic molecules. We then discuss the displacement
of a string under static forces. Finally, simulations of waves on
a string and on a membrane are discussed.

To solve these problems, we introduce matrix algebra and

eigenvalue formulation of linear systems. We also discuss finite
difference and finite element methods for boundary value
problems involving ordinary and partial differential (wave)
equations. Various visualization techniques including
animation of waves on strings and membranes using objects in
our VPython modules are also presented.

.1 A damped harmonic oscillator

The best-known oscillator in the physical world is also the
simplest: the simple harmonic oscillator (2.46), with its
ubiquitous acronym – SHO. Figure 6.1 captures several frames
of the animated motion of an SHO over half a period. It is
produced with Program 6.1, a slightly modified version of
Program 2.1.
 Figure 6.1: Animated motion of a simple harmonic oscillator.

Program listing 6.1: Simple harmonic oscillator ( sho.py)

import visual as vp
2

ball = vp.sphere(pos=(−2, −2,0), radius=1, color=vp.color.green) # ball

4 wall = vp.box(pos=(−4, −2,0), length=.2, height=4, width=4) # wall
floor = vp.box(pos=(0, −3.1,0), length=8, height=0.2, width=4) # floor
6 spring = vp.helix(pos=wall.pos, thickness=0.1, radius=0.5) # spring

8 h, v = 0.1, 0.0 # step size, initial velocity

while True:
10 vp.rate(50)
ball.pos.x = ball.pos.x + v*h
12 v = v − ball.pos.x*h # Euler−Cromer method
spring.length = ball.pos.x + 3 # stretch spring (offset by 3)

The spring is drawn as a helix object in VPython (line 6),

which has its position at the tail end of the coil (at the wall in
this case), and extends in the direction of its axis vector
(default to the x-axis, (1,0,0)). The length of the spring is
changed as it is being compressed or stretched by the ball (line
13, plus an offset from the wall to the equilibrium position).
The integration uses the Euler-Cromer method (line 12), a first-
order symplectic integrator for conservative systems (Section
2.4, and Exercise E2.6). The numerical method is stable, and
the SHO will oscillate periodically and perpetually according to
Eq. (2.47) because there is no dissipation.

Realistic oscillators are dissipative, or damped, due to

forces like friction or air resistance. A damped harmonic
oscillator of mass m and spring constant k may be described by
the linearized version of Eq. (5.30) (small oscillations and a
change of variable θ → x) as

Two generic, system-independent constants are introduced

where b is the linear coefficient in the damping force, −bv (see

Eq. (3.3)).

Given ω0 and the dimensionless γ, Eq. (6.1) is the universal

equation for a damped harmonic oscillator. It occurs frequently
in physics and engineering problems, and has wide-ranging
applications. For example, an RLC circuit (Figure 6.2) behaves
exactly like the mechanical oscillator we assumed in Eq. (6.1).
The only difference is that the displacement x would be
replaced with the charge q (Exercise E6.2).
6.1.1 CRITICAL, OVER- AND UNDER-
DAMPING
Let us first examine the solutions of Eq. (6.1) numerically using
RK4 for several cases of damping as shown in Figure 6.3.

With no damping, the oscillation is sinusoidal as expected.

When damping is small, the oscillation persists but the
amplitude is reduced. When damping is further increased to γ
≥ 1, the oscillation is totally wiped out, as the amplitude does
not cross the zero axis even once. For the smaller γ = 1,
damping happens quicker and more effective than the larger γ
= 2. This seems counterintuitive at first, but it is all about the
balance of forces. If damping is large, the spring force needs a
longer time to restore the system to equilibrium, losing all
energy and getting stuck at the equilibrium. The smaller the
damping, the faster the process. However, there must be a
threshold where if the damping is further reduced, the
oscillator will be unable to lose all its energy by the time it
returns to equilibrium. If it still has a positive amount of energy
left when it reaches the equilibrium position, it will keep going,
crossing zero to the other side of the equilibrium. Indeed, this
threshold exists (γ = 1, Figure 6.3), and is called critical
damping. Below the threshold γ < 1, the oscillation continues,
but with reduced peaks and valleys in each cycle, and
eventually stops.
 Figure 6.2: RLC circuit.

Figure 6.3: The damped harmonic oscillator for different damping

coefficients γ: no damping, γ = 0; underdamping, γ = 0.2; critical
damping, γ = 1; and overdamping, γ = 2.

The numerical results above can be confirmed by analytical

analysis [10, 31]. The general solution x(t) to Eq. (6.1) is
(Exercise E6.3)

where A and B are constants, determined by the initial

conditions x(0) and (0). Without damping (γ = 0), Eq. (6.3)
reduces to a perfect harmonic (2.47).

Because of the overall damping factor exp(−γω0t), the

solution will eventually reach zero. We expect this because
damping will dissipate all the energy in the system so it stays
put at equilibrium. How it gets there, however, is not trivial,
and quite interesting as discussed above following Figure 6.3.

Qualitatively, the exponential damping factor is determined

by the smaller exponent (A term) in Eq. (6.3), for γ
≥ 1. Its maximum is 1 when γ = 1, the critical damping – the
fastest damping possible. If γ 1, then . The
larger the γ, the less effective the damping. This is
overdamping.

If γ < 1, the factor becomes imaginary, so the terms

in the brackets of Eq. (6.3) are oscillatory (sinusoidal)
functions of time. This leads to underdamping of the kind

where A′ and φ are new constants related to A and B. Both

overdamping and underdamping increase the time it takes to
reach equilibrium.

6.1.2 DRIVEN OSCILLATOR AND

RESONANCE
In our discussion so far, the system loses energy continuously
and stops at equilibrium on its own. What happens if energy is
added to the system via an external driving force? We then
speak of a driven, linear oscillator. How will it be different from
a driven nonlinear oscillator of Section 5.3?

Let the driving force be F(t). We simply add it to Eq. (6.1) to

obtain the equation of a driven oscillator

An interesting case involves a periodic driving force F(t) =

Fd cos(ωdt). Given ω0, there are essentially three parameters
we can play with: the damping coefficient γ, the driving
frequency ωd and amplitude Fd. The amplitude Fd has
dimensions of force/mass, but its units are arbitrary depending
on the units of x and t.

The results of the driven linear oscillator obtained with RK4

are shown in Figure 6.4 where we vary the driving frequency ωd
but keep γ and Fd fixed.
 Figure 6.4: The driven harmonic oscillator for different driving
frequencies: ωd = 0.5ω0, ω0, and 1.5ω0, with γ = 0.1 and
in each case.

The oscillations start out in irregular fashion, but eventually

settle into regular sinusoidal oscillations. This happens quicker
for critical damping and overdamping, somewhat slower for
underdamping. The irregular part of the oscillation is due to
transient solutions, and the regular part due to steady state
solutions. If we look at the steady state solutions carefully, we
find that the oscillation frequency is the same as the driving
frequency. This is because damping will cause any initial
perturbations to subside, and in the end, the system is
responding to the driving force only. Mathematically, the
transient solution is the general solution (6.3) to the
homogeneous equation (6.1), and the steady state solution is a
particular solution to the inhomogeneous equation (6.5) [10].

Unlike the chaotic nonlinear oscillator (Section 5.3), the

response of the linear oscillator is proportional to the driving
 amplitude (see Exercise E6.4). Incidentally, there is no
nonlinear dynamics, hence no chaos, in a driven linear
oscillator.

Though the driving amplitude is the same in each case, we

see a big difference in the steady state amplitudes. For the case
shown with relatively small damping, the amplitude is
significantly enhanced when the driving frequency is close to
the natural frequency, ωd ω0. This is resonance. The driving
force is in near lock-step with the natural oscillation that
energy is most easily absorbed, producing an enhanced
amplitude.

The general resonance frequency where the amplitude is

maximum occurs at (Exercise E6.4)

Resonance is an important phenomenon in physics and in

nature.1 We will see it in quantum transitions (Sections 8.4.2
and S:8.2) and scattering (Sections 12.4.1 and S:8.1.2).

.2 Vibrations of triatomic molecules

Instead of external driving forces, coupled oscillators can drive
each other with internal forces, conserving the energy of the
system. Like the simple two- and three-body problems seen
before (Section 4.2), Figure 6.5 shows two systems consisting
of two and three particles, respectively. We assume the
particles are connected by identical springs with spring
constant k and unstretched length l. The spring forces could
represent bonds in diatomic or triatomic molecules such as CO
(C–O) or CO2 (O–C–O), for instance.

Figure 6.5: The diatomic and triatomic molecules.

Displacement from equilibrium

For the two-particle system, the equations of motion are

The addition (or subtraction) of l takes into account that, in

Hooke's law F = −kx, the variable x refers to the stretching (or
compression) of the spring from the unstretched length (l). We
can make Eq. (6.7) tidier by measuring the displacement from
equilibrium positions. Let and represent the equilibrium
positions of particle 1 and 2, respectively. We then have
. Introducing new displacement variables as

we can rewrite Eq. (6.7) as

 Using the center of mass (CM) coordinates similar to Eq.
(4.4),

we obtain

The oscillation (u) of two particles is effectively one SHO with

angular frequency ω and reduced mass μ. It is a perfect
harmonic (2.47).

6.2.1 NUMERICAL SOLUTIONS AND POWER

SPECTRUM
A pair of true coupled oscillators are contained in the three-
particle system in Figure 6.5, referred to as the triatomic
molecule from now on. We can write down the equations of
motion directly in displacement variables ui from Eq. (6.8) as
 Figure 6.6: The displacement of particle 1 in the triatomic molecule.

Let us first examine the numerical solutions of Eq. (6.12),

which may be solved with a code like Program 6.5. The results
are displayed in Figure 6.6. The parameters used are k = 1, m1
= m3 = 1/4, m2 = 2/5, in arbitrary units.2

The initial condition was set to (u1, u2, u3) = (−0.2, 0.2, 0)
and zero velocities. Only the displacement u1 is shown (the
other two are qualitatively similar). We see regular, periodic
oscillations, but they are not sinusoidal. The positive and
negative displacements are not symmetric about the
equilibrium. The period appears to be 2π for this set of initial
conditions.

The nice periodicity in the data tells us that it can be

decomposed into a Fourier series to reveal the fundamental
frequencies in it. We calculate the power spectrum of the
displacement u1 using the FFT method, and show the results in
Figure 6.7.

A total of 1,024 data points are used in the Fourier

transform. We interpret the spectrum as ranging from negative
to positive frequencies, or ωm in [−512, 511] (see Sections 5.5
and S:5.B). There are only five peaks in the spectrum,
symmetrically distributed about the origin at ωm = 0, ±10, and
±15. Using the time interval of the data points, we can figure
out that they correspond to actual frequencies 0, ±2, and ±3.
Since a positive-negative frequency pair ±ω are physically
indistinguishable, we conclude that there are three frequencies
in the oscillation: ω = 0, 2, and 3. Had we done the same
spectrum analysis of the other two particles, we would have
found the same frequencies. As we will see next, these are the
exact frequencies of the triatomic molecule.

Figure 6.7: The spectrum of the displacement of particle 1 in the

triatomic molecule.
6.2.2 NORMAL MODES
The simulation result in Figure 6.7 shows that the triatomic
molecule have discrete frequencies. We can also find these
frequencies without numerical integration by a totally different
approach: the linear algebraic method. It represents a class of
powerful methods involving systems of linear equations and
matrix algebra. We introduce this method here, and will rely on
it heavily in subsequent problems.

We expect that the solutions to Eq. (6.12) are a superposition

of fundamental harmonics (2.47), ui = ai cos(ωt + φ) (see Sec.
3.12 of Ref. [10]). Substituting this form of solution into Eq.
(6.12) and rearranging the terms, we obtain

These linear equations may be expressed in matrix form,

Let λ = ω2 and

Equation (6.14) can be more conveniently written as

 The matrix A comes from the coupling forces and is so
appropriately named the stiffness matrix because of the spring
constants. Note that A is symmetric about the diagonal, i.e., Aij
= Aji. The matrix B is due to kinetic energy. For general small
oscillations, A and B can be determined systematically from
kinetic and potential energies (see Project P6.2).

Equation (6.16) is a so-called (generalized) eigenvalue

equation. As we shall see shortly, only discrete, characteristic
values of λ are allowed by Eq. (6.16). Moreover, once they are
determined, the solutions u can also be found, one for each λ
value. A characteristic value of λ is called an eigenvalue, and
the corresponding solution u is called an eigenvector, a column
vector.

We wish to determine the eigenvalues λ and the

eigenvectors u. Linear algebra dictates that (e.g., see Sec. 3.8 of
Ref. [10]), aside from the trivial solution u = 0, Eq. (6.16) has
nonzero solutions if and only if the determinant is zero

The problem now is to solve for λ from Eq. (6.17), then

substitute it back into Eq. (6.16) to obtain u, one for each λ. So
an eigenvalue equation like (6.16) determines both the
eigenvalues and the eigenvectors.
 A direct way to solve Eq. (6.17) is to expand the
determinant, which is a cubic polynomial of λ,

Thus we will have three eigenvalues, or eigenfrequencies. For

the parameters used in Figure 6.7, we obtain three
eigenfrequencies, ω1 = 0, ω2 = 2, and ω3 = 3 (see Exercise
E6.5), in agreement with the Fourier analysis.

Substituting the eigenfrequencies back into Eq. (6.13), we

obtain the corresponding unnormalized eigenvectors as

and the final solutions are (up to some phase factors)

Equations (6.19) and (6.20) are the normal modes. They are
the fundamental harmonic oscillations of the triatomic system.

Figure 6.8 shows graphically the motion associated with the

normal modes (run Program 6.5 to see animated motion). In
mode 1, ω1 = 0, all three particles have the same, constant
displacement. There is no oscillation at all. This is the motion
of the CM, a pure translation.
 In mode 2 (ω2 = 2), the two particles on the sides oscillate
about the center particle which stays at rest. The outer particles
move in opposite directions of each other with the same
amplitude. This is the symmetric stretch. Since the center
particle does not participate in the vibration, the frequency is
the same as an SHO, namely, (recall k = 1, m1 = m3
= 1/4 in our example).

All three particles vibrate in mode 3 (ω3 = 3), with the outer
particles moving in unison, and the center particle vibrating in
the opposite direction. This is the asymmetric stretch. The
amplitudes of the outer particles are the same, but are smaller
than the amplitude of the center particle. Their ratio is such
that the CM is at rest in the middle, the same as in mode 2. Any
oscillation of the triatomic molecule is a linear superposition of
the three normal modes. This is what we have seen in Figure
6.6.

Figure 6.8: The normal modes of the triatomic molecule.

6.2.3 EIGENFREQUENCIES AND MATRIX
DIAGONALIZATION
Direct calculation of eigenfrequencies via determinant
expansion as above works for a small system, but quickly
becomes unmanageable for larger systems. For an N-particle
system, the matrices A, B, and the determinant matrix will have
dimensions N × N. There would be N! terms in the determinant
alone (merely ~ 3.6 × 106 for N = 10), not to mention a highly
oscillatory polynomial. In such cases, matrix diagonalization
methods explained below should be used.

To illustrate this, let us assume that in Eq. (6.15) matrix A is

already diagonal, and matrix B is the identity matrix.3 Then the
determinant in Eq. (6.17) is

Finding eigenvalues of the above equation is simple: they are

just the diagonal elements of A.

So, the goal of matrix diagonalization is to nudge a non-

diagonal A toward diagonal form while preserving the
eigenvalues. It is a complicated business. One of the simpler
methods is the Jacobi transformation algorithm for symmetric
matrices. It uses a series of rotation transformations to
successively eliminate the off-diagonal elements. When the
latter are all zero (or close to it), the diagonal elements are the
eigenvalues.

Let us illustrate this approach with a very simple example.

Suppose N = 2, and B is the identity matrix. Then, matrix A and
the eigenvalue equation are

The Jacobi diagonalization method uses a pair of rotation

matrices to transform A to A′ as

where R(θ) is the rotation matrix (4.75). The rotation R(θ) and
its inverse R(−θ) pair are necessary so the transformation is
orthogonal and preserves the eigenvalues [10].

Denoting c = cos θ, s = sin θ, and carrying out the matrix

multiplication in Eq. (6.23), we arrive at4

where we have used the symmetry a21 = a12.

We can eliminate the off-diagonal elements by choosing the

rotation angle to satisfy
which yields

Substituting this θ back into the diagonal elements gives us the

eigenvalues.

Suppose we have for A

which has exact eigenvalues λ1 = 1 and λ2 = 3. According to Eq.

(6.25), θ = π/4 which, when substituted into Eq. (6.24), gives
the transformed matrix A′

We see that the diagonal elements are the correct eigenvalues

when the off-diagonals are zero.

We can essentially apply the Jacobi transformation to any

symmetric N × N matrix, eliminating one pair of off-diagonal
elements each time.5 But there is more to it than meets the eye.
When N > 2, eliminating one off-diagonal pair introduces
nonzero contributions to other off-diagonal elements, though
their magnitudes decrease successively [71]. Complications
such as stability and convergence must also be addressed when
N is moderately large. There are other complicating factors
including condition of the matrix, etc. Clearly, the art of solving
eigenvalue problems is very involved.

For practical reasons, discussing other algorithms and

designing codes that can tackle general eigenvalue problems
reliably and efficiently would give us a very limited return
within the scope of the text. This is one of the few areas where
we defer to “blackbox” solvers for eigenvalue problems.
Fortunately, there are several well-tested and widely available
open-source packages for such purposes, including BLAS
(Basic Linear Algebra Subprograms) and LAPACK (Linear
Algebra Package), both available from the Netlib Repository
(netlib.org).

We can access BLAS and LAPACK routines in Python via

the SciPy libraries (Section 1.3.2), which make available linear
algebra functions among other things, at the speed of compiled
Fortran codes. For our purpose, the relevant function is eigh(),
which solves for eigenvalues and eigenvectors of symmetric (or
Hermitian) matrices. The following code solves the eigenvalues
of our triatomic molecule. It requires the SciPy library (Section
1.B).

Program listing 6.2: Diagonalization of triatomic systems ( eigh.py)

import numpy as np
2 from scipy.linalg import eigh # Hermitian eigenvalue solver

4 k = 1.0
 m1, m2, m3 = 1./4., 2./5., 1./4.
6 A = np.array([[k, −k, 0], [−k, 2*k, −k], [0, −k, k]])
B = np.array([[m1, 0, 0], [0, m2, 0], [0, 0, m3]])
8

lamb, u = eigh(A, B) # eigenvalues and eigenvectors

10 print (np.sqrt(lamb)) # print omega

The program prepares the matrices A and B according to

Eq. (6.15) as NumPy arrays, the expected input format of SciPy
routines. The symmetric eigenvalue solver (line 9) takes A and
B as input, and returns a pair of arrays: lamb is a 1D array
containing the eigenvalues, and u a 2D array filled with the
eigenvectors. The i-th column u[:,i] is the eigenvector
corresponding to the eigenvalue lamb[i] (see Program 6.6).
Executing the program gives the following output:

[ 1.47451053e−08 2.00000000e+00 3.00000000e+00]

We recognize the last two elements are the exact

eigenvalues. The first one, however, is not exactly zero. The
reason is that , and λ was “zero”, as far as the computer
is concerned, on the order of machine accuracy 10−16 for double
precision (Section 1.2).

For larger matrices, it is more efficient to use sparse matrix

functions from SciPy.6 We will encounter this and other linear
 algebra routines in the sections and chapters ahead.

.3 Displacement of a string under a load

Suppose we attach an extra atom to the triatomic system of
Figure 6.5 and wish to know how this will affect its motion.
Well, this does not pose a serious problem. We could follow the
same procedure and integrate a four-particle system in place of
a three-particle one (6.12). What about five-, six-, or N-particle
systems, where N may be very large (think N ~ 1023)? We now
have a string of atoms forming a continuous system which can
support wave motion. There is no hope of treating this many
atoms individually even with all the computing resources in the
world combined. Nor would we want to if we want to make
sense of collective vibrational and wave motion. Here, we must
treat the string as a continuous mass and use concepts such
mass density found in continuum mechanics.

6.3.1 THE DISPLACEMENT EQUATION

Before studying vibrations on a string, we discuss a simpler
problem: the shape of an elastic string in equilibrium under a
static, external force. Consider a piece of string between x and x
+ Δx illustrated in Figure 6.9.
 Figure 6.9: The displacement of an elastic string under tension and
external forces.

We denote the transverse displacement by u. The piece is

held by three forces, the tension to the left T1 = T(x) and to the
right T2 = T(x + Δx), and the external transverse load f which is
defined as force per unit length. The equilibrium condition is
that the longitudinal (horizontal) and transverse (vertical)
forces cancel each other, namely,

Equation (6.26a) tells us that the longitudinal tension, T = T(x)

cos(θ), is the same throughout the string. It is a parameter
measuring how stretched (tautness) the string is.

Rewriting Eq. (6.26b) in terms of T gives us

 The net transverse tension force is T tan θ2 − T tan θ1. Since
tan θ1 = u′(x), and tan θ2 = u′(x + Δx), we can rearrange Eq.
(6.27) to read

In the limit Δx → 0, the ratio in the first term becomes a

second derivative of u, and we have

This equation describes the displacement of an elastic string

under an external load f(x). It is an example of a problem
reducing to a differential equation under infinitesimal changes
mentioned in Chapter 2.

Given f(x), Eq. (6.29) can be solved if boundary conditions

are specified. For instance, if f(x) = f0 is a constant, then

where a and b are constants to be determined by the boundary

conditions u(0) and u(L) at the ends 0 and L, respectively. We
see that it is a parabola.

6.3.2 FINITE DIFFERENCE METHOD

For a general f(x), we can solve Eq. (6.29) using the finite
difference method (FDM). Without loss of generality, we
assume the two ends of the string are separated by a distance
(L) of unity at x = 0 and x = 1. We divide the distance into N
equal intervals, each size h (Figure 6.10). There are N + 1 grid
points, including N −1 interior points and two boundary points.
Over the grid, let xi = ih, u(xi) = ui, and f(xi) = fi, i = 0, 1, 2, …,
N. The boundary points are at u0 and uN.

Figure 6.10: The finite difference grid.

Once space is discretized, we can approximate the

derivatives by numerical differentiation using a three-point
central difference scheme. By adding or subtracting the Taylor
series such as in Eqs. (2.9) and (2.10), we obtain the first and
second derivatives as

Equation (6.31a) is the same as the midpoint method (2.13),

and the error in both cases is second order in h.

Substituting the discretized second derivative (6.31b) into

(6.29) at grid point i, we obtain the FDM displacement
equation

We have simplified the LHS by moving h2 and T to the RHS.

We also restrict the grid points to the interior points only. This
is a three-term recursion relation, a result of three-point
approximation of the second derivative. Even so, all the N − 1
equations must be solved simultaneously since a single grid
point can propagate its influence through its neighbors to its
neighbors’ neighbors, etc.

Defining gi = −h2fi/T, we can write out Eq. (6.32) explicitly,

ending up with

The first and the last rows on the LHS have two terms since we
have moved to the RHS the boundary values which are given
and need not to be solved. That is how the boundary conditions
determine the solutions. The other rows have three terms each.

In matrix form, we have a tridiagonal system,

We can express it in compact matrix form

where A is the (N − 1, N − 1) tridiagonal matrix (stiffness like),

B the column matrix on the RHS (N − 1, 1), and u as usual is a
column vector containing the N − 1 unknowns u1, u2, … uN−1.

Equation (6.34) (or (6.35)) is a linear system of equations.

It can be solved by linear equation solvers once the load f(x) is
given, and thus B known. For example, assuming zero
boundary values u0 = uN = 0, matrix B for a constant load f(x)
= f0 is simply

6.3.3 SOLUTIONS OF LINEAR SYSTEM OF

EQUATIONS
There are well-known methods for solving a linear system of
equations, including Gauss elimination and a closely related
variant, the Gauss-Jordan elimination. These methods plus the
iterative Gauss-Seidel method are discussed in Appendix S:6.A.
Programs implementing these methods are included in
Program S:6.1.

However, in keeping with our approach to eigenvalue

problems, we will use LAPACK routines via SciPy which
provides a general purpose solver solve(A, B). Below we
illustrate its use in solving Eq. (6.34).

First let us consider the external force to be constant, f(x) =

f0. The program below solves Eq. (6.34) for this case.

Program listing 6.3: String under external forces ( stringfdm.py)

1 from scipy.linalg import solve # SciPy linear eqn solver

import numpy as np, matplotlib.pyplot as plt
3

N, u0, uN= 20, 0., 0. # number of intervals, boundary values

5 x = np.linspace (0., 1., N+1) # grid
h, T, f = x[1]−x[0], 1.0, −1.0 # bin size, tension, load
7

A = np.diag([−2.]*(N−1)) # diagonal
9 A += np.diag([1.]*(N−2),1) + np.diag([1.]*(N−2), −1) # off diagonals

11 B = np.array([−h*h*f/T]*(N−1)) # B matrix
B[0], B[−1] = B[0]−u0, B[−1]−uN # boundary values, just in case
13

u = solve(A, B) # solve
15 u = np.insert(u, [0, N−1], [u0, uN]) # insert BV at 1st and last pos
 17 plt.plot(x, u, label=’ $f=-1$’), plt. legend(loc=’ center’) # legend
plt.xlabel(’ $x$’), plt.ylabel(’ $u$’), plt.show()

After setting the number of intervals, the boundary values

and related parameters, the program generates matrix A
according to Eq. (6.34). First, the diagonal of A is created using
np.diag(a,k) (line 8), which returns a 2D array with the k-th
(default 0) diagonal given by the 1D input array a. It can also
extract the diagonal from a 2D input array (not used here). The
next line fills the upper and lower diagonals (k = ±1, and one
fewer element than the diagonal), respectively, completing the
tridiagonal matrix A. Next, matrix B is created (recall gi = −h2 *
f/T, Eq. (6.36)), and the boundary values are accounted for.
Once A and B are prepared, solve() from SciPy is called to
obtain the solution u (line 14). It contains the displacements at
the interior grid points.

Before plotting the results, the two boundary values are

inserted into array u, one at the beginning and one at the end.
The pair of lists [0,N-1] and [u0,uN] in the np.insert function
(line 15) will insert u0 before the first element (index 0) and uN
before (the yet nonexistent) index N − 1 which is one position
behind the last element, effectively appending to the end (recall
that the length of u was N − 1, with the last element at index N
− 2). Finally, the results are plotted with legends from the
labels given to the curves (line 17).
 Figure 6.11 shows the results for two external forces: a
constant force f = −1, and a linear force f = −2x (per unit
length, of course). The tension parameter is T = 1 in both cases,
and the space between the end points were divided into twenty
intervals, N = 20. Because the forces are negative, the string
droops downward as it is being pulled down by the forces. For
the constant force, the shape is a perfect parabola as expected
from the analytic solution (6.30), and is symmetric about the
center point.

Figure 6.11: The displacement of an elastic string under external

forces.

For the linear force, the magnitude of the load increases

linearly with x, so the symmetry is broken. The string is
sheared toward the right end. The bottom of the valley is
shifted to x ~ 0.6, instead of at the center in the case of
constant load. The shape can be described by a cubic
polynomial. Even though the FDM method is second order, and
 should be exact only for a second-order polynomial, it turns out
that the solutions are actually exact in this case due to a quirk.

The FDM method can be used for any continuous load

distribution f(x). If the load is a point source acting on a single
point, say a Dirac function f(x) = f0δ(x − x0) (see Section S:5.1,
Figure S:5.1), then the point x0 must be treated separately. A
different method to be described below, however, can deal with
point sources gracefully.

.4 Point source and finite element

method
The finite element method (FEM) is complementary to FDM,
but is more versatile and advantageous for certain problems
such as point sources just mentioned and irregular domains in
higher dimensions. The FEM is commonly used in science and
engineering.

Below we introduce FEM in 1D space, which will also be

extended to 2D in Chapters 7 and 9. The development of FEM
consists of four steps: choosing the elements and basis
functions; converting a differential equation to integral form;
applying boundary conditions; and finally assembling system
matrices and solving the resultant linear system of equations.
6.4.1 FINITE ELEMENTS AND BASIS
FUNCTIONS
In FEM, the space is divided into discrete elements. Figure 6.12
shows an example with N = 4 elements labeled e1 to e4. Each
element ei has a length hi, which need not be the same. To keep
things tidy, we will assume the elements are identical, h1 = h2 =
· · · = hN = h. There are N + 1 grid points, called nodes in FEM.
The positions of the nodes are xi = ih, same as in FDM.

Figure 6.12: The finite elements and tent basis functions.

The difference between FEM and FDM is that, rather than

discretizing the differential operator, the FEM tries to
interpolate the solution over the domain. To that end, we
expand the solution (displacement) u in a basis set as
where ui are the expansion coefficients to be determined, and
φi the basis functions.

In general, the basis functions are arbitrary. For instance,

orthogonal basis functions are often used in physics (Sections
9.4 and S:8.2). However, it is crucial to FEM that the basis
functions be chosen such that only neighboring basis functions
overlap (the reason will be clear in a moment).

The simplest function between two points is a straight line.

We construct a basis function φi centered at node i, consisting
of two such lines and making a tent-shaped function shown in
Figure 6.12. In terms of node number i and width h, we can
write it as

The basis function φi is nonzero only in the two elements

sharing the common node i. Functions of this type are said to
be compact support. Furthermore, φi is unity at node i and zero
at other nodes,

where δij is the Kronecker delta function, δij = 1 if i = j, and 0 if

i ≠ j.

Therefore, the solution u (6.37) at node i is

In other words, the value of the solution at xi is just the
expansion coefficient, ui.

Figure 6.13: Interpolation (dashed line) of 4x(1−x) (solid line) using

FEM basis functions.

Figure 6.13 illustrates the interpolation of the function y =

4x(1 − x) (logistic map) over the unity interval with four finite
elements, spanned by five basis functions. Setting ui = y(xi), the
values u0 to u4 are , so the interpolated function is
. Over each element, the function
is approximated by a straight line. It shows that the more
elements are used, the more accurate the interpolation.

We will need the derivatives of the basis functions later.

They are

The derivative is discontinuous at node i. The discontinuity at

the nodes is carried over to interpolation (Figure 6.13).
6.4.2 INTEGRAL FORM OF SOLUTIONS
Having chosen the basis functions, we next turn our attention
to finding the coefficients ui. Instead of solving Eq. (6.29)
directly, FEM solves the integral form (also called the weak
form) of the equation [7].

Multiplying both sides of Eq. (6.29) by φj(x) and integrating

over the interval [a, b] (which are the end points of the
domain) leads to

Integrating the LHS by parts, we have

The quantity qj is zero unless j = 0 or j = N because of the basis

function evaluated at the boundaries, i.e., qj = u′(b)δjN − u′
(a)δj0.

We differentiate Eq. (6.37) as , and

substitute it into Eq. (6.43), obtaining

Now let
Because the basis functions and their derivatives have no
overlap unless they share a common node, we see that Aij = 0
unless i = j or j ± 1. This is the reason the basis functions are
chosen to be compact support, so the matrix Aij is sparse. It is
also symmetric, Aij = Aji.

With Eqs. (6.44) and (6.45), Eq. (6.42) can be expressed as

This is a system of N + 1 linear equations, one for each node.

Explicitly, Eq. (6.46) can be expanded as

Equation (6.47) is the final form of the FEM approach.

Except for the first and last rows with two terms each, all other
rows have three terms, reflecting the fact only neighboring
elements contribute to matrix Aij.

6.4.3 BOUNDARY CONDITIONS

The unknowns ui in Eq. (6.47) depend on the boundary
conditions, q0 and qN. There are two types of boundary
conditions.

The Neumann boundary condition specifies the derivatives,

u′(a) and u′(b). According to Eqs. (6.39) and (6.43), we replace
q0 = −u′(a) and qN = u′(b) in Eq. (6.47). All N+1 node values,
u0 to uN, can be determined in the Neumann boundary
problem as

The other type is the Dirichlet boundary condition, in which

the boundary values u(a) = u0 and u(b) = uN are specified. We
are mainly interested in this type of boundary conditions here.
Because u0 and uN are already given, we need not to solve for
them. We therefore cross off the first and the last rows in Eq.
(6.47). Furthermore, we move the first term A10u0 in the
second row, and the last term AN−1,NuN in the second to last
row, to the RHS. The remaining rows form a linear system of N
− 1 equations, which determine the N − 1 interior node values,
u1 to uN−1,
 In matrix form, let

The solutions in the Dirichlet boundary problems are given by

In both FDM and FEM, we end up with a linear system like

Eq. (6.35) or (6.52). The difference lies in how the boundary
conditions are treated, and how the system matrices are set up.

6.4.4 ASSEMBLING SYSTEM MATRICES

Matrix A in FEM is independent of load. From Eqs. (6.41) and
(6.45), we find that the integral over the domain is effectively
over the elements ei and ej only,

The full matrix A reads

This is the stiffness matrix in FEM (Dirichlet type), with a

dimension (N − 1)×(N −1). It is the same as the FDM stiffness
matrix (6.34) (for identical elements of same size h).

For a given load, matrix B will be evaluated according to

Eqs. (6.45) and (6.51), where pi given by

Here we have again used the fact that φi(x) is nonzero only if
xi−1 ≤ x < xi+1. For constant load f(x) = f0, pi = hf0. Assuming u0
= uN = 0, matrix B is
 Once A and B have been prepared, we can obtain the
solutions by solving either Eq. (6.48) for the Neumann
boundary type or (6.52) for the Dirichlet boundary type.
Comparing the system matrices between FDM and FEM in this
case, we find that they are the same (besides the placement of
h), so the solutions are identical. It turns out they are also
identical for linear load, and differences start to develop only
for quadratic and higher order load.

6.4.5 POINT SOURCE SOLUTIONS

Figure 6.14: FEM solutions for combinations of a point source load

and a constant load.
 For a point source, the FEM is easier to apply. For example,
consider the following load,

The Dirac δ function (see Section S:5.1 and Figure S:5.1) makes
direct evaluation of f(xn) impossible in FDM. In FEM, however,
it poses no special difficulty for this integrable singularity, as

where we have used the property φn(xn) = 1 from Eq. (6.39). All
elements of B are the same as Eq. (6.56) except at node n,
where it is (hf0 + α)/T. Solving for u using this modified B, we
obtain the results shown in Figure 6.14.

With only a single point-source force, the solution consists

of two segments of straight lines. It is what we expect if we pull
a string and hold it in position. The “kink” at the point of the
applied force indicates a discontinuity of the first derivative,
i.e.,7

If the force is suddenly withdrawn, as if plucking the string,

we expect waves to be formed on the string. This is discussed
shortly.
 When there is an additional constant load, the same
discontinuity exists. On each side of xn, the string is a shallower
curve relative to the constant force alone, akin to pulling up a
suspended cable.

.5 Waves on a string
When a string, held by a point-like force as depicted in Figure
6.14, is released, wave motion will be generated on it. If the
string is coupled to a sound chamber such as that of a cello,
vibration energy of the string will be transferred to the
chamber, and musical notes can be heard. Modeling the energy
loss would be similar to but more complicated than damping in
a harmonic oscillator (Section 6.1). We are interested instead in
describing an ideal vibrating string without dissipation.

6.5.1 THE WAVE EQUATION

The starting point for the wave equation is Figure 6.9 and Eq.
(6.28). The net (transverse) force per unit length on the piece
of string from x to x + dx is given by Eq. (6.28),

where T is the tension, f the external load. For a static string,

the net force is zero, giving us the displacement equation
(6.29).
 For a vibrating string, this is no longer the case. The
displacement depends on both position x and time t, u = u(x, t).
From Newton's second law, the net force (per unit length) is
equal to the linear mass density (ρ) times acceleration. The
acceleration at x is given by ∂2u/∂t2, the second derivative of
displacement holding x constant. Replacing u″ by ∂2u/∂x2 in
Eq. (6.60), we obtain

The above equation describes the general motion of a string

under an external load f.

For a free vibrating string, f = 0, and we have the following

wave equation,

Here, v is the speed of the wave. It increases with increasing

tension or decreasing mass density.8

So far we have mostly dealt with ODEs. The wave equation

(6.62) is a PDE. In general, it is much harder to solve PDEs
than ODEs because a PDE has at least two independent
variables whereas an ODE only one. We have systematic
methods such as the Runge-Kutta scheme that are applicable to
most initial value problems involving ODEs. For PDEs,
however, few such widely applicable approaches exist. The
correct method depends on the kind of problems, and the type
of boundary conditions.

6.5.2 ANALYTICAL PROPERTIES

Before solving Eq. (6.62) numerically, we describe analytic
properties as a guide to our subsequent discussions. If u(x) is a
solution at t = 0, then u(x ± vt) is also a solution. Because Eq.
(6.62) is linear, we can form a superposition of counter-
propagating waves as

with A and B being constants. The function f(x−vt) describes a

traveling waveform to the right, for if ϕ = x − vt is kept
constant, then a fixed value f(ϕ) moves to the right (positive x
direction) with speed v. Conversely, g(x + vt) is a traveling
wave to the left. Whether f or g (or both) is present depends the
initial condition.

A useful technique for PDEs is the separation of variables

that can help reduce the dimensionality. Let us try a solution
which is a product of space and time variables separately,

Substitute it into Eq. (6.62) and rearrange the resulting

equation so that x and t are on the opposite sides as
 For Eq. (6.65) to hold for all x and t which can vary
independently, each side must be a constant, call it −k2,

The solutions are

from which we can construct the general solution

We have four constants (C, k, ϕ, θ) to be determined by the

initial condition [u(x, 0), ∂u(x, 0)/∂t], and by the boundary
condition [u(0, t), u(L, t)].

Let us consider the case where the two ends of a string are
held fixed at zero, u(0, t) = u(L, t) = 0. Equation (6.68) dictates
that

The first term can be satisfied by choosing ϕ = 0. The second

term implies
meaning that only discrete values of wavevector k are allowed.

Let

The corresponding discrete solution follows from Eq. (6.68),

This is a standing wave. The λn and fn defined in Eq. (6.71) are

the fundamental wavelengths and frequencies
(eigenfrequencies). The lowest fundamental frequency is v/2L,
and higher harmonics are integer multiples of that. Equation
(6.72) is called an eigenmode. The general solution is a linear
superposition of all eigenmodes like the normal modes (Section
6.2.2).

6.5.3 FDM FORMULATION

Let us try to numerically solve the wave equation using FDM.
We need to discretize both space and time. As before, let the
space interval be h, and xm = mh, m = 0, 1, 2, …, N. For time, let
Δt be the time interval, and tn = nΔt, n = 0, 1, 2, …. The
displacement at xm and tn is written as , so the
subscript refers to space, and the superscript to time.
 We use the second-order approximation for the derivatives
(6.31b),

Evaluating the above at xm and tn, and substituting into Eq.

(6.62), we have

The terms on the LHS refer to different space grids at the same
time grid, whereas the reverse is true on the RHS.

For stepping forward, we solve for from Eq. (6.74),

obtaining

This is the FDM form of the wave equation. It is a recursion

relating the solution at tn+1 to the solutions at previous two
steps tn and tn−1 (for all space grid points). The wave equation
is a second-order differential equation, so we expect to have
two initial values to determine the solution.

6.5.4 VON NEUMANN STABILITY ANALYSIS

As discussed earlier (Section 1.2.2), recursion relations
notoriously tend to be bimodal, either they are very stable or
very unstable. If it is unstable, the recursion relation is useless
numerically, as a small error will be amplified exponentially,
totally overwhelming the true solution. Therefore, before we
plunge ahead solving Eq. (6.75) iteratively (or using any
recursions), we need to be sure of its stability. We can check
this from the von Neumann stability analysis.

Essentially, we represent the solution as

where ρ and k are constants. The recursion is stable if |ρ| ≤ 1.

To find ρ, we substitute Eq. (6.76) into (6.75). After

cancellations and simplification, we obtain

This is the von Neumann characteristic equation. It has two

roots,

The dividing line is |1−2β2s2| = 1, corresponding to βs = 1,

or vΔts = h. If |1−2β2s2| > 1, or vΔts > h, we have two real
roots. However, one of them is always greater than unity, so it
is unstable.
 If |1 − 2β2s2| ≤ 1, or vΔts ≤ h, the two roots are complex
conjugates of each other,

such that the magnitude is unity

Because s = sin(kh/2) ≤ 1, the sufficient condition for stability

is

Physically, this means that the time step must be smaller than
the time it takes for the wave to travel one space grid.

It is instructive to note that if vΔt s is slightly greater than h,

then Eq. (6.81) will not be satisfied near the maximum s =
sin(kmaxh/2) = 1, i.e., at kmaxh/2 = π/2, or,

This is just the Nyquist sampling theorem (S:5.44) (Section

S:5.B). If we take Δk = 2kmax and Δx = h as the uncertainties in
wavevector and in position, respectively, we can restate Eq.
(6.82) as ΔpΔx = ħΔkΔx = 2πħ, which satisfies Heisenberg's
uncertainty principle. Moreover, using k = 2π/λ, we have from
Eq. (6.82)
which is the minimum wavelength representable over space
intervals of h.

6.5.5 SIMULATION OF WAVE MOTION

From the above analysis, we can now solve Eq. (6.75)
iteratively and maintain stability by choosing β ≤ 1, the single
parameter in the wave equation. We use two arrays to hold the
solutions at times tn−1 and tn, advance the solution to tn+1 and
store it in a third array, then swap the appropriate arrays to
advance to tn+2, and so on. This is done in Program 6.4.

Program listing 6.4: Waves on a string ( waves.py)

import numpy as np, visual as vp, vpm

2

def wavemotion(u0, u1):

4 u2 = 2*(1−b)*u1 − u0 # unshifted terms
u2[1:−1] += b*(u1[0:−2] + u1[2:]) # left, right
6 return u2

8 def gaussian(x):
return np.exp(−(x−5)**2)
10

L, N = 10.0, 100 # length of string, number of intervals

12 b = 1.0 # beta ˆ2
14 x, z = np.linspace (0., L, N+1), np.zeros(N+1)
u0, u1 = gaussian(x), gaussian(x)
16

scene = vp.display(title =’ Waves’,background=(.2,.5,1),center=(L/2,0,0))

 18 string = vpm.line(x, u0, z, (1,1,1), 0.05) # string

20 while(True):
vp.rate(100), vpm.wait(scene) # pause if key press
22 u2 = wavemotion(u0, u1)
u0, u1 = u1, u2 # swap u’s
24 string.move(x, 2*u0, z) # redraw string

To start the iterative process, we need the initial solutions at

two successive steps. There are several ways to do this. A
practical way is to choose u0, then shift it by an amount
proportional to vΔt and set it to u1. We use the simplest choice
in Program 6.4: we set both to the same Gaussian.

A more important issue is how to handle boundary

conditions. When m = 0, for instance, we need from Eq.
(6.75), which is not defined on our grid. The simplest choice is
to keep both ends fixed, , so we do not have to worry
about them. This is the approach taken in Program 6.4. Or we
could use periodic conditions, and . It is
equivalent to identical strings placed to both the left and right
sides of the grid. A string with one end fixed and one end open
could also be considered (Project P6.6).

Rather than looping through the array index explicitly,

wavemotion() iterates Eq. (6.75) implicitly via element-wise
operations of NumPy arrays. Whereas the former may seem
more direct, the latter is more suitable in terms of speed and
simplicity. At a given grid m, the last term in Eq. (6.75) involves
the sum of the left and right neighbors, . It suggests a sum
of the form Hj = aIj−1 + bIj + cIj+1 can be done via shifting and
slicing, illustrated in Figure 6.15 for a 5-element array.

We can see from the shaded elements that, except for the
end points, the interior elements (middle row) are lined up
with their neighbors with the shifted arrays. We could
accomplish the conceptual shifting via slicing of NumPy arrays
(Section 1.D). Guided by Figure 6.15, we can find Hj by

H[1:−1] = a*I[:3] + b*I[1:−1] + c*I [2:]

Note that I[1:−1] is the same as I[1:4] for a 5-element array.

Figure 6.15: Conceptual alignment of arrays by shifting and slicing.

Using this strategy, wavemotion() has the uncanny

resemblance to the written form of Eq. (6.75), reproducing it
almost verbatim: line 4 corresponds to the first two terms in
Eq. (6.75), and the next line to the third term via slicing. We
gain not only speed, but also simplicity. Because discretized
operators like Eq. (6.31b) always involve coupling of close
neighbors, we will often encounter such situations. As a result,
element-wise operations with slicing will be the preferred
method from now on.

The main program animates waves on a string with both

ends fixed. The pair of statements (lines 18 and 24) work
together to create and redraw the string using the VPM
(VPython modules, Program S:6.4) library functions line()
and line.move(), respectively. The animation can be paused by
a key press (line 21).

Figure 6.16 displays the results from Program 6.4. It shows

a single Gaussian peak in the beginning. As time increases, it
splits into two smaller peaks, traveling in opposite directions.
When they reach the boundaries, there is no possibility of
moving forward. The tension is downward, pulling the string in
that direction. As a result, the displacement is inverted, and the
waves move toward each other, until they merge into one.
Thereafter, they split again, repeating the cycle.

The fact that there are two counter-propagating waves is a

result of our choice of initial conditions and the principle of
linear superposition (6.63). By choosing the two initial waves
to be the same, we are in effect forcing two waves into one. It
shows we can choose the kind of wave by setting the
appropriate initial conditions, from traveling waves to standing
waves.

We have used a combination of h and Δt such that β = 1 in

Program 6.4. From the von Neumann stability analysis (6.81),
this is the maximum β for stable solutions. Could we obtain
more accurate solutions if we had used a smaller Δt? Our
experience and expectation would tell us yes. As it turns out, we
would be wrong in this case: we would get worse, less accurate
solutions. Owing to a quirk in the 1D wave equation (6.62), we
actually obtain the exact solution if β = 1. The quirk has to do
with the fact ∂4u/∂x4 = v−4∂4u/∂t4 [24]. By a happy accident,
this leads to total cancellation of the third and higher order
terms in the discretized wave equation (6.62) if β = 1. For 1D
waves, the optimal value is β = 1, which simplifies Eq. (6.75) to

Figure 6.16: Traveling waves on a string. Time increases from top

down.
 This result is almost too simple to be true.

However, the exact algorithm is striking in the symmetry

about space and time. Re-writing Eq. (6.84) as

it tells us that the sum of the amplitudes at a grid point m

immediately before and after the current time is equal to the
sum of the amplitudes at the current time n immediately ahead
and behind the current grid point.

.6 Standing waves
Like the normal modes for a systems of particles, standing
waves are the fundamental building blocks of waves on a string.
Any wave can be represented as a linear superposition of these
fundamental harmonics (6.72).

Standing waves can be formed by proper choice of initial

conditions and their time-dependent motion can be simulated
with Program 6.4. Even though analytically solvable (Eq.
(6.71)), we are interested in computing the fundamental
frequencies using the same FDM scheme as the simulations.

The standing waves satisfy Eq. (6.66)

subject to the boundary condition X(0) = X(L) = 0. Equation
(6.86) is analogous to the static string equation (6.29) if T = 1
and f(x) = k2X. Thus, we can solve it by following the same
steps as in Section 6.3.2. In particular, the FDM equation
(6.34) as applied to Eq. (6.86) becomes

This is an eigenvalue problem, AX = λX, analogous to Eq.

(6.16) but simpler since . It can be solved using the same
method as in Program 6.2 with the eigenvalue solver eigh().
This is done in Program 6.6. The results for the first few
standing waves are shown in Figure 6.17.

The FDM results are plotted as symbols and the analytic

solutions as curves (sine waves, Eq. (6.72)) for a string of unit
length. The FDM results and the analytic solutions are
normalized at the first point, X1. A total of twenty grid
segments (N = 20) are used in the FDM calculation. We
recognize the fundamental wavelengths λn = 2/n at harmonic
frequencies f1 to 4f1 where f1 = v/2 (recall L = 1). The FDM
waveforms agree exactly with the analytic solutions.
 Figure 6.17: The first four harmonics on a string. The symbols are
FDM results and the solid curves are exact solutions.

Table 6.1: The FDM eigenwavelengths and relative error.

The FDM eigenvalues from Program 6.6 are listed in Table

6.1. They are slightly off in comparison to the exact results. The
relative error increases with n. The most accurate result is for
the lowest harmonic n = 1. This is expected since we are
approximating the full spectrum by a finite number of spectral
lines (N − 1). Since higher harmonics would require more grid
points, we expect larger errors for higher n. Increasing N will
improve the accuracy of the lower harmonics.9

It is remarkable that, even though the eigenvalues are only

approximate, the FDM waveforms are exact as seen above
 (Figure 6.17). The exact waveform holds even for the highest n
= N − 1, while the relative error in λn is fairly large, ~ 50%. The
reason has to do with the special form of the eigenequation
(6.86), a coincidence that led to the FDM producing exact
solutions for the 1D wave equation (6.62) at optimal value of β
= 1.

Finally let us consider why only discrete frequencies are

allowed. Imagine we pluck the string close to the left end point.
The displacement will travel along the string at a fixed speed v
in (6.62). By the time it reaches the right end point, the
displacement must be zero, for otherwise it does not satisfy the
boundary condition, and is not allowed. This means that in the
time interval Δt = L/v, an allowed wave must complete an
integer n number of half sinusoidal periods, i.e., Δt = nT/2, or
T = 2L/nv. In terms of frequency, it is f = nv/2L, same as Eq.
(6.71). We see that discrete frequencies arise naturally from the
wave equation. We will see essentially the same idea leading to
discrete energies in quantum mechanics (Section 9.1.1).

.7 Waves on a membrane
When the vibrating medium is a surface rather than a string,
two-dimensional waves are created. With a slight modification,
we can generalize the one-dimensional wave equation (6.62) to
two dimensions as
In effect, the differential operator ∂2/∂x2 is replaced by the
Laplacian in 2D. Equation (6.88) describes two-dimensional
waves on a membrane such as a drum. Holding either x or y
constant, we recover the 1D wave equation in the other
direction.

Discretizing ∂2/∂y2 in the same way as Eq. (6.73a), the

Laplacian is

where hx and hy are the grid sizes in x and y directions,

respectively.

Assuming hx = hy = h, and letting , we

have the 2D FDM wave equation (β = vΔt/h)
 Figure 6.18: Waves on a membrane at different times.

Equation (6.90) may be solved in a similar way to Eq.

(6.75), except the displacement arrays are two dimensional.
The code is given in Program 6.7. The core function wave2d() is
below.

1 u2 = 2*(1−b−b)*u1 − u0
u2[1:−1,1:−1] += b*(u1[1:−1,0:−2] + u1[1:−1,2:] # left, right
3 + u1[0:−2,1:−1] + u1[2:,1:−1]) # top, bottom
 The waves in the previous two iterations are stored on a 2D
grid as and ( u0 and u1), respectively. Like the 1D case
(Section 6.5.5), the solution for the next step is obtained via
element-wise operations. The first line accounts for the first
two terms in Eq. (6.90). The third term, which is proportional
to the sum of the four nearest neighbors, is computed in the
next two lines via slicing, again bearing striking simplicity and
resemblance to the written form. This is a natural extension of
the 1D case, as illustrated below for a 4 × 4 array.

Slicing on the left side of line 2 in the above code selects the
center subarray (boxed) from the left array in Eq. (6.91). It
contains the interior points to be updated (compare with
Figure 6.15). The boundary points where ui,j = 0 are excluded.
The first term in the brackets on the right side of line 2 is
depicted in the right array of Eq. (6.91). Here, slicing refers to
the boxed subarray, which is shifted to the left by one column
relative to the center box. Therefore, entries in u[1:-1,0:-2]
are the left neighbors to u[1:-1,1:-1]. Similarly, the other
three terms on the right side of the statement (lines 2 and 3)
correspond to the right, top, and bottom neighbors,
respectively. Here, left-to-right refers to moving across
columns and top-to-bottom to running down the rows.

Through slicing and element-wise operations, the algorithm

(6.90) is thus expressed succinctly in two compact statements.
Because no explicit looping is used, computation is fast and
highly efficient. We will encounter similar situations later and
use the same technique (e.g., Program 6.8 and Section 7.2.1).

The results from Program 6.7 are shown in Figure 6.18 at

several instants as contour plots and snapshots of the animated
wave are shown in Figure 6.19. The initial wave is a product of
symmetric Gaussians in both directions, localized at the center
of the membrane (Figure 6.18, upper left). As time increases it
spreads outward in the form of circular waves (Figure 6.18,
upper right). It is rather like the ripple effect caused by a
pebble hitting a water surface.

When the waves hit the boundary wall where the membrane
is fixed, they are no longer circular. Instead, an inversion
occurs, similar to that observed in waves on a string at the end
points. But it is more complicated for 2D waves because the
wavefront reaches the boundary at different times. The
wavefront first hits the central portion of the walls (Figure 6.18,
lower left). By the time it spreads toward the corner of the
walls, the central portion has already inverted and started
going toward the center (Figure 6.18, lower right). The
wavefronts are distorted by the boundary walls.
 Figure 6.19: Snapshots of animated waves on a membrane.

After many encounters with the boundary, the wave

develops numerous nodes and crests. It is thoroughly mixed,
and the original Gaussian shape is unrecognizable. This is
shown in Figure 6.19. Depending on the initial and boundary
conditions, many interesting landscapes can occur, none
repeating themselves, even after t L/v. For more
explorations, see Project P6.8.

Of course, to maintain stability of evolution, we must be

sure the algorithm is stable. A value of β = 0.5 was used in
generating the above results. For 2D waves, the range of
 stability of β is slightly different than for 1D waves. A von
Neumann stability analysis can be carried out as discussed in
Section 6.5.4. It is found that the FDM solutions are stable for
. The number 2 under the square root comes as a
result of two-dimensional space. Details are left to Exercise
E6.8.

.8 A falling tablecloth toward

equilibrium
The ideal waves on a membrane will oscillate forever. A
realistic wave would suffer dissipation. We are interested in
modeling such a realistic wave on a surface such as a tablecloth.

Figure 6.20: An array of particles on a grid. The right figure shows

the nearest neighbor interactions.

We construct a tablecloth model as a two-dimensional array

of particles arranged on a grid shown in Figure 6.20. There are
N rows, and each row has M particles, for a total of M × N
particles. Nearest neighbors are connected by springs. The
particle at column i (x) and row j (y) interacts with its four
nearest neighbors: left, right, top, and bottom. We assume
initially the grid is laid out in the horizontal x-y plane. The
force on the particle at (i, j) includes four spring forces plus
gravity and damping as

The equations of motion for this system of particles are

qualitatively the same as Eq. (S:6.5) (or (4.41) and (4.42) with
Hooke's law), and can be solved accordingly. The included
Program 6.8 closely mirrors Program S:6.2. It combines the
numerical integration technique of Program S:6.2 and the
visualization technique of Program 6.7. The visual
representation of motion is based on dividing the particle grid
(Figure 6.20) into triangular meshes, approximating a
vibrating surface.
 Figure 6.21: A falling tablecloth held at the corners.

Program 6.8 can be used to simulate 2D waves with or

without damping, for small or large amplitudes. It can also
accommodate various types of boundary conditions, as well as
constant or variable mass density with a slight modification.
Figure 6.21 shows the results of a uniform surface (tablecloth)
held at the corners and falling under its own weight. The
tablecloth was initially flat and at rest. As it falls, it gains both
kinetic energy and elastic potential energy. Toward the bottom,
it rebounds, and oscillations start to propagate throughout the
grid. Except the corner points, all the boundary points are free.
For zero or small damping, very complex waves can be formed.
After a while, the tablecloth settles into equilibrium. The
overall shape is like the catenary (Figure S:6.1) spun around
the central axis.

Our model can also be used to study relaxation time, rate of

energy dissipation, boundary effects, etc. We will revisit the
model for energy minimization in thermal physics (see Section
11.3).

Computationally, modeling a vibrating surface is much

more expensive for a 2D particle array than for a 1D string. The
core of Program 6.8 is cloth() which computes the forces in
the equations of motion. If N is the linear dimension, the
number of operations scales like N2 in 2D. For medium to large
N, it would be a speed bottleneck for an interpretative language
like Python if explicit looping were used (Section 1.3.2). But,
like Program 6.7, cloth() uses implicit looping of NumPy
arrays with efficient vectorized operations, enabling us to
simulate 2D waves without the need of language extensions.

Programs 6.7 and 6.8 are prime examples where Python

with NumPy provides the best of both worlds: the versatility
and elegance of Python programming and the computational
speed of compiled codes. Because this combination results in a
flexible yet efficient workflow, use of NumPy (and SciPy) is
indispensable in the chapters ahead. In most cases, this is
sufficient. If extra speed boost is needed, we can turn to
language extensions (see Program 5.7 for Mandelbrot fractals
and Program S:11.4 for molecular dynamics).

Chapter summary

We began discussion of oscillations and waves with a damped,

driven harmonic oscillator, including critical, over- and under-
damping, and resonance. We discussed the oscillations of small
systems such as triatomic molecules, its power spectrum, and
normal modes motion. Reduction to linear systems of
equations, eigenvalue problems, and their solutions are
described using the SciPy linear algebra library.

The displacement of a string, representing the continuum

limit of an array of particles under static forces, was discussed.
The finite difference method (FDM) was applied to solving the
displacement equation. We also introduced the basic idea of
the finite element method (FEM) for 1D problems. In Chapters
7 and 9, we will extend the FEM to 2D problems in classical
and quantum physics.

Simulations of one- and two-dimensional waves on a string

and on a membrane as described by second-order PDEs were
presented last. We discussed stability of solutions in terms of
von Neumann stability conditions for the FDM scheme. We
also presented a model using a grid of particles to approximate
an oscillating membrane.
 We have illustrated the use of several composite objects
from the VPython modules (VPM) library to visualize
oscillations and wave motion. The VPM library defines several
classes of objects, including strings, slinkies, nets (grids), and
meshes (triangular faces). These objects will be used in
subsequent chapters.

.9 Exercises and Projects

EXERCISES
E6.1 (a) Modify Program 6.1 so the oscillation is in the
vertical direction. Do not forget gravity. How is the
motion different from the horizontal case? Where is the
equilibrium position? Explain.

(b) Show analytically that the only effect of gravity is to

shift the equilibrium position of the oscillator.

E6.2 Show that the RLC circuit in Figure 6.2 can be

described by

Here q is the charge, ∈(t) the driving voltage. Find γ and

ω0 in terms of resistance R, inductance L, and
capacitance C.
E6.3 The solutions to Eq. (6.1) may be obtained by assuming
a trial solution of the form x(t) = exp(λt). Substitute it
into Eq. (6.1) and solve the resulting characteristic
equation for λ. Obtain Eq. (6.3) by a linear
superposition of exp(λ±t) (see Sec. 8.5 of Ref. [10]).
E6.4 (a) Assume a driving force of the form F = Fd exp(iωt),
and a trial solution x = A(ω) exp(i(ωt + ϕ)) for Eq.
(6.5). Show that A(ω) is proportional to Fd. (b) Verify
the resonance condition (6.6) by finding the ωr for
which A is maximum.
E6.5 (a) Solve Eq. (6.18) for λ in the special case m1 = m3 =
m and m2 = M. (b) Verify that if k = 1, m = 1/4, and M =
2/5, there are three eigenfrequencies ωi = 0, 2, and 3, i =
1 to 3. Remember λ = ω2. (c) Solve Eq. (6.13) for each
ωi to obtain the eigenvector [ai1, ai2, ai3]. You will only
be able to determine uniquely two of the three
components. Normalize it such that the largest entry is
unity. (d)* All of the above can be studied with SymPy
as well. For example, factor Eq. (6.18) to show that zero
is always a root (see Exercise E5.1). Next solve it
symbolically for arbitrary masses.
E6.6 (a) Program 6.5 shows the animated motion of the
triatomic molecule in the symmetric vibration mode.
Change the initial condition such that it represents the
asymmetric vibrational motion.

(b) Start with an arbitrary initial condition, analyze the

motion using the FFT method, and confirm the results
in Figure 6.7.

(c) Replace the Euler-Cromer method with the standard

Euler method, and run the simulation again. What is the
long-term consequence?
 E6.7 (a) Verify the FEM matrix Aij in Eq. (6.53).

(b) Evaluate pi given in Eq. (6.55) for linear and

quadratic loads, f1 = a + bx and f2 = cx2, respectively.
Express your results in terms of the node index i and
grid spacing h.

E6.8 Carry out a von Neumann stability analysis of the FDM

equation (6.90) for waves in 2D to determine the
stability condition for β = vΔt/h.
E6.9 Starting from Eq. (6.68), derive the wavevectors for
standing waves subject to one fixed end and one open
end, Eq. (6.94). Do the same for two open ends. Sketch
the first few standing waves in both cases.
E6.10 Suppose a membrane is open on one side, say x = L.
Derive the relation describing the open boundary
condition in 2D.
E6.11 We usually choose higher-order methods over lower-
order ones, hoping to gain better accuracy and
efficiency. Most of the time, it works out this way.
However, higher-order methods are not always better.
Strictly speaking, higher orders translate into better
accuracy only if the problem is sufficiently smooth. For
highly oscillatory problems like those we are dealing
with in this chapter, a higher-order method can be worse
than a lower-order one.

As an example, run Program 6.8 with the fourth-order

Runge-Kutta RK4 and the fifth-order Runge-Kutta-
Felhberg RK45n integrators. Observe the system's
behavior with the latter. Is it physical? Vary the step
size and watch where the problem starts, and analyze
what caused the behavior (by plotting the energy of the
 problematic part, for example). Find ways to resolve the
problem.

PROJECTS
P6.1 Consider the longitudinal oscillations of an N-particle
array, Figure 6.22. Assume the interaction between
neighboring particles obeys Hooke's law, and all the
masses are the same.

Figure 6.22: A particle array.

(a) Set up the equations of motion for the system, and the
A and B matrices as in Eq. (6.14).

(b) Let N = 3, what kind of motion do you predict?

Briefly write down your predictions. Integrate the
equations with an ODE solver. Choose a suitable k and
m, e.g., unity, and use an arbitrary initial condition, say
u1 = 0.2, all others including velocities are zero.
 Optionally animate the motion of the particles with
VPython.

Plot the displacement of a particle as a function of time,

u3(t), for instance. Discuss your results vs. predictions.

(c) Obtain the power spectrum of the data plotted above.

Identify the frequencies in the spectrum, remembering a
± frequency pair counts as one. Make sure you have
enough data points which should be a power of two. Are
the frequencies as you expected? Hint: To obtain a
“clean” spectrum, adjust your step size so that the peaks
are well separated and not “contaminated”.

(d) Calculate the normal modes of the system. Compare

the eigenfrequencies with your power spectrum results.
Explain the motion associated with the normal modes.

(e) Repeat the above calculations for N = 5. Identify all

harmonics. Compare and discuss your results with the
exact value, ωi = 2ω0 sin(iπ/(2N + 2)), i = 1, 2, …, N,

where .

P6.2 Two coupled oscillators are shown in Figure 6.23. The

horizontal oscillator (mass m1 and spring constant k)
moves on a frictionless surface. A pendulum (mass m2
and cord length l) is connected to m1 via a cord.
 Figure 6.23: Two coupled oscillators.

(a) Show that the kinetic and potential energies are

(b) Take the small oscillation limit, expand T and V in

powers of θ and retain terms only to second order in any

combination of x, θ, , and .

(c) Let q1 = x and q2 = θ, and define

Compute the A and B matrices as

 This is how A and B in Eq. (6.16) may be found for
general small oscillations [40]. Calculate the
eigenfrequencies and eigenvectors. Describe the normal
modes.

P6.3 This project and the next one (Project P6.4) should
ideally be treated as team projects. Each individual team
member can focus on one primary project, but work as a
team exchanging ideas and results.

(a) Run Program 6.3 for the constant load f(x) = −2.
Compare your results with Figure 6.11 (f = −1), and
discuss the shapes of the strings.

(b) Assume a linear load f(x) = −2x. Work out the exact
solution from Eq. (6.29).

(c) Modify your program to calculate the displacement

numerically for the linear load. You need to change
matrix B. Plot both numerical and analytical results.
Compare each other and with Figure 6.11 for the linear
case.

(d) Predict what the shape will look like for a quadratic
load, f(x) = −3x2. Sketch it. Carry out the FDM
calculation. How did your prediction turn out? Explain.
P6.4 Investigate FEM solutions for the displacement of a
string under Dirichlet boundary conditions, u0 = uN = 0.
It is helpful if you had gone through Project P6.3
individually or as a team.

(a) Modify Program 6.3 for the constant load, and make
sure you can reproduce the FDM results shown in Figure
6.11. You can decide whether or not to move the factor h
in Eq. (6.54) to the RHS of Eq. (6.52). Verify that doing
so does not alter the solutions.

(b) Consider the combination of constant and Dirac

point-source loads shown in Figure 6.14. Reproduce the
graphs. Your FEM program should be considered
validated.

(c) Calculate the displacement for the same linear and

quadratic loads in Project P6.3. Results from Exercise
E6.7 would be helpful. Compare the results. Plot the
error between FDM and FEM solutions. Explain the
discrepancy.

P6.5 (a) Set up a traveling Gaussian wavepacket on a string.

Do this by changing the initial condition in Program 6.4
to the following:

u1 = gaussian(x+h) or u1 = gaussian(x-h)

Which would result in a right-traveling wave? Observe

the actual wave. Is it a pure traveling wave?
 (b) Let us change u1 = gaussian(x+2*h). Predict the
kind of wave that will be formed. Discuss the actual
waves you see.

Figure 6.24: Standing waves on a string.

(c) Figure 6.24 shows the second harmonic on the string.

Implement initial conditions that will produce the
standing wave, as well as the first, third, and fourth
harmonics.

P6.6 Study waves on a string with an open end. Suppose one

end of the string is tied to a small, frictionless, massless
ring that can move freely along a transverse thin rod at
the boundary (say at x = L).

(a) By considering the balance of forces similar to Eqs.

(6.26a) to (6.26b), show that an open end (or free)
boundary is equivalent to
 (b) Simulate waves on a string with a closed end and an
open end. Modify the FDM scheme in Program 6.4 to
take into account the boundary condition above and u0 =
0. Plot the waves as presented in Figure 6.16. What are
the differences when one end is open?

P6.7 Calculate the fundamental wavelengths of standing

waves on a string with either one or two open ends.

(a) Assume only one open end, so the boundary

condition is X(0) = 0, and ∂X/∂x|x=1 = 0. Implement the
boundary condition to obtain the appropriate
eigenequation analogous to Eq. (6.87). Plot the first four
harmonics, discuss and compare your results to the exact
values.

(b) Repeat above for two open ends.

P6.8 There are many more modes of oscillation with two-

dimensional waves on a membrane. Let us sample a few
of them.

(a) The waves shown in Figure 6.18 propagate outward

from the center. Suppose we start the wave off center at
(L/2, L/4). What do you expect the wave motion to be?
Modify Program 6.7 and simulate this case. Describe
your results.
 (b) What if the initial wave is asymmetric? Simulate this
effect by introducing different Gaussian widths in the x
and y directions. Which direction does the wave broaden
faster? Why?

(c) Is it possible to set up a pure traveling wave on a

membrane equivalent to such a wave on a string (Project
P6.5)? Make an educated guess. Test your hypothesis by
changing u1 to a Gaussian (symmetric) displaced in the
diagonal direction by a distance of v × dt relative to u0.
Describe the resulting wave you see. Explain the wake.

(d) What would standing waves look like in 2D? Set up a

standing wave that is the first harmonic in the x direction
and the second harmonic in the y direction.

(e) So far we have assumed fixed boundary conditions.

Now suppose we use periodic boundary conditions in the
x direction, i.e., u(−h, y, t) = u(L, 0, t) and u(L + h, y, t) =
u(0, y, t). Implement this boundary condition in your
program. Discuss the differences with fixed boundary
conditions.

(f)* Simulate the waves on a membrane with an open

boundary condition on one side, say at x = L.

P6.9 Let us explore the falling tablecloth model in some

detail.
(a) Study Program 6.8 to understand the organization of
the code and the workings of the function cloth().

(b) Add a function that calculates the energy, E, of the

system at a given time. The energy should include the
kinetic and potential (elastic and gravitational) energies.
Calculate the energy in the main loop, and plot it as a
function of time for a given damping factor, say b = 0.2.
Test that your code works correctly by checking some
limits, e.g., without damping (b = 0 in Eq. (6.92)), the
energy should be conserved.

Find the final energy, Ef, when the system reaches

equilibrium. Here equilibrium is defined as when the
energy has reached a certain limit relative to the final
energy, say 99%. Record the net energy loss as ΔE = Ei −
Ef, where Ei is the initial energy.

(c) Vary b from 0.1 to 1 in steps of 0.1. For each b,

simulate the system until it reaches equilibrium. Record
the time it takes for the system to lose half of the amount
ΔE. Denote this time as the relaxation time. Plot the
relaxation time as a function of b. Discuss your results.
Can you classify the system in terms of under- and over-
damping?

For each b, also graph the change in energy (dE) as a

function of energy difference E − Ef at each step.
Characterize the trend between the rate of energy loss dE
 and E − Ef. Analyze the relationship between relaxation
times and rate of energy loss.

(d) Suppose we want to change the boundary condition.

Instead of fixing only the corners, let us set the particles
on opposite sides of the square fixed, e.g., at x = 0 and x
= L. Predict how this will affect the relaxation time and
the rate of energy loss.

Change the boundary condition in cloth(), and run the

simulation again. Compare the results with your
predictions, and briefly explain.

(e)* Currently, cloth() calculates the forces from the

right and top neighbors directly, then uses the third law
for the left and bottom neighbors (Figure 6.20). Re-code
this function so the process is reversed, i.e., compute the
forces from the latter directly, then apply 3rd law.

.A Program listings and descriptions

Program listing 6.5: Triatomic vibrations ( triatom.py)

1 import visual as vp, numpy as np

3 def accel(u):
 return −k*np.array([u[0]−u[1], 2*u[1]−u[0]−u[2], u[2]−u[1]])/m
5

col = [(1,0,0), (0,1,0), (0,0,1)] # RGB colors

7 x0, atom, r = np.array([−3., 0., 3.]), [0]*3, 1.0
for i in range(3):
9 atom[i] = vp.sphere(pos=(x0[i],0,0), radius=r, color=col[i]) # atoms
floor = vp.box(pos=(0, −1.1,0), length=8, height=0.2, width=4) # floor
11 s1 = vp.helix(pos=(x0[0],0,0), thickness=0.1, radius=0.5) # spring
s2 = vp.helix(pos=(x0[1],0,0), thickness=0.1, radius=0.5)
13

h, k, m = 0.1, 1.0, np.array ([1./4, 2./5, 1./4])

15 u, v = np.array ([−1.,0.,1.]), np.zeros(3) # symmetric init cond
while True:
17 vp.rate(50)
u = u + v*h # Euler−Cromer method
19 v = v + accel(u)*h
x = x0 + u
21 for i in range(3): # animation
atom[i]. pos = (x[i],0,0)
23 s1. axis, s1.pos, s1.length= (1,0,0), x[0]+r, x[1]−x[0]−2*r # move
s2. axis, s2.pos, s2.length= (1,0,0), x[1]+r, x[2]−x[1]−2*r

Program 6.5 simulates the motion a triatomic system by

solving Eq. (6.12) using the Euler-Cromer method. Other than
dealing with three atoms, it works just like Program 6.1. The
springs are attached to and moving with atoms 1 and 2 (line
23). They compress or stretch to fill the distances to their right
neighbors, atoms 2 and 3, respectively.

Program listing 6.6: Standing waves on a string ( stringfun.py)

 1 from scipy.linalg import eigh # Hermitian eigenvalue solver
import numpy as np, matplotlib.pyplot as plt
3
N = 20 # number of intervals
5 h, s = 1./N, 5 # bin size, skip
7 A = np.diag([−2.]*(N−1)) # diagonal
A += np.diag([1.]*(N−2),1) + np.diag([1.]*(N−2), −1) # off
diagonals
9
A = −A/(h*h) # solve −h−2AX = k2X
11 lamb, u = eigh(A) # so eigenvalues lamb = kˆ2

13 x, sty = np.linspace (0.,1., s*N + 1), [’ o’,’ ^’,’ s’,’ v’]

fig, f, pi = plt. figure (), np.sin, np.pi # some aliases
15 print (2*pi/np.sqrt(lamb[:len(sty)])) # wavelength
Eq. (6.71)

17 for i in range(1, len(sty)+1):

ui = np.insert(u [:, i−1], [0, N−1], [0., 0.]) # insert BV
19 plt.plot(x, f (i*pi*x)*ui[1]/ f (i*pi*x[s])) # normalize
soln
plt.plot(x [:: s], ui, sty [i−1], label=repr(i))
21
plt.xlabel(’x’), plt.ylabel(’u’)
23 plt.legend(loc=’ lower right’), plt.show()

This program computes the eigenvalues of the wavevector

k2 on a string of unit length. After preparing matrix A the same
way as in Program 6.3 using np.diag, it is scaled by −1/h2, such
that the eigenvalues are actually for k2 instead of −h2k2. This is
convenient since they will be positive and ordered in ascending
order. Because this is a normal eigenvalue problem, matrix B is
omitted from eigh().
 The first four eigenvectors, stored in column i of u[:,i] for
eigenvector i, are plotted with legends (mode number). As in
Program 6.3, boundary values are inserted before plotting. To
compare with analytical solutions directly, they are normalized
at the first point from the boundary. A skip factor s is used to
ensure that FDM symbols do not obscure the smooth analytical
curves.

Program listing 6.7: Waves on a membrane ( waves2d.py)

1 import numpy as np, visual as vp, vpm

3 def wave2d(u0, u1):

u2 = 2*(1−b−b)*u1 − u0
5 u2[1:−1,1:−1] += b*(u1[1:−1,0:−2] + u1[1:−1,2:] # left, right
+ u1[0:−2,1:−1] + u1[2:,1:−1]) # top, bottom
7 return u2
9 def gaussian(x):
return np.exp(−(x−5)**2)
11

L, N = 10.0, 40 # length of square, number of intervals

13 b = 0.25 # betaˆ2

15 x = np.linspace (0., L, N+1) # mesh grid

x, y = np.meshgrid(x, x)
17 u0, u1 = gaussian(x)*gaussian(y), gaussian(x)*gaussian(y)

19 scene = vp.display(title =’ 2D waves’, background=(.2,.5,1),

center=(L/2,L/2,0), up=(0,0,1), forward=(1,2, −1))
21 mesh = vpm.mesh(x, y, 2*u0, (.9,.8,0), (1,0,0)) # mesh

23 while(True):
vp.rate (40), vpm.wait(scene) # pause if key press
 25 u2 = wave2d(u0, u1)
u0, u1 = u1, u2 # swap u’s
27 mesh.move(x, y, 2*u0) # redraw mesh

The above code works in nearly the same way as Program

6.4. The only differences are that the arrays u0, u1, and u2 are
two dimensional and the animation uses a mesh redrawn at
each time step. The core function wave2d() relies on 2D slicing
as described in Section 6.7. The VPM mesh object (line 21) is
created with mesh(x,y,z, topcolor, bottomcolor) over the
meshgrid (x,y) and displacement z. It is animated with the
mesh.move() method (line 27).

Program listing 6.8: A falling tablecloth ( tablecloth.py)

1 import ode, vpm, numpy as np, visual as vp

3 def force( r ): # force of particle pair, with relative pos r

s = np.sqrt(np.sum(r*r, axis=−1)) # distance
5 s3 = np.dstack((s, s, s)) # make (m,n,3) array
return −spring_k*(1.0 − spring_l/s3)*r # Hooke’s law
7

def cloth(Y, t): # tablecloth

9 r, v, f = Y[0], Y[1], np.zeros((N,M,3))
11 rtop = r[0:−1, :] − r [1:, :] # rel pos to top neighbor
rright = r [:, 0:−1] − r [:, 1:] # rel pos to right neighbor
13 ftop, fright = force(rtop), force ( rright ) # forces from top, right
f[0:−1, :] = ftop # force from top
15 f [:, 0:−1] += fright # force from right
f [1:, :] −= ftop # below, left : use 3rd law
 17 f [:, 1:] −= fright
a = (f − damp*v)/mass + gvec
19 v [0,0], v[0, −1], v[−1,0], v[−1, −1]=0, 0, 0, 0 # fixed coners
return np.array([v,a])
21

L, M, N = 2.0, 15, 15 # size, (M,N) particle array

23 h, mass, damp = 0.01, 0.004, 0.01 # keep damp between [.01,.1]
x, y = np.linspace(0,L,M), np.linspace(0,L,N) # particle grid
25 r, v = np.zeros((N,M,3)), np.zeros((N,M,3))
spring_k, spring_l = 50.0, x[1]−x[0] # spring const., relaxed length
27 r [:,:, 0], r [:,:, 1] = np.meshgrid(x,y) # initialize pos
Y, gvec = np.array([r, v]), np.array ([0,0, −9.8]) # [v,a], g vector
29

scene = vp.display(title =’ Tablecloth’, background=(.2,.5,1),

31 up=(0,0,1), center=(L/2,L/2, −L/4), forward=(1,2, −1))
vp.points(pos=[(0,0,0),(0, L,0),(L,L,0),(L, 0,0)], size=50) # corners
33 x, y, z = r [:,:,0], r [:,:,1], r [:,:,2] # mesh points
net = vpm.net(x, y, z, vp.color.yellow, 0.005) # mesh net
35 mesh = vpm.mesh(x, y, z, vp.color.red, vp.color.yellow)

37 while (1):
vp.rate(100), vpm.wait(scene) # pause if key pressed
39 Y = ode.RK4(cloth, Y, 0, h)
x, y, z = Y [0,:,:,0], Y [0,:,:,1], Y [0,:,:,2]
41 net.move(x, y, z), mesh.move(x, y, z)

We can use this program to simulate a falling tablecloth

made of a 2D array of particles. The particles are initially
arranged on a (M × N) grid (see Figure 6.20) but can move in
3D space. Like Program S:6.2 for 1D, neighboring particles
interact through Hooke's force. The function force() computes
this force given the relative position vector r between the
particles. It first calculates the distance squared (line 4) by
summing over the three components via np.sum along the last
axis (i.e., x2 + y2 + z2), followed by the square-root. Because r
is actually a 3D array (see below), the distance is a 2D array
after the sum. Next the distance grid is stacked depth-wise with
np.dstack (laying over each other) to form a 3D array, so that it
has the correct shape for element-wise operations when
multiplied by r to obtain the force (line 6). That is to say, every
component of is divided by r3, akin to ,
for every particle at once. We need to do this because automatic
broadcasting is ambiguous here.

The core function is cloth() which returns the equations of

motion. The input Y=[r,v] contains the position r and velocity
v vectors, respectively, each a (N, M, 3) array corresponding to
the particles on the grid. Like Program 6.7, implicit looping via
slicing is heavily used. For instance, we find the relative
positions of the particles to the neighbors on top (one row
above, Figure 6.20) by subtracting the rows shifted up by one
from the current rows (line 11). The relative positions to the
neighbors on the right can be obtained in a similar fashion. The
forces can then be calculated from these relative positions and
recorded accordingly (lines 14 and 15). We use Newton's third
law (line 16) for the forces from the neighbors below and to the
left, cutting the computational time by half. Note that slicing
keeps the proper boundary conditions throughout. For
example, the last column does not have neighbors to the right,
so it is excluded correctly in f[:,0:-1] (line 15).
 Once the forces are known, acceleration can be obtained for
all particles at once. Finally, the velocities of the corner
particles are set to zero (line 19), so they never move. The result
is returned in a 4D array [v,a]. This again reaffirms the vector
nature of our ODE solvers (Section 2.3.2) that can handle
scalar (one ODE) and multidimensional vector equations
automatically and transparently.

Visual representation of the tablecloth consists of a net

overlaid onto a mesh. Like Program 6.7, we create a net with
the VPM function net() which takes as input x, y, and z grid
points, color, and thickness of the links. It is moved with
net.move() as usual.

1
The Millennium Bridge in London had to be closed just two days after opening in June,
2000 due to resonant swaying. The bridge swayed laterally when crowds walked over it,
causing them to sway in step to keep balance, and leading to resonant vibrations. Fixed with
more damping, it reopened 20 months later, nicknamed the “wobbly bridge”.

2
The time and length scales are determined by the choice of units for k and mi. We can
set them to arbitrary units without loss of generality. If they are in SI units, for example, then
time and length would be in seconds and meters, respectively.

3
This would be the case if the particles have the same, unity mass, and the couplings
between them are neglected.

4
You may also use SymPy's symbolic matrix manipulation to verify the results.

5
The rotation matrix would be an N × N identity matrix except for rows i and j: aii = ajj =
cosθ, aij = −aji = sinθ.

6
Aside from efficiency derived from special algorithms for sparse matrices, we can also
reduce memory bandwidth of the program, making it more efficient by minimizing the
amount of data being moved, which may be as important as computational speed.

7 ±
We use the shorthand for x = x ± ∈ where ∈ is a positive infinitesimal.

8
For instance, the speed on the thickest string of a cello (C string) is v ~ 100 m/s.
−2
Assuming a mass density of ρ ~ 1.5 × 10 kg/m for a metal string, the tension is T ~ 150 N,
a strongly-strung string!

9
The same phenomenon exists in quantum mechanics. See Section 9.6.2.
Chapter 7
Electromagnetic fields
In the simulation of few-body problems in Chapter 4, gravity
was the sole interaction at large distances. If the particles are
charged, they will interact via electromagnetic forces which are
much stronger than gravity at the same distance. For example,
the ratio of the Coulomb force to gravity between a proton and
an electron is about 1039 at a given separation, making gravity
utterly negligible. In fact, macroscopic properties of all
ordinary matter are determined by electromagnetic
interactions. Forces such as tension in a string (Section 6.3),
spring forces, friction, etc., are all manifestation of
electromagnetic forces between atoms.

Like gravity, electromagnetic interactions occur through

action at a distance via electromagnetic fields described by
classical field equations, the Maxwell’s equations. In this
chapter, we are interested in the simulation of electric
potentials and fields in electrostatics and propagation of
electromagnetic waves. Both scalar and vector fields in
electromagnetism will be studied by solving the appropriate
 PDEs first encountered in Chapter 6. We discuss two mesh-
based methods for solving the 2D Poisson and Laplace
equations and related boundary value problems: an iterative
self-consistent (relaxation) method, and a noniterative finite
element method first introduced for 1D problems in Chapter 6.
Complementing mesh-based methods, we introduce a meshfree
method using radial basis functions for solving PDEs. Finally,
we model the vector fields and electromagnetic radiation due to
electric and magnetic dipoles.

.1 The game of electric field hockey

Let us illustrate electric fields via a game of electric field
hockey. The puck is electrically charged and moves in a
rectangular playing area, the rink. Instead of contact forces by
sticks involving a great number of atoms, Coulomb forces from
stationary charges placed around the rink and at the center
control the motion of the puck. A particular configuration of
charges is illustrated in Figure 7.1.
 Figure 7.1: Three shots in a game of electric field hockey.

The goal of the game is to score goals, of course. Two such

goals are placed at the opposite ends of the rink. The rule of the
game is simple: shoot the puck from one end with a given
velocity; a point is awarded if it goes into the goal at the
opposite end. If the puck goes out of bound, your turn ends.

Figure 7.1 shows three shots, two misses and one score. The
success rate depends on the initial condition (skill) and the
underlying physics for a given charge configuration. The
configuration in Figure 7.1 consists of seven charges: five are
positive, one at each corner and one at the center; two negative
ones are placed in the middle of each side. In this example, the
puck is negatively charged, so the forces on the puck are
repulsive from the two negative charges on the sides and
attractive from the corners and the center.
 In the first instance, the puck was aimed at the lower side.
When it got close to the negative charge, the repulsive force
slowed it down, and turned it back up, resulting in a miss. The
other instances had the same speed but different launching
angles. In each case, the puck moved toward the center charge,
orbiting around it once under its attractive force. The puck with
the smaller angle suffered a greater pull, over rotated, and went
out of bound from below, another miss. The third one had just
the right pull, was catapulted toward the goal, and a score.

The hockey game simulator is given in Program 7.1. Most of

the code is to set up visual animation. The actual game
dynamics is modeled in hockey(), which returns the equations
of motion as

where q is the charge of the puck, Qi and the charge and

position, respectively, of the stationary charges. The factor k =
1/4πε0 = 9 × 109 Nm2/C2 is the electrostatic constant for the
Coulomb force analogous to gravity (4.1). In the simulation, we
set k/m = 1 for simplicity. This can always be done with a
proper choice of units for charge and mass (Exercise E7.1).

The main program integrates the equations of motion (7.1),

and checks whether the puck is in bound after each step. If it is
out of bound, the code decides whether it is a miss or a goal.
 As seen in Chapter 4 in the restricted three-body problem
(Figure 4.17), the motion of the particle is more readily
understood in terms of the potential surface it navigates on.
Figure 7.2 shows the equipotential lines and forces (electric
field) on the puck. The figure is plotted with Program 4.6 where
the effective potential is replaced by the Coulomb potentials
between the puck and stationary charges,
(Project P7.1).

The five positive charges at the corners and the center act
like sink holes the puck is attracted to. There are two saddle
points, one on each side of the playing field, situated
approximately in the middle of each half. Therefore, there is a
minimum energy threshold for the puck to overcome the saddle
point. Once over the threshold, the puck must navigate through
the repulsive and attractive fields, and the trajectory can
become very complex, as some in Figure 7.1 illustrate. This is
true especially at lower energies when the puck must orbit
around the center charge in order to escape to the other side.
Occasionally, the particle can even become trapped to the
center of the field, orbiting indefinitely. At higher energies, it is
possible to deflect directly off the negative charge in the middle
of the side and score a goal.
 Figure 7.2: The potential and forces corresponding to the
configuration in Figure 7.1.

For the configuration of point charges considered here, it is

straight forward to compute the potential surface and electric
fields (forces) from Coulomb’s law. For other charge
distributions (including continuous distributions) and arbitrary
boundary conditions, we need to approach the problem
differently. We discuss this next.

.2 Electric potentials and fields

In electrostatics, Gauss’ law tells us that the electric flux
through a closed surface is equal to the enclosed charge divided
by a constant ε0 (permittivity of free space). We can also
express this law in a differential form using vector calculus.
This differential form of Gauss’ law is called the Poisson
equation [43],
where V is the electric potential, and ρ the charge density. The
electric field can be obtained from V by

If the region within the boundary is charge free, i.e., ρ = 0

everywhere inside, Eq. (7.2) reduces to the Laplace equation.
We are mainly interested in the two-dimensional Laplace
equation,

Solutions to Eq. (7.4) are uniquely determined by the boundary

conditions.

Like the displacement and wave equations discussed in

Chapter 6, there are two types of boundary conditions: the
Neumann and Dirichlet types. Under the Neumann boundary
condition, the derivative, or more accurately the gradient, of V
is specified on the boundary. This is equivalent to specifying
the electric field on the boundary. On the other hand, the
Dirichlet boundary condition deals with the values of V itself on
the boundary. There can also be a mixture of both types of
boundary conditions. Below, we will discuss solutions of Eq.
(7.4) assuming Dirichlet boundary conditions where the values
of V are given on the boundary points unless noted otherwise.
7.2.1 SELF-CONSISTENT METHOD
The general idea of self-consistent methods is to solve a given
problem iteratively, starting from an initial guess. In each
iteration, we use the previous iteration to obtain an improved
solution. The process continues until the solution becomes self-
consistent or relaxed toward the “equilibrium” value (see
Section S:6.1.2 for the catenary problem solved with this
method).

We will follow this approach here to solve the Laplace

equation by relaxation. We shall first discretize Eq. (7.4), which
is the same as the LHS of the wave equation (6.88).
Accordingly, we can discretize the Laplacian over a 2D grid just
like Eq. (6.89). Dropping the time variable from Eq. (6.90), the
Laplace equation in FDM is

or equivalently,

As before, h is the grid size (we assume the same h for x and
y), and we have used the following convention
 Equation (7.6) states that the correct, self-consistent
solution at any point is just the average of its four nearest
neighbors. It is independent of the grid size h (though the
accuracy is). Its simplicity is strikingly similar to Eq. (6.85) for
an exact wave on a string.

Accordingly, if we find a solution satisfying Eq. (7.6) and

the boundary condition, we have obtained the correct solution.
So here is how the self-consistent method works: we start with
an initial guess, say zero everywhere, and set the boundary
values; then we go through every grid point not on the
boundary, assigning the average of its four neighbors as its new
value. In the beginning, we expect large changes in the grid
values. But as the iterations progress, they should become more
self-consistent, and only small changes will occur. We repeat
the process until the solution has converged.

Program 7.2 implements this strategy (Section 7.A) to

calculate the potential of two finite parallel plates kept at
opposite potentials and placed inside a grounded box. The
details of the program is described following the listing, but the
central function is relax() below, which finds the self-
consistent solution by relaxation.

1 def relax(V, imax=200): # find self−consistent soln

for i in range(imax):
3 V[1:−1,1:−1] = ( V[1:−1,:−2] + V[1:−1,2:] # left, right
+ V[:−2,1:−1] + V[2:,1:−1] )/4 # top, bottom
5 V = set_boundary(V) # enforce boundary condition
 draw_pot(V), vp.rate(1000)
7 return V

The potential V is stored on a 2D grid as V [i, j] where, by

convention of Eq. (7.7), indices i and j respectively refer to the x
and y directions. The key statement is line 3. It uses the same
slicing and element-wise operations as in 2D waves (see
explanation in Section 6.7 and Eq. (6.91)) to compute the
average of the four nearest neighbors, in remarkable
resemblance to Eq. (7.6). Besides elegantly expressing the
algorithm in one concise statement, we gain speed and
efficiency since otherwise two nested loops would be required
without implicit looping. Furthermore, the averaged values are
assigned back to the same array without the need for a second
array.
 Figure 7.3: Relaxation of the self-consistent method.

The statement is repeated many times, and the difference

between iterations becomes progressively smaller. After each
iteration, the boundary condition on the plates is applied, and
the solution is visually displayed as colored points (Figure 7.3).
Toward the end, very little change occurs, and we say the
solution has become self-consistent, or it has converged.
Finally, the potential and electric fields are plotted in 7.4. See
Program 7.2 for the overall flow and explanation.

Figure 7.3 shows the progress of the solution as a function

of iterations. At the start, the trial solution is set to zero
everywhere except on the plates. After 20 iterations (top right),
the potential has visibly spread to the immediate areas
surrounding the plates. The outward spreading continues,
although at a slower rate, as seen at iteration 60 (bottom left)
and 200 (bottom right). After the last picture, it is hard to spot
changes to the naked eye between iterations. But it is in fact
surely getting closer and closer to the correct self-consistent
solution, only too slowly (see below). This standard approach is
known as the Jacobi scheme.
 Figure 7.4: The potential and electric field of two, finite parallel
plates.

The potential contour lines plus electric field vectors and

surface plots are shown in Figure 7.4. Constant contour lines of
positive potential wrap around the top plate, and those of
negative potential around the bottom plate as expected. In
between the plates, we have approximately equidistant
potential lines, consistent with the surface plot which shows a
roughly linear slope inside as well.

Consequently, the electric field vector inside the plates is

nearly constant, pointing from top to bottom. Near the edges,
there are large fringe fields. Outside the plates, the “leaked”
electric field is considerably weaker. In contrast, in the ideal
case of infinite parallel plates, the electric field would be
constant inside the plates and zero outside. The large fringe
fields and nonzero fields outside are due to the plates being
finite. They should become increasingly less significant if the
plate size increases.

7.2.2 ACCELERATED RELAXATION

At first, we can see visual changes in the standard relaxation
shown in Figure 7.3 because our initial guess, zero everywhere
except on the two plates, is not close to the self-consistent
(correct) solution. After a while, smaller changes occur with
each iteration, the rate of convergence is rather slow. For
instance, for the grid size used above (60×60), the solution is
within a fraction of a percent of the correct solution in about
the first 50 iterations. Thereafter, accuracy improves at a low
power law (Figure 7.5). To gain one more significant digit,
approximately 25 times more iterations are required. For larger
grid sizes, the standard relaxation scheme is time consuming,
and becomes impractical for calculations requiring higher
accuracy. The root cause of the problem is that change at a
particular grid point propagates slowly to neighbors far away,
like in a diffusion process.

Another scheme, called the Gauss-Seidel (GS) method,

makes the system relax faster. Instead of using only old values
before the current iteration, the GS scheme uses the current
values of the nearest neighbors as they become available in the
current iteration. For example, if we start from the first row,
and go across the column in each row, the modified loop in
relax() in the GS scheme for one iteration would look like

for i in range(1, M−1):

for j in range(1, N−1):
if (not_on_plates( i, j )):
V1[i, j]=(V1[i−1,j] + V0[i+1,j] + V1[i,j−1] + V0[i,j+1])/4

Here, V0 is the old solution of the last iteration, and V1 the new
solution. The function not_on_plates() checks that the point (i,
j) is not on the plates (boundary) before updating. Because
V1[i-1, j] and V1[i, j-1] (previous row and column of the
current iteration) are available by the time the grid point (i, j) is
being calculated, we make use of them. It mixes old and new
values, and should converge faster. Indeed, it is faster than the
Jacobi scheme by about a factor of two. However, its advantage
does not stop there.

It turns out that the GS scheme is suitable for accelerated

relaxation (successive overrelaxation) through the introduction
of an acceleration parameter, α. Let V0 and V1 be the values of
the last (old) and current iterations, respectively. We can
express the accelerated relaxation method as

If α = 1, we recover the Gauss-Seidel scheme. For α < 1, the

process is under-relaxed, and usually results in slower
convergence. Over-relaxation is achieved for α > 1, and
generally speeds up the convergence. However, we must keep α
< 2 as the method becomes unstable beyond 2. Even so, we
need to be careful not to set α too close to 2, as it can cause
fluctuations (noise) that do not vanish. For the grid sizes and
geometries used here, we find it best to keep α ~ 1.85.

Figure 7.5 shows the average difference of the grid values

between successive iterations, , which we take as
a measure of absolute error in the solution. The error analysis
was performed on the parallel plates system. The initial guess
was the same as in the standard (Jacobi) and the accelerated
GS methods.

At the beginning, the Jacobi method produced smaller

differences than the accelerated GS method, presumably due to
the mixing of old and new values in the latter. The slope of the
error, however, is much steeper in the accelerated GS method.
On the semilog scale, it is approximately a straight line,
indicating a fast, exponential rate of convergence. Fully self-
consistent solutions within machine accuracy are obtained in
about 100 iterations. The standard relaxation method
converges very slowly as a power scaling. The advantage of
accelerated GS method producing accurate solutions efficiently
is evident.1

Figure 7.5: The error of the standard and accelerated Gauss-Seidel

methods.

It is curious as to why the accelerated method works so

efficiently. Physically, we imagine a situation analogous to the
 falling tablecloth (Section 6.8). The standard method would
correspond to uninterferred relaxation into equilibrium, and
the accelerated method to forced relaxation. The role of the
parameter α can be thought of as an applied force, in addition
to gravity, that pulls the tablecloth into equilibrium faster than
gravity alone.

.3 Laplace equation and finite element

method
The self-consistent method solves the Laplace equation
iteratively by successive relaxation. We can also solve it
noniteratively with other methods. One such method is the
finite element method (FEM) introduced in Chapter 6 for 1D
problems. In this section we will extend the FEM to 2D in order
to treat the Laplace equation. The FEM offers several
advantages including easy adaptation to irregularly-shaped
domains and accommodating different types of boundary
conditions. More broadly, the FEM is widely used in science
and engineering problems.

However, it takes some preparation to develop and

understand the FEM framework. Hence, it is helpful to be
familiar with 1D FEM first (Section 6.4), as our discussions
follow a similar approach here in 2D.

The FEM development is organized in five steps:

 Choose the elements and basis functions;

Set up meshes and tent functions for interpolation;

Reduce the Laplace equation to integral form;

Impose the boundary conditions;

Build the linear system of equations (system matrix).

To see FEM in action first, you can skip ahead to Section 7.4
for a worked example and sample Program 7.3. You may also
switch back and forth between FEM development and
application.

7.3.1 FINITE ELEMENTS AND BASIS

FUNCTIONS IN 2D
We wish to obtain the solution in a two-dimensional domain, a
region surrounded by an arbitrary boundary. As in 1D, we
expand the solution u in a basis set as

Analogous to Eq. (6.37), ui is the expansion coefficient and ϕi

the tent function to be formed with basis functions. Note that
in 2D, we make a distinction between tent functions and basis
functions for clarity.
 In 1D, the basic elements were line segments, and the basis
functions were linear functions between neighboring elements
and zero elsewhere (Figure 6.12). In 2D, we can form a surface
with a minimum of three points, so the simplest element is a
triangle. We will divide the domain into triangular meshes, or
elements. The vertices of the triangles are the nodes (we will
use the terms nodes and vertices interchangeably in this
section). Each node will belong to at least one element, but can
belong to many more.

What basis function do we want to use over each element?

Like the 1D case, the simplest function is a linear function in
both x and y, which is an equation describing a plane. Since
there are three nodes for each element, there will be three basis
functions covering an element, one for each node.2
 Figure 7.6: An element and its basis functions.

Sample basis functions are shown in Figure 7.6. Each basis

function is a triangular plane defined to be unity at its own
node and zero at the other two. The basis function is nonzero
only over the element containing the node. It is defined to be
zero outside the element (compact support).

Let the vertices of the triangle be (xi, yi), i = 1, 2, and 3. The

basis function for node 1 can be written as [75]

The constant 2A is the magnitude of the cross product of

the two vectors from vertex 1 to vertices 2 and 3, respectively,
and is equal to twice the area of the triangle. We can verify that
φ1(x1, y1) = 1 and φ1(x2, y2) = φ1(x3, y3) = 0 at the nodes, and
φ1(x, y) = 0 along the base opposite node 1 (Figure 7.6, top
left). The basis function φ1(x, y) is the shaded triangular plane,
which can be viewed as being formed by pulling one vertex of
an elastic triangle up vertically, while holding the base down.
Similarly, two other surfaces, φ2(x, y) and φ3(x, y), can be
formed about nodes 2 and 3, respectively.

Let us denote the element by e. We label the nodes on

element e as [1, 2, 3] in counterclockwise order. Also, let [i, j, k]
represent a cyclic permutation of [1, 2, 3], including [2, 3, 1]
and [3, 1, 2]. With these notations, we can generalize Eq. (7.10)
to be

where Ae is the area of the element, same as A in Eq. (7.10).

Suppose we want to represent an arbitrary flat surface over

element e. We can do so with a linear combination of the basis
functions

This is illustrated in Figure 7.6 (bottom right). By varying the

coefficients ci, the patch can be stretched and oriented in any
direction. Since our solution u(x, y) will ultimately be
represented by the basis functions (through tent functions to be
discussed below), u(x, y) will be composed of flat, triangular
patches over the domain.

7.3.2 MESHES, TENT FUNCTIONS AND

INTERPOLATION
We illustrate with an example how the meshes, basis functions,
and the tent functions fit together. Figure 7.7 shows a ¬-shaped
domain, a rectangle with a missing corner. We divide it into
four elements (meshes), numbered from e1 to e4. There are six
nodes in total, all on the boundary. Node 1 is a member of all
four elements, nodes 2 and 6 of a single element, and nodes 3
to 5 of two elements.

We associate a tent function with each node for

interpolation as given by Eq. (7.9). A tent function is just a
collection of basis functions from all elements containing the
node. Two tent functions are drawn in Figure 7.8.

The first tent function at node 1 consists of four basis

functions since node 1 belongs to all four elements. This tent
function would be closed except for the fact that the node is on
the boundary, and the lower-left corner is therefore open. We
can express this tent function in terms of the basis functions as

Figure 7.7: The triangular meshes in a domain (unshaded).

 Figure 7.8: The tent functions at nodes 1 and 3.

The tent function is continuous, though not smooth, across

the element boundaries (the ridges). Because the basis
functions are zero outside their own element boundaries, the
value of ϕ1 at a given point (x, y) can be found on one, and one
only, basis function of the element containing that point.

This is true for all points except those on the ridges and the
exact node point for which the tent function is defined. To
avoid double counting, we require that only one term in the
sum (7.12) is actually selected: the basis function of the first
element determined to contain the point. The value of the tent
function at the node is unity by definition.
 Likewise, tent functions at other nodes can be built in the
same way. For example, the tent function at node 3 (Figure 7.8,
bottom) has two basis functions, one each from elements e1 and
e2. There are six such tent functions, one for each node. We
note that two tent functions overlap each other only if the two
nodes are directly connected by an edge. For instance, the two
tent function shown in Figure 7.8 overlap because nodes 1 and
3 are connected by the common edge between e1 and e2. In
contrast, the tent functions at node 3 and node 5 would not
overlap.

Suppose we are given the values of a function at the nodes

ui, and we wish to interpolate the function over the whole
domain. Then from Eq. (7.9), we just need to run the sum over
all the tent functions thus defined.

To see it in action, we show the interpolation of the function

f(x, y) = sin(πx) sin(πy) over the unit square in Figure 7.9. The
meshes are shown in the left panel, and the function
(wireframe) and its interpolations (solid pyramids) in the right
panel.

Two different sets of meshes are used. In the first instance,

the domain is divided into eight elements, with one internal
node and eight boundary nodes. The values of ui = f(xi, yi) are
zero at the boundary nodes and 1 at the lone internal node (the
center). Of the nine tent functions, only the one at the center
contributes. The interpolation is therefore given by this tent
function which is a six-sided pyramid. The front and back
facets have been sliced off because all node values of elements
e4 and e5 are zero. The interpolation is below the actual
function over most of the domain, except at the other two edges
along the diagonal (between e1/e2 and e7/e8).

The second set (Figure 7.9, second row) increases the

number of meshes to 18. Now there are four internal nodes
with the same value . Summing up the four tent functions
gives us a flat-top pyramid. Again the front and back edges
corresponding to elements e6 and e13 are sliced off, but by a
smaller amount than before. There are no improvements over
the other two edges. However, overall fill volume has
improved, mainly due to the facets becoming closer to the
function.
 Figure 7.9: Interpolation with triangular elements.

We mention two observations. First, if we increase the

number of meshes to make them finer, the interpolation would
become better. However, one thing remains: the interpolation
is made up of flat, finite patches that are continuous across
element boundaries, but their derivatives are discontinuous.3
Second, the way we have divided the domain up along one
diagonal of the unit square breaks some symmetries of the
original function. This impacts the quality of the interpolation
and of the solution when symmetries are important in a
problem. Mesh generation should take these factors into
account.

7.3.3 REDUCTION TO A LINEAR SYSTEM OF

EQUATIONS
Because the gradient of the FEM interpolation is discontinuous
across element boundaries, we cannot work directly with
second-order equations like the Laplace equation in differential
form. Rather, like in 1D, we convert the problem into one
involving only first derivatives.

Let us consider the Poisson equation in 2D (change V to u

in Eq. (7.2)),

where σ is the surface charge density. Similar to Eq. (6.42), we

multiply both sides of Eq. (7.13) by ϕj(x, y) and integrate over
the domain S as

Using the identity v∇2u = ∇ · (v∇u)−∇v · ∇u, we can reduce

Eq. (7.14) to
The first term can be simplified with Green’s theorem, which
converts a surface integral of divergence over S into a line
integral along the boundary C

The contour of integration C encloses the domain S in the

counterclockwise direction, and the normal vector points
outward and perpendicular to the tangent of the contour [10].

With the help of Eq. (7.16), we can rewrite Eq. (7.15) as

Note that for the Laplace equation, f(x, y) = 0, so pj = 0, but we

keep it for generality. We substitute the interpolation (7.9) into
(7.17) to arrive at

Let

we can express Eq. (7.18) for each node j as

This is a linear system of equations, to be solved subject to
boundary conditions.

7.3.4 IMPOSING BOUNDARY CONDITIONS

We can specify two types of boundary conditions in Poisson (or
Laplace) problems: the values of the electric field, ∇u
(Neumann type), or the values of the potential, u (Dirichlet
type). Using two parallel plates as an example, the Neumann
type would correspond to specifying the electric field on the
plates (analogous to setting up surface charge densities), and
the Dirichlet type to keeping the plates at certain voltages. We
are mostly interested in the latter here, and restrict our
discussion below to Dirichlet boundary values.

Suppose node k is on the boundary with a given value uk.

Since this node no longer needs to be solved for, we remove the
kth equation from Eq. (7.20), and move the term Ajkuk(j ≠ k) in
other equations to the RHS so the new system is

We have one fewer equation in (7.21) than (7.20).

We can eliminate all other boundary nodes the same way,

crossing off the equations corresponding to these nodes and
moving them to the RHS in the rest, so we end up with

The sums on the LHS and the RHS involve internal and
boundary nodes only, respectively.

There is one more simplification we can make. Because the

index j in Eq. (7.22) runs over internal nodes only, and the tent
function centered on an internal node is zero on the boundary
(see Figure 7.8 and discussion following it), qj as given in Eq.
(7.17) is zero since the line integral is on the boundary where ϕj
= 0. Therefore, we can drop qj from Eq. (7.22). This is an
expected result since we are concerned with the Dirichlet
boundary condition here, and qj involves the Neumann type
(∇u on the boundary).

Expressing Eq. (7.22) in matrix form, we have

It is understood that indices of the matrix entries refer to the

internal nodes. This is the final FEM result. Solving Eq. (7.23)
will give us the FEM solutions uniquely determined by the
boundary condition. For the Laplace equation, we can also
drop pj’s from Eq. (7.23), leaving only bj's.
7.3.5 BUILDING THE SYSTEM MATRIX
The matrix entries Aij in Eq. (7.19) involve the gradient of the
tent functions ϕi and ϕj which are composed of basis functions.
The evaluation of Aij will necessarily operate on the basis
functions of the elements containing the nodes i and j.

To make this explicit, let us denote the list of elements

connected to node i by e(i). For example, referring to Figure
7.7, four elements contain node 1, so the list for node 1 is e(1) =
{e1, e2, e3, e4}; and for node 2, e(2) = {e1}, etc.

Similar to Eq. (7.12), the tent functions ϕi and ϕj can be

written as

where the basis functions are given in Eq. (7.11). Substitution of

ϕi and ϕj into Eq. (7.19) yields

This result tells us that only basis functions in the elements

containing nodes i and j contribute to Aij. Furthermore,
because a basis function is zero outside its own element, there
will be no contribution to Aij unless e(i) and e(j) refer to the
same element. In other words, Aij will be zero unless nodes i
and j are directly connected.
 It seems that a straightforward way to calculate Aij is to
proceed as follows. For each node, we build a list containing
the elements the node belongs to. When evaluating the sum in
Eq. (7.25), we just go through the lists associated with the pair
of nodes. But this node-oriented approach would be
cumbersome and inelegant. For instance, additional
bookkeeping would be necessary to decide whether a pair of
nodes are directly connected, as well as what the common
edges are.

A more natural way is the element-oriented approach. We

specify each element by listing the three nodes at its vertices (in
counterclockwise order). Then, we iterate through the
elements, computing the contributions from each element (e =
[i, j, k]) to the six entries of matrix A: three diagonals
and three off-diagonals plus the
symmetric pairs , etc. After iterating over all the elements,
all contributions will have been accounted for, and the matrix A
will be completely built.
 Figure 7.10: Contributions to off-diagonal entries of matrix A from
individual elements.

Let us illustrate the element-oriented approach using the

mesh shown in Figure 7.7 and reproduced in Figure 7.10. The
six nodes are numbered 1 to 6 and have coordinates (xi, yi), i =
1 to 6. The four finite elements are specified as e1 = [1, 2, 3], e2
= [1, 3, 4], etc. Let

This is the contribution to Aij from element e. It is symmetric

about the diagonal, . Note that we have changed the
integration domain S to Se, the domain over element e.

Figure 7.10 shows the contributions to off-diagonal entries

of matrix A. We start from e1 with nodes [1, 2, 3]. It contributes
to matrix A along the three edges as , and . There are
also symmetric pairs , etc.) and diagonal entries (e.g.,
, not shown in Figure 7.10).

After e1 has been processed, entries A12 and A23 and their
symmetric pairs are complete. No contributions will come from
other elements (besides e1) because node 2 is connected only to
nodes 1 and 3. By the same token, after processing e2, all
entries associated with e1 (plus A34 with e2) are complete with
the exception of A11 which awaits further contribution from
elements e3 and e4. For example, the entry A13 will be complete
to become . We see that after going through
every element, all entries of matrix A will be complete.

We can accomplish the matrix-building process with the

following pseudo code:

for e in [e1, e2, e3, e4]:

for m = 0,1,2:
for n = m to 2:
i = e[m], j = e[n]
Aij = Aij + from (7.27)
if i ≠ j:
Aij = Aji

The two inner loops cycle through the nodes of a given element,
assigning the contribution to the appropriate entry of A, and
taking advantage of symmetry about the diagonal.

It is now just a matter of evaluating Eq. (7.26) to complete

the construction of matrix A. The gradient may be obtained
from Eq. (7.11) as , which is a constant
vector over element e. Therefore, Eq. (7.26) becomes

Once matrix A is prepared, the linear system (7.23) is

readily solved.
7.3.6 MESH GENERATION
Mesh generation is a major effort in FEM. The quality of the
mesh can affect the quality of solutions and computational
efficiency. There are special Python packages dedicated to just
mesh generation. We will discuss the basic idea and the data
structure in the generation of triangular meshes.

Figure 7.11: The mesh over a rectangular domain. Solid symbols are
boundary nodes, and open symbols are internal nodes.

Let us consider a rectangular domain shown in Figure 7.11,

over which the Laplace equation is to be solved. We divide the
domain into N intervals along the x-axis and M intervals along
the y-axis. The grid will have N×M sub-rectangles and (N + 1)
× (M + 1) grid points, i.e., nodes. Each sub-rectangle is
subdivided into two triangles along the diagonal. Each triangle
is a finite element, so there are a total of 2N × M elements.
Figure 7.11 shows a mesh grid where the unit square is divided
with N = M = 3. There are 16 nodes (consecutively numbered),
including 12 boundary nodes and 4 internal nodes, and 18
elements.

We store the nodes as a master list of coordinates (n × 2

array). The elements are also stored in a list, each entry having
three numbers: the indices of the nodes (vertices) in
counterclockwise order. The boundary and internal nodes are
indexed to the master list as well. Table 7.1 illustrates these
variables and sample values.

Table 7.1: FEM variables and data structure of the mesh in Figure
7.11.

A node may belong to several elements, e.g., node 5 is a

member of six elements, e0−2,7−9. The coordinates of an
element’s vertices may be determined by finding the indices
associated with the nodes. For example, the second vertex of
the first element e0 is node elm[0][1]=5, and the coordinates
are node[elm[0][1]], or from Table 7.1 and Figure 7.11. See
Figure 9.24 for another sample mesh.

With the sample mesh of Figure 7.11, the dimension of the

full matrix is 16 × 16. After removing the boundary nodes, the
actual linear system (7.23) to be solved would be 4 × 4. As we
 will see next, the matrix is banded (Figure 7.12), and most
entries are zero (see Project P7.5).

.4 Boundary value problems with FEM

7.4.1 A WORKED EXAMPLE
We present a worked example of boundary value problems with
FEM as applied to the Laplace equation over a rectangular
domain. We assume a unit square such as the one shown in
Figure 7.11. The boundary conditions are that the potential is
zero on the left and bottom sides, and a quadratic curve on the
right and top sides. They are (we make a notation change from
V to u to be consistent with the FEM development)

The FEM program designed to solve the Laplace problem

with these boundary conditions is given in full in Section 7.A
(Program 7.3). The program has two key parts: mesh
generation and system-matrix construction.

Mesh generation is discussed in detail following the

program. The construction of matrix A is done as follows.

def fem_mat(node, elm): # fills matrix Aij = 〈∇ϕi · ∇ϕj〉

2 A = np.zeros((len(node),len(node)))
for e in elm:
 4 (x1,y1), (x2,y2), (x3,y3) = node[e [0]], node[e [1]],
node[e [2]]
beta, gama = [y2−y3, y3−y1, y1−y2], [x3−x2, x1−x3,
x2−x1]
6 ar = 2*(x1*(y2−y3) + x2*(y3−y1) + x3*(y1−y2))
for i in range(3):
8 for j in range(i,3):
A[e[i], e[j]] += (beta[i]*beta[j] +
gama[i]*gama[j])/ar
10 if ( i != j ): A[e[ j ], e[ i ]] = A[e[i ], e[ j ]] #
symmetry
return A

The routine goes through the element list elm. For each
element, the coordinates of the vertices are extracted from the
node list, and the parameters β0−2, γ0−2 and Ae are calculated
according to Eq. (7.11). The contributions to matrix A, ,
where m and n are the nodes of the current element, are
computed from Eq. (7.27). Three of the six off-diagonal entries
are computed, the other three are obtained via symmetry.

We show in Figure 7.12 a sample system matrix with N = 10.

The boundary points have been removed, leaving the matrix
dimension at 81 × 81. The matrix is sparse, consisting of three
bands. We expect a sparse matrix, since Aij is nonzero only if
the nodes i and j share a common edge (directly connected).
From Figure 7.11, we see a node has a maximum of seven
connections (six neighboring nodes plus self). However, Figure
7.12 shows that a row (or column) has at most five nonzero
entries, two less than the number of connections.
 The reason turns out to be that our finite elements are all
right triangles. As such, the entries Aij on the hypotenuse are
zero. Take node 5 in Figure 7.11 as an example, A5,0 and A5,10
are zero. We can visualize that the basis function at node 5, ,
changes along the x direction only; while the basis function at
node 0, , changes along the y direction only; such that their
gradients are perpendicular to each other, resulting in zero
contribution to A5,0. The same reason applies to contributions
to A5,0 from e1 and contributions to A5,10 from e8 and e9.

Figure 7.12: The system matrix for a mesh like Figure 7.11 with N =
10.

The sparseness of the matrix means that the number of

nonzero matrix entries scales linearly with N2, rather than N4.
With special storage schemes, we could deal with a large
number of finite elements before running out of memory (see
Section 8.3.1 for band matrix representation).
 We have the necessary ingredients to solve for the potential
values at the internal nodes. The rest of Program 7.3 is
described following its listing. We show the results in Figure
7.13. The potential slopes down monotonically from its highest
point at (1, 1), and is symmetric about the ridge running down
the diagonal line to the origin. The slope is greatest near the
top, producing the greatest electric fields there (arrows on the
contour plot).

The electric field near the top and right sides initially points
away from the sides near the highest point. Further left and
down, it becomes weaker and its directions turn toward each
side. By the midpoint, they are parallel to the respective sides,
but eventually point into the sides past the midpoint. On the
left and bottom sides, however, the electric fields are
perpendicular to the boundaries because the potentials are
constant on these sides. As the contour lines extend closer to
the bottom-left corner, they start to mimic the shape of the
walls, running nearly parallel to them and bending more
sharply near the origin.
 Figure 7.13: The potential and fields by FEM of a unit square with
boundary conditions (7.28).

To approximate the solution near the corner more

accurately, finer meshes would be necessary. How fine? The
answer lies in the accuracy of FEM solutions considered next.

7.4.2 ACCURACY AND ERROR OF FEM

SOLUTIONS
To analyze the accuracy and error of FEM solutions, we
consider an example which admits compact analytic solutions.
We assume the same unit-square domain as in the worked
example above but with sinusoidal boundary conditions as

The analytic solution can be shown to be

Figure 7.14: FEM solutions with sinusoidal boundary conditions

(7.29).

We only need to change the mesh() routine slightly to take

into account the new boundary conditions to obtain the FEM
solutions from Program 7.3. The results are displayed in Figure
7.14.

The boundary conditions dictate that the solutions be zero

at all four corners. Because of this, there is a saddle point in the
potential surface near where the potential shows
considerable variation, in contrast to the monotonic solutions
of Figure 7.13. The contour lines as well as electric fields (not
shown in Figure 7.14) show symmetry about the diagonal line y
= x. Away from the saddle point and toward the left and bottom
sides, the potential distribution approaches the boundary
values, a situation similar to that in Figure 7.13.

Figure 7.15: The difference between the analytic and FEM solutions
for boundary conditions (7.29).

To assess the accuracy of FEM, we show in Figure 7.15 the

absolute difference between the analytic and FEM solutions
(Project P7.6). At each node, we compute the difference as

where uexact is the analytic solution (7.30), and uFEM the FEM
solution (Figure 7.14). The parameter N = 40 is used in the
FEM calculation, corresponding to grid spacing of 0.025, and
3, 200 finite elements. The error is displayed as a color-coded
image using plt.imshow() (see Program 7.4).

The difference ranges between 0 and ~ 3×10−4, and is

always positive. The FEM results are consistently below the
analytic ones, even though the solution is not monotonic as
discussed above. The smallest error occurs near the boundary
as expected. The largest error is slightly off the saddle point
where the potential values are largest away from the
boundaries.

In Figure 7.16, we show the average error of the FEM

solutions as a function of N. The error decreases with
increasing N, but the slope after the second point (N = 4) is a
straight line. Since the plot is on a log-log scale, it indicates that
the error decreases as a power law. Careful examination of the
trend shows that the slope is −2, or error ~ N−2. Doubling N
reduces the error by a factor of 4.
 Figure 7.16: Accuracy of FEM solutions.

Since the grid spacing h ~ 1/N, we conclude that FEM with

triangular meshes is second order in accuracy, the same as
FDM. In hind sight, we probably could have guessed this,
because the linear interpolation in FEM is like the trapezoid
rule, which is also second order (Section 8.A.1). Higher order
polynomial basis functions could also be used, but the resulting
system matrix would be more complex [38].

.5 Meshfree methods for potentials and

fields
The methods we have discussed so far all involve certain ways
of discretizing space into grids or meshes, over which the
differential operators are approximated. The accuracy of mesh-
based methods can generally be improved by reducing the
mesh size h.

There is another class of meshfree methods taking a

different approach. Rather than discretizing space, the
solutions are interpolated over a chosen set of basis functions.
We discuss one meshfree method here, namely, the radial basis
function method.4 The advantage of this method is that it is
fast, straightforward to implement, and can easily handle
irregular domains. It may also be readily scaled up to higher
dimensions. The drawback is that its accuracy depends on the
quality and size of the basis set, and it is trickier to
systematically improve accuracy.

7.5.1 RADIAL BASIS FUNCTION

INTERPOLATION
The basic idea of meshfree methods is scattered data
interpolation [96]. We assume that, like in FEM, the solution u
can be approximated by expansion in a distance-based radial
basis function (RBF) set as

where ai are the expansion coefficients.

Figure 7.17: Scattered RBFs (left) and the resultant interpolation

(right).

There are many possible functional forms for ϕi. We cite

two commonly used RBFs as example, the Gaussian (GA) and
the multiquadric (MQ),
where ε is the shape (control) parameter, and are the nodes
(centers) scattered over the domain of interest.

We can use Eq. (7.31) for scattered data interpolation

(Figure 7.17). For example, suppose we have four known data
points, ui at to 4. The scattered data points are not
restricted to any particular geometry. The coefficients ai can be
determined by requiring

where we have taken the nodes to coincide with the scattered

data points. The result can be expressed as a matrix equation

Equation (7.34) can be solved for the RBF expansion

coefficients a1−4. The matrix entries are defined at the RBF
collocation points as

where ϕ is either a GA or MQ RBF in Eq. (7.32). Because the

RBFs are based on the radial distance from the
nodes, the system matrix is positive and definite. Furthermore,
its determinant is invariant under the exchange of any pair of
nodes because the scalar distances rij remain the same.
Therefore, the system matrix is well-conditioned, and the
solution is assured to be nonsingular (Exercise E7.6). In fact,
the tent basis functions in the 1D FEM (Figure 6.12, Eq. (6.38))
are linear RBFs, and share some of above traits, except they are
not infinitely differentiable like the GA or MQ. In general,
however, if the basis functions were position (rather than
distance) based, the above mentioned properties would no
longer hold. In this respect, scattered data interpolation by
RBFs has unique advantages.

Figure 7.17 illustrates an example consisting of four

Gaussian RBFs centered at the scattered data points. After
obtaining the expansion coefficients a1−4 (vertical lines at the
nodes in Figure 7.17, left), the resultant interpolant (7.31) can
be used to approximate the solution over the whole domain.

7.5.2 RBF COLLOCATION METHOD

The RBF collocation method is a way to solve PDEs. It is based
on differentiating a RBF interpolant, from which the
differentiation operator matrices can be built [27]. As an
example, we use the method to solve the 2D Poisson equation
with the Gaussian RBF (equally applicable to 3D).

We consider a region surrounded by an arbitrary boundary

(the domain). Let N be the total number of scattered nodes. Of
these, we assume the number of internal nodes is n, and the
number of boundary nodes is N −n.
 Substituting the interpolant (7.31) into the Poisson equation
(7.13), we obtain

The collocation method, also known as Kansa’s method

[52], stipulates that the Poisson equation (7.36) be satisfied at
the internal nodes , j ∈ [1, n], and simultaneously, the
boundary conditions be also satisfied at the boundary nodes ,
j ∈ [n + 1, N].

For simplicity, we assume Dirichlet boundary conditions.

We have for the collocation method

where , and bj are the potential values on the

boundary. Casting Eqs. (7.37a) and (7.37b) into matrix form,
we have

where a = [a1, a2,…, aN]T is a column vector to be determined,

and
The differentiation operators and RBF values are defined as

The system matrix A is square but asymmetric. It can be

evaluated given the specific RBF (GA or MQ) and the nodes. If
the operator is different than the Laplacian, it can be modified
in a straightforward way. The vector b consists of the charge
densities fj and the boundary values bj. Once the necessary
parameters are known, we can obtain the solutions a by solving
Eq. (7.38) in the same manner as Eq. (7.23).
 Figure 7.18: The disk-in-a-box configuration. There are n = 344
internal nodes ( ) and 185 boundary nodes ( ) for a total of N = 529
nodes. The potential is constant on the octagon and zero along the
walls.

Let us apply the RBF collocation method to solve the

Laplace equation with Gaussian RBFs. The system geometry is
a disk (roughly an octagon at the current node spacing) in the
center of a square box, Figure 7.18. We assume the box is kept
at zero potential and the disk at a constant potential (say 1). We
simply distribute the nodes uniformly over the domain
(23×23), though other choices are also possible (e.g., the
Halton sequence [27]). The number of internal nodes is n =
344, and the number of boundary nodes is N − n = 185. The b
vector in (7.39) consists of zeros in the top n rows and ones in
the bottom N − n rows.

The required differentiation operator is (Exercise E7.6)

It can be readily evaluated to form the system matrix A given

the nodes and the shape parameter ε. The value of ε turns out
to be critical to the accuracy and success of the RBF method. It
is known that the accuracy depends on ε, sometimes very
sensitively, given the choice of the RBF type and the placement
of the nodes [55]. Typically, the extreme values, either very
small or very large ε, lead to very bad approximations. It is
unknown, however, where the sweet spot is in general.
Sometimes we must find it by trial and error. From the applied
physicists’ perspective, we resort to physics to guide us, e.g.,
Eq. (6.82). Suppose h is the typical distance between the nodes.
We expect that the resolution should be such that εh ~ 1, or ε ~
1/h.

Figure 7.19: The potential for the disk-in-a-box configuration.

Figure 7.19 shows the results of the RBF collocation method

from Program 7.4. It is structurally similar to the FEM code
and is fully explained in Section 7.A. The value ε = 20 is used
(the actual spacing is 1/22). However, test shows that values
from ~ 10 to 25 works as well. The potential is highest, and
approximately constant, within the disk (or the octagon), and
decreases to zero toward the walls as expected. It also shows
the correct symmetry for the configuration.

Compared to FDM, the accuracy of the RBF method is good

(Project P7.9). Considering that sharp corners usually require
 higher density of nodes for the scattered data interpolation to
be accurate, the RBF results with a modest number of uniform
nodes in the present case are satisfactory. The RBF method is
fast, and easier to use than the FEM. The disadvantage is that it
is more subtle to control accuracy systematically.

With FDM or FEM, we can progressively reduce the error

with smaller meshes (within the limits of machine accuracy or
memory). In the RBF method, we expect that increasing the
number of nodes will lead to more accurate results. However, if
the number of nodes changes, we need to change the shape
parameter accordingly, whose optimal value is not known a
priori. And the situation may be different for different RBFs
(Project P7.9).

Finally, the location of the nodes may represent a potential

degree of freedom that can be exploited to our advantage,
similar to the location of the abscissa in Gaussian integration
(Section 8.A.3). For instance, it is thought that some other
independent way (equilibrium points, standing wave nodes,
etc.) may foretell a set of good nodes from bad ones, proof
again that sometimes numerical computing is indeed a form of
art.

.6 Visualization of electromagnetic
fields
We have discussed electrostatic problems where the electric
charges are stationary. In this section we are interested in
visualization of electromagnetic vector fields produced by
either steadily moving charges or accelerating charges. The
vector fields can be effectively visualized using the tools and
techniques we have employed so far.

7.6.1 MAGNETIC FIELDS DUE TO

CURRENTS
In magnetostatics, the source of magnetic fields is moving
charges, i.e., electric currents. Like Coulomb’s law for electric
fields, the Biot-Savart law describes the magnetic fields due to
a current segment as

where I is the current, the current-carrying line segment

(Figure 7.20), the position vector relative to , and
μ0/4π = 10−7 T·m/A. The total magnetic field is the sum over
all segments, and for a continuous current distribution such as
shown in Figure 7.20, it approaches a line integral.
 Figure 7.20: Magnetic fields due to a current loop.

As a first example, we simulate the magnetic field of a

closed loop. We consider the loop as a unit circle placed in the
x-y plane, forming a magnetic dipole.5

Figure 7.21: Left: magnetic dipole fields; right: zoom-in around the
wire.

Figure 7.21 displays the magnetic fields calculated from Eq.

(7.42) in Program 7.5 using 36 circular segments. Because it is
rotationally symmetric about the z-axis, we show a cut of the
field in the x-z plane, perpendicular to the loop. The current
comes out of the page in the left eye in Figure 7.21 and goes
into the page in the right eye. The magnetic field is strongest
inside the loop (between the eyes), and weakens significantly
outside it. This is because the magnetic fields from all line
segments are additive inside, but are canceling outside due to
the vector product nature of Eq. (7.42). Along the central z-
axis, the magnetic field is perpendicular to the loop.

3D vector field visualization

By swinging around the central z-axis in Figure 7.21, we could
imagine how the magnetic fields look like in 3D. But they can
also be directly visualized using VPython. This can give a new
perspective to viewing vector fields. Figure 7.22 shows two 3D
examples of the magnetic fields of a dipole and of a long,
current-carrying wire.

Figure 7.22: Magnetic fields of a dipole and of a long, current-

carrying wire.
 For the dipole, the calculation of the magnetic fields
remains the same and only the representation is different. To
preserve symmetry, the spatial points are chosen in a axially
symmetric manner (see Program 7.6).

For the long, current-carrying wire, we use the analytic

results for the magnetic fields,

The direction of the current is out of the page, so the magnetic

fields are in counterclockwise tangential directions as given by
the right-hand rule. In both examples, we can rotate the scene
to view the fields from arbitrary viewpoints.

7.6.2 ELECTROMAGNETIC RADIATION OF

AN OSCILLATING DIPOLE
In electrodynamics, if moving charges are also accelerating,
e.g., moving along a curved path or oscillating back and forth,
they emit electromagnetic radiation. One of the most important
sources of radiation is an oscillating electric dipole. This can be
thought of as a pair of opposite charges oscillating sinusoidally
about their center. Let us assume the center at the origin and
the oscillation in the z direction. The electric dipole may be
written as
where p0 is the magnitude of the electric dipole, and ω the
angular frequency. The resulting electromagnetic fields in 3D
visualization obtained from Program 7.7 are shown in Figure
7.23.

A way from the dipole in the radiation zone (kr 1), the
electromagnetic fields have particularly simple expressions
[50],

Here c is the speed of light, k = ω/c = 2π/λ the wavevector (λ

the wavelength).

The magnitudes of the electromagnetic fields decrease

inversely proportional to r. This ensures that the radiative
power, proportional to , is constant through a spherical
surface on average.6 This is in contrast to electrostatic dipole
electric fields which fall off much faster at 1/r3 at large
distances.
 Figure 7.23: Electromagnetic fields of an oscillating dipole, with
and in the longitudinal and latitudinal directions, respectively.

Figure 7.24: Angular distribution of electric dipole radiation and a

cross-sectional view (right).

On the spherical surface of a given radius r, the magnitudes

vary as B, E ∝ sin θ, with θ being the polar angle (see Figure
7.20). Therefore, the radiation is anisotropic and highly
directional. Figure 7.24 shows the angular distribution of field
magnitudes from Program 7.8.
 The pattern displayed in Figure 7.24 is characteristic of a
dipole distribution. The fields are zero at the north and south
poles along the dipole axis (θ = 0 and π), and maximum at the
equator. Furthermore, as Figure 7.23 shows, the electric field
is longitudinal and magnetic field latitudinal, such that they
are perpendicular to each other at a given point. Their cross
product is along the radial vector, the direction of field
propagation. As a result, the radiative power is largest near the
equator in the direction perpendicular to the electric dipole.

Plane electromagnetic waves

As seen above, the electric and magnetic fields in free space are
perpendicular to each other, with the direction of propagation
given by . They are transverse waves. The dipole radiation
is an outgoing spherical wave.

The plane electromagnetic waves are idealized waves

propagating in free space described by

where is the 3D wavevector.

Figure 7.25 illustrates the spatial and temporal visualization

of a plane electromagnetic wave at four different instants. If
is in the x direction, in the y direction, then the direction of
propagation is the z direction. At a given time, the plane wave
is an infinite train of sinusoidal wave along the z-axis. At a
given position, the field vectors oscillate sinusoidally and
indefinitely. Program 7.9 simulating plane electromagnetic
waves is given in Section 7.A.

Chapter summary

We introduced electric fields of point charges through the

simulation of the electric hockey game. For the general
boundary value problems in electrostatics, we studied
numerical solutions of Poisson and Laplace equations. We
presented two methods, the iterative, self-consistent FDM and
the noniterative FEM.

The self-consistent FDM is simple to implement and yields

accurate solutions. The simplicity of the algorithm, namely, the
potential at a given point is equal to the average of its
neighbors, offers a clear physical insight to the meaning of the
Laplace equation, and enables us to visualize the relaxation
process. The standard self-consistent iteration suffers from
slow convergence, but that can be overcome to certain extent
via accelerated schemes such as the Gauss-Seidel or the
overrelaxation methods. It is less straightforward, however, to
deal with Neumann boundary conditions or point charge
distributions.
 Figure 7.25: Plane electromagnetic waves.

We described the 2D FEM for boundary value problems as

applied to the Poisson equation from a physicist’s point of view
for practical use, rather than mathematical rigor. The FEM is
more adapt to handling different types of boundary conditions
and charge distributions. Its versatility is not limited to
electrostatics, and we will encounter it again in quantum
eigenvalue problems.

We introduced the meshfree RBF collocation method for

boundary value problems. The method is fast, and can easily
handle irregular domains and mixed boundary conditions. It
 can be scaled up to higher dimensions without drastically
altering the basic framework.

We have also modeled the vector fields and electromagnetic

radiation due to electric and magnetic dipoles, using both 2D
vector graphs and 3D representations.

.7 Exercises and Projects

EXERCISES
E7.1 Suppose the SI (MKS) unit system is chosen for Eq.
(7.1). What is the mass m such that the magnitude of k/m
is unity?
E7.2 Consider the 2 × 2 grid shown in Figure 7.28. All
potential values on the boundary points are zero except
V5 = 1 at grid point 5. Find, by hand, the potential at the
internal point V4 using FDM.
E7.3 Verify that the basis function (7.10) satisfies φ1(x1, y1) =
1 and φ1(x2, y2) = φ1(x3, y3) = 0 at the nodes, and that
φ1(x, y) = 0 along the base line between the vertices (x2,
y2) and (x3, y3).
E7.4 (a) Calculate the volume under the surface f(x, y) =
sin(πx) sin(πy) over the unit square 0 ≤ x ≤ 1, 0 ≤ y ≤ 1.

(b) Calculate the fill volumes of the interpolations in

Figure 7.9. Break down each pyramid into tetrahedrons.
The volume of a tetrahedron is given by
 , where , , and are the position

vectors of the three vertices relative to the fourth one.

E7.5 Suppose we wish to impose the Neumann boundary

condition where the electric field vector is specified on

the boundary. Starting from Eq. (7.20), express the FEM

system analogous to Eq. (7.23) for Neumann boundary
conditions.
E7.6 (a) Let us swap nodes 1 and 2 in Eq. (7.34). By
exchanging rows and columns 1 and 2, respectively,
show that the determinant of the system matrix is
invariant, and hence never changes sign or crosses zero,
making the matrix nonsingular.

(b) Derive ∇ϕ and ∇2ϕ where ϕ is either a Gaussian or

multiquadric radial basis function (7.32). Verify Eq.
(7.41).

(c) Assume Neumann boundary conditions, i.e., ∇nu is

specified along the normal at the boundary nodes. Obtain
the corresponding form of the system matrix (7.39).

E7.7 Plot the magnetic filed patterns of the following currents,

two of which are shown in Figure 7.26. We are interested
only in the relative magnitudes, and assume all currents
have unity magnitude. Predict the field patterns in each
case, and use VPython to generate a 3D vector field.
Then choose an appropriate cross section that best
represents the symmetry, and use quiver() to produce a
cross-sectional (2D) visual representation.
 (a) Two parallel, long wires separated by a distance d = 1
and carrying parallel and opposite currents, respectively.

Figure 7.26: Current loops, Exercise E7.7.

(b) Two semicircular loops of radii 1 and 2 (Figure

7.26(b)).

(c) Two coaxial circular loops, Figure 7.26(c), stacked a

distance d = 1 apart.

(d) A solenoid of radius 1 and length 2 with N equi-

spaced turns, N ≥ 10.

E7.8 Modify the hockey game (Program 7.1) and add a

magnetic field perpendicular to the playing surface.
Simulate the game to include the Lorentz force

. Vary B from ~ 0.2 to 1. Compare the

trajectories with and without the magnetic field.

E7.9 Simulate the temporal and spatial variation of electric
 dipole radiation.

(a) First, modify Program 7.8 so the sinusoidal time

oscillation at a given r is simulated.

(b)* Program both the r and t dependence (7.45) in select

directions, say along polar angles 30°, 60°, and 90°. How
are they different from the plane electromagnetic waves
(Figure 7.25)?

PROJECTS
P7.1 Generate the potential surface (Figure 7.2) of the hockey
field. Use Program 4.6 and replace the effective potential
by the Coulomb potentials between the puck and
stationary charges. What would the potential surface look
like if the charge of the puck is positive? Is it possible to
score a goal?
P7.2 (a) Implement the Gauss-Seidel acceleration and
overrelaxation methods discussed in Section 7.2.2, using
Program 7.2 as the basis. Verify that your program
produces the same results as the standard self-consistent
method for the parallel plates.

(b) For the overrelaxation method, vary the acceleration

parameter α between 1 ≤ α < 2. Record the number of
iterations, n(α), required to reach a given accuracy, say
10−10. Plot n(α) as a function of α. Discuss the results and
trends.
P7.3 Consider the geometries shown in Figure 7.27. Assume
in each case the central object is kept at a constant
potential (1) and the potential is zero on the rectangular

boundary. Take the size of the object to be about of the

domain size.

(a) Predict the potential lines and electric field patterns in

each geometry, and sketch them.

(b) For each geometry, find the numerical solutions with

the standard FDM relaxation method. Use a grid
dimension of 30 or more (for larger grid dimensions, use
the overrelaxation method). Display the solutions
graphically.

(c)* No doubt you will have noticed that the square

solution seems to be made up of two halves: one is the
wedge solution and the other its reflection about the
diagonal. In fact, the Laplace equation is linear, and
solutions corresponding to piece-wise boundary
conditions are additive. Let u1 and u2 be the solutions for
the boundary conditions b1 and b2, respectively. Then, u
= u1 + u2 is the full solution for the combined boundary
condition b = b1 + b2. We call this the additivity rule.
 Figure 7.27: The square and wedge geometries for
Project P7.3.

In the geometries of Figure 7.27, change the boundary

condition of the wedge and of the wedge reflected about
the diagonal so the combined effect is the same as the
square. Find the two solutions, and verify numerically
the above additivity rule.

(d) Using E⊥ = σ/ε0, compute the surface charge densities

on the square and the wedge, respectively. Find a way to
visualize σ.

P7.4 The Laplace operator may be approximated by a higher-

order, nine-point scheme as
 (a) With the Laplacian above, obtain the FDM equation
equivalent to Eq. (7.6). Let us call this approximation
FDM9.

(b) Devise a standard relaxation scheme for FDM9,

program and test it. Use Program 7.2 as the base program
for the same parallel plates system. In particular, decide
how to implement the boundary condition. One choice is
to treat the points beyond the boundary the same as their
nearest boundary point. Compare the FDM9 results with
FDM for the same grid size.

(c) Once your program is working, apply FDM9 to other

geometries, such as a square with Eq. (7.29). Study
convergence and accuracy of FDM9 by varying the grid
sizes and comparing with the exact solutions. Plot the
scaling of error vs. grid size analogous to Figure 7.16.

(d)* Apply the standard Gauss-Seidel acceleration and

overrelaxation methods to FDM9. Investigate the rate of
convergence in each case.

P7.5 Consider a domain of eight finite elements shown in

Figure 7.28.
 Figure 7.28: A sample FEM mesh (Project P7.5).

(a) Make a table listing the nodes, elements, and internal

and boundary nodes according to the data structure
outlined in Table 7.1.

(b) Sketch the tent functions at nodes 0, 1, 2, and 4.

Optionally draw them as surface plots with Matplotlib, or
as faces objects with VPython (see VPM mesh objects,
Program S:6.4).

(c) Verify that the system matrix (7.25) is given by

 Check the first and second row entries by calculating
them explicitly.

(d) Explain why the entries A13 and A15 are zero. Indicate
what nodes are involved. The entry A55 = 8/2 = 4 is the
largest. What is the reason?

(e) Suppose we wish to solve the Laplace equation, and

all boundary values are zero except u5 = 1 at node 5.
Obtain the FEM equation (7.23) and solve it. Discuss
your results and compare with the FDM result from
Exercise E7.2.

P7.6 Investigate the FEM solution for the boundary condition

(7.29).

(a) Impose the boundary condition by modifying

Program 7.3. Only minimal change to mesh() is required.
Reproduce the solutions shown in Figure 7.14.
 (b) Based on the contour plot, sketch the electric fields.
Calculate the actual electric field vectors and plot them
over the contour plot. Discuss your results.

(c) Compare the numerical and analytic results (7.30),

and plot the absolute error as shown in Figure 7.15 for a
given grid dimension N ~ 40. In a separate graph, plot the
relative error in the interior region. Compared to the
absolute error, is the relative error localized? Can you tell
if the FEM solution is symmetric about the diagonal line?

(d) Vary N from 2 to ~ 100 (push to the highest limit

possible), doubling N each time. For each N, compute the
average error as

and plot Δ vs. N as in Figure 7.16. Describe the trend.

(e)* Study the accuracy of the self-consistent method

with FDM by repeating the last two parts. Use the code
from Project P7.2 if available.

P7.7 Solve the boundary value problems in Figure 7.27 with

FEM.
 Figure 7.29: The FEM mesh for the wedge (N = 10).

(a) For the square problem, use the same mesh as

presented in Program 7.3, set the inner boundary values
to 1 but exclude the area within the inner square. For the
wedge, generate a mesh as shown in Figure 7.29, a
sample mesh with N = 10. The outer boundary values are
zero on the box ( ) and the inner boundary values are 1

(•) on the wedge. Plot the solutions using the Matplotlib

function tripcolor() (see Program 9.8). Compare the
FEM and FDM solutions (Project P7.3 if available) for a
given N ≥ 30.

(b) The wedge is symmetric about the upper-left to

lower-right diagonal. Does the FEM preserve this
symmetry? Devise a way to check it.
 (c)* Generate a mesh so that the dividing line of each
subsquare is parallel to the upper-left to lower-right
diagonal line above. Repeat the above calculation, and
discuss your results.

P7.8 Solve the Poisson equation with FEM for a point charge
inside a unit square that is kept at zero potential. Treat
the point charge at (x0, y0) as a Dirac charge distribution
f(x, y) = δ(x − x0)δ(y − y0).

(a) Assume the charge is just off the center of the square,

(x0, y0) = . Choose a mesh (Figure 7.11) with

an even N, say N = 10. The charge would be in only one

element just to the southeast of the center (in Figure 7.28,
it would be e2).

Prepare the FEM matrices in Eq. (7.23). When evaluating

the column matrix on the RHS, the only nonzero pj’s are
on the three nodes of the element containing the charge.
Furthermore, because ε ~ 0, only one is nonzero (using
the 2 × 2 mesh above, it would be p4). Solve Eq. (7.23),
and display the solutions graphically. Are the results as
expected?

(b) How would the results change if the charge is exactly

at the center, Explain your answer both

physically and numerically.

 (c)* Write a program that accepts the charge at an
arbitrary location. Optionally, generalize it so it is able to
handle n charges at different locations. Test it with n = 4.

P7.9 Study the RBF collocation method pertaining to the

solutions of the Laplace and Poisson equations.

(a) Perform an error analysis of the RBF method. For the

same grid size, carry out calculations using FDM and
RBF methods, Program 7.2 and 7.4, respectively, for the
disk-in-the-box configuration (Figure 7.18). Either
combine the programs, or write the results to files for
separate processing (for file handling, see Program 4.4
on text I/O or Program 9.7 using pickle). Plot the
absolute error as shown in Figure 7.15. Repeat the
calculations for several shape parameters ε = 10 and 15.
Discuss your finding. Optionally, perform the same
analysis by comparing with the FEM results.

(b) Instead of constant potential, suppose the disk is

uniformly charged with a surface charge density of 1
(i.e., σ/ε0 = 1). Change the initialization function in
Program 7.4 to solve the Poisson equation. Calculate the
potential and explain the results.

(c) Modify Program 7.4 to use the multiquadric RBF

(7.32) instead of the Gaussian RBF (use result of
Exercise E7.6 if available). Repeat the above
investigations.
 (d)* Apply the RBF method to the configuration in
Figure S:7.3. Compare results with Project S:P7.1.
Optionally time the program execution speed for the
FDM and RBF methods to compare their performance
(exclude plotting, and use time library). Program
profiling is discussed in Section S:8.B.

.A Program listings and descriptions

Program listing 7.1: Electric field hockey ( hockey.py)

import visual as vp, numpy as np, ode

2

def hockey(Y, t): # return eqn of motion

4 accel = 0.
for i in range(len(loc )):
6 accel += Q[i]*(Y[0]−loc[i])/(vp.mag(Y[0]−loc[i]))**3
return [Y[1], q*accel] # list for non–vectorized solver
8

a, b, w = 1., 0.5, 0.125 # rink size, goal width

10 q, qcor, qmid, qcen = −1.0, 1.0, −2., 2. # Qs: puck, cor., mid, cen.
Q = [qcor, qmid, qcor, qcor, qmid, qcor, qcen] # charges, locations
12 loc = [[−a, b], [0, b], [a, b], [a, −b], [0, −b], [−a, −b], [0,0]]

14 scene = vp.display( title = ’Electric hockey’, background=(.2,.5,1))

puck = vp.sphere(pos=(−a,0,0), radius = 0.05, make_trail=True) # trail
16 rink = vp.curve(pos=loc[:−1]+[loc[0]], radius=0.01) # closed curve
goalL = vp.curve(pos=[(−a,w,0),(−a, −w,0)], color=(0,1,0), radius=.02)
 18 goalR = vp.curve(pos=[( a,w,0),( a, −w,0)], color=(0,1,0), radius=.02)
for i in range(len(loc )): # charges, red if Q>0, blue if Q<0
20 color = (1,0,0) if Q[i]>0 else (0,0,1)
vp.sphere(pos=loc[i ], radius = 0.05, color=color)
22

v, theta = input( ’enter speed, theta; eg, 2.2, 19:’) # try 2.2, 18.5
24 v, theta = min(4, v), max(1,theta)*np.pi/180. # check valid input
Y = np.array([[−a,0], [v*np.cos(theta), v*np.sin(theta )]])

26 while True:
vp.rate(200)
28 Y = ode.RK45n(hockey, Y, t=0., h=0.002)
x, y = Y[0][0], Y[0][1]
30 if (abs(x) > a or abs(y) >b):
txt = ’Goal!’ if (x > 0 and abs(y) < w) else ’Miss!’
32 vp.label (pos=(x, y+.2), text=txt, box=False)
break
34 puck.pos = Y[0]

Most of the program is to set up visual animation. The

dynamics of motion (7.1) is modeled in hockey(). The input Y
holds the position and velocity vectors of the puck. The motion
is in the x-y plane, so all vectors are two-dimensional.
Calculation of acceleration utilizes vector operations with
NumPy arrays. The velocity and acceleration vectors are
returned as a list of NumPy arrays for non-vectorized ODE
solvers.

The playing area is a rectangle bounded by |x| ≤ a and |y| ≤

b with the origin at the center of the field. After the charges and
their locations are initialized, the scene is drawn. Each charge
is represented as a sphere, including the puck which is set up to
leave a trail as it moves (line 15).

The speed and launching angle are read from the user, with
an upper speed limit and a lower cutoff for the angle. The latter
is to prevent the puck from falling into the sink hole at the
center of the field. The main loop integrates the motion with
the RK45n solver for improved accuracy. Because hockey()
returns a list of vectors, RK45n can be readily used, because
vector objects represented by NumPy arrays can be
manipulated like scalars in arithmetics (addition, subtraction,
and multiplication). The rest of the loop checks collisions with
the wall, decides if a goal has been scored, and the outcome is
labeled at the end.

Program listing 7.2: Self-consistent method for parallel plates

( laplace.py)

import matplotlib.pyplot as plt

2 from mpl_toolkits.mplot3d import Axes3D
import visual as vp, numpy as np, copy
4
def relax(V, imax=200): # find self−consistent soln
6 for i in range(imax):
V[1:−1,1:−1] = ( V[1:−1,:−2] + V[1:−1,2:] # left,
right
8 + V[:−2,1:−1] + V[2:,1:−1] )/4 # top,
bottom

V = set_boundary(V) # enforce boundary condition

10 draw_pot(V), vp.rate(1000)
 return V
12
def set_boundary(V): # set interior BC values
14 V[w:2*w, top], V[w:2*w, bot] = 1., −1. # plates voltage
return V
16
def draw_pot(V): # refresh potential, slow version
18 for i in range(M):
for j in range(N):
20 q = abs(V[i,j ]) # graduated color mix
if (V[i, j ] >0): grid. color [N*i + j] = (q,.5*q,.2*q)
22 else: grid. color [N*i + j] = (.2*q,.5*q, q)

24 def draw_efield(V, scale ): # draw electric field

Ex, Ey = np.gradient(−V)
26 Emag = np.sqrt(Ex*Ex + Ey*Ey)
for i in range(2, M−1, 2):
28 for j in range(2, N−1, 2):
vp.arrow(pos=(i,j), axis=(Ex[i,j ], Ey[i, j ]),
30 length=Emag[i,j]*scale)
vp.rate(100)
32 return Ex, Ey

34 M, N, s = 61, 61, 10 # M x N = grid dim, s = point

size
w, d, h = M//3, N//6, N//2 # plates width, separation, half
N
36 bot, top = h − d//2, h + d//2 # bottom and top plates

38 scene = vp.display(width=M*s, height=N*s, center=

(M//2,N//2))
grid = vp.points(pos=[(i,j) for i in range(M) for j in
range(N)], # grid
40 color =[(0,0,0)]*(M*N), size=s,
shape= ’square’)

42 V = np.zeros((M,N)) # initialze V on grid, apply BC

 V = set_boundary(V)
44 V = relax(V) # solve by relaxation

46 Ex, Ey = draw_efield(V, scale = 16)

V, Ex, Ey = np.transpose(V), np.transpose(Ex), np.transpose(Ey)
48 X, Y = np.meshgrid(range(M), range(N))

50 plt. figure () # Fig.1, contour plot

plt.contour(V, 14)
52 plt.quiver(X [::2,], Y [::2,], Ex [::2,], Ey [::2,], # stride 2 in y
dir
width=0.004, minshaft=1.5, minlength=0, scale=10.)
54 plt. xlabel( ’x’), plt. ylabel( ’y’)

56 plt. figure () # Fig.2, surface plot

ax = plt.subplot(111, projection= ’3d’)
58 ax. plot_surface (X, Y, V, rstride=1, cstride=1, cmap=plt.cm.jet,
lw=0)
ax. set_xlabel ( ’x’), ax. set_ylabel ( ’y’), ax. set_zlabel ( ’V’)
60 plt.show()

The code implements the self-consistent method to solve

the Laplace equation for two parallel plates held at a potential
difference. It solves the problem by iterative FDM over a grid of
M × N.

The main program sets the grid size, the parallel plates
dimensions, and visually represents the grid by square points
(line 39). It then initializes the potential array and applies the
boundary values. The boundary conditions include the two
parallel plates held at values of ±1 (in arbitrary units), and the
four sides of the box held at zero. The plates run horizontally
from M/3 to 2M/3. The vertical positions of the bottom and top
plates are (N −d)/2 respectively, with separation d. The unit
of distance is also arbitrary since it is scalable.

The heavy work is done in relax(), which attempts to find

the self-consistent solution via relaxation. It works the same
way as waves2d() in Program 6.7 with slicing as discussed in
Section 6.7. It iterates for a maximum of imax times (default
value can be changed by the caller), applying the boundary
conditions and redrawing the potential grid after each
iteration. The function draw_pot() displays the potential on the
grid as hues of orange or blue colors for positive and negative
potentials, respectively, similar to graduated colors used in
Program 5.7.

Once the solution is returned, draw_efield() is called to

calculate and draw the electric field using the NumPy gradient
function. VPython arrows representing the electric field are
scaled to their magnitudes. Contour and surface plots are also
generated. Similarly, vector electric fields are also overlaid on
the contour plot.

One note about an unfortunate source of confusion caused

by inconsistent mathematical conventions. We are used to the
normal convention of labeling a point by its coordinates (x, y),
or (i, j) on a grid, with i running horizontally and j vertically.
However, after conceptually setting up the problem this way,
when it is time to represent the potential grid as a 2D array, we
face a dilemma: V[i,j] means a vertical position i (row) and
horizontal position j (column) in the interpretation of a matrix.
This is opposite to our conceptual setup. We had this problem
in Program 4.6 for the effective potential of the restricted
three-body motion.

We believe it is important to keep conceptual clarity, so we

will set up the problem in the normal (coordinate) convention,
and do calculations accordingly. But when it comes to graphing
the results, we have to transpose the arrays first. Fortunately,
the NumPy function np.transpose eases the pain considerably.

Speaking of clarity, we chose code readability over speed in

draw_pot(), which updates the colors of the grid one point at a
time inside a double loop. This slows down the simulation
because it is called repeatedly, and explicit looping is relatively
slow in Python (Section 1.3.2). For the grid size used in this
program, draw_pot() actually uses most of the computing time,
since the core computation in relax() is already highly
efficient. This will only get worse for larger grid sizes.

Fortunately, we can eliminate the bottleneck if we wish to

retain visualization. We do so by entirely eliminating the
double loop as follows:

def draw_pot(V): # refresh potential, fast version

hue, neg = np.dstack((V, V*.5, V*.2)), V<0. # stack a 3D
array
hue[neg] = − hue[neg ][:,[2,1,0]] # swap R,B for
neg pot
 grid. color = hue.reshape(M*N,3) # reshape to
(M*N,3)

To associate each point with an RGB color code, we use a

base code (1, 0.5, 0.2) and scale it by the potential to obtain
shades of orange. First, we stack the potential grid depth-wise
(on top of each other) to form a 3D array (M×N×3), with
appropriate weighing by the base color components at each
point. Next, color codes for negative potentials, indicated by
the truth array neg, are turned positive. Their red and blue
components are also swapped at the same time to make shades
of blue. All this is done with advanced indexing (Section 1.D).
Finally, the array is reshaped to a 2D array (MN × 3) before
assignment to the grid.

Now the visualization code is as fast and efficient as the core

computation, and we can take on larger grids, at the expense of
code readability. Use the new draw_pot() in Program 7.2,
quadruple the grid size (say 128×128, also remember to halve
the point size), and run!

Program listing 7.3: Laplace equation by FEM ( laplacefem.py)

from scipy.linalg import solve

2 from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt, numpy as np
4
def mesh(L, N): # generate mesh
6 elm, bv = [], [] # elements: [[n1, n2, n3],..], bndry value
x, y = np.linspace(0,L,N+1), np.linspace(0,L,N+1) # same
x,y grids
 8 ndn = np.arange((N+1)*(N+1)).reshape(N+1,N+1) # node
number
node = [[xi, yj] for xi in x for yj in y] # nodes
10 for j in range(N):
for i in range(N):
12 elm.append([ndn[i,j], ndn[i+1,j+1], ndn[i, j+1]]) #
upper
elm.append([ndn[i,j], ndn[i+1,j], ndn[i+1,j+1]]) #
lower
14
ip = ndn[1:−1,1:−1].flatten() # internal nodes
16 bp = np.delete(ndn, ip) # boundary
nodes=all−internal
for p in bp:
18 bv.append((node[p][0]*node[p][1])**2) #
boundary values
return node, elm, bp, ip, bv, x, y
20

def fem_mat(node, elm): # fills matrix Aij = 〈∇ϕi · ∇ϕj〉

22 A = np.zeros((len(node),len(node)))
for e in elm:
24 (x1,y1), (x2,y2), (x3,y3) = node[e [0]], node[e [1]],
node[e [2]]
beta, gama = [y2−y3, y3−y1, y1−y2], [x3−x2, x1−x3,
x2−x1]
26 ar = 2*(x1*(y2−y3) + x2*(y3−y1) + x3*(y1−y2))
for i in range(3):
28 for j in range(i,3):
A[e[ i ], e[ j ]] += (beta[i]*beta[j] +
gama[i]*gama[j])/ar
30 if (i != j): A[e[j], e[i]] = A[e[i ], e[ j ]] #
symmetry
return A
32
L, N = 1.0, 40 # length of square, number of
intervals
34 node, elm, bp, ip, bv, x, y = mesh(L, N) # generate mesh
ip. sort () # sort ip, just in case

36 A, b = fem_mat(node,elm), np.zeros(len(ip)) # build matrices

for j in range(len(ip)):
38 b[ j ] = np.dot(A[ip[j ], bp], bv) # boundary
condition Eq. (7.23)

40 A = np.delete(A, bp, axis=0) # delete rows specified by bp

A = np.delete(A, bp, axis=1) # delete cols specified by bp
42 u = solve(A, −b) # solve

44 u = np.concatenate((u, bv)) # combine internal+boundary

values
all = np.concatenate((ip, bp)) # internal+boundary nodes
46 idx = np.argsort(all) # index sort nodes
u = np.take(u, idx) # now u[n] is the value at node n
48 u = np.reshape(u, (N+1, N+1)) # reshape grid for graphing
x, y = np.meshgrid(x, y)
50
plt. figure ()
52 ax = plt.subplot(111, projection= ’3d’)
ax. plot_surface (x, y, u, rstride=1, cstride=1,
54 linewidth=0, cmap=plt.cm.jet)
ax. set_xlabel ( ’x’), ax. set_ylabel ( ’y’), ax. set_zlabel ( ’V’)
56 plt. figure ()
plt.subplot(111, aspect= ’equal’)
58 plt.contour(x, y, u, 26)
plt.show()

Mesh generation is done through mesh() in accordance to

the data structure outlined in Table 7.1. We divide the
rectangular domain into N intervals in both x and y directions
(Figure 7.11). A node ni,j is a grid point at (i, j) which has the
coordinates (xi, yj), with xi = ih and yj = jh, i, j = 0, 1,…,N, and h
= 1/N being the grid spacing. The nodes, numbered from 0 to
N(N + 2), are reshaped to a (N + 1, N + 1) array, ndn. As
illustrated in Figure 7.11, the node numbers increase from left
to right horizontally then up the rows. The node number at grid
(i, j) is ni,j = j × (N + 1) + i. The (x, y) coordinates of the nodes
are stored in a nested list node (line 9) in the same order as the
node numbers.

We next build the triangular elements. Each element is a list

of three node numbers defining the triangle. The process
traverses the sub-squares (Figure 7.11) from the bottom row
moving left to right (i-loop) then going up the rows (j-loop).
For a sub-square whose lower-left corner is at (i, j), two
elements are appended to the elm list (lines 12 and 13): the
upper one (above diagonal) with nodes [ni,j, ni+1,j+1, ni,j+1], and
the lower one (below diagonal) with [ni,j, ni+1,j, ni+1,j+1].

We can obtain the internal points from the nodes ndn by

taking only the interior points via slicing, excluding points on
the four sides (line 15). The resultant nodes are flattened to a
1D array and assigned to ip. Now, the boundary points bp can
be found by deleting the internal nodes ip from all nodes ndn
(line 16). This is actually done by advanced indexing since ip is
an integer array. The array ndn is flattened first, then elements
indexed by ip are removed.

Having found bp, we can evaluate the boundary values bv

according to Eq. (7.28). Let (x, y) be the coordinates of a given
boundary point. Because it is on the boundary, the expression
x2y2 yield a zero value at the bottom and left sides, x2 at the top
(y = 1), and y2 on the right side (x = 1), so they are the correct
boundary conditions.

For other domain or boundary configurations, the details of

mesh generation may be different, but the general flow is
similar. In fact, meshes generated by other means including
Python meshing packages such as MeshPy can be used equally
well (see Programs 9.7 and 9.9).

The function fem_mat() for building the system matrix has

been explained in the text. The main program obtains the
nodes, elements and boundary conditions by calling mesh(). As
a precaution, the internal points ip are sorted (line 35) upon
return. Once we have the mesh structure, the system matrix A
is built. The boundary terms bj are computed from Eq. (7.23)
using the np.dot function for the inner product bj = Σk Ajkuk
over the boundary points k (recall the pj’s are zero for the
Laplace equation, no charges).

Next, we delete the rows and columns corresponding to the

boundary points from matrix A. This is done via the np.delete
function (line 40) which removes entries contained in bp from
the dimension specified by the axis parameter: 0 for rows and
1 for columns. Finally, the linear system is solved with solve().
 The returned solution is stored in the array u in the order of
the internal nodes after the removal of boundary nodes from
matrix A. We must be certain what internal node corresponds
to a given index in u. This is why we sort the internal nodes ( ip)
before deletion as a precaution. If they are already in sorted
order, this step would be unnecessary.

This is important as we combine the internal and boundary

values next (lines 44 to 47) for graphing (or other purposes
such as data analysis). First, both the node values and the
nodes themselves are concatenated. Since the internal ( ip) and
boundary ( bp) nodes already correspond to the right order in u
and bv arrays, respectively, the concatenated nodes in all
correlate one-to-one with the node values in the concatenated
array u. Next, we index-sort the nodes in array all using
np.argsort, which returns the indices idx pointing to the order
that would sort the array. The same idx will also sort the
concatenated array u such that the sorted node values
correspond to the sorted nodes in all. Thus, we sort u by
taking the elements in the order in idx (line 47). Now, u
contains the node values in the correct order, i.e., the value at
node n is given by u[n].

Last, we reshape the 1D array u into the shape of our 2D

grid (line 48) for surface and contour plotting at the end. If the
grid is not square, use the Matplotlib function tripcolor(),
which works well for plotting data over triangular meshes. See
Program 9.8.

Program listing 7.4: Laplace equation by RBF ( laplacerbf.py)

from scipy.linalg import solve

2 import matplotlib.pyplot as plt, numpy as np

4 def initialize (L, R, N=22): # calc nodes and boundary

values
inode, bnode, b = [], [], [] # intnl, bndry nodes, bndry
values
6 x, y = np.linspace(0,L,N+1), np.linspace(0,L,N+1) # same
x,y grids
for j in range(N+1):
8 for i in range(N+1):
r2 = (x[i]−L/2.)**2 + (y[j]−L/2.)**2
10 if (i==0 or i==N or j==0 or j==N or r2<=R*R):
bnode.append([x[i], y[j ]]) # bndry node
12 b.append(1. if r2<=R*R else 0.)
else:
14 inode.append([x[i ], y[ j ]])
ni, node = len(inode), inode+bnode # combine
nodes
16 return np.array(node), [0.]*ni+b, ni, len(node)

18 def phi(i, r ): # Gaussian ϕi and ∇2ϕi

r2, e2 = (node[i,0]−r[0])**2 + (node[i,1]−r [1])**2,
eps*eps
20 f = np.exp(−e2*r2)
return f, 4*e2*f*(e2*r2−1)

22
def rbf_mat(ni, nt ): # fills matrix or ϕij
24 A = np.zeros((nt, nt))
 for j in range(nt):
26 f, df = phi(np.arange(nt), node[j ]) # vector operation
A[j,:] = df if j < ni else f
28 return A

30 def usum(a, x, y): # calc u at (x,y)

u = 0.
32 for i in range(len(a )):
u += a[i]*phi(i, [x, y ])[0] # [0]= first return
value
34 return u

36 L, R, eps = 1.0, 0.25, 20. # sizes of square, disk; shape

parameter

38 node, b, ni, nt = initialize (L, R) # ni, nt = num. intrnl/tot

nodes
A = rbf_mat(ni, nt)
40 ai = solve(A, b) # solve

42 x = np.linspace(0, L, 41) # same x,y plotting grids

x, y = np.meshgrid(x, x)
44 u = usum(ai, x, y)

46 plt. figure ()
img = plt.imshow(u, cmap=plt.cm.jet) # plot as image
48 plt.colorbar(img), plt. axis( ’off’)
plt.show()

We solve the Laplace problem with the RBF collocation

method in Program 7.4. The program is structurally similar to
Program 7.3. The first function initialize() sets up the nodes
and boundary values, playing the role of mesh generation in
FEM. It creates a uniform node distribution along x and y
directions. The two loops iterate through the rows j and
columns i. The function keeps two lists of nodes, internal or
boundary nodes. At each node, it checks whether it is internal
or on the boundary. If a node is within a radius R of a disk at
the center of the box, or if it is on the walls of the box, it is
added to the boundary nodes list (line 11). The next line sets the
boundary value to 1 or 0 depending on its being on the disk or
the walls. Otherwise, the node is added to the internal nodes
list. For finite number of nodes, the disk is actually a polygon.
In this case, it is approximately an octagon (Figure 7.18).

At the end, the internal and boundary nodes are added to

create the total nodes list on line 15. When lists are added, the
elements are combined in the same order to form a larger list,
essentially like concatenating them. The node is returned as a
NumPy array for vector operations (line 26, see below),
together with the column vector b, which is formed by adding
to the boundary values list a number of zeros (zero charge
density) equal to the number of internal nodes. The other two
returned values are the numbers of internal and total nodes.
Note that the internal nodes come first, and the boundary
nodes second.

The next function phi() computes the Gaussian RBF value

and its derivative (Laplacian) value . It is called by
rbf_mat() to form the system matrix A. It iterates over the
rows, and for each row, it calculates all the entries in the entire
row (all columns) via vector operation (line 26). To do so, it
passes a vector list of indices in place of a single index, and
because the node is stored as a NumPy array, phi() interprets
this as an implicit element-wise vector operation. This is much
faster than going through columns one by one. The next line
fills the A matrix with the derivatives for the internal nodes,
and RBF values the for boundary nodes. The last function
usum() just calculates the interpolant (7.31) given the expansion
coefficients a. Because phi() returns a list of values, it picks the
first value in line 33, which is the RBF value.

The main code sets the parameters including the box size,
radius of the disk, and the shape parameter ε. Then it generates
the nodes and boundary values, the system matrix, and solves
the linear system to obtain the expansion coefficients a. The
potential is calculated over a grid for plotting. The result is
shown as a color-coded image via imshow() on line 47, with a
color bar and the axis (frame) turned off.

Program listing 7.5: Magnetic dipole fields ( bdipole.py)

import visual as vp, numpy as np

2 import matplotlib.pyplot as plt

4 def bdipole(r ): # calc magnetic fields at

b, rel = vp.vector (0,0,0), r−rs
6 for i in range(len(ds )): # sum over segments
b += vp.cross(ds[i ], rel [ i ])/vp.mag(rel[i ])**3 #

8 return b

10 divs, m, n = 36, 21, 16 # circle divisions, space

 grid
phi = np.linspace (0., 2*np.pi, divs+1)[:−1] # omit last
pt
12 rs = np.column_stack((np.cos(phi), np.sin(phi), 0*phi)) #
segments
ds = np.column_stack((−np.sin(phi), np.cos(phi), 0*phi)) #
tangents
14 x, z = np.linspace(−2, 2, m), np.linspace(−1.5, 1.5, n)
x, z = np.meshgrid(x, z)
16
bx, bz, big = np.zeros((n,m)), np.zeros((n,m)), 100.
18 for i in range(n):
for j in range(m):
20 b = bdipole(vp.vector(x[i, j ], 0., z[ i, j ]))
if (vp.mag(b) < big): # exclude large fields
22 bx[i, j ], bz[ i, j ] = b.x, b.z

24 plt.subplot(111, aspect= ’equal’)

plt.quiver(x, z, bx, bz, width=0.004, minshaft=1.5,
minlength=0)
26 plt. plot ([−1],[0], ’bo’, [1],[0], ’bx’) # eyes
plt.show()

The code calculates and displays the magnetic fields due to

a circular loop in the x-y plane. The function bdipole()
assumes the loop divided into a number of segments at
positions . We approximate the magnetic field at a given
location by summing over all the segments, each contributing
an amount , Figure 7.20). This is
conveniently accomplished via the vector product operations
vp.cross in VPython.
 The main program divides the current loop into divs
segments, and hence divs+1 points for the variable phi between
0 and 2π. The position vector of each segment on the unit
circle and the associated tangent vector ( ) are computed and
stored in rs and ds, respectively, as 2D arrays of vectors. These
vectors are formed by column stacking the (x, y, z) components
using the NumPy function np.column_stack (line 12), which
puts 1D arrays as columns side-by-side to make a 2D or 3D
array. Note that the tangent vector is just a rotation of the unit
vector in Eq. (4.63).

The magnetic fields are plotted using the quiver() function.

The length of the arrow indicates the relative magnitude of the
magnetic field, so that we need not worry about units (such as
the units of current or the length of the segments). The
approach works well for grid points not too close to the loop
itself. Because of the 1/r2 divergence, the small neighborhood
around the wire is treated separately. There, the magnetic
fields diverge and would dominate the graph. For this reason, a
number big is set to exclude these regions. To accurately
represent these regions, we need to plot the fields along
concentric circles around the wire as shown in Figure 7.21
because they then have comparable magnitudes.

Program listing 7.6: Magnetic fields of a current-carrying wire

( longwire.py)
 import visual as vp, numpy as np
2

m, n = 6, 30 # r, phi intervals
4 r = np.linspace(0.15, 0.4, m)
phi = np.linspace (0., 2*np.pi, n+1)
6 r, phi = np.meshgrid(r, phi) # n x m grid

8 x, y = r*np.cos(phi), r*np.sin(phi) # pos and dirs of B−fields

ax, ay = −np.sin(phi), np.cos(phi) # tangential vectors
10

scene = vp.display( title = ’Magnetic fields’, background=(.2,.5,1))

12 wire = vp.curve(pos=[(0,0,.6), (0,0, −.6)], radius = 0.01)

14 scale, q = 0.008, 1.0

for z in np.linspace (.5, −.5, 6):
16 for i in range(n):
for j in range(m):
18 vp.arrow(pos=(x[i,j ], y[ i, j ], z ), axis=(ax[i, j ], ay[ i, j ]),
length=scale/r[ i, j ], color=(q, q, 0))
20 q = q −.05

The magnitudes of the magnetic fields of a long current-

carrying wire vary as 1/r (7.43), and their directions (line 9) are
tangential to concentric cylinders around the wire. The latter
are calculated similarly to Program 7.5 with the aid of Eq.
(4.63). The color intensity variable q dims the VPython arrows
further into the page. This helps with depth perception because
as they move further back (and out), the arrows are too small to
judge from light reflection.
Program listing 7.7: Electric dipole fields ( edipole.py)

import visual as vp, numpy as np

2
r, scale, m, n = 0.5, 0.05, 11, 19 # parameters
4 scene = vp.display( title = ’Electric dipole’, background=
(.2,.5,1),
forward=(0, −1, −.5), up=(0,0,1))
6 zaxis = vp.curve(pos=[(0,0, −r),(0,0,r )])
qpos = vp.sphere(pos=(0,0,.02), radius=0.01, color=(1,0,0))
8 qneg = vp.sphere(pos=(0,0, −.02), radius=0.01, color=(0,0,1))
c1 = vp.ring(pos=(0,0,0), radius=r, axis=(0,0,1),
thickness=0.002)
10 c2 = vp.ring(pos=(0,0,0), radius=r, axis=(0,1,0),
thickness=0.002)

12 theta, phi = np.linspace(0, np.pi, m), np.linspace (0, 2*np.pi, n)

# grid
phi, theta = vp.meshgrid(phi, theta)
14 rs = r*np.sin(theta)
x, y, z = rs*np.cos(phi), rs*np.sin(phi), r*np.cos(theta) #
coord.
16 for i in range(m):
for j in range(n):
18 rvec = vp.vector(x[i, j ], y[ i, j ], z[ i, j ])
B = scale*vp.cross(rvec, vp.vector (0,0,1))/( r*r) #

20 E = vp.cross(B, rvec)/r #

vp.arrow(pos=rvec, axis=E, length=vp.mag(E), color=

(1,1,0))
22 vp.arrow(pos=rvec, axis=B, length=vp.mag(B), color=
(0,1,1))
 The program first draws the z-axis, a pair of charges
representing the dipole, and two great circles in the x-y and x-z
planes. Real work begins at line 12 which divides the sphere
into grids over the polar (θ) and azimuthal (φ) angles. The grid
is converted into Cartesian coordinates (line 15) where the field
vectors will be drawn.

The double loop calculates the electromagnetic fields from

Eq. (7.45) at a fixed time, so the factor cos(kr − ωt) is omitted
and absorbed into the scaling factor scale which also controls
the proportion of the arrows. We use VPython’s vector product
operations to simplify the calculations.

Program listing 7.8: Angular distribution of dipole radiation

( dipole.py)

1 import numpy as np
import matplotlib.pyplot as plt
3 from mpl_toolkits.mplot3d import Axes3D

5 theta = np.linspace(0, np.pi, 21)

phi = np.linspace(0, 2*np.pi, 41)
7 theta, phi = np.meshgrid(phi, theta)
x = np.sin(theta)*np.cos(phi)
9 y = np.sin(theta)*np.sin(phi)
z = np.sin(theta)*np.cos(theta) # z=radial times cos(theta)
11

plt. figure ()
13 ax = plt.subplot(111, projection= ’3d’, aspect=.5)
ax. plot_surface (x, y, z, rstride=1, cstride=1, color= ’w’)
 15 plt. axis( ’off’)
plt.show()

We first generate a grid over polar and azimuthal angles,

from which we calculate the Cartesian grids, x = sin θ cos φ and
y = sin θ sin φ, over the unit circle. The dipole radiation pattern
is a polar plot revolved around the z-axis. At a given θ, the
magnitude (radial distance) is proportional to r ~ sin θ, and as
a result, z = r cos θ ~ sin θ cos θ (line 10). The actual surface is
made with plot_surface with an aspect ratio less than one to
flatten the donut-shaped surface.

Program listing 7.9: Plane electromagnetic waves ( plane.py)

import visual as vp, numpy as np, vpm

2

t, dt, v, L = 0.0, 0.01, 0.2, 1.0 # time, speed, wavelength

4 n, scale, E, B = 61, 0.2, [], [] # grid, scale

6 scene=vp.display( title= ’Electromagnetic waves’, background=(.2,.5,1),

center=(0,0,L), forward=(−.4, −.3, −1))
8 ax, ay, az = (1,0,0), (0,1,0), (0,0,1) # axes and labels
vp.arrow(pos=(.2, −.2, L), axis=ax, length=0.2, color=(1,1,0))
10 vp.arrow(pos=(.2, −.2, L), axis=ay, length=0.2, color=(0,1,1))
vp.arrow(pos=(.2, −.2, L), axis=az, length=0.2, color=(1,1,1))
12 vp.label (pos=(.45, −.2, L), text= ’E’, box=False, height=30)
vp.label (pos=(.25, −.0, L), text= ’B’, box=False, height=30)
14 vp.label (pos=(.2, −.15, L+.3), text= ’v’, box=False, height=30)

16 idx, z = np.arange(n), np.linspace(−L, 2*L, n) # order of vectors

 mag = scale*np.sin(2*np.pi*z/L) # sine envelope

18 ewave = vp.curve(color=(1,1,0), pos=np.column_stack((mag,0*z,z)))

bwave = vp.curve(color=(0,1,1), pos=np.column_stack((0*z,mag,z)))
20 for i in idx:
E.append( vp.arrow(pos=(0, 0, z[i ]), axis=ax, length=mag[i],
22 color=(1,1,0)) )
B.append( vp.arrow(pos=(0, 0, z[i ]), axis=ay, length=mag[i],
24 color=(0,1,1)) )
while True:
26 vp.rate(100), vpm.wait(scene) # hit a key to pause
t, mg = t + dt, mag*np.cos(t) # sinusoidal wave
28 for i in range(n):
E[i ]. pos.z += v*dt # traveling wave
30 B[i ]. pos.z += v*dt
if (E[i ]. pos.z > 2*L): # wrap around
32 E[i ]. pos.z, B[i ]. pos.z = −L, −L
idx = np.insert(idx, 0, i )[:−1] # move to end
34

E[i ]. axis, B[i ]. axis = ax, ay # reset axis to

36 E[i ]. length, B[i ]. length = mg[i], mg[i] # draw correctly
id = idx[i ]
38 ewave.pos[i ] = (mg[id], 0, E[id ]. pos.z) # envelope curves
bwave.pos[i ] = (0, mg[id], B[id ]. pos.z)

After setting up the axes and labels without boxes,

sinusoidal electromagnetic field vectors are created along the z-
axis. The E vectors are in the x direction, and B vectors in the y
direction. We also place envelope curves over the tips of the
vectors. The points on the curves are created by column
stacking as in Program 7.5. In this case, the points in the E-
curve are in a (n×3) array formed by stacking E magnitudes for
x, n zeros for y, and n grid points for z. It is similar for the B
curve.

The propagation of the electromagnetic wave is simulated

by moving the vectors in the z direction and varying their
magnitudes sinusoidally. When the vectors at the wavefront
move beyond the set z limit, we assume they fall off the front
and append them to the end, effectively enforcing periodic
condition. To effect this strategy, an index variable idx is
created to maintain the order of the vectors. When they move
out of the front range, they are inserted back to the other end of
the array (line 33).

Finally, a caution on working with VPython arrows. As of

VPthon version 5.7x, when specifying arrows with a negative
length, the axis is flipped. This is a useful behavior, and we
have used it to our advantage. On the other hand, if we are
working with the assumption that they are relative to a fixed
direction, we must reset the axis to ensure this consistent
default behavior.

1
Accurate to within the discretization scheme, of course, which in the present case is of
second-order accuracy (6.89).

2
We assumed one basis function in 1D for each node (tent function, Figure 6.12).
Technically, we could split the tent into two, so we would also have two basis functions for
each element.

3
This is the reason we must reduce second-order PDEs to an integral form involving only
first derivatives.
 4
Other meshfree methods are discussed in solutions of quantum systems in Chapters 8
and 9.

5
Unlike electric fields, the basic units producing magnetic fields are magnetic dipoles.
There is at present no evidence of magnetic monopoles equivalent to electric charges.

6
The power in an electromagnetic field is given by the Poynting vector, ,
with units of power/area.
Chapter 8
Time-dependent quantum
mechanics
Can a particle be at two places simultaneously? Could a cat be
dead and alive at the same time? If we asked these questions in
previous chapters on classical physics, the answer would seem
to be obviously silly. However, in the context of quantum
physics, the answer is neither obvious nor as silly as it may
seem. Enter the fascinating world of quantum physics at the
atomic scale where classical physics breaks down, and where
few classical beings like us really have any intuitive experience.
Even now, quantum mechanics is still somewhat mysterious,
though its validity bas been confirmed by available experiments
for almost a century since its firm establishment.

The student of classical physics begins by studying motion.

To gain a feel for quantum physics, it is important that we also
begin with motion in the quantum world. What is free fall
motion like in quantum mechanics? How about a simple
harmonic oscillator? By studying, simulating, and observing
motion in quantum physics, we begin to build up intuition and
to understand the conceptual underpinning of quantum
mechanics. This may be done even without prior knowledge of
quantum mechanics, at least for Sections 8.2 and 8.3, if we
treat the Schrödinger equation that governs the quantum world
as just a wave equation analogous to wave motion discussed in
Section 6.5.

But, quantum motion is not as direct or as easy to study as

classical motion. The Schrödinger equation is a partial
differential wave equation, and its solutions are not local like
Newton's second law. Unlike deterministic Newtonian
mechanics, quantum mechanics is both deterministic and
probabilistic. As a result, quantum mechanics suffers from
what may be called the “unsolvability” problem, i.e.,
meaningful time-dependent quantum systems that are
analytically solvable are few and far in between.

This is where numerical simulations are particularly

valuable to reveal aspects of quantum mechanics that cannot
be easily perceived. We will describe several methods suitable
for simulating time-dependent quantum systems. We begin
with a direct method which converts the Schrödinger equation
into a system of ODEs, and use it to study the quantum motion
of a simple harmonic oscillator. This is the most direct and
straightforward approach. Next, we discuss a split-operator
method to deal with motion in open space such as free fall. We
also introduce a two-state model to study transitions and to
illustrate the concept of coherent states. Finally, we briefly
discuss quantum waves in 2D and quantum revival.
.1 Time-dependent Schrödinger
equation
In quantum mechanics the notion that a particle's position and
velocity can be simultaneously determined is abandoned.
Instead, owing to particle-wave duality, a particle is described
by the wave function, ψ(x, t) [44]. The interpretation of the
wave function is such that |ψ|2dx is the probability of finding
the particle in space from x to x + dx at time t. For normalized
wave function, the probability over all space is unity,

The wave function evolves in time according to the time-

dependent Schrödinger equation (TDSE)

Here, V (x, t) is the potential, ħ the rationalized Planck

constant, and m the mass of the particle. The TDSE is
fundamental to quantum mechanics as Newton's second law is
to classical mechanics.

Compared to waves on a string (6.62), the TDSE for matter

waves is first order in time and second order (same) in space.1
Furthermore, it contains the unit imaginary number ,
making the time-dependent wave function necessarily complex.
 Except for a few trivial interaction potentials, no analytic
solutions exist for the wave function ψ(x, t). The simplest case
is the motion of a free particle, V (x, t) = 0, whose solution is
called a plane wave, ψ(x, t) = A exp(i(kx−ωt)). It is similar to
plane electromagnetic waves (Figure 7.25), where k is the wave
vector and ω = ħk2/2m. The wave function at different times
are shown in Figure 8.1 (animated in Program 8.1).

Figure 8.1: The wave function (solid curves: real and imaginary
parts) and probability density (dashed) of a free particle. Dotted line:
a wavefront.

The real and imaginary parts of the wave function are

sinusoidal waves, phase shifted by . However, the probability
density is constant (a bit of oddity already).2 A plane wave has
a well-defined momentum (ħk) and no uncertainty, Δp = 0.
According to Heisenberg's uncertainty principle ΔpΔx ≥ ħ/2, it
has an infinite uncertainty in position, i.e., no beginning or
ending. This gives rise to the constant probability density over
all space. It is a useful idealization of a free particle that does
not exist in reality.
 In most cases when the potential is nonzero, Eq. (8.2) must
be solved numerically using appropriate computational
methods. In such cases, a sensical unit system should be used
as discussed below.

ATOMIC UNITS
Given the small constants such as ħ in the TDSE, it would be
cumbersome to use SI units in actual calculations. For
quantum systems in atomic, molecular, or solid state physics, a
natural unit system is the atomic units (a.u.). It is based on the
Bohr model of the hydrogen atom.

In the a.u. system, the units for mass and charge are the
electron mass me and basic charge e, respectively. The length is
measured in Bohr radius a0, and energy is chosen equal to the
magnitude of the potential energy of the electron at the radius
a0 in the Bohr atom (Hartree). They can be most conveniently
expressed in terms of the universal fine structure constant α
and speed of light c as

Given the chosen units for mass, charge, length, and energy,
all other units can be derived from these four. Their
relationship and numerical values (in SI) are summarized in
Table 8.1.
 Table 8.1: The atomic units (a.u.) system.

We can express the TDSE in a.u. for an electron as (Exercise

E8.1)

Comparing Eqs. (8.2) and (8.4), we see that the net effect in
switching to atomic units is dropping all constants, equivalent
to setting ħ = me = e = 1 in the equations.3 In the rest of this
chapter, we will use atomic units unless otherwise noted. We
also assume the particles to be electrons.

.2 Direct simulation
We first consider a direct method for solving the TDSE (and
other time-dependent PDEs alike) by converting it to a set of
ODEs.4 This will enable us to start simulating quantum
systems with minimum delay using standard ODE solvers.
8.2.1 SPACE DISCRETIZED LEAPFROG
METHOD
We discretize space into grid points xj = jh, where h is the grid
size, and j = 0, 1, …, N. The second-order differential operator
∂2/∂x2 is approximated the same way as before, Eq. (6.73a),
such that

Let ψj(t) = ψ(jh, t) and Vj(t) = V (jh, t) as usual, we obtain

the space discretized TDSE by substituting Eq. (8.5) into (8.4),

This is a set of N + 1 ODEs, to be solved given the initial wave

function over space, ψj(0). This way, the solution to the TDSE
is reduced an initial value problem.

Before we can integrate Eq. (8.6), we need to consider the

fact the Schrödinger equation preserves the normalization
(8.1), so an ODE solver that preserves the norm (or flux) should
be used. We have discussed in Chapter 2 that the leapfrog
method has this special property. It is our method of choice
here.

To rearrange Eq. (8.6) suitable for leapfrog integration, we

separate the wave function into its real and imaginary parts, Rj
and Ij, respectively. Let
Substitution of Eq. (8.7) into (8.6) yields (Exercise E8.2)

We have assumed that the potential is real, as physical

potentials are.

Besides being real, Eqs. (8.8a) and (8.8b) are in the proper
form for leapfrog integration. The derivative of the real part
depends only on the imaginary part, and vice versa. If we
regard Rj as the general “coordinates”, and Ij as the general
“velocities”, these equations mirror exactly the pair of
equations (2.44) and (2.45), with Rj corresponding to rj and Ij
to vj.

Let us express Eqs. (8.8a) and (8.8b) in a more compact

matrix form.

Let
The matrix A is tridiagonal, so it is a band, sparse matrix.

In terms of these matrices, Eqs. (8.8a) and (8.8b) become

Note the opposite signs in front of the matrix A. This result is

formally equivalent to Eqs. (2.39) and (2.40) in that the change
of the “velocity” vector I depends on the “acceleration”, −AR,
which is a function of “position” vector R only, like in a
Hamiltonian system. In fact, the analogy goes further: Eq.
(8.10) describes a general “harmonic oscillator” in the same
way Eqs. (2.39) and (2.40) do a simple harmonic oscillator,
recalling a(x) = −ω2x in the latter. Consequently, we can
formally view RTR + ITI as a general energy, and conservation
of energy in a Hamiltonian system requires that

which is a statement on the conservation of probability (8.1).

We can prove this result analytically (Exercise E8.2).

Accordingly, we can formally port over the leapfrog method

from Eqs. (2.41) and (2.42) to
where Δt is the step size. Equation (8.12) is the space
discretized leapfrog (SDLF) method. It advances the solution
one step forward, from (R0, I0) to (R1, I1), and more
importantly, preserves the norm.

When the initial wave function is given at t = 0, we can

march forward to obtain the wave function at later times. But,
with what step size? As we saw earlier (Section 6.5.4), we also
need to check the stability of an algorithm whenever finite
differencing is involved in time stepping. A similar von
Neumann stability analysis can be carried out for the SDLF
(Exercise E8.3), and the stability criterion is

where 0 < λ < 1 is a coefficient dependent on the potential V.

For free particle motion (V = 0), λ = 1. In general, the greater
the potential, the smaller the λ. From experimentation, λ =
should work for most cases.

Recall that for waves on a string, the stability condition

(6.81) is Δt ∝ h. Why is Δt ∝ h2 in the present case? As
mentioned earlier, the Schrödinger equation is first order in
time, whilst the wave equation is second order in time. This
accounts for the h2 dependence mathematically. Physically, the
Schrödinger equation is similar to the diffusion equation. In
diffusion, typified by Brownian motion such as a drop of ink in
water, the particles spread out at a rate 〈x2〉 ∝ t (see Section
10.3 and Eq. (10.40)). Equation (8.13) requires that the time
step must be such that a particle does not exceed the physically
allowed diffusion distance.

8.2.2 IMPLEMENTATION
We have everything in place to start implementing the direct
simulation method. We need to write a derivative function,
similar to Program 4.3, appropriate to the leapfrog integrator.
First we declare two arrays, R and I, to hold the real and
imaginary parts of the wave function over the spatial grid. They
will be initialized to the wave function at t = 0, and stepped
forward according to Eqs. (8.8a) to (8.8b).

It seems straightforward enough to program this.

Depending on whether dRj/dt or dIj/dt is needed, it requires
only one loop to sum up the three terms in each equation. But
(there is always a but, isn't there? this but is different this
time), consider this: each loop needs about N operations, and
since the grid size is h ~ N−1, the time step Δt required for
stable integration scales as Δt ∝ h2 ~ N−2. It means to advance
the solution over some time interval (say 1 unit), the number of
operations scales like Nop ~ N3. Suppose N = 1000, then Nop ~
109. This is large and taxing for an interpretive language like
Python.

Thus, we need to make the above mentioned loop as

efficient as possible. Fortunately, as in previous situations (see
Sections 6.5.5, 6.7, or 7.2.1), the NumPy library makes it easy.5
There are two approaches. The first is to use fast matrix
multiplication, realizing that A in Eq. (8.12) is tridiagonal and
banded. The second is to use element-wise operations with
array slicing to avoid explicit looping as in Program 6.4. We
choose the second approach here for it is simpler and just as
fast.

Observing from Eq. (8.8a), for instance, dRj/dt depends on

Ij and its left and right neighbors, Ij−1 and Ij+1, respectively. We
can thus obtain dR/dt at once by adding three arrays (up to
some scaling constants) as: I, I right-shifted by one, and I left-
shifted by one. This is the same method as in 1D waves
discussed in Section 6.5.5 and illustrated in Figure 6.15.

The end points can be resolved assuming a given boundary

condition. We will use periodic boundary conditions. Consider
the actual system consisting of N + 1 grid points with ψj, j = 0,
1, …, N. Imagine identical copies of the system are placed next
to each other along the x-axis. If we step past the right end
point of the (actual) system, we will be at the first (left-most)
point of the (virtual) system to the right. Likewise, if we go the
other way and move past the left end point, we will be at the
right-most point of the system to the left. We can express the
periodic boundary condition as

With Eq. (8.14), the actual derivative function, sch_eqn(), is

given below.

1 def sch_eqn(id, R, I, t): # return dR/dt, dI/dt

of Sch. eqn.
b = 0.5/(h*h)
3 if (id == 0): # gen. velocity
tmp = −b*I # 1st/3rd term in Eq.
(8.8a)
5 dydt = (2*b + V)*I # 2nd term
else: # gen. acceleration
7 tmp = b*R # 1st/3rd term in Eq.
(8.8b)
dydt = (−2*b − V)*R # 2nd term
9
dydt[1:−1] += tmp[:−2] + tmp[2:] # add bψj−1, bψj+1 to
grid j
11 dydt[0] += tmp[−1] + tmp[1] # 1st point, periodic
BC
dydt[−1] += tmp[−2] + tmp[0] # last point
13 return dydt

It returns dR/dt if id is zero, else dI/dt. In each case, a

temporary array tmp is used to hold the first and the third
terms of Eq. (8.8a) or (8.8b) before shifting, and the middle
term is stored in dydt. The potential array V is built in the main
program (Program 8.2). Then, the interior elements are added
via slicing discussed above. Note tmp[:-2] implies all elements
from the beginning up to (but not including) the second last,
and tmp[2:] from the third to the end. The end points are
treated separately according to the periodic boundary
condition (8.14), which means that the left neighbor of the first
point is the same as the last point, tmp[−1] (line 11), and the
right neighbor of the last point is the same as the first point,
tmp[0].

Similarly, we can use the vectorized leapfrog code (Program

2.6) which succinctly expresses the algorithm (8.12). In
addition, it has the advantage of speed gain. Each of the three
steps in the leapfrog method is an implicit loop, and can be
computed much faster for large arrays, as is the case here.
Profiling (Section S:8.B) shows that we gain about 50x speedup
compared to explicit looping for N = 500.

8.2.3 QUANTUM HARMONIC MOTION

 Figure 8.2: The probability density (top), the real and imaginary parts
of the wave function (middle and bottom, respectively), in the SHO
potential.

The complete code is given in Program 8.2, which yields the

results below (slightly modified for plotting, Project P8.1).
Figure 8.2 displays the motion of an electron in the simple
harmonic oscillator (SHO) potential (2.46), . The
initial wave function is a normalized Gaussian wavepacket,
The parameters x0 and σ are the center and width of the
Gaussian, respectively. The variable k0 is the average
momentum (at t = 0) of the wavepacket.

The results shown in Figure 8.2 are for , x0 = −3 and k0

= 0. The spatial range is set to −10 ≤ x ≤ 10, divided into N =
500 intervals (for a total of 501 grid points), so h = 0.04 and
. This value of N gives us good balance between
accuracy and speed. The top graph shows the probability
distribution at five equidistant locations. The potential is also
drawn for easy reference. The wavepacket moves to the right,
accelerates and broadens as it approaches the center. Past the
center, it starts to slow down and becomes narrower until the
right end point where the wavepacket looks nearly identical to
the original one (refocusing). Afterward, it reverses direction,
and moves toward the starting position.

The middle and bottom graphs in Figure 8.2 show the real
and imaginary parts, R and I respectively, of the wave function
at three positions (left end, center, and right end). Initially,
there is only the real part. With increasing t, both parts
develop, and become more oscillatory toward the center. The
peaks of R and the nodes of I coincide with each other (vice
versa), making the probability density smooth throughout.
Surprisingly, the real part does not vanish at the right end,
where the imaginary part dominates. We would not have
suspected this behavior based on the appearance of the
probability density graph alone.
 The evolution of the probability density is shown in Figure
8.3 as a function of space and time simultaneously. We can see
more clearly from the surface plot the periodic motion of the
wavepacket as it zigzags back and forth in time. The width of
the wavepacket also oscillates periodically. We can observe in
the contour plot the periodicities of both the probability density
and the width more clearly. The period is close to 2π, which
matches very well with the classical period of the SHO, 2π/ω
from Eq. (6.11), when the mass and spring constant are unity.
We can also understand these oscillations as a result of
quantum revival, discussed in Section 8.5.

Based on the results so far, we should expect that the

average position, i.e., the expectation value of position 〈x〉, to
give a quantitative depiction of the periodic motion. Given the
wave function ψ(x, t), we can calculate the expectation value for
position as

For momentum p and kinetic energy T, respectively, we have

 Figure 8.3: The surface and contour plots of probability density.

The last expressions in Eqs. (8.17) and (8.18) are useful for
numerical calculations (see Exercise E8.5 and Project P8.2). For
time-dependent systems, the above expectation values are generally a
function of t.
 We need to evaluate the integral (8.16) numerically. Like solving
ODEs, numerical integration is a common procedure in scientific
computing. We discuss several integration techniques in Section 8.A.

Figure 8.4: The expectation value of position (solid line) and the
maximum of the probability density (dashed line).

Since the wave function is discretized over a regular grid, it is

simplest to use Simpson's rule (Project P8.1) to evaluate Eq. (8.16).
The expectation value 〈x〉 as a function of time is graphed in Figure
8.4. As expected, it indeed follows a sinusoidal pattern like the
classical SHO (2.47). From visual inspection, we infer 〈x〉 −A cos t.

The quantity 〈x〉 provides a bridge between classical and quantum

mechanics through the Ehrenfest theorem, which describes the
dynamics of physical observables governed by a set of relationships
very similar to classical equations of motion. For instance, the
expectation values of position and momentum satisfy (Exercise E8.4)
We see that this is just like Newton's second law with a force given
by F = −〈∂V/∂x〉 (see Eq. (2.46)).

In our case, , thus F = −〈x〉. Substituting it into Eq. (8.19)

yields 〈x〉 ~ −A cos t, just as observed earlier. Therefore, we can
numerically confirm Eq. (8.19), m〈 〉 = F.6 So classical mechanics is
recovered for space-averaged observables.

2
Figure 8.5: The normalization error, 1 −∫ |ψ| dx.

Also shown in Figure 8.4 is the value of maximum probability

density (right y axis) as a function of time. The maximum value,
Pmax, refers to the peak of the distribution, which is approximately at
the center of the wavepacket. We can think of Pmax as a measure of
how quickly the wavepacket broadens since it is inversely
proportional to the width (roughly). Pmax starts from the peak value at
t = 0 when the Gaussian is at its narrowest. It reaches the lowest
 value at the moment the center of the Gaussian is at x = 0, where the
wavepacket is broadest, before recovering its peak value at the right
end. The cycle repeats afterwards, so the period of Pmax is half the
period of the SHO. Because the SDLF method preserves the norm,
the peak values of Pmax stays constant.

We can keep track of normalization continuously by computing

Eq. (8.1) at regular time intervals. Figure 8.5 shows the deviation of
normalization from unity, i.e., 1 − ∫ |ψ|dx. The absolute error is on the
order of 10−6 given the same grid size h and step size Δt as in Figure
8.2. The absolute error is less significant than its oscillatory behavior:
it shows that the error is bounded, much like what we saw for the
energy of the SHO using the leapfrog method (Figure 2.7). Here we
see again that the leapfrog method, being finite order and symplectic,
exactly preserves a numerically inexact result. Overall, the SDLF is a
direct, robust method for solving the TDSE.

It is interesting to observe the absolute error exhibiting a period

equal to half of the SHO's period, just like Pmax. Correlating with
Figure 8.4, we see that the error is largest when the wavepacket is at
the end points when it is narrowest. A sharper wave function requires
a denser grid to maintain accuracy. Making h or Δt smaller can help
reduce the magnitude of the error. A more effective method is to
make our numerical scheme unitary (norm preserving) to begin with,
an approach we take next.

.3 Free fall, the quantum way

The SDLF is an explicit method that marches forward given the
initial wave function. It is direct and robust, provided that the
stability condition (8.13) is fulfilled, which is also its major
limitation. In this section, we discuss a first order split-operator
method, simply referred to as the split-operator method below, which
removes this limitation. The method is particularly useful for motion
in open space such as scattering problems or unidirectional motion
like free fall. We apply it to simulate the free fall of a quantum
particle.

8.3.1 THE SPLIT-OPERATOR METHOD

To develop the split-operator method, we rewrite Eq. (8.4) more
concisely utilizing the Hamiltonian H,

where we assume for the moment that the potential is time-

independent.

Formally, we can write the solution to Eq. (8.20) as [80]

It is understood that the function of an operator is defined through

its Taylor series
If the effect of is small, we can retain the first few terms for
accurate representation of the operator function.

The operator e−iH(t−t0) is called the evolution operator for it

advances the wave function from t0 to t. The evolution operator is
unitary, that is, it preserves the normalization at all time, ∫|ψ(x, t)|2 dx
= ∫|ψ(x, t0)|2 dx (see Exercise E8.6). Since its exact solution is
unknown, we must approximate it.

Let Δt = t − t0, ψ0 = ψ(x, t0), and ψ1 = ψ(x, t0 + Δt). We have from

Eq. (8.21)

Assuming Δt is small, we expand e−iHΔt according to Eq. (8.22), and

to first order we have

This is Euler's method for the TDSE. But, this form turns out to be
numerically unusable, because it is nonunitary and unstable no matter
how small Δt is (Exercise E8.3).

The split-operator method, also known as the Crank-Nicolson

method originally applied to heat conduction problems [21], takes a
symmetric approach by splitting the exponential operator in halves,
and applying the first-order truncation to each half as
It can be shown that this result is accurate to first order in Δt and
agrees with the Euler method (8.24) (Exercise E8.6). However,
though an approximation, Eq. (8.25) is unitary,

Physically, it means that the split operator preserves the

normalization of the wave function regardless of the size of Δt.

Substituting Eq. (8.25) into (8.23) and acting on both sides by the
operator in the denominator, we obtain

To look at this relationship another way, we have

If we were to replace ψ1 on the RHS by ψ0, we would recover Euler's

method (8.24). The effect of the split-operator method is to replace
ψ0 in Euler's method by the average (ψ1+ψ0)/2. This little trick makes
a big difference.

To construct a numerical scheme, we discretize the Hamiltonian

operator in Eq. (8.27) the same way as in SDLF over the spatial grid
xj = jh. Let . After some algebra
(Exercise E8.7), the final result is
 Unlike the explicit SDLF method (8.12), the split-operator
method is an implicit method: the value depends not only on
previous values at t0, but on the current values at t as well.
Rather than simply marching forward, we have to invert Eq. (8.29)
for all values of simultaneously at each step. This is the price we
pay, in exchange for absolute stability guaranteed in Eq. (8.29)
(Exercise E8.8).

Equation (8.29) had first been solved using clever recursion

techniques [37]. We use a much simpler matrix method. Let us
assume that, for the time interval of interest, the wave function at the
boundaries are so small as to be negligible, ψ0 = ψN = 0 (fixed ends).
This can be interpreted in two ways. One is that the spatial range is
so large that the particle (wavepacket) is far away from either end of
the box. The other is that the boundaries are hard walls (infinite
potentials) so the particle does not penetrate them. We will revisit this
point shortly in discussing the results (Section 8.3.2).

Let
and assuming fixed-end boundary conditions, we can express Eq.
(8.29) in matrix form as

Equation (8.31) is the final result of the split-operator (Crank-

Nicolson) method in matrix formulation. Given the wave function u0
at t0, we obtain the new wave function u1 at t + Δt by solving a linear
system of equations. Because we must do this at each step, it would
be very costly in general. However, we are helped by two facts: the
matrix A is a tridiagonal, band matrix, and it is time-independent (if
V = V (x)). The tridiagonal nature of A is due to approximating ∂2/∂x2
by a three-point scheme (8.5).

We can use special matrix solvers designed for band matrices in

the SciPy library, which reduce the number of operations linearly in
N rather than N2 in general. To use band matrix solvers, we first need
to express the matrix A in the required format.
 Figure 8.6: A tridiagonal matrix (5×5) and its band representation
(3×5).

Figure 8.6 illustrates an example of representing a tridiagonal

matrix as a band matrix. Our original matrix (A, source) has a
maximum of three nonzero elements in each column. The band
representation (B, target) is chosen to be a matrix with three rows and
the same number of columns. The resultant matrix is just the band of
the original matrix being flattened. The first and the last elements of
the target, B11 and B35 respectively (•), are dummy fillers not used in
actual computation, so they can be set arbitrarily. For general band
matrices, we just need to increase the number of rows to represent it
properly. If u is the number of elements above the diagonal, then Aij
= Bi−j+u+1,j, i ≤ j. For instance, we have u = 1 for the tridiagonal
matrix, such that A34 = B14 = 1, and A44 = B24 = 4, etc.

The actual implementation of the split-operator method is

straightforward. It consists of six lines shown below, taken from the
full Program 8.3.
 1 A = np.ones((3, N+1), dtype=complex) # prepare band matrix A, B
A[1,:] = 2*(2j*h*h/dt − 1 − h*h*V)
3 dB = − np.conjugate(A[1,:]) # diagonal of B
......
5 C = dB*psi # prepare RHS Eq. (8.31)
C[1:−1] −= (psi[:−2] + psi [2:])
7 psi = solve_banded((1,1), A, C) # band matrix solver

The first line creates a 3 × (N + 1) matrix, data-typed complex and

filled with ones initially. The second line replaces the second row
with the complex diagonal elements from Eq. (8.30) using vector
operation, completing the band representation of matrix A. The third
line creates a 1D array containing the diagonal of matrix B in Eq.
(8.30) only, which is the negative complex conjugate of the diagonal
of A.

Next, we prepare the RHS of Eq. (8.31). Rather than direct matrix
multiplication, we obtain it from the RHS of Eq. (8.29) because
SciPy does not yet have a banded matrix multiplication routine. Line
5 computes the first term on the RHS of Eq. (8.29), i.e., the diagonal
elements of Bu0 in Eq. (8.31). This is why we did not need the full
matrix B above. The next line subtracts the last two terms, the left
and right neighbors, respectively, using the same shift-slicing
technique as before (Section 8.2.2). For fixed ends, we neglect the
boundary points, since . Now, we have completed the
RHS of Eq. (8.31).
 The last line (line 7) calls the SciPy band matrix solver,
solve_banded(), to obtain the solution ψ1 at t + Δt. The function
requires a minimum of three parameters as

solve_banded((l,u), A, b)

It solves a linear system, Ax = b, assuming A is in band matrix

format. The (l,u) pair specifies the lower and upper diagonals, i.e.,
the numbers of elements below and above the diagonal in the band,
respectively, such that the band width is l+u+1. For our tridiagonal
matrix, we have (l,u)=(1,1) in line 7.

8.3.2 QUANTUM FREE FALL

As a test case, we apply the split-operator method to the free fall of a
quantum particle. We choose the linear potential to be

The parameter a plays the same role as the gravitational acceleration

in free fall, mgy. Such a scenario exists for electrons in a constant
electric field.
 Figure 8.7: A quantum particle in free fall: probability density |ψ(x,
2
t)| .

Figure 8.7 shows the results from Program 8.3. The initial wave
function is again a Gaussian (8.15), and the acceleration is a = 2. The
center of the wavepacket falls toward the negative x direction as
expected. It also broadens as before, but unlike the SHO potential,
there is no refocusing effect in the linear potential, and the
wavepacket just keeps broadening.

Since a wavepacket is made up of different momentum

components which spread out at different speeds, the broadening is
irreversible if there is no refocusing force to restore the motion. The
SHO potential provided a restoring force causing the wavepacket to
narrow and broaden periodically (Section 8.2.3). The linear potential,
however, yields a constant force that accelerates different parts of the
wavepacket equally, and the spreading continues unabated.

Another difference between the linear and SHO potentials is that

free fall motion is unidirectional and has no lower bound. As a result,
when waves inevitably reach the finite boundaries in our simulation,
it will be reflected due to our assumption that the boundary values
are small, ψ0 = ψN 0. We have seen similar reflections on a
vibrating string (Section 6.5.5). This causes the leading edge of the
waves to interfere with its self reflection. The growing ripples we see
in the last three frames (Figure 8.7) are self-interference patterns.
These patterns are clearly an artifact of our domain being finite, and
not real physical effects. In infinite space, the wavepacket would just
keep spreading, so the amplitude would become ever smaller while
its width ever larger. This cautions us that, in interpreting simulation
results or comparing with observations such as experimental data, we
should consider boundary effects carefully, and take steps to
minimize them if possible. Conversely, as stated earlier, we can
interpret the results as free fall between hard walls, in which case the
interference patterns are real and due to actual reflections.

We note that the split-operator method as implemented initially

preserves the normalization (8.1) to a high degree of accuracy. As
time increases, error starts to grow due to waves reaching the
boundaries. The leading waves have higher momentum so they are
highly oscillatory as seen in Figure 8.7. To maintain accuracy,
smaller grids are needed (see Project S:P8.1). A more accurate,
second-order split evolution operator (SEO) method is developed in
Section S:8.1.
 Figure 8.8 shows a contour plot of the probability density as a
function of time using the same parameters as in Figure 8.7. In
addition to broadening and self-interference discussed above, we see
the center of the wavepacket following a parabolic curve (before
reaching the boundary). The classical free fall motion is given by Eq.
(2.2), which in our case is x = 5−t2 (a = 2). This is drawn as the
dashed line in Figure 8.8. We see good agreement and an interesting
connection between quantum waves and classical trajectory motion,
which is a another clear demonstration of the Ehrenfest theorem
(8.19). A quantum wave moves along the classical path of a particle
in terms of average position and force (expectation values).

Figure 8.8: The probability density of a quantum particle in free fall.

The dashed line shows the classical path.
8.3.3 MOMENTUM SPACE
REPRESENTATION
From the motion of the wavepacket in coordinate space above, we
have learned a good deal about the momentum (velocity) of the
particle. Better yet, quantum mechanics allows us to get an entirely
different, but equivalent, view from the wave function in momentum
space. In a.u., the momentum is equal to the wavevector ħk = k.

The wave function in momentum space, ϕ(k, t), and in coordinate

space, ψ(x, t), are related by a pair of reciprocal Fourier transforms
(5.34) and (5.35),

It is sometimes convenient to use a shorthand notation for these

Fourier transforms, , and (we have
omitted t in the shorthand to denote the actual variables x and k).

Just like |ψ|2 giving the probability in position space, |ϕ(k)|2 gives
the probability of finding the particle with momentum k. And ϕ(k) is
normalized if ψ(x) is as well, i.e., ∫|ϕ(k, t)|2dk = ∫|ψ(x, t)|2dx = 1.

Since our wave function ψ(x, t) is discrete over the space grid, it
is efficient and straightforward to evaluate Eq. (8.33a) with the FFT
method (if you have not done so already, now would be a good time
to review Section S:5.B).
 Suppose we sample M points at equal intervals δx = h over the
spatial range (i.e., period) d = Mh. From Eqs. (S:5.18) and (S:5.20),
and according to Section S:5.B.3, the momentum grid points are
determined by

where δk = 2π/d is the interval in momentum. The range of

momentum is q = Mδk = 2π/h, again in accordance with Eq. (S:5.21).
Like δk, the spatial interval is related to q reciprocally by δx = 2π/q.
To use FFT, we make the number of grid points M a power of 2, i.e.,
M = 2L, L = integer. By our convention (Program 8.3), if N is the
number of intervals, then N = M − 1.

In Eq. (8.34), we are interpreting the Fourier components as made

up of positive and negative momentum values. Because the transform
has a period M, we have ϕ(kj) = ϕ(kj+M). Even though all momentum
values can be treated as being positive mathematically, we require
half of them to be negative from physical point of view. The FFT
storage scheme for the components is outlined in Table S:5.1.

The momentum distribution is presented as a profile on the k-axis

in Figure 8.9 and as a density contour plot in Figure 8.10 in terms of
k and t (Project P8.4). The results are obtained from a slight
modification of Program 8.3 where line 37 is replaced by the
following,
 phi = np.array(fft. fft_rec (psi, L)) # FFT
phi = np.concatenate((phi[M//2:], phi[:M//2])) # re−order neg to pos k
pb = phi.real**2 + phi.imag**2

The parameter L is equal to 10, so M = 1024 and N = 1023. After

taking the FFT using the recursive function fft_rec() from our
library, the array phi[] contains the resultant momentum wave
function. The second line pulls the second half of the array in front of
the first half so it is in the order of increasing k per Eq. (8.34) after
concatenation (see Eqs. (S:5.41) and (S:5.42)).
 Figure 8.9: A quantum particle in free fall: momentum profile |ϕ(k,
2
t)| .

Initially the momentum profile is a Gaussian (in k space), since

the Fourier transform of a Gaussian is another Gaussian.
Furthermore, the widths in coordinate and momentum spaces are
related by the uncertainty principle (do not confuse Δx and Δk with
sampling intervals δx and δk)

In the present case, we had Δx ~ 2σ = 1, so Δk should be on the order

of one. This may be assessed from Figure 8.10. If we increase σ, Δk
will decrease proportionally, and vice versa.

Both figures show that, at the beginning, the profile keeps its
initial shape with no apparent broadening, in stark contrast to
position space where broadening happens rather quickly. Imagine the
wavepacket was in free space. Since there would be no force, the
momentum as well as the kinetic energy of the particle must be
conserved, and we would expect no spreading at all.
 Figure 8.10: The momentum density of a quantum particle in free
fall. The dashed line shows the classical relation v = v0 − at (a = 2).

In free fall, there is a constant acceleration in the negative

direction. As a result, we see the entire momentum profile moving in
that direction, following the classical linear relation, v = v0 − at
(Figure 8.10), just like its classical path (Figure 8.8). As time
increases, the leading edge reaches the boundary, equivalent to
hitting a hard wall. Part of the wave is being reflected and self-
interference occurs as discussed earlier. The momentum distribution
begins to deform from a perfect Gaussian, and smaller structures
develop on top of the Gaussian. Toward the end at t ~ 3, we see a
spike at a positive momentum k ~ 7. In large part the spike is due to
the acceleration of the center-left portion of the wavepacket reaching
that speed, and being reflected by the wall.
 The center of the momentum is zero initially, but since Δk ~ 1,
there is a significant portion of the distribution near k ~ ±1.
Classically, the leading edge (k = v0 ~ −1) at t ~ 3 would have a
displacement of (Figure 8.8), and a velocity of v =
v0 − at ~ −7. Its position, assuming an initial position x0 = 5 and a
spread of ±1, would be roughly x ~ −7 to −9, just reaching the wall
located at −10. Upon colliding with the wall, the velocity would be
reversed, giving rise to the spike at k = −v ~ 7.

As seen earlier, this effect is due to our domain being finite. If we

interpret the physical boundaries as actual hard walls, the particle
would bounce up and down, with self-interference making it look
less and less like a localized classical particle with time. Interestingly,
if we evolve the system long enough until the so-called revival time
(8.55), nearly all the distribution will be contained in the spike, and
the cycle repeats.

.4 Two-state systems and Rabi flopping

In the last two sections we discussed quantum motion under time-
independent potentials. There are situations where a quantum system
is subject to a time-dependent potential, causing it to change states, a
process called quantum transition. For instance, if a hydrogen atom is
exposed to an oscillating laser field, the electron can jump to an
excited state. When a system interacts with an external field, it is an
open system.
 We are interested in quantum transitions in this section. For
simplicity and clarity, we will consider only two-state systems here,
and will keep the model brief. A full, unlimited model with a worked
example is developed in Section S:8.2, which should be reviewed for
more details.

8.4.1 TWO-STATE MODEL FOR TRANSITION

Let H0 be the unperturbed Hamiltonian of the system in the absence
of the laser field, and u1 and u2 be its two states, called eigenstates
(like eigenmodes in Chapter 6; see Chapter 9 for eigenstate
problems). The solutions of the unperturbed system satisfy

where E1 and E2 are the eigenenergies of states 1 and 2, respectively.

We assume u1, u2 and E1, E2 are known, e.g., they may be the up or
down states of a spin, or the lowest two states of a particle in a box
(S:8.39). The eigenstates are also orthonormal, i.e., (integration limits
omitted)

We prepare the system so that the particle is in state 1 at t = 0,

and then turn on the laser. The particle will respond to the laser force,
and start mixing with state 2. We expect the wave function of the
particle at t, ψ(x, t), to be a superposition of the two states as
where a1(t) and a2(t) are the expansion coefficients (generally
complex). Note that we have included the exponential time factors of
stationary states in Eq. (8.38).

The interpretation of the coefficients is that the probabilities of

finding the particle in states 1 or 2 are given by |a1(t)|2 and |a2(t)|2,
respectively. They are also called occupation (or transition)
probabilities. Because the total probability is conserved, we have

To solve for a1(t) and a2(t), we substitute ψ(x, t) of Eq. (8.38) into
the Schrödinger equation iħ∂ψ/∂t = (H0 + V (x, t))ψ, where V (x, t)
represents the interaction with the laser field (see Eq. (S:8.40)). After
some algebra and using Eq. (8.36) for cancellations (see Section
S:8.2), we obtain

The notation 1,2 represents derivative with respect to time.

Equation (8.40) is valid for all space and time. To extract the
coefficients, we project it to the eigenstates u1 and u2. For instance,
we can multiply both sides of Eq. (8.40) by , and integrate over x.
Using the orthogonality relation (8.37), the 2 term drops out. The
same can be done with . The final result after projection is
where ω = (E2−E1)/ħ is the transition frequency, and the matrix
elements Vmn are defined as

Equation (8.41) is the central result of the two-state model (a subset

of Eq. (S:8.33)). It can be integrated if the laser field V (x, t) is given.

8.4.2 RABI FLOPPING

Figure 8.11: State population for a laser duration τ = 600. The dotted
2
line shows a shifted Rabi flopping function cos (ΩRt + φ), Eq.
(8.49a).

As an example, we carry out a calculation for a particle in the box

(infinite potential well) with Program S:8.1 which integrates Eq.
(8.41) with the RK4 method. The ground and the first excited states
are included (N = 2), and the laser field interaction is given by Eqs.
(S:8.40) and (S:8.41). The laser has a center frequency equal to the
transition frequency between the two states, and a pulse duration of τ
= 600 a.u. The rest of the parameters are kept the same as in Figure
S:8.8. The results are shown in Figure 8.11.

The system starts out in state 1, its population P1 decreases

continuously to very nearly zero. At the same time, the population P2
of state 2 increases in such a way that the net loss in P1 becomes the
net gain in P2. The two states oscillate back and forth between
practically 0 and 1. This behavior may be understood in a simplified
picture known as Rabi flopping.

Rabi flopping is an important phenomenon in externally driven

two-state systems. To avoid unnecessary clutter, we assume a
monochromatic laser field tuned to (resonant with) the transition
frequency,

This form of the field will help us see more clearly the core idea of
Rabi flopping. In numerical calculations such as Figure 8.11 we still
use realistically enveloped pulses like Eq. (S:8.41).

For the particle in a box, the transition matrix elements can be

written explicitly from Eq. (S:8.44) as

where V0 = −16eaF0/9π2 (e is electron charge, a box width, F0 field

amplitude). The diagonal matrix elements being zero (or close to
zero) is a common property in dipole transitions of many systems.

Substituting Eq. (8.44) into (8.41) gives us

Equation (8.45) is analytically solvable, but the solutions may be

simplified with a useful approximation. Over a time period long
compared to 1/ωL, the factors exp(±2iωLt) in Eq. (8.45) will undergo
many oscillations. Their net average values will be close to zero, i.e.,
〈exp(±2iωt)〉 0. We can then drop them from Eq. (8.45). This is
called the “rotating wave approximation” (RWA) [64]. Within RWA,
we arrive at two simplified equations

Differentiating both sides of Eq. (8.46), we obtain the following

equivalent second-order differential equations,

The quantity ΩR is the Rabi frequency. It depends on the

interaction strength only, not on the laser frequency.

Equation (8.47) is a pair of well-known SHO equations whose

solutions, subject to the initial condition a1(0) = 1 and a2(0) = 0, are
 The occupation probabilities of the two states at time t are

Clearly, P1(t) + P2(t) = 1, as required by Eq. (8.39). The two states

flip-flop back and forth with the Rabi frequency ΩR. This is Rabi
flopping [74]. The frequency can be controlled through the laser field
strength V0. Rabi flopping occurs in many situations, such as nuclear
magnetic resonance, quantum computing and optics. It is one of the
most important ideas in quantum optics.

Going back to Figure 8.11, we can understand the oscillations

between 0 and 1 as a result of Rabi flopping. Recall that the results
are for a realistic pulse, Rabi flopping will be significant only after
the pulse has ramped up. In a complete Rabi cycle the two states are
populated and depopulated completely and complementarily. We
have superimposed P1(t) from Eq. (8.49a) in Figure 8.11 (dotted line)
to simulate the oscillation for an idealized, infinitely long,
monochromatic laser. It is phase shifted to account for the ramping
up stage. We can see that during the middle part of the laser there is
good agreement between numerical results and the ideal solutions.

Although no simple analytic solutions for Rabi flopping are

known with an enveloped laser pulse described by Eq. (S:8.41), our
numerical simulations are straightforward and the results demonstrate
the universal nature of the important phenomenon (see Project S:P8.9
for Rabi flopping in hydrogen atoms). The oscillation frequency is
dependent on the laser strength, and independent of laser frequency.
The longer or stronger the pulse, the more cycles of Rabi oscillations.
The laser frequency is responsible for the small variations (step
structures) as discussed in Section S:8.2.2.

8.4.3 SUPERPOSITION AND THE

MEASUREMENT PROBLEM
In Rabi flopping (8.48), we have a system whose wave function ψ is
a superposition of two states ϕ1 and ϕ2,

where ϕ1,2 = u1,2 exp(−iE1,2t/ħ) (see Eq. (8.38)). At any given time,
the wave function is a coherent sum of the two states, i.e., both states
exist simultaneously within ψ. The probabilities of finding the states
are cos2(ΩRt) and sin2(ΩRt), respectively. Quantum mechanics
prescribes exactly, and deterministically, how the probabilities
change with time.

How do we realize the probabilities? Measurement, of course.

When we make a measurement, we will find the system in either
state 1 or 2, not both. But quantum mechanics never tells us which
state we will get, only the probabilities. The system would evolve
just fine without any measurement. A measurement seems to force
the system to reveal itself. How does it happen? What state was the
system in just before we made the measurement? If we regard the
 two states as a cat being dead or alive, we have the well-known
Schrödinger's cat problem.

Because the action of measurement is not modeled in quantum

mechanics, the answers to the above questions are unknowable
within the confines of quantum mechanics. Essentially, this is the
measurement problem, and is open to interpretation. A commonly
accepted view is the Copenhagen interpretation which holds that the
system ceases to be a coherent superposition of states and collapses
to one state or the other when a measurement is made. To put it
another way, measurement destroys the system. Even though this
does not quite answer how the system stops being a superposition of
states, it is an answer. In fact, this is one of the intriguing aspects of
quantum mechanics, and searching for an answer can get quite
philosophical, and metaphysical sometimes. Fortunately, it does not
hinder our ability to do quantum mechanics, and least of all, to
simulate quantum systems.

.5 Quantum waves in 2D
As seen from above, we can create a superposition of states, or a
coherent state, with external perturbations like laser excitation. After
the external interaction is turned off, the coherent state will continue
to evolve but with different characteristics than single stationary
states. The numerical methods we discussed above also enable us to
study coherent states in higher dimensions. Below we will explore an
example in 2D and examine the evolution in both coordinate and
momentum spaces.
 Let us consider a coherent state prepared at t = 0 in a 2D box
(infinite potential well) as

Figure 8.12: Probability density in a 2D well. The side length is a =

5.3 Å.
Figure 8.13: Momentum density in a 2D well. The kx or ky scale is
[−4, 4].
This is a normalized compact wave function that is nonzero only
within a rectangular region σx × σy centered at (x0, y0).

The state will evolve similarly to the expansion (8.38), which in

2D reads

The basis functions um and un are the same as Eq. (S:8.39), and the
eigenenergies are Emn = Em + En, the sum of individual eigenenergies
Em and En from Eq. (S:8.39). We can determine the amplitudes amn
from the wave function at t = 0 as (see Exercise E8.9)

A time sequence of probability densities |ψ|2 is shown in Figure

8.12. The calculations are done with Program 8.4. We choose a
square well of size a = 10, and a symmetric coherent state with σx =
σy = 2 placed at the center of the well. This is a relatively narrow
wavepacket, and to adequately represent the state, we include N = 20
basis states in the expansion in each dimension, so the total number
of states is N2 = 400.

Initially the wavepacket broadens as expected (Figure 8.12, top

row). Starting t = 1.4, we see interference effects on the outer edges
of the distribution from reflected waves from the wall. Increasingly
intricate interference patterns are developed at later times (t = 2 and
3), showing the level of complexity a quantum wave is capable of
exhibiting. You should play with Program 8.4, e.g., setting the initial
wavepacket off center, looking at the imaginary part, etc. We had
seen 2D classical waves on a membrane (Section 6.7) also showing
similarly interesting interference patterns (Figures 6.18 and 6.19).
But quantum waves can display more entangled wave pockets
(Figure 8.12, middle row) at a comparable level of energy, because
the time-dependent wave functions are complex.

As time increases further, the number of pockets decreases and

the wave function starts to reassemble itself into the original state at t
= tR ~ 63.7. Thereafter, it repeats as if time had been reset
periodically at tR, e.g., ψ(t = 0.5) = ψ(t = 64.2), etc. This phenomenon
is known as quantum revival, and tR the revival time (see below).

The corresponding momentum probability density |ϕ(kx, ky, t)|2 is

shown in Figure 8.13. To obtain the momentum wave function, we
calculate the 2D FFT of ψ(x, y, t) by inserting the following line in
Program 8.4

phi = np.fft. fft2 (wf)

The results show that the momentum distribution changes little in

the beginning, t = 0 and 0.5. When the leading edge of the wave
comes in contact with the wall, the momentum distribution begins to
change substantially due to the “external” forces from the wall. The
large momentum components on the perimeter (t =1.4 to 40) are
significantly well separated from the initial distribution. All these are
in accord with the 1D case (Figure 8.9).

What is new, however, is the clear reciprocal relationship

between the position and momentum spaces depicted in Figures 8.12
and 8.13. They are essentially optical diffraction or interference
patterns of each other [8]. For instance, we can think of the initial
wave function at t = 0 in position space as a single aperture. The
diffraction pattern through a single aperture would correspond to the
momentum distribution. The smaller the aperture, the wider the
diffraction spot. At t = 40, we can think of the four peaks as small
“holes” spaced-out in position space (Figure 8.12, bottom left). The
resulting interference pattern in momentum space has rows of small
dots stacked over each other, forming a larger square. The size of the
square is related to the size of the holes in position space, and the size
of the dots to the spacing between the holes. For example, if the size
of the holes is reduced, the square would grow; and if the spacing
between the holes is reduced, the dots would get bigger. Conversely,
we can run the argument backward from momentum space to
position space, and arrive at similar conclusions.

Of course, interference of waves can be mathematically

represented as Fourier transforms. Evolution of 2D quantum waves
in Figures 8.12 and 8.13 graphically describes these relationships
very clearly. As in position space, we observe quantum revival in the
momentum distribution in the last two frames in Figure 8.13 (t = 63.7
and 64.2).
 In the above example, the initial interference occurs primarily
between the expanding wave and the reflected wave off the walls. We
look in the next example at intrawave interference. We construct the
initial state by a superposition of two wavepackets that have the same
form as Eq. (8.51) but centered at different locations (Project P8.6).
Shown in Figure 8.14 are two localized wavepackets initially placed
at (4, 4) and (6, 6), respectively. The corresponding momentum
density is given in Figure 8.15.

In position space, the waves merge and intrawave interference is

significant up to t = 1. Because no external forces are involved at this
stage, the momentum distribution remains substantially the same and
little visible changes are seen between t = 0 and 1. Starting from t =
1.3, collision and reflection from the walls cause the wave function in
both spaces to form highly entangled pockets, as was seen earlier.
 Figure 8.14: Probability density in 2D well of two localized
wavepackets.

Note that in the beginning, the wave function in position

space is along the upper-left to lower-right diagonal, but in
momentum space the distribution is in bands oriented from the
lower-left to upper-right direction, such that the two
distributions are approximately perpendicular to each other.
The reason can be understood in analogy to optical diffraction
alluded to earlier. The lengths and widths of the bands, as well
as the fine structures within each band, are related to the size
and separation of wavepackets in position space.

QUANTUM REVIVAL
A coherent state of an isolated, finite quantum system has an
interesting property: periodic revival of its wave function. We
have seen an example of this from Figures 8.12 and 8.13. Let us
consider a coherent state expanded in the basis functions for a
particle in a box (S:8.39)

Figure 8.15: Momentum density corresponding to Figure 8.14.

where E1 = π2ħ2/2mea2 is the ground state energy.

We see that the exponential factors are periodic if E1t/ħ =

2πj, j = 1, 2, 3, etc. Taking the smallest multiple, we define the
revival time as
The wave function is periodic in tR, ψ(t + tR) = ψ(t). For an
electron in a box with width a = 10, the period is tR = 127.3 a.u.

The period observed from the 2D box (Figure 8.12) is half

this value, 63.7. The ground state energy in 2D is twice that in
1D, so the revival time is halved according to Eq. (8.55).

Classically, a particle with energy E (or ) in a

box would bounce from wall to wall periodically at

This classical result depends on the energy, as we intuitively

expect. However, the quantal result (8.55) does not show any
dependence on the energy of the coherent state at all. It is
determined solely by the system parameters. The two results
seem inconsistent. Worse, they also seem incompatible with
the Bohr-Sommerfeld correspondence principle which states
that the behavior of a quantal system should approach its
classical description in the limit of large quantum numbers
(semiclassical limit).

This suggests a deeper connection than the wave function

alone. Though the wave function revives periodically, physical
observables do not necessarily vary with the same revival
period. In fact, we have seen hints to a possible resolution of
this incompatibility problem in the connection between the
expectation value of position and the classical motion in the
SHO (Figure 8.4).

To be concrete, let us assume only two states are included,

The expectation value of position is given by

We see that 〈x〉 oscillates with a period 2πħ/ |Em − En|, which is
different than the revival period (8.55). What is more, the
energies of the states enter the equation. It shows that the
oscillation period is inversely proportional to the difference of
eigenenergies, not the eigenenergies themselves.7

Let us take the semiclassical limit, m, n 1 but keep m = n

+ 1, the oscillation period reduces to

where we have used En = n2π2ħ2/2mea2. This semiclassical

result agrees with the classical prediction, Eq. (8.56). In this
way, we have resolved the apparent incompatibility in the
correspondence principle. This oscillation period tsc, rather
than the revival time tR, should be compared between classical
and quantal predictions.
 In general, we can construct a coherent state in the
semiclassical limit by including a number of eigenstates
clustered around n0 1 [87]. Then, Eq. (8.59) can be
generalized to

The period tsc can be much smaller than tR. We leave further
exploration of quantum revival to Project P8.7.

Chapter summary

We studied numerical simulations of time-dependent quantum

systems including dynamic evolution of closed systems and
quantum transitions in open systems. We presented several
efficient and easy-to-use methods appropriate for solving the
TDSE, breaking down the traditional unsolvability barrier in
quantum mechanics where very few meaningful time-
dependent problems are analytically solvable. This enabled us
to learn by doing quantum mechanics rather than by just
studying it.

The easiest way to get started is the explicit SDLF method,

Program 8.2. This method converts the TDSE into a familiar
system of ODEs, and we only need to change the initialization
for different energies, potentials, etc. It is norm preserving and
robust, provided the time step satisfies the stability criterion.
 The drawback is that this limits its speed for very accurate
calculations. It is the recommended method because it is
simple to understand and program, easy to implement
boundary conditions, and requires only knowledge of ODE
solvers.

Alternatively, we can use the first order split-operator

method (Crank-Nicolson) which is fast and stable. But it is
implicit and requires a banded matrix solver for speed. It is
suitable for short-time evolution or weak potentials.

We discussed quantum transitions in external fields within

a two-state model. We can obtain state occupation probabilities
directly, which oscillate between the two states in a
superposition. We also studied coherent states in position and
momentum spaces, including quantum revival. In addition, we
discussed general numerical integration techniques and
banded matrix handling.

.6 Exercises and Projects

EXERCISES
E8.1 (a) Check the atomic unit system in Table 8.1, evaluate
the numerical values for the fine structure constant α, the
Bohr radius a0, and the Hartree E0, in Eq. (8.3). Also,

prove the identity . (b) Let us choose a

 unit system: x0, m0, E0, and t0 for length, mass, energy,
and time, respectively. Express the variables as x = x′x0,
m = m′m0, t = t′t0, and V = E0V′. Show that, if the units
are chosen to be the atomic units in Table 8.1, the TDSE
(8.2) becomes in a.u.

For electrons, m′ = 1, and the above equation reduces to

Eq. (8.4).

E8.2 (a) Fill in the steps from Eq. (8.6) to Eqs. (8.8a) and
(8.8b). (b) Prove Eq. (8.11) by explicit differentiation.
E8.3 Analyze the stability condition of the space discretized
TDSE. Assume the potential is zero in this exercise.

(a) Apply Euler's method to Eq. (8.6), and carry out a

von Neumann stability analysis. Show that there is no
stable step size in general.

(b) Repeat the above with the leapfrog method to find the
stability criterion for the SDLF.

E8.4 The dynamical evolution of an observable O is given by

the Ehrenfest theorem [44]
 where [O,H] is the commutator between the operator O
and the Hamiltonian H. Using Eq. (8.62), verify Eqs.
(8.17) and (8.19).

E8.5 Decompose the wave function into real and imaginary

parts as ψ(x, t) = R(x, t) + iI(x, t). Show that:

(a) the expectation value of momentum from Eq. (8.17)

is

(b) the expectation value of kinetic energy (8.18) is

similarly

(c) the probability current defined as

is
 The results show that if the wave function is purely real
or imaginary aside from an overall multiplication
constant, then 〈p〉 = 0 and J = 0. Stationary states are in
this category. For nontrivial time-dependent processes
we are dealing with in this chapter, they are nonzero.

E8.6 (a) Let U = e−iH(t−t0) be the evolution operator. Show that

U is unitary, i.e., the product with its adjoint is unity,
U†U = 1.

Given U and Eq. (8.21), show that ∫ |ψ(x, t)|2 dx = ∫ |ψ(x,

t0)|2 dx.

(b) Expand the denominator in Eq. (8.25) in a Taylor

series, and show that the split operator agrees with e−iHΔt
to first order.

E8.7 Using Eq. (8.5), discretize the split operators in Eq.

(8.27) to prove Eq. (8.29).
E8.8 Conduct a von Neumann stability analysis of the split-
operator result (8.29). Show that it is stable regardless of
the time step.
E8.9 Given the initial wave function (8.51), prove the
expansion coefficients (8.53).

PROJECTS
P8.1 Explore the evolution of a wavepacket in the SHO
potential, and use Program 8.2 as the base program. In
this and other projects below, assume the wavepacket
describes an electron.

(a) Generate the spatial and temporal probability density

shown in Figure 8.3. To prepare data, declare two empty
lists before the main loop, say ta and Z. Inside the loop,
periodically append the probability density array (over
the spatial grid) to Z and time to ta. This is one slice of
the probability density at a given time. The above
procedure should be the same as in Program 8.3.
Terminate the loop after certain amount of time, e.g., t =
8. Adjust the frequency of sampling so that roughly 50
slices of data are recorded. After exiting the loop, the
data Z is a list of arrays, i.e., a two-dimensional array.

Prepare the space-time grid using X, Y =

np.meshgrid(x, ta) where x is the spatial grid. Make a

surface plot as usual (see Programs 8.3 or 7.2, for
example). You can exclude the end regions |x| 1 where

the probability density is small via index slicing (you can

safely cut off a quarter from each end without affecting
the appearance of the plot). This reduces the amount of
unnecessary data to be plotted. Also use rstride or
cstride parameters to control the number of meshes in
the surface plot for efficiency and easy manipulation.

With the same data, reproduce the contour plot in Figure

8.3. Also try plt.imshow() to change the appearance of
the contour plot to an image (see Program 8.4).

(b) Calculate and graph the expectation value of position

〈x〉 as a function of time. Let pb be the probability density
array at t. An easy way to approximate 〈x〉 is to convert
the integral (8.16) into a sum with the NumPy sum()
function,

xexp = h*np.sum(x*pb)

Here, h is the grid size. The expression np.sum(x*pb) is

equivalent to ∑ xi|ψi|2. Note that xexp is a number
(scalar). Put this line inside the main loop, append xexp
to a list, and plot it at the end. If everything works
correctly, you should see a graph like Figure 8.4.

Do a more accurate calculation of 〈x〉 using Simpson's

rule with

import integral as itg

......
xexp = itg.simpson(x*pd, h)

Compare your results with the above approximation.

P8.2 We will calculate the average momentum in this project.
We have seen the oscillatory motion of a wavepacket in
the SHO potential. Classically, both position and velocity
oscillate complementarily. Even though we cannot
specify position and velocity simultaneously in quantum
mechanics, we examine the expectation values to see if
they exhibit similar behavior.

(a) Predict the momentum as a function of time,

assuming the same initial condition as in Figure 8.3.
Sketch your 〈p〉 vs. t curve, and briefly describe your
reasoning.

(b) Calculate the average momentum 〈p〉 from either the

modified code from Project P8.1 or Program 8.2. Use the
results from Exercise E8.5, and carry out the integral by
Simpson's rule. Evaluate the derivatives in the integrand
using the central difference formula (6.31a).

Plot the results for time up to 20 a.u. Discuss them and

compare with your prediction. Estimate the period from
the graph. What function best describes the oscillation?
Are they consistent with the average position 〈x〉 shown
in Figure 8.4?

(c)* Study the uncertainty relations between position and

momentum. Let and

. Calculate ΔxΔp and plot the results

 to t = 20 a.u. What are the maximum and minimum
values of the uncertainty? Does it ever violate the
uncertainty principle?

P8.3 Simulate the motion of a wavepacket in a rigid box, V =

0 if 0 < x < a and V = ∞ outside the box. Use the SDLF
method (Program 8.2), and set zero values ψ(0, t) = ψ(a,
t) = 0 for the boundary conditions. This may be done
either by setting the derivatives at the end points zero in
sch_eqn(), or by requiring R0 = RN = I0 = IN = 0 after
each iteration. Choose a reasonable set of parameters for
the width of the box and the Gaussian wavepacket.

Suggested values are a = 5 and . Place the center of

the initial wavepacket away from the walls so that the

wave function is negligible at the boundaries, say x0 = 1
to 2 for the suggested a and σ.

(a) Give the initial wavepacket zero average momentum

(k0 = 0). Run the simulation with animation on, and
observe the wavepacket spreading and colliding with the
walls. Summarize your observations, and briefly explain.
Wait long enough to see the wavepacket approximately
coming back together, i.e., revival of the wave function.
You can speed up the process if you set N proportional to
a, keeping the grid size h ~ 0.02 to 0.04. Add a
vp.label() so the time t is displayed on the screen. Note

the revival time.

(b) Repeat the simulation, but give the wavepacket a

nonzero initial momentum, k0 = nπ/a, with n = 1 to 3.
 Turn off animation, and modify the program to plot the
probability density as a contour plot like Figure 8.3. In
addition, calculate and plot the expectation value of
position as a function of time (refer to Project P8.1 for a
procedure on doing so). Generate a pair of plots for each
k0. The time should be long enough so you can clearly
identify a couple of periods in the probability density
plot.

Discuss your findings in terms of quantum revival. How

do your results depend on energy ? What do you

expect from classical motion of a particle trapped

between two walls? How are the quantal and classical
results different?

P8.4 (a) For free fall, compute the expectation value of

momentum as a function of time with the split-operator
method from Program 8.3. Use the results of Exercise
E8.5 and follow the procedure outlined in Project P8.2.
Plot the results.

(b) Generate the momentum density plot of Figure 8.10.

Modify Program 8.3 to use the FFT algorithm discussed
in Section 8.3.2 with the NumPy version, np.fft(), for
speed. Compare the center of the momentum distribution
with results above.

(c)* Obtain the expectation value of momentum as a

function of time directly from the momentum wave
 function of part (b). How do your results compare with
those from part (a)? Plot the difference between them and
briefly explain.

P8.5 Consider the absolute linear potential, V = a|x|, shown in

Figure 8.16.

Figure 8.16: The wedged linear potential.

(a) Solve the motion classically for a given energy E.

Find the position as a function of time and determine the
period.

(b) Study the motion of a wavepacket with the split-

operator method. Choose a reasonable a ~ 2. Place the
wavepacket like in Program 8.3 at x0 ~ 5 from origin,
plot the probability density distribution (contour plots)
and the expectation value of position as a function of
time for a few periods. Discuss your results and compare
with classical motion at the same energy E ~ a|x0|.
 (c)* Repeat the above with the SDLF method. Use a
conservative λ in Eq. (8.13) for stability. Follow a similar
approach to the SHO potential. Discuss and compare
with results above.

P8.6 In this project, let us further explore our quantum quilt,

the ever changing patterns of quantum waves in 2D.

(a) Modify Program 8.4 to calculate and plot the wave

function in momentum space. Use 2D FFT to transform
the wave function from coordinate to momentum space
(Section 8.5). Make sure to reorder momentum as
negative and positive components. Also note that most of
the momentum distribution is concentrated near (kx, ky) =
(0, 0). Cut off large |kx,y| values via slicing. You may
wish to test the 2D FFT on some simple functions to
increase familiarity with reordering. You should see
pictures as shown in Figure 8.13.

(b) Explore intrawave interference by constructing the

initial state from two wavepackets (8.51) centered at
different locations. First assume they are identical. Note
that the expansion coefficients are additive from the two
wavepackets. Simulate the motion, you should see
patterns shown in Figures 8.14 and 8.15.

Next, suppose the wavepacket is narrower in one

direction, say σx = 0.9 and σy = 1.8. Predict what the
 evolution would look like. In particular, predict the
momentum distribution before strong interference sets in.

Run the simulation, first with animation on to observe

the evolution process. Then, turn off animation, increase
the number of grid points in order to produce accurate
snapshots of the wave function in momentum space.
Comment on your results.

P8.7 Construct a wavepacket in the infinite potential well as

follows,

where un(x) are the eigenstates (S:8.39). Set N = 20, m =

5, and choose equal weights, an = (2m + 1)−1/2. Evolve
the state according to Eq. (8.38) or (S:8.23). Plot the
average position, and optionally the probability density,
as a function of time. What is the average energy? What
is the revival time?

Next change the mixing coefficients, e.g., an =

(−1)n(2m+1)−1/2 and repeat the calculation. How did the
results change?

P8.8 The symmetric anharmonic potential is given by V =

V0(x4 − ax2). The potential, graphed in Figure 8.17, is
double-welled.
Simulate the motion of a Gaussian wavepacket in the
potential. Use your method of choice (including the SEO
method, Section S:8.1). Assume V0 = 1 and a = 4 for the
potential. Place the wavepacket in one of the wells, e.g.,

the left well at and set . Describe what

motion is expected before running the simulation.

Because of the steep rise of the potential for large |x|, the
wavepacket is limited to a compact spatial range. For the
suggested parameters above, it is sufficient to set the
range to −5 ≤ x ≤ 5 in the actual simulation.

Figure 8.17: The anharmonic potential.

Discuss the actual motion. Does the wavepacket stay in

one well, or vacillate between the wells? Produce
probability density or average position plots, explain
their features. Do they show quantum revival?
.A Numerical integration
The need for numerical evaluation of integrals occurs
frequently in scientific computing. In certain cases, computing
integrals accurately, especially oscillatory or multiple integrals,
is the core operation. It is important to understand the basic
technique of numerical integration so we can choose or tailor a
technique to balance speed and accuracy.

The basic problem of numerical integration is, given a

known integrand f(x), find I as

Most numerical techniques are geared toward finding the

approximate area under the curve, shown in Figure 8.18, as
efficiently as possible. In other words, we want to use the
fewest number of function evaluations to achieve the maximum
possible accuracy.
 Figure 8.18: Numerical integration of f(x) by finding the area under
the curve at discrete intervals.

8.A.1 TRAPEZOID RULE

To begin, let us divide the range [a, b] into N equidistant
intervals of width h. As usual, let

If we connect two neighboring points, say (x0, f0) and (x1,

f1), by a straight line, we obtain a trapezoid. Within this
interval, we can use linear interpolation (3.51) for f(x),

Then the integral in the first interval of Figure 8.18 is

Summing up all N intervals, we have the total integral

This is the trapezoid rule, exact for a linear function. The local
error in Eq. (8.66) is O(h3), and the overall error is O(h2), or
O(1/N2).
 Except for the end points f0 and fN that have a coefficient ,
the interior points appear twice in the sum (8.67), so their
coefficients are 1. We can rewrite Eq. (8.67) a little differently
as (dropping the error)

This way, the integral in reduced to a sum of function values at

abscissa xi with weight wi.

8.A.2 SIMPSON'S RULE

We can improve the accuracy by using a three-point quadratic
interpolation, e.g., between x0 and x2,

The integral between x0 and x2 becomes

Assuming even N and summing up all pairs of intervals, we

obtain the total integral
This is Simpson's rule, with error scaling O(1/N4). It is exact up
to cubic polynomials.

Similar to Eq. (8.69), we present Simpson's rule in the

abscissa-weight form (8.68) with weights

Simpson's rule performs adequately for sufficiently smooth

integrands. It is useful for piece-wise continuous functions. If
we are integrating over discrete data points, Simpson's rule
should be used. We give an implementation in Program 8.5 as
part of our numerical integration library.

Let our test integral be sin x dx = 2. If N = 2, we have

, , and f = [0, 1, 0]. Calling Simpson's rule with

I = integral.simpson(f, h)

returns a value 2.0944, an error of ~ 5%. Doubling N gives a

result 2.0046, an error of ~ 0.2%.

To further improve the accuracy, we could increase N, or

use a higher order interpolation. For instance, the Boole's rule
uses five points interpolation,
which can be readily cast into the abscissa-weight form like Eq.
(8.68). However, a better alternative is discussed next.

8.A.3 GAUSSIAN INTEGRATION

If we view numerical integration as a process of finding optimal
weights, then it effectively is an optimization problem. For both
the trapezoid and Simpson's rules, the weights wi are obtained
at equidistant abscissas xi.

Equidistant abscissas are reasonable, but their locations

represent un-used degrees of freedom in the optimization
process. What if the abscissas are allowed to float? As it turns
out, doing so can double the order of accuracy with the same
number of nonequidistant abscissas compared to equidistant
ones.

The Gaussian integration scheme uses nonequidistant

abscissas and associated weights. The well known Gaussian-
Legendre method takes the zeros of the Legendre polynomials
as the abscissas. The corresponding weights can be found by
table lookup or calculation for a given order n (see Program
8.6). The integral can be obtained from a shifted abscissa-
weight sum (see Eq. (S:8.55), Section S:8.A).
 For instance, the abscissas and weights for n = 3 are
, . Evaluating the sum (S:8.55) with
these x and w values, we obtain for our test integral sin(x) dx
2.0014, an error ~ 0.07%. To obtain a similar level of
accuracy with Simpson's rule, we need to roughly double the
number of abscissas.

In Program 8.5, we give a Gaussian integration routine for n

= 36. The routine gauss(f,a,b) may be used as a general
purpose integrator. It is sufficiently accurate, provided the
integrand f is smooth enough and the range [a, b] is not too
great. Furthermore, it can handle integrable singularities at the
end points. The routine needs a user-supplied integrand
function. The following illustrates its usage for our test integral.

import integral as itg, numpy as np

def f(x):
return np.sin(x)

I = itg.gauss(f, 0, np.pi)
print (I)

The returned integral is 2, practically exact within machine

precision. The test integral is smooth. In general, we will not be
so lucky. The theory of Gaussian integration is discussed in
Section S:8.A.
8.A.4 INTEGRATION BY DIFFERENTIATION
AND MULTIPLE INTEGRALS
Let us write the integral (8.63) as

It means that

We know how to integrate a single ODE, Eq. (8.76), by RK4

for instance. Thus, we start with the initial condition y(a) = 0
at t = a (not 0), and integrate to t = b, at which point we have
found our integral, I = y(b). This method is of low efficiency.
But it may be useful in some special situations.

Often we need to do double, triple, or higher multiple

integrals. For instance, a double integral looks like

We can treat the integral as two nested, single integrals,

Each single integral can be done using Gaussian integration,

where x is a parameter in the second integral.
 We can do up to 4D or perhaps 5D multiple integrals this
way, depending on the integrand. Higher dimensional integrals
would require Nd number of points, where N is the abscissa
and d the dimension, making it impractical for higher d. In
such cases, the integral should be done by Monte Carlo
methods (see Section 10.4).

.B Program listings and descriptions

Program listing 8.1: A quantum plane wave ( qmplane.py)

1 import matplotlib.pyplot as plt, numpy as np

import matplotlib.animation as am
3
def updatefig(*args): # args[0] = frame
5 wf = np.exp(1j*(x−0.1*args[0])) # traveling plane
wave
w = [prob, wf.real, wf.imag]
7 for i in range(len(w)):
plot [ i ]. set_data(x, w[i]) # update data
9
x = np.linspace(−2*np.pi, 2*np.pi, 101)
11 wf, prob = np.exp(1j*x), [1]*len(x)
fig = plt.figure ()
13 plot = plt.plot(x,prob, x,wf.real, x,wf.imag) # plot
object
ani = am.FuncAnimation(fig, updatefig, interval=10) #
animate
15 plt.ylim(−1.2, 1.2), plt.show()

The program animates a traveling plane wave given by ψ =

ei(kx−ωt), with k = ω = 1 in this case. It uses complex numbers
like in Program 5.7, and the identity |ψ|2 = 1 for the probability
density.

Three line objects are created and stored in the list plot
(line 13). The Matplotlib animation function (line 14) calls
updatefig() at regular intervals (in ms) to update the figure,
passing the frame number in the arguments by default. In
updatefig(), we scale the frame number as time, and update
the (x, y) data set for each of the lines in plot.

Program listing 8.2: Space discretized leapfrog method ( sdlf.py)

1 import numpy as np, visual as vp, ode, vpm

3 def sch_eqn(id, R, I, t): # return dR/dt, dI/dt of

Sch. eqn.
b = 0.5/(h*h)
5 if (id == 0): # gen. velocity
tmp = −b*I # 1st/3rd term in Eq.
(8.8a)
7 dydt = (2*b + V)*I # 2nd term
else: # gen. acceleration
9 tmp = b*R # 1st/3rd term in Eq.
(8.8b)
dydt = (−2*b − V)*R # 2nd term
11
dydt[1:−1] += tmp[:−2] + tmp[2:] # add bψj−1, bψj+1 to
grid j
13 dydt[0] += tmp[−1] + tmp[1] # 1st point, periodic
BC
dydt[−1] += tmp[−2] + tmp[0] # last point
15 return dydt
17 def gaussian(s, x0, x): # normalized Gaussian,
s=width, x0=center
c = 1.0/np.sqrt(s*np.sqrt(np.pi))
19 return c*np.exp(−((x−x0)/s)**2/2)

21 def initialize (a, b, N): # set parameters

x = np.linspace(a, b, N+1) # grid points
23 V = 0.5*x*x # SHO potential
s, x0 = 0.5, −5.0 # width (sigma), center of
gaussian
25 R, I = gaussian(s, x0, x), np.zeros(N+1) # real, imag
w.f.
return x[1]−x[0], x,V,R,I # grid size, h=x[1]−x[0]
27
a, b, N = −10., 10., 500 # space range [a,b], num.
intervals
29 z, mag = np.zeros(N+1), 4 # zeros, magnifying factor
31 h, x, V, R, I = initialize (a, b, N) #
initialization
scene = vp.display(background=(1,1,1), ambient=1) # set
scene
33 bars = vpm.bars(x, z, z, z, h, 0.05, (1,0,1)) # wave
function
line = vpm.line(x, z, z, (1,0,1), 0.02)
35 pot = vpm.line(x, V*0.1, z, (0,0,0), 0.02) #
potential line

37 t, ic, cycle = 0.0, 0, 20 # t, animation cycle

dt = h*h*0.5 # time step, dt < h*h
39 while True:
R, I = ode.leapfrog(sch_eqn, R, I, t, dt) # main
work
41 ic += 1
if (ic % cycle == 0): # animate
43 pb = R*R + I*I # probability
line.move(x, mag*pb, z)
45 bars.move(x, z, z, mag*pb)
vp.rate(3000), vpm.wait(scene)

The program starts by calling initialize() which initializes

the space grid, the potential, and the Gaussian wavepacket
returned by gaussian(). The probability |ψ|2 is represented by
a line on top of a bar graph which creates a distribution of solid
colors. Both the line and bar objects are created from VPM
modules of the same name, and can be animated with the move
method. The main loop integrates the Schrödinger equation,
sch_eqn(), with the vectorized leapfrog method as explained in
the text. The time t is not updated in the loop. However, if the
potential was time-dependent, it would be necessary to update
t and V after each step. A magnification factor is necessary to
scale the distribution so it is visible over the large spatial range.

Program listing 8.3: Quantum free fall ( splitop.py)

1 import numpy as np, visual as vp, vpm

from scipy.linalg import solve_banded
3 import matplotlib.pyplot as plt

5 def gaussian(s, x0, x): # normalized Gaussian, s=width,

x0=center
c = 1.0/np.sqrt(s*np.sqrt(np.pi))
7 return c*np.exp(−((x−x0)/s)**2/2)
9 def initialize (a, b, N): # set parameters
x = np.linspace(a, b, N+1) # grid points
11 V = 2*x # linear potential
s, x0 = 0.5, 5.0 # width (sigma), center of
gaussian
13 R, I = gaussian(s, x0, x), np.zeros(N+1) # real, imag
 w.f.
return x[1]−x[0], x,V,R,I # grid size, h=x[1]−x[0]
15
a, b, N = −10., 10., 500 # space range [a,b], num.
intervals
17 z, mag = np.zeros(N+1), 6 # zeros, magnifying factor

19 h, x, V, R, I = initialize (a, b, N) #
initialization
scene = vp.display(background=(1,1,1), ambient=1) # set
scene
21 bars = vpm.bars(x, z, z, z, h, 0.05, (1,0,1)) # wave
function
line = vpm.line(x, z, z, (1,0,1), 0.02)
23 pot = vpm.line(x, V*0.05, z, (0,0,0), 0.02) #
potential line

25 t, ic, cycle = 0.0, 0, 10 # t, animation cycle

dt, psi = 0.001, R + 1j*I # initialize dt,
complex psi
27 ta, pba = [], [] # time, prob. arrays
A = np.ones((3, N+1), dtype=complex) # prepare band matrix
A, B
29 A[1,:] = 2*(2j*h*h/dt − 1 − h*h*V)
dB = − np.conjugate(A[1,:]) # diagonal of B
31 while(t<=3):
C = dB*psi # prepare RHS Eq.
(8.31)
33 C[1:−1] −= (psi[:−2] + psi [2:])
psi = solve_banded((1,1), A, C) # band matrix solver
35 t, ic = t+dt, ic+1
if (ic % cycle == 0):
37 pb = psi.real**2 + psi.imag**2 # probability
ta.append(t), pba.append(pb) # store data
39 line.move(x, mag*pb, z)
bars.move(x, z, z, mag*pb)
41 vp.rate(1200), vpm.wait(scene)
43 (X, Y), ta = np.meshgrid(x, ta), np.array(ta)
plt.figure ()
45 plt.contour(Y, X, pba, 36, linewidths=1)
plt.plot(ta, 5 − ta*ta, ’r--’) # classical free fall
47 plt.xlabel( ’t (a.u.)’), plt.ylabel( ’x (a.u.)’)
plt.show()

The overall program flow is similar to Program 8.2. Lines

28 to 34 are the core of the first order split-operator method,
and have been explained in the main text. The variable pba[]
stores the probability distribution as a list of arrays, appended
to periodically. It contains data for contour plotting at the end.

Program listing 8.4: Quantum waves in 2D ( qmwaves2d.py)

import matplotlib.pyplot as plt

2 import numpy as np, visual as vp, vpm

4 def initialize (a=10., N=12, nx=40, ny=40): # size, max QM

num, grid
s, x0, y0 = 1.9, a/2, a/2 # width, center of
wavepacket
6 na, pi = np.arange(1, N+1), np.pi # QM numbers
En = pi*pi*na**2/(2*a*a) # all eigenenergies
8 x, y = np.linspace(0,a,nx), np.linspace (0,a,ny) #
grid
X, Y = np.meshgrid(x, y)
10 umx, uny = np.zeros((N,ny,nx)), np.zeros((N,ny,nx))
for n in range(N): # compute basis
functions
12 umx[n] = np.sqrt(2/a)*np.sin((n+1)*pi*X/a)
uny[n] = np.sqrt(2/a)*np.sin((n+1)*pi*Y/a)
14 amn0 = expansion(a, na, s, x0, y0) # expansion coeff.
 return a, N, En, amn0, X, Y, umx, uny
16
def expansion(a, n, s, x0, y0): # calc expansion
coefficients
18 r, q, pi = s/a, n*s/a, np.pi # warning: ’a/s’ must not be
integer
cm = np.cos(q*pi/2)*np.sin(n*pi*x0/a)/((1−q)*(1+q))
20 cn = np.cos(q*pi/2)*np.sin(n*pi*y0/a)/((1−q)*(1+q))
return 16*r*np.outer(cm, cn)/(pi*pi) # amn(0)
22
def psi(amn0, t): # calc wavefunction at time
t
24 wf, phase = 0.0, np.exp(−1j*En*t)
amnt = amn0*np.outer(phase, phase) # amn(t) =
−iEmnt
amn(0)e
26 for m in range(N):
s = 0.0
28 for n in range(N): s += amnt[m,n]*uny[n] # vector
operation
wf += s*umx[m] # ∑amnum(x)un(y)
30 return wf

32 t, num, plot = 0., 1, False # set plot=False to animate

forever
time = [0.0, 0.5, 0.9, 1.4, 2.0, 3.0, 40, 63.7, 64.2] # snapshots
34 a, N, En, amn0, X, Y, umx, uny = initialize()

36 scene = vp.display(background=(.2,.5,1), center=(a/2,a/2, −0.5),

up=(0,0,1), forward=(1,2, −1))
38 mesh = vpm.mesh(X, Y, 0*X, vp.color.yellow, vp.color.red)
info = vp.label(pos=(a/4,.8*a,1), height=20)
40 while True:
wf = psi(amn0, t)
42 mesh.move(X, Y, wf.real*3) # show real part, times 3
info . text= ’%5.2f’ %(t)
44 vpm.wait(scene), vp.rate(40)
if (plot):
46 plt.subplot(3, 3, num)
plt.imshow(np.abs(wf)**2, cmap=plt.cm.jet) # try
contourf()
48 plt.xticks ([],[]), plt.yticks ([],[]) # omit
ticks
plt.xlabel( ’t=%s’ %(t))
50 if (num > len(time)−1): break
t, num = time[num], num+1
52 else:
t = t + 0.05
54 if (plot): plt.show()

The function initialize() initializes some parameters and

sets up the space grid. The basis functions um(x) and un(y) for
each state n = 1 to N are calculated over the grid. It also calls
expansion() which computes the expansion coefficients of the
compact wavepacket at t = 0 from Eq. (8.53). Both Cm and Cn
are calculated first, each a 1D array of length N, then an N × N
matrix amn is formed with np.outer (line 21) which produces an
outer product such that amn = CmCn, similar to Program S:6.1.

The same outer product is used in psi() to calculate the

time-dependent amplitudes amn(t) = amn exp(−i(Em+En)t),
from which the wave function (8.52) is computed. Because
um(x) and un(y) have been pre-computed, the sum in Eq. (8.52)
is performed efficiently using element-wise (vector) operations
over the space grid.

The main program sets up a VPM mesh to animate the real

part of the wave function and displays the time with vp.label.
If plot is set to True, nine subplots (3×3) at pre-selected times
are created using imshow() which produces a smooth, contour-
like image. The tick marks are suppressed by supplying empty
lists (line 48). If plot is False, the program just steps through
time indefinitely.

Program listing 8.5: Numerical integration ( integral.py)

def simpson(f, h): # Simpson's rule over discrete points

2 n, extra = len(f), 0.0
if (n−1)%2 != 0: # numbers of bins not even
4 n=n−1
extra = f[−2]+f[−1] # trapzoid for last bin
6

sum, w = f[0]+f[n−1], 4
8 for i in range(1, n−1):
sum = sum + w*f[i]
10 w=6−w # alternate between 4, 2
return h*(sum/3. + extra/2.)
12

def gauss(f, a, b): # Order 36: abscissas and weights

14 x=[0.9978304624840857, 0.9885864789022123, 0.9720276910496981,
0.9482729843995075, 0.9174977745156592, 0.8799298008903973,
16 0.8358471669924752, 0.7855762301322065, 0.7294891715935568,
0.6680012365855210, 0.6015676581359806, 0.5306802859262452,
18 0.4558639444334203, 0.3776725471196892, 0.2966849953440283,
0.2135008923168655, 0.1287361038093848, 0.04301819847370858]
20

w=[.005565719664245068, 0.01291594728406522, 0.02018151529773513,

22 0.02729862149856854, 0.03421381077030707, 0.04087575092364476,
0.04723508349026585, 0.05324471397775972, 0.05886014424532480,
24 0.06403979735501547, 0.06874532383573639, 0.07294188500565307,
0.07659841064587077, 0.07968782891207167, 0.08218726670433972,
26 0.08407821897966186, 0.08534668573933869, 0.08598327567039468]
 28 p = 0.5*(b+a) # map from [a,b] to [−1,1]
q = 0.5*(b−a)
30 n = len(x)
sum = 0.0
32 for i in range(n):
sum += w[i]*(f(p + q*x[i]) + f(p − q*x[i]))
34 return q*sum

The function simpson() assumes discrete data points. The

number of intervals should be even. If it is odd, the first n−2
intervals (n = number of data points) are computed by
Simpson's rule, and the last one by trapezoid rule. If the latter
contribution is not small, the accuracy may be affected.

The Gaussian-Legendre function gauss() can be used with

any abscissa-weight data. SciPy provides an easy way to
generate these data sets from its special functions library. For
orthogonal polynomials, the library can return objects called
orthopoly1d containing the abscissa-weight data. The following
code illustrates its use.

Program listing 8.6: Gauss abscissa and weight ( gaussxw.py)

from scipy.special import legendre

2

n = 36 # order
4 p = legendre(n)
x = p.weights [:,0] # abscissa
6 w = p.weights [:,1] # weight
 8 for i in range(n):
print ( ’%20.16f %20.16f’ %(x[i], w[i]))

The data can be used directly in gauss(). If the order is

even, no modification is necessary (other than dropping the
negative abscissas). If it is odd, make sure to include the x = 0
point only once in the loop. If the default order here (36) is not
enough to give accurate results, our integrand is probably ill-
behaved (highly oscillatory, stiff, or singular), or the integration
limit is too great. Increasing the order alone rarely cures the
problem. A better strategy is to break the range into several
subintervals and add up their contributions, or to make a
proper change of variable. The latter is particularly important if
one end of the integral is infinity, e.g.,

Program 8.5 serves as our numerical integration library. If

more sophisticated integration technique is required, such as
the need for error control, look to SciPy library
scipy.integrate. It has several routines, including a general
purpose routine quad for single integrals, and dblquad and
tplquad for double and triple integrals, respectively.
 1
If you are unfamiliar with Eq. (8.2) but familiar with wave motion, it may be best to
think of it as a wave equation for now, and explore its behavior purely as a PDE.

2
Many aspects of quantum mechanics do not seem intuitive to us classical beings,
making computer simulations of quantum systems all the more expedient and interesting.

3 2
For Coulomb potentials, we can set e /4πε0 = 1 in atomic units.

4
This technique is also known as the method of lines.

5
An alternative is to use Numba, or call compiled codes in Fortran or C/C++ using F2Py,
Weave, or Cython (see Section 1.3.2, and Programs 5.7 and S:11.3).

6
“It is weird that quantum mechanics is not weird”, Nevin (a student of mine) thus
exclaimed after seeing quantum mechanics in action and working like classical mechanics.

7
Another example is found in the hydrogen transition frequency which also depends on
the difference of eigenenergies.
Chapter 9
Time-independent quantum
mechanics
We saw in Chapter 8 that time-dependent quantum motion
could be studied in its own right, but some features such as
quantum transitions are connected to the stationary states of
time-independent quantum systems where the potential has no
explicit dependence on time. Transitions between these states
give rise to discrete emission lines which could not be
explained by classical physics when they were discovered
around the turn of the twentieth century. In fact, one of the
most spectacular successes of quantum mechanics was its full
explanation of the hydrogen spectra, helping to establish
quantum physics as the correct theory at the atomic scale.

In this chapter, we discuss the structure of time-

independent quantum systems. We are interested in seeking
solutions to the bound states of the time-independent
Schrödinger equation (see Eq. (9.1) below). We will discuss
several methods, beginning with shooting methods for 1D and
central-field problems. They are the most direct methods with
 a high degree of flexibility in choosing the core algorithm,
requiring no more than an ODE solver and a root finder at the
basic level. We use it with the efficient Numerov algorithm to
study band structure of quasi-periodic potentials, atomic
structure, and internuclear vibrations.

We also apply the 1D finite difference and finite element

methods introduced previously for boundary value problems to
eigenvalue problems, including point potentials in the Dirac δ-
atom and δ-molecules. The basis expansion method is
introduced as a general meshfree method that depends on the
basis functions rather than the space grid.

To simulate quantum dots, we develop a full FEM program

to solve the Schrödinger equation in arbitrary 2D domains. The
program is robust and simple to use, the core having less than
10 lines of code. We will explore the rich physics in these
structures, including the triangle and hexagon quantum dots.

Unless explicitly noted, we use atomic units throughout.

.1 Bound states by shooting methods

Our study begins with the time-independent Schrödinger
equation, which can be obtained from the time-dependent case
(8.2) by the separation of space and time variables such as in
Eq. (6.64),
This is a second order, homogeneous ODE. For most
potentials, the stationary solutions fall into two categories,
localized bound states that vanish at infinity (i.e., u(±∞) = 0),
and continuum states that oscillate forever. As stated earlier,
we focus on the bound states in this chapter, and will consider
continuum states in Chapter 12.

Subject to the boundary condition u(±∞) = 0, Eq. (9.1)

determines simultaneously the correct solutions and the
allowed energies E, in the same way that Eq. (6.86) determines
both the shapes and frequencies of standing waves. The
shooting method (Section 3.7) helps us see how this works and
why only discrete values of E are allowed.

Since Eq. (9.1) is an ODE, it suggests that we could try

solving it like a two-point boundary value problem, u(−∞) =
u(∞) = 0, similar to shooting a projectile illustrated in Figure
3.22. Suppose we start with a trial E, integrate from x = −R to +
R (R large enough to approximate ∞) with the initial value
u(−R) = 0, and check the final value u(R). If it is not zero, we
adjust E and repeat. When u(R) is zero, we have found the right
energy and solution because they satisfy the correct boundary
conditions. This in essence is the shooting method.

9.1.1 VISUALIZING EIGENSTATES

 Figure 9.1: Visual eigenstates of a square well potential by the
shooting method.

The idea of the shooting method is animated in Program 9.1

for the square well potential and illustrated in Figure 9.1 with
snapshots taken from the program. We assume the potential is
symmetric about the origin for now, i.e., V = −V0 if |x| ≤ , and
zero elsewhere. The width and depth of the well are a = 4 a.u.
(2.1 Å) and V0 = 4 a.u. (109 eV), respectively.

Program 9.1 steps through the energy range −V0 < E < 0 at
fixed increments. One can show analytically and graphically
that no bound states exist outside this range (Project P9.1). At a
given energy E, the program integrates the Schrödinger
equation from x = −R upward to x = 0, instead of x = +R. The
reason is that as x → ±∞, there are two mathematical solutions,
one exponentially decaying and another exponentially growing.
The former is physically acceptable, but the latter is
numerically dominant. A small error will be exponentially
magnified and overwhelm the correct solution. We saw the
same kind of divergence in the calculation of the golden mean
(Section 1.2.2). This means that if we were to integrate to x = ∞,
the wave function will never be zero at the boundaries, and will
fail the test u(∞) = 0, no matter how exact the energy E is.

The correct technique to avoid divergence is to integrate

inward from both ends: upward from x = −R and downward
from x = R, and meet somewhere in the middle at a matching
point, xm. We choose xm = 0 in this case. For a symmetric
potential, V(−x) = V(x), the solutions are either even or odd,
i.e., the wave function has definite parities. Because of this, we
integrate from x = −R to 0, and obtain the wave function in the
other half by symmetry. We will discuss general, asymmetric
cases shortly.

If the wave functions match each other smoothly at xm, we

have a solution that satisfies Eq. (9.1), vanishes at ±∞, is
differentiable, and therefore must be a correct solution. For
even states shown in Figure 9.1, the derivatives of correct
solutions will be zero at the origin. Initially at E = −V0 = −4, the
wave functions inside the well are straight lines meeting at x =
0, creating a cusp. The energy is too low, and the wave function
has not bent enough. The situation improves a little at a higher
energy E = −3.9, and the wave functions bend downward and
are no longer straight lines. But it is still not smooth across the
origin. The next frame at E = −3.78, the wave functions connect
smoothly to each other, and the kink has disappeared. We have
found the first allowed energy – an eigenenergy, and with it,
the correct solution – an eigenfunction.

Increasing the trial energy further, the wave function over

bends. By the time it again becomes smooth at E = −2.06, we
have found the second, and last, even eigenstate with two
nodes. There are no more even states to be found after this. The
last frame (E = 0) shows a would-be solution with 4 nodes, but
it is not smooth at the matching point. There are also a finite
number of odd states that can be found similarly (Project P9.1).

This example shows graphically how discrete energies

naturally emerge in quantum mechanics. There had been
considerable attention to visual representation of eigenstates
even before the age of personal computers. In Ref. [23],1 for
instance, a second-order recursion method similar to RK2 had
been used to generate a film like Figure 9.1. If we accept normal
modes of oscillations (Section 6.2.2) or standing waves (Section
6.6) in classical mechanics as fact of life in the land of waves,
these discrete energies in quantum mechanics may not seem so
strange anymore. They are a result of finding well-behaved
solutions to a wave equation, the Schrödinger equation.

9.1.2 GENERAL SHOOTING METHOD

We just illustrated the idea of the shooting method with the
symmetric square well as a simple example. For it to be
applicable to arbitrary potentials, we need to generalize the
method so that it can deal with any symmetry and can narrow
down the precise value of the eigenenergies.

To begin, we introduce an auxiliary variable w = u′ to

represent the first derivative of the wave function, and convert
the second-order Schrödinger equation into two first-order
ODEs (in a.u.)

We solve Eq. (9.2) in the range xL ≤ x ≤ xR over the grid

points xj = x0 + jh, j = −N, …, −1, 0, 1, …,N, h = (xR − xL)/2N, so
is the midpoint. For open space, the value of xR
− xL should be large compared to the characteristic scale of the
system. We assume the wave function is zero outside the range.
As stated above, we must integrate inward from x = xL and xR
in order to maintain stability. Usually, we can set the boundary
values u(x±N) ≡ u±N to zero, u−N = uN = 0, and w−N and wN to a
small but arbitrary nonzero value (e.g., 0.1). The latter affects
only the overall scale (normalization) of the wave function,
which cancels out as we will see shortly.

Let u↑ and u↓ represent the solutions of upward and

downward integration, respectively. If the energy is a correct
eigenenergy, the matched wave functions will be smooth at the
matching point xm. This means that the wave functions and
their derivatives at xm can differ at most by an overall constant
C,

Eliminating C, we have an equivalent matching condition

Equation (9.4) is satisfied only if the trial energy E is an

exact eigenenergy. In other words, allowed energies are roots of
Eq. (9.4). We can use this fact to locate the exact energy: pick a
trial E, increase E until Eq. (9.4) changes sign, then we know a
root has been bracketed. We can narrow the bracket down with
a root solver like the bisection method.

Let us define a matching (shooting) function in terms of u

and w on the grid,

where we made E explicit, and as usual. We also

substituted . This is the function we will evaluate
numerically,2 in much the same way as Eq. (3.49b) for
projectile motion.

Program 9.2 implements the shooting algorithm. Key parts

of the program are given below.
 def sch(psi, x): # Schrodinger eqn
2 return [psi [1], 2*(V(x)−E)*psi[0]]

4 def intsch(psi, n, x, h): # integrate Sch eqn for n steps

psix = [psi [0]]
6 for i in range(n):
psi = ode.RK45n(sch, psi, x, h)
8 x += h
psix.append(psi[0])
10 return psix, psi # return WF and last point

12 def shoot(En): # calc diff of derivatives at given E

global E # global E, needed in sch()
14 E = En
wfup, psiup = intsch ([0., .1], N, xL, h) # march
upward
16 wfdn, psidn = intsch ([0., .1], N, xR, −h) # march
downward

return psidn[0]*psiup[1] − psiup[0]*psidn[1]

18
……
20 while (E1 < 0): # find E, calc and plot wave function
if (shoot(E1) * shoot(E1 + dE) < 0): # bracket E
22 E = rtf. bisect (shoot, E1, E1 + dE, 1.e−8)
……

The function sch() returns the Schrödinger equation (9.2).

The wave function and its derivative are contained in a 2-
element array, psi=[u,w]. The main workhorse is the function
intsch(). It integrates sch() for n steps from the initial value
using the non-vectorized RK45 (Runge-Kutta-Felhberg)
method for faster speed because the ODEs in sch() are simple
to evaluate and called repeatedly. A vectorized version would
be inefficient for only two ODEs in this case.

The shooting function (9.5) is calculated in shoot(). First, it

sets energy E which is declared to be global because it is
required in a different function sch(). Then it marches upward
from x = xL for N steps and downward from x = xR for an equal
number of steps, so the matching point xm is at the midpoint
(origin in this case). Note that in the downward march the step
size is −h. The main loop scans through the energy. When a
root is bracketed, the bisection root finder is called to
accurately find the eigenenergy.

Table 9.1: Eigenenergies of the square well potential (a = 4, V0 = 4).

As our first test case, we apply the shooting method to the

square well potential in Figure 9.1. The results from Program
9.2 are given in Table 9.1 (along with other results to be
discussed later). They agree very well with the exact values
obtained by numerically solving the analytic eigenequations
(Exercise E9.1). The first and third states are even, and the
second and fourth odd. We had seen the even states in Figure
9.1.
 As well as the shooting method works for the square well, it
performs even better for continuous potentials such as the
simple harmonic oscillator (SHO), and yields practically exact
results (see Project P9.2).

Figure 9.2: The double square well potential.

As the second test case, we use the shooting method to

calculate the eigenstates of a double square well potential
defined below and graphed in Figure 9.2,

The results from Program 9.2 are shown in Figure 9.3. The
parameter values are a = 4, b = 1, and V0 = 6. We again see four
bound states, two even and two odd, the same as the single well
potential. But there are several differences. The wave functions
are very different. For the ground state (−4.907), we see a
double hump structure rather than a single peak. This is caused
by the barrier in the middle. Inside the barrier, the wave
function does not oscillate and must either decay or grow
exponentially. We can understand this from Eq. (9.2), u″ = 2(V
− E)u. Below the potential, V − E is positive, so u″ and u have
the same sign. If u, and hence its second derivative u″, is
positive, the wave function must be concave up, so it will stay
positive (remember no nodes allowed for the ground state).
Similarly for negative u, the opposite is true. Therefore, the
wave function in each well has to bend and passes a maximum
so as to decrease to zero at x = ±∞. In the next even state
(−2.005), a local peak is formed within the barrier for the same
reason, the only way to have two nodes.

In contrast, the odd states look similar to their counter

parts for a single well potential (see Project P9.1). The
difference is a change of curvature in the middle of the barrier,
with the second derivative changing sign due to u crossing zero
(visible in the first excited state at −4.858, but hardly
noticeable at −1.69).
 Figure 9.3: The eigenstates in the double square well potential by the
shooting method.

The ground and the first excited states are separated in

energy by 0.049, which is only 0.8% of the depth of the wells.
The wave function of the first excited state looks like that of the
ground state if one of its humps is flipped. In fact, we can think
of the wave functions as a linear superposition of one basic
state shifted to different centers ±xc,

The plus and minus signs correspond to the even and odd
states, respectively, i.e., u±(−x) = ±u±(x), assuming φ(−x) =
φ(x). Equation (9.7) represents a linear combination of atomic
orbitals (LCAO). The function φ would describe the ground
 state in one of the wells (single-particle state). As the barrier
width becomes large, the two wells are increasingly
independent, so the two states become approximately
degenerate with the same energy.

.2 Periodic potentials and energy bands

Suppose we add many potential wells to Figure 9.2 to form an
extended periodic system. The shooting method would require
a large number of grid points, slowing down the speed. The
bottleneck is the calculation of the matching function f(E), Eq.
(9.5), which relies on a general ODE solver (RK45 in this case)
to integrate the Schrödinger equation. Because f(E) is called
repeatedly, the efficiency of the ODE solver is crucial.

As it turns out, we can solve the Schrödinger equation much

more efficiently using Numerov's method rather than using a
general ODE solver. This method is specialized to second-order
ODEs which do not contain first derivatives explicitly. It is
presented in Section 9.A.

To use Numerov's method, we write the Schrödinger

equation (9.1) as

The method gives the solution as a three-term recursion (9.59),

where fi = 2m(E−V(xi))/ħ2. Equation (9.9) is symmetric
whether marching up or down, i.e., the subscripts i±1 can be
swapped with i 1 without any change.

The recursion is stable and is accurate to fifth order in h, the

same order as the RK45 method. Although it requires three
calls to f(x), two of the values can be reused in subsequent
iterations, so it needs only one new call per iteration. In
contrast, RK45 requires six function calls per step.

However, there are two issues to consider. First, Numerov's

method is not self starting, because two seed values are
necessary to get started as expected from a second order ODE.
This poses no difficulty for us. For instance, marching
downward from the boundary point N, we can set uN = 0 and
uN−1 to a small value (say 0.1) as before, affecting only the
normalization. Now, we can obtain uN−2 from Eq. (9.9), then
uN−3, etc. Upward recursion is similar.

Secondly, Numerov's method does not provide first

derivatives on its own. But we need them when matching the
wave functions. We can calculate the first derivative using the
central difference formula (6.31b). Better yet, we can do it more
accurately with a similar formula (9.60) which takes advantage
of the special form of the ODE the same way Numerov's
method does. Either way, we must march one step past the
matching point in both directions in order to compute the first
derivatives. If the matching point is m, we must march upward
from −N to m + 1, and downward from N to m − 1, calculate the
derivatives at xm, and match the solutions with Eq. (9.5). We
leave the implementation to Project P9.3.

Figure 9.4: Multiple potential wells (top) and their eigenenergies

forming energy bands.

The speed gain for shooting with Numerov's method is

significant. We can use it to extend our study to large systems
requiring large number of grid points. As an example, we show
the results obtained this way for multiple identical potential
wells (1, 2, 4, and 8, Figure 9.4). In each case, the width of the
well is a = 1, depth V0 = 6, and the thickness of the walls
(barriers) between the wells is 0.5.

There are only two states for a single well. For the double
well, there are three states in total. The extra state comes from
the ground state of the single well which splits into two
 adjacent levels, one slightly below and the other slightly above
the original level. The two levels come from the near
degeneracy we discussed regarding the double well potential in
Figure 9.3.

The excited state of the single well is too close to zero to

split, as one would end up above zero, so it remains one bound
state below the original energy. For four wells, we see more
splitting, with the states clustering into two groups at the top
and bottom. We are beginning to see energy bands. At eight
wells, the bands become more well defined and more densely
filled, particularly the bottom band. The gap between the bands
remains empty.

We can imagine as the number of wells increases, we have a

quasi-periodic system with filled bands and empty gaps. This is
how energy bands form in solids [6]. Atoms pack together in
periodic lattice structures in solids. Multiple states cluster
around single atomic states, forming energy bands and gaps.
Details may differ, but the qualitative features are the same.
Our example shows not only that energy bands form, but how
they form, namely via state splitting. Each pair of new states
are approximately degenerate. When many nearly degenerate
states cluster together, we have completely filled bands.

.3 Eigenenergies by FDM and FEM

methods
The shooting method discussed so far scans the energy range
and finds one eigenstate at a time. It is suitable primarily for 1D
problems. We describe two other grid-based methods, FDM
and FEM, that solve for all states simultaneously, and can be
extended to higher dimensions.

9.3.1 EIGENENERGIES BY FDM

We described the FDM method in connection with the
displacement of an elastic string under external loads (Section
6.3.2). We just need to make a slight change so as to convert it
from a linear system to an eigenvalue system.

The displacement equation (6.29) and the Schrödinger

equation (9.1) are structurally similar. Comparing the two, we
identify “tension” −ħ2/2m (−1/2 in a.u.) and “load” f(x) = (V(x)
− E)u(x) for the latter. The function f now depends on the
solution u itself, owing to the linearity of the Schrödinger
equation. This leads to an eigenvalue problem.

We can discretize the kinetic energy operator

and follow the discussion of the FDM method in Section 6.3.2
up to Eq. (6.33). There, the f values on the grid are replaced by
fi = Viui − Eui, even though E and ui are unknown yet. In matrix
form, let
Note the negative sign in is absorbed inside the
matrix (compare with Eq. (6.34)), and is the potential energy
operator in matrix form. The eigenvalue equation can be
expressed as

This is the same eigenvalue problem as in Section 6.6. In Eqs.

(9.10) and (9.11), we assume the usual notations: Vi = V(xi) on
the grid (Figure 6.10), and u is a column vector containing the
N − 1 unknown wave function values, u1, u2, …, uN−1. We have
excluded the boundary values u0 = uN = 0 in Eq. (9.11) because
the wave function vanishes on the boundary for bound states
(Dirichlet boundary condition).

The kinetic energy matrix is tridiagonal (same as Eq.

(6.34) except for an overall negative sign), and the potential
energy matrix is diagonal. Equation (9.11) may be solved to
obtain the eigenenergies and associated eigenvectors using
SciPy functions like eigh() in Program 6.2, or the more
efficient sparse eigenvalue solver, eigsh() (see Program 9.3,
line 40).
 The FDM method is direct and easy to use. The matrix is
independent of potential, and the matrix has only diagonal
elements . As a test, we give the results of the FDM
method for the square well in Table 9.1. The step size is the
same as in the shooting method. The eigenequation (9.11) has
N − 1 eigenvalues (N = 500 in the present case). The four
lowest eigenvalues are negative and are listed in Table 9.1, so
the FDM gives the correct number of true bound states. The
rest are positive, and they represent pseudo-continuum states.3

The first three bound states agree with the exact results to
four digits, and the fourth to two digits. The accuracy (second
order) is good except for the fourth bound state. The reason is
that the highest bound state is close to the continuum, and
coupling to (i.e., mixing with) the pseudo-continuum makes its
accuracy worse. The FDM results are less accurate than the
shooting method which is of a higher order with either RK45 or
Numerov's method, and does not suffer from pseudo-
continuum coupling. If we need only a few discrete states or
high accuracy, the shooting method is better suited than the
FDM. But the FDM is much simpler to use and can produce
bound states en masse.

9.3.2 EIGENENERGIES BY FEM

Similar to the FDM, we developed the FEM method and first
applied it to the displacement solution of an elastic string. It is
equally applicable to quantum eigenvalue problems as we show
below. The development of 1D FEM will also help us better
understand 2D FEM in the study of quantum dots (Section
9.6).

We start from the integral formulation of FEM in Section

6.4. In complete analogy to the FDM description above, we
identify from Eq. (6.42) and f(x) = (V(x) − E)u(x) by
comparing the displacement equation (6.29) and the
Schrödinger equation (9.1). Analogous to Eq. (6.42), we have
for the Schrödinger equation in FEM form

where a and b are the boundary points.

Following exactly the same steps from Eq. (6.42), namely,

integrating the kinetic energy term by parts, and substituting u′
and u from Eq. (6.37), we obtain (Exercise E9.4)

where qj is given by Eq. (6.43), Aij by Eq. (6.45), and N the

number of finite elements (intervals). The first two terms
belong to the kinetic energy matrix. The potential matrix Vij
and basis overlap matrix Bij are defined as
As discussed in Section 6.4 (see Eq. (6.53)), even though the
integrals are over the whole domain [a, b], they are effectively
over the elements ei or ej only (see Figure 6.12). Furthermore,
because the basis functions do not overlap unless they share a
common node, Vij and Bij are nonzero only if |i − j| = 0,±1.

Like Aij in Eq. (6.53), the overlap matrix can be evaluated

analytically,

From Eq. (6.43) and the discussion following it, the value qj
is zero unless j = 0 or j = N. Because the boundary values u0 =
uN = 0 for bound states are already known, we can remove the
indices j = 0 and j = N from Eq. (9.13). As a result, we can
eliminate qj entirely from Eq. (9.13).

Collecting Aij from (6.54) and Bij above, we can express the
kinetic, potential, and overlap matrices as
All three matrices are tridiagonal with dimensions
(N−1)×(N−1). Except for a multiplication factor, the matrix
in FEM basis is identical to the FDM version, Eq. (9.10).

Finally, we can write the FEM eigenvalue equation as

Comparing with the FDM equation (9.11), we have an extra

matrix B on the RHS of Eq. (9.17). Equations of this type are
known as the generalized eigenvalue equations (see also Eq.
(6.16)). It is common in quantum problems when the basis
functions are not orthogonal.

Given a potential V(x), we can evaluate Vij between nodes i

and j from Eq. (9.14) analytically if possible or numerically if
necessary. In the latter case, it is more convenient to use
element-oriented rather than node-oriented approach.4 We can
rewrite Eq. (9.14) as a sum over all elements e
Here, xe denotes the domain over finite element e (compare
with Eq. (7.26)). Except on the boundary, each domain
contributes three terms to the matrix: two diagonal entries to
the left and right nodes, and one off-diagonal cross-node entry.
After going through all finite elements, we will have built the
full matrix.

Upon solving Eq. (9.17), we will have both the eigenenergies

E and the wave functions u. We can determine any property of
the system, such as the expectation values of the kinetic and
potential energies (Exercise E9.5),

where the normalization of the wave function is assumed to be

9.3.3 THE FEM PROGRAM

A complete FEM program is given in Program 9.3. It
implements the above strategy for an arbitrary potential. The
code segment for building the matrix is given below.

1 def Vij(i, j): # pot. matrix Vij over [xi, xi+1] by order−4
Gaussian
 x=np.array([0.3399810435848564, 0.8611363115940525])
# abscissa
3 w=np.array([0.6521451548625463, 0.3478548451374545])
# weight
phi = lambda i, x: 1.0 − abs(x−xa[i])/h # tent
function
5 vV, hh = np.vectorize(V), h/2.0 #
vectorize V
x1, x2 = xa[i] + hh − hh*x, xa[i] + hh + hh*x
7 return hh*np.sum(w * (vV(x1) * phi(i, x1) * phi(j, x1) +
vV(x2) * phi(i, x2) * phi(j, x2)) )
9
def V_mat(): # fill potential matrix
11 Vm = np.zeros((N−1,N−1))
for i in range(N): # for each element # contribution
to:
13 if (i>0): Vm[i−1,i−1] += Vij(i, i) # left
node
if (i < N−1): Vm[i,i] += Vij(i+1, i+1) # right
node
15 if (i>0 and i< N−1):
Vm[i−1,i] += Vij(i, i+1) # off
diagonals
17 Vm[i,i−1] = Vm[i−1,i] #
symmetry
return Vm

The function Vij() calculates in Eq. (9.18) over a finite

element between xi and xi+1. We use a fourth order Gaussian
abscissa and weight for numerical integration (see Section
8.A.3). Because our FEM is a second order method, the
relatively low-order Gaussian integration is sufficient. It
defines an inline lambda function for the tent function centered
at node i as
which is equivalent to Eq. (6.38). Then, the Gaussian
integration formula (S:8.55) is applied where the sum Σwkf(xk)
with f(x) = Vφiφj is performed vectorially using np.sum. For this
reason, the potential V(x) is vectorized to ensure it can operate
on an array of abscissas (a ufunc).

The next routine V_mat() builds the potential matrix by

iterating over all finite elements. For each element, it calls
Vij() to compute its contribution to the diagonal entries and
of the left and right nodes, and the off-diagonal entries
. Since the dimension of the matrix is (N − 1, N −
1), its indices are offset by 1 relative to the element number.

Once the necessary input is prepared, we are ready to solve

Eq. (9.17). This is done with the sparse matrix solver from
SciPy, eigsh(), in Program 9.3. The FEM results for the square
well potential are shown in Table 9.1. Its accuracy is
comparable to FDM, as expected because both are second
order. Usually, the smaller the element size, the better the
results. However, per the sharp walls of the well, actual
accuracy could suffer because of the discontinuity. For best
result, choose N such that the walls bisect the elements
covering the walls, or linearly interpolate the potential (Project
P9.5). This poses no problem for continuous potentials, e.g.,
FEM results for the SHO are rather accurate. Go ahead and try
it: replace the potential, and start SHOwing!
 def V(x): return 0.5*x*x # potential

9.3.4 THE DIRAC Δ-ATOM AND Δ-

MOLECULE

Figure 9.5: The bound state wave function of the Dirac δ atom.

One advantage of the FEM over the FDM is that it is

applicable to integrable singular potentials such as the Dirac
delta potentials. For instance, for V = −αδ(x), referred to as the
Dirac δ atom, there is only one bound state at , and u
= A exp(−α|x|) [44]. With FEM, we get one bound state as well.
At α = 1, the FEM result is −0.499882 using the same element
size as in Program 9.3, h ~ 0.05 (Project P9.6).

The wave function of the bound state is shown in Figure 9.5.

It is obtained along with eigenenergies from Program 9.3. The
discontinuity in the first derivative is accurately reproduced,
agreeing with the analytic “cusp” condition
Equation (9.22) is similar to (6.59) for a point source on a
string. The piece-wise basis functions in FEM are flexible
enough to handle this kind of discontinuities efficiently.

We can also determine the average kinetic and potential

energies from Eq. (9.19) in a straightforward manner (Project
P9.6). For α = 1, the results are and
, which compare well with the exact values 0.5
and −1, respectively. As expected, is equal to the total
energy quoted earlier.

When there are two δ potentials next to each other (Figure

9.6), we have a Dirac δ molecule,

Figure 9.6: The Dirac δ molecule.

The analytic eigenequation of the δ molecule is (Exercise

E9.6)
 For arbitrary potential strengths α and β, Eq. (9.24) needs
to be solved numerically. There are two limits where the
solutions are readily obtained. The first is the separate atom
limit, a → ∞. We have two isolated δ atoms, so k = α or β. The
second is the united atom limit, a = 0, and k = α + β. This is a
single δ atom with an effective potential strength α+β as
expected.

If the potential strengths are equal, α = β, we can solve Eq.

(9.24) analytically in terms of the Lambert W function
discussed in Chapter 3 (Section 3.4). The solutions are [90]
(Exercise E9.6)

Figure 9.7: The universal energy-level diagram of the symmetric

Dirac δ molecule, where .
 We see from Eqs. (9.24) and (9.25) that if we scale the
energy E by , the magnitude of the isolated δ-atom
energy, then E becomes a function of the scaled variable αa
only. We can make a universal energy-level diagram as a
function of the scaled variables, shown in Figure 9.7. For a
given α, the diagram shows the energy as a function of
internuclear distance a, also known as the molecular potential
curve. The δ-molecule has been a useful model in atomic
ionization studies [93].

For large a, the energy approaches E = −E0, the separate

atom limit. As discussed above, this limit corresponds to two
degenerate atomic levels. As a decreases, the degeneracy is
lifted and the two levels begin to split: the ground state bends
downward and the excited state upward. This is fully analogous
to state-splitting in multiple-well potentials (Figure 9.4). Their
wave functions would resemble the first two shown in Figure
9.3 for the double-well potential, with the former being
symmetric and the latter antisymmetric.5 We expect that they
may be approximately described by the LCAO according to Eq.
(9.7) [6].

The molecule supports two bound states until the critical

internuclear distance

below which the excited state ceases to exist. We can see that
the critical value ac comes from Eq. (9.25) by requiring k− = 0
 and noting the special value W(−e−1) = −1 in Eq. (3.22). The
energy E−, ever increasing with decreasing a, has reached zero
at ac and entered the continuum for a < ac. On the other hand,
the energy of the ground state continues to decrease below the
critical distance, finally approaching E+ = −4E0, the united
atom limit at a = 0.

The case for asymmetric potentials α ≠ β may not be as tidy

analytically, but is certainly as approachable numerically with
the FEM. We leave its investigation to Project S:P9.1.

.4 Basis expansion method

We had encountered the general idea of the basis expansion
method (BEM) previously in electrostatic and time-dependent
quantum systems (Sections 7.5 and 8.4.1). The key was that,
rather than discretizing space, the solution was expanded in a
(finite) basis set. We discuss the same approach for eigenvalue
problems. This method is easy to set up and can be very
accurate and efficient. Usually, we have the freedom to choose
from many basis sets the one that best suits the problem. When
a good basis set is used, a relatively small number of basis
states can give very accurate results, saving both computing
time and memory (compact matrices).

Let the basis set be un(x) satisfying the boundary conditions

un(a) = un(b) = 0. Like Eq. (7.31) or (S:8.23), we expand the
wave function as
Unlike the time-dependent case, the expansion coefficients an
are constants independent of time.

The Schrödinger equation to be solved is

As before, H is the Hamiltonian representing the sum of kinetic

and potential energies defined over the same range [a, b].

We can substitute u into Eq. (9.28) and project onto on

both sides, following analogous steps from Eq. (S:8.24).
Alternatively, we can start from Eq. (9.12) but keep the kinetic
energy operator as is without integration by parts. Either way,
after projection, we obtain the familiar eigenequation,
(Exercise E9.8). Though this is identical in
form to Eq. (9.17), the operator representations are different in
BEM than in FEM. Here, the kinetic, potential, and overlap
matrices are respectively defined as

Depending on the basis set, the BEM matrices are not

necessarily sparse like in FDM or FEM, but they often are for
the problems we are interested in.
 The eigenvector u is a column vector similar to Eq. (6.15),
i.e., u = [a1, a2, …]T. The and B matrices depend on the basis
set only, but the matrix depends on both the basis set and the
potential. If the same basis set is used for different problems,
only the potential matrix needs to be evaluated for each
problem, while and B remain the same.

If the basis set is orthogonal, then Bmn = δmn, so is

the identity matrix. We assume orthonormal basis sets in the
rest of this section. In this case, we need to solve the standard
eigenvalue problem (9.11) rather than a generalized eigenvalue
problem.

9.4.1 THE BOX BASIS SET

In our first example, let us illustrate the BEM using the familiar
basis set of a particle in the box, Eq. (S:8.39). Since they are
eigenfunctions of the kinetic energy operator, the matrix is
diagonal. The diagonal elements are the eigenvalues associated
with un, i.e., Tmn = (π2ħ2n2/2meL2)δmn, where L is the box
width. The full matrix is

As discussed above, this is the same for any problem as long

as we choose the same box basis set. The potential matrix is
specific to a given system. Let us apply the BEM to the common
test case, the square well potential. The matrix element is from
Eq. (9.29)

where a and V0 are the width and depth of the square well as
usual. It can be evaluated either numerically or analytically
(Exercise E9.9). We have assumed the box is centered at origin,
−L/2 ≤ x ≤ L/2, so Vmn is zero unless m + n is even.

We leave the calculations to Project P9.7 and show the

results in Table 9.1. The accuracy depends on the box width L
and the number of included states N. We chose the box width L
= 8a, and used N = 300 states. The results are in good
agreement with the exact values.

The advantage of using the box basis set is that the basis
functions are simple, elementary functions. The required
matrices are easy to generate. In fact, Vmn as given by Eq. (9.31)
can be efficiently evaluated using FFT. But convergence can be
slow with increasing N for the upper excited states close to the
continuum, mainly caused by coupling to pseudo-continuum
states discussed earlier.

9.4.2 THE SHO BASIS SET

The simple harmonic oscillator offers another useful basis set
for the BEM. The SHO potential is , where
is the natural frequency.

The SHO is investigated numerically in Project P9.2.

Analytically, its eigenenergies are

and the associated eigenfunctions can be written as

In Eq. (9.33), An is the normalization constant, and a0 the

characteristic length. Either ω or a0 serves as a free controlling
parameter of the SHO basis set.6

The function Hn(y) are the Hermite polynomials [10], and

the first few series are

Higher orders can be obtained from the recurrence formula

This formula is stable for upward recursion, in the direction of

increasing n. The SHO basis is defined for all space −∞ < x < ∞,
so the BEM using this basis is applicable in principle to all
systems whose boundary conditions are u(±∞) = 0.

The kinetic energy operator in the SHO basis set is given by

(Exercise E9.10)

This is a sparse matrix consisting of three single-element

bands, the diagonal and two symmetric off-diagonals. Each row
or column, other than the first and last two, has three nonzero
elements, Tmm and Tm,m±2. The diagonal elements are equal to
one half of the eigenenergies.

We choose the wedged linear potential, V(x) = β|x| (Figure

8.16), as the test case because, like the SHO, it supports only
bound states and is also analytically solvable in terms of Airy
functions [60] (see Exercise E9.11, Section 9.B). Program 9.4
solves the problem using BEM with the SHO basis set. The
results are given in Table 9.2. A total of N = 20 basis states are
used, and the free parameter is ω = 1.

Table 9.2: The first several eigenenergies of the potential V = |x|.

 The analytic (exact) values are also computed in the
program and given in the same table. The numerical results
agree very well with the analytic results, mostly to six digits.
This level of accuracy was obtained with only N = 20, a very
small and efficient basis set. Of course, increasing N improves
the agreement. For a given N, the adjustable parameter ω
affects the accuracy as well. It controls the shape of the basis
functions (like radial basis functions, Section 7.5.2), which in
turn affects the quality of the basis expansion.

It is curious, though, that the accuracy for the ground state

is only four digits, worse than for the excited states, when we
expect the BEM (and other methods) to give the most accurate
results for low-lying states. The wedged potential is an
exception to the rule. The reason has to do with the wedge at x
= 0, which causes a discontinuity in the third derivative of the
wave function. For smoother wedges, the agreement becomes
better.

Scanning through Table 9.2 reveals that the gap between

adjacent energy levels decreases with increasing n. For
instance, the gap is ΔE21 = 1.05 between n = 1 and 2, and ΔE65
= 0.556 between 5 and 6. This behavior is compared in Figure
9.8 to two other well-known potentials, the infinite potential
well (rigid box) and the SHO. The level spacing increases in the
rigid box, remains equidistant in the SHO, and decreases in the
linear potential. It suggests that the asymptotic break-even
point that determines the gap's behavior is quadratic. If the
potential increases faster than x2 as x → ±∞, the gap will
increase with increasing n. Right at x2, the gap is constant. If
the potential increases slower than quadratic, the gap will
decrease. Realistic potentials are in the last category.

Figure 9.8: Energy-level diagrams for a rigid box (left), the SHO
(center), and the wedged linear potential (right).

Half-open space
The full SHO basis set spans the whole space. For bound
systems that exist in the half-open space, 0 ≤ x < ∞, the
boundary conditions are u(0) = u(∞) = 0. For example, one
such system is a particle bouncing between a hard wall and a
linear potential similar to quantum free fall in Section 8.3.2.
The potential is V(0) = ∞, and V(x) = βx for x > 0 (Figure 9.9).
 Figure 9.9: A hard wall at x = 0 plus a linear potential for x > 0.

We can still use the SHO basis set, but only half of it. The
even-n basis states of Eq. (9.33) are nonzero at the origin, and
must be discarded because they do not satisfy the boundary
conditions. The odd-n basis states, however, do satisfy the
boundary conditions. They are what we need. In fact, these
states are eigenstates of the SHO in the half-open space:
, for x > 0. Therefore, they are a complete
basis set for the half-open space.

We can use the odd states as is in the BEM, provided we

renormalize them by multiplying the normalization constants
An by a factor . This accounts for the loss of probability in
the other half space. With only odd states, we are effectively
removing from the kinetic and potential matrices the rows and
columns due to even states (Project S:P9.2).

Similarly, for the potential shown in Figure 9.9, a valid wave

function must be zero at x = 0. The odd solutions given by Eq.
(9.61) satisfy the Schrödinger equation everywhere in the half-
open space including the correct boundary conditions, so they
are the eigenfunctions of the system. Accordingly, the valid
eigenenergies are those corresponding to odd states. For
instance, we can obtain the first several values from Table 9.2
by omitting entries for n = 1, 3, 5 and keeping n = 2, 4, 6. We
can confirm this numerically (Project S:P9.2) with the values
determined by the zeros of the Airy function (9.63).
 The Morse potential is another interesting case defined in
the half space x ≥ 0 as

This potential approximately describes the interaction

between two atoms in a diatomic molecule at internuclear
distance x. The parameter r0 represents the equilibrium
distance where the potential is a minimum, −V0 (Figure 9.10).
The potential rises rapidly as x → 0, simulating the hard wall
(e.g., the Coulomb repulsion), and decreases exponentially at
large x → ∞.

Around the minimum, the potential is approximately like an

SHO potential, so the Morse potential is a useful model to
describe vibrational states of a molecule. The energy levels
computed by the BEM (Project S:P9.3) show roughly
equidistant level spacing for low-lying states. As energy
increases, the potential becomes anharmonic, and the spacing
decreases, in accord with our expectation (Figure 9.8) for
realistic potentials.
 Figure 9.10: Energy-level diagram for the Morse potential.

Quantitatively, the results shown in Figure 9.10 were

obtained with realistic parameters for molecular hydrogen, H2
(Project S:P9.3). The BEM code found 17 bound states. The
largest gap is ΔE10 = 0.51 eV (wavenumber 4100 cm−1) between
the ground state and the first excited state, and the smallest
gap at the top is 0.04 eV (300 cm−1). These are typical of
vibrational excitation energies in molecules. They are larger
than thermal energies at room temperature , so
vibrational degrees of freedom are usually frozen at these
temperatures.

.5 Central field potentials

An important class of systems consists of central field
potentials such as the Coulomb potentials and central (mean)
field approximations, e.g., Section 4.3.4. In these systems, the
three-dimensional Schrödinger equation (S:12.1) can be
separated into radial and angular solutions, ψ = Rnl(r)Ylm(θ,
φ). The latter, Ylm, are the universal spherical harmonics [4].
They are eigenfunctions of angular momentum which is
conserved in central fields.

The radial wave function, Rnl, satisfies the radial

Schrödinger equation

The first several hydrogen radial and angular wave functions

are given in Project S:P8.9.
 Figure 9.11: Energy-level diagrams of hydrogen (−1/r, left), and
1.1
modified potential (−1/r , right).

For a general potential, the energy E depends on both n and

l, the principal and angular momentum numbers, respectively.
For a pure 1/r potential like in the hydrogen atom, the energy
becomes independent of l, i.e., it is degenerate with respect to l
(Figure 9.11, left). The Coulomb potential is the only potential
having this degeneracy. When modified slightly (Figure 9.11,
right), the degeneracy disappears (see Project P9.8).

Equation (9.38) can be simplified further by making a

substitution u = rR to arrive at

The effective potential Veff is the same as Eq. (4.8) for planetary
motion, with the replacement of L2 → l(l + 1)ħ2. The limiting
behavior of u is , and .

Equation (9.39) is identical to the one-dimensional

Schrödinger equation (9.8) with the introduction of Veff. We
can apply all the methods discussed so far to (9.39), subject to
the boundary conditions u(0) = u(∞) = 0. In particular,
Numerov's method is very efficient for solving central potential
problems. Program 9.5 combines the Numerov and shooting
methods to solve for the eigenenergies and wave functions in
central fields such as the hydrogen atom, shown in Figure 9.11
and Figure 9.12, respectively.

Figure 9.12: The radial wave functions of the first six states of
hydrogen. The dotted lines represent the effective potential.

The effective potential for each angular momentum number

l supports an infinite number of states in hydrogen, all
converging toward the continuum E = 0. Each state can be
labeled by a pair of quantum numbers, n and l. The lowest state
in each l-series starts from the second-lowest level of the
previous series. For a given n, the maximum value of l is lmax =
n − 1. For a given l (fixed angular momentum), the smallest
permitted n is l + 1. The results for hydrogen are in good
agreement with the exact eigenenergies .

Even though the Coulomb potential vanishes at r → ∞, the

number of bound states is still infinite, unlike most potentials
of similar asymptotic behavior. This is attributed to the
particularly long range nature of the Coulomb potential. The
rate of decrease as 1/r is slow enough that a particle “feels” the
effect of the potential even as r → ∞.7

The radial wave functions of the first several states of

hydrogen are shown in Figure 9.12. They are qualitatively the
same for other central fileds. The states are labeled by the
principal quantum number n followed by a letter according to
spectroscopic notation where the values l = 0, 1, 2, 3, … are
represented by s, p, d, f…, etc. For the ground state 1s, the wave
function is tightly localized around r = 1. With increasing n and
l, it becomes more spread out. Higher energy pushes the
electron further outward, resulting in a higher potential energy
and a lower kinetic energy. We can show that on average, the
two energies in hydrogen are related by

The above relationship is called the virial theorem. The

increase in potential energy is twice the decrease in kinetic
energy, so the total energy increases.
 We see from Figure 9.12 that for a given n, the wave
function becomes more localized toward larger r with
increasing l. The centrifugal potential wall keeps the wave
function small at small r (e.g., compare 2s with 2p, or 3s and
3p with 3d). When l = lmax = n − 1, there is a single peak
centered at r = n2. This result can also be proven analytically
(Exercise S:E9.1). It shows two things: the size of the atom
scales as n2, and the radial probability distribution is
increasingly confined to a spherical shell.8 If we make a cut
through the shell structure, we would obtain a circular
distribution, or a ring. For this reason, states of maximum
angular momentum l = lmax are called circular Rydberg states
[11]. Figure 9.13 displays the radial probability densities of 4s
to 4f states, showing the approach to circular states with
increasing l. In the semiclassical limit n 1, these circular
Rydberg states look increasingly like the circular orbits of
classical motion. This is the limit of the Bohr model, a fact used
later (see Section S:12.5.2).
 Figure 9.13: The probability density for 4s to 4f states in the x-y
plane.

We also observe that the number of nodes for a given state

is equal to n − l − 1, which is zero for circular states such as 3d.
This may seem paradoxical to the notion we have developed so
far, which is that the k-th excited state should have k nodes.
For three-dimensional systems, the wave function can have
nodes in radial as well as angular variables, so the number of
nodes should be the sum in all directions. The fact that the 3d
state has fewer nodes than the 2s state in the radial wave
function means that there must be more nodes in the angular
wave function for 3d than for 2s. This is in fact true.
 Figure 9.14: The probability density of the circular state (n, l) = (4, 3)
for m = 0 to 3 (different orientations) in the x-z plane.

In Figure 9.14 we plot the radial and angular probability

density distribution, |unlYlm|2. Due to azimuthal symmetry, we
show a cut in the x-z plane. We choose the circular state (n, l) =
(4, 3), and vary the quantum number m = 0 to 3. Because m
measures the z component of the angular momentum (Lz), we
can view different m values as representing the orientation of
the classical circular orbit. The maximum Lz occurs when |m| =
l, and we expect the orbit to be in the x-y plane so the central
axis (normal) of the orbit is aligned with the z-axis. Figure 9.14
shows that the probability density in this case (m = 3) is mostly
 confined to the space near the x-y plane where , or |z| ~ 0.
Conversely, if m = 0, the central axis should be perpendicular
to the z-axis. We expect the probability density to be
concentrated near the poles, θ ~ 0 and π. This case is depicted
in Figure 9.14 (top-left, m = 0). For 0 < |m| < l, the density
evolves between the two limits.

Furthermore, we can count the number of nodes in the

angular wave function from Figure 9.14. For example, we see
three valleys (zero density) between θ = 0 and π in the m = 0
case, corresponding to three angular nodes. Because the radial
wave function u4,3 has zero nodes, the total number of nodes is
three in the state nlm = (4, 3, 0). This is equal to the number of
radial nodes in the 4s state which has zero angular nodes.

The number of angular nodes has no direct impact when we

are only interested in computing the radial wave function. But
counting radial nodes in a given l-series is helpful numerically
because it lets us know if we have missed any state when
searching for eigenenergies.

.6 Quantum dot
We have discussed solutions to 1D quantum systems and 3D
central field problems that are effectively one-dimensional in
the radial direction. In this section, we will study 2D quantum
systems.
 Perhaps the most interesting among them are quantum
dots [45], systems confined to sizes in the nanometer range.
Some occur in lattice structure, but they are often fabricated in
semiconductors using techniques such as precision
lithography, and are sometimes called designer atoms.9 Their
shapes can be simple or highly irregular. In the latter case,
enforcing boundary conditions can be problematic. One of the
advantages of the FEM is its ability to adapt to flexible
boundaries. We will extend the FEM to solve the Schrödinger
equation in 2D and use it to study quantum dots.

9.6.1 FEM SOLUTIONS OF SCHRÖDINGER

EQUATION IN 2D
The Schrödinger equation in two-dimensional space is

where is the Laplacian in 2D.

We can follow two paths to develop the FEM for the

Schrödinger equation: starting with explicit expansion
analogous to Eq. (9.12) in 1D, or using the readily available
results from Chapter 7 (Section 7.3) for 2D FEM. We choose
the latter.

For the purpose of FEM development, the Schrödinger

equation (9.41) can be cast into an equivalent form to the
Poisson equation (7.13). If we multiply both sides of Eq. (9.41)
by −2 and compare with Eq. (7.13), we can identify the
equivalent function in the latter as f = 2(V − E)u.

The framework of FEM in 2D had been fully developed in

Chapter 7 (Section 7.3) with the Poisson equation as the prime
example. Therefore, we can follow the exact development down
to a tee for application to the Schrödinger equation. The only
difference is in the imposition of boundary conditions (7.22).
That is where we start.

Equation (7.22) reads,

where the sums on the LHS and RHS refer to internal and
boundary nodes, respectively. As before, ui is the solution in
the expansion (7.9), and Aij, qj, and pj are respectively defined
in Eqs. (7.19) and (7.17).

We are interested in Dirichlet boundary conditions for

bound states, where the wave function vanishes on the
boundary, i.e., uk = 0 if node k is on the boundary.

Let us consider each of the three terms on the RHS of Eq.

(9.42). The first term, qj, is zero if j is an internal node, because
the tent function ϕj in Eq. (7.17) centered on an internal node is
zero on the boundary (see Figure 7.8 and discussion following
it, as well as the discussion right after Eq. (7.22)). Hence, qj
drops out. The third term, , is also zero since the
sum involves only boundary nodes where uk = 0.

That leaves the second term, pj, as the only nonzero term.
The expression for pj follows from Eq. (7.17) with f identified
above for the Schrödinger equation

Substituting the basis expansion u = Σ uiϕi from Eq. (7.9)

into (9.43), we obtain

Like Eq. (9.14) in 1D FEM, Vij is the potential matrix and Bij the
overlap matrix between the tent functions ϕi and ϕj.

Substituting pj into Eq. (9.42), dividing both sides by 2, and

moving the potential matrix to the LHS, we obtain

In matrix form, Eq. (9.46) can be written as

where is the kinetic energy matrix. This final result is
again a generalized eigenvalue problem that is formally the
same as Eq. (9.17) for the 1D Schrödinger equation. In
retrospect, we may have guessed Eq. (9.47).

We have discussed triangular mesh generation in Section

7.3. On a given mesh, the kinetic and overlap matrices can be
calculated from

It is assumed that the summation runs over all elements e that

contain nodes i and j. We have already encountered which is
given by Eq. (7.27). For , we can evaluate it similarly as [75]

The quantity Ae is the area of the triangle whose vertices are

given by (xk, yk) in Eq. (7.10). The parameters αi, βi, and γi
define the basis functions over the triangular element e given in
Eq. (7.11).

For a given potential, we can evaluate the potential matrix

on the mesh by numerical integration. We will assume simple
potentials for the quantum dot: either zero or constant in the
domain (of course, V = ∞ outside where u = 0). As a result, the
potential matrix either vanishes or is proportional to the
overlap matrix .10 Once we have all three matrices, we can
solve the generalized eigenvalue problem (9.47).

9.6.2 FEM IMPLEMENTATION

We break down the FEM approach into three separate tasks:
mesh generation, matrix preparation, and actual computation.

We treat mesh generation as an independent process. As

discussed in Chapter 7 (Section 7.3), proper meshing is
important to the quality of the solution (see Table S:9.1) and
computational efficiency. We generate all the meshes for our
systems in simple domains (see Figure 7.11, Table 7.1, and
examples in Programs 7.3 and 9.9). They can also be generated
independently by other methods including MeshPy and can be
read from a file (see Figure 9.24 and associated sample file
format).

On a given mesh, we calculate the A and B matrices defined

in Eqs. (9.48), (7.27), and (9.49). The necessary routines are
included in the FEM library (Program 9.6). It includes a pair of
functions A_mat() and B_mat() that accept the nodes and
elements defined according to the data structure given in Table
7.1.

Actual computation and results presentation are performed

with Program 9.7. We consider it a universal FEM program for
quantum dots where the potential is zero in the domain. It
requires only mesh data stored in a file. Additionally, it
calculates eigenenergies and eigenfunctions only once, and
writes the data to a file. Subsequently, the data can be loaded,
analyzed and graphed without re-computation.

The triangular box

Figure 9.15: The FEM mesh for the right triangular box.

As a test, we apply Program 9.7 to the isosceles right

triangular box (see Exercise E9.12). This system is exactly
solvable so we can directly compare numerical and analytical
results. A sample mesh is shown in Figure 9.15. The isosceles
sides are divided into N = 10 intervals each. There are 100
elements. The mesh setup properly preserves the symmetry of
the geometry, including reflection symmetry about the bisector
of the right angle.
 The actual mesh used below is for N = 20, or a total of 400
elements. Of 231 total nodes, 171 are internal, so the
dimensions of A and B matrices are 171 × 171, and there are a
total of 171 states. The mesh data is stored in a file and is read
in by Program 9.7. Figures 9.16 and 9.17 display the
eigenenergies and wave functions, respectively, of low-lying
states. The latter is obtained from a separate code, Program
9.8, which plots the wave function over a triangle mesh.

Figure 9.16: The first 20 eigenenergies of the right triangular box.

The open circles ( ) are FEM results, and filled circles (•) analytic
results.

Comparing numerical and analytic eigenenergies, we

observe good agreement for the lower states, within ~ 1% for
the first 5 states. Analytically, the solutions can be obtained
from the 2D box as
where a is the side length. In Eq. (9.50), m and n are nonzero
integers, and m < n. The wave function is a combination of 1D
wave functions in the x and y directions (S:8.39), satisfying the
boundary condition that the wave function is zero on the three
sides, umn(x, 0) = umn(a, y) = umn(x, y = x) = 0. The
eigenenergies are given by

In contrast to the rectangular box which has degeneracies,

the triangular box removes degeneracy all together, a result of
boundary conditions requiring that the wave function vanish
on the hypotenuse. This restricts the way the wave function can
oscillate. As Figure 9.17 shows, the ground state has one peak
in the center of the triangle. With increasing state number,
more pockets are formed, which are generally correlated with
the state number, but not always. Sometimes, the pockets are
broadly-connected local maxima or minima (e.g., states 3 and
4). They develop into clear peaks at later stages (e.g., states 4 →
5).
 Figure 9.17: Wave functions for the first 9 states in a triangular box.

The wave function (9.50) in the triangular box is not

separable into a product of x and y variables. Therefore, the
concept of nodes is not readily applicable. Instead, we must
identify the wave function by nodal lines or the pattern of the
extrema. A higher number of extrema corresponds to a higher
state. If two states have the same number of extrema, the wave
function with sharper and better developed peaks belongs to
the higher state.

Going back to the eigenenergies shown in Figure 9.16, the

discrepancy between the numerical and analytic results
increases toward higher states, and numerical values are
always above the analytic values for a given state. At the 20th
state, the error is about ~ 15%. The difference comes from two
sources, the accuracy of the FEM, and the related number of
representable states. The difference can be reduced with a finer
mesh, which in turn increases the number of states in the FEM
representation. This will delay the onset of large errors, but
they will happen eventually.

In other words, the fundamental problem remains, which is

to represent a system having an infinite number of states with a
finite number of representable states in a numerical method.
We had seen the same phenomenon for standing waves on a
string (Section 6.6). Essentially, we are trying to mimic an
infinite Hilbert space with a finite number of states in our basis
set. So it is really not the fault of the numerical method. What
we must keep in mind is that higher states have larger errors.
In the present case, we can trust the energies of states up to 8
at 1% accuracy, or about 5% of the total states (171). A general
guide requires a statistical analysis of energy levels (Section
S:9.1).

9.6.3 THE HEXAGON QUANTUM DOT

The hexagon quantum dot is very interesting. Its shape is
highly symmetric, and appears naturally in semiconductors
such as GaAs or in a honeycomb as the basic cell.

Figure 9.18 shows a sample mesh of the hexagon quantum

dot. The side length, assumed to be 1, is divided into N = 2
intervals for illustration. It is generated with Program 9.9. The
elements are equilateral triangles, preserving all the rotation
and reflection symmetries. Note that the node number starts
from the bottom-left corner and increases consecutively from
left to right and bottom to top, until the end of the center row,
where it jumps to the top-left corner. This has to do with the
way the mesh was made, in which the lower half was generated
first and the upper half obtained by reflection (see explanation
of Program 9.9).

Table 9.3: The lowest 12 eigenenergies of the hexagon quantum dot.

Figure 9.18: Sample mesh for the hexagon quantum dot.

 The number of elements is equal to 6N2. We use more
elements than shown in Figure 9.18 in actual computation. We
have performed a series of calculations with N = 10 to 40. This
is done in part to gauge accuracy and to understand the rate of
convergence. In Table 9.3 and Figure 9.19, we show
eigenenergies and wave functions calculated with N = 40 and
9600 elements. The hexagon quantum dot does not yield
analytic solutions in general. Analytic solutions can be obtained
only for a limited subset of highly symmetric states. Even then,
they are not in closed form [57]. Judging from the convergence
sequence, the relative error in the eigenenergies of Table 9.3 is
about ~ 10−3.

The energy spectrum is very interesting. The ground state,

which must be nondegenerate, has an energy 3.57866. With
regard to geometry, we can compare this value to three wells of
comparable size, two circular wells – an inner circular well of
diameter that just fits inside the hexagon and an outer one
of diameter 2 that just encloses the hexagon, and a rectangular
well of dimensions . The eigenenergies of the circular
wells are related to the zeros of the Bessel function, and that of
the rectangular well is well known. In order, their values are
3.85546, 2.89159, and 2.87863, respectively (Exercise S:E9.5).
The inner circular well gives the best fit to the hexagon, and its
energy is the closest. It is higher than the actual energy because
its area is smaller than the area of the hexagon. This is due to
the uncertainty principle which generally dictates a larger
momentum for a smaller well, thus a higher energy. We see the
same trend for the outer circular well with an area π and the
rectangular well with an area .

Figure 9.19: Wave functions in the hexagon quantum dot.

Immediately above the ground state, the first and second

excited states are degenerate, as are the next pair, the third and
fourth excited states. After states 6 to 8 which are
nondegenerate, degeneracy resumes for the next two pair of
states, (9, 10) and (11, 12). The pattern continues for later
states.

The high frequency of degeneracy is a reflection of the high

symmetry of the hexagon structure. It is symmetric under
rotations of about the center, and under reflections about the
“diagonals” and bisectors.11 In addition, we can imagine our
hexagon is one unit in an infinite honeycomb. Then the system
has translation symmetry in several directions. In contrast, the
triangular quantum dot discussed earlier lacks these
symmetries, and as a result, shows no degeneracy.

Even though the numerical results may have only limited

accuracy, the degenerate energies are exactly equal. This is
because our mesh preserves the exact symmetry of the system.
We expect this symmetry to be carried over to the wave
functions.

The corresponding wave functions are shown in Figure 9.19.

The ground state has a single maximum as expected. States 2
and 3, which are degenerate, have a peak and a valley each,
separated by nodal lines. In state 2, the nodal line runs along a
vertical bisector and divides the hexagon into two irregular
pentagons. In state 3, it is along the horizontal diagonal and
divides the hexagon into two trapezoids. The next pair of
degenerate states 4 and 5 have similar saddle-like patterns
slightly rotated off each other, but no simple nodal lines. State
4 has two straight nodal lines running along a diagonal and a
bisector (like × but in perpendicular directions), while state 5
has two curvy nodal lines running roughly along two bisectors
(like χ).

With increasing state number, the patterns become more

complex. State 6 has no side-touching nodal lines, but the
central peak looks like the peak of the ground state stuck in a
bowl. Figure 9.20 compares the two more clearly. The very
similar patterns of states 7 and 8 suggest they might be
degenerate, but are in fact not. A closer examination of state 8
shows that each of the six peaks or valleys is completely
enclosed by an equilateral triangle, so they are the ground state
of the equilateral triangle quantum dot. This hexagon state
consists of 6 triangular states stitched together. The last two
degenerate pairs of states, (9, 10) and (11, 12), show the rapid
transition from underdeveloped extrema to sharp peaks and
valleys (see Figure 9.20). Electron densities of quantum dots
have been imaged experimentally [26], and the single-electron
states form the basis for understanding many-electron
quantum dots [77].
 Figure 9.20: Surface plot of select wave functions from Figure 9.19.

Chapter summary
The focal area of this chapter is the simulation of time-
independent quantum systems. We studied systems such as
simple wells, linear potentials, central field potentials, and
quantum dots. We discussed several methods for solving the
Schrödinger equation to obtain the energy spectrum and the
wave function. Solutions to any given potential including
singular potentials can be found using at least one of the
methods presented, making the study of any quantum system
within our grasp. Furthermore, each of the methods can be
used with minimum modification, at most requiring the user to
supply a custom potential function.

With little preparation, we can use the standard shooting

method, first introduced for projectile motion, with an ODE
solver and a root finder (e.g., RK4 and bisection) to obtain
solutions directly. A more efficient approach is to replace the
ODE solver with Numerov's method for increased speed and
accuracy, while maintaining robustness and versatility.

We also discussed the application of the familiar finite

difference and finite element methods previously used for
boundary value problems to eigenvalue problems. For 1D
eigenvalue problems, the FDM is slightly simpler than the
FEM, but has no performance advantage over the FEM. Except
for pedagogical reasons, the FEM is preferred because it is able
to deal with point potentials.

The basis expansion method can be used as a general

meshfree method that depends on the basis functions rather
than a space grid since space is not discretized. With an
appropriate basis set to a given problem, this method can lead
to very accurate solutions even for large systems.

We have devoted considerable effort to the study of

quantum dots in this chapter, and developed a full FEM
program to solve the Schrödinger equation in 2D for arbitrary
boundaries. The program is robust and simple to use, including
a small FEM library fem.py that can be used for other general
 purposes. One may start using the code first, and going through
its development after you are familiar with its flow and
operation.

Throughout, we have made use of several functions (Airy,

Bessel, Hermite, etc.) from the SciPy special function library.
We have also discussed file I/O for storing data and avoiding
recalculation. Effective graphing techniques have been
demonstrated using Matplotlib's triangle plotting capability,
which proved to be an excellent fit for working with data over
triangular meshes such as in FEM.

.7 Exercises and Projects

Exercises
E9.1 Solve for the eigenenergies of the square well potential
shown in Figure 9.1. The eigenequations for even and
odd states are given by (in a.u.)

Assume a = 4 and V0 = 4. Also, convert your results to

eV and J, respectively. Does it make sense to use the
latter?
 You can find the roots one by one using the bisection or
Newton's method, or SciPy's equation solver fsolve
(see Exercise E3.8). You may also wish to try SymPy's
solver which can find them all at once. Give it a try, see
Exercise E3.10.

E9.2 (a) Use the same method as in Exercise E9.1 to

compute the eigenenergies of a single-well potential of
width (a − b)/2, where a and b are the parameters for
the double-well potential (9.6). Compare with the
double-well energies in Figure 9.3 using the same
parameters.

(b) Shrink the barrier width b by half, and calculate the

wave functions with Program 9.2. What happens to the
double hump?

Predict what will happen to the energy and wave

function if the barrier width is increased, but the well
width (a − b)/2 is kept the same. Repeat the calculation
by doubling b. Discuss your results, and check
degeneracy.

E9.3 (a) Verify the first derivative formula (9.60). Subtract

the Taylor series y(x ± h) from Eqs. (2.9) and (2.10),
keep terms up to h3, and approximate y′″ from Eq.
(9.53).

(b) Carry out a von Neumann stability analysis (6.76)

for Numerov's method (9.59), and show that it is stable
 regardless of the step size. Assume a simple function,
e.g., f = c.

(c) Experiment with instability in unidirectional

integration. Pick a correct energy for the single well
potential, e.g., the ground state energy from Table 9.1 or
Exercise E9.1, integrate the Schrödinger equation from
−R to R (or where the wave function diverges badly)
with Program 9.1 or 9.2. Plot the wave function.
Reverse direction of integration. Discuss your
observations.

E9.4 Fill in the steps leading from Eq. (9.12) to (9.13) in 1D

FEM.
E9.5 Show that the expectation values of kinetic and
potential energies

are given by Eq. (9.19) in terms of the respective and

matrices, Eqs. (9.16a) and (9.16b).

E9.6 (a) Derive the eigenequation (9.24) for the Dirac δ

molecule. Divide the space into three regions separated
by the δ potentials, apply the boundary condition at x =
 ±∞, and finally match the wave function at ,

including the cusp condition (9.22).

(b) Solve Eq. (9.24) for the symmetric potential (α = β)

to express the solutions (9.25) in terms of the Lambert
W function. Refer to Chapter 3 (Section 3.4) for an
example on the general approach, e.g., after Eq. (3.29).
You can also solve it more quickly with SymPy, see
Exercise E3.10.

E9.7 A delta-wall potential, shown in Figure 9.21, is

composed of a delta potential placed to the

right of a hard wall at x = 0.

Figure 9.21: The delta-wall potential.

Show that there exists a bound state only if αa > 1, the

same condition for the existence of the excited state in
the δ molecule (9.26), and that the energy is also the
same (Eq. (9.25), k−).
E9.8 Show that in the BEM, the Schrödinger equation (9.28)
can be reduced to Eq. (9.17) with the matrix elements
given by Eq. (9.29). Fill in the detail following similar
projection methods from Eq. (S:8.24), or from Eq.
(9.12) keeping the kinetic energy operator intact, i.e., no
integration by parts.
E9.9 Derive the potential matrix element Vmn (9.31) for the
square well potential. Show that Vmn = 0 unless m + n is
even, in which case

E9.10 Show that the matrix elements for x, x2, p, and p2 in the
SHO basis are

where mj = m + j. The easiest way is to express the

position and momentum operators in terms of the
raising and lowering operators as
 where a0 is the length scale given in Eq. (9.33). When
acting on an eigenstate, the operators â and â+
respectively lowers or raises the state by 1 [44],

E9.11 (a) Given the potential V = β|x|, use variable substitution

to arrange the Schrödinger equation (9.1) in the form of
Eq. (9.62), the differential equation for Airy functions.
Show that the solutions are given by Eq. (9.61).

(b) Match the wave functions at x = 0 and obtain the

eigenenergy equations (9.63) for even and odd states,
respectively.

E9.12 Use the supplied mesh data file to calculate the

eigenenergies and wave functions of the triangle
quantum dot with Program 9.7. Compare the
eigenenergies with the exact values (9.51) and plot the
relative error for all states. Also plot the wave functions
of the first 12 states like Figure 9.17 or Figure 9.20.

Projects
P9.1 (a) Run Program 9.1 with E starting below the bottom
of the well, say −2V0 < E < −V0. Describe the shape of
the wave function and explain why no bound state
exists.
 (b) Modify Program 9.1 to find the odd bound states.
Instead of the derivative being zero at x = 0, the wave
function itself is zero. Modify the conditional statement
in the main loop to detect the sign change in the wave
function. Also add a negative sign to the wave function
for x > 0.

With the same parameters as in the program, how many

odd states are there? Where do the energies fall in
relation to the energies of even states?

(c) Modify the program so it can find both even and odd
states. Double the width of the well, find the number of
all bound states (both even and odd). Do the same, but
double the depth. Which way is more effective at
increasing the number of bound states? Briefly explain.

(d) Change the potential to the SHO potential ,

and find all states up to E = 10. Compare with Eq.

(9.32) and comment on your results.

P9.2 Consider the distorted harmonic potential V(x) = V0 (|x|

− x0)2, graphed in Figure 9.22. If x0 = 0, it is the SHO
potential. Otherwise, the potential has a double well at x
= ±x0, and approaches the SHO potential for large |x|

x0.
 (a) Let and x0 = 0. Use the shooting method

(Program 9.2) to compute the first 10 to 20

eigenenergies. Plot and inspect a few wave functions.
Compare the spectrum with Eq. (9.32). Are the states
equidistant? Experiment with the parameters, including
range, number of grids, and energy increment, etc.

Figure 9.22: The distorted harmonic potential.

(b) Now let x0 = 1.2. Repeat the above calculations.

Discuss your results in relation to the SHO energies (x0
= 0).

(c) Move the matching point M around, say just below

or above x0. Calculate the energy levels for three
different M values. Do the energies change? Should
they? Why?

P9.3 Modify Program 9.2 to use Numerov's method (9.59).

Let the matching point be M. Integrate the Schrödinger
equation upward or downward one step beyond M in
 both directions, so we can calculate the derivative using
Eq. (9.60). See Program 9.5.

Profile your code to test the performance gain between

Numerov's method and RK45 using the same step size
for Project P9.2, for example. Optionally repeat the test
for an SHO if the same accuracy is required.

P9.4 (a) Write a program to calculate the eigenenergies in the

square well potential using the FDM method. Prepare
the and matrices according to Eq. (9.10), and use

the same step size h as the shooting method (Program

9.2).

(b) The square well is infinitely sharp, but numerically

our grid is finite. This causes numerical error if the
potential jumps from −V0 to 0 at the walls. Rather than a
steep rise, the result can be improved if we linearly
interpolate the potential across the edges (see Eq.
(3.51), Project P3.2). Modify your potential matrix so
that the two values at the grid points immediately

outside the well on each side is . Repeat the

calculation. Comment on the improvement of accuracy.

P9.5 (a) Evaluate the analytical matrix elements (9.18) for

the square well potential. Replace the potential matrix
in Program 9.3 by your analytical results. Repeat the
calculation to reproduce the results in Table 9.1.
 (b) The sharp edges of the square well affects the
accuracy of the results. To improve it, linearly
interpolate the potential between the two elements
bracketing the edges at ±a/2 as

where h is the element size. The interpolated potential is

identical to the actual potential outside the zones
between the two immediate elements sandwiching the
edges. In the zones, it varies linearly from −V0 at |x| =
a/2 − h to 0 at |x| = a/2 + h. For best result, set the grid
so the edges (±a/2) are also the nodes.

Use this potential in Program 9.3 and redo the

calculation. Discuss your results.

P9.6 (a) Calculate the energy and wave function of the bound
state of the Dirac δ atom. Assume α = 1. Use the FEM
code, Program 9.3, and modify the potential matrix so
that the only nonzero matrix element is at the node
where the δ potential is located. Plot the numerical and
analytic wave functions in the same figure, normalize
both at the center. Compare numerical and exact results
for the energy and wave function.

(b) Check the quality of the wave function near the

center. What are the numerical cusp values, u′(0±)/u(0)?
How do they compare with the exact value?
 (c) Calculate the expectation values of kinetic and
potential energies from Eq. (9.19). See description of
Program 9.3 on how to proceed.

(d) Compute the expectation values of 〈x〉, 〈x2〉, 〈p〉, and

〈p2〉. Use the matrix representations of the operators
from Exercise S:E9.3 if available. If not, either derive
them, or use numerical integration to find 〈x〉 and 〈x2〉.
Find the uncertainty ΔxΔp, and verify that it satisfies
the uncertainty principle.

(e)* Predict and sketch the wave function in momentum

space. Calculate and plot the wave function using FFT.
Estimate the widths Δx′ and Δp′ from their respective
wave functions. Compare Δx′Δp′ with the result above.

(f)* Simulate the δ potential with the FDM. Represent

the singularity as at x = 0 and zero elsewhere,

where h is the grid spacing. Calculate the energy and

cusp condition, and compare with the FEM results
above.

P9.7 (a) Using the results from Exercise E9.9, modify

Program 9.4 to compute the bound state energies for the
square well potential with the box basis set. Also plot
the corresponding wave functions.

(b) Calculate the eigenenergy of the Dirac δ-atom using

the BEM. Compare with results from Project P9.6 for
 the same α. Change the number of states N in your
basis, and determine the minimum N to achieve a
comparable accuracy as the FEM.

P9.8 Study the energy structure of several model central

potentials. Choose one potential, and use Program 9.5
as the base code.

(a) First, let us consider a potential that deviates slightly

from the Coulomb potential in the power law,

Vary ∈ = 0.01 to 0.1. Compute the energy-level diagram

as shown in Figure 9.11. Compare with the hydrogen
atom (Z = 1), and summarize the similarities and
differences.

Note that all states are higher than hydrogen, except the
1s state. Explain why (a figure similar to Figure 9.12
should be helpful).

(b) Next, consider the screened Coulomb potential

(Yukawa),
The parameter a is the screening length. This potential
behaves like the Coulomb potential for r → 0 but has a
much shorter range. Let a = 20 and Z = 1. Calculate the
energy-level diagram as above. Find the energy
difference between 2s and 2p states. Is the number of
bound states finite?

Double Z = 2, repeat the calculation. Observe the

difference with Z = 1.

(c) Last, consider the Hulthén potential,

where Z is the nuclear charge, and a the screening

length. What is the potential like in the limits r → 0 and
∞? Based on your observations from above, sketch the
energy-level diagram.

Choose Z = 1 and a = 10, and calculate the energy-level

diagram. Find the largest l supported by the potential.
Plot the wave function for the s states, and compare
both with the hydrogen atom. Examine the number of
nodes in the wave function so as not to miss any states.

Repeat the calculation for Z = 2, keep the same a.

 (d)* Modify your program to compute the expectation
values of kinetic and potential energies, respectively.
Since the wave function is obtained on a regular grid,
we can use Simpson's rule (Section 8.A.2) to carry out
the numerical integral. For 〈T〉, do not compute the
derivatives directly. Rather, use either the first
derivative (9.60) in Eq. (8.18), or the second derivative
from Eq. (9.8) with the integral in Exercise E9.5. In
addition, properly scale your results by C = ∫ u*udx,
because the wave function is not normalized.

Test your program on hydrogen atom to make sure it

gives the correct results according to the virial theorem
(9.40). For potentials given above, compute 〈T〉 and 〈V〉
for the lowest state in each l-series. Do they still obey
the virial theorem? Comment on your results.

P9.9 (a) Calculate and plot the wave functions of the unit
circle quantum dot (billiard, Section S:9.2.1) using the
supplied mesh data file. Discuss the symmetry of the
degenerate states.

(b) Generate a mesh for the circular billiard so it

preserves rotational symmetry within the grid
representation. One way to do this is start with the mesh
grid of the hexagon or octagon, scale the points off the
circle symmetrically about the radial they are on.
Calculate the eigenenergies, and compare the
degenerate states against Table S:9.1.
 P9.10 (a) Generate a mesh for the hexagon using Program 9.9.
Plot the kinetic energy matrix (or equivalently the
overlap matrix) similar to Figure 7.12. You should see
bands running along the diagonal initially, but a little
later some bands break, jump away from the diagonal,
and continue from there. This is caused by the
discontinuity in numbering the nodes due to reflection.

(b) Modify the program so the nodes are consecutively

numbered, or post-process the mesh to achieve the same
goal. Check the bandedness by plotting the matrix
again.

(c)* Represent the new matrix in banded matrix format,

see Figure 8.6 and Program 8.3. Solve the eigenvalue
problem using the SciPy eig_banded() function.

Profile your programs and compare speeds between the

standard and banded eigensolvers.

.A Numerov's method
The Numerov method is useful for a second order ODE of the
following form that does not contain first derivatives

It turns out that we can apply a special technique to

discretize it over a grid such that odd derivatives up to the fifth
order can be canceled in the central difference scheme, giving
us a method accurate to h5, h being the grid size as usual.

To show this, we apply an operator to Eq. (9.53),

Using the Taylor series for y(x ± h) from Eqs. (2.9) and (2.10)
and adding them up, we have,

or in terms of the y″

which is accurate to order h4 (compare with Eq. (6.31b)).

Substituting Eq. (9.56) into (9.54), we see that the terms y″

″ cancel, leaving us with

To approximate (fy)″, we can use Eq. (9.56) but keep only

the first term so the error is still of the order h4. Switching to
the usual subscript notation, Eq. (9.57) becomes
 Solving for yi+1, we obtain Numerov's method

Numerov's method is a three-term recursion relation. It is

stable (Exercise E9.3) and has a local error of the order O(h6).
This is the same order as the Runge-Kutta-Felhberg method
(RK45, Section 2.3.2), which requires six function calls.
Numerov's method is more efficient with apparent three calls
to f(x), but effectively only one new call because the other two
can be reused in subsequent iterations. However, it is not self
starting, as two seed values are necessary to get started.
Program 9.5 contains an implementation of the method. We
will include it in our ODE library ode.py from now on.

We can also obtain the first derivative to a higher order

taking advantage of the special form Eq. (9.53). It can be shown
that the central difference method leads to (Exercise E9.3)

The error is two orders smaller than the standard central

difference formula (6.31a), and is the same as the standard five
point scheme. Equation (9.60) is useful for wave function
matching in quantum problems.
 We note that the standard Numerov method can be
modified so it is applicable to ODEs containing first
derivatives. The interested reader should check relevant
literature if the need arises.

.B The linear potential and Airy function

In the wedged linear potential V = β|x|, the wave function can
be solved (see Exercise E9.11) in terms of a special function
called the Airy function, Ai(x),

The Airy function is defined as a solution to the differential

equation [10]

It has two solutions, the non-diverging regular solution Ai(x)

and a diverging solution Bi(x). We are interested in the regular
solution Ai(x) which is graphed in Figure 9.23.
 Figure 9.23: The Airy function.

Since the potential is even, the eigenfunctions (9.61) have

definite even or odd parities. As we have seen (e.g., Figure 9.3),
even eigenfunctions must have a local maximum at the origin,
u′(0) = 0, and odd ones must have a node, u(0) = 0.
Accordingly, the eigenenergies must satisfy

Let zn and be the n-th zero of Ai(x) and its derivative Ai′
(x), respectively. The eigenenergies can be found from Eq.
(9.63) as

To find the eigenenergies, we need to locate the zeros of the

Airy function. The first several zeros are listed in Table 9.4.

Table 9.4: The first six zeros of Airy function and its derivative.
 If other zeros are needed, they may be generated from the
code below using the SciPy special function library.

from scipy.special import ai_zeros

n = 10 # number of zeros
zn, zpn, zbn, zbpn = ai_zeros(n)
print (zn, zpn)

On return, zn and zpn contain the zeros of Ai(x) and Ai′(x),

respectively. The zeros of Bi(x) and Bi′(x) are in zbn and zbpn,
which are of no interest to us here.

.C Program listings and descriptions

Program listing 9.1: Visual eigenstates in the square well
( squarewell.py)

import numpy as np, visual as vp, ode, vpm

2
def V(x): # potential
4 return 0. if abs(x) > a/2. else −V0

6 def sch(psi, x): # Schrodinger eqn

return [psi [1], 2*(V(x)−E)*psi[0]]
8
# initialization and animation setup
10 a, V0 = 4.0, 4. # well width, depth
R, N = 4*a, 200 # limit, intervals
12 xa = np.linspace(−R, R, 2*N+1) # grid
h, z = xa[1]−xa[0], np.zeros(2*N+1) # step size
14 E, dE, dpsi, psix = −V0, 0.001, 1.0, np.zeros(2*N+1)

16 scene = vp.display(background=(.2,.5,1), range=1.5*a)

wf = vpm.line(xa, psix, z, vp.color.red, .05)
18 pot = vpm.line(xa, .5*np.vectorize (V)(xa), z, (1,1,1), .04) # pot.
V
info = vp.label(pos=(0, −0.6*a, 0), box=False, height=20)
20
while (E < 0.0):
22 psi, x = [.0, .1], −R
for i in range(N): # WF for x <=0
24 psi = ode.RK45n(sch, psi, x, h)
x += h
26 psix[i+1] = psi[0]
psix[N+1:] = psix[N−1::−1] # WF for x > 0 by reflection
28 if (dpsi*psi [1] < 0.): # dpsi/dx changes sign
info. text= ’Energy found, E=%5.4f’ %(E−dE/2)
30 vpm.pause(scene) # any key to continue
else:
32 info. text= ’E=%5.3f’ %(E)
wf.move(xa, 2*psix/max(psix), z), vpm.wait(scene),
vp.rate(2000)
34 dpsi = psi[1] # old dpsi/dx at E
E += dE

The program assumes a symmetric potential well centered

at the origin, and calculates the energies of even states. The
function sch() is explained in detail in Section 9.1.2. The range
of integration is set to [−R, R] where R should be large
compared to the width of the well. The range is divided into 2N
intervals (2N + 1 grid points). To display the potential, the
NumPy vectorize function is used (line 18), which vectorizes a
scalar function so it operates on an input array and returns an
output array.

The program scans energy E from −V0 to 0. At each E, we

integrate the Schrödinger equation from x = −R to the origin,
starting with ψ(−R) = 0 ( psi[0]) and a small, arbitrary value
for the derivative ψ′(−R) ( psi[1]). The wave function for x > 0
is obtained by symmetry via slicing (line 27). We count
backward from N −1 to 0 to get the reflected wave function
from x = h to R.

For even states, the derivative will be zero at the origin if

the energy is an eigenenergy. When the derivative changes
sign, we assume a correct eigenenergy has been found. The
program terminates when E becomes positive, and we have
found all even eigenstates.

Program listing 9.2: Shooting method for double well ( qmshoot.py)

import numpy as np, rootfinder as rtf, ode

2 import matplotlib.pyplot as plt

4 def V(x): # potential

return 0 if (abs(x) > a/2. or abs(x) < b/2.) else −V0
6
def sch(psi, x): # Schrodinger eqn
8 return [psi [1], 2*(V(x)−E)*psi[0]]

10 def intsch(psi, n, x, h): # integrate Sch eqn for n steps

psix = [psi [0]]
12 for i in range(n):
 psi = ode.RK45n(sch, psi, x, h)
14 x += h
psix.append(psi[0])
16 return psix, psi # return WF and last point

18 def shoot(En): # calc diff of derivatives at given E

global E # global E, needed in sch()
20 E = En
wfup, psiup = intsch ([0., .1], N, xL, h) # march
upward
22 wfdn, psidn = intsch ([0., .1], N, xR, −h) # march
downward
return psidn[0]*psiup[1] − psiup[0]*psidn[1]
24
a, b, V0 = 4.0, 1.0, 6. # double well widths, depth
26 xL, xR, N = −4*a, 4*a, 500 # limits, intervals
xa = np.linspace(xL, xR, 2*N+1) # grid
28 h, M = xa[1]−xa[0], 2 # step size, M=matching
point
E1, dE, j = −V0, 0.01, 1
30
plt.figure ()
32 while (E1 < 0): # find E, calc and plot wave function
if (shoot(E1) * shoot(E1 + dE) < 0): # bracket E
34 E = rtf. bisect (shoot, E1, E1 + dE, 1.e−8)
print (’ Energy found: %.3f’ %(E))
36 wfup, psiup = intsch ([0., .1], N+M, xL, h) #
compute WF
wfdn, psidn = intsch ([0., .1], N−M, xR, −h)
38 psix = np.concatenate((wfup[:−1], wfdn[::−1])) #
combine WF
scale = psiup[0]/psidn[0]
40 psix [N+M:] *= scale # match
WF
ax = plt.subplot (2,2, j)
42 ax.plot(xa, psix/max(psix)) # plot
WF
 ax.plot(xa, np.vectorize (V)(xa)/(2*V0)) #
overlay V
44 ax.set_xlim(−a,a)
ax.text (2.2,0.7, ’ %.3f’ %(E))
46 if (j == 1 or j == 3): ax.set_ylabel(r’ψ’)
if (j == 3 or j == 4): ax.set_xlabel( ’x’)
48 if (j<4): j += 1 # 4 plots max
E1 += dE
50 plt.show()

The potential function V() returns the double square well

potential. The functions sch(), intsch() and shoot() have been
explained earlier in the text. The limits xL and xR are coupled to
the width of the double well, which we take as the characteristic
length of the system.

The main loop iterates through the trial energy in equal

increment. When a root is bracketed, it is solved with the
bisection method. Once a true eigenenergy is found, the wave
function is obtained by integrating inward, and matching at
grid M, which should be off-center to avoid the node point for
symmetric potentials. The wave function is combined by
np.concatenate into one array psix for plotting. Note that the
order for the downward wave function is reversed via slicing
wfdn[::-1] before the combination, which is scaled (matched)
according to Eq. (9.3) afterward. The layout of the plots are
assumed to be 2 × 2. For other potentials with more bound
states, it should be modified accordingly.

Program listing 9.3: Eigenenergies by FEM ( qmfem.py)

 import numpy as np
2 from scipy.sparse. linalg import eigsh

4 def V(x): # potential

return 0. if (abs(x) >= a/2.) else −V0
6
def TB_mat(n): # fill in T and and B
matrices
8 Tm, B = np.diag([2.]*n), np.diag ([4.]* n)
Tm += np.diag([−1.]*(n−1),1) + np.diag([−1.]*(n−1), −1) #
off diag
10 B += np.diag([1.]*(n−1),1) + np.diag([1.]*(n−1), −1)
return Tm/(2.*h), B*h/6.
12
def Vij(i, j): # pot. matrix Vij over [xi, xi+1] by order−4
Gaussian
14 x=np.array([0.3399810435848564, 0.8611363115940525])
# abscissa
w=np.array([0.6521451548625463, 0.3478548451374545])
# weight
16 phi = lambda i, x: 1.0 − abs(x−xa[i])/h # tent function
vV, hh = np.vectorize(V), h/2.0 # vectorize V
18 x1, x2 = xa[i] + hh − hh*x, xa[i] + hh + hh*x
return hh*np.sum(w * (vV(x1) * phi(i, x1) * phi(j, x1) +
20 vV(x2) * phi(i, x2) * phi(j, x2)) )
22 def V_mat(): # fill potential matrix
Vm = np.zeros((N−1,N−1))
24 for i in range(N): # for each element # contribution
to:
if (i>0): Vm[i−1,i−1] += Vij(i, i) # left node
26 if (i < N−1): Vm[i,i] += Vij(i+1, i+1) # right node
if (i>0 and i< N−1):
28 Vm[i−1,i] += Vij(i, i+1) # off
diagonals
Vm[i,i−1] = Vm[i−1,i] # symmetry
30 return Vm
32 a, V0 = 4.0, 4.0 # well width, depth
xL, xR, N = −4*a, 4*a, 600 # boundaries, num. of
elements
34 xa = np.linspace(xL, xR, N+1) # nodes
h = xa[1]−xa[0] # size of element
36
Tm, B = TB_mat(N−1) # obtain T, B, V matrices
38 Vm = V_mat()
40 E, u = eigsh(Tm + Vm, 10, B, which= ’SA’) # get lowest 10
states
print (E)

This is a general FEM program for 1D potentials. The

routine TB_mat() prepares the kinetic energy and basis overlap
matrices, both tridiagonal, according to Eqs. (9.16a) and
(9.16c) using np.diag (see Program 6.3). The next couple of
functions Vij() and V_mat() work together to compute the
potential matrix (9.18) as explained in the text.

The main code initializes parameters such as the potential,

space range, the number of finite elements, and nodes, etc. It
then prepares the matrices for the kinetic energy Tm, overlap B,
and the potential energy Vm. The eigenenergies and
eigenvectors are obtained with the SciPy sparse matrix
eigenvalue solver eigsh (line 40), which is more efficient than
the standard eigh routine for large matrices. It works on
symmetric real or complex Hermitian matrices. The
eigenvalues are returned in E as a 1D array, and eigenvectors in
u as a 2D array, in such a way that u[:,i], a 1D array over the
space grid, is the eigenvector corresponding to E[i].
 In many cases, we are interested in only a select group of
eigenvalues, such as the low-lying bound states. It is also faster
to solve only for a subset of the eigenvalues. We specify the
number of eigenvalues (10 in this case) and the sorting option
’SA’ (lowest value first), so this combination of switches gives
us the lowest 10 eigenstates. For larger dimensions, consider
the banded eigenvalue solver eig_banded in SciPy for better
convergence.

The eigenvector, i.e., the wave function, returned by eigsh

is properly normalized with B as the weight (9.20), meaning
that the statement

np.dot(u [:, i], np.dot(B, u [:, i]))

evaluates to 1 for any state i. Therefore, we can readily compute

any average such as the kinetic or potential energies (9.19). For
example, the kinetic energy of the first state (the ground state)
is simply

ke = np.dot(u [:,0], np.dot(Tm, u[:,0]))

The function np.dot calculates the scalar product if the

arguments are 1D arrays. If one of arguments is a 2D array, it is
the same as matrix multiplication. This is the case for dot(Tm,
u[:,0]), which returns a 1D array as a result of a square matrix
Tm multiplied by a column matrix u[:,0]. A second dot
operation computes the scalar product, a single number, from
the resultant and the wave function, both 1D arrays.

Program listing 9.4: Eigenenergies by BEM ( bem.py)

1 import numpy as np, integral as itg

from scipy.sparse. linalg import eigsh
3 from scipy.special import eval_hermite, gamma, ai_zeros

5 def V(x): # potential

return beta*abs(x)
7
def uVu(x): # integrand umVun
9 y, c = x/a0,
a0*np.sqrt(np.pi*gamma(m+1)*gamma(n+1)*2**(m+n))
return
V(x)*eval_hermite(m,y)*eval_hermite(n,y)*np.exp(−y*y)/c
11
beta, omega, N = 1., 1., 20 # pot slope, nat freq, num. basis
states
13 a0, x0 = 1/np.sqrt(omega), 1/(2*beta)**(1./3) # length
scale, x0

15 m, Vm = np.arange(N−2), np.zeros((N,N))
mm = np.sqrt((m+1)*(m+2)) # calc T
matrix
17 Tm = np.diag(np.arange(1, N+N, 2,float)) # diagonal
Tm −= np.diag(mm, 2) + np.diag(mm, −2) # off
diagonal by +/−2
19 for m in range(N): # calc V matrix
for n in range(m, N, 2): # m+n is even every 2
steps
21 Vm[m, n] = 2*itg.gauss(uVu, 0., 10*a0) # use symm.
if (m != n): Vm[n,m] = Vm[m,n]
23 Tm = Tm*omega/4.
E, u = eigsh(Tm + Vm, 6, which= ’SA’) # get lowest 6 states
25
zn, zpn, zbn, zbpn = ai_zeros(3) # find exact values
27 Ex = − beta*x0*np.insert(zpn, range(1, 4), zn) # combine
even/odd
print (E, E/Ex)

For a given potential defined by the user, V(), the program

calculates eigenenergies using the SHO basis set. It generates
the kinetic energy matrix from Eq. (9.36). Since it is banded,
we use np.diag to build the diagonal and off-diagonal (±2)
bands. The potential matrix is computed using numerical
integration. The integrand is defined in uVu() which returns
the product in Eq. (9.29). The basis functions un are
evaluated using the Hermite polynomials from SciPy special
functions library. You can also switch to the recurrence formula
(9.35) to generate your own Hermite polynomials efficiently.

Note that the numerical integral is performed for positive x

only and multiplied by 2 to account for the negative x by
symmetry. This needs to be modified if the potential is not
symmetric. Another caution is in order. If higher basis states
are used, the integrand will be highly oscillatory, and the
integral may become inaccurate. Make sure to check
convergence, e.g., by breaking the interval into two, or varying
the upper limit, etc.

The eigenenergies obtained with eigsh are compared with

exact values from Eq. (9.64) in terms of the zeros of the Airy
function. The odd states are inserted after every even state (line
27).

Program listing 9.5: Hydrogen atom by Numerov's method

( hydrogen.py)

1 import matplotlib.pyplot as plt

import numpy as np, rootfinder as rtf
3
def Veff(r): # effective potential
5 return (L*(L+1)/(2*mass*r)−1)/r

7 def f(r): # Sch eqn in Numerov form

return 2*mass*(E−Veff(r))
9
def numerov(f, u, n, x, h): # Numerov integrator for u″ +
f(x)u = 0
11 nodes, c = 0, h*h/12. # given [u0, u1], return [u0, u1,
…, un+1]
f0, f1 = 0., f (x+h)
13 for i in range(n):
x += h
15 f2 = f(x+h) # Numerov method below, Eq.
(9.59)
u.append((2*(1−5*c*f1)*u[i+1] −
(1+c*f0)*u[i])/(1+c*f2))
17 f0, f1 = f1, f2
if (u[−1]*u[−2] < 0.0): nodes += 1
19 return u, nodes # return u, nodes

21 def shoot(En):
global E # E needed in f(r)
23 E, c, xm = En, (h*h)/6., xL + M*h
wfup, nup = numerov(f, [0,.1], M, xL, h)
25 wfdn, ndn = numerov(f, [0,.1], N−M, xR, −h) # f′ from Eq.
 (9.60)
dup = ((1+c*f(xm+h))*wfup[−1] −
(1+c*f(xm−h))*wfup[−3])/(h+h)
27 ddn = ((1+c*f(xm+h))*wfdn[−3] −
(1+c*f(xm−h))*wfdn[−1])/(h+h)
return dup*wfdn[−2] − wfup[−2]*ddn
29
xL, xR, N = 0., 120., 2200 # limits, intervals
31 h, mass = (xR−xL)/N, 1.0 # step size, mass
Lmax, EL, M = 4, [], 100 # M = matching point
33
Estart, dE = −.5/np.arange(1, Lmax+1)**2−.1, 0.001 # ∼
2
−1/2n
35 for L in range(Lmax):
n, E1, Ea = L+1, Estart[L], []
37 while (E1 < −4*dE): # sweep E range for each L
E1 += dE
39 if (shoot(E1)*shoot(E1 + dE) > 0): continue
E = rtf. bisect (shoot, E1, E1 + dE, 1.e−8)
41 Ea.append(E)
wfup, nup = numerov(f, [0,.1], M−1, xL, h) # calc wf
43 wfdn, ndn = numerov(f, [0,.1], N−M−1, xR, −h)
psix = np.concatenate((wfup[:−1], wfdn[::−1]))
45 psix [M:] *= wfup[−1]/wfdn[−1] # match
print ( ’nodes, n,l,E=’, nup+ndn, n, L, E)
47 n += 1
EL.append(Ea)
49
plt.figure () # plot energy levels
51 for L in range(Lmax):
for i in range(len(EL[L])):
53 plt.plot ([L−.3, L+.3], [EL[L][i]]*2, ’k-’)
plt.xlabel(’l’), plt.ylabel(’E’)
55 plt.ylim(−.51, 0), plt.xticks (range(Lmax))
plt.show()
 The functions Veff() and f() calculate the effective
potential and the Schrödinger equation (9.8), respectively. The
routine numerov() is a standalone Numerov integrator. It
receives f(x) to be integrated as input, together with initial
values [u0, u1], number of steps n to advance, starting position
x, and step size h. On return, the n new values plus the initial
pair are contained in u = [u0, u1, ···, un+1]. The same module is
included in the ODE library ode.py for convenience.

The module shoot() works the same way as the one in

Program 9.2. It calls numerov() twice to integrate the
Schrödinger equation inward from both ends, going one step
past the matching point M in each direction (recall the number
of steps counts from u1 upward or uN−1 downward). This is so
that we can use the three-point formula (9.60) to compute the
first derivative at M.

The main code sets parameters including the maximum

angular momentum lmax. For each l, the main loop scans the
energy range for eigenenergies. When one is found, the
principal number n is increased. If no valid eigenenergies are
skipped, n − l − 1 is equal to the number of nodes. The energy
levels are plotted at the end, grouped by l. The plotting
statements for the wave function are similar to those in
Program 9.2 and thus omitted.

For large n, l, the code can be modified to use the more

efficient logarithmic grid. See Exercise S:E9.2.
Program listing 9.6: FEM library ( fem.py)

import numpy as np
2
def abg(p1, p2, p3): # return alpha, beta, gamma, area of
element
4 [x1,y1], [x2,y2], [x3,y3] = p1, p2, p3
alfa = [x2*y3 − x3*y2, x3*y1 − x1*y3, x1*y2 − x2*y1]
6 beta, gama = [y2−y3, y3−y1, y1−y2], [x3−x2, x1−x3,
x2−x1]
area = 0.5*(alfa [0] + alfa [1] + alfa [2]) # area of
triangle
8 return alfa, beta, gama, area

10 def overlap(i, j, p1, p2, p3): # return ∫φiφjdxdy, Eq.

(9.49)
a, b, c, area = abg(p1, p2, p3) # over eltriangle and pts
p1−p3
12 X, Y, XY, X2, Y2 = 0., 0., 0., 0., 0.
for [x, y] in [p1, p2, p3]:
14 X, Y, XY, X2, Y2 = X+x, Y+y, XY+x*y, X2+x*x,
Y2+y*y
return ((a[i]*b[j]+b[i]*a[j])*X + (a[i]*c[j]+c[i]*a[j])*Y +
16 (b[i]*b[j]*(X2+X*X) +(b[i]*c[j]+c[i]*b[j])*
(XY+X*Y) +
c[i]*c[j]*(Y2+Y*Y))/4 + 3*a[i]*a[j])/(12*area)
18
def A_mat(node, elm): # fills matrix ∫ ∇ϕi · ∇ϕjdxdy, Eq.
(7.19)
20 A = np.zeros((len(node),len(node)))
for e in elm:
22 [x1,y1], [x2,y2], [x3,y3] = node[e [0]], node[e [1]],
node[e [2]]
a = 2*(x1*(y2−y3) + x2*(y3−y1) + x3*(y1−y2)) #
4Ae
24 if (a<=0.0): print ( ’Warning: zero or negative elm
 area’)
beta, gama = [y2−y3, y3−y1, y1−y2], [x3−x2, x1−x3,
x2−x1]
26 for i in range(3):
for j in range(i ,3): # Eq.
(7.27)
28 A[e[i], e[j]] += (beta[i]*beta[j] +
gama[i]*gama[j])/a
if (i != j): A[e[j], e[i]] = A[e[i], e[j]]
30 return A # A=twice KE,

32 def B_mat(node, elm): # overlap matrix, ∫ ϕiϕjdxdy, Eq.

(9.48)
B = np.zeros((len(node),len(node)))
34 for e in elm:
p1, p2, p3 = node[e [0]], node[e [1]], node[e [2]]
36 for i in range(3):
for j in range(i ,3):
38 B[e[i], e[j]] += overlap(i, j, p1, p2, p3)
if (i != j): B[e[j], e[i]] = B[e[i], e[j]]
40 return B

This FEM library includes the necessary functions for FEM

solutions of 2D quantum systems. Two functions are meant to
be internal: abg() returning α, β, γ and area of a basis function
(7.11) at the three nodes of an element; and overlap()
calculating the overlap integral (9.49) between two nodes.

The two external functions, A_mat() and B_mat(), compute

the A and B matrices, respectively, as defined in Eqs. (9.48),
(7.27), and (9.49). They iterate through the elements, calling
the internal functions to compute the contributions from each
element, and assigning them to the appropriate matrix entry
(see Figure 7.10 and associated discussion). After processing all
the elements in this element-oriented approach, all
contributions will have been accounted for, and the matrices A
and B will be completely built.

Program listing 9.7: Quantum dot ( qmdot.py)

import numpy as np, pickle, fileio, fem

2 import matplotlib.pyplot as plt
from scipy.sparse.linalg import eigsh
4 from mpl_toolkits.mplot3d import Axes3D

6 nt, n = 12, 0 # nt = tot num of states, n=state to plot

meshfile, eigenfile = ’meshdata.txt’, ’eigendata.dat’ #
data files
8 node, elm, bp, ip = fileio .readmesh(meshfile)
print (’nodes/elements’, len(bp), len(ip), len(elm))
10
try:
12 file = open(eigenfile, ’r’) # if prev eigen data
exists,
E, u = pickle. load( file ) # read from file
14 except IOError:
Tm = 0.5*fem.A_mat(node, elm) # no eigendata,
recalculate
16 B = fem.B_mat(node, elm)
Tm = np.delete(Tm, bp, axis=0) # delete boundary rows
18 Tm = np.delete(Tm, bp, axis=1) # delete boundary cols
B = np.delete(B, bp, axis=0)
20 B = np.delete(B, bp, axis=1)
E, u = eigsh(Tm, nt, B, which= ’SA’) # solve
22 file = open(eigenfile, ’w’) # file overwritten
pickle.dump((E, u), file) #
24 file.close ()
26 print (E) # print E, prep for wf
node, wf = np.asarray(node), u[:,n]
28 for i in bp: wf = np.insert(wf, i, 0.) # add boundary values

30 plt.figure () # draw mesh

plt.subplot(111, aspect= ’equal’)
32 plt.triplot (node [:,0], node [:,1], elm, ’o-’, linewidth=1)
plt.xlabel(’x’, size=20), plt.ylabel(’y’, size=20)
34
fig = plt.figure () # plot wave function
36 ax = fig.add_subplot(111, projection= ’3d’)
ax. plot_trisurf (node [:,0], node [:,1], wf, cmap=plt.cm.jet,
linewidth=.2)
38 plt.axis( ’off’)

40 plt.show()

This is a universal FEM program for quantum dot

calculations. It requires the mesh data file (see Figure 9.24 and
its associated mesh file format).

After reading in the mesh data, it tries to check the

existence of the eigenvalue data file. If it exists, presumably
from a previous calculation, it simply loads the data using the
pickle file handler (line 13). This handler is very handy for
outputting data to a file. It works with the dump function that
can pickle, or package, any data and write it to a file. For our
use, it is most convenient to form a tuple of eigenenergies and
eigenfunctions (line 23) and dump it in one operation.

If the eigenvalue data file does not exist, indicated by the

exception IOError, the program starts a new calculation.
Matrices and B are obtained from the FEM library, boundary
nodes are deleted (same as in Program 7.3), and the
generalized eigenvalue equation is solved using the sparse
eigensolver from SciPy, eigsh. Since we usually do not want all
states, we specify the number of states we do want, nt, which
must be less than the rank of the matrix, or the number of
internal nodes in our case. The results are dumped to the
eigendata file, so it can be independently analyzed without
repeating the calculation which could be long for large
problems. It is possible that pickle can run out of memory
storing the data set (E, u) for large matrices. If this is the case,
either break up the set into smaller chunks, or storing all
eigenenergies and only a subset of eigenfunctions.

At the end, the program prints the eigenenergies, and

inserts the boundary values back into the wave function for
graphing, assuming bp is already sorted. The wave function of a
chosen state is graphed with the newer function
plot_trisurf(x,y,z). It interprets the x, y, and z values as 1D
arrays defining the points of triangular patches. It fits our
purpose well, and we do not have to regenerate the grid.

If the mesh is such that the system matrices are banded, the
program can be modified to use a banded eigensolver,
eig_banded. The program could then handle very large matrices

efficiently. See Figure 8.6 and corresponding discussion for

band matrix representation.
Program listing 9.8: Triangle-mesh plotting of wave function
( tripwf.py)

1 import numpy as np, pickle, fileio

import matplotlib.pyplot as plt
3
meshfile, eigenfile = ’meshdata.txt’, ’eigendata.dat’ #
data files
5 node, elm, bp, ip = fileio .readmesh(meshfile)
file = open(eigenfile, ’r’) # read pickled eigendata
7 E, u = pickle. load(file)
file. close ()
9
node, st = np.asarray(node), range(12) # change st for other
states
11 fig = plt.figure ()
for n in range(len(st)):
13 wf = u[:, st [n]]
for i in bp: # bp should be sorted
15 wf = np.insert(wf, i, 0.) # add boundary values,
ax = fig.add_subplot(4, 3, n+1, aspect= ’equal’) # 4x3
plots
17 plt.tripcolor (node [:,0], node [:,1], wf, shading= ’gouraud’)
plt.title (repr(st [n]+1)), plt.axis( ’off’)
19
plt.show()

This program plots the wave function over a triangular

mesh using the Matplotlib function tripcolor(), which fits our
need perfectly with shading. The code requires the mesh data
and eigenvectors from Program 9.7 stored in respective files.
The boundary points stored in bp must be sorted if meshes
generated from other sources are used. As it is, the first 12
states are graphed, but we can change the state list st to graph
any states desired.

Program listing 9.9: Hexagon mesh generator ( meshhex.py)

import numpy as np, fileio

2
def mesh(a, N): # generate Hexagon mesh
4 node, elm = [], [] # nodes, elements
bp, ip, h = [], [], a/N # boundary and internal nodes,
size
6 M = (N*(3*N+1))//2 # last node before y=0 (center
row)
K = M + 2*N +1 # last node at end of center row
8 ndn = lambda i, j: j*N + (j*(j+1))//2 + i # node number at
i,j

10 for j in range(N+1): # go up till y=0

x, y = − 0.5*(a + j*h), (j*h − a)*np.sqrt(3.0)/2.0
12 for i in range(N+1+j):
node.append([x + i*h, y])
14 if (j != N):
elm.append([ndn(i,j), ndn(i+1,j+1),ndn(i,j+1)])
16 if (i != N+j):
elm.append([ndn(i,j), ndn(i+1,j),ndn(i+1,j+1)])
18 if (j == 0 or i == 0 or i == N+j): bp.append(ndn(i,j))
else: ip.append(ndn(i,j))
20
node, elm = np.array(node), np.array(elm) # get y>0 by
reflection
22 flip = np.column_stack((node[:M][:,0], −node[:M][:,1])) #
c−stack
node = np.concatenate((node, flip))
24
flip = elm [:,[1, 0, 2]] # swap nodes so they are ccw
26 flip [flip <M] += K # add offset, excluding center
row
elm = np.concatenate((elm, flip))
28
bp = np.concatenate((bp, K + np.array(bp[:−2])))
30 ip = np.concatenate((ip, K + np.array(ip[:−(N+N−1)])))
fileio.writemesh( ’meshHexagon.txt’, node, elm.tolist(),
bp, ip)
32 print ( ’Hex mesh: %d nodes (%d/%d bndry/intrnl), %d
elements’
%(len(node), len(bp), len(ip), len(elm)))
34
a, N = 1.0, 10 # side length, num intervals
36 mesh(a, N)

We can generate a mesh similar to Figure 9.18 with this

program. It starts from the bottom row (yj = 0), going from left
to right, and up to the center row (y = 0). The nodes above the
center row are obtained by reflection.

Below y = 0, the nodes are enumerated sequentially.

Assuming each side divided into N intervals, the node number
at row yj and column xi (counting from left) is given by
. This is calculated by ndn defined as a
lambda function. Unless at the end of a row, two elements are
formed at each node (i, j) in CCW order, the elements above
and to the right (note that the row above is effectively left-
shifted by one). This is similar to Program 7.3. The boundary
and internal nodes are also recorded. The process stops after
the center row. The number of nodes below the center row is
.
 For the other half above y > 0, we use reflection to copy the
nodes below, reversing the sign of y coordinates. This is
efficiently done using NumPy array slicing and column
stacking (line 22), which puts the same x-column and the
negative y-column side-by-side to make a 2D array. The flipped
nodes are added (concatenated) to the nodes for y ≤ 0. The
elements are also copied over, but nodes 0 and 1 are swapped
so they are in CCW order (line 25). An offset is added to the
copied nodes so their numbers start after the right-most node
of the center row, K = M + 2N + 1. We used array mask, i.e.,
truth array (line 26, see Section 1.D), to exclude the nodes on
the center row. When finished, the total number of elements is
6N2.

Finally, the mesh data is written to a file containing, in

order, the nodes, elements, boundary and internal nodes. The
elm array is converted to a list via tolist method for easy

formating by the file writer. As an example, the format of the

file for the mesh shown in Figure 9.24 is illustrated below,
which can be read by readmesh().
 Figure 9.24: A sample mesh for a rectangular domain.

# this is a comment; blank lines start a new section

# each line can have an unlimited number of items, separated by
commas

# nodes
[0.0, 0.0], [1.0, 0.0], [2.0, 0.0],
[0.0, 0.5], [1.0, 0.5], [2.0, 0.5],
[0.0, 1.0], [1.0, 1.0], [2.0, 1.0],

# elements
[0, 4, 3], [0, 1, 4], [1, 5, 4], [1, 2, 5],
[3, 7, 6], [3, 4, 7], [4, 8, 7], [4, 5, 8],

# boundary nodes
0, 1, 2, 3, 5, 6, 7, 8,

# internal nodes
4,

1
The study concluded: “Students … are weaned on continuous, well-behaved functions
and variables. It is therefore often a shock to them when they study quantum mechanics and
find, suddenly, discrete variables. The purpose of the computer-generated film described in
this paper is to illustrate that these discrete variables are not so peculiar after all – that, in
fact, they arise in a natural way from requiring the solution of the Schrödinger equation to be
continuous, differentiable, and finite”.

2
A mathematically equivalent form to Eq. (9.5) is , but this is
not stable for numerical work, because the wave function or its derivative can become close
to zero, and f(E) could fluctuate wildly.

3
Owing to the boundary conditions, these positive energy states are bound states in the
(numerically finite) box, mimicking the true continuum states of the actual, infinite system.
 4
This was discussed in Chapter 7 and illustrated in Figure 7.10 for FEM in 2D. The
reason is still true in 1D.

5
The Dirac δ molecule is a good model for understanding the qualitative features of
diatomic molecules such as , where the terms “gerade” and “ungerade” are used to
describe the symmetric ground and the antisymmetric excited states, respectively.

6
The wave functions un are available from SymPy: sympy.physics.qho_1d.psi_n.

7
Technically, a particle never becomes free in the Coulomb potential, even though it can
escape to infinity if its energy is positive. In this case, the continuum wave function is
distorted compared to the plane wave representing a truly free particle. See Section 12.B and
Exercise S:E12.7.

Because 〈r〉 scales like n , numerical integration must extend to large distances for
8 2

moderate to large n. To maintain efficiency, numerical integration is often done on a

logarithmic grid. See Exercise S:E9.2.

9
Quantum dots are literally shining a new light on quantum mechanics. Since the energy
levels, hence light emission, can be manually controlled via shape and size, they are
beginning to usher in new technologies such as the quantum dot display.

10
For other nontrivial potentials, numerical integration is more conveniently carried out
by mapping the triangles to a simpler shape such as isosceles right triangles through
coordinate transformations.

11
Here we ascribe the term “diagonals” to lines connecting the opposite vertices, and
bisectors to lines connecting the centers of opposite edges.
Chapter 10
Simple random problems
Many systems behave unpredictably and randomly, or at least
seem to. Some are random due to a lack of information such as
a coin toss or the golden autumn leaves fluttering and falling in
the wind. Other systems, though perfectly deterministic and
well defined, are still random because of their intrinsic,
probabilistic nature, such as the radioactive particle decay and
measurement of quantum systems.

We consider several simple random systems in this chapter

so as to be familiar with random sampling and distributions
needed in the next two chapters. First we introduce random
numbers and model particle decay by random sampling. We
also discuss random walks in 1D and 2D, as well as a stochastic
model for Brownian motion, a simple but important
phenomenon. Finally we introduce Monte Carlo integration to
evaluate multidimensional integrals including electrostatic
potential energies.
0.1 Random numbers and radioactive
decay
The first step to represent a random problem starts with the
generation of random numbers. At initial glance, this appears
paradoxical: on the one hand, the digital computer is as
deterministic a machine as it gets, yet on the other hand we
want it to produce random numbers. This is a dilemma. We
cannot generate truly random numbers on the computer. We
can, however, hope to generate a sequence of pseudo-random
numbers that mimics the properties of a series of true random
numbers.

The generation of pseudo-random numbers usually involves

bits overflow in integer multiplication at the low level. For our
purpose, Python has a random number library that provides
several random distributions. For example, the random()
function returns a uniform distribution in the interval x ∈ [0,
1),

We can repeat a random sequence by initializing it with the

seed function seed(n), where n is an identifier, typically an
integer.

Truly random numbers have no correlation between them.

Suppose we have a sequence of random numbers, {x}. Let us
define a joint probability P(x, x′) as the probability of finding x′
after x. No correlation means that the chance of finding x′ must
not depend on x before it. In other words,

As a result, a correlation test between two random numbers

is

Similarly, the moment tests are defined as

We can perform the tests numerically by generating a

sequence of N random numbers {xi} and calculate the following
averages

Table 10.1: Moments and correlation of pseudo-random numbers

generated from random().
 Table 10.1 shows the results of moments and correlation of
various order k. A million random numbers (N = 106) were
drawn from random() in each case.1 The values match their
theoretical predictions to within the statistical uncertainty of O
.

Radioactive nuclear decay by random sampling

Radioactive nuclear decay is an intrinsically random process
because it occurs via quantum transitions which are
probabilistic in nature as seen in Section 8.4. Let N be the
number of nuclei alive at time t, and let −ΔN be the number of
nuclei that will decay between t and t + Δt. Then, −ΔN will be
proportional to the existing nuclei and the time step as

where λ is the decay constant related to the lifetime of the

unstable nuclei as τ = 1/λ (Exercise E2.7).

We can interpret the LHS of Eq. (10.6) as the probability of

decay in one step, given by p, a constant in terms of λ and Δt.
To simulate the decay process, we draw a uniform random
number x in [0, 1) for each undecayed nucleus and compare it
to p. If x < p, the particle decays; otherwise, the particle lives
on. The code segment for each step is (import random as rnd)

Figure 10.1: Nuclear decay by random sampling.

deltaN = 0
for i in range(N):
x = rnd.random()
if (x < p):
deltaN = deltaN + 1
N = N − deltaN

We show in Figure 10.1 representative results for p = 0.02.

The overall trend resembles an exponential decay law as
expected. There are clear fluctuations in the number of decayed
nuclei in each step (Figure 10.1, right). But their cumulative
effect yields a relatively smooth curve of the remaining
particles (Figure 10.1, left), which is a Poisson distribution. The
 solution thus obtained by random sampling is an example of
Monte Carlo simulations.

In our simulation, the step size is related to p and λ. In other

words, the probability p implicitly determines the time scale of
the decay process. If the rate of decay λ is large, Δt will be
small, vice versa.

0.2 Random walk

Like radioactive decay, a random walk is another simple model
defined in a straightforward way. A walker takes one unit step
in one unit time, but the direction of each step is randomly
distributed, uniformly or otherwise. Many practical
applications are related to some features of random walks,
including diffusive processes, Brownian motion (Section 10.3)
and transport phenomena, etc.

Random walk in 1D
In one-dimensional random walks, there are only two possible
directions, left or right. Let the unit step be l, so each step has a
displacement of ±l. If a walker starts from the origin, the
position x, and its square x2, after n steps are

where si is distributed according to

subject to p + q = 1.

Figure 10.2 shows the results of ten separate walkers, each

moving ten steps with p = q = and l = 1. The paths for
different walkers show considerable variation (left panel).
Because they move left or right with equal probabilities, the
average position should be zero statistically. The solid line
represents the average position of the walkers at each step
number n. It fluctuates around zero with no clear trend. The
average position squared (right panel) shows an upward trend,
though there is fluctuation.

Let N be the number of walkers, and 〈x(n)〉 and 〈x2(n)〉 be

the average values of x(n) and x2(n) of the walkers after n steps,
respectively. In the limit N 1, the position follows a binomial
distribution (Figure 10.3), and the distribution near the center
resembles a Gaussian for large steps n (Exercise E10.3). The
average values approach (see Exercise E10.4)
 Figure 10.2: Random walks for 10 walkers moving 10 steps with
equal probabilities p = q = . The solid lines are the averages in each
case.

As a measure of spread (dispersion), we can calculate the

uncertainty as 〈(Δx)2〉 = 〈x2(n)〉 − 〈x(n)〉2. Given the expressions
(10.9), we have

The relationship is linear in step number n. It is an important

result from random walks, and is typical of random motion like
Brownian motion. For continuous time, t ∝ n, and the linear
spread is , C being some constant. For p = q =
, 〈(Δx)2〉 = 〈x2(n)〉, so Figure 10.2 (right) shows the approximate
linear trend. We can interpret the linear spread as the mean
distance the walkers can travel in time t. To travel twice as far,
we would have to wait four times as long.

Random walk in 2D
We can generalize 1D random walks to 2D via the displacement
vector . We need to determine x and y in some
random fashion.

Unlike the 1D random walk, we have more freedom

choosing the directions. There are many possibilities,
including:

Random walk on a lattice. Choose with equal

probability in the cardinal directions: left, right, up
and down. Set the (x, y) pair accordingly as (−1, 0), (1,
0), (0, 1), (0, −1). One random number is needed. This
seems to be the most direct way.

Figure 10.3: The distribution of position for 2000 walkers

moving 20 steps each (p = ). The dotted line is the
(unnormalized) binomial distribution. The upper scale nr is
the number of right steps.
 Random directions. Choose a random angle θ in [0, 2π],
and set x = l cos θ and y = l sin θ. Only one random
number is needed.

Variable step size. Choose x and y randomly in [−l, l].

Two random numbers are needed.

Figure 10.4 shows four walkers starting from the origin and
moving with unit step size and uniform angular distribution
(isotropic). The individual walks vary greatly as seen before in
1D random walks. We have to characterize an ensemble of
walkers statistically. For instance, the average position and
position squared are defined accordingly as ensemble averages,

These averages will differ according to the chosen model. But,

the general linear relationship (10.10) still holds, 〈(Δr)2〉 ∝ n.
We shall see an actual example with continuous time next in
Brownian motion.
 Figure 10.4: Four random walks taking unit steps in random
directions.

0.3 Brownian motion

Brownian motion refers to random motion of macroscopic
particles such as fine dust grains in liquids observed under a
microscope similar to Figure 10.4. It behaves in many respects
like random walks, but is a real phenomenon. Brownian
motion has great importance in physics and biophysical
systems, and often is the foundation toward understanding
more complex phenomena in such systems. We model
Brownian motion below.

10.3.1 A STOCHASTIC MODEL

Consider a particle suspended in a liquid and moving with
velocity . Intuitively we would expect the particle to slow
down due to drag (Section 3.2), and eventually to stop moving.
But such particles observed in Brownian motion never stop
moving. Rather, the motion carries on but has random
fluctuations.

This means there has to be some random force, F(t), besides

drag. Since the speed is usually small, we can model the drag
force as a viscous linear force (3.9), . The equation of
motion including both forces is

where m is the mass, and b1 the linear drag coefficient. This is

known as the Langevin equation. It is just Newton’s second law
with random forces.

Both forces in Eq. (10.12) come from interactions with the

same liquid, but why do we separate them? The reason has to
do with different time scales. Roughly, we can think of the drag
as representing some average force due to collective effects on a
larger time scale, and the random force due to collisions with
individual atoms and molecules on a smaller time scale [76].

Clearly, must be rapidly fluctuating and very complex.

Its precise form cannot be known, given the large number of
atoms moving about. As a simplification, we model as a
series of stochastic “kicks”
where is the strength of the kick at ti. Each kick is
represented by a Dirac delta function δ(t − ti) like in the kicked
rotor (Figure S:5.1). The effect is to deliver an impulse to the
particle after each kick. In general, both the strength and the
time ti of the kicks are random.

Between kicks, there is only the drag force in Eq. (10.12),

and the solutions can be generalized from the 1D results (3.12)
and (3.13),

where b = b1/m, and and are the initial position and

velocity. Note that 1/b has the dimension of time, so we can
regard it as the larger time scale associated with drag stated
earlier.

Let be the velocity right before ti+1. The velocity

immediately after the kick at ti+1 is increased to

With the δ-kick model, Eq. (10.12) becomes a discrete linear

map between kicks.
10.3.2 MODELING BROWNIAN MOTION
The model outlined above assumes arbitrary distributions for
and ti. Now we just need to specify them to go forward. For
simplicity, we assume has a constant magnitude and random
directions (this is not too crucial, see Project P10.2). For the
distribution of ti, it should be random, but we also expect that
after a kick, the probability of not being kicked again should
decrease with time. We choose to model the interval between
kicks as an exponential distribution,

This is a nonuniform distribution discussed in the next

section (see also Section 10.B). In Eq. (10.16), τ is the average
interval between kicks, τ = P(t)t dt. It represents the smaller
time scale of the rapidly fluctuating . It should be small
relative to 1/b, i.e., τ 1/b.
 Figure 10.5: Snapshots of 2D Brownian motion.

Here is our strategy for simulating Brownian motion. For

each particle, sample the time interval ti to the next kick
according to Eq. (10.16), and randomly orient . Calculate the
position and velocity from the kick-free motion (10.14) until ti,
then increment the velocity right after the kick from Eq.
(10.15). Repeat the above steps.

Figure 10.5 shows snapshots from animation by Program

10.2 using this strategy. The time scales are 1/b = 10 and τ = 2
(arbitrary units). Note the different length scales between the
two rows. All particles are “dropped” at the origin with zero
initial velocity. In the beginning (t 10), the size of the drop
grows linearly with time, i.e., = Ct. After that, the drop
spreads at a slower square-root rate, , the same as
random walks (10.10).

Figure 10.6: The average squared-distance of travel in Brownian

motion.
 The different power scalings are more clearly seen in Figure
10.6 showing the square of the distance (10.11), averaged over
1000 independent particles (an ensemble). We see two
regimes: t t1 = 10 and t t2 = 30, where the curve is roughly a
straight line with different slopes (note the log-log scale). The
slope is about 2 as expected below t1, and almost 1 above t2.
Between t1 and t2, there is a transition region connecting the
regimes. This different behavior at small and large time scales
is predicted in a statistical model (see Eqs. (10.39) and (10.40),
Section 10.A).

Physically, the small stochastic forces cause little effect over

a short time period, and we expect the motion to be linear. We
can see this from Eq. (10.14) as well, as for bt 1.
Over longer periods of time, the stochastic forces are more
important since they provide the energy to the particle, so the
motion resembles random walks with the same scaling (10.10).

We have determined the power laws for the distance of

travel in Brownian motion. What about the constant C or the
average speed? We suspect that they should depend on the
strength and frequency of the kicks. Indeed they do, and are
related to temperature discussed in Chapter 11. Further
investigation of Brownian motion is left to Project P10.2.

0.4 Potential energy by Monte Carlo

integration
We have just described simulations of select simple problems
with random numbers. It turns out that problem solving by
random sampling is a powerful method and, for some complex
problems, is the best method available. It falls into a category
of the so-called Monte Carlo methods. Below we illustrate this
method with an electrostatic problem.

Let us consider the electrostatic potential energy, U,

between two charged spheres of radii r1 and r2, respectively,
separated by a distance d between their centers on the x axis (d
> r1 + r2, nonoverlapping). We can find U by

where k is the electrostatic constant in Eq. (7.1), ρ1 and ρ2 the

charge densities in spheres 1 and 2, respectively. Equation
(10.17) is a six-dimensional integral over the volumes of the
spheres.

For arbitrary charge densities, Eq. (10.17) must be

evaluated numerically (Project P10.3). We discussed grid-based
numerical integration in Section 8.A. This approach becomes
inefficient for higher multiple integrals like Eq. (10.17) because
the number of grid points increases rapidly as a power of the
dimensions. It can be expensive if this type of integrals must be
calculated repeatedly during the course of a simulation.

Monte Carlo integration by random sampling can be useful

in such cases. Its accuracy is independent of the dimensionality
in principle. We discuss briefly this method below.

10.4.1 WEIGHTED AVERAGE METHOD

Let us consider the weighted average of a function f(x) defined
as

where P(x) is the weighting function. We assume it to be a

normalized probability distribution, P(x)dx = 1.

We divide the space into small intervals as before (Figure

8.18) and convert Eq. (10.18) into a sum

If we randomly sample the points, P(xj)Δx tells us the

probability with which the points should fall in the interval [xj,
xj+1]. Let nj be the number of points in that interval, and we
have

Substituting Eq. (10.20) into (10.19), we obtain

 We have turned the group sum over j into an individual sum
over i in Eq. (10.21). Once we have chosen a weighting function
P(x), we can compute the weighted average 〈f〉, and the integral
(10.18). Let us choose a weighting function as

so that x is uniformly distributed in [a, b]. With this P(x), Eq.

(10.18) becomes

It follows that the integral is

The notation 〈f〉mc reminds us that the weighted average is to be

calculated subject to Monte Carlo sampling, a uniform
distribution in this case. We can generalize Eq. (10.25) to n-
dimensional multiple integrals of the type

where V is the hyper volume over which the integration is to be

done. Monte Carlo integration in this manner boils down to
sampling the points uniformly within the volume of
integration.

Table 10.2: The electrostatic potential energy between two uniformly

charged spheres of radii r1 = 1 and r2 = 2 separated by a distance d =
4, in units of kQ1Q2, by Monte Carlo integration. The exact result is
.

We apply Monte Carlo integration to the evaluation of the

electrostatic potential energy U in Eq. (10.17). If we assume
uniform charge densities, the result is known, U = kQ1Q2/d,
where Q1 and Q2 are the charges on the spheres. This is the
energy required to bring the two charges in from infinity. The
Monte Carlo results are listed in Table 10.2, obtained from
Program 10.3.

The main effort is to generate two points in a trial, one

inside each sphere. This is done by sample().

def sample(r): # return a random point inside a sphere

while True:
x, y, z = rnd.random(), rnd.random(), rnd.random()
x, y, z = r*(x+x−1), r*(y+y−1), r*(z+z−1) # map to [−r, r]
if (x*x + y*y + z*z <= r*r): break
return x, y, z
The function samples points inside a sphere of radius r
uniformly by the rejection method. A point at coordinates (x, y,
z) is randomly selected within a cube of sides 2r containing the
sphere. If the point is inside the sphere, it is selected and
returned. Otherwise, it is rejected and a new point is generated
until one is found inside. The main program simply evaluates
the integrand at these points.

The Monte Carlo results are in satisfactory agreement with

the exact value kQ1Q2 for d = 4, even for N in the low
hundreds. As N increases, the result stabilizes at the exact
value. We expect the relative error to scale as 1/ , so at the
largest N = 104 in Table 10.2, the error should be about 1%. We
can easily control the relative error in Monte Carlo integration.
Furthermore, we can improve the final result by cumulatively
averaging different runs with proper weighting because they
are independent.2 Doing this with the seven runs in Table 10.2,
we obtain 0.2501, the closest to the exact value yet. The overall
convergence of the Monte Carlo approach is good with
increasing N in this example. Its performance and simplicity
cannot be matched by nested, grid-based integrals.

10.4.2 IMPORTANCE SAMPLING

The integrand in the above example is monotonic and smooth.
Like any numerical integration scheme, Monte Carlo
integration suffers from ill-behaving integrands. The problem
is particularly acute if the integrand is highly oscillatory or
strongly peaked. Excessive oscillations cause large fluctuations
in random sampling. Sharp peaks mean that most
contributions to the integral come from small localized regions,
making uniform sampling inefficient, and often inadequate.

We can address the problem by the technique of importance

sampling. Suppose we want to integrate f(x) which is not
smooth. We attempt to separate it into two parts as

The new function g(x) is assumed to be smoother than the

original integrand f(x). Like in Eq. (10.18), P(x) is a probability
distribution function, and Eq. (10.27) is the weighted average
of g(x) under P(x). In other words, we have

We stress that 〈g〉mc in Eq. (10.28) is calculated with xi

distributed according to probability P(x). This is the basic idea
of importance sampling. For a given integrand f(x), we should
choose P(x) such that the modified integrand g(x) becomes as
smooth as possible. The distribution P(x) guides us to sample
the important regions. The integration volume does not appear
explicitly in Eq. (10.28). It is implicitly taken into account by
the sampling process. Of course, if we choose a uniform
probability distribution as in Eq. (10.22), then Eq. (10.28) is
reduced to (10.25).
 As an example, let us evaluate the following integral

The integrand is strongly peaked at the origin, where we expect

most contributions to come from. At the same time, it also
oscillates. But relatively speaking, the more pressing problem is
to deal with the sharp exponential factor, not the cosine
oscillation. Therefore, we choose an exponential probability
function as P(x) = exp(−x), which is already normalized.

The modified integrand is g(x) = f(x)/P(x) = cos(x). The

integral (10.29) becomes

The value 〈cos(x)〉mc is to be calculated with the exponential

probability distribution. The following code illustrates the
procedure.

f, N = 0., 1000
for i in range(N):
x = − np.log(rnd.random()) # exponential distribution
f += np.cos(x)

print ( ’MC=’, f/N)

One trial run gave a result 0.517, or about %3 error, consistent
with the number of sampling points N. Without importance
sampling, the error would be unacceptably large, even for a
reasonably large N, as to make the result meaningless (see
Exercise E10.7).

The key to the above code is the statement x = −

ln(random()), which transforms the logarithm of uniform
random numbers in [0, 1) into a series satisfying the
exponential distribution in (0, ∞). It ensures that the small-x
region, where the most contribution comes from, is sampled
more frequently. We discuss this and general nonuniform
sampling in Section 10.B.

Chapter summary

We discussed a few simple random problems, including

radioactive particle decay by random sampling, and random
walks in 1D and 2D. We constructed a stochastic model for
Brownian motion, and compared simulation results with a
statistical treatment. We introduced Monte Carlo integration
and importance sampling methods for obtaining nonuniform
distributions.

The basic usage of the intrinsic Python random library was

illustrated through the selected problems.
0.5 Exercises and Projects
EXERCISES
E10.1 (a) Test the uniformness of random number generation.
Generate N = 104 samples and plot the distribution as a
histogram of 100 bins (see Program S:9.1).

(b) Write a program to calculate the moments and

correlation listed in Table 10.1.

E10.2 (a) Suppose someone prefers white socks to red socks.

On each shopping trip, he opens packages – sealed
blackboxes each containing a pair of white or red socks,
until he finds a pair of white socks. Then he stops, and
must buy all the opened packages. After many such
trips, will he have an oversupply of white socks relative
to red ones? Prove it one way or the other by random
sampling.

(b) The SOS game (stick or switch, similar to the

Monty Hall game show Let’s Make a Deal) is a game of
chance and works as follows. The host hides a gold coin
in one of four boxes. You pick a box first, and the host
opens an empty box. You decide whether to stick with
the box you picked, or switch to one of the two
remaining boxes. Which way gives you a better chance
of winning?

Simulate the SOS game by random sampling. Tabulate

the results for N trials. Choose a large enough N so that
 you can draw definitive conclusions.

Assume on average half of the players switch and the

other half stick. How much should the host charge the
players per game in order to break even?

E10.3 (a) Generate random walk distributions analogous to

Figure 10.3 for various p values, e.g., 0.3, 0.6, 0.9, etc.
Use enough walkers so the fluctuations are small. Show
that as the maximum step number nmax increases, the
distribution becomes sharper and more Gaussian-like.

(b) Two random walkers start at the same position.

What is the probability that they will meet again after
walking 20 steps each? Assume equal probability going
left or right.

Simulate the process numerically. Use enough trials

such that the event occurs about 100 times. How does
the numerical result compare?

E10.4
Prove the relations (10.9). For 〈x2(n)〉, group the terms
as

Compute the averages of the two terms on the RHS.

The first term is independent of direction, and the
 double sum in the second term can be evaluated
independently.

E10.5 Calculate the spread in 2D random walks

for the following models (see Section 10.2): (a) random

walks on a lattice; and (b) random directions.

For part (b), introduce z = Σ zj, zj = xj + iyj = exp(iθj),

and calculate 〈r2〉 = 〈|z|2〉. Separate the equal and
unequal indices in the sums as in Exercise E10.4.

E10.6 (a) Consider the n-fold integral

With the parametric identity

show that
 (b) Calculate I(m, n) using Monte Carlo integration.
First choose a fixed m = 3 and n = 10, and vary the
number of sampling points from N = 10 to 106 by
factors of 10. Compare the Monte Carlo and analytic
results. Plot the results on a semilog N scale, and
discuss convergence.

Next, fix N, e.g., N = 105, and calculate the integral for

n = 10 and m = 0 to 5. What do you expect for n = 20?
Try it.

E10.7 Calculate the integral (10.29) by Monte Carlo

integration by uniform sampling x in [0, 8]. Compare
the accuracy with the exponential distribution by
importance sampling for the same N (Section 10.4).
E10.8 (a) Show that the transform for the sine angular
distribution is given by

and for the Lorentzian distribution by

 (b) Generate these distributions, plot them as
histograms (see Program S:9.1). Use enough samplings
so the fluctuations are small. Compare with the exact
results.

(c) A Gaussian distribution can be generated using a

pair of random variables as

This is known as the Box-Muller method. Because x1

and x2 are independent, the pair y1 and y2 are also
independent, and both values can be used. Produce a
histogram as above, and verify that it yields a Gaussian
distribution. Find the standard deviation.

PROJECTS
P10.1 Simulate 1D random walks. Construct a program that
calculates individual path of a single walker, as well as
the averages 〈x(n)〉 and 〈x2(n)〉 for N walkers as a
function of step number n.

Write a standalone subroutine walknsteps() as the

workhorse. It should be similar to the function of the
same name in Program 10.1. The function should
 assume arbitrary step size and probability p. At each
step, draw a random number. If it is less than p, the
walk is to the right, otherwise it is to the left. The main
program should call walknsteps N times for a given
maximum step number nmax. Compute the averages
〈x(n)〉 and 〈x2(n)〉 after N walks.

(a) Assume unit step size l = 1, and choose nmax ∼ 50

and N ∼ 100. Vary p values, e.g., 0.2, 0.5, 0.9, etc.
Predict and sketch 〈x(n)〉 and 〈x2(n) in each case. Plot
and discuss the actual results.

(b)* Modify walknsteps() to allow the step size l to

vary between some minimum and maximum limits, l ∈
[a, b]. You can draw a random l in this range using
uniform(a, b) which is equivalent to a + (b − a) × r

where r is a random number between [0, 1). Set

reasonable limits, e.g., [0, 2]. Plot the results, and
discuss differences and similarities with the above
results.

P10.2 Let us investigate Brownian motion in this project. First

explore Program 10.2 with animation on, change some
parameters such as F, b, etc., but keep τ < 1/b.

(a) Modify the program to calculate 〈r2〉. You can

disable animation. Add a function to the Brownian-
motion class that returns 〈r2〉 as a function of time. The
average 〈r2〉 at an instant can be obtained from
 r2 = np.sum(self.r* self. r)/ self .N

The number of particles N should be 1000 or more to

obtain good statistics. Append this to a list, and plot it
as shown in Figure 10.6.

(b) Obtain 〈v2〉 in a similar way, and plot it as a function

of time. Discuss the results. What is the temperature kT
from the equipartition theorem (11.41) (set m = 1)?

(c) Reduce τ by a factor of two, and repeat the

calculations for 〈r2〉 and 〈v2〉. Do the same with the kick
strength. Discuss the results. Is the temperature higher
or lower in each case? Why?

(d) Change the exponential distribution for ti to a

uniform distribution between 0 and 2τ, and repeat the
calculations. Also, instead of a constant F, assume it
follows a Gaussian distribution (see Eq. (S:10.4)).
Sample it using rnd.gauss() in Python (see also
Exercise E10.8), with a mean and standard deviation
equal to the default F value. Discuss how these results
differ from earlier ones.

(e)* Generalize the program to the 3D case. The

position and velocity should now be N × 3 arrays. The
 force should be distributed uniformly in all

directions. One way is, for a given magnitude F, to

sample the angle φ uniformly in [0, 2π], and then
sample θ as a sine distribution in Exercise E10.8.

Another method is to randomize by an Euler rotation

(see Eq. (S:12.125)). Compute 〈r2〉 and 〈v2〉 and compare

with 2D results. Optionally visualize 3D Brownian
motion using VPython.

P10.3 Solve the following problems by Monte Carlo

integration. In each case, obtain the results to three
digits of accuracy. Assume arbitrary units.

(a) Calculate the center of mass of a hemisphere of

radius r with uniform mass density. Compare with the
analytic result.

(b) Two spheres of radii r1 and r2 are separated by a

distance d between the centers. Each carries a unit
charge. The charge densities obey a linear relationship
of the form ρi = Ci(1 − r/ri), with i = 1 and 2, and Ci are
constants.

Assume r1 = 1 and r2 = 2. Determine the constants C1

and C2 analytically or numerically via Monte Carlo
integration. Find the electrostatic potential energy
between the spheres for d = 4. Discuss and compare
 your results with two uniformly charged spheres (Table
10.2).

(c) Consider the same setup as above, but the charge

densities have dipole angular distributions (Figure 7.24)
ρi = Ci sin θi, where θi is the polar angle relative to
central axis connecting the spheres. Predict the energy
of this configuration relative to part (b) above, as well
as to the uniformly charged spheres. Calculate the
actual result, and compare the three systems.

P10.4 The hit-or-miss method can be used to find an area or

an integral.

(a) Find the approximate value of π by this method.

Imagine a unit circle is enclosed in a square board of
side length 2, and we throw N darts randomly at the
board. Afterwards we inspect the board, and count the
number of hits inside the circle which is proportional to
the area. We can calculate π as

Simulate the dart game. For each throw, generate two

random numbers x and y between [−1, 1], and record
the number of hits within the unit circle. Vary N, and
calculate π. How many throws are needed to obtain π to
 two digits of accuracy consistently? How many throws
to match the accuracy of 22/7? How about 355/113?3

(b) Design a Monte Carlo program to calculate the

integral f(x)dx by the hit-or-miss method (see Section

10.B). Test your code with sin x dx, and compare the

accuracy with the weighted average method.

0.A Statistical theory of Brownian

motion
We briefly describe a statistical method for Brownian motion
where the unknown force in the Langevin equation (10.12)
is absorbed in the thermal properties of a system. We restrict
ourselves to 1D motion.

Let us multiply both sides of Eq. (10.12) by x and divide by

m, obtaining

where b = b1/m as before. For convenience, we introduce s =

xv, so that
 Substituting s and xdv/dt above into the RHS and LHS of
Eq. (10.31), respectively, we have

To eliminate F(t), let us take statistical (ensemble) averages on

both sides of Eq. (10.33) to obtain

with .

Because x and F are not correlated, the last term in Eq.

(10.34) is zero since 〈xF〉 = 〈x〉〈F〉 = 0. In thermal equilibrium,
the first term can be obtained from the so-called equipartition
theorem (11.41), which in 1D is

where k is the Boltzmann constant and T the temperature.

Using Eq. (10.35), we can simplify Eq. (10.34) to

It yields the following solution, assuming initially at t = 0,

Recalling from Eq. (10.32), we can obtain 〈x2〉 as

subject to the initial condition 〈x2〉 = 0 at t = 0. This is the

central result.

It seems that we have managed to eliminate the rapidly

fluctuating force F(t) in Eq. (10.38). But, its effect is hidden in
temperature T. The strength (impulse) and frequency of F(t)
clearly depend on the temperature.

Still, the solution lets us obtain the limiting behavior of 〈x2〉

at different time scales. For small t 1/b, exp
, and we have

For large t 1/b, we can drop the second term in the bracket of
Eq. (10.38) all together, so that

This linear scaling is the same as in random walks (10.10) and

other diffusive processes.
0.B Nonuniform distributions
The need for nonuniform probability distributions arises
frequently in practical situations as demonstrated by several
examples we have discussed so far. A number drawn from a
nonuniform distribution is still random, but after many such
drawings, the distribution becomes skewed. The general
approach is to generate a nonuniform distribution from a
uniform one.

Let us denote the nonuniform variable by y, and the

uniform variable by x. We wish to generate a certain
distribution py(y) via a transformation of x. We will discuss two
methods below.

INVERSE TRANSFORM METHOD

In certain cases, it is possible to obtain a nonuniform
distribution analytically via the inverse transform method. The
basic premise is that there exists a transformation, say y =
G(x), which transforms the uniform distribution px(x) to the
nonuniform distribution py(y).

Since probability is conserved, we have the following

relationship
The LHS of Eq. (10.41) gives the probability in the y-domain
from y to y + dy. Because it is generated from x to x + dx one-
to-one by G(x), it must be equal to the probability in the x-
domain – the RHS of Eq. (10.41).

Because px(x) = 1 from Eq. (10.1), we can integrate Eq.

(10.41) on both sides to obtain

In obtaining Eq. (10.42) we have dropped the || signs for

convenience.

To the extent that the integral in Eq. (10.42) is analytically

solvable for F(y), and subsequently, the inverse of F(y) exists,
then we have found the transform

where G = F−1 is the inverse function of F.

As an example, let us obtain the exponential (Poisson)

distribution

Though this is also a standard distribution, we choose it for the

simplicity. We find the integral from Eq. (10.42) as
from which we can obtain the inverse transform

This shows that the logarithm of a uniform distribution

produces an exponential distribution. The two functions are
just the inverse of each other in this case.

We have used this distribution in the exponential integral

(10.29) in Section 10.4 with λ = 1, which controls the mean.
Basically, the −ln x function compresses x ∼ 1 toward y ∼ 0 and
stretches x ∼ 0 toward y ∼ ∞, so the resulting distribution is an
exponential, like radioactive nuclear decay (Figure 10.1). More
examples are given in Exercise E11.8.

REJECTION METHOD
Often, it is not possible to find the inverse function analytically.
For example, one of the most useful distributions, the Gaussian
distribution, cannot be generated by the inverse transform
method (though it is possible using two variables, see Exercise
E10.8). Sometimes the distribution itself cannot be expressed
as a simple, compact function. Rather, it may be in the form of
discrete data like a lookup table. In such cases, we can use the
rejection method which always works.
 The idea is similar to the hit-or-miss method (Project
P10.4). Let H be greater or equal to the maximum value of
py(y) in the range y ∈ [a, b]. We put a box stretching from the
lower-left corner [a, 0] to the upper-right corner [b, H]. The
sampling procedure is as follows (as usual, x is uniform in [0,
1)):

Generate y randomly in [a, b] by y = a + (b − a)x

Generate p randomly in [0, H] by p = Hx

If p < py(y), accept and return y

Otherwise, reject y, repeat the first step

Each successful sampling requires at least two random

numbers x. After many samplings, the number of y values
accepted is proportional to py(y), the desired distribution. The
rejection method also requires the values of py(y), in either
functional or numerical form, and its upper limit H. The closer
H is to the actual maximum of py(y), the more efficient the
rejection method. The efficiency of the rejection method can be
improved if a better comparison function closer to the actual
py(y), rather than the box, is suitably constructed.

0.C Program listings and descriptions

Program listing 10.1: Random walk in 2D ( walk2d.py)
 1 import numpy as np, random as rnd
import matplotlib.pyplot as plt
3

def walknsteps(n): # walk n steps

5 x, y = np.zeros(n+1), np.zeros(n+1)
for i in range(n):
7 phi = rnd.random()*2*np.pi # random in [0, 2π]
x[i+1] = x[i] + np.cos(phi)
9 y[i+1] = y[i] + np.sin(phi)
return x, y
11

nstep, N = 20, 4 # steps, num of walkers

13 col = [ ’k’, ’r’, ’g’, ’b’, ’c’, ’m’] # color codes
plt.subplot(111, aspect= ’equal’)
15 for i in range(N):
x, y = walknsteps(nstep)
17 plt.quiver(x[:−1], y[:−1], x[1:]−x[:−1], y[1:]−y[:−1], scale = 1,
scale_units= ’xy’, angles= ’xy’, color=col[i%len(col)])
19 plt.plot ([0],[0], ’y*’, markersize=16)
plt. xlabel(’x’), plt.ylabel(’y’)
21 plt.show()

The function walknsteps() simulates one walker taking n

steps of unit length, in random directions between [0, 2π],
starting from the origin. The main program plots the paths of N
walkers with quiver() which connects the path by arrows
having the same components as the steps.

Program listing 10.2: Brownian motion ( brownian.py)

 1 import numpy as np, random as rnd
import matplotlib.pyplot as plt
3 import matplotlib.animation as am

5 class BrownianMotion: # Brownian motion class

def _init_ (self, N=400, F=0.005, b=0.1, tau=2., h = 0.5):
7 self .N, self .F, self .b, self .tau, self .h = N, F, b, tau, h
self .r, self .v = np.zeros((N, 2)), np.zeros((N, 2))
9 self .t = −np.log(np.random.rand(N))*tau # initial kick times

11 def move(self, r, v, dt): # move between kicks

edt = np.exp(−self.b*dt)
13 return r + v*(1−edt)/self.b, v*edt

15 def iterate (self): # advance one step

r, v, t, h, F = self.r, self .v, self .t, self .h, self .F # alias
17 for i in range(self .N):
if t[i] > h: # no kick within current step
19 dt = h # dt= time to end of step
else: # suffers kicks before end of step
21 tot, dt = 0., t[i] # tot=time to last kick
while t[i] <= h:
23 r[i], v[i] = self .move(r[i], v[i], dt)
phi = rnd.random()*2*np.pi # apply kick
25 v[i] += [F*np.cos(phi), F*np.sin(phi)]
tot += dt
27 dt = −np.log(rnd.random())*self.tau # sample dt
t[i] += dt # next kick
29 dt = h − tot # dt= to end of current step
r[i], v[i] = self.move(r[i], v[i], dt) # no kick, just move
31 t[i] −= h

33 def updatefig(*args): # update figure data

bm.iterate()
35 plot.set_data(bm.r [:,0], bm.r [:,1]) # update data
return [plot] # return plot object
 37

bm = BrownianMotion() # create Brownian model

39 fig = plt. figure ()
plt.subplot(111, aspect= ’equal’)
41 plot = plt.plot(bm.r [:,0], bm.r [:,1], ’o’)[0] # create plot object
ani = am.FuncAnimation(fig, updatefig, interval=10, blit=True) # animate
43 plt .xlim(−1., 1.), plt .ylim(−1., 1.), plt .show()

Program 10.2 simulates Brownian motion according to the

stochastic model in Section 10.3. We use object-oriented
programming in this case since the model is self-contained and
there is little interaction with the main program. We define a
Brownian-motion class consisting of an ensemble of N
independent particles. The _ _init_ _() function is called
when an object is created. In this case, it initializes the default
parameters, sets the initial position and velocity vectors to zero,
and samples the times of the initial kicks using the NumPy
function np.random.rand(N) (line 9), which generates an array
of N random numbers at once. The function move() calculates
the position and velocity under the drag force only between
kicks from Eq. (10.14).

The next module, iterate(), advances the solution forward

by one step h. For each particle in the ensemble, we check
whether it receives any kick within this step. That information
is stored in t[i] (ti), which indicates the time of its next kick
from the start of the current step. If ti > h, it means there is no
kick within the current step. So Δt is set to h, the time to the
end of the current step, and the particle will be moved
according to Eq. (10.14) after the if-else statement.

On the other hand, if ti ≤ h, there will be at least one kick.

So we move the particle to obtain and just before ti with
move(). Then we sample a randomly oriented force and
increase the velocity by the impulse received (10.15). The
variable tot keeps track of the time elapsed from the beginning
of the step to the last kick. Now we sample from an exponential
distribution (10.46) the interval to the next kick (line 27), and
add it to ti. If this ti is still within the current step, we repeat the
above treatment, until the next sampled kick is after the
current step. We then set Δt to the end of the current step,
which is the difference between the step size h and the time of
the last kick tot.

After the if-else block, the particle is moved to the end of

the current step (start of the next step), and ti is adjusted
accordingly.

The updatefig() function calls the iterate() method of a

given object to update the x and y positions of the particles for
animation. The plot object is returned as a list (line 36)
required by the animation function.

The main program makes an instance of Brownian-motion

objects bm with default parameters. It also creates a plot object
for Matplotlib animation using updatefig() at regular intervals
(10 ms) to update the figure as in Program 8.1. The blit option
makes drawing faster.

Because updatefig() neither receives nor returns any data

variable, the object-oriented approach is more convenient here
because all data and computation (methods) are contained
within the Brownian-motion object.

Program listing 10.3: Electrostatic energy ( mcint.py)

1 import numpy as np, random as rnd

3 def sample(r): # return a random point inside a sphere

while True:
5 x, y, z = rnd.random(), rnd.random(), rnd.random()
x, y, z = r*(x+x−1), r*(y+y−1), r*(z+z−1) # map to [−r, r]
7 if (x*x + y*y + z*z <= r*r): break
return x, y, z
9

r1, r2, d = 1.0, 2.0, 4.0 # radii and separation, d>r1+r2

11

f, V, N = 0., (4*np.pi/3.)**2*(r1*r2)**3, 100

13 for i in range(N):
x1, y1, z1 = sample(r1)
15 x2, y2, z2 = sample(r2)
f += 1./np.sqrt((x1−x2−d)**2+(y1−y2)*(y1−y2)+(z1−z2)*(z1−z2))
17

print ( ’MC=’, V*f/N, ’exact=’, V/d)

 This program calculates the electrostatic potential energy
between two spheres of uniform charge densities (ρ1 = ρ2 = 1,
and electrostatic constant k = 1 in Eq. (10.17)). The function
sample() returns a uniformly distributed point inside a sphere
by the rejection method as explained in the text. The main
program draws N points inside the spheres and computes the
weighted average according to Eq. (10.17). The final result is
multiplied by the combined volumes of the spheres.

1
NumPy also has a random library that is especially useful for generating arrays of
random distributions. For example, see the use of np.random.rand(N) in Program 10.2.

2
This is true to the extent the pseudo-random numbers are uncorrelated, of course.

3
These were the ratios used to approximate π by the fifth century Chinese mathematician
Zu Chongzhi.
Chapter 11
Thermal systems
Up to the last chapter, we had built simulations from first
principles, i.e., calculations were done from Newton's laws or
the Schrödinger equation. Thermal systems, however, require a
different, statistical approach since they are made up of large
numbers of atoms and molecules. Macroscopic thermal
properties are determined by the microscopic interactions
between these particles. Given that the number of particles is at
least on the order of the Avogadro number ∼ 1023, we cannot
hope to simulate thermal systems using first-principle
calculations by tracking all the individual particles.1 Nor would
we want to. Even if their individual properties such as the
energies or velocities were available, we would not be able to
make sense out of that many particles without statistics. It
turns out that we can model much smaller thermal systems and
still simulate the statistical properties of large systems,
provided we sample them correctly and be mindful of the
limitations.
 We begin with thermodynamics of equilibrium, exploring
the role of energy sharing, entropy, and temperature as the
driving forces toward equilibrium: the Boltzmann distribution.
We then introduce the Metropolis algorithm and apply it to
study 1D and 2D Ising models, carefully comparing numerical
and analytic solutions where possible, including phase
transitions. After fully assessing the Metropolis method, we
extend it to the study of non-thermal systems such as the
hanging tablecloth via simulated annealing. Finally, we
investigate the thermalization of N-body atomic systems and
kinetic gas theory by molecular dynamics, connecting first-
principle calculations to thermal properties.

1.1 Thermodynamics of equilibrium

The central idea in statistical thermodynamics is the
Boltzmann distribution. It describes the energy distribution of
a thermal system, and is the most powerful tool we have in
statistical mechanics. In this section we explore the equilibrium
process and show, through numerical (Monte Carlo)
experiments, that Boltzmann statistics arises naturally in
thermal equilibrium as a matter of overwhelming probability.

11.1.1 TOWARD THERMAL EQUILIBRIUM

To understand probabilities of states, we begin with counting of
states at the microscopic level. Imagine a thermal system made
up of microscopic particles that can store energy internally in
translational motion, rotation, or vibration. The specific
mechanism is unimportant. For example, consider the Einstein
solid model, one of the few analytically solvable problems. The
system consists of N independent quantum simple harmonic
oscillators (SHO). Each SHO has an infinite number of
equidistant states (see Eq. (9.32)), and can store an unlimited
quanta of energy. Suppose the system has a total energy of qε
with ε as the basic unit of energy. The energy can be shared in
many ways among the N oscillators. A microstate refers to a
particular distribution of energy.

At the macroscopic level, we are interested in the net,

externally distinguishable effects, called macrostates. Consider
two microstates in the Einstein solid example: one in which a
single oscillator has all q units of energy and the rest N − 1
oscillators have none, and another in which the first q
oscillators have one unit of energy each and none for the rest.
These two different microstates correspond to the same
macrostate, a state of total energy q. In fact, all different
microstates of an Einstein solid with a given energy is just one
macrostate, irrespective of how it is distributed internally.

As there can be multiple microstates for a given macrostate,

the correspondence between microstates and macrostates is
not unique. The number of microstates in a given macrostate is
called the multiplicity, denoted by Ω. The multiplicity, Ω(N, q),
of the Einstein solid for a given N and q, can be found
analytically [41, 82] (see Exercise E11.1).
 To study equilibrium, let us consider an isolated system
composed of two interacting solids, A and B, with NA and NB
oscillators respectively, sharing a total of q units of energy, qA +
qB = q. We denote a macrostate n as a particular pair of
numbers, n ≡ (qA, qB), where qB = q − qA would be fixed for a
given qA. The multiplicity of a macrostate n is equal to the
product of multiplicities of the two solids,

It tells us the number of possible ways (microstates) of

arranging energies qA and qB within solids A and B,
respectively.

To associate the multiplicity with a probability, we invoke

the statistical postulate which states that all accessible
microstates are equally probable in an isolated system. This is a
fundamental assumption in thermal physics.2 With this
postulate, we can assert that the probability of finding the
system in macrostate n is

where ΩT = ∑n Ω(n) is the total multiplicity.

We could calculate the probability distribution from Eq.

(11.2) (“theory”, Project P11.2), or we could simulate how this
happens naturally (“experiment”). We choose the latter. To do
so, we consider an asymmetric pair, NA NB, a small solid A
interacting with a large solid B. This simulates the common
situation in which a system of interest is in thermal equilibrium
with a large environment (the reservoir).

It is more convenient to define an Einstein-solid object via

object-oriented programming rather than through a function as
usual. This object will be used in more than one program.
Furthermore, it can be expanded to two interacting solids. This
object is defined as follows.

Program listing 11.1: Einstein solid class ( einsteinsolid.py)

1 import random as rnd

3 class EinsteinSolid: # Einstein solid object

def _init_ (self, N=400, q=10):
5 self .N = N
self. cell = [q]*N # q units energy per cell
7

def _add_ (self, other): # combine two solids

9 self .N += other.N
self. cell += other.cell
11 return self

13 def exchange(self, L=20): # iterate L times

for i in range(L):
15 take = rnd.randint(0, self .N−1) # random pair
give = rnd.randint(0, self .N−1)
17 while self. cell [take] == 0: # find a nonzero−energy cell
take = rnd.randint(0, self .N−1)
19 self. cell [take] −= 1 # exchange energy
self. cell [give] += 1
 The function _init_() is called when an object is
instantiated to initialize N oscillators and to store their energies
in the attribute cell as a list.3 The next function _add_()
defines the addition operator so two solids can be combined
such that if C = A+B, then NC = NA + NB and the cells (lists) are
concatenated (Project P11.2). This is also called operator
overloading. The double underscore marks a method as private
to a class.

Because all allowed microstates are equally probable by the

statistical postulate, we will let the oscillators exchange energy
freely and randomly. This is carried out in exchange() which
makes a specified number of iterations through the oscillator
list. In each iteration, it randomly picks a pair of oscillators for
energy exchange using randint(n1,n2), which returns a
random integer between n1 and n2, inclusive. After finding an
oscillator with a nonzero energy, it takes one unit of energy
from that oscillator and gives it to the other. As discussed
below, this is equivalent to the interaction between a single
oscillator and the rest.

Given an initial state, we allow many interactions to occur

to reach equilibrium. We can count the number of cells with a
given amount of energy to obtain the energy distribution. What
is the distribution after equilibrium is established?
 Figure 11.1: The energy distribution of an Einstein solid (256
oscillators) at given iterations (time). The darker the cell, the higher
the energy.

We first present results from Program 11.4 for a small

Einstein-solid object (256 oscillators) in Figure 11.1, which
shows the energy distribution in a 16 × 16 intensity image as a
function of iterations elapsed between samplings. Darker cells
correspond to higher energies. Not to play favoritism, we give
every oscillator the same amount of energy in the initial
microstate, indicated by the uniform intensity at the start (first
frame). The visual effect clearly shows that, with increasing
iterations, the number of white cells increases quickly, followed
by ever smaller numbers of lighter to darker gray levels. We get
the picture that the most probable states are at low energies.

Figure 11.2: The energy distribution of an Einstein solid (1024

oscillators) at given iterations (time). The smooth curve is an
−αn
exponential e .

To be more quantitative, we show the results from Program

11.5 in Figure 11.2 for a larger solid with N = 1024. It displays
as histograms the relative fraction of oscillators having n units
of energy between samplings. Because every oscillator is
initialized to one unit of energy, we see a single peak at n = 1 in
the first frame. As the number of iterations increases, the
distribution widens, and the number of oscillators having zero
energy grows rapidly, as observed in the intensity plot Figure
11.1. After 1500 cumulative iterations from the start (the fifth
frame), the zero-energy state is the most probable (to check if
this might be due to the low initial energy, increase it and run
Program 11.5). By the last three frames, the distribution has
only small changes. It signals that the system has reached
equilibrium, with what looks like an exponential distribution

Drawing smooth curves through the distribution, we find α ∼ ln

2 gives the best visual fit. We will see why later (Section 11.1.4).

Figure 11.3: The energy distribution in equilibrium (last frame of

−αn
Figure 11.2) on a semilog scale. The solid line is N(0)e , α = ln 2.

To confirm this, we plot the distribution in Figure 11.3

where N(n) is the number of oscillators having n units of
energy, this time on a semilog y scale. The data is from the last
frame of Figure 11.2. We can see a clear trend of the data points
following closely the exponential function (11.3) which is a
straight line on the semilog scale. In addition, we can also see
the exponential tail at larger n which is too small to see on a
linear scale (Figure 11.2). The error bars are equal to the
statistical fluctuation, . They are produced from
Program 11.5 with

plt.errorbar(range(len(bin)), bin, np.sqrt(bin), fmt=None)

In the simulation, we only know the total number of

oscillators (N = NA + NB = 1024), but what are the implicit
values of NA and NB? From the way we modeled the system, we
can interpret the results as caused by the interaction between
two solids, NA = 1 and NB = 1023. When we iterated the system,
we picked two individual oscillators randomly to exchange
energy. Because all oscillators are equivalent, each action is in
effect an interaction between a single oscillator (A) and the rest
(B). When we plot the histogram, we are effectively plotting the
results of N iterations of a single oscillator.

This is the subtlety of the Monte Carlo experiment. We

could have focused on a particular oscillator (say the first one),
and exchanged energy between it and the rest of the system.
Over the long run, we would still have the same cumulative
distribution. However, it would be extremely inefficient and
would take much longer to obtain results as accurate as these in
Figure 11.3.

As we will see shortly (Section 11.1.4), the remarkable

“experimental” results, obtained with a system so tiny as to be
utterly insignificant compared to a normal thermal system, in
fact do agree with Boltzmann statistics in actual systems.
Although we used the Einstein solid model, the results are
rather general, since the exchange of energy is generic and does
not assume any particular mechanism. It illustrates that we can
efficiently simulate the behavior of much larger systems using a
small but finite model system with correct sampling techniques
and proper interpretation. Now we have seen what equilibrium
looks like, we try to understand why next.

11.1.2 ENTROPY
We have seen from above that the thermal system marches
from an ordered initial configuration toward somewhat
disordered equilibrium configuration. The energy distribution
follows a probabilistic course of action given by the exponential
factor. The probability is extremely low that the system would
spontaneously go back to the ordered initial state, even for our
very tiny system of a few thousand oscillators. This all seems
purely mathematical.

However, in thermal physics we deal with measurable

physical quantities such as energy, heat, etc., not just
probability theory. How do we characterize the trend of
thermal systems toward disorder? The answer is a physical
quantity called entropy.

Entropy measures the degree of disorder, or more precisely,

the number of ways a thermal system can be rearranged,
namely the multiplicity Ω. Boltzmann defined entropy S as

where k is the Boltzmann constant. Because the multiplicity Ω

is a very large number, the logarithm ln Ω will be a large
number. After multiplied by the small k, entropy S will be
merely an ordinary number. More importantly, entropy thus
defined is additive: the total entropy of a composite system is
equal to the sum of individual entropies of its subsystems,
because the total multiplicity is equal to the product of
multiplicities (11.1).

We can calculate the change of entropy in the above

simulation of the Einstein solid. Because we know the
probability distributions of an oscillator from Figure 11.2, it is
more convenient to calculate entropy using an equivalent
expression as [9] (see Section 11.A, Eq. (11.55))

where Pn is the probability of finding the system in state n.

 The function below computes the entropy of one oscillator
in the system.

Program listing 11.2: Entropy of an Einstein solid ( entropy.py)

1 def entropy(cell): # entropy of Einstein solid

N, n, nt, s = len(cell), 0, 0, 0.
3 while nt < N: # until all cells are counted
cn = cell.count(n) # num. of cells with En
5 n, nt = n + 1, nt + cn # increase energy, cumul. cells
p = cn/float(N) # probability
7 if (cn != 0): s −= p*np.log(p) # entropy/k, Eq. (11.5)
return s

It accepts the configuration (microstate) in cell, counts the

number of oscillators with energy n (line 4), converts it to a
probability, and adds it to the entropy, until all oscillators are
accounted for.

We can put the above function in Program 11.5 (or make it a

method of the Einstein-solid object in Program 11.1) to obtain
the results in Figure 11.4, showing the entropy S of a single
oscillator in a system of N = 1024 oscillators, in units of k to
make its numerical value tidy. The total entropy of the system
is N times larger because it is additive. Initially the entropy is
zero because the initial state, where P(1) = 1 and P(n ≠ 1) = 0, is
perfectly ordered. As time increases, S grows rapidly, noting
the semilog x scale. Near equilibrium at the end, S is maximum
and stabilizes to a plateau. Numerical experiments lead us to
this observation:

Figure 11.4: The entropy of an oscillator as a function iteration

number.

Entropy tends to increase.

This is the second law of thermodynamics [82]. Because

entropy measures multiplicity, we can state the second law a
little differently: a system in equilibrium will be found in a
macrostate of greatest multiplicity (apart from fluctuations).

We can now understand why the most probable state is the

ground state (lowest energy state) in terms of entropy. For a
given amount of energy, the multiplicity of a macrostate as
measured by the entropy is increased by a much greater factor
if the energy is deposited in the large system B (reservoir) than
in the small system A (microsystem). The energy makes many
more microstates accessible in the reservoir than in the
microsystem. Using our example NA = 1, NB = 1023 and q = 1,
for instance, increasing the energy of system A by one unit has
no effect on the number of microstates, it is always 1. But, there
are 1023 possibilities, i.e., additional microstates, of
distributing one unit of energy in system B. Evidently, of all
possibilities, an exponential distribution (i.e., Boltzmann
distribution, Section 11.1.4) gives the maximum possible
entropy for a system in equilibrium.

11.1.3 TEMPERATURE
We intuitively know that when two systems are in equilibrium,
they have the same temperature. This leads us to expect that
there is a connection between temperature and entropy
reaching maximum in the above example.

Figure 11.5: The changes of entropy per oscillator vs. iterations (left)
and vs. energy exchange (right) between two equal-sized Einstein
solids.
 Our small microsystem has very little effect on the
temperature of the reservoir because the amount of energy
exchanged is limited. We can gain more insight into the
equilibrium process by changing the relative sizes of the
interacting systems.

Figure 11.5 shows the changes of entropy for two Einstein

solids of equal size, NA = NB = 512. We set up the systems so
that initially solid A has 3 units of energy per oscillator and
solid B has 1. Both are in equilibrium before they are allowed to
interact. Once interactions begin, the hotter solid A loses
energy more readily than solid B. The change of entropy of A,
ΔSA, is negative and decreasing relative to the initial entropy.
Conversely, the change of entropy of B, ΔSB, is positive and
increasing due to the absorbed energy ΔqB > 0 (Figure 11.5,
right). But, the positive gain ΔSB is more than the negative loss
ΔSA so the net change ΔST = ΔSA + ΔSB is positive.

Energy will continue to flow from solid A to solid B as long

as the net change in entropy is positive. But as the energy
transfer continues, the rate of entropy gain decreases. At the
point of equilibrium (ΔqB = 512, dashed line in Figure 11.5,
right), the net entropy stabilizes and fluctuates. We see the
systems reach equilibrium, and their energies are equal qA =
qB, apart from fluctuations.
 Figure 11.6: The multiplicity (top) and entropy (bottom) of two
interacting solids NA = NB = 512, sharing a total energy q = qA + qB =
2048.

The rate of entropy change is an important parameter.

From Figure 11.5 (right) we can see that as solid B gains energy,
the rate of ΔSB decreases, whereas the rate of ΔSA increases in
magnitude, though the clarity of the trends is affected by
fluctuations. A clearer picture is given in Figure 11.6 where the
total multiplicity and changes of entropy are calculated from
Eqs. (11.1) and (11.4) for the same parameters as in Figure 11.5
(Project P11.2). The scaled multiplicity shows a sharp peak near
the equilibrium, which would become increasingly narrower for
larger systems.

The values of entropy (Figure 11.6, bottom) are relative to a

single solid at the beginning qB ∼ 500. The arrows indicate the
rates of changes at the beginning and also at the equilibrium
(dashed line). When the rates are equal in magnitude, the
solids are in equilibrium. Because the total entropy is
maximum at that point, we have the equilibrium condition

We have used qB = q−qA in the last term. Equation (11.6) is

a statement on temperature which is the “thing” that is the
same for the solids or any systems in equilibrium. Considering
that the temperature of a solid rises with increasing energy, we
can therefore define the temperature as the inverse of the rate,

We can see from the slope change in Figure 11.6 that the
difference in temperature measures the tendency to give off
energy. A body at a higher temperature tends to lose heat, and
a body at a lower temperature tends to gain heat. Restated in
temperature, the equilibrium condition (11.6) is TA = TB.
 Figure 11.7: The temperatures of two interacting solids. The
parameters are the same as in Figure 11.6.

Figure 11.7 shows the temperatures from our simulation.

We calculate the temperature via the average energy from Eq.
(11.13) rather than from the derivative relation (11.7).4 The
temperature of solid A steadily decreases while that of solid B
increases. The solids reach an equilibrium temperature kT ∼
2.5ε. The many data points around qB ∼ 1024 indicate the
amount of fluctuation. We see the overall trends to be linear to
a very good approximation. This is because the average energy
〈E〉 is high. For smaller 〈E〉, the lines would curve (Exercise
E11.3).

The entropy in Einstein solids increases with energy (Figure

11.6), accompanied with increasing temperature. This is the
normal behavior. However, it is not always the case. For
systems such as a paramagnet that can store only a limited
amount of energy, the entropy can decrease with increasing
energy. The slope ∂S/∂E then is negative. According to Eq.
(11.7), we have a negative temperature. What does it mean,
though? We leave further exploration of such systems to
Project P11.3.

11.1.4 BOLTZMANN DISTRIBUTION

Through numerical simulations, we have established that the
energy distribution in equilibrium is an exponential factor
(11.3), and temperature is related to entropy by Eq. (11.7).

We only have to relate α in Eq. (11.3) to temperature (see

Section 11.A.1). Comparing Eq. (11.3) with the thermodynamic
result (11.51) and noting that En = nε, we find α = ε/kT. So Eq.
(11.3) becomes

This is the Boltzmann distribution. It describes the relative

probabilities of states of a system in thermal equilibrium [82].

The 1/Z factor may be determined through normalization of

probabilities ∑n P(En) = 1, which yields the so-called partition
function

If we know the partition function Z, we can determine all

properties of a thermal system in principle. For example, we
can calculate the average energy as a weighted average from
Eq. (11.8),

Heat capacity is another important thermodynamic

property. It is defined as

The heat capacity measures the amount of energy required (or

released) to raise (or lower) the temperature of a system by one
degree. For numerical work, it is more convenient to use an
alternative expression [41]

where Δ2E is the energy fluctuation (Exercise E11.2).

Many other quantities can be obtained this way. The

exponential Boltzmann distribution (11.8) is central to
thermodynamics.

As an illustration, we can finally determine why α = ln 2 in

Figures 11.2 and 11.3. We expect α is related to temperature,
which in turn should be related to the average energy of the
system. For an SHO, the average energy is analytically
 available, and the temperature can be expressed as (see
Exercise E11.3, ignoring zero point energy)

where ε (= ħω) is the unit of energy.

For the Einstein solid in our example above (Figure 11.2),

the average energy is 1, 〈E〉 = ε, so α = ln 2. Equivalently, the
temperature is kT = ε/ ln 2. We have used this value for the
exact Boltzmann distribution in Figures 11.2 and 11.3 that show
very good agreement between numerical (“experimental”) and
analytical results.

1.2 The Ising model

The Einstein model considered above consists of independent
oscillators exchanging energy with each other. Now we discuss
a model in which the particles are interdependent, i.e., coupled
to each other. The simplest, yet nontrivial, model is the Ising
model that can exhibit a range of thermodynamic properties. It
is useful for understanding critical behaviors including self-
organization in magnetism such as ferromagnetism where
magnetic dipole moments align themselves spontaneously in
the same direction, giving rise to macroscopic domains of net
magnetization. In terms of modeling, the simplicity of the Ising
model is ideal for us to clearly highlight the important
Metropolis algorithm whose basis rests on the Boltzmann
distribution.

11.2.1 THE 1D ISING MODEL

We consider the 1D Ising model consisting of N identical spins
(dipole moments), {si}, arranged in a chain as shown in Figure
11.8.

Figure 11.8: The one dimensional Ising model.

Each spin has two possible values: either up si = 1, or down

si = −1. The interaction energy between any two spins is
represented as

The parameter ε is the pair-wise energy (coupling). If ε > 0,

the interaction favors parallel spins, leading to
ferromagnetism. For antiferromagnetism, ε < 0, anti-parallel
spins are preferred. The specific value ε depends on the
properties of the interaction between atoms, not the direct
dipole-dipole interaction. The latter is too weak in comparison
to the exchange energies arising from symmetries of the wave
functions [76] (e.g., see Eq. (9.7) and Figure 9.3). For our
purpose, we assume ε > 0, treating it as the basic energy unit of
a ferromagnetic system.

In principle, all pairs interact with each other. To keep with

simplicity, the Ising model takes into account only nearest
neighbor interactions, neglecting all long-range interactions
that are presumably weak. We can express the total energy of
the 1D Ising model as the sum of neighboring pairs

Here, the last term involves sN+1, which is undefined. We could

just neglect that term, which is justified if N is large. However,
a better way to deal with the situation without unnecessary
complication is to use periodic boundary conditions,

In effect, we assume that there were virtual identical copies of

the chain to the left and to the right of the actual chain. This
also helps to minimize finite size effects in Monte Carlo
simulations. Another way to visualize Eq. (11.16) is the Ising
chain arranged in a ring. The 1D Ising model is analytically
solvable.

11.2.2 METROPOLIS SIMULATION

To simulate the dynamics of the Ising model, we need to put it
in contact with a reservoir at temperature T which we just
introduced in Eq. (11.7). The reservoir acts as a large heat bath
whose temperature remains unchanged while exchanging
energy with the Ising system.

Qualitatively, for a given initial configuration and a

specified T, the Ising chain will start exchanging energy by
flipping the spins in order to reach equilibrium. If the energy of
a microstate as given by Eq. (11.15) is too low, the system will
absorb energy from the heat bath. Spins will tend to align anti-
parallel to each other so the energy of the system is increased.
Conversely, if the energy of the microstate is too high, the
system will give off energy to the heat bath. This causes the
spins to flip toward parallel alignment.

These parallel and anti-parallel spin flips will continue

until, after sufficient time passes, the system reaches
equilibrium. Then sporadic flips will still occur, but on average,
the two types of flips balance each other out. The system will
just fluctuate around equilibrium (see Project P11.4 and Figure
11.26 for an animated Ising model).

The question now is how to simulate this equilibrium

process. A direct way is to use importance sampling (Section
10.4) to build up a set of microstates that obey the Boltzmann
distribution. We would sample randomly a microstate {si},
record its energy E from Eq. (11.15), and accept it with a
probability exp(−E/kT). The problem with the direct method is
that the number of microstates is large even for a tiny system.
For instance, if N = 1000, the number is already 2N ∼ 10300.
Out of the uniformly-sampled microstates, only a very small
fraction will be near the equilibrium.5 The direct approach
would be highly inadequate, unless the process could be biased
toward the equilibrium.

The Metropolis algorithm applies a proper bias to efficiently

guide the system into equilibrium and keep it there [62]. It
works as follows for the Ising model [41]. Starting from a given
microstate, we pick a spin randomly and propose to flip it. Let
ΔE be the energy difference caused by the flip. If the flip
produces a lower energy, i.e., ΔE < 0, the flip is accepted and
microstate changed. The reason is because lower energy
configurations are always preferred according to the
Boltzmann distribution. If the proposed flip produces a higher
energy, ΔE > 0, we do not discard the flip entirely. Instead, we
accept it with a probability exp(−ΔE/kT) according to Eq.
(11.8). This is to account for thermal fluctuations, for otherwise
the system would descend to the lowest possible state
monotonically with no fluctuation whatsoever, a behavior
contradictory to real systems.

This process is repeated many times. If we started with a

configuration having an energy above its equilibrium value, the
flips will lower the energy on average, and the Metropolis
method will accept the flips. This drives the system toward
lower energy configurations closer to equilibrium. The opposite
happens if the initial configuration is below the equilibrium.
Either way, once equilibrium is reached, the flips tend to raise
or lower the energy equally, producing only fluctuations. In this
pick-and-go manner, the Metropolis method samples
microstates obeying the Boltzmann distribution.

Let us first see how the method works in a simulation and

analyze it afterward. As stated earlier, we assume the Ising
model is ferromagnetic, and without loss of generality, we can
set ε = 1. The temperature is measured in terms of the ratio kT/
ε. The Metropolis sampling works as follows:

1. Set the initial microstate {si}. In principle, any random

configuration will do. But at low temperatures kT < 1,
the system takes longer to equilibrate. It is better in
practice to start with a cold configuration, i.e., more
spins parallel to each other than anti-parallel.
2. Pick a spin at random (say the i-th), and calculate the
energy difference if the spin is to flip (a trial flip).
Rather than calculating the energy anew, it is easier and
more efficient to just find the energy difference due to
the interaction with the nearest neighbors by

Note, the value of si in Eq. (11.17) is the pre-flip value.

3. If ΔE < 0, the trial flip is accepted, set si → −si.

Otherwise, draw a random number x. If x <
exp(−ΔE/kT), accept the flip and set si → −si; if not,
 reject the trial flip and keep the microstate unchanged.
This is the Metropolis algorithm.
4. Repeat steps 2 and 3 until equilibrium is established. As
a rule of thumb, each spin should be given a sufficient
number of chances to flip, say 10 to 100 (or more,
depending on temperature).

Program 11.6 simulates the 1D Ising model with the

Metropolis algorithm. The core module, update(), is listed
below.

def update(N, spin, kT, E, M): # Metropolis sampling

2 i, flip = rnd.randint(0, N−1), 0
dE = 2*spin[i]*(spin[i−1] + spin[(i+1)%N]) # Eq. (11.17) , periodic bc
4 if (dE < 0.0): flip=1 # flip if dE<0, else flip
else: # according to exp(−dE/kT)
6 p = np.exp(−dE/kT)
if (rnd.random() < p): flip=1
8 if (flip == 1): # accept
E = E + dE
10 M = M − 2*spin[i]
spin[i] = −spin[i]
12 return E, M

The Ising chain (N spins) is stored in the array spin[]. The

parameters kT, E, and M are respectively the temperature,
energy and magnetization (11.18) of the current state. The
module picks a random spin index between 0 and N − 1
(inclusive), and calculates the change of energy ΔE from Eq.
(11.17) if it is to be flipped. Note that the periodic boundary
condition (i = 0 or N − 1) is automatically enforced via modulo
operations. If the change ΔE is negative, the flip is accepted
and the flag set accordingly. If not, it is accepted with a
probability exp(−ΔE/kT). At the end, the energy,
magnetization, and the spin itself are updated if the trial flip is
accepted, flip=1.

Figure 11.9: The energy of an Ising chain (N = 1000) equilibrating at

three temperatures. The dashed lines are the analytic values in
equilibrium.

Figure 11.9 shows the energies toward the equilibrium

values of an Ising chain with N = 1000 spins as a function of
iterations (time). Three temperatures are used: low, kT/ε = 0.5
(ε = 1); intermediate, kT = 1; and high kT = 2. We initialize the
system to a configuration whose energy is below the high
temperature but above the low and intermediate temperatures.
In the beginning, the system rapidly exchanges energy with the
heat bath, and quickly approaches their respective equilibrium
values from below or above. This is evidenced by the steep
slope near the beginning.

In this specific run, the intermediate temperature case (kT

= 1) reaches equilibrium first, taking a bit less than one pass
(one pass = N iterations, each spin having one chance to flip on
average). The quick equilibrium is helped by the initial energy
closest to the analytic value for kT = 1.

Figure 11.10: The equilibrium energy (left) and magnetization (right)

per spin as a function of temperature. The solid line is the analytic
result (11.19).

The high temperature case (kT = 2) rises to its equilibrium level

around five passes. In both cases, the energy fluctuates around
equilibrium as expected. After a relatively quick initial dive, the
rate of energy loss in the low temperature case (kT = 0.5)
decreases substantially as the system becomes cooler. The
hotter initial state tends to give off energy at a faster rate. But
as it cools down, the temperature difference becomes smaller,
resulting a slower rate. The system still has not reached
equilibrium after ten passes. There is virtually no movement in
the last two passes shown. The spins are more tightly aligned,
and the chance of flipping to a lower energy is greatly reduced.
In this case, only after more than 50 passes did the system
reach equilibrium.

After equilibrium is achieved, we can start taking averages.

Figure 11.10 displays the results for the average energy and
magnetization as a function of temperature. The averages in
Monte Carlo Metropolis simulations are calculated as simple
algebraic averages. For instance, the average energy and
magnetization are

where E(n) is the energy of microstate n (configuration {sn}),

computed from Eq. (11.15), and M(n) = ∑i si is the
magnetization (net spin). The value Nmc is the total number of
Monte Carlo samplings. Because we sample the microstates
with the Metropolis algorithm, the importance-sampled
microstates obey the Boltzmann distribution. As a result, Eq.
(11.18) uses algebraic averages without the explicit weighting
factor exp(−E/kT) as would be the case from Eq. (11.10) if the
microstates were uniformly sampled. This is a another example
of importance sampling (see Section 10.4.2).
 The average energy per spin (Figure 11.10) is flat and very
close to −1 at T → 0, corresponding to all spins parallel to each
other. As seen from Figure 11.9, we must take considerable care
to make sure the Ising chain reaches equilibrium at low
temperatures before taking averages. The energy grows with
increasing temperature, and should approach zero as T → ∞, a
state of totally random spin alignment.

The exact energy is given by [82] (Exercise E11.7)

Overall, we see good agreement with analytic results for the

average energy.

For magnetization, the agreement depends on temperature.

Theoretically, magnetization is identically zero in the 1D Ising
model, even at low temperature when all spins are aligned. The
reason is two-fold. First, absent any external magnetic field,
there is no preferred direction for the spins. Second, in
practically every microstate, a spin can spontaneously flip
without changing the total energy. Sometimes this can lead to a
domino effect where the spins flip one after another, so the net
spin is reversed. The average magnetization is therefore zero.

We see good agreement between numerical and analytic

results for magnetization from intermediate to high
temperatures (Figure 11.10). But for low temperature kT < 1,
numerical data has considerable scatter and the disagreement
is large. Increasing the time for equilibrium and for sampling
did not seem to noticeably improve the disagreement. On the
one hand, most spins are aligned at low T, and there is a low
probability of finding a spin with anti-parallel neighbors where
a flip requires no extra energy, effectively blocking the most
favorable path. On the other hand, a positive amount of energy
(11.17) is needed to flip a spin with parallel neighbors, and the
probability of accepting the flip is prohibitively low due to the
small exponential factor. For example, in such a flip, ΔE = 4. If
kT = 0.5, the probability is exp(−ΔE/kT) = exp(−8) ∼ 3 × 10−4.
Basically, the spins are frozen in their alignment at low
temperatures, and the time scale for the whole Ising chain to
flip is very large. This also happens in actual thermal systems,
which can get stuck in a state of finite entropy at low
temperatures.

Figure 11.11: The heat capacity per spin as a function of temperature.

The solid line is the analytic result.
 We show the heat capacity per spin, C(T)/Nk, in Figure
11.11. Numerically, it is calculated from energy fluctuation
(11.12) (Project P11.4). The analytic result (solid line) is given
by

Qualitatively, numerical and analytic results behave

similarly. The heat capacity is small at low temperature. This is
consistent with the average energy (Figure 11.10) which shows
only small changes in energy when temperature is varied. At T
= 0, C(T) = 0 analytically, a result often attributed to the third
law of thermodynamics. At high temperature, the heat capacity
again approaches zero as C(T) → 1/T2, since rising temperature
is accompanied with little change in energy. In between we
observe a maximum at kT ∼ 0.8, corresponding to the change of
curvature (bend) in the energy diagram (Figure 11.10).

Quantitatively, the agreement between numerical and

analytic results is satisfactory from intermediate to high
temperatures (kT ≥ 1). At low temperature, the difference is
significant, in line with similar observations for magnetization.
For the same reasons as explained earlier, it is tricky to reduce
the considerable scatter in the numerical results because of
long relaxation times.
 Figure 11.12: The entropy per spin as a function of temperature. The
solid line is the analytic result.

Finally, we show in Figure 11.12 the entropy of as a function

of temperature. To compute the entropy S in the simulation, we
use the thermodynamic relationship (11.7)

We interpret δS as the change of entropy caused by the

absorbed energy δE. The results in Figure 11.12 are the
cumulative change of entropy, assuming S = 0 at T = 0.
Equation (11.21) directly relates entropy to heat and
temperature, all experimentally measurable.

The entropy starts from zero, increases quickly between kT

= 0.3 and 1, and approaches the asymptotic limit ln 2, i.e.,
random dipole orientations. The agreement is good between
the numerical and analytic results. The latter is given by
(Exercise E11.6)

Comparing entropy (Figure 11.12) with energy (Figure 11.10),

the trends are very similar. Upon closer examination of Eq.
(11.22), we see the first term dominates at higher temperatures,
but has a weaker temperature dependence because of the
logarithm. The second term, which is proportional to 〈E〉/T per
Eq. (11.19), dominates at intermediate temperatures. As a
result, the entropy curve takes on the trend of the average
energy at kT ∼ 1, but it levels off faster at higher temperatures
due to the 1/T factor.

Summarizing, all thermodynamic quantities presented are

continuous and smooth, including energy, magnetization, heat
capacity, and entropy. Therefore, the 1D Ising model does not
exhibit phase transitions. Per our discussion on magnetization,
a spin has only two neighbors in 1D, and the Ising chain can
flip-flop freely without energy constraint. It makes critical
phenomena impossible in 1D. However, it is possible in 2D
Ising model discussed shortly.

The Metropolis algorithm analyzed

We see from above discussions that the 1D Ising model is well
described by simulations with the Metropolis algorithm in
comparison with analytic results both qualitatively and
quantitatively. We wish to analyze more closely the underlying
mechanism.

Let E1 and E2 be the energies of two states in a trial

transition (flip), and assume E1 < E2. Further, let R(i → f)
denote the transition rate from state i to state f. According to
the Metropolis algorithm, transitions from states 2 → 1 occur
with unit probability, or R(2 → 1) = 1, since it is going from a
state of higher energy (E2) to a lower one (E1). Conversely,
transitions 1 → 2 has a rate R(1 → 2) = exp(−ΔE/kT) with ΔE =
E2−E1. In equilibrium, we expect the number of transitions to
be equal in both directions 1 ⇔ 2. Since that number is
proportional to the product of the transition rate and the
probability of finding the system in the initial state, we have

where P(E1) and P(E2) are the probabilities of finding the

system in states 1 and 2, respectively. Substituting the
Metropolis transition rates into Eq. (11.23) we obtain

This is precisely the Boltzmann distribution (11.8).

Therefore, the Metropolis algorithm leads the system,

sometimes slowly but surely, to its correct thermodynamic
equilibrium. This algorithm is essential to thermodynamic
simulations, and can be adapted to problems that are non-
thermodynamic in nature (see Section 11.3).

11.2.3 THE 2D ISING MODEL

The 1D Ising model offers many useful insights, but does not
exhibit phase transitions. The 2D Ising model addresses this
problem. It consists of spins arranged in rows and columns on
a lattice, say an N × N square grid for a total of N2 spins (Figure
11.13).

Figure 11.13: The 2D Ising model. Each spin interacts with four
nearest neighbors.

As in the 1D case, we still assume nearest neighbor

interactions. Each spin now interacts with four nearest
neighbors: up, down, left, and right, doubling the number of
nearest neighbors relative to its 1D sibling. The energy for a
given microstate is
where the sum is over all (unique) nearest neighboring pairs, (i,
j).

The Monte Carlo simulation of the 2D Ising model runs

parallel to that of the 1D Ising model. Let si,j be the spin picked
for a trial flip. The change of energy if the spin were to flip is

This is similar to Eq. (11.17) but with two more neighbors. If the
picked spin is on the edge, we use the periodic boundary
conditions (11.16) horizontally (column-wise) and vertically
(row-wise).

The Metropolis algorithm can be applied in nearly the same

manner as before, allowing for minor differences. The
equivalent module for the 2D Ising model is given below.

Program listing 11.3: Ising model in 2D ( metropolis2.py)

def update2d(N, spin, kT, E, M): # 2D Ising model, N x N lattice

2 i, j, flip = rnd.randint(0, N−1), rnd.randint(0, N−1), 0
dE = 2*spin[i][j]*(spin[i−1][j] + spin[(i+1)%N][j]
4 + spin[i][j−1] + spin[i][(j+1)%N])
if (dE < 0.0): flip=1 # flip if dE<0, else flip
6 else: # according to exp(−dE/kT)
p = np.exp(−dE/kT)
8 if (rnd.random() < p): flip=1
if (flip == 1):
10 E = E + dE
M = M − 2*spin[i][j]
 12 spin[i][j] = −spin[i][j]
return E, M

Compared to the 1D case (Program 11.6), the only difference

is that the spins are stored in a 2D N × N list. To select a spin,
we draw a pair of random integers (i, j) representing the
column and row indices, respectively. The change of energy is
calculated as discussed above from Eq. (11.26). The rest is
identical to the 1D case.

Below we discuss simulation results and leave the details of

modeling to Project P11.5. We consider a system made up of
1024 spins arranged on a 32 × 32 square lattice. First, we show
the spin alignment in Figure 11.14 at several temperatures.
Starting from random initial states, the system is sampled after
waiting sufficiently long so it has reached equilibrium. The
domains may shift depending on when the sampling takes
place, but the ratio of “up” to “down” spins should be constant
within thermal fluctuations.
 Figure 11.14: The Ising square lattice (32 × 32) in equilibrium. Filled
squares ( ) are “up” spins, and open squares ( ) are “down” spins.

At the lowest temperature shown (kT = 1), all spins are

aligned spontaneously (up). With increasing temperature,
more down spins gradually appear. At the higher temperatures
kT ≥ 2.8, the numbers of up and down spins are roughly equal,
a totally random distribution. However, something dramatic
seems to occur between kT = 2.2 and 2.4, which upsets the
gradual change in the up/down ratio. The abrupt change is a
telltale sign of phase transitions [82].

Phase transitions

Figure 11.15: The energy and magnetization per spin as a function of

temperature in the 2D Ising model. The solid lines are the analytic
results. The phase transition occurs at Tc = 2.2692 ε/k.

To investigate further, we show the equilibrium energy and

magnetization as a function of temperature in Figure 11.15. The
energy is continuous and looks qualitatively similar to the 1D
case. At low temperature, the energy per spin approaches −2ε,
doubling the amount in the 1D Ising model. In 2D, a spin has
four nearest neighbors. Since they are all aligned, the total
energy is −4ε. Because this energy is mutual between each of
the four pairs, the energy per spin is half the total. Another
property, difficult to visually spot, is a discontinuity of the slope
between kT = 2.2 and 2.4, the same interval observed above
(Figure 11.14). Clearly, this is no coincidence.
 The magnetization shows that the critical behavior is a
phase transition occurring at the critical temperature kTc/ε
2.2692. Above Tc, the magnetization is identically zero. Below
Tc, it rapidly rises to full, spontaneous alignment within a small
temperature window.

The analytic results in Figure 11.15 are given in Section 11.B.

The energy and magnetization per spin are [41]

Figure 11.16: The parameter m (11.57) as a function of temperature.

The value of m is 1 at Tc = 2.2692.

The function K(m) in Eq. (11.27) is the complete elliptic

integral of the first kind (11.61), and m is a temperature-
dependent dimensionless parameter (11.57). The temperature
dependence of m is shown in Figure 11.16.

The value of m approaches 1 from both sides of Tc, the

critical temperature. However, the function K(1) is infinite
because of a logarithmic singularity at m = 1 (Figure 11.27).
Because energy must remain finite, we require the factor in
front of the second term in Eq. (11.27) to vanish at Tc,

yielding the critical temperature

The energy at the critical temperature is .

From the magnetization in Figure 11.15, we see the system

undergoing a phase transition at the critical temperature Tc,
known as the Curie temperature. Spin orientations are
completely random just above Tc, whilst they spontaneously
order themselves just below. Note, however, spins of the same
orientation tend “stick” together due to the ferromagnetic
nature, forming pockets of various sizes like pieces on a go
board (Figure 11.14). With increasing temperature, the sizes of
the pockets become smaller.

Expanding Eq. (11.28) around Tc, we obtain

The power law scaling like Eq. (11.31) is a common feature in
phase transitions. The critical exponent in this case is 1/8.
Because the magnetization changes continuously from zero
across Tc (even though its derivative is discontinuous), the
phase transition may be classified as a continuous one.

Figure 11.17: The heat capacity per spin as a function of temperature

in the 2D Ising model. The solid line is the analytic result.

Figure 11.17 displays the heat capacity as a function of

temperature. The most prominent feature is the sharp spike at
the critical temperature, another sign of possible phase
transitions. Unlike energy, there is no requirement that the
heat capacity be finite. The critical behavior around Tc can be
derived from the analytic expression (11.60),
This is a logarithmic discontinuity of the derivative of energy.
The agreement between simulation and analytic results are
good in regions away from the singularity around Tc. Near the
singularity, the fluctuations in energy is large, resulting in
larger scatter in the heat capacity.

We can now compare the 1D and 2D Ising models and

understand why the 1D model does not show phase transitions.
In 1D, we can reverse the whole spin chain without energy cost.
The initial spin flip may require some extra energy, but it is
given back at the end. This is always possible at finite
temperatures. The exception is T = 0, when the spins are fully
aligned. The Ising chain cannot reverse direction
spontaneously without additional energy.

In the 2D model, a spontaneous full reversal of the lattice is

not possible once the temperature is below some critical value
when net magnetization becomes none zero. On average,
additional energies are required to get the process going due to
the constraints of the domain walls and neighbors. For
instance, consider one row of the spin lattice, the equivalent of
a 1D spin chain. This row cannot flip spontaneously because of
the interactions with the rows just above and below.

Finally, we discuss finite-size (or edge) effects. All of our

results have been obtained with a square lattice of 32 × 32, tiny
compared to realistic systems. In most cases, our results are in
good agreement with analytic results for N → ∞. We conclude
that finite-size effects are small except near phase transitions.
Certainly, periodic boundary conditions help to minimize these
effects. Nonetheless, we expect the accuracy to improve with
larger systems in the region surrounding the phase transition.

In Figure 11.18, we show a larger Ising model containing

1048576 spins on a 1024 × 1024 square lattice at the critical
temperature Tc = 2.2692. The initial microstate was hot with
random spin directions. The snapshot was taken when
equilibrium was well established after each spin had on average
3000 chances to flip, roughly 3 × 109 iterations. The energy of
the sampled state is −1.399, within ∼ 1% of the analytic value
. The error remains stable. Analytically, the magnetization
should be zero, compared to the sampled value −9.7 × 10−4. It
means that only about 500 spins need to flip to achieve a
perfect zero magnetization. Generally, the sampled
magnetization has a higher fluctuation than energy, typically in
the range ±0.02.

We again see the tendency of parallel spins sticking together

as the lattice is covered by large connected domains
(continents), some spanning the full extent of the lattice. In the
limit N → ∞, the continents grow to infinity. But there are also
smaller islands of varying sizes down to single sites, coexisting
with the continents.
 Figure 11.18: Snapshot of a 2D Ising model on a square lattice (1024
× 1024, 1048576 spins) in equilibrium at the critical temperature Tc =
2.2692.

1.3 Thermal relaxation by simulated

annealing
As the previous examples illustrate, a thermal system at high
temperatures will typically be energetic and well mixed. As
temperature decreases, it will become more ordered
(homogeneous) and equilibrate toward a lower energy state. In
the limit T → 0, the system should approach the ground state,
provided the temperature decreases gradually. This can take a
long time.

This thermal relaxation occurs naturally in many processes

such as annealing in materials. For example, heating a metal to
higher temperatures and cooling it slowly can improve its
structure and properties. Heating helps atoms to mix up with
their neighbors, and gradual cooling ensures that they have
enough time to find the lowest possible energy states and the
most stable structure.

We can simulate the annealing process with the Metropolis

method. There are many applications in which simulated
annealing is useful and where the problems may not be
thermodynamic in nature, including minimization problems in
particular where finding the global minimum is essential. In
these problems, we first find the equivalent parameters that
play the role of energy and state (configuration), then apply the
Metropolis method, starting from high to low temperature. At
each temperature, sufficient time (Monte Carlo samplings) is
allowed for equilibration. The temperature is then lowered, and
the process is repeated. Hopefully, the system will be gradually
brought to the lowest possible state at the lowest temperature.
Below we discuss an example (more examples can be found in
Section S:11.1).
The falling tablecloth revisited
Let us revisit the relaxation of a free hanging tablecloth
discussed in Section 6.8, where it was modeled as a square
array of particles (Figure 6.20) connected by springs.

The relaxation problem was solved by integrating equations

of motion in Section 6.8. If we treat it as a thermal process, it is
essentially a minimization problem. Given a system of N × N
particles of mass m on a square lattice of side L, suspended
from the corners, and connected via springs of relaxed length l
and spring constant k, what is the minimum energy of the
system?

Figure 11.19: Snapshots of a hanging tablecloth in thermal relaxation.

Starting from a flat surface (upper left), the temperature decreases in
−7
clock-wise order, ending at kT/ε = 10 and a minimum energy
−2.718ε.

To solve this problem by simulated annealing, we randomly

select a particle at grid (i, j). We then jiggle it slightly, causing a
displacement of , and a new position vector of
. We assume the (x, y) coordinates in the horizontal
plane, and z in the vertical plane. The change of potential
energy (see also Eq. (S:11.1)) for this proposed move is

where n denotes the nearest neighbors at (i±1, j) and (i, j±1),

is the new distance to the neighbor n, and dn the
original distance.

Having found ΔE, it is straightforward to implement the

Metropolis method (Project P11.6). We show the results in
Figure 11.19 for a 13 × 13 = 169 particle system. The parameters
k, l, and m are the same as in Program 6.8.

We start from the flat configuration (Figure 11.19, upper

left, E = 0) at T = 10−3 (set ε = 1 as usual). The next state (upper
right) was taken after a few passes so the system is not in
equilibrium yet. Its energy is becoming negative, but the
surface is rather rough. Temperature is decreased in clockwise
order, and the next two states (lower right and left) are
snapshots in equilibrium at T = 5 × 10−4 and 10−7 respectively.
The energy changes very little after the last state. The final
energy is −2.718ε. Though the exact solution is unknown, it
should be very close to the global minimum energy.

Compared to the direct integration method (Figure 6.21,

Section 6.8), simulated annealing is simpler to implement, but
 is slower to converge to the global minimum. In direct
integration, if the damping coefficient is chosen properly (at
critical value), the system can relax to the global minimum
quickly. There is no single parameter in simulated annealing to
control this rate. It is the nature of thermal systems that
changes occur very slowly at low temperatures. But simulated
annealing is more efficient at exploring a larger parameter
space and can be applied to solving very difficult problems such
as the traveling salesman problem (Section S:11.1).

1.4 Molecular dynamics

We began this chapter by choosing statistical methods over
first-principle calculations. Now we return to the latter for
molecular dynamics which simulates the motion of N-body
systems (atoms or molecules) classically. Because the masses of
atoms are much larger than the electron mass (∼ 2000 times),
the motion of an atom can be adequately described classically
by Newton's laws at all but very low temperatures.

The reason for first-principle molecular dynamics

simulations is not to replace statistical mechanics but to
complement it. For instance, if we wish to study kinetic and
thermal properties of gases such as Maxwell speed
distributions (Section 11.4.4), we can use molecular dynamics
to follow the motion of each atom classically and obtain
thermal properties statistically.6 Furthermore, many systems
such as molecular protein or motor simulations involve
thousands to millions of atoms in non-equilibrium motion, and
molecular dynamics is a very useful tool. It can be used to study
atomic and chemical structures or reactions because the
internuclear degrees of freedom are effectively separated from
the electronic degrees of freedom.

11.4.1 INTERACTION POTENTIALS AND

UNITS
In a sense, molecular dynamics, an N-body problem, is an
extension of the few-body problems discussed in Chapter 4. For
instance, we can generalize the three-body equations of motion
(4.42) to an N-body system as

where is the force between atoms i and j. We typically

consider systems of identical particles, so the mass mi = m will
be the same for all N atoms.

Unlike gravity, the forces are Coulombic in nature, and

there are no exact expressions for them due to many-body
interactions (electrons and nuclei). For practical reasons, we
use phenomenological forces that approximate the interactions
between atoms, including the Morse potential (9.37) (Figure
9.10) and the Lennard-Jones potential (Project S:P9.3). The
latter is a two-parameter empirical potential popular among
chemists and commonly used in molecular dynamics
simulations

where V0 is a positive constant and r0 the equilibrium

internuclear distance (Figure 11.20). The first term, 1/r12,
describes the strong repulsive wall for r r0 where the
electrons repel each other (Coulomb and Pauli exclusion), but
the 12th power law is chosen more for convenience than for
fundamental significance. The second term, 1/r6, describes the
long-range attraction between neutral atoms due to dipole-
dipole interactions (van der Waals force).7 The switch-over
between repulsive and attractive forces occurs at r0, where the
potential is minimum at −V0.

The force on atom i by atom j is equal to the gradient −∇V,

where is the relative position vector between atoms i

and j, and is the magnitude. As Figure 11.20
shows, the force behaves qualitatively similar to the potential, a
sharply rising (1/r13) repulsive force for small r and a rapidly
decreasing (1/r7) attractive force for large r. In between, the
force is most negative at r = (13/7)1/6r0 ∼ 1.11r0, slightly larger
than the equilibrium distance, with the value f ∼ −2.69V0/r0.
This force is effective within a small radius r/r0 2, and is
negligible for larger values r/r0 3. It is a short-ranged force.
We will use Eq. (11.37) in the equations of motion (11.35).

Figure 11.20: The Lennard-Jones potential and force.

Like before, we need to choose a convenient unit system for

the problem at hand. We could use atomic units (Table 8.1),
but it is unwieldy as the mass would be unnaturally large, and
velocities and energies very small. Since we will be using the
Lennard-Jones potential, we shall define energy in units of V0,
length in units of r0 as listed in Table 11.1.

Table 11.1: Unit system for molecular dynamics based on the

Lennard-Jones potential.

Like the unit system for restricted three-body problems

(Table 4.3), Table 11.1 is also a floating unit system whose
absolute scale is determined by the Lennard-Jones parameters
and the mass of atoms. In this sense, simulation results with
this unit system are universal. It is particular tidy now for
numerical work, for we can just drop V0, r0 and m from Eqs.
(11.37) and (11.35). For example, the parameters for argon are
V0 = 0.01 eV, r0 = 3.9 Å, and m = 40 amu (6.7 × 10−26 kg). The
other derived units are: velocity, v0 = 150 m/s; time, t0 = 2.5 ×
10−12 s; temperature T0 = 120 K; and pressure P0 = 2.7 × 107 Pa
= 270 atm.

11.4.2 PERIODIC BOUNDARY CONDITIONS

There is just one more problem before we start writing the
program, and it is this: what boundary conditions to use? In
actual thermal systems, there are around 1023 particles.
Collisions among the particles determine the thermal
properties of the systems. Boundary effect are negligible, for we
know that air in a ball, a bottle, or a box behaves the same way.
We cannot possibly keep track of that many particles. However,
we have seen that it is possible to simulate behaviors of real
thermal systems with much smaller model systems if we are
careful with boundary conditions.

Suppose we set up the system in a cube, for instance, and let

the atoms interact and move about. Some atoms will eventually
collide with the walls. To prevent them from escaping, we could
calculate the point of collision and reflect the atom off the wall
elastically, as was done in the stadium billiard problem
(Section S:5.1). For a few hundred or thousand particles,
collisions with the boundaries will be much more frequent than
collisions between the particles themselves. So the boundaries
will artificially affect the behaviors, contrary to how real
systems behave.

It turns out that, again, we can call on periodic boundary

conditions to come to the rescue. We imagine that identical
cubes containing identical but virtual systems surround our
central cube [42]. For every atom moving in our system, there
is an atom with exactly the same position (relative) and velocity
in every other virtual cube (Figure 11.21). If an atom escapes
our cube, say from the right side (Figure 11.21, atom ),
another one ( ) enters from the left side to replace it. This way,
we do not have to worry about collisions with the walls,
because there are no collisions, and we never lose atoms. When
a particle exits the cube on one side, it is “teleported” to the
opposite side ( , Figure 11.21) without altering its velocity.

But teleporting atoms this way creates another problem. As

Figure 11.21 illustrates, the distance between atoms and
changes abruptly when the former is teleported, from r to r′.
This causes a discontinuous and unphysical jump in the
potential energy and the force. Worse, the teleported atom can
exert such a strong, and sudden, force on the atoms already in
the close vicinity of its new position that some can be ejected
from the system in just a few steps due to numerical error,
causing the system to disintegrate.
 Figure 11.21: Periodic boundary conditions in molecular dynamics
simulations. The central real box is surrounded by virtual copies all
around.

A careful look of Figure 11.21 between the real atom and

the virtual atom (in the right box) reveals that if we use the
distance r′ instead of r, we can avoid the discontinuity in the
force. It suggests that we should use the smaller of the two
horizontal separations, and , between the atoms.
Physically, this means that an atom always finds another atom,
be it a real atom or a virtual clone, that is closer along a given
axis to interact with. Let us call this close-neighbor interaction.

Examination of Figure 11.21 shows that, within close-

neighbor interaction, the largest horizontal or vertical
separation between two atoms (real or virtual) is equal to half
the width of the box. For example, if we shift horizontally
between and , the largest separation occurs when is
exactly in the middle of the two atoms.

Let L be the width of the box, and Δx = x1 − x2. If ,

we accept it as is. But if , we require the following
correction

Similar relationships hold for y and z directions.

With the simple and elegant periodic boundary conditions

(11.38) as described above, we solve both the boundary-wall
collision problems and the abrupt jumps in interaction energies
and forces. There is one drawback, however. At exactly
, the force is ambiguous. Using atom as the
example (Figure 11.21), the force would change sign depending
whether it interacts with to the left or with the virtual clone,
, to the right. We justify this situation by arguing that, for a
box size L r0, the forces between atoms separated at are
small as to be negligible. For the Lennard-Jones potential
(Figure 11.20), this is true to a very good degree of
approximation since the force is short-ranged and is negligible
for r 3 as stated earlier (Figure 11.20). Take L = 10, for
instance, the magnitude of the force at is ∼ 10−4. In this case,
we expect that the periodic boundary conditions should work
well.

Of course, if there were significant spurious effects arising

from the use of Eq. (11.38), we would need to amend our
approach. Fortunately, Eq. (11.38) works well for molecular
dynamics simulations using potentials that decrease rapidly for
large r as does the Lennard-Jones potential.
11.4.3 SIMULATION AND VISUALIZATION
We can now write a molecular dynamics program employing
periodic boundary conditions. Taken from the complete
Program 11.7, the heart of the simulation computing the core
N-body dynamics is given below.

1 def nbody(id, r, v, t): # N−body MD

if (id == 0): # velocity
3 return v
a = np.zeros((N,3)) # acceleration
5 for i in range(N):
rij = r[i]−r[i+1:] # rij for all j>i
7 rij [rij > HL] −= L # periodic bc
rij [rij < −HL] += L
9 r2 = np.sum(rij*rij, axis=1) # |rij|ˆ2
r6 = r2*r2*r2
11 for k in [0,1,2]: # L−J force in x,y,z
fij = 12.*(1. − r6)*rij [:, k]/(r6*r6*r2)
13 a[i,k] += np.sum(fij)
a[i+1:,k] −= fij # 3rd law
15 return a

The module nbody() returns the equations of motion (11.34)

and (11.35) in the required format for leapfrog integration. The
variables r and v are arrays of vectors, effected by 3-element
NumPy arrays, containing the positions and velocities of the
atoms. If id=0, velocities are returned. Otherwise, acceleration
is calculated in an optimized manner with NumPy arrays.
 For a given atom i, we obtain the forces, , between it and
all higher-numbered atoms j = i + 1 to N (of course j ≠ i, an
atom does not exert forces on itself). The forces are
found by Newton's third law, reducing computation by half.
The cumulative effect is that for atom i we only need to find
for j > i since the other forces (j < i) had been found already in
prior iterations. For instance, when i = 1, forces , , etc., will
be calculated. When i = 2, forces , , etc., are needed, but
was already available and recorded and need not be
recalculated.

Normally, one would need an explicit inner loop over j, but

we avoid it for the sake of speed. We first find the relative
position vectors (line 6) between i and all j > i,
simultaneously. Here, is broadcast into a (N − i − 1) × 3 array
(shape of r[i+1:]), equivalent to itself duplicated N − i − 1
times, and then for j > i is subtracted from it via element-
wise operations. Next, the components of are checked for
periodic boundary conditions (11.38) to ensure close-neighbor
interactions. This is accomplished with truth array indexing
(see Section 1.D, Section 4.4.2), so that separations greater
than L/2 in any direction (e.g., |xij | > L/2) are mapped
according to Eq. (11.38), again all at once. The distances
squared between the atoms are obtained by summing |rij|2 over
the three directions via np.sum along the second axis, i.e.,
across columns (line 9). It is slightly faster to compute r6 and
r14 from r2 using multiplications rather than powers ( r**n)
which is slower for low n.
 Finally, each component of the net force on atom i due to
atoms j > i is summed up, and the same component on atom j
due to i is obtained via Newton's third law. On exit,
accelerations are returned as an N × 3 array of vectors.

Figure 11.22 shows two snapshots of an animated molecular

dynamics simulation. We used N = 16 atoms, initially randomly
distributed in a cube of side length L = 10. The initial velocities
were set randomly between [−1, 1]. We can see from the trails
that close-by atoms interact strongly (sharp turns), and distant
ones interact weakly (nearly straight line trajectories). In
particular, the dark-colored (red) atom (Figure 11.22, left)
suffers a strong force near the beginning, being close to several
atoms to the right. It makes a sharp turn toward the left wall.
After a short time, it exits the cube, and re-enters from the
right side by the periodic boundary conditions. For the
duration shown between the two frames, this is the only atom
escaping the cube. But several atoms near the top and back in
the second frame are just about to exit the cube. When they do,
the same periodic boundary conditions will be applied, and
they will continue to interact with other atoms without
collisions with boundary walls.
 Figure 11.22: Animated molecular dynamics simulation. The left-
most particle (red/dark color) in the left frame exited the cube and is
wrapped around by periodic boundary conditions (right frame).

11.4.4 MAXWELL DISTRIBUTIONS AND

KINETIC PROPERTIES
We can obtain a slew of properties by analyzing relevant data.
We show only results and leave the work to Project P11.7. All
results are in the floating-scale units (Table 11.1). Figure 11.23
displays the velocity P(vx) (similar for vy and vz) and speed
P(v) distributions. We divide the velocity and speed ranges into
certain number of bins. Data is sampled periodically (say every
20 time steps). Each time, the atoms are added to the
appropriate bins according to their velocities and speeds. The
cumulative results are plotted as histograms in Figure 11.23,
normalized to one at the maximum.
 The results are broken down into three time intervals to
show the distribution at different stages of evolution. Toward
the beginning (t = 0 to 10), the initial conditions are heavily
weighted with large scatters. By the second interval 10 to 50,
the distributions look more and more like the equilibrium
Maxwell velocity distribution [82]

Figure 11.23: Velocity (left) and speed (right) distributions for N = 40

atoms. Data are taken in different time intervals: •, t = 0 to 10; , 10
to 50; , 50 to 400. The solid curves are the Maxwell distributions.

and the speed distribution

Roughly speaking, Eqs. (11.39) and (11.40) are a result of the

Boltzmann distribution (11.8), with and ,
respectively. The prefactor 4πv2 in Eq. (11.40) comes from the
volume element d3v = 4πv2dv.
 In the last interval (50 to 400), the results are in very good
agreement with the respective Maxwell distributions, including
the symmetry about zero in the velocity distribution, and the
forward-backward asymmetry about the maximum in the
speed distribution. This indicates that the system has reached
equilibrium for t 50. Considering that we have only N = 40
atoms in the simulation, the convergence is quite fast, a big
factor due to the use of periodic boundary conditions.

Figure 11.24: The average of speed squared as a function of time.

The system is the same as in Figure 11.23.

The equilibrium temperature is determined by the energy of

the initial condition. The average kinetic energy is related to
temperature by the equipartition theorem: a quadratic degree
of freedom carries of energy [82]. An atom has three
translational degrees of freedom in 3D, so its average kinetic
energy is
We can obtain the equilibrium temperature from Eq. (11.41).
Figure 11.24 shows the speed squared as a function of time. It is
calculated periodically by averaging the speed squared v2 of all
atoms at once. The average value of v2 rises steadily in the
beginning, indicating an oversupply of potential energy in the
initial condition. It quickly reaches a plateau around 〈v2〉 ∼ 1.85
at t ∼ 40, consistent with the equilibration time observed from
Figure 11.23. Using this value, the temperature from Eq. (11.41)
is kT = 〈v2〉/3 = 0.62. We have used this temperature in the
Maxwell velocity and speed distributions in Figure 11.23.

There is considerable fluctuation in v2. The main cause is

due to the small number of atoms. During motion, some atoms
can “bunch” up, creating local concentrations that have higher
or lower than normal potential energies. As a result, kinetic
energies rise or fall accordingly, leading to the observed
fluctuations.
 Figure 11.25: The number of collisions (left) and the average
pressure (right) as a function of time. The system is the same as in
Figure 11.23. The solid curve shows the expected pressure for an
ideal gas at kT = 0.62.

We can also obtain pressure from our simulation (Figure

11.25), even though using periodic boundary conditions we
have managed to avoid any atom-wall collisions, which
generates pressure in real systems. Pressure is calculated as
follows. When an atom escapes the cube, say from the side, we
record its velocity vx before teleporting it. We assume that, as
in a real system without periodic boundary conditions, the
atom would be spectrally reflected from the wall, reversing the
x velocity to −vx. The change of momentum (impulse) would be
Δpi = −2mvx. We keep cumulating the impulse within a given
time interval Δt (e.g., 20 time steps). The force exerted on the
wall would be F = ∑iΔpi/Δt, which yields the pressure, P =
F/L2.

Figure 11.25 shows the number of collisions per Δt = 0.2

(left) and the pressure (right) obtained in the manner
described. For this simulation, there are on average about 30
atoms escaping one side of the cube in each interval (about 1.5
atoms per step), a relatively low number. The fluctuations are
again mainly due to the atoms bunching up as discussed
earlier. The pressure generally follows the trend in the number
of collisions as expected, but it is modified by the momentum
carried by the atom. We also show the pressure for an ideal gas,
PV = NkT, assuming the equilibrium temperature found
earlier. There is good agreement between numerical results and
the ideal gas law. We conclude that, although our gas is not
ideal because of short-range interactions instead of point-like
hard spheres, the system is sufficiently dilute that the ideal gas
law is approximately valid.

However, the actual equation of state for an interacting gas

deviates from the ideal gas law. According to the theory of
weakly interacting gases, the pressure can be expanded as

where n = N/V is the particle density. The constants B2 and B3,

etc., are called the virial coefficients [76], which are zero for
non-interacting, ideal gases. For low density gases, the leading
correction is the second term, B2(T)n2. The second virial
coefficient B2 is given by

If we take V to be the Lennard-Jones potential (11.36), the

average potential energy should be negative at low
temperatures, and we expect the integrand in Eq. (11.43) to be
positive, yielding a negative B2. At high temperatures, higher
(positive) values of the potential energy near the hard wall
becomes important. The value B2 should be positive because
the integrand will be negative (e−V/kT < 1 if V > 0). With regard
to the results in Figure 11.25, the particle density is n = 4 ×
10−2. With n this small and only one temperature, we cannot
clearly extract the effect of the correction term. Further
investigation of the second virial coefficient is left to Project
P11.7.

Chapter summary

In the preceding sections we discussed equilibrium

thermodynamics and Monte Carlo simulations of thermal
systems. With the Einstein solid model, we presented
simulations that showed the Boltzmann distributions as a
natural outcome of energy exchange maximizing the
multiplicity, i.e., probability. This enabled us to develop the
concept of entropy and temperature, and to construct
algorithms to calculate them. We showed that the total entropy
of interacting solids increases and drives the systems toward
equilibrium.

We introduced the Metropolis algorithm in the discussion

of 1D and 2D Ising models. It showed that we can simulate
statistical properties of large systems using much smaller
thermal systems. We did a careful examination of numerical
and analytic results, including phase transitions absent in 1D
but present in 2D Ising models. Simulated annealing, i.e.,
Monte Carlo sampling with the Metropolis algorithm, was
introduced to study non-thermal problems such as the hanging
tablecloth by treating it as a thermal relaxation process.

Thermalization of N-body systems was studied with

molecular dynamics simulations, showing the importance of
 periodic boundary conditions to avoid the artifacts of particle-
wall collisions. At the same time, we used the method of virtual
particle-wall collisions to extract pressure and to study the
kinetic properties of weakly interacting gases.

We have used extensively the capabilities of Matplotlib and

NumPy libraries for visualizing data including computation
and presentation of various distributions. We also find that the
combination of the leapfrog method and vector operations
simulated by NumPy arrays is adequate for medium scale N-
body simulations.

1.5 Exercises and Projects

EXERCISES
E11.1 (a) Consider a chain of N spins, each can be up or
down. A microstate refers to any ordering of the spins,
and a macrostate refers to the number of “up” spins (see
e.g., Ref. [82]). How many microstates are there? How
many macrostates?

For a given macrostate N↑, show that the multiplicity is

This is a binomial coefficient ( ), which reads “m

choose n”.

(b) For an Einstein solid with N SHOs and q units of

energy, show that the multiplicity is

It gives the number of ways of distributing q quanta

among N oscillators, like putting q eggs in N baskets.

One method is to convert the problem into one similar

to the spin chain above. Let q be the eggs ( , “up

spins”), and the SHOs be the baskets ( , “down

spins”). Consider the number of eggs to the left of a

basket as the units of energy in that SHO, with no eggs
allowed to the right of the last SHO. For instance, the
following layout illustrates three units of energy in a
system of four SHOs:
 We interpret this configuration as follows: the first SHO
has one unit of energy, the second none, the third two,
and the last none. The effective length of the “spin”
chain is N − 1 + q since the last SHO does not move.

E11.2 (a) Verify the relationship (11.10) between the average

energy and the partition function.

(b) Derive the heat capacity (11.12). First rewrite Eq.

(11.11) in terms of ∂〈E〉/∂β, β = 1/kT, and show that

E11.3 (a) Show that the partition function and the average
energy for an SHO are

Assume , n = 0, 1, 2, …, and ε = ħω is a

unit of energy.

The first term in 〈E〉 is the zero point energy at T = 0.

Argue that it represents an energy shift and can be
 safely ignored in practical calculations. Obtain the low
and high T limits.

(b) Calculate the heat capacity per oscillator from Eq.

(11.11). Explain the limiting behavior for low and high
T.

E11.4
(a) For N 1, Stirling's approximation can be used to

compute N! as

Compare the exact values with Stirling's approximation

for N = 1, 5, 10, 50, and 100. Calculate the absolute and
relative error in each case.

(b) A related result is the logarithm of N!,

Calculate the absolute and relative error for N = 10n, n =

0 to 6.

E11.5 Working with large numbers in thermodynamics,

numerical problems such as over- or underflow can
occur in intermediate steps even though the final results
are reasonable and well defined. A useful technique is
to use logarithm for the intermediate calculations and
 convert back in the final step. For example, with the
identity x = exp(lnx), we can calculate m!/n! = exp(ln m!
− ln n!) by computing the exponent first and the
exponential last. The logarithm of the Gamma function
can be found from scipy.special, gammaln().

(a) Let N! = 10x. Find x for N = 1000.

(b) Evaluate exactly, and compare with the

value using Stirling's approximation for the factorials.

E11.6 Consider a paramagnetic system with N = N↑ + N↓

dipoles. Each dipole can be in one of two states, E =

ε, depending on whether it is aligned (↑) or antialigned

(↓) with the external magnetic field B. The energy unit
is ε = μB with μ being the magnetic dipole moment. The
total energy is E/ε = −N↑ + N↓ = N − 2N↑. For all the
quantities below, sketch their temperature dependence,
and derive the low and high T limits.

(a) Write down the partition function for a single dipole,

and show that the average energy per dipole is

The total energy is then 〈E〉 = N〈∈〉 = −Nε tanh(βε).

 (b) Calculate the entropy of the paramagnetic system.
Express the number of “up” dipoles N↑ in terms of
energy q = 〈E〉/ε from above. Using the multiplicity
(11.44) from Exercise E11.1, show that for large N the
entropy is given by

Rewrite it in terms of temperature to show the

equivalent expression (11.22).

(c) The magnetization is the average dipole moment.

Show that for a single dipole, the magnetization is

The total magnetization is 〈M〉 = N〈m〉. Compare the low

T limit with Curie's law, M ∝ 1/T.

(d) Derive the heat capacity

E11.7 Consider the 1D Ising model consisting of N dipoles.

The partition function can be written as
 Assuming periodic boundary conditions, sN+1 = s1,
show that

Prove that the properties of the 1D Ising system are

identical to the paramagnetic system (Exercise E11.6).

E11.8 (a) Derive the energy (11.59) and heat capacity (11.60)
of the 2D Ising model.

(b) Calculate and graph the entropy of the 2D Ising

model as a function of temperature. Use Eq. (11.56)
with the partition function (11.58). It may be convenient
to expand the logarithm inside the integral as ln(1 + x) ∼
x and use the elliptic integral (11.62). Comment on your
result around the critical temperature Tc.

PROJECTS
P11.1 Explore the Einstein solid model.
 (a) Make a table listing all possible microstates of an
Einstein solid for N = 3 and q = 4. Verify that the total
number agrees with the multiplicity from Eq. (11.45).

(b) Assuming N = 400, graph the entropy S/k as a function

of energy q = 0 to 500 from Eq. (11.4). To avoid overflow,
either use the techniques of Exercise E11.5, Stirling's
approximation (Exercise E11.4), or the combination
function comb from scipy.misc.

(c) Modify Program 11.5 to calculate the entropy of an

Einstein solid in equilibrium. Use the same N as part (b)
above. Vary q in the range 0 to 500 at equal increments
(say 20). Initialize the solid for each q, equilibrate the
system, and record the stabilized value of the entropy (the
plateau part of Figure 11.4). To reduce fluctuation, sample
entropy multiple times after equilibrium and take the
average. Graph the results and compare with the analytic
results above.

(d)* Choose a q (say 200), plot the progress toward

equilibrium as intensity images similar to Figure 11.1.
Produce the energy distribution histograms, determine the
temperature and compare with the Boltzmann distribution.

P11.2 Consider an isolated system consisting of two interacting

Einstein solids with NA and NB oscillators, respectively,
and sharing q = qA + qB units of energy. You can use
SciPy's combination function (Project P11.1) for the first
two parts.
(a) Let NA = 6, NB = 4, and q = 10. Calculate the
multiplicity as a function of qA from Eqs. (11.1) and
(11.45), and plot the results. What is the probability of
finding the most probable macrostate?

(b) Let NA = 600, NB = 400, and q = 300. What is the most

probable macrostate? Plot the multiplicity distribution as a
function of qA, normalized to 1 at the maximum. Also plot
the entropy analogous to Figure 11.6.

(c) Simulate the interaction process of the solids. Create a

program similar to Program 11.5. Initialize the solids with
qA = 100 and qB = 200, equilibrate each well before
interactions can begin. Monitor the entropy or the
Boltzmann distribution to ensure equilibrium is reached.
Calculate the initial temperatures, and the final
equilibrium temperature.

Allow the systems to interact by combining them into one

solid as

C=A+B # combine solids

C.exchange(L) # energy exchange

This enables both inter- and intra-solid interactions. At

any instant, solid A is in the first NA elements
 ( C.cell[:Na]) and solid B in the next NB elements
( C.cell[Na:]).

From the start of the interaction, calculate the changes of

entropy and temperature in each solid. Sample the systems
at intervals In+1 = f In (geometric distribution) between
iterations, starting with I1 = 2. First set f = 2, and once In is
larger than some value, say 100, reduce f ∼ 1.2 or so. This
speeds up the simulation in later rounds without
sacrificing details in the early stages. Append the results in
arrays, including the energies qA and qB. Stop sampling
once equilibrium has been well established.

Plot ΔSA, ΔSB, and ΔST vs. qA on the linear scale and vs.
the (cumulative) iteration number on the semilog x scale.

(d) Calculate the heat capacity of solid A (or B) from Eq.

(11.12). Plot the numerical results as a function of
temperature. Compare with analytic results (Exercise
E11.3).

P11.3 Study the two-state paramagnetic system outlined in

Exercise E11.6, which also contains the analytic results
needed in this project.

(a) First consider a single system with N = 10000 dipoles.

Graph the entropy and temperature as a function of q =
[−N, N] using the analytic expression. Explain the general
shapes of the plots.
Next, divide the system into two interacting subsystems A
and B with NA = NB = N/2 sharing a total of q = 0 units of
energy. Plot the multiplicity and entropy of the system as a
function of qA from analytic formulas.

(b) Write a program to simulate the interaction between A

and B. Initialize the subsystems with qA = −2000, qB =
1000. This determines the number of “up” (N↑) and
“down” (N↓) dipoles within each subsystem. Let the
dipoles interact with each other.

In each iteration, pick a pair of dipoles randomly, and

swap their orientations. If they are from the same
subsystem, do nothing. If they are from different
subsystems, adjust the dipoles and energies appropriately.
For instance, if an up dipole in A is swapped with a down
dipole from B, reduce NA↑ by one and raise NA↓ by one,
and do the opposite for NB↑ and NB↓. At the same time,
increase qA by two and decrease qB by two. Sample the
subsystems at regular iteration intervals to calculate
entropies from Eq. (11.5), where Pn is now the probability
of a dipole pointing up or down. Also calculate
temperatures and heat capacities of the subsystems.

Record the data points until equilibrium has been well

established.

Plot the results as a function of the change of energy, ΔqA.

What are the trends of the changes of entropy? Are the
 temperatures of either subsystem negative? If so, what
does negative temperature mean? Discuss what happened
to the temperature of subsystem B when its energy qB
crossed zero and became negative. Is there a singularity in
entropy, heat capacity, or temperature?

P11.4 Explore the standard and modified 1D Ising model

including varying the number and strength of interacting
neighbors.

(a) For the standard Ising model, calculate the heat

capacity and entropy as a function of temperature. Modify
Program 11.6 to compute the fluctuation of energy and
heat capacity from Eq. (11.12). For the entropy, use Eq.
(11.21). Let S1 be the entropy at T1, and δE be the
difference of energy between T1 and T2. Compute the

entropy at T2 as with .

Assume S(T = 0) = 0. Watch fluctuations in your results,

particularly in heat capacity. You may need many passes
to keep the data scatter small.

Plot both the heat capacity and entropy along with

analytical results. Compare your results with Figures
11.11 and 11.12.

(b) An improved method for calculating entropy is to

integrate the energy-temperature curve (Figure 11.10). We
already have the energy at each temperature. Use the
trapezoid rule (Section 8.A, Eq. (8.69)) to obtain S(T).
Compare with results above.

Try a different way using Eq. (11.5). Assume Pn is the

probability of a spin pointing up or down, same as in the
paramagnetic system (Exercise E11.6). Plot this result
together with the above results. Does this method work?
Why?

(c) Test a modified 4-neighbor 1D Ising model. Increase

the number of interacting neighbors to include the nearest
and the next nearest neighbors. The change of energy of a
spin flip is

The constant r is a reduction factor simulating weaker

interactions between next nearest neighbors. If r = 0, we
recover the standard Ising model.

First set r = 1. Modify the program used in part (a) to

calculate the energy, magnetization, heat capacity, and
entropy as a function of temperature between [0.1, 5] in
steps of 0.1 or less. Discuss your results and note any
differences with the standard Ising model.

Based on your observations above, predict how the results

will change if r is decreased. Run the simulation with r =
 0.5. Discuss your findings.

Figure 11.26: Animated Ising model.

(d)* Use VPython to animate the 1D Ising chain.

Represent the spins as arrows (see Program 7.9), and
choose N ∼ 50 to 100. Start with an initial microstate,
animate spin flips toward equilibrium at a given
temperature. Integrate keyboard detection in the program
to change the temperature, e.g., “up” or “down” key for
increasing or decreasing the temperature by a certain
amount. A sample scene is shown in Figure 11.26. Note
like spins tend to be clustered.

P11.5 Investigate the properties of the 2D Ising model.

(a) Write a program to simulate the Ising model on an N ×

N lattice. Use Program 11.6 or the code developed from
Project P11.4 as a template, implement the Metropolis
algorithm in Section 11.2.3 into your program. Test it and
reproduce the results shown in Figures 11.15 and 11.17.
(b) Set N = 32. Calculate the entropy of the system from
your simulation as a function of temperature. Use the
method discussed in Project P11.4 (the integral method).
Compare with analytic results (see Exercise E11.8). What
is the reason for the high numerical values near Tc?

(c) Study hysteresis of the 2D Ising model in an external

magnetic field. Let the magnetic interaction energy be
−ηsi, so the energy of the system is modified from Eq.
(11.25)

The parameter η = μB is the strength of the magnetic

interaction. Accordingly, we add an extra term, 2ηsi,j, to
ΔE in Eq. (11.26).

Assume ε = 1. Calculate the magnetization as a function of

temperature at η = ±1, , and 0. Discuss your results.

Calculate and plot the magnetization as a function of η at

several temperatures, from T = 1.5 to 2.5 in steps 0.2. For
a given temperature, plot the magnetization for increasing
η = −1 → +1 in small steps (≤ 0.02); then do the same but
reverse the field η = +1 → −1. At each η value, make 10
passes to obtain the magnetization.
 Discuss the hysteresis loops. What are the differences
below and above the critical temperature? Why does
hysteresis occur? Does it change if you vary the number of
passes between changes of η?

(d)* Explore the ferromagnetic domain on a larger system,

N = 1024 or larger. Plot the equilibrium results (after 1000
passes) similar to Figure 11.18 at T = 2, 2.2692, and 2.5
(use imshow, see Program 7.4). This will be computation
intensive, so consider using speed boost such as Program
S:11.4. Or, write your program such that you can dump
the data to a file with pickle after a number of passes,
and reload it to continue. Observe the qualitative changes
of the domain across the critical temperature. Present a
few unique plots and discuss what you think is interesting.

P11.6 Simulate the hanging tablecloth problem and find the

minimum energy by thermal relaxation.

(a) Write a program based on Program 6.8. Design a

Metropolis updater using Eq. (11.33) as the change of
energy. Convergence may be faster by sampling Δx, Δy,
and Δz individually rather than all simultaneously in each
iteration. Furthermore, because the relaxation is driven
mostly by gravity, the vertical z direction should be
sampled more frequently. Pick a particle to move,
excluding the four corner particles. Sample x and y with

probabilities each, and sample z with , illustrated below.

 dx, dy, dz = 0., 0., 0.
t = rnd.random()
if (t < .25):
dx = rnd.gauss(0., sd)
elif (t < .5):
dy = rnd.gauss(0., sd)
else:
dz = rnd.gauss(0., sd*2)

In the above code, we also double the standard deviation

in the z direction to reflect larger movements in that
direction.

Use the same parameters for k, l, and m as in Program 6.8.

Set the standard deviation .

Run the simulation. Start at T = 10−3 and a flat

configuration, let the system reach equilibrium, then
decrease T by a factor of 2, and repeat. Check the final
results against those in Section 11.3 or Section 6.8.
Increase the animation rate vp.rate to gain execution
speed. How do you check whether the system has reached
equilibrium at a given T?

(b) Disable animation, and automate the process of finding

the minimum energy for a given N × N system. Calculate
the minimum for N = 3 to 20 (9 to 400 particles). Plot the
 minimum energy per particle, excluding the particles held
fixed at the corners, as a function of temperature. Explain
the results.

P11.7 Use molecular dynamics simulations to investigate the

behavior of an interacting gas.

(a) Run Program 11.7 with different parameters to get a

feel on the program execution and speed. For example,
vary N = 40 to 100, step size h = 0.001 to 0.01, etc. Fine
tune the code for optimal performance, observing the
points raised below.

Add a function that calculates the total energy of the

system. Plot the relative error (see Project P4.1). How well
is the energy conserved?

If you run the program multiple times, it is likely that you

will observe the system collapse for some initial
conditions where the atoms too close to one another can
fly off. You can either re-run the program, or avoid it by
following the prescription given in Program 11.7.

Another issue is speed. Consider using compiled code in

Programs S:11.3 and S:11.4. Push the upper limits of N
and h while preserving accuracy, by monitoring the total

energy for instance. Since the force is negligible for r 3

(Figure 11.20), modify nbody() such that it is evaluated

only if r2 < 9 ( r2). For the tasks below, use animation
sparingly or just turn it off.

(b) Calculate the velocity and speed distributions. Write a

module that evolves the system and samples periodically
the distributions to build cumulative histograms. For
instance, the following code can be used for this purpose.

vxbin = np.zeros(nvbin, dtype=int)

speedbin = np.zeros(nsbin, dtype=int)
……
v2 = np.sum(v*v, axis=1)
speed = np.sqrt(v2)
speedbin += np.histogram(speed, nsbin, (0., smax))[0]
vxbin += np.histogram(v[:,0], nvbin, (−vmax, vmax))[0]

The variables vxbin and speedbin are integer ndarrays

for storing the cumulative histograms of x velocity and
speed, respectively. The NumPy function histogram
accepts an input array, the number of bins and the range (a
pair of numbers), and returns two arrays: the counts and
the bin edges. We only need the first. For the speed
distribution, we calculate the square of speed first using

the np.sum function along axis 1 (i.e., ) for all

particles at once, and then take the square root by element-

wise operations to obtain speed (similar to the calculation
of distances in Program 11.7).
Compute the histogram periodically (e.g., every 20 steps),
and cumulate the counts separately in time windows t = 0
to 10, 10 to 50, and 50 to 200. In the last time window,
compute the average kinetic energy and the corresponding
temperature. Plot the results similar to Figure 11.23, and
compare with the Maxwell velocity and speed
distributions.

Optionally change the initial condition to a higher

equilibrium temperature, e.g., set the range of the initial
velocities to [−2, 2]. Repeat the simulation. Beware that
you may need to adjust the step size or find a new initial
condition to maintain stable integration.

(c) Calculate pressure on one side of the cube. Track the

atoms exiting the side, find the impulse over regular time
intervals to compute the pressure as described following
Figure 11.25. Plot the pressure as a function of time, and
compare the equilibrium pressure with the ideal gas value.

(d)* Extract numerically the second virial coefficient B2 of

Eq. (11.42). To do so, vary the number of atoms N = 100
to 10000 (adjust the upper limit according to your
computing power, and consider speed boost mentioned
earlier). Note that as the particle density increases, it is
crucial to start the atoms somewhat uniformly in space to
avoid large forces destroying the system, and to use small
step sizes to keep numerical error low. Lowering the initial
speeds also helps.
 At each N, calculate the pressure as described above.
Obtain at least five data points. Plot P/nkT vs. n. The
linear slope is equal to B2. Compare B2 with theoretical
results in Exercise S:E11.2. The effect is more pronounced
at low temperatures. Change the initial condition
corresponding to half the temperature and repeat the
simulation.

1.A Boltzmann factor and entropy

11.A.1 BOLTZMANN FACTOR
Consider an isolated system consisting of a microsystem (e.g.,
an oscillator) in contact with a reservoir. To keep it simple, we
assume the microsystem is nondegenerate, i.e., there is only
one microstate for a given energy. In other words, the
multiplicity of the microsystem is 1. Suppose the microsystem
is found in state En. The multiplicity for any macrostate is 1 ×
Ωn = Ωn, where Ωn is the multiplicity of the reservoir with
energy ER = E − En, E being the total energy.

We can write Ωn according to Eq. (11.4) as

where SR(n) ≡ SR(E − En) is the entropy of the reservoir.

 The probabilities of finding the microsystem in states En
and Em are proportional to the multiplicities of the
macrostates, so their ratio is

Because ΔE = En − Em is small compared to the total energy E,

we can approximate the exponent in Eq. (11.47) by a Taylor
series,

Relating the second term to temperature from Eq. (11.7)

and ignoring the second-order correction, we simplify Eq.
(11.48) to

Substitution of this SR(m) into Eq. (11.47) gives

Equation (11.50) shows that the probability of a

microsystem with energy En is given by the Boltzmann factor,

where C is a normalization constant. It is the same as the

numerical result (11.3), of course.
 If the microsystem is degenerate, we can generalize Eq.
(11.51) to include degeneracy as Pn = Cdn exp(−En/kT), where
dn is the number of states with En.

11.A.2 ENTROPY AND PROBABILITIES

We are now interested in the entropy of the microsystem. Let S,
SR, and ST be the entropies of the microsystem, the reservoir,
and the total (isolated) system, respectively. Since entropy is
additive, we have

Let Pn be the probability that the microsystem will be found

in the state n. It is the same as the probability that the total
system will be found in the macrostate associated with n
because the microsystem is assumed to be nondegenerate.

Statistically, the entropy of the reservoir is given by the

weighted average

where we have substituted SR(n) = k ln Ωn according to Eq.

(11.4). The multiplicity of the reservoir Ωn is related to the total
multiplicity of the isolated system ΩT as Ωn = PnΩT by Eq.
(11.2). Substituting it into Eq. (11.53), we obtain
where we have used the normalization ∑n Pn = 1 to obtain the
second term. We recognize that ST = k ln ΩT is the total entropy
of the isolated system.

Substituting Eq. (11.54) into (11.52) and eliminating ST on

both sides, we obtain the entropy S of the microsystem of
interest as

This expression gives the entropy in terms of probabilities. It

can be extended to non-thermal systems such as information
analysis. In that case, the Boltzmann constant k is often set to 1.
Equation (11.55) shows directly that entropy is related to
disorder. The more possibilities, the higher the entropy. For
instance, suppose there are N possible events. If only one event
occurs with unit probability, the entropy is S = 0. But, if all
events can occur with equal probability 1/N, the entropy is S =
k ln N.

Another useful expression for entropy can be obtained if the

partition function Z (11.9) for a thermal system is known.
Putting Pn = exp(−En/kT)/Z into ln Pn in Eq. (11.55) and using
Eq. (11.10), the entropy can be rewritten as

where E is the average energy, and F is called the Helmholtz

free energy.
1.B Exact solutions of the 2D Ising model
The derivation of the exact solution of the 2D Ising model, first
worked out by Onsager, is very involved. Here, we only quote
the results [41].

Let m be a dimensionless parameter

It varies between 0 and 1 and is graphed in Figure 11.16.

The partition function can be written as

From Eq. (11.10), it can be shown that the energy per spin is
(Exercise E11.8)

and the heat capacity per spin is, from Eq. (11.11),
In Eqs. (11.59) and (11.60), the functions K(m) and E(m) are
the complete elliptic integrals of the first and second kind (Sec.
11.12 of Ref. [10]), respectively,

These integrals first appeared in Chapter 4, Eq. (4.30). They

are graphed in Figure 11.27. The function K(m) has a
logarithmic singularity at m = 1.

Figure 11.27: The complete elliptic integrals of the first and second
kind. The first kind K(m) diverges logarithmically as m → 1.

We can find the elliptic integrals as defined in Eqs. (11.61)

and (11.62) in the SciPy special function library called ellipk
and ellipe. If other sources are used, beware about the
argument m to the elliptic integrals. Some sources use m2 in
 the definitions of Eqs. (11.61) and (11.62) . In such cases, give
as the argument.

The magnetization per spin can be obtained separately as

The above results are valid for both ferromagnetic (ε > 0)

and antiferro-magnetic (ε < 0) systems. For the latter, the
magnetization (11.63) should be interpreted as the staggered
magnetization (see Project S:P11.1).

1.C Program listings and descriptions

Program listing 11.4: Equilibrium energy sharing ( equishare.py)

1 from einsteinsolid import EinsteinSolid

import matplotlib.pyplot as plt, numpy as np
3 import matplotlib.animation as am

5 def updateimg(*args): # args[0] = frame

L = 20 if args[0]<400 else 200 # slower init rate
7 solid.exchange(L)
plot. set_data(np.reshape(solid. cell, (K,K))) # update image
9 return [plot] # return line object in a list

11 K = 20 # grid dimension
solid = EinsteinSolid(N = K*K, q=10) # 10 units of energy/cell
13 fig = plt. figure ()
img = np.reshape(solid. cell, (K,K)) # shape to KxK image
 15 plot = plt.imshow(img, interpolation= ’none’, vmin=0, vmax=50)
plt. colorbar(plot)
17

anim = am.FuncAnimation(fig, updateimg, interval=1, blit=True) # animate

19 plt.show()

The process of thermal equilibrium energy sharing is

visualized on a square lattice composed of K × K interacting
oscillators.

The main program creates an Einstein-solid object with 10

units of energy per oscillator. The energy distribution is
represented by imshow() from Matplotlib. Like in Program
10.2, we use the Matplotlib animation function to update the
figure with the supplied function updateimg(). It calls the
exchange() method for a number of iterations, which randomly
exchanges energy between the oscillators as explained in
Program 11.1, then updates image data.

Program listing 11.5: Boltzmann distribution ( boltzmann.py)

1 from einsteinsolid import EinsteinSolid

import matplotlib.pyplot as plt, numpy as np
3

solid = EinsteinSolid(N=1024, q=1) # initialize solid

5 L, M, kT = 100, 6, 1./np.log(2.) # L = base iteration num.
E, bin = np.linspace (0., M, 100), np.zeros(M+1)
7 fig = plt. figure ()
for i in range(9): # 3x3 subplots
 9 ax = fig.add subplot(3, 3, i+1)
for n in range(M+1):
11 bin[n] = solid. cell.count(n) # count n
plt.step(range(len(bin)), bin/max(bin), where= ’mid’)
13 plt. plot(E, np.exp(−E/kT))
if (i == 3): plt.ylabel(’P(n)’, fontsize=14)
15 if (i == 7): plt.xlabel(’n’, fontsize=14)
if (i <= 5):
17 plt. xticks (range(M+1), [’’ for j in range(M+1)])
if (i > 0): plt. text(M−2, 0.8, repr(L*2**(i−1)))
19 solid.exchange(L*2**i) # double L each time
plt.show()

This program simulates the equilibrium distribution of an

N-oscillator Einstein solid. It uses the same exchange() method
as in Program 11.4. The average energy per oscillator is 1. This
determines the temperature according to Eq. (11.13). The main
loop plots the energy distribution as histograms between
iterations. The number of oscillators with a given amount of
energy, n, is conveniently tallied by the list count method
cell.count(n), which returns the number of times n appears in

cell. The histograms are made with the step plotter plt.step()
(see Figure S:9.1), normalized to 1 at the maximum and
centered at the midpoints. To reduce crowdedness, the x tick
labels for the first two rows are omitted.

Program listing 11.6: Ising model in 1D ( ising1d.py)

 1 import random as rnd, numpy as np
import matplotlib.pyplot as plt
3
def initialize (N): # set initial spins
5 p, spin, E, M = 0.5, [1]*N, 0., 0. # p = order para.
for i in range(1, N):
7 if (rnd.random() < p): spin[i]=−1
E = E − spin[i−1]*spin[i]
9 M = M + spin[i]
return spin, E − spin[N−1]*spin[0], M+spin[0]
11
def update(N, spin, kT, E, M): # Metropolis sampling
13 i, flip = rnd.randint(0, N−1), 0
dE = 2*spin[i]*(spin[i−1] + spin[(i+1)%N]) # periodic bc
15 if (dE < 0.0): flip=1 # flip if dE<0, else flip
else: # according to exp(−dE/kT)
17 p = np.exp(−dE/kT)
if (rnd.random() < p): flip=1
19 if (flip == 1): # accept
E = E + dE
21 M = M − 2*spin[i]
spin[i] = −spin[i]
23 return E, M

25 N, passes = 1000, 10
iter, Nmc = passes*N, passes*N
27 T, Eavg, Mavg = [], [], []
for i in range(1,41): # temperature loop
29 kT = 0.1*i # kT = reservoir temperature
spin, E, M = initialize (N)
31 for k in range(iter): # let it equilibrate
E, M = update(N, spin, kT, E, M)
33
E1, M1 = 0., 0.
35 for k in range(Nmc): # take averages
E, M = update(N, spin, kT, E, M)
37 E1, M1 = E1 + E, M1 + M
E1, M1 = E1/Nmc, M1/Nmc
39 T.append(kT), Eavg.append(E1/N), Mavg.append(M1/N)
41 plt. figure ()
plt. plot(T, Eavg, ’o’, T, −np.tanh(1./np.array(T)))
43 plt. xlabel(’kT/ε’), plt.ylabel(r’〈E〉/Nε’)
plt. figure ()
45 plt. plot(T, Mavg, ’o’)
plt. xlabel(’kT/ε’), plt.ylabel(r’〈M〉/N’)
47 plt.show()

Program 11.6 simulates the 1D Ising model using the

Metropolis algorithm (Section 11.2.2). The module
initialize() initializes the spin chain stored in a list. Plain
lists are faster to manipulate in explicit (serial) iterations than
NumPy arrays. The p-value is an order parameter that specifies
the probability of down spins. The net spin is given by 1 − 2p. A
p-value different from may be preferred in some situations
such as at low temperatures to reduce equilibration time.

The Metropolis algorithm is programmed in update(),

explained in the text. The main code steps through temperature
T. At a given T, it initializes the spin chain, waits for it to
equilibrate, and samples the system Nmc times to obtain the
average energy and magnetization. Note that the cumulative
energy and magnetization, E1 and M1, are defined to be floats to
avoid potential inaccuracies by integer division (Python 2.xx).
Though this is not an issue in Python 3.xx which defaults to
floating point divisions, it is still good practice to be precise
(Section 1.3.1). At low temperatures, the number of passes to
reach equilibrium will need to be increased for accurate
sampling afterwards (Figure 11.9). The averages are plotted at
the end.

Program listing 11.7: N-body molecular dynamics ( md.py)

1 import ode, vpm, random as rnd

import numpy as np, visual as vp
3

def nbody(id, r, v, t): # N−body MD

5 if (id == 0): # velocity
return v
7 a = np.zeros((N,3)) # acceleration
for i in range(N):
9 rij = r[i]−r[i+1:] # rij for all j>i
rij [rij > HL] −= L # periodic bc
11 rij [rij < −HL] += L
r2 = np.sum(rij*rij, axis=1) # |rij|ˆ2
13 r6 = r2*r2*r2
for k in [0,1,2]: # L−J force in x,y,z
15 fij = 12.*(1. − r6)*rij [:, k]/(r6*r6*r2)
a[i,k] += np.sum(fij)
17 a[i+1:,k] −= fij # 3rd law
return a

19

L, N = 10.0, 32 # cube size, num. atoms

21 atoms, HL, t, h = [], L/2., 0., 0.002
r, v = np.zeros((N,3)), np.zeros((N,3))
23

scene = vp.display(background=(.2,.5,1), center=(L/2, L/3, L/2))

25 vp.box(pos=(HL,HL,HL), length=L, height=L, width=L, opacity=0.3)
for i in range(N): # initial pos, vel
 27 for k in range(3):
r[i,k] = L*rnd.random()
29 v[i,k] = 1−2*rnd.random()
atoms.append(vp.sphere(pos=r[i], radius=0.04*L, color=(1,0,1)))
31 v −= np.sum(v, axis=0)/N # center of mass frame

33 while (1):
vpm.wait(scene), vp.rate(1000)
35 r, v = ode.leapfrog(nbody, r, v, t, h)
r [r > L] −= L # periodic bc
37 r [r < 0.] += L
for i in range(N): atoms[i]. pos = r[i] # move atoms

We can simulate full 3D molecular dynamics with Program

11.7. The N-body dynamics is modeled in nbody(), which is
written in the required format for leapfrog integration and is
explained in Section 11.4. The main code initializes the
parameters such as the cube size and number of atoms, etc. The
position and velocity vectors are stored in the variables r and v
which are created as N × 3 arrays. After randomly initializing
positions within the cube and velocity components between
[−1, 1], the center of mass velocity is computed by
summing over the three components of all atoms separately,
via np.sum along the first axis, i.e., the particle index. To
eliminate the net drift caused by the center of mass motion, we
subtract the average from the velocities. Here (line 31),
is broadcast into an N × 3 array before element-wise
operations are carried out.
 Animation elements include a transparent cube and a list of
atoms, atoms. In the infinite loop, the leapfrog integrator is
called to evolve the N-body system. After each step, the atoms
are checked whether they have escaped the cube. We do so with
the same truth array indexing as in nbody(), so the position in
any direction outside the cube (e.g., x < 0 or x > L) is mapped
inside by periodic conditions. Finally, the atoms list is updated
to move the atoms.

Because the initial positions are randomly selected, there

are situations when atoms are placed too close to each other.
This can cause a very large force due to strong repulsion 1/r13
of the Lennard-Jones potential. Unless the time step h is very
small, numerical error can destroy the system, often ejecting
the atoms into a distant oblivion. If this happens, just re-run
the simulation. Note that this rarely happens once the
simulation starts normally. To guard against this situation, we
can require a minimum initial distance between the atoms,
seed the random number generator for a stable initial
condition (Section 10.1, Program S:11.4), or use the N-body
leapfrog method with time transformation (see Program
S:12.1).

Our code is fairly optimized and efficient. It is adequate for

N-body simulations up to perhaps N 1000. For larger
simulations, it would be too slow, with most time spent
computing the forces. In this case, we can use compiled codes
for nbody() in Fortran via F2Py, or in C/C++ via Cython or
Weave (see Section 1.3.2). We describe an F2Py modification in
Programs S:11.3 and S:11.4, which show a dramatic increase in
speed by a factor of about 100.

1
The amount of memory in all the computers in the world would not be nearly enough,
short by a few orders of magnitude, to just store the positions of the particles in a can of air.

2
We cannot prove it, but there are no other more plausible alternatives.

3
A Python list is faster than an ndarray for explicit looping (see Section 2.3.2).

4
Numerically calculating derivatives of a fluctuating function is worse than undesirable
and should be attempted only for educational purposes.

5
A more serious technical barrier is the period of the pseudo-random number generator,
19937
which is 2 as currently implemented in Python. It means that we could simulate a
4
system of no more than 10 spins by brute force.

6
It is as if Richard Feynman was referring to molecular dynamics when he wrote [29]
“what statement would contain the most information in the fewest words? I believe it is the
atomic hypothesis that all things are made of atoms − little particles that move around in
perpetual motion, attracting each other when they are a little distance apart, but repelling
upon being squeezed into one another”. feynmanlectures.caltech.edu/I_01.html

7
Naturally, the Lennard-Jones potential is also given the moniker of “6−12” potential.
Chapter 12
Classical and quantum
scattering
Most problems we have discussed so far, from harmonic
oscillations to planet motion, are bound systems. There are
unbound systems in our physical world, such as light scattering
from water droplets making rainbows, as well as proton-proton
collisions leading to the recent discovery of Higgs boson in the
world's largest atom smasher. These are scattering problems.
To study them, we need a fundamentally new approach to
account for the fact that particles come in from infinity and go
out to infinity.

Much of what we know about the fundamental forces (e.g.,

Coulomb, strong, and weak) comes from particle scattering
studies. In a typical scattering setup, a beam of particles
(projectiles), prepared in well-defined initial states, is sent to a
target region for collisions with the constituents. After the
beam passes through, the outgoing particles will be scattered
relative to their incoming direction, possibly accompanied by
other fragments or secondary emissions. By detecting the
 scattered particles, on the one hand, we can infer the
underlying forces that caused the deflection or scattering. This
was how the atomic nucleus was discovered in Rutherford's
alpha-particle and gold-foil experiment. On the other hand, if
the interaction forces are presumed to be well known,
scattering studies can test and improve our understanding of
collision dynamics by comparing experimental observations
with theoretical models and predictions. The study of
scattering is vital not only to understanding the fundamental
interactions, but also to many practical applications, including
radiation and transport physics (Section S:11.2), fusion energy
research, and the scattering and absorption of radiation by the
atmosphere, etc.

We start by introducing the concept of scattering cross

sections and numerical determination of deflection functions.
After studying general potential scattering, we present classical
analysis of rainbow and glory scattering. Quantum
mechanically, we define the scattering amplitude and relate it
to the cross section in analogy to scattering of waves. Then, the
scattering (continuum) wave functions and amplitudes are
calculated in terms of phase shifts from partial wave analysis.

2.1 Scattering and cross sections

In Chapter 4 we discussed bound (planetary) motion in central
fields for negative energies E < 0. Since scattering is a natural
extension from bound to unbound motion for positive energies
(4.11), we begin our study of classical scattering from central
field motion for E > 0.

12.1.1 DEFLECTION FUNCTION AND

SCATTERING ANGLE
Figure 12.1 (top) illustrates the schematics of the scattering
process. The incoming particle comes in from far left (infinity),
where the force is negligible, with a given kinetic energy E and
velocity toward the target (scattering center). Taking the
incident direction to be horizontal, the initial vertical distance b
from the target is called the impact parameter. The impact
parameter may not be available experimentally, but we can
specify it theoretically. After passing through the target region,
the outgoing particle is scattered to a deflection angle Θ at
infinity where the force again becomes negligible.

Each value of b produces a well-defined deflection angle Θ.

But in general we cannot say that a given Θ corresponds to a
unique b. There are cases where different values of b lead to the
same Θ as we will see shortly. The goal of scattering studies is
to determine the relationship between the deflection angle Θ(b)
and the impact parameter b. The function Θ(b) is called the
deflection function.
 Figure 12.1: Schematics of particle scattering from a central field
(top). All particles entering the shaded ring of cross section dσ = 2πb
db are scattered to the angle between θ and θ + dθ (bottom).

Given a collision energy E, we can find the incident speed v

and angular momentum L as

Because the angular momentum L depends on b, a typical

scattering experiment samples outcomes of different angular
momenta even though the energy is fixed. In other words,
angular momentum may be conserved in each individual
collision, but the overall outcome has contributions from
different angular momenta.
 The deflection angle Θ for a given b is defined as the angle
of rotation of the velocity vector from the incident direction to
the final outgoing direction (Figure 12.1, top). To find Θ, it is
easier to work with the polar angle of the radial vector.
Consider the radial bisector as the straight line from the target
to the point of closest approach (r = rmin) bisecting the
trajectory. Let α be the angle swept by the radial vector from
the radial bisector to infinity (r = ∞). From symmetry, this is
also equal to the angle swept by the radial vector in the
incoming direction. Accordingly, the deflection angle can be
written as

The derivation for Θ and α is given in Section 12.A, and the

result is

where V(r) is the interaction potential. The parameter rmin is

the distance of closest approach where the denominator in Eq.
(12.3) is zero, defined in Eq. (12.48). For certain potentials, the
deflection function can be calculated analytically [40]. A well-
known example is Rutherford scattering with the Coulomb
potential (12.52). For most potentials, the integral in Eq. (12.3)
can only be evaluated numerically.
 We make a distinction between the deflection angle Θ and
the scattering angle θ. The scattering angle θ is the physically
observed angle after the particle leaves the scattering region,
and it is restricted to 0 ≤ θ ≤ π. On the other hand, the
deflection angle Θ can be positive or negative (attractive
potentials), and is not limited to a fixed range. For instance, in
some attractive potentials, the particle could be orbiting quite a
few times before leaving the interaction region, resulting in a
deflection angle exceeding −nπ, n being a positive integer.
However, the deflection angle is not directly observable. It is
related to the observable scattering angle through

For negative Θ, Eq. (12.4) maps it to the observable θ range

and accounts for the possibility of orbiting (Θ < −2π, Figure
12.8).

12.1.2 CROSS SECTIONS

To measure the scattering events, we imagine placing particle
detectors far outside the target region. The outcomes as
measured experimentally consist of the number of particles
(dN) recorded by the detector subtending a solid angle, dΩ.
That number is proportional to the beam flux I, defined as the
number of incident particles per unit area. It is useful to
introduce a quantity independent of I as
The value σ(Ω) gives the likelihood of a particle being scattered
to the angle θ, but has the dimensions of an area. Therefore,
σ(Ω) is called the differential scattering cross section, or simply
the cross section. It is the single, most useful quantity in
scattering.

As Figure 12.1 (bottom) shows, all particles (dN) entering

the ring in the impact parameter range b to b + db are scattered
to the angle between θ and θ + dθ. The area of the ring is dσ =
2πb db, so dN = Idσ = I2πb db. Because scattering in central
fields is independent of the azimuthal angle φ, the solid angle
element is dΩ = 2π sin θdθ, and σ(Ω) = σ(θ). Substituting dN
and dΩ into Eq. (12.5), we obtain

We take the absolute value of the derivative |db/dθ| in Eq.

(12.6) to ensure that σ(θ) is positive. The classical
interpretation of dσ is that of the cross-sectional area around
the scattering center (Figure 12.1, bottom). Note that the flux I
drops out of σ(θ).

The cross section σ(θ) can be measured experimentally

(12.5) and calculated theoretically (12.6). Since the latter
requires the interaction force as input, comparison between
experiment and theory can reveal the nature of that interaction.
 Theoretically, because Θ is not necessarily bounded or
single-valued, it is possible that different deflection angles
contribute to the same scattering angle θ (see Eq. (12.4)). To be
general, we can extend the definition of the cross section (12.6)
to include this possibility as

where the sum is over all values of Θ and b that yield the same
observable scattering angle θ. We shall see an example of this
later on.

The total cross section, σt, can be obtained from Eq. (12.6)
by

In classical mechanics, the total elastic cross section is infinite

if the potential has an infinite range (long-ranged), no matter
how small or how rapidly the force decreases at large distances.
This is because there is always a small deflection at any large
but finite b (at least theoretically). In quantum mechanics,
however, the total cross section is finite if the potential
decreases faster than 1/r (see Section 12.4.5).

12.1.3 RUTHERFORD SCATTERING

Rutherford scattering is an example of scattering by long-
ranged potentials, the Coulomb interaction in this case, V =
k/r. The trajectories of scattering motion are described by Eq.
(12.50) (Section 12.A). They are plotted in Figure 12.2.

Figure 12.2: Scattering from repulsive (top) and attractive (bottom)

potentials.

For repulsive potentials, the trajectories curve to the same

side of the incoming particle, and for attractive potentials they
curve to the opposite side. Accordingly, the deflection angles
have the same sign as the potential. In either case, the
scattering angle is the absolute value of the deflection angle θ =
|Θ| (no orbiting). At small impact parameters, the scattering
angle approaches π due to the singularity at the origin
(hardcore potentials). For repulsive potentials, the particle is
pushed back, and for attractive potentials, it is turned around
the target. As the impact parameter increases, the scattering
angle decreases as expected. We can see graphically (Figure
12.2) that the scattering angle for a given impact parameter is
the same regardless of the sign of the Coulomb potential.1

Figure 12.3: Universal deflection function (left) and cross section

(right) for Rutherford scattering.

The deflection function is given in Eqs. (12.52) and (12.54).

Differentiating db/dΘ from Eq. (12.54), we obtain the cross
section (12.7) as

This is the celebrated Rutherford cross section. It is the same

for repulsive or attractive potentials because of the k2.
Moreover, it is also identical in both classical and quantum
mechanics.

We plot the deflection function and cross section in Figure

12.3, displaying both as a function of the dimensionless
variable η = 2Eb/k. Consequently, the plots are universal. For
 instance, if energy E is doubled, the impact parameter b should
be halved (or k doubled) to obtain the same value for Θ or σ.
The net effect is to shift the curves to the left by the same
factor. For small values of η → 0 (low energy, small impact
parameter, or strong potential), the deflection angle is near π.
In the limit of large η 1, it approaches zero.

The cross section (Figure 12.3) is scaled by σ0 = k2/16E2, the

value at η = 0. Because it is plotted in terms of η rather than
the conventional scattering angle θ, the Rutherford cross
section can be displayed as a universal curve. We can relate η
and θ graphically via the deflection function (Figure 12.3, left),
or analytically via Eq. (12.54). We use semilog and log-log plots
to show the details over the full scales of η and σ.

The Rutherford cross section diverges at large impact

parameters as b4, or equivalently as θ−4 at small angles. This
gives rise to an infinite total cross section. At small impact
parameters (large scattering angle ~ π), the cross section is
finite. In the original Rutherford experiment, alpha particles
were sent to gold foils. From the number of back-scattered
alpha particles, Rutherford concluded that there had to be a
point-like charge at the scattering center, the nucleus. Without
a hardcore potential (singularity), large angle scattering would
not be possible, as evidenced below.

2.2 Rainbow and glory scattering

Unlike the Coulomb potential, for the majority of potentials
encountered in actual scattering, the integral in Eq. (12.3)
cannot be carried out analytically or elegantly. It is easier to
obtain the deflection function by numerical integration. Below
we discuss two examples, rainbow and glory scattering from
softcore potentials.

12.2.1 RAINBOW SCATTERING

Consider the plum-pudding potential given by Eq. (S:7.6)
(Project S:P7.2),

Figure 12.4: The repulsive plum-pudding potential (left) and force

(right). The radius of the sphere is 1 (dotted line).

It describes the potential between an electron and a uniformly

charged sphere of charge Z and radius R (in a.u., which is
implicitly assumed thereafter). The “plum” potential is a
softcore potential since the force is finite (zero in this case) at
the center. Outside the sphere, it is the normal Coulomb
potential (Figure 12.4). The force is linear in r inside the sphere
and 1/r2 outside. The potential is attractive when the nuclear
charge Z is positive, and repulsive if Z is negative.

The deflection functions for the repulsive plum-pudding

potential (Z = −1 and R = 1) are shown in Figure 12.5. They are
computed from Program 12.1 which calculates the integral
(12.49) in u = 1/r space. At the lower energy E = 1.2, the
scattering angle spans the full range [0, π]. The particle does
not have sufficient energy to penetrate the repulsive core at
small impact parameters, and is deflected backward including
θ = π (Figure 12.6, top).

At the higher energy E = 2, the particle is energetic enough

to move straight through the softcore potential, and the
scattering angle is zero at b = 0 (Figure 12.6, bottom). This
cannot happen if the potential is singular as discussed earlier
regarding Rutherford scattering (Figure 12.2). With increasing
b, the scattering angle increases at first due to the rising
repulsive force (Figure 12.4). As b further increases, the effect
of the stronger force is neutralized by the reduced time spent in
the charged sphere, indicated by the shrinking portion of
trajectories inside the circle in Figure 12.6 (bottom).
Consequently, the scattering angle reaches a maximum value
near b ~ 0.4, and decreases thereafter. The maximum
scattering angle, ~ π/4 in this case, is called the rainbow angle.
For a given angle below the rainbow angle, there are two
impact parameters contributing to the cross section, indicated
by two crossings on the deflection function (Figure 12.5, dotted
line).

Figure 12.5: Deflection functions of the plum potential at two

energies. The dotted line shows that there is only one impact
parameter contributing at E = 1.2 but there are two at E = 2.

At large impact parameters, both deflection functions

behave like Rutherford scattering because the potential is
Coulombic outside the sphere.

Computing cross sections

Knowing the deflection functions, we can obtain the cross
sections that are directly observable experimentally from Eq.
(12.7). A direct but less accurate method is to compute the
numerical derivative dΘ/db of the deflection function from Eq.
(2.13) as
Equation (12.11) is useful if Θ is known at discrete impact
parameters b.

To obtain the cross section at arbitrary b values, the

derivative dΘ/db = (db/dΘ)−1 would need to be evaluated. It
can be obtained from Eq. (12.49) as (Exercise E12.2)

Figure 12.6: Scattering from the plum potential at two energies. The
circle represents the size of the plum sphere.

where f = 1 − V (1/u)/E − b2u2, and f′ and f″ are the first and

second derivatives with respect to u. Equation (12.12) can be
accurately calculated, and has the advantage of being on equal
footing as the deflection function (12.3) (see Project P12.4).
Finally, we can also obtain the cross section directly by virtual
experiments (see Section S:12.B, Project S:P12.2).

Below we will use the direct method (12.11) for the results
(Program 12.1). Figure 12.7 shows the cross sections for the
plum potential. At the lower energy (Figure 12.7, left), the cross
section behaves like Rutherford scattering from small angles to
just below π/2, where a small plateau develops. The energy is
such that the particle can enter the outer layers of the core, but
is eventually deflected because the energy is still below the
peak of the potential. We can see a small shoulder in the
deflection function responsible for this effect (a slight change in
curvature, barely visible in Figure 12.5 near ). Beyond
the plateau, large scattering angles are predominantly due to
scattering at the core.

At the higher energy (Figure 12.7, right), the cross section is

nonzero in a limited range only, from zero up to the rainbow
angle. At every angle, there are two contributing impact
parameters. They converge to the rainbow angle where the
cross section is infinite. It is caused by the singularity in the
derivative db/dΘ = ∞ at the rainbow angle (Figure 12.5). This is
known as rainbow scattering (see Exercise E12.3). The
terminology comes from the analogy to the optical
phenomenon of rainbows formed by light rays incident on
water droplets.2 The primary rainbow, observed at the angle ~
42° with the back against the incident light, is due to a
reflection between two refractions inside the droplet (Figure
12.20). This angle occurs at an impact parameter just shy of the
radius of the droplet, gathering light from a ring and giving rise
to enhanced light intensity, the rainbow [8, 69]. The
enhancement of cross sections by rainbow scattering is also
present in quantum mechanics (Figure 12.19), and has been
observed experimentally.

Figure 12.7: The cross sections for the plum potential at two
energies. The cross section diverges at the rainbow angle (E = 2,
right).

12.2.2 GLORY SCATTERING

For attractive potentials, the deflection angle is negative. As
alluded to earlier, in some cases the particle could be deflected
by more than −π. We discuss a representative case involving
the screened Coulomb potential (Yukawa),
where a is the screening length, and Z the nuclear charge as
usual. The potential is widely used to model ion-atom
interactions (Section S:11.2.3). It is a short-range potential
because of the exponential reduction relative to the unscreened
Coulomb potential which is long-ranged. As a result, scattering
at intermediate energies or above is predominantly at forward
angles. At low energies, however, several interesting behaviors
emerge. We discuss this case below. The results are obtained by
modifying Program 12.1 for the Yukawa potential. Calculations
are left to Project P12.4.

Figure 12.8: The deflection (left) and scattering (right) angles for the
Yukawa potential (Z = 2, a = 1) at E = 0.1. Five impact parameters
contribute to the scattering angle , marked as (•) on the curves.

Figure 12.8 shows the deflection and the scattering angles

as a function of impact parameters at energy E = 0.1. Starting
from large b when the deflection angle is zero, the value of Θ
steadily decreases (more negative) for decreasing b, until the
onset of a precipitous drop around b ~ 3.5. The deflection angle
reaches −π at b ~ 3, crosses it and falls rapidly past to the
lowest point Θ ~ −2.7π, giving rise to a rainbow. It rebounds
back to −π at b ~ 0 (hardcore potential) from below.

Figure 12.9: The cross sections for the Yukawa potential with the
same parameters as in Figure 12.8.

When Θ crosses −π, it produces an infinite cross section

(12.6) due to sin θ → 0 but db/dΘ being finite. The is known as
the glory scattering.3 The infinite cross section is due to the
vanishing solid-angle element dΩ. Physically, it means
channeling a finite flux into an infinitely small area, so the
cross section is greatly enhanced.

Immediately following the glory, a rainbow occurs as the

deflection angle has reached the minimum. This is not the
minimum for the scattering angle θ, however. The latter can be
obtained from Eq. (12.4), shown side by side in Figure 12.8.
The sharp tips are the result of mapping Θ into the range [0, π].
We can see that the deflection and the scattering angles can be
 quite different. For instance, for the scattering angle ,
there are three Θ values contributing, , , and . In
terms of the impact parameter, there are five b values
contributing to the cross section for all scattering angles θ
below the rainbow angle (~ 0.7π). The five impact parameters
for are marked by the crossings in Figure 12.8. Between
the rainbow angle and π, we find three contributing impact
parameters, one from small b, and two from both sides of the
glory.

The cross section shown in Figure 12.9 is dominated by

small-angle scattering, the same as in Rutherford scattering.
The smooth background is punctuated by two singularities, the
rainbow and the glory, which theoretically are infinite above
the background. The cross section drops to a lower background
value immediately past the rainbow angle, because the number
of contributing impact parameters has decreased from five to
three as discussed above. Because of the glory, the cross section
does not remain finite as θ approaches π as would be expected
otherwise for hardcore potentials.

Like the rainbow, the glory scattering has also been

observed in electron-atom scattering experiments and
predicted by quantum scattering discussed next.

2.3 Quantum scattering amplitude

Classical scattering discussed above breaks down at the atomic
scale. To correctly describe scattering phenomena of atomic
particles, we again turn to quantum mechanics as we did
previously (Chapters 8 and 9). Full quantum scattering theory
is itself a rich and expansive topic [51]. We will only briefly
summarize the key elements of this theory within a limited
scope for our purpose. We use atomic units (Table 8.1) below
unless otherwise noted.

Figure 12.10: Schematics of quantum scattering from a central field.

The basic premise in quantum scattering is the same as

classical scattering of waves such as sound waves. There are
differences, of course, that in quantum mechanics, we are
dealing with matter waves that obey the Schrödinger equation
(8.2) instead of classical wave equations (6.62). Nonetheless,
we can freely borrow ideas from classical wave scattering to
construct a qualitatively correct picture of quantum scattering.

Figure 12.10 shows the schematics of wave scattering from a

potential field. The incident wave is dispersed into scattered
waves upon encountering the scattering center. We can write
the total wave function as a superposition of the incident wave
ψin and the scattered wave ψsc,

We take the incident wave as an idealized plane wave, ψin = eikz

(Figure 8.1). Far from the scattering center, the scattered wave
is an outgoing spherical wave, its magnitude modulated by an
angle-dependent factor f(θ). We need to find the scattering
solutions satisfying the boundary condition of outgoing
spherical waves,

where is the wave vector, E > 0 the energy, and m

the mass of the particle. The factor f(θ) is called the scattering
amplitude. Equation (12.15) makes it clear that the amplitude
f(θ) has the dimension of length. In contrast to bound states,
scattering wave functions (12.15) for continuum states do not
vanish at infinity.

The amplitude of the incident plane wave is chosen to be

one, which sets the overall particle flux and does not affect the
cross sections. The incident flux is (Exercise E8.5)
 Similarly, we can find the outgoing radial flux, far from the
scattering center r → ∞, from the asymptotic spherical waves
(12.15) to be

Suppose a particle detector is placed at r and θ outside the

interaction region (Figure 12.1). Let ds and dΩ be the surface
area and the solid angle element subtended by the detector,
respectively. They are related by ds = r2dΩ. The number of
particles going into the detector is

We have used Eq. (12.17) to eliminate Ir in Eq. (12.18).

Substituting dN into Eq. (12.5), the product IdΩ cancels, and
we obtain the scattering cross section

The calculation of cross sections in quantum scattering

boils down to finding the scattering amplitude f(θ), which is to
be extracted from the scattering wave function ψ in Eq. (12.14)
(see also Eq. (S:12.10)). In essence, the scattering amplitude is
to quantum scattering as the deflection function is to classical
scattering.

2.4 Partial waves

The partial wave method is an effective way to calculate the
scattering amplitude f(θ) just defined. In this method, we
decompose the incident waves into components, or partial
waves, of fixed angular momenta. This is in exact analogy to
the analysis of classical scattering in terms of the impact
parameter. The commonality exists because the angular
momentum is conserved in central field potentials. In quantum
scattering, each partial wave can be treated separately to find
the phase shift, which determines the scattering amplitudes
and cross sections.

12.4.1 PHASE SHIFT

Before discussing scattering in 3D space, let us introduce phase
shifts in scattering from a 1D potential with a hard wall at x =
0, V (0) = ∞. Figure 12.11 (top) shows a square well potential,
but it can be any short-ranged potential decaying faster than
x−1−∈, ∈ > 0, as x → ∞.4
 Figure 12.11: Scattering in 1D (top) and the phase shifts (bottom, δ >
0).

We assume an incident plane wave e−ikx traveling from right

to left. The scattered wave can be quite complex in the
interaction region, but in the asymptotic region where the
potential vanishes, it emerges again as a plane wave, moving to
the right and phase-shifted by 2δ, −e2iδeikx. According to Eq.
(12.14), we have the scattering wave function in the asymptotic
region as

The negative sign in the scattered wave comes from

collision with the hard wall, similar to inverted waves on a
string (Figure 6.16, Section 6.5.5). The magnitude of the phase
factor is one, |e2iδ| = 1, because the incident flux must be equal
to the outgoing flux. Therefore, the phase shift δ is real. The
factor of two is included for convenience to account for the fact
that

We can now interpret the phase shift in real sine waves. Figure
12.11 (bottom) shows two cases, a positive phase shift
corresponding to a sine wave pulled back, and a negative one to
a sine wave pushed forward (see Figure 12.17 for an actual
case). Attractive potentials produce positive phase shifts, and
repulsive ones produce negative phase shifts. Another
explanation for the factor of two is that the wave is shifted once
on the way in and once on the way out.

Regardless, the phase shift provides information on the

potential. It is analogous to probing a dark well by sending
waves into it, and inferring the structure of the well from the
phase shift of the returned waves.

Figure 12.12: Phase shift from the delta-wall potential (Figure 9.21).
 Figure 12.12 shows an example of phase shifts from the
delta-wall potential, a delta potential −αδ(x − a/2) placed to
the right of a hard wall (Figure 9.21). We can find the phase
shift analytically (Exercise E12.6),

The results in Figure 12.12 confirm that the phase shift is

positive for attractive potentials (α > 0) and negative for
repulsive potentials (α < 0). The phase shifts in this example
vanish exactly whenever ka/2 = nπ, n being an integer. This is
the condition when standing waves are formed between the
wall and the delta potential. When this happens, the scattered
waves are not shifted at all (other than being reflected). This is
an extreme case of resonance.

At low energies, Figure 12.12 shows that the phase shift

approaches zero for the repulsive potential. But for the
attractive potential, it approaches π. The latter is due to an
interesting but general property known as Levinson's theorem
[60], which states that the phase shift at k = 0 is equal to

where nb is the number of bound states supported by the

potential.
 The parameters used in Figure 12.12 are such that there is
one bound state (Exercise E9.7). To ensure the correct
calculation of the phase shift, we require that δ is continuous
and δ → 0 as k → ∞ (usually). In plotting Figure 12.12, we have
made corrections to account for the phase shift jumping by π
due to the multi-valuedness of the inverse tangent function.
Only then does δ(k = 0) approach the correct limit of
Levinson's theorem (12.23). Further explorations can be found
in Exercise E12.6.

12.4.2 PARTIAL WAVE EXPANSION

The above concept of phase shifts in 1D is equally applicable to
3D. We just need to separate the radial and angular variables in
the 3D Schrödinger equation as ψ = R(r)P(θ) (Section 9.5), so
we obtain an effective 1D radial equation. Furthermore, the
angular solution is given by Pl(cos θ), the well-known Legendre
polynomials (Section 8.A.3), independent of the potential (so
long it is central). The order parameter l corresponds to a
discrete angular momentum state. As such, any wave function
can be written as a superposition ∑ AlRlPl.

For instance, the incident plane wave eikz, a free-particle

solution (V = 0, see Eq. (S:12.2)), can be written as [4]
where jl(kr) is the spherical Bessel function of the first kind
(see Section 12.B). For nonzero potentials, we seek solutions of
the form [80]

The radial wave function Rl(k, r) satisfies the radial

Schrödinger equation (9.38) for scattering states E > 0,

The free-particle radial wave functions, jl(kr), are solutions

to Eq. (12.26) for V = 0 (see Eq. (12.55)). They have the
asymptotic behavior [4]

i(kr−lπ/2)
Like the 1D case (12.20), we recognize e /r in Eq.
(12.27) as the incoming and outgoing spherical waves,
respectively. This fact readily lends itself to the interpretation
that the incoming wave is scattered into an outgoing wave with
zero phase shifts because V = 0.

When the interactions are not zero, we expect the

asymptotic behavior of Rl(k, r) to be (see Section 12.B, Eqs.
(12.66a) to (12.66c))
in full analogy to Eqs. (12.20) and (12.21). In Eq. (12.28a),
nl(kr) is the spherical Bessel function of the second kind.
Comparing Eq. (12.28b) with (12.27), we conclude that the
outgoing wave is phase-shifted by 2δl, yet to be determined.

Substituting Eqs. (12.24) and (12.25) into (12.14), we can

express the scattered wave as

With Eqs. (12.27) and (12.28b), we can obtain the asymptotic

behavior of the scattered wave,

where the incoming waves cancel out.

Equation (12.30) is a pure outgoing wave, . We

identify f(θ) as the scattering amplitude from Eq. (12.15),
where we have used e−ilπ/2 = (−i)l. The scattering amplitude is
complex, so the cross section is

We can obtain the total cross section by integrating Eq. (12.32)

(Exercise S:E12.5)

12.4.3 EXTRACTION OF PHASE SHIFTS

The above discussion shows that the central quantity is the
phase shift δl. Knowing the asymptotic form of Rl(k, r), we can
extract the phase shift from Eqs. (12.28a) to (12.28c).
Numerically, it is more accurate to use Eq. (12.28a) in the
potential free region where kr is just large enough that the
solutions are well described by a combination of jl(kr) and
nl(kr), but not so large that the asymptotic forms (12.27) and
(12.60) are sufficiently satisfied.

Suppose the matching radius is r = a, beyond which the

potential is negligible. Differentiating Eq. (12.28a) on both
sides with respect to r, we obtain the logarithmic condition
where and are derivatives with respect to x, evaluated
at x = ka. Rearranging Eq. (12.34), we obtain the phase shift as

The phase shift δl depends on l and k, but should be

independent of the matching radius a. The latter can be a
useful numerical check.

Like bound states in central field potentials (Section 9.5, Eq.

(9.39)), it is easier to work with ul(k, r) = rRl(k, r), satisfying

In terms of ul, we can express the phase shift as

Given the logarithmic derivatives of the radial wave functions β

or γ, analytically or numerically, phase shifts can be calculated
from Eqs. (12.35) or (12.37). They can also be obtained from
the phase shift integral (S:12.85) (see Section S:12.A).

12.4.4 SCATTERING BY A HARD SPHERE

As an example, consider scattering from a hard sphere,
The wave function vanishes inside the sphere at r ≤ a. Outside
the sphere r > a, the radial wave function is a solution to the
free-particle radial equation (12.55)

We can obtain the phase shifts by the continuity condition,

Rl(k, a) = 0. It leads to the analytic result

The same result can be derived from Eq. (12.35) by setting β →

∞.

Figure 12.13: Phase shifts for scattering from a hard sphere.

 Figure 12.13 shows the phase shift as a function of wave
vector (ka) for the first several partial waves l = 0 to 4 (this and
the following results can be obtained as described in Project
P12.5). The values of jl(ka) and nl(ka) are obtained from SciPy
special functions (see Program 12.2 for examples). All phase
shifts start from zero at ka = 0, and are negative (repulsive
potential). At low energies, the predominant partial wave is the
s-wave (l = 0), the lowest angular momentum state. As energy
increases, higher partial waves start to contribute more to
scattering.

The phase shifts at large energies ka 1 approach δl = lπ/2

− ka, linearly decreasing with ka. This behavior is atypical, as
we expect the phase shift to approach zero as ka → ∞ for finite
potentials. For ideal hard spheres, however, no matter how
high the energy is, a particle will never penetrate the sphere.
The wave function remains zero at r = a (Figure 12.14) while
the wavelength decreases, causing the magnitude of the phase
shifts to increase without bound.

The wave functions squared are shown in Figure 12.14 for

two energies ka = 0.01 and 2 in the left and right columns,
respectively. They are computed from Eq. (12.39). The incident
wave ψin moves horizontally from left to right. At each energy,
the total wave function (|rψ|2, Eq. (12.25)) is shown above the
scattered wave function (|rψsc|2, Eq. (12.29)). We plot |rψ|2
instead of |ψ|2 to emphasize the details in the asymptotic
region. On the sphere, the full wave function vanishes as
required, and the scattered wave function is maximum.
Because the incident waves are not zero on the sphere, the
scattered waves must be properly phase-shifted such that they
cancel each other, ψsc = −ψin at r = a.

2 2
Figure 12.14: The total and scattered waves, |rψ| and |rψsc| , shown
in the top and the bottom rows, respectively, for scattering from a
hard sphere at two energies, ka = 0.01 (left column) and 2 (right
column).

At the lower energy, both the total and the scattered waves
are nearly independent of directions, showing a high degree of
isotropy. Here, the de Broglie wavelength is long compared to
the sphere or any structure of the potential, so the scattering
amplitude f(θ) has little angular dependence.

At the higher energy, the total wave peaks right in front of

the sphere, and there is a dark shadow immediately behind.
The overall shape illustrates the scattering of plane waves by a
hard sphere, resembling sheets of paper bending under stress.
There must be strong destructive interference between the
incident and the scattered waves to create the shadow. Indeed,
the scattered waves are strongest in the shadow behind the
sphere, and are predominantly focused in the forward
direction. Because of this, we expect that the scattering
amplitude f(θ) must have a strong anisotropy. We also expect
the focus to be increasingly narrower at higher energies.

Figure 12.15: The differential (left) and total cross sections (right) for
scattering from a hard sphere. The dotted line indicates the classical
cross section.

The cross sections, both differential and total, are shown in

Figure 12.15. At the lowest energy (ka = 0.01), the differential
cross section is flat, i.e., isotropic. As we have seen from Figure
12.13, low-energy scattering is dominated by the s-wave, and
hence is non-directional because P0(cos θ) = 1 in Eq. (12.32).
This is consistent with the isotropy in the wave functions
(Figure 12.14).

The differential cross section peaks at increasingly forward

angles as energy increases, over a smooth background
extending to the backward scattering angles up to 180°. The
background value happens to coincide with the classical
differential cross section, which remains constant over the
entire angular range, irrespective of energy (Exercise E12.4).
This is no coincidence, of course. At higher energies, or
equivalently at large momentum transfers, the effective de
Broglie wavelength becomes very small. The waves behave
more like geometric rays scattering in the same way classical
particles reflect off a surface. At smaller forward angles,
interference effects, absent from classical particle scattering,
are important.

It is interesting to note that the total cross section (Figure

12.15) is maximum at ka → 0, 4πa2. This is equal to the surface
area of the sphere, four times the geometric cross-sectional
area. It is consistent with the differential cross section being
isotropic at σ(θ) = a2 for ka ~ 0. With increasing energy, the
total cross section decreases steadily, approaching a limit σt →
2πa2 as ka → ∞, though the convergence to this limit is slow.
The number of partial waves contributing to the total cross
section increases for large ka. Numerically, we included l = 0 to
40 to obtain convergence at ka = 20. Even though the phase
shifts (Figure 12.13) grow without bound, the cross section
remains well-defined.

The high-energy limit of σt = 2πa2 agrees with classical

optical scattering. It is interpreted as reflection plus shadow
effects. All rays are reflected, creating a dark shadow right
behind the sphere. However, the space far behind the sphere is
still filled with waves (Figure 12.14), so this can be construed as
waves scattered from behind the sphere. If detectors are placed
away from the sphere, they will detect the actual reflections
(background) in front of the sphere, and the scattered waves
behind the sphere (shadow effect). The two effects account for
the total cross section equal to twice the geometric cross-
sectional area.

This hard-sphere model lets us extract phase shifts without

having to solve the Schrödinger equation (12.36). It ceases to
be the case for other realistic potentials to be considered next.

12.4.5 SCATTERING FROM THE YUKAWA

POTENTIAL
For most potentials, we have to calculate the phase shifts
numerically. This entails computing the radial wave function
for a given energy and partial wave l from Eq. (12.26) or
(12.36), and matching it to the free-particle solutions (12.34).
 It is most convenient for us to solve Eq. (12.36) with
Numerov's method, which we have used for bound states in
central field potentials such as the hydrogen atom (Section
9.5). For scattering states, the problem is even easier to handle.
Recall that for bound states the solutions diverge in the upward
direction of integration as r → ∞ (Section 9.1.1). We do not
have this difficulty for scattering (continuum) states, since
scattering wave functions oscillate rather than decay at large
values of r. It is not necessary to integrate Eq. (12.36) upward
and downward and match in the middle.

The procedure is very similar to Program 9.5. We only need

to start from the origin with u(0) = 0, and integrate Eq. (12.36)
upward until the field-free region, r ≥ a, is reached. We apply
the outgoing boundary condition to obtain the correct wave
function (12.28a), hence the phase shift (12.37). The full
modified program is given in Program 12.2. The following code
illustrates the core function.

def wf(M, xm): # find w.f. and deriv at xm

c = (h*h)/6.
wfup, nup = ode.numerov(f, [0,.1], M, xL, h) # 1 step past xm
dup = ((1+c*f(xm+h))*wfup[−1] − (1+c*f(xm−h))*wfup[−3])/(h+h)
return wfup, dup/wfup[−2]
……
jl, dj = sph_jn(Lmax, k*a) # (j_l, j_l ’) tuple
nl, dn = sph_yn(Lmax, k*a) # (n_l, n_l ’)
 The module wf() integrates the radial equation upward for
M steps using Numerov's method from the ODE library. Its
function is similar to shoot() in Program 9.5. The energy E and
angular momentum l are properly set before wf() is called. It
returns the wave function on the grid, as well as . The
latter is used to calculate the phase shift from Eq. (12.37),
which requires the spherical Bessel functions and their
derivatives at ka, namely, jl, nl, , and . Like the Hermite
polynomials (Program 9.4), we can obtain these values from
the SciPy special functions library as sph_jn and sph_yn. Each
returns a pair of arrays with indices from 0 to Lmax, containing
the function values and the derivatives. Once the phase shifts
have been calculated, we can obtain the cross sections and
normalized wave functions.

We first test Program 12.2 against a known case of

scattering from the square spherical barrier,

The solutions are analytically available in terms of spherical

Bessel functions (see Exercise E12.7). But, only for l = 0 can the
phase shift be easily calculated analytically. For l > 0, we still
need to rely on numerical evaluation of these functions.
 Figure 12.16: Phase shifts by a square spherical barrier, a = 1 and V0
= 2. The dashed line is the δ0 result by a hard sphere.

We calculate the phase shifts from Program 12.2, shown in

Figure 12.16. When compared to the analytic results (Project
P12.6), they are identical within numerical accuracy. The
spherical barrier does not support bound states. All phase
shifts are negative, approach zero as k → 0 with zero slope,
except l = 0 with a finite slope. The latter is similar to the s-
wave for the hard sphere, with different slopes. For a finite V0,
the waves can enter the barrier, however slightly at low
energies, leading to a less steep slope. As V0 → ∞, the result for
the finite barrier will approach that for a hard sphere.

With increasing energies, the phase shifts reach some

minima and rise up sharply, followed by plateaus and smaller
oscillations. The sharp edges are well-known features of
resonances in the effective potential (see Section S:8.1.2, Figure
S:8.5 for the 1D analog). At high energies, the phase shifts
vanish.

We now turn our attention to scattering from the Yukawa

potential (12.13). The nuclear charge and screening length are Z
= 2 and a = 1, respectively. Note that a is not the matching
radius, which is set to 10 in this case, with a grid size h = 0.05.

Figure 12.17 shows the first two radial wave functions ul(kr)
at the energy E = 1. They start from zero and show faster
oscillations than free-particle solutions in the inner region.5
For direct comparison, the peak values are normalized to be
equal. The faster oscillations, or shorter wavelengths, cause the
scattered waves to accumulate phase at a faster rate than the
unscattered free waves, giving rise to positive phase shifts.
Conversely, if the potential was repulsive, the scattered waves
would be suppressed (larger wavelengths or decaying waves),
leading to negative phase shifts. Summarizing these
observations, we have for the sign of phase shifts,
 Figure 12.17: Radial wave functions ul for l = 0 and 1 for the Yukawa
potential with Z = 2 and a = 1. The energy is E = 1, or .
The dashed lines are the field-free solutions jl(kr).

For potentials with mixed signs, the sign of the phase shift is
indeterminate, depending on the average effects of the
interaction.

For the Yukawa potential, which decreases exponentially,

the field-free region is established quickly at r ~ 3. As a result,
starting at kr 5, the scattered waves behave just like the
phase-shifted free-particle waves. In this case , the
phase shifts are δ0 ~ 1.9 and δ1 ~ 0.7, as can be estimated
visually from Figure 12.17.
 Figure 12.18: Phase shifts vs. partial waves l (left) and vs. wave
vector ka (right). The latter is scaled by 2l + 1, the multiplicity of
degeneracy. The potential parameters are the same as Figure 12.17.

We show in Figure 12.18 the phase shifts as a function of l

for fixed energies, and as a function of energy for fixed l values.
At low energies, only low partial waves are important. Higher
partial waves become significant for increasing energies. To
show this more clearly, we have multiplied the phase shifts by
the factor 2l + 1, the multiplicity of level degeneracy of partial
wave l (Figure 12.18, right).

For a given energy, we can estimate the typical range of

significant partial waves as lmax ~ k〈r〉, where 〈r〉 is the range of
the potential. For the Yukawa potential, it is on the order of a,
the screening length. If the energy is high, the value of lmax can
be large in order to achieve convergence. In that case,
perturbative methods such as the Born approximation (Section
S:12.3.2) may be more appropriate.

As k → 0, only s-wave phase shift remains, approaching δ0

= π. According to Levinson's theorem (12.23), we expect that
there is one bound state for l = 0. Indeed, this is the case for the
potential parameters used. For l ≠ 0, no bound states exist, as
indicated by the vanishing phase shifts for δl≠0 = 0 as k → 0.

Figure 12.19: Cross sections at two energies by the partial wave

method. The classical and Born (E = 2 only) results are also plotted.
The potential parameters are the same as Figure 12.17.

We present in Figure 12.19 the cross sections (12.32) at low

and intermediate energies. The classical (Figure 12.9) and Born
(Eq. (S:12.13)) results are also plotted for comparison. The
latter is valid from intermediate to high energies, so only E = 2
results are shown.

At low energies, the quantum mechanical results show a

minimum at θ ~ 50°. It is due to interference of the s- and p-
waves (l = 0 and 1), which are the predominant partial waves at
E = 0.1 (Figure 12.18). For stronger potentials, the minimum
can develop into a narrow, sharp dip that has been observed
experimentally.
 The differences between classical and quantal results are
very large in forward scattering angles up to 90°. We expect the
classical results to fail at low energies in general, since the de
Broglie wavelength is large, and the scattering is highly
quantum mechanical. In the forward angles, in particular, we
expect the disagreement to be even worse, because any
potential without a cut-off radius will cause a classical particle
to deflect, no matter how large the impact parameter is. This
causes the total cross section to always diverge as stated earlier
(Section 12.1.2). Quantum mechanically, however, the
uncertainty principle means that we can resolve Δθ only to
within some finite interval limited by the spread of angular
momentum, which is finite. As a result, quantal total cross
sections are finite for short-range potentials.

Toward large angles, agreement starts to emerge between

quantal and classical results. The classical rainbow and glory
singularities manifest as enhanced but smooth quantal cross
sections which, in fact, become maximum at θ = 180°.

At the intermediate energy E = 2, the quantal results start to

favor forward scattering. The agreement with the classical
results is much better for θ > 30°. There is still some
discrepancy at backward angles (150° to 180°), mainly due to a
handful of partial waves interfering with each other. The
differences disappear at higher energies (not shown) when a
large number of partial waves contribute, and interference
effects are insignificant.
 The Born results are not terribly accurate, but not grossly
inaccurate either. It is often useful as a quick check on the
order-of-magnitude estimate at this energy. Like the classical
results, it becomes accurate at higher energies (Section
S:12.3.2).

Chapter summary

We discussed potential scattering in both classical and

quantum mechanics, and compared the two where appropriate.

The direct method for classical potential scattering is to

calculate the deflection function on a grid, and obtain the cross
section by numerical differentiation. We can also conduct
numerical experiments to determine the cross section. We
studied rainbow and glory scattering in some detail.

Quantum mechanically, the partial wave analysis is the

easiest way to compute the cross sections via phase shifts. Used
with Numerov's method, the radial Schrödinger equation can
be efficiently solved for scattering states just as it could for
bound states (Chapter 9). At low energies, only the s-wave is
important, which also contains bound state information
(Levinson's theorem). Quantum scattering cross sections show
enhancement, rather than singularities, at the classical rainbow
and glory angles.
 In the calculation of phase shifts and cross sections, we
made extensive use of the SciPy special function library,
including spherical Bessel functions and Legendre
polynomials.

2.5 Exercises and Projects

EXERCISES
E12.1 (a) From Eq. (12.48), show that the distance of closest
approach in Rutherford scattering (V = k/r) is given by

The ± signs correspond to repulsive (k > 0) and

attractive (k < 0) potentials, respectively.

(b) Integrate Eq. (12.49) directly to derive the deflection

function (12.52) for the Coulomb potential. Optionally
try your luck with SymPy's symbolic integration
(Project P2.7).

E12.2 Derive the derivative dΘ/db (12.12) from Eq. (12.49).

Note that the upper limit umin depends on the impact
parameter b. Carefully handle the diverging terms
arising from the differentiation of the upper limit and
the denominator.
E12.3 The primary rainbow refers to light rays back-scattered
from water droplets. Figure 12.20 illustrates how the
rainbow is formed.

Figure 12.20: Schematics of optical rainbow scattering.

The incident light ray entering the droplet undergoes a

refraction, reflection, and another refraction, finally
exiting as the rainbow ray at a scattering angle θ.
Derive a formula for θ using Snell's law in terms of the
impact parameter b, droplet radius R, and refractive
index n.

Plot θ vs. b, assuming n = 1.33 and R = 1. Compare the

plot with the deflection function for the attractive plum
potential (Project P12.2 below) if available, or the
repulsive potential (Figure 12.5).

Show that θm ≤ θ ≤ 180°. Find θm and the value of b

where the minimum angle occurs. Optionally, draw the
 paths of rainbow rays over select b values between [0,
R].

E12.4 (a) Derive the deflection function for scattering from a

hard sphere (12.38). Sketch the Θ-b curve.

(b) Show that the differential cross section is

, independent of energy or the scattering angle, and the

total cross section is therefore σt = πa2. Explain both
results.

E12.5 Similar to the incident flux (12.16), the radial flux is

given by

where ψr = f(θ)eikr/r is the asymptotic outgoing

spherical waves (12.15). Calculate Ir and prove that it is
given by Eq. (12.17).

E12.6 (a) Show that the phase shift of the delta-wall potential
(Figure 9.21, Exercise E9.7) is given by Eq. (12.22).
Assuming m = 1 and α = 1, calculate and plot the phase

shifts for , 1, and 2 (all in a.u.). Make sure the

curves are continuous and δ → 0 as ka 1.

 (b) Calculate the phase shift from the square well
potential of width a and depth V0 (Figure S:8.4), and
show that it is given by

where and .

Plot the phase shift as a function of k ∈ [0, 10] for m =

1, V0 = 1 and a = 1. Do the same with the same
parameters as in Figure S:8.5 (a = 3.8). Discuss your
results in relation to the transmission and reflection
coefficients there.

Optionally explore the phase shift at small k ~ 0 by

increasing the parameters a and V0. Verify that δ(k ~ 0)
= nπ, n being the number of bound states according to
Levinson's theorem. Check n against the case shown in
Table 9.1.

E12.7 (a) Show that for the square spherical potential well, V
= −V0 for r ≤ a, and V = 0 for r > a, the β value in Eq.
(12.35) is given by
 Prove that for l = 0, with the β value above, Eq. (12.35)
is identical to the 1D result of Exercise E12.6.

(b)* For the square spherical potential barrier, V0 < 0,

show that the β value is the same as above for E > |V0|,
and for E < |V0|, it is

where il(x) = (−i)ljl(ix) is the modified spherical Bessel

function of the first kind.

PROJECTS
P12.1 (a) Explore Rutherford scattering numerically. Plot the
trajectories (12.50) for different impact parameters as in
Figure 12.2. Assume atomic units. Consider low energy

and intermediate energy E/k = 2 regimes. Vary

b in the range [10−2, 2]. At each b, find the distance of

closest approach rmin (Exercise E12.1). Plot the
trajectories as polar plots using polar projection as

plt.subplot(111, polar=True)
plt.plot(theta, r)
or convert (r, θ) to (x, y) plots. In either case, note that
the angle −α < θ < α in Eq. (12.50) is relative to the
radial bisector (Figure 12.1).

(b) Calculate the deflection function from Eq. (12.47).

First graph the integrand for a given E and b. Note that
it has a singularity at rmin, but it is an integrable
singularity. Estimate what the upper limit should be for
the integral to converge.

Evaluate the integral numerically with an open-end

integrator such as Gaussian integration. Incrementally
increase the upper limit until the integral converges to at
least three digits. Break up the interval as necessary to
maintain accuracy. Compute the deflection function at
low and intermediate energies for b in [10−2, 102]. Plot
the results on a semilog scale together with the analytic
result.

Accelerate convergence by isolating the integrable

singularity (using a technique similar to Eq. (8.79) or
Project S:P11.7). Make a variable substitution r − rmin =
x2, and repeat the evaluation. Explain the reason for the
improvement in the results.

Apply the above techniques to u-space (12.49). Assess

the advantage or disadvantage relative to r-space
(12.47).
P12.2 Study scattering from the attractive plum-pudding
potential (12.10). Use the parameters R = 1 and Z = −1.

(a) Make predictions about the deflection function and

the cross sections. Sketch them for low and intermediate
energies.

Run Program 12.1 to compute the deflection function

and the cross sections at E = 0.5 and 2, respectively.
Discuss the results and compare with the repulsive
potential (Figures 12.5 and 12.7). Record the rainbow
angles and impact parameters. What parts of the results
are the same for the deflection function? Cross section?

(b) Based on the deflection function, sketch the particle

trajectories. Write a program to compute and plot the
trajectories. Use the leapfrog solver to integrate the
equations of motion, and set the initial condition to

and , where a is large compared

to the range of the potential (say a = 10), b the impact

parameter, and v according to the collision energy
(12.1). Divide b evenly between [0, bmax], e.g., bmax = 2,
in roughly 10 intervals. Stop the integration when the
particle leaves the interaction region, for instance when
r ≥ 1.5a.

Plot the trajectories for all impact parameters for a

given energy. Discuss how they relate to the rainbow
 angles and impact parameters.

P12.3 Consider classical scattering from the square spherical

well V = −V0 for r ≤ a, and V = 0 for r > a.

(a) Sketch a typical trajectory for b < a and V0 > 0. Also

sketch the deflection function as a function of b.
Explain your reasoning.

Modify Program 12.1 to calculate the deflection

function Θ and the cross section σ(θ). Use a = 1 and V0
= 1, and plot Θ(b) for b ≤ a and σ(θ) up to the maximum
scattering angles for two energies E = 0.5 and 1. Take
precaution to treat the discontinuity in the potential at r
= a, e.g., by breaking the integration range into two
parts on either side of a and properly shifted for the
integration variable. The plots should be sufficiently
smooth. Comment on your results. Can you tell if σ(θ)
is finite as θ → 0?

(b) Show that the scattering is equivalent to an optical

refraction problem if the refractive index of the sphere
is defined as

Calculate the deflection function from Snell's law, and

compare with the numerical results from part (a).
 Because you have the exact Θ(b), calculate the cross
section analytically from Θ(b) (evaluated numerically,
of course).

Discuss the source of discrepancies, particularly at

small scattering angles, between the cross sections
obtained by finite difference and by analytic
differentiation. How can the differences be reduced?

(c)* Let V0 = −1, so the scattering is from a spherical

barrier. Repeat the above parts, but for energies E = 0.5,
1.2, and 2. Compare the results for the lowest energy
with scattering from the hard sphere, Exercise E12.4.

At higher energies E > |V0|, investigate the existence of

critical scattering angles where, according to part (b)
above, the index of refraction n is less than 1 when V0 <
0. Explore the dependence of the critical angle on the
impact parameter and energy. What is the physical
explanation of the critical angle in the case of
scattering?

P12.4 Carry out the calculations of classical scattering for the

Yukawa potential (12.13), assuming parameters Z = 2, a
= 1, and E = 0.1.

(a) Compute the deflection function Θ for b ∈ [10−2, 10]

using Program 12.1 or codes developed from above
projects. Plot the results (Figure 12.8). Ensure the data
points are dense enough so as to obtain an overall
smooth curve, especially around the sharp dip at the
rainbow angle. Map the deflection angle to the
scattering angle θ according to Eq. (12.4), and plot it
alongside Θ.

(b) Obtain the cross sections. It may be easier to search

for all contributing impact parameters from the mapped
scattering angle using Eq. (12.11). But, watch for
artificial jumps in the angle due to the mapping (e.g., at
b ~ 2.5 and 3), and resolve them appropriately to ensure
continuity of the derivatives (12.11). Plot the results.

Depending on the spacing of data points for the impact

parameter, your results may not be smooth in the
neighborhood of the rainbow and glory angles. If so,
increase the density of data points in the neighborhood
until the curve is satisfactorily smooth.

(c)* Accurately calculate the cross section from Eq.

(12.12). For a given scattering angle θ, we need to know
the contributing impact parameters. To do so, identify
all Θ values corresponding to a given θ from the
deflection function Θ (already available from above),
and bracket the contributing b values. Use the bisection
root solver to find the exact b values from Eq. (12.3)
(see Project S:P11.7 for solving equations involving
integrals). Plug these values into Eqs. (12.12) and (12.7)
to obtain the cross sections. Plot the results and
 compare with the direct but crude approximations
above.

P12.5 Numerically study quantum scattering from the ideal

hard sphere (12.38), V = ∞ for r ≤ a, and V = 0 for r >
a. Assume a = 1.

(a) Calculate the phase shifts δl from Eq. (12.40) for l ≤

lmax = 20 as a function of k from 0 to 10 in equal steps
of 0.1 or smaller. Obtain jl and nl values from the SciPy
special function library as demonstrated in Program
12.2. Adjust δl by adding or subtracting π so that they
are continuous and approach δl(k → 0) = 0. Plot the
results as in Figure 12.13.

Compute phase shifts by integrating the Schrödinger

equation with Program 12.2. Modify the starting point
to begin from r = a. Integrate to a short distance away,
say 2a, and match the wave functions. Determine the
phase shifts and compare with the analytic results
above.

(b) First calculate the differential cross sections at

several k values from low to high energies, e.g., k =
10−2, 1 and 10. Plot the results as in Figure 12.15.

Next, calculate the total cross section σt(k) as a function

of k. Extend the upper end of k to 102 or higher. Make
sure to adjust lmax for larger k. Plot the shifted results,
σt/π−2, on a log-log scale. How do you know if the lmax
is adequate? For large k 1, you should see the curve

approaching a straight line. Determine the slope. What

is the meaning of the slope? Explain.

(c) Generate and plot the scattering wave functions as

shown in Figure 12.14. Divide the radius between [a,
rmax] into m intervals, and θ between [0, 2π] into n
intervals (though the maximum θ is π, it is easier to
cover the full angular range without the need for
reflection). Form an m × n mesh over (r, θ) and
subsequently a grid over (x, y) (see Program 7.6 for an
example).

On the (r, θ) mesh, calculate the full wave function ψ in

Eq. (12.25) using the radial wave function (12.39). At
each grid point, you must perform the sum over l. Do
the same for the scattered wave function ψsc by
subtracting jl(kr) from the radial wave function (12.39).
Once the wave functions are generated, plot |rψ|2 and
|rψsc|2 over the (x, y) grid as filled contours with

plt.contourf(x, y, wf, 256, cmap=plt.cm.jet)

This creates a 256-level contour plot. Plot the wave

functions for k in the low, intermediate, and high
 energies, e.g, k = 10−2, 1, and 5. Adjust the grid sizes so
the plots are smooth.

(d)* Calculate and plot the scattered radial flux r2Ir(θ)

(Exercise E12.5). Plot it as a vector field with the
plt.quiver() function (see Program 7.5) over the
same grid and with the same k values as above.

P12.6 Investigate quantum scattering from the square

spherical potential well or barrier, V = −V0 for r ≤ a, and
V = 0 for r > a, considered in Project P12.3 for classical
scattering.

(a) First assume a potential well, V0 = 1 and a = 1.

Replace the potential in Program 12.2, and calculate the
phase shifts δl for l ≤ lmax as a function of k ∈ [10−2, 10]
incremented by a constant factor (say 1.1). Set lmax such
that higher partial waves than lmax are negligible at the
largest k. Ensure that the jumps in π are accounted for.
Plot the phase shifts as a function of k.

Calculate the differential cross section at energies E =

0.5 and 1. Compare with the classical results from
Project P12.3 if available.

(b) Due to the sharp cut-off of the potential, accuracy of

results will suffer unless the discontinuity is properly
treated. To do so, linearly interpolate the potential
between the two immediate grid points sandwiching the
radius a according to Eq. (9.52) (Project P9.4).
 With the same grid size h, calculate the phase shifts.
Compute the relative errors of the results with and
without the interpolation from the exact results
(Exercise E12.7). Plot the relative errors for the first
several partial waves as a function of k. Explain the
improvement.

(c) Check that Levinson's theorem (12.23) works

correctly. Compute the phase shifts for l = 0, 1, and 2 as
a function of k. Use the same parameters for V0 and a as
in Table 9.1. For δ0, the number of bound states nb is the
same in 1D and 3D. For l ≠ 0, use Program 9.5 to find
nb. Show that δl approaches the correct limit as k → 0.

(d) Repeat the above investigation for the square barrier,

V0 = −1.

P12.7 Comparatively study quantum scattering from the

Yukawa potential (12.13) against several other
potentials including,

(i) the Gaussian potential, ;

(ii) the Hulthén potential, ;

 (iii) the Buckingham polarization potential,

(a) Plot (or sketch to scale) these potentials for the same
parameters Z = 2 and a = 1. Compare the shapes of the
potential. Predict which potential will have the greatest
effect on scattering relative to the Yukawa potential.

(b) Using Program 12.2 or a similar one you have

developed, calculate the phase shifts at E = 2 for the
potential predicted above. Plot the results along with
those for the Yukawa potential on the same graph as a
function of l. Also calculate and plot the differential
cross sections. Discuss the similarities and differences
in the results. Do they support your prediction?

(c) Calculate the total cross sections for each of the

potentials as a function of energy from E = 0.1 to 20.
Comment on the results. Which potential has the largest
effect? The least effect? Explain.

P12.8 The cross section for electron scattering from light

atoms, represented by weak Yukawa potentials (Eq.
(12.13), Z ~ 1), is relatively featureless. For heavier
atoms, however, some interesting effects can occur,
including the so-called Ramsauer-Townsend effect.
Investigate this effect in this project.
 Figure 12.21: The Ramsauer-Townsend effect (Project
P12.8).

Let Z = 10 and . Calculate the phase shifts as a

function of k = [10−2, 10] for l = 0 to 5. Compute and

plot the total cross section as a function of k. You
should see a graph like Figure 12.21. Note the peak at k
~ 3 and the quick descend into the valley at k ~ 1. The
Ramsauer-Townsend effect refers to the sharp decrease
of the cross section at low energies. Why does this
happen?

To explore this effect, plot the phase shifts. Below k

5, are higher partial waves l = 2 to 5 negligible?

Confirm by plotting the partial cross sections
 Now consider only the first two partial waves l = 0 and
1. How does δ0 change when k decreases from 3 to 1?
How does σ0 behave? How does that explain the rapid
drop in the cross section around k ~ 2? What is the role
of σ1?

Repeat the above calculation for a = 0.15. Does the

Ramsauer-Townsend effect occur? Does your

explanation apply? How about for Z = 5 and ?

2.A Derivation of the deflection function

To find the polar angle in terms of the radial vector, we first
express their time derivatives and from Eqs. (4.6) and (4.8),

We can derive the relationship between θ and r by eliminating

time from Eq. (12.43) with
Substituting Eq. (12.43) into (12.44) and integrating both sides,
we obtain

Since α in Eq. (12.2) is the angle swept by the radial vector

from rmin to ∞ (Figure 12.1), we can obtain α by replacing r0
and r with rmin and ∞ in Eq. (12.45), respectively,

Here, rmin is the turning point where .

Using Eq. (12.1) to simplify α explicitly in terms of the

impact parameter b, the resulting expression for the deflection
function from Eq. (12.2) is

where the distance of closest approach (turning point) satisfies

Sometimes it is useful to convert the integral of an infinite

range into a finite range by u = 1/r. This gives
 For Coulomb potentials V (r) = k/r, the integral (12.47) can
be carried out analytically. However, we had already described
the result for the orbit of the Kepler problem, which is
hyperbolic for positive energies, Eq. (4.11). Combining Eqs.
(4.13) and (4.14), we have

where e > 1 is the eccentricity (4.10). The ± signs correspond to

repulsive and attractive potentials, respectively. Equation
(12.50) is equivalent to setting the reference point θ = 0 at the
distance of closest approach. Therefore, the value of α is equal
to θ when r = ∞ in Eq. (12.50) (Figure 12.1, top), which means
that the denominator is zero,

From Eq. (12.2), we have , so it follows from Eq.

(12.51) that

Because the eccentricity e is positive, the deflection angle Θ

must be negative for attractive potentials.
 Equation (12.52) gives Θ as an implicit function of the
impact parameter and energy through the eccentricity e. To
make it explicit, we use Eq. (4.10) and the relations (12.1) to
obtain

Substituting Eq. (12.52) into (12.53), we find

We see that as b → 0, Θ → π, and as b → ∞, Θ → 0.

2.B Partial wave analysis

Phase shifts are determined by the behavior of the radial wave
function in the asymptotic region where the potential is
negligible. If V = 0, the radial equation (12.26) can be
simplified to

It admits two linearly independent solutions, jl(kr) and nl(kr),

the spherical Bessel functions of the first and second kind,
respectively.

The first several spherical Bessel functions are (x = kr) [10]

Higher orders of jl and nl can be obtained via the recurrence
relation

The same holds true for nl(x).6

For small x ~ 0, jl(x) and nl(x) approach the limits

For large x → ∞, the asymptotic behaviors are

The function jl(kr) is regular in the entire range r ∈ [0, ∞),

and is always a solution to Eq. (12.55), as discussed earlier
relative to the incident wave (12.24). But, nl(kr) diverges at the
origin, and is therefore a valid solution only if the origin is
excluded. This is the case in the asymptotic region of Eq.
(12.26) where the general solution Rl(k, r) can be written as
We have used the asymptotic behavior (12.59) and (12.60) in
Eq. (12.61), where the first and second terms are the incoming
and outgoing spherical waves, respectively.

In comparison to the incident wave (12.27), we require the

incoming wave be unmodified,

so that Eq. (12.61) becomes

Analogous to the 1D case (12.20), we also require that the

incident flux be equal to the outgoing flux,

where δl is a real number, to be identified as the phase shift.

Solving Eqs. (12.62) and (12.64), we obtain the coefficients

Al and Bl as

Finally, substituting Eq. (12.65) into (12.61) and (12.63), we

obtain the asymptotic limit of the radial wave function in terms
 of δl,

We can see from Eq. (12.66c) that the meaning of δl is a phase

shift.

2.C Program listings and descriptions

Program listing 12.1: Deflection function ( deflect.py)

1 import matplotlib.pyplot as plt

import numpy as np, rootfinder as rtf, integral as itg
3

def V(r): # plum potential

5 return Z*(3−r*r/(R*R))/(R+R) if (r<R) else Z/r

7 def fu(u): # f(u), turning point eqn

return 1 − V(1./u)/E − b*b*u*u
9

def fx(x): # integrand, called by gauss

11 u = umin − x*x
return 2*x*b/np.sqrt(fu(u))
13

def xection(theta, ba, da): # cross section

15 cs = 0.0 # ba=impact para, da=deflection angle
for i in range(len(ba)−1):
 17 if ((theta−da[i])*(theta−da[i+1]) < 0.): # theta bracketed
db = ba[i+1] − ba[i]
19 cs += (ba[i] + db/2.)*abs(db/(da[i+1]−da[i]))
return cs/np.sin(theta)
21

Z, R = 1.0, 1.0 # nuclear charge Z, radius of plum potential

23 E, b, bmax = 1.2, 0.01, 20. # energy, initial b, bmax
eps, tiny =1.E−14, 1.E−5 # rel error, u−limit
25 ba, theta = [], [] # impact para, deflection
while (b <= bmax):
27 umin = rtf.bisect(fu, tiny, 1./b, eps) # find turning pt
alpha = itg.gauss(fx, 0., np.sqrt(umin))
29 ba.append(b), theta.append(np.pi − 2*alpha)
b *= 1.02
31

plt.figure (), plt.plot(ba, theta) # plot deflection function

33 plt.xlabel(’ $b$ (a.u.)’), plt.ylabel(’ $\Theta$’, rotation=0)
plt.yticks ([0, np.pi/2, np.pi], [’ $0$’,’ $\pi/2$’, ’ $\pi$’])
35 plt.xlim(0,3)

37 cs, sa = [], np.linspace (0.01, np.pi−.01, 500) # scattering angle

for x in sa: # calc, plot cross section
39 cs.append(xection(x, ba, theta))
plt.figure (), plt.plot(sa, cs)
41 plt.xlabel(r’ $\theta$’), plt.ylabel(r’ $\sigma(\theta)$’)
plt.xticks ([0, np.pi/2, np.pi], [’ $0$’,’ $\pi/2$’, ’ $\pi$’])
43 plt.xlim(0, np.pi), plt.semilogy()

45 plt.show()

Given a potential V (r), Program 12.1 calculates the

deflection function and the differential cross sections, Eqs.
(12.3) and (12.7), respectively. The function fu() returns the
equation defining the turning points (12.48) in u = 1/r space.
The other function, fx(), computes the integrand (12.49), but
with a change of variable u = umin − x2 to improve convergence
(see Project P12.1 and Project S:P11.7).

The main code sets the relevant parameters, and iterates

through the impact parameter range. In each iteration, it finds
the turning point umin. The bisection method is used to solve
Eq. (12.48). Since the plum potential is assumed to be positive,
the turning point is such that b < rmin < ∞, or equivalently 0 <
umin < 1/b. A small value tiny is used for zero to avoid
overflow. Having obtained umin, the α value is calculated from
Eq. (12.49) by Gaussian integration, using the variable
substitution above.

The impact parameter is increased by a constant factor in

each iteration, so we can step through a large range without
wasting time. This is required if we are interested in small
angle scattering. The rest of the code plots the deflection
function and the cross section.

The cross section is calculated by the module xection()

from the deflection function with the approximation (12.11),
requiring the impact parameter array ba and the corresponding
deflection angle array da. For a given scattering angle θ, it
searches through the deflection angles that bracket θ (line 17).
If found, is approximated at the midpoint. The cumulative
contributions from all impact parameters (12.7) are included.
Program listing 12.2: Partial wave method ( qmscatt.py)

1 from scipy.special import sph_jn, sph_yn, lpn

import matplotlib.pyplot as plt, numpy as np, ode
3

def V(r): # Yukawa potential

5 Z, sa = 2., 1.0 # nuclear charge, screening length
return −Z*np.exp(−r/sa)/r
7

def f(r): # Sch eqn in Numerov form

9 return 2*(E − V(r)) − L*(L+1)/(r*r)

11 def wf(M, xm): # find w.f. and deriv at xm

c = (h*h)/6.
13 wfup, nup = ode.numerov(f, [0,.1], M, xL, h) # 1 step past xm
dup = ((1+c*f(xm+h))*wfup[−1] − (1+c*f(xm−h))*wfup[−3])/(h+h)
15 return wfup, dup/wfup[−2]

17 xL, a, M = 0., 10., 200 # limits, matching point

h, Lmax, E =(a−xL)/M, 15, 2. # step size, max L, energy
19

k, ps = np.sqrt(2*E), np.zeros(Lmax+1) # wave vector, phase shift

21 jl, dj = sph_jn(Lmax, k*a) # (j_l, j_l ’) tuple
nl, dn = sph_yn(Lmax, k*a) # (n_l, n_l ’)
23

for L in range(Lmax+1):
25 u, g = wf(M, a) # g= u’/u
x = np.arctan(((g*a−1)*jl[L] − k*a*dj[L])/ # phase shift Eq. (12.37)
27 ((g*a−1)*nl[L] − k*a*dn[L]))
while (x < 0.0): x += np.pi # handle jumps by pi
29 ps[L] = x

31 theta, sigma = np.linspace(0., np.pi, 100), []

cos, La = np.cos(theta), np.arange(1,2*Lmax+2,2)
33 for x in cos: # calc cross section
 pl = lpn(Lmax, x)[0] # Legendre polynomial
35 fl = La* np.exp(1j*ps)*np.sin(ps)*pl # amplitude
sigma.append(np.abs(np.sum(fl))**2/(k*k))
37

plt.figure () # plot phase shift vs L

39 plt.plot(range(Lmax+1), ps, ’ -o’)
plt.xlabel(’ $l$’), plt.ylabel(r’ $\delta_l$’, fontsize=16)
41

plt.figure ()
43 plt.plot(theta*180/np.pi, sigma) # plot cross sections
xtck = [0, 30, 60, 90, 120, 150, 180]
45 plt.xticks (xtck, [repr(i) for i in xtck]) # custom ticks
plt.xlabel(r’ $\theta$ (deg)’)
47 plt.ylabel(r’ $\sigma(\theta)$ (a.u.)’), plt.semilogy()

49 plt.show()

Phase shifts, cross sections, and normalized wave functions

may be obtained from Program 12.2. It efficiently solves
quantum scattering problems from central field potentials by
partial wave expansion with Numerov's method. It is based on
Program 9.5.

It starts by importing the spherical Bessel functions and

other packages. The function V(r) calculates the Yukawa
potential, and f() returns the corresponding function f(r) in
Eq. (12.36) dependent on the energy E and the effective
potential. The code up to line 22 had been explained in the
main text, except for the two lines in the middle initializing
parameters including the matching radius a, the number of
intervals M, step size h, maximum partial waves lmax, and
scattering energy E. Depending on how quickly the potential
decays, or how high the energy is, it may be necessary to adjust
a, M (h) to ensure accuracy. Generally speaking, the higher the
energy, the more rapidly the wave function oscillates, and
therefore a larger M should be used.

The next L-loop calls wf(M,a) integrating the radial

equation for a given l to obtain the wave function u0, u1, …,
uM+1. Note that the derivative is evaluated at ,
however. The phase shift δl is calculated from Eq. (12.37) with
the returned γ (variable g). For attractive potentials, line 28
ensures that the phase shift is positive due to jumps by π in the
arctan function. All phase shifts are stored in the array ps.

The cross section is calculated next. For each angle θ, the

SciPy special function lpn() is called to calculate the Legendre
polynomial Pl(cos θ). Like the spherical Bessel functions, lpn()
also returns a pair of arrays, the function values and the
derivatives, for all orders 0 − lmax. We only need the function
values, so we select the first array in line 34. Next (line 35), the
scattering amplitude is calculated from Eq. (12.31) for all
partial waves at once, and the cross section from Eq. (12.32). At
higher energies, lmax should be sufficiently large to include all
contributions.

The remaining code plots the phase shifts and the cross
sections. We use customized tick marks (line 45) for the latter.
 1
This fact is an exception rather than the rule. It is related to the special behavior
(symmetry) of the Coulomb potential which is also responsible for the Runge-Lenz vector
(4.20) being a constant of motion in 1/r fields. In general, the scattering angle depends on
the sign of noncoulombic potentials.

2
A rainbow in the sky looks slightly different to observers at different locations, so
everyone sees their own rainbow. You can easily make your own rainbow by holding a mist-
emitting hose facing away from the afternoon sun.

3
Like the rainbow, the glory has its origin in sunlight back-scattered by water droplets.
Basically, light rays entering the sides of a droplet are first refracted critically, then reflected
from the back to the other side, and refracted again, finally emerging from the droplet in the
opposite direction to the incident light [13]. The net effect is a colored ring around the
shadow of an observer's head on the cloud. It is rarer to observe the glory than the rainbow.

4 −1
Potentials decaying at or slower than x such as the Coulomb potential are long-
ranged. Particles are never free in long-ranged potentials, suffering distortions even at
infinity (see Exercise S:E12.7). As a result, scattering in these potentials does not have well-
defined phase shifts. The same effect causes an infinite number of bound states in the
Coulomb potential (Chapter 9, Section 9.5).

5
For l ≥ 1, true oscillations begin only after entering the classically allowed region
beyond the turning point (Figure S:12.1).

6
The recurrence relation (12.57) is stable in the downward direction for jl and in the
upward direction for nl (see Exercise S:E12.6).
List of programs
We have built many programs in this book. The table below
summarizes these programs and dependencies. Standard
libraries such as Matplotlib (2D), NumPy, and SciPy are
omitted. Library abbreviations are: fft – fast Fourier transform
( fft.py, Chapter 5); fem – finite element method ( fem.py,
Chapter 9); fileio – file input/output ( fileio.py, Chapter 9); itg
– numerical integration ( integral.py, Chapter 8); ode –
ordinary differential equation ( ode.py, Chapter 2); rtf – root
finders ( rootfinder.py, Chapter 3); and vpm – VPython
Modules ( vpm.py, Chapter 6). In addition, we also note the use
of 3D and animation libraries: Axes3D – Matplotlib 3D
plotting; am – Matplotlib animation; and vp – VPython.

Page numbers with the “S:” prefix are entries from the
Digital Online Companion.

Program Description Dependence

3body, 206 Three-body motion ode, rtf, vp
balltoss, 84 Ideal projectile motion vp
baseball, 132 Flight of baseball ode, vp
bdipole, 391 Magnetic dipole field vp
bem, 522 Basis expansion method itg
bisect, 130 Bisection root finder
boltzmann, 627 Boltzmann distribution
bouncing_ball, Bouncing ball vp
40
brownian, 559 Brownian motion am
coupled, S:112 Quantum transitions ode, vp, vpm
ctmc, S:263 Classical ion-atom ode, rtf. vp, vpm
collisions
deflect, 679 Deflection function itg, rtf
dipole, 394 Dipole distribution Axes3D
earth, 136 Planetary motion ode, vp
edipole, 393 Electric dipole radiation vp
eigh, 285 Eigenvalue problem
einsteinsolid, 566 Einstein solid class
entropy, 571 Entropy of an Einstein
solid
equishare, 626 Thermal energy sharing am
fem, 525 FEM library
fft, S:44 Fast Fourier transform
fractal, 269 Mandelbrot fractal
freefall_euler, 57 Free fall with modular
Euler
freefall_plot, 44 Free fall with inline Euler
gauss_elim, S:57 Gauss elimination
gaussxw, 457 Gauss abscissa and weight
hockey, 381 Electric field hockey ode, vp
hydrogen, 523 Atomic structure rtg
integral, 456 Numerical integration
ising1d, 627 Ising model
lambertw, 101 Lambert W function
laplace, 382 Relaxation solutions Axes3D, vp
laplacefem, 386 Laplace equation by FEM Axes3D
laplacerbf, 389 Meshfree method for
PDEs
leapfrog, 65 Leapfrog method
leapfrog_tt, 151 Time-transformed leapfrog
leapfrog_ttN, N-body transformed
S:224 leapfrog
life, S:149 Game of life
logisdiff, 265 Difference of logistic map
logisticmap, 216 Logistic map iterates
longwire, 393 Magnetic fields of a long vp
wire
lyapunov, 265 Lyapunov exponent
mcint, 561 Monte Carlo integration
md, 629 Molecular dynamics ode, vp, vpm
mdf2py, S:192 Molecular dynamics F2Py ode, nbodyf
mercury, 200 Precession of Mercury ode, vp
meshhex, 529 FEM meshes of a hexagon fileio
metropolis2, 589 Ising model in 2D
motion, 10 1D motion and graphing
nbody, S:266 N-body Coulomb
interaction
nbodyf, S:189 N-body interaction,
Fortran
newton, 131 Newton's root finder
nns, S:135 Energy spectrum
unfolding
nonlindro, 266 Nonlinear pendulum ode
odewrap, 80 SciPy ODE wrapper
perioddbl, 222 Bifurcation diagram
plane, 395 Plane electromagnetic vp, vpm
wave
poincare, 267 Poincaré map ode
projectile, 91 Motion with linear drag
qmdot, 526 Quantum dot Axes3D, fem,
fileio
qmfem, 519 Eigenenergies by FEM
qmplane, 450 Quantum plane wave am
qmscatt, 681 Central field scattering ode
qmshoot, 518 Quantum shooting method ode, rtf
qmwaves2d, 454 Quantum waves in 2D vp, vpm
r3body, 211 Restricted 3-body motion ode, vp
r3body_veff, 209 Lagrange points Axes3D
range, 97 Range with linear drag rtf
relax, S:59 Relaxation of a string Axes3D, ode
relaxtd, S:187 Thermal relaxation
rk2, 51 Runge-Kutta order 2
rk4, 53 Runge-Kutta order 4
rk45n, 79 Runge-Kutta-Felhberg
rk4n, 79 Nonvector RK4
rvfit, 202 Radial velocity modeling
sdlf, 451 Quantum wavepacket ode, vp, vpm
 motion
sho, 272 Simple harmonic oscillator vp
sho_lf, 66 Oscillator with leapfrog ode
shoot, 120 Shooting method ode, rtf
slinky, S:60 Motion of a slinky ode, vp, vpm
splitop, 452 Quantum free fall vp, vpm
squarewell, 516 Visualizing eigenstates ode, vp, vpm
stringfdm, 290 Displacement of a string
stringfun, 326 Fundamental frequencies
tablecloth, 328 Falling tablecloth ode, vp, vpm
transport, S:188 Particle transport
triatom, 326 Triatomic vibration vp
tripwf, 528 Triangular mesh plots fileio
vpm, S:61 VPython modules vp
walk2d, 558 Random walk in 2D
waves, 305 Waves on a string vp, vpm
waves2d, 327 Waves on a membrane vp, vpm
 Bibliography
1] M. Abramowitz and I. A. Stegun. Handbook of mathematical
functions: with formulas, graphs, and mathematical tables.
(Dover, New York), 1970.

2] R. K. Adair. The physics of baseball. Physics Today, 48(5):26–

31, (1995).

3] R. K. Adair. The physics of baseball. (Perennial, New York),

2002.

4] G. B. Arfken and H. J. Weber. Mathematical methods for

physicists. (Academic Press, New York), 6th edition, 2005.

5] T. Asai, K. Seo, O. Kobayashi, and R. Sakashita. Fundamental

aerodynamics of the soccer ball. Sports Eng., 10:101–110,
(2007).

6] N. W. Ashcroft and N. D. Mermin. Solid state physics.

(Saunders College, New York), 1976.

7] E. B. Becker, G. F. Carey, and J. T. Oden. Finite elements: an

introduction. (Prentice Hall, Englewood Cliffs, NJ), 1981.
8] C. A. Bennett. Principles of physical optics. (Wiley, New York),
2008.

9] S. J. Blundell and K. M. Blundell. Concepts in thermal physics.

(Oxford University Press, New York), 2010.

10] M. L. Boas. Mathematical methods in the physical sciences.

(Wiley, New York), 3rd edition, 2006.

11] B. H. Bransden and C. J. Joachain. Physics of atoms and

molecules. (Prentice Hall, New York), 2002.

12] K. Briggs. A precise calculation of the Feigenbaum constants.

Math. Comp., 57:435–439, (1991).

13] H. C. Bryant and N. Jarmie. The glory. Sci. Am., 231:60–71,

July (1974).

14] R. P. Butler, J. T. Wright, G. W. Marcy, D. A. Fischer, S. S.

Vogt, C. G. Tinney, H. R. A. Jones, B. D. Carter, J. A. Johnson,
C. McCarthy, and A. J. Penny. Catalog of nearby exoplanets.
Astrophys. J., 646:505–522, (2006).

15] A. Chenciner and R. Montgomery. A remarkable periodic

solution of the three-body problem in the case of equal masses.
Ann. of Math., 152:881–901, (2000).

16] G. M. Clemence. The relativity effect in planetary motions. Rev.

Mod. Phys., 19:361–364, (1947).
17] C. J. Cohen and E. C. Hubbard. Libration of the close
approaches of Pluto to Neptune. Astrono. J., 70:10–13, (1965).

18] J. W. Cooley and J. W. Tukey. An algorithm for the machine

calculation of complex Fourier series. Math. Comp., 19:297–
301, (1965).

19] S. N. Coppersmith. A simpler derivation of Feigenbaum's

renormalization group equation for the period-doubling
bifurcation sequence. Am. J. Phys., 67:52–54, (1999).

20] R. M. Corless, G. H. Gonnet, D. E. G. Hare, D. J. Jeffrey, and

D. E. Knuth. On the Lambert W function. Adv. Comp. Math.,
5:329–359, (1996).

21] J. Crank and E. Nicolson. A practical method for numerical

evaluation of solutions of partial differential equations of the
heat-conduction type. Adv. Comp. Math., 6:207–226, (1996).

22] R. Cross. Physics of baseball and softball. (Springer, New

York), 2011.

23] C. R. Davis and H. M. Schey. Eigenvalues in quantum

mechanics: a computer-generated film. Am. J. Phys., 40:1502–
1508, (1972).

24] P. DeVries and J. Hasbun. A first course in computational

physics. (Jones & Bartlett, Sudbury, MA), 2010.
25] E. Doolittle. The secular variations of the elements of the orbits
of the four inner planets computed for the epoch 1850.0 G.M.T.
Trans. Am. Phil. Soc., 22:37–189, (1912).

26] P. Fallahi, A. C. Bleszynski, R. M. Westervelt, J. Huang, J.

Walls, and E. J. Heller. Imaging a single-electron quantum dot.
Nano Lett., 5:223–226, (2005).

27] G. E. Fasshauer. Meshfree approximation methods with

MATLAB. (World Scientific, Hackensack, NJ), 2007.

28] J. Feigenbaum. Quantitative universality for a class of

nonlinear transformations. J. Stat. Phys., 19:25–52, (1978).

29] R. Feynman. The Feynman Lectures on Physics, volume I.

(Addison Wesley, New York), 1970.
feynmanlectures.caltech.edu.

30] Debra A. Fischer, R. Paul Butler, Geoffrey W. Marcy, Steven S.

Vogt, and Gregory W. Henry. A Sub-Saturn mass planet
orbiting HD 3651. Astrophys. J., 590:1081–1087, (2003).

31] G. R. Fowles and G. L. Cassiday. Analytical mechanics.

(Thomson Brooks/Cole, Belmont, CA), 7th edition, 2004.

32] C. Frohlich. Aerodynamic drag crisis and its possible effect on

the flight of baseballs. Am. J. Phys., 52:325–334, (1984).

33] T. Gay. The physics of football. (HarperCollins, New York),

2005.
34] N. Giordano and H. Nakanishi. Computational physics.
(Benjamin Cummings, New York), 2nd edition, 2005.

35] J. Gleick. Chaos: making a new science. (Penguin, New York),

1988.

36] J. E. Goff and M. J. Carré. Trajectory analysis of a soccer ball.

Am. J. Phys., 77:1020–1027, (2009).

37] A. Goldberg, H. M. Schey, and J. L. Schwartz. Computer-

generated motion pictures of one-dimensional quantum-
mechanical transmission and reflection phenomena. Am. J.
Phys., 35:177–186, (1967).

38] J. Golde, J. Shertzer, and P. Oxley. Finite element solution of

Laplaces equation for ion-atom chambers. Am. J. Phys., 77:81–
86, (2009).

39] H. Goldstein. Prehistory of the Runge-Lenz vector. Am. J.

Phys., 43:737–738, (1975).

40] H. Goldstein, C. Poole, and J. Safko. Classical mechanics.

(Addison Wesley, New York), 2002.

41] H. Gould and J. Tobochnik. Statistical and thermal physics

with computer applications. (Princeton University Press,
Princeton, NJ), 2010.

42] H. Gould, J. Tobochnik, and W. Christian. An introduction to

computer simulation methods: Applications to physical
 systems. (Addison-Wesley, New York), 3rd edition, 2007.

43] D. J. Griffiths. Introduction to electrodynamics. (Prentice

Hall, Upper Saddle River, NJ), 1999.

44] D. J. Griffiths. Introduction to quantum mechanics. (Prentice

Hall, Upper Saddle River, NJ), 2005.

45] P. Harrison. Quantum wells, wires and dots: theoretical and

computational physics of semiconductor nanostructures.
(Wiley, New York), 3rd edition, 2010.

46] B. Hayes. Why W? Am. Scientist, 93:104–108, (2005).

47] W. H. Heintz. Determination of the Runge-Lenz vector. Am. J.

Phys., 42:1078–1082, (1974).

48] P. Hellings. Astrophysics with a PC. (Willmann-Bell Inc.,

Richmond, VA), 1994. An introduction to computational
astrophysics.

49] D. Hestenes. New foundations for classical mechanics.

(Kluwer Pub., New York), 1999. Ch. 8, Celestial mechanics.

50] J. D. Jackson. Classical electrodynamics. (Wiley, New York),

1975.

51] C. J. Joachain. Quantum collision theory. (North-Holland,

Amsterdam), 1983.
52] E. J. Kansa. Multiquadrics – a scattered data approximation
scheme with applications to computational fluid dynamics II:
Solutions to parabolic, hyperbolic, and elliptic partial
differential equations. Comp. Math. Appl., 19:147–161, (1990).

53] R. Landau, M. Páez, and C. Bordeianu. Computational physics:

Problem solving with computers. (Wiley, New York), 2nd
edition, 2007.

54] H. P. Langtangen. A primer on scientific programming with

Python. (Springer, New York), 3rd edition, 2012.

55] E. Larsson and B. Fornberg. A numerical study of some radial

basis function based solution methods for elliptic PDEs. Comp.
Math. Appl., 46:891–902, (2003).

56] S. M. Lea. Mathematics for physicists. (Thomson Brooks/Cole,

Belmont, CA), 2004.

57] R. L. Liboff and J. Greenberg. The hexagon quantum billiard. J.

Stat. Phys., 105:389–402, (2001).

58] J. J. Lissauer. Chaotic motion in the Solar System. Rev. Mod.

Phys., 71(3):835–845, (1999).

59] E. N. Lorenz. Deterministic nonperiodic flow. J. Atmo. Sci.,

20:130–141, (1963).

60] G. D. Mahan. Quantum mechanics in a nutshell. (Princeton

University Press, Princeton, NJ), 2009.
61] B. Marmaras and J. Wang. Simulation and visualization of few-
body systems and the differential precession of Mercury.
Computing in Sci. & Eng., 16:42–50, (2014).

62] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H.

Teller, and E. Teller. Equation of state calculations by fast
computing machines. J. Chem. Phys., 21:1087–1092, (1953).

63] S. Mikkola and S. Aarseth. A time-transformed leapfrog

scheme. Integration of few-body systems with large mass
ratios. Celes. Mech. Dyn. Astrono., 84:343–354, (2002).

64] M. H. Mittleman. Introduction to the theory of laser-atom

interactions. (Plenum, New York), 1993.

65] A. Morbidelli. Modern integrations of solar system dynamics.

Annu. Rev. Earth Planet. Sci., 30:89–112, (2002).

66] NASA exoplanet archive.

http://exoplanetarchive.ipac.caltech.edu/.

67] A. M. Nathan. The effect of spin on the flight of a baseball. Am.

J. Phys., 76:119–124, (2008).

68] M. Newman. Computational physics. (CreateSpace

Independent Publishing), 2012.

69] H. M. Nussenzveig. The theory of the rainbow. Sci. Am.,

236:116–127, (1977).
70] T. Pang. An introduction to computational physics.
(Cambridge Univ. Press, Cambridge), 2nd edition, 2006.

71] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T.

Vetterling. Numerical recipes: the art of scientific computing.
(Cambridge Univ. Press, Cambridge), 1992.

72] M. P. Price and W. F. Rush. Nonrelativistic contribution to

Mercurys perihelion precession. Am. J. Phys., 47:531–534,
(1979).

73] E. Rabe. Determination and survey of periodic Trojan orbits in

the restricted problem of three bodies. Astrono. J., 66:500–
513, (1961).

74] I. I. Rabi. Space quantization in a gyrating magnetic field.

Phys. Rev., 51:652–654, (1961).

75] J. N. Reddy. An introduction to the finite element method.

(McGraw-Hill, New York), 1993.

76] F. Reif. Fundamentals of statistical and thermal physics.

(Waveland Press, Long Grove, IL), 2009.

77] S. M. Reimann and M. Manninen. Electronic structure of

quantum dots. Rev. Mod. Phys., 74:1283–1342, (2002).

78] M. J. Romanelli. Runge-kutta methods for the solution of

ordinary differential equations. In A. Ralston and H. S. Wiff,
 editors, Mathematical methods for digital computers, pages
110–120. (Wiley, New York), 1960.

79] A. E. Roy. Orbital motion. (IOP Publishing, Bristol), 2005.

80] J. J. Sakurai and J. Napolitano. Modern quantum mechanics.

(Addison Wesley, New York), 2011.

81] G. S. Sawicki, M. Hubbarda, and W. J. Stronge. How to hit

home runs: Optimum baseball bat swing parameters for
maximum range trajectories. Am. J. Phys., 71:1152–1162,
(2003).

82] D. V. Schroeder. An introduction to thermal physics. (Addison

Wesley, New York), 1999.

83] R. Seydel. Determinant factors of the table tennis game:

measurement and simulation of ball-flying curves. Int. J. Tab.
Tenn. Sci., 1:1–7, (1992).

84] C. Simó. New families of solutions in N-body problems.

Proceedings of the ECM 2000, Barcelona (July 10–14),
(2000).

85] M. G. Stewart. Precession of the perihelion of Mercury's orbit.

Am. J. Phys., 73:730–734, (2005).

86] S. H. Strogatz. Nonlinear dynamics and chaos. (Westview

Press, Cambridge, MA), 1994.
87] D. F. Styer. Quantum revivals versus classical periodicity in the
infinite square well. Am. J. Phys., 69:56–62, (2001).

88] G. J. Sussman and J. Wisdom. Numerical evidence that the

motion of Pluto is chaotic. Science, 241:433–437, (1988).

89] J. Thijssen. Computational physics. (Cambridge University

Press, Cambridge), 2nd edition, 2007.

90] S. R. Valluri, D. J. Jeffrey, and R. M. Corless. Some

applications of the Lambert W function to physics. Can. J.
Phys., 78(9):823–831, (2000).

91] M. C. Vargas, D. A. Huerta, and V. Sosa. Chaos control: The

problem of a bouncing ball revisited. Am. J. Phys., 77:857–861,
(2009).

92] L. Verlet. Computer ‘experiments’ on classical fluids. I.

Thermodynamical properties of Lennard-Jones molecules.
Phys. Rev., 159:98–103, (1967).

93] J. Wang, J Burgdörfer, and A Bárány. Ionization spectrum for

ion-atom collisions with short-ranged potentials in one and
three dimensions. Phys. Rev. A, 43:4036–4039, (1991).

94] R. Warburton and J. Wang. Analysis of asymptotic projectile

motion with air resistance using the Lambert W function. Am.
J. Phys., 72:1404–1407, (2004).
95] P. R. Weissman and G. W. Wetherill. Periodic Trojan-type
orbits in the Earth-Sun system. Astrono. J, 79:404–412,
(1974).

96] H. Wendland. Scattered data approximation. (Cambridge

University Press, Cambridge), 2005.

97] J. Wesson. The science of soccer. (IOP Publishing, Bristol),

2002.

98] F. Yamamoto, Y. Tsuji, and G. Chen. Basic theory and

experiment for the simulation of ball trajectory. Int. J. Tab.
Tenn. Sci., 3:1–15, (1996).
 Index
Page numbers with the “S:” prefix are entries from the Digital
Online Companion.

accelerated relaxation, 339

Gauss-Seidel method, 339

air resistance, 85–89

linear, 91

quadratic, 103

animation

ant raids, S:145

atomic collision, S:228

baseball, realistic flight, 110

bouncing ball, 40

dipole radiation field, 369

electric field hockey, 332

falling tablecloth

mechanical, 316
thermal, 596

game of life, S:138

Halley's comet, S:15

laser-driven coherent state, S:94

magnetic field, 368

Mandelbrot fractals, 254

N-body simulator, 604

oscillation of slinky, S:52

plane electromagnetic wave, 371

planetary motion

Earth, 136

Mercury, 144

precession of Mercury, 148

projectile motion, 84

quantum revival, 2D, 429

quantum scattering from a barrier, S:79

quantum wavepacket

in free fall, 418

in SHO, 408

relaxation of electric potential, 336

shooting for eigenenergies, 461

simple harmonic oscillator, 272

soccer, 117

spin flip, 589, 617

strange butterfly attractor, 246

thermal equilibrium, 626

Thomson problem, S:67

three-body motion

choreography, 168

collinear, 167

Trojan asteroids, 179

wave on a membrane, 313

ants raiding pattern, S:142, S:147

atomic form factor, S:214

elastic, S:220

atomic reaction, S:211–S:221

antiproton impact, S:247

capture, S:229, S:232

cross section, S:230, S:233

Thomas mechanism, S:234

excitation, S:162, S:214

cross section, S:215, S:216, S:230

electron impact, S:245

ionization, S:162, S:218

cross section, S:218, S:230

electron impact, S:248

free-particle model, S:220

positron impact, S:248

atomic structure, S:65, see also hydrogenic atom

atomic units, 402

attractor, 219, 236, 245

dimension, see fractal

band matrix, 416

representation, 416

solver, see SciPy

baseball, 107

animation, 110

curveball, 110

drag coefficient, 89

lift coefficient, 107

basis expansion method, 479

box basis, 480

half-open space, 484

SHO basis, 481

bifurcation, 222, see also chaos

binomial

coefficient, 610

distribution, 537

Bohr model, 490, S:225, S:232

Boltzmann distribution, 576, 587

Bose-Einstein condensation, S:170

chemical potential, S:171

critical temperature, S:172

scattering length, S:206

bound states, 459

central field potentials, 486

double square well, 466

gerade and ungerade, 479

Morse potential, 485

periodic multiple wells, 469

square well, 465

boundary value problem

Dirichlet boundary condition, 297, 335, 471, 493

mixed boundary conditions, S:135

Neumann boundary condition, 296, 335, 373, 374

Brownian motion, 405, 537, 540–544, 554

simulator, 560

C/C++, 17, 37, 406, S:193

catenary, S:47

celestial mechanics, 136

central field approximation, 154, 486, S:165

central field motion, 139

centrifugal

force, 173, 177, 196

potential, 141, 173, 177, 489, S:127

chaos, 227

bifurcation, 225, 240

kicked rotor, S:19, S:27

Lorenz model, 242

Lyapunov exponent, 229

nonlinear driven oscillator, 232

Poincaré map, 237, 247, S:134

Poincaré surface of section, 239

stadium billiard, S:22, S:28

strange attractor, 245, 263

time scale, 236

weather system, 244

chemical potential, S:170

circle

approximation of π, 554

midpoint drawing algorithm, S:71

quantum dot, S:118

classical scattering, see scattering

classical trajectory Monte Carlo, S:221–S:236

animation, S:228

microcanonical ensemble, S:225, S:260

straightline approximation, S:229, S:248

coherent state, 429, S:88

measurement problem, 429

comet, S:15–S:18

Halley's, S:15

ISON, S:15

commutator, S:76

Coriolis effect, 173, 178

coupled channel method, S:86, S:90, S:110

Crank-Nicolson method, 414

Curie's law, 613

Cython, 17, 406, 631, S:110, S:193

debugging, 20

deflection function, see scattering

density of states, S:171, S:258

differentiation operator, 362, see also radial basis function

dipole selection rule, S:93

Dirac δ atom, 476, 510

δ comb, S:129

δ molecule, 477, 506, S:128

cusp, 477

displacement of a string, 286

drag force, see also air resistance

Brownian motion, 540

coefficient, 88

empirical formula, 90

quadratic, 88

viscosity, 87

eccentric anomaly, 162, S:260

eccentricity, 141, 676

Ehrenfest theorem, 411, 438, S:96

eigenvalue problem, 283, 309, 471, 480

generalized, 281, 474, 494, 528

Jacobi transformation, 284

Einstein solid, 564–576

energy distribution, 567

entropy, 571, 573, 614

interacting systems, 573, 614

temperature, 577

electric field hockey, 332

electric potentials and fields, 334

disk in a box, 364

parallel plates, 336

unit square, 356

electromagnetic waves

dipole radiation, 369

plane waves, 371

electrostatic equilibrium

on a sphere, S:65

plum-pudding model, S:65

electrostatic potential energy, 544, 553

energy band, 470

entropy, 570, 623

Einstein solid, 571

heat and temperature, 586

Ising model, 586

paramagnetic system, 613

envelope function, S:92, S:96

equipartition theorem, 555, 607

error, see numerical error

Euler method, 42

Euler rotation, S:261

Euler-Cromer method, 71, 273, 326

evolution operator, 413

approximate, S:77

exoplanets, 158–164

HD 139357, 159

HD 3651, 163, 192

modeling RV datasets, 162

radial velocity method, 158

expectation value, 409, 474, S:90, S:96

F2Py, 17, 406, 631, S:110, S:191, S:193, S:247, S:266

falling tablecloth

mechanical, 315

thermal, 596

fast Fourier transform, 250, S:30–S:44

aliasing, S:40

iterative FFT, S:38

Nyquist frequency, 305, S:44

Parseval's relation, 250

positive and negative frequencies, S:42

recursive FFT, S:34

two-dimensional, 432

wave function, 421

fast multipole method, S:193

finite difference method

displacement of a string, 288

error, 359

Laplace equation, 335

quantum dot, 493

quantum eigenenergies, 470

standing waves, 309

waves on a membrane, 311

waves on a string, 303

finite element method

accuracy, 359

basis functions, 293, 342

building system matrix, 350

data structure, 353

Dirac δ atom, 476, 510

Dirac δ molecule, S:129

displacement of a string, 292

error, 499

FEM library, 495

Laplace equation, 341

mesh file format, 531

mesh generation, 347, 353, 387, 495

mixed boundary conditions, S:135

nodes and elements, 344

Schrödinger equation, 472

stiffness matrix, 298

tent functions, 342, 475

fitting, 163

fixed-point number, 23
floating point, 24

bit shifting, 26

byte string, 25

machine accuracy, 5, 26

mantissa, 4, 24

phantom bit, 25

round-off error, 26

football, 128

Fortran, 17, 285, 406, S:191

Fourier transform, 249, see also fast Fourier transform

fractals, 251

Cantor set, 258

correlation dimension, 254

Hausdorff dimension, 252

Julia set, 259

Koch curve, 252

Mandelbrot fractals, 254

Sierpinski carpet, 259

free fall, 42

animation, 40

Euler's method, 44
momentum profile, 423

quantum mechanical, 413, 418

Runge-Kutta methods, 51

game of life, S:137

Gauss elimination, S:55

Gauss-Jordan method, S:56

Gauss-Seidel method, S:56

Gaussian distribution, 551, S:140

golden mean, 8, 462

recursion, 8

golf, 112

drag and lift, 113

Green's function, S:199

Halton sequence, 364

heat capacity, 577

Heisenberg's uncertainty principle, 305, 401, 422, 501, S:44

Hilda asteroids, see restricted three-body problem

Hooke's law, 145, 278, 320

hydrogen molecule, vibrational states, 486

hydrogenic atom, 486–492

l degeneracy, 487

angular probability density, 491

Hulthén potential, 512

1+∈
modified potential, 1/r , 487, 511

radial equation, 487

radial probability density, 490

radial wave function, 489, S:106, S:256

screened Coulomb potential, 511

shell structure, 490

ideal gas law, 609

importance sampling, 547

installation, 26–28

integral equation, S:198

IPython, 4, 26, 29, 32

IVisual, 19

Matplotlib inline, 17

Ising model, 577–588

1D, 578

2D, 588–595, 618

3D, S:179

antiferromagnetism, 578, S:178, S:187

critical temperature, 591, 594

energy, 583, 591

entropy, 586, 613, 616, 618

computation, 616

exact solution in 2D, 624

ferromagnetism, 578

heat capacity, 585, 593

hysteresis, 618, S:181

magnetization, 583, 591

staggered, 626, S:178

mean field approximation, S:177, S:180, S:181, S:184

partition function, 613

phase transition, 591

spin domains, 589

toward equilibrium, 582

IVisual, 19, 26

Jupiter, see precession of Mercury, see also restricted three-body problem

pull on the Sun, 159

Kansa's method for PDEs, 363, see also radial basis function

Kepler orbit, 141, 160

Kepler's third law, 144

planets, 143

Kepler's equation, 162, S:260

Lagrange points, 175, S:233

Lambert W function, 98–103, 506, S:129

approximate formulas, S:7

Bose-Einstein condensation, S:171

Dirac δ molecule, 478

evaluation, 100

projectile motion, linear drag, 101

laminar flow, 88

Langevin equation, 541, 554

Laplace equation, 335

additivity rule, 376

Laplace operator

nine-point discretization, 376

laser-electron interaction, S:92

strong fields, S:98

leapfrog method, 62–69, 76

area-preserving, 64

space discretized, 403

time dependent, S:261

time transformation, 149

N-body system, S:222

least square fitting, 163, 204

Lennard-Jones potential, 599, S:132

Levinson theorem, 652, 663, 668, 673

lift force, 105, see also Magnus force

linear combination of atomic orbitals (LCAO), 467, 479, S:129

linear interpolation, 125, 510

Lippmann-Schwinger equation, S:200

logistic map, 214–227

bifurcation, 225

Feigenbaum number, 225, 226

fixed points, 218

Lyapunov exponent, 230

period doubling, 222

renormalization, S:28

Lorenz flow, 240, see also chaos

Lyapunov exponent, 229, see also chaos

magnetic field, 366

closed loop, 368

long wire, 368

magnetization, 583, 591, 613

Magnus force, 106

lift coefficient, 107

Matlab, 17

Matplotlib, 17, 30

2D plots, plot, 12, 30–31, 45

3D plots

Axes3D, plot_surface, 211, 384, 389, 395

scatter, 262, S:60

animation, 451, 561, 627, S:148

aspect ratio, 211, 395

axis

label, 12

limit, 206, 269

off, 391

semilog scale, 265

width, 30

bitmap images, imshow, 269, 391, 455, 627, S:150

color, 30

colorbar, 391
configuration records, 31

contour

filled, contourf, 672, S:96

lines, contour, 211, 384, 389

error bar, 206, 569

font size, 30

frame

off, 391

spacing, 30

histogram, 627, S:136

IPython inline, 17

legend, 291, 327, S:114

frame off, S:136

line

style, 30

width, 30, 211

marker, 30

color, S:60

multiple plots, subplot, 267, 269, 455

polar plot, 668

pylab, 17
step plot, step, S:117

text label, 30

math mode, 211

tick marks, 455, 683

triangular mesh plot, tripcolor, 379, 389, 529

triangular mesh surface, plot_trisurf, 528

vector fields, quiver, 211, 384, 392, 559

Maxwell distributions, 605

mean free path, S:161

energy dependent, S:183

Mercury, see precession of Mercury

meshfree method, 360, see also radial basis function

MeshPy, 388, 495

Metropolis algorithm, 579–581, 587–588

model building, 89

molecular dynamics, 598–605, S:222

close-neighbor interaction, 602

equipartition theorem, 607

ideal gas law, 609

initial condition, 604, 631

Maxwell distributions, 606, 621

optimization, 631, S:191

periodic boundary condition, 601–605

pressure, 608

second virial coefficient, 609, 621

units, 600

Monte Carlo integration, 450, 544, 553

error, 547

hit-or-miss method, 554

Monte Carlo simulation

ants, S:145

Einstein solid, 564

nuclear decay, 536

particle transport, S:160

simulated annealing, 596

falling tablecloth, 596

hanging chain, S:153

hanging tablecloth, 619

traveling salesman problem, S:157, S:181

Morse potential, 485, 599, S:131

Navier-Stokes equations, see Lorenz flow

Newton
second law, 39

third law, 105

nuclear decay, 535

Numba, 16, 17, 270, 406, S:110, S:193

numerical differentiation

first order, 43

midpoint method, 48

second order, 288

numerical error, 5

global error, 45

in energy, 68

round-off, 6

truncation, 7

numerical integration, 445, see also Monte Carlo integration

Gaussian, 448, S:107

abscissa and weight, 457

multiple integral, 450

Simpson's rule, 447

trapezoid rule, 446

Numerov's method, 468, 513

first derivative, 514

logarithmic scale, S:127

NumPy, 16, 31–38

advanced indexing, 34

row or column swap, 36, 385, 530

with boolean array, 34, 205, 211, 604, 631

with integer array, 34, 388

argmin, 206

array creation, 31

broadcasting, 33, 111, 330, 604, 630

concatenate, 205, 389, 421, 519, 530, S:43

conversion to list, 531

copy, 33

data type, 31, 417

diagonal, 291, 327, 521, 522

dot product, 152, 388, 521

element insertion or deletion, 32, 291, 327, 388, 396, 523, 528

element-wise operations, 33, 37, 205, 306–307, 312–313, 330, 337, 391, 406, 455, 604,
621, 630, S:58, S:60

F2Py, 17

FFT, S:86

in 2D, 432
flatten, 388

gradient, 211, 384

histogram, 621, S:189

linspace, 205, 210

matrix multiplication, 521

maximum element, S:58

meshgrid, 210

nearest difference, 205

outer method, 205

outer product, 455, S:58

random distribution, 535, 560

reshape, 385, 387, 389

row and column convention, 211, 384

row and column insertion or deletion, 36, 388, S:130

shape, 35

slicing, 32, 211, 306–307, 312–313, 330, 337, 384, 388, 417, 518, S:60

sorting, 389

stacking, S:60

column, 392, 396, 530

depth, 330, 385

summing arrays, 329, 440, 475, 604, 621, 630

take, 389

transpose, 385

truth array, 34, 205, 211, 385, 530

universal functions (ufunc), 16, 36, 37, 210, 475

vector operations, 111

vectorizing functions, 37, 475, 517

object-oriented programming, 14, 560, 565

orbiting, S:195, see also scattering

ordinary differential equation, 39, see also Euler, leapfrog, Numerov, and Runge-Kutta
methods

implicit method, 62

oscillation

damping, 273

resonance, 275

RLC circuit, 273

paramagnetic system, 576, 612, 615

partial differential equation, 271, see also Laplace, Schrödinger, and wave equations

particle transport, S:160

angular scattering, S:182

energy deposition, S:161, S:169

energy-dependent mean free path, S:183

range distribution, S:164, S:168

partition function, 576

harmonic oscillator, 577

phase transition, 591

Ising model, 591

ping pong, 114

spin effects, 116

planetary motion, 136–146

open orbits, 144

properties, 139

simulation, 136

units, 143

Pluto, see restricted three-body problem

Poincaré map, see chaos

Poisson distribution, 536, 557, S:122

Poisson equation, 335

power spectrum, 249, 279

precession of Mercury, 146–158

by other planets, 154

oscillations, 153

relativistic, 147, 152

scaling law, 156

probability density, 412

profiling, 381, 407, S:109

program profiling, S:85

programs list, 685

projectile motion, 83

linear drag, 91

quadratic drag, 103

visualizing, 84

pseudospectral method, S:78

Python, 10

2.7x vs. 3.xx compatibility, 14

assignment and type, 12

complex number, 269, 451

conditional, 13

deep copy, S:148

eval function, 204

exception, 528

file I/O, 14, 204

formatting string, 202, 216

global variables, 13, 52, 465

IDLE, 29

indentation, 12

inline if, 13, 206, 216

input, 12

installation, 26

lambda function, 475, 530

list, 12

append, 12

concatenate, 391, 566

count, 627

delete element, S:148

nested, 387

slicing, 33

sorting, 388

vs. ndarray, 33, 566

online help, 15, 31

operator overloading, 566

pickle file handler, 527

profiling, S:109

random integer, 566

random number, 534

speed boost, 16, 270, 317

with-as statement, 204

quantum chaos, S:117–S:125

chaoticity, S:125

energy level statistics, S:121

nearest neighbor spacing, S:121

histogram, S:124

scars, S:125

spectrum unfolding, S:122

stadium billiard, S:117

quantum dot, 492–503

circle, 512, S:128

degeneracy, 501

energy level distribution, S:117

hexagon, 499

isosceles right triangle, 496

stadium, S:118

triangle, 508

wave function, 498, 502, S:119, S:125

quantum mechanics, see Schrödinger equation

quantum quilt, 443

quantum revival, 432, 434

revival time, 435

semiclassical limit, 437

quantum scattering, 647

T-matrix, S:212

amplitude, 648, 654, S:200, S:211

atomic form factor, S:214

Born approximation, 663, 665, S:201, S:206, S:209, S:213

Buckingham potential, 673

cross section, 654, 658, 664, S:212

elastic, S:245

Fermi's golden rule, S:201

Gaussian potential, 673

hard sphere, 655

shadow effect, 659

Hulthén potential, 673

inelastic, S:211

optical theorem, S:238

partial wave expansion, 652–655

phase shift, 649, 654, 661, 663, S:206, S:208, S:209

potential barrier, S:79

potential well, S:83

Ramsauer-Townsend effect, 674, S:205

resonance, 651, S:84

scattering length, S:203, S:242

square spherical barrier, 660, 668

square spherical well, 668

WKB approximation, S:207, S:209

Yukawa potential, 659, S:166, S:201, S:209

quantum transitions, S:86

amplitudes, S:88

dipole allowed, S:98, S:216

dipole forbidden, S:98, S:216

in hydrogen, S:105

in the SHO, S:104

laser driven, S:91

multiphoton transition, S:98

occupation probability, 425, S:88, S:96

Rabi flopping, 426

two-photon transition, S:98

two-state system, 424

Rabi flopping, 426

in hydrogen, S:106

Rabi frequency, 427

rotating wave approximation, 427

radial basis function, 361

collocation method, 362, S:71

differentiation operator, 363, 364

Gaussian and multiquadric RBF, 361

scattered data interpolation, 361

shape parameter, 361, 365

radial velocity method, see exoplanets

radial velocity transformation, 198

random number, 533

correlation and moment tests, 534

integer, 566

nonuniform distribution, 551, 556

Lorentzian, 551

rejection method, 557

transform method, 556

seed, 534

uniform range, 552

random walk, 537

binomial distribution, 537

in 2D, 538

recursion

stability, 9, 304, S:238

reflection coefficient, S:81, S:83

restricted three-body problem, 171–178

Earth-Moon system, 175

Hilda asteroids, 181, 193

Lagrange points, 175

orbital resonance, 181

Pluto libration, 183, S:14

Pluto's motion, 181

Sun-Jupiter system, 178

Sun-Neptune system, 181

units, 173

Reynolds number, 87

root finding, 94

bisection, 94, 129, 465

false position, S:6

Newton's method, 95, 131, 259

SciPy equation solver, 94, 122, 505

secant method, S:5

rotating frame, 194

rotation matrix, 197

round-off error, see numerical error

Runge-Kutta methods, 46

characteristic time, 61

non-vectorized, 60

SciPy wrapper, 81

step size control, 61

Runge-Kutta-Felhberg method, 62, 514

Runge-Lenz vector, 147, 153

Rutherford scattering, 638, S:66

cross section, 639

Rydberg states, 490, S:236

scattering, 634, see also quantum scattering

cross section, 636, S:253

deflection function, 634, 675, S:208

Yukawa potential, 645

glory, 644, 646, 665

impact parameter, 634, S:197

orbiting, S:195, S:240

plum potential, 641, 680

rainbow, 640, 644, 645, 665, 666, 669

Snell's law, 666, 670

square spherical barrier, 670

square spherical well, 670

Yukawa potential, 645

scattering length, S:203, see also quantum scattering

Schrödinger equation, time dependent, 400, see also wavepacket

average position, 409, 436

boundary effects, 419

coupled channel method, S:86

direct simulation, 403

periodic boundary condition, 407

split evolution operator, S:78

split-operator method, 415

Schrödinger equation, time independent, 460, see also bound states

animated eigenstates, 461

basis expansion method, 479

discrete energies, 311, 463

integral equation, S:199

matching condition, 464

pseudo-continuum states, 471, 472, 481

shooting methods, 463

Schrödinger's cat, 429

SciPy, 16

Airy function and its zeros, 516, 523

band eigenvalue solver, 513, 521, 528

band matrix solver solve_banded, 416

Bessel function

spherical, 660

zeros, S:128

BLAS and LAPACK, 285

combination, 614

eigenvalue solver eigh, 285, 327, 521

elliptic integral, 625

gamma function, 612

Hermite polynomial, 522, S:131

integration, 457, S:243

Lambert W function, 100

least square fitting, 163, 204

Legendre polynomial, 683

linear system solver solve, 290, 291, 388

ODE solvers, 40, 61, 81

orthogonal polynomial, 457

root solver fsolve, 94, 122, 505

sparse eigenvalue solver eigsh, 471, 475, 521, 523, 528

Weave, 17

self-consistent methods, 335, S:48

relaxation error, 340

shooting methods, 118, 463, 465

simple harmonic oscillator

animation, 273

classical, 66

quantum mechanical, 408

Simpson's rule, 411

Snell's law, see scattering

snub cube, S:69

soccer, 116

space discretized leapfrog method, 403, 405

normalization error, 412

stability, 405

sparse matrix, 355

special function
Airy function, 483, 507, 515, S:130

zeros, 516

Bessel function, 500, S:128

modified spherical, 668

recurrence, 677

spherical, 652, 660, 677, S:251

zeros, S:128

elliptic integral, 592, 625

Hermite polynomial, 482, S:131

Laguerre polynomial, S:127

Lambert W function, 478

Legendre polynomial, 448, S:108

in plane waves, 652

spectral staircase, S:115

spherical harmonics, 486, S:105

addition, S:251

in plane waves, S:251

orthogonality, S:251

spinning balls, 105

spin parameter, 107

split evolution operator, S:75, S:78

split-operator method, first order, 413

error, 419

stiff differential equation, 62

Stirling's approximation, 612

Stoke's law, 87

stopping power, S:162

symplectic methods, 69, see also leapfrog method

first order, 71

SymPy, 18, 26, 29, 319

factorization, 257

integrate, 71, 666

Lambert W function, 100

physics

hydrogen atom, S:106

quantum oscillator, 482

series, 186

solve, 123, 505, 506

table tennis, 114, see also ping pong

temperature

Curie, 592

Einstein solid, 576, 577

negative, 576, 616

thermodynamics, 564

second law, 572

third law, 585

Thomson model, S:65

three-body problem, 164–171

choreography, 168

dynamics, S:228, see also classical trajectory Monte Carlo

Euler's collinear motion, 165

Euler's quintic equation, 167

planar motion, 164

traffic flow, S:139

fundamental diagram, S:140

hybrid model, S:146

transmission coefficient, S:81, S:83, S:103

tridiagonal matrix, see band matrix

Trojan asteroids, 178

truncation error, see numerical error

tunneling, S:82, S:234

turbulent flow, 88

unitarity, S:76
Verlet, see leapfrog method

vibration, 277

normal modes, 280

string, 300

triatomic molecules, 277

virial theorem, 489, 512

viscosity, 86–87

air, 122

visualization, 3, 40, 110, 369

von Neumann stability, 304

VPython, 18

arrow, 134, 202, 208, 393

axis flip, 397

box, 19, 41, 133

camera angle, 20

curve, 133

faces, 377

helix, 273

in GlowScript, 30

IVisual, 19

key detection, 134, 187

label, 133, 187, 202, 382, 455

box, 396

light source, 138

make_trail, 138, 382

making movies, 30

opacity, 133, S:67

rate requirement, 19

retain, 85

ring, 212

rotate, 20, 134

sphere, 41

vector operations, 111, 202, 392, 394

VIDLE, 29

VPython modules (VPM), 14, 139, 307, 328, 330, 377, 452, 455, S:52, S:61–S:64

wave function, 400

laser driven, S:96

momentum space, 420

normalization, 412

conservation, 405

plane wave, 401

scarring, S:125
scattering, 656

wavepacket, 408, 418

broadening, 418

in 2D, 432

momentum distribution, 421, 432

optical diffraction, 433

refocusing, 409

scattering from a barrier, S:79

scattering from a well, S:85

self interference, 419

waves, 300–309

on a membrane, 311

standing, 303, 307, 309, S:85

traveling, 301, 307, 323

wave equation, 301, 311

Weave, 17, 406, 631, S:110, S:193

Weyl formula, S:116

Wigner distribution, S:122

WKB approximation, S:207, see also quantum scattering

Yukawa potential, 511, 645, 659, S:201