Scientific Computing

and
Programming Problems

by
Willi-Hans Steeb
International School for Scientific Computing
at
University of Johannesburg, South Africa

Yorick Hardy
Department of Mathematical Sciences
at
University of South Africa, South Africa

George Dori Anescu

email: george.anescu@gmail.com
Preface

The purpose of this book is to supply a collection of problems in matrix calculus.

Prescribed books for problems.

1) Matrix Calculus and Kronecker Product with Applications and C++ Pro-
grams

by Willi-Hans Steeb
World Scientific Publishing, Singapore 1997
ISBN 981 023 2411
http://www.worldscibooks.com/mathematics/3572.html

2) Problems and Solutions in Introductory and Advanced Matrix Calculus

by Willi-Hans Steeb
World Scientific Publishing, Singapore 2006
ISBN 981 256 916 2
http://www.worldscibooks.com/mathematics/6202.html

3) Continous Symmetries, Lie Algebras, Differential Equations and Computer

Algebra, second edition

by Willi-Hans Steeb
World Scientific Publishing, Singapore 2007
ISBN 981-256-916-2
http://www.worldscibooks.com/physics/6515.html

4) Problems and Solutions in Quantum Computing and Quantum Information,

second edition

by Willi-Hans Steeb and Yorick Hardy

World Scientific, Singapore, 2006
ISBN 981-256-916-2
http://www.worldscibooks.com/physics/6077.html

v
The International School for Scientific Computing (ISSC) provides certificate
courses for this subject. Please contact the author if you want to do this course
or other courses of the ISSC.

e-mail addresses of the author:

steebwilli@gmail.com
steeb_wh@yahoo.com
Home page of the author:

http://issc.uj.ac.za

vi
vii
Contents

Preface v

Notation x

1 Quickies 1

2 Bitwise Operations 11

3 Maps and Functions 16

4 Number Manipulations 22

5 Combinatorical Problems 35

6 Matrix Calculus 42

7 Recursion 53

8 Numerical Techniques 61

9 Random Numbers 72

10 Optimization Problems 73

11 String Manipulations 74

12 Programming Problems 76

13 Applications of STL in C++ 82

14 Particle Swarm Optimization 87

Bibliography 93

Index 94

viii
x

Notation
:= is defined as
belongs to (a set)

/ does not belong to (a set)

intersection of sets
union of sets
empty set
N set of natural numbers
Z set of integers
Q set of rational numbers
R set of real numbers
R+ set of nonnegative real numbers
C set of complex numbers
Rn n-dimensional Euclidean space
space of column vectors with n real components
Cn n-dimensional complex linear space
space of column vectors with n complex components
H Hilbert
space
i 1
<z real part of the complex number z
=z imaginary part of the complex number z
|z| modulus of complex number z
|x + iy| = (x2 + y 2 )1/2 , x, y R
T S subset T of set S
ST the intersection of the sets S and T
ST the union of the sets S and T
f (S) image of set S under mapping f
f g composition of two mappings (f g)(x) = f (g(x))
x column vector in Cn
xT transpose of x (row vector)
0 zero (column) vector
k.k norm
x y x y scalar product (inner product) in Cn
xy vector product in R3
A, B, C m n matrices
det(A) determinant of a square matrix A
tr(A) trace of a square matrix A
rank(A) rank of matrix A
AT transpose of matrix A
A conjugate of matrix A
 xi

A conjugate transpose of matrix A

A conjugate transpose of matrix A
(notation used in physics)
A1 inverse of square matrix A (if it exists)
In n n unit matrix
I unit operator
0n n n zero matrix
AB matrix product of m n matrix A
and n p matrix B
AB Hadamard product (entry-wise product)
of m n matrices A and B
[A, B] := AB BA commutator for square matrices A and B
[A, B]+ := AB + BA anticommutator for square matrices A and B
AB Kronecker product of matrices A and B
AB Direct sum of matrices A and B
jk Kronecker delta with jk = 1 for j = k
and jk = 0 for j 6= k
eigenvalue
 real parameter
t time variable
H Hamilton operator

The Pauli spin matrices are used extensively in the book. They are given by
     
0 1 0 i 1 0
1 := , 2 := , 3 := .
1 0 i 0 0 1

In some cases we will also use x , y and z to denote 1 , 2 and 3 .

Chapter 1

Quickies

Problem 1. How would we calculate more efficiently the following functions

(x, x1 , x2 , x3 R)
f1 (x) = 2 sinh(x) cosh(x)
f2 (x) = cosh2 (x) sinh2 (x)
f3 (x) = cos2 (x) sin2 (x)
f4 (x) = ex1 ex2 ex3
1
f5 (x) = 1
1 + ex
1
f6 (x) = q .
1 tanh2 (x)

Problem 2. (i) Let be the frequency and t the time. Simplify

eit eit
eit + eit , .
2i
(ii) Let a, b R and a, b > 0. Can the expression
(a + ib)1/3 + (a ib)1/3
be simplified? Show that the number is actually real. Hint. For the complex
numbers z = a + ib and z = a ib set z = rei and z = rei .
(iii) Let a, b R and a2 6= b2 . Simplify
 
1 i i
+ .
2ia a b a + b

1
2 Problems and Solutions

Problem 3. (i) Let n be a positive integer. Can the calculation of

1 1 1
+ + +
12 23 n(n + 1)

be simplified for computation?

(ii) Let n be a positive integer. Can the calculation of

13 + 23 + + n3

be simplified for computation? Hint. The ansatz

13 + 23 + + n3 = an4 + bn3 + cn2

could be helpful with the coefficients a, b, c to be determined.

p
Problem 4. Can the expression 3 2 2be simplified for computation?
Hint. Let a > 0 and b > 0. Calculate ( a b)( a b) and compare coefficients.

Problem 5. Let n be a positive integer and x, y R. Can the calculation of

n  
X n
xnk y k
k
k=0

be simplified for computation?

Problem 6. The square of the Planck length is defined as

G~
`2P :=
c3
where G is the gravitational constant, ~ is the Planck constant divided by 2
and c is the speed of light. We have in the MKSA-system

m3
G = 6.6732 1011
sec2 kg
kgm2
~ = 1.0545919 1034
sec
m
c = 2.9979250 108 .
sec
Write a C++ program that calculates this quantity. Is the data type double
sufficient? Extend the calculation to find the Planck time interval and the Planck
mass r r
G~ c~
tP = , mP = .
c5 G
 Quickies 3

Problem 7. Let n N. How can the calculation of

x(ex + e2x + + enx )
be simplified for n large?

Problem 8. Let Z be the integer numbers. Let N0 be the natural numbers

including 0. Find a 1 1 map
f : Z Z Z N0
with f (0, 0, 0) = 0 and f (1, 0, 0) = 1. The number of nearest neighbours are 6.
Let (j1 , j2 , j3 ) Z Z Z. Then the six nearest neighbours are
(j1 + 1, j2 , j3 ), (j1 , j2 + 1, j3 ), (j1 , j2 , j3 + 1)
(j1 1, j2 , j3 ), (j1 , j2 1, j3 ), (j1 , j2 , j3 1).
Give a C++ implementation using the Verylong class of SymbolicC++. Give
a Java implementation using the BigInteger class.

Problem 9. (i) Let  R and  > 0. Let v be a nonzero vector in Rn . Assume

that kvk  . Show that
p 
vT v  kvk T .
2v v
(ii) Let x, ` 0 and x  `. Show that
p x2
`2 + x2 ` .
2`

Problem 10. (i) Let N1 , N2 be given positive integers. Let n1 = 0, 1, . . . , N1

1, n2 = 0, 1, . . . , N2 1. There are N1 N2 points. The points (n1 , n2 ) are a
subset of N0 N0 and can be mapped one-to-one onto a subset of N0
j(n1 , n2 ) = n1 N2 + n2
where j = 0, 1, . . . , N1 N2 1. Find the inverse of this map. Consider first the
case N1 = N2 = 2.
(ii) Give a C++ implementation of the map and the inverse.

Problem 11. Let n N. The map f : [0, 2n] [0, 2n] on the integers defined
by
f (0) = n
f (k) = 2n + 1 k for 0 < k n
f (k) = 2n k for n < k 2n
4 Problems and Solutions

plays a role for the converse of Sarkovskiis theorem.

(i) Let n = 2. Starting with 1 find

f (1), f (f (1)), f (f (f (1))), f (f (f (f (1)))), f (f (f (f (f (1))))).

Discuss.
(ii) Give a C++ implementation of this map. The user provides the n.

Problem 12. Show that

 
1 1 1 1
+ = .
a+nk+1 ab+1 nk+b (a b + 1)(n k + b)

Problem 13. Given a vector of length n. Write a C++ program that checks
whether all entries are pairwise different.

Problem 14. The sinc function f : R R

sin(x)
f (x) =
x
can be evaluated using the series expansion
1 1
f (x) = 1 (x)2 + (x)4
3! 5!
However the sinc function could also be evaluated from

x2
Y 
f (x) = 1 2 .
k
k=1

Compare the two methods.

Problem 15. The quadratic equation x2 = x + 1 has the solutions

1 1
= (1 + 5), = (1 5)
2 2
(golden mean numbers). Let k Z. Can the expressions

k1 + k2 , k1 + k2

be simplified?

Problem 16. How would one calculate more efficiently (i.e. minimizing the
number of multiplications) the analytic function f : R R

f (x) = x + 2x2 + 3x3

 Quickies 5

for a given x.

Problem 17. Consider the analytic function f : R R

x
f (x) = .
1 + x2
Simplify the calculation of the integral
Z 2
f (x)dx.
1

Hint. First show that f (x) = f (x).

Problem 18. (S) Solve the quadratic equation 2 + + 1 = 0 by multiplying

this equation with and inserting the quadratic equation and then solving the
resulting cubic equation. Select the solutions from the cubic equation which are
also solutions of the quadratic equation. Note that 1 exp(i2).

Problem 19. Find numerically solutions of the transcendental equation

x
ex + 1=0
5
for x 0.

Problem 20. (i) Let m be a mass, E an energy, a a length and ~ the Planck
constant (divided by 2). Show that
p
2m|E|a
 :=
~
be dimensionless.
(ii) Let e be the charge, E the electric field, m the mass, the frequency and c
the speed of light (all in SI units). Is

eE
=
mc
a dimensionless quantity so that cases such as  1 can be studied?

Problem 21. Let n0 , n1 , n2 N0 . Implement the function

(n0 + n1 + n2 )!
f (n0 , n1 , n2 ) =
n0 !n1 !n2 !
in SymbolicC++ utilizing the Verylong class and Rational class.
6 Problems and Solutions

Problem 22. Calculate efficiently

Z 7 Z 7
sin(x)dx, cos(x)dx.
0 0

Problem 23. (S) Let i, j, k N0 . Find all solutions of i + j + k = 3. Give the

solution in lexicographical order.

Problem 24. The number can be calculated from the expansion

X (j!)2 2j+1
= .
j=0
(2j + 1)!

Let n be positive integer. Give an implementation of the sum

n
X (j!)2 2j+1
s=
j=0
(2j + 1)!

with SymbolicC++ using the Verylong and Rational class and so find an ap-
proximation of .

Problem 25. Let N0 be the set of natural numbers including 0. Let n1 , n2 , n3

N0 . An invertible function f : N0 N0 N0 7 N0 is defined as follows. Let
n = n1 + n2 + n3 . For a fixed n we have f (n) < f (n + 1) and within a fixed n a
lexicographical ordering is assumed. For the first ten elements one has

(0, 0, 0) 0, (0, 0, 1) 1, (0, 1, 0) 2, (1, 0, 0) 3,

(0, 0, 2) 4, (0, 1, 1) 5, (1, 0, 1) 7, (1, 1, 0) 8, (2, 0, 0) 9

Give a C++ implementation of the function f and its inverse f 1 using tem-
plates so that the Verylong class of SymbolicC++ can be used. Give a Java
implementation using the BigInteger class.

Problem 26. Let f L2 (R). Poissons summation formula in one dimension

is given by
+
X + Z
X +
f (n) = f (x)e2iqx dx.
n= q=

Show that if f is an even function of x, then the summation formula can be

written as
+ Z + + Z +
X 1 X
f (n) = f (0) + f (x)dx + 2 f (x) cos(2qx)dx.
n=1
2 0 q=1 0
 Quickies 7

Apply it to the function f (x) = e|x| .

Problem 27. The Catalan constant is defined as

G = 1 32 + 52 72 +

Give a SymbolicC++ implementation using the Verylong and Rational class

to find an approximation of the constant.

Problem 28. Let n Z. Simplify sin(n), cos(2n), cos((2n + 1)).

Problem 29. Consider the normalized vectors

sin(j ) cos(j ) sin(k ) cos(k )
nj := sin(j ) sin(j ) , nk := sin(k ) sin(k )
cos(j ) cos(k )

in R3 . Find the scalar product

nj nk
and simplify it. The scalar product is the angle between the two vectors.

Problem 30. (i) Let u, v be (column) vectors in the Euclidean space Rn .

Now uT , vT are the corresponding row vectors (T denotes transpose) and thus
uT v is the scalar product of u and v. What does
q
A := |(uT u)(vT v) (uT v)2 |

calculate?
(ii) Consider R4 and the vectors

1 1

1 0
u = , v = .
1 0
1 1
Calculate A.

Problem 31. (S) (i) Let u, v be column vectors in Cn and thus u , v

(transpose and complex conjugate) are row vectors. Calculate efficiently

tr(uu vv ).

Note that uu vv is an n n matrix. Could one utilize that matrix multiplica-

tion is associative? Discuss. Is

tr(uu vv ) 0 ?
8 Problems and Solutions

Prove or disprove.
(ii) Let A be a 2 2 matrix. Calculate efficiently tr(A2 ).

Problem 32. Consider the 2 2 matrices

   
0 1 a11 a12
C= , A= , a11 , a12 R.
1 0 a12 a11

Can the expression

A3 + 3AC(A + C) + C 3
be simplified for computation?

Problem 33. Given two invertible n n matrices A and B.

(i) How can we calculate B 1 A1 more efficiently?
(ii) Let be the Kronecker product. How can we calculate B 1 A1 more
efficiently?

Problem 34. (i) Let

x1 y1
x2 , y2 R3 .
x3 y3
What does
1 x1 y1
1
det 1 x2 y2
2
1 x3 y3
calculate?
(ii) Consider the coordinates

p1 = (x1 , y1 , z1 )T , p2 = (x2 , y2 , z2 )T , p3 = (x3 , y3 , z3 )T

with p1 6= p2 , p2 6= p3 , p3 6= p1 . We form the vectors

x2 x1 x3 x1
v21 = y2 y1 , v31 = y3 y1 .
z2 z1 z3 z1

Let be the vector product. What does

1
|v21 v31 |
2
calculate? Apply it to p1 = (0, 0, 0)T , p2 = (1, 0, 1)T , p3 = (1, 1, 1)T .

Problem 35. What is the output of the following C++ program

 Quickies 9

// whileloop.cpp

#include <iostream>
using namespace std;

int main(void)
{
int x = 0; int t = 0; int p = 0;
while(t < 100) {
if(p==0) x = x + 2;
if(p==1) x = x - 1;
p = 1 - p;
t++;
} // end while
cout << "x = " << x << endl;
cout << "p = " << p << endl;
cout << "t = " << t << endl;
return 0;
}

Problem 36. The surface area of a torus with inner radius a and outer radius
b is a given by
A = 2 (b2 a2 ).
The formula for the volume of a torus is given by

2
V = (a + b)(b a)2 .
4
Simplify the calculation of V given A.

Problem 37. Let a, b be non-negative integers.

(i) Simplify the expression

q q
E1 = a + b + a b.

(ii) Simplify the expression

q q
E2 = a+ b a b.

Problem 38. Let x [1, 1]. Simplify arcsin(x) + arccos(x).

Problem 39. Simplify

f (, , ) = sin( + ) + sin( + ) + sin( + ) sin( + + ).

10 Problems and Solutions

Problem 40. Let f : R R be an analytic function. Consider the central

difference operator defined by
1 1
(f (x)) := f (x + h) f (x h)
2 2
where h > 0 is the step length. The operator is linear. Find ((f (x)).

Problem 41. Consider the coordinates in R3

p1 = (x1 , y1 , z1 ), p2 = (x2 , y2 , z2 ), p3 = (x3 , y3 , z3 )

with p1 6= p2 , p2 6= p3 , p3 6= p1 . We form the two vectors

v = p2 p1 , w = p3 p1 .

What does 21 |v w| calculate?

Problem 42. Let n be a positive integer. Give a C++ implementation of sum

n X
n  
X k
`
k=0 `=0

using templates so that the Verylong class of SymbolicC++ can be used.

Problem 43. Let x R. Show that

1 1
1 x .
ex + 1 e +1
Chapter 2

Bitwise Operations

Problem 1. Let x, y {0, 1}. Show that the NOT-gate, AND-gate, OR-
gate, NAND-gate, NOR-gate and XOR-gate can be expressed using arithmetic
operations (i.e. addition, subtraction, multiplication).

Problem 2. Consider a binary nn matrix, where we count the entries from 0.

We have b00 = 1 and bn1n1 = 1. The other 0-1 entries are generated randomly.
An ant at entry (0, 0) can only move to the right or down (not diagonal) when
this entry contains a 1. Write a C++ program that checks whether the ant could
reach the entry (n 1, n 1). For example, consider the matrix
1 0 0 0 0

1 1 1 0 0
1 0 1 0 1.

0 0 1 1 0

0 0 0 1 1
For this case one path

(0, 0) (1, 0) (1, 1) (1, 2) (2, 2) (3, 2) (3, 3) (4, 3) (4, 4)

would be possible. Note that the ant could also get stuck at (2, 0).

Problem 3. Linear feedback shift registers play a role in the generation of

pseudo-random numbers. A linear feedback shift register is a shift register whose
input is a linear function of its previous state. What is the output of the following
C++ program?
// lfsr.cpp

11
12 Problems and Solutions

#include <iostream>
using namespace std;

int main(void)
{
unsigned short i = 0xACE1u; // hex notation
cout << "i = " << i << endl;
unsigned short b;
unsigned short counter = 0u;

do
{
b = ((i)^(i >> 2)^(i >> 3)^(i >> 5)) & 1;
i = (i >> 1) | (b << 15);
counter++;
cout << "b = " << b << endl;
cout << "i = " << i << endl;
}
while(i != 0xACE1u);
cout << "leaving while-loop" << endl;
cout << "b = " << b << endl;
cout << "i = " << i << endl;
cout << "counter = " << counter << endl;
return 0;
}
Note that ^ is the XOR-operation, | is the OR-operation and & is the AND-
operation. Note that unsigned short has 16 bits.

Problem 4. A boolean function f : {0, 1}n {0, 1} (xj {0, 1}, j = 1, . . . , n)

can be transformed from the domain {0, 1} into the spectral domain by a linear
transformation
Ty = s
where T is a 2n 2n orthogonal matrix, y = (y0 , y1 , . . . , y2n 1 )T is the two-
valued ({+1, 1} with 0 1, 1 1) truth table of the boolean function and
spectral coefficients sj . Since T is invertible we have
T 1 s = y.
For T we select the Hadamard matrix. The (n + 1) (n + 1) Hadamad matrix
is recursively defined as
 
H(n 1) H(n 1)
H(n) = , n = 1, 2, . . .
H(n 1) H(n 1)
with H(0) = (1) ((1 1) matrix). The inverse of H(n) is given by
1
H(n)1 = H(n).
2n
 Bitwise Operations 13

Now any boolean function can be expanded as the arithmetical polynomial

1
f (x1 , . . . , xn ) = (2n s0 s1 (1)xn s2 (1)xn1 s2n 1 (1)x1 x2 xn
2n+1
where denotes the modulo-2 addition and the

(s0 , s1 , . . . , s2n 1 ) = s

are the spectral coefficients. Consider the boolean function f : { 0, 1 }3 { 0, 1 }

f (x1 , x2 , x3 ) = x1 x2 x3 + x1 x2 x3 + x1 x2 x3 .

Find the truth table, the vector y and then using H(3) calculate the spectral
coefficients sj (j = 0, 1, . . . , 7).

Problem 5. Analogously to the Hamming distance for finite sequences, a

metric can be used to compute distances between infinite u(j) and v(j), where
jZ
X |u(j) v(j)|
d(u, v) := .
jZ
2|j|

Consider the infinite alternating sequences

...010101010... = u
...101010101... = v
^
|
0
Find the distance between u and v.

Problem 6. Let x, y {0, 1} and the AND operation. Is the circuit

x0 = x, y0 = x y

a reversible gate?

Problem 7. Consider the unit cube with the vertices

(0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), (1, 0, 1), (1, 1, 0), (1, 1, 1)

The majority gate is given by

(0, 0, 0) 7 0, (0, 0, 1) 7 0
(0, 1, 0) 7 0, (0, 1, 1) 7 1
(1, 0, 0) 7 0, (1, 0, 1) 7 1
(1, 1, 0) 7 1, (1, 1, 1) 7 1.
14 Problems and Solutions

Find the boolean expression.

Problem 8. Consider the truth table

(0, 0, 0) 7 1, (0, 0, 1) 7 0
(0, 1, 0) 7 0, (0, 1, 1) 7 0
(1, 0, 0) 7 0, (1, 0, 1) 7 0
(1, 1, 0) 7 0, (1, 1, 1) 7 1.

Find the boolean expression.

Problem 9. Let x0 , y0 {0, 1}. Solve the system of boolean equations

xt+1 = xt yt , yt+1 = xt yt

where x0 = 0, y0 = 1 and t = 0, 1, . . .. Here denotes the XOR operation and

denotes the AND operation. First find the fixed points, i.e. solve x y = x,
x y = y. Does the sequence xt , yt tend to a fixed point?

Problem 10. (i) Let s1 (0), s2 (0), s3 (0) {+1, 1}. Study the time-evolution
(t = 0, 1, 2, . . .) of the coupled system of equations

s1 (t + 1) = s2 (t)s3 (t)
s2 (t + 1) = s1 (t)s3 (t)
s3 (t + 1) = s1 (t)s2 (t)

for the eight possible initial conditions, i.e. (i) s1 (0) = s2 (0) = s3 (0) = 1, (ii)
s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (iii) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (iv)
s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (v) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (vi)
s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (vii) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1,
(viii) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1. Which of these initial conditions are
fixed points?
(ii) Let s1 (0), s2 (0), s3 (0) {+1, 1}. Study the time-evolution (t = 0, 1, 2, . . .)
of the coupled system of equations

s1 (t + 1) = s2 (t)s3 (t)
s2 (t + 1) = s1 (t)s2 (t)s3 (t)
s3 (t + 1) = s1 (t)s2 (t)

for the eight possible initial conditions, i.e. (i) s1 (0) = s2 (0) = s3 (0) = 1, (ii)
s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (iii) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (iv)
s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (v) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (vi)
s1 (0) = 1, s2 (0) = 1, s3 (0) = 1, (vii) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1,
(viii) s1 (0) = 1, s2 (0) = 1, s3 (0) = 1. Which of these initial conditions are
fixed points?
 Bitwise Operations 15

Problem 11. Let x1 (0), x2 (0), x3 (0) {0, 1} and let be the XOR-operation.
Study the time-evolution (t = 01, 2, . . .) of the coupled system of equations

x1 (t + 1) = x2 (t) x3 (t)
x2 (t + 1) = x1 (t) x3 (t)
x3 (t + 1) = x1 (t) x2 (t)

for the eight possible initial conditions, i.e. (i) x1 (0) = x2 (0) = x3 (0) = 0, (ii)
x1 (0) = 0, x2 (0) = 0, x3 (0) = 1, (iii) x1 (0) = 0, x2 (0) = 1, x3 (0) = 0, (iv)
x1 (0) = 1, x2 (0) = 0, x3 (0) = 0, (v) x1 (0) = 0, x2 (0) = 1, x3 (0) = 1, (vi)
x1 (0) = 1, x2 (0) = 0, x3 (0) = 1, (vii) x1 (0) = 1, x2 (0) = 1, x3 (0) = 0, (viii)
x1 (0) = 1, x2 (0) = 1, x3 (0) = 1. Which of these initial conditions are fixed
points?

Problem 12. Show that one Fredkin gate is sufficient to implement the XOR
gate.

Problem 13. Show that a b = a + b using (i) truth tables and (ii) properties
of boolean algebra (with a + 1 = 1).
Chapter 3

Maps and Functions

Problem 1. The straight line Hough transform maps a line in R2 into a point
in the Hough transform space. The polar definition of the Hough transform is
based on the representation of the lines by the parameters (, ) via the equation

= xj cos() + yj sin().

with 0 and [0, 2). All points (xj , yj ) of a given line correspond to a point
(, ) in the Hough transform space. Any point (xj , yj ) is mapped to a sinusoidal
curve in the Hough transform space. Consider the two points (x0 , y0 ) = (1, 0)
and (x1 , y1 ) = (0, 1) on a line. Find , .

Problem 2. Let f, g : R R be analytic functions and n 1. Then

Z b b b b Z b
f (n) gdx = f (n1) g  f (n2) g 0  + f (n3) g 00  (1)n f g (n) dx.
  
a a a a a

Here f (n) denotes the n-th derivative. This identity is called generalized inte-
gration by parts. Let  > 0. Find
Z 1
ex xn dx
0

using generalized integration by parts.

Problem 3. Show that expanding the function


sin(2x) x [0, 1]
f (x) :=
0 otherwise

16
 Maps and Functions 17

on the Hilbert space of square integrable functions L2 (R) in terms of the Haar
basis
{ j,k (x) := 2j/2 (2j x k) : j, k Z }
where
1 x [0, 1/2]
(x) := 1 x (1/2, 1] .
0 otherwise

yields the expansion

2j 1
" #
2 k + 12


1 XX j 2 (k + 1) 2k
f (x) = 2 2 2 cos cos cos j
2 j=0 2j 2j 2
k=0

by considering j 1 and j 0 separately.

Problem 4. Find a polynomial

p(x) = ax4 + bx3 + cx2 + dx + e

which satisfies the conditions

p(0) = 0, p(1) = 0, p(1/2) = 0

p(1/4) = 1, p(3/4) = 1/2.

Problem 5. Let A, B and C be arbitrary non-empty sets and let f : A B

and g : B C. The composite function of f and g is the function

g f : A C, (g f )(x) = g(f (x)).

Notice that g f reads from right to left; it means first apply f , then apply g to
the result. Note that function composition is associative.
(i) Let f : R R, f (x) = x2 , and g : R R, g(x) = 3x 1. Find g f and
f g.
(ii) Write a C++ program which implements these compositions with x of data
type double.

Problem 6. Let f1 , f2 be continuous functions over an interval [a, b]. Then

we have the identities
1
min(f1 , f2 ) (f1 + f2 |f1 f2 |)
2
1
max(f1 , f2 ) (f1 + f2 + |f1 f2 |).
2
Write a C++ program that finds the min and max for two given continuous
functions f1 and f2 using the function
18 Problems and Solutions

void minmax(double (*f1)(double),double (*f2)(double),

double x,double& min,double& max)
where x is the function parameter. Apply it to the sine function and cosine
funtion in the interval [0, 2].

Problem 7. Given a smooth surface in the Euclidean space R3 described by

x1 (u, v)
x(u, v) = x2 (u, v) .
x3 (u, v)
The Gaussian curvature is calculated as follows. First we calculate E(u, v),
F (u, v), G(u, v) of the first fundamental form
x x x x x x
E(u, v) = , F (u, v) = , G(u, v) =
u u u v v v
where denotes the scalar product. Next we calculate
x x
u
n+ (u, v) :=  x v 
x 

u v

where denotes the vector product. Using n+ we calculate L(u, v), M (u, v),
N (u, v) of the second fundamental form
2x 2x 2x
L(u, v) = n+ , M (u, v) = n+ , N (u, v) = n+ .
u2 uv v 2
Then the Gaussian curvature K(u, v) is given by
LN M 2
K := .
EG F 2
Write a SymbolicC++ program that calculates K and apply it to the Mobius
band given by
(2 v sin(u/2)) sin(u)
x(u, v) = (2 v sin(u/2)) cos(u) .
v cos(u/2)

Problem 8. Consider the two membership functions f : R [0, 1], g : R

[0, 1] in fuzzy logic
2
f (x) = ex /2
, g(x) = 1/(1 + ex ).
Write a C++ program that finds the algebraic sum (page 524, Nonlinear Work-
book 5th edition).

Problem 9. What is the output of the following C++ program

 Maps and Functions 19

// fcomposition.cpp

#include <iostream>
#include <cmath>
using namespace std;

double f(double x) { return x*x; }

double g(double x) { return 3.0*x-1.0; }

double comp(double (*f)(double),double (*g)(double),double x)

{ f(g(x)); }

int main(void)
{
double x = 2.5;
cout << "f(" << x << ") = " << f(x) << endl;
cout << "g(" << x << ") = " << g(x) << endl;
cout << comp(f,g,x) << endl;
return 0;
}

Problem 10. Let N0 be the set of natural numbers including 0. The Cantor
pairing function f : N0 N0 N0 is defined by
1
f (x, y) = y + (x + y)(x + y + 1).
2
(i) Find the inverse function, i.e. given s = f (x, y) find x and y. Set
1 2
a := x + y, (a + a).
b :=
2
(ii) Give a C++ implementation utilizing Verylong of SymbolicC++.

Problem 11. Consider the mathematical expression

sin(b) + a b + c d + (a b).
|{z}
Write this mathematical expression as a binary tree with the root indicated by
the brace. Then evaluate this binary tree from bottom to top with the values
a = 2, b = /2, c = 4, d = 1.

voluntary. An alternative to represent a mathematical expression as tree is mul-

tiexpression programming (see next page). Use multiexpression programming
to evaluate the expression.

Problem 12. (i) Let r > 0 (fixed) and x > 0. Consider the map
1 r
fr (x) = x+
2 x
20 Problems and Solutions

or written as difference equation

 
1 r
xt+1 = xt + , t = 0, 1, 2, . . . x0 > 0.
2 xt
Find the fixed points of fr . Are the fixed points stable?
(ii) Let r = 3 and x0 = 1. Find limt xt . Discuss.

Problem 13. Let A be a given 3 3 matrix over R with det(A) 6= 0. Is the

transformation
a11 x + a12 y + a13
x0 (x, y) =
a31 x + a32 y + a33
a21 x + a22 y + a23
y 0 (x, y) =
a31 x + a32 y + a33
invertible? If so find the inverse.

Problem 14. Let N1 , N2 , N3 be positive integers. Let n1 = 0, 1, . . . , N1 1,

n2 = 0, 1, . . . , N2 1, n3 = 0, 1, . . . , N3 1. There are N1 N2 N3 points and the
points (n1 , n2 , n3 ) are a subset of N0 N0 N0 and can be mapped one-to-one
onto a subset of N0

j(n1 , n2 , n3 ) = (n1 N2 + n2 )N3 + n3

where j = 0, 1, . . . , N1 N2 N3 1. Consider first the case N1 = N2 = N3 = 2.

Problem 15. Give an implementation of the function

1 X i i i
ji11 ij22...i
...jk :=
k
sgn()j(1) j(2) j(k) .
k! 1 2 k
Sk

For example
ij 1 i j
k` = ( `i kj ).
2 k `

Problem 16. Let i0 , i1 , . . . , ik1 N0 . Given a vector (i0 , i1 , . . . , ik1 ). Give

an implementation of the function

1 for i0 > i1 > > ik1
i0 ,i1 ,...,ik1 = 1 for i0 < i1 < < ik1
0 otherwise

Problem 17. Let n1 , n2 , n3 N. Consider the equation

1 1 1 1
+ + = .
n1 n2 n3 2
 Maps and Functions 21

Write a SymbolicC++ program utilizing the class Rational and Verylong to test
whether there are solutions for n1 , n2 , n3 {1, 2, . . . , 50}.
Chapter 4

Number Manipulations

Problem 1. Let p be a prime number (base 10) with p > 2. Convert the
prime number into binary. Then consider this number in base 10. Test whether
this number is prime again. For example, consider the prime number 5. Then
in binary we have 101. Now 101 (base 10) is a prime number.
(i) Study this question for the first 10 prime numbers.
(ii) Write a C++ program that could do the job.

Voluntary. Study the same question for base 3.

Problem 2. (i) Consider the first 10 prime twins, i.e.

(3, 5), (5, 7), (11, 13), (17, 19), (29, 31),

(41, 43), (59, 61), (71, 73), (101, 103), (107, 109)
Let (p1 , p2 ) be a set of prime twins. Is

p1 p2 (p1 + p2 )

a prime numbers? Test for the first ten prime twins.

(ii) Write a C++ program that implements this test.

Problem 3. For elliptic curve cryptography we consider elliptic curves that

are defined over a finite field, i.e the elliptic group mod p, where p is a prime
number. This is defined as follows. Choose two nonnegative integers, a and b,
less then p that satisfy
(4a3 + 27b2 ) mod p 6= 0. (1)

22
 Number Manipulations 23

Then EP (a, b) denotes the elliptic group mod p whose elements (x, y) are pairs
of non-negative integers less than p satisfying

y 2 x3 + ax + b (mod p)

together with the point at infinity. Let p = 23 and consider the elliptic curve
y 2 = x3 + x + 1.
(i) Show that the condition (1) is satisfied.
(ii) To find the points in EP (a, b) one proceeds as follows:
1. For each x such that 0 x < p, calculate x3 + ax + b (mod p).
2. For each x from step 1 determine if it has a square root mod p. If not, there
are no points in EP (a, b) with this value of x. If so, there will be two values of
y that satisfy the square root operation root operation (unless the value is the
single y value of 0). These (x, y) values are points in EP (a, b). Find the points
in E23 (1, 1).

Problem 4. Apply the Chinese remainder theorem to solve the set of equations

x 7 (mod 8)
x 2 (mod 9)
x 1 (mod 5).

Chinese remainder theorem. Suppose that the positive integers m1 , m2 , . . . , mt

are relatively prime in pairs; that is, gcd(mi , mj ) = 1 if i 6= j, 1 i, j t. Let
b1 , b2 , . . . , bt be arbitrary integers. Then the congruences

x b1 (mod m1 )
x b2 (mod m2 )
..
.
x bt (mod mt )

have a simultaneous solution. Moreover, the simultaneous solution is unique

modulo m1 m2 mt . That is, if y is another solution, then x y (mod m1 m2 mt ).

To find x we write it in the form

x = y1 b1 + + yt bt

where y1 1 (mod m1 and y1 0 (mod mi (2 i t), y2 1 (mod m2 )

and y2 0 (mod mi ) (i = 1, 3, 4, . . . , t) and similarly for y3 , . . . , yt . To have
y1 0 (mod mi ) (2 i t) we must have m2 m3 mt |y1 , since the mi are
relative prime in pairs. Thus, in general, set
m1 m2 mt
m0i = .
mi
24 Problems and Solutions

Then gcd(mi , m0i ) = 1 since m1 , m2 , . . . , mt are relatively prime in pairs. Thus

m0i has an arithmetic inverse m0i mod mi , i.e.

m0 0
i mi = 1 (mod mi ).

We set yi = m0 0
i mi and correspondingly set

x = m0 0 0 0
1 m1 b1 + + mt mt bt .

We have x b1 (mod m1 ) since for 2 i t we have m1 |m0i so that m0 0

i m i bi
0 0 0 0
0 (mod m1 ) for 2 i t. We also have m1 m1 1 (mod m1 ) so that m1 m1 b1
b1 (mod m1 ). Thus

x b1 + 0 + + 0 b1 (mod m1 ).

It follows similarly that x = bi (mod mi ) for all i, 1 i t.

Problem 5. Let p and q be prime numbers with p, q 3 and p 6= q. Let

n = pq. Assume that d, e N be two integer numbers with the properties

3 < e < (p 1)(q 1)

de = 1 mod (p 1)(q 1)
gcd(e, n) = gcd(e, (p 1)(q 1)) = 1.

With these properties we can prove that for M { 0, 1, 2, . . . , n 1 } the defini-

tion
C := M e mod n
yields M = C d mod n. Consequently

C = C ed mod n

(i) Let s N such that C = C ed mod n. Show that

M = C s mod n.

(ii) Show how the order r of C under modulo n arithmetic can be used to obtain
a linear diophantine equation for s.
(iii) Let p = 3, q = 17, e = 5 and M = 10. Find s and d.

Problem 6. Let F be a field (F = R, C). Consider polynomials. The division

algorithm is as follows. Let g be a nonzero polynomial in F[x]. Then every p in
F[x] can be written as
p = qg + r
where q anbd r are in F[x], and either r = 0 or deg(r) < deg(g). Furthermore q
and r are unique and can be found by the following algorithm
 Number Manipulations 25

input: p, g
output: q, r
q := 0; r := p
while(r <> 0) and LT(g) divides LT(r)) do
q := q + LT(r)/LT(g)
r := r - (LT(r)/LT(g))

where LT is the leading term, i.e. the term with the highest degree. Apply the
division algorithm to

p(x) = x4 1, g(x) = x3 x2 + x 1.

Write a C++ program using SymbolicC++ that implements the algorithm.

Problem 7. Consider the bijective spiral map on page 79 (problem 19). Can
we find an explicit expression for f ? Could a polynomial ansatz work
N
X
f (x, y) = cij xi y j , (x, y) Z Z.
i,j=0

Problem 8. Let x Z. Consider the equation

5x2 + 9x + 11 0 (mod 13).

Write a C++ program that checks whether there are solutions in the range
20 x 20.

Problem 9. What is the output of the following C++ program

// successor.cpp

#include <iostream>
using namespace std;

template<int n> class number

{
private:
number<n-1> predecessor;
public:
number<n+1> successor(void)
{ return number<n+1>(); }

ostream& output(ostream& o)
{ o << "{" << predecessor << "}"; return o; }
}; // end class number
26 Problems and Solutions

class number<0>
{
public:
number<1> successor(void)
{ return number<1>(); }
}; // end class number<0>

template <int n> ostream& operator << (ostream& o,number<n> n)

{ return n.output(o); }

ostream& operator << (ostream& o,number<0> n)

{ o << "{ }"; return o; }

int main(void)
{
number<0> zero;
cout << zero << endl;
cout << zero.successor() << endl;
cout << zero.successor().successor() << endl;
return 0;
}

Problem 10. The following program uses the Verylong class of Symbol-
icC++. What is the program doing?
// wormell.cpp

#include <iostream>
#include "verylong.h"
using namespace std;

int main(void)
{
Verylong two("2");
for(Verylong x("3");x<=Verylong("100");x+=2)
{
Verylong p("1");
Verylong t = x/two;
for(Verylong a("2");a<=t;a++)
{ for(Verylong b("2");b<=t;b++) { p = p*(x-a*b); } }
cout << "x = " << x << " " << "p = " << p << endl;
} // end x for loop
return 0;
}

Problem 11. What is the output of the following program?

// cypher.cpp
 Number Manipulations 27

#include <iostream>
#include <string>
using namespace std;

int main(void)
{
string input = "PLEASE CONFIRM RECEIPT 471";
string output;
char t = 3;

for(int i=0;i<input.length();i++)
{
if((a <= input[i]) && (input[i] <= z))
output += (input[i] - a + t)%26 + a;
else if((A <= input[i]) && (input[i] <= Z))
output += (input[i] - A + t)%26 + A;
else if((0 <= input[i]) && (input[i] <= 9))
output += (input[i] - 0 + t)%10 + 0;
else output += input[i];
}
cout << "output = " << output << endl;
return 0;
}

Problem 12. Let n be a positive integer. We define the set Zn as the set of
nonnegative integers less than n

Zn := { 0, 1, 2, . . . , (n 1) }.

This is referred to as the set of residues modulo n. If we perform modular

arithmetic within this set the following properties hold

Commutative laws (w + x) mod n = (x + w) mod n

(w x) mod n = (x w) mod n
Associative laws ((w + x) + y) mod n = (w + (x + y)) mod n
((w x) y) mod n = (x (w y)) mod n
Distributive law (w (x + y)) mod n = ((w x) + (w y)) mod n
Identities (0 + w) mod n = w mod n
(1 w) mod n = w mod n

and we have an additive inverse (w), i.e. for each w Zn there exists a z such
that w + z = 0 mod n. Note that
if (a*b) = (a*c) mod n then b = c mod n if a is relatively
prime to n
28 Problems and Solutions

Write a C++ program using templates that implements modular arithmetic.

Problem 13. The change problem is as follows. Convert some amount of

money M into given denominations, using the smallest possible number of coins.
This means the input is an amount of money M and an array of d denominations
c = (c0 , c1 , . . . , cd1 ) in decreasing order of value, i.e. c0 > c1 > > cd1 ,
where cd1 = 1. The output is an array of d integers i0 , i1 , . . . , id1 such that

c0 i0 + c1 i1 + + cd1 id1 = M

and i0 + i1 + + id1 is a small as possible. The pseudocode is as follows

smallestNumberOfCoins = M

for each (i0 , . . . , id1 ) from (0, . . . , 0) to (M/c0 , . . . , M/cd1 )

Pd1
valueOfCoins = k=0 ik ck

if valueOfCoins = M
Pd1
numberOfcoins = k=0 ik

if numberOfCoins < smallestNumberOfCoins

smallestNumberOfCoins = numberOfCoins

bestChange = (i0 , i1 , . . . , id1 )

return array bestChange

Write the pseudocode as a C++ program.

Problem 14. Write a C++ program using Verylong of SymbolicC++ that

finds all integer solutions of

2437x + 51329y = 1 x, y Z

in the range x, y [100000, 100000].

Problem 15. Let n be a positive integer to be tested for primality. Let a be

an integer less than n. The following algorithm due to Miller and Rabin tests n
for primality. Write the algorithm as a C++ program using templates so that
also the Verylong class of SymbolicC++ can be used.
primality(a,n)
1. let b_k b_{k-1}...b_0 be the binary representation of (n-1)
 Number Manipulations 29

2. d <- 1
3. for i <- k downto 0
4. do x <- d
5. d <- (d*d) mod n
6. if d = 1 and x neq 1 and x neq n-1 then return true
7. if b_i = 1 then d <- (d*a) mod n
8. if d neq 1 then return true
9. return false

Problem 16. We define a mapping from the natural numbers N0 to sets as

0 { }, n + 1 { n }.

Give a C++ implementation of this representation of natural numbers.

Problem 17. Let m and n be positive integers. We define the map NN N

the lowest common multiple of m and n

mn := .
the highest common factor of m and n

For example
60
1230 = = 10.
6
Is the composition associative, i.e.

(mn)p = m(np) ?

Write a C++ program using templates (so that Verylong of SymbolicC++ can
also be used) that implements this composition.

Problem 18. Consider the following arithmetic problem

# # * #
--------
# #
+ # #
----
# #

where denotes multiplication and + denotes addition. Each # should be one

of the digit 1,2,3,4,5,6,7,8,9, where the condition is that each digit occurs only
once. Write a C++ program that finds all the solutions.

Problem 19. Show that any positive integer n can be written uniquely as

n = 2j + k
30 Problems and Solutions

where 0 k < 2j . For example 10 = 23 + 2. Write a C++ program that

finds j and k for a given n. Use templates so that also the Verylong class of
SymbolicC++ can be used.

Problem 20. Give a C++ implementation of modular arithmetic using the

Verylong class of SymbolicC++. Let a, b Z. Recall that
a mod b
denotes the remainder obtained by dividing integer b into integer a, which is a
number less than b. Thus a = mod b if a = b + kn for some integer k. This is
expressed by saying that a is congruent to b modulo n or that b is the residue of
a modulo n. Modular arithmetic is commutative, associative, and distributive
(a b) mod = ((a mod n) (b mod n)) mod n
(a b) mod = ((a mod n) (b mod n)) mod n
(a (b + c)) mod = (((ab) mod n) + ((ac) mod n)) mod n

Problem 21. Let n be a positive integer. There are exactly as many irre-
ducible representations of the permutation group Sn (order of Sn is n!) as there
are partitions { pj } of n
n
X
pj = n, p1 p2 pn 0.
j=1

For example for n = 4 we have 5 partitions with

4000, 3100, 2200, 2110, 1111.
The number of partitions is given by p(1, k), where p(k, n) represents the number
of partitions of n using only natural numbers at least as large as k. A recursion
relation for p(k, n) is given by

0 if k>n
p(k, n) = 1 if k=n
p(k + 1, n) + p(k + 1, n) + p(k, n k) otherwise

(i) Give a C++ implementation of this recursion.

(ii) Give a C++ implementation that finds the partitions for a given n without
the trailing 0s.

Problem 22. Consider the sequence (Ballot numbers)

B0 = B1 = 1,
L2
X
BL = BL1 + B` BL2` , L = 2, 3, . . . .
`=0
 Number Manipulations 31

(i) Give a C++ implementation of this recursion.

(ii) Show that the generating function

X
B(x) = BL xL
L=0

is given by
x2 (B(x))2 (1 x)B(x) + 1 = 0.

Problem 23. A perfect number is a natural number with the properties that
the sum of the factors gives twice the number, for example

26=1+2+3+6

so that 6 is a perfect number. A Mersenne prime is a Mersenne number 2n 1,

where n is chosen such that 2n 1 is prime (for example n = 5).
(i) Show that 2n1 (2n 1) is perfect. For example for n = 5 we have 496 with

496 = 248 + 124 + 62 + 31 + 16 + 8 + 4 + 2 + 1.

(ii) Prove that if 2n1 p is perfect for p prime then p is a Mersenne prime and
p = 2n 1.
(iii) Prove that for any even perfect number q there exists n N such that
q = 2n1 (2n 1).

Problem 24. Consider the one-dimensional map (logistic map) f : [0, 1]

[0, 1]
f (x) = 4x(1 x).
A computational analysis using a finite state machine with base 2 arithmetic in
fixed point operation provides one-dimensional maps with a lattice of 2N sites
labeled by numbers
N
X j
x= , j { 0, 1 }
j=1
2j

and N defines the machines precision. Consider N = 8 bits and x = 1/8.

Calculate the orbit f (x), f (f (x)), f (f (f (x))), . . . with this precison. Discuss.

Problem 25. Find the solution of the system

5x = 2 mod 3
4x = 7 mod 9
2x = 4 mod 10.
32 Problems and Solutions

Problem 26. Let n be a postive integer. A numerical partition of n is a

sequence
p1 p2 pk 1
such that
p1 + p2 + + pk = n.
Each pj is called a part. For example

18 = 7 + 4 + 4 + 1 + 1 + 1

is a partition of the integer 18 into 6 parts. The number of partitions of n into

k parts is denoted by p(n, k).
(i) Find p(7, 3).
(ii) Show that the recurrence for p(n, k) is given by

p(n, k) = p(n 1, k 1) + p(n k, k)

with the initial conditions p(n, 0) = 0, p(k, k) = 1. Obviously we have p(n, 1) =

1.
(iii) Write a C++ or Java program that implements this recurrence.
(iv) Every numerical partition of n corresponds to a unique Ferrers diagram.
A Ferrers diagram of a partition is an arrangement of n dots on a square grid,
where a part j in the partion is represented by placing pj dots in a row. Thus we
represent each term of the partition by a row of dots, the terms in descending
order with the largest at the top. Often it is more convenient to use squares
instead of dots. In this case the diagram is called a Young diagram. Draw
Ferrers diagram for the partition of 18 given above. The partition we obtain by
reading Ferrers diagram by columns instead of rows is called the conjugate of
the original partition. Find the conjugate of partion of 18 given above.

Problem 27. The Bernoulli numbers B0 , B1 , B2 , . . . are defined by the series

X Bj xj
x
=
ex 1 j=0 j!

where B0 = 1, B1 = 1/2, B2 = 1/6, B3 = 0, B4 = 1/30. Note that

B2j+1 = 0 for all j = 1, 2, . . .. Give an efficient way to calculate the Bernoulli
numbers. Then write a C++ program using the Verylong and Rational class
of SymbolicC++.

Problem 28. In the following arithmetic problem with one multiplication

and one sum each digit (except one with the digit 0) has been replaced by an
asterisks.

* * * x
 Number Manipulations 33

* *
-------
* * * *
* * * 0 +
-------
* * * *
Solve the problem when the asterisks can only be one of the prime numbers 2,
3, 5, 7.

Problem 29. Consider the nine numbers

111, 222, 333, 444, 555, 666, 777, 888, 999.
Find all the postive integers larger 1 that divide these numbers without remain-
der.

Problem 30. Let n be a positive integer. Any partition of n into parts

n = p1 + p2 + + pn , 0 p1 p2 pn n can be rearranged in 2 r
matrix  
a1 a2 ar
b1 b2 br
called the Frobenius symbol, such that
r
X r
X
0 a1 < a2 < < ar , 0 b1 < b2 < < br , n=r+ aj + bj .
j=1 j=1

Each partition has a unique representation as a Frobenius symbol.

(i) Find the Frobenius symbol for n = 20.
(ii) Write a C++ program that finds the Frobenius symbol for a given n.

Problem 31. Show that (continued-fraction representation)

1
= [1, 2, 1, 2, 1, 2 . . .].
3

Problem 32. The Mobius arithmetic functions : N 7 {0, 1} is defined by

1 if n=1
(n) := (1)m if n is the product of m distinct primes
0 otherwise

Give a C++ implementation using Verylong of SymbolicC++ and BigInteger

using Java. For m N one has
X
(n) = m,1
n|m
34 Problems and Solutions

where denotes the Kronecker delta.

Problem 33. Let x, y be integers. Assume they satisfy the equation

x2 8y 2 = 17.

(i) Show that (x0 , y0 ) = (5, 1) is a solution. Show that (x0 , y0 ) = (7, 2) is a
solution.
(ii) Let n = 0, 1, 2, . . .. Show that if (xn , yn ) satisfy the equation, then

xn+1 = 3xn + 8yn , yn+1 = xn + 3yn

also satisfy the equation.

Problem 34. Nobles are irrationals with continued fraction representations of

the form
= a0 + 1/(a1 + 1/(a2 + + 1/(1 + 1/(1 +
which eventually have elements all equal to one. Find the number with all aj s
equal to one.

Problem 35. Let p and q be prime with p q. Solve for p and q such that
p + q is prime.

Problem 36. Let n = 0, 1, 2, . . .. The Fermat numbers Fn are given by

n
Fn = 2(2 )
+1

(i) Show that the Fermat numerbers satisfy the recurrence relation

Fn+1 = (Fn 1)2 + 1

with F0 = 3.
(ii) Calculate the first ten Fermat numbers using this recurrence relation and
the Verylong class of SymbolicC++.
(iii) Calculate the first ten Fermat numbers using this recurrence relation and
the BigInteger class of Java.

Problem 37. Consider a palindrome consisting of digits. A palindrome is a

string of characters that reads the same from left to right or from right to left.
Show that if n N is a palindrome with an even number of digits, then n is a
multiple of 11. For example 3223 : 11 = 293. Write a C++ program utilizing
the string class to find out whether a string with an even number of digits is a
palindrome.
Chapter 5

Combinatorical Problems

Problem 1. Consider n distinct objects. Then there are n! permutations.

Without loss of generality we consider the integer numbers 1, 2, . . . , n. Write a
C++ program that uses recursion to find all permutations of these numbers.

Problem 2. Let X := { x0 = 0, x1 , x2 , . . . , xn1 } be a set of n points on the

real axis with xj < xj+1 for j = 0, 1, . . . , n 2. Let X denote the sequence of
all  
n n!

2 (n 2)!2!
pairwise distance
between points in X with the ordering xj xj+1 for
j = 0, 1, . . . , n2 1. For example, let X = { 0, 2, 4, 7, 10}. Then

X = {2, 2, 3, 3, 4, 5, 6, 7, 8, 10}.

Given X write a C++ program that generates X. Can one reconstruct X

from X? For the given example look at
0 2 4 7 10
0 2 4 7 10
2 2 5 8
4 3 6
7 3
10

Problem 3. Let n be an integer n 1. Consider a 2 n rectangular array

of boxes (a lattice space) and r dumbbell-shaped objects . Let A(r, n)
be the number of ways in which the r dumbbells may be placed in the 2 n

35
36 Problems and Solutions

rectangular array such that the two ends of each dumbbel are in two horizontally
or vertically adjacent boxes and no two dumbells have ends which share a box.
Obviously, A(r, n) = 0 if r > n. For example a configuration with n = 3 and
r = 3 is

(i) Find A(1, 1), A(1, 2), A(2, 2).

(ii) A recurrence is given by

A(r, n) = A(r, n 1) + 2A(r 1, n 1) + A(r 1, n 2) A(r 3, n 3)

with the initial conditions A(1, 1), A(1, 2), A(2, 2) and A(0, n) = 1. Use this
implementation to find A(1, 5), A(2, 5), A(3, 5), A(4, 5), A(5, 5).
(iii) Give a C++ implementation of this recurrence together with the initial
conditions.

Problem 4. The rank of an element in a sequence (one-dimensional array) of

numbers is the number of smaller elements in the sequence plus the number of
equal elements that appear to its left. For example, if the sequence is given as
the one-dimensional array a = [4, 3, 9, 3, 7], then the ranks are r = [2, 0, 4, 1, 3].
Write a C++ program with a function void rank(T* a,int n,int* r) that
computes the ranks of the elements of the array a[0 : n 1]. Once the elements
have been ranked using the function rank() write a function rearrange() that
rearrange them in nondecreasing order so that a[0] a[1] a[n 1] by
moving elements to positions corresponding to their ranks.

Problem 5. (i) Let N0 be the set of all positive integers including 0. Let
i, j, k N0 . Find all solutions of i + j + k = 3. Write down the solution in
lexigographical order.
(ii) Write a C++ program that finds all solutions of i + j + k = n for a given
n N0 .

Problem 6. Suppose we list all the 2n 1 nonempty subsets of the set of

numbers {1, 2, . . . , n}. Then, for each subset, we write down the product of
its elements. Finally, we add these 2n 1 numbers to obtain the number sn .
Obviously s1 = 1. For n = 2 we have {1}, {2}, {1, 2}. Thus s2 = 5 = 1 + 2 + 2.
For n = 3 the seven products one obtains are 1, 2, 3, 1 2, 1 3, 2 3 and
1 2 3. Thus s3 = 23.
(i) Find s4 .
(ii) Find a recursion sn+1 = f (n, sn ) for sn+1 with s= 1.
 Combinatorical Problems 37

(iii) Write a C++ program that implements this recurrence relation with the
initial value s1 = 1.

Problem 7. How many arrangements of a, a, a, b, b, b, c, c, c are there so that

no 2 letters of the same type appear consecutively? For example abcabcabc would
be such an arrangement. Could a tree structure be used to find the solution?
Write a C++ program that finds all sequences.

Problem 8. A coin is tossed eight times in a row.

(i) What is the probability of getting exactly four heads in a row?
(ii) What is the probability of getting at least four heads in a row?

Problem 9. Show that the number of ways in which n different objects can
be arranged in a a ring, if only relative order matters, is (n 1)!. First give an
example with 3 objects.

Problem 10. (i) How many n-digit ternary sequences (using only 0,1,2) are
there with k 1s?
(ii) Find the sequences for n = 3 and k = 2.
(iii) Write a C++ program that finds these sequences for given n and k in
lexicographical order.

Problem 11. Suppose that we list all the 2n 1 nonempty subsets of the set
of numbers { 1, 2, . . . , n }. Then, for each subset, we write down the product of
its elements. Finally, we add these 2n 1 numbers to obtain the number sn .
Obviously, s1 = 1. For n = 3 the seven products we obtain are 1, 2, 3, 1 2, 1
3, 2 3 and 1 2 3. Thus s3 = 1 + 2 + 3 + 2 + 3 + 6 + 6 = 23. Find a recurrence
relation for sn . Write a C++ program that implements this recurrence relation
with the initial value s1 = 1.

Problem 12. How many arrangements of a, a, a, b, b, b, c, c, c are there so that

no 2 letters of the same type appear consecutively? For example, abcabcabc
would be such an arrangement.

Problem 13. Let n be a positive integer. Use the inclusion-exclusion principle

to prove that
n  
X n nk
n= (1)k1 k 2 .
k
k=1

Problem 14. Show that the number of ways of writing the positive integer n
as a sum of positive integers, where the order of the summands is significant, is
2n1 for n 1. For example, for n = 3 we have 3 = 3, 3 = 2 + 1, 3 = 1 + 2,
38 Problems and Solutions

3 = 1 + 1 + 1.

Problem 15. The number xn of steps required to solve the Chinese rings
puzzle with n rings satisfies x1 = 1 and the recurrence relation

2xn n odd
xn+1 =
2xn + 1 n even

where n = 1, 2, . . ..
(i) Prove that xn+2 = xn+1 + 2xn + 1.
(ii) Find a formula for xn .

Problem 16. A derangement (or fixed-point-free permutation) of { 1, 2, . . . , n }

is a permutation f such that f (j) 6= j for all j = 1, 2, . . . , n. What is the number
of derangements of n objects?

Problem 17. A numerical partition of a positive integer n is a sequence

p1 p2 pk 1

such that
p1 + p2 + + pk = n.
Each pj is called a part. For example, 18 = 7 + 4 + 4 + 1 + 1 + 1 is a partition of
18 into 6 parts. The number of partitions of n into k parts is denoted by p(n, k).
(i) Find p(7, 3).
(ii) Show that the recurrence for p(n, k) is given by

p(n, k) = p(n 1, k 1) + p(n k, k)

with the initial conditions p(n, 0) = 0, p(k, k) = 1. Obviously, we have p(n, 1) =

1.
(iii) Write a C++ program that implements this recurrence..
(iv) Every numerical partition of a positive integer n corresponds to a unique
Ferrers diagram. A Ferrers diagram of a partition is an arrangement of n dots
on a square grid, where a part j in the partition is represented by placing pj
dots in a row. This means we represent each term of the partition by a row
of dots, the terms in descending order with the largest at the top. Sometimes
it is more convenient to use squares instead of dots (in this case the diagram
is called a Young diagram). Draw the Ferrers diagram for the partition 18 =
7 + 4 + 4 + 1 + 1 + 1. The partition we obtain by reading the Ferrers diagram by
columns instead of rows is called the conjugate of the original partition. Find
the conjugate of the partition 18 = 7 + 4 + 4 + 1 + 1 + 1.

Problem 18. The Bell numbers count (starting from 0) the ways that n
distinguishable objects can be grouped into sets if no set can be empty. Thus
 Combinatorical Problems 39

the Bell numbers are given by the sequence

{ 1, 1, 2, 5, 15, 52, 203, 877, 4140, . . . }.

For example the numbers 1, 2, 3 can be grouped into sets so that

1) 1, 2, 3 are in three separate sets: {1}, {2}, {3}

2) 1 and 2 are together and 3 is separate: {1, 2}, {3}
3) 1 and 3 together and 2 separate: {1, 3}, {2}
4) 2 and 3 together and 1 separate: {2, 3}, {1}
5) 1, 2, 3 are all together in a single set: {1, 2, 3}

Hence for n = 3 there are five partitions and thus the third Bell number is 5.
(i) Let Pn denote the nth Bell number, i.e. the number of all partitions of n
objects. Then we have

1 X kn
Pn = .
e k!
k=0

(i) Find a recurrence relation for Pn .

(ii) Write a C++ program that implements this recursion. Apply Verylong of
SymbolicC++.
(iii) Find the MacLaurin expansion (expansion around 0) of exp(exp(x)) and
establish a connection with the Bell numbers.

Problem 19. Let N be a positive integer. Consider the set of numbers

S = { 0, 1, 2, . . . , N }

How many pairs (m, n) (m, n S) can be formed with the condition that m < n.

Problem 20. Let n be the number of discrete symbols s1 , s2 , . . . ,sn that can
be used. Let m be the length of the message string. Find the number M of
messages. Then consider the special case n = m = 2.

Problem 21. Consider a bitstring of length m which has exactly m1 ones and
m2 zeros (m1 + m2 = m).
(i) Find the number of different possible bitstrings.
(ii) Consider the special case m = 4, m1 = m2 = 2. Write down the bitstrings
in lexicographical order.

Problem 22. Consider the n-dimensional unit cube in Rn . How many k-

dimensional surfaces (1 k < n) does the n-dimensional unit cube has?
40 Problems and Solutions

Problem 23. Let n1 , n2 , n3 be nonnegative integers. Let n = n1 + n2 + n3 .

To what combinatorical problem can
n!
n1 !n2 !n3 !
be associated?

Problem 24. A dice is thrown twice. The first throw determines the tens
digit and the second throw the ones digit of the two-digit number. Find the
probability that this two-digit number is a perfect square.

Problem 25. How many different numbers of 7 digits can be formed with the
digits 1122334 ?

Problem 26. Let i, j, k N0 . Find all solutions of

i + j + k = 3.
Give the solution in lexicographical order.

Problem 27. Suppose three fair coins are flipped. Let X be the event that
they same face. Let Y be the event that there is at most one head. Show that
X and Y are independent.

Problem 28. The Stirling number of the second kind S(n, k) is the number
of partitions of a set with n elements into k classes. Let b , b be Bose creation
and annihilation operators with the commutation relations
[b, b ] = bb b b = I
where I is the identity operator. Then S(n, k) can be defined by
n
X
(b b)n = S(n, k)(b )k bk .
k=1

Let n = 3. Use this definition to find S(3, 1), S(3, 2), S(3, 3).

Problem 29. Let n 1. Let a1 , a2 , . . . ,an be real numbers. How many terms
are there in the sum
X X X
aj1 aj2 aj3 .
1j1 <j2 <j3 n

Problem 30. Let n 1. Let c1 , c2 , . . . , cn be spin-less Fermi creation

operators and c1 , c2 , . . . , cn be spin-less Fermi annihilation operators. Thus
[cj , ck ]+ = jk I, [cj , ck ]+ = 0, [cj , ck ]+ = 0
 Combinatorical Problems 41

where 0 is the zero operator, [ , ]+ denotes the anti-commutator and j, k =

1, . . . , n. Then (cj )2 = 0 and (cj )2 = 0. Let |0i be the vacuum state with
cj |0i = 0|0i and j = 1, . . . , n. Then for n = 1 we can form the two-dimensional
basis |0i, c1 |0i. For n = 2 we can form the four dimensional basis

|0i, c1 |0i, c2 |0i, c2 c1 |0i.

(i) Find the basis for n = 3.

(ii) Find the basis for n = 4.
(iii) Extend to arbitrary n.
Chapter 6

Matrix Calculus

Problem 1. Let A be an n n matrix over C. We define sin(A) as

X (1)j
sin(A) := A2j+1 .
j=0
(2j + 1)!

Consider the 2 2 matrix (x, y R)

 
x y
B= .
0 x

Calculate sin(B) efficiently. Decompose the matrix B as B = C + D, where

   
x 0 0 y
C= , D= .
0 x 0 0

Problem 2. Let A be a square matrix over R. We define the nm approximant

of eA as  k !n
m
X 1 A
fn,m (A) := .
k! n
k=0
We have the error estimation
1
keA fn,m (A)k kAkm+1 ekAk
nm (m + 1)!

and fn,m (A) converges to eA

lim fn,m (A) = lim fn,m (A) = eA .

n m

42
 Matrix Calculus 43

Consider
 
1 1
A= .
1 1

(i) Calculate eA .
(ii) Calculate f2,2 (A), the norm of A and the error estimation. The norm is
given by
kAk := max kAxk
kxk=1

where x R2 and kAxk denotes the Euclidean norm.

Problem 3. Consider an 8 8 matrix numbered as follows

0 1 2 3 4 5 6 7

8 9 10 11 12 13 14 15
16 17 18 19 20 21 22 23

24 25 26 27 28 29 30 31

.
32 33 34 35 36 37 38 39

40 41 42 43 44 45 46 47

48 49 50 51 52 53 54 55

56 57 58 59 60 61 62 63

Let (j, k) be the matrix elements with j, k = 0, 1, . . . , 7. Write a C++ program

that writes the matrix elements given as floating point numbers into a vector
with 64 elements using the numbering given above. Use the vector class of the
Standard Template Library for the vector and the matrix.

Problem 4. An n n matrix T is called a Toeplitz matrix if it satisfies the

relation T (k, j) = T (k 1, j 1), for 2 j, k n. In other words, the entries
on each diagonal of T are all equal. Hence, such a matrix is determined by the
2n 1 entries appearing in the first row and first column. We denote these
entries by t0 , t1 , . . . , t2n2 such that

tn1 tn2 ... t2 t1 t0

tn tn1 tn2 ... t2 t1
tn+1 tn tn1 tn2 ... t2


. ..
..

T = .. . .

.
. .. ..
.. . .

t t2n2 ... tn1 tn2
2n3
t2n2 t2n3 ... tn+1 tn tn1

We say that the vector t = (t0 , t1 , . . . , t2n2 ) defines the Toeplitz matrix T .
Thus the 4 4 Toeplitz matrix defined by the vector t = (t0 , t1 , t2 , t3 , t4 , t5 , t6 )
44 Problems and Solutions

is
t3 t2 t1 t0

t4 t3 t2 t1
T = .
t5 t4 t3 t2
t6 t5 t4 t3
Write a C++ program that generates the Toeplitz matrix T from a given vector
t. Vice versa given a Toeplitz matrix T find the vector t.

Problem 5. Let A1 , A2 , . . . , Ap be square matrices of the same size. Let

fn,1 (A1 , A2 , . . . , Ap ) := (eA1 /n eA2 /n eAp /n )n .

Then
2
p p p
X 2 X n + 2 X
k exp( Aj ) fn,1 (A1 , A2 , . . . , Ap )k kAj k exp kAj k
j=1
n j=1
n j=1

and
Xp
lim fn,1 (A1 , A2 , . . . , Ap ) = exp Aj .
n
j=1

For p = 2 we obtain
eA1 +A2 = lim (eA1 /n eA2 /n )n . (1)
n

Let      
0 1 0 0 0 1
A= , A1 = , A2 = .
1 1 0 1 1 0
Then A = A1 + A2 . Note that [A1 , A2 ] 6= 0.
(i) Calculate eA by diagonalizing A.
(ii) Calculate eA using equation (1).
(iii) Calculate f2,1 (A1 , A2 ) and the error estimation.

Problem 6. The discrete Fourier transform over n points can be written in

matrix form
1 1 1 ... 1

1 w w2 ... w n1

1 w2 w4 . . . w2(n1)

Fn :=
.
.. .. .. .. ..
. . . .
(n1)2
1 wn1 w2(n1) ... w
where w := e2i/n is the n-th root of unity. We obtain the discrete Fourier
transform from
(x1 , x2 , . . . , xn )T = Fn (x1 , x2 , . . . , xn )T .
 Matrix Calculus 45

Apply F4 to the data (1, 0, 0, 1)T and (0, 1, 1, 0)T and interpret the results to
find the underlying periodicity.

Problem 7. (i) The discrete Fourier transform over n points can be written
in matrix form
1 1 1 ... 1

1 2 n1
w w ... w
2 4 2(n1)
1 w w . . . w

Fn :=

... .. .. .. ..
. . . .
n1 2(n1) (n1)2
1 w w ... w

where w = e2i/n is the n-th root of unity. Show that the matrix 1 Fn is
n
unitary.
(ii) Use trigonometric interpolation to find

f (x) = c0 + c1 eix + c2 e2ix + c3 e3ix

which interpolates the points

x0 = 0, y0 = 0, x1 = , y1 = 1,
2
3
x2 = , y2 = 1, x3 = , y3 = 0
2
i.e. solve the equation F4 (c0 , c1 , c2 , c3 )T = (y0 , y1 , y2 , y3 )T .

Problem 8. A 1-inverse of the m n matrix A is an n m matrix A such

that AA A = A.

(i) Suppose that m = n and that A1 exists, find A .

(ii) Is the 1-inverse unique? Prove or disprove.
(iii) The Moore-Penrose pseudoinverse of the m n matrix A is the 1-inverse
A of A which additionally satisfies

A AA = A (AA ) = AA (A A) = A A.

Let A = U V be the singular value decomposition of A. Show that A =

V U is a Moore-Penrose pseudoinverse of A, where
 1
()kj 6= 0
( )jk = ()kj
0 ()kj = 0
Hint: First show that is the Moore-Penrose pseudoinverse of . Find the
Moore-Penrose pseudoinverse of
 
1 1
.
1 1
46 Problems and Solutions

Problem 9. Find the eigenvectors and generalized eigenvectors of

0 1 0
0 0 0.
0 1 0

Problem 10. Let

0 i 0
A = 0 0 0.
0 i 0
Calculate exp(A), sinh(A), cosh(A), sin(A), cos(A) efficiently.

Problem 11. Find the Cosine-Sine decomposition of

0 1 1 0

1 1 0 0 1
.
2 1 0 0 1
0 1 1 0

Problem 12. Solve the system of linear equations

2 1 1 x 1
0 2 1y = 1
1 0 2 z 1

using the Jacobi method. Start from the initial guess x0 = y0 = z0 = 0.

Problem 13. How would one store a matrix using a linked list?

Problem 14. Given an n n matrix over C. How would we efficiently test

whether the matrix is unitary? Apply your approach to

1 1 1
1
U= i i i .
3 i i i

Problem 15. The sum 12 + 22 + 32 + + n2 can be written as

12 + 22 + 32 + + n2 = an3 + bn2 + cn

where the unknown coefficients a, b, c can be determined from a system of linear

equations obtained from n = 1, n = 2, n = 3. Find this system of linear
 Matrix Calculus 47

equations and write a C++ program using Gauss elimination that finds the
solution.

Problem 16. The sum 13 + 23 + 33 + + n3 can be written as

13 + 23 + 33 + + n3 = an4 + bn3 + cn2

where the unknown coefficients a, b, c can be determined from a system of linear

equations obtained from n = 1, n = 2, n = 3. Find this system of linear
equations and write a C++ program using Gauss elimination that finds the
solution.

Problem 17. Let A, B be m n matrices over C. The Hadamard product

A B is defined as the m n matrix (entrywise multiplication)

A B = (ajk bjk ).

Write a C++ program using vector<vector<complex> > that implements the

Hadamard product.

Problem 18. Consider the set, M , containing all matrices of M2 (R) of the
form  
a b
b a
where a, b R. Mapping this matrix onto a + i b we obtain a field iso-
morphism between M and C. Write a C++ program using complex<T> and
vector<vector<T> > and the function
void isomorphism(complex<T>& z,vector<vector<T> >& m)
that maps a complex number to the corresponding 2 2 matrix.

Problem 19. Let A be an n n symmetric matrix over R. Write a C++

program that uses Givens transform to cast the matrix into tridiagonal form.

Problem 20. Given an n n tridiagonal matrix over R with n 3. Write a

C++ program that finds the characteristic polynomial.

Problem 21. Let A be an n n matrix over R. Let u be a nonzero column

vector in Rn . Computing the n n matrix

2uuT
 
In T A
u u
can be done as follows.
48 Problems and Solutions

Step 1. Compute the number = 2/(uT u).

Step 2. for j = 1, 2, . . . , n do
= u1 a1j + u2 a2j + + un anj
=
for i = 1, 2, . . . , n do aij = aij ui

Write a C++ program that implements this algorithm.

Problem 22. Consider the two polynomials

p1 (x) = a0 + a1 x + + an xn , p2 (x) = b0 + b1 x + + bm xm

where n = deg(p1 ) and m = deg(p2 ). Assume that n > m. Let r(x) =

p2 (x)/p1 (x). We expand r(x) in powers of 1/x, i.e.
c1 c2
r(x) = + 2 +
x x
From the coefficients c1 , c2 , . . . , c2n1 we can form an n n Hankel matrix
c
1 c2 cn
c2 c3 cn+1
Hn =
.. .. .. .
..
. . . .
cn cn+1 c2n1

The determinant of this matrix is proportional to the resultant of the two poly-
nomials. If the resultant vanishes, then the two polynomials have a non-trivial
greates common divisor. Implement this algorithm in SymbolicC++ where
aj , bj Q and apply it to the polynomials

p1 (x) = x3 + 6x2 + 11x + 6, p2 (x) = x2 + 4x + 3.

Problem 23. Given an m n matrix A = (ajk ). We define a norm as

n
!
X
kAk := max |ajk | .
1jm
k=1

Give a C++ implementation using templates.

Problem 24. Write a C++ program that transposes a square matrix in-place.

Problem 25. Consider a binary n n matrix, where we count the entries

from 0. We have b00 = 1 and bn1n1 = 1. The other 0-1 entries are generated
 Matrix Calculus 49

randomly. An ant at entry (0, 0) can only move to the right or down (not diag-
onal) when this entry contains a 1. Write a C++ program that check whether
the ant could reach the entry (n 1, n 1). For example, consider the matrix

1 0 0 0 0

1 1 1 0 0
1 0 1 0 1.

0 0 1 1 0

0 0 0 1 1

For this case one path

(0, 0) (1, 0) (1, 1) (1, 2) (2, 2) (3, 2) (3, 3) (4, 3) (4, 4)

would be possible. Note that the ant could also get stuck at (2, 0).

Problem 26. Let z C and A be an n n matrix over C with A2 = In .

Calculate exp(zA).

Problem 27. Let z C and A be an n n matrix over C with A2 = A.

Calculate exp(zA).

Problem 28. Apply the Leverrier method to find the determinant of the
matrix
1 0 1
A() = 0 1 0
1 0 
where  R. What is the condition on  such that the inverse of A() exists?

Problem 29. Let  

1/4 1/2
A= .
1/2 1/4
Let
(A) = max |j |
1j2

where j are the eigenvalues of A.

(i) Check that (A) < 1.
(ii) If (A) < 1, then

(I2 A)1 = I2 + A + A2 + .

Calculate (I2 A)1 .

(iii) Calculate
(I2 A)(I2 + A + A2 + + Ak ).
50 Problems and Solutions

Problem 30. Let A be an n n matrix over R. Consider the system of linear

equations
Ax = b
or
n
X
aij xj = bi , i = 1, 2, . . . , n.
j=1

We assume that A is invertible. Let A = C R. This is called a splitting of the

matrix A and R is the defect matrix of the splitting. Consider the iteration

Cx(t + 1) = Rx(t) + b, t = 0, 1, . . .

with a given x(0).

Let

4 1 0 4 0 0 3 0
A = 1 4 1 , C = 0 4 0, b = 2, x(0) = 0 .
0 2 4 0 0 4 2 0

Perform the iteration. Does x(k) (k = 0, 1, . . .) converge to the solution of

Ax = b. Write a C++ program that implements this iteration.

Problem 31. Let A be an n n matrix over C. Then any eigenvalue of A

satisfies the inequality
n
X
|| max |ajk |.
1jn
k=1

Write a C++ program that calculates the right-hand side of the inequality for
a given matrix. Apply the complex class of STL. Apply it to the matrix

i 0 0 i

0 2i 2i 0
A= .
0 3i 3i 0
4i 0 0 4i

Problem 32. We count the entries of the n n matrix from (0, 0) to (n

1, n 1). Let A = (ajk ) be a real n n matrix (j, k = 0, 1, . . . , n 1). The
permanent of A is defined to be the real number

X Y
perm(A) := Aj,(j)
Sn j[n]

where the summation is over all n! permutations of the set [n] := {0, 1, . . . , n1}.
Give an implementation with SymbolicC++ to find the permanent of a given
 Matrix Calculus 51

matrix A.

Note that the definition of the determinant for the matrix A is

X Y
det(A) := sgn() Aj,(j)
Sn j[n]

where the summation is over all n! permutations of the set [n] := {0, 1, . . . , n1}
and sgn() equals +1 if is an even permutation and equals 1 if is an odd
permutation.

Problem 33. We count the entries of the n n matrix F from (0, 0) to

(n 1, n 1). We define
n = exp(i2/n).
Now the n n matrix F is defined by

F = njk , j, k = 0, 1, . . . , n 1.

Give a SymbolicC++ implementation for F .

Problem 34. Consider the linear equation written in matrix form

1 0 1 x1 1
0 1 1 x2 = 2 .
1 0 2 x3 1

First show that the determinant of the 3 3 matrix is nonzero. Apply two
different methods (Gauss elimination and the Leverriers method) to find the
solution. Compare the two methods and discuss.

Problem 35. Consider the two 3 3 permutation matrices (which are of

course then also unitary matrices)

0 1 0 0 0 1
U1 = 0 0 1 , U2 = 1 0 0.
1 0 0 0 1 0

We want to find efficiently K1 and K2 such that U1 = eK1 and U2 = eK2 . We

would apply the spectral decomposition theorem to find K1 , i.e.
3
X
K1 = ln(j )vj vj
j=1

where j are the eigenvalues of U1 and vj are the corresponding normalized

eigenvectors. But then to find K2 we would apply the property that U12 = U2 . Or
52 Problems and Solutions

could we actually apply that U2 = U1T ? Note that U1 , U2 , I3 form a commutative

subgroup of the group of 3 3 permutation under matrix multiplication.

Problem 36. Let A, B, C be n n matrices. Simplify

(A In In )(In B In (In In C) .

Problem 37. Consider the 3 3 normal matrix

0 0 1
A = 0 1 0.
1 0 0
Study the following four methods to calculate exp(A). Discuss.
(i) Apply the definition

X Aj
exp(A) := .
j=0
j!
(ii) Apply the definition
 n
A
exp(A) := lim I3 + .
n n
(iii) Apply the spectral theorem, i.e. use the eigenvalues and normalized eigen-
vectors of A.
(iv) Apply the Cayley-Hamilton theorem which also needs the eigenvalues of A.
Keep in mind that one eigenvalue is degenerate.

Problem 38. Consider an n n symmetric tridiagonal matrix over R. Let

fn () := det(A In ) and
1 1 0 0

1 2 2 0
..
0

2 . 0

fk () = det . .. .. .. ..
.
. . . . .

0 k1 k1

0 0 k1 k
for k = 1, 2, . . . , n and f0 () = 1, f1 () = 0. Then
2
fk () = ( )fk1 () k1 fk2 ()
for k = 2, 3, . . . , n. Find f4 () for the 4 4 matrix

0 1 0 0

1 0 2 0
.
0 2 0 3

0 0 3 0
Chapter 7

Recursion

Problem 1. Let a, b be realpositive numbers. If A = (a + b)/2 denotes

the arithmetic mean and B = ab denotes the geometric mean, then A B
with equality precisely when a = b. Given two positive real numbers, a0 and b0 ,
where we suppose that a0 b0 , we consider the recursion
1 p
aj+1 = (aj + bj ), bj+1 = aj bj
2
so that aj+1 is the arithmetic mean of aj and bj , while bj+1 is their geometric
mean. Write a C++ program that implements this recursion with a0 = 2.0 and
b0 = 1.0.

Problem 2. The Catalan recurrence is given by

n
X
h[n] = h[k 1] h[n k]
k=1

with n 0 and the initial value h[0] = 1. The Catalan number h[1], h[2], . . .
arise in a number of problems in combinatorics. Write a C++ program that
finds the Catalan numbers h[1], h[2], . . . , h[6] using the Catalan recurrence.

Problem 3. For all p > 1, the iteration (k = 0, 1, 2, . . .)

1
xk+1 = ((p 1)xk + x1p
k a), x0 = 1
p
converges quadratically to a1/p if a belongs to
a { z C : <z > 0 and |z| 1 } R+ .

53
54 Problems and Solutions

Write a C++ program that implements this iteration for p = 3 and a = 27.

Problem 4. Let 0 < x < 2. The computation of 1/x can be done with
addition and multiplication using the following recurrence relation

aj+1 = aj (1 + cj ), cj+1 = c2j (1)

with j = 0, 1, 2, . . . and the initial values a0 = 1, c0 = 1 x.

(i) Show that
1 cj
aj = (2)
x
j
and since cj = c20 with |c0 | < 1, it follows that
1
lim aj = .
j x
(ii) Write a C++ program that implements this recurrence relation.

Problem 5. Let n = 1, 2, . . .. Consider the definite integral

Z /2
In = cosn (x)dx.
0

Apply integration by parts to find a recursion relation for In , i.e. express In with
In1 , In2 . Note that
Z /2 Z /2

I1 = cos(x)dx = 1, I2 = cos2 (x)dx = .
0 0 4

Problem 6. Find a recursion for

Z /4
In = tann (x)dx
0

where n = 0, 1, . . . and
Z /4

I0 = dx =
0 4
Z /4
/4

I1 = tan(x)dx = ln(cos(x))|0 = ln(cos(/4)) + ln(cos(0)) = ln( 2).
0

Problem 7. Let x [0, 1]. Then x can be approximated by the sequence of
polynomials
1
pk+1 (x) = pk (x) + (x (pk (x))2 ), k = 0, 1, 2, . . .
2
 Recursion 55

and k the sequences converges pointwise to x with p0 (x) = x. Write
a
C++ program that implements this sequence to find an approximation for x.

Problem 8. The number /2 can be calculated using the iteration

p
xk+1 = xk yk , yk+1 = 2yk /(yk + 1), k = 0, 1, 2, . . .

where
x0 = 1, y0 = 2.
Then

lim xk =
.
k 2
Write a C++ program that implements this iteration and thus finds an approx-
imation of /2.

Problem 9. The number can be calculated using the iteration

2xk yk
xk+1 = , yk+1 = xk+1 yk , k = 0, 1, 2, . . .
xk + yk
where
x0 = 2 3, y0 = 3.
Then
lim xk = lim yk = .
k k
Write a C++ program that implements this iteration and thus finds an approx-
imation of .

Problem 10. (i) Let r be a real nonzero number. Then 1/r can be calculated
to ever increasing accuracy applying the map

xt+1 = xt (2 rxt ), t = 0, 1, . . .

provided the initial estimate x0 is sufficientlty close to 1/r. The number of digits
of accuracy approximately doubles with each iteration. First find the fixed points
of the map. Let r = 3 and x0 = 2. Find x1 , x2 , . . . . Discuss.
(ii) The same iteration can be applied to find the multiplicative inverse modulo
any power of 2. For example, to find the multiplicative inverse of 5, modulo 256,
we start with x0 = 1 (any odd number will do). Then

x1 = x0 (2 5 x0 ) = 3
x2 = 3(2 5 (3)) = 51
x3 = 51(2 5 (51)) = 13107.

Thus 13107 mod 256 = 205. Thus the multiplicative inverse of 5 (modulo 256)
is 205. Write a C++ program that implements this algorithm.
56 Problems and Solutions

Problem 11. According to Gauss, the elliptic integral

Z /2
2 dx
I=
0 (a2 cos2 (x) + b2 sin2 (x))1/2

is equal to the limit of any of the two convergent sequences

s0 , s1 , s2 , . . . or t0 , t1 , t2 . . .

as defined by the recurrence relations for j > 0

sj+1 = (sj + tj )/2

p
tj+1 = sj tj

and s0 = a, t0 = b. The calculation of the two sequences is called the arithmetic-

geometric mean method. Write a C++ program that implements this method.

Problem 12. Given the sequence of terms

t0 , t1 , t2 , . . .

the series of partial sums

s0 , s1 , s2 , . . .
is defined such that

sj := t0 + t1 + + tj j = 0, 1, 2, . . .

If the sequence is given by the recurrence relation

tj+1 = f (tj ) for j 0

then the series sj is determined by

sj+1 = sj + tj+1 for j 0

s0 = t0 .

Give a C++ implementation for the sequence sj and the recursion

tj
tj+1 = f (tj ) = .
j+1

Problem 13. Let m be a positive integer and x be a fixed real number. Then
we can calculate cos(mx) using the recursion

cos((m + 1)x) = 2 cos(x) cos(mx) cos((m 1)x)

 Recursion 57

Write a C++ program that implements this recursion. Use the values m = 10
and x = 0.1.

Problem 14. John McCarthy introduced the following recurrence equation

(n N)
f (n) = if n > 100 then n 10 else f (f (n + 11)).
Write a C++ program that implements the recurrence equation. What is the
output if n 101?

Problem 15. Let k be a positive integer k 2. Consider the recursion

xt+1 = xt + kyt , yt+1 = xt + yt

where t = 0, 1, 2, . . . and x0 = y0 = 1. Study xt+1 /yt+1 for t and k = 5.

Problem 16. Give a C++ implementation of the recursion

n X
X k
c0 = 1, c1 = 1, cn+1 = ci cki cnk+1 .
k=1 i=0

Problem 17. Give a C++ implementation of the recursion

1
xj (t + 1) = (1 )xj (t) + (xj1 (t) + xj+1 (t)), t = 0, 1, 2, . . .
2
where j = 0, 1, 2, 3 and 1 3, 4 0.

Problem 18. Consider the function

Z
fn (x) = y n exp(y 4 xy 2 )dy, n = 0, 1, . . .
0

(i) Show that

n+1 x
fn+4 (x) = fn (x) fn+2 (x)
4 2
using integration by parts.
(ii) Show that
dfn (x)
= fn+2 (x).
dx

Problem 19. (i) Consider the recursion

xt+1 = xt + 5yt
yt+1 = xt + yt
58 Problems and Solutions

where t = 0, 1, . . .. Let x0 = y0 = 1. Calculate x1 , y1 , x2 , y2 , x3 , y3 and x0 /y0 ,

x1 /y1 , x2 /y2 , x3 /y3 .
(ii) Define
xt
zt := .
yt
Find the recursion for zt . Find the fixed points of this recursion. Are the fixed
point stable? Find
lim zt
t

with z0 = x0 /y0 = 1.

Problem 20. Let n be a positive integer. A Dyk word is a string of length 2n

with n xs and n ys such that no initial segment of the string of length 2n has
more ys than xs.
(i) Give the Dyk words for n = 1, n = 2 and n = 3.
(ii) Describe an algorithm to generate the Dyk words for a given n. Give a
recursion.

Problem 21. Let k and n be positive integers. Implement in C++ the

recursive function

0 if k>n
p(k, n) = 1 if k=n
p(k + 1, n) + p(k, n k) otherwise

where p(1, 1) = 1.

Problem 22. Consider the alphabet {A, B, C} and the Fredholm subsitution

A 7 AB, B 7 BC, C 7 CC.

Start of with A and find the sequence. Then set A = C = 0 and B = 1 and find
the bitstring.

Problem 23. Let k = 0, 1, . . . and R. Consider the integral defined by

Z
cos(k) cos(k)
Ik () := d .
0 cos() cos()

Show that I0 () = 0 and I1 () = . Show that Ik () satisfies the second order

difference equation

Ik+2 () 2 cos()Ik+1 () + Ik () = 0, k = 0, 1, . . .

with I0 () = 0 and I1 () = . Solve the second order difference equation.

 Recursion 59

Problem 24. Let m = 1, 2, . . .. Consider the recursion

m
X
cm = ck1 cmk
k=1

with c0 = 1. Define a generating function

X
P (x) = cm xm = c0 + c1 x1 + c2 x2 + 1 + c1 x + c2 x2 + .
m=0

Then
X
X m X
X
P (x) = 1+ ck1 cmk xm = 1+x cmk xmk ck1 xk1 = 1+xP 2 (x)
m=1 k=1 k=1 m=k

with the solution for P (x)

1 1 4x
P (x) =
2x
with P (0) = 1. Taylor expansion of P (x) provides
4n 2
cm = cm1 , m = 1, 2, . . .
n+1
and thus (c0 = 1)
(2m)!
cm = .
m!(m + 1)!
Give an implementation of the recursion for cm and of the two last equations for
cm using SymbolicC++ and the Verylong of SymbolicC++.

Problem 25. Consider the tridiagonal n n matrix

a1 b2 0 ... 0 0

c2 a2 b3 ... 0 0
0 c3 a3 ... 0 0


A= .. .. .. .. .. ..
. . . . . .


0 0 0 ... an1 bn
0 0 0 ... cn an
with a1 , aj , bj , cj C (j = 1, 2, . . . , n). It has in general n complex eigenvalues
the n roots of the characteristic polynomial p(). Show that this polynomial
can be evaluated by the recursive formula

pk () = ( ak )pk1 bk ck pk2 (), k = 2, 3, . . . , n

p1 () = a1
p0 () = 1.
60 Problems and Solutions

Problem 26. Let I (x) be the modified Bessel function. For the asymptotic
expansion we have

I+1 (x) X
cj xj
I (x) j=0

The expansion coefficients cj obey the quadratic recursive relation

   
1 1 1
c0 = 1, c1 = + , c2 = c3 = 2 ,
2 2 4
j2
X
2cj = (j 1)cj1 c` cj` , j 4.
`=2

Give a SymbolicC++ implementation of this recursion applying the Verylong

class. Then apply it to = 1/2 and = 1/2.
Chapter 8

Numerical Techniques

Problem 1. The exponential function ex (x R) can be defined as

ex := lim (1 + x/n)n (1)

n

or

X xk
ex := . (2)
k!
k=0

The above two formulae give methods to calculate ex numerically. The second
expression is more convenient for such a purpose, because the convergence of (2)
is better than of (1). Is there a way to combine the two definitions for a more
rapidly converging expression for ex ?

Problem 2. Calculating 2 an elementary and ancient recursion consists of
the double sequence

pj+1 = pj + 2qj , qj+1 = pj + qj

with j = 0, 1, . . . and
pj
lim = 2.
j qj

(i) Calculate the first three terms with the initial values p0 = q0 = 1.

(ii) Give an error estimation with j := | 2 pj /qj |.
(iii) Write the problem in matrix notation and solve it.

61
62 Problems and Solutions

Problem 3. Let f be an analytic function. Konigs root-finding algorithm is

given by the iteration

(1/f )[n2] (xj )

xj+1 = xj + (n 1)
(1/f )[n1] (xj )

where n 2, (1/f )[k] denotes the kth derivative of 1/f and j = 0, 1, 2, . . . with
x0 to be the initial value. Show that for n = 2 we obtain Newtons method and for
n = 3 we obtain Halleys method. Derive the iteration for the case n = 4. Write
a C++ program for this case with the analytic function f (x) = sin(2x) sinh(x)
and the initial condition x0 = 1.

Problem 4. Consider the case of solving the quadratic equation

ax2 + bx + c = 0, a 6= 0.

The usual way the two roots x1 and x2 are computed is

b + b2 4ac b b2 4ac
x1 = , x2 = .
2a 2a

If a, b, and c numbers such that b is about the same size as b2 4ac (with
respect to the arithmetic used), then a catastrophic cancellation will occur in
computing x2 . As a result, the computed value of x2 can be completely erro-
neous.
(i) Consider the case with a = 1, b = 105 and c = 1 and eight-digit arithmetic.
(ii) How can this problem be avoided?

Problem 5. Let a > 0. Find an iteration to approximate a. Use the fact
that if x (x > 0) is the actual square root of a, then x = a/x; that is, the two
factors x and a/x are equal.

Problem 6. For any positive number h we define an operator Sh which replaces

a continuous function f : R R by its average over an interval with length h
and centre x
1 x+h/2
Z
(Sh f )(x) := f (t)dt.
h xh/2
We find that
lim (Sh f )(x) = f (x).
h0

(i) Show that the operator Sh is linear.

(ii) Show that the operator Sh leaves linear functions unchanged.
(iii) Calculate Sh f for f (t) = e|t| sin(t) with h = 0.1.

Problem 7. (i) Provide a fast algorithm to calculate 2.
 Numerical Techniques 63

(ii) Provide a fast algorithm to calculate the golden mean number = (1+ 5)/2.

Problem 8. If a beam runs into an obstacle a part of the signal is transmitted

the rest reflected. The difference between the frequency of the reflected part
and the initial frequency 0 is the so-called Doppler shift caused by a particle
moving with velocity in the direction opposite to the transmitted signal. It
can be calculated as
2c0
= 0 = 2
c 2
where c denotes the velocity of sound within the medium. We have  c. Can
the calculation be simplified?

Problem 9. Given pairs of single precision numbers (x1 , y1 ), (x2 , y2 ), (x3 , y3 ),

(x4 , y4 ). Decide whether the line segement (x1 , y1 ) (x2 , y2 ) intersects the line
determined by (x3 , y3 )(x4 , y4 ) at a unique point. If so, compute the coordinates
(x, y) of the intersection point accurate to single precision. That is, if the exact
intersection point is (x , y ), then return a point (x, y) such that x is the nearest
single precision number to x0 and y is the nearest single precision number to y 0 .

Problem 10. The harmonic series can be approximated by

n
X 1 1
0.5772 + ln(n) + .
j=1
j 2n

Calculate the left and right hand side for n = 1 and n = 10.

Problem 11. Consider the linear first order delay-differential equation

du
= u(t 1).
dt
We can find the solution with the ansatz

u(t) = Cet .

Since du/dt = Cet and u(t 1) = Ce(t1) we obtain

= e .

There is no solution if is real. If is complex then there are an infinite number

of solutions. They are given by the Lambert W function. Write a C++ program
that finds some of the solutions.

Problem 12. Let a, b, c, d > 0 and a + b + c > d. Consider the problem of

relating the input and output crank angles of a four-bar mechanism. The angles
64 Problems and Solutions

and , respectively, are measured from the line of the fixed pivots. The moving
links of fixed length are a, b, c. The fixed link is d. This provides us with the
equation

b2 = c2 + d2 + a2 2dc cos() 2ac cos() cos() 2ac sin() sin() + 2ad cos().

With C1 = d/c, C2 = d/a, C3 = (d2 + a2 b2 + c2 )/(2ac) we obtain the

Freudenstein equation

C1 cos() C2 cos() + C3 cos( ) = 0.

Set a = 1, b = c = d = 2. Solve this transcendental equation with the Newton

method for different fixed and solve for .

Problem 13. Let f : R R be an analytic function. We define

f (x ) + f (x + ) 2f (x)
f (x) := .
2
(i) Let f (x) = x2 . Find f (x). Study  0.
(ii) Give a C++ implementation for f (x).
(iii) Calculate the derivative of f (x) = x2 using

11f (x) + 18f (x + ) 9f (x + 2) + 2f (x + 3h)

f 0 (x) := lim .
0 6

Problem 14. Given a sequence of ordered parameters (knots): (x0 , x1 , . . . , xm ),

the ith normalized B-spline basis function (B-function) Ni,k of order k is defined
recursively as

1 if xi x < xi+1
Ni,k (x) = if k = 1
0 otherwise
x xi xi+k x
Ni,k (x) = Ni,k1 (x) + Ni+1,k1 (x) if k > 1
xi+k1 xi xi+k xi+1
with i = 0, 1, . . . , k and k < m. The properties of the B-spline basis functions
are:
1) Partition of unity, i.e.
mk
X
Ni,k (x) = 1.
i=0

2) Positivity
Ni,k (x) 0.
3) Local support
Ni,k (x) = 0 for x
/ [xi , xi+k ].
 Numerical Techniques 65

4) C k2 continuity. If the knots { xi } are pairwise different from each other,

then Ni,k (x) C k2 , i.e., the function Ni,k (x) is (k 2) times continuously
differentiable.

Let m = 4 and

x0 = 0, x1 = 0.5, x2 = 1.0, x3 = 1.5, x4 = 2.0.

Let k = 2. Find N0,2 (x), N1,2 (x) and N2,2 (x). Draw the functions.

Problem 15. Let f be a continuos function in the interval [a, b] (b > a). Then
n   b
ba X k(b a)
Z
lim f a+ = f (x)dx.
n n n a
k=1

Implement in C++ the left-hand side for a given f , a, b and n with

double integrate(double (*f)(double),double a,double b,int n)

Apply it to the function

f (x) = sin(x)
and the interval [0, ] and to the function

f (x) = exp(x ln(x))

with the interval [0, 1]. Choose n = 10, 100, 1000000.

Problem 16. (i) Use the class Derive of SymbolicC++ to find the derivative
of
y = 2x3 5x 1
at the point x = 2. Use the data type double for x.
(ii) Use the class Derive of SymbolicC++ and the class complex over double
to find the derivative of the complex-valued function

w = 2z 2 5z 1

at the point z = i.

Problem 17. Let r be a real number with r 6= 0. Then 1/r can be calculated
to ever increasing precision by using the iteration

xt+1 = xt (2 rxt ), t = 0, 1, 2, . . .

provided the initial value x0 is sufficiently close to 1/r. The number of digits of
precision approximately doubles with each iteration. Write a C++ program to
66 Problems and Solutions

find the inverse of r = 2 with the initial value x0 = 0.8. Why does the initial
value x0 = 1 not work?

Problem 18. In C and C++ the function fabs finds the absolute value of
a floating point number. How can fabs be replaced by an if condition and
multiplication by 1.0?

Problem 19. What is the following code doing

// InvSqrt.cpp

#include <iostream>
using namespace std;

float invSqrt(float x)
{
float xhalf = 0.5f*x;
int i = *(int*)& x; // get bits for floating value
i = 0x5f3759df - (i >> 1); // initial guess
x = *(float*)& i; // convert bits back to float
x *= 1.5f - xhalf*x*x;
x *= 1.5f - xhalf*x*x;
return x;
}

int main(void)
{
float x1 = 10.0f;
float r1 = invSqrt(x1);
cout << "r1 = " << r1 << endl;

float x2 = 100.0f;
float r2 = invSqrt(x2);
cout << "r2 = " << r2 << endl;
return 0;
}

Problem 20. Consider the function f : Rn R, where it is assumed that

the function f is at least twice continuous differentiable. We want to find the
minimum of the function f . Let
 2 
f (x)
H(x) :=
xj xk
be the Hessian matrix and j, k = 1, 2, . . . , n. In the Levenberg-Marquardt algo-
rithm we apply
xt+1 = xt (H(xt ) + diag(H(xt ))1 f (xt )
 Numerical Techniques 67

where t = 0, 1, 2, . . ., is the step length and given initial values. We need

matrix inversion as part of the update. Since the determinant of the Hessian
matrix is proportional to the curvature of f , the iteration implies a large step in
the direction with low curvature (i.e., an almost flat terrain) and a small step in
the direction with hight curvature (i.e. a steep incline). Write a C++ program
that applies this method to solve the system of equation

x1 = sin(x1 + x2 ), x2 = cos(x1 x2 )

by finding the minimum the function

f (x1 , x2 ) = (x1 sin(x1 + x2 ))2 + (x2 cos(x1 x2 ))2 .

Use the initial values x1,0 = x2,0 = 0.

Problem 21. Given a polynomial of degree n which admits n real roots. Write
a C++ program using the Newton method that finds all real roots. Apply the
program to the polynomial

p(x) = x4 7x3 + 8x2 + 2x 1.

Problem 22. A polygon is a closed plane figure with n sides. If all sides and
angles are equivalent the polygon is called regular. The area of a planar convex
polygon with vertices

(x0 , y0 ), (x1 , y1 ), . . . , (xn1 , yn1 )

is given by
n1
1X
A= (xi yi+1 xi+1 yi ), xn x0 , yn y0 .
2 i=0

A polygon in the plane R2 is a closed figure with n sides. If all sides and angles
are equal the polygon is called regular. The area of a planar convex polygon
with vertices
(x0 , y0 ), (x1 , y1 ), . . . , (xn1 , yn1 )
is given by
 
n1 
1  X 
A=  (xj yj+1 xj+1 yj ) , xn x0 , yn y0 .
2  j=1 

(i) Write a C++ program that finds the area of a given planar convex polygon.
Apply the modulus operator % to identify n and 0.
68 Problems and Solutions

(ii) Write a Java program that finds the area of a given planar convex polygon.
Apply the modulus operator % to identify n and 0.

Problem 23. Given a time series { xi : i = 0, 1, . . . , N 1 }, the linear

correlation of an epoch consisting of K points { xi : i = 0, 1, . . . , K 1 } and
another epoch of the same length K covering a different time interval { xk : k =
j, j + 1, . . . , K + j 1 }, K + j N , is given by the cross-correlation coefficient
K1
X (xi hx0 i)(xi+j hxj i)
rj :=
i=0
0 j

where hx0 i and hxj i are, respectively, the mean values of the epochs starting at
x0 and xj and 0 and j the corresponding standard deviations. Write a C++
program that find the correlation coefficient for the logistic map

xt+1 = 4xt (1 xt ), t = 0, 1, 2, . . .

and x0 = 1/3. Let N = 2048 and K = 1024.

Problem 24. The standard Hermite polynomial satisfy the recursion relations
dHn (x)
Hn+1 (x) = 2xHn (x) 2nHn1 (x), = 2nHn1 (x)
dx
with H0 (x) = 1. Combining these two relations we get the recursion relation
 
d
Hn+1 (x) = 2x Hn (x).
dx
Write a C++ program using SymbolicC++ which implement this recursion re-
lation.

Problem 25. Write a C++ program to implement an approximation to the

integral Z x
x3 x5 x7
exp(s2 )ds = x + +
0 3 1! 5 2! 7 3!

Problem 26. Consider the 3 3 matrix

0 1 1
A = 1 0 1.
1 1 0

Find A2 and A3 . We know that

tr(A) = 1 + 2 + 3 , tr(A2 ) = 21 + 22 + 23 , tr(A3 ) = 31 + 32 + 33 .

 Numerical Techniques 69

Use Newtons method to solve this system of equations to find the eigenvalues
of A.

Problem 27. What is the output of the following C++ code

// epsilon.cpp

#include <iostream>
using namespace std;

int main(void)
{
double eps = 1.0, x = 2.0, y = 1.0;
while(y < x) { eps *= 0.5; x = 1.0 + eps; }
eps *= 2.0;
cout << "eps = " << eps;
}

Problem 28. Let x > 1. Then ln(x) can be calculated using the iteration
x0 = x
2x
xj+1 = p j , j = 0, 1, 2, . . .
1 + 1 + 2j xj
Write a C++ program that implements this iteration. Apply it to x = 10 and
x = e.

Problem 29. The junction between p- and n-type semiconductors has prop-
erties which make it the basis of many electronic devices. For a p-n junction
diode we have the equation
 1/2
20 r (nA + nD )(VD U )
d=
enA nD
where d has the dimension of a length (thickness of the deplection layer) and

e = 1.6021 1019 As elementary charge

0 = 8.8542 1012 As/V m permittivity of the vacuum
r = 16 for Germanium
nA = nD = 1022 m3 donor and acceptor concentrations
VD = 0.358 V diffusion voltage
6 2
a = 10 m area of the junction

Write a C++ program that finds d in dependence of the voltage U with 10V
U 0V and step size 0.5V and the capacity

C(d) = 0 r a/d.
70 Problems and Solutions

All the units are in the MKSA system. Thus the result for d will be in meters
and the result for the capacity C will be in Farad (= s4 A2 /m2 kg).

Problem 30. Use numerical integration to show that

Z 1
x
2
ln(1 + x)dx = 0.162865007.
0 1+x

Problem 31. Consider the one-dimensional Schrodinger equation (eigenvalue

problem)
~2 d2 u(x)
+ V (x)u(x) = Eu(x).
2m dx2
For numerical studies the eigenvalue equation is replaced by the difference equa-
tion
~2 un+1 + un1 2un
 
+ Vn un = Eun
2m 2
where := xn+1 xn (step size), un := u(xn ), Vn = V (xn ). Imposing the bound-
ary conditions the set of linear equations can be solved as eigenvalue problem of
a symmetric matrix over the real numbers. Show that the error in the represen-
tation relative to (2mE/~2 )un is
1 2 (4)
un .
12
The Numerov-Cooley approximation uses the second order difference equation

~2 un+1 + un1 2un

 
5 1 1
2
= (EVn )un + (EVn+1 )un+1 + (EVn1 )un1 .
2m 6 12 12

Show that relative error using this approximation is

29 4 (6)
un .
300
Note that this approximation gives an asymmetric matrix equation.

Problem 32. Let x > 0. Find the solution of the equation

(x 2)2 = ln(x).

First show that the equation has a root in [1, 2].

Problem 33. Consider the mathematical expression

sin(b) + a b + c d + (a b).
|{z}
 Numerical Techniques 71

(i) Write this mathematical expression as a binary tree with the root indicated
by the underbrace. Then evaluate this binary tree from bottom to top with the
values a = 2, b = /2, c = 4, d = 1.
(ii) Am alternative to represent a mathematical expression as tree is multiex-
pression programming. Use multiexpression programming to evaluate the math-
ematical expression given above.

Problem 34. Find an approximation of 30 utilizing

30 = 25 + 5 = 5 1 + 0.2.
Chapter 9

Random Numbers

Problem 1. Write a C++ program that implements the Durstenfeld algorithm

for randomly shuffling a one-dimensional array of integers.

Problem 2. Calculate the integral

Z 1
| cos(2x)|dx
0

using the random numnber generator described in problem 7, chapter 10, page
250, Problems and Solutions in Scientific Computing. Compare to the exact
result by solving the integral.

72
Chapter 10

Optimization Problems

Problem 1. Consider an overdetermined linear system Ax = b, where A is

an m n matrix with m > n. Thus x is a column vector with n rows and b
is a column vector with m rows. Write a genetic algorithm program that finds
the Chebyshev or minmax solution to set of overdetermined linear equations
Ax = b, i.e. the column vector x which minimizes
 

Xn 
 
c = max ci max bi
 aij xj  .
1im 1im 
j=1 

Apply the program to the overdetermined linear system

1 1 1 1


1 0.5 0.25 x1 0.5
1 0 0 x2 = 0 .

1 0.5 0.25 x3 0.5

1 1 1 2.0

73
Chapter 11

String Manipulations

Problem 1. A protein sequence can be considered as a sequence from a 20-

letter alphabet A = { a1 , a2 , . . . , an } where n = 20 and ai is called the letter
of type i (1 i n). Let A` be the set of sequences of length ` over A for
a positive integer `. Given a sequence S in A` , we denote by vi the number
of occurrences of ai in S (1 i n). We have v1 + v2 + + vn = `. The
vector V = (v1 , v2 , . . . , vn ) is called the frequency vector of S. For a sequence
S = s1 s2 . . . s`1 s` of length `, the Shannon entropy of S is defined as

n
X vi v 
i
H(S) = log .
i=1
` `

This can be regarded as the average information per position in the sequences.
Write a C++ program that implements H(S).

Problem 2. Given a string of digits with or without a decimal point, for

example 345678 or 12.3456. Write a C++ program that converts the string
to a floating point number double.

Problem 3. Write a C++ program that implements the Levenshtein distance

(also called the edit distance).

Problem 4. Write a C++ program that find the first character in an ASCII
string that occurs only once. Find a solution that minimizes the number of
comparisons between characters. Note that \0 denotes the null character.

74
 String Manipulations 75

Problem 5. Given a totally ordered infinite alphabet A, represented by

{ 1, 2, 3, . . . }. The free associative algebra over an alphabet A, i.e., the al-
gebra spanned by words, the product being the concatenation, is denoted by
KhAi, and its unity (the empty word) is denoted by . Here, K is some field
of characteristic zero. An algebraic structure on KhAi is known as the shuffle
product. Let w1 and w2 be two words. Then the shuffle product w1 t w2 is
recursively defined by

w1 t  = w1
 t w2 = w2
au t bv = a(u t bv) + b(au t v)

where a, b are letters and u, v are words. For example

12 t 43 = 1(2 t 43) + 4(12 t 3)

= 12( t 43) + 14(2 t 3) + 41(2 t 3) + 43(12 t )
= 1243 + 142( t 3) + 143(2 t ) + 412( t 3) + 413(2 t ) + 4312
= 1243 + 1423 + 1432 + 4123 + 4132 + 4312.

Write a C++ program that implements this product.

Problem 6. (i) Write a C++ program that uses the string class and the line
string* sa = new string[N];

and then concatentes the strings.

(ii) Write a C++ program that uses the string class and the line
string* sa = new string(6, );

and then concatentes the characters.

Chapter 12

Programming Problems

Problem 1. The rank of an element in a sequence (one-dimensional array) of

numbers is the number of smaller elements in the sequence plus the number of
equal elements that appear to its left. For example, if the sequence is given as
the one-dimensional array a = [4, 3, 9, 3, 7], then the ranks are r = [2, 0, 4, 1, 3].
Write a C++ program with a function void rank(T* a,int n,int* r) that
computes the ranks of the elements of the array a[0 : n 1]. Once the elements
have been ranked using the function rank() write a function rearrange() that
rearrange them in nondecreasing order so that a[0] a[1] a[n 1] by
moving elements to positions corresponding to their ranks.

Problem 2. Write a C++ program that transposes a square matrix in-place.

Problem 3. Given an array of 20 integer numbers. We would like to assign

the numbers 0, 4, 8, . . . , 76 to the array, i.e.,
array[0] = 0, array[1] = 4, ... , array[19] = 76.

Can the multiplication i*4 in the following C++ program forloop1.cpp be

avoided and replaced by addition? Can the C++ program be even more efficient?
// forloop1.cpp

#include <iostream>
using namespace std;

int main(void)
{
int* array = new int[20];

76
 Programming Problems 77

for(int i=0;i<20;i++) { array[i] = i*4; }

delete[] array;
return 0;
}

Problem 4. In wavelet theory we start from a function (the so-called mother

wavelet) and then by scaling and translation of the dependent variable x we
generate a set of new functions. The most studied mother wavelet is the Haar
function implemented in the following C++ program.

// wavelets.cpp

#include <iostream>
using namespace std;

template <class T> T transform(T (*f)(T),T x,T a,T b)

{ return f(a*x + b); }

double H(double x)
{
if((x >= 0.0) && (x <= 0.5)) return 1.0;
if((x > 0.5) && (x <= 1.0)) return -1.0;
return 0.0;
}

int main(void)
{
double r1 = transform(H,-4.0,0.5,4.0);
cout << "r1 = " << r1 << endl;
double r2 = transform(H,-1.6,2.0,4.0);
cout << "r2 = " << r2 << endl;
double r3 = transform(H,2.6,4.0,-10.0);
cout << "r3 = " << r3 << endl;
return 0;
}

What is the output of the program?

Problem 5. Let n be a positive integer. Then the solutions of z n = 1 are

called the roots of unity. For example, if n = 4 we have z1 = 1, z2 = i, z3 = 1,
z4 = i. Write a C++ program using the class complex<double> that stores
the unit roots in an array for a given n.

Problem 6. Extend the C++ program Gauss0.cpp to a template basis

so that it can also be used for other data types such as Rational<int> and
Rational<Verylong>.
78 Problems and Solutions

Problem 7. Consider the system of nonlinear equations

3x2 2y 2 4z 2 + 54 = 0, 5x2 3y 2 7z 2 + 74 = 0.

Write a C++ program that finds all integer solutions in the range 0 x 100,
0 y 100, 0 z 100.

Problem 8. Let a0 , a1 , a2 , . . . , an1 be a finite sequence of numbers. Its

Cesaro sum is defined as
1
(s0 + s1 + s2 + + sn1 )
n
where
sk = a0 + a1 + + ak
for each k, 0 k (n 1). Write a C++ program that finds the Cesaro sum
for a given finite sequence of numbers. Use templates so that the program can
also be used for complex numbers, rational numbers etc.

Problem 9. Write a C++ program that finds the maximum of the function

f (x) = x21 2x1 + x22 + 3x2 + x23 + 4x3

subject to the constraint

x21 + 2x22 + 3x23 17
where x1 , x2 , x3 are non-negative integers.

Problem 10. Let x, y Z. Consider the equation

x4 + y 4 + 79 = 48xy.

Write a C++ program that finds all solutions in the range 10 x 10 and
10 y 10.

Problem 11. LISP is latently typed, i.e. does not explicitly specify the
underlying data type for variables. Is it possible to achieve the same with C++?
In other words, can be construct an abstract datatype in C++ such that the
underlying data type can be stored and used in a form that does not explicitly
state the underlying data type? Give a possible implementation in standard
C++.

Problem 12. Write a C++ program that finds the numbers of integer solu-
tions of the equation i1 + i2 + i3 = 12 satisfying the following constraints

0 i1 6, 0 i2 6, 0 i3 3.
 Programming Problems 79

Problem 13. The vector class of the Standard Template Library in C++
is a container class. Arithmetic operation such as addition of vectors are not
implemented. Write a C++ program that overloads + so that one can add two
vectors.

Problem 14. Let x be the number of man, y be the number of woman and
z be the number of children. Altogether there are 100 persons. Given 100 kg
of potatos. Every man gets 3 portions, a woman gets 2 portions and a child
gets 1/2 a portions. Find all (integer) solutions. We have two linear equations
with three unkowns, however we have the constraint that x, y, z are nonnegative
integers. Write a C++ program that finds all these integer solutions.

Problem 15. Consider a linked list. Determine if the linked list loops using
only two pointers.

Problem 16. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Consider the program (page 25)
au.cpp. Where is the amplitude? Extend the program to two or more sine
waves (for example frequency 880 besides 440).

Problem 17. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Consider the program (page 8)
SineSound.java. Run the Java program with the first line (page 10) changed
to
...new File("sine.au"));
with WAVE also changed. Compare to the previous problem. Extend the program
to get in more frequencies.

Problem 18. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Consider the program (page 31)
LGB.java. Rewrite the program in C++ either with the vector class of STL or
plain (just functions).

Problem 19. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Consider the program (page 44)
Noise.java. Rewrite the program into C++.

Problem 20. The problem refers to the book Mathematical Tools in Sig-
nal Processing with C++ and Java Simulations. Consider the Matlab Filter
Implementation (page 49). Rewrite the code in C++.

Problem 21. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Consider the program (page 60)
80 Problems and Solutions

NonCircular.java. Rewrite the code in C++.

Problem 22. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Give a Java implementation of
the two-dimensional convolution (page 61).

Problem 23. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Implement the two-dimensional
Fourier transform in C++ (page 72).

Problem 24. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Implement the two-dimensional
Cosine transform (page 76).

Problem 25. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Use the complex class of STL and
do someting useful with the z-transform (chapter 8).

Problem 26. The problem refers to the book Mathematical Tools in Sig-
nal Processing with C++ and Java Simulations. Implement two-dimensional
wavelets (page 89).

Problem 27. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. In the C++ program (page 138)
the total number of distinct observations is 2. Extend the program to more
observations.

Problem 28. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Extend the C++ code fragment
(page 205) to a complete C++ program.

Problem 29. The problem refers to the book Mathematical Tools in Signal
Processing with C++ and Java Simulations. Implement the decompression
procedure (page 216) in C++.

Problem 30. Write a Java program Gauss.java that implements Gauss elim-
ination to solve linear equation with n equations and n unkowns. Apply the
program to the system

0 0 1 x0 1
0 1 0 x1 = 2 .
1 0 0 x2 3
 Programming Problems 81

Problem 31. Which of the following C++ program fragments will loop for-
ever?
int i = 1;
while(i != 0) i = i + 1;

int j = 1;
while(j != 0) j = 2*j + 1;

double x = 1.0;
while(x != 0) x = x/2.0;

Problem 32. Write down the tree expression for

((a/b) 4) + ((1 + c) (2 d)).

Then evaluate the tree expression for a = 1, b = 2, c = 3, d = 4.

Problem 33. How would one store a matrix using a linked list?
Chapter 13

Applications of STL in
C++

Problem 1. Given two sets of strings, for example,

{ "Jones", "Miller", "Steeb", "Smith" }
{ "Hardy", "Copper", "Steeb" }
Apply the set class of STL to find the union (method set_union), the intersec-
tion (method set_intersection), and the difference (method set_difference)
of the two sets. Also apply the method includes to test for subsets. Finally
use the method size() to find the number of elements in the sets.

Problem 2. Given a set of n strings, say

"ballon", "tree", "fish", "dog", "cat"
with n = 5. We want to display all possible m = 2n subsets of S (including the
set itself and the empty set represented by { }). Use recursion.

Problem 3. Given a set of positive integers, for example,

{ 2, 3, 11, 7, 4, 28, 41, 16 }
Write a C++ program using set<int> that partitions the set into two subsets
of even and odd numbers, respectively.

Problem 4. Write a useful C++ program that uses the function find_if().
The function find_if takes a predicate (function object or function) as param-
eter.

82
 Applications of STL in C++ 83

Problem 5. A function f is a set of pairs (a, b) where a A and b B are

elements of the sets A and B, respectively, such that for each a A there is
exactly one b B such that (a, b) f .

Example. Consider the sets

A = {one, two, three, f our, f ive}.
and
B = {1, 2, 3, 4, 5}.
Let
f = {(one, 1), (two, 2), (three, 3), (f our, 4), (f ive, 5)}
then f (one) = 1, f (two) = 2 etc. The set f denotes a function since each ele-
ment of A appears as the first of a pair exactly once in f . The set A is called
the domain of f and the set B is called the range of f .

We usually write f : A B and f (a) = b where (a, b) f .

Example. Consider
A = {1, 2, 3, 4, 5}
and
B = {true, f alse}.
Let
g = {(1, f alse), (2, true), (3, true), (4, f alse), (5, true)}
then g(2) = true, g(4) = f alse etc. The function g associates with each number
in A the value in B which indicates whether the number is prime.

Let f : A B and g : B C where A, B and C are sets. Then, by convention,

a A f (a) B. Consequently
a A f (a) B g(f (a)) C.
Thus we define function composition
g f : A C, g f = {(a, g(f (a))) | a A}.
Note that the functions f and g need to be compatible for function composition,
i.e. the range of f must be contained in (subset of) the domain of g.

Example. Let f and g denote the two functions from the examples above and
(we rename the sets for clarity)
A = {one, two, three, f our, f ive},
B = {1, 2, 3, 4, 5},
C = {true, f alse}.
84 Problems and Solutions

Thus f : A B and g : B C. Consequently

g f = {(one, g(f (one))), (two, g(f (two))), (three, g(f (three))),

(f our, g(f (f our))), (f ive, g(f (f ive)))}
f = {(one, g(1)), (two, g(2)), (three, g(3)), (f our, g(4)), (f ive, g(5))}
f = {(one, f alse), (two, true), (three, true), (f our, f alse), (f ive, true)}.

In other words (gf )(two) = g(f (two)) = g(2) = true and (gf )(f our) = f alse.

Finite sets of a homogeneous nature, such as integers (represented by int in

C++) allow a simple representation of functions in C++ using the STL data type
map. The template class map<A,B> associates pairs of type A and B. Implement
the examples above using the map data type. Implement function composition
for two arbitrary (compatible) maps.

Problem 6. The spiral map (Problems and Solutions in Scientific Computing,

page 79) is a 1 to 1 map (i.e. invertible) which maps Z2 Z. Write a C++
program using the map class and pair that stores the elements of the map as
map<int,pair<int,int> >

Problem 7. The group table for a group with three elements a,b,e is given
by

e a b
e e a b
a a b e
b b e a

where e is the neutral element (identity) of the group. Write a C++ program
that implements this group table using the map class and the string class. For
example, the user should enter a*b and the output should be e.

Problem 8. The STL set class is a template class. Thus we may construct for
example set<int>, set<double>, and set<string>. In mathematics sets are
not required to be of a homogeneous type. We should be able to store for example
both integers and strings in a mathematical set. Implement a class AnyData such
that set<AnyData> can contain elements of any type. For example, it should be
possible to do the following:
set<AnyData> s;
s.insert(2);
s.insert(string("string"));
s.insert(3.5);
set.insert(2);
 Applications of STL in C++ 85

Problem 9. Let C be the complex plane. Let c C. The Mandelbrot set M

is defined as

M := { c C : c, c2 + c, (c2 + c)2 + c, . . . , 6 }.

To find the Mandelbrot set we study the recursion relation

zt+1 = zt2 + c, t = 0, 1, 2, . . .

with the initial value z0 = 0 and whether zt escapes to infity. For example
c = 0 and c = 1/4 + i/4 belong to the Mandelbrot set. The point c = 1/2 does
not belong to the Mandelbrot set. Write a C++ program using the complex
class of STL to find the Mandelbrot set. The output should be written to a file
Mandel.pnm (portable anymap utilities). This file can then be used to display
the fractal.

Problem 10. The spiral map is a 1 : 1 map from Z Z2 . Write a C++

program using the map class and pair that stores the elements of the map as

map<int,pair<int,int> >

Problem 11. A priority queue is a type of queue that assigns a priority to

every element that it stores. New elements are added to the queue using the
push() function. Thus it is a set for which two operations are defined:
1) Adding an item (using push())
2) Extracting the item that has the highest priority using top() and pop().

We may think of a priority queue as a set of tasks with priorities. At any time
a new task can be added. A task can also be removed from the priority queue,
but this can only be the one with the highest priority. If this highest priority
is shared by more than one task, we do not care which one is taken. Write a
C++ program that uses the priority queue from STL. Apply it to floating point
numbers so that bigger numbers get a higher priority. Apply it to strings so that
strings lexicographicaly higher get a higher priority (case sensitive).

Problem 12. The ancient puzzle of the Tower of Hanoi consists of a number of
wooden disks mounted on three poles, which are in turn attached to a baseboard.
The disks each have different diameters and a hole in the middle large enough
for the poles to pass through. At the beginning all disks are on the left pole
with the smallest at the top, the second smallest one down etc. The object of
the puzzle is to move all the disks over to the right pole, one at the time, so that
they end up in the original order on that pole. One uses the middle pole as a
temporary resting place for the disks. However it is allowed for a larger disk to
be on top of a smaller one. For example if we have three disks then the moves
are
86 Problems and Solutions

move disk A from pole 1 to 3

move disk B from pole 1 to 2
move disk A from pole 3 to 2
move disk C from pole 1 to 3
move disk A from pole 2 to 1
move disk B from pole 2 to 3
move disk A from pole 1 to 3
total number of moves: 7

(i) Write a C++ program using recursion to implement the Tower of Hanoi.
(ii) Write a C++ program using the stack class of the standard template li-
braray to implement the Tower of Hanoi.

Problem 13. Using the Verylong class of SymbolicC++ and the complex
class (of STL) that finds positive integer solutions (a, b, c) of the equation

c = (a + bi)3 107i

where i2 = 1.

Problem 14. Let i = 1. Calculate ii . Use the complex class of the
standard template library of C++ to calculate ii . Discuss.
Chapter 14

Particle Swarm
Optimization

Problem 1. Particle Swarm Optimization (PSO) is based on the behavior of

a colony or swarm of insects, such as ants, termites, bees, and wasps; a flock
of birds; or a school of fish. The particle swarm optimization algorithm mimics
the behavior of these social organisms. The word particle denotes, for example,
a bee in a colony or a bird in a flock. Each individual or particle in a swarm be-
haves in a distributed way using its own intelligence and the collective or group
intelligence of the swarm. As such, if one particle discovers a good path to food,
the rest of the swarm will also be able to follow the good path instantly even if
their location is far away in the swarm. Optimization methods based on swarm
intelligence are called behaviorally inspired algorithms as opposed to the genetic
algorithms, which are called evolution-based procedures. The PSO algorithm
was originally proposed by Kennedy and Eberhart in 1995.

In the context of multivariable optimization, the swarm is assumed to be of

specified or fixed size with each particle located initially at random locations
in the multidimensional design space. Each particle is assumed to have two
characteristics: a position and a velocity. Each particle wanders around in the
design space and remembers the best position (objective function value) it has
discovered. The particles communicate information or good positions to each
other and adjust their individual positions and velocities based on the informa-
tion received on the good positions.
The PSO is developed based on the following model:

1. When one particle locates an extremum point of the objective function, it

87
88 Problems and Solutions

instantaneously transmits the information to all other particules.

2. All other particles gravitate to the extremum point of the objective function,

but not directly.

3. There is a component of each particles own independent thinking as well as

its past memory.

Thus the model simulates a random search in the design space for the extremum

points of the objective function. As such, gradually over many iterations, the
particles go to the target (the extremum point of the objective function).
The algorithm for determining the maximum of a function f (x) (with x an n

dimensional vector) is as follows:

1) Initialize the number of particles N , the search intervals for each dimension
(ai , bi ), i = 1, . . . , n (n being the dimension of the search space), the search
precision for each dimension i , the maximum number of iterations imax .
Initialize the positions of the particles xj (0) = rand(), j = 1, . . . , N randomly
in the search domain.
Initialize the speeds of the particles vj (0) = 0, j = 1, . . . , N .
Initialize the individual best positions of the particles xbest,j (0) = xj (0), j =
1, . . . , N .
Initialize the iteration count k = 0.

2) Check the stop conditions: the diameter of the swarm in each dimension is
less then the dimensions precision i , or the maximum number of iterations imax
was reached. If yes then terminate else continue to step 3).

3) Calculate the values of the function f (x) in the current positions of the par-
ticles, f (xj (k)), j = 1, . . . , N . Update the values of the best individual points
for each particle xbest,j (k), j = 1, . . . , N and the value of the global best point
xbest (k). Continue to 4).

4) Update the particles speeds by applying the formula:

vj (k) = (k)vj (k 1) + c1 r1 [xbest,j (k) xj (k 1)] + c2 r2 [xbest (k) xj (k 1)]

where j = 1, . . . , N and r1 and r2 are random number between 0 and 1. The

parameters c1 and c2 have usually the value 2 so that the particles would overfly
the target about half of the time. (k) is the inertia weight dependent of the
iteration count according to the formula:
 
max min
(k) = max k
imax
 Particle Swarm Optimization 89

where max is a maximum value and min is a minimum value, typically max =
0.9 and min = 0.4. Continue to 5).
5) Update the particles positions by applying the formula:

xj (k) = xj (k 1) + vj (k), j = 1, . . . , N

6) Increment the iteration count k and go to 2).

Determine the maximum of the function

f (x) = x2 + 2x + 11, 2 x 2

by applying the PSO (Particle Swarm Optimization) method. The required

precision is 104 .

Problem 2. Minimize

f (x1 , x2 ) = x21 + x22 2x1 4x2

subject to the constraints

x1 + 4x2 5 0, 2x1 + 3x2 6 0, x1 0, x2 0

by applying the Particle Swarm Optimization (PSO) method. The required pre-
cision is 103 .

Problem 3. Find the global minimum of the De Jongs function (or sphere
model)
n
X
f (x) = x2i , n 2
i=1

by applying the Particle Swarm Optimization (PSO) method. The required

precision is 103 .

Problem 4. Differntial evolution (DE) is a population-based optimization

method that attacks the starting point problem by sampling the objective func-
tion at multiple, randomly chosen initial points. At initialization a vector pop-
ulation of dimension Np is generated such that the allowed parameter region is
entirely covered. Each vector is indexed with a number from 0 to Np 1 for
bookkeeping because each of them has to enter a competition. Like other Evolu-
tionary Strategy population-based methods, DE generates new points that are
perturbations of existing points, but unlike other Evolutionary Strategy meth-
ods, DE perturbs population vectors with the scaled difference of two randomly
90 Problems and Solutions

selected population vectors to produce the trial vectors. Assume that we pro-
duce the trial vector with index 0, u0 . DE selects randomly two distinct vectors
xr1 and xr2 from the population and adds the scaled perturbation xr1 xr2 to
a third vector also randomly selected from the population xr3 , distinct from the
first two vectors. The procedure is repeated in order to generate all the set of
trial vectors u0 , u1 , . . . , uNp . In the selection stage, each trial vector competes
against the vector in the population vectors with the same index. The vector
with the lower objective function value is selected as a member of the next gen-
eration. The survivors of the Np pairwise competitions become parents for the
next generation in the evolutionary cycle. The evolutionary cycles are repeated
until a termination criteria is meet, for example the diameter of the population
becomes less than a small limit value  or a maximum number of population
generations were generared.
The more detailed algorithm for determining the global minimum of a function

f (x) (with x an n dimensional vector) is as follows:

1) Initialization - Initialize the number of vectors Np , the search intervals for

each dimension (ai , bi ), i = 1, . . . , n, the precision for the termination crite-

ria , the maximum number of iterations imax , the scale factor F (0, 1+),
the crossover probability Cr [0, 1). Initialize the positions of the particles
(0)
xi,j = aj + randi,j ()(bj aj ), i = 1, . . . , Np j = 1, . . . , n randomly in the
search domain (the random number generator, rand(), returns a uniformly dis-
tributed random number from within the range [0, 1)). Initialize the iteration
count k = 0.
2) Mutation - differential mutation adds a scaled, randomly sampled, vector

difference to a third randomly sampled vector to create a mutant vector

(k) (k) (k) (k)
vi = xr3 + F (xr1 xr2 ) i = 1, . . . Np
The scale factor, F (0, 1+), is a positive real number that controls the rate
at which the population evolves. While there is no upper limit on F , effective
values are seldom greater than 1. The base vector index, r3, can be determined
in a variety of ways, but for now it is assumed to be a randomly chosen vector
index that is different from the target vector index, i. Except for being distinct
from each other and from both the base and target vector indices, the difference
vector indices, r1 and r2, are also randomly selected once per mutant.
3) Crossover - to complement the differential mutation search strategy, DE also

employs uniform crossover. Sometimes referred to as discrete recombination,

(dual) crossover builds trial vectors out of parameter values that have been
copied from two different vectors. In particular, DE crosses each vector with a
mutant vector:
(
(k)
(k) vi,j if rand()i,j < Cr or j = jrand
ui,j = (k) i = 1, . . . , Np , j = 1, . . . , n
xi,j otherwise
 Particle Swarm Optimization 91

The crossover probability, Cr [0, 1), is a user-defined value that controls the
fraction of parameter values that are copied from the mutant. To determine
which source contributes a given parameter, uniform crossover compares Cr to
the output of a uniform random number generator. If the random number is less
than or equal to Cr , the trial vector component is inherited from the mutant
(k) (k)
vi ; otherwise, the trial vector component is copied from the vector, xi . In
addition, the trial vector component with randomly chosen index, jrand , is taken
(k)
from the mutant to ensure that the trial vector does not duplicate xi . Because
of this additional demand, Cr only approximates the true probability, pCr , that
a trial parameter will be inherited from the mutant.
(k)
4) Selection - if the trial vector, ui , has an equal or lower objective function

(k)
value than that of its target vector, xi , it replaces the target vector in the next
generation; otherwise, the target retains its place in the population for at least
one more generation:
(
(k) (k) (k)
(k) ui if f (ui ) < f (xi )
xi = (k) i = 1, . . . , Np
xi otherwise

By comparing each trial vector with the target vector from which it inherits pa-
rameters, DE more tightly integrates recombination and selection than do other
Evolutionary Algorithms.
5) Termination - check the termination conditions: the diameter of the pop-

ulation is less then the preset precision , or the maximum preset number of
generations imax was reached. If yes then terminate, else increment the itera-
tion count k and go to 2).

Determine the global minimum of the function

2
+(y+1)2 ) x 2 2 1 2 2
f (x, y) = 3(1 x)2 e(x 10( x3 y 5 )e(x +y ) e((x+1) +y )
5 3
where
4 x 4, 4 y 4
by applying the DE method. The required precision is 104 .
92 Problems and Solutions
Bibliography

Steeb W.-H., Hardy Y., Hardy A. and Stoop R.

Problems and Solutions in Scientific Computing with C++ and Java Simula-
tions
World Scientific Publishing, Singapore (2004)

Steeb W.-H.
The Nonlinear Workbook: Chaos, Fractals, Cellular Automata, Neural Networks,
Genetic Algorithm, Gene Expression Programming, Wavelets, Fuzzy Logic, fifth
edition
World Scientific Publishing, Singapore 2011
ISBN 978-981-4335-77-5
http://www.worldscibooks.com/chaos/8050.html

Hardy Y., Kiat Shi Tan and Steeb W.-H.

Computer Algebra with SymbolicC++

World Scientific Publishing, Singapore 2008

ISBN-13: 978-981-283-360-0
http://www.worldscibooks.com/mathematics/6966.html

Steeb W.-H.,
Mathematical Tools in Signal Processing with C++ and Java Simulations
World Scientific Publishing, Singapore 2005
ISBN 981 256 500 0
http://www.worldscibooks.com/engineering/5939.html

93
Index

B-spline basis function, 64 Givens transform, 47

Arithmetic mean, 53 Hadamard product, 47

Halleys method, 62
Ballot numbers, 30 Hankel matrix, 48
Bell numbers, 38 Harmonic series, 63
Bernoulli numbers, 32 Hessian matrix, 66
Hough transform, 16
Cantor pairing function, 19
Catalan constant, 7 Integration by parts, 54
Catalan numbers, 53
Catalan recurrence, 53 Konigs root-finding algorithm, 62
Catastrophic cancellation, 62
Cesaro sum, 78 Lambert W function, 63
Chinese remainder theorem, 23 Levenberg-Marquardt algorithm, 66
Chinese rings puzzle, 38 Levenshtein distance, 74
Cross-correlation coefficient, 68
Mobius band, 18
Difference operator, 10 Majority gate, 13
Division algorithm, 24 Mandelbrot set, 85
Doppler shift, 63 minmax solution, 73
Dumbells, 36 Modulus operator, 68
Durstenfeld algorithm, 72
Dyk word, 58 Newtons method, 62
Nobles, 34
Elliptic curve cryptography, 22
Palindome, 34
Fermat numbers, 34 Partition of unity, 64
Ferrers diagram, 32, 38 Partitions, 30
Freudenstein equation, 64 Perfect number, 31
Frobenius symbol, 33 Permanent, 50
Planck length, 2
Gauss elimination, 47 Poisson summation formula, 6
Gaussian curvature, 18 Polygon, 67
Generalized integration by parts, 16 Prime number, 22
Generating function, 59
Geometric means, 53 Quadratic equation, 62

94
Index 95

Resultant, 48

Shannon entropy, 74
Shuffle product, 75
Sinc function, 4
Spiral map, 85
Stirling number, 40

Toeplitz matrix, 43
Tridiagonal form, 47

Wavelet theory, 77