Dynamic

Programming

Dynamic Programming
Dynamic

Programming is an algorithm design

method that can be used when the solution to
a problem may be viewed as the result of a
sequence of decisions

Example Fibonacci Numbers

using recursion
Function f(n)
if n = 0
output 0
else
if n = 1
output 1
else
output f(n-1) + f(n-2)

Example Fibonacci Numbers

using recursion
Run

time: T(n) = T(n-1) + T(n-2)

This function grows as n grows
The run time doubles as n grows and is order
O(2n).
This is a bad algorithm as there are
numerous repetitive calculations.

Example Fibonacci Numbers

using Dynamic programming
f(5)
f(4)
f(3)
f(2)

f(3)
f(2)

f(1)

f(2)

f(1)

Example Fibonacci Numbers

using Dynamic programming
Dynamic

Programming calculates
bottom to top
Values are stored for later use.
This reduces repetitive calculation.

from

Greedy vs. Dynamic Programming :

Both techniques are optimization techniques, and both

build solutions from a collection of choices of individual
elements.
The greedy method computes its solution by making its
choices in a serial forward fashion, never looking back or
revising previous choices.
Dynamic programming computes its solution bottom up by
synthesizing them from smaller sub-solutions, and by trying
many possibilities and choices before it arrives at the
optimal set of choices.
There is no priori test by which one can tell if the Greedy
method will lead to an optimal solution.
By contrast, there is a test for Dynamic Programming,
called the principle of optimality

Statement of Principle of Optimality

Principle of optimality: Suppose that in solving a

problem, we have to make a sequence of
decisions D1, D2, , Dn. If this sequence is optimal,
then the last k decisions, 1 k n must be optimal.

Divide and Conquer

vs.
Dynamic Programming:

Both techniques split their input into parts, find

subsolutions to the parts, and synthesize larger
solutions from smaller ones.
Divide and Conquer splits its input at pre-specified
deterministic points (e.g., always in the middle)
Dynamic Programming splits its input at every
possible split points rather than at a pre-specified
points. After trying all split points, it determines which
split point is optimal.

Multistage Graphs

To find a shortest path in a multi-stage graph

Apply the greedy method :

the shortest path from S to T :
1+2+5=8

The shortest path in multistage

A
D
graphs
e.g.
4

11

18

16

13

The greedy method can not be applied to this

case:
(S, A, D, T) 1+4+18 = 23.
The real shortest path is:
(S, C, F, T) 5+2+2 = 9.

Dynamic programming approach

Dynamic programming approach (forward approach):

A

1
11

d(A, T)

18

16

B
5

d(B, T)

13

d(C, T)

d(S, T) =
4

11

D
E

d(D, T)

d(E, T)

d(A,T) = min{4+d(D,T), 11+d(E,T)}

= min{4+18, 11+13} = 22.

Dynamic programming
4

1
11

18

16

13

d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}

= min{9+18, 5+13, 16+2} = 18.
d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
The above way of reasoning is called backward
reasoning.

Backward approach
(forward reasoning)

d(S, A) = 1
d(S, B) = 2
d(S, C) = 5

1
11

18

16

13

d(S,D)=
= min{ 1+4, 2+9 } = 5
d(S,E)=min{d(S, A)+d(A, E),d(S, B)+d(B, E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S, A)+d(A, F),d(S, B)+d(B, F)}
= min{ 2+16, 5+2 } = 7

d(S,T) = min{d(S, D)+d(D, T),d(S,E)+d(E,T),

d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
=9

Principle of optimality-

the shortest

path problem

If i, i1, i2, , j is a shortest path from i to j, then i1, i2,

, j must be a shortest path from i1 to j

In summary, if a problem can be described by a

multistage graph, then it can be solved by dynamic
programming.

Dynamic programming

Forward approach and backward approach:

Note: If the recurrence relations are formulated

using the forward approach then the relations are
solved backwards . i.e., beginning with the last
decision
On the other hand if the relations are formulated
using the backward approach, they are solved
forwards.

To solve a problem by using dynamic

programming:

Find out the recurrence relations.

Represent the problem by a multistage graph.

0-1 Knapsack problem

Given a knapsack with maximum capacity
W, and a set S consisting of n items
Each item i has some weight wi and benefit
value bi (all wi , bi and W are integer values)
Problem: How to pack the knapsack to
achieve maximum total value of packed
items?

Using only item1 to itemi and knapsack

weight at most w

The Traveling Salesperson

Problem
Given

a list of cities and their pairwise

distances, the task is to find the shortest
possible route that visits each city exactly
once.

Salesman wishes to travel around a given

set of cities, and return to the beginning,
covering the smallest total distance

The Traveling Salesperson Problem

Instance: n vertices
(cities), distance
between every pair of
vertices
Question: Find
shortest (simple)
cycle that visits every
city

2
3
1

2
4
3

5
3

2
2

4
3

2
5

2
3

Applications of TSP
Collection

and delivery problems

Robotics
Board

drilling

The Traveling Salesperson Problem

Assumption: A tour starts and ends at vertex 1

The principle of optimality holds
g(i,S) = length of a shortest path starting at vertex i, going through all
vertices in S, and terminating at vertex 1

g(1,V-{1}) =
g(i,S) =

Solving the recurrence relation

g(i,) = c , 1in
i1

Then obtain g(i,S) for all S of size 1

Then obtain g(i,S) for all S of size 2
Then obtain g(i,S) for all S of size 3

Finally obtain g(1,V-{1})

The Traveling Salesperson Problem

Example
For |S|=0
g(2,)=c =5
21

g(3,)=c31=6

g(4,)=c41=8

For |S|=1
g(2,{3})=c +g(3,)=15
23

g(2,{4})=c24+g(4,)=18

g(3,{2})=c32+g(2,)=18

g(3,{4})=c34+g(4,)=20

g(4,{2})=c42+g(2,)=13

g(4,{3})=c43+g(3,)=15

3
(a)

(b)

The Traveling Salesperson Problem

Example
For |S|=2
g(2,{3,4}) = min {c +g(3,{4}), c +g(4,{3})} = 25
23
24

g(3,{2,4}) = min {c32+g(2,{4}), c34+g(4,{2})} = 25

g(4,{2,3}) = min {c42+g(2,{3}), c43+g(3,{2})} = 23

For |S|=3
g(1,{2,3,4} = min {c +g(2,{3,4}), c +g(3,{2,4}),
12
13
c14+g(4,{2,3})} =
= min {35, 40, 43} = 35
g(1,V-{1}) =

Example 2

The Traveling Salesperson Problem

Time complexity
Let N be the number of g(i,S) that have to be
computed before it can be used to compute g(1,V-{1})

N=

Total time = O(n22n)

This is better than enumerating all n! different tours

Optimal Binary Search

Tree

Optimal Binary Search Tree

We now want to focus on the construction of
binary search trees for a static set of identifiers.
And only searches are performed.
To find an optimal binary search tree for a given
static file, a cost measure must be determined
for search trees.
Its reasonable to use the level number of a node
as the cost.

Binary Search Tree Example

for

for

while

do

return

if
4 comparisons
in worst case

return

do

if
3 comparisons
in worst case

while

Extended Binary Tree

Example
for

for

while

do

return

return

do

if

while

if

(b)
(a)

External Path Length and Internal

Path Length

External path length of a binary tree is the sum

over all external nodes of the lengths of the
paths from the root to those nodes.

Internal path length is the sum over all internal

nodes of the lengths of the paths from the root to
those nodes.

Let the internal path length be I and external

path length E, then the binary tree of (a) has I =
0+1+1+2+3 = 7, E = 2+2+4+4+3+2 = 17.

Binary Search Tree

Containing A Symbol Table

Lets look at the problem of representing a symbol

table as a binary search tree.
If a binary search tree contains the identifiers a1, a2,
, an with a1 < a2 < < an, and the probability of
searching for each ai is pi, then the total cost of any
binary search tree is

when only successful searches are made.

Binary Search Tree Containing A

Symbol Table

For unsuccessful searches, lets partitioned the identifiers

not in the binary search tree into n+1 classes Ei, 0 i n. If
qi is the probability that the identifier being sought is in Ei,
then the cost of the failure node is

Therefore, the total cost of a binary search tree is

Binary Search Tree With Three

Identifiers Example
do

if

while
if

do

if

while

while

do
(c)

(b)

(a)

while

do

do

while
if

(d)

if
(e)

Cost of Binary Search Tree In The

Example
With equal probabilities, pi = qj = 1/7 for all i and j, we
have
cost(tree a) = 12/7; cost(tree b) = 10/7
cost(tree c) = 12/7; cost(tree d) = 12/7
cost(tree e) = 12/7
Tree b is optimal.

With p1=0.5, p2=0.1, p3=0.05, q0=0.15, q1=0.1, q2=0.05,

and q3=0.05 we have
cost(tree a) = 2.65; cost(tree b) = 1.9
cost(tree c) = 1.5; cost(tree d) = 2.05
cost(tree e) = 1.6
Tree c is optimal.

Determine Optimal Binary

Search Tree

So to determine which is the optimal binary search, it is not

practical to follow the above brute force approach since the
complexity is O(n4n/n3/2).
Now lets take another approach. Let Tij denote an optimal
binary search tree for ai+1, , aj, i<j.
Let cij be the cost of the search tree Tij.

Let rij be the root of Tij and let wij be the weight of Tij, where

Therefore, by definition rii=0, wii=qi, 0 i n. T0n is an optimal

binary search tree for a1, , an. Its cost function is e0n, it
weight w0n, and it root is r0n.

Determine Optimal Binary

Search Tree (Cont.)

If Tij is an optimal binary search tree for ai+1, , aj,

and rij =k, then i< k <j. Tij has two subtrees L and R.
L contains ai+1, , ak-1, and R contains ak+1, , aj. So
the cost cij of Tij is
cij = cost(L) + cost(R) + weight(L) + weight(R)
cij = ci,k-1+ ckj + wi,k-1+ wkj
=min i<k<=j { ci,k-1+ ckj }+ wij

OPTIMALBST(p,q,n)

Example:
i

pi

qi

0.05

0.15

0.10

0.05

0.10

0.20

0.10

0.05

0.05

0.05

0.10

e
1

5
3

0.45
0.05

1.75

0.90

1
0

1.20
0.70

0.10

0.05

1.30
0.60

0.25

0.40

2.00

0.90

0.05

0.55

0.45

0.30

0.10

0.05

0.80
0.50

0.35
0.25

0.10

1.00
0.70

0.50

0.30
0.05

2.75

1.25

0.60
0.30

0.15
0.05

0.20
0.05

0.05

3
2

2
2

2
4

2
2

1
2

3
5

4
5

4
4

0.35

root
5

0.50

5
5

The tables e[i,j], w[i,j], and root [i,j]computed by Optimal-BST

0.10

k2

k1
d0

k5
d1

d5

k4
i

d4
k3

pi

0.15

0.10

0.05

0.10

0.20
d2

qi

0.05

0.10

0.05

0.05

0.05

0.10

d3

Example: key
A B C D
probability 0.1 0.2 0.4 0.3
The tables below are filled diagonal by diagonal: the left one is filled
using the recurrence and the right one, for trees roots, records ks
values giving the minima

.1

.4

1.1 1.7

A
optimal BST

.2

.8 1.4

Example

Let n=4, (a1, a2, a3, a4) = (do, if return, while). Let (p1, p2, p3,
p4)=(3,3,1,1) and (q0, q1, q2, q3, q4)=(2,3,1,1,1). wii = qii, cii=0,
and rii=0, 0 i 4.

Example
0

3
4

0
w00=2
c00=0
r00=0
w01=8
c01=8
r01=1
w02=12
c02=19
r02=1
w03=14
c03=25
r03=2
w04=16
c04=32
r04=2

1
w11=3
c11=0
r11=0
w12=7
c12=7
r12=2
w13=9
c13=12
r13=2
w14=11
c14=19
r14=2

2
w22=1
c22=0
r22=0
w23=3
c23=3
r23=3
w24=5
c24=8
r24=3

3
w33=1
c33=0
r33=0
w34=3
c34=3
r34=4

4
w44=1
c44=0
r44=0

Computation Complexity of
Optimal Binary Search Tree

To evaluate the optimal binary tree we need to

compute cij for (j-i)=1, 2, ,n in that order. When ji=m, there are n-m+1 cijs to compute.
The computation of each cijs can be computed in
time O(m).
The total time for all cijs with j-i=m is therefore
O(nm-m2). The total time to evaluate all the cijs
and rijs is

All pairs shortest path

Floyds Algorithm

All pairs shortest path

In

the all pairs shortest path problem, we want to

find the shortest path from every possible source
to every possible destination.

All pairs shortest path

The problem: find the shortest path between every pair

of vertices of a graph

The graph: may contain negative edges but no

negative cycles

A representation: a weight matrix where

W(i,j)=0 if i=j.
W(i,j)= if there is no edge between i and j.
W(i,j)=weight of edge

The weight matrix and the

graph
1
v1

3
5
v5
3

v2
9

v4

2
2
4

v3

The subproblems
How can we define the shortest distance di,j in
terms of smaller problems?
One way is to restrict the paths to only include
vertices from a restricted subset.
Initially, the subset is empty.
Then, it is incrementally increased until it
includes all the vertices.

The sub-problems
Let

D(k)[i,j]=weight of a shortest path from vi

to vj using only vertices from {v1,v2,,vk}
as intermediate vertices in the path

D(0)=W
D(n)=D which is the goal matrix

How
Floyds Algorithm 95

do we compute D(k) from D(k-1) ?

The Recursive Definition:

Case 1: A shortest path from vi to vj restricted to using only
vertices from {v1,v2,,vk} as intermediate vertices does not use
vk.
Then D(k)[i,j]= D(k-1)[i,j].
Case 2: A shortest path from vi to vj restricted to using only
vertices from {v1,v2,,vk} as intermediate vertices does use vk.
Then D(k)[i,j]= D(k-1)[i,k]+ D(k-1)[k,j].
Vk

Vi

Shortest path using intermediate vertices

{V1, . . . Vk }

Vj
Shortest Path using intermediate vertices { V1, . . . Vk -1 }

The recursive definition

Since
D(k)[i,j]= D(k-1)[i,j] or
D(k)[i,j]= D(k-1)[i,k]+ D(k-1)[k,j].
We conclude:
D(k)[i,j]= min{ D(k-1)[i,j], D(k-1)[i,k]+ D(k-1)[k,j] }.

Shortest path using intermediate vertices

{V1, . . . Vk }
Vk

Vi

Vj
Shortest Path using intermediate vertices { V1, . . . Vk -1 }

The pointer array P

Used to enable finding a shortest path

Initially the array contains 0
Each time that a shorter path from i to j is found the k
that provided the minimum is saved (highest index
node on the path from i to j)
To print the intermediate nodes on the shortest path a
recursive procedure that print the shortest paths from
i and k, and from k to j can be used

Floyd's Algorithm
//Computes shortest distance between all pairs of
//nodes, and saves P to enable finding shortest paths
1. D0 W // initialize D array to W [ ]
2. P 0 // initialize P array to [0]
3. for k 1 to n
4.
do for i 1 to n
5.
do for j 1 to n
6.
if (Dk-1[ i, j ] > Dk-1 [ i, k ] + Dk-1 [ k, j ] )
7.
then Dk[ i, j ] Dk-1 [ i, k ] + Dk-1 [ k, j ]
8.
P[ i, j ] k;
9.
else Dk[ i, j ] Dk-1 [ i, j ]

Floyd//Computes shortest distance between all pairs

of nodes

1. D0 W // initialize D array to W [ ]
2. P 0 // initialize P array to [0]
3. for k 1 to n
4.
do for i 1 to n
5.
do for j 1 to n
6.
Dk[i, j ] min(Dk-1[i, k] +Dk-1[k,j ])

Example
W=D =
0

1
4

-3

3
-3
P=

5
2
-3

D1 =

P=

D =
0

-3

-3

k=1
Vertex 1 can be
intermediate
node

D1[2,3] = min( D0[2,3], D0[2,1]+D0[1,3] )

= min (, 7)
=7

D1[3,2] = min( D0[3,2], D0[3,1]+D0[1,2] )

= min (-3,)
= -3

5
2
-3

D2 =

P=

D1 = 1

-3

-1

-3

k=2
Vertices 1, 2 can
be intermediate

D2[1,3] = min( D1[1,3], D1[1,2]+D1[2,3] )

= min (5, 4+7)
=5

D2[3,1] = min( D1[3,1], D1[3,2]+D1[2,1] )

= min (, -3+2)
= -1

2
-3

D =
3

P=

-1

-3

D2 =
1
3

-1

-3

k=3
Vertices 1, 2, 3
can be
intermediate

D3[1,2] = min(D2[1,2], D2[1,3]+D2[3,2] )

= min (4, 5+(-3))
=2

D3[2,1] = min(D2[2,1], D2[2,3]+D2[3,1] )

= min (2, 7+ (-1))
=2

Apply the Floyds algorithm to the following problem

instance:
0 3
(0)
D = 2 0
7 0 1
6 0

Time Complexity

FLOYD(G)
for i,j in [1..n]
d[i,j]=w(ui,uj)
for k in [1..n]
for i in [1..n]
for j in [1..n]
d[i,j]=min(d[i,j],d[i,k]+d[k,j])
Time complexity: O(n3)

Advantages of Dynamic
programming
A

recursive procedure does not have memory

Dynamic programming stores previous
values to avoid multiple calculations

Space and Time

The

space can be reduced to order of

O(min{m,n})
It is enough to keep only two rows j and j-1
After calculating the entries for row j move
that row up to row j-1 and delete row j-1 and
get the new entries for row j.
The time cannot be reduced

Space reduction
To

compare for a smaller area w, where w is

the window size, the matrix size will reduce.
S1: a b c d a c e
S2: b a d c a b e
3
w

3
w

Space reduction
Specifying

the window size reduces the

number of calculation
The runtime of this algorithm is:
O(2w * min{m,n})
It is a linear time algorithm and the space
required: O(w)