Dynamic Programming

7 -1
 Dynamic Programming
 Dynamic Programming is an
algorithm design method that can
be used when the solution to a
problem may be viewed as the
result of a sequence of decisions

7 -2
 Dynamic Programming (DP)
 Dynamic programming (DP) applies when a problem has both of
these properties:
1. Optimal substructure: “optimal solutions to a problem

incorporate optimal solutions to related subproblems, which we

may solve independently”.
2. Overlapping subproblems: “a recursive algorithm revisits the

same problem repeatedly”.

 Dynamic programming is typically used to:

 Solve optimization problems that have the above properties.
 Solve counting problems –e.g. Matrix Traversal.
 Speed up existing recursive implementations of problems that
have overlapping subproblems (property 2) – e.g. Fibonacci.


3
Iterative or Bottom-Up
Dynamic Programming
• Main type of solution for DP problems
• We can define the problems size and solve problems
from size 0 going up to the size we need.
• Iterative – because it uses a loop
• Bottom-up because you solve problems from the
bottom (the smallest problem size) up to the original
problem size.

4
Bottom-Up vs. Top Down
• There are two versions of dynamic programming.
• Bottom-up.
• Top-down (or memoization).

• Bottom-up:
• Iterative, solves problems in sequence, from smaller to bigger.
• Top-down:
• Recursive, start from the larger problem, solve smaller problems as
needed.
• For any problem that we solve, store the solution, so we never have
to compute the same solution twice.
• This approach is also called memoization.

5
Top-Down Dynamic Programming
( Memoization )

• Maintain an array/table where solutions to problems

can be saved.
• To solve a problem P:
• See if the solution has already been stored in the
array.
• If yes, return the solution.
• Else:
• Issue recursive calls to solve whatever smaller
problems we need to solve.
• Using those solutions obtain the solution to
problem P.
• Store the solution in the solutions array.
• Return the solution.

6
Fibonacci Numbers

7
 Fibonacci Numbers
• Generate Fibonacci numbers
– 3 solutions: inefficient recursive, memoization (top-down
dynamic programming (DP)), bottom-up DP.
– Not an optimization problem but it has overlapping
subproblems => DP eliminates recomputing the same
problem over and over again.

8
 Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2:
Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• How can we write a function that computes Fibonacci
numbers?

9
 Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Consider this function: what is its running time?

int Fib(int i)
{
if (i < 1) return 0;
if (i == 1) return 1;
return Fib(i-1) + Fib(i-2);
} 10
 Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Consider this function: what is its running time?
– g(N) = g(N-1) + g(N-2) + constant
 g(N) ≥ Fibonacci(N) => g(N) = Ω(Fibonacci(N))
 Also g(N) ≤ 2g(N-1)+constant => g(N) ≤ c2N => g(N) = O(2N)
=> g(N) is exponential
– We cannot compute Fibonacci(40) in a reasonable amount of time
(with this implementation). int Fib(int i)
{
– See how many times this function is executed. if (i < 1) return 0;
if (i == 1) return 1;
return Fib(i-1) + Fib(i-2);
– Draw the tree
}
11
 Fibonacci sequence

• Fibonacci sequence: 0 , 1 , 1 , 2 , 3 , 5 , 8 , 13 , 21 , …
Fi = i if i  1
Fi = Fi-1 + Fi-2 if i  2
• Solved by a recursive program:
f5

f4 f3

f3 f2 f2 f1

f2 f1 f1 f0 f1 f0

f1 f0

• Much replicated computation is done.

• It should be solved by a simple loop. 7 -12
 Fibonacci Numbers
• Fibonacci(0) = 0
• Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Alternative to inefficient recursion: compute from small to
large and store data in an array.

Notice the mapping/correspondence of the mathematical expression and code.

linear version (Iterative, bottom-up ):
int Fib_iter (int i) {
exponential version:
int F[i+1];
F[0] = 0; F[1] = 1; int Fib(int i) {
int k; if (i < 1) return 0;
for (k = 2; k <= i; k++) F[k] = F[k-1] + F[k-2]; if (i == 1) return 1;
return F[i]; return Fib(i-1) + Fib(i-2);
} }
13
 Applied scenario
• F(N) = F(N-1)+F(N-2), F(0) = 0, F(1) = 1,
• Consider a webserver where clients can ask what the value of a
certain Fibonacci number, F(N) is, and the server answers it.
How would you do that? (the back end, not the front end)
(Assume a uniform distribution of F(N) requests over time most F(N) will be asked.)
• Constraints:
– Each loop iteration or function call costs you 1cent.
– Each loop iteration or function call costs the client 0.001seconds wait time
– Memory is cheap
• How would you charge for the service? (flat fee/function calls/loop
iterations?)
• Think of some scenarios of requests that you could get. Think of it
with focus on:
– “good sequence of requests”
– “bad sequence of requests”
14
– Is it clear what good and bad refer to here?
•
Fibonacci Numbers
Fibonacci(0) = 0 , Fibonacci(1) = 1
• If N >= 2: Fibonacci(N) = Fibonacci(N-1) + Fibonacci(N-2)
• Alternative: remember values we have already computed.

memoized version:
int Fib_mem_wrap(int i) {
int sol[i+1];
if (i<=1) return i;
sol[0] = 0; sol[1] = 1;
for(int k=2; k<=i; k++) sol[k]=-1;
Fib_mem(i,sol);
return sol[i];
}
exponential version:
int Fib_mem (int i, int[] sol) {
if (sol[i]!=-1) return sol[i]; int Fib(int i) {
int res = Fib_mem(i-1, sol) + Fib_mem(i-2, sol); if (i < 1) return 0;
sol[i] = res; if (i == 1) return 1;
return res; return Fib(i-1) + Fib(i-2);
} } 15
 The shortest path
 To find a shortest path in a multi-stage
graph 3 2 7

1 4
S A B 5
T

5 6

 Apply the greedy method :

the shortest path from S to T :
1+2+5=8
7 -16
 The shortest path in
multistage graphs
 e.g. A
4
D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F
 The greedy method can not be applied to
this case: (S, A, D, T) 1+4+18 = 23.
 The real shortest path is:
(S, C, F, T) 5+2+2 = 9.

7 -17
 Dynamic programming
approach
 Dynamic programming approach (forward
approach):

A
4
D 1 A
1 18 d(A, T)
11 9

2 d(B, T)
S
2
B
5
E
13
T S B T
16 2

5 d(C, T)
5
C 2
F C

 d(S,=
d(A,T) T)min{4+d(D,T),
= min{1+d(A, T), 2+d(B, T),4 5+d(C, T)}
A D
11+d(E,T)} d(D, T)

= min{4+18, 11+13} = 22. T

11
E d(E,
7 T)
-18
 d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F
 d(C, T) = min{ 2+d(F, T) } = 2+2 = 4
 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
 The above way of reasoning is called
backward reasoning.

7 -19
 Backward approach
(forward reasoning)
4
A D
1 18
11 9

2 5 13
S B E T
16 2
 d(S, A) = 1
5
d(S, B) = 2 C 2
F
d(S, C) = 5
 d(S,D)=min{d(S,A)+d(A,D), d(S,B)+d(B,D)}
= min{ 1+4, 2+9 } = 5
d(S,E)=min{d(S,A)+d(A,E), d(S,B)+d(B,E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S,B)+d(B,F), d(S,C)+d(C,F)}
= min{ 2+16, 5+2 } = 7
7 -20
 d(S,T) = min{d(S, D)+d(D, T), d(S,E)+
d(E,T), d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
=9 4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F

7 -21
 Principle of optimality
 Principle of optimality: Suppose that in solving a
problem, we have to make a sequence of
decisions D1, D2, …, Dn. If this sequence is
optimal, then the last k decisions, 1  k  n must
be optimal.
 e.g. the shortest path problem
If i, i1, i2, …, j is a shortest path from i to j, then
i1, i2, …, j must be a shortest path from i1 to j
 In summary, if a problem can be described by a
multistage graph, then it can be solved by
dynamic programming.

7 -22
Example 2

7 -23
Example 3
 Dynamic programming
 Forward approach and backward
approach:
 Note that if the recurrence relations are
formulated using the forward approach then
the relations are solved backwards . i.e.,
beginning with the last decision
 On the other hand if the relations are
formulated using the backward approach, they
are solved forwards.
 To solve a problem by using dynamic
programming:
 Find out the recurrence relations.
 Represent the problem by a multistage graph.

7 -27
Travelling Salesperson Problem
 The traveling salesperson problem

 Figure Directed graph and edge length matrix c

 Example 2
 For the given TSP find a tour of minimum cost

A B C D

A 0 16 11 6

B 8 0 13 16

C 4 7 0 9

D 5 12 2 0

7 -33