Chapter 4

Dynamic Programming
and traversal techniques

Always remember answers to the sub-problems you've already solved. 1

Outline
 The general Method of dynamic programming
 Fibonacci Series
 0/1 Knapsack Problem
 Finding the actual Knapsack

2
Divide-and-conquer
1. split problem into smaller problems
2. solve each smaller problem recursively
3. recombine the results

3
 Dynamic Programming

In Computer Science, Mathematics, Management,

Economics and Bioinformatics, dynamic
programming (also known as dynamic optimization)
is a method for solving a complex problem by breaking
it down into a collection of simpler sub-problems,
solving each of those sub-problems just once, and
storing their solutions.

The technique of storing solutions to sub-problems

instead of re-computing them is called "memorization".

4
 Dynamic Programming

Dynamic Programming is a general algorithm

design technique for solving problems defined by or
formulated as recurrences with overlapping sub-
instances
• Main idea:
- set up a recurrence relating a solution to a
larger instance to solutions of some smaller
instances
- solve smaller instances once
- record solutions in a table
- extract solution to the initial instance from that
table

5
Dynamic programming
 It is like a divide-and-conquer …
but store the solutions to sub-problems
for possible reuse.
 Invented by American mathematician
Richard Bellman in the 1950s to solve
optimization problems and later
assimilated by CS
 “Programming” here means “planning”

6
Dynamic Programming
Dynamic Programming is a powerful
technique that allows one to solve
different types of problems in time O(n2)
or O(n3) for which a naive approach
would take exponential time.

7
Example Fibonacci series
 0, 1, 1, 2, 3, 5, 8, 13, 21, …
 f(0) = 0. f(7)
 f(1) = 1.
 f(N) = f(N-1) + f(N-2) f(6) f(5)
if N 2.
f(5) f(4) f(4) f(3)
int f(int n) {
if n < 2
f(4) f(3)f(3)f(2)f(3)f(2) f(2)f(1)
return n
… … … … … … … 1
else
return f(n-1) + f(n-2)
}

8
Reuse earlier results! f(7)
(“memorization” or “tabling”) f(6)
 0, 1, 1, 2, 3, 5, 8, 13, 21, …
f(5)
 f(0) = 0.
 f(1) = 1. f(4)
 f(N) = f(N-1) + f(N-2) …
if N 2.

int f(int n) {
if n < 2
return n
else
return fmemo(n-1) + fmemo(n-2)
}

9
0-1 Knapsack problem

10
Knapsack problem
Given some items, pack the knapsack to get the
maximum total value. Each item has some weight and
some value. Total weight that we can carry is no more
than some fixed number W.
So we must consider weights of items as well as
their values.

Item # Weight Value

1 1 8
2 3 6
3 5 5
11
Knapsack problem

There are two versions of the problem:

1. “0-1 knapsack problem”
 Items are indivisible; you either take an item
or not. each item must be entirely accepted or
rejected

2. “Fractional knapsack problem”

 Items are divisible: you can take any fraction
of an item

12
0-1 Knapsack problem
 Given a knapsack with maximum capacity W, and
a set S consisting of n items
 Each item i has some weight wi and benefit value
bi (all wi and W are integer values)
 Problem: How to pack the knapsack to achieve
maximum total value of packed items?

max  bi subject to w i W
iT iT

13
0-1 Knapsack problem:
brute-force approach
Let’s first solve this problem with a
straightforward algorithm
 Since there are n items, there are 2n possible
combinations of items.
 We go through all combinations and find the one
with maximum value and with total weight less or
equal to W
 Running time will be O(2n)

14
0-1 Knapsack problem:
dynamic programming approach
 We can do better with
an algorithm based on
dynamic programming
 We need to carefully
identify the sub-
problems

15
Defining a Sub-problem
 Given a knapsack with maximum capacity W, and n
items
 Each item i has some weight wi and benefit value bi (all
wi and W are integer values)
 Problem: How to pack the knapsack to achieve maximum
total value of packed items?
 Find
» V[i,w]???

16
 Recursive Formula
for sub-problems

Recursive formula for subproblems:

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi} else
It means that, the best subset of Sk that has total weight w is:
1) the best subset of Si-1 that has total weight  w, or
2) the best subset of Si-1 that has total weight  w-wi plus the item i

17
Recursive Formula
 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi} else
 The best subset of Si that has the total weight  w,
either contains item i or not.
 First case: wi>w. Item i can’t be part of the solution,
since if it was, the total weight would be > w, which
is unacceptable.
 Second case: wi  w. Then the item i can be in the
solution, and we choose the case with greater value.

18
0-1 Knapsack Algorithm
for w = 0 to W
V[0,w] = 0
for i = 1 to n
V[i,0] = 0
for i = 1 to n
for w = 0 to W
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
19
Running time
for w = 0 to W
O(W)
V[0,w] = 0
for i = 1 to n
V[i,0] = 0
for i = 1 to n Repeat n times
for w = 0 to W
O(W)
< the rest of the code >

What is the running time of this

algorithm?
O(W) + O(n*W) ~ O(n)
Remember that the brute-force algorithm
takes O(2n) 20
Example

Let’s run our algorithm on the

following data:

n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)

21
Example (2)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1
2
3
4

for w = 0 to W
V[0,w] = 0

22
Example (3)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0

for i = 1 to n
V[i,0] = 0

23
 Items:
1: (2,3)
Example (4) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0
wi=2
2 0
3 0 w=1
4 0 w-wi =-1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
24
 Items:
1: (2,3)
Example (5) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3
wi=2
2 0
3 0 w=2
4 0 w-wi =0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
25
 Items:
1: (2,3)
Example (6) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3
wi=2
2 0
3 0 w=3
4 0 w-wi =1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
26
 Items:
1: (2,3)
Example (7) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3
wi=2
2 0
3 0 w=4
4 0 w-wi =2
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
27
 Items:
1: (2,3)
Example (8) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3 3
wi=2
2 0
3 0 w=5
4 0 w-wi =3
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
28
 Items:
1: (2,3)
Example (9) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0
3 0 w=1
4 0 w-wi =-2
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
29
 Items:
1: (2,3)
Example (10) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3
3 0 w=2
4 0 w-wi =-1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
30
 Items:
1: (2,3)
Example (11) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4
3 0 w=3
4 0 w-wi =0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
31
 Items:
1: (2,3)
Example (12) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4
3 0 w=4
4 0 w-wi =1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
32
 Items:
1: (2,3)
Example (13) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4 7
3 0 w=5
4 0 w-wi =2
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
33
 Items:
1: (2,3)
Example (14) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 w= 1..3
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
34
 Items:
1: (2,3)
Example (15) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 w= 4
4 0 w- wi=0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
35
 Items:
1: (2,3)
Example (16) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 w- wi=1
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
36
 Items:
1: (2,3)
Example (17) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 1..4
4 0 0 3 4 5
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
37
 Items:
1: (2,3)
Example (18) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 0 3 4 5 7 w- wi=0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
38
How to find actual Knapsack Items

39
How to find actual Knapsack
Items
 All of the information we need is in the table.
 V[n,W] is the maximal value of items that can be
placed in the Knapsack.
 Let i=n and k=W
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 // Assume the ith item is not in the knapsack

40
 Items:
1: (2,3)
Finding the Items 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 41
 Items:
1: (2,3)
Finding the Items (2) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 42
 Items:
1: (2,3)
Finding the Items (3) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=5
2 0 0 3 4 4 7 wi=4
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 43
 Items:
1: (2,3)
Finding the Items (4) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=4
2 0 0 3 4 4 7 wi=3
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =3
i=n, k=W
k  wi=2
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 44
 Items:
1: (2,3)
Finding the Items (5) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 k= 2
1 0 0 3 3 3 3 bi=3
2 0 0 3 4 4 7 wi=2
3 0 0 3 4 5 7 V[i,k] = 3
4 0 0 3 4 5 7 V[i1,k] =0
i=n, k=W
k  wi=0
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 45
 Items:
1: (2,3)
Finding the Items (6) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=0 4: (5,6)
0 0 0 0 0 0 0 k= 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
knapsack
4 0 0 3 4 5 7
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k]  V[i1,k] then
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1 46
 Items:
1: (2,3)
Finding the Items (7) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 4: (5,6)
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
knapsack
4 0 0 3 4 5 7
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k]  V[i1,k] then
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1 47
Conclusion
 Dynamic programming is a useful technique of
solving certain kind of problems
 When the solution can be recursively described in
terms of partial solutions, we can store these
partial solutions and re-use them as necessary
(memorization)
 Running time of dynamic programming algorithm
vs. naïve algorithm:
» 0-1 Knapsack problem: O(W*n) vs. O(2n)

48
Group Assignment (10%)
 Optimal binary search trees
 Reliability design
 Multistage graphs
 All pairs shortest pattern

49