5-Dynamic Programming (1)
5-Dynamic Programming (1)
Programming
1
What is Dynamic Programming?
• Dynamic programming is a very powerful, general tool for solving
optimization problems.
• Once understood it is relatively easy to apply, but many people have
trouble understanding it.
• Why not Greedy?
• Greedy algorithms focus on making the best local choice at each decision point.
• For example, a natural way to compute a shortest path from x to y might be to
walk out of x, repeatedly following the cheapest edge until we get to y. WRONG!
• In the absence of a correctness proof, greedy algorithms are very likely to fail.
2
Example 1 – Fibonacci number generation
Problem:
Let’s consider the calculation of Fibonacci numbers:
3
Example 1 – Fibonacci number generation
Recursive Algorithm:
Fib(n)
{
if (n == 0)
return 0;
if (n == 1)
return 1;
Return Fib(n-1)+Fib(n-2)
}
4
Example 1 – Fibonacci number generation
Recursive Algorithm:
if (n == 1)
return 1;
Return Fib(n-1)+Fib(n-2)
}
5
Example 1 – Fibonacci number generation
Recursion tree
What’s the problem?
6
Example 1 – Fibonacci number generation
Memoization:
Fib(n)
{
if (n == 0)
return M[0];
if (n == 1)
return M[1];
if (Fib(n-2) is not calculated)
call Fib(n-2);
if(Fib(n-1) is not calculated)
call Fib(n-1);
//Store the ${n}^{th}$ Fibonacci no. in memory & use previous results.
M[n] = M[n-1] + M[n-2]
Return M[n];
}
7
Example 1 – Fibonacci number generation
already calculated …
8
Dynamic programming - Approach
- Main approach: recursive, holds answers to a sub problem in
a table, can be used without recomputing.
9
Elements of Dynamic
Programming
• Optimal Substructure
• An optimal solution to a problem contains within it an optimal
solution to subproblems
• Optimal solution to the entire problem is build in a bottom-up
manner from optimal solutions to subproblems
• Overlapping Subproblems
• If a recursive algorithm revisits the same subproblems over and
over the problem has overlapping subproblems
10
0-1 Knapsack – DP approach
11
0-1 Knapsack – Algorithm
for w=0 to W do
B[w] = 0
for k=1 to n do
{ k=W
for j=W downto wk do for i = n downto 1 do
{ {
if B[j-wk]+bk > B[j] if keep[i, k] == 1
{
{ B[j]= B[j-wk]+bk
output i “as selected”
keep[k, j] = 1 k = k-wi
} }
else }
keep[k, j] = 0
}
12
}
0-1 Knapsack – Example 1
• Find the solution for the 0-1 knapsack problem with benefit-weight
pairs as (10, 5), (40, 4), (30, 6), (50, 3) and total sack capacity as 10.
13
0-1 Knapsack – Example 1
• Find the solution for the 0-1 knapsack problem with benefit-weight
pairs as (10, 5), (40, 4), (30, 6), (50, 3) and total sack capacity as 10.
B[0] 0
B[1] 0
B[2] 0 0 1 2 3 4 5 6 7 8 9 10
B[3] 0 1
B[4] 0 2
B[5] 0 3
B[6] 0 4
B[7] 0
B[8] 0 Initialize the Keep array
B[9] 0
Initialize the
B[10] 0 Benefit array 14
0-1 Knapsack – Algorithm
for w=0 to W do
B[w] = 0
for k=1 to n do
{ k=W
for j=W downto wk do for i = n downto 1 do
{ {
if B[j-wk]+bk > B[j] if keep[i, k] == 1
{
{ B[j]= B[j-wk]+bk
output i “as selected”
keep[k, j] = 1 k = k-wi
} }
else }
keep[k, j] = 0
}
15
}
0-1 Knapsack – Example 1
• Find the solution for the 0-1 knapsack problem with benefit-weight
pairs as (10, 5), (40, 4), (30, 6), (50, 3) and total sack capacity as 10.
B[0] 0
B[1] 0
B[2] 0 0 1 2 3 4 5 6 7 8 9 10
B[3] 0 1
B[4] 0 2
B[5] 0 3
B[6] 0 4
B[7] 0
B[8] 0 Initialize the Keep array
B[9] 0
Initialize the
B[10] 0 Benefit array 16
0-1 Knapsack – Example 1
• Find the solution for the 0-1 knapsack problem with benefit-weight
pairs as (10, 5), (40, 4), (30, 6), (50, 3) and total sack capacity as 10.
B[0] 0
B[1] 0
B[2] 0 0 1 2 3 4 5 6 7 8 9 10
B[3] 0 1 1 1 1 1 1 1
B[4] 0 2
B[5] 0 10 3
B[6] 0 10 4
B[7] 0 10
B[8] 0 10 Keep array
B[9] 0 10
B[10] 0 10 17
0-1 Knapsack – Example 1
• Find the solution for the 0-1 knapsack problem with benefit-weight
pairs as (10, 5), (40, 4), (30, 6), (50, 3) and total sack capacity as 10.
B[0] 0
B[1] 0
B[2] 0 0 1 2 3 4 5 6 7 8 9 10
B[3] 0 1 1 1 1 1 1 1
B[4] 0 40 2 1 1 1 1 1 1 1
B[5] 0 10 40 3
B[6] 0 10 40 4
B[7] 0 10 40
B[8] 0 10 40 Keep array
B[9] 0 10 50
B[10] 0 10 50 18
0-1 Knapsack – Example 1
• Find the solution for the 0-1 knapsack problem with benefit-weight
pairs as (10, 5), (40, 4), (30, 6), (50, 3) and total sack capacity as 10.
B[0] 0
B[1] 0
B[2] 0 0 1 2 3 4 5 6 7 8 9 10
B[3] 0 1 1 1 1 1 1 1
B[4] 0 40 2 1 1 1 1 1 1 1
B[5] 0 10 40 3 0 0 0 0 1
B[6] 0 10 40 4
B[7] 0 10 40
B[8] 0 10 40 Keep array
B[9] 0 10 50
B[10] 0 10 50 70 19
0-1 Knapsack – Example 1
• Find the solution for the 0-1 knapsack problem with benefit-weight
pairs as (10, 5), (40, 4), (30, 6), (50, 3) and total sack capacity as 10.
B[0] 0
B[1] 0
B[2] 0 0 1 2 3 4 5 6 7 8 9 10
B[3] 0 50 1 1 1 1 1 1 1
B[4] 0 40 50 2 1 1 1 1 1 1 1
B[5] 0 10 40 50 3 0 0 0 0 1
B[6] 0 10 40 50 4 1 1 1 1 1 1 1 1
B[7] 0 10 40 90
B[8] 0 10 40 90 Keep array
B[9] 0 10 50 90
B[10] 0 10 50 70 90 Maximum Profit 20
0-1 Knapsack – Algorithm
for w=0 to W do
B[w] = 0
for k=1 to n do
{ k=W
for j=W downto wk do for i = n downto 1 do
{ {
if B[j-wk]+bk > B[j] if keep[i, k] == 1
{
{ B[j]= B[j-wk]+bk
output i “as selected”
keep[k, j] = 1 k = k-wi
} }
else }
keep[k, j] = 0
}
21
}
0-1 Knapsack – Example 1 I1 - (10, 5),
0 1 2 3 4 5 6 7 8 9 10
I2 - (40, 4),
1 1 1 1 1 1 1
2 1 1 1 1 1 1 1
I3 - (30, 6),
3 0 0 0 0 1 I4 - (50, 3)
4 1 1 1 1 1 1 1 1
k=10
i=4 to 1
When i=4, keep[4,10]==1??
Yes, pick item 4,
When i=3, keep[3,7]==1??
k=10-3 = 7
Final Solution,
No [0, 1, 0, 1] = 90
When i=2, keep[2,7]==1??
Yes, pick item 2, k=7-4 = 3
When i=1, keep[1,3]==1?? No 22
0-1 Knapsack – Algorithm
for w=0 to W do
B[w] = 0
for k=1 to n do
{ k=W
for j=W downto wk do for i = n downto 1 do
{ {
if B[j-wk]+bk > B[j] if keep[i, k] == 1
{
{ B[j]= B[j-wk]+bk
output i “as selected”
keep[k, j] = 1 k = k-wi
} }
else }
keep[k, j] = 0
} Complexity?? O(nW)
23
}
0-1 Knapsack – Example 2
• Consider the given benefit-weight list for four items as {1, 4, 5, 7} and
{1, 3, 4, 5}. Solve the 0-1 knapsack problem assuming W=7.
B[0] 0 0 1 2 3 4 5 6 7
B[1] 0 1
B[2] 0 2
B[3] 0 3
B[4] 0 4
B[5] 0
B[6] 0 Keep array
B[7] 0
Benefit array
24
0-1 Knapsack – Example 2
• Consider the given benefit-weight list for four items as {1, 4, 5, 7} and {1, 3,
4, 5} respectively. Solve the 0-1 knapsack problem assuming W=7.
B[0] 0 0 1 2 3 4 5 6 7
B[1] 0 1 1 1 1 1 1 1 1 1
B[2] 0 1 2
B[3] 0 1 3
B[4] 0 1 4
B[5] 0 1
B[6] 0 1 Keep array
B[7] 0 1
Benefit array
25
0-1 Knapsack – Example 2
• Consider the given benefit-weight list for four items as {1, 4, 5, 7} and {1, 3,
4, 5} respectively. Solve the 0-1 knapsack problem assuming W=7.
B[0] 0 0 1 2 3 4 5 6 7
B[1] 0 1 1 1 1 1 1 1 1 1
B[2] 0 1 2 1 1 1 1 1
B[3] 0 1 4 3
B[4] 0 1 5 4
B[5] 0 1 5
B[6] 0 1 5 Keep array
B[7] 0 1 5
Benefit array
26
0-1 Knapsack – Example 2
• Consider the given benefit-weight list for four items as {1, 4, 5, 7} and {1, 3,
4, 5} respectively. Solve the 0-1 knapsack problem assuming W=7.
B[0] 0 0 1 2 3 4 5 6 7
B[1] 0 1 1 1 1 1 1 1 1 1
B[2] 0 1 2 1 1 1 1 1
B[3] 0 1 4 3 0 1 1 1
B[4] 0 1 5 4
B[5] 0 1 5 6
B[6] 0 1 5 6 Keep array
B[7] 0 1 5 9
Benefit array
27
0-1 Knapsack – Example 2
• Consider the given benefit-weight list for four items as {1, 4, 5, 7} and {1, 3,
4, 5} respectively. Solve the 0-1 knapsack problem assuming W=7.
B[0] 0 0 1 2 3 4 5 6 7
B[1] 0 1 1 1 1 1 1 1 1 1
B[2] 0 1 2 1 1 1 1 1
B[3] 0 1 4 3 0 1 1 1
B[4] 0 1 5 4 1 1 0
B[5] 0 1 5 6 7
B[6] 0 1 5 6 8 Keep array
B[7] 0 1 5 9 Maximum Profit
Benefit array 28
0-1 Knapsack – Example 1 I1 - (1, 1),
0 1 2 3 4 5 6 7 I2 - (4, 3),
1 1 1 1 1 1 1 1
I3 - (5, 4),
2 1 1 1 1 1
3 0 1 1 1
I4 - (7, 5)
4 1 1 0
k=7
i=4 to 1
When i=4, keep[4,7]==1??
No
When i=3, keep[3,7]==1??
Final Solution,
Yes, pick item 3, k=7-4 = 3 [0, 1, 1, 0] = 9
When i=2, keep[2,3]==1??
Yes, pick item 2, k=3-3 = 0
When i=1, keep[1,0]==1?? No 29
Matrix Chain Multiplication
• Finding the most efficient way to multiply a given sequence
of matrices.
• The problem is not actually to perform the multiplications,
but merely to decide the sequence of the matrix
multiplications involved.
• Matrix chain multiplication is an optimization problem.
30
Parenthesizing the matrices
• Consider a chain of ‘n’ matrices. The number of ways to parenthesize
the matrices can be found by
31
Example 1
• How many ways you can parenthesize 3 and 4 matrices and what are
they? Assume the matrices as A1, A2, A3, A4.
• P(2) = P(1).P(1) = 1.1=1
3 matrices:
• P(3) = P(1)P(2) + P(2).P(1) Solution 1: (A1.A2).A3
= 1.1 + 1.1 = 2 ways Solution 2: A1.(A2.A3)
• P(4) = P(1).P(3)+P(2).P(2)+P(3).P(1) 4 matrices:
= 1.2 + 1.1 + 2.1 = 5 ways Solution 1: ((A1.A2).(A3.A4))
Solution 2: (A1.((A2.A3).A4))
Solution 3: ((A1.(A2.A3)).A4)
Solution 4: (((A1.A2).A3).A4)
Solution 5: (A1.(A2.(A3.A4))) 32
Example 2
• Consider the chain of 4 matrices with the size 30, 35, 10, 15, 12. Find
the number of scalar multiplications in all the possible ways and find
the optimal way of multiplying these matrices.
• Solution 1: ((A1.A2).(A3.A4))
A1 – 30 x 35
= 30 x 35 x 10 + 10 x 15 x 12 + 30 x 10 x 12 A2 – 35 x 10
=15900 A3 – 10 x 15
• Solution 2: (A1.((A2.A3).A4)) A4 – 15 x 12
= 35 x 10 x 15 + 35 x 15 x 12 + 30 x 35 x 12
= 24150
33
Example 2
A1 – 30 x 35
• Solution 3: ((A1.(A2.A3)).A4) A2 – 35 x 10
A3 – 10 x 15
= 35 x 10 x 15 + 30 x 35 x 15 + 30 x 15 x 12 A4 – 15 x 12
= 26400
• Solution 4: (((A1.A2).A3).A4)
= 30 x 35 x 10 + 30 x 10 x 15 + 30 x 15 x 12
= 20400
• Solution 5: (A1.(A2.(A3.A4)))
= 10 x 15 x 12 + 35 x 10 x 12 + 30 x 35 x 12
=18600
34
Matrix Chain Multiplication
• Given a chain of matrices namely A1, A2, A3…An,
matrix Ai has the dimensions Pi-1xPi, completely
parenthesize the product in a way such that it
minimizes the number of scalar multiplications.
m[i, j]
35
Computing the Optimal Costs
0 if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i
<j
ik<j
• Length = 1: i = j, i = 1, 2, …, n n d
co t
sen firs
• Length = 2: j = i + 1, i = 1, 2, …,1n-12 3
n
m[1, n] gives the optimal
solution to the problem
j
Compute rows from bottom to top 3
and from left to right 2
1
i 36
Matrix Chain Multiplication
Example: min {m[i, k] + m[k+1, j] + pi-1pkpj}
m[2, 2] + m[3, 5] + p1p2p5 k=2
m - table s - table
40
Algorithm
O(N3)
41
1 2 3 4 5
4 3 3 3
3 1 2
PRINT-OPT-PARENS(s, i, j) 2 1
if i = j S, 1, 6
PRINT-OPT-PARENS(s, s[i, j] + 1, j) A6
A1 ( S, 2, 2 ( )
S, 3, 3 ) S, 4, 4 S, 5, 5
print “)”
A2 A3 A4 A5
((A1(A2A3))((A4A5)A6)) 42
Example 2
• Consider the chain of 4 matrices with the size 30, 35, 10, 15, 12.
A1 – 30 x 35
A2 – 35 x 10
A3 – 10 x 15 P0 = 30
A4 – 15 x 12 P1 = 35
P2 = 10
P3 = 15
P4 = 12
43
Example 1
1 2 3 4 1 2 3 4
4 15900 6000 1800 0 4 2 2 3
3 15000 5250 0 3 2 2
2 10500 0
2 1
1 0
m - table s - table
((A1A2)(A3A4))
44
Longest Common Subsequence
• Given two sequences
X = x1, x2, …, xm
Y = y1, y2, …, yn
find a maximum length common subsequence (LCS) of X and Y
• E.g.:
X = A, B, C, B, D, A, B
• Subsequences of X:
• A subset of elements in the sequence taken in order
A, B, D, B, C, D, B, etc.
45
Example
X = A, B, C, B, D, A, B X = A, B, C, B, D, A, B
Y = B, D, C, A, B, A Y = B, D, C, A, B, A
46
Brute-Force Solution
47
Making the choice
Scenario 1
X = A, B, D, E
Y = Z, B, E
• Choice: include one element into the common sequence (E) and solve the
resulting subproblem
Scenario 2
X = A, B, D, G
Y = Z, B, D
• Choice: exclude an element from a string and solve the resulting subproblem
48
Notations
• Given a sequence X = x1, x2, …, xm we define the i-th prefix of X, for i
= 0, 1, 2, …, m
• c[i, j] = the length of a LCS of the sequences Xi = x1, x2, …, xi and Yj
e.g.: Xi = A, B, D, E
Yj = Z, B, E
c[i, j] =c[i - 1, j - 1] + 1
e.g.: Xi = A, B, D, G
Yj = Z, B, D
max { c[i - 1, j], c[i, j-1] }
c[i, j] =
• we may need to find the LCS between X and Yn-1 and that of Xm-1 and Y
• Both the above subproblems has the subproblem of finding the LCS of Xm-1
and Yn-1
0 xi 0 0 0 0 0 0 • If xi = yj
1 A 0 b[i, j] = “ ”
2 B 0 • Else, if
c[i-1,j]
i c[i - 1, j] ≥ c[i, j-1]
3 C 0 c[i,j-1]
0 b[i, j] = “ ”
0 else
m D
b[i, j] = “ ”
j
54
LCS-LENGTH(X, Y, m, n)
The length of the LCS if one of the sequences
1. for i ← 1 to m
is empty is zero
2. do c[i, 0] ← 0
3. for j ← 0 to n
4. do c[0, j] ← 0
5. for i ← 1 to m
6. do for j ← 1 to n
7. do if xi = yj
Case 1: xi = yj
8. then c[i, j] ← c[i - 1, j - 1] + 1
9. b[i, j ] ← “ ”
10. else if c[i - 1, j] ≥ c[i, j - 1]
11. then c[i, j] ← c[i - 1, j]
12. b[i, j] ← “↑” Case 2: xi yj
13. else c[i, j] ← c[i, j - 1]
14. b[i, j] ← “←”
15. return c and b
Running time: (mn)
55
Example 1 0 if i = 0 or
X = A, B, C, B, D, A, B j=0
Y = B, D, C, A, B, A c[i, j] = c[i-1, j-1] + 1 if xi = yj
0 1 2
max(c[i, 3
j-1], 4
c[i-1,5j]) if6 xi yj
If xi = yj yj B D C A B A
0 xi
b[i, j] = “ ” 0 0 0 0 0 0 0
1 A
Else if c[i - 0 0 0 0 1 1 1
2 B
1, j] ≥ c[i, j-1] 0 1 1 1 1 2 2
b[i, j] = “ ”3 C 0 1 1 2 2 2 2
else 4 B 0 1 1 2 2 3 3
b[i, j] = “ 5” D 0
1 2
2
2
3
3
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
56
Constructing a LCS
• Start at b[m, n] and follow the arrows
• When we encounter a “ “ in b[i, j] xi = yj is an element of
the LCS 0 1 2 3 4 5 6
yj B D C A B A
0 xi 0 0 0 0 0 0 0
1 A
0 0 0 0 1 1 1
2 B
0 1 1 1 1 2 2
3 C
0 1 1 2 2 2 2
4 B 0 1
1
2
2 3 3 Solution:
5 D 0 1 2 2 2 3 3 BCBA
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
57
PRINT-LCS(b, X, i, j)
1. if i = 0 or j = 0 Running time: (m + n)
2. then return
3. if b[i, j] = “ ”
4. then PRINT-LCS(b, X, i - 1, j - 1)
5. print xi
6. elseif b[i, j] = “↑”
7. then PRINT-LCS(b, X, i - 1, j)
8. else PRINT-LCS(b, X, i, j - 1)
X/Y 0 1 2 3 4 5 6
Yi B A C B C A
0 Xi 0 0 0 0 0 0 0
1 A 0 0 1 1 1 1 1
2 B 0 1 1 1 2 2 2 Solution:
3 C 0 1 1 2 2 3 3
ABCA
4 A 0 1 2 2 2 3 4
5 B 0 1 2 2 3 3 4
6 A 0 1 2 2 3 3 4
59