Vivekananda

College of Engineering & Technology

Nehru Nagar Post,
P Puttur, D.K. 574203

Lecture Notes on
15CS43
Design and Analysis of
Algorithms
(CBCS Scheme)

Prepared by

Harivinod N
Dept. of Computer Science and Engineering,
VCET Puttur

April 2017

Module : Dynamic Programming

Module-4

Contents

1. Introduction to Dynamic Programming 4. Optimal Binary Search Trees

1.1. General method with Examples 5. Knapsack problem
1.2. Multistage Graphs 6. Bellman-Ford
Ford Algorithm
2. Transitive Closure: 7. Travelling Sales Person problem
2.1. Warshall’s Algorithm, 8. Reliability design
3. All Pairs Shortest Paths:
3.1. Floyd's Algorithm,

Course Website
www.TechJourney.in
 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

1. Introduction to Dynamic Programming

Dynamic programming is a technique for solving problems with overlapping subproblems.
subproblems
Typically, these subproblems arise from a recurrence relating a given problem’s solution to
solutions of its smaller subproblems. Rather than solving overlapping subproblems again and
again, dynamic programming suggests solving each of the smaller subproblems only once
and recording the results in a table from which a solution to the original
al problem can then
t be
obtained. [From T1]
The Dynamic programming can also be used when the solution to a problem can be viewed
as the result of sequence of decisions.
decisions [ From T2]. Here are some examples.
Example 1

Example 2

Example 3

Example 4

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 2

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 3

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

1.2 Multistage Graphs

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 4

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Figure: Five stage graph

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 5

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 6

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Backward Approach

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 7

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

2. Transitive Closure using Warshall’s Algorithm,

Definition: The transitive closure of a directed graph with n vertices can be defined as the n
× n boolean matrix T = {tij }, in which the element in the ith row and the jth column is 1 if
there exists a nontrivial path (i.e., directed path of a positive length) from the ith vertex to the
jth vertex; otherwise, tij is 0.
Example: An example of a digraph, its adjacency matrix, and its transitive closure is given
below.

(a) Digraph. (b) Its adjacency matrix. (c) Its transitive closure.

We can generate the transitive closure of a digraph with the help of depthfirst search or
first search. Performing either traversal starting at the ith vertex gives the information
breadth-first
about the vertices reachable from it and hence the columns that contain 1’s in the ith row of
the transitive closure. Thus, doing such a traversal for every vertex as a starting point yields
the transitive closure in its entirety.
Since this method traverses the same digraph several times, we can use a better algorithm
called Warshall’s algorithm.. Warshall’s algorithm constructs the transitive closure through
a series of n × n boolean matrices:

Each of these matrices provides certain information about directed paths in the digraph.
Specifically, the element in the ith row and jth column of matrix R(k) (i, j = 1, 2, . . . , n, k
= 0, 1, . . . , n) is equal to 1 if and only if there exists a directed path of a positive length from
the ith vertex to the jth vertex with each intermediate vertex, if any, numbered not higher than
k.
Thus, the series starts with R(0) , which does not allow any intermediate vertices in its paths;
hence, R(0) is nothing other than the adjacency matrix of the digraph.
digraph R(1) contains the
information about paths thatat can use the first vertex as intermediate. The last matrix in the
(n)
series, R , reflects paths that can use all n vertices of the digraph as intermediate and hence
is nothing other than the digraph’s transitive closure.
This means that there exists a path from the ith vertex vi to the jth vertex vj with each
intermediate vertex numbered not higher than k:
vi, a list of intermediate vertices each numbered not higher than k, vj . --- (*)
Two situations regarding this path are possible.

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 8

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

1. In the first, the list of its intermediate vertices does not contain the kth vertex. Then this
−1. i.e. r
path from vi to vj has intermediate vertices numbered not higher than k− 1
2. The secondd possibility is that path (*)(* does contain the kth vertex vk among the
intermediate vertices. Then path (*) can be rewritten as;
vi, vertices numbered ≤ k − 1, vk, vertices numbered ≤ k − 1, vj .

i.e r 1 and r 1

Thus, we have the following formula for generating the elements of matrix R(k) from the
elements of matrix R(k−1)

The Warshall’ss algorithm works based on the above formula.

As an example, the application of Warshall’s algorithm to the digraph is shown below. New
1’s are in bold.

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 9

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Analysis
Its time efficiency is Θ(n3). We can make the algorithm to run faster by treating matrix rows
as bit strings and employ the bitwise or operation available in most modern computer
languages.
Space efficiency: Although separate matrices for recording intermediate results of the
algorithm are used, that can be avoided.

3. All Pairs Shortest Paths using Floyd's Algorithm,

Problem definition: Given a weighted connected graph (undirected or directed), the all-pairs
all
shortest paths problem asks to find the distances—i.e.,
distances the lengths of the shortest paths - from
each vertex to all other vertices.
Applications: Solution to this problem finds applications in communications, transportation
networks,
tworks, and operations research. Among recent applications of the all--pairs shortest-path
problem is pre-computing
computing distances for motion planning in computer games.
We store the lengths of shortest paths in an n x n matrix D called the distance matrix: the
element dij in the ith row and the jth column of this matrix indicates the length of the shortest
path from the ith vertex to the jth vertex.

(a) Digraph. (b) Its weight matrix. (c) Its distance matrix
We can generate the distance matrix with an algorithm that is very similar to Warshall’s
algorithm. It is called Floyd’s algorithm.
Floyd’s algorithm computes the distance matrix of a weighted graph with n vertices through a
series of n × n matrices:

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 10

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

The element in the ith row and the jth column of matrix D(k) (i, j = 1, 2, . . . , n, k = 0, 1,
he length of the shortest path among all paths from the i vertex to the jth
. . . , n) is equal to the th

vertex with each intermediate vertex, if any, numbered not higher than k.
As in Warshall’s algorithm, we can compute all the elements of each matrix D(k) from its
immediate predecessor D(k−1)

If 1,, then it means that there is a path;

vi, a list of intermediate vertices each numbered not higher than k, vj .

We can partition all such paths into two disjoint subsets: those that do not use the kth vertex vk
as intermediate and those that do.
i. Since the paths of the first subset have their intermediate vertices numbered not higher
than k − 1, the shortest of them is, by definition of our matrices, of length
ii. In the second subset the paths are of the form
vi, vertices numbered ≤ k − 1, vk, vertices numbered ≤ k − 1, vj .

The situation is depicted symbolically in Figure,

Figure which shows
the underlying
nderlying idea of Floyd’s algorithm.

Taking into account the lengths of the shortest paths in both subsets leads to the following
recurrence:

Analysis: Its time efficiency is Θ(n3), similar to the warshall’s algorithm.

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 11

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Application of Floyd’s algorithm to the digraph is shown below.. Updated elements are shown
in bold.

4. Optimal Binary Search Trees

A binary search tree is one of the most important data structures in computer science. One of
its principal applications is to implement a dictionary, a set of elements with the operations of
searching, insertion, and deletion.
If probabilities of searching for elements of a set are known e.g., from accumulated data
about past searches it is natural to pose a question about an optimal binary search tree for
which the average number of comparisons in a search is the smallest possible.
As an example, consider four keys A, B, C, and D
to be searched for with probabilities 0.1, 0.2, 0.4,
and 0.3, respectively. The figure depicts two out of
14 possible binary search trees containing these
keys.

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 12

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

The average number of comparisons in a successful search in the first of these trees is 0.1 *
1+ 0.2 * 2 + 0.4 * 3+ 0.3 * 4 = 2.9, and
and for the second one it is 0.1 * 2 + 0.2 * 1+ 0.4 * 2 +
0.3 * 3= 2.1. Neither of these two trees is, in fact, optimal.
For our tiny example, we could find the optimal tree byb generating all 14 binary search trees
with these keys. As a general algorithm, this exhaustive-search
exhaustive approach is unrealistic: the
total number of binary search trees with n keys is equal to the nth Catalan number,

which grows to infinity as fast as 4n / n1.5

So let a1, . . . , an be distinct keys ordered from the smallest to the largest and let p1, . . . , pn be
the probabilities of searching for them. Let C(i, j) be the smallest average number of
comparisons made in a successful search in a binary searchse tree Tij made up of keys ai, . . , aj,
where i, j are some integer indices, 1≤ 1 i ≤ j ≤ n.
Following the classic dynamic programming approach, we will find values of C(i, j) for all
smaller instances of the problem, although we are interested just in C(1, n). To derive a
recurrence underlying a dynamic programming algorithm, we will consider all possible ways
to choose a root ak among the keys ai, . . . , aj . For such a binary search tree (Figure 8.8), the
root contains key ak, the left subtree Tik−1 contains keys ai, . . . , ak−1 optimally arranged, and
the right subtree Tjk+1contains keys ak+1, . . . , aj also optimally arranged. (Note how we are
taking advantage of the principle of optimality here.)

If we count tree levels starting with 1 to make the comparison numbers equal the keys’ levels,
the following recurrence relation is obtained:

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 13

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

The two-dimensional
dimensional table in Figure 8.9 shows the values needed for computing C(i, j) by
formula (8.8): they are in row i and the columns to the left of column j and in column j and
the rows below row i. The arrows point to the pairs of entries whose sums are computed in
order to find the smallest one to be recorded as the value of C(i, j). This suggests filling the
table along its diagonals, starting with all zeros on the main diagonal and given probabilities
pi, 1≤ i ≤ n, right above it and moving toward the upper right corner.
The algorithm we just sketched computes C(1, n)—the n) the average number of comparisons for
successful searches in the optimal binary tree.
tree. If we also want to get the optimal tree itself,
we need to maintain another two-dimensional
two dimensional table to record the value of k for which the
minimum in (8.8) is achieved. The table has the same shape as the table in Figure 8.9 and is
filled in the same manner,
nner, starting with entries R(i, i) = i for 1≤
1 i ≤ n. When the table is filled,
its entries indicate indices of the roots of the optimal subtrees, which makes it possible to
reconstruct an optimal tree for the entire set given.

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 14

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Example: Let us illustrate the algorithm by applying it to the four-key

four key set we used at the
beginning of this section:

The initial tables look like this:

Let us compute C(1, 2):

Thus, out of two possible binary trees containing the first two keys, A and B, the root of the
optimal tree has index 2 (i.e., it contains B), and the average number of comparisons in a
successful search in this tree is 0.4. On finishing the computations we get the following final
tables:

Thus, the average number of key comparisons

comparisons in the optimal tree is equal to 1.7. Since R(1,
4) = 3, the root of the optimal tree contains the third key, i.e., C. Its left subtree is made up of
keys A and B, and its right subtree contains just key D. To find the specific structure of these
subtrees,, we find first their roots by consulting the root table again as follows. Since R(1, 2) =
2, the root of the optimal tree containing A and B is B, with A being its left child (and the
root of the one-node tree: R(1, 1) = 1). Since R(4, 4) = 4, the root of this one-node
one optimal
tree is its only key D. Figure given below presents the optimal tree in its entirety.

Here is Pseudocode of the dynamic programming algorithm.

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 15
 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

5. Knapsack problem
We start this section with designing a dynamic programming algorithm for the knapsack
problem: given n items of known weights w1, . . . , wn and values v1, . . . , vn and a knapsack
of capacity W, find the most valuable subset of the items that fit into the knapsack.
To design a dynamic programming algorithm, we need to derive a recurrence relation that
expresses a solution to an instance of the knapsack problem in terms of solutions to its
smaller subinstances.
Let us consider an instance defined by the first i items, 1≤1 i ≤ n, with weights w1, . . . , wi,
values v1, . . . , vi , and knapsack capacity j, 1 ≤ j ≤ W. Let F(i, j) be the value of an optimal
solution to this instance. We can divide all the subsets of the first i items that fit the knapsack
th ith item and those that do. Note
of capacity j into two categories: those that do not include the
the following:
i. Among the subsets that do not include the ith item, the value of an optimal subset is,
by definition, F(i − 1, j).
ii. Among the subsets that do include the ith item (hence, j − wi ≥ 0), an optimal subset is
made
de up of this item and an optimal subset of the first i−1
i−1 items that fits into the
knapsack of capacity j − wi . The value of such an optimal subset is vi + F(i − 1, j −
wi).

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 16

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Thus, the value of an optimal solution among all feasible subsets of the first I items is the
maximum of these two values.

It is convenient to define the initial conditions as follows:

F(0, j) = 0 for j ≥ 0 and F(i, 0) = 0 for i ≥ 0.
Our goal is to find F(n, W), the maximal value of a subset of the n given items that fit into
the knapsack
psack of capacity W, and an optimal subset itself.

Example-1: Let us consider the instance given by the following data:

The dynamic programming table, filled by applying formulas is given below

Thus, the maximal value is F(4, 5) = $37.

We can find the composition of an optimal subset by backtracing the computations of this
entry in the table. Since F(4, 5) > F(3, 5), item 4 has to be included in an optimal solution
along with an optimal subset for filling 5 − 2 = 3 remaining units of the knapsack capacity.
ca
The value of the latter is F(3, 3). Since F(3, 3) = F(2, 3), item 3 need not be in an optimal
subset. Since F(2, 3) > F(1, 3), item 2 is a part of an optimal selection, which leaves element
F(1, 3 − 1) to specify its remaining composition. Similarly,
Similarly, since F(1, 2) > F(0, 2), item 1 is
the final part of the optimal solution {item 1, item 2, item 4}.

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 17

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Analysis
The time efficiency and space efficiency of this algorithm are both in Θ(nW). The time
needed to find the composition of an optimal solution is
i in O(n).

Memory Functions
The direct top-down
down approach to finding a solution to such a recurrence leads to an algorithm
that solves common subproblems more than once and hence is very inefficient.
inefficient
The classic dynamic programming approach, on the other hand, works bottom up: it fills a
table with solutions to all smaller subproblems, but each of them is solved only once. An
unsatisfying aspect of this approach is that solutions to some of these smaller subproblems
are often not necessary for getting a solution
so to the problem given. Since this drawback is not
present in the top-down
down approach, it is natural to try to combine the strengths of the top-down
top
and bottom-up approaches. The goal is to get a method that solves only subproblems that are
necessary andnd does so only once. Such a method exists; it is based on using memory
functions.
This method solves a given problem in the top-down top down manner but, in addition, maintains a
table of the kind that would have been used by a bottom-up
bottom up dynamic programming algorithm.
algori
Initially, all the table’s entries are initialized with a special “null” symbol to indicate that they
have not yet been calculated. Thereafter, whenever a new value needs to be calculated, the
method checks the corresponding entry in the table first: if this entry is not “null,” it is simply
retrieved from the table; otherwise, it is computed by the recursive call whose result is then
recorded in the table.
The following algorithm implements this idea for the knapsack problem. After initializing the
table,
ble, the recursive function needs to be called with i = n (the number of items) and j = W
(the knapsack capacity).
Algorithm MFKnapsack(i, j )
//Implements the memory function method for the knapsack problem
//Input: A nonnegative integer i indicating the number of the first items being
considered and a nonnegative integer j indicating the knapsack capacity
//Output: The value of an optimal feasible subset of the first i items
//Note: Uses as global variables input arrays
arrays Weights[1..n], V alues[1..n], and
table F[0..n, 0..W ] whose entries are initialized with −1’s except for
row 0 and column 0 initialized with 0’s

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 18

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Example-2 Let us apply the memory function method to the instance considered in Example
1. The table in Figure given below gives the results. Only 11 out of 20 nontrivial values (i.e.,
not those in row 0 or in column 0) have been computed. Just one nontrivial entry, V (1, 2), is
retrieved rather than being recomputed. For larger instances,
stances, the proportion of such entries
can be significantly larger.

Figure: Example of solving an instance of the knapsack problem by the memory function algorithm

In general, we cannot expect more than a constant-factor

constant factor gain in using the memory function
method for the knapsack problem, because its time efficiency class is the same as that of the
bottom-up algorithm

6. Bellman-Ford
Ford Algorithm (Single source shortest path with –ve
ve weights)
Problem definition
Single source shortest path - Given a graph
gra and a source vertex s in graph, find shortest paths
from s to all vertices in the given graph. The graph may contain negative weight edges.
Note that wee have discussed Dijkstra’s algorithm for single source shortest path problem.
Dijksra’s algorithm is a Greedy algorithm and time
ti complexity is O(VlogV)
ogV). But Dijkstra
doesn’t work for graphs
raphs with negative weight edges.
edges
Bellman-Ford
Ford works for such graphs. Bellman
Bellman-Ford
Ford is also simpler than Dijkstra and suites
well for distributed systems. But time complexity of Bellman
Bellman-Ford
Ford is O(VE), which is more
than Dijkstra.
How it works?
Like other Dynamic Programming Problems, the algorithm calculates shortest paths in
bottom-up
up manner. It first calculates the shortest distances for the shortest paths which have
at-most
most one edge in the path. Then, it calculates
cal shortest paths with at-most
ost 2 edges, and so
th
on. After the i iteration of outer loop, the shortest paths with at most i edges are calculated.
There can be maximum |V| – 1 edges in any simple path, that is why the outer loop runs |v| –
1 times. The idea is, assuming that there is no negative weight cycle, if we have calculated
shortest paths with
th at most i edges, then an iteration over all edges guarantees to give shortest
path with at-most (i+1) edges

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 19

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Bellman-Ford
Ford algorithm to compute shortest path

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 20

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

7. Travelling Sales Person problem (T2:5.9),

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 21

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 22

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 23

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

8. Reliability design

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 24

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 25

 Lecture Notes | 10CS43 – Design & Analysis
A of Algorithms | Module 4: Dynamic Programming

Prerpared by Harivinod N, Dept of CSE, VCET Puttur Techjourney.in Page| 26