Please read this disclaimer before proceeding:

This document is confidential and intended solely for the educational purpose of
RMK Group of Educational Institutions. If you have received this document
through email in error, please notify the system manager. This document
contains proprietary information and is intended only to the respective group /
learning community as intended. If you are not the addressee you should not
disseminate, distribute or copy through e-mail. Please notify the sender
immediately by e-mail if you have received this document by mistake and delete
this document from your system. If you are not the intended recipient you are
notified that disclosing, copying, distributing or taking any action in reliance on
the contents of this information is strictly prohibited.
 22CS306
Design and Analysis of
Algorithms
Department: CSE
Batch/Year: 2023-2027/II
Created by:
Dr. D.Rajalakshmi, Associate Professor
Mr. K Mohana Sundaram , Asst.Professor
Mr. D Jayakumar, Asst Professor
Ms. J. Sherine Glory, Asst Professor
Table of Contents
Sl. Topics Page
No. No.

1. Contents 5

2. Course Objectives 6

3. Pre Requisites (Course Name with Code) 8

4. Syllabus (With Subject Code, Name, LTPC details) 10

5. Course Outcomes (6) 12

6. CO-PO/PSO Mapping 14

Lecture Plan (S.No., Topic, No. of Periods, Proposed date, 16

7. Actual Lecture Date, pertaining CO, Taxonomy level, Mode of
Delivery)
8. Activity based learning 18

Lecture Notes ( with Links to Videos, e-book reference, PPTs, 21

9.
Quiz and any other learning materials )
Assignments ( For higher level learning and Evaluation - 51
10.
Examples: Case study, Comprehensive design, etc.,)
11. Part A Q & A (with K level and CO) 53

12. Part B Qs (with K level and CO) 69

Supportive online Certification courses (NPTEL, Swayam, 62

13.
Coursera, Udemy, etc.,)
14. Real time Applications in day to day life and to Industry 64

Contents beyond the Syllabus ( COE related Value added 66

15.
courses)
16. Assessment Schedule ( Proposed Date & Actual Date) 71

17. Prescribed Text Books & Reference Books 73

18. Mini Project 75

Course Objectives
Course Objectives
 Critically analyse the efficiency of alternative
algorithmic solutions for the same problem
 Illustrate brute force and divide and conquer design
techniques.
 Explain dynamic programming for solving various
problems.
 Apply greedy technique and iterative improvement
technique to solve optimization problems
 Examine the limitations of algorithmic power and
handling it in different problems.
PRE REQUISITES
Prerequisites

22CS201-Data Structures 22CS101-Problem Linear

Solving using Programming
C++ Problem &
Geometric
Trees Problem

Graphs

Searching
& Sorting
Syllabus
 Syllabus L T P C
22CS306 DESIGN AND ANALYSIS OF ALGORITHMS
3 0 2 4
Unit I : INTRODUCTION 9+6
Notion of an Algorithm – Fundamentals of Algorithmic Problem Solving – Fundamentals of the
Analysis of Algorithmic Efficiency – Asymptotic Notations and their properties. Analysis
Framework – Empirical analysis - Mathematical analysis for Recursive and Non-recursive
algorithms.
List of Exercise/Experiments:
1.Perform the recursive algorithm analysis.
2.Perform the non-recursive algorithm analysis.
Unit II : BRUTE FORCE AND DIVIDE AND CONQUER 9+6
Brute Force – String Matching – Exhaustive Search - Knapsack Problem. Divide and Conquer
Methodology – Binary Search – Merge sort- Quick sort – Multiplication of Large Integers –
Closest-Pair and Convex Hull Problems. -Transform and Conquer Method: Heap Sort
List of Exercise/Experiments:
1.Write a program to search an element using binary search
2.Write a program to sort the elements using merge sort and find time complexity.
3.Write a program to sort the elements using quick sort and find time complexity.
4.Write a program to sort the elements using heap sort
Unit III : DYNAMIC PROGRAMMING 9+6
Dynamic programming – Principle of optimality - Floyd‘s algorithm – Multi stage graph -
Optimal Binary Search Trees – Longest common subsequence - Matrix-chain multiplication –
Travelling Salesperson Problem – Knapsack Problem and Memory functions.
List of Exercise/Experiments:
1.Solve Floyd’s algorithm
2.Write a program to find optimal binary search tree for a given list of keys.
3.Solve the multi-stage graph to find shortest path using backward and forward approach
4.Write a program to find the longest common subsequence
Unit IV : GREEDY TECHNIQUE AND ITERATIVE IMPROVEMENT 9+6
Greedy Technique - Prim‘s algorithm and Kruskal’s Algorithm, Huffman Trees. The Maximum-
Flow Problem – Maximum Matching in Bipartite Graphs- The Stable marriage Problem.
List of Exercise/Experiments:
1.Write a program to find minimum spanning tree using Prim’s algorithm
2.Implement Kruskal’s algorithm to find minimum spanning tree
3.Write a program to solve maximum flow problem
Unit V : BACKTRACKING AND BRANCH AND BOUND 9+6
P, NP, NP - Complete and NP Hard Problems. Backtracking – N-Queen problem - Subset Sum
Problem. Branch and Bound – LIFO Search and FIFO search - Assignment problem –
Knapsack Problem - Approximation Algorithms for NP-Hard Problems – Travelling Salesman
problem
List of Exercise/Experiments:
1.Write a program to implement sum of subset problem.
2.Write a program to solve N-Queen problem
3.Solve the assignment problem using branch and bound technique
4.Solve knapsack problem using branch and bound technique
Course Outcomes
Course Outcomes
CO# COs K Level

CO1 Evaluate the efficiency of algorithms using asymptotic notations. K6

Design, implement, and analyze the time complexity of various
CO2 K4
sorting algorithms.
Develop solutions for optimization problems using dynamic
CO3 K3
programming techniques.
CO4 Implement and analyze greedy algorithms for optimization problems. K3
Develop solutions for complex problems using backtracking and
CO5 K3
branch and bound techniques..
Analyze NP-complete problems and implement approximation
CO6 K4
algorithms for these problems.

Knowledge Level Description

K6 Evaluation

K5 Synthesis

K4 Analysis

K3 Application

K2 Comprehension

K1 Knowledge
CO – PO/PSO Mapping
 CO – PO /PSO Mapping Matrix

CO PO PO PO PO PO PO PO PO PO PO PO PO PSO PSO PS0

# 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3

CO1 3 3 - 1 - - 2 2 2 - 2 3 2 -

CO2 3 2 - 2 2 - - 2 2 2 - 2 3 2 -

CO3 3 3 2 - 2 - - 2 2 2 - 2 3 2 -

CO4 3 2 2 2 - - - 2 2 2 - 2 3 3 -

CO5 3 2 2 - 2 - - 2 2 2 - 2 3 2 -

CO6 3 2 2 - - - - 2 2 2 - 2 2 2 -
Lecture Plan
Unit III
Lecture Plan – Unit 3 - DYNAMIC PROGRAMMING
Sl. Topic Num Proposed Actual CO Taxo Mode
No ber Date Lecture nomy of
. of Date Level Deliver
Perio y
ds
Dynamic
programming – Chalk &
1 Principle of 1 CO3 K3 Talk,
optimality-Floyd‘s Demo
algorithm
Chalk &
Multi stage graph –
2 1 CO3 K3 Talk,
Forward Approach
Demo
Chalk &
Multi stage graph –
3 1 CO3 K3 Talk,
Backward Approach
Demo
Chalk &
Optimal Binary
4 1 CO3 K3 Talk,
Search Trees
Demo
Chalk &
Longest common
5 1 CO3 K3 Talk,
subsequence
Demo
Chalk &
Matrix-chain
6 1 CO3 K3 Talk,
multiplication
Demo
Travelling Chalk &
7 Salesperson 1 CO3 K3 Talk,
Problem Demo
Chalk &
8 Knapsack Problem 1 CO3 K3 Talk,
Demo
Chalk &
9 Memory functions 1 CO3
Talk
Activity Based Learning
Unit III
Activity Based Learning
Word Hunt
Activity Based Learning

Activity Link

Travelling Salesman Metric No-Repeat Traveling

Problem - Simulation Salesperson Problem -
VisuAlgo
Longest Common Suffix Array - VisuAlgo
Subsequence
Lecture Notes – Unit 3
 UNIT 3 DYNAMIC PROGRAMMING
Sl. No. Contents Page No.

1 Dynamic Programming 23

2 Principle of Optimality 23

3 Floyd’s Algorithm (All-Pairs Shortest-Paths Problem) 24

4 Multi Stage Graphs 28

5 Optimal Binary Search Trees 32

6 Knapsack Problem and Memory Functions 37

7 Longest common subsequence 41

8 Matrix-chain multiplication 44

9 Travelling Salesperson Problem 47

 Unit III DYNAMIC PROGRAMMING
3.1. Dynamic Programming:
Dynamic Programming was invented in 1950 by Richard Bellman an U.S
mathematician as a general method for optimizing multistage decision processes.
It is a planning concept, which is a technique for solving problems with
overlapping subproblems. This arises from a recurrence relating a solution to a
given problem with solutions to its smaller subproblems of the same type.
Rather than solving overlapping subproblems it suggests solving each of the
smaller subproblems only once and recording the results in a table from which
we can then obtain a solution to the original problem. It can also be used for
avoiding using extra space. Bottom-up approach all the smaller subproblems
to be solved. In top-down it avoids solving the unnecessary subproblems, which
is called memory-functions.

3.2. Principle of Optimality:

Dynamic Programming makes use of principle of Optimality when finding solution to
given problem
• It states that in an optimal sequence of choices or decisions, each sub sequence
must be optimal
• when it is not possible to apply principle of optimality, it is almost impossible
to obtain the solution using dynamic programming approach.
A fundamental difference between the greedy strategy and dynamic programming
is that in the greedy strategy only one decision sequence is generated, wherever
in the dynamic programming, a number of them may be generated. Dynamic
programming technique guarantees the optimal solution for a problem whereas
greedy method never gives such guarantee.

Applications of Dynamic Programming

 Longest common subsequence

 Shortest path problems
 Multi Stage Graphs
 All Pair Shortest Path
 Travelling Sales Person Problem
 0/1 Knapsack
 Matrix chain multiplication
 Optimal Merge portions
 Mathematical optimization
 Optimal Binary Search Tree
 Unit III DYNAMIC PROGRAMMING

Differentiate between Divide & Conquer Method vs Dynamic Programming .

Divide & Conquer Method Dynamic Programming

1.It deals (involves) three steps at each 1.It involves the sequence of four steps:
level of recursion:
Divide the problem into a number of  Characterize the structure of optimal
subproblems. solutions.
Conquer the subproblems by solving them  Recursively defines the values of
recursively. optimal solutions.
Combine the solution to the subproblems  Compute the value of optimal
into the solution for original subproblems. solutions
in a Bottom-up minimum.
 Construct an Optimal Solution from
computed information.

2. It is Recursive. 2. It is non Recursive.

3. It does more work on subproblems and 3. It solves subproblems only once and then
hence has more time consumption. stores in the table.
4. It is a top-down approach. 4. It is a Bottom-up approach.
5. In this subproblems are independent of 5. In this subproblems are interdependent.
each other.
6. For example: Merge Sort & Binary Search 6. For example: Matrix Multiplication.
etc.

3.3. Floyd’s Algorithm (All-Pairs Shortest-Paths Problem):

Floyd’s algorithm is an algorithm for finding shortest paths for all pairs in a weighted
connected graph (undirected or directed) with (+/-) edge weights.

A distance matrix is a matrix (two-dimensional array) containing the distances, taken

pairwise, between the vertices of graph.

The lengths of shortest paths in an n × n matrix D called the distance matrix:

the element dij in the ith row and the jth column of this matrix indicates the
length of the shortest path from the ith vertex to the jth vertex.
 Unit III DYNAMIC PROGRAMMING

We can generate the distance matrix with an algorithm that is very similar to
Warshall’s algorithm is called Floyd’s algorithm.

Floyd’s algorithm computes the distance matrix of a weighted graph with n vertices
through a series of n × n matrices: D(0), . . . , D(k−1), D(k), . . . , D(n)

The element dij(k) in the ith row and the jth column of matrix D(k) (i, j = 1, 2, . . . , n, k

= 0, 1,. . . , n) is equal to the length of the shortest path among all paths from the
ith vertex to the jth vertex with each intermediate vertex, if any, numbered not
higher than k.

Steps to compute D(0), . . . , D(k−1), D(k), . . . , D(n)

The series starts with D(0), which does not allow any intermediate vertices in its
paths; hence, D(0) is simply the weight matrix of the graph.

As in Warshall’s algorithm, compute all the elements of each matrix D(k) from its
immediate predecessor D(k−1).

The last matrix in the series, D(n), contains the lengths of the shortest paths
among all paths that can use all n vertices as intermediate and hence is
nothing other than the distance matrix.

Let dij(k) be the element in the ith row and the jth column of matrix D(k). This
means that dij(k) is equal to the length of the shortest path among all paths

from the ith vertex vi to the jth vertex vj with their intermediate vertices
numbered not higher than k.
 Unit III DYNAMIC PROGRAMMING

Fig.3.1.Underlying idea of Floyd’s algorithm.

The length of the shortest path can be computed by the following recurrence:

ALGORITHM Floyd(W[1..n, 1..n])

//Implements Floyd’s algorithm for the all-pairs shortest-paths problem
//Input: The weight matrix W of a graph with no negative-length cycle
//Output: The distance matrix of the shortest paths’ lengths D ←W //is not necessary
if W can be overwritten
for k←1 to n do
for i ←1 to n do
for j ←1 to n do
D[i, j ]←min{D[i, j ], D[i, k]+ D[k, j]}
return D
Floyd’s Algorithm’s time efficiency is only Θ(n3).
Space efficiency is Θ(n2). i.e matrix size.
 Unit III DYNAMIC PROGRAMMING

Lengths of the shortest paths with no

intermediate vertices (D(0) is simply the
weight matrix).

Lengths of the shortest paths with

intermediate vertices numbered not
higher than 1, i.e., just a (note two
new shortest paths from b to c and
from d to c ).
Lengths of the shortest paths with
intermediate vertices numbered not
higher than 2, i.e., a and b (note a
new shortest path from c to a).
Lengths of the shortest paths with
intermediate vertices numbered not
higher than 3, i.e., a, b, and c (note
four new shortest paths from a to b,
from a to d, from b to d, and from d to
b).
Lengths of the shortest paths
with intermediate vertices numbered
not higher than 4, i.e., a, b, c, and
d (note a new shortest path from c to
a).

Application of Floyd’s algorithm to the digraph shown. Updated

elements are shown in bold.
 Unit III DYNAMIC PROGRAMMING
3.4. Multi Stage Graphs:
A multistage graph G = (V, E) is a directed graph in which the vertices are partitioned into k >2

disjoint sets Vi, 1 <i <k. In addition, if <u, v> is an edge in E, then u Vi and v Vi+1 for
some i, 1 <i <k.
Let the vertex ‘s’ is the source, and ‘t’ the sink. Let c (i, j) be the cost of edge <i, j>. The cost
of a path from ‘s’ to ‘t’ is the sum of the costs of the edges on the path. The multistage graph
problem is to find a minimum cost path from ‘s’ to ‘t’. Each set Vi defines a stage in the
graph. Because of the constraints on E, every path from ‘s’ to ‘t’ starts in stage 1, goes to
stage 2, then to stage 3, then to stage 4, and so on, and eventually terminates in stage k.

A dynamic programming formulation for a k-stage graph problem is obtained

by first noticing that every stop path is the result of a sequence of k–2 decisions. The ith
decision involves determining which vertex in vi+1, 1 <i <k - 2, is to be on the path.
Let c (i, j) be the cost of the path from source to destination. Then using the forward approach, we
obtain:

cost (i, j) = min {c (j, l) + cost (i + 1,l)} l in Vi+1 <j, l> in E

Algorithm Fgraph(G, k, n, p)
// The input is a k-stage graph G = (V, E) with n vertices indexed in order or stages.
//E is a set of edges and c [i,j] is the cost of (i, j).
//p [1 : k] is a minimum cost path.
{
cost [n] :=0.0;
for j:= n - 1 to 1 step – 1 do
// compute cost[j] let r be a vertex such that (j, r) is an edge of G and c [j, r] + cost
[r] is minimum;
cost [j] := c [j, r] + cost[r]; d [j] :=r:

// Find a minimum cost path.

p [1] := 1; p [k] :=n;
for j := 2 to k - 1 do
p [j] := d [p [j -1]];
}

The multistage graph problem can also be solved using the backward approach. Let
bp(i, j) be a minimum cost path from vertex s to j vertex in Vi. Let B cost(i, j) be
the cost of bp(i, j).
 Unit III DYNAMIC PROGRAMMING
From the backward approach we obtain:

Bcost (i, j) = min { Bcost (i –1, l) + c (l, j)} 1 in Vi -1 <l, j> in E

Algorithm Bgraph(G, k, n,p)

// Same function as Fgraph
{
Bcost [1] :=0.0;
for j := 2 to n do
{// Compute Bcost[j]. Let r be such that (r, j) is an edge of G and Bcost [r] + c [r, j] is
minimum;
Bcost [j] := Bcost [r] + c [r,j]; D [j] :=r;
}
p [1] := 1; p [k] :=n;
for j:= k - 1 to 2 do p [j] := d [p [j +1]]; //find a minimum costpath

EXAMPLE1:

Find the minimum cost path from s to t in the multistage graph of five stages shown below.
Do this first using forward approach and then using backward approach.

FORWARD APPROACH:

We use the following equation to find the minimum cost path from s to t: cost
(i, j) = min {c (j, l) + cost (i + 1,l)} l in Vi +1 <j, l>in E
 Unit III DYNAMIC PROGRAMMING
Step 1
Step 1 uses the forwarded approach (cost (5,12) = 0 ).
Here, 5 represents the stage number and 12 represents a node in that stage. Since
there are no outgoing edges from vertex 12, the cost is 0.
Step 2
cost(4,9)=c(9,12)=4 , d[4,9]=12
cost(4,10)=c(10,12)=2, d[4,10]=12
cost(4,11)=c(11,12)= 5, d[4,11]=12
Step 3
cost(3,6)=min{c(6,9)+cost(4,9),c(6,10)+cost(4,10)}
min{6+4,5+2}
min{10,7}=7
cost(3,6)=7, d[3,6]=10
cost(3,7)=min{c(7,9)+cost(4,9),c(7,10)+cost(4,10} = min{4+4, 3+2}
min{8, 5}=5
cost(3,7)=5, d[3,7]=10
cost(3,8)=min{c(8,10)+cost(4,10),c(8,11)+cost(4,11)}=min{5+2,6+5}
min{7,11}=7
cost(3,8)=7, d[3,8]=10

Step 4
cost(2,2)=min{c(2,6)+cost(3,6),c(2,7)+cost(3,7),c(2,8)+cost(3,8)} min{4+7,2+5,1+7}
min{11,7,8}=7
cost(2,2)=7, d[2,2]=7
cost(2,3)=min{c(3,6)+cost(3,6),c(3,7)+cost(3,7)} min{2+7,7+5}
min{9,12}=9
cost(2,3)=9, d[2,3]=6
cost(2,4)=c(4,8)+cost(3,8)=11+7=18
cost(2,4)=18, d[2,4]=8
cost(2,5)=min{c(5,7)+cost(3,7),c(5,8)+cost(3,8)}=min{11+5,8+7}=min{16,15}=15
cost(2,5)=15 d(2,5)=8
 Unit III DYNAMIC PROGRAMMING
Step-5
cost(1,1)=min{c(1,2)+cost(2,2),c(1,3)+cost(2,3),c(1,4)+cost(2,4),c(1,5)+cost(2,5)} min
{9+7,7+9,3+18,2+15}
min {16,16,21,17}=16
cost(1,1)=16, d[1,1]=2 or 3
To find the path
d[1,1]=2 or 3
d[2,2]=7 or d[2,3]=6
d[3,7]=10 or d[3,6]=10
d[4,10]=12
So the path is 1, 2, 7, 10, 12 with cost = 16 or 1,3,6,10,12 with cost = 16

Backward Approach
Step 1
Bcost(1,1)=0
Step 2
Bcost(2,2)=c(1,2)+bcost(1,1)=9+0=9 d(2,2)=1
Bcost(2,3)=c(1,3)+bcost(1,1)=7+0=7 d(2,3)=1
Bcost(2,4)=c(1,4)+bcost(1,1)=3+0=3 d(2,4)=1
Bcost(2,5)=c(1,5)+bcost(1,1)=2+0=2 d(2,5)=1
Step 3
Bcost(3,6)=min(Bcost(2,2)+c(2,6),Bcost(2,3)+c(3,6)} = min{9+4,7+2}=min{13,9}=9
Bcost(3,6)=9 d(3,6)=3
Bcost(3,7)=min(Bcost(2,2)+c(2,7),Bcost(2,3)+c(3,7),Bcost(2,5)+c(5,7)}
=min{9+2,7+7,2+11}=min{11,14,13}=11
Bcost(3,7)=11 d(3,7)=2
Bcost(3,8)=min(Bcost(2,2)+c(2,8),Bcost(2,4)+c(4,8),Bcost(2,5)+c(5,8)}
=min{9+1,3+11,2+8}=min{10,14,10}=10
Bcost(3,8)=10 d(3,8)=2 or 5
Step 4
Bcost(4,9)=min{Bcost(3,6)+c(6,9),Bcost(3,7)+c(7,9)}
=min(9+6,11+4}=min{15,15}=15
Bcost(4,9)=15 d(4,9)=6 or 7
 Unit III DYNAMIC PROGRAMMING
Bcost(4,10)=min{Bcost(3,6)+c(6,10),Bcost(3,7)+c(7,10),Bcost(3,8)+c(8,10)}
=min(9+5,11+3,10+5}=min{14,14,15}=14
Bcost(4,10)=14 d(4,10)=6 or 7
Bcost(4,11)=Bcost(3,8)+c(8,11)
=10+6=16
Bcost(4,11)=16 d(4,11)=8
Step 5
Bcost(5,12)=min{Bcost(4,9)+c(9,12),Bcost(4,10)+c(10,12),Bcost(4,11)+c(11,12)}
=min{15+4,14+2,16+5}=min{19,16,21}=16
Bcost(5,12)=16 d(5,12)=10
d(5,12)=10
d(4,10)=6 or 7
d(3,6)=3 or d(3,7)=2
d(2,3)=1 or d(2,2)=1
So the path is 1, 3, 6, 10, 12 with cost = 16 or path is 1, 2, 7, 10, 12 with cost =16

3.5. Optimal Binary Search Trees:

A binary search tree is one of the most important data structures in computer
science. One of its principal applications is to implement a dictionary, a set of
elements with the operations of searching, insertion, and deletion.

Figure 3.2.Two out of 14 possible binary search trees with keys A, B, C, and D.

Consider four keys A, B, C, and D to be searched for with probabilities 0.1, 0.2, 0.4, and
0.3, respectively. Figure 3.6 depicts two out of 14 possible binary search trees
containing these keys.

The average number of comparisons in a successful search in the first of these trees
is 0.1 x 1+ 0.2 x 2 + 0.4 x 3+ 0.3 x 4 = 2.9, and for the second one it is
 Unit III DYNAMIC PROGRAMMING
0.1 x 2 + 0.2x 1+ 0.4 x 2 + 0.3 x 3= 2.1.
Neither of these two trees is optimal.

The total number of binary search trees with n keys is equal to the nth Catalan
number,

c(n)=(2n)!/(n+1)!n!

Let a1, . . . , an be distinct keys ordered from the smallest to the largest
and let p1, . . . pn be the probabilities of searching for them. Let C(i, j) be the
smallest average number of comparisons made in a successful search in a binary
search tree Ti j
made up of keys ai, . . . , aj, where i, j are some integer indices, 1≤ i
≤ j ≤ n.

Consider all possible ways to choose a root ak among the keys ai, . . . , aj . For such
k−1
a binary search tree (Figure 3.7), the root contains key ak, the left subtree Ti
j
contains keys ai, . . . , ak−1 optimally arranged, and the right subtree T k+1 contains
keys ak+1, . . ., aj also optimally arranged.

If we count tree levels starting with 1 to make the comparison numbers equal the keys’
levels, the following recurrence relation is obtained:

Figure 3.3 Binary search tree (BST) with root ak and two optimal binary search
subtrees Ti k−1 and T k+1 j.
 Unit III DYNAMIC PROGRAMMING

We assume in above formula that C(i, i − 1) = 0 for 1≤ i ≤ n + 1, which can be

interpreted as the number of comparisons in the empty tree. Note that this formula
implies that C(i, i) = Pi for 1≤ i ≤ n, as it should be for a one-node binary search

tree containing ai.

The two-dimensional table in Figure 3.6.3 shows the values needed for
computing C(i, j). They are in row i and the columns to the left of column j and in
column j and the rows below row i. The arrows point to the pairs of entries whose
sums are computed in order to find the smallest one to be recorded as the value of
C(i, j). This suggests filling the table along its diagonals, starting with all zeros
on the main diagonal and given probabilities Pi, 1≤ i ≤ n, right above it and moving
toward the upper right corner.

Figure 3.4 Table of the dynamic programming algorithm for

construction of OBST
 Unit III DYNAMIC PROGRAMMING

ALGORITHM OptimalBST(P [1..n])

//Finds an optimal binary search tree by dynamic programming
//Input: An array P[1..n] of search probabilities for a sorted list of n keys
//Output: Average number of comparisons in successful searches in the
// optimal BST and table R of subtrees’ roots in the optimal BST
for i ←1 to n do
C[i, i − 1]←0
C[i,i]←P[i]
R[i,i]←i
C[n + 1, n]←0
for d ←1 to n − 1 do //diagonal count
for i ←1 to n − d do
minval← ∞
for k←i to j do
if C[i, k − 1]+ C[k + 1, j]< minval
minval←C[i, k − 1]+ C[k + 1, j]; kmin←k
R[i, j]←kmin
sum←P[i];
for s ←i + 1 to j do
sum←sum + P[s]
C[i, j ]←minval + sum
return C[1, n], R

The algorithm’s space efficiency is clearly quadratic, ie, : Θ(n3); the time
efficiency of this version of the algorithm is cubic. It is Possible to reduce the
running time of the algorithm to Θ(n2) by taking advantage of monotonicity of
entries in the root table, i.e., R[i,j] is always in the range between R[i,j-1] and
R[i+1,j].

EXAMPLE: Let us illustrate the algorithm by applying it to the four-key set we

used at the beginning of this section:

Key A B C D
Probability 0.1 0.2 0.4 0.3
 Unit III DYNAMIC PROGRAMMING

Let us compute C(1, 2):

Thus, out of two possible binary trees containing the first two keys, A
and B, the root of the optimal tree has index 2 (i.e., it contains B), and the
average number of comparisons in a successful search in this tree is 0.4.

We arrive at the following final tables:

Thus, the average number of key comparisons in the optimal tree is equal to 1.7.
Since R(1,4) = 3, the root of the optimal tree contains the third key, i.e., C. Its left
subtree is made up of keys A and B, and its right subtree contains just key D. To
find the specific structure of these subtrees, we find first their roots by
consulting the root table again as follows. Since R(1, 2) = 2, the root of
the optimal tree containing A and B is B, with A being its left child (and the
root of the one node.
Tree: R(1, 1) = 1). Since R(4, 4) = 4, the root of this one-node optimal tree is its only
key D. Figure 3.10 presents the optimal tree in its entirety.

Figure 3.5 Optimal binary search tree for the above example.
 Unit III DYNAMIC PROGRAMMING

3.6.Knapsack Problem and Memory Functions:

Designing a dynamic programming algorithm for the knapsack problem:
 Given n items of known weights w1, . . . , wn and values v1, . . . , vn
and a knapsack of capacity W, find the most valuable subset of the
items that fit into the knapsack.

 Assume that all the weights and the knapsack capacity are
positive integers; the item values do not have to be integers.

0 / 1 knapsack problem means, the chosen item should be either null or whole.
Recurrence relation that expresses a solution to an instance of the
knapsack problem:
Let us consider an instance defined by the first i items, 1≤ i ≤ n, with
weights w1, . . . , wi, values v1, . . . , vi , and knapsack capacity j, 1 ≤ j ≤ W.
Let F(i, j) be the value of an optimal solution to this instance, i.e., the
value of the most valuable subset of the first i items that fit into the knapsack of
capacity j.
We can divide all the subsets of the first i items that fit the knapsack of capacity
j into two categories: those that do not include the ith item and those that do.
Note the following:

1.Among the subsets that do not include the ith item, the value of an optimal
subset is, by definition, F(i − 1, j).
Among the subsets that do include the ith item (hence, j – wi ≥ 0), an
optimal subset is made up of this item and an optimal subset of the first i − 1 items
that fits into the knapsack of capacity j − wi. The value of such an optimal subset
is vi + F(i − 1, j − wi).
Thus, the value of an optimal solution among all feasible subsets of the first I
items is the maximum of these two values. Of course, if the ith item does not
fit into the knapsack, the value of an optimal subset selected from the first i items
is the same as the value of an optimal subset selected from the first i − 1
items.
 Unit III DYNAMIC PROGRAMMING

These observations lead to the following recurrence:

It is convenient to define the initial conditions as follows:

F(0, j) = 0 for j ≥ 0 and F(i, 0) = 0 for i ≥ 0.

Goal is to find F(n, W), the maximal value of a subset of the n given items that fit into the
knapsack of capacity W, and an optimal subset itself.
For F(i, j),compute the maximum of the entry in the previous row and the same
column and the sum of vi and the entry in the previous row and wi columns to the
left. The table can be filled either row by row or column by column.

ALGORITHM DPKnapsack(w[1..n], v[1..n], W)

for j := 0 to W do
F[0,j] := 0
for i := 0 to n do
F[i,0] := 0
for i := 1 to n do
for j := 1 to W do
if w[i] <= j then
F[i,j]=max{F[i-1,j],v[i]+F[i-1,j-w[i]]}
else
F[i,j] := F[i-1,j];
return V[n,W] and the optimal subset by backtracking

Note: Running time and space: O(nW).

Table 3.1 for solving the knapsack problem by dynamic programming.

 Unit III DYNAMIC PROGRAMMING
EXAMPLE 1 Let us consider the instance given by the following data: Table
An instance of the knapsack problem:

item weight value capacity

1 2 $12
2 1 $10
W=5
3 3 $20
4 2 $15

The maximal value is F(4, 5) = $37. We can find the composition of an optimal
subset by backtracing (Back tracing finds the actual optimal subset. i.e.
solution), the computations of this entry in the table. Since F(4, 5) > F(3, 5), item
4 has to be included in an optimal solution along with an optimal subset for
filling 5 − 2 = 3 remaining units of the knapsack capacity.

Similarly, since F(1, 2) > F(0, 2), item 1 is the final part of the optimal
solution {item 1, item 2, item 4}. Solving an instance of the knapsack problem by
the dynamic programming algorithm.
Capacity j

i 0 1 2 3 4 5

0 0 0 0 0 0 0

w1 = 2, v1 = 12 1 0 0 12 12 12 12

w2 = 1, v2 = 10 2 0 10 12 22 22 22

w3 = 3, v3 = 20 3 0 10 12 22 30 32

w4 = 2, v4 = 15 4 0 10 15 25 30 37

Memory Functions
The direct top-down approach to finding a solution to such a recurrence leads to an
algorithm that solves common subproblems more than once and hence is very
inefficient.

The bottom up fills a table with solutions to all smaller subproblems, but each of
them is solved only once. An unsatisfying aspect of this approach is that
solutions to some of these smaller subproblems are often not necessary for getting a
solution to the problem given.
 Unit III DYNAMIC PROGRAMMING
Since this drawback is not present in the top-down approach, it is natural to try to
combine the strengths of the top-down and bottom-up approaches. The goal is
to get a method that solves only subproblems that are necessary and does so only
once. Such a method exists; it is based on using memory functions.

This method solves a given problem in the top-down manner but, in addition,
maintains a table of the kind that would have been used by a bottom-up
dynamic programming algorithm.

Initially, all the table’s entries are initialized with a special “null” symbol to
indicate that they have not yet been calculated. Thereafter, whenever a new
value needs to be calculated, the method checks the corresponding entry in the
table first: if this entry is not “null,” it is simply retrieved from the table;
otherwise, it is computed by the recursive call whose result is then recorded in the
table. The following algorithm implements this idea for the knapsack problem.

After initializing the table, the recursive function needs to be called with i = n
(the number of items) and j = W (the knapsack capacity).

ALGORITHM MFKnapsack(i, j )
//Implements the memory function method for the knapsack problem
//Input: A nonnegative integer i indicating the number of the first items being
considered
// and a nonnegative integer j indicating the knapsack capacity
//Output: The value of an optimal feasible subset of the first i items
//Note: Uses as global variables input arrays Weights [1..n], Values[1..n],
// and table F[0..n, 0..W ] whose entries are initialized with −1’s except for
// row 0 and column 0 initialized with 0’s
if F[i, j ]< 0
if j <Weights[i]
value←MFKnapsack(i − 1, j)
else
value←max(MFKnapsack(i − 1, j),
Values[i]+ MFKnapsack(i − 1, j −Weights[i]))
F[i, j ]←value
return F[i, j ]
 Unit III DYNAMIC PROGRAMMING
EXAMPLE 2 Let us apply the memory function method to the instance considered in
Example 1.

Capacity j
I 0 1 2 3 4 5

0 0 0 0 0 0 0

w1 = 2, v1 = 12 1 0 0 12 12 12 12

w2 = 1, v2 = 10 2 0 - 12 22 - 22

w3 = 3, v3 = 20 3 0 - - 22 - 32

w4 = 2, v4 = 15 4 0 - - - - 37

Only 11 out of 20 nontrivial values (i.e., not those in row 0 or in column 0) have been
computed. Just one nontrivial entry, V (1, 2), is retrieved rather than being
recomputed. For larger instances, the proportion of such entries can be significantly
larger.

3.7. Longest Common Subsequence:

The longest common subsequence problem is finding the longest sequence which exists in
both the given strings.

Subsequence
Let us consider a sequence S = < s1, s2, s3, s4, …,sn>.

A sequence Z = < z1, z2, z3, z4, …,zm> over S is called a subsequence of S, if and only if

it can be derived from S deletion of some elements.

Common Subsequence

Suppose, X and Y are two sequences over a finite set of elements. We can say that Z is a
common subsequence of X and Y, if Z is a subsequence of both X and Y.
 Unit III DYNAMIC PROGRAMMING

Longest Common Subsequence

If a set of sequences are given, the longest common subsequence problem is to find a
common subsequence of all the sequences that is of maximal length.
The longest common subsequence problem is a classic computer science problem, the
basis of data comparison programs such as the diff-utility, and has applications in
bioinformatics.
It is also widely used by revision control systems, such as SVN and Git, for reconciling
multiple changes made to a revision-controlled collection of files.

By Dynamic Programming:
Let X = < x1, x2, x3,…, xm> and Y = < y1, y2, y3,…, yn > be the sequences. To compute
the length of an element the following algorithm is used.
In this procedure, table C[m, n] is computed in row major order and another
table B[m,n] is computed to construct optimal solution.

Algorithm: LCS-Length-Table-Formulation (X, Y):

m := length(X)
n := length(Y)
for i = 1 to m do
C[i, 0] := 0
for j = 1 to n do
C[0, j] := 0
for i = 1 to m do
for j = 1 to n do
if xi = yj
C[i, j] := C[i - 1, j - 1] + 1
B[i, j] := ‘D’
else
if C[i -1, j] ≥ C[i, j -1]
C[i, j] := C[i - 1, j]
B[i, j] := ‘U’
else
C[i, j] := C[i, j - 1]
B[i, j] := ‘L’
return C and B
 Unit III DYNAMIC PROGRAMMING

Algorithm: Print-LCS (B, X, i, j)

if i = 0 and j = 0
return
if B[i, j] = ‘D’
Print-LCS(B, X, i-1, j-1)
Print(xi)
else if B[i, j] = ‘U’
Print-LCS(B, X, i-1, j)
else
Print-LCS(B, X, i, j-1)

This algorithm will print the longest common subsequence of X and Y.

Analysis

To populate the table, the outer for loop iterates m times and the inner for loop
iterates n times.

Hence, the complexity of the algorithm is O(m, n), where m and n are the length of
two strings.
Example

In this example, we have two strings X = BACDB and Y = BDCB to find the longest
common subsequence.
Following the algorithm LCS-Length-Table-Formulation (as stated above), we have
calculated table C (shown on the left hand side) and table B (shown on the right hand
side).
In table B, instead of ‘D’, ‘L’ and ‘U’, we are using the diagonal arrow, left arrow and up
arrow, respectively. After generating table B, the LCS is determined by function LCS-
Print. The result is BCB.
 Unit III DYNAMIC PROGRAMMING

3.8. Matrix-chain Multiplication:

Suppose we have a sequence or chain A1, A2, …, An of n matrices to be multiplied. That

is, we want to compute the product A1A2…An

There are many possible ways (parenthesizations) to compute the product.

Example: consider the chain A1, A2, A3, A4 of 4 matrices

Let us compute the product A1*A2*A3*A4

There are 5 possible ways:

1. (A1(A2(A3A4)))
2. (A1((A2A3)A4))
3. ((A1A2)(A3A4))
4. ((A1(A2A3))A4)
5. (((A1A2)A3)A4)

To compute the number of scalar multiplications necessary, we must know:

Algorithm to multiply two matrices

Matrix dimensions

Algorithm to Multiply 2 Matrices

Input: Matrices Ap×q and Bq×r (with dimensions p×q and q×r)

Result: Matrix Cp×r resulting from the product A·B

MATRIX-MULTIPLY(Ap×q , Bq×r)
1. for i ← 1 to p
2. for j ← 1 to r
3. C[i, j] ← 0
4. for k ← 1 to q
5. C[i, j] ← C[i, j] + A[i, k] · B[k, j]
6. return C
 Unit III DYNAMIC PROGRAMMING

Example: Consider three matrices A10100, B1005, and C550

There are 2 ways to parenthesize

((AB)C) = D105 · C550

AB  10·100·5=5,000 scalar multiplications
DC  10·5·50 =2,500 scalar multiplications
=7,500

(A(BC)) = A10100 · E10050

BC  100·5·50=25,000 scalar multiplications
AE  10·100·50 =50,000 scalar multiplications
=75,000

Matrix-chain multiplication problem

• Given a chain A1, A2, …, An of n matrices, where for i=1, 2, …, n,
matrix Ai has dimension pi-1pi
• Parenthesize the product A1A2…An such that the total number of
scalar multiplications is minimized

Brute force method of exhaustive search takes time exponential in n.

Dynamic Programming Approach

The structure of an optimal solution
Let us use the notation Ai..j for the matrix that results from the product Ai
Ai+1 … Aj
An optimal parenthesization of the product A1A2…An splits the product
between Ak and Ak+1 for some integer k where1 ≤ k < n

First compute matrices A1…k and Ak+1…n ; then multiply them to get the
final matrix A1…n
Key observation: parenthesizations of the sub chains A1A2…Ak and
Ak+1Ak+2…An must also be optimal if the parenthesization of the
chain A1A2…An is optimal (why?)
That is, the optimal solution to the problem contains within it the optimal
solution to sub problems.
 Unit III DYNAMIC PROGRAMMING

Recursive definition of the value of an optimal solution

Let m[i, j] be the minimum number of scalar multiplications necessary to compute Ai....j .

Minimum cost to compute A1…..n is m[1, n]

Suppose the optimal parenthesization of Ai..j splits the product between Ak and
Ak+1 for some integer k where i ≤ k < j

Ai....j = (Ai Ai+1 Ak )·(Ak +1 Ak +2 ….. Aj)= Ai…..k · Ak +1…..j

Cost of computing Ai….j = cost of computing Ai….k + cost of computing Ak+1….j +

cost of multiplying Ai…..k and Ak+1…..j

Cost of multiplying Ai…..k and Ak+1....j is pi-1 pk pj

m[i, j ] = m[i, k] + m[k+1, j ] + pi-1 pk pj for i ≤ k < j

m[i, i ] = 0 for i=1,2,…,n

But… optimal parenthesization occurs at one value of k among all possible i ≤ k < j
Check all these and select the best one.

To keep track of how to construct an optimal solution, we use a table s

s[i, j ] = value of k at which Ai Ai+1 ……Aj is split for optimal parenthesization

• First computes costs for chains of length l=1

• Then for chains of length l=2,3, … and so on

• Computes the optimal cost bottom-up

Algorithm to Compute Optimal Cost

Input: Array p[0…n] containing matrix dimensions and n

Result: Minimum-cost table m and split table s
MATRIX-CHAIN-ORDER(p[ ], n)
for i ← 1 to n
m[i, i] ← 0
for l ← 2 to n
for i ← 1 to n-l+1
j ← i+l-1
m[i, j] ← 
for k ← i to j-1
q ← m[i, k] + m[k+1, j] + p[i-1] p[k] p[j]
if q < m[i, j]
m[i, j] ← q
s[i, j] ← k
return m and s
 Unit III DYNAMIC PROGRAMMING

Takes O(n3) time

Requires O(n2) space

The algorithm computes the minimum-cost table m and the split table s
The optimal solution can be constructed from the split table s.
Each entry s[i, j ]=k shows where to split the product Ai Ai+1 …… Aj for the minimum
cost show how to multiply this matrix chain optimally.

Assume the following Matrix Dimension:

A1-30×35
A2-35×15
A3-15×5
A4-5×10
A5-10×20
A6-20×25

Solution on the board

Minimum cost 15,125

Optimal parenthesization ((A1(A2A3))((A4 A5)A6))

3.9. Traveling Salesman Problem – Dynamic Programming

Approach:

Introduction to TSP:

In the TSP, given a set of cities and the distance between each pair of cities, a
salesman needs to choose the shortest path to visit every city exactly once and return
to the city from where he started.
Let’s take an example to understand the TSP in more detail:
 Unit III DYNAMIC PROGRAMMING

Here, the nodes represent cities, and the values in each edge denote the distance between
one city to another. Let’s assume a salesman starts the journey from the city A. According
to TSP, the salesman needs to travel all the cities exactly once and get back to the city A by
choosing the shortest path. Here the shortest path means the sum of the distance between
each city travelled by the salesman, and it should be less than any other path.
With only 5 cities, the problem already looks quite complex. As the graph in
the example is a complete graph, from each city, the salesman can reach any other cities in
the graph. From each city, the salesman needs to choose a route so that he doesn’t have to
revisit any city, and the total distance travelled should be minimum.
Let’s discuss an algorithm to solve the TSP problem.
Dynamic Programming Approach for Solving TSP:
In this algorithm, take a subset N of the required cities needs to be visited,
distance among the cities and starting city s as inputs. Each city is identified by unique city
id like (1,2,3,4…n).
Algorithm: Traveling-Salesman-Problem

C ({1}, 1) = 0
for s = 2 to n do
for all subsets S Є {1, 2, 3, … , n} of size s and containing 1
C (S, 1) = ∞
for all j Є S and j ≠ 1
C (S, j) = min {C (S – {j}, i) + d(i, j) for i Є S and i ≠ j}
Return minj C ({1, 2, 3, …, n}, j) + d(j, i)

Initially, all cities are unvisited, and the visit starts from the city s. We assume
that the initial travelling cost is equal to 0. Next, the TSP distance value is calculated based
on a recursive function. If the number of cities in the subset is two, then the recursive
function returns their distance as a base case.
On the other hand, if the number of cities is greater than 2, then we’ll
calculate the distance from the current city to the nearest city, and the minimum distance
among the remaining cities is calculated recursively.
 Unit III DYNAMIC PROGRAMMING

Finally, the algorithm returns the minimum distance as a TSP solution. Use a
dynamic approach to calculate the cost function Cost(). Using recursive calls, calculate the
cost function for each subset of the original problem.

Time Complexity Analysis:

In the dynamic algorithm for TSP, the number of possible subsets can be at most N*2N.
Each subset can be solved in O(N)times. Therefore, the time complexity of this algorithm
would be O(N2*2N).

Conclusion:
TSP is a popular NP-Hard problem, but depending on the size of the input cities, it is
possible to find an optimal or a near-optimal solution using various algorithms.

Illustrate the steps to solve travelling salesman problem with the following example:

From the above graph, the following table is prepared.

1 2 3 4
1 0 10 15 20
2 5 0 9 10
3 6 13 0 12
S=Φ 4 8 8 9 0

Cost(2,Φ,1)=d(2,1)=5

Cost(3,Φ,1)=d(3,1)=6

Cost(4,Φ,1)=d(4,1)=8
 Unit III DYNAMIC PROGRAMMING
Cost( i, s)=min{Cost(j, s-(j))+d[ i, j]}

S=1

Cost(2,{3},1)=d[2,3]+Cost(3,Φ,1)=9+6=15

Cost(2,{4},1)=d[2,4]+Cost(4,Φ,1)=10+8=18

Cost(3,{2},1)=d[3,2]+Cost(2,Φ,1)=13+5=18

Cost(3,{4},1)=d[3,4]+Cost(4,Φ,1)=12+8=20

Cost(4,{3},1)=d[4,3]+Cost(3,Φ,1)=9+6=15

Cost(4,{2},1)=d[4,2]+Cost(2,Φ,1)=8+5=13

S=2

Cost(2,{3,4},1)=min{d[2,3]+Cost(3,{4},1), d[2,4]+Cost(4,{3},1)}

= min {9+20,10+15} = min{29,25} = 25

Cost(3,{2,4},1)=min{d[3,2]+Cost(2,{4},1), d[3,4]+Cost(4,{2},1)}

=min {13+18,12+13} = min {31, 25} = 25

Cost(4,{2,3},1)=min{d[4,2]+Cost(2,{3},1), d[4,3]+Cost(3,{2},1)}

=min {8+15,9+18} = min {23,27} =23

S=3

Cost(1,{2,3,4},1)=min{ d[1,2]+Cost(2,{3,4},1), d[1,3]+Cost(3,{2,4},1),

d[1,4]+cost(4,{2,3},1)}

=min{10+25, 15+25, 20+23} = min{35,40,43}=35

Time Complexity Analysis

In the dynamic algorithm for TSP, the number of possible subsets can be at most N*2N.
Each subset can be solved in O(N)times. Therefore, the time complexity of this algorithm
would be O(N2*2N).

Conclusion
TSP is a popular NP-Hard problem, but depending on the size of the input cities, it is
possible to find an optimal or a near-optimal solution using various algorithms.
Assignments
Unit 3
 Assignments

Sl. Knowledge
Question CO#
No. Level
Solve the multi-stage graph problem using forward and
backward approach

1. CO3 K3

Solve the all-pairs shortest path problem for the digraph

with the following weight matrix

2. CO3 K3

Apply the bottom-up dynamic programming algorithm

to the following instance of the knapsack problem:
Item weight value
1 3 $25
3.
2 2 $20 CO3 K3
3 1 $15
4 4 $40
5 5 $50
capacity W=6.
Construct the optimal binary search tree for the given
data
4. CO6 K3
Key else if int while
Prob 0.1 0.2 0.4 0.3
Solve the following LCS Problem
S1=BCDAACD
5 CO3 K3
S2=ACDBAC
Part A – Q & A
Unit - 3
 PART - A Questions
1. What is Dynamic programming? (CO3,K1)

Dynamic programming is an algorithm design technique for solving problem with

overlapping subprograms. The smaller subprograms are solved only once and recording the
results in a table from which the solution to the original problem is obtained.

2. What are the features of dynamic programming? (CO3,K1)

o Optimal solutions to sub problems are retained so as to avoid recomputing their
values.
o Decision sequences containing subsequences that are sub optimal are not
considered.
o It definitely gives the optimal solution always.
3. What do you mean by multi stage graph? (CO3,K1)

A Multistage graph is a directed graph in which the nodes can be divided into a set of
stages such that all edges are from a stage to next stage only. A multistage graph G = (V,
E) is a directed graph where vertices are partitioned into k (where k > 1) number of
disjoint subsets S = {s1,s2,…,sk}such that edge (u, v) is in E, then u Є si and v Є s1 + 1
for some subsets in the partition and |s1| = |sk| = 1. The vertex s Є s1 is called the source
and the vertex t Є sk is called sink.

4. Write the difference between the Greedy method and Dynamic programming.
(CO3,K1)

Greedy method
i. Only one sequence of decision is generated.
ii. It does not guarantee to give an optimal solution always.
Dynamic programming
● Many numbers of decisions are generated.
● It definitely gives an optimal solution always

5. What is Catalan number? (CO3,K1)

The total number of binary search trees with n keys is equal to the nth Catalan number,
PART - A Questions
6. Define principle of optimality (CO3,K1)

It says that an optimal solution to any instance of an optimization problem is composed

of optimal solutions to its sub instances.

7. Give the Floyd’s algorithm (CO3,K1)

Floyd(W[1..n,1..n])
//Implements Floyd’s algorithm for the all-pair shortest–path problem //Input The
weight matrix W of a graph
//Output The distance matrix of the shortest paths’ lengths D W
for k <- 1 to n do
for i <- 1 to n do
for j <- 1 to n do
D[i,j] = min{D[i,j], D[i,k] + D[k,j]}

8. Define optimal binary search tree (CO3,K1)

A binary search tree which has minimal average comparisons for all its search keys is
called as optimal binary search tree.

9. Define all pair shortest path algorithm. (CO3,K1)

Floyd’s algorithm is otherwise called as all pair shortest path algorithm. It is used to find
the shortest path between every pair of vertices.

10.What is the time complexity of Floyd’s algorithm?

O(n3)

11. What is distance matrix?

A distance matrix is a matrix (two-dimensional array) containing the distances,
taken pairwise, between the vertices of graph.

The lengths of shortest paths in an n × n matrix D called the distance

matrix: the element dij in the ith row and the jth column of this matrix
indicates the length of the shortest path from the ith vertex to the jth vertex.
PART - A Questions
12. Compare Feasible and Optimal solution. (CO3,K2)

Feasible Solution Optimal Solution

A feasible solution is a set of values for the An optimal solution is a feasible solution
decision variables that satisfies all of the where the objective function reaches its
constraints in an optimization problem. maximum (or minimum) value – for
example, the most profit or the least cost.
The set of all feasible solutions defines the A globally optimal solution is one where
feasible region of the problem. there are no other feasible solutions with
better objective function values.

13. Give any two properties of Dynamic programming approach. (CO3,K1)

The properties of are
● overlapping sub-problems
● optimal substructure

14.What is the use of memory function? (CO3,K1)

The memory function solves only subproblems that are necessary and does so only
once. The memory function solves a given problem in the top-down manner but, in
addition, maintains a table of the kind that would have been used by a bottom-up
dynamic Programming algorithm.

15.What are the different approaches used to solve the multi-stage graph.
(CO3,K1)
Forward approach and Backward approach.

16. What is bottom up approach? (CO3, K1)

The smaller problems from the bottom are solved initially and the solution is used to
solve the upper level problems to get the solution for the original given problem.
PART - A Questions
17. Which method is used to solve matrix chain multiplication. (CO3,K1)

Parenthesization method is used to solve matrix chain multiplication.

18. What is longest common subsequence problem? (CO3,K1)

If a set of sequences are given, the longest common subsequence problem is to

find a common subsequence of all the sequences that is of maximal length.

The longest common subsequence problem is a classic computer science problem,

the basis of data comparison programs such as the diff-utility, and has
applications in bioinformatics.

19. What is Travelling Salesman Problem? (CO3,K1)

The nodes represent cities, and the values in each edge denote the distance
between one city to another. A salesman starts the journey from the city A.
According to TSP, the salesman needs to travel all the cities exactly once and get
back to the city A by choosing the shortest path. Here the shortest path means the
sum of the distance between each city travelled by the salesman, and it should be
less than any other path.

20. Give the recurrence relation for knapsack problem. (CO3,K1)

21. What is the time complexity of OBST algorithm? (CO3,K1)

O(n3)
 PART - A Questions
22. What is the use of root table? (CO3,K1)

Root table is used to find the root in the optimal binary search tree and subtrees.

23. What is the time complexity of knapsack problem? (CO3,K1)

O(nW)

24. How Floyd’s algorithm finds the shortest path between pair of vertices?
(CO3,K1)

Generating series of distance matrices D(0), D(1), D(2)…… D(n)

25. Does Floyd’s algorithm work well on negative edge graph? (CO3.K1)

Floyd’s all pairs shortest paths algorithm works for graphs with negative edge weights
because the correctness of the algorithm does not depend on edge's weight being non-
negative.

26. What is the time complexity of Travelling Salesman Problem? (CO3,K1)

O(N2*2N).

27. List out the applications of Longest Common Subsequence problem.

(CO3,K1)

Bioinformatics, natural language processing, and text comparison

28. Define Catalan Number. (CO3,K1)

c(n)=(2n)!/(n+1)!n!
29. What are the uses of Muti-stage graph? (CO3,K1)
Social networks, transportation systems, and communication networks

30. What is the formula used for finding the average number of comparisons in
OBST? (CO3,K1)
Part B – Questions
 Part-B Questions
Q. Questions CO K Level
No. Level

1 Discuss Knapsack problem with memory functions. CO3 K2

Describe the pseudo code to find optimal binary

2 CO3 K2
search tree using dynamic programming
Analyze the algorithm by applying the following keys
and probabilities to obtain the optimal binary tree.
3 CO3 K3
Key A B C D
Probability 0.1 0.2 0.4 0.3
Consider 4 elements a1< a2< a3< a4with q0=
0.25,q1=3/16,q2=q3=q4=1/16.
4 P1=1/4,P2=1/8,P3=P4=1/16. CO3 K3
Construct the optimal binary search tree as a
minimum cost tree.
Consider 5 items along their respective weights and
values:
5 CO3 K3
N=5, W=6, W=(3,2,1,4,5), V=(25,20,15,40,50). Find
Optimal solution.
Explain about Longest Common Subsequence with
6 CO3 K2
suitable Example
Explain about matrix chain Multiplication with suitable
7 CO3 K2
example
Explain about Travelling Salesman Problem using
8 CO3 K2
Dynamic Programming
Apply dynamic programming to find the optimal tour
in the following graph

9 CO3 K3

Solve the following graph for all pair shortest path.

10 CO3 K3
 Part-B Questions
Q. Questions CO K Level
No. Level
Write the Huffman’s Algorithm. Construct the
Huffman’s tree for the following data and obtain its
Huffman’s Code.

11 CO3 K3

Encode the characters BAD

Decode the bit string 001110010110011100
Find the Compression ratio
Solve the graph using backward approach algorithm.

12 CO3 K3

Define multi stage graph. Solve the following

graph using forward approach

13 CO3 K3
Supportive online
Certification courses
(NPTEL, Swayam,
Coursera, Udemy, etc.,)
Supportive Online Courses

Sl. Courses Platform

No.
1 Design and analysis of algorithms NPTEL
2 The Design and Analysis of Algorithm Udemy
3 Algorithms Specialization Coursera
4 Algorithm Design and Analysis edX
Real time Applications in
day to day life and to
Industry
 Real Time Applications
1. In Google Maps to find the shortest path between source and the series of destinations
(one by one) out of the various available paths.

2. In networking to transfer data from a sender to various receivers in a sequential manner.

3.Document Distance Algorithms- to identify the extent of similarity between two text
documents used by Search engines like Google, Wikipedia, Quora, and other websites

4. Edit distance algorithm used in spell checkers.

5. Databases caching common queries in memory: through dedicated cache tiers storing
data to avoid DB access, web servers store common data like configuration that can be
used across requests. Then multiple levels of caching in code abstractions within every
single request that prevents fetching the same data multiple times and save CPU cycles by
avoiding recomputation. Finally, caches within your browser or mobile phones that keep the
data that doesn't need to be fetched from the server every time.

6. Git merge. Document diffing is one of the most prominent uses of LCS.

7. Dynamic programming is used in TeX's system of calculating the right amounts of

hyphenations and justifications.

8. Genetic algorithms.
Content Beyond Syllabus
 Unit III Content Beyond Syllabus

WARSHALL’S ALGORITHM
A directed graph (or digraph) is a graph, or set of vertices connected by edges, where
the edges have a direction associated with them.
An Adjacency matrix A = {aij} of a directed graph is the boolean matrix that has 1 in its
ith row and jth column if and only if there is a directed edge from the ith vertex to the jth
vertex.
The transitive closure of a directed graph with n vertices can be defined as the n x n
boolean matrix T = {tij }, in which the element in the ith row and the jth column is 1 if
there exists a nontrivial path (i.e., directed path of a positive length) from the ith
vertex to the jth vertex; otherwise, tij is 0.

The transitive closure of a digraph can be generated with the help of depth-first
search or breadth-first search. Every vertex as a starting point yields the transitive
closure for all.

FIGURE 3.13.1 (a) Digraph. (b) Its adjacency matrix. (c) Its transitive closure.
 Unit III Content Beyond Syllabus

Warshall’s algorithm constructs the transitive closure through a series of n

× n boolean matrices: R(0), . . . , R(k−1), R(k), . . . R(n).

The element rij(k) in the ith row and jth column of matrix R(k) (i, j = 1, 2, . .
. , n, k = 0, 1, . . ., n) is equal to 1 if and only if there exists a directed path of a
positive length from the ith vertex to the jth vertex with each intermediate vertex,
if any, numbered not higher than k.

Steps to compute R(0), . . . , R(k−1), R(k), . . . R(n).

The series starts with R(0), which does not allow any intermediate vertices in
its paths; hence, R(0) is nothing other than the adjacency matrix of the digraph.

R(1) contains the information about paths that can use the first vertex as
intermediate. it may contain more 1’s than R(0).
The last matrix in the series, R(n), reflects paths that can use all n vertices of
the digraph as intermediate and hence is nothing other than the digraph’s
transitive closure.
In general, each subsequent matrix in series has one more vertex to use as
intermediate for its paths than its predecessor.
The last matrix in the series, R(n), reflects paths that can use all n vertices of
the digraph as intermediate and hence is nothing other than the digraph’s
transitive closure.

FIGURE 3.13.2 Rule for changing zeros in Warshall’s algorithm.

 Unit III Content Beyond Syllabus

All the elements of each matrix R(k) is computed from its immediate predecessor
R(k−1). Let r (k)
, the element in the ith row and jth column of matrix R(k), be
equal to 1. This means that there exists a path from the ith vertex vi to the
jth vertex vj with each intermediate vertex numbered not higher than k.

The first part of this representation means that there exists a path from v i to vk
(k−1)
with each intermediate vertex numbered not higher than k − 1 ( ihence, r =
1), and the second part means that there exists a path from v k to vj with each
intermediate vertex numbered not higher than k − 1 (hence, r kj(k−1) = 1).

Thus the following formula generates the elements of matrix R (k) from the
elements of matrix R(k−1):

Applying Warshall’s algorithm by hand:

If an element rij is 1 in R(k−1), it remains 1 in R(k).

If an element rij is 0 in R(k−1), it has to be changed to 1 in R(k) if and only if the
element in its row i and column k and the element in its column j and row k are
both 1’s in R(k−1).

ALGORITHM Warshall(A[1..n, 1..n])

//ImplementsWarshall’s algorithm for computing the transitive closure
//Input: The adjacency matrix A of a digraph with n vertices
//Output: The transitive closure of the digraph R(0)←A
for k←1 to n do
for i ←1 to n do
 Unit III Content Beyond Syllabus
for j ←1 to n do
R(k)[i, j ]←R(k−1)[i, j ] or (R(k−1)[i, k] and R(k−1)[k, j])
return R(n)

Warshall’s algorithm’s time efficiency is only Θ(n3). Space efficiency is Θ(n2). i.e matrix size.

1’s reflect the existence of paths

with no intermediate vertices (R(0) is
just the adjacency matrix); boxed
row and column are used for getting
R(1).

1’s reflect the existence of paths

with intermediate vertices numbered not
higher than 1, i.e., just vertex a (note a
new path from d to b); boxed
row and column are used for getting
R(2).
1’s reflect the existence of paths
with intermediate vertices numbered not
higher than 2, i.e., a and b (note two
new paths); boxed row and column
are used for getting R(3).
1’s reflect the existence of paths
with intermediate vertices numbered not
higher than 3, i.e., a, b, and c (no new
paths);
boxed row and column are used for
getting R(4).
1’s reflect the existence of paths
with intermediate vertices numbered not
higher than 4, i.e., a, b, c, and d (note
five new paths).

Application of Warshall’s algorithm to the digraph shown. New 1’s are in bold.
Assessment Schedule
(Proposed Date & Actual
Date)
 Assessment Schedule

Assessment Proposed Date Actual Date Course Outcome Program

Tool Outcome
(Filled Gap)
Assessment I CO1, CO2

Assessment II CO3, CO4

Model CO1, CO2, CO3,
CO4, CO5
Prescribed Text Books &
Reference
Prescribed Text & Reference
Books

Sl. Book Name & Author Book

No.
1 Anany Levitin, ―Introduction to the Design and Analysis of Text Book
Algorithms‖, Third Edition, Pearson Education, 2012.
2 Ellis Horowitz, Sartaj Sahni and Sanguthevar Rajasekaran, Text Book
Computer Algorithms/ C++, Second Edition, Universities Press,
2019
3 Thomas H.Cormen, Charles E.Leiserson, Ronald L. Rivest and Reference
Clifford Stein, ―Introduction to Algorithms‖, Third Edition, PHI Book
Learning Private Limited, 2012.
4 Alfred V. Aho, John E. Hopcroft and Jeffrey D. Ullman, ―Data Reference
Structures and Algorithms‖, Pearson Education, Reprint 2006. Book
5 Harsh Bhasin, ―Algorithms Design and Analysis‖, Oxford Reference
university press, 2016 Book
6 S. Sridhar, ―Design and Analysis of Algorithms‖, Oxford Reference
university press, 2014. Book
7 http://nptel.ac.in/ Reference
Book
8 https://doc.lagout.org/science/0_Computer%20Science/2_Algor E-Book
ithms/Introduction%20to%20the%20Design%20and%20Analy
sis%20of%20Algorithms%20%283rd%20ed.%29%20%5BLevit
in%202011-10-09%5D.pdf
Mini Project Suggestions
 Mini Project
Dynamic Programming
1. Travel Buddies (CO3,K6)
Much like the traveling salesperson, you and a friend must visit n cities. A fixed distance
between every two pairs of cities is given in advance. You and your friend must divide the
list of cities into two lists, A and B. You visit all the cities on list A and your friend visits all the
cities on list B. Each of you must visit the list of cities in order on your list – the order is
determined by the city subscripts (you must visit city ci before city cj for all i < j). The goal is
to divide the original list of n cities into two sublists so as to minimize the sum of distances
traveled by both you and your friend. Assume there is no cost to arrive at the first city on
each list, and you can end your tour in the last city (no final cost to return home).
Construct a dynamic programming solution that can take any arbitrary list of cities and divide
them so as to minimize the sum of distances traveled. Your input will be integer n [1 to 100]
followed by n × n distances between cities. Yes, the distances are symmetric (distance from
ci to cj is the same as cj to ci so each distance will be given twice). Costs from ci to ci will
always be 0. All distances will be positive floating point values, 0 ≤ di ≤ 100 (no negative
distances or negative cycles).
Read data from stdin. Your first input will be integer n followed by an n × n array of
numbers (may be floating point values) indicating the distances between each pair of
cities.

0 2 10 3 11 13
2 0 13 4 11 15
10 13 0 9 2 3
3 4 9 0 8 12
11 11 2 8 0 4
13 15 3 12 4 0

2. Partition Problem (CO3,K3)

Given a set of positive integers, check if it can be divided into two subsets with equal sum.
S = {3, 1, 1, 2, 2, 1}
3. Bus Stop Prize (CO3,K6)
As a promotion to promote mass transit, Gotham city has awarded you a chance to win
some great prizes by riding one of their busses. The city bus has n stops on its schedule
(1 ≤ n ≤ 100, 000 – it’s a very long bus route). At each stop there is a prize that
has integer monetary value of
−100 ≤ pi ≤ 100. Yes, some prizes are negative. You get to pick the stop where you
board the bus (and hence win that first prize). Then you win each prize along the route
for each stop that you are still on the bus. You select some other stop to deboard the bus
(and win that prize too). Your goal is to determine where you should board and deboard
the bus in order to maximize your total prize winnings. Note that you may be willing to
visit a few negative stops if by doing so you reach bigger prizes down the road. You are
given the n prize winnings ahead of time to help you make your decision: p1, . . . , pn.

Develop a dynamic programming solution to determine the optimal point of boarding and
deboarding given a sequence of n prizes.

Read data from stdin. Your first input will be integer n followed by the integers 1..n
indicating the prizes at each bus stop. An example input is given below:

10
4 -1 5 -4 2 6 -8 -2 9 -3

4. Coin Change Problem (CO3,K3)

Given an unlimited supply of coins of given denominations, find the minimum number of
coins required to get the desired change.
For example, consider S = { 1, 3, 5, 7 }. C=15

5. Rod Cutting Problem (CO3,K3)

Given a rod of length n and a list of rod prices of length i, where 1 <= i <= n, find the
optimal way to cut the rod into smaller rods to maximize profit.
length[]=[1, 2, 3, 4, 5, 6, 7, 8]
price[] = [1, 5, 8, 9, 10, 17, 17, 20]. Rod length: 4
 Thank you

Disclaimer:

This document is confidential and intended solely for the educational purpose of RMK Group of
Educational Institutions. If you have received this document through email in error, please notify the
system manager. This document contains proprietary information and is intended only to the
respective group / learning community as intended. If you are not the addressee you should not
disseminate, distribute or copy through e-mail. Please notify the sender immediately by e-mail if you
have received this document by mistake and delete this document from your system. If you are not
the intended recipient you are notified that disclosing, copying, distributing or taking any action in
reliance on the contents of this information is strictly prohibited.