Daa
Daa
Daa
CHAPTER 1
INTRODUCTION
Algorithm:
It is a sequence of unambiguous instructions for solving a problem to obtain a
required output for any legitimate input in a finite time.
PROBLEM
ALGORITHM
INPUT
CPU
OUTPUT
Validation process contains two forms one is producing respective outputs for
given inputs and the other is specification.
These two forms are shown equivalent for a given input.
The purpose of the validation of an algorithm is to assure that this algorithm will
work correctly independently of the issues such as machine, language, platform etc. If
an algorithm is according to given specifications and is producing the desired outputs
for a given input then we can say it is valid.
Order of magnitude
It refers to the frequency of execution of a statement. The order of magnitude of
an algorithm is sum of all frequencies of all statements. The running time of an
algorithm increases with size of input and number of operations.
It is sum of all frequencies of execution of statements in a program.
Example (1):
3
In linear search, if the element to be found is in the first position, the basic operation
done is one. If it is some where in array, basic operation (comparison) is done n times.
If in an algorithm, the basic operations are done same number of times every time then it
is called Every time complexity analysis T(n).
Example (2):
Addition of n numbers
Input : n
Alg: for 1 to n
Sum = sum + n;
T(n) = n;
Example (3):
Matrix multiplication
Input : n
Alg : for i = 1 to m
for j = 1 to n for k = 1 to n
c[i] [j] = c[I] [j] + a[i] [k] * b[k] [i]
T(n) = n3
A(n) =
KX
k 1
1
1 n(n 1)
n 1
=
=
2
n
2
n
Case 2:
If x is not in array
Probability that x is in Kth slot = p/n
Probability that x is not in array = 1 P
n
A(n) =
k 1
ORDER
O(1)
O(log n)
O(n)
O(n log n)
O(n2)
O(n3)
O(2 n)
O(n!)
1.
2.
3.
KX
P
P n(n 1)
p
p
+ n(1 P) = n 1 .
= n(1 P) =
n
2
n
2
n
NAME
EXAMPLES
Constant
Few algorithms without any loops
Logarithmic
Searching algorithms
Linear
Algorithms that scan a list of size n
n-log-n
Divide and conquer algorithms
Quadratic
Operations on n by n matrices
Cubic
Nontrivial algorithms from linear algebra
Exponential
Algorithms that generate all subsets of a set
Factorial
Algorithms that generate permutations of set
Algorithm with time complexities of O(n) and 100n are called linear time
algorithms since time complexity is linear with input size.
Algorithm such as O(n2), 0.001 n2 are called quadratic time algorithms.
Any linear time algorithm is efficient than quadratic time algorithm.
T(n)
O(2n)
O(n2)
O(logn)
O(n)
O(logn)
n
If algorithm is run on same complexity on same type of data, but with higher
magnitude of n, the resulting time is less than some constant time f(n).
Thus
O(1) O(log n) O (n) O (nlogn) O(n2) O(2n) for a given N.
ASYMPTOTIC NOTATIONS:
In priori analysis, all factors regarding machine and language are ignored and
only inputs and number of operations are taken into account. The principle indicator of
the algorithms efficiency is order of growth of an algorithms basic operation count.
Three notations O (big oh), (big omega) and (big theta) are used for this purpose.
Big O:
5
f(n)
n
N
Example (1):
If 100n +5 O(n2) , find C and N.
For all n>= 5
100n + 5 <=100n + n
= 101n <=101n2
thus C =101 and N =5
Example (2):
If n2 + 10 n O( n2) , find C and N
f(n) = n2 + 10 n , g(n) = n2
f(n) O(g(n))
When C = 2 and N = 10
If n <= 10 then c* g(n) f(n)
Thus N = 10 keeps an upper bound complexity of above function
Theorem 1:
| a0 | | a1 |
m 1 | am | n m
n n
Example:
If f(n) = aknk + ..a0
= O(nk) and f(n) ~ O(aknk).
6
Big :
Definition: A function f(n) is said to be in (g(n)) ,
denoted f(n) (g(n)) , if f(n) is bounded below by
some positive constant multiple of g(n) for all large
n i.e., iff there exists positive constants C and N such that
f(n) >= C * g(n) for all N>=N
T(n)
g(n)
cf(n)
Example (1):
Show that n3 ( n2 )
n3 >= n2 for all n>=0
for C=1 and N =0 we can say n3 ( n2 )
Theorem 2:
T(n)
T(n)
O(n)
g(n)
(n)
(n)
(1)
(2)
(3)
Example (1):
If we say n(n-1)/2 (n2), find c1, C2 and N
To prove upper bound
n(n-1)/2 = n2/2 - 1/2n <= n2/2 for all n>=0
To prove lower bound
n(n-1)/2 = n2/2 - 1/2n >= n2/2 - n2/4 = n2/4, for all n>=2
hence C1 = , C2 =1/4 and N=2
Theorem 3:
If f1(n) O(g1(n)) and f2(n) O(g2(n)), then f1(n) + f2(n) O(max{ g1(n) , g2(n)} )
7
Proof:
Let a1, a2,bq,b2 be real numbers and if a1 <=, bq and a2 <=b2
then a1 + a2 <=2 max(b1,b2)
if f1(n) O(g1(n))
then f1(n) <= C1 * (g1(n)) for n>=N1
and if f2(n) O(g2(n))
then f2(n) <= cfor n>=N2
Let C3 = max {C1, C2} and N >= max{ N1, N2}
f1(n) + f2(n) <= C1 * (g1(n)) + C2 * (g2(n))
<= C3 * (g1(n)) + C3 * (g2(n))
<= C3 { (g1(n)) + (g2(n)) } = C3 * 2 max { (g1(n)) , (g2(n)) }
hence f1(n) + f2(n) O(max{ g1(n) , g2(n)} )
with the constants C and N required by O definition being 2C3 = 2 max {C1, C2} and
max { N1, N2}
CHAPTER 2
Introduction:
The strategy of divide and conquer which was successfully employed by British
rulers in India may also be applied to develop efficient algorithms for complicated
problems.
DAC algorithms works according to following plan
1.Problems instance is divided into several smaller instances of the same problem.
2.Smaller instances are solved and the solutions thus obtained are combined to get
a solution to original problem.
It is similar to top down approach but, TD approach is a design methodology
(procedure) where as DAC is a plan or policy to solve the given problem. TD approach
deals with only dividing the program into modules where as DAC divides the size of
input..
In DAC approach the problem is divided into sublevels Where it is abstract. This
process continues until we reach concrete level, where we can conquer(get solution).
Then combine to get solution for the original problem. The problem containing n inputs
is divided into into K distinct sets, yielding k sub problems. These sub problems must
be solved and sub solutions are combined to form solution.
Problem of size n
n
n/2
n/2
Solution to original
Problem
Control Abstraction:
Procedure DAC(p, q)
{
if RANGE(p, q) is small
Then return G(p, q)
9
Else
{
m = DIVIDE(p, q)
return (COMBINE (DAC(p, m), DAC(m + 1, q)))
}
}
Step 1:
Step 2:
Step 3:
g ( n)
2T (n / 2) f (n)
T(n) =
n is small
otherwise
T (2 m )
T (2 m 1 ) f (2 m )
2m
2m
2m
T (2 m 1 ) k (2 m )
=
2 m 1
2m
Let k=1
T (2 m ) T (2 m 1 )
1
2m
2 m 1
T (2m2 )
=
11
2 m2
T (2 m m )
=
+m
2 m m
= T(1) + m
=m+1
m
T(2 ) = 2m(m + 1)
T(n) = n(logn + 1)
= O(nlogn)
Example (1):
Find time complexity for DAC
a)
If g(n) = O(1) and f(n) = O(n),
b)
If g(n) = 0(1) and f(n) = O(1)
a)
It is the same as above case
10
= nT(1) +
i 1
i 0
=n+
i 1
- 2-1
i 0
= n + (n 1 )
= 2n 3/2
T(n) = O(n)
Time complexity:
The given data set is divided into two half and search is done in only one half
wither left half or right half depending on the value of x
n is 1
g ( n)
T ( n / 2) g ( n ) n 1
T(n) =
Example (1):
Take instances 12
14
18 22
24
38
1
2
3
4
5
6
The element to be found is the key for searching and let Key = 24
Therefore mid = 3
low
high
mid
1
6
3
4
6
5
Thus number of comparisons needed are two.
Theorem 1:
Prove that E = I + 2n, for a binary search tree.
E = sum of all length, of external nodes
1
I = sum of all length, internal nodes
n = internal nodes
2
for level 2 for n = 3
4
5
4
E = 22.2 = 8
E = I + 2n
I = 21 > ! + 2
8=2+6=8
By induction if we add these branches to form a tree it satisfied E = I + 2n
Theorem 2:
Prove average successful search time S is equal to average unsuccessful search time U,
if n .
ltn Savg = ltn Uavg
E
1
Uavg =
Savg =
+1
n 1
n
I 2n l 2n
Uavg =
n 1
n
I
I 2n
Eavg
= 2
n
n
I 2n
U=
n 1
12
I = U(n + 1) 2n
S=
u ( n 1) 2n
1
n
ltn S = U 2 + 1 = U 1 U
Total int enal path lengths
1
Savg =
n
Total external path lengths
Uavg =
n 1
Max-Min Problem using DAC
Problem:
To find out maximum and minimum elements in a given list of elements.
Analysis:
According to divide and conquer algorithm the given instance is divided into
smaller instances.
Step 1:For example an instance I=(n,A(1),A(2)..A(n)) is divided into I1=(n/2,A(1),A(2),
.A(n/2)) and I2=(n-n/2, A(n/2 +1),..A(n)).
Step 2: If Max(I) and Min(I) are the maximum and minimum of the elements in I trhe
Max(I) = the larger of Max(I1) and Max(I2) and Min(I)=the smaller of Min(I1) and Min(I2).
Step 3:The above procedure is repeated until I contains only one element and then the
answer is computed.
Example:Let us consider an instance 22,13,-5,-8,15,60,17,31,47 from which maximum
and minimum elements to be found.
22, 13, -5, -8, 15, 60, 17, 31, 47
1
2 3 4 5 6 7 8 9
1, 9, 60, -8
i, j, max,min
1, 5, 22, -8
i, j, max,min
1, 3, 22, -5
i, j, max,min
1, 2, 22, 13
i, j, max,min
4, 5, 15, -8
i, j, max,min
6, 9, 60, 17
i, j, max,min
6, 7, 60, 17
i, j, max,min
3, 3, -5, -5
i, j, max,min
13
8, 9, 47, 31
i, j, max,min
Algorithm 1:
Procedure:
Max Min (A[1]..A[n])
{
for i = 2 to n do
if A[i] > max
then min = A[i]
endif
if A[i] < min
then min = A[i]
endif
endfor
}
This algorithm requires (n1) comparisons for finding largest number and (n1)
comparisons for smallest number.
T(n) = 2(n 1)
Algorithm 2:
If the above algorithm is modified as
If A[i] > max
then max = A[i]
Else
A[i] < min
then min = A[i]
Best case occurs when elements are in increasing order. Number of comparisons for this
case is (n 1). Worst case occurs when elements are in decreasing order. Number of
comparisons for this case is 2(n-1).
Average number of comparisons =
(n 1) 2( n 1) 3n
1
2
2
If i = j then
Max = Min = A[I]
else
If j i = 1
If A[i] < A[j]
max = A[j]
min = A[j]
else
max = A[j]
min = A[I]
else
{
i j
p=
2
MAX MIN ((i, p, hmax, hmin)
MAX MIN ((p + 1, j, Lmax, Lmin)
Max = large (hmax, Lmax)
Min = small (hmin, Lmin)
Time complexity:
0
n 1
1
n
2
T(n) =
2T ( n / 2) 2 n 2
Thus
n
T(n) = 2T 2
2
n/2
2} 2
2
= 2{ 2T
= 22 T(n/22) + 22 + 2
= 2k 1T(n/2k-1) + 2k-1 +++ 2
let n = 2k
k 1
=2
T(2) +
k1
i 1
but T(2) = 1
15
k 1
= 2k 1 +
-1
i 1
= 2k 1 + 2k1 1
= 2k/2 + 2k 2
= 3/2 2k 2 = 3/2 n 2
T(n) = O(3/2 n)
Theorem 3:
Prove that T(n) = O(nlogm)
Ans:
= m T(n/2 ) + an
k
i 1
i 1
n=2
= mk + an2
i 1
l / m
i 1
m
1
= mk + an2
m
1
m
k
m 1 1
= mk + an2
m 1 m
k 11
m (1 + an /m)
k
if m >> n
= mk = mlogn
but O(m ) = O(nlogm)
logn
1.Merge Sort:
Problem:
Sorting the given list of elements in ascending or decreasing order.
Analysis:
Step 1: Given a sequence of n elements, A(1),A(2),.A(n). Split them into two sets A(1)
..A(n/2) and A(n/2+1)..A(n).
Step 2: Splitting is carried over until each set contain only one element.
Step 3: Each set is individually sorted and merged to produce a single sorted sequence.
Step 4:Step 3 is carried over until a single sequence of n elements is produiced
16
Example: Let us consider the instances 10. They are 310,285, 179, 652, 351, 423, 861,
254, 450, 520
310, 285, 179, 652, 351, 423, 861, 254, 450, 520
1
2
3
4
5
6
7
8
9
10
For first recursive call left half will be sorted (310) and
1,10
(285) will be merged, then (285, 310) and 179 will be
1,5
6,10
merged and at last (179, 285, 310) and (652, 351) are
merged to give 179, 285, 310, 351, 652 Similarly, for
1,3
4,5
6,8
9,10
second recursive call right half will be sorted to give
254, 423, 520, 681
A partial view of merge sorting is shown in fig
1,1 1,2 2,2 3,3
Finally merging of these two lists produces
179, 254, 285, 310, 351, 423, 450, 520, 652, 820
Algorithm:
Step 1: Given list of elements
I = {A(1), A(2) A(n)} is divided into two sets
I1 = {A(1), A(2).A(n/2)}
I2 = {A(n/2 + 1..A(n)}
Step 2: Sort I1 and sort I2 separately
Step 3: Merge them
Step 4: Repeat the process
Procedure merge sort (p, q)
{
if p < q
mid = p + q/2:
merge sort (p, mid);
merge sort (mid + 1, q);
merge (p, mid, q);
endif
}
Procedure Merge (p, mid, q)
{
let h = p; I = p; j = mid + 1
while (h mid and j q)
do
{
if A(h) A(j)
B(I) = A(h);
h = h + 1;
else
B(i) = A(j);
17
j = j + 1;
end if
I = I + 1;
if h > mid
for k = j to q
B(i) = A(k);
I = I + 1;
repeat
else
if j > q
for k = h to mid
B(i) = A(k);
I = I + 1;
repeat
endif
for k = p to q
A(k) = B(k);
Repeat
}
Time Complexity:
a
n 1
T(n) = 2T n cn n 1
= 2(2T(n/4) + 2cn
= 4T(n/4) + 2cn
= 2kT(n/2k) + kcn
n = 2k
= n.a + kcn
= na = cn logn
= O(n logn)
QUICK SORT
Problem: Arranging all the elements in a list in either acending or descending order.
Analysis:
File A(1n) was divided into subfiles, so that sorted subfiles does not need merging.
This can be achieved by rearranging the elements in A(1n) such that A(i)<=A(j) for all
1<I<m and all m+1<j<n for some m, 1<=m<=n. Thus the lements in A(1.m) and
A(m+1,..n)may be independently sorted.
Step 1: Select first element as pivot in the given sequence A(1n)
Step 2: Two variables are initialized
Lower = a + 1
upper = b
Lower scans from left to right
18
A11
A
21
A12 B11
A22 B21
B12
C
11
B22
C21
C12
C22
Q = (A21 + A22)B11
S = A22(B21 B11)
U = (A21 A11) (B11 + B12)
C11 = P + S T + V
C21 = Q + S
C12 = R + T
C22 = P + R Q + U
T(n) =
2
8T (n / 2) cn n 2
Where b and c are constants
This recurrence can be solved in the same way as earlier recurrences to obtain T(n) +
O(n ). Hence no improvement over the conventional method has been made. Since matrix
multiplications are more expensive than matrix addition O(n3) versus O(n2), we can attempt to
reformulate the equations for Cij so as to have fewer multiplications and possibly more
additions. Volker Stressen has discovered a way to compute the C ijs of using only 7
multiplications and 18 additions or subtractions. This method involves first computing the
seven n/2 n/2 matrices P, Q, R, S, T, U and V. Then the C ijs are computed using the
3
20
T(n) =
2
7T (n / 2) an n 2
Where a and b are constants. Working with this formula, we get
T(n) = an2[1 + 7/4 + (7/4)2 + + (7/4)k 1] + 7kT(1)
cn2 (7/4)logn + 7logn, c a constant
= 0(nlog7) O(n2.81).
21
Chapter 3
Greedy Method
Greedy method is a general design technique despite the fact that it is applicable to
optimization problems only. It suggests constructing a solution through a sequence of steps
each expanding a partially constructed solution obtained so far until a complete solution is
reached.
According to greedy method an algorithm works in stages. At each
stage a decision is made regarding whether an input is optimal or
not.
To find an optimal solution two parameters must be considered. They are
1.
Feasibility: Any subset that satisfies some specified constraints, then it is said to be
feasible.
2.
Objective function: This feasible solution must either maximize or minimize a given
function to achieve an objective.
3.
Irrevocable: Once made, it cannot be changed on subsequent stages.
Step 1: A decision is made regarding whether or not a particular input is in an optimal
solution. Select an input
Step 2: If inclusion of input results infeasible solution, then delete form solution.
Step 3: Repeat until optimization is achieved.
Control abstract:
Procedure GREEDY(A[], n)
{
Solution =
for i = 1 to n do
{
x = select (A(i))
if FEASIBLE (solution, x)
Solution = UNION(solution, x)
}
return(solution);
}
1 k j
ik
.
22
If all the programs are retrieved with equal probabilities, mean retrieval time MRT =
1
t j . To minimize MRT, we must store the programs such that their lengths are in non
n 1 j n
MRT = n
1 j n
1 k j
ik
Example 1:
There are 3 programs of length I = (5, 10, 3)
1, 2, 3 5 + 15 + 18 = 38
1, 3, 2 5 + 5 + 18 = 31
3, 1, 2 3 + 8 + 18 = 29 is the best case where retrieval time is minimized, if the next program
of least length is added.
Time complexity:
If programs are assumed to be in order, then there is only on loop in algorithm, thus
Time complexity = O(n). If we consider sorting also, sorting requires O(nlogn)
T(n) = O(nlogn) + O(n)
= O(nlogn)
Problem 2:
To store different programs of length l1, l2, l3.ln with different frequencies f1,
f2fn on a single tape so that MRT is less.
Solution:
f ik
. The programs must be
1 k j lik
arranged such that ratio of frequency and length must be in non increasing order.
l1 l2 l3..ln
f1 f2 f3..fn
f1 f 2
fn
l1 l2
ln
Knapsack problem
Given n objects and a knapsack, object i has a weight w i and knapsack has capacity of
M. If a fraction xi, 0 xj 1 of object i is placed into knapsack then a profit pjxI is earned.
Problem: To fill the knapsack so that total profit is maximum and satisfies two conditions.
1)
Objective
px
1 i n
= maximum
2)
Feasibility
w x
1i n
Types of knapsack:
1.
Normal knapsack: in normal knapsack, a fraction of the last object is included to
maximize the profit.
23
2.
i.e., 0 x 1 so that w1 x1 M
O/I knapsack: In 0/1 knapsack, if the last object is able to insert completely, then only it
is selected otherwise it not selected.
i.e., x = 1/0 so that w1 x1 M
Pi
P
i 1 and
wi wi 1
P1/W1P7/W7 =
5
1.6
3
1
6
4.5
3
Arrange in non increasing order of Pi/WI
Order = 5
1
6
3
7
2
4
P/w = 6
3
4.5
3
3
1.6
1
C = 15 x = 0
1)
x[1] = 1 x = 0
2)
x[2] = 1
c = 15 1 = 14
3)
x[3] = 1 c = 14 2 = 12
4)
x[4] = 1
c = 12 4 = 8
5)
x[5] = 1 c = 8 5 = 3
6)
x[6] = 2/3
c=31=2
7)
x[7] = 0 c = 2 2/3.3 = 0
X=1 1
1
1
1
2/3
0
Rearranging into original form
X = 1 2/3
1
0
1
1
1
Solution vector for O/I knapsack is X = 1010111
Total profit pi xi = 10 + 15 + 0 + 6 + 18 + 3 = 52
Theorem:
If p1/w1 p2/w2 ..pn/wn then greedy knapsack generate optimal solution.
Proof: Let X = (x1 = (x1,..x2) be solution vector, let j be the least index such that
xj 1
Xi = 1 for 1 i j
Xi = 0 for j i n
Xi = 0 for 0 xj 1
Let y = (y1 ..yn) be optimal solution
wi xi M
2)
3)
if k > j
1)
w x
i
Job Sequencing:
Let there are n jobs, for any job i, profit Pi is earned if it is completed in dead line di.
Object: Obtain a feasible solution j = (j1, j2..) so that the profits i.e.,
p
i j
is maximum.
The set of jobs J is such that each job completes within deadline.
Job sequencing
Problem: There are n jobs, for any job I a profit PI is earned iff it is completed in a dead line di.
Feasibility: Each job must be completed in the given dead line.
Objective: The sum of the profits of doing all jobs must be minimum, i.e. PI maximum.
25
Solution: Try all possible permutations and check if jobs in J, can be processed in any of these
permutations without violating dead line. If J is a set of K jobs and = i1, i2,ik a
permutation of jobs in J such that di1 <= di2 <=..dik. Then J is a feasible solution.
Control abstract:
Procedure JS(D, J, N, K)
{
D(0) = J(0) = 0;
K = 1, J(1) = 1
For I = 2 to n do
{
r=k
while D(J(r)) > max (D(i), r)
{
r = r 1;
}
if D(i) > r then
for (L = k; 1 = r + 1, 1 = 1 1)
J(1 + 1) = J91)
}
J(r + 1) = i;
k = k + 1;
}
}
}
Example: Given 5 jobs, such that they have the profits and should be complete within dead
lines as given below. Find and optimal sequence to achieve maximum prodit.
P1..P5 =
20
15
10
5
1
D1d5 =
2
2
1
3
3
Processing sequence
Value
Feasible
solutio
n
1)
1, 2
2, 1 or 1, 2
35
2)
1, 3
3, 1
30
3)
1, 4
1, 4
25
4)
1, 2, 3
5)
1, 2, 4
1, 2, 4
40
Optical solution is 1, 2, 4
Try all possible permutations and if J can be produced in any of these permutations
without violating dead line.
If J is a set of k jobs and = i 1, i2,.i k, such that d i1 di2 ..dik then j is a
feasible solution.
Optimal Merge pattern:
Merging of an n record file and m record file requires n + m records to move. At each
step merge smallest size files. Two way merge pattern is represented by a merge tree.
26
Leaf nodes (squares) represent the given files. File obtained by merging the files is a
parent file(circles). Number in each node is length of file or number of records.
Problem: To merge n sorted files, at the cost of minimum comparisons.
Objective: To merge given field.
If di is the distance from root to external node for file f i , qI is the length of f i, then total
n
d q
i 1
Example:
5 7 9 13
3
13
10
2 3 5 7 9 13
5
39
23
5
3
16
10
5
Algorithm:
2 3
Procedure Tree(l, n)
For I = l to n
{
Call GETNODE(T)
L CHILD(T) = LEAST(L);
R CHILD(T) = LEAST (L);
WEIGHT(T)
WEIGTH(LCHILD(T)) + WEIGHT(RECHILD(T));
Call INSERT(L, T);
}
return(LEAST(L))
}
ANALYSIS:
Main loop is executing (n 1) times. If L is in non decreasing order
LEAST(L) requires only O(l) and INSERT(L, T) can be done in O(n) times.
Total time taken = O(n2)
Minimum spanning Tree:
Let G = (V, E) is an undirected connected graph. A sub graph T = (V, E) of G is a
spanning tree of F iff T is a tree. 0
Any connected graph with n vertices must have at least n 1 edges and all connected
graphs with n 1 edge are trees. Each edge in the graph is associated with a weight.
Such a weighted graph is used for construction of a set of communication links at a
minimum cost. Removal of any one of the links in the graph if it has cycles will result
a spanning tree with minimum cost. The cost of minimum spanning tree is sum of costs
of edges in that tree.
A greedy method is to obtain a minimum cost spanning tree. The next edge to include
is chosen according to optimization criteria in the sum of costs of edges so far
included.
1) If a is he set of edges in a spanning tree.
2) The next edge (u, v) to be included in A is a minimum cost edge and A U(u, v) is also a
tree.
PRIMS algorithm:
1
27
30
4
2
40
20
45
35
3
5 35
Cost
10
30
20
35
ST
1
2
1
10 2
30 1
2
10
30
41 10
20 2
Algorithm:
5
304
Procedure PRIM(E, COST, n, mincost, int NEAR(n)<
[n] [n], I, j, k, l)
{
(k, 1) = edge with mini cost
4
min cost = cost(k, 1)
20
35
(T(1, 1), T(1, 2)) = (k, 1)
3
5
for i = 1 to n
{
if COST (i, l) < cost (i, k)
then NEAR(i) = 1
else
NEAR(i) = k
}
}
NEAR(k) = NEAR(l) = 0
For j = 2 to n 1
{
If Near(j) = 0 find cost(j, NEAR(j)) which is mini
T(i, l), T(i, 2) = (j, NEAR(j))
NEAR(j) = 0
For k = 1 to n
{
If NEAR(k) ! = 0 and cost (k, NEAR(k)) >COST(k, j)
Then NEAR(k) = j;
}
}
Time complexity:
The total time required for the algorithm is (n2).
Krushkals algorithm:
If E is the set of all edges of G, determine an edge with minimum.
Cost(v, w) and delete from E. If the edge(v, w) does not create any cycle in the tree T,
add(v, w) to T. Resent the process until all edges are covered.
Algorithm:
Procedure KRUSKAL(E, COST, n, T)
{
construct a heap with edge costs
I=0
Min cost = 0
Parent (l, n) = -1
28
I = I + 1;
T(I, 1) = u;
T(1, 2) = v;
Mincost = mincost + cost(u, v)
UNION(j, k)
}
}
}
Procedure FIND(i)
{
J =I
While PARENT(j) > 0
{
J = PARENT(j)
}
K = I;
While K! = j
{
T = PARENT(k);
PARENT(k) = j;
K=T
}
Return(j)
}
Else
Procedure UNION(i, j)
{
x = PARENT(i) + PARENT(j)
If PARENT(i) > PARENT(j)
{
PARENT(i) = j
PARENT(j) = x
PARENT(j) = i
PARENT(i) = x
}
}
Time Complexity:
29
To develop a heap the time taken is O(n). To adjust the heap it takes a time complexity
of O(logn). The set of edges to be included in minimum cost spanning tree is n, hence it
is O(n).
Total time complexity is in the order of O(nlogri)
Krushkals Alg
10
1
2 50
Example:
30
45
4
20
Edge
1,
3,
4,
2,
1,
3,
2
4
6
6
4
5
25
6
40 3
5 35
55
Cost
10
15
20
25
30
35
15
SF
6
1
3
Single source shortest paths
Graphs may be used to represent highway structure in which vertices represent cities
6
4
and edges represent sections of highway. The edges are assigned with weight which
might be distance or time.
1 shortest
2 path between given
Now the problem of single source shortest paths is to find
3
source and distination. In the problem a directed graph G = 4
(V, E), a weighting function
C(e) for the edges of G and source vertex V, are given. So we must6find the shortest
paths from v0 to all remaining vertices of G.
Greedy algorithm for single source shortest path
Step 1 : A shortest paths are build one by one so that the problem becomes multisage
1
2
solution problem
3 so3 far generated
Step 2 : For optimization, at every stage sum of the lengths4 all paths
is minimum.
6
The greedy way to generate shortest paths from v0 to remaining vertices woulbe be to
generate these paths in non-decreasing order of path length.
The algorithm for the single source shortest paths using greedy method leads to a
simple algorithm called Dijkstras algorithm.
Procedure SHORTEST PATHS (v, COST, DIST, n)
{
for i = 1 to n
{
S[i] = 0
30
2
300
1 90
2 300
3 1000
4
5
6
7
8 1700
800
1200
1000
1500
5
250
1000
1
0
80017000
1200
1400
900
1500 1000
0 250
1000
0
900 1400
0
1000
5
5,
5,
5,
5,
5,
6
6,
6,
6,
6,
7
7,4
7, 4, 8
7,4,8,3
Vertex
Selected
6
7
4
8
3
2
3350
3350
3250
2450
2450
2450
1250
1250
1250
1250
1250
1250
0
0
0
0
0
0
250
250
250
250
250
250
1150
1150
1150
1150
1150
1650
1650
1650
1650
1650
31
ANS:
a) T(n) = 2T(n/2) + n
= 2{ 2T(n/4) +n/2} +n
=22T(n/22) + 2n
=
= 2mT(n/2m) + mn
=
=
=
=
nT(n/n) + nlog n
nT(1) + nlog n
n + nlog n
n log n
for n=1
for n>1
ANS:
a) T(n) = T(n/2) + cn
= { T(n/4) +nc/2} +cn
since f(n) = O(n)
=T(n/22) + 3cn/2
=
= T(n/2m) + cn(1/2m-1 +1/2m-2 + 1/2m-3 +.1)
let 2m =n, m= log n
T(n)
3.
Write the algorithm for trinary search and derive the expression for its time
complexity
TRINARY SEARCH
Procedure: TRISRCH (p, q)
{
a = pt((q p)/3)
b = p + ((2*(q p))/3
if A[a] = x then
return(a);
else
else
if A[b] = x then
return(b);
if A[b] = x then
return(b);
else
if A[a] > x then
TRISRCH(p, (a I))
else
Time complexity:
g ( n)
n2
T(n) = n
T
f ( n) n 2
3
n = 3k
= log3n
T(n) = O(log3n)
4.
Give an algorithm for binary search so that the two halfs are unequal. Derive the
expression for its time complexity?
Binary Search:
(Split the input size into 2 sets such that 1/3rd and 2/3rd)
procedure BINSRCH(p, q)
{
mid = p + (q p)/3
if A(mid) = x then
return(mid);
else
if A(mid) > x then
BINSRCH (p, mid 1);
Else
BINSRCH (mid + 1, q);
}
}
Time Complexity:
g ( n)
2
n
T(n) =
T
g ( n)
3
if n 1
T
/ T g ( n)
3
Worst case
best case
T(n) = T(cn/3) + g(n)
= T(ck n/3k) + kg(n)
let n = 3k
T(n) = T(ck) + log3n
if c = 2
T(n) = O(2k) + logn = O(2k)
C=1
T(n) = O(1) + logn = logn
5.
Give the algorithm for finding min-max elements when the given list is divided into
3 sub sets. Derive expression for its time complexity?
MAX MIN
If each leaf node has 3 elements
If I = j then
Max = min = a[i]
Else
If j I = 1 then
If a[i] > a(j)
Max = a[i];
Min = a[j];
Else
Max = a[j];
35
Min = a[i];
}
if j i = 2 then
if a[i] > a[j]
{
then
max = a[i];
{
then
}
else
if
max = a[j+1];
min = a[j];
}
}
max = a[I + 1]
min = a[I]2
else
p = I + j/2
MAX MIN (I, p), hmax, Imin)
MAX MIN((p + 1), j), Imax, Imin)
FMax = large(hmax, hmin)
fMin = small(lmax,
}
Time Complexity:
0
T(n) =
3
2T (n / 2) 2
T(n) = 2T(n/2) + 2
36
lmin)
n 1
n2
n3
n3
= 2(2T(n/4) + 2) + 2
= 2k(T(n/2k)) + 2k + ..2
2k = n/3
kth level there are 3 elements for each node.
T(n) = 2k.T(2) +
2k
i 1
= 3.2 + 2
2
= n + 2n/3 2
= 5n/3 2
O(n) = O(5n/3)
k
k+1
Nov2003
Nov2003
Nov2003
Nov2003
May2004
38
(n) c g(n) , n n0
f(n) c g(n) , n n0
b)
(n) = c g(n) , n n0
c)
(n) c g(n) , n n0
d)
3) The recurrence relation T(1) = 2 ; T(n) 3 T(n/4) + n equal to has the solution T(n)
equal to
a) O(n)
b) O(logn)
c) O(n3/4)
d) none
4) If T1 (N) = O(f(n)) and T2 (N) = O(g(N)) then T1 (N)+ T2 (N) =
a) O(f(N)
b) min (O(f(N), O(g(N)))
b) O(g(N)
d) max (O(f(N), O(g(N)))
5) The time complexity of binary search is
a) O(logn)
b) O(n)
c) O(nlogn)
d) O(n2)
6) The technique used by merge sort is
a) dynamic programming
b) Greedy method
c) Backtracking
d) Divide and Conquer
7) When the relation f(N) = (T(N)) then the T(N) is _______________ bound on f(N)
a) Upper
b) lower
c) center
d) none
8) Which of the following shows correct relationship
O(logn) < O(n)< O(nlogn) < O(2n) < O(n2)
a)
O(n) < O(logn) < O(nlogn) < O(2n) < O(n2)
b)
O(n)< O(logn) < O(nlogn) < O(n2 < O(2n))
c)
O(logn) < O(n)< O(nlogn) < O(n2) < O(2n)
d)
9) The strassens matrix multiplication requires only __________ multiplications,
___________ additions or subtractions.
a) 7, 18
b) 18, 7
c) 7, 10
d) 7, 8
10) The time complexity of merge sort is
a) O(n3)
b) O(n2 logn)
c) O(logn)
d) O(n logn)
11) In greedy method, a feasible solution that either maximiser or minimizes objective
function is called
a) basic solution
b) base solution
c) optimal solution d) none
12) The time complexity of strassens matrix multiplication is
a) O(n2)
b) O(n2.81)
c) O(n3.24)
d) O(n2.18)
13) Consider Knapsack problem n=3, m=20 (p1, p2, p3) = (25, 24, 15)
(w1, w2, w3) = (18, 15, 10)
a) , 1/3,
b) 2, 1, 3
c) 1,2,3
d) 3,2,1
14) The stack space required by quicksort is
a)
39
a) O(logn)
b) O(n)
c) O(n logn)
d) none
15) If there are 3 programs of length (l1, l2, l3) = (11,1,14). The optimal ordering of storage
on tape is
a) (3,1,2)
b) (2,1,3)
c) (1,2,3)
d) (3,2,1)
16) The Knapsack algorithm generates an optimal solution to given instance if
p1 / w1 p2 / w2 --------- pn / wn
a)
p1 / w1 p2 / w2 --------- pn / wn
b)
p1 / w1 p2 / w2 --------- pn wn
c)
none
d)
17) The Greedy method is applicable to
optimal storage on tapes
a)
knapsack problem
b)
minimum spanning tree
c)
all the above
d)
18) The worst case time complexity of quick sort is
a) O(n logn)
b) O(n3)
c) O(n2)
d) O(n)
19) The programs on the tape are stored such that mean retrieval time is
a) minimized
b) maximized
c) infinity
d) none
20) The Knapsack problem, optimal storage on tapes belongs to ________, ________
versions of greedy method respectively
subset, ordering paradigms
a)
subset, subset
b)
ordering, subset
c)
ordering, ordering
d)
Answers for the above:
1) b 2) a 3) a 4) d 5) a 6) d 7) b 8) d 9)a
10) d
11) c 12) b 13) b 14) b 15) b 16) a 17) d 18) c 19) a 20) a
1)
2)
3)
4)
5)
6)
7)
UNIT - II
DYNAMIC PROGRAMMING
Introduction:
Dynamic programming results when the solution is a result of a sequence of decisions. For
some problems it is not possible to make a sequence of stepwise decisions leading to optimal
solution. Therefore, we can try out all possible decisions sequences. This is the concept
behind dynamic programming
40
Principle of Optimality: An optimal sequence of decisions has the property that what
ever the initial state and decisions are, the remaining decisions must constitute an
optimal decision sequence with regard to the state resulting from the first decision.
The major difference between greedy method and dynamic programming is that
in the greedy method only one decision sequence is ever generated. Where as in
dynamic programming many decision sequences may be generated. Another important
feature of dynamic programming is that optimal solutions to sub problems are retained
so as to avoid recomputing their values.
.
Multistage Graph Problem:
A multistage graph G = (V, E) is a directed graph in which the vertices are
partitioned into two or more disjoint sets as shown in figure.
The multistage graph problem is to find the minimum cost path from a source to
target. Let c(i, j) be the cost of edge <i, j>. The cost of a path from source to target is the
sum of the costs of edges on the path.
FORWARD APPROACH:
Dynamic programming formulation for a K-stage graph problem is obtained by
first noticing k that every source to target path is the result of a sequence of k-2
decisions. The ith decision involves determining which vertex in level i+1 , 1<=i<=k-2 is
to be on the path.
L2
2
L1 9
3
1 7
3
2 4
5
L3
6
6 5
2
4
2
7
7
3
5
1
8 6
1
18
L4
9
1
10
4 L5
2 12
11 5
Let p(i,j) be a minimum cost path form vertex j in stage i to vertex target. Let cost(I,j) be
the cost of its path, then Cost(i, j) = min{c(j, l) + cost(i + 1, l)} where <j,l> is an
intermediate edge, where c(j, l) is cost of edge with vertices (j, l) and COST(i + 1, l) is the
cost to reach a node l in level i + 1, the
1)
COST(1, 1) = min{9 + COST(2, 2), 7 + COST(2, 3), 3 + COST(2, 4), 2 + COST(2, 5)} = 16
2)
COST(2, 2) = min{4 + COST(3, 6), 2 + COST(3, 7), 1 + COST(3, 8)} = 7
COST(2, 3) = min{2 + COST(3, 6), 2 + COST(3, 7)} = 12
COST(2, 4) = 11 + COST(3, 8)} = 18
COST(2, 5) = min{11 + COST(3, 7), 8 + COST(3, 8)} = 16
3)
COST(3, 6) = min{6 + COST(4, 9), 5 + COST(4, 10)} = 7
COST(3, 7) = min{4 + COST(4, 9), 3 + COST(4, 10)} = 5
COST(3, 8) = min{5 + COST(4, 10), 6 + COST(4, 11)} = 7
4)
COST(4, 9) = 4 + COST(5, 12) = 4
41
P(1) = 1; P(k) = n
For j = k 1 to 2
{
P(j) = D(p(j + 1)
}
}
Backward approach
For the previous example shown in forward approach,
1.BCOST(5, 12) = min {BCOST(4, 9) + 4, BACOST(4, 10) + 2, BACOST(4, 11) + 5}
2.BCOST(4, 9) = min {BCOST(3, 6) + 6, BACOST(3, 7) + 4}
BCOST(4, 10) = min {BCOST(3, 7) + 3, BACOST(3, 8) + 5, BACOST(3, 6) + 5}
BCOST(4, 11) = min {BCOST(3, 8) + 6}
3.BCOST(3, 6) = min {BCOST(2, 2) + 4, BACOST(2, 3) + 2},BCOST(3, 7)}
= min {BCOST(2, 2) + 2, BACOST(2, 3) + 7, BACOST(2, 5) + 11}
BCOST(3, 8) = min {BCOST(2, 2) + 1, BACOST(2, 4) + 11,BACOST(2, 5) + 8}
4. BCOST(2, 2) = min {BCOST(1, 1) + 9} = 9
Since BCOST(1, 1) = 0
BCOST(2, 3) = 7
BCOST(2, 4) = 3
BCOST(2, 5) = 2
Substituting these values in the above equations and going back
BCOST(3, 8) = 10
BCOST(3, 7) = 11
BCOST(3, 6) = 9
BCOST(4, 9) = 15
BCOST(4, 10) = 14
BCOST(4, 11) = 16
BCOST(5, 12) = 16
This algorithm also has same time complexity as that of forward approach
Reliability Design
Reliability design problem is concerned with the design of a system using several
devices connected in series with high reliability. Let r i be the reliability of device Di. Then
the reliability of entire system is ri. Even the reliability of individual device is good the
reliability of entire system may not be good.
Example: If n = 10, ri = 0.99
D1
D2
D3
ri = 0.904
Hence it is desirable to use multiple copies of devices
in parallel in each stage to improve the reliability.
D1
Dn
D2
D1
D3
Dn
.
D2
D1
Dn
If stage i contains mi copies of device Di
mi
then the probability that all mi have a malfunction is (1 ri)
Hence reliability of stage i is 1 (1 ri)mi
43
A(i, j) cost(i, j)
}
}
For k 1 to n
{
For I 1 to n
{
For J 1 to n
A(i, j) = min{A(i, j), (A(i, k) + A(k, j)}
}
}
Example: Let us consider the graph whose cost matrix is shown.
A(0)
A(1)
11
11
A(2)
7 1) + (1,
- 2)
(3,3 2) = (3,
In general.
10
15
20
15
10
15
13
12
10
Solution:
When |s| =
g(2, ) = c21 = 5
g(3, ) = c31 = 6
g(4, ) = c41 = 8
g(5, ) = c51 = 2
When |s| = I
g(2, {3}) = c23 + g(3, ) = 9 + 6 = 15
g(2, {4}) = c24 + g(4, ) = 10 + 8 = 18
g(2, {5}) = c25 + g(5, ) = 15 + 2 = 17
g(3, {2}) = 18, g(4{2}) = 13
g(3, {4}) = 20, g(4{3}) = 15
g(3, {5}) = 9, g(4{5}) = 6
g(5, {2}) = 10 = c52 + g(2, ) = 5 + 5
g(5, {3}) = 13 = c53 + g(3, ) = 7 + 6
g(5, {4}) = 18 = c54 + g(4, ) = 10 + 8
When |s| = 2
g(2, {3, 4}) = min(c23 + g(3, {4}) = 29 or c24 + g(4, {3}) = 25
= 25
g(2, {3, 5}) = min(c23 + g(3, {5}) = 18 or c25 + g(5, {3}) = 28
= 18
g(2, {4, 5}) = 16
g(3, {2, 5}) = 17
g(3, {2, 4}) = 25
g(3, {4, 5}) = 23
g(5, {2, 4}) = 23
g(4, {2, 3}) = 23
g(4, {3, 5}) = 17
g(5, {3, 4}) = 25
g(4, {2, 5}) = 14
g(5, {2, 3}) = 20
When |s| = 3
g(2, {3, 4, 5}) = min
g(3, {2, 4, 5}) = min
g(5, {2, 3, 4}) = min
9 + 23, 10 + 17, 15 + 25 = 27
13 + 16, 12 + 14, 7 + 23 = 26
5 + 25, 7 + 25, 10 + 23 = 30
46
8 + 18, 9 + 17, 4 + 20 = 24
When |s| = 4
g(1, {2, 3, 4, 5}) = min
minimum cost path
vertices 1 2 4
cost 10
10
4
31
6
#####
Time complexity
N be the number of g(i, s)s to be computed. For each of S there are
(n 1) choices for i.
Number of sets s if size k (excluding 1 and i) = ?????
?????
Therefore time complexity of the algorithm is (n22n) and space complexity O(n2n)
Optimal Binary search Tree:
A BST is a binary tree, either empty or each node is an identifier and
i)
All identifiers in LST are less than identifier at root node.
ii)
All identifiers in RST are greater than identifier at root node.
To find whether an identifier X is present are not.
1)
X is compared with root
2)
If X is less than identifier at root, then search continues in left sub tree.
3)
If X is greater than id at root, then search continues in right sub tree.
Procedure SEARCH(T, X, I)
{
iT
While i ! = 0
{
case
:X < IDENT(i): LCHILD(i)
:X < IDENT(i): return
:X < IDENT(i): i RCHILD(i)
endcase
}
}
Let us assume that given set of identifiers is {a 1, a2,..an} and let P(i) be the
probability of successful search of an identifier a i, and let Q(i) be the probability
unsuccessful search of ai.
Therefore p(i) + Q(i) = 1
To obtain a cost function for binary tree, it is useful to add a fictions node in place of
every empty node.
If n identifiers are there, there will be n internal nodes and n + 1 external nodes. Every
external node represents an unsuccessful search.
The identifiers not in binary tree are partitioned into n + 1 equivalence classes E i.
47
P(1 : 4) = (3, 3, 1, 1)
Q(0 : 4) = (2, 3, 1, 1, 1)
Initially w(, ) = Q(i), c(, ) = 0, R(, ) = 0
We have W(, j) = P(j) + Q(j) + w(, j 1)
W(0, 1) = P(1) + Q(1) + w(0, 0) = 8
C(0, 1) = W(0, 1) + min{c(0, 0) + c(1, 1)} = 8
R(0, 1) = 1
W(1, 2) = P(2) + Q(2) + w(1, 1) = 7
C(1, 2) = W(1, 2) + min{c(1, 1) + c(2, 2)} = 7
R (0, 2) = 2
W(i, j) = P(j) + Q(j) + w(Ij 1)
W(2, 3) = P(3) + Q(3) + w(2, 2) = 3
C(2, 3) = W(2, 3) + min{c(2, 2) + c(3, 3)} = 3
R(2, 3) = 3
W(I, j) = P(j) + Q(j) + w(Ij 1)
W(3, 4) = P(4) + Q(4) + w(3, 3) = 3
C(3, 4) = W(3, 4) + min{c(3, 3) + c(4, 4)} = 3
R(3, 4) = 4
W(0, 2) = P(2) + Q(2) + W(0, 1) = 3 + 13 + 8 = 12
C(0, 2) = w(0, 2) + min{C(0, 1) + C(2, 2) or c(0, 0) + c(1, 2)
= 14 + MIN{8 + 0) or {0 + 7} = 12 + 7 = 19
w(2, 4) = w(2, 4) + min{c(2, 3) + c(3, 4) = 6 or c(2, 2) + (3, 4) = 3} = 8
w(1, 3) = P(3) + P(3) + W(1, 2) = 2 + 7 = 9
C(1, 3) = W(1, 3) + min{C(1, 1) + C(2, 3) = 3 or C(1, 2) + C(3, 3) = 7} = 12
R(1, 3) = 2
W(0, 3) = P(3) + Q(3) + w(0, 2) = 14
C(0, 3) = w(0, 3) + min{c(0, 1) + c(2, 3)
C(0, 2) + c(3, 3)}
= 14 + min????
= 25
First solve all c(i, j) such that j I = 1 next compute c(I, j) such that j 1 = 2 and so on
during the computation record R(I, j) root of tree T ij. By observation we have w(I, j) = P(j)
+ Q(j) + W(i, j 1)
Row Col
0
1
2
3
4
0
2, 0, 0
3, 0, 0
1, 0, 0
1, 0, 0
1, 0, 0
j-i=1
8, 8, 1
7, 7, 2
3, 3, 3
3, 3, 4
2
12, 19, 1
9, 12, 2
5, 8, 3
3
16, 25, 2
11, 19, 2
4
16, 32, 2
C(0, 4) = 32 is minimum
The tree with root as 2 node is OBST.
Algorithm:
Procedure OBST(P, Q, n)
49
####
50
51
120
80
60
85
100
150
3
1
2
2
C
-
Print A
Print C
A
C
B
D
F
E
G
A
C
B
D
F
E
G
If (T1 = 0)
{
then call POSTORDER(LCHILD(T))
call POSTORDER9RCHILD(T))
call PRINT(T)
}
}
Example:
The post order traversal of the tree shown is
FHIGDEBCA
Time and space complexities of tree traversals:
If the time and space needed to visit a node is
(1) then t(n) = (n) and S(n) = 0(n)
#####
54