1 - Design and Analysis of Algorithms by Karamagi, Robert

Contents
Algorithm
Asymptotic Analysis
Recurrence
Substitution Method
Iteration Method
Recursion Tree Method
Master Method
Sorting Analysis
Bubble Sort
Selection Sort
Insertion Sort
Divide and Conquer
Max - Min Problem
Binary Search
Merge Sort
Tower of Hanoi
Sorting
Heap Sort
Quick Sort
Stable Sorting
Lower Bound Theory
Sorting in Linear Time
Linear Time Sorting
Counting Sort
Bucket Sort
Radix Sort
Hashing
Hash Tables
Methods of Hashing
Open Addressing Techniques
Hash Function
Binary Search Trees
Red Black Tree
Dynamic Programming
Fibonacci Sequence
Matrix Chain Multiplication
Longest Common Sequence (LCS)
0/1 Knapsack Problem
Dutch National Flag
Longest Palindrome Subsequence
Greedy Algorithm
Activity Selection Problem
Fractional Knapsack
Huffman Codes
Activity or Task Scheduling Problem
Travelling Sales Person Problem
Backtracking
Recursive Maze Algorithm
Hamiltonian Circuit Problems
Subset-Sum Problem
N-Queens Problem
Minimum Spanning Tree
Kruskal's Algorithm
Prim's Algorithm
Shortest Paths
Negative Weight Edges
Representing Shortest Path
Relaxation
Dijkstra's Algorithm
Bellman-Ford Algorithm
Single Source Shortest Path in Directed Acyclic Graphs
All-Pairs Shortest Paths
Floyd-Warshall Algorithm
Johnson's Algorithm
Flow
Network Flow Problems
Ford-Fulkerson Algorithm
Maximum Bipartite Matching
Sorting Networks
Comparison Networks
Bitonic Sorting Network
Merging Network
Complexity Theory
Polynomial Time Verification
NP-Completeness
Circuit Satisfiability
3CNF Satisfiability
Clique
Vertex Cover
Subset Cover
Approximate Algorithms
Vertex Cover
Traveling-salesman Problem
String Matching
Naive String Matching Algorithm
Rabin-Karp-Algorithm
String Matching with Finite Automata
Knuth-Morris-Pratt (KMP)Algorithm
Boyer-Moore Algorithm
Algorithm
A finite set of instruction that specifies a sequence of operation is to be
carried out in order to solve a specific problem or class of problems is called
an Algorithm.
Why study Algorithm?
As the speed of processor increases, performance is frequently said to be less
central than other software quality characteristics (e.g. security, extensibility,
reusability etc.). However, large problem sizes are commonplace in the area
of computational science, which makes performance a very important factor.
This is because longer computation time, to name a few mean slower results,
less through research and higher cost of computation (if buying CPU Hours
from an external party). The study of Algorithm, therefore, gives us a
language to express performance as a function of problem size.
Algorithm
An algorithm can be defined as a well-defined computational procedure that
takes some values, or the set of values, as an input and produces some value,
or the set of values, as an output. An algorithm is thus a sequence of
computational steps that transform the input into output.
It describes specific computational procedures for achieving the input-output
relationship.
For Example, We need to sort the sequence of number into ascending order.
Here is how we define the sorting problem.
Input:
A sequence of n number (a1, a2,......an)
Output:
A permutation (reordering) (a'1, a'2 .... a'n) of the input sequence such that (a'1
≤ a'2 ≤ ....≤ a'n)
An algorithm must have the following properties:
Correctness: It should produce the output according to the

requirement of the algorithm
Finiteness: Algorithm must complete after a finite number of
instructions have been executed.
An Absence of Ambiguity: Each step must be defined, having
only one interpretation.
Definition of Sequence: Each step must have a unique defined
preceding and succeeding step. The first step and the last step
must be noted.
Input/output: Number and classification of needed inputs and
results must be stated.
Feasibility: It must be feasible to execute each instruction.
Flexibility: It should also be possible to make changes in the
algorithm without putting so much effort on it.
Efficiency: Efficiency is always measured in terms of time and
space requires implementing the algorithm, so the algorithm
uses a little running time and memory space as possible within
the limits of acceptable development time.
Independent: An algorithm should focus on what are inputs,
outputs and how to derive output without knowing the language
it is defined. Therefore, we can say that the algorithm is
independent of language.
Need of Algorithm
1. To understand the basic idea of the problem.
2. To find an approach to solve the problem.
3. To improve the efficiency of existing techniques.
4. To understand the basic principles of designing the algorithms.
5. To compare the performance of the algorithm with respect to other
techniques.
6. It is the best method of description without describing the implementation
detail.
7. The Algorithm gives a clear description of requirements and goal of the
problem to the designer.
8. A good design can produce a good solution.
9. To understand the flow of the problem.
10. To measure the behavior (or performance) of the methods in all cases
(best cases, worst cases, average cases)
11. With the help of an algorithm, we can also identify the resources
(memory, input-output) cycles required by the algorithm.
12. With the help of algorithm, we convert art into a science.
13. To understand the principle of designing.
14. We can measure and analyze the complexity (time and space) of the
problems concerning input size without implementing and running it; it will
reduce the cost of design.
Algorithm vs Program
A finite set of instructions that specifies a sequence of operations to be
carried out to solve a specific problem of a class of problem is called an
algorithm.
On the other hand, the Program doesn't have to satisfy the finiteness
condition. For example, we can think of an operating system that continues in
a "wait" loop until more jobs are entered. Such a program doesn't terminate
unless the system crashes.
Given a Problem to solve, the design Phase produces an algorithm, and the
implementation phase then generates a program that expresses the designed
algorithm. So, the concrete expression of an algorithm in a particular
programming language is called a program.
Complexity of Algorithm
It is very convenient to classify algorithm based on the relative amount of
time or relative amount of space they required and specify the growth of
time/space requirement as a function of input size.
Time Complexity: Running time of a program as a function of the size of the
input.
Space Complexity: Some forms of analysis could be done based on how
much space an algorithm needs to complete its task. This space complexity
analysis was critical in the early days of computing when storage space on
the computer was limited. When considering this algorithm are divided into
those that need extra space to do their work and those that work in place.
But now a day's problem of space rarely occurs because space on the
computer (internal or external) is enough.
Broadly, we achieve the following types of analysis -
Worst-case: f (n) defined by the maximum number of steps

taken on any instance of size n.
Best-case: f (n) defined by the minimum number of steps taken
on any instance of size n.
Average case: f (n) defined by the average number of steps
taken on any instance of size n.
Algorithm Design Techniques

The following is a list of several popular design approaches:
1. Divide and Conquer Approach: It is a top-down approach. The algorithms
which follow the divide & conquer techniques involve three steps:
Divide the original problem into a set of subproblems.

Solve every subproblem individually, recursively.
Combine the solution of the subproblems (top level) into a
solution of the whole original problem.
2. Greedy Technique: Greedy method is used to solve the optimization

problem. An optimization problem is one in which we are given a set of input
values, which are required either to be maximized or minimized (known as
objective), i.e. some constraints or conditions.
Greedy Algorithm always makes the choice (greedy criteria) looks best at the
moment, to optimize a given objective.
The greedy algorithm doesn't always guarantee the optimal solution however
it generally produces a solution that is very close in value to the optimal.
3. Dynamic Programming: Dynamic Programming is a bottom-up approach
we solve all possible small problems and then combine them to obtain
solutions for bigger problems.
This is particularly helpful when the number of copying subproblems is
exponentially large. Dynamic Programming is frequently related
to Optimization Problems.
4. Branch and Bound: In Branch & Bound algorithm a given subproblem,
which cannot be bounded, has to be divided into at least two new restricted
subproblems. Branch and Bound algorithm are methods for global
optimization in non-convex problems. Branch and Bound algorithms can be
slow, however in the worst case they require effort that grows exponentially
with problem size, but in some cases we are lucky, and the method coverage
with much less effort.
5. Randomized Algorithms: A randomized algorithm is defined as an
algorithm that is allowed to access a source of independent, unbiased random
bits, and it is then allowed to use these random bits to influence its
computation.
6. Backtracking Algorithm: Backtracking Algorithm tries each possibility
until they find the right one. It is a depth-first search of the set of possible
solution. During the search, if an alternative doesn't work, then backtrack to
the choice point, the place which presented different alternatives, and tries the
next alternative.
7. Randomized Algorithm: A randomized algorithm uses a random number at
least once during the computation make a decision.
Example 1: In Quick Sort, using a random number to choose a pivot.
Example 2: Trying to factor a large number by choosing a random number as
possible divisors.
Loop Invariants
This is a justification technique. We use loop invariant that helps us to
understand why an algorithm is correct. To prove statement S about a loop is
correct, define S concerning series of smaller statement S0 S1....Sk where,
The initial claim so is true before the loop begins.
If Si-1 is true before iteration i begin, then one can show that Si will be true
after iteration i is over.
The final statement Sk implies the statement S that we wish to justify as being
true.
Asymptotic Analysis
Resources for an algorithm are usually expressed as a function regarding
input. Often this function is messy and complicated to work. To study
Function growth efficiently, we reduce the function down to the important
part.
Let f (n) = an2+bn+c
In this function, the n2 term dominates the function that is when n gets
sufficiently large.
Dominate terms are what we are interested in reducing a function, in this; we
ignore all constants and coefficient and look at the highest order term
concerning n.
Asymptotic Notation
The word Asymptotic means approaching a value or curve arbitrarily closely
(i.e., as some sort of limit is taken).
Asymptotic Analysis
It is a technique of representing limiting behavior. The methodology has the
applications across science. It can be used to analyze the performance of an
algorithm for some large data set.
In computer science in the analysis of algorithms, considering the
performance of algorithms when applied to very large input datasets.
The simplest example is a function ƒ (n) = n2+3n, the term 3n becomes
insignificant compared to n2 when n is very large. The function "ƒ (n) is said
to be asymptotically equivalent to n2 as n → ∞", and here is written
symbolically as ƒ (n) ~ n2.
Asymptotic notations are used to write fastest and slowest possible running
time for an algorithm. These are also referred to as 'best case' and 'worst case'
scenarios respectively.
"In asymptotic notations, we derive the complexity concerning the size of the
input. (Example in terms of n)"
"These notations are important because without expanding the cost of
running the algorithm, we can estimate the complexity of the algorithms."
Why is Asymptotic Notation Important?
1. They give simple characteristics of an algorithm's efficiency.
2. They allow the comparisons of the performances of various algorithms.
Asymptotic Notations
Asymptotic Notation is a way of comparing function that ignores constant
factors and small input sizes. Three notations are used to calculate the
running time complexity of an algorithm:
1. Big-oh notation: Big-oh is the formal method of expressing the upper
bound of an algorithm's running time. It is the measure of the longest amount
of time. The function f (n) = O (g(n)) [read as "f of n is big-oh of g of n"] if
and only if exist positive constant c and such that
f(n) ⩽ k.g (n)f(n) ⩽ k.g(n) for n>n0n>n0 in all case
Hence, function g (n) is an upper bound for function f (n), as g (n) grows
faster than f (n)
Example
1. 3n+2=O(n) as 3n+2≤4n for all n≥2
2. 3n+3=O(n) as 3n+3≤4n for all n≥3
Hence, the complexity of f(n) can be represented as O(g(n))
2. Omega() Notation: The function f(n) = Ω (g(n)) [read as "f of n is omega
of g of n"] if and only if there exists positive constant c and n0 such that F(n)
≥ k* g (n) for all n, n≥ n0
Example
F(n) =8n2+2n-3≥8n2-3
=7n2+(n2-3)≥7n2 (g(n))
Thus, k1=7
Hence, the complexity of f (n) can be represented as Ω (g (n))
3. Theta (θ): The function f (n) = θ (g (n)) [read as "f is the theta of g of n"] if
and only if there exists positive constant k1, k2 and k0 such that
k1 * g (n) ≤ f(n)≤ k2 g(n)for all n, n≥ n0

Example
3n+2= θ (n) as 3n+2≥3n and 3n+2≤ 4n, for n
k1=3,k2=4, and n0=2
Hence, the complexity of f (n) can be represented as θ (g(n)).
The Theta Notation is more precise than both the big-oh and Omega notation.
The function f (n) = θ (g (n)) if g(n) is both an upper and lower bound.
Analyzing Algorithm Control Structure
To analyze a programming code or algorithm, we must notice that each
instruction affects the overall performance of the algorithm and therefore,
each instruction must be analyzed separately to analyze overall performance.
However, there are some algorithm control structures which are present in
each programming code and have a specific asymptotic analysis.
Some Algorithm Control Structures are:
1. Sequencing
2. If-then-else
3. for loop
4. While loop
1. Sequencing
Suppose our algorithm consists of two parts A and B. A takes time tA and B
takes time tB for computation. The total computation "tA + tB" is according to
the sequence rule. According to maximum rule, this computation time is
(max (tA,tB)).
Example
Suppose tA =O (n) and tB = θ (n2).
Then, the total computation time can be calculated as
Computation Time = tA + tB
= (max (tA,tB)
= (max (O (n), θ (n2)) = θ (n2)
2. If-then-else
The total time computation is according to the condition rule-"if-then-else."
According to the maximum rule, this computation time is max (tA,tB).
Example
Suppose tA = O (n2) and tB = θ (n2)
Calculate the total computation time for the following:
Total Computation = (max (tA,tB))

= max (O (n2), θ (n2) = θ (n2)
3. For loop
The general format of for loop is:
For (initialization; condition; updation)
Statement(s);
Complexity of for loop
The outer loop executes N times. Every time the outer loop executes, the
inner loop executes M times. As a result, the statements in the inner loop
execute a total of N * M times. Thus, the total complexity for the two loops is
O (N2)
Consider the following loop:
for i ← 1 to n
{
P (i)
}
If the computation time ti for ( PI) various as a function of "i", then the total
computation time for the loop is given not by a multiplication but by a sum
i.e.
For i ← 1 to n
{
P (i)
}
Takes
If the algorithms consist of nested "for" loops, then the total computation
time is
For i ← 1 to n
{
For j ← 1 to n
{
P (ij)
}
}
Example
Consider the following "for" loop, Calculate the total computation time for
the following:
For i ← 2 to n-1
{
For j ← 3 to i
{
Sum ← Sum+A [i] [j]
}
}
Solution
The total Computation time is:
4. While loop
The Simple technique for analyzing the loop is to determine the function of
variable involved whose value decreases each time around. Secondly, for
terminating the loop, it is necessary that value must be a positive integer. By
keeping track of how many times the value of function decreases, one can
obtain the number of repetition of the loop. The other approach for analyzing
"while" loop is to treat them as recursive algorithms.
Algorithm
1. [Initialize] Set k: =1, LOC: =1 and MAX: = DATA [1]
2. Repeat steps 3 and 4 while K≤N
3. if MAX<DATA [k],then:
Set LOC: = K and MAX: = DATA [k]
4. Set k: = k+1
[End of step 2 loop]
5. Write: LOC, MAX
6. EXIT
Example
The running time of algorithm array Max of computing the maximum
element in an array of n integer is O (n).
Solution
array Max (A, n)
1. Current max ← A [0]
2. For i ← 1 to n-1
3. do if current max < A [i]
4. then current max ← A [i]
5. return current max.
The number of primitive operation t (n) executed by this algorithm is at least.
2 + 1 + n +4 (n-1) + 1=5n
2 + 1 + n + 6 (n-1) + 1=7n-2
The best case T(n) =5n occurs when A [0] is the maximum element. The
worst case T(n) = 7n-2 occurs when element are sorted in increasing order.
We may, therefore, apply the big-Oh definition with c=7 and n0=1 and
conclude the running time of this is O (n).
Recurrence
Recurrence
A recurrence is an equation or inequality that describes a function in terms of
its values on smaller inputs. To solve a Recurrence Relation means to obtain
a function defined on the natural numbers that satisfy the recurrence.
For Example, the Worst Case Running Time T(n) of the MERGE SORT
Procedures is described by the recurrence.
T (n) = θ (1) if n=1
2T + θ (n) if n>1
There are four methods for solving Recurrence:
1. Substitution Method
2. Iteration Method
3. Recursion Tree Method
4. Master Method
Substitution Method
The Substitution Method Consists of two main steps:
Guess the Solution.

Use the mathematical induction to find the boundary condition
and shows that the guess is correct.
Example
Solve the equation by Substitution Method.
T (n) = T +n
We have to show that it is asymptotically bound by O (log n).
Solution
For T (n) = O (log n)
We have to show that for some constant c
T (n) ≤c logn.
Put this in given Recurrence Equation.
T (n) ≤c log +1
≤c log + 1 = c logn-clog2 2+1

≤c logn for c≥1
Thus T (n) =O logn.
Example
Consider the Recurrence
T (n) = 2T + n n>1
Find an Asymptotic bound on T.
Solution
We guess the solution is O (n (logn)).Thus for constant 'c'.
T (n) ≤c n logn
Put this in given Recurrence Equation.
Now,
T (n) ≤2c log +n

≤cnlogn-cnlog2+n
=cn logn-n (clog2-1)
≤cn logn for (c≥1)
Thus T (n) = O (n logn).
Iteration Method
It means to expand the recurrence and express it as a summation of terms of n
and initial condition.
Example
T (n) = 1 if n=1
= 2T (n-1) if n>1
Solution
T (n) = 2T (n-1)
= 2[2T (n-2)] = 22T (n-2)
= 4[2T (n-3)] = 23T (n-3)
= 8[2T (n-4)] = 24T (n-4) (Eq.1)
Repeat the procedure for i times
T (n) = 2i T (n-i)
Put n-i=1 or i= n-1 in (Eq.1)
T (n) = 2n-1 T (1)
= 2n-1 .1 {T (1) =1 .....given}
= 2n-1
Example
T (n) = T (n-1) +1 and T (1) = θ (1).
Solution
T (n) = T (n-1) +1
= (T (n-2) +1) +1 = (T (n-3) +1) +1+1
= T (n-4) +4 = T (n-5) +1+4
= T (n-5) +5= T (n-k) + k
Where k = n-1
T (n-k) = T (1) = θ (1)
T (n) = θ (1) + (n-1) = 1+n-1=n= θ (n).
Recursion Tree Method

1. Recursion Tree Method is a pictorial representation of an iteration method
which is in the form of a tree where at each level nodes are expanded.
2. In general, we consider the second term in recurrence as root.
3. It is useful when the divide & Conquer algorithm is used.
4. It is sometimes difficult to come up with a good guess. In Recursion tree,
each root and child represents the cost of a single subproblem.
5. We sum the costs within each of the levels of the tree to obtain a set of pre-
level costs and then sum all pre-level costs to determine the total cost of all
levels of the recursion.
6. A Recursion Tree is best used to generate a good guess, which can be
verified by the Substitution Method.
Example
Consider T (n) = 2T + n2
We have to obtain the asymptotic bound using recursion tree method.
Solution
The Recursion tree for the above recurrence is
Example
Consider the following recurrence
T (n) = 4T +n
Obtain the asymptotic bound using recursion tree method.
Solution
The recursion trees for the above recurrence
Example
Consider the following recurrence
Obtain the asymptotic bound using recursion tree method.
Solution
The given Recurrence has the following recursion tree
When we add the values across the levels of the recursion trees, we get a
value of n for every level. The longest path from the root to leaf is
Master Method
The Master Method is used for solving the following types of recurrence
T (n) = a T + f (n) with a≥1 and b≥1 be constant & f(n) be a function and
can be interpreted as
Let T (n) is defined on non-negative integers by the recurrence.
T (n) = a T + f (n)
In the function to the analysis of a recursive algorithm, the constants and
function take on the following significance:
n is the size of the problem.

a is the number of subproblems in the recursion.
n/b is the size of each subproblem. (Here it is assumed that all
subproblems are essentially the same size.)
f (n) is the sum of the work done outside the recursive calls,
which includes the sum of dividing the problem and the sum of
combining the solutions to the subproblems.
It is not possible always bound the function according to the
requirement, so we make three cases which will tell us what
kind of bound we can apply on the function.
Master Theorem
It is possible to complete an asymptotic tight bound in these three cases:
Case1
If f (n) = for some constant ε >0, then it follows that:

T (n) = Θ
Example:
T (n) = 8 T apply master theorem on it.

Solution
Compare T (n) = 8 T with
T (n) = a T
a = 8, b=2, f (n) = 1000 n2, logba = log28 = 3
Put all the values in: f (n) =

1000 n2 = O (n3-ε )
If we choose ε=1, we get: 1000 n2 = O (n3-1) = O (n2)
Since this equation holds, the first case of the master theorem applies to the
given recurrence relation, thus resulting in the conclusion:
T (n) = Θ
Therefore: T (n) = Θ (n3)
Case 2
If it is true, for some constant k ≥ 0 that:
F (n) = Θ
then it follows that: T (n) = Θ
Example
T (n) = 2 , solve the recurrence by using the master method.
As compare the given problem with T (n) = a T

a = 2, b=2, k=0, f (n) = 10n, logba = log22 =1
Put all the values in f (n) =Θ , we will get

10n = Θ (n1) = Θ (n) which is true.
Therefore: T (n) = Θ
= Θ (n log n)
Case 3
If it is true f(n) = Ω for some constant ε >0 and it also true that: a
f for some constant c<1 for large value of n ,then :

T (n) = Θ((f (n))
Example: Solve the recurrence relation:
T (n) = 2
Solution
Compare the given problem with T (n) = a T

a= 2, b =2, f (n) = n2, logba = log22 =1
Put all the values in f (n) = Ω ..... (Eq. 1)

If we insert all the value in (Eq.1), we will get
n2 = Ω(n1+ε) put ε =1, then the equality will hold.
n2 = Ω(n1+1) = Ω(n2)
Now we will also check the second condition:
2
If we will choose c =1/2, it is true:
∀ n ≥1
So it follows: T (n) = Θ ((f (n))
T (n) = Θ(n2)
Sorting Analysis
Bubble Sort
Bubble Sort also known as Exchange Sort, is a simple sorting algorithm. It
works by repeatedly stepping throughout the list to be sorted, comparing two
items at a time and swapping them if they are in the wrong order. The pass
through the list is duplicated until no swaps are desired, which means the list
is sorted.
This is the easiest method among all sorting algorithms.
BUBBLE SORT (A)
1. for i ← 1 to length [A]
2. for k ← length [A] down to i+1
3. if A[k] <A[k-1]
4. exchange (A[k], A [k-1])
Analysis
Input: n elements are given.
Output: the number of comparisons required to make sorting.
Logic: If we have n elements in bubble sort, then n-1 passes are required to
find a sorted array.
In pass 1: n-1 comparisons are required
............................................................................
...............................................................................
In pass n-1: 1 comparisons is required
Total comparisons: T (n) = (n-1) + (n-2) +...........+ 1
=
= o (n2)
Therefore complexity is of order n2
Example
Unsorted List: A = {7, 2, 1, 4, 5, 9, 6}
i.e., A [] =
7 2 1 4 3 6 9
Here length [A] =7
i=1 to 7 and j=7 to 2
i=1, j=7
A [7] =6 and A [6] =9. So A [7] <A [6]
Now Exchange (A [7], A [6])
Now, i=1, j=6 then A [6] =6
A [5] = 3 and A [5] < A [6]
Now, i=1, j=5 then A [5] = 3
A [4] = 4 and A [5] < A [4]
So, exchange (A [5], A [4])
And A [] =
7 2 1 3 4 6 9
Now, i=1, j=4 then A [4] =3

A [3] = 1 and A [4] > A [3]
Now, i=1, j=3 then A [3] = 1
A [2] = 2 and A [3] < A [2]
Then A [] =
7 1 2 3 4 6 9
Now, i=2, j-2 then A [2] =1
A [1] =7 and A [2] <A [1]
Then A [] =
1 7 2 3 4 6 9
Now, i=1, j=7 then A [7] =9

A [6] =6 and A [7] > A [6], No Exchange
Similarly, i=2, j=6, 5, 4. No change.
Then i = 2, j=3
A [3] = 2
A [2] = 7 and A [3] < A [2]
And A [] =
1 2 7 3 4 6 9
Now, i=3, j=5, 6, 7 No change
Then i=3, j=4
A [4] = 3
A [3] = 7 and A [4] < A [3]
Then A [] =
1 2 3 7 4 6 9
Now, i=4, j= 6, 7 No change

Now, i=4, j=5 then A [5] = 4
A [4] = 7 and A [5] < A [4]
So exchange (A [5], A [4])
And A [] =
1 2 3 4 7 6 9
Now, i=5, j= 6, 7 No change
Now, i=5, j=6 then A [6] = 6
A [5] = 7 and A [6] < A [5]
So exchange (A [6], A [5])
And A [] =
1 2 3 4 6 7 9
is the sorted array.
Selection Sort
The selection sort enhances the bubble sort by making only a single swap for
each pass through the rundown. Indorder to do this, a selection sort searches
for the biggest value as it makes a pass and, after finishing the pass, places it
in the best possible area. Similarly as with a bubble sort, after the first pass,
the biggest item is in the right place. After the second pass, the following
biggest is set up. This procedure proceeds and requires n-1 goes to sort n
item, since the last item must be set up after the (n-1) st pass.
Algorithm
SELECTION SORT (A)
1. k ← length [A]
2. for j ←1 to n-1
3. smallest ← j
4. for I ← j + 1 to k
5. if A [i] < A [ smallest]
6. then smallest ← i
7. exchange (A [j], A [smallest])
Analysis
1. Input: n elements are given.

2. Output: the number of comparisons required to make sorting.
3. Logic: If we have n elements in selection sort, then n-1 passes
are required to find a sorted array.

............................................................................
...............................................................................
In pass n-1: 1 comparison is required
Total comparisons: T (n) = (n-1) + (n-2) + (n-3) +........+ 1
=
= o (n2)
Example
Sort the following array using selection sort: A [] = (7, 4, 3, 6, 5).
A [] =
7 4 3 6 5
1 Iteration:
Smallest =7
4 < 7, smallest = 4
3 < 4, smallest = 3
6 > 3, smallest = 3
5 > 3, smallest = 3
Swap 7 and 3
3 4 7 6 5
2nd iteration:
Smallest = 4
4 < 7, smallest = 4
4 < 6, smallest = 4
4 < 5, smallest = 4
No Swap
3 4 7 6 5
3rd iteration:
Smallest = 7
6 < 7, smallest = 6
5 < 7, smallest = 5
Swap 7 and 5
3 4 5 6 7
4th iteration:
Smallest = 6
6< 7, smallest = 6
No Swap
3 4 5 6 7
Finally, the sorted list is:

3 4 5 6 7
Insertion Sort
It is a very simple method to sort the number in an increasing or decreasing
order.
It has various advantages:
1. It is simple to implement.
2. It is efficient on small datasets.
3. It is stable (does not change the relative order of elements with
equal keys)
4. It is in-place (only requires a constant amount O (1) of extra
memory space).
5. It is an online algorithm, in that it can sort a list as it receives it.
Algorithm
INSERTION SORT (A)
1. For k ← 2 to length [A]
2. Do key ← A[k]
3. i=k-1
4. while i>0 and A[i]>key
5. do A[i+1] ← A[i]
6. i=i-1
7. A[i+1] ← key
Analysis
Input: n elements are given.
Output: the number of comparisons required to make sorting.
Logic: If we have n elements in insertion sort, then n-1 passes are required to
find a sorted array.
In pass 1: no comparison is required
In pass 2: 1 comparison is required
In pass 3: 2 comparisons are required
............................................................................
...............................................................................
In pass n: n-1 comparisons are required
Total comparisons: T (n) = 1+2+3+...........+ n-1
=
= o (n2)
Example
Illustrate the operation of INSERTION SORT on the array A = (4, 15, 7, 18,
and 16).
Solution
A [] =
4 15 7 18 16
For j=2 to 5
J=2, key=A [2]
Key=15
I=2-1=1, i=1
While i>0 and A [1]>15
A Condition false, so no change
Now=3, key=A [3] =7
I=3-1=2
I=2, key=7
While i>0 and A [2]>key
Condition is true
So A [2+1] ← A [2]
A [3] ← A [2]
i.e.
4 15 18 16
and i=2-1=1, i=1

while i>0 and A [1]>key
A Condition false. So no change
Then A [1+1] ← key
A [2] ← 7
That is
4 7 15 18 16
For j=4
Key=A [4]
Key = 18, i=3
Now, while 3>0 and A [3]>18
The Condition is false, No change.
Similarly, j=5
Key=A [5]
So key=16, i=4
Now while 4>0 and A [4]>16
Condition is true
So A [5] =16 and i=4-1=3
Now while 3>0 and A [3]>16
Condition is false
So A [3+1] =A [4] =16
And the sorted array is:
4 7 15 16 18
Divide and Conquer

Divide and Conquer is an algorithmic pattern. In algorithmic methods, the
design is to take a dispute on a huge input, break the input into minor pieces,
decide the problem on each of the small pieces, and then merge the piecewise
solutions into a global solution. This mechanism of solving the problem is
called the Divide & Conquer Strategy.
Divide and Conquer algorithm consists of a dispute using the following three
steps.
1. Divide the original problem into a set of subproblems.

2. Conquer: Solve every subproblem individually, recursively.
3. Combine: Put together the solutions of the subproblems to get
the solution to the whole problem.
Generally, we can follow the divide-and-conquer approach in a three-step

process.
Examples
The specific computer algorithms are based on the Divide & Conquer
approach:
1. Maximum and Minimum Problem

2. Binary Search
3. Sorting (merge sort, quick sort)
4. Tower of Hanoi.
Fundamental of Divide & Conquer Strategy
There are two fundamental of Divide & Conquer Strategy:
1. Relational Formula
2. Stopping Condition
1. Relational Formula: It is the formula that we generate from the given

technique. After generation of Formula we apply D&C Strategy, i.e. we break
the problem recursively & solve the broken subproblems.
2. Stopping Condition: When we break the problem using Divide & Conquer
Strategy, then we need to know that for how much time, we need to apply
divide & Conquer. So the condition where the need to stop our recursion
steps of D&C is called as Stopping Condition.
Max - Min Problem

Problem
Analyze the algorithm to find the maximum and minimum element from an
array.
Algorithm
Max ?Min Element (a [])
Max: a [i]
Min: a [i]
For i= 2 to n do
If a[i]> max then
max = a[i]
if a[i] < min then
min: a[i]
return (max, min)
Analysis
Method 1: if we apply the general approach to the array of size n, the number
of comparisons required are 2n-2.
Method 2: In another approach, we will divide the problem into sub-problems
and find the max and min of each group, now max. Of each group will
compare with the only max of another group and min with min.
Let n = is the size of items in an array
Let T (n) = time required to apply the algorithm on an array of size n. Here
we divide the terms as T(n/2).
2 here tends to the comparison of the minimum with minimum and maximum
with maximum as in above example.
T (n) = 2 T → Eq (i)
T (2) = 1, time required to compare two elements/items. (Time is measured in
units of the number of comparisons)
→ Eq (ii)
Put eq (ii) in eq (i)
Similarly, apply the same procedure recursively on each subproblem or

anatomy
{Use recursion means, we will use some stopping condition to stop the
algorithm}
Recursion will stop, when → (Eq. 4)

Put the eqn.4 into equation3.
Number of comparisons requires applying the divide and conquering
algorithm on n elements/items =
Number of comparisons requires applying general approach on n elements =
(n-1) + (n-1) = 2n-2
From this example, we can analyze, that how to reduce the number of
comparisons by using this technique.
Analysis
Suppose we have the array of size 8 elements.
Method 1: requires (2n-2), (2x8)-2=14 comparisons
Method 2: requires
It is evident; we can reduce the number of comparisons (complexity) by
using a proper technique.
Binary Search
1. In Binary Search technique, we search an element in a sorted array by
recursively dividing the interval in half.
2. Firstly, we take the whole array as an interval.
3. If the Pivot Element (the item to be searched) is less than the item in the
middle of the interval, We discard the second half of the list and recursively
repeat the process for the first half of the list by calculating the new middle
and last element.
4. If the Pivot Element (the item to be searched) is greater than the item in the
middle of the interval, we discard the first half of the list and work
recursively on the second half by calculating the new beginning and middle
element.
5. Repeatedly, check until the value is found or interval is empty.
Analysis
1. Input: an array A of size n, already sorted in the ascending or

descending order.
2. Output: analyze to search an element item in the sorted array of
size n.
3. Logic: Let T (n) = number of comparisons of an item with n
elements in a sorted array.
Set BEG = 1 and END = n
Find mid =
Compare the search item with the mid item.
Case 1: item = A[mid], then LOC = mid, but it the best case and T (n) = 1
Case 2: item ≠A [mid], then we will split the array into two equal parts of
size .
And again find the midpoint of the half-sorted array and compare with search
element.
Repeat the same process until a search element is found.
T (n) = ...... (Equation 1)

{Time to compare the search element with mid element, then with half of the
selected half part of array}
At least there will be only one term left that's why that term will compare out,
and only one comparison be done that's why
Is the last term of the equation and it will be equal to 1

Merge Sort
It closely follows the divide & Conquers paradigm.
Conceptually, it works as follows:
1. Divide: Divide the unsorted list into two sublists of about half
the size.
2. Conquer: Sort each of the two sublists recursively until we have
list sizes of length 1, in which case the list items are returned.
3. Combine: Join the two sorted Sub lists back into one sorted list.
The Main purpose is to sort the unsorted list in nondecreasing order.
Algorithm-Merge Sort
1. If p<r
2. Then q ← ( p+ r)/2
3. MERGE-SORT (A, p, q)
4. MERGE-SORT ( A, q+1,r)
5. MERGE ( A, p, q ,r)
The following figure illustrates the dividing (splitting) procedure.
Functions: MERGE (A, p, q, r)

1. n 1 = q-p+1
2. n 2= r-q
3. create arrays [1.....n 1 + 1] and R [ 1.....n 2 +1 ]
4. for i ← 1 to n 1
5. do [i] ← A [ p+ i-1]
6. for j ← 1 to n2
7. do R[j] ← A[ q + j]
8. L [n 1+ 1] ← ∞
9. R[n 2+ 1] ← ∞
10. I ← 1
11. J ← 1
12. For k ← p to r
13. Do if L [i] ≤ R[j]
14. then A[k] ← L[ i]
15. i ← i +1
16. else A[k] ← R[j]
17. j ← j+1
In this method, we split the given list into two halves. Then recursively
analyzing merge sort and dividing. We get many sorted lists.
At last, we combine the sorted lists.
Analysis of Merge Sort
Let T (n) be the total time taken in Merge Sort
Sorting two halves will be taken at the most 2T( ) time

When we merge the sorted lists, we have a total n-1 comparison because the
last element which will be left will just need to be copied down in the
combined list and there will be no comparison.
So, the relational formula becomes
But we ignore '-1' because the element will take some time to be copied in
merge lists.
So T (n) = 2T + n.........equation 1
Note: Stopping Condition T (1) =0 because at last there will be only 1
element left which need to be copied and there will be no comparison.
Put 2 equation in 1 equation
Putting 4 equation in 3 equation
From Stopping Condition:

Apply log both sides:
log n=log2i
logn= i log2
=i
log2n=i
From 6 equation
Tower of Hanoi
1. It is a classic problem where you try to move all the disks from one peg to
another peg using only three pegs.
2. Initially, all of the disks are stacked on top of each other with larger disks
under the smaller disks.
3. You may move the disks to any of three pegs as you attempt to relocate all
of the disks, but you cannot place the larger disks over smaller disks and only
one disk can be transferred at a time.
This problem can be easily solved by Divide & Conquer algorithm
In the above 7 step all the disks from peg A will be transferred to C given
Condition
1. Only one disk will be shifted at a time.
2. Smaller disk can be placed on larger disk.
Let T (n) be the total time taken to move n disks from peg A to peg C
Moving n-1 disks from the first peg to the second peg. This can be done in T
(n-1) steps.
1. Moving larger disks from the first peg to the third peg will
require first one step.
2. Recursively moving n-1 disks from the second peg to the third
peg will require again T (n-1) step.
So, total time taken T (n) = T (n-1)+1+ T(n-1)

Relation formula for Tower of Hanoi is:
We get,
It is a Geometric Progression Series with common ratio, r=2

First term, a=1(20)
B equation is the required complexity of technique tower of Hanoi when we
have to move n disks from one peg to another.
T (3) = 23- 1
= 8 - 1 = 7 Ans
[As in concept we have proved that there will be 7 steps now proved by
general equation]
Program of Tower of Hanoi
#include<stdio.h>
void towers(int, char, char, char);
int main()
{
int num;
printf ("Enter the number of disks : ");
scanf ("%d", &num);
printf ("The sequence of moves involved in the Tower of Hanoi are
:\n");
towers (num, 'A', 'C', 'B');
return 0;
}
void towers( int num, char from peg, char topeg, char auxpeg)
{
if (num == 1)
{
printf ("\n Move disk 1 from peg %c to peg %c", from peg, topeg);
return;
}
Towers (num - 1, from peg, auxpeg, topeg);
Printf ("\n Move disk %d from peg %c to peg %c", num, from peg,
topeg);
Towers (num - 1, auxpeg, topeg, from peg);
}
Output
Enter the number of disks: 3
The sequence of moves involved in the Tower of Hanoi is
Move disk 1 from peg A to peg C
Move disk 2 from peg A to peg B
Move disk 1 from peg C to peg B
Move disk 3 from peg A to peg C
Move disk 1 from peg B to peg A
Move disk 2 from peg B to peg C
Move disk 1 from peg A to peg
Sorting
Heap Sort
Binary Heap
Binary Heap is an array object can be viewed as Complete Binary Tree. Each
node of the Binary Tree corresponds to an element in an array.
1. Length [A],number of elements in array

2. Heap-Size[A], number of elements in a heap stored within
array A.
The root of tree A [1] and gives index 'i' of a node that indices of its parents,
left child, and the right child can be computed.
PARENT (i)
Return floor (i/2)
LEFT (i)
Return 2i
RIGHT (i)
Return 2i+1
Representation of an array of the above figure is given below:

The index of 20 is 1
To find the index of the left child, we calculate 1*2=2
This takes us (correctly) to the 14.
Now, we go right, so we calculate 2*2+1=5
This takes us (again, correctly) to the 6.
Now, 4's index is 7, we want to go to the parent, so we calculate 7/2 =3 which
takes us to the 17.
Heap Property
A binary heap can be classified as Max Heap or Min Heap.
1. Max Heap: In a Binary Heap, for every node I other than the root, the
value of the node is greater than or equal to the value of its highest child
A [PARENT (i) ≥A[i]
Thus, the highest element in a heap is stored at the root. Following is an
example of MAX-HEAP
2. MIN-HEAP: In MIN-HEAP, the value of the node is lesser than or equal
to the value of its lowest child.
A [PARENT (i) ≤A[i]
Heapify Method
1. Maintaining the Heap Property: Heapify is a procedure for manipulating
heap Data Structure. It is given an array A and index I into the array. The
subtree rooted at the children of A [i] are heap but node A [i] itself may
probably violate the heap property i.e. A [i] < A [2i] or A [2i+1]. The
procedure 'Heapify' manipulates the tree rooted as A [i] so it becomes a heap.
MAX-HEAPIFY (A, i)
1. l ← left [i]
2. r ← right [i]
3. if l≤ heap-size [A] and A[l] > A [i]
4. then largest ← l
5. Else largest ← i
6. If r≤ heap-size [A] and A [r] > A[largest]
7. Then largest ← r
8. If largest ≠ i
9. Then exchange A [i] A [largest]
10. MAX-HEAPIFY (A, largest)
Analysis
The maximum levels an element could move up are Θ (log n) levels. At each
level, we do simple comparison which O (1). The total time for heapify is
thus O (log n).
Building a Heap
BUILDHEAP (array A, int n)
1 for i ← n/2 down to 1
2 do
3 HEAPIFY (A, i, n)
Heap-Sort Algorithm
HEAP-SORT (A)
1. BUILD-MAX-HEAP (A)
2. For I ← length[A] down to Z
3. Do exchange A [1] ←→ A [i]
4. Heap-size [A] ← heap-size [A]-1
5. MAX-HEAPIFY (A,1)
Analysis: Build max-heap takes O (n) running time. The Heap Sort algorithm
makes a call to 'Build Max-Heap' which we take O (n) time & each of the (n-
1) calls to Max-heap to fix up a new heap. We know 'Max-Heapify' takes
time O (log n)
The total running time of Heap-Sort is O (n log n).
Example
Illustrate the Operation of BUILD-MAX-HEAP on the array.
A = (5, 3, 17, 10, 84, 19, 6, 22, 9)
Solution
Originally:
Heap-Size (A) =9, so first we call MAX-HEAPIFY (A, 4)
And I = 4.5= 4 to 1
After MAX-HEAPIFY (A, 4) and i=4

L ← 8, r ← 9
l≤ heap-size[A] and A [l] >A [i]
8 ≤9 and 22>10
Then Largest ← 8
If r≤ heap-size [A] and A [r] > A [largest]
9≤9 and 9>22
If largest (8) ≠4
Then exchange A [4] ←→ A [8]
MAX-HEAPIFY (A, 8)
l← 6, r ← 7
6≤ 9 and 19>17
Largest ← 6
7≤9 and 6>19
If largest (6) ≠3
Then Exchange A [3] ←→ A [6]
MAX-HEAPIFY (A, 6)
l ← 4, r ← 5
4≤9 and 22>3
Largest ← 4
5≤9 and 84>22
Largest ← 5
If largest (4) ≠2
MAX-HEAPIFY (A, 5)
l ← 2, r ← 3
2≤9 and 84>5
Largest ← 2
3≤9 and 19<84
If largest (2) ≠1
MAX-HEAPIFY (A, 2)
Priority Queue
As with heaps, priority queues appear in two forms: max-priority queue and
min-priority queue.
A priority queue is a data structure for maintaining a set S of elements, each
with a combined value called a key. A max-priority queue guides the
following operations:
Insert(S, x): inserts the element x into the set S, which is proportionate to the
operation S=S ∪ [x].
Maximum(S) returns the element of S with the highest key.
Extract-Max(S) removes and returns the element of S with the highest key.
Increase-Key(S, x, k) increases the value of element x's key to the new value
k, which is considered to be at least as large as x's current key value.
Let us discuss how to implement the operations of a max-priority queue. The
procedure HEAP-MAXIMUM consider the MAXIMUM operation in θ (1)
time.
Heap-Maximum(A)
1. return A [1]
The procedure HEAP-EXTRACT-MAX implements the EXTRACT-MAX
operation. It is similar to the for loop of Heap-Sort procedure.
HEAP-EXTRACT-MAX (A)
1 if A. heap-size < 1
2 error "heap underflow"
3 max ← A [1]
4 A [1] ← A [heap-size [A]]
5 heap-size [A] ← heap-size [A]-1
6 MAX-HEAPIFY (A, 1)
7 return max
The procedure HEAP-INCREASE-KEY implements the INCREASE-KEY
operation. An index i into the array identify the priority-queue element whose
key we wish to increase.
HEAP-INCREASE-KEY(A, i, key)
1 if key < A[i]
2 errors "new key is smaller than current key"
3 A[i] = key
4 while i>1 and A [Parent (i)] < A[i]
5 exchange A [i] with A [Parent (i)]
6 i =Parent [i]
The running time of HEAP-INCREASE-KEY on an n-element heap is O (log
n) since the path traced from the node updated in line 3 to the root has length
O (log n).
The procedure MAX-HEAP-INSERT implements the INSERT operation. It
takes as an input the key of the new item to be inserted into max-heap A. The
procedure first expands the max-heap by calculating to the tree a new leaf
whose key is - ∞. Then it calls HEAP-INCREASE-KEY to set the key of this
new node to its right value and maintain the max-heap property.
MAX-HEAP-INSERT (A, key)
1 A. heap-size = A. heap-size + 1
2 A [A. heap-size] = - ∞
3 HEAP-INCREASE-KEY (A, A. heap-size, key)
The running time of MAX-HEAP-INSERT on an n-element heap is O (log
n).
Example
Illustrate the operation of HEAP-EXTRACT-MAX on the heap
A= (15,13,9,5,12,8,7,4,0,6,2,1)
In this figure, that max-heap with a node whose index is 'i' heavily shaded
In this Figure, this node has its key increased to 15.

After one iteration of the while loop of lines 4-6, the node and its parent have
exchanged keys, and the index i moves up to the parent.
The max-heap after one more iteration of the while loops, the A [PARENT (i)
≥A (i)] the max-heap property now holds and the procedure terminates.
Heap-Delete
Heap-DELETE (A, i) is the procedure, which deletes the item in node 'i' from
heap A, HEAP-DELETE runs in O (log n) time for n-element max heap.
HEAP-DELETE (A, i)
1. A [i] ← A [heap-size [A]]
2. Heap-size [A] ← heap-size [A]-1
3. MAX-HEAPIFY (A, i)
Quick Sort
It is an algorithm of Divide & Conquer type.
Divide: Rearrange the elements and split arrays into two sub-
arrays and an element in between search that each element in
left sub array is less than or equal to the average element and
each element in the right sub- array is larger than the middle
element.
Conquer: Recursively, sort two sub arrays.
Combine: Combine the already sorted array.
Algorithm
QUICKSORT (array A, int m, int n)
1 if (n > m)
2 then
3 i ← a random index from [m,n]
4 swap A [i] with A[m]
5 o ← PARTITION (A, m, n)
6 QUICKSORT (A, m, o - 1)
7 QUICKSORT (A, o + 1, n)
Partition Algorithm
Partition algorithm rearranges the sub arrays in a place.
PARTITION (array A, int m, int n)
1 x ← A[m]
2o←m
3 for p ← m + 1 to n
4 do if (A[p] < x)
5 then o ← o + 1
6 swap A[o] with A[p]
7 swap A[m] with A[o]
8 return o
Example of Quick Sort
44 33 11 55 77 90 40 60 99 22 88
Let 44 be the Pivot element and scanning done from right to left
Comparing 44 to the right-side elements, and if right-side elements are
smaller than 44, then swap it. As 22 is smaller than 44 so swap them.
22 33 11 55 77 90 40 60 99 44 88
Now comparing 44 to the left side element and the element must
be greater than 44 then swap them. As 55 are greater than 44 so swap them.
22 33 11 44 77 90 40 60 99 55 88
Recursively, repeating steps 1 & steps 2 until we get two lists one left from
pivot element 44 & one right from pivot element.
22 33 11 40 77 90 44 60 99 55 88
Swap with 77:
22 33 11 40 44 90 77 60 99 55 88
Now, the element on the right side and left side are greater than and smaller
than 44 respectively.
Now we get two sorted lists:
And these sublists are sorted under the same process as above done.
These two sorted sublists side by side.
Merging Sublists:
SORTED LISTS
Worst Case Analysis
It is the case when items are already in sorted form and we try to sort them
again. This will takes lots of time and space.
Equation
T (n) =T(1)+T(n-1)+n
T (1) is time taken by pivot element.
T (n-1) is time taken by remaining element except for pivot element.
N: the number of comparisons required to identify the exact position of itself
(every element)
If we compare first element pivot with other, then there will be 5
comparisons.
It means there will be n comparisons if there are n items.
Relational Formula for Worst Case

Note: for making T (n-4) as T (1) we will put (n-1) in place of '4' and if
We put (n-1) in place of 4 then we have to put (n-2) in place of 3 and (n-3)
In place of 2 and so on.

T(n)=(n-1) T(1) + T(n-(n-1))+(n-(n-2))+(n-(n-3))+(n-(n-4))+n
T (n) = (n-1) T (1) + T (1) + 2 + 3 + 4+............n
T (n) = (n-1) T (1) +T (1) +2+3+4+...........+n+1-1
[Adding 1 and subtracting 1 for making AP series]
T (n) = (n-1) T (1) +T (1) +1+2+3+4+........ + n-1
T (n) = (n-1) T (1) +T (1) + -1

Stopping Condition
T (1) =0
Because at last there is only one element left and no comparison is required.
T (n) = (n-1) (0) +0+ -1
Worst Case Complexity of Quick Sort is T (n) =O (n2)

Randomized Quick Sort [Average Case]
Generally, we assume the first element of the list as the pivot element. In an
average Case, the number of chances to get a pivot element is equal to the
number of items.
Let total time taken =T (n)
For eg: In a given list
p 1, p 2, p 3, p 4............pn
If p 1 is the pivot list then we have 2 lists.
I.e. T (0) and T (n-1)
If p2 is the pivot list then we have 2 lists.
I.e. T (1) and T (n-2)
p 1, p 2, p 3, p 4............pn
If p3 is the pivot list then we have 2 lists.
I.e. T (2) and T (n-3)
p 1, p 2, p 3, p 4............p n
So in general if we take the Kth element to be the pivot element.
Then,
Pivot element will do n comparison and we are doing average case so,
So Relational Formula for Randomized Quick Sort is:

= n+1 + (T(0)+T(1)+T(2)+...T(n-1)+T(n-2)+T(n-3)+...T(0))
= n+1 + x2 (T(0)+T(1)+T(2)+...T(n-2)+T(n-1))
n T (n) = n (n+1) +2 (T(0)+T(1)+T(2)+...T(n-1)........eq 1

Put n=n-1 in eq 1
(n -1) T (n-1) = (n-1) n+2 (T(0)+T(1)+T(2)+...T(n-2)......eq2
From eq1 and eq 2
n T (n) - (n-1) T (n-1)= n(n+1)-n(n-1)+2 (T(0)+T(1)+T(2)+?T(n-2)+T(n-
1))-2(T(0)+T(1)+T(2)+...T(n-2))
n T(n)- (n-1) T(n-1)= n[n+1-n+1]+2T(n-1)
n T(n)=[2+(n-1)]T(n-1)+2n
n T(n)= n+1 T(n-1)+2n
Put n=n-1 in eq 3
Put 4 eq in 3 eq
Put n=n-2 in eq 3
Put 6 eq in 5 eq
Put n=n-3 in eq 3
Put 8 eq in 7 eq
From 3eq, 5eq, 7eq, 9 eq we get
From 10 eq
Multiply and divide the last term by 2
Is the average case complexity of quick sort for sorting n elements.

Quick Sort [Best Case]
In any sorting, best case is the only case in which we don't make any
comparison between elements that is only done when we have only one
element to sort.
Stable Sorting
A sorting algorithm is said to be stable if two objects with equal keys appear
in the same order in sorted output as they appear in the input unsorted array.
Some Sorting Algorithm is stable by nature like Insertion Sort, Merge Sort
and Bubble Sort etc.
Sorting Algorithm is not stable like Quick Sort, Heap Sort etc.
Another Definition of Stable Sorting
A Stable Sort is one which preserves the original order of input set, where the
comparison algorithm does not distinguish between two or more items. A
Stable Sort will guarantee that the original order of data having the same rank
is preserved in the output.
In Place Sorting Algorithm:
1. An In-Place Sorting Algorithm directly modifies the list that is
received as input instead of creating a new list that is then
modified.
2. In this Sorting, a small amount of extra space it uses to
manipulate the input set. In other Words, the output is placed in
the correct position while the algorithm is still executing, which
means that the input will be overwritten by the desired output
on run-time.
3. In-Place, Sorting Algorithm updates input only through
replacement or swapping of elements.
4. An algorithm which is not in-place is sometimes called not-in-
Place or out of Place.
5. An Algorithm can only have a constant amount of extra space,
counting everything including function call and Pointers,
Usually; this space is O (log n).
Note
1. Bubble sort, insertion sort, and selection sort are in-place

sorting algorithms. Because only swapping of the element in
the input array is required.
2. Bubble sort and insertion sort can be applying as stable
algorithms but selection sort cannot (without significant
modifications).
3. Merge sort is a stable algorithm but not an in-place algorithm.
It requires extra array storage.
4. Quicksort is not stable but is an in-place algorithm.
5. Heap sort is an in-place algorithm but is not stable.
Lower Bound Theory
Lower Bound Theory Concept is based upon the calculation of minimum
time that is required to execute an algorithm is known as a lower bound
theory or Base Bound Theory.
Lower Bound Theory uses a number of methods/techniques to find out the
lower bound.
Concept/Aim: The main aim is to calculate a minimum number of
comparisons required to execute an algorithm.
Techniques
The techniques which are used by lower Bound Theory are:
1. Comparisons Trees
2. Oracle and adversary argument
3. State Space Method
1. Comparison Trees
In a comparison sort, we use only comparisons between elements to gain
order information about an input sequence (a1; a2......an).
Given ai,aj from (a1, a2.....an). We Perform One of the Comparisons
ai < aj less than

ai ≤ a j less than or equal to
ai > aj greater than
ai ≥ a j greater than or equal to
ai = aj equal to
To determine their relative order, if we assume all elements are distinct, then
we just need to consider ai ≤ aj '=' is excluded &, ≥,≤,>,< are equivalent.
Consider sorting three numbers a1, a2, and a3. There are 3! = 6 possible
combinations:
(a1, a2, a3), (a1, a3, a2),
(a2, a1, a3), (a2, a3, a1)
(a3, a1, a2), (a3, a2, a1)
The Comparison based algorithm defines a decision tree.
Decision Tree: A decision tree is a full binary tree that shows the
comparisons between elements that are executed by an appropriate sorting
algorithm operating on an input of a given size. Control, data movement, and
all other conditions of the algorithm are ignored.
In a decision tree, there will be an array of length n.
So, total leaves will be n! (I.e. total number of comparisons)
If tree height is h, then surely
n! ≤2n (tree will be binary)
Taking an Example of comparing a1, a2, and a3.
Left subtree will be true condition i.e. ai ≤ aj
Right subtree will be false condition i.e. ai >aj
So from above, we got

N! ≤2n
Taking Log both sides
Comparison Tree for Binary Search
Example
Suppose we have a list of items according to the following Position:
1,2,3,4,5,6,7,8,9,10,11,12,13,14
And the last midpoint is:
2, 4, 6, 8, 10, 12, 14
Thus, we will consider all the midpoints and we will make a tree of it by
having stepwise midpoints.
The Bold letters are Mid-Points Here
According to Mid-Point, the tree will be:
Step 1: Maximum number of nodes up to k level of the internal node is 2k-1

Example
2k-1
23-1= 8-1=7
Where k = level=3
Step 2: Maximum number of internal nodes in the comparisons tree is n!
Note: Here Internal Nodes are Leaves.
Step 3: From Condition1 & Condition 2 we get
N! ≤ 2k-1
14 < 15
Where N = Nodes
Step 4: Now, n+1 ≤ 2k
Here, Internal Nodes will always be less than 2k in the Binary Search.
Step 5:
n+1<= 2k
Log (n+1) = k log 2
k >=
k >=log2(n+1)
Step 6:
T (n) = k
Step7:
T (n) >=log2(n+1)
Here, the minimum number of Comparisons to perform a task of the search
of n terms using Binary Search
2. Oracle and adversary argument
Another technique for obtaining lower bounds consists of making use of an
"oracle."
Given some model of estimation such as comparison trees, the oracle tells us
the outcome of each comparison.
In order to derive a good lower bound, the oracle efforts it's finest to cause
the algorithm to work as hard as it might.
It does this by deciding as the outcome of the next analysis, the result which
matters the most work to be needed to determine the final answer.
And by keeping step of the work that is finished, a worst-case lower bound
for the problem can be derived.
Example
(Merging Problem) given the sets A (1: m) and B (1: n), where the
information in A and in B are sorted. Consider lower bounds for algorithms
combining these two sets to give an individual sorted set.
Consider that all of the m+n elements are specific and A (1) < A (2) < ....< A
(m) and B (1) < B (2) < ....< B (n).
Elementary combinatory tells us that there are C ((m+n), n)) ways that the A's
and B's may merge together while still preserving the ordering within A and
B.
Thus, if we need comparison trees as our model for combining algorithms,
then there will be C ((m+n), n)) external nodes and therefore at least log C
((m+n), m) comparisons are needed by any comparison-based merging
algorithm.
If we let MERGE (m, n) be the minimum number of comparisons used to
merge m items with n items then we have the inequality
Log C ((m+n), m) MERGE (m, n) m+n-1.
The upper bound and lower bound can get promptly far apart as m gets much
smaller than n.
3. State Space Method
1. State Space Method is a set of rules that show the possible states (n-tuples)
that an algorithm can assume from a given state of a single comparison.
2. Once the state transitions are given, it is possible to derive lower bounds
by arguing that the finished state cannot be reached using any fewer
transitions.
3. Given n distinct items, find winner and loser.
4. Aim: When state changed count it that is the aim of State Space Method.
5. In this approach, we will count the number of comparison by counting the
number of changes in state.
6. Analysis of the problem to find out the smallest and biggest items by using
the state space method.
7. State: It is a collection of attributes.
8. Through this we sort out two types of Problems:
Find the largest & smallest element from an array of elements.
To find out largest & second largest elements from an array of an element.
9. For the largest item, we need 7 comparisons and what will be the second
largest item?
Now we count those teams who lose the match with team A
Teams are: B, D, and E
So the total no of comparisons are: 7
Let n is the total number of items, then
Comparisons = n-1 (to find the biggest item)
No of Comparisons to find out the 2nd biggest item = log2n-1
10. In this no of comparisons are equal to the number of changes of

states during the execution of the algorithm.
Example
State (A, B, C, D)
No of items that can never be compared.
No of items that win but never lost (ultimate winner).
No of items that lost but never win (ultimate loser)
No of items who sometimes lost & sometimes win.
Firstly, A, B compare out or match between them. A wins and in C, D.C wins
and so on. We can assume that B wins and so on. We can assume that B wins
in place of A it can be anything depending on our self.
In Phase-1 there are 4 states
If the team is 8 then 4 states
As if n team the n/2 states.
4 is Constant in C-State as B, D, F, H are lost teams that never win.
Thus there are 3 states in Phase-2,
If n is 8 then states are 3
If n teams that are states.

Phase-3: This is a Phase in which teams which come under C-State are
considered and there will be matches between them to find out the team
which is never winning at all.
In this Structure, we are going to move upward for denoting who is not
winning after the match.
Here H is the team which is never winning at all. By this, we fulfill our
second aim to.
Thus there are 3 states in Phase -3
If n teams that are states

Note: the total of all states value is always equal to 'n'.
Thus, by adding all phase's states we will get:-
Phase1 + Phase 2 + Phase 3
Suppose we have 8 teams then states are there (as minimum)

to find out which one is never wins team.
Thus, Equation is:
Lower bound (L (n)) is a property of the particular issue i.e. the sorting
problem, matrix multiplication not of any particular algorithm solving that
problem.
Lower bound theory says that no calculation can carry out the activity in less
than that of (L (n)) times the units for arbitrary inputs i.e. that for every
comparison based sorting algorithm must take at least L (n) time in the worst
case.
L (n) is the base overall conceivable calculation which is greatest finished.
Trivial lower bounds are utilized to yield the bound best alternative is to
count the number of elements in the problems input that must be prepared
and the number of output items that need to be produced.
The lower bound theory is the method that has been utilized to establish the
given algorithm in the most efficient way which is possible. This is done by
discovering a function g (n) that is a lower bound on the time that any
algorithm must take to solve the given problem. Now if we have an algorithm
whose computing time is the same order as g (n) , then we know that
asymptotically we cannot do better.
If f(n) is the time for some algorithm, then we write f (n) = Ω (g (n)) to mean
that g (n) is the lower bound of f(n) . This equation can be formally written, if
there exists positive constants c and n0 such that |f (n)| >= c|g (n)| for all n >
n0. In addition for developing lower bounds within the constant factor, we
are more conscious of the fact to determine more exact bounds whenever this
is possible.
Deriving good lower bounds is more challenging than arrange efficient
algorithms. This happens because a lower bound states a fact about all
possible algorithms for solving a problem. Generally, we cannot enumerate
and analyze all these algorithms, so lower bound proofs are often hard to
obtain.
Sorting in Linear Time

Linear Time Sorting
We have sorting algorithms that can sort "n" numbers in O (n log n) time.
Merge Sort and Heap Sort achieve this upper bound in the worst case, and
Quick Sort achieves this on Average Case.
Merge Sort, Quick Sort and Heap Sort algorithm share an interesting
property: the sorted order they determined is based only on comparisons
between the input elements. We call such a sorting algorithm "Comparison
Sort".
There is some algorithm that runs faster and takes linear time such as
Counting Sort, Radix Sort, and Bucket Sort but they require the special
assumption about the input sequence to sort.
Counting Sort and Radix Sort assumes that the input consists of an integer in
a small range.
Bucket Sort assumes that a random process that distributes elements
uniformly over the interval generates the input.
Counting Sort
It is a linear time sorting algorithm which works faster by not making a
comparison. It assumes that the number to be sorted is in range 1 to k where k
is small.
Basic idea is to determine the "rank" of each number in the final sorted array.
Counting Sort uses three arrays:
1. A [1, n] holds initial input.

2. B [1, n] holds sorted output.
3. C [1, k] is the array of integer. C [x] is the rank of x in A where
x ∈ [1, k]
Firstly C [x] to be a number of elements of A [j] that is equal to x
Initialize C to zero
For each j from 1 to n increment C [A[j]] by 1
We set B[C [x]] =A[j]

If there are duplicates, we decrement C [i] after copying.
Counting Sort (array P, array Q, int k)
1. For i ← 1 to k
2. do C [i] ← 0 [ θ(k) times]
3. for j ← 1 to length [A]
4. do C[A[j]] ← C [A [j]]+1 [θ(n) times]
5. // C [i] now contain the number of elements equal to i
6. for i ← 2 to k
7. do C [i] ← C [i] + C[i-1] [θ(k) times]
8. //C[i] now contain the number of elements ≤ i
9. for j ← length [A] down to 1 [θ(n) times]
10. do B[C[A[j] ← A [j]
11. C[A[j] ← C[A[j]-1
Explanation
Step1: for loop initialize the array R to 'o'. But there is a contradict in the first
step initialize of loop variable 1 to k or 0 to k. As 0&1 are based on the
minimum value comes in array A (input array). Basically, we start I with the
value which is minimum in input array 'A'
For loops of steps 3 to 4 inspects each input element. If the value of an input
element is 'i', we increment C [i]. Thus, after step 5, C [i] holds the number of
input element equal to I for each integer i=0, 1, 2.....k
Step 6 to 8 for loop determines for each i=0, 1.....how many input elements
are less than or equal to i
For loop of step 9 to 11 place each element A [j] into its correct sorted
position in the output array B. for each A [j],the value C [A[j]] is the correct
final position of A [j] in the output array B, since there are C [A[j]] element
less than or equal to A [i].
Because element might not be distinct, so we decrement C[A[j] each time we
place a value A [j] into array B decrement C[A[j] causes the next input
element with a value equal to A [j], to go to the position immediately before
A [j] in the output array.
Analysis of Running Time
For a loop of step 1 to 2 take θ(k) times

For a loop of step 3 to 4 take θ(n) times
For a loop of step 6 to 7 take θ(k) times
For a loop of step 9 to 11 take θ(n) times
Overall time is θ(k+n) time.

Note
Counting Sort has the important property that it is stable: numbers with the
same value appears in the output array in the same order as they do in the
input array.
Counting Sort is used as a subroutine in Radix Sort.
Example
Illustration the operation of Counting Sort in the array.
A= ( 7,1,3,1,2,4,5,7,2,4,3)
Solution
Initial A and C Arrays

For j=1 to 11
J=1, C [1, k] =
A [1] = 7 Processed
J=2, C [1, k] =
A [2] = 1 Processed
J=3, C [1, k]
A [3] = 3 Processed
J=4, C [1, k]
A [4] = 1 Processed
J=5, C [1, k]
A [5] = 2 Processed
UPDATED C is:
C now contains a count of elements of A

Note: here the item of 'A' one by one get scanned and will become a position
in 'C' and how many times the item get accessed will be mentioned in an item
in 'C' Array and it gets updated or counter increased by 1 if any item gets
accessed again.
Now, the for loop i= 2 to 7 will be executed having statement:
C [i] = C [i] + C [i-1]
By applying these conditions we will get C updated as i stated from 2 up to 7
C [2] = C [2] + C [1] C [3] = C [3] + C [2]
C [2] = 2 + 2 C [3] = 2 + 4
C [2] = 4 C [3] = 6
C [4] = C [4] + C [3] C [5] = C [5] + C [4]
C [4] = 2 + 6 C [5] = 1 +8
C [4] = 8 C [5] = 9
C [6] = C [6] + C [5] C [7] = C [7] + C [6]
C [6] = 0 + 9 C [7] = 2 + 9
C [6] = 9 C [7] = 11
Thus the Updated C is:
C set to rank each number of A
Now, we will find the new array B

Now two Conditions will apply:
1. B[C[A[j] ← A [j]
2. C[A[j] ← C[A[j]-1
We decrease counter one by one by '1'

We start scanning of the element in A from the last position.
Element in A became a position in C
For j ← 11 to 1
Step 1
B [C [A [11]]] = A [11] C [A [11] = C [A [11]-1
B [C [3] = 3 C [3] = C [3] -1
B [6] = 3 C [3] = 5
A [11] placed in Output array B

Step 2
B [C [A [10]]] = A [10] C [A [10]] = C [A [10]]-1
B [C [4]] =4 C [4] = C [4] -1
B [8] = 4 C [4] = 7
Step 3
B [C [A [9]] = A [9] C [A [9] = C [A [9]]-1
B [C [2]] = A [2] C [2] = C [2]-1
B [4] = 2 C [2] = 3
Step 4
B [C [A [8]]] = A [8] C [A [8]] =C [A [8]] -1
B [C [7]] =7 C [A [8]] = C [7]-1
B [11] =7 C [7] = 10
Step 5
B [C [A [7]]] = A [7] C [A [7]] = C [A [7]] - 1
B [C [5]] = 5 C [5] = C [5] - 1
B [9] = 5 C [5] =8
Step 6
B [C [A [6]]] = A [6] C [A [6]] = C [A [6]] - 1
B [C [4]] = 4 C [4] = C [4] - 1
B [7] = 4 C [4] = 6
Step 7
B [C [A [5]]] = A [5] C [A [5] = C [A [5]] -1
B [C [2] =2 C [2] = C [2] - 1
B [3] = 2 C [2] = 2
Step 8
B [C [A [4]]] = A [4] C [A [4]] = C [A [4]] - 1
B [C [1] = 1 C [1] = C [1] - 1
B [2] = 1 C [1] = 1
Step 9
B [C [A [3]]] = A [3] C [A [3]] = C [A [3]] - 1
B [C [3] = 3 C [3] = C [3] - 1
B [5] = 3 C [3] = 4
Step 10
B [C [A [2]]] = A [2] C [A [2]] = C [A [2]] - 1
B [C [1]] = 1 C [1] = C [1] - 1
B [1] = 1 C [1] = 0
Step 11
B [C [A [1]]] = A [1] C [A [1]] = C [A [1]] - 1
B [C [7]] = 7 C [7] = C [7] - 1
B [10] = 7 C [7] = 9
B now contains the final sorted data.
Bucket Sort
Bucket Sort runs in linear time on average. Like Counting Sort, bucket Sort is
fast because it considers something about the input. Bucket Sort considers
that the input is generated by a random process that distributes elements
uniformly over the intervalμ=[0,1].
To sort n input numbers, Bucket Sort
1. Partition μ into n non-overlapping intervals called buckets.

2. Puts each input number into its buckets
3. Sort each bucket using a simple algorithm, e.g. Insertion Sort
and then
4. Concatenates the sorted lists.
Bucket Sort considers that the input is an n element array A and that each
element A [i] in the array satisfies 0≤A [i] <1. The code depends upon an
auxiliary array B [0....n-1] of linked lists (buckets) and considers that there is
a mechanism for maintaining such lists.
Bucket-Sort (A)
1. n ← length [A]
2. for i ← 1 to n
3. do insert A [i] into list B [n A[i]]
4. for i ← 0 to n-1
5. do sort list B [i] with insertion sort.
6. Concatenate the lists B [0], B [1] ...B [n-1] together in order.
Example
Illustrate the operation of BUCKET-SORT on the array.
A = (0.78, 0.17, 0.39, 0.26, 0.72, 0.94, 0.21, 0.12, 0.23, 068)
Solution
Bucket sort: step 1, placing keys in bins in sorted order

Bucket sort: step 2, concatenate the lists
Bucket sort: the final sorted sequence
Radix Sort
Radix Sort is a Sorting algorithm that is useful when there is a constant'd'
such that all keys are d digit numbers. To execute Radix Sort, for p =1
towards 'd' sort the numbers with respect to the Pth digits from the right using
any linear time stable sort.
The Code for Radix Sort is straightforward. The following procedure
assumes that each element in the n-element array A has d digits, where digit 1
is the lowest order digit and digit d is the highest-order digit.
Here is the algorithm that sorts A [1.n] where each number is d digits long.
RADIX-SORT (array A, int n, int d)
1 for i ← 1 to d
2 do stably sort A to sort array A on digit i
Example
The first Column is the input. The remaining Column shows the list after
successive sorts on increasingly significant digit position. The vertical arrows
indicate the digits position sorted on to produce each list from the previous
one.
576 49[4] 9[5]4 [1]76 176
494 19[4] 5[7]6 [1]94 194
194 95[4] 1[7]6 [2]78 278
296 → 57[6] → 2[7]8 → [2]96 → 296
278 29[6] 4[9]4 [4]94 494
176 17[6] 1[9]4 [5]76 576
954 27[8] 2[9]6 [9]54 954
Hashing
Hashing is the transformation of a string of character into a usually shorter
fixed-length value or key that represents the original string.
Hashing is used to index and retrieve items in a database because it is faster
to find the item using the shortest hashed key than to find it using the original
value. It is also used in many encryption algorithms.
A hash code is generated by using a key, which is a unique value.
Hashing is a technique in which given key field value is converted into the
address of storage location of the record by applying the same operation on it.
The advantage of hashing is that allows the execution time of basic operation
to remain constant even for the larger side.
Why we need Hashing?
Suppose we have 50 employees, and we have to give 4 digit key to each
employee (as for security), and we want after entering a key, direct user map
to a particular position where data is stored.
If we give the location number according to 4 digits, we will have to reserve
0000 to 9999 addresses because anybody can use anyone as a key. There is a
lot of wastage.
In order to solve this problem, we use hashing which will produce a smaller
value of the index of the hash table corresponding to the key of the user.
Universal Hashing
Let H be a finite collection of hash functions that map a given universe U of
keys into the range {0, 1..... m-1}. Such a collection is said to be universal if
for each pair of distinct keys k,l ∈ U, the number of hash functions h ∈ H
for which h(k)= h(l) is at most |H|/m. In other words, with a hash function
randomly chosen from H, the chance of a collision between distinct keys k
and l is no more than the chance 1/m of a collision if h(k) and h(l)were
randomly and independently chosen from the set {0,1,...m-1}.
Rehashing
If any stage the hash table becomes nearly full, the running time for the
operations of will start taking too much time, insert operation may fail in
such situation, the best possible solution is as follows:
1. Create a new hash table double in size.

2. Scan the original hash table, compute new hash value and insert
into the new hash table.
3. Free the memory occupied by the original hash table.
Example
Consider inserting the keys 10, 22, 31,4,15,28,17,88 and 59 into a hash table
of length m = 11 using open addressing with the primary hash function h' (k)
= k mod m .Illustrate the result of inserting these keys using linear probing,
using quadratic probing with c1=1 and c2=3, and using double hashing with
h2(k) = 1 + (k mod (m-1)).
Solution
Using Linear Probing the final state of hash table would be:
Using Quadratic Probing with c1=1, c2=3, the final state of hash table would
be h (k, i) = (h' (k) +c1*i+ c2 *i2) mod m where m=11 and h' (k) = k mod m.
Using Double Hashing, the final state of the hash table would be:
Hash Tables
It is a collection of items which are stored in such a way as to make it easy to
find them later.
Each position in the hash table is called slot, can hold an item and is named
by an integer value starting at 0.
The mapping between an item and a slot where the item belongs in a hash
table is called a Hash Function. A hash Function accepts a key and returns its
hash coding, or hash value.
Assume we have a set of integers 54, 26, 93, 17, 77, 31. Our first hash
function required to be as "remainder method" simply takes the item and
divide it by table size, returning remainder as its hash value i.e.
h item = item % (size of table)
Let us say the size of table = 11, then
54 % 11 = 10 26 % 11 = 4 93 % 11 = 5
17 % 11 = 6 77 % 11 = 0 31 % 11 = 9
Item Hash
Value
54 10
26 4
93 5
17 6
77 0
31 9
Now when we need to search any element, we just need to divide it by the
table size, and we get the hash value. So we get the O (1) search time.
Now taking one more element 44 when we apply the hash function on 44, we
get (44 % 11 = 0), But 0 hash value already has an element 77. This Problem
is called as Collision.
Collision: According to the Hash Function, two or more item would need in
the same slot. This is said to be called as Collision.
Using a hash function h to map keys to hash-table slots. Because keys K2 and
k5 map to the same slot, they collide.
Why use HashTable?
1. If U (Universe of keys) is large, storing a table T of size [U]

may be impossible.
2. Set k of keys may be small relative to U so space allocated for
T will waste.
So Hash Table requires less storage. Indirect addressing element with key k is
stored in slot k with hashing it is stored in h (k) where h is a hash fn and hash
(k) is the value of key k. Hash fn required array range.
Application of Hash Tables
Some application of Hash Tables are:
1. Database System: Specifically, those that are required efficient

random access. Usually, database systems try to develop
between two types of access methods: sequential and random.
Hash Table is an integral part of efficient random access
because they provide a way to locate data in a constant amount
of time.
2. Symbol Tables: The tables utilized by compilers to maintain
data about symbols from a program. Compilers access
information about symbols frequently. Therefore, it is essential
that symbol tables be implemented very efficiently.
3. Data Dictionaries: Data Structure that supports adding,
deleting, and searching for data. Although the operation of hash
tables and a data dictionary are similar, other Data Structures
may be used to implement data dictionaries.
4. Associative Arrays: Associative Arrays consist of data arranged
so that nth elements of one array correspond to the nth element
of another. Associative Arrays are helpful for indexing a logical
grouping of data by several key fields.
Methods of Hashing
There are two main methods used to implement hashing:
1. Hashing with Chaining

2. Hashing with Open Addressing
1. Hashing with Chaining

In Hashing with Chaining, the element in S is stored in Hash table T [0...m-1]
of size m, where m is somewhat larger than n, the size of S. The hash table is
said to have m slots. Associated with the hashing scheme is a hash function h
which is mapping from U to {0...m-1}.Each key k ∈ S is stored in location
T [h (k)], and we say that k is hashed into slot h (k). If more than one key in S
hashed into the same slot then we have a collision.
In such case, all keys that hash into the same slot are placed in a linked list
associated with that slot, this linked list is called the chain at slot. The load
factor of a hash table is defined to be ∝ =n/m it represents the average
number of keys per slot. We typically operate in the range m=θ(n), so ∝ is
usually a constant generally ∝ <1.
Collision Resolution by Chaining
In chaining, we place all the elements that hash to the same slot into the same
linked list, As fig shows that Slot j contains a pointer to the head of the list of
all stored elements that hash to j ; if there are no such elements, slot j contains
NIL.
Collision resolution by chaining.

Each hash-table slot T [j] contains a linked list of all the keys whose hash
value is j.
For example, h (k1) = h (k4) and h (k5) = h (k7) =h (K2). The linked list can
be either singly or doubly linked; we show it as doubly linked because
deletion is faster that way.
Analysis of Hashing with Chaining

Given a hash table T with m slots that stores n elements, we define the load
factors α for T as n/m that is the average number of elements stored in a
chain. The worst case running time for searching is thus θ(n) plus the time to
compute the hash function- no better than if we used one linked list for all the
elements. Clearly, hash tables are not used for their worst-case performance.
The average performance of hashing depends on how well the hash function
h distributes the set of keys to be stored among the m slots, on the average.
Example
Let us consider the insertion of elements 5, 28, 19,15,20,33,12,17,10 into a
chained hash table. Let us suppose the hash table has 9 slots and the hash
function be h (k) =k mod 9.
Solution
The initial state of chained-hash table
Insert 5:
h (5) = 5 mod 9 =5
Create a linked list for T [5] and store value 5 in it.
Similarly, insert 28. h (28) = 28 mod 9 =1. Create a Linked List for T [1] and
store value 28 in it. Now insert 19 h (19) = 19 mod 9 = 1. Insert value 19 in
the slot T [1] at the beginning of the linked-list.
Now insert h 15, h (15) = 15 mod 9 = 6. Create a link list for T [6] and store
value 15 in it.
Similarly, insert 20, h (20) = 20 mod 9 = 2 in T [2].
Insert 33, h (33) = 33 mod 9 = 6
In the beginning of the linked list T [6]. Then,
Insert 12, h (12) = 12 mod 9 = 3 in T [3].
Insert 17, h (17) = 17 mod 9 = 8 in T [8].
Insert 10, h (10) = 10 mod 9 = 1 in T [1].
Thus the chained- hash- table after inserting key 10 is
2. Hashing with Open Addressing

In Open Addressing, all elements are stored in hash table itself. That is, each
table entry consists of a component of the dynamic set or NIL. When
searching for an item, we consistently examine table slots until either we find
the desired object or we have determined that the element is not in the table.
Thus, in open addressing, the load factor α can never exceed 1.
The advantage of open addressing is that it avoids Pointer. In this, we
compute the sequence of slots to be examined. The extra memory freed by
not sharing pointers provides the hash table with a larger number of slots for
the same amount of memory, potentially yielding fewer collision and faster
retrieval.
The process of examining the location in the hash table is called Probing.
Thus, the hash function becomes
h : U x {0,1,....m-1} → {0,1,....,m-1}.
With open addressing, we require that for every key k, the probe sequence
{h, (k, 0), h (k, 1)....h (k, m-1)}
Be a Permutation of (0, 1...... m-1)
The HASH-INSERT procedure takes as input a hash table T and a key k
Hash-Insert (T, k)
1. i ← 0
2. repeat j ← h (k, i)
3. if T [j] = NIL
4. then T [j] ← k
5. return j
6. else ← i= i +1
7. until i=m
8. error "hash table overflow"
The procedure HASH-SEARCH takes as input a hash table T and a key k,
returning j if it finds that slot j contains key k or NIL if key k is not present in
table T.
Hash-Search.T (k)
1. i ← 0
2. repeat j ← h (k, i)
3. if T [j] =j
4. then return j
5. i ← i+1
6. until T [j] = NIL or i=m
7. return NIL
Open Addressing Techniques

Three techniques are commonly used to compute the probe sequence required
for open addressing:
1. Linear Probing
2. Quadratic Probing
3. Double Hashing
1. Linear Probing
It is a Scheme in Computer Programming for resolving collision in hash
tables.
Suppose a new record R with key k is to be added to the memory table T but
that the memory locations with the hash address H (k). H is already filled.
Our natural key to resolve the collision is to crossing R to the first available
location following T (h). We assume that the table T with m location is
circular, so that T [i] comes after T [m].
The above collision resolution is called "Linear Probing".
Linear probing is simple to implement, but it suffers from an issue known as
primary clustering. Long runs of occupied slots build up, increasing the
average search time. Clusters arise because an empty slot proceeded by i full
slots gets filled next with probability (i + 1)/m. Long runs of occupied slots
tend to get longer, and the average search time increases.
Given an ordinary hash function h': U {0, 1...m-1}, the method of linear
probing uses the hash function.
h (k, i) = (h' (k) + i) mod m
Where 'm' is the size of hash table and h' (k) = k mod m. for i=0, 1....m-1.
Given key k, the first slot is T [h' (k)]. We next slot T [h' (k) +1] and so on up
to the slot T [m-1]. Then we wrap around to slots T [0], T [1]....until finally
slot T [h' (k)-1]. Since the initial probe position dispose of the entire probe
sequence, only m distinct probe sequences are used with linear probing.
Example
Consider inserting the keys 24, 36, 58,65,62,86 into a hash table of size m=11
using linear probing, consider the primary hash function is h' (k) = k mod m.
Solution
Initial state of hash table
Insert 24. We know h (k, i) = [h' (k) + i] mod m

Now h (24, 0) = [24 mod 11 + 0] mod 11
= (2+0) mod 11 = 2 mod 11 = 2
Since T [2] is free, insert key 24 at this place.
Insert 36. Now h (36, 0) = [36 mod 11 + 0] mod 11
= [3+0] mod 11 = 3
Since T [3] is free, insert key 36 at this place.
Insert 58. Now h (58, 0) = [58 mod 11 +0] mod 11
= [3+0] mod 11 =3
Since T [3] is not free, so the next sequence is
h (58, 1) = [58 mod 11 +1] mod 11
= [3+1] mod 11= 4 mod 11=4
T [4] is free; Insert key 58 at this place.
= (10 +0) mod 11= 10
T [10] is free. Insert key 65 at this place.
= [7 + 0] mod 11 = 7
Insert 86. Now h (86, 0) = [86 mod 11 + 0] mod 11
= [9 + 0] mod 11 = 9
Thus,
2. Quadratic Probin
Suppose a record R with key k has the hash address H (k) = h then instead of
searching the location with addresses h, h+1, and h+ 2...We linearly search
the locations with addresses
h, h+1, h+4, h+9...h+i2
Quadratic Probing uses a hash function of the form
h (k,i) = (h' (k) + c1i + c2i2) mod m
Where (as in linear probing) h' is an auxiliary hash function c1 and c2 ≠0 are
auxiliary constants and i=0, 1...m-1. The initial position is T [h' (k)]; later
position probed is offset by the amount that depend in a quadratic manner on
the probe number i.
Example
Consider inserting the keys 74, 28, 36,58,21,64 into a hash table of size m
=11 using quadratic probing with c1=1 and c2=3. Further consider that the
primary hash function is h' (k) = k mod m.
Solution
For Quadratic Probing, we have
h (k, i) = [k mod m +c1i +c2 i2) mod m
This is the initial state of hash table

Here c1= 1 and c2=3
h (k, i) = [k mod m + i + 3i2 ] mod m
Insert 74
h (74,0)= (74 mod 11+0+3x0) mod 11
= (8 +0+0) mod 11 = 8
T [8] is free; insert the key 74 at this place.
Insert 28
h (28, 0) = (28 mod 11 + 0 + 3 x 0) mod 11
= (6 +0 + 0) mod 11 = 6.
T [6] is free; insert key 28 at this place.
Insert 36
h (36, 0) = (36 mod 11 + 0 + 3 x 0) mod 11
= (3 + 0+0) mod 11=3
Insert 58
h (58, 0) = (58 mod 11 + 0 + 3 x 0) mod 11
= (3 + 0 + 0) mod 11 = 3
T [3] is not free, so next probe sequence is computed as
h (59, 1) = (58 mod 11 + 1 + 3 x12) mod 11
= (3 + 1 + 3) mod 11
=7 mod 11= 7
Insert 21
h (21, 0) = (21 mod 11 + 0 + 3 x 0)
= (10 + 0) mod 11 = 10
Insert 64
h (64, 0) = (64 mod 11 + 0 + 3 x 0)
= (9 + 0+ 0) mod 11 = 9.
Thus, after inserting all keys, the hash table is
3. Double Hashing
Double Hashing is one of the best techniques available for open addressing
because the permutations produced have many of the characteristics of
randomly chosen permutations.
Double hashing uses a hash function of the form
h (k, i) = (h1(k) + i h2 (k)) mod m
Where h1 and h2 are auxiliary hash functions and m is the size of the hash
table.
h1 (k) = k mod m or h2 (k) = k mod m'. Here m' is slightly less than m (say m-
1 or m-2).
Example
Consider inserting the keys 76, 26, 37,59,21,65 into a hash table of size m =
11 using double hashing. Consider that the auxiliary hash functions are
h1 (k)=k mod 11 and h2(k) = k mod 9.
Solution
Initial state of Hash table is
1. Insert 76.
h1(76) = 76 mod 11 = 10
h2(76) = 76 mod 9 = 4
h (76, 0) = (10 + 0 x 4) mod 11
= 10 mod 11 = 10
T [10] is free, so insert key 76 at this place.
2. Insert 26.
h1(26) = 26 mod 11 = 4
h2(26) = 26 mod 9 = 8
h (26, 0) = (4 + 0 x 8) mod 11
= 4 mod 11 = 4
3. Insert 37.
h1(37) = 37 mod 11 = 4
h2(37) = 37 mod 9 = 1
h (37, 0) = (4 + 0 x 1) mod 11 = 4 mod 11 = 4
T [4] is not free, the next probe sequence is
h (37, 1) = (4 + 1 x 1) mod 11 = 5 mod 11 = 5
4. Insert 59.
h1(59) = 59 mod 11 = 4
h2(59) = 59 mod 9 = 5
h (59, 0) = (4 + 0 x 5) mod 11 = 4 mod 11 = 4
Since, T [4] is not free, the next probe sequence is
h (59, 1) = (4 + 1 x 5) mod 11 = 9 mod 11 = 9
5. Insert 21.
h1(21) = 21 mod 11 = 10
h2(21) = 21 mod 9 = 3
h (21, 0) = (10 + 0 x 3) mod 11 = 10 mod 11 = 10
h (21, 1) = (10 + 1 x 3) mod 11 = 13 mod 11 = 2
6. Insert 65.
h1(65) = 65 mod 11 = 10
h2(65) = 65 mod 9 = 2
h (65, 0) = (10 + 0 x 2) mod 11 = 10 mod 11 = 10
h (65, 1) = (10 + 1 x 2) mod 11 = 12 mod 11 = 1
Thus, after insertion of all keys the final hash table is
Hash Function
Hash Function is used to index the original value or key and then used later
each time the data associated with the value or key is to be retrieved. Thus,
hashing is always a one-way operation. There is no need to "reverse
engineer" the hash function by analyzing the hashed values.
Characteristics of Good Hash Function
1. The hash value is fully determined by the data being hashed.

2. The hash Function uses all the input data.
3. The hash function "uniformly" distributes the data across the
entire set of possible hash values.
4. The hash function generates complicated hash values for
similar strings.
Some Popular Hash Function is:

1. Division Method
Choose a number m smaller than the number of n of keys in k (The number
m is usually chosen to be a prime number or a number without small divisors,
since this frequently a minimum number of collisions).
The hash function is:
Example
If the hash table has size m = 12 and the key is k = 100, then h (k) = 4. Since
it requires only a single division operation, hashing by division is quite fast.
2. Multiplication Method
The multiplication method for creating hash functions operates in two steps.
First, we multiply the key k by a constant A in the range 0 < A < 1 and
extract the fractional part of kA. Then, we increase this value by m and take
the floor of the result.
The hash function is:
Where "k A mod 1" means the fractional part of k A, that is, k A - ⌊ k A ⌋ .
3. Mid Square Method
The key k is squared. Then function H is defined by
H (k) = L
Where L is obtained by deleting digits from both ends of k2. We emphasize
that the same position of k2 must be used for all of the keys.
4. Folding Method
The key k is partitioned into a number of parts k1, k2.... kn where each part
except possibly the last, has the same number of digits as the required
address.
Then the parts are added together, ignoring the last carry.
H (k) = k1+ k2+.....+kn
Example
Company has 68 employees, and each is assigned a unique four- digit
employee number. Suppose L consist of 2- digit addresses: 00, 01, and
02....99. We apply the above hash functions to each of the following
employee numbers:
3205, 7148, 2345
(a) Division Method: Choose a Prime number m close to 99, such as m =97,
Then
H (3205) = 4, H (7148) = 67, H (2345) = 17.
That is dividing 3205 by 17 gives a remainder of 4, dividing 7148 by 97
gives a remainder of 67, dividing 2345 by 97 gives a remainder of 17.
(b) Mid-Square Method:
k = 3205 7148 2345
2
k = 10272025 51093904 5499025
h (k) = 72 93 99
Observe that fourth & fifth digits, counting from right are chosen for hash
address.
(c) Folding Method: Divide the key k into 2 parts and adding yields the
following hash address:
H (3205) = 32 + 50 = 82 H (7148) = 71 + 84 = 55
H (2345) = 23 + 45 = 68
Binary Search Trees

A Binary Search tree is organized in a Binary Tree. Such a tree can be
defined by a linked data structure in which a particular node is an object. In
addition to a key field, each node contains field left, right, and p that point to
the nodes corresponding to its left child, its right child, and its parent,
respectively. If a child or parent is missing, the appropriate field contains the
value NIL. The root node is the only node in the tree whose parent field is
Nil.
Binary Search Tree Property
Let x be a node in a binary search tree.
If y is a node in the left subtree of x, then key [y] ≤key [k].

If z is a node in the right subtree of x, then key [x] ≤ key [y].
In this tree key [x] = 15

If y is a node in the left subtree of x, then key [y] = 5.
i.e. key [y] ≤ key[x].
If y is a node in the right subtree of x, then key [y] = 20.
i.e. key [x] ≤ key[y].
Traversal in Binary Search Trees
1. In-Order-Tree-Walk (x): Always prints keys in binary search tree in
sorted order.
Inorder-Tree-Walk(x) - Running time is θ(n)
1. If x ≠ NIL.
2. then INORDER-TREE-WALK (left [x])
3. print key [x]
4. INORDER-TREE-WALK (right [x])
2. Preorder-Tree-Walk(x)
In which we visit the root node before the nodes in either subtree.
Preorder-Tree-Walk (x):
1. If x ≠ NIL.
2. then print key [x]
3. PREORDER-TREE-WALK (left [x]).
4. PREORDER-TREE-WALK (right [x]).
3. Postorder-Tree-Walk (x): In which we visit the root node after the nodes
in its subtree.
Postorder-Tree-Walk (x)
1. If x ≠ NIL.
2. then POSTORDER-TREE-WALK (left [x]).
3. POSTORDER-TREE-WALK (right [x]).
4. print key [x]
Querying a Binary Search Trees
1. Searching: The TREE-SEARCH (x, k) algorithm searches the tree node at
x for a node whose key value equal to k. It returns a pointer to the node if it
exists otherwise NIL.
TREE-SEARCH (x, k)
1. If x = NIL or k = key [x].
2. then return x.
3. If k < key [x].
4. then return TREE-SEARCH (left [x], k)
5. else return TREE-SEARCH (right [x], k)
Clearly, this algorithm runs in O (h) time where h is the height of the tree.
The iterative version of the above algorithm is very easy to implement
ITERATIVE-TREE- SEARCH (x, k)
1. while x ≠ NIL and k ≠ key [k].
2. do if k < key [x].
3. then x ← left [x].
4. else x ← right [x].
5. return x.
2. Minimum and Maximum: An item in a binary search tree whose key is a
minimum can always be found by following left child pointers from the root
until a NIL is encountered. The following procedure returns a pointer to the
minimum element in the subtree rooted at a given node x.
TREE- MINIMUM (x)
1. While left [x] ≠ NIL.
2. do x←left [x].
3. return x.
TREE-MAXIMUM (x)
1. While left [x] ≠ NIL
2. do x←right [x].
3. return x.
3. Successor and predecessor: Given a node in a binary search tree,
sometimes we used to find its successor in the sorted form determined by an
in order tree walk. If all keys are specific, the successor of a node x is the
node with the smallest key greater than key[x]. The structure of a binary
search tree allows us to rule the successor of a node without ever comparing
keys. The following action returns the successor of a node x in a binary
search tree if it exists, and NIL if x has the greatest key in the tree:
TREE SUCCESSOR (x)
1. If right [x] ≠ NIL.
2. Then return TREE-MINIMUM (right [x]))
3. y←p [x]
4. While y ≠ NIL and x = right [y]
5. do x←y
6. y←p[y]
7. return y.
The code for TREE-SUCCESSOR is broken into two cases. If the right
subtree of node x is nonempty, then the successor of x is just the leftmost
node in the right subtree, which we find in line 2 by calling TREE-
MINIMUM (right [x]). On the other hand, if the right subtree of node x is
empty and x has a successor y, then y is the lowest ancestor of x whose left
child is also an ancestor of x. To find y, we quickly go up the tree from x
until we encounter a node that is the left child of its parent; lines 3-7 of
TREE-SUCCESSOR handle this case.
The running time of TREE-SUCCESSOR on a tree of height h is O (h) since
we either follow a simple path up the tree or follow a simple path down the
tree. The procedure TREE-PREDECESSOR, which is symmetric to TREE-
SUCCESSOR, also runs in time O (h).
4. Insertion in Binary Search Tree: To insert a new value into a binary
search tree T, we use the procedure TREE-INSERT. The procedure takes a
node ´ for which key [z] = v, left [z] NIL, and right [z] = NIL. It modifies T
and some of the attributes of z in such a way that it inserts into an appropriate
position in the tree.
TREE-INSERT (T, z)
1. y ←NIL.
2. x←root [T]
3. while x ≠ NIL.
4. do y←x
5. if key [z]< key [x]
6. then x←left [x].
7. else x←right [x].
8. p [z]←y
9. if y = NIL.
10. then root [T]←z
11. else if key [z] < key [y]
12. then left [y]←z
Example
Working of TREE-INSERT
Suppose we want to insert an item with key 13 into a Binary Search Tree.
x=1
y = 1 as x ≠ NIL.
Key [z] < key [x]
13 < not equal to 12.
x ←right [x].
x ←3
Again x ≠ NIL
y ←3
key [z] < key [x]
13 < 18
x←left [x]
x←6
Again x ≠ NIL, y←6
13 < 15
x←left [x]
x←NIL
p [z]←6
Now our node z will be either left or right child of its parent (y).
key [z] < key [y]
13 < 15
Left [y] ← z
Left [6] ← z
So, insert a node in the left of node index at 6.
5. Deletion in Binary Search Tree: When Deleting a node from a tree it is
essential that any relationships, implicit in the tree can be maintained. The
deletion of nodes from a binary search tree will be considered:
There are three distinct cases:
1. Nodes with no children: This case is trivial. Simply set the

parent's pointer to the node to be deleted to nil and delete the
node.
2. Nodes with one child: When z has no left child then we replace
z by its right child which may or may not be NIL. And when z
has no right child, then we replace z with its right child.
3. Nodes with both Childs: When z has both left and right child.
We find z's successor y, which lies in right z's right subtree and
has no left child (the successor of z will be a node with
minimum value its right subtree and so it has no left child).
If y is z's right child, then we replace z.

Otherwise, y lies within z's right subtree but not z's
right child. In this case, we first replace z by its own
right child and the replace z by y.
TREE-DELETE (T, z)
1. If left [z] = NIL or right [z] = NIL.
2. Then y ← z
3. Else y ← TREE- SUCCESSOR (z)
4. If left [y] ≠ NIL.
5. Then x ← left [y]
6. Else x ← right [y]
7. If x ≠NIL
8. Then p[x] ← p [y]
9. If p[y] = NIL.
10. Then root [T] ← x
11. Else if y = left [p[y]]
12. Then left [p[y]] ← x
13. Else right [p[y]] ← y
14. If y ≠ z.
15. Then key [z] ← key [y]
16. If y has other fields, copy them, too.
17. Return y
The Procedure runs in O (h) time on a tree of height h.
Example
Deleting a node z from a binary search tree. Node z may be the root, a left
child of node q, or a right child of q.
Z has the only right child.
Z has the only left child. We replace z by l.

Node z has two children; its left child is node l, its right child is its successor
y, and y's right child is node x. We replace z by y, updating y's left child to
become l, but leaving x as y's right child.
Node z has two children (left child l and right child r), and its successor y ≠ r
lies within the subtree rooted at r. We replace y with its own right child x,
and we set y to be r's parent. Then, we set y to be q's child and the parent of l.
Red Black Tree

A Red Black Tree is a category of the self-balancing binary search tree. It
was created in 1972 by Rudolf Bayer who termed them "symmetric binary B-
trees."
A red-black tree is a Binary tree where a particular node has color as an extra
attribute, either red or black. By check the node colors on any simple path
from the root to a leaf, red-black trees secure that no such path is higher than
twice as long as any other so that the tree is generally balanced.
Properties of Red-Black Trees
A red-black tree must satisfy these properties:
1. The root is always black.

2. A nil is recognized to be black. This factor that every non-NIL
node has two children.
3. Black Children Rule: The children of any red node are black.
4. Black Height Rule: For particular node v, there exists an integer
bh (v) such that specific downward path from v to a nil has
correctly bh (v) black real (i.e. non-nil) nodes. Call this portion
the black height of v. We determine the black height of an RB
tree to be the black height of its root.
A tree T is an almost red-black tree (ARB tree) if the root is red, but other
conditions above hold.
Operations on RB Trees
The search-tree operations TREE-INSERT and TREE-DELETE, when runs
on a red-black tree with n keys, take O (log n) time. Because they customize
the tree, the conclusion may violate the red-black properties. To restore these
properties, we must change the color of some of the nodes in the tree and also
change the pointer structure.
1. Rotation
Restructuring operations on red-black trees can generally be expressed more
clearly in details of the rotation operation.
Clearly, the order (Ax By C) is preserved by the rotation operation.

Therefore, if we start with a BST and only restructure using rotation, then we
will still have a BST i.e. rotation do not break the BST-Property.
LEFT ROTATE (T, x)
1. y ← right [x]
1. y ← right [x]
2. right [x] ← left [y]
3. p [left[y]] ← x
4. p[y] ← p[x]
5. If p[x] = nil [T]
then root [T] ← y
else if x = left [p[x]]
then left [p[x]] ← y

else right [p[x]] ← y
6. left [y] ← x.
7. p [x] ← y.
Example
Draw the complete binary tree of height 3 on the keys {1, 2, 3... 15}. Add the
NIL leaves and color the nodes in three different ways such that the black
heights of the resulting trees are: 2, 3 and 4.
Solution
Tree with black-height-2

2. Insertion
Insert the new node the way it is done in Binary Search Trees.
Color the node red
If an inconsistency arises for the red-black tree, fix the tree
according to the type of discrepancy.
A discrepancy can decision from a parent and a child both having a red color.
This type of discrepancy is determined by the location of the node concerning
grandparent, and the color of the sibling of the parent.
RB-INSERT (T, z)
1. y ← nil [T]
2. x ← root [T]
3. while x ≠ NIL [T]
4. do y ← x
5. if key [z] < key [x]
6. then x ← left [x]
7. else x ← right [x]
8. p [z] ← y
9. if y = nil [T]
10. then root [T] ← z
11. else if key [z] < key [y]
12. then left [y] ← z
13. else right [y] ← z
14. left [z] ← nil [T]
15. right [z] ← nil [T]
16. color [z] ← RED
17. RB-INSERT-FIXUP (T, z)
After the insert new node, Coloring this new node into black may violate the
black-height conditions and coloring this new node into red may violate
coloring conditions i.e. root is black and red node has no red children. We
know the black-height violations are hard. So we color the node red. After
this, if there is any color violation, then we have to correct them by an RB-
INSERT-FIXUP procedure.
RB-INSERT-FIXUP (T, z)
1. while color [p[z]] = RED
2. do if p [z] = left [p[p[z]]]
3. then y ← right [p[p[z]]]
4. If color [y] = RED
5. then color [p[z]] ← BLACK //Case 1
6. color [y] ← BLACK //Case 1
7. color [p[z]] ← RED //Case 1
8. z ← p[p[z]] //Case 1
9. else if z= right [p[z]]
10. then z ← p [z] //Case 2
11. LEFT-ROTATE (T, z) //Case 2
12. color [p[z]] ← BLACK //Case 3
13. color [p [p[z]]] ← RED //Case 3
14. RIGHT-ROTATE (T,p [p[z]]) //Case 3
15. else (same as then clause)
With "right" and "left" exchanged
16. color [root[T]] ← BLACK
Example
Show the red-black trees that result after successively inserting the keys
41,38,31,12,19,8 into an initially empty red-black tree.
Solution
Insert 41
Insert 19
Thus the final tree is

3. Deletion
First, search for an element to be deleted
If the element to be deleted is in a node with only left child,

swap this node with one containing the largest element in the
left subtree. (This node has no right child).
If the element to be deleted is in a node with only right child,
swap this node with the one containing the smallest element in
the right subtree (This node has no left child).
If the element to be deleted is in a node with both a left child
and a right child, then swap in any of the above two ways.
While swapping, swap only the keys but not the colors.
The item to be deleted is now having only a left child or only a
right child. Replace this node with its sole child. This may
violate red constraints or black constraint. Violation of red
constraints can be easily fixed.
If the deleted node is black, the black constraint is violated. The
elimination of a black node y causes any path that contained y
to have one fewer black node.
Two cases arise:
The replacing node is red, in which case we merely
color it black to make up for the loss of one black
node.
The replacing node is black.
The strategy RB-DELETE is a minor change of the TREE-DELETE

procedure. After splicing out a node, it calls an auxiliary procedure RB-
DELETE-FIXUP that changes colors and performs rotation to restore the red-
black properties.
RB-DELETE (T, z)
1. if left [z] = nil [T] or right [z] = nil [T]
2. then y ← z
3. else y ← TREE-SUCCESSOR (z)
4. if left [y] ≠ nil [T]
5. then x ← left [y]
6. else x ← right [y]
7. p [x] ← p [y]
8. if p[y] = nil [T]
9. then root [T] ← x
10. else if y = left [p[y]]
11. then left [p[y]] ← x
12. else right [p[y]] ← x
13. if y≠ z
14. then key [z] ← key [y]
15. copy y's satellite data into z
16. if color [y] = BLACK
17. then RB-delete-FIXUP (T, x)
18. return y
RB-DELETE-FIXUP (T, x)
1. while x ≠ root [T] and color [x] = BLACK
2. do if x = left [p[x]]
3. then w ← right [p[x]]
4. if color [w] = RED
5. then color [w] ← BLACK //Case 1
6. color [p[x]] ← RED //Case 1
7. LEFT-ROTATE (T, p [x]) //Case 1
8. w ← right [p[x]] //Case 1
9. If color [left [w]] = BLACK and color [right[w]] = BLACK
10. then color [w] ← RED //Case 2
11. x ← p[x] //Case 2
12. else if color [right [w]] = BLACK
13. then color [left[w]] ← BLACK //Case 3
14. color [w] ← RED //Case 3
15. RIGHT-ROTATE (T, w) //Case 3
16. w ← right [p[x]] //Case 3
17. color [w] ← color [p[x]] //Case 4
18. color p[x] ← BLACK //Case 4
19. color [right [w]] ← BLACK //Case 4
20. LEFT-ROTATE (T, p [x]) //Case 4
21. x ← root [T] //Case 4
22. else (same as then clause with "right" and "left" exchanged)
23. color [x] ← BLACK
Example
In a previous example, we found that the red-black tree that results from
successively inserting the keys 41,38,31,12,19,8 into an initially empty tree.
Now show the red-black trees that result from the successful deletion of the
keys in the order 8, 12, 19,31,38,41.
Solution
Delete 38
Delete 41
No Tree.
Dynamic Programming
Dynamic Programming is the most powerful design technique for solving
optimization problems.
Divide & Conquer algorithm partition the problem into disjoint subproblems
solve the subproblems recursively and then combine their solution to solve
the original problems.
Dynamic Programming is used when the subproblems are not independent,
e.g. when they share the same subproblems. In this case, divide and conquer
may do more work than necessary, because it solves the same sub problem
multiple times.
Dynamic Programming solves each subproblems just once and stores the
result in a table so that it can be repeatedly retrieved if needed again.
Dynamic Programming is a Bottom-up approach- we solve all possible small
problems and then combine to obtain solutions for bigger problems.
Dynamic Programming is a paradigm of algorithm design in which an
optimization problem is solved by a combination of achieving sub-problem
solutions and appearing to the "principle of optimality".
Characteristics of Dynamic Programming
Dynamic Programming works when a problem has the following features:-
Optimal Substructure: If an optimal solution contains optimal

sub solutions then a problem exhibits optimal substructure.
Overlapping subproblems: When a recursive algorithm would
visit the same subproblems repeatedly, then a problem has
overlapping subproblems.
If a problem has optimal substructure, then we can recursively define an
optimal solution. If a problem has overlapping subproblems, then we can
improve on a recursive implementation by computing each subproblem only
once.
If a problem doesn't have optimal substructure, there is no basis for defining a
recursive algorithm to find the optimal solutions. If a problem doesn't have
overlapping sub problems, we don't have anything to gain by using dynamic
programming.
If the space of subproblems is enough (i.e. polynomial in the size of the
input), dynamic programming can be much more efficient than recursion.
Elements of Dynamic Programming
There are basically three elements that characterize a dynamic programming
algorithm:-
1. Substructure: Decompose the given problem into smaller

subproblems. Express the solution of the original problem in
terms of the solution for smaller problems.
2. Table Structure: After solving the sub-problems, store the
results to the sub problems in a table. This is done because
subproblem solutions are reused many times, and we do not
want to repeatedly solve the same problem over and over again.
3. Bottom-up Computation: Using table, combine the solution of
smaller subproblems to solve larger subproblems and
eventually arrives at a solution to complete problem.
Note: Bottom-up means:-
1. Start with smallest subproblems.

2. Combining their solutions obtain the solution to sub-problems
of increasing size.
3. Until solving at the solution of the original problem.
Components of Dynamic programming
1. Stages: The problem can be divided into several subproblems,

which are called stages. A stage is a small portion of a given
problem. For example, in the shortest path problem, they were
defined by the structure of the graph.
2. States: Each stage has several states associated with it. The
states for the shortest path problem was the node reached.
3. Decision: At each stage, there can be multiple choices out of
which one of the best decisions should be taken. The decision
taken at every stage should be optimal; this is called a stage
decision.
4. Optimal policy: It is a rule which determines the decision at
each stage; a policy is called an optimal policy if it is globally
optimal. This is known as Bellman principle of optimality.
5. Given the current state, the optimal choices for each of the
remaining states does not depend on the previous states or
decisions. In the shortest path problem, it was not necessary to
know how we got a node only that we did.
6. There exist a recursive relationship that identify the optimal
decisions for stage j, given that stage j+1, has already been
solved.
7. The final stage must be solved by itself.
Development of Dynamic Programming Algorithm

It can be broken into four steps:
1. Characterize the structure of an optimal solution.

2. Recursively defined the value of the optimal solution. Like
Divide and Conquer, divide the problem into two or more
optimal parts recursively. This helps to determine what the
solution will look like.
3. Compute the value of the optimal solution from the bottom up
(starting with the smallest subproblems)
4. Construct the optimal solution for the entire problem form the
computed values of smaller subproblems.
Applications of Dynamic Programming
1. 0/1 knapsack problem

2. Mathematical optimization problem
3. All pair Shortest path problem
4. Reliability design problem
5. Longest common subsequence (LCS)
6. Flight control and robotics control
7. Time-sharing: It schedules the job to maximize CPU usage
Divide and Conquer Method vs. Dynamic Programming

Divide & Conquer Method Dynamic Programming
1.It deals (involves) three steps at 1.It involves the sequence of four
each level of recursion: steps:
Divide the problem into a number Characterize the structure of optimal
of subproblems. solutions.
Conquer the subproblems by Recursively defines the values of
solving them recursively. optimal solutions.
Combine the solution to the Compute the value of optimal
subproblems into the solution for solutions in a Bottom-up minimum.
original subproblems. Construct an Optimal Solution from
computed information.
2. It is Recursive. 2. It is non Recursive.
3. It does more work on 3. It solves subproblems only once
subproblems and hence has more and then stores in the table.
time consumption.
4. It is a top-down approach. 4. It is a Bottom-up approach.
5. In this subproblems are 5. In this subproblems are
independent of each other. interdependent.
6. For example: Merge Sort & 6. For example: Matrix
Binary Search etc. Multiplication.
Fibonacci Sequence
Fibonacci sequence is the sequence of numbers in which every next item is
the total of the previous two items. And each number of the Fibonacci
sequence is called Fibonacci number.
Example
0 ,1,1,2,3,5,8,13,21,....................... is a Fibonacci sequence.
The Fibonacci numbers F_nare defined as follows:
F0 = 0
Fn=1
Fn=F(n-1)+ F(n-2)
FIB (n)
1. If (n < 2)
2. then return n
3. else return FIB (n - 1) + FIB (n - 2)
The Figure shows four levels of recursion for the call fib (8):
Recursive calls during computation of Fibonacci number

A single recursive call to fib (n) results in one recursive call to fib (n - 1), two
recursive calls to fib (n - 2), three recursive calls to fib (n - 3), five recursive
calls to fib (n - 4) and, in general, Fk-1 recursive calls to fib (n - k) We can
avoid this unneeded repetition by writing down the conclusion of recursive
calls and looking them up again if we need them later. This process is called
memorization.
Here is the algorithm with memorization
MEMOFIB (n)
1 if (n < 2)
2 then return n
3 if (F[n] is undefined)
4 then F[n] ← MEMOFIB (n - 1) + MEMOFIB (n - 2)
5 return F[n]
If we trace through the recursive calls to MEMOFIB, we find that array F []
gets filled from bottom up. I.e., first F [2], then F [3], and so on, up to F[n].
We can replace recursion with a simple for-loop that just fills up the array F
[] in that order
ITERFIB (n)
1 F [0] ← 0
2 F [1] ← 1
3 for i ← 2 to n
4 do
5 F[i] ← F [i - 1] + F [i - 2]
6 return F[n]
This algorithm clearly takes only O (n) time to compute Fn. By contrast, the
original recursive algorithm takes O ( ∅ n;), ∅ = = 1.618. ITERFIB

conclude an exponential speedup over the original recursive algorithm.
Matrix Chain Multiplication
It is a Method under Dynamic Programming in which previous output is
taken as input for next.
Here, Chain means one matrix's column is equal to the second matrix's row
[always].
In general:
If A = ⌊ aij ⌋ is a p x q matrix
B = ⌊ bij ⌋ is a q x r matrix
C = ⌊ cij ⌋ is a p x r matrix
Then
Given following matrices {A1,A2,A3,...An} and we have to perform the matrix
multiplication, which can be accomplished by a series of matrix
multiplications
A1 xA2 x,A3 x.....x An
Matrix Multiplication operation is associative in nature rather commutative.

By this, we mean that we have to follow the above matrix order for
multiplication but we are free to parenthesize the above multiplication
depending upon our need.
In general, for 1≤ i≤ p and 1≤ j ≤ r
It can be observed that the total entries in matrix 'C' is 'pr' as the matrix is of
dimension p x r Also each entry takes O (q) times to compute, thus the total
time to compute all possible entries for the matrix 'C' which is a
multiplication of 'A' and 'B' is proportional to the product of the dimension p
q r.
It is also noticed that we can save the number of operations by reordering the
parenthesis.
Example
Let us have 3 matrices, A1,A2,A3 of order (10 x 100), (100 x 5) and (5 x 50)
respectively.
Three Matrices can be multiplied in two ways:
1. A1,(A2,A3): First multiplying(A2 and A3) then multiplying and

resultant withA1.
2. (A1,A2),A3: First multiplying(A1 and A2) then multiplying and
resultant withA3.
No of Scalar multiplication in Case 1 will be:

(100 x 5 x 50) + (10 x 100 x 50) = 25000 + 50000 = 75000
No of Scalar multiplication in Case 2 will be:
(100 x 10 x 5) + (10 x 5 x 50) = 5000 + 2500 = 7500
To find the best possible way to calculate the product, we could simply
parenthesis the expression in every possible fashion and count each time how
many scalar multiplication are required.
Matrix Chain Multiplication Problem can be stated as "find the optimal
parenthesization of a chain of matrices to be multiplied such that the number
of scalar multiplication is minimized".
Number of ways for parenthesizing the matrices:
There are very large numbers of ways of parenthesizing these matrices. If
there are n items, there are (n-1) ways in which the outer most pair of
parenthesis can place.
(A1) (A2,A3,A4,................An)
Or (A1,A2) (A3,A4 .................An)
Or (A1,A2,A3) (A4 ...............An)
........................
Or(A1,A2,A3.............An-1) (An)
It can be observed that after splitting the kth matrices, we are left with two
parenthesized sequence of matrices: one consist 'k' matrices and another
consist 'n-k' matrices.
Now there are 'L' ways of parenthesizing the left sublist and 'R' ways of
parenthesizing the right sublist then the Total will be L.R:
Also p (n) = c (n-1) where c (n) is the nth Catalon number
c (n) =
On applying Stirling's formula we have
c (n) = Ω
Which shows that 4n grows faster, as it is an exponential function, then n1.5.
Development of Dynamic Programming Algorithm
1. Characterize the structure of an optimal solution.

2. Define the value of an optimal solution recursively.
3. Compute the value of an optimal solution in a bottom-up
fashion.
4. Construct the optimal solution from the computed information.
Dynamic Programming Approach

Let Ai,j be the result of multiplying matrices i through j. It can be seen that the
dimension of Ai,j is pi-1 x pj matrix.
Dynamic Programming solution involves breaking up the problems into

subproblems whose solution can be combined to solve the global problem.
At the greatest level of parenthesization, we multiply two matrices
A1.....n=A1....k x Ak+1....n)
Thus we are left with two questions:
How to split the sequence of matrices?

How to parenthesize the subsequence A1.....k andAk+1......n?
One possible answer to the first question for finding the best value of 'k' is to
check all possible choices of 'k' and consider the best among them. But that it
can be observed that checking all possibilities will lead to an exponential
number of total possibilities. It can also be noticed that there exists only O
(n2 ) different sequence of matrices, in this way do not reach the exponential
growth.
Step 1: Structure of an optimal parenthesization
Our first step in the dynamic paradigm is to find the optimal substructure and
then use it to construct an optimal solution to the problem from an optimal
solution to subproblems.
Let Ai....j where i≤ j denotes the matrix that results from evaluating the product
Ai Ai+1....Aj.
If i < j then any parenthesization of the product Ai Ai+1 ......Aj must split that
the product between Ak and Ak+1 for some integer k in the range i ≤ k ≤ j.
That is for some value of k, we first compute the matrices Ai.....k & Ak+1....j and
then multiply them together to produce the final product Ai....j. The cost of
computing Ai....k plus the cost of computing Ak+1....j plus the cost of multiplying
them together is the cost of parenthesization.
Step 2: A Recursive Solution
Let m [i, j] be the minimum number of scalar multiplication needed to
compute the matrixAi....j.
If i=j the chain consist of just one matrix Ai....i=Ai so no scalar multiplication
are necessary to compute the product. Thus m [i, j] = 0 for i= 1, 2, 3....n.
If i<j we assume that to optimally parenthesize the product we split it
between Ak and Ak+1 where i≤ k ≤j. Then m [i,j] equals the minimum cost for
computing the subproducts Ai....k and Ak+1....j+ cost of multiplying them
together. We know Ai has dimension pi-1 x pi, so computing the product
Ai....k and Ak+1....jtakes pi-1 pk pj scalar multiplication, we obtain
m [i,j] = m [i, k] + m [k + 1, j] + pi-1 pk pj
There are only (j-1) possible values for 'k' namely k = i, i+1.....j-1. Since the
optimal parenthesization must use one of these values for 'k' we need only
check them all to find the best.
So the minimum cost of parenthesizing the product Ai Ai+1......Aj becomes
To construct an optimal solution, let us define s [i,j] to be the value of 'k' at

which we can split the product Ai Ai+1 .....Aj To obtain an optimal
parenthesization i.e. s [i, j] = k such that
m [i,j] = m [i, k] + m [k + 1, j] + pi-1 pk pj
Example of Matrix Chain Multiplication
We are given the sequence {4, 10, 3, 12, 20, and 7}. The matrices have size 4
x 10, 10 x 3, 3 x 12, 12 x 20, 20 x 7. We need to compute M [i,j], 0 ≤ i, j≤ 5.
We know M [i, i] = 0 for all i.
Let us proceed with working away from the diagonal. We compute the
optimal solution for the product of 2 matrices.
Here P0 to P5 are Position and M1 to M5 are matrix of size (pi to pi-1)

On the basis of sequence, we make a formula
In Dynamic Programming, initialization of every method done by '0'.So we

initialize it by '0'.It will sort out diagonally.
We have to sort out all the combination but the minimum output combination
is taken into consideration.
Calculation of Product of 2 matrices:
1. m (1,2) = m1 x m2
= 4 x 10 x 10 x 3
= 4 x 10 x 3 = 120
2. m (2, 3) = m2 x m3
= 10 x 3 x 3 x 12
= 10 x 3 x 12 = 360
3. m (3, 4) = m3 x m4
= 3 x 12 x 12 x 20
= 3 x 12 x 20 = 720
4. m (4,5) = m4 x m5
= 12 x 20 x 20 x 7
= 12 x 20 x 7 = 1680
We initialize the diagonal element with equal i,j value with '0'.
After that second diagonal is sorted out and we get all the
values corresponded to it
Now the third diagonal will be solved out in the same way.
Now product of 3 matrices:
M [1, 3] = M1 M2 M3
1. There are two cases by which we can solve this multiplication:
( M1 x M2) + M3, M1+ (M2x M3)
2. After solving both cases we choose the case in which minimum
output is there.
M [1, 3] =264
As Comparing both output 264 is minimum in both cases so we insert 264 in
table and ( M1 x M2) + M3 this combination is chosen for the output making.
M [2, 4] = M2 M3 M4
There are two cases by which we can solve this multiplication: (M2x M3)+M4,
M2+(M3 x M4)
After solving both cases we choose the case in which minimum output is
there.
M [2, 4] = 1320
As Comparing both output 1320 is minimum in both cases so we
insert 1320 in table and M2+(M3 x M4) this combination is chosen for the
output making.
M [3, 5] = M3 M4 M5
There are two cases by which we can solve this multiplication:
( M3 x M4) + M5, M3+ ( M4xM5)
After solving both cases we choose the case in which minimum output is
there.
M [3, 5] = 1140
As Comparing both output 1140 is minimum in both cases so we
insert 1140 in table and ( M3 x M4) + M5this combination is chosen for the
output making.
Now Product of 4 matrices:

M [1, 4] = M1 M2 M3 M4
There are three cases by which we can solve this multiplication:
( M1 x M2 x M3) M4
M1 x(M2 x M3 x M4)
(M1 xM2) x ( M3 x M4)
After solving these cases we choose the case in which minimum output is
there
M [1, 4] =1080
As comparing the output of different cases then '1080' is minimum output, so
we insert 1080 in the table and (M1 xM2) x (M3 x M4) combination is taken
out in output making,
M [2, 5] = M2 M3 M4 M5
There are three cases by which we can solve this multiplication:
(M2 x M3 x M4)x M5
M2 x( M3 x M4 x M5)
(M2 x M3)x ( M4 x M5)
there
M [2, 5] = 1350
As comparing the output of different cases then '1350' is minimum output, so
we insert 1350 in the table and M2 x( M3 x M4 xM5)combination is taken out
in output making.
Now Product of 5 matrices:
M [1, 5] = M1 M2 M3 M4 M5
There are five cases by which we can solve this multiplication:

(M1 x M2 xM3 x M4 )x M5
M1 x( M2 xM3 x M4 xM5)
(M1 x M2 xM3)x M4 xM5
M1 x M2x(M3 x M4 xM5)
there
M [1, 5] = 1344
As comparing the output of differrent cases then '1344' is minimum output,
so we insert 1344 in the table and M1 x M2 x(M3 x M4 x M5)combination is
taken out in output making.
Final Output is:
Computing Optimal Costs: let us assume that matrix Ai has dimension pi-1x
pi for i=1, 2, 3....n. The input is a sequence (p0,p1,......pn) where length [p] =
n+1. The procedure uses an auxiliary table m [1....n, 1.....n] for storing m [i, j]
costs an auxiliary table s [1.....n, 1.....n] that record which index of k achieved
the optimal costs in computing m [i, j].
The algorithm first computes m [i, j] ← 0 for i=1, 2, 3.....n, the minimum
costs for the chain of length 1.
Algorithm of Matrix Chain Multiplication
MATRIX-CHAIN-ORDER (p)
1. n length[p]-1
2. for i ← 1 to n
3. do m [i, i] ← 0
4. for l ← 2 to n // l is the chain length
5. do for i ← 1 to n-l + 1
6. do j ← i+ l -1
7. m[i,j] ← ∞
8. for k ← i to j-1
9. do q ← m [i, k] + m [k + 1, j] + pi-1 pk pj
10. If q < m [i,j]
11. then m [i,j] ← q
12. s [i,j] ← k
13. return m and s.
We will use table s to construct an optimal solution.
Step 1: Constructing an Optimal Solution
PRINT-OPTIMAL-PARENS (s, i, j)
1. if i=j
2. then print "A"
3. else print "("
4. PRINT-OPTIMAL-PARENS (s, i, s [i, j])
5. PRINT-OPTIMAL-PARENS (s, s [i, j] + 1, j)
6. print ")"
Analysis: There are three nested loops. Each loop executes a maximum n
times.
1. l, length, O (n) iterations.
2. i, start, O (n) iterations.
3. k, split point, O (n) iterations
Body of loop constant complexity

Total Complexity is: O (n3)
Question
P [7, 1, 5, 4, 2}
Solution
Here, P is the array of a dimension of matrices.
So here we will have 4 matrices:
A17x1 A21x5 A35x4 A44x2
i.e.
First Matrix A1 have dimension 7 x 1
Second Matrix A2 have dimension 1 x 5
Third Matrix A3 have dimension 5 x 4
Fourth Matrix A4 have dimension 4 x 2
Let say,
From P = {7, 1, 5, 4, 2}
(Given)
And P is the Position
p0 = 7, p1 =1,p2 = 5, p3 = 4, p4=2.
Length of array P = number of elements in P
∴ length (p)= 5
From step 3
Follow the steps in Algorithm in Sequence
According to Step 1 of Algorithm Matrix-Chain-Order
Step 1:
n ← length [p]-1
Where n is the total number of elements
And length [p] = 5
∴ n=5-1=4
n=4
Now we construct two tables m and s.
Table m has dimension [1.....n, 1.......n]
Table s has dimension [1.....n-1, 2.......n]
Now, according to step 2 of Algorithm

for i ← 1 to n
this means: for i ← 1 to 4 (because n =4)
for i=1
m [i, i]=0
m [1, 1]=0
Similarly for i = 2, 3, 4
m [2, 2] = m [3,3] = m [4,4] = 0
i.e. fill all the diagonal entries "0" in the table m
Now,
l ← 2 to n
l ← 2 to 4 (because n =4 )
Case 1
1. When l - 2
for (i ← 1 to n - l + 1)
i ← 1 to 4 - 2 + 1
i ← 1 to 3
When i = 1
do j ← i + l - 1
j←1+2-1
j←2
i.e. j = 2
Now, m [i, j] ← ∞
i.e. m [1,2] ← ∞
Put ∞ in m [1, 2] table
for k ← i to j-1
k ← 1 to 2 - 1
k ← 1 to 1
k=1
Now q ← m [i, k] + m [k + 1, j] + pi-1 pk pj
for l = 2
i=1
j =2
k=1
q ← m [1,1] + m [2,2] + p0x p1x p2
and m [1,1] = 0
for i ← 1 to 4
∴ q←0+0+7x1x5
q ← 35
We have m [i, j] = m [1, 2] = ∞
Comparing q with m [1, 2]
q < m [i, j]
i.e. 35 < m [1, 2]
35 < ∞
True
then, m [1, 2 ] ← 35 ( ∴ m [i,j] ← q)
s [1, 2] ← k
and the value of k = 1
s [1,2 ] ← 1
Insert "1" at dimension s [1, 2] in table s. And 35 at m [1, 2]
2. l remains 2
L=2
i ← 1 to n - l + 1
i ← 1 to 4 - 2 + 1
i ← 1 to 3
for i = 1 done before
Now value of i becomes 2
i=2
j← i+l-1
j← 2+2-1
j← 3
j=3
m [i , j] ← ∞
i.e. m [2,3] ← ∞
Initially insert ∞ at m [2, 3]
Now, for k ← i to j - 1
k ← 2 to 3 - 1
k ← 2 to 2
i.e. k =2
Now, q ← m [i, k] + m [k + 1, j] + pi-1 pk pj
For l =2
i=2
j=3
k=2
q ← m [2, 2] + m [3, 3] + p1x p2 x p3
q←0+0+1x5x4
q ← 20
Compare q with m [i ,j ]
If q < m [i,j]
i.e. 20 < m [2, 3]
20 < ∞
True
Then m [i,j ] ← q
m [2, 3 ] ← 20
and s [2, 3] ← k
and k = 2
s [2,3] ← 2
3. Now i become 3
i=3
l=2
j←i+l-1
j←3+2-1
j←4
j=4
Now, m [i, j ] ← ∞
m [3,4 ] ← ∞
Insert ∞ at m [3, 4]
for k ← i to j - 1
k ← 3 to 4 - 1
k ← 3 to 3
i.e. k = 3
Now, q ← m [i, k] + m [k + 1, j] + pi-1 pk pj
i=3
l=2
j=4
k=3
q ← m [3, 3] + m [4,4] + p2 x p3 x p4
q←0+0+5x2x4
q 40
Compare q with m [i, j]
If q < m [i, j]
40 < m [3, 4]
40 < ∞
True
Then, m [i,j] ← q
m [3,4] ← 40
and s [3,4] ← k
s [3,4] ← 3
Case 2: l becomes 3
L=3
for i = 1 to n - l + 1
i = 1 to 4 - 3 + 1
i = 1 to 2
When i = 1
j←i+l-1
j←1+3-1
j←3
j=3
Now, m [i,j] ← ∞
m [1, 3] ← ∞
for k ← i to j - 1
k ← 1 to 3 - 1
k ← 1 to 2
Now we compare the value for both k=1 and k = 2. The minimum of two will
be placed in m [i,j] or s [i,j] respectively.
Now from above

Value of q become minimum for k=1
∴ m [i,j] ← q
m [1,3] ← 48
Also m [i,j] > q
i.e. 48 < ∞
∴ m [i , j] ← q
m [i, j] ← 48
and s [i,j] ← k
i.e. m [1,3] ← 48
s [1, 3] ← 1
Now i become 2
i=2
l=3
then j ← i + l -1
j← 2+3-1
j←4
j=4
so m [i,j] ← ∞
m [2,4] ← ∞
Insert initially ∞ at m [2, 4]
for k ← i to j-1
k ← 2 to 4 - 1
k ← 2 to 3
Here, also find the minimum value of m [i,j] for two values of k = 2 and k =3
But 28 < ∞
So m [i,j] ← q
And q ← 28
m [2, 4] ← 28
and s [2, 4] ← 3
e. It means in s table at s [2,4] insert 3 and at m [2,4] insert 28.
Case 3: l becomes 4
L=4
For i ← 1 to n-l + 1
i ← 1 to 4 - 4 + 1
i←1
i=1
do j ← i + l - 1
j←1+4-1
j←4
j=4
Now m [i,j] ← ∞
m [1,4] ← ∞
for k ← i to j -1
k ← 1 to 4 - 1
k ← 1 to 3
When k = 1
q ← m [i, k] + m [k + 1, j] + pi-1 pk pj
q ← m [1,1] + m [2,4] + p0xp4x p1
q ← 0 + 28 + 7 x 2 x 1
q ← 42
Compare q and m [i, j]
m [i,j] was ∞
i.e. m [1,4]
if q < m [1,4]
42< ∞
True
Then m [i,j] ← q
m [1,4] ← 42
and s [1,4] 1 ? k =1
When k = 2
L = 4, i=1, j = 4
q ← m [i, k] + m [k + 1, j] + pi-1 pk pj
q ← m [1, 2] + m [3,4] + p0 xp2 xp4
q ← 35 + 40 + 7 x 5 x 2
q ← 145
Compare q and m [i,j]
Now m [i, j]
i.e. m [1,4] contains 42.
So if q < m [1, 4]
But 145 less than or not equal to m [1, 4]
So 145 less than or not equal to 42.
So no change occurs.
When k = 3
l=4
i=1
j=4
q ← m [i, k] + m [k + 1, j] + pi-1 pk pj
q ← m [1, 3] + m [4,4] + p0 xp3 x p4
q ← 48 + 0 + 7 x 4 x 2
q ← 114
Again q less than or not equal to m [i, j]
i.e. 114 less than or not equal to m [1, 4]
114 less than or not equal to 42
So no change occurs. So the value of m [1, 4] remains 42. And value of s [1,
4] = 1
Now we will make use of only s table to get an optimal solution.
Longest Common Sequence (LCS)
A subsequence of a given sequence is just the given sequence with some
elements left out.
Given two sequences X and Y, we say that the sequence Z is a common
sequence of X and Y if Z is a subsequence of both X and Y.
In the longest common subsequence problem, we are given two sequences X
= (x1 x2....xm) and Y = (y1 y2 yn) and wish to find a maximum length common
subsequence of X and Y. LCS Problem can be solved using dynamic
programming.
Characteristics of Longest Common Sequence
A brute-force approach we find all the subsequences of X and check each
subsequence to see if it is also a subsequence of Y, this approach requires
exponential time making it impractical for the long sequence.
Given a sequence X = (x1 x2.....xm) we define the ith prefix of X for i=0, 1,
and 2...m as Xi= (x1 x2.....xi). For example: if X = (A, B, C, B, C, A, B, C)
then X4= (A, B, C, B)
Optimal Substructure of an LCS: Let X = (x1 x2....xm) and Y = (y1 y2.....) yn)
be the sequences and let Z = (z1 z2......zk) be any LCS of X and Y.
If xm = yn, then zk=x_m=yn and Zk-1 is an LCS of Xm-1and Yn-1
If xm ≠ yn, then zk≠ xm implies that Z is an LCS of Xm-1and Y.
If xm ≠ yn, then zk≠yn implies that Z is an LCS of X and Yn-1
Step 2: Recursive Solution: LCS has overlapping subproblems property
because to find LCS of X and Y, we may need to find the LCS of Xm-1 and
Yn-1. If xm ≠ yn, then we must solve two subproblems finding an LCS of X
and Yn-1.Whenever of these LCS's longer is an LCS of x and y. But each of
these subproblems has the subproblems of finding the LCS of Xm-1 and Yn-1.
Let c [i,j] be the length of LCS of the sequence Xiand Yj.If either i=0 and j
=0, one of the sequences has length 0, so the LCS has length 0. The optimal
substructure of the LCS problem given the recurrence formula
Step 3: Computing the length of an LCS: let two sequences X = (x1 x2.....xm)
and Y = (y1 y2..... yn) as inputs. It stores the c [i,j] values in the table c
[0......m,0..........n].Table b [1..........m, 1..........n] is maintained which help us
to construct an optimal solution. c [m, n] contains the length of an LCS of
X,Y.
Algorithm of Longest Common Sequence
LCS-LENGTH (X, Y)
1. m ← length [X]
2. n ← length [Y]
3. for i ← 1 to m
4. do c [i,0] ← 0
5. for j ← 0 to m
6. do c [0,j] ← 0
7. for i ← 1 to m
8. do for j ← 1 to n
9. do if xi= yj
10. then c [i,j] ← c [i-1,j-1] + 1
11. b [i,j] ← "↖"
12. else if c[i-1,j] ≥ c[i,j-1]
13. then c [i,j] ← c [i-1,j]
14. b [i,j] ← "↑"
15. else c [i,j] ← c [i,j-1]
16. b [i,j] ← "← "
17. return c and b.
Example of Longest Common Sequence
Given two sequences X [1...m] and Y [1.....n]. Find the longest common
subsequences to both.
here X = (A,B,C,B,D,A,B) and Y = (B,D,C,A,B,A)

m = length [X] and n = length [Y]
m = 7 and n = 6
Here x1= x [1] = A y1= y [1] = B
x2= B y2= D
x3= C y3= C
x4= B y4= A
x5= D y5= B
x6= A y6= A
x7= B
Now fill the values of c [i, j] in m x n table

Initially, for i=1 to 7 c [i, 0] = 0
For j = 0 to 6 c [0, j] = 0
That is:
Now for i=1 and j = 1

x1 and y1 we get x1 ≠ y1 i.e. A ≠ B
And c [i-1,j] = c [0, 1] = 0
c [i, j-1] = c [1,0 ] = 0
That is, c [i-1,j]= c [i, j-1] so c [1, 1] = 0 and b [1, 1] = ' ↑ '
x1 and y2 we get x1 ≠ y2 i.e. A ≠ D
c [i-1,j] = c [0, 2] = 0
c [i, j-1] = c [1,1 ] = 0
That is, c [i-1,j]= c [i, j-1] and c [1, 2] = 0 b [1, 2] = ' ↑ '
x1 and y3 we get x1 ≠ y3 i.e. A ≠ C
c [i-1,j] = c [0, 3] = 0
c [i, j-1] = c [1,2 ] = 0
so c [1,3] = 0 b [1,3] = ' ↑ '
x1 and y4 we get. x1=y4 i.e A = A
c [1,4] = c [1-1,4-1] + 1
= c [0, 3] + 1
=0+1=1
c [1,4] = 1
b [1,4] = ' ↖ '
x1 and y5 we get x1 ≠ y5
c [i-1,j] = c [0, 5] = 0
c [i, j-1] = c [1,4 ] = 1
Thus c [i, j-1] > c [i-1,j] i.e. c [1, 5] = c [i, j-1] = 1. So b [1, 5] = '←'
x1 and y6 we get x1=y6
c [1, 6] = c [1-1,6-1] + 1
= c [0, 5] + 1 = 0 + 1 = 1
c [1,6] = 1
b [1,6] = ' ↖ '
We get x2 and y1 B = B i.e. x2= y1
c [2,1] = c [2-1,1-1] + 1
= c [1, 0] + 1
=0+1=1
c [2, 1] = 1 and b [2, 1] = ' ↖ '
Similarly, we fill the all values of c [i, j] and we get
Step 4: Constructing an LCS: The initial call is PRINT-LCS (b, X, X.length,
Y.length)
PRINT-LCS (b, x, i, j)
1. if i=0 or j=0
2. then return
3. if b [i,j] = ' ↖ '
4. then PRINT-LCS (b,x,i-1,j-1)
5. print x_i
6. else if b [i,j] = ' ↑ '
7. then PRINT-LCS (b,X,i-1,j)
8. else PRINT-LCS (b,X,i,j-1)
Example
Determine the LCS of (1,0,0,1,0,1,0,1) and (0,1,0,1,1,0,1,1,0).
Solution
let X = (1,0,0,1,0,1,0,1) and Y = (0,1,0,1,1,0,1,1,0).
We are looking for c [8, 9]. The following table is built.
From the table we can deduct that LCS = 6. There are several such
sequences, for instance (1,0,0,1,1,0) (0,1,0,1,0,1) and (0,0,1,1,0,1)

Knapsack is basically means bag. A bag of given capacity.
We want to pack n items in your luggage.
The ith item is worth vi dollars and weight wi pounds.

Take as valuable a load as possible, but cannot exceed W
pounds.
vi wi W are integers.
W ≤ capacity
Value ← Max
Input
Knapsack of capacity
List (Array) of weight and their corresponding value.
Output: To maximize profit and minimize weight in capacity.

The knapsack problem where we have to pack the knapsack with maximum
value in such a manner that the total weight of the items should not be greater
than the capacity of the knapsack.
Knapsack problem can be further divided into two parts:
1. Fractional Knapsack: Fractional knapsack problem can be solved
by Greedy Strategy where as 0 /1 problem is not.
It cannot be solved by Dynamic Programming Approach.
In this item cannot be broken which means thief should take the item as a
whole or should leave it. That's why it is called 0/1 Knapsack Problem.
Each item is taken or not taken.

Cannot take a fractional amount of an item taken or take an
item more than once.
It cannot be solved by the Greedy Approach because it is enable
to fill the knapsack to capacity.
Greedy Approach doesn't ensure an Optimal Solution.
Example of 0/1 Knapsack Problem

Example: The maximum weight the knapsack can hold is W is 11. There are
five items to choose from. Their weights and values are presented in the
following table:
The [i, j] entry here will be V [i, j], the best value obtainable using the first
"i" rows of items if the maximum capacity were j. We begin by initialization
and first row.
V [i, j] = max {V [i - 1, j], vi + V [i - 1, j -wi]

The value of V [3, 7] was computed as follows:
V [3, 7] = max {V [3 - 1, 7], v3 + V [3 - 1, 7 - w3]
= max {V [2, 7], 18 + V [2, 7 - 5]}
= max {7, 18 + 6}
= 24
Finally, the output is:
The maximum value of items in the knapsack is 40, the bottom-right entry).
The dynamic programming approach can now be coded as the following
algorithm:
Algorithm of Knapsack Problem
KNAPSACK (n, W)
1. for w = 0, W
2. do V [0,w] ← 0
3. for i=0, n
4. do V [i, 0] ← 0
5. for w = 0, W
6. do if (wi≤ w & vi + V [i-1, w - wi]> V [i -1,W])
7. then V [i, W] ← vi + V [i - 1, w - wi]
8. else V [i, W] ← V [i - 1, w]
Dutch National Flag

Dutch National Flag (DNF) - It is a programming problem proposed by
Edsger Dijkstra. The flag of the Netherlands consists of three colors: white,
red, and blue. The task is to randomly arrange balls of white, red, and blue in
such a way that balls of the same color are placed together. For DNF (Dutch
National Flag), we sort an array of 0, 1, and 2's in linear time that does not
consume any extra space. We have to keep in mind that this algorithm can be
implemented only on an array that has three unique elements.
Algorithm
Take three-pointers, namely - low, mid, high.

We use low and mid pointers at the start, and the high pointer
will point at the end of the given array.
Cases
If array [mid] =0, then swap array [mid] with array [low] and
increment both pointers once.
If array [mid] = 1, then no swapping is required. Increment mid
pointer once.
If array [mid] = 2, then we swap array [mid] with array [high]
and decrement the high pointer once.
Code
C
#include<bits/stdc++.h>
using namespace std;
// Function to sort the input array where the array is assumed to have values
in {0, 1, 2}
// We have to take 3 distint or unique elements
void JTP(int arr[], int arr_size)
{
int low = 0;
int high = arr_size - 1;
int mid = 0;
// We have keep iterating till all the elements are sorted
while (mid <= high)
{
switch (arr[mid])
{
// Here mid is 0.
case 0:
swap(arr[low++], arr[mid++]);
break;
// Here mid is 1.
case 1:
mid++;
break;
// Here mid is 2.
case 2:
swap(arr[mid], arr[high--]);
break;
}
}
}
// Now, we write the function to print array arr[]
void printArray(int arr[], int arr_size)
{
// To iterate and print every element, we follow these steps
for (int i = 0; i < arr_size; i++)
cout << arr[i] << " ";
}
//Main Code
int main()
{
int arr[] = {0,1,0,1,2,0,1,2};
int n = sizeof(arr)/sizeof(arr[0]);
cout << "Array before executing the algorithm: ";
printArray(arr, n);
JTP(arr, n);
cout << "\nArray after executing the DNFS algorithm: ";
printArray(arr, n);
return 0;
}
Output
Array before executing the algorithm: 0 1 0 1 2 0 1 2
Array after executing the DNFS algorithm: 0 0 0 1 1 1 2 2
Java
import java.io.*;
class DNF {
static void JTP(int arr[], int arr_size)
{
int low = 0;
int high = arr_size - 1;
int mid = 0, temp=0; // We use temporary variable for swapping
while (mid <= high)
{
switch (arr[mid])
{
case 0: // Here mid pointer points is at 0.
{
temp = arr[low];
arr[low] = arr[mid];
arr[mid] = temp;
low++;
mid++;
break;
}
mid++;
break;
{
temp = arr[mid];
arr[mid] = arr[high];
arr[high] = temp;
high--;
break;
}
}
}
}
// Now we have to call function to print array arr[]
static void printArray(int arr[], int arr_size)
{
int i;
for (i = 0; i < arr_size; i++)
System.out.print(arr[i]+" ");
System.out.println("");
}
//Now we use driver function to check for above functions
public static void main (String[] arguments)
{
int arr[] = {0, 1, 0, 1, 2, 0, 1, 2};
int arr_size = arr.length;
System.out.println("Array before executing the DNFS algorithm : ");
printArray(arr, arr_size);
JTP(arr, arr_size);
System.out.println("\nArray after executing the DNFS algorithm : ");
printArray(arr, arr_size);
}
}
Output
Array before executing the DNFS algorithm : 0 1 0 1 2 0 1 2
Array after executing the DNFS algorithm : 0 0 0 1 1 1 2 2
Python
def JTP( arr, arr_size):
low = 0
high = arr_size - 1
mid = 0
while mid <= high:
if arr[mid] == 0:
arr[low],arr[mid] = arr[mid],arr[low]
low = low + 1
mid = mid + 1
elif arr[mid] == 1:
mid = mid + 1
else:
arr[mid],arr[high] = arr[high],arr[mid]
high = high - 1
return arr
# Function to print array
def printArray(arr):
for k in arr:
print k,
print
# Driver Program
arr = [0, 1, 0, 1, 1, 2, 0, 1, 2]
arr_size = len(arr)
print " Array before executing the algorithm: ",
printArray(arr)
arr = JTP(arr, arr_size)
print "Array after executing the DNFS algorithm: ",
printArray(arr)
Output
Array before executing the algorithm: 0 1 0 1 1 2 0 1 2
Array after executing the DNFS algorithm: 0 0 0 1 1 1 1 2 2
Longest Palindrome Subsequence

It is a sequence of characters in a string that can be spelled and read the same
both ways, forward and backward. Words like civic, redivider, deified, radar,
level, madam, rotor, refer, kayak, racecar, and reviver. But in palindromic
subsequence, a sequence can but not necessarily appear in the same relative
order, but the chance of being necessarily contiguous and palindromic in
nature is negligible.
Dynamic Programming Solution
Example - We have been given a sequence as "BDBADBDCBDCADB."
Then the longest palindrome will be eleven - "BDABDCDBADB", this is the
longest palindromic subsequence here. 'BBABB,' 'DAD,' 'BBDBB' and many
more are also palindromic subsequences of the given sequence, but they are
not the longest. In simpler words, sequences generate subsequences, then we
compare their length and find which palindromic subsequence is the longest.
We follow these steps to achieve the most extended palindrome sequence
using Dynamic Programming Solution.
First, we reverse the sequence.
Then we use the LCS algorithm (Longest Common Subsequence) to find the
longest common subsequence among the original sequence and reversed
sequence. Here original LCS and reverse LCS are a function that returns the
longest common subsequence between the pair of strings, now the answer
from LCS will be the longest palindromic subsequence.
Let LP (a, b) = Length of longest palindromic subsequence in array Z from
index a to b
LP (a, b) = LP (a+1, b-1) + 2: if Z [a] = X [b]
= max [LP (a+1, b), LP (a, b-1)]: if Z [a]! = Z [b]
= 1 if a = b
= 1 if a = b - 1 and Z [a]! = Z [b]
= 2 if a = b - 1 and Z [a] = Z [b]
Code
#include<iostream>
using namespace std;
int max (int a, int b) { return (a > b)? a : b;
}
int palSubseqLen(string str) {
int n = str.size();
int lenTable[n][n];
for (int i = 0; i < n; i++)
lenTable[i][i] = 1;
for (int col=2; col<=n; col++) { for (int i=0; i<n-col+1; i++) {
int j = i+col-1; if (str[i] == str[j] && col == 2)
lenTable[i][j] = 2;
else if (str[i] == str[j])
lenTable[i][j] = lenTable[i+1][j-1] + 2;
else
lenTable[i][j] = max(lenTable[i][j-1], lenTable[i+1][j]); }
} return lenTable[0][n-1];
}
int main() {
string sequence = "BDBADBDCBDCADB";
int n = sequence.size();
cout << "The length of the longest palindrome subsequence is: " <<
palSubseqLen(sequence);
}
Output
The length of the longest palindrome subsequence is: 11
Now, if we were to combine all the above cases into a mathematical equation:
We call the original sequence X = (x1x2 ... xm ) and reverse as Y = ( y1y2 ...
ym). Here, the prefixes of X are X1, 2, 3 ... m and the prefixes of Y are Y1, 2,
3 ... m.
Let LCS (XI, Yj) represent the set of the longest common subsequence of
prefixes Xi and Yj.
Then:
LCS (Xi , Yj) = ∅ ; if i = 0 or j = 0
LCS (Xi , Yj) = LCS(Xi-1, Yj-1) ^ xi ; if i > 0, j > 0 & xi = yj
LCS (Xi , Yj) = max{ LCS (Xi, Yj-1), LCS (Xi-1, Yj) } ; if i > 0, j > 0 & xi ≠
yj
If the last characters match, then the sequence LCS (Xi-1 , Yj-1) is extended
by that matching character xi. Otherwise, the best result from LCS (Xi , Yj-1)
and LCS (Xi-1 , Yj) is used.
In the recursive method, we compute some sub-problem, divide it, and
repeatedly perform this kind of task. So it's a simple but very tedious method.
The time complexity in recursive solution is more. The worst-case time
complexity is exponential O (2^n), and auxiliary space used by the program
is O (1).
In X, if the last and first characters are the same:-
X (0, n - 1) = X (1, n - 2) + 2
If not, then
X (0, n - 1) = MAX (X (1, n - 1), X (0, n - 2)).
Code
class Main
{
public static String longestPalindrome(String X, String Y, int m, int n, int[][]
T)
{
if (m == 0 || n == 0) {
return "";
}
if (X.charAt(m - 1) == Y.charAt(n - 1))
{
return longestPalindrome(X, Y, m - 1, n - 1, T) + X.charAt(m - 1);
}
if (T[m - 1][n] > T[m][n - 1]) {
return longestPalindrome(X, Y, m - 1, n, T);
}
return longestPalindrome(X, Y, m, n - 1, T);
}
public static int LCSLength(String X, String Y, int n, int[][] T)
{
for (int i = 1; i <= n; i++)
{
for (int j = 1; j <= n; j++)
{
if (X.charAt(i - 1) == Y.charAt(j - 1))
{
T[i][j] = T[i - 1][j - 1] + 1;
}
else
{
T[i][j] = Integer.max(T[i - 1][j], T[i][j - 1]);
}
}
}
return T[n][n];
}
public static void main(String[] args)
{
String X = "BDBADBDCBDCADB";
String Y = new StringBuilder(X).reverse().toString();
int[][] T = new int[X.length() + 1][X.length() + 1];
System.out.println("The length of Longest Palindromic Subsequence is "
+ LCSLength(X, Y, X.length(), T));
System.out.println("The Longest Palindromic Subsequence is "
+ longestPalindrome(X, Y, X.length(), X.length(), T));
}
}
Output
The length of the Longest Palindromic Subsequence is 11
The Longest Palindromic Subsequence is BDADBCBDADB
Now, if we were to combine all the above cases into a mathematical equation:
We call the original sequence X = (x1x2 ... xm ) and reverse as Y = ( y1y2 ...
ym). Here, the prefixes of X are X1, 2, 3 ... m and the prefixes of Y are Y1, 2,
3 ... m.
Let LCS (XI, Yj) represent the set of the longest common subsequence of
prefixes Xi and Yj.
Then:
LCS (Xi , Yj) = ∅ ; if i = 0 or j = 0
LCS (Xi , Yj) = LCS(Xi-1, Yj-1) ^ xi ; if i > 0, j > 0 & xi = yj
LCS (Xi , Yj) = max{ LCS (Xi, Yj-1), LCS (Xi-1, Yj) } ; if i > 0, j > 0 & xi ≠
yj
If the last characters match, then the sequence LCS (Xi-1 , Yj-1) is extended
by that matching character xi. Otherwise, the best result from LCS (Xi , Yj-1)
and LCS (Xi-1 , Yj) is used.
In the recursive method, we compute some sub-problem, divide it, and
repeatedly perform this kind of task. So it's a simple but very tedious method.
The time complexity in recursive solution is more. The worst-case time
complexity is exponential O (2^n), and auxiliary space used by the program
is O (1).
In X, if the last and first characters are the same -
X (0, n - 1) = X (1, n - 2) + 2
If not, then
X (0, n - 1) = MAX (X (1, n - 1), X (0, n - 2)).
Code
class Main
{
public static String longestPalindrome(String X, String Y, int m, int n, int[][]
T)
{
if (m == 0 || n == 0) {
return "";
}
if (X.charAt(m - 1) == Y.charAt(n - 1))
{
return longestPalindrome(X, Y, m - 1, n - 1, T) + X.charAt(m - 1);
}
if (T[m - 1][n] > T[m][n - 1]) {
return longestPalindrome(X, Y, m - 1, n, T);
}
return longestPalindrome(X, Y, m, n - 1, T);
}
public static int LCSLength(String X, String Y, int n, int[][] T)
{
for (int i = 1; i <= n; i++)
{
for (int j = 1; j <= n; j++)
{
if (X.charAt(i - 1) == Y.charAt(j - 1))
{
T[i][j] = T[i - 1][j - 1] + 1;
}
else
{
T[i][j] = Integer.max(T[i - 1][j], T[i][j - 1]);
}
}
}
return T[n][n];
}
public static void main(String[] args)
{
String X = "BDBADBDCBDCADB";
String Y = new StringBuilder(X).reverse().toString();
int[][] T = new int[X.length() + 1][X.length() + 1];
System.out.println("The length of Longest Palindromic Subsequence is "
+ LCSLength(X, Y, X.length(), T));
System.out.println("The Longest Palindromic Subsequence is "
+ longestPalindrome(X, Y, X.length(), X.length(), T));
}
}
Output
The length of the Longest Palindromic Subsequence is 11
The Longest Palindromic Subsequence is BDADBCBDADB
Optimal Substructure
It satisfies overlapping subproblem properties. In a two dimensional array,
Longest Common Subsequence can be made as a memo, where LCS[X][Y]
represents the length between original with length X and reverse, with length.
The longest palindromic subsequence can be generated by backtracking
technique, after filling and using the above algorithm.
Greedy Algorithm
"Greedy Method finds out of many options, but you have to choose the best
option."
In this method, we have to find out the best method/option out of many
present ways.
In this approach/method we focus on the first stage and decide the output,
don't think about the future. This method may or may not give the best
output.
Greedy Algorithm solves problems by making the best choice that seems best
at the particular moment. Many optimization problems can be determined
using a greedy algorithm. Some issues have no efficient solution, but a
greedy algorithm may provide a solution that is close to optimal. A greedy
algorithm works if a problem exhibits the following two properties:
1. Greedy Choice Property: A globally optimal solution can be

reached at by creating a locally optimal solution. In other
words, an optimal solution can be obtained by creating
"greedy" choices.
2. Optimal substructure: Optimal solutions contain optimal
subsolutions. In other words, answers to subproblems of an
optimal solution are optimal.
Example
1. Machine Scheduling
2. Fractional Knapsack Problem
3. Minimum Spanning Tree
4. Huffman Code
5. Job Sequencing
6. Activity Selection Problem
Steps for achieving a Greedy Algorithm are:
1. Feasible: Here we check whether it satisfies all possible

constraints or not, to obtain at least one solution to our
problems.
2. Local Optimal Choice: In this, the choice should be the
optimum which is selected from the currently available
3. Unalterable: Once the decision is made, at any subsequence
step that option is not altered.
Activity Selection Problem

The activity selection problem is a mathematical optimization problem. Our
first illustration is the problem of scheduling a resource among several
challenge activities. We find a greedy algorithm provides a well designed and
simple method for selecting a maximum- size set of manually compatible
activities.
Suppose S = {1, 2....n} is the set of n proposed activities. The activities share
resources which can be used by only one activity at a time, e.g., Tennis
Court, Lecture Hall, etc. Each Activity "i" has start time si and a finish time fi,
where si ≤fi. If selected activity "i" take place meanwhile the half-open time
interval [si,fi). Activities i and j are compatible if the intervals (si, fi) and [si,
fi) do not overlap (i.e. i and j are compatible if si ≥fi or si ≥fi). The activity-
selection problem chosen the maximum- size set of mutually consistent
activities.
Algorithm Of Greedy- Activity Selector
GREEDY- ACTIVITY SELECTOR (s, f)
1. n ← length [s]
2. A ← {1}
3. j ← 1.
4. for i ← 2 to n
5. do if si ≥ fi
6. then A ← A ∪ {i}
7. j ← i
8. return A
Example
Given 10 activities along with their start and end time as
S = (A1 A2 A3 A4 A5 A6 A7 A8 A9 A10)
Si = (1,2,3,4,7,8,9,9,11,12)
fi = (3,5,4,7,10,9,11,13,12,14)
Compute a schedule where the greatest number of activities takes place.
Solution
The solution to the above Activity scheduling problem using a greedy
strategy is illustrated below:
Arranging the activities in increasing order of end time
Now, schedule A1
Next schedule A3 as A1 and A3 are non-interfering.
Next skip A2 as it is interfering.
Next, schedule A4 as A1 A3 and A4 are non-interfering, then next, schedule A6
as A1 A3 A4 and A6 are non-interfering.
Skip A5 as it is interfering.
Next, schedule A7 as A1 A3 A4 A6 and A7 are non-interfering.
Next, schedule A9 as A1 A3 A4 A6 A7 and A9 are non-interfering.
Skip A8 as it is interfering.
Next, schedule A10 as A1 A3 A4 A6 A7 A9 and A10 are non-interfering.
Thus the final Activity schedule is:
Fractional Knapsack
Fractions of items can be taken rather than having to make binary (0-1)
choices for each item.
Fractional Knapsack Problem can be solvable by greedy strategy whereas 0 -
1 problem is not.
Steps to Solve the Fractional Problem
1. Compute the value per pound for each item.
2. Obeying a Greedy Strategy, we take as possible of the item
with the highest value per pound.
3. If the supply of that element is exhausted and we can still carry
more, we take as much as possible of the element with the next
value per pound.
4. Sorting, the items by value per pound, the greedy algorithm run
in O (n log n) time.
Fractional Knapsack (Array v, Array w, int W)

1. for i= 1 to size (v)
2. do p [i] = v [i] / w [i]
3. Sort-Descending (p)
4. i ← 1
5. while (W>0)
6. do amount = min (W, w [i])
7. solution [i] = amount
8. W= W-amount
9. i ← i+1
10. return solution
Example
Consider 5 items along their respective weights and values: -
I = (I1,I2,I3,I4,I5)
w = (5, 10, 20, 30, 40)
v = (30, 20, 100, 90,160)
The capacity of knapsack W = 60
Now fill the knapsack according to the decreasing value of pi.
First, we choose the item Ii whose weight is 5.
Then choose item I3 whose weight is 20. Now,the total weight of knapsack is
20 + 5 = 25
Now the next item is I5, and its weight is 40, but we want only 35, so we
chose the fractional part of it,
Solution
ITEM wi vi
I1 5 30
I2 10 20
I3 20 100
I4 30 90
I5 40 160
Taking value per weight ratio i.e. pi=

ITEM wi vi
Pi=
I1 5 30 6.0
I2 10 20 2.0
I3 20 100 5.0
I4 30 90 3.0
I5 40 160 4.0
Now, arrange the value of pi in decreasing order.

ITEM wi vi
pi=
I1 5 30 6.0
I3 20 100 5.0
I5 40 160 4.0
I4 30 90 3.0
I2 10 20 2.0
Huffman Codes
(i) Data can be encoded efficiently using Huffman Codes.
(ii) It is a widely used and beneficial technique for compressing data.
(iii) Huffman's greedy algorithm uses a table of the frequencies of
occurrences of each character to build up an optimal way of representing each
character as a binary string.
Suppose we have 105 characters in a data file. Normal Storage: 8 bits per
character (ASCII) - 8 x 105 bits in a file. But we want to compress the file and
save it compactly. Suppose only six characters appear in the file:
How can we represent the data in a Compact way?

(i) Fixed length Code: Each letter represented by an equal number of bits.
With a fixed length code, at least 3 bits per character:
Example
a 000
b 001
c 010
d 011
e 100
f 101
For a file with 105 characters, we need 3 x 105 bits.
(ii) A variable-length code: It can do considerably better than a fixed-length
code, by giving many characters short code words and infrequent character
long codewords.
For example:
a 0
b 101
c 100
d 111
e 1101
f 1100
Number of bits = (45 x 1 + 13 x 3 + 12 x 3 + 16 x 3 + 9 x 4 + 5 x 4) x 1000
= 2.24 x 105bits
Thus, 224,000 bits to represent the file, a saving of approximately 25%.This
is an optimal character code for this file.
Prefix Codes
The prefixes of an encoding of one character must not be equal to complete
encoding of another character, e.g., 1100 and 11001 are not valid codes
because 1100 is a prefix of some other code word is called prefix codes.
Prefix codes are desirable because they clarify encoding and decoding.
Encoding is always simple for any binary character code; we concatenate the
code words describing each character of the file. Decoding is also quite
comfortable with a prefix code. Since no codeword is a prefix of any other,
the codeword that starts with an encoded data is unambiguous.
Greedy Algorithm for constructing a Huffman Code
Huffman invented a greedy algorithm that creates an optimal prefix code
called a Huffman Code.
The algorithm builds the tree T analogous to the optimal code in a bottom-up
manner. It starts with a set of |C| leaves (C is the number of characters) and
performs |C| - 1 'merging' operations to create the final tree. In the Huffman
algorithm 'n' denotes the quantity of a set of characters, z indicates the parent
node, and x & y are the left & right child of z respectively.
Algorithm of Huffman Code
Huffman (C)
1. n=|C|
2. Q ← C
3. for i=1 to n-1
4. do
5. z= allocate-Node ()
6. x= left[z]=Extract-Min(Q)
7. y= right[z] =Extract-Min(Q)
8. f [z]=f[x]+f[y]
9. Insert (Q, z)
10. return Extract-Min (Q)
Example
Find an optimal Huffman Code for the following set of frequencies:
a: 50 b: 25 c: 15 d: 40 e: 75
Solution
i.e.
Again for i=2
Similarly, we apply the same process we get
Thus, the final output is:
Activity or Task Scheduling Problem

This is the dispute of optimally scheduling unit-time tasks on a single
processor, where each job has a deadline and a penalty that necessary be paid
if the deadline is missed.
A unit-time task is a job, such as a program to be rush on a computer that
needed precisely one unit of time to complete. Given a finite set S of unit-
time tasks, a schedule for S is a permutation of S specifying the order in
which to perform these tasks. The first task in the schedule starts at time 0
and ends at time 1; the second task begins at time 1 and finishes at time 2,
and so on.
The dispute of scheduling unit-time tasks with deadlines and penalties for
each processor has the following inputs:
a set S = {1, 2, 3.....n} of n unit-time tasks.

a set of n integer deadlines d1 d2 d3...dn such that di satisfies 1≤
di ≤ n and task i is supposed to finish by time di and
a set of n non-negative weights or penalties w1 w2....wn such
that we incur a penalty of wi if task i is not finished by time di,
and we incurred no penalty if a task finishes by its deadline.
Here we find a schedule for S that minimizes the total penalty incurred for
missed deadlines.
A task is late in this schedule if it finished after its deadline. Otherwise, the
task is early in the schedule. An arbitrary schedule can consistently be put
into early-first form, in which the first tasks precede the late tasks, i.e., if
some new task x follows some late task y, then we can switch the position of
x and y without affecting x being early or y being late.
An arbitrary schedule can always be put into a canonical form in which first
tasks precede the late tasks, and first tasks are scheduled in order of
nondecreasing deadlines.
A set A of tasks is independent if there exists a schedule for the particular
tasks such that no tasks are late. So the set of first tasks for a schedule forms
an independent set of tasks 'l' denote the set of all independent set of tasks.
For any set of tasks A, A is independent if for t = 0, 1, 2.....n we have Nt(A) ≤
t where Nt(A) denotes the number of tasks in A whose deadline is t or prior,
i.e. if the tasks in A are expected in order of monotonically growing
deadlines, then no task is late.
Example
Find the optimal schedule for the following task with given weight (penalties)
and deadlines.
1 2 3 4 5 6 7
di 4 2 4 3 1 4 6
wi 70 60 50 40 30 20 10
Solution
According to the Greedy algorithm we sort the jobs in decreasing order of
their penalties so that minimum of penalties will be charged.
In this problem, we can see that the maximum time for which uniprocessor
machine will run in 6 units because it is the maximum deadline.
Let Ti represents the tasks where i = 1 to 7
T5 and T6 cannot be accepted after T7 so penalty is

w5 + w6 = 30 + 20 = 50 (2 3 4 1 7 5 6)
Other schedule is
(2 4 1 3 7 5 6)
There can be many other schedules but (2 4 1 3 7 5 6) is optimal.
Travelling Sales Person Problem

The traveling salesman problems abide by a salesman and a set of cities. The
salesman has to visit every one of the cities starting from a certain one (e.g.,
the hometown) and to return to the same city. The challenge of the problem is
that the traveling salesman needs to minimize the total length of the trip.
Suppose the cities are x1 x2..... xn where cost cij denotes the cost of travelling
from city xi to xj. The travelling salesperson problem is to find a route starting
and ending at x1 that will take in all cities with the minimum cost.
Example
A newspaper agent daily drops the newspaper to the area assigned in such a
manner that he has to cover all the houses in the respective area with
minimum travel cost. Compute the minimum travel cost.
The area assigned to the agent where he has to drop the newspaper is shown
in fig:
Solution
The cost- adjacency matrix of graph G is as follows:
costij =
The tour starts from area H1 and then select the minimum cost area reachable
from H1.
Mark area H6 because it is the minimum cost area reachable from H1 and then
select minimum cost area reachable from H6.
Mark area H7 because it is the minimum cost area reachable from H6 and then
select minimum cost area reachable from H7.
Mark area H8 because it is the minimum cost area reachable from H8.
Mark area H4 and then select the minimum cost area reachable from H4 it is
H1.So, using the greedy strategy, we get the following.
4 3 2 4 3 2 1 6
H1 → H6 → H7 → H8 → H5 → H2 → H3 → H4 → H1.
Thus the minimum travel cost = 4 + 3 + 2 + 4 + 3 + 2 + 1 + 6 = 25

Matroids
A matroid is an ordered pair M(S, I) satisfying the following conditions:
S is a finite set.
I is a nonempty family of subsets of S, called the independent subsets of S,
such that if B ∈ I and A ∈ I. We say that I is hereditary if it satisfies this
property. Note that the empty set ∅ is necessarily a member of I.
If A ∈ I, B ∈ I and |A| <|B|, then there is some element x ∈ B ? A such
that A ∪ {x} ∈ I. We say that M satisfies the exchange property.
We say that a matroid M (S, I) is weighted if there is an associated weight
function w that assigns a strictly positive weight w (x) to each element x ∈
S. The weight function w extends to a subset of S by summation:
w (A) = ∑x ∈ A w(x)
for any A ∈ S.
Differentiate between Dynamic Programming and Greedy Method
Dynamic Programming Greedy Method
1. Dynamic Programming is used 1. Greedy Method is also used to
to obtain the optimal solution. get the optimal solution.
2. In Dynamic Programming, we 2. In a greedy Algorithm, we make
choose at each step, but the choice whatever choice seems best at the
may depend on the solution to moment and then solve the sub-
sub-problems. problems arising after the choice is
made.
3. Less efficient as compared to a 3. More efficient as compared to a
greedy approach greedy approach
4. Example: 0/1 Knapsack 4. Example: Fractional Knapsack
5. It is guaranteed that Dynamic 5. In Greedy Method, there is no
Programming will generate an such guarantee of getting Optimal
optimal solution using Principle of Solution.
Optimality.
Backtracking
The Backtracking is an algorithmic-method to solve a problem with an
additional way. It uses a recursive approach to explain the problems. We can
say that the backtracking is needed to find all possible combination to solve
an optimization problem.
Backtracking is a systematic way of trying out different sequences of
decisions until we find one that "works."
In the following Figure:
Each non-leaf node in a tree is a parent of one or more other

nodes (its children)
Each node in the tree, other than the root, has exactly one parent
Generally, however, we draw our trees downward, with the root at the top.
A tree is composed of nodes.

Backtracking can understand of as searching a tree for a particular "goal" leaf
node.
Backtracking is undoubtedly quite simple - we "explore" each node, as
follows:
To "explore" node N:
1. If N is a goal node, return "success"
2. If N is a leaf node, return "failure"
3. For each child C of N,
Explore C
If C was successful, return "success"
4. Return "failure"
Backtracking algorithm determines the solution by systematically searching
the solution space for the given problem. Backtracking is a depth-first
search with any bounding function. All solution using backtracking is needed
to satisfy a complex set of constraints. The constraints may be explicit or
implicit.
Explicit Constraint is ruled, which restrict each vector element to be chosen
from the given set.
Implicit Constraint is ruled, which determine which each of the tuples in the
solution space, actually satisfy the criterion function.
Recursive Maze Algorithm

Recursive Maze Algorithm is one of the best examples for backtracking
algorithms. Recursive Maze Algorithm is one of the possible solutions for
solving the maze.
Maze
The maze is an area surrounded by walls; in between, we have a path from
starting point to ending position. We have to start from the starting point and
travel towards from ending point.
Principle of Maze
As explained above, in the maze we have to travel from starting point to
ending point. The problem is to choose the path. If we find any dead-end
before ending point, we have to backtrack and move the direction. The
direction for traversing is North, East, West, and South. We have to continue
"move and backtrack" until we reach the final point.
Consider that we are having a two-dimensional maze cell [WIDTH]
[HEIGHT]. Here cell [x] [y] = 1 denotes wall and cell [x] [y] = 0 denotes free
cell in the particular location x, y in the maze. The directions we can change
in the array are North, East, West, and South. The first step is to make the
boundary of the two - dimensional array as one so that we won't go out of the
maze and usually reside inside the maze at any time.
Example Maze
1 1 1 1 1 1 1
1 0 0 0 1 1 1
1 1 1 0 1 1 1
1 1 1 0 0 0 1
1 1 1 1 1 0 1
1 1 1 1 1 1 1
Now start changing from the starting position (since the boundary is filled by
1) and find the next free cell then turn to the next free cell and so on. If we
grasp a dead-end, we have to backtrack and make the cells in the path as 1
(wall). Continue the same process till the final point is reached.
Hamiltonian Circuit Problems

Given a graph G = (V, E) we have to find the Hamiltonian Circuit using
Backtracking approach. We start our search from any arbitrary vertex say 'a.'
This vertex 'a' becomes the root of our implicit tree. The first element of our
partial solution is the first intermediate vertex of the Hamiltonian Cycle that
is to be constructed. The next adjacent vertex is selected by alphabetical
order. If at any stage any arbitrary vertex makes a cycle with any vertex other
than vertex 'a' then we say that dead end is reached. In this case, we backtrack
one step, and again the search begins by selecting another vertex and
backtrack the element from the partial; solution must be removed. The search
using backtracking is successful if a Hamiltonian Cycle is obtained.
Example
Consider a graph G = (V, E) shown in fig. we have to find a Hamiltonian
circuit using Backtracking method.
Solution
Firstly, we start our search with vertex 'a.' this vertex 'a' becomes the root of
our implicit tree.
Next, we choose vertex 'b' adjacent to 'a' as it comes first in lexicographical

order (b, c, d).
Next, we select 'c' adjacent to 'b.'

Next, we select 'd' adjacent to 'c.'
Next, we select 'e' adjacent to 'd.'

Next, we select vertex 'f' adjacent to 'e.' The vertex adjacent to 'f' is d and e,
but they have already visited. Thus, we get the dead end, and we backtrack
one step and remove the vertex 'f' from partial solution.
From backtracking, the vertex adjacent to 'e' is b, c, d, and f from which
vertex 'f' has already been checked, and b, c, d have already visited. So, again
we backtrack one step. Now, the vertex adjacent to d are e, f from which e
has already been checked, and adjacent of 'f' are d and e. If 'e' vertex,
revisited them we get a dead state. So again we backtrack one step.
Now, adjacent to c is 'e' and adjacent to 'e' is 'f' and adjacent to 'f' is 'd' and
adjacent to 'd' is 'a.' Here, we get the Hamiltonian Cycle as all the vertex other
than the start vertex 'a' is visited only once. (a - b - c - e - f -d - a).
Again Backtrack
Here we have generated one Hamiltonian circuit, but another Hamiltonian
circuit can also be obtained by considering another vertex.
Subset-Sum Problem
The Subset-Sum Problem is to find a subset's' of the given set S =
(S1 S2 S3...Sn) where the elements of the set S are n positive integers in such a
manner that s' ∈ S and sum of the elements of subset's' is equal to some
positive integer 'X.'
The Subset-Sum Problem can be solved by using the backtracking approach.
In this implicit tree is a binary tree. The root of the tree is selected in such a
way that represents that no decision is yet taken on any input. We assume that
the elements of the given set are arranged in increasing order:
S1 ≤ S2 ≤ S3... ≤ Sn
The left child of the root node indicated that we have to include 'S1' from the
set 'S' and the right child of the root indicates that we have to execute 'S1'.
Each node stores the total of the partial solution elements. If at any stage the
sum equals to 'X' then the search is successful and terminates.
The dead end in the tree appears only when either of the two inequalities
exists:
The sum of s' is too large i.e.
s'+ Si + 1 > X
The sum of s' is too small i.e.
Example
Given a set S = (3, 4, 5, 6) and X =9. Obtain the subset sum using
Backtracking approach.
Solution
Initially S = (3, 4, 5, 6) and X =9.
S'= ( ∅ )
The implicit binary tree for the subset sum problem is shown as fig:
The number inside a node is the sum of the partial solution elements at a
particular level.
Thus, if our partial solution elements sum is equal to the positive integer 'X'
then at that time search will terminate, or it continues if all the possible
solution needs to be obtained.
N-Queens Problem
N - Queens problem is to place n - queens in such a manner on an n x n
chessboard that no queens attack each other by being in the same row,
column or diagonal.
It can be seen that for n =1, the problem has a trivial solution, and no solution
exists for n =2 and n =3. So first we will consider the 4 queens problem and
then generate it to n - queens problem.
Given a 4 x 4 chessboard and number the rows and column of the chessboard
1 through 4.
Since, we have to place 4 queens such as q1 q2 q3 and q4 on the chessboard,

such that no two queens attack each other. In such a conditional each queen
must be placed on a different row, i.e., we put queen "i" on row "i."
Now, we place queen q1 in the very first acceptable position (1, 1). Next, we
put queen q2 so that both these queens do not attack each other. We find that
if we place q2 in column 1 and 2, then the dead end is encountered. Thus the
first acceptable position for q2 in column 3, i.e. (2, 3) but then no position is
left for placing queen 'q3' safely. So we backtrack one step and place the
queen 'q2' in (2, 4), the next best possible solution. Then we obtain the
position for placing 'q3' which is (3, 2). But later this position also leads to a
dead end, and no place is found where 'q4' can be placed safely. Then we have
to backtrack till 'q1' and place it to (1, 2) and then all other queens are placed
safely by moving q2 to (2, 4), q3 to (3, 1) and q4 to (4, 3). That is, we get the
solution (2, 4, 1, 3). This is one possible solution for the 4-queens problem.
For another possible solution, the whole method is repeated for all partial
solutions. The other solutions for 4 - queens problems is (3, 1, 4, 2) i.e.
The implicit tree for 4 - queen problem for a solution (2, 4, 1, 3) is as follows:
Fig shows the complete state space for 4 - queens problem. But we can use
backtracking method to generate the necessary node and stop if the next node
violates the rule, i.e., if two queens are attacking.
4 - Queens solution space with nodes numbered in DFS

It can be seen that all the solutions to the 4 queens problem can be
represented as 4 - tuples (x1, x2, x3, x4) where xi represents the column on
which queen "qi" is placed.
One possible solution for 8 queens problem is shown in fig:
Thus, the solution for 8 -queen problem for (4, 6, 8, 2, 7, 1, 3, 5).
If two queens are placed at position (i, j) and (k, l).
Then they are on same diagonal only if (i - j) = k - l or i + j = k + l.
The first equation implies that j - l = i - k.
The second equation implies that j - l = k - i.
Therefore, two queens lie on the duplicate diagonal if and only if |j-l|=|i-k|
Place (k, i) returns a Boolean value that is true if the kth queen can be placed
in column i. It tests both whether i is distinct from all previous costs x1,
x2,....xk-1 and whether there is no other queen on the same diagonal.
Using place, we give a precise solution to then n- queens problem.
Place (k, i)
{
For j ← 1 to k - 1
do if (x [j] = i)
or (Abs x [j]) - i) = (Abs (j - k))
then return false;
return true;
}
Place (k, i) return true if a queen can be placed in the kth row and ith column
otherwise return is false.
x [] is a global array whose final k - 1 values have been set. Abs (r) returns
the absolute value of r.
N - Queens (k, n)
{
For i ← 1 to n
do if Place (k, i) then
{
x [k] ← i;
if (k ==n) then
write (x [1....n));
else
N - Queens (k + 1, n);
}
}

Tree
A tree is a graph with the following properties:
1. The graph is connected (can go from anywhere to anywhere)

2. There are no cyclic (Acyclic)
Spanning Tree
Given a connected undirected graph, a spanning tree of that graph is a
subgraph that is a tree and joined all vertices. A single graph can have many
spanning trees.
Example
For the above-connected graph. There can be multiple spanning Trees like
Properties of Spanning Tree
1. There may be several minimum spanning trees of the same

weight having the minimum number of edges.
2. If all the edge weights of a given graph are the same, then every
spanning tree of that graph is minimum.
3. If each edge has a distinct weight, then there will be only one,
unique minimum spanning tree.
4. A connected graph G can have more than one spanning trees.
5. A disconnected graph can't have to span the tree, or it can't span
all the vertices.
6. Spanning Tree doesn't contain cycles.
7. Spanning Tree has (n-1) edges where n is the number of
vertices.
Addition of even one single edge results in the spanning tree losing its
property of Acyclicity and elimination of one single edge results in its losing
the property of connectivity.
Minimum Spanning Tree is a Spanning Tree which has minimum total cost.
If we have a linked undirected graph with a weight (or cost) combine with
each edge. Then the cost of spanning tree would be the sum of the cost of its
edges.
Application of Minimum Spanning Tree
1. Consider n stations are to be linked using a communication

network & laying of communication links between any two
stations involves a cost.
The ideal solution would be to extract a subgraph termed as
minimum cost spanning tree.
2. Suppose you want to construct highways or railroads spanning
several cities then we can use the concept of minimum
spanning trees.
3. Designing Local Area Networks.
4. Laying pipelines connecting offshore drilling sites, refineries
and consumer markets.
5. Suppose you want to apply a set of houses with
Electric Power
Water
Telephone lines
Sewage lines
To reduce cost, you can connect houses with minimum cost
spanning trees.
For Example, Problem laying Telephone Wire.

Methods of Minimum Spanning Tree
There are two methods to find Minimum Spanning Tree (MST)
1. Kruskal's Algorithm
2. Prim's Algorithm
Kruskal's Algorithm
An algorithm to construct a Minimum Spanning Tree for a connected
weighted graph. It is a Greedy Algorithm. The Greedy Choice is to put the
smallest weight edge that does not because a cycle in the MST constructed so
far.
If the graph is not linked, then it finds a Minimum Spanning Tree.
Steps for finding MST using Kruskal's Algorithm
1. Arrange the edge of G in order of increasing weight.

2. Starting only with the vertices of G and proceeding sequentially
add each edge which does not result in a cycle, until (n - 1)
edges are used.
3. EXIT.
MST- KRUSKAL (G, w)
1. A ← ∅
2. for each vertex v ∈ V [G]
3. do MAKE - SET (v)
4. sort the edges of E into non decreasing order by weight w
5. for each edge (u, v) ∈ E, taken in non decreasing order by weight
6. do if FIND-SET (μ) ≠ if FIND-SET (v)
7. then A ← A ∪ {(u, v)}
8. UNION (u, v)
9. return A
Analysis
Where E is the number of edges in the graph and V is the number of vertices,
Kruskal's Algorithm can be shown to run in O (E log E) time, or simply, O (E
log V) time, all with simple data structures. These running times are
equivalent because:
E is at most V2 and log V2= 2 x log V is O (log V).

If we ignore isolated vertices, which will each their components
of the minimum spanning tree, V ≤ 2 E, so log V is O (log E).
Thus the total time is

O (E log E) = O (E log V).
Example
Find the Minimum Spanning Tree of the following graph using Kruskal's
algorithm.
Solution
First we initialize the set A to the empty set and create |v| trees, one
containing each vertex with MAKE-SET procedure. Then sort the edges in E
into order by non-decreasing weight.
There are 9 vertices and 12 edges. So MST formed (9-1) = 8 edges
Now, check for each edge (u, v) whether the endpoints u and v belong to the
same tree. If they do then the edge (u, v) cannot be supplementary.
Otherwise, the two vertices belong to different trees, and the edge (u, v) is
added to A, and the vertices in two trees are merged in by union procedure.
Step 1: So, first take (h, g) edge
Step 2: then (g, f) edge.

Step 3: then (a, b) and (i, g) edges are considered, and the forest becomes
Step 4: Now, edge (h, i). Both h and i vertices are in the same set. Thus it
creates a cycle. So this edge is discarded.
Then edge (c, d), (b, c), (a, h), (d, e), (e, f) are considered, and the forest
becomes.
Step 5: In (e, f) edge both endpoints e and f exist in the same tree so
discarded this edge. Then (b, h) edge, it also creates a cycle.
Step 6: After that edge (d, f) and the final spanning tree is shown as in dark
lines.
Step 6: This step will be required Minimum Spanning Tree because it

contains all the 9 vertices and (9 - 1) = 8 edges
e → f, b → h, d → f [cycle will be formed]
Minimum Cost MST
Prim's Algorithm
It is a greedy algorithm. It starts with an empty spanning tree. The idea is to
maintain two sets of vertices:
Contain vertices already included in MST.

Contain vertices not yet included.
At every step, it considers all the edges and picks the minimum weight edge.
After picking the edge, it moves the other endpoint of edge to set containing
MST.
Robert Clay Prim
Steps for finding MST using Prim's Algorithm
1. Create MST set that keeps track of vertices already included in

MST.
2. Assign key values to all vertices in the input graph. Initialize all
key values as INFINITE (∞). Assign key values like 0 for the
first vertex so that it is picked first.
3. While MST set doesn't include all vertices.
a. Pick vertex u which is not is MST set and has
minimum key value. Include 'u'to MST set.
b. Update the key value of all adjacent vertices of u.
To update, iterate through all adjacent vertices. For
every adjacent vertex v, if the weight of edge u.v
less than the previous key value of v, update key
value as a weight of u.v.
MST-PRIM (G, w, r)
1. for each u ∈ V [G]
2. do key [u] ← ∞
3. π [u] ← NIL
4. key [r] ← 0
5. Q ← V [G]
6. While Q ? ∅
7. do u ← EXTRACT - MIN (Q)
8. for each v ∈ Adj [u]
9. do if v ∈ Q and w (u, v) < key [v]
10. then π [v] ← u
11. key [v] ← w (u, v)
Example
Generate minimum cost spanning tree for the following graph using Prim's
algorithm.
Solution
In Prim's algorithm, first we initialize the priority Queue Q. to contain all the
vertices and the key of each vertex to ∞ except for the root, whose key is set
to 0. Suppose 0 vertex is the root, i.e., r. By EXTRACT - MIN (Q) procure,
now u = r and Adj [u] = {5, 1}.
Removing u from set Q and adds it to set V - Q of vertices in the tree. Now,
update the key and π fields of every vertex v adjacent to u but not in a tree.
Taking 0 as starting vertex
Root = 0
Adj [0] = 5, 1
Parent, π [5] = 0 and π [1] = 0
Key [5] = ∞ and key [1] = ∞
w [0, 5) = 10 and w (0,1) = 28
w (u, v) < key [5] , w (u, v) < key [1]
Key [5] = 10 and key [1] = 28
So update key value of 5 and 1 is:
Now by EXTRACT_MIN (Q) Removes 5 because key [5] = 10 which is

minimum so u = 5.
Adj [5] = {0, 4} and 0 is already in heap
Taking 4, key [4] = ∞ π [4] = 5
(u, v) < key [v] then key [4] = 25
w (5,4) = 25
w (5,4) < key [4]
date key value and parent of 4.
Now remove 4 because key [4] = 25 which is minimum, so u =4

Adj [4] = {6, 3}
Key [3] = ∞ key [6] = ∞
w (4,3) = 22 w (4,6) = 24
w (u, v) < key [v] w (u, v) < key [v]
w (4,3) < key [3] w (4,6) < key [6]
Update key value of key [3] as 22 and key [6] as 24.
And the parent of 3, 6 as 4.
π[3]= 4 π[6]= 4
u = EXTRACT_MIN (3, 6) [key [3] < key [6]]
u=3 i.e. 22 < 24
Now remove 3 because key [3] = 22 is minimum so u =3.
Adj [3] = {4, 6, 2}

4 is already in heap
4 ≠ Q key [6] = 24 now becomes key [6] = 18
Key [2] = ∞ key [6] = 24
w (3, 2) = 12 w (3, 6) = 18
w (3, 2) < key [2] w (3, 6) < key [6]
Now in Q, key [2] = 12, key [6] = 18, key [1] = 28 and parent of 2 and 6 is 3.
π [2] = 3 π[6]=3
Now by EXTRACT_MIN (Q) Removes 2, because key [2] = 12 is minimum.
u = EXTRACT_MIN (2, 6)
u=2 [key [2] < key [6]]
12 < 18
Now the root is 2
Adj [2] = {3, 1}
3 is already in a heap
Taking 1, key [1] = 28
w (2,1) = 16
w (2,1) < key [1]
So update key value of key [1] as 16 and its parent as 2.
π[1]= 2
Now by EXTRACT_MIN (Q) Removes 1 because key [1] = 16 is minimum.

Adj [1] = {0, 6, 2}
0 and 2 are already in heap.
Taking 6, key [6] = 18
w [1, 6] = 14
w [1, 6] < key [6]
Update key value of 6 as 14 and its parent as 1.
Π [6] = 1
Now all the vertices have been spanned, Using above the table we get
Minimum Spanning Tree.
0→5→4→3→2→1→6
[Because Π [5] = 0, Π [4] = 5, Π [3] = 4, Π [2] = 3, Π [1] =2, Π [6] =1]
Thus the final spanning Tree is
Total Cost = 10 + 25 + 22 + 12 + 16 + 14 = 99
Shortest Paths
In a shortest- paths problem, we are given a weighted, directed graphs G =
(V, E), with weight function w: E → R mapping edges to real-valued
weights. The weight of path p = (v0,v1,..... vk) is the total of the weights of its
constituent edges:
We define the shortest - path weight from u to v by δ(u,v) = min (w (p):

u→v), if there is a path from u to v, and δ(u,v)= ∞, otherwise.
The shortest path from vertex s to vertex t is then defined as any path p with
weight w (p) = δ(s,t).
The breadth-first- search algorithm is the shortest path algorithm that works
on unweighted graphs, that is, graphs in which each edge can be considered
to have unit weight.
In a Single Source Shortest Paths Problem, we are given a Graph G = (V, E),
we want to find the shortest path from a given source vertex s ∈ V to every
vertex v ∈ V.
Variants
There are some variants of the shortest path problem.
Single- destination shortest - paths problem: Find the shortest

path to a given destination vertex t from every vertex v. By shift
the direction of each edge in the graph, we can shorten this
problem to a single - source problem.
Single - pair shortest - path problem: Find the shortest path
from u to v for given vertices u and v. If we determine the
single - source problem with source vertex u, we clarify this
problem also. Furthermore, no algorithms for this problem are
known that run asymptotically faster than the best single -
source algorithms in the worst case.
All - pairs shortest - paths problem: Find the shortest path from
u to v for every pair of vertices u and v. Running a single -
source algorithm once from each vertex can clarify this
problem; but it can generally be solved faster, and its structure
is of interest in the own right.
Shortest Path: Existence

If some path from s to v contains a negative cost cycle then, there does not
exist the shortest path. Otherwise, there exists a shortest s - v that is simple.
Negative Weight Edges

It is a weighted graph in which the total weight of an edge is negative. If a
graph has a negative edge, then it produces a chain. After executing the chain
if the output is negative then it will give - ∞ weight and condition get
discarded. If weight is less than negative and - ∞ then we can't have the
shortest path in it.
Briefly, if the output is -ve, then both condition get discarded.
1. -∞
2. Not less than 0.
And we cannot have the shortest Path.

Example
Beginning from s
Adj [s] = [a, c, e]
Weight from s to a is 3
Suppose we want to calculate a path from s→c. So We have 2 paths /weight
s to c = 5, s→c→d→c=8
But s→c is minimum
So s→c = 5
Suppose we want to calculate a path from s→e. So we have two paths again
s→e = 2, s→e→f→e=-1
As -1 < 0 ∴ Condition gets discarded. If we execute this chain, we will get
- ∞. So we can't get the shortest path ∴ e = ∞.
This figure illustrates the effects of negative weights and negative weight
cycle on the shortest path weights.
Because there is only one path from "s to a" (the path <s, a>), δ (s, a) = w (s,
a) = 3.
Furthermore, there is only one path from "s to b", so δ (s, b) = w (s, a) + w (a,
b) = 3 + (-4) = - 1.
There are infinite many path from "s to c" : <s, c> : <s, c, d, c>, <s, c, d, c, d,
c> and so on. Because the cycle <c, d, c> has weight δ (c, d) = w (c, d) + w
(d, c) = 6 + (-3) = 3, which is greater than 0, the shortest path from s to c is
<s, c> with weight δ (s, c) = 5.
Similarly, the shortest path from "s to d" is <s, c, d> with weight δ (s, d) = w
(s, c) + w (s, d) = 11.
Analogously, there are infinite many paths from s to e: <s, e>, <s, e, f, e>, <s,
e, f, e, f, e> and so on. Since the cycle <e, f, e> has weight δ (e, f) = w (e, f) +
w (f, e) = 3 + (-6) = -3. So - 3 < 0, however there is no shortest path from s to
e. B8y traversing the negative weight cycle <e, f, e>. This means path from s
to e has arbitrary large negative weights and so δ (s, e) = - ∞.
Similarly δ (s, f) = - ∞ because g is reachable from f, we can also find a path
with arbitrary large negative weight from s to g and δ (s, g) = - ∞
Vertices h, i, g also from negative weight cycle. They are also not reachable
from the source node, so distance from the source is - ∞ to three of nodes (h,
i, j).
Representing Shortest Path

Given a graph G = (V, E), we maintain for each vertex v ∈ V
a predecessor π [v] that is either another vertex or NIL. During the execution
of shortest paths algorithms, however, the π values need not indicate shortest
paths. As in breadth-first search, we shall be interested in the predecessor
subgraph Gn= (Vn,En) induced by the value π. Here again, we define the
vertex set Vπ, to be the set of vertices of G with non - NIL predecessors, plus
the source s:
Vπ= {v ∈ V: π [v] ≠ NIL} ∪ {s} }
The directed edge set EΠ is the set of edges induced by the Π values for
vertices in VΠ:
EΠ= {(Π[v], v) ∈ E: v ∈ VΠ - {s}}
A shortest - paths tree rooted at s is a directed subgraph G = (V' E'), where
V' ∈ V andE' ∈ E, such that
1. V' is the set of vertices reachable from s in G

2. G' forms a rooted tree with root s, and
3. For all v ∈ V', the unique, simple path from s to v in G' is the
shortest path from s to v in G.
Shortest paths are not naturally unique, and neither is shortest - paths trees.
Properties of Shortest Path
1. Optimal substructure property: All subpaths of shortest paths are shortest
paths.
Let P1 be x - y sub path of shortest s - v path. Let P2 be any x - y path. Then

cost of P1≤ cost of P2,otherwise P not shortest s - v path.
2. Triangle inequality: Let d (v, w) be the length of shortest path from v to w.
Then,
d (v, w) ≤ d (v, x) + d (x, w)
3. Upper-bound property: We always have d[v] ≥ δ(s, v) for all vertices v ∈

V, and once d[v] conclude the value δ(s, v), it never changes.
4. No-path property: If there is no path from s to v, then we regularly have
d[v] = δ(s, v) = ∞.
5. Convergence property: If s->u->v is a shortest path in G for some u, v ∈
V, and if d[u] = δ(s, u) at any time prior to relaxing edge (u, v), then d[v] =
δ(s, v) at all times thereafter.
Relaxation
The single - source shortest paths are based on a technique known
as relaxation, a method that repeatedly decreases an upper bound on the
actual shortest path weight of each vertex until the upper bound equivalent
the shortest - path weight. For each vertex v ∈ V, we maintain an attribute
d [v], which is an upper bound on the weight of the shortest path from source
s to v. We call d [v] the shortest path estimate.
INITIALIZE - SINGLE - SOURCE (G, s)
1. for each vertex v ∈ V [G]
2. do d [v] ← ∞
3. π [v] ← NIL
4. d [s] ← 0
After initialization, π [v] = NIL for all v ∈ V, d [v] = 0 for v = s, and d [v]
= ∞ for v ∈ V - {s}.
The development of relaxing an edge (u, v) consists of testing whether we
can improve the shortest path to v found so far by going through u and if so,
updating d [v] and π [v]. A relaxation step may decrease the value of the
shortest - path estimate d [v] and updated v's predecessor field π [v].
Fig: Relaxing an edge (u, v) with weight w (u, v) = 2. The shortest-path
estimate of each vertex appears within the vertex.
(a) Because v. d > u. d + w (u, v) prior to relaxation, the value of v. d

decreases
(b) Here, v. d < u. d + w (u, v) before relaxing the edge, and so the relaxation
step leaves v. d unchanged.
The subsequent code performs a relaxation step on edge (u, v)
RELAX (u, v, w)
1. If d [v] > d [u] + w (u, v)
2. then d [v] ← d [u] + w (u, v)
3. π [v] ← u
Dijkstra's Algorithm
It is a greedy algorithm that solves the single-source shortest path problem
for a directed graph G = (V, E) with nonnegative edge weights, i.e., w (u, v)
≥ 0 for each edge (u, v) ∈ E.
Dijkstra's Algorithm maintains a set S of vertices whose final shortest - path
weights from the source s have already been determined. That's for all
vertices v ∈ S; we have d [v] = δ (s, v). The algorithm repeatedly selects
the vertex u ∈ V - S with the minimum shortest - path estimate, insert u
into S and relaxes all edges leaving u.
Because it always chooses the "lightest" or "closest" vertex in V - S to insert
into set S, it is called as the greedy strategy.
Dijkstra's Algorithm (G, w, s)
1. INITIALIZE - SINGLE - SOURCE (G, s)
2. S← ∅
3. Q←V [G]
4. while Q ≠ ∅
5. do u ← EXTRACT - MIN (Q)
6. S ← S ∪ {u}
7. for each vertex v ∈ Adj [u]
8. do RELAX (u, v, w)
Analysis
The running time of Dijkstra's algorithm on a graph with edges E and vertices
V can be expressed as a function of |E| and |V| using the Big - O notation.
The simplest implementation of the Dijkstra's algorithm stores vertices of set
Q in an ordinary linked list or array, and operation Extract - Min (Q) is
simply a linear search through all vertices in Q. In this case, the running time
is O (|V2 |+|E|=O(V2 ).
Example
Solution
Step 1: Q =[s, t, x, y, z]
We scanned vertices one by one and find out its adjacent. Calculate the
distance of each adjacent to the source vertices.
We make a stack, which contains those vertices which are selected after
computation of shortest distance.
Firstly we take's' in stack M (which is a source)
M = [S] Q = [t, x, y, z]
Step 2: Now find the adjacent of s that are t and y.
Adj [s] → t, y [Here s is u and t and y are v]
Case - (i) s → t
d [v] > d [u] + w [u, v]
d [t] > d [s] + w [s, t]
∞ > 0 + 10 [false condition]
Then d [t] ← 10
π [t] ← 5
Adj [s] ← t, y
Case - (ii) s→ y
d [v] > d [u] + w [u, v]
d [y] > d [s] + w [s, y]
∞>0+5 [false condition]
∞>5
Then d [y] ← 5
π [y] ← 5
By comparing case (i) and case (ii)
Adj [s] → t = 10, y = 5
y is shortest
y is assigned in 5 = [s, y]
Step 3: Now find the adjacent of y that is t, x, z.
Adj [y] → t, x, z [Here y is u and t, x, z are v]
Case - (i) y →t
d [v] > d [u] + w [u, v]
d [t] > d [y] + w [y, t]
10 > 5 + 3
10 > 8
Then d [t] ← 8
π [t] ← y
Case - (ii) y → x
d [v] > d [u] + w [u, v]
d [x] > d [y] + w [y, x]
∞>5+9
∞ > 14
Then d [x] ← 14
π [x] ← 14
Case - (iii) y → z
d [v] > d [u] + w [u, v]
d [z] > d [y] + w [y, z]
∞>5+2
∞>7
Then d [z] ← 7
π [z] ← y
By comparing case (i), case (ii) and case (iii)
Adj [y] → x = 14, t = 8, z =7
z is shortest
z is assigned in 7 = [s, z]
Step 4: Now we will find adj [z] that are s, x

Adj [z] → [x, s] [Here z is u and s and x are v]
Case - (i) z → x
d [v] > d [u] + w [u, v]
d [x] > d [z] + w [z, x]
14 > 7 + 6
14 > 13
Then d [x] ← 13
π [x] ← z
Case - (ii) z → s
d [v] > d [u] + w [u, v]
d [s] > d [z] + w [z, s]
0>7+7
0 > 14
∴ This condition does not satisfy so it will be discarded.
Now we have x = 13.
Step 5: Now we will find Adj [t]
Adj [t] → [x, y] [Here t is u and x and y are v]
Case - (i) t → x
d [v] > d [u] + w [u, v]
d [x] > d [t] + w [t, x]
13 > 8 + 1
13 > 9
Then d [x] ← 9
π [x] ← t
Case - (ii) t → y
d [v] > d [u] + w [u, v]
d [y] > d [t] + w [t, y]
5 > 10
∴ This condition does not satisfy so it will be discarded.
Thus we get all shortest path vertex as
Weight from s to y is 5
Weight from s to z is 7
Weight from s to t is 8
Weight from s to x is 9
These are the shortest distance from the source's' in the given graph.
Disadvantage of Dijkstra's Algorithm
1. It does a blind search, so wastes a lot of time while processing.

2. It can't handle negative edges.
3. It leads to the acyclic graph and most often cannot obtain the
right shortest path.
4. We need to keep track of vertices that have been visited.
Bellman-Ford Algorithm
Solves single shortest path problem in which edge weight may be negative
but no negative cycle exists.
This algorithm works correctly when some of the edges of the directed graph
G may have negative weight. When there are no cycles of negative weight,
then we can find out the shortest path between source and destination.
It is slower than Dijkstra's Algorithm but more versatile, as it capable of
handling some of the negative weight edges.
This algorithm detects the negative cycle in a graph and reports their
existence.
Based on the "Principle of Relaxation" in which more accurate values
gradually recovered an approximation to the proper distance by until
eventually reaching the optimum solution.
Given a weighted directed graph G = (V, E) with source s and weight
function w: E → R, the Bellman-Ford algorithm returns a Boolean value
indicating whether or not there is a negative weight cycle that is attainable
from the source. If there is such a cycle, the algorithm produces the shortest
paths and their weights. The algorithm returns TRUE if and only if a graph
contains no negative - weight cycles that are reachable from the source.
Recurrence Relation
distk [u] = [min[distk-1 [u],min[ distk-1 [i]+cost [i,u]]] as i except u.
k → k is the source vertex
u → u is the destination vertex
i → no of edges to be scanned concerning a vertex.
BELLMAN -FORD (G, w, s)
1. INITIALIZE - SINGLE - SOURCE (G, s)
2. for i ← 1 to |V[G]| - 1
3. do for each edge (u, v) ∈ E [G]
5. for each edge (u, v) ∈ E [G]
6. do if d [v] > d [u] + w (u, v)
7. then return FALSE.
8. return TRUE.
Example
Here first we list all the edges and their weights.
Solution
distk [u] = [min[distk-1 [u],min[distk-1 [i]+cost [i,u]]] as i ≠ u.
dist2 [2]=min[dist1 [2],min[dist1 [1]+cost[1,2],dist1 [3]+cost[3,2],dist1

[4]+cost[4,2],dist1 [5]+cost[5,2]]]
Min = [6, 0 + 6, 5 + (-2), ∞ + ∞ , ∞ +∞] = 3
[4]+cost[4,3],dist1 [5]+cost[5,3]]]
Min = [5, 0 +∞, 6 +∞, ∞ + ∞ , ∞ + ∞] = 5
[3]+cost[3,4],dist1 [5]+cost[5,4]]]
Min = [∞, 0 +∞, 6 + (-1), 5 + 4, ∞ +∞] = 5
[3]+cost[3,5],dist1 [4]+cost[4,5]]]
Min = [∞, 0 + ∞,6 + ∞,5 + 3, ∞ + 3] = 8
[4]+cost[4,2],dist2 [5]+cost[5,2]]]
Min = [3, 0 + 6, 5 + (-2), 5 + ∞ , 8 + ∞ ] = 3
[4]+cost[4,3],dist2 [5]+cost[5,3]]]
Min = [5, 0 + ∞, 3 + ∞, 5 + ∞,8 + ∞ ] = 5
[3]+cost[3,4],dist2 [5]+cost[5,4]]]
Min = [5, 0 + ∞, 3 + (-1), 5 + 4, 8 + ∞ ] = 2
[3]+cost[3,5],dist2 [4]+cost[4,5]]]
Min = [8, 0 + ∞, 3 + ∞, 5 + 3, 5 + 3] = 8
[4]+cost[4,2],dist3 [5]+cost[5,2]]]
Min = [3, 0 + 6, 5 + (-2), 2 + ∞, 8 + ∞ ] =3
[4]+cost[4,3],dist3 [5]+cost[5,3]]]
Min = 5, 0 + ∞, 3 + ∞, 2 + ∞, 8 + ∞ ] =5
[3]+cost[3,4],dist3 [5]+cost[5,4]]]
Min = [2, 0 + ∞, 3 + (-1), 5 + 4, 8 + ∞ ] = 2
[3]+cost[3,5],dist3 [5]+cost[4,5]]]
Min = [8, 0 +∞, 3 + ∞, 8, 5] = 5
Single Source Shortest Path in Directed Acyclic Graphs

By relaxing the edges of a weighted DAG (Directed Acyclic Graph) G = (V,
E) according to a topological sort of its vertices, we can figure out shortest
paths from a single source in ∅ (V+E) time. Shortest paths are always well
described in a dag, since even if there are negative-weight edges, no negative-
weight cycles can exist.
DAG - SHORTEST - PATHS (G, w, s)
1. Topologically sort the vertices of G.
2. INITIALIZE - SINGLE- SOURCE (G, s)
3. for each vertex u taken in topologically sorted order
4. do for each vertex v ∈ Adj [u]
The running time of this data is determined by line 1 and by the for loop of
lines 3 - 5. The topological sort can be implemented in ∅ (V + E) time. In
the for loop of lines 3 - 5, as in Dijkstra's algorithm, there is one repetition
per vertex. For each vertex, the edges that leave the vertex are each examined
exactly once. Unlike Dijkstra's algorithm, we use only O (1) time per edge.
The running time is thus ∅ (V + E), which is linear in the size of an
adjacency list depiction of the graph.
Example
Step1: To topologically sort vertices apply DFS (Depth First Search) and then
arrange vertices in linear order by decreasing order of finish time.
Now, take each vertex in topologically sorted order and relax each edge.
adj [s] →t, x

0+3<∞
d [t] ← 3
0+2<∞
d [x] ← 2
adj [t] → r, x
3+1<∞
d [r] ← 4
3+5≤2
adj [x] → y
2-3<∞
d [y] ← -1
adj [y] → r
-1 + 4 < 4
3 <4
d [r] ← 3
Thus the Shortest Path is:

s to x is 2
s to y is -1
s to t is 3
s to r is 3
All-Pairs Shortest Paths
It aims to figure out the shortest path from each vertex v to every other u.
Storing all the paths explicitly can be very memory expensive indeed, as we
need one spanning tree for each vertex. This is often impractical regarding
memory consumption, so these are generally considered as all pairs-shortest
distance problems, which aim to find just the distance from each to each node
to another. We usually want the output in tabular form: the entry in u's row
and v's column should be the weight of the shortest path from u to v.
Three approaches for improvement:
Algorithm Cost
Matrix Multiplication O (V3 logV)
Floyd-Warshall O (V3)
Johnson O (V2 log?
V+VE)
Unlike the single-source algorithms, which assume an adjacency list

representation of the graph, most of the algorithm uses an adjacency matrix
representation. (Johnson's Algorithm for sparse graphs uses adjacency lists.)
The input is a n x n matrix W representing the edge weights of an n-vertex
directed graph G = (V, E). That is, W = (wij), where
Floyd-Warshall Algorithm
Let the vertices of G be V = {1, 2........n} and consider a subset {1, 2........k}
of vertices for some k. For any pair of vertices i, j ∈ V, considered all paths
from i to j whose intermediate vertices are all drawn from {1, 2.......k}, and
let p be a minimum weight path from amongst them. The Floyd-Warshall
algorithm exploits a link between path p and shortest paths from i to j with all
intermediate vertices in the set {1, 2.......k-1}. The link depends on whether or
not k is an intermediate vertex of path p.
If k is not an intermediate vertex of path p, then all intermediate vertices of
path p are in the set {1, 2........k-1}. Thus, the shortest path from vertex i to
vertex j with all intermediate vertices in the set {1, 2.......k-1} is also the
shortest path i to j with all intermediate vertices in the set {1, 2.......k}.
If k is an intermediate vertex of path p, then we break p down into i → k → j.
Let dij(k) be the weight of the shortest path from vertex i to vertex j with all
intermediate vertices in the set {1, 2.......k}.
A recursive definition is given by
FLOYD - WARSHALL (W)

1. n ← rows [W].
2. D0 ← W
3. for k ← 1 to n
4. do for i ← 1 to n
6. do dij(k) ← min (dij(k-1),dik(k-1)+dkj(k-1) )
7. return D(n)
The strategy adopted by the Floyd-Warshall algorithm is Dynamic
Programming. The running time of the Floyd-Warshall algorithm is
determined by the triply nested for loops of lines 3-6. Each execution of line
6 takes O (1) time. The algorithm thus runs in time θ(n3 ).
Example
Apply Floyd-Warshall algorithm for constructing the shortest path. Show that
matrices D(k) and π(k) computed by the Floyd-Warshall algorithm for the
graph.
Solution
Step (i) When k = 0
Step (ii) When k =1

Step (iii) When k = 2
Step (iv) When k = 3
Step (v) When k = 4
Step (vi) When k = 5
TRANSITIVE- CLOSURE (G)
1. n ← |V[G]|
2. for i ← 1 to n
4. do if i = j or (i, j) ∈ E [G]
5. the ←1
6. else ←0
7. for k ← 1 to n
8. do for i ← 1 to n
10. dod ij(k) ←

11. Return T(n).
Johnson's Algorithm
The problem is to find the shortest path between every pair of vertices in a
given weighted directed graph and weight may be negative. Using Johnson's
Algorithm, we can find all pairs shortest path in O (V2 log ? V+VE ) time.
Johnson's Algorithm uses both Dijkstra's Algorithm and Bellman-Ford
Algorithm.
Johnson's Algorithm uses the technique of "reweighting." If all edge weights
w in a graph G = (V, E) are nonnegative, we can find the shortest paths
between all pairs of vertices by running Dijkstra's Algorithm once from each
vertex. If G has negative - weight edges, we compute a new - set of non -
negative edge weights that allows us to use the same method. The new set of
edges weight w must satisfy two essential properties:
For all pair of vertices u, v ∈ V, the shortest path from u to v using weight
function w is also the shortest path from u to v using weight function w.
For all edges (u, v), the new weight w (u, v) is nonnegative.
Given a weighted, directed graph G = (V, E) with weight function w: E→R
and let h: v→R be any function mapping vertices to a real number.
For each edge (u, v) ∈ E define
Where h (u) = label of u

h (v) = label of v
JOHNSON (G)
1. Compute G' where V [G'] = V[G] ∪ {S} and
E [G'] = E [G] ∪ {(s, v): v ∈ V [G] }
2. If BELLMAN-FORD (G',w, s) = FALSE
then "input graph contains a negative weight cycle"
else
for each vertex v ∈ V [G']
do h (v) ← δ(s, v)
Computed by Bellman-Ford algorithm
for each edge (u, v) ∈ E[G']
do w (u, v) ← w (u, v) + h (u) - h (v)
for each edge u ∈ V [G]
do run DIJKSTRA (G, w, u) to compute
δ (u, v) for all v ∈ V [G]
for each vertex v ∈ V [G]
do duv← δ (u, v) + h (v) - h (u)
Return D.
Example
Step 1: Take any source vertex's' outside the graph and make distance from's'
to every vertex '0'.
Step 2: Apply Bellman-Ford Algorithm and calculate minimum weight on
each vertex.
Step 3: w (a, b) = w (a, b) + h (a) - h (b)
= -3 + (-1) - (-4)
=0
w (b, a) = w (b, a) + h (b) - h (a)
= 5 + (-4) - (-1)
=2
w (b, c) = w (b, c) + h (b) - h (c)
w (b, c) = 3 + (-4) - (-1)
=0
w (c, a) = w (c, a) + h (c) - h (a)
w (c, a) = 1 + (-1) - (-1)
=1
w (d, c) = w (d, c) + h (d) - h (c)
w (d, c) = 4 + 0 - (-1)
=5
w (d, a) = w (d, a) + h (d) - h (a)
w (d, a) = -1 + 0 - (-1)
=0
w (a, d) = w (a, d) + h (a) - h (d)
w (a, d) = 2 + (-1) - 0 = 1
Step 4: Now all edge weights are positive and now we can apply Dijkstra's
Algorithm on each vertex and make a matrix corresponds to each vertex in a
graph
Case 1: 'a' as a source vertex
Case 2: 'b' as a source vertex
Case 3: 'c' as a source vertex
Case 4:'d' as source vertex

Step 5:
duv ← δ (u, v) + h (v) - h (u)
d (a, a) = 0 + (-1) - (-1) = 0
d (a, b) = 0 + (-4) - (-1) = -3
d (a, c) = 0 + (-1) - (-1) = 0
d (a, d) = 1 + (0) - (-1) = 2
d (b, a) = 1 + (-1) - (-4) = 4
d (b, b) = 0 + (-4) - (-4) = 0
d (c, a) = 1 + (-1) - (-1) = 1
d (c, b) = 1 + (-4) - (-1) = -2
d (c, c) = 0
d (c, d) = 2 + (0) - (-1) = 3
d (d, a) = 0 + (-1) - (0) = -1
d (d, b) = 0 + (-4) - (0) = -4
d (d, c) = 0 + (-1) - (0) = -1
d (d, d) = 0
Flow
Flow Network is a directed graph that is used for modeling material Flow.
There are two different vertices; one is a source which produces material at
some steady rate, and another one is sink which consumes the content at the
same constant speed. The flow of the material at any mark in the system is
the rate at which the element moves.
Some real-life problems like the flow of liquids through pipes, the current
through wires and delivery of goods can be modeled using flow networks.
Definition
A Flow Network is a directed graph G = (V, E) such that
1. For each edge (u, v) ∈ E, we associate a nonnegative weight

capacity c (u, v) ≥ 0.If (u, v) ∉ E, we assume that c (u, v) = 0.
2. There are two distinguishing points, the source s, and the sink t;
3. For every vertex v ∈ V, there is a path from s to t containing
v.
Let G = (V, E) be a flow network. Let s be the source of the network, and let t
be the sink. A flow in G is a real-valued function f: V x V→R such that the
following properties hold:
Capacity Constraint: For all u, v ∈ V, we need f (u, v) ≤ c (u,

v).
Skew Symmetry: For all u, v ∈ V, we need f (u, v) = - f (u, v).
Flow Conservation: For all u ∈ V-{s, t}, we need
The quantity f (u, v), which can be positive or negative, is known as the net
flow from vertex u to vertex v. In the maximum-flow problem, we are given
a flow network G with source s and sink t, and we wish to find a flow of
maximum value from s to t.
The three properties can be described as follows:
1. Capacity Constraint makes sure that the flow through each edge
is not greater than the capacity.
2. Skew Symmetry means that the flow from u to v is the negative
of the flow from v to u.
3. The flow-conservation property says that the total net flow out
of a vertex other than the source or sink is 0. In other words, the
amount of flow into a v is the same as the amount of flow out
of v for every vertex v ∈ V - {s, t}
The value of the flow is the net flow from the source,
The positive net flow entering a vertex v is described by

The positive net flow leaving a vertex is described symmetrically. One
interpretation of the Flow-Conservation Property is that the positive net flow
entering a vertex other than the source or sink must equal the positive net
flow leaving the vertex.
A flow f is said to be integer-valued if f (u, v) is an integer for all (u, v) ∈
E. Clearly, the value of the flow is an integer is an integer-valued flow.
Network Flow Problems

The most obvious flow network problem is the following:
Problem 1: Given a flow network G = (V, E), the maximum flow problem is
to find a flow with maximum value.
Problem 2: The multiple source and sink maximum flow problem is similar
to the maximum flow problem, except there is a set {s1,s2,s3.......sn} of sources
and a set {t1,t2,t3..........tn} of sinks.
Fortunately, this problem is no solid than regular maximum flow. Given
multiple sources and sink flow network G, we define a new flow network G'
by adding
A super source s,
A super sink t,
For each si, add edge (s, si) with capacity ∞, and
For each ti,add edge (ti,t) with capacity ∞
Figure shows a multiple sources and sinks flow network and an equivalent
single source and sink flow network.
Residual Networks: The Residual Network consists of an edge that can admit
more net flow. Suppose we have a flow network G = (V, E) with source s and
sink t. Let f be a flow in G, and examine a pair of vertices u, v ∈ V. The
sum of additional net flow we can push from u to v before exceeding the
capacity c (u, v) is the residual capacity of (u, v) given by
When the net flow f (u, v) is negative, the residual capacity cf (u,v) is greater
than the capacity c (u, v).
Example: if c (u, v) = 16 and f (u, v) =16 and f (u, v) = -4, then the residual
capacity cf (u,v) is 20.
Given a flow network G = (V, E) and a flow f, the residual network of G
induced by f is Gf = (V, Ef), where
That is, each edge of the residual network, or residual edge, can admit a
strictly positive net flow.
Augmenting Path: Given a flow network G = (V, E) and a flow f,
an augmenting path p is a simple path from s to t in the residual network Gf.
By the solution of the residual network, each edge (u, v) on an augmenting
path admits some additional positive net flow from u to v without violating
the capacity constraint on the edge.
Let G = (V, E) be a flow network with flow f. The residual capacity of an
augmenting path p is
The residual capacity is the maximal amount of flow that can be pushed
through the augmenting path. If there is an augmenting path, then each edge
on it has a positive capacity. We will use this fact to compute a maximum
flow in a flow network.
Ford-Fulkerson Algorithm
Initially, the flow of value is 0. Find some augmenting Path p and increase
flow f on each edge of p by residual Capacity cf (p). When no augmenting
path exists, flow f is a maximum flow.
FORD-FULKERSON METHOD (G, s, t)
1. Initialize flow f to 0
2. while there exists an augmenting path p
3. do argument flow f along p
4. Return f
FORD-FULKERSON (G, s, t)
1. for each edge (u, v) ∈ E [G]
2. do f [u, v] ← 0
3. f [u, v] ← 0
4. while there exists a path p from s to t in the residual network Gf.
5. do cf (p)←min?{ Cf (u,v):(u,v)is on p}
6. for each edge (u, v) in p
7. do f [u, v] ← f [u, v] + cf (p)
8. f [u, v] ←-f[u,v]
Example
Each Directed Edge is labeled with capacity. Use the Ford-Fulkerson
algorithm to find the maximum flow.
Solution
The left side of each part shows the residual network Gf with a shaded
augmenting path p,and the right side of each part shows the net flow f.
Maximum Bipartite Matching
A Bipartite Graph is a graph whose vertices can be divided into two
independent sets L and R such that every edge (u, v) either connect a vertex
from L to R or a vertex from R to L. In other words, for every edge (u, v)
either u ∈ L and v ∈ L. We can also say that no edge exists that connect
vertices of the same set.
Matching is a Bipartite Graph is a set of edges chosen in such a way that no

two edges share an endpoint. Given an undirected Graph G = (V, E), a
Matching is a subset of edge M ⊆ E such that for all vertices v ∈ V, at
most one edge of M is incident on v.
A Maximum matching is a matching of maximum cardinality, that is, a
matching M such that for any matching M', we have|M|>|M' |.
Finding a maximum bipartite matching

We can use the Ford-Fulkerson method to find a maximum matching in an
undirected bipartite graph G= (V, E) in time polynomial in |V| and |E|. The
trick is to construct a flow network G= (V',E') for the bipartite graph G as
follows. We let the source s and sink t be new vertices not in V, and we let
V'=V ∪ {s,t}.If the vertex partition of G is V = L ∪ R, the directed edges of
G' are the edges of E, directed from L to R, along with |V| new directed
edges:
Fig: A Bipartite Graph G = (V, E) with vertex partition V = L ∪ R.
Sorting Networks
Comparison Networks
A comparison network is made of wires and comparators. A comparator is a
device with two inputs, x and y, and two outputs, x' and y,' where
x'= min (x, y)
y' = max (x, y)
In Comparison Networks input appear on the left and outputs on the right,
with the smallest input value appearing on the top output and the largest input
value appearing on the bottom output. Each comparator operates in O (1)
time. In other words, we consider that the time between the appearance of the
input values x and y and the production of the output values x' and y' is a
constant.
A wire transmits a value from place to place. A comparison network contains
n input wires a1,a2,........an through which the benefits to be sorted enter the
network, and n output wires b1,b2,......bn which produce the results computed
by the network.
Comparison Network is a set of comparators interconnected by wires.

Running time of comparator can define regarding depth.
Depth of a Wire: An input wire of a comparison network has depth 0. Now, if
a comparator has two input wires with depths dx and dy' then its output wires
have depth max (dx,dy) + 1.
A sorting network is a comparison network for which the output sequence is
monotonically increasing (that is b1≤ b2 ≤ ....bn) for every input sequence.
A Sorting network based on Insertion Sort
Bitonic Sorting Network

A sequence that monotonically increases and then monotonically decreases,
or else monotonically decreases and then monotonically increases is called a
bitonic sequence. For example: the sequence (2, 5, 6, 9, 3, 1) and (8, 7, 5, 2,
4, 6) are both bitonic. The bitonic sorter is a comparison network that sorts
bitonic sequence of 0's and 1's.
Half-Cleaner: A bitonic sorter is containing several stages, each of which is
called a half-cleaner. Each half-cleaner is a comparison network of depth 1 in
which input line i is compared with line 1+ for i = 1, 2..... .
When a bitonic sequence of 0's and 1's is practiced as input to a half-cleaner,
the half-cleaner produces an output sequence in which smaller values are in
the top half, larger values are in the bottom half, and both halves are bitonic,
and at least one of the halves is clean.
Bitonic Sorter: By recursively connecting half-cleaners, we can build a
bitonic sorter, which is a network that sorts bitonic sequences. The first stage
of BITONIC-SORTER [n] consists of HALF-CLEANER [n], which
produces two bitonic sequences of half the size such that every element in the
top half is at least as small as each element in the bottom half. Thus, we can
complete the sort by utilizing two copies of BITONIC-SORTER [n/2] to sort
the two halves recursively.
The depth D (n) of BITONIC-SORTER [n] is given by recurrence whose

solution is D (n) = log n.
Merging Network
Merging Network is the network that can join two sorted input sequences into
one sorted output sequence. We adapt BITONIC-SORTER [n] to create the
merging network MERGER [n].
The merging network is based on the following assumption:
Given two sorted sequences, if we reverse the order of the second sequence
and then connect the two sequences, the resulting sequence is bitonic.
Example: Given two sorted zero-one sequences X = 00000111 and Y
=00001111, we reverse Y to get YR = 11110000. Concatenating X and
YR yield 0000011111110000, which is bitonic.
The sorting network SORTER [n] need the merging network to implement a
parallel version of merge sort. The first stage of SORTER [n] consists of n/2
copies of MERGER [2] that work in parallel to merge pairs of a 1-element
sequence to produce a sorted sequence of length 2. The second stage subsists
of n/4 copies of MERGER [4] that merge pairs of these 2-element sorted
sequences to generate sorted sequences of length 4. In general, for k = 1, 2.....
log n, stage k consists of n/2k copies of MERGER [2k] that merge pairs of the
2k-1 element sorted sequence to produce a sorted sequence of length2k. At the
last stage, one sorted sequence consisting of all the input values is produced.
This sorting network can be shown by induction to sort zero-one sequences,
and therefore by the zero-one principle, it can sort arbitrary values.
The recurrence given the depth of SORTER [n]
Whose solution is D (n) = θ (log2n). Thus, we can sort n numbers in parallel

in ₒ (log2n) time.
Complexity Theory
Complexity Classes
NP Class Problem
The set of all decision-based problems came into the division of NP Problems
who can't be solved or produced an output within polynomial time but
verified in the polynomial time. NP class contains P class as a subset. NP
problems being hard to solve.
Note: - The term "NP" does not mean "not polynomial." Originally, the term
meant "non-deterministic polynomial. It means according to the one input
number of output will be produced.
P Class Problem
The set of decision-based problems come into the division of P Problems
who can be solved or produced an output within polynomial time. P problems
being easy to solve
Polynomial Time
If we produce an output according to the given input within a specific amount
of time such as within a minute, hours. This is known as Polynomial time.
Non-Polynomial Time
If we produce an output according to the given input but there are no time
constraints is known as Non-Polynomial time. But yes output will produce
but time is not fixed yet.
Decision Based Problem
A problem is called a decision problem if its output is a simple "yes" or "no"
(or you may need this of this as true/false, 0/1, accept/reject.) We will phrase
many optimization problems as decision problems. For example, Greedy
method, D.P., given a graph G= (V, E) if there exists any Hamiltonian cycle.
Np-Hard Class
Here you to satisfy the following points to come into the division of NP-hard
1. If we can solve this problem in polynomial time, then we can

solve all NP problems in polynomial time
2. If you convert the issue into one form to another form within
the polynomial time
NP-Complete Class
A problem is in NP-complete, if
1. It is in NP
2. It is NP-hard
Pictorial representation of all NP classes which includes NP, NP-hard, and
NP-complete
Polynomial Time Verification

Before talking about the class of NP-complete problems, it is essential to
introduce the notion of a verification algorithm.
Many problems are hard to solve, but they have the property that it easy to
authenticate the solution if one is provided.
Hamiltonian Cycle Problem
Consider the Hamiltonian cycle problem. Given an undirected graph G, does
G have a cycle that visits each vertex exactly once? There is no known
polynomial time algorithm for this dispute.
Note: - It means you can't build a Hamiltonian cycle in a graph with a
polynomial time even if there is no specific path is given for the Hamiltonian
cycle with the particular vertex, yet you can't verify the Hamiltonian cycle
within the polynomial time
Hamiltonian Cycle
Let us understand that a graph did have a Hamiltonian cycle. It would be easy
for someone to convince of this. They would similarly say: "the period is hv3,
v7, v1....v13i.
We could then inspect the graph and check that this is indeed a legal cycle
and that it visits all of the vertices of the graph exactly once. Thus, even
though we know of no efficient way to solve the Hamiltonian cycle problem,
there is a beneficial way to verify that a given cycle is indeed a Hamiltonian
cycle.
Note:-For the verification in the Polynomial-time of an undirected
Hamiltonian cycle graph G. There must be exact/specific/definite path must
be given of Hamiltonian cycle then you can verify in the polynomial time.
Certificate
A piece of information which contains in the given path of a vertex is known
as certificate
Relation of P and NP classes
P contains in NP
P=NP
1. Observe that P contains in NP. In other words, if we can solve a

problem in polynomial time, we can indeed verify the solution
in polynomial time. More formally, we do not need to see a
certificate (there is no need to specify the vertex/intermediate of
the specific path) to solve the problem; we can explain it in
polynomial time anyway.
2. However, it is not known whether P = NP. It seems you can
verify and produce an output of the set of decision-based
problems in NP classes in a polynomial time which is
impossible because according to the definition of NP classes
you can verify the solution within the polynomial time. So this
relation can never be held.
Reductions
The class NP-complete (NPC) problems consist of a set of decision problems
(a subset of class NP) that no one knows how to solve efficiently. But if there
were a polynomial solution for even a single NP-complete problem, then
every problem in NPC will be solvable in polynomial time. For this, we need
the concept of reductions.
Suppose there are two problems, A and B. You know that it is impossible to
solve problem A in polynomial time. You want to prove that B cannot be
explained in polynomial time. We want to show that (A ∉ P) => (B ∉ P)
Consider an example to illustrate reduction: The following problem is well-
known to be NPC:
3-color: Given a graph G, can each of its vertices be labeled with one of 3
different colors such that two adjacent vertices do not have the same label
(color).
Coloring arises in various partitioning issues where there is a constraint that
two objects cannot be assigned to the same set of partitions. The phrase
"coloring" comes from the original application which was in map drawing.
Two countries that contribute a common border should be colored with
different colors.
It is well known that planar graphs can be colored (maps) with four colors.
There exists a polynomial time algorithm for this. But deciding whether this
can be done with 3 colors is hard, and there is no polynomial time algorithm
for it.
Example of 3-colorable and non-3-colorable graphs
Polynomial Time Reduction
We say that Decision Problem L1 is Polynomial time Reducible to decision
Problem L2 (L1≤p L2) if there is a polynomial time computation function f
such that of all x, xϵL1 if and only if xϵL2.
NP-Completeness
A decision problem L is NP-Hard if
L' ≤p L for all L' ϵ NP.
L is NP-complete if
1. L ϵ NP and
2. L' ≤ p L for some known NP-complete problem L.' Given this
formal definition, the complexity classes are:
P: is the set of decision problems that are solvable in

polynomial time.
NP: is the set of decision problems that can be verified in
polynomial time.
NP-Hard: L is NP-hard if for all L' ϵ NP, L' ≤p L. Thus if we
can solve L in polynomial time, we can solve all NP problems
in polynomial time.
NP-Complete L is NP-complete if
1. L ϵ NP and
2. L is NP-hard
If any NP-complete problem is solvable in polynomial time, then every NP-
Complete problem is also solvable in polynomial time. Conversely, if we can
prove that any NP-Complete problem cannot be solved in polynomial time,
every NP-Complete problem cannot be solvable in polynomial time.
Reductions
Concept
If the solution of NPC problem does not exist then the conversion from one
NPC problem to another NPC problem within the polynomial time. For this,
you need the concept of reduction. If a solution of the one NPC problem
exists within the polynomial time, then the rest of the problem can also give
the solution in polynomial time (but it's hard to believe). For this, you need
the concept of reduction.
Example
Suppose there are two problems, A and B. You know that it is impossible to
solve problem A in polynomial time. You want to prove that B cannot be
solved in polynomial time. So you can convert the problem A into
problem B in polynomial time.
Example of NP-Complete Problem
NP problem: - Suppose a DECISION-BASED problem is provided in which
a set of inputs/high inputs you can get high output.
Criteria to come either in NP-hard or NP-complete.
1. The point to be noted here, the output is already given, and you
can verify the output/solution within the polynomial time but
can't produce an output/solution in polynomial time.
2. Here we need the concept of reduction because when you can't
produce an output of the problem according to the given input
then in case you have to use an emphasis on the concept of
reduction in which you can convert one problem into another
problem.
Note 1:- If you satisfy both points then your problem comes into the category
of NP-complete class
Note 2:- If you satisfy the only 2nd points then your problem comes into the
category of NP-hard class
So according to the given decision-based NP problem, you can decide in the
form of yes or no. If, yes then you have to do verify and convert into another
problem via reduction concept. If you are being performed, both then
decision-based NP problems are in NP compete.
Here we will emphasize NPC.
Circuit Satisfiability
According to given decision-based NP problem, you can design the circuit
and verify a given mentioned output also within the P time. The circuit is
provided below:-
Note:- You can design a circuit and verified the mentioned output within
Polynomial time but remember you can never predict the number of gates
which produces the high output against the set of inputs/high inputs within a
polynomial time. So you verified the production and conversion had been
done within polynomial time. So you are in NPC.
SAT (Satisfiability)
A Boolean function is said to be SAT if the output for the given value of the
input is true/high/1
F=X+YZ (Created a Boolean function by CIRCUIT SAT)
These points you have to be performed for NPC
CONCEPTS OF SAT
CIRCUIT SAT≤ρ SAT
SAT≤ρ CIRCUIT SAT
SAT ϵ NPC
1. CONCEPT: - A Boolean function is said to be SAT if the

output for the given value of the input is true/high/1.
2. CIRCUIT SAT ≤ ρ SAT: - In this conversion, you have to
convert CIRCUIT SAT into SAT within the polynomial time as
we did it
3. SAT ≤ ρ CIRCUIT SAT: - For the sake of verification of an
output you have to convert SAT into CIRCUIT SAT within the
polynomial time, and through the CIRCUIT SAT you can get
the verification of an output successfully
4. SAT ϵ NPC: - As you know very well, you can get the SAT
through CIRCUIT SAT that comes from NP.
Proof of NPC: Reduction has been successfully made within the polynomial
time from CIRCUIT SAT TO SAT. Output has also been verified within the
polynomial time as you did in the above conversation.
So concluded that SAT ϵ NPC.
3CNF Satisfiability
Concept: - In 3CNF SAT, you have at least 3 clauses, and in clauses, you will
have almost 3 literals or constants
Such as (X+Y+Z) (X+Y+Z) (X+Y+Z)
You can define as (XvYvZ) ᶺ (XvYvZ) ᶺ (XvYvZ)
V=OR operator
^ =AND operator
These all the following points need to be considered in 3CNF SAT.
To prove: -
Concept of 3CNF SAT

SAT≤ρ 3CNF SAT
3CNF≤ρ SAT
3CNF ϵ NPC
1. CONCEPT: - In 3CNF SAT, you have at least 3 clauses, and in clauses,

you will have almost 3 literals or constants.
2. SAT ≤ρ 3CNF SAT:- In which firstly you need to convert a Boolean
function created in SAT into 3CNF either in POS or SOP form within the
polynomial time
F=X+YZ
= (X+Y) (X+Z)
= (X+Y+ZZ') (X+YY'+Z)
= (X+Y+Z) (X+Y+Z') (X+Y+Z) (X+Y'+Z)
= (X+Y+Z) (X+Y+Z') (X+Y'+Z)
3. 3CNF ≤p SAT: - From the Boolean Function having three literals we can
reduce the whole function into a shorter one.
F= (X+Y+Z) (X+Y+Z') (X+Y'+Z)
= (X+Y+Z) (X+Y+Z') (X+Y+Z) (X+Y'+Z)
= (X+Y+ZZ') (X+YY'+Z)
= (X+Y) (X+Z)
= X+YZ
4. 3CNF ϵ NPC: - As you know very well, you can get the 3CNF through
SAT and SAT through CIRCUIT SAT that comes from NP.
Proof of NPC
1. It shows that you can easily convert a Boolean function of SAT

into 3CNF SAT and satisfied the concept of 3CNF SAT also
within polynomial time through Reduction concept.
2. If you want to verify the output in 3CNF SAT then perform the
Reduction and convert into SAT and CIRCUIT also to check
the output
If you can achieve these two points that means 3CNF SAT also in NPC
Clique
To Prove: - Clique is an NPC or not?
For this you have to satisfy the following below-mentioned points: -
Clique
3CNF ≤ρ Clique
Clique ≤ρ 3CNF≤SAT
Clique ϵ NP
1) Clique
Definition: - vertex, and the number of vertices in the Clique represents the
Size of Clique.
CLIQUE COVER: - Given a graph G and an integer k, can we find k subsets
of verticesV1, V2...VK, such that UiVi = V, and that each Vi is a clique of G.
The following figure shows a graph that has a clique cover of size 3.
2) 3CNF ≤ρ Clique
For the successful conversion from 3CNF to Clique, you have to follow the
two steps:-
Draw the clause in the form of vertices, and each vertex represents the literals
of the clauses.
1. They do not complement each other

2. They don't belong to the same clause. In the conversion, the
size of the Clique and size of 3CNF must be the same, and you
successfully converted 3CNF into Clique within the polynomial
time
3) Clique ≤ρ 3CNF
As you know that a function of K clause, there must exist a Clique of size k.
It means that P variables which are from the different clauses can assign the
same value (say it is 1). By using these values of all the variables of the
CLIQUES, you can make the value of each clause in the function is equal to
1
Example
You have a Boolean function in 3CNF:-
(X+Y+Z) (X+Y+Z') (X+Y'+Z)
After Reduction/Conversion from 3CNF to CLIQUE, you will get P variables
such as: - x +y=1, x +z=1 and x=1
Put the value of P variables in equation (i)
(1+1+0)(1+0+0)(1+0+1)
(1)(1)(1)=1 output verified
4) Clique ϵ NP
As you know very well, you can get the Clique through 3CNF and to convert
the decision-based NP problem into 3CNF you have to first convert into SAT
and SAT comes from NP.
So, concluded that CLIQUE belongs to NP.
Proof of NPC
1. Reduction achieved within the polynomial time from 3CNF to

Clique
2. And verified the output after Reduction from Clique To 3CNF
above
So, concluded that, if both Reduction and verification can be
done within the polynomial time that means Clique also in
NPC.
Vertex Cover
1) Vertex Cover
It represents a set of vertex or node in a graph G (V, E), which gives the
connectivity of a complete graph
According to the graph G of vertex cover which you have created, the size of
Vertex Cover =2
2) Vertex Cover ≤ρ Clique

In a graph G of Vertex Cover, you have N vertices which contain a Vertex
Cover K. There must exist of Clique Size of size N-K in its complement.
According to the graph G, you have
Number of vertices=6
Size of Clique=N-K=4
You can also create the Clique by complimenting the graph G of Vertex
Cover means in simpler form connect the vertices in Vertex Cover graph G
through edges where edges don?t exist and remove all the existed edges
You will get the graph G with Clique Size=4
3) Clique ≤ ρ Vertex Cover
Here through the Reduction process, you can get the Vertex Cover form
Clique by just complimenting the Clique graph G within the polynomial time.
4) Vertex Cover ϵ NP
As you know very well, you can get the Vertex Cover through Clique and to
convert the decision-based NP problem into Clique firstly you have to
convert into 3CNF and 3CNF into SAT and SAT into CIRCUIT SAT that
comes from NP.
Proof of NPC
1. Reduction from Clique to Vertex Cover has been made within
the polynomial time. In the simpler form, you can convert into
Vertex Cover from Clique within the polynomial time
2. And verification has also been done when you convert Vertex
Cover to Clique and Clique to 3CNF and satisfy/verified the
output within a polynomial time also, so it concluded that
Reduction and Verification had been done in the polynomial
time that means Vertex Cover also comes in NPC
Subset Cover
1) Subset Cover
Number of a subset of edges after making the union for a get all the edges of
the complete graph G, and that is called Subset Cover.
According to the graph G, which you have created the size of Subset
Cover=2
v1{e1,e6} v2{e5,e2} v3{e2,e4,e6} v4{e1,e3,e5} v5{e4} v6{e3}

v3Uv4= {e1, e2, e3, e4, e5, e6} complete set of edges after the union of
vertices.
2) Vertex Cover ≤ρ Subset Cover
In a graph G of vertices N, if there exists a Vertex Cover of size k, then there
must also exist a Subset Cover of size k even. If you can achieve after the
Reduction from Vertex Cover to Subset Cover within a polynomial time,
which means you did right.
3) Subset Cover ≤ρ Vertex Cover
Just for verification of the output perform the Reduction and create Clique
and via an equation, N-K verifies the Clique also and through Clique you can
quickly generate 3CNF and after solving the Boolean function of 3CNF in
the polynomial time. You will get output. It means the output has been
verified.
4) Subset Cover ϵ NP
As you know very well, you can get the Subset-Cover through Vertex Cover
and Vertex Cover through Clique and to convert the decision-based NP
problem into Clique firstly you have to convert into3CNF and 3CNF into
SAT and SAT into CIRCUIT SAT that comes from NP.
Proof of NPC
The Reduction has been successfully made within the polynomial time form
Vertex Cover to Subset Cover
Output has also been verified within the polynomial time as you did in the
above conversation so, concluded that SUBSET COVER also comes in NPC.
Independent Set
An independent set of a graph G = (V, E) is a subset V' ⊆ V of vertices such
that every edge in E is incident on at most one vertex in V.' The independent-
set problem is to find a largest-size independent set in G. It is not hard to find
small independent sets, e.g., a small independent set is an individual node,
but it is hard to find large independent sets.
Approximate Algorithms
An Approximate Algorithm is a way of approach NP-COMPLETENESS for
the optimization problem. This technique does not guarantee the best
solution. The goal of an approximation algorithm is to come as close as
possible to the optimum value in a reasonable amount of time which is at the
most polynomial time. Such algorithms are called approximation algorithm
or heuristic algorithm.
For the traveling salesperson problem, the optimization problem is to find the
shortest cycle, and the approximation problem is to find a short cycle.
For the vertex cover problem, the optimization problem is to find the vertex
cover with fewest vertices, and the approximation problem is to find the
vertex cover with few vertices.
Performance Ratios
Suppose we work on an optimization problem where every solution carries a
cost. An Approximate Algorithm returns a legal solution, but the cost of that
legal solution may not be optimal.
For Example, suppose we are considering for a minimum size vertex-cover
(VC). An approximate algorithm returns a VC for us, but the size (cost) may
not be minimized.
Another Example is we are considering for a maximum size Independent set
(IS). An approximate Algorithm returns an IS for us, but the size (cost) may
not be maximum. Let C be the cost of the solution returned by an
approximate algorithm, and C* is the cost of the optimal solution.
We say the approximate algorithm has an approximate ratio P (n) for an input
size n, where
Intuitively, the approximation ratio measures how bad the approximate

solution is distinguished with the optimal solution. A large (small)
approximation ratio measures the solution is much worse than (more or less
the same as) an optimal solution.
Observe that P (n) is always ≥ 1, if the ratio does not depend on n, we may
write P. Therefore, a 1-approximation algorithm gives an optimal solution.
Some problems have polynomial-time approximation algorithm with small
constant approximate ratios, while others have best-known polynomial time
approximation algorithms whose approximate ratios grow with n.
Vertex Cover
A Vertex Cover of a graph G is a set of vertices such that each edge in G is
incident to at least one of these vertices.
The decision vertex-cover problem was proven NPC. Now, we want to solve
the optimal version of the vertex cover problem, i.e., we want to find a
minimum size vertex cover of a given graph. We call such vertex cover an
optimal vertex cover C*.
An approximate algorithm for vertex cover:
Approx-Vertex-Cover (G = (V, E))
{
C = empty-set;
E'= E;
While E' is not empty do
{
Let (u, v) be any edge in E': (*)
Add u and v to C;
Remove from E' all edges incident to
u or v;
}
Return C;
}
The idea is to take an edge (u, v) one by one, put both vertices to C, and
remove all the edges incident to u or v. We carry on until all edges have been
removed. C is a VC. But how good is C?
VC = {b, c, d, e, f, g}
In the traveling salesman Problem, a salesman must visits n cities. We can
say that salesman wishes to make a tour or Hamiltonian cycle, visiting each
city exactly once and finishing at the city he starts from. There is a non-
negative cost c (i, j) to travel from the city i to city j. The goal is to find a tour
of minimum cost. We assume that every two cities are connected. Such
problems are called Traveling-salesman problem (TSP).
We can model the cities as a complete graph of n vertices, where each vertex
represents a city.
It can be shown that TSP is NPC.
If we assume the cost function c satisfies the triangle inequality, then we can
use the following approximate algorithm.
Triangle inequality
Let u, v, w be any three vertices, we have
One important observation to develop an approximate solution is if we

remove an edge from H*, the tour becomes a spanning tree.
Approx-TSP (G= (V, E))
{
1. Compute a MST T of G;
2. Select any vertex r is the root of the tree;
3. Let L be the list of vertices visited in a preorder tree walk of T;
4. Return the Hamiltonian cycle H that visits the vertices in the order L;
}
Intuitively, Approx-TSP first makes a full walk of MST T, which visits each
edge exactly two times. To create a Hamiltonian cycle from the full walk, it
bypasses some vertices (which corresponds to making a shortcut)
String Matching
String Matching Algorithm is also called "String Searching Algorithm." This
is a vital class of string algorithm is declared as "this is the method to find a
place where one is several strings are found within the larger string."
Given a text array, T [1.....n], of n character and a pattern array, P [1......m],
of m characters. The problems are to find an integer s, called valid
shift where 0 ≤ s < n-m and T [s+1......s+m] = P [1......m]. In other words, to
find even if P in T, i.e., where P is a substring of T. The item of P and T are
character drawn from some finite alphabet such as {0, 1} or {A, B .....Z, a,
b..... z}.Given a string T [1......n], the substrings are represented as T [i......j]
for some 0≤i ≤ j≤n-1, the string formed by the characters in T from index i to
index j, inclusive. This process that a string is a substring of itself (take i = 0
and j =m).
The proper substring of string T [1......n] is T [1......j] for some 0<i ≤ j≤n-1.
That is, we must have either i>0 or j < m-1.
Using these descriptions, we can say given any string T [1......n], the
substrings are
T [i.....j] = T [i] T [i +1] T [i+2]......T [j] for some 0≤i ≤ j≤n-1.
And proper substrings are
T [i.....j] = T [i] T [i +1] T [i+2]......T [j] for some 0≤i ≤ j≤n-1.
Note: If i>j, then T [i.....j] is equal to the empty string or null, which has
length zero.
Algorithms used for String Matching
There are different types of method is used to finding the string
1. The Naive String Matching Algorithm

2. The Rabin-Karp-Algorithm
3. Finite Automata
4. The Knuth-Morris-Pratt Algorithm
5. The Boyer-Moore Algorithm
Naive String Matching Algorithm

The naïve approach tests all the possible placement of Pattern P [1.......m]
relative to text T [1......n]. We try shift s = 0, 1.......n-m, successively and for
each shift s. Compare T [s+1.......s+m] to P [1......m].
The naïve algorithm finds all valid shifts using a loop that checks the
condition P [1.......m] = T [s+1.......s+m] for each of the n - m +1 possible
value of s.
NAIVE-STRING-MATCHER (T, P)
1. n ← length [T]
2. m ← length [P]
3. for s ← 0 to n -m
4. do if P [1.....m] = T [s + 1....s + m]
5. then print "Pattern occurs with shift" s
Analysis
This for loop from 3 to 5 executes for n-m + 1(we need at least m characters
at the end) times and in iteration we are doing m comparisons. So the total
complexity is O (n-m+1).
Example
Suppose T = 1011101110
P = 111
Find all the Valid Shift
Solution
Rabin-Karp-Algorithm
The Rabin-Karp string matching algorithm calculates a hash value for the
pattern, as well as for each M-character subsequences of text to be compared.
If the hash values are unequal, the algorithm will determine the hash value for
next M-character sequence. If the hash values are equal, the algorithm will
analyze the pattern and the M-character sequence. In this way, there is only
one comparison per text subsequence, and character matching is only
required when the hash values match.
RABIN-KARP-MATCHER (T, P, d, q)
1. n ← length [T]
2. m ← length [P]
3. h ← dm-1 mod q
4. p ← 0
5. t0 ← 0
6. for i ← 1 to m
7. do p ← (dp + P[i]) mod q
8. t0 ← (dt0+T [i]) mod q
9. for s ← 0 to n-m
10. do if p = ts
11. then if P [1.....m] = T [s+1.....s + m]
12. then "Pattern occurs with shift" s
13. If s < n-m
14. then ts+1 ← (d (ts-T [s+1]h)+T [s+m+1])mod q
Example
For string matching, working module q = 11, how many spurious hits does
the Rabin-Karp matcher encounters in Text T = 31415926535.......
T = 31415926535.......
P = 26
Here T.Length =11 so Q = 11
And P mod Q = 26 mod 11 = 4
Now find the exact match of P mod Q...
Solution
Complexity
The running time of RABIN-KARP-MATCHER in the worst case scenario O
((n-m+1) m but it has a good average case running time. If the expected
number of strong shifts is small O (1) and prime q is chosen to be quite large,
then the Rabin-Karp algorithm can be expected to run in time O (n+m) plus
the time to require to process spurious hits.
String Matching with Finite Automata

The string-matching automaton is a very useful tool which is used in string
matching algorithm. It examines every character in the text exactly once and
reports all the valid shifts in O (n) time. The goal of string matching is to find
the location of specific text pattern within the larger body of text (a sentence,
a paragraph, a book, etc.)
Finite Automata
A finite automaton M is a 5-tuple (Q, q0,A,∑δ), where
Q is a finite set of states,

q0 ∈ Q is the start state,
A ⊆ Q is a notable set of accepting states,
∑ is a finite input alphabet,
δ is a function from Q x ∑ into Q called the transition function
of M.
The finite automaton starts in state q0 and reads the characters of its input
string one at a time. If the automaton is in state q and reads input character a,
it moves from state q to state δ (q, a). Whenever its current state q is a
member of A, the machine M has accepted the string read so far. An input
that is not allowed is rejected.
A finite automaton M induces a function ∅ called the called the final-state
function, from ∑* to Q such that ∅ (w) is the state M ends up in after
scanning the string w. Thus, M accepts a string w if and only if ∅ (w) ∈ A.
The function f is defined as
∅ ( ∈ )=q0
∅ (wa) = δ (( ∅ (w), a) for w ∈ ∑*,a ∈ ∑)
FINITE- AUTOMATON-MATCHER (T,δ, m),
1. n ← length [T]
2. q ← 0
3. for i ← 1 to n
4. do q ← δ (q, T[i])
5. If q =m
6. then s←i-m
7. print "Pattern occurs with shift s" s
The primary loop structure of FINITE- AUTOMATON-MATCHER implies
that its running time on a text string of length n is O (n).
Computing the Transition Function: The following procedure computes the
transition function δ from given pattern P [1......m]
COMPUTE-TRANSITION-FUNCTION (P, ∑)
1. m ← length [P]
2. for q ← 0 to m
3. do for each character a ∈ ∑*
4. do k ← min (m+1, q+2)
5. repeat k←k-1
6. Until
7. δ(q,a)←k
8. Return δ
Example
Suppose a finite automaton which accepts even number of a's where ∑ = {a,
b, c}
Solution
q0 is the initial state.
Knuth-Morris-Pratt (KMP)Algorithm
Knuth-Morris and Pratt introduce a linear time algorithm for the string
matching problem. A matching time of O (n) is achieved by avoiding
comparison with an element of 'S' that have previously been involved in
comparison with some element of the pattern 'p' to be matched. i.e.,
backtracking on the string 'S' never occurs
Components of KMP Algorithm
1. The Prefix Function (Π): The Prefix Function, Π for a pattern encapsulates
knowledge about how the pattern matches against the shift of itself. This
information can be used to avoid a useless shift of the pattern 'p.' In other
words, this enables avoiding backtracking of the string 'S.'
2. The KMP Matcher: With string 'S,' pattern 'p' and prefix function 'Π' as
inputs, find the occurrence of 'p' in 'S' and returns the number of shifts of 'p'
after which occurrences are found.
The Prefix Function (Π)
Following pseudo code compute the prefix function, Π:
COMPUTE- PREFIX- FUNCTION (P)
1. m ←length [P] //'p' pattern to be matched
2. Π [1] ← 0
3. k ← 0
4. for q ← 2 to m
5. do while k > 0 and P [k + 1] ≠ P [q]
6. do k ← Π [k]
7. If P [k + 1] = P [q]
8. then k← k + 1
9. Π [q] ← k
10. Return Π
Running Time Analysis
In the above pseudo code for calculating the prefix function, the for loop
from step 4 to step 10 runs 'm' times. Step1 to Step3 take constant time.
Hence the running time of computing prefix function is O (m).
Example
Compute Π for the pattern 'p' below:
Solution
Initially: m = length [p] = 7
Π [1] = 0
k=0
After iteration 6 times, the prefix function computation is complete:
The KMP Matcher

The KMP Matcher with the pattern 'p,' the string 'S' and prefix function 'Π' as
input, finds a match of p in S. Following pseudo code compute the matching
component of KMP algorithm:
KMP-MATCHER (T, P)
1. n ← length [T]
2. m ← length [P]
3. Π← COMPUTE-PREFIX-FUNCTION (P)
4. q ← 0 // numbers of characters matched
5. for i ← 1 to n // scan S from left to right
6. do while q > 0 and P [q + 1] ≠ T [i]
7. do q ← Π [q] // next character does not match
8. If P [q + 1] = T [i]
9. then q ← q + 1 // next character matches
10. If q = m // is all of p matched?
11. then print "Pattern occurs with shift" i - m
12. q ← Π [q] // look for the next match
Running Time Analysis
The for loop beginning in step 5 runs 'n' times, i.e., as long as the length of
the string 'S.' Since step 1 to step 4 take constant times, the running time is
dominated by this for the loop. Thus running time of the matching function is
O (n).
Example
Given a string 'T' and pattern 'P' as follows:
Let us execute the KMP Algorithm to find whether 'P' occurs in 'T.'
For 'p' the prefix function, ? was computed previously and is as follows:
Solution
Initially: n = size of T = 15
m = size of P = 7
Pattern 'P' has been found to complexity occur in a string 'T.' The total
number of shifts that took place for the match to be found is i-m = 13 - 7 = 6
shifts.
Boyer-Moore Algorithm
Robert Boyer and J Strother Moore established it in 1977. The B-M String
search algorithm is a particularly efficient algorithm and has served as a
standard benchmark for string search algorithm ever since.
Robert Stephen Boyer
The B-M algorithm takes a 'backward' approach: the pattern string (P) is
aligned with the start of the text string (T), and then compares the characters
of a pattern from right to left, beginning with rightmost character.
If a character is compared that is not within the pattern, no match can be
found by analyzing any further aspects at this position so the pattern can be
changed entirely past the mismatching character.
For deciding the possible shifts, B-M algorithm uses two preprocessing
strategies simultaneously. Whenever a mismatch occurs, the algorithm
calculates a variation using both approaches and selects the more significant
shift thus, if make use of the most effective strategy for each case.
The two strategies are called heuristics of B - M as they are used to reduce
the search. They are:
Bad Character Heuristics

Good Suffix Heuristics
1. Bad Character Heuristics

This Heuristics has two implications:
Suppose there is a character in a text in which does not occur in

a pattern at all. When a mismatch happens at this character
(called as bad character), the whole pattern can be changed,
begin matching form substring next to this 'bad character.'
On the other hand, it might be that a bad character is present in
the pattern, in this case, align the nature of the pattern with a
bad character in the text.
Thus in any case shift may be higher than one.

Example
Let Text T = <nyoo nyoo> and pattern P = <noyo>
Example
If a bad character doesn't exist the pattern then.
Problem in Bad-Character Heuristics
In some cases, Bad-Character Heuristics produces some negative shifts.
Example
This means that we need some extra information to produce a shift on

encountering a bad character. This information is about the last position of
every aspect in the pattern and also the set of characters used in a pattern
(often called the alphabet ∑of a pattern).
COMPUTE-LAST-OCCURRENCE-FUNCTION (P, m, ∑ )
1. for each character a ∈ ∑
2. do λ [a] = 0
3. for j ← 1 to m
4. do λ [P [j]] ← j
5. Return λ
2. Good Suffix Heuristics
A good suffix is a suffix that has matched successfully. After a mismatch
which has a negative shift in bad character heuristics, look if a substring of
pattern matched till bad character has a good suffix in it, if it is so then we
have an onward jump equal to the length of suffix found.
Example
COMPUTE-GOOD-SUFFIX-FUNCTION (P, m)
1. Π ← COMPUTE-PREFIX-FUNCTION (P)
2. P'← reverse (P)
3. Π'← COMPUTE-PREFIX-FUNCTION (P')
4. for j ← 0 to m
5. do ɣ [j] ← m - Π [m]
6. for l ← 1 to m
7. do j ← m - Π' [L]
8. If ɣ [j] > l - Π' [L]
9. then ɣ [j] ← 1 - Π'[L]
10. Return ɣ
BOYER-MOORE-MATCHER (T, P, ∑)
1. n ←length [T]
2. m ←length [P]
3. λ← COMPUTE-LAST-OCCURRENCE-FUNCTION (P, m, ∑ )
4. ɣ← COMPUTE-GOOD-SUFFIX-FUNCTION (P, m)
5. s ←0
6. While s ≤ n - m
7. do j ← m
8. While j > 0 and P [j] = T [s + j]
9. do j ←j-1
10. If j = 0
11. then print "Pattern occurs at shift" s
12. s ← s + ɣ[0]
13. else s ← s + max (ɣ [j], j - λ[T[s+j]])
Complexity Comparison of String Matching Algorithm
Algorithm Preprocessing Matching Time
Time
Naive O (O (n - m + 1)m)
Rabin-Karp O(m) (O (n - m + 1)m)
Finite Automata O(m|∑|) O (n)
Knuth-Morris-Pratt O(m) O (n)
Boyer-Moore O(|∑|) (O ((n - m + 1) + |
∑|))

1 - Design and Analysis of Algorithms by Karamagi, Robert

Uploaded by

Copyright:

Available Formats

1 - Design and Analysis of Algorithms by Karamagi, Robert

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1 - Design and Analysis of Algorithms by Karamagi, Robert

Uploaded by

Copyright:

Available Formats

Contents

Correctness: It should produce the output according to the

Worst-case: f (n) defined by the maximum number of steps

Algorithm Design Techniques

Divide the original problem into a set of subproblems.

2. Greedy Technique: Greedy method is used to solve the optimization

k1 * g (n) ≤ f(n)≤ k2 g(n)for all n, n≥ n0

Total Computation = (max (tA,tB))

Guess the Solution.

≤c log + 1 = c logn-clog2 2+1

T (n) ≤2c log +n

Recursion Tree Method

n is the size of the problem.

If f (n) = for some constant ε >0, then it follows that:

T (n) = 8 T apply master theorem on it.

Compare T (n) = 8 T with

Put all the values in: f (n) =

T (n) = 2 , solve the recurrence by using the master method.

As compare the given problem with T (n) = a T

Put all the values in f (n) =Θ , we will get

f for some constant c<1 for large value of n ,then :

Compare the given problem with T (n) = a T

Put all the values in f (n) = Ω ..... (Eq. 1)

Now, i=1, j=4 then A [4] =3

Now, i=1, j=7 then A [7] =9

Now, i=4, j= 6, 7 No change

1. Input: n elements are given.

In pass 1: n-1 comparisons are required

Finally, the sorted list is:

and i=2-1=1, i=1

Divide and Conquer

1. Divide the original problem into a set of subproblems.

Generally, we can follow the divide-and-conquer approach in a three-step

1. Maximum and Minimum Problem

1. Relational Formula: It is the formula that we generate from the given

Max - Min Problem

Similarly, apply the same procedure recursively on each subproblem or

Recursion will stop, when → (Eq. 4)

1. Input: an array A of size n, already sorted in the ascending or

Set BEG = 1 and END = n

T (n) = ...... (Equation 1)

Is the last term of the equation and it will be equal to 1

Functions: MERGE (A, p, q, r)

Sorting two halves will be taken at the most 2T( ) time

Put 2 equation in 1 equation

Putting 4 equation in 3 equation

From Stopping Condition:

So, total time taken T (n) = T (n-1)+1+ T(n-1)

It is a Geometric Progression Series with common ratio, r=2

1. Length [A],number of elements in array

Representation of an array of the above figure is given below:

After MAX-HEAPIFY (A, 4) and i=4

In this Figure, this node has its key increased to 15.

Relational Formula for Worst Case

In place of 2 and so on.

T (n) = (n-1) T (1) +T (1) + -1

Worst Case Complexity of Quick Sort is T (n) =O (n2)

So Relational Formula for Randomized Quick Sort is:

n T (n) = n (n+1) +2 (T(0)+T(1)+T(2)+...T(n-1)........eq 1