Design and Analysis of Algorithms

Lecture-1

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
1
 Syllabus
Unit-I:
Introduction: Algorithms, Analyzing Algorithms, Complexity of
Algorithms, Growth of Functions, Performance
Measurements, Sorting and Order Statistics - Shell Sort, Quick
Sort, Merge Sort, Heap Sort, Comparison of Sorting
Algorithms, Sorting in Linear Time.

Unit-II:
Advanced Data Structures: Red-Black Trees, B–Trees,
Binomial Heaps, Fibonacci Heaps, Tries, Skip List.

2
 Syllabus
Unit-III:
Divide and Conquer with Examples Such as Sorting, Matrix
Multiplication, Convex Hull and Searching. Greedy Methods
with Examples Such as Optimal Reliability Allocation,
Knapsack, Minimum Spanning Trees – Prim’s and Kruskal’s
Algorithms, Single Source Shortest Paths - Dijkstra’s and
Bellman Ford Algorithms.

3
 Syllabus
Unit-IV:
Dynamic Programming with Examples Such as Knapsack. All
Pair Shortest Paths – Warshal’s and Floyd’s Algorithms,
Resource Allocation Problem. Backtracking, Branch and
Bound with Examples Such as Travelling Salesman Problem,
Graph Coloring, n-Queen Problem, Hamiltonian Cycles and
Sum of Subsets.

Unit-V:
Selected Topics: Algebraic Computation, Fast Fourier
Transform, String Matching, Theory of NP Completeness,
Approximation Algorithms and Randomized Algorithms
4
 Text books

1. Thomas H. Coreman, Charles E. Leiserson and Ronald L.

Rivest, “Introduction to Algorithms”, Printice Hall of India.

2. E. Horowitz & S Sahni, "Fundamentals of Computer

Algorithms",

5
 Course Outcome
CO1 Design new algorithms, prove them correct, and analyze
their asymptotic and absolute runtime and memory
demands.
CO2 Find an algorithm to solve the problem (create) and prove
that the algorithm solves the problem correctly (validate).
CO3 Understand the mathematical criterion for deciding
whether an algorithm is efficient, and know many
practically important problems that do not admit any
efficient algorithms.
CO4 Apply classical sorting, searching, optimization and graph
algorithms.
CO5 Understand basic techniques for designing algorithms,
including the techniques of recursion, divide-and-conquer,
6
and greedy.
 Definition of Algorithm

An algorithm is any well-defined computational

procedure that takes some value, or set of values, as input
and produces some value, or set of values, as output.

An algorithm is thus a sequence of computational

steps that transform the input into the output.

An algorithm is a set of steps to accomplish or

complete a task that is described precisely enough
that a computer can run it.

7
 Characteristics of an algorithm

 Input: Algorithm must contain 0 or more input values.

Output: Algorithm must contain 1 or more output
values.
Finiteness: Algorithm must complete after finite
number of steps.
Definiteness: Each instruction of the algorithm must
be defined precisely or clearly.
Effectiveness: Every instruction must be basic i.e.
simple instruction.

8
 Algorithm vs Program

Algorithm Program
•Simple English like language, •Predefined syntax and grammar,
with no predefined syntax. compatible to a programming
•Person writing algorithm must language.
have domain knowledge. •Developed and made by
•Critical usage in Design Phase of programmer
SDLC. •Used in Coding phase of SDLC.
•H/W and OS independent. •H/W and OS dependent.
•Analysis is done to ensure its •Testing is done to check its
efficiency. correctness.

9
 Pseudo Code
Pseudo code gives a high-level description of an algorithm
without the ambiguity associated with plain text but also
without the need to know the syntax of a particular
programming language.
It is an artificial and informal language that helps
programmer to develop a program.
It is text based detailed design tool.
Pseudo Code is an intermediatary between an algorithm
and a program.

10
 Pseudo Code Vs Algorithm: Example

Algorithm: Insertion Sort:

Input: A list L of integers of length n
Output: A sorted list L1 containing those integers present in L
Step 1: Keep a sorted list L1 which starts off empty
Step 2: Perform Step 3 for each element in the original list L
Step 3: Insert it into the correct position in the sorted list L1.
Step 4: Return the sorted list
Step 5: Stop

11
 Pseudo Code Vs Algorithm: Example
Pseudo Code : Insertion Sort
for j=2 to A.length
Key=A[j]
//Insert A[j] into sorted sequence A[1…j-1
i=j-1
while(i>0 && A[i]> key)
A[i+1]=A[i]
i=i-1
A[i+1]=key

12
 Pseudo code conventions

We use the following conventions in our pseudo code.

 Indentation is used to indicate a block.
 The looping constructs while, for, and repeat-until
and the if-else conditional construct have
interpretations similar to those in C.
 The symbol “//” indicates that the remainder of the
line is a comment.
 A multiple assignment of the form ij  e assigns to
both variables i and j the value of expression e; it
should be treated as equivalent to the assignment j  e
followed by the assignment i  j .
13
 Pseudo code conventions

 We access array elements by specifying the array name

followed by the index in square brackets. For example,
A[i] indicates the ith element of the array A.
 The notation “..” is used to indicate a range of values
within an array. Thus, A[1..j] indicates the sub-array of
A consisting of the j elements A[1], A[2],…………,A[j].
 The Boolean operators “and” and “or” are used.
 A return statement immediately transfers control back
to the point of call in the calling procedure.

14
 Design Strategy of Algorithm

 Divide and Conquer

 Dynamic Programming
 Branch and Bound
 Greedy Approach
 Backtracking
 Randomized Algorithm

15
 Analysis of algorithms

Analysis of algorithms is the determination of the

amount of time and space resources required to execute it.

Time complexity: T he amount of time required by an

algorithm is called time complexity.

Space complexity: T he amount of space/memory

required by an algorithm is called space complexity.

16
 Analysis of algorithms

The main concern of analysis of algorithms is the

required time or performance. Generally, we perform the
following types of analysis −

Worst case − The maximum number of steps taken on

any instance of size n.
Best case − The minimum number of steps taken on any
instance of size n.
Average case − An average number of steps taken on any
instance of size n.

17
 Running time of an algorithm

 Running time of an algorithm is the time taken by the

algorithm to execute it successfully.
 The running time of an algorithm on a particular input
is the number of primitive operations or "steps"
executed.
 It is convenient to define the notion of step so that it is
as machine independent as possible.
 It is denoted by T(n), where n is the input size.

18
 Insertion Sort

Ex. Sort the following elements by insertion sort:- 4, 3, 2,

10, 12, 1, 5, 6.

19
 Insertion Sort Algorithm

Insertion_Sort(A)
1. n length[A]
2. for j2 to n
1. aA[j]
2. //Insert A[j] into the sorted sequence A[1 .. j-1].
3. i j-1
4. While i > 0 and a < A[i]
1. A[i+1]A[i]
2. i i-1
5. A[i+1]  a

20
 Insertion Sort Algorithm Analysis
Insertion_Sort(A)
Instruction cost times
n length[A] c1 1
for j2 to n c2 n
aA[j] c3 n-1
//Insert A[j] into the sorted 0 n-1
sequence A[1 .. j-1].
i j-1 c4 n-1
𝑛
∑𝑗=
While i > 0 and a < A[i] c5 2 𝑡𝑗𝑗
c6 𝑛
∑𝑗=
A[i+1]A[i] 2 (𝑡𝑗𝑗-1)
i i-1 c7 𝑛
∑𝑗= (𝑡𝑗𝑗-1)
2
A[i+1]  a c8 n-1

21
 Time complexity of Insertion sort

T(n) = c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5.∑𝑗= 𝑛 𝑡

2 𝑗
+ c6.∑𝑛𝑗=2 (𝑡𝑗𝑗-1) + c7 𝑗=2 (𝑡𝑗𝑗-1) + c8.(n-1) ………..(1)
.∑𝑛
Here, there will be three case.
(1) Best case
(2) Worst case
(3) Average case

22
 Time complexity of Insertion sort

Best case: This case will be occurred when data is already

sorted.
In this case, value of t j will be 1. Put the value of t j in
equation (1) and find T(n). Therefore,
𝑛
T(n) = c1.1 + c2.n + c 3.(n-1) + c 4.(n-1) + c 5.∑ 𝑗= 2 1
+ c6.∑ 𝑗𝑛=2 (1-1) + c7.∑𝑗𝑛=2 (1-1) + c8.(n-1)
= (c2+c3+c4+c5+c8).n + (c1- c3 - c4 - c5 - c8)
= an + b
Clearly T(n) is in linear form, therefore,
T(n) = θ(n)

23
 Time complexity of Insertion sort
Worst case: This case will be occurred when data is in reverse sorted
order.
In this case, value of tj will be j. Put the value of tj in equation (1) and
find T(n). Therefore,
T(n) = c1.1 + c2.n + c3.(n-1) + c 4.(n-1) + c 5.∑ 𝑗𝑛=2 𝑗
+ c6.∑ 𝑗𝑛=2 (j-1) + c7.∑𝑛𝑗=2 (𝑗𝑗-1) + c .(n-1)
8
= c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. (n+2).(n-1)/2
+ c6.n(n-1)/2+ c7. n(n-1)/2+ c8.(n-1)
= (c5+c6+c7).n2/2+ (c2+c3+c4+(c5 – c6- c7)/2+c8).n +
(c1- c3 - c4 - c5 - c8)
= an2 + bn + c
Clearly T(n) is in quadratic form, therefore, T(n) = θ(n2)

24
 Time complexity of Insertion sort
Average case: This case will be occurred when data is in any order
except best and worst case.
In this case, value of tj will be j/2. Put the value of tj in equation (1) and
find T(n). Therefore,
T(n) = c1.1 + c2.n + c3.(n-1) + c 4.(n-1) + c 5.∑ 𝑗𝑛=2 (𝑗𝑗/2)
+ c6.∑ 𝑗𝑛=2 (j/2-1) + c7.∑𝑛𝑗=2 (𝑗𝑗/2-1) + c .(n-1)
8
= c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. (n+2).(n-1)/4
+ c6.(n-2)(n-1)/4+ c7. (n-2)(n-1)/4+ c8.(n-1)
= (c5+c6+c7).n2/4+ (c2+c3+c4+(c5 – 3c6- 3c7)/4+c8).n +
(c1- c3 - c4 - c5/2 + c6/2+ c7/2 - c8)
= an2 + bn + c
Clearly T(n) is in quadratic form, therefore, T(n) = θ(n2)

25
Design and Analysis of Algorithms

Lecture-2

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
26
 Some other sorting algorithms
 Selection Sort

27
 Some other sorting algorithms
Selection_sort(A)
n ←length[A]
for i←1 to n-1
min ← i
for j←i+1 to n
if(A[j] < A[min])
min ←j
Interchange A[i] ↔ A[min]

Time complexity, T(n) = θ (n2)

28
Some other sorting algorithms
Bubble Sort

29
Some other sorting algorithms
Bubble _Sort(A)

30
 Divide and Conquer approach
The divide-and-conquer paradigm involves three steps at each level
of the recursion:

Divide the problem into a number of sub-problems that are smaller

instances of the same problem.

Conquer the sub-problems by solving them recursively. If the sub-

problem sizes are small enough, however, just solve the sub
problems in a straightforward manner.

Combine the solutions to the sub-problems into the solution for

the original problem.

31
Analysis of Divide and Conquer based algorithm

When an algorithm contains a recursive call to itself, we

can often describe its running time by a recurrence
equation or recurrence, which describes the overall
running time on a problem of size n in terms of the
running time on smaller inputs. We can then use
mathematical tools to solve the recurrence and provide
bounds on the performance of the algorithm.

32
Analysis of Divide and Conquer based algorithm
A recurrence equation for the running time of a divide-
and-conquer algorithm uses the three steps of the basic
paradigm.

The recurrence equation for the running time of a divide-

and-conquer algorithm is the following:-

T(n) = aT(n/b) + D(n) + C(n), if n > c

= θ(1) , otherwise
Where, n is the size of original problem. a is the number
of sub-problems in which the original problem divided at
an instant. Each sub-problems has size n/b. c is a small
integer.
33
Merge Sort

34
Merge Sort Algorithm

35
Merge Sort Algorithm

36
 Merge Sort Algorithm Analysis

The recurrence equation for the running time of

merge sort algorithm will be

T(n) = 2T(n/2) + θ(1)+ θ(n), if n > 1

= θ(1) , otherwise

It can be modified as:-

T(n) = 2T(n/2) + θ(n) , if n > 1
= θ(1) , otherwise

37
 Merge Sort Algorithm Analysis

When we solve recurrence equation, modify it

as:-
T(n) = 2T(n/2) + cn , if n > 1
= d , otherwise
Here, c and d are some constants.

38
 Merge Sort Algorithm Analysis
Iterative method:
T(n) = 2T(n/2) + cn
= 2(2T(n/4) + cn/2) + cn
= 2 2 T(n/4) + 2cn
= 2 2 (2T(n/8)+cn/4) + 2cn
= 2 3 T(n/8)+ 3cn
=………………….
= 2 kT(n/2k) + kcn
= nT(1) + c nlog(n) (Let n = 2k )
= dn + cnlog(n) (since T(1) = d)
= θ(nlog(n))

Therefore, T(n) = θ(nlog(n)) 39

Design and Analysis of Algorithms

Lecture-3

Manoj Mishra (Assistance Professor)

Department of Computer Science and
Engineering United College of Engineering and
Research, Prayagraj
40
 Asymptotic Notations
 The notations we use to describe the asymptotic
running time of an algorithm are defined in terms of
functions whose domains are the set of natural
numbers N={ 0, 1, 2, … … … … } .
 We will use asymptotic notations to describe the
running times of algorithms.
 Following notations are used to define the running
time of algorithms.
 notation
2. O-notation
3. Ω-notation
4. o-notation
5. ω-notation
41
 Asymptotic Notations
notation ( Theta notation )
 For a given function g(n), it is denoted by (g(n)).
 It is defined as following:-
(g(n)) = { f(n) ! ∃positive constants c1, c2 and n 0
such that
0 ≤ c1g(n) ≤f(n) ≤ c2g(n), ∀ n ≥ n0 }

 This notation is
said to be tight
bound.
 If f(n) ∈ (g(n)) then
f(n) = (g(n)) 42
 Asymptotic Notations
notation ( Big-oh notation )
 For a given function g(n), it is denoted by (g(n)).
 It is defined as following:-
(g(n)) = { f(n) ! ∃ positive constants c and n 0 such
that
0 ≤ f(n) ≤ cg(n), ∀ n ≥ n0 }

 This notation is said

to be upper bound.
 If f(n) ∈ (g(n)) then
f(n) = (g(n))
 If f(n) = (g(n)) then
f(n) = (g(n))
43
 Asymptotic Notations
Ωnotation ( Big-omega notation )
 For a given function g(n), it is denoted by Ω(g(n)).
 It is defined as following:-
Ω(g(n)) = { f(n) ! ∃ positive constants c and n 0 such
that
0 ≤ cg(n) ≤f(n), ∀ n ≥ n0 }

 This notation is said

to be lower bound.
 If f(n) ∈ Ω(g(n)) then
f(n) = Ω(g(n))
 If f(n) = (g(n)) then
f(n) = Ω(g(n))
44
 f(n) = (g(n)) iff f(n) = Ω(g(n)) and f(n) = (g(n))
 Asymptotic Notations
notation ( little-oh notation )
 The asymptotic upper bound provided by O-
notation may or may not be asymptotically tight.
 o-notation denotes an upper bound that is not
asymptotically tight.
 For a given function g(n), it is denoted by (g(n)).
 It is defined as following:-
(g(n)) = { f(n) ! for any positive constants c, there
exists a constant n0 such that
0 ≤ f(n) < cg(n), ∀ n ≥n0 }

45
 Asymptotic Notations
𝜔notation ( little-omega notation )
 The asymptotic lower bound provided by 𝛀-
notation may or may not be asymptotically tight.
 𝜔-notation denotes an upper bound that is not
asymptotically tight.
 For a given function g(n), it is denoted by 𝜔(g(n)).
 It is defined as following:-
𝜔 (g(n)) = { f(n) ! for any positive constants c, there
exists a constant n0 such that
0 ≤ cg(n) < f(n), ∀ n ≥n0 }

46
 Asymptotic Notations
Example: Show that (1/2)n2 - 3n = θ(n2).
Solution: Using definition of θ-notation,
c1g(n) ≤ f(n) ≤c2g(n), ∀ n ≥n0
In this question, f(n) = (1/2)n2 - 3n and g(n) = n2 , therefore
c1 n 2 ≤(1/2)n2 - 3n ≤c2 n 2, ∀ n ≥n0
We divide above by n2, we get
c1 ≤(1/2)- (3/n) ≤c2 , ∀ n ≥n 0 … … … … … … . .(1)
Now, we have to find c1, c2 and n0, such that equation (1) is satisfied.
Consider, left part of (1), c1 ≤(1/2)- (3/n) … … … … . (2)
The value of c1 will be positive value less than or equal to the minimum value
of (1/2)- (3/n). Minimum value of (1/2)- (3/n) = 1/14. Therefore, c1 =
1 /14 . This value of c1 will satisfyequation(2)forn≥7.
Here, c1 = 1 /14 and n ≥7 which satisfy (2). 47
 Asymptotic Notations
Consider, right part of (1), (1/2)- (3/n) ≤c2 , … … … … . (3)
The value of c2 will be positive value greater than or equal
to the maximum value of (1/2)- (3/n). Maximum value of
(1/2) - (3/n) = 1/2. Therefore, c2 = 1/2 . This value of c2
will satisfy equation (3) for n ≥1.
Here, c2 = 1/ 2 and n ≥1 which satisfy (3).
Therefore, for c1 = 1/14 , c2 = 1/2 and n0 = 7, equation (1)
is satisfied.
Hence by using definition of θ-notation ,
(1/2)n 2 - 3n = θ(n2).
It is proved.
48
 Asymptotic Notations
Example: Show that 2n+5 = O(n2).
Solution: Using definition of O-notation,
f(n) ≤cg(n) , ∀ n ≥n0
In this question, f(n) = 2n+5 and g(n) = n2 , therefore
2n+5≤cn 2 ∀ n ≥n 0 We divide
above by n2, we get
(2/n)+(5 /n 2) ≤c , ∀ n ≥n0… … … (1)
Now, we have to find c and n0, such that equation (1) is
satisfied.

49
 Asymptotic Notations
The value of c will be positive value greater than or equal to
the maximum value of (2/n)+(5/n 2 ) .
Maximum value of (2/n)+(5/n 2 ) = 7.
Therefore, c = 7.
Clearly equation (1) is satisfied for c = 7 and n ≥1.
Hence by using definition of O-notation ,
2n+5 = O(n2).
It is proved.

50
 Asymptotic Notations
Example: Show that 2n 2 +5n+6 = 𝛀(n).
Solution: Using definition of 𝛀 -notation,
cg(n) ≤f(n) , ∀ n ≥n0
In this question, f(n) = 2n2+5n+6 and g(n) = n , therefore
cn ≤2n2+5n+6 , ∀ n ≥n0
We divide above by n, we get
c ≤2n +5 +(6/n) , ∀ n ≥n0 ………(1)
Now, we have to find c and n0, such that equation (1) is
always satisfied.

51
 Asymptotic Notations
The value of c will be positive value less than or equal to the
minimum value of 2n + 5 + (6/n) .
Minimum value of 2n + 5 + (6/n) = 12.
Therefore, c = 12.
Clearly equation (1) is satisfied for c = 12 and n ≥2.
Hence by using definition of 𝛀 -notation ,
2n 2 +5n+6 = 𝛀 (n).
It is proved.

52
 Asymptotic Notations
Example: Show that 2n2 = o(n3).
Solution: Using definition of o-notation,
f(n) < cg(n) , ∀ n ≥n 0
Here, f(n) = 2n2, and g(n) = n3. Therefore,
2n2 < cn3 , ∀ n ≥n 0
We divide above by n3, we get
(2/n) < c , ∀ n ≥n 0 … … … … (1)
for c = 1, there will be n0 = 3, which satisfy (1).
for c = 0.5, there will be n0 = 7, which satisfy (1).
Therefore, for every c, there exists n0 which satisfy (1).
Hence 2n2 = o(n3). 53
 Asymptotic Notations
Example: Show that 2n2 ≠o(n2).
Solution: Using definition of o-notation,
f(n) < cg(n) , ∀ n ≥n 0
Here, f(n) = 2n2, and g(n) = n2. Therefore,
2n2 < cn2 , ∀ n ≥n 0
We divide above by n2, we get
2 < c, ∀ n ≥n 0 … … … … (1)
Clearly for c = 1, inequality (1) does not satisfy.
Therefore, for every c, there does not exist n0 which satisfy
(1). Hence 2n2 ≠o(n2).
54
 Asymptotic Notations
Example: Show that 2n2 = 𝜔(n).
Solution: Using definition of 𝜔 -notation,
cg(n) < f(n) , ∀ n ≥n 0
Here, f(n) = 2n2, and g(n) = n. Therefore,
cn < 2n2 , ∀ n ≥n 0
We divide above by n, we get
c < 2n , ∀ n ≥n 0 … … … … (1)
for c = 1, there will be n0 = 1, which satisfy (1).
for c = 10,there will be n0 = 6, which satisfy (1).
Therefore, for every c, there exists n0 which satisfy (1).
Hence 2n2 = 𝜔(n). 55
 Asymptotic Notations
Example: Show that 2n2 ≠𝜔 (n2).
Solution: Using definition of 𝜔 -notation,
cg(n) < f(n) , ∀ n ≥n 0
Here, f(n) = 2n2, and g(n) = n2. Therefore,
cn2 < 2n2 , ∀ n ≥n 0
We divide above by n2, we get
c < 2, ∀ n ≥n 0 … … … … (1)
Clearly for c = 3, there does not exists n0, which satisfy (1).
Therefore, for every c, there does not exist n 0 which satisfy
(1). Hence 2n2 ≠𝜔 (n2).
56
Design and Analysis of Algorithms

Lecture-4

Manoj Mishra(Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
57
 Asymptotic Notations
Example: Show that using definition of notations
(a) 3n3-10n+50 = θ(n3)
(b) 5n2-100n ≠ θ(n3)
(c) 3n3-10n+50 = O(n3)
(d) 5n2-100n ≠ O(n)
(e) 3n3-10n+50 = 𝛀(n3)
(f) 5n2-100n ≠ 𝛀(n3)

58
 Asymptotic Notations
Limit based method to compute notations for a function
𝑓 𝑛
First compute lim = c.
𝑛→∞ 𝑔 𝑛

(1) If c is a constant such that 0< c < ∞, then f(n) = θ(g(n)).

(2) If c is a constant such that 0 ≤ c < ∞, then f(n) = O(g(n)).

(3) If c is a constant such that 0< c ≤ ∞, then f(n) = 𝛀(g(n)).

(4) If c is a constant such that c = 0, then f(n) = o(g(n)).

(5) If c is a constant such that c = ∞, then f(n) = 𝜔(g(n)).

59
 Asymptotic Notations
Exercises

1. Let f(n) and g(n) be asymptotically non-negative

functions. Using the basic definition of ‚θ-notation,
prove that max(f(n), g(n)) = θ(f(n)+g(n)).

2. Show that for any real constants a and b, where b > 0,

(n+a)b = θ(nb)

3. Solve the followings:-

(a) Is 2n+1 = O(2n) ?
(b) Is 22n = O(2n) ? 60
 Asymptotic Notations
Exercise(cont.)
7.Arrange the following in ascending order of growth
or rank the following functions by order of growth.
n3, (3/2)n, 2n, n2, log(n), 22n, loglog(n), n!, en.
8.Let f(n) and g(n) be two asymptotically positive
functions. Prove or disprove the following:-
(a) f(n) = O(g(n)) implies g(n) = O(f(n)).
(b) f(n) + g(n) = θ(min(f(n),g(n))).
(c) f(n) = O(g(n)) implies lg(f(n)) = O(lg(g(n))), where
lg(g(n)) ≥1 and f(n) ≥ 1 for all sufficiently large n.
(d)f(n) = O(g(n)) implies 2f(n) = O(2g(n)).
(e) f(n) = O(((f(n))2)
61
 Asymptotic Notations
AKTU questions
1. Take the following list of functions and arrange them in
ascending order of growth rate. That is, if function g(n)
immediately follows function f(n) in your list, then it
should be the case that f(n) is O(g(n)). f1(n) = n2.5, f2(n) =
√2n, f3(n) = n + 10, f4(n) = 10n, f5(n) = 100n, and
f6(n) = n2 log n

2. Rank the following by growth rate: n, 2lg √n, log n, log

(logn), log2n, (lgn)lgn, 4, (3/2)n, n!

62
Design and Analysis of Algorithms

Lecture-5
Recurrence Relation

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
63
 Recurrence Relation

•A Recurrence is an equation or inequality that

describes a function in terms of its value on smaller
input.
•The recurrence describes the running time of an
algorithm that divides a problem of size n into a
number of sub-problems.

64
65
66
 Recurrence relation
Recurrence equations will be of the following form:-

(1) T(n) = aT(n/b) + f(n)

(2) T(n) = T(n-1) + n
(3) T(n) = T(n/3) + T(2n/3) + n
(4) T(n) = T(n-1) + T(n-2)

Some approaches to sole recurrence relations

(1) Iterative method
(2) Substitution method
(3) Recurrence Tree
(4) Master theorem method
67
 Substitution method

The substitution methodfor solving recurrences comprises

two steps:

1. Guess the form of the solution.

2.Use mathematical induction to find the constants and show

that the solution works.

68
 Back Substitution Method

Example: Solve the given recurrence

relation using substitution method:
T(n) = 1, n=0
T(n – 1) + 1, n > 0

69
Solution

70
 Example

Solve the given recurrence relation

using substitution method:
T(n) = 1, n=0
T(n – 1) + n, n > 0

71
Solution

72
 Example

Solve the given recurrence relation

using substitution method:
T(n) = 1, n=1
T(n/2) + n, n > 1

73
Solution

74
 Substitution method

Example: Find the upper bound of following recurrence

relation T(n) = 2T(⌊n/2⌋) + n. ………..(1)

Solution: (Do yourself)

75
 Substitution method
Example: Find the upper bound of following recurrence
relation T(n) = 2T(⌊n/2⌋ + 17) + n

Solution: ( Do yourself)

76
 Substitution method
Example: Solve the following recurrence relation
T(n) = 2T(⌊√n⌋ ) + lg n
Solution: ( Do yourself)

77
 Substitution method
Example: Solve the following recurrence relation T(n) = T(⌊n/2⌋ )
+ T(⌈ n/2 ⌉ ) + 1

Solution: (Do yourself)

78
Design and Analysis of Algorithms

Lecture-6
Recurrence Relation

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
79
 Recurrence tree method
In a recursion tree, each node represents the cost of a
single sub-problem somewhere in the set of recursive
function invocations. We sum the costs within each level of
the tree to obtain a set of per-level costs, and then we sum
all the per-level costs to determine the total cost of all
levels of the recursion.

A recursion tree is best used to generate a good guess,

which you can then verify by the substitution method.

80
 Example

Solve the given recurrence relation using

recurrence tree method:
T(n) = 1, n=0
T(n – 1) + n, n > 0

81
Solution

82
 Example

Solve the given recurrence relation using

recurrence tree method:
T(n) = 1, n=1
T(n/2) + n, n > 1

83
Solution

84
 Recurrence tree method
Example: Solve the following recurrence equation
T(n) = 3T(⌊n/4⌋) + θ(n2)
Solution: (Do Yourself)

85
 Recurrence tree method
Example: Solve the following recurrence relation using recurrence
tree method
T(n) = T(n/3) + T(2n/3) + θ(n)
Solution: (Do Yourself)

86
 Recurrence tree method
Exercise
(1)Draw the recursion tree for T(n) = 4 T(⌊n/2⌋) + cn, where
c is a constant and provide a tight asymptotic bound on its
solution. Verify your bound by the substitution method.
(2)Use a recursion tree to give an asymptotically tight
solution to the recurrence T(n) = T(n-a) + T(a) + cn, where a
≥1 and c > 0 are constants.
(3)Use a recursion tree to give an asymptotically tight
solution to the recurrence T(n) = T(αn) + T((1- α)n) + cn,
where α is a constant in the range 0 < α < 1 and c > 0 is also
a constant.

87
Design and Analysis of Algorithms

Lecture-7
Recurrence Relation

Manoj Mishra (Assistance Professor)

Department of Computer Science and
Engineering United College of Engineering and
Research, Prayagraj
88
 Master Theorem
Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and let
T(n) be defined on the nonnegative integers by the
recurrence
T(n) = aT(n/b) + f(n);
Where we interpret n/b to mean either ⌊n/b⌋ or ⌈n/b⌉. Then
T(n) has the following asymptotic bounds:
1. If f(n) = 𝑂(nlogba − ϵ) for some constant ϵ> 0, then
T(n) = θ(nlogba).
2. If f(n) =θ(nlog ba) , then T(n) = θ(nlog balg n).
3. If f(n) = 𝛺(nlogba + ϵ) for some constant ϵ> 0, and if
af(n/b) ≤ cf(n) for some constant c < 1 and all sufficiently
large n, then T(n) = θ(f(n).
89
 Solution Steps using Master theorem
•General form of recurrence relation is T(n) = =aT(n/b) + f(n)
where a>= 1, b > 1 and f(n) = Θ(nk logpn)
From above we find two values as:
(1) logba (2) k
Based on above two values there are three cases to solve recurrence relation:
Case I
If logba > k then solution is Θ(nlogba )
Case II

If logba = k,
(i) if p > -1 then solution will be Θ(nk logp+1 n)
(ii) if p = -1 then Θ(nk log logn)
(iii) ifp < -1 then Θ (nk )

Case III
if logba < k,
(i) p>= 0 then solution is Θ(nk logp n)
(ii) p < 0 then solution is Θ (nk )
90
 Example

1- Solve the following recurrence relation :

(i)T(n) = 2T(n/2) + 1
(ii)T(n) = 4T(n/2) + n
(iii)T(n) 8T(n/2) + n
(iv) T(n) = 9T(n/3) +1
2- Solve the following recurrence relation:
(i)T(n) = 2T(n/2) + n
(ii)T(n) = 2T(n/2) + n/logn
(iii) T(n) = 2T(n/2) + n/ log2 n
3- Solve the following recurrence relation:
T(n) = T(n/2) + n2

91
 Master Theorem Method
Example: Solve the following recurrence relations using
master theorem method
(a) T(n) = 9T(n/3) + n
(b) T(n) = T(2n/3) + 1
(c) T(n) = 3T(n/4) + n log n
Solution:
(a) Consider T(n) = 9T(n/3) + n
In this recurrence relation, a = 9, b= 3 and f(n) = n.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog 39 = n2
Clearly, n𝑙𝑜𝑔 𝑏𝑎 > f(n) , therefore case1 can be applied.
Now determine ϵ such that f(n) = 𝑂(n2−ϵ) . Here ϵ = 1.
Therefore case 1 will be applied.
Hence solution will be T(n) = θ(n2).
92
 Master Theorem Method
Solution:
(b) Consider T(n) = T(2n/3) + 1
In this recurrence relation, a = 1, b= 3/2 and f(n) = 1.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog3/2 1 = 0
Clearly, f(n) =θ(nlog ba) , therefore case 2 will be applied.
Hence solution will be T(n) = θ(log n).

93
 Master Theorem Method
Solution:
(c) Consider T(n) = 3T(n/4) + n log n
In this recurrence relation, a = 3, b= 4 and f(n) = n log n.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog 43 = n0.793
Clearly, n𝑙𝑜𝑔 𝑏𝑎 < f(n) , therefore case 3 can be applied.
Now determine ϵ such that f(n) =𝛺(n0.793+ϵ) . Here ϵ = 0.207.
Now, af(n/b) ≤ cf(n) imply that 3f(n/4) ≤ cf(n)
⇒ 3(n/4)log (n/4) ≤ c n log n
⇒ (¾)log(n/4) ≤ c log n
Clearly above inequality is satisfied for c = 3/ 4. Therefore
case 3 will be applied.
Hence solution will be T(n) = θ(n log n).
94
 Master theorem method
Example: Solve the following recurrence relation
T(n) = 2T(n/2) + n log n
Solution: Here, a = 2 , b=2 and f(n) = n log n.
nlogba = nlog 22 = n
If we compare nlog a and f(n), we get f(n) is greater than nlog a
b b
. Therefore, case 3 may be applied.
Now we have to determine ϵ >0 which satisfy f(n) = 𝛺
(nlogba+ϵ), i.e. n logn = 𝛺(n1+ϵ). Clearly there does not exist
any ϵ which satisfy this condition. Therefore case 3 can not
be applied. Other two cases are also not satisfied. Therefore
Master theorem can not be applied in this recurrence
relation. 95
 Generalized Master theorem
Theorem: If f(n) = θ(n log ba lgkn), where k ≥ 0, then the
solution of recurrence will be T(n) = θ(nlogba lgk+1n).

Now, consider the previous example:- T(n) =

2T(n/2) + n log n
Solve it using aboe theorem,
Here a =2, b= 2, and k = 1. T herefore, the solution of this
recurrence will be
T(n) = θ(nlog 2 lg1+1n)
2

= θ(n lg2n)

Hence, T(n) = θ(n lg2n)

96
 Recurrence relation
Exercise
1.Use the master method to give tight asymptotic bounds for
the following recurrences:-
(a) T(n) = 8T(n/2) + θ(n2)
(b) T(n) = 7T(n/2) + θ(n2)
(c) T(n) = 2T(n/4) + 1
(d) T(n) = 2T(n/4) + √n
2.Can the master method be applied to the recurrence
4T(n/2) + n2 log n ? Why or why not? Give an asymptotic
upper bound for this recurrence.
97
Design and Analysis of Algorithms

Lecture-8

Manoj Mishra (Assistance Professor)

Department of Computer Science and
Engineering United College of Engineering and
Research, Prayagraj
98
 Recurrence relation
3. Give asymptotic upper and lower bounds for T(n) in each
of the following recurrences. Assume that T(n) is constant
for n ≤2. Make your bounds as tight as possible, and justify
your answers.
(a) T(n) = 2T(n/2) + n4
(b) T(n) = T(7n/10) + n
(c) T(n) = 16T(n/4) + n2
(d) T(n) = 2T(n/4) + √n
(e) T(n) = T(n-2) + n2
(f) T(n) = 7T(n/3) + n2
(g) T(n) = 3T(n/3 -2) + n/2
99
 Recurrence relation
4. Give asymptotic upper and lower bounds for T(n) in each of the
following recurrences. Assume that T(n) is constant for sufficiently small
n. Make your bounds as tight as possible, and justify your answers.
(a) T(n) = 4T(n/3) + n lgn
(b) T(n) = 3T(n/3) + n/lgn
(c) T(n) = 2T(n/2) + n/lgn
(d) T(n) = T(n/2) + T(n/4) + T(n/8) + n
(e) T(n) = T(n-1) + 1/n
(f) T(n) = T(n-1) + lg n
(g) T(n) = T(n-2) + 1/lg n
(h) T(n) = √n T(√n) + n

100
Design and Analysis of Algorithms

Lecture-9
Recurrence Relation

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
101
 AKTU Examination Questions
1. Solve the recurrence T (n) = 2T(n/2) + n 2 + 2n+ 1
2.Solve the recurrence using recursion tree method:
T (n) = T (n/2) + T (n/4) + T (n/8) + n
3. Use a recursion tree to give an asymptotically tight solution
to the recurrence T (n) = T (αn) + T ((1 - α)n) + cn, where α is a
constant in the range 0 <α< 1 and c > 0 is also a constant.
4.The recurrence T (n) = 7T (n/3) + n2 describes the running
time of an algorithm A. Another competing algorithm B has a
running time of S (n) = a S (n/ 9) + n2. What is the smallest
value of ‘a’ such that A is asymptotically faster than B?
5. Solve the recurrence relation by substitution method
T(n)= 2T(n/2) + n
102
 AKTU Examination Questions
6. Show that the solution to T (n) = 2T (⌊n/2⌋ + 17) + n is
O(nlgn).
7. Solve the recurrence: T (n) = 50 T (n/49) + log n!
8.Solve the following recurrence using Master method:
T (n) = 4T (n/3) + n2
9.Find the time complexity of the recurrence relation
T(n) = n +T(n/10)+T(7n/10)
10.Solve the following By Recursion Tree Method
T(n) = n + T(n/5)+T(4n/5)
11.The recurrence T (n) =7T (n/2) +n 2 describe the running
time of an algorithm A. A competing algorithm A has a running
time of T’ (n) =aT’ (n/4) +n2. What is the largest integer value
for a A’ is asymptotically faster than A? 103
Design and Analysis of Algorithms

Lecture-10

Manoj Mishra(Assistance Professor)

Department of Computer Science and
Engineering United College of Engineering and
Research, Prayagraj
104
 Merge sort

Merge sort is defined as a sorting algorithm that

works by dividing an array into smaller subarrays,
sorting each subarray, and then merging the sorted
subarrays back together to form the final sorted
array.
•we can say that the process of merge sort is to divide
the array into two halves, sort each half, and then
merge the sorted halves back together. This process is
repeated until the entire array is sorted.

105
How does Merge Sort work?

Merge sort is a recursive algorithm that continuously splits the

array in half until it cannot be further divided i.e., the array has
only one element left (an array with one element is always
sorted). Then the sorted subarrays are merged into one sorted
array.

106
 Example

Lets consider an array arr[] = {38, 27, 43, 10}

Initially divide the array into two equal halves:
Solution:

107
 Solution Contd..
These subarrays are further divided into two halves. Now they
become array of unit length that can no longer be divided and
array of unit length are always sorted.

108
 Solution Contd..
These sorted subarrays are merged together, and we
get bigger sorted subarrays.

109
 Solution Contd..
This merging process is continued until the sorted array is
built from the smaller subarrays.

110
 Algorithm: Merge Sort
Algorithm MergeSort(l,h)
{
If l < h
{
mid = (l + h) /2 ;
MergeSort(l,mid);
MergeSort(mid + 1, h);
Merge(l,mid,h);
}
}

111
 Complexity Analysis of Merge Sort:

Time Complexity: O(N log(N)), Merge Sort is a recursive

algorithm and time complexity can be expressed as following
recurrence relation.
T(n) = 2T(n/2) + θ(n)
The above recurrence can be solved either using the
Recurrence Tree method or the Master method. It falls in case
II of the Master Method and the solution of the recurrence is
θ(Nlog(N)). The time complexity of Merge Sort is θ(Nlog(N))
in all 3 cases (worst, average, and best) as merge sort always
divides the array into two halves and takes linear time to
merge two halves.
Auxiliary Space: O(N), In merge sort all elements are copied
into an auxiliary array.

112
 Applications of Merge Sort:
Sorting large datasets: Merge sort is particularly well-suited for
sorting large datasets due to its guaranteed worst-case time
complexity of O(n log n).
External sorting: Merge sort is commonly used in external
sorting, where the data to be sorted is too large to fit into
memory.
Custom sorting: Merge sort can be adapted to handle different
input distributions, such as partially sorted, nearly sorted, or
completely unsorted data.

113
Advantage s and Disadvantages of Merge Sort
Advantages of Merge Sort:
Stability: Merge sort is a stable sorting algorithm, which means it maintains
the relative order of equal elements in the input array.
Guaranteed worst-case performance: Merge sort has a worst-case time
complexity of O(N logN), which means it performs well even on large
datasets.
Parallelizable: Merge sort is a naturally parallelizable algorithm, which
means it can be easily parallelized to take advantage of multiple processors
or threads.
Drawbacks of Merge Sort:
Space complexity: Merge sort requires additional memory to store the
merged sub-arrays during the sorting process.
Not in-place: Merge sort is not an in-place sorting algorithm, which means it
requires additional memory to store the sorted data. This can be a
disadvantage in applications where memory usage is a concern.
Not always optimal for small datasets: For small datasets, Merge sort has a
higher time complexity than some other sorting algorithms, such as
insertion sort. This can result in slower performance for very small
114
datasets.
 Heapsort
Heap

 The (binary) heap data structure is an array object

that we can view as a nearly complete binary tree.

 Each node of the tree corresponds to an element

of the array. The tree is completely filled on all
levels except possibly the lowest, which is filled
from the left up to a point.

115
 Heapsort

Max-heap Array
116
 Heapsort
Index: If i the index of a node, then the index of parent and its
child are the following:-
Parent(i) = ⌊i/2⌋
Left(i) = 2i
Right(i) = 2i+1
Note: Root node has always index 1 i.e. A[1] is root element.
Heap-size: Heap-size is equal to the number of elements in the heap.
Height of a node: The height of a node in a heap is the
number of edges on the longest simple downward path from the
node to a leaf.
Height of heap: The height of the heap is equal to the height of its
root.
117
 Heapsort
Types of heap
There are two kinds of binary heaps:
(1) max-heaps (2) min-heaps
Max-heap: The heap is said to be max-heap if it satisfy the
max-heap property.
The max-heap property is that the value at the parent node
is always greater than or equal to value at its children.
Min-heap: The heap is said to be min-heap if it satisfy the
min-heap property.
The min-heap property is that the value at the parent node
is always less than or equal to value at its children.
118
 Heapsort
Heap sort algorithm consists of the following two sub-
algorithms.
(1)Max-Heapify: It is used to maintain the max-heap
property.
(2)Build-Max-Heap: It is used to construct a max-heap for
the given set of elements.

119
 Heapsort
Max-Heapify Algorithm
Action done by max-heapify algorithm is shown in the following
figures:-

120
 Heapsort
Max-Heapify Algorithm

121
 Heapsort
Time complexity of Max-Heapify Algorithm
The running time of max-heapify is determined by the
following recurrence relation:-
T(n) ≤T(2n/3)+ θ(1)
Here n is the size of the sub-tree rooted at node i.
Using master theorem, the solution of this recurrence
relation is
T(n) = θ(lg n)

122
Design and Analysis of Algorithms

Lecture-11

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
123
 Heapsort
Build-Max-Heap Algorithm
Example: Construct max-heap corresponding to the
following elements
4, 1, 3, 2, 16, 9, 10, 14, 8, 7.
Solution:

124
 Heapsort
Build-Max-Heap Algorithm (cont.)

125
 Heapsort
Build-Max-Heap Algorithm (cont.)

The running time of this algorithm is

T(n) = O(n lg n)
But this upper bound is not asymptotically tight.
Now, we shall calculate tight upper bound.
126
 Heapsort
Time complexity of Build-Max-Heap Algorithm
We can derive a tighter bound by observing that the time for MAX-
HEAPIFY to run at a node varies with the height of the node in the tree,
and the heights of most nodes are small.

Our tighter analysis relies on the properties that an n-element heap has
height ⌊lg n⌋ and at most ⌈n/2h+1⌉ nodes of any height h.

If h is the height of the sub-tree then running time of max-

heapify is O(h).
Therefore, total cost of build-max-heap is
T(n) =

127
 Heapsort
Time complexity of Build-Max-Heap Algorithm
Now, T(n) =

=
Since

Therefore,
T(n) = O( 2n)
= O(n)

128
 Heapsort Algorithm
Example: Sort the following elements using heapsort
5, 13, 2, 25, 7, 17, 20, 8, 4.
Solution: ( Do Yourself)

129
 Heapsort Algorithm

Time complexity: The HEAPSORT procedure takes time

O(nlgn), since the call to BUILD-MAX-HEAP takes time
O(n) and each of the n-1 calls to MAX-HEAPIFY takes time
O(lgn).
130
 Heapsort Algorithm
Exercise
1. What are the minimum and maximum numbers of
elements in a heap of height h?
2. Show that an n-element heap has height ⌊lgn⌋.
3. Is the array with values 23, 17, 14, 6, 13, 10, 1, 5, 7, 12 a
max-heap?
4. Show that there are at most ⌈n/2h+1⌉nodes of height h in
any n-element heap.
5. What is the running time of HEAPSORT on an array A of
length n that is already sorted in increasing order? What
about decreasing order?
131
 Heapsort Algorithm
AKTU Examination Questions

1. Sort the following array using Heap-Sort techniques—

5,8,3,9,2,10,1,35,22
2.How will you sort following array A of elements using
heap sort: A = (23, 9, 18, 45, 5, 9, 1, 17, 6).

132
Design and Analysis of Algorithms

Lecture-12

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
133
 Quicksort

Quicksort, like merge sort, applies the divide-and-

conquer paradigm.

Quicksort uses three steps to sort the elements.

 Divide
 Conquer
 Combine

134
 Quicksort
Algorithm
It divides the large array into smaller sub-arrays. And then quicksort
recursively sort the sub-arrays.
Pivot
1. Picks an element called the "pivot".
Partition
2.Rearrange the array elements in such a way that the all values lesser
than the pivot should come before the pivot and all the values greater
than the pivot should come after it.
This method is called partitioning the array. At the end of the partition
function, the pivot element will be placed at its sorted position.
Recursive
3.Do the above process recursively to all the sub-arrays and sort the
elements.
135
 Quicksort
Example: Sort the following elements using quicksort
2, 8, 7, 1, 3, 5, 6, 4
Solution: Here, we pick the last element in the list as a pivot
1 2 3 4 5 6 7 8

2 8 7 1 3 5 6 4

2 8 7 1 3 5 6 4

2 8 7 1 3 5 6 4

2 8 7 1 3 5 6 4

2 1 7 8 3 5 6 4

2 1 3 8 4 5 6 4
136
 Quicksort
1 2 3 4 5 6 7 8

2 1 3 8 7 5 6 4

2 1 3 4 7 5 6 8
Partition completed in first pass

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8
137
Partition completed in second pass
 Quicksort
1 2 3 4 5 6 7 8

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8

1 2 3 4 5 7 6 8

1 2 3 4 5 6 7 8
Partition completed in third pass
1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8
Process
138
completed
1 2 3 4 5 6 7 8
 Quicksort Algorithm

Quicksort(A, p, r)
1 if p < r
2 q = PARTITION(A, p, r)
3 QUICKSORT(A, p, q-1)
4 QUICKSORT(A, q+1, r)

To sort an entire array A, the initial call is QUICKSORT(A, 1,

length[A]).

139
Quicksort Algorithm

140
 Quicksort Algorithm Analysis
Time Complexity:

The running time of quicksort algorithm is

T(n) = T(n1) + T(n2) + θ(n) ……………….(1)
Here n1 and n2 are the size of sub-problems.

The running time of quicksort depends on whether the

partitioning is balanced or unbalanced, which in turn depends
on which elements are used for partitioning.
If the partitioning is balanced, the algorithm runs
asymptotically as fast as merge sort.
If the partitioning is unbalanced, however, it can run
asymptotically as slowly as insertion sort.
141
 Quicksort Algorithm Analysis
Worst-case partitioning
The worst-case behavior for quicksort occurs when the
partitioning routine produces one sub-problem with n-1
elements and one with 0 elements.

Let us assume that this unbalanced partitioning arises in

each recursive call. Therefore,

T(n) = T(0) + T(n-1) + θ(n)

T(n) = T(n-1) + θ(n) (since T(0) = θ(1) )

142
 Quicksort Algorithm Analysis
Worst-case partitioning
After solving this recurrence relation, we get
T(n) = θ(n2)

Therefore the worst-case running time of quicksort is no better

than that of insertion sort.
Moreover, the θ(n2) running time occurs when the input array
is already completely sorted—a common situation in which
insertion sort runs in O(n) time.

143
 Quicksort Algorithm Analysis
Best-case partitioning
If PARTITION produces two sub-problems, each of size no
more than n / 2, since one is of size ⌊n /2⌋ and one of size
⌈n/2⌉-1 . In this
case, quicksort runs much faster. The recurrence for the
running time is then
T(n) = 2T(n/2) + θ(n)
After solving this recurrence relation, we get
T(n) = θ(n lgn)

Note: By equally balancing the two sides of the partition at

every level of the recursion, we get an asymptotically faster
algorithm.
144
 Quicksort Algorithm Analysis
Balanced partitioning
The average-case running time of quicksort is much closer to
the best case than to the worst case.
Suppose that the partitioning algorithm always produces a 9-
to-1 proportional split. In this case, running time will be
T(n) = T(9n/10) + T(n/10) + θ(n)
This relation is equivalent to the following
T(n) = T(9n/10) + T(n/10) + cn
We can solve this recurrence using recurrence tree method.

145
 Quicksort Algorithm Analysis
Recurrence tree will be

146
 Quicksort Algorithm Analysis
Therefore, the solution of recurrence relation will be
T(n)≤cn + cn + cn+……………..+cn
= cn(1+lg10/9n)
= cn + cnlg 10/9n

Therefore, T(n) = O(nlgn)

147
 Quicksort Algorithm Analysis
Exercise
1. Sort the following elements using quicksort
13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11.
2.What is the running time of QUICKSORT when all elements
of array A have the same value?
3.Show that the running time of QUICKSORT is‚ θ(n2) when
the array A contains distinct elements and is sorted in
decreasing order.
4.Suppose that the splits at every level of quicksort are in the
proportion 1-α to α, where 0 < α ≤ 1/2 is a constant. Show that
the minimum depth of a leaf in the recursion tree is
approximately lgn/ lgα and the maximum depth is
approximately lgn/ lg(1-α) . (Don’t worry about integer round-
off.) 148
A randomized version of quicksort

149
 A randomized version of quicksort

The running time of randomized quick sort will be

T(n) = O(nlgn)
150
Design and Analysis of Algorithms

Lecture-13

Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
151
 Sorting in Linear Time
Comparison Sort
In a comparison based sort, we use only comparisons between
elements to gain order information about an input sequence
<a 1, a2,……………., an>. That is, given two elements
ai and aj , we perform one of the tests ai < aj, ai ≤aj , ai=aj , ai
≥aj, or ai > aj to determine their relative order.

The decision-tree model

We can view comparison sorts abstractly in terms of decision
trees.
A decision tree is a full binary tree that represents the
comparisons between elements that are performed by a
particular sorting algorithm operating on an input of a given
152
size.
 Sorting in Linear Time
In a decision tree, we annotate each internal node by i:j for
some i and j in the range 1≤i, j ≤ n, where n is the number of
elements in the input sequence. We also annotate each leaf by
a permutation < 𝑟(1), 𝑟(2), 𝑟(3), … … … … , 𝑟(n) >.

 The execution of the sorting algorithm corresponds to

tracing a simple path from the root of the decision tree
down to a leaf.
 Each internal node indicates a comparison ai ≤ aj . The left
sub-tree then dictates subsequent comparisons once we
know that ai ≤ aj , and the right sub-tree dictates subsequent
comparisons knowing that ai > aj .
 When we come to a leaf, the sorting algorithm has
established the ordering <a𝑟(1), a 𝑟(2) ,…………., a𝑟(n)>. 153
 Sorting in Linear Time
The decision tree operating on three elements is the
following:-

154
 Sorting in Linear Time
Theorem:
Any comparison sort algorithm requires 𝛺(nlg n) comparisons
in the worst case.
Proof: The worst-case number of comparisons for a given
comparison sort algorithm equals the height of its decision
tree.
Consider a decision tree of height h with l reachable leaves
corresponding to a comparison sort on n elements. Because
each of the n! permutations of the input appears as some
leaf, we have n! ≤ l . Since a binary tree of height h has no
more than 2h leaves, therefore
n! ≤l ≤2h ⇒ h ≥ lg(n!)
Therefore h = 𝛺(nlgn)
155
 Counting sort
 Counting sort assumes that each of the n input elements
is an integer in the range 0 to k, for some integer k.

 When k = O(n), the sorting algo. runs in θ(n) time.

 Counting sort determines, for each input element x, the

number of elements less than x. It uses this information to
place element x directly into its position in the output
array.

156
 Counting sort
Example: Sort the following elements using counting sort
2, 5, 3, 0, 2, 3, 0, 3
Solution:

157
Counting sort

158
 Counting sort

 Time complexity of counting sort is

T(n) = θ(n+k)
 If k ≤ n, then T(n) = θ(n)
159
Design and Analysis of Algorithms

Lecture-14

Manoj Mishra (Assistane Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
160
 Stable sorting
A sorting algorithm is said to be stable if two objects with
equal keys appear in the same order in sorted output as they
appear in the input array to be sorted.

Some stable sorting algorithms

1. Counting sort
2. Insertion sort
3. Merge Sort
4. Bubble Sort

Some unstable sorting algorithms

1. Selection sort
2. Quicksort
161
3. Heap sort
 Radix sorting
 This sorting algorithm assumes all the numbers must be of
equal number of digits.

 This algorithm sort all the elements on the basis of digits

used in the number.

 First sort the numbers on the basis of least significant digit.

Next time on the basis of 2nd least significant digit. After it
on the basis of 3rd least significant digit. And so on. At the
last, it sort on the basis of most significant digit.

162
 Radix sorting
Example: Sort the following elements using radix sort
326, 453, 608, 835, 751, 435, 704, 690.
Solution: In these elements, number of digits is 3. Therefore, we
have to used 3 iterations.

163
Radix sorting

164
 Radix sorting
Time complexity of radix sort is
T(n) = θ(d(n+k))

Note: When d is constant and k=O(n), we can make radix

sort run in linear time i.e T(n) = θ(n).

165
 Radix sorting
Time complexity of radix sort is
T(n) = θ(d(n+k))

Note: When d is constant and k=O(n), we can make radix

sort run in linear time i.e T(n) = θ(n).

166
 Bucket sorting
 Bucket sort assumes that the input is generated by a
random process that distributes elements uniformly and
independently over the interval [0,1).

 Bucket sort divides the interval [0,1) into n equal-sized

subintervals, or buckets, and then distributes the n input
numbers into the buckets. Since the inputs are uniformly
and independently distributed over [0,1), we do not expect
many numbers to fall into each bucket. To produce the
output, we simply sort the numbers in each bucket and
then go through the buckets in order, listing the elements
in each.
167
 Bucket sorting
Example: Sort the following elements using bucket sort
0.78, 0.17, 0.39, 0.26, 0.72, 0.94, 0.21, 0.12, 0.23, 0.68

168
 Bucket sorting

Time complexity T(n) = θ(n)

169
 Linear search

 It is a simple search algorithm that search a target value

in an array or list.
 It sequentially scan the array, comparing each array item
with the search element.
 If search element is found in an array, then the index of
an element is returned.
 The time complexity of linear search is O(n).
 It can be applied to both sorted and unsorted array.

170
 Binary search

 Binary search is the most popular Search algorithm. It is

efficient and also one of the most commonly used
techniques that is used to solve problems.

 Binary search works only on a sorted set of elements. To

use binary search on a collection, the collection must first
be sorted.

 When binary search is used to perform operations on a

sorted set, the number of iterations can always be reduced
on the basis of the value that is being searched.
171
Binary search process

172
Binary search
Binary-search(A, n, x)
l=1
r=n
while l ≤ r
do
m = ⌊(l + r) / 2⌋
if A[m] < x then
l=m+1
else if A[m] > x then
r=m−1
else
return m
return unsuccessful

Time complexity T(n) = O(lgn)

173
 AKTU Previous Year Questions
1. How analyze the performance of an algorithm in different cases?
2. Derive the time complexity of Merge sort.
3. Solve the recurrence i) T (n) =3T (n/4) + cn2 using recursion tree method. ii)
T (n) = n + 2T (n/2) using Iteration method. (Given T(1)=1)
4. Write Merge sort algorithm and sort the following sequence {23, 11, 5, 15,
68, 31, 4, 17} using merge sort.
5. What do you understand by stable and unstable sorting? Sort the following
sequence {25, 57, 48, 36, 12, 91, 86, 32} using heap sort.
6. What is recurrence relation? How is a recurrence relation solved using
Master’s theorem?
7. What is asymptotic notation? Explain Omega(Ω) notation.
8. Write an algorithm for counting sort. Illustrate the operation of counting
sort on the following array: A = {4,0,2,0,1,3, 5,4,1,3,2,3}.
9. Solve the following recurrence relations:-
(a) T(n) = T(n-1)+n4
(b) T(n) = T(n/4)+T(n/2) + n2
10.Write an algorithm for insertion sort? Find the time complexity in all the
174
case.
 Comparison of Sorting
Sorting is a fundamental operation in computer science and plays a crucial role in a wide range of
applications. There are many sorting algorithms available, each with its own advantages and
disadvantages. Here's a comparison of some commonly used sorting algorithms based on various
factors:
1- Time Complexity:
Best Case: The time complexity in the best-case scenario when the input data is already sorted.
Average Case: The time complexity in an average-case scenario.
Worst Case: The time complexity in the worst-case scenario.
Bubble Sort:
Best Case: O(n)
Average Case: O(n^2)
Worst Case: O(n^2)
Insertion Sort:
Best Case: O(n)
Average Case: O(n^2)
Worst Case: O(n^2)
Selection Sort:
Best Case: O(n^2)
Average Case: O(n^2)
Worst Case: O(n^2)

175
 Comparison of Sorting
Merge Sort:
Best Case: O(n log n)
Average Case: O(n log n)
Worst Case: O(n log n)
Quick Sort:
Best Case: O(n log n)
Average Case: O(n log n)
Worst Case: O(n^2) - but rarely occurs with good pivot selection
Heap Sort:
Best Case: O(n log n)
Average Case: O(n log n)
Worst Case: O(n log n)
Radix Sort:
Best Case: O(n * k)
Average Case: O(n * k)
Worst Case: O(n * k)

176
 2- Space Complexity:
The amount of additional memory required for
sorting.
Bubble Sort, Selection Sort, Insertion Sort:
O(1) - They are in-place sorting algorithms and do not
require additional memory.
Merge Sort, Quick Sort:
O(n) - They require additional memory for temporary
storage.
Heap Sort:
O(1) - It's an in-place sorting algorithm but involves
restructuring the heap.
Radix Sort:
O(n + k) - It depends on the range of the input data.177
 3- Stability:
Whether the sorting algorithm preserves the relative
order of equal elements.
Bubble Sort, Insertion Sort, Merge Sort, and Radix Sort:
Stable - They preserve the order of equal elements.
Selection Sort, Quick Sort, and Heap Sort:
Not stable - They may change the order of equal
elements.

178
 4-Use Cases:
Different sorting algorithms are suitable for different
scenarios.
Bubble Sort, Insertion Sort, and Selection Sort:
Small data sets or nearly sorted data.
Merge Sort and Quick Sort:
General-purpose sorting for larger data sets.
Heap Sort:
In-place sorting with better average-case performance than
Bubble, Insertion, or Selection Sort.
Radix Sort:
When sorting integers with a fixed number of digits.

179
Advantages and Disadvantages:
•Each algorithm has its unique advantages and
disadvantages in terms of implementation complexity,
stability, and performance.

•The choice of a sorting algorithm depends on the

specific requirements of the problem, the size of the
data, and whether stability is a concern. It's essential
to consider these factors when selecting the
appropriate sorting algorithm for a given task.

180
Thank you.

181