Amity School of Engineering and Technology

Amity School of Engineering & Technology

B.Tech CSE , 5th Semester

Analysis & Design of Algorithms

Dr. Garima Aggarwal
 Amity School of Engineering & Technology

Module Objectives
Discuss Key Concepts of Divide n Conquer Approach

2
Contents

3
 Amity School of Engineering & Technology

Learning Outcomes
Students will be able to

4
 Introduction

• Merge sort algorithm is a classic example of divide and conquer approach.

• To sort an array, recursively, sort its left and right halves separately and
then merge them.
• The fundamental operation in this algorithm is merging two sorted lists.
• Merge sort repeatedly breaks down a list into several sub-lists until each
sub-list consists of a single element and merging those sub-lists in a
manner that results into a single sorted list.
• Merge Sort works solving a problem in a top-down manner.

5
 Merge Sort Approach
• To sort an array A[p . . r]:
• Divide
• Divide the n-element sequence to be sorted into two subsequences of n/2
elements each
• Conquer
• Sort the subsequences recursively using merge sort
• When the size of the sequences is 1 there is nothing more to do
• Combine
• Merge the two sorted subsequences

6
 Mergesort
8 3 2 9 7 1 5 4

8 3 2 9 7 1 5 4

8 3 2 9 7 1 5 4

8 3 2 9 7 1 5 4

3 8 2 9 1 7 4 5

2 3 8 9 1 4 5 7

1 2 3 4 5 7 8 9

Fig 1: Merge sort representation [1]

 Merge Sort
p q r
1 2 3 4 5 6 7 8

Alg.: MERGE-SORT(A, p, r) 5 2 4 7 1 3 2 6

if p < r Check for base case

then q ← (p + r)/2 Divide

MERGE-SORT(A, p, q) Conquer

MERGE-SORT(A, q + 1, r) Conquer

MERGE(A, p, q, r) Combine

• Initial call: MERGE-SORT(A, 1, n)

8
 Example – n Power of 2
1 2 3 4 5 6 7 8

Divide 5 2 4 7 1 3 2 6 q=4

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

9
 Example – n Power of 2
1 2 3 4 5 6 7 8

Conquer 1 2 2 3 4 5 6 7
and
Merge 1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6

1 2 3 4 5 6 7 8

2 5 4 7 1 3 2 6

1 2 3 4 5 6 7 8

5 2 4 7 1 3 2 6

10
 Example – n Not a Power of 2
1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6 q=6
Divide
1 2 3 4 5 6 7 8 9 10 11

q=3 4 7 2 6 1 4 7 3 5 2 6 q=9

1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6

1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6

1 2 4 5 7 8

4 7 6 1 7 3

11
 Example – n Not a Power of 2
1 2 3 4 5 6 7 8 9 10 11

Conquer 1 2 2 3 4 4 5 6 6 7 7
and
Merge
1 2 3 4 5 6 7 8 9 10 11

1 2 4 4 6 7 2 3 5 6 7

1 2 3 4 5 6 7 8 9 10 11

2 4 7 1 4 6 3 5 7 2 6

1 2 3 4 5 6 7 8 9 10 11

4 7 2 1 6 4 3 7 5 2 6

1 2 4 5 7 8

4 7 6 1 7 3

12
 Merging
p q r
1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6

• Input: Array A and indices p, q, r such that

p≤q<r
– Subarrays A[p . . q] and A[q + 1 . . r] are sorted
• Output: One single sorted subarray A[p . . r]

13
 Merging
p q r
• Idea for merging: 1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6
– Two piles of sorted cards
• Choose the smaller of the two top cards
• Remove it and place it in the output pile

– Repeat the process until one pile is empty

– Take the remaining input pile and place it face-
down onto the output pile
A1 A[p, q]
A[p, r]

A2 A[q+1, r]

14
Example: MERGE(A, 9, 12, 16)
p q r

15
Example: MERGE(A, 9, 12, 16)

16
Example (cont.)

17
Example (cont.)

18
Example (cont.)

Done!

19
 Merge - Pseudocode
p q r
Alg.: MERGE(A, p, q, r) 1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6
1. Compute n1 and n2
2. Copy the first n1 elements into n1 n2
L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 + 1]
3. L[n1 + 1] ← ; R[n2 + 1] ←  p q

4. i ← 1; j ← 1 L 2 4 5 7 
q+1 r
5. for k ← p to r
R 1 2 3 6 
6. do if L[ i ] ≤ R[ j ]
7. then A[k] ← L[ i ]
8. i ←i + 1
9. else A[k] ← R[ j ]
10. j←j+1 20
 Running Time of Merge
(assume last for loop)
• Initialization (copying into temporary arrays):
 (n1 + n2) = (n)
• Adding the elements to the final array:
- n iterations, each taking constant time  (n)
• Total time for Merge:
 (n)

21
 Analyzing Divide-and Conquer Algorithms

• The recurrence is based on the three steps of the paradigm:

– T(n) – running time on a problem of size n
– Divide the problem into a subproblems, each of size n/b:
takes D(n)
– Conquer (solve) the subproblems aT(n/b)
– Combine the solutions C(n)

T(n)= (1) if n ≤ c ;
aT(n/b) + D(n) + C(n) otherwise

22
 MERGE-SORT Running Time
• Divide:
– compute q as the average of p and r: D(n) = (1)
• Conquer:
– recursively solve 2 subproblems, each of size n/2  2T
(n/2)
• Combine:
– MERGE on an n-element subarray takes (n) time 
C(n) = (n)
(1) if n =1
T(n) = 2T(n/2) + (n) if n > 1

23
 Solve the Recurrence
T(n) = c if n = 1
2T(n/2) + cn if n > 1

Use Master’s Theorem:

Compare n with f(n) = cn

Case 2: T(n) = Θ(nlgn)

24
 Merge Sort - Discussion

• Running time insensitive of the input

• Advantages:
– Guaranteed to run in (nlgn)

• Disadvantage
– Requires extra space N

25
 Test your Skills…

• Illustrate the operation of merge sort on an array

A={3,41,52,26,38,57,9,49}
• Apply merge sort to sort the list E, X, A, M, P, L, E in
alphabetical order.

26
 Summary

• Merge Sort is useful for sorting linked lists.

• Merge Sort is a stable sort which means that the same element in an array
maintain their original positions with respect to each other.
• Overall time complexity of Merge sort is O(nlogn). It is more efficient as it
is in worst case also the runtime is O(nlogn)
• The space complexity of Merge sort is O(n). This means that this algorithm
takes a lot of space and may slower down operations for the last data sets.

27
 Summary

• Time complexity of Merge Sort is O(n*log n) in all the 3 cases (worst,

average and best) as merge sort always divides the array in two halves and
takes linear time to merge two halves.

• It requires equal amount of additional space as the unsorted array. Hence

its not at all recommended for searching large unsorted arrays.

• It is the best Sorting technique used for sorting Linked Lists.

28
 Quicksort
A[p…q] ≤ A[q+1…r]

• Sort an array A[p…r]

• Divide
– Partition the array A into 2 subarrays A[p..q] and A[q+1..r],
such that each element of A[p..q] is smaller than or equal to
each element in A[q+1..r]
– Need to find index q to partition the array

29
 Quicksort
A[p…q] ≤ A[q+1…r]

• Conquer
– Recursively sort A[p..q] and A[q+1..r] using
Quicksort

• Combine
– Trivial: the arrays are sorted in place
– No additional work is required to combine them
– The entire array is now sorted

30
 QUICKSORT

Initially: p=1, r=n

Alg.: QUICKSORT(A, p, r)

if p < r

then q  PARTITION(A, p, r)

QUICKSORT (A, p, q)

QUICKSORT (A, q+1, r)

Recurrence:
T(n) = T(q) + T(n – q) + f(n) PARTITION())

31
 Partitioning the Array
• Choosing PARTITION()
– There are different ways to do this

– Each has its own advantages/disadvantages

• Hoare partition (see prob. 7-1, page 159)

– Select a pivot element x around which to partition
– Grows two regions
A[p…i]  x A[p…i]  x x  A[j…r]

x  A[j…r]

i j
32
 Example
A[p…r] pivot x=5

5 3 2 6 4 1 3 7 5 3 2 6 4 1 3 7

i j i j

3 3 2 6 4 1 5 7 3 3 2 6 4 1 5 7

i j i j
A[p…q] A[q+1…r]

3 3 2 1 4 6 5 7 3 3 2 1 4 6 5 7

i j j i
33
Example

34
 Partitioning the Array
Alg. PARTITION (A, p, r)
p r
1. x  A[p]
A: 5 3 2 6 4 1 3 7
2. i  p – 1
3. j  r + 1 i j
A[p…q] ≤ A[q+1…r]
4. while TRUE
5. do repeat j  j – 1 A: ap ar

6. until A[j] ≤ x
j=q i
7. do repeat i  i + 1
8. until A[i] ≥ x
Each element is
9. if i < j visited once!
10. then exchange A[i]  A[j] Running time: (n)
n=r–p+1
11. else return j
35
 Recurrence

Initially: p=1, r=n

Alg.: QUICKSORT(A, p, r)

if p < r

then q  PARTITION(A, p, r)

QUICKSORT (A, p, q)

QUICKSORT (A, q+1, r)

Recurrence:
T(n) = T(q) + T(n – q) + n
36
 Worst Case Partitioning

• Worst-case partitioning
– One region has one element and the other has n – 1 elements

– Maximally unbalanced
n n
• Recurrence: q=1 1 n-1 n
1 n-2 n-1
T(n) = T(1) + T(n – 1) + n, n-2
n 1 n-3
T(1) = (1) 1
2 3
T(n) = T(n – 1) + n 1 1 2
 n 
n    k   1 (n)  (n 2 ) (n 2 ) (n2)
=  k 1 
When does the worst case happen? 37
 Best Case Partitioning
• Best-case partitioning
– Partitioning produces two regions of size n/2
• Recurrence: q=n/2
T(n) = 2T(n/2) + (n)
T(n) = (nlgn) (Master theorem)

38
 Difference Between Worst and
Best
• 9-to-1 proportional split
Q(n) = Q(9n/10) + Q(n/10) + n

39
 Test your Skills…

• What is the running time of Quicksort when all elements of Array A

have the same value?
• Show that the running time of Quick Sort is ɵ(n2) when the Array A
contains distinct elements and is sorted in decreasing order.
• Illustrate Quick Sort Operation on the array A=
{13,19,9,5,12,8,7,4,11,2,6,21}
• Apply Quick Sort to sort the list ‘QUESTION’ in alphabetical
order.Draw the tree of recursive calls made.

40
 Summary
• Learned Quick Sort.
• Without recursion implementation of quicksort is complicated.
• Worst case running time of Quick Sort is O(n2)>> ɵ(nlogn) for large n.
• Average Case complexity of Quick Sort is ɵ(nlogn) to sort n items.
• Merge Sort requires an auxiliary array of size n. Since it requires 2n spaces
of sorting. It’s called not in-place sorting.
• Merge Sort requires O(nlogn) time comparison based sorting algorithm.

41
 References

[1] https://medium.com/karuna-sehgal
[2] https://www.studytonight.com

42