Sorting Algorithms

Introduction
• Rearranging elements of an array in some order.
• Various types of sorting are:
– Bubble Sort
– Selection Sort • In place.
– Insertion Sort
• Stable.
– Shell Sort
– Quick Sort • Online.
– Merge Sort
– Counting Sort 10 30 10 50 70 40 10 20

– Radix Sort
10 10 10 20 30 40 50 70
– Bucket Sort
Bubble Sort
Bubble Sort – Ascending

[0]
8 1 1 1 1 1 1 1
[1] 1 8 7 7 7 7
5 5
2 2
[2] 7 7 8 5 5 5
7
2 2
5 5
[3] 5 5 5 8 2 2
7 7 7
[4] 2 2 2 2 8 8 8 8
Pass 1st 2nd 3rd 4th
Bubble Sort – Descending

[0]
8 8 8 8 8 8 8 8
[1] 1 1 1 7 7 7 7 7
[2] 7 7 7 1 1 5 5 5
[3] 5 5 5 5 5 1 2 2
[4] 2 2 2 2 2 2 1 1
Pass 1st 2nd 3rd 4th
Algorithm – Bubble Sort
Algorithm bubbleSort(A,n)
Input: An array A containing n integers.
Output: The elements of A get sorted in increasing order.
1. for i = 1 to n – 1 do

2. for j = 0 to n – i – 1 do

3. if A[j] > A[j + 1]

4. Exchange A[j] with A[j+1]

In all the cases, complexity is of the order of n2.

Optimized Bubble Sort?

[0]
5 1 1 1 1 1
[1] 1 5 2 2 2 2
[2] 2 2 5 3 3 3
[3] 3 3 3 5 4 4
[4] 4 4 4 4 5 5
Pass 1st 2nd
Algorithm – Optimized Bubble Sort
Algorithm bubbleSortOpt(A,n)
Input: An array A containing n integers.
Output: The elements of A get sorted in increasing order.
1. for i = 1 to n – 1 The best case
2. flag = true complexity reduces
3. for j = 0 to n – i – 1 do to the order of n,
4. if A[j] > A[j + 1] but the worst and
5. flag = false average is still n2.
6. Exchange A[j] with A[j+1] So, overall the
7. if flag == true complexity is of the
8. break; order of n2 again.
Selection Sort
Selection Sort
• In-place comparison-based algorithm.
• Divides the list into two parts
– The sorted part, which is built up from left to right at the front
(left) of the list, and
– The unsorted part, that occupy the rest of the list at the right end.
• The algorithm proceeds by
– Finding the smallest (or the largest) element in the unsorted array
– Swapping it with the leftmost (or the rightmost) unsorted element
– Moving the boundary one element to the right.
– This process continues till the array gets sorted.
• Not suitable for large data sets.
• Complexity is Ο(n2), where n is the number of elements.
Example
3 6 1 8 4 5 7 2 3 6 1 8 4 5 7 2

1 6 3 8 4 5 7 2 3 6 1 2 4 5 7 8

1 2 3 8 4 5 7 6 3 6 1 2 4 5 7 8

1 2 3 8 4 5 7 6 3 5 1 2 4 6 7 8

1 2 3 4 8 5 7 6 3 4 1 2 5 6 7 8

1 2 3 4 5 8 7 6 3 2 1 4 5 6 7 8

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Algorithm
• Algorithm selectionSort(a[], n)
• Input: An array a containing n elements.
• Output: The elements of a get sorted in increasing order.
1. for i = 0 to n – 2
2. min = i
3. for j = i+1 to n – 1
4. if a[j] < a[min]
5. min = j
6. if min != i
7. Exchange a[min] with a[i]
Insertion Sort
Insertion Sort
• An in-place, stable, online comparison-based sorting
algorithm.
• Always keeps the lower part of an array in the sorted
order.
• A new element will be inserted in the sorted part at an
appropriate place.
• The algorithm searches sequentially, move the
elements, and inserts the new element in the array.
• Not suitable for large data sets
• Complexity is Ο(n2), where n is the number of elements.
• Best case complexity is O(n).
Example
6 3 9 1 8 temp = 3
6 3 9 1 8
6 3 9 1 8 6 6 9 1 8

3 6 9 1 8
temp = 1
3 6 9 1 8
3 6 9 1 8
3 36 9
6 9 8
1 3 6 9 8

temp = 8
1 3 6 8 9 1 3 6 9 8

1 3 6 9 9
Algorithm
• Algorithm insertionSort(a[], n)
• Input: An array a containing n elements.
• Output: The elements of a get sorted in increasing order.
1. for i = 1 to n – 1
2. temp = a[i]
3. j=i
4. while j > 0 and a[j-1] > temp
5. a[j] = a[j-1]
6. j=j–1
7. a[j] = temp
Shell Sort
Shell Sort
• Improved or generalized insertion sort.
• An in–place comparison sort.
• Also known as diminishing increment sort.
• Breaks the original list into a number of smaller sublists, each
of which is sorted using an insertion sort.
• Instead of breaking the list into sublists of contiguous items,
the algorithm uses a unique way to choose the sublists.
– An increment (say h), sometimes called the gap or the interval.
– A sublist contains all the elements that are i elements apart.
• Complexity lies in between O(n) and O(n2). Still an open
problem.
Contd…
• Increment Sequences:
– Shell's original sequence: N/2 , N/4 , …, 1
– Hibbard's increments: 1, 3, 7, …, 2k – 1
– Knuth's increments: 1, 4, 13, …, (3k + 1)
– Sedgewick's increments: 1, 5, 19, 41, 109, ….
• Merging of (9 × 4i) – (9 × 2i) + 1 and 4i – (3 × 2i) -1
• Start with higher intervals and then reduce the
interval after each pass as per the chosen sequence.
Example
• Using original sequence
– h = N/2 = 8/2 = 4. – h = h/2 = 4/2 = 2.
45 43 52 20 24 29 37 54 24 29 37 20 45 43 52 54

45 24
24 37 45 52
43 29

52 37
29 20 43 54
20 54

24 29 37 20 45 43 52 54 24 20 37 29 45 43 52 54
 24 20 37 29 45 43 52 54
Contd… 24 20 37 29 45 43 52 54

– h = h/2 = 2/2 = 1. 20 24 37 29 45 43 52 54
20 24 37 29 45 43 52 54
20 24 37 29 45 43 52 54
20 24 29 37 45 43 52 54
20 24 29 37 45 43 52 54
20 24 29 37 45 43 52 54
20 24 29 37 43 45 52 54
20 24 29 37 43 45 52 54
20 24 29 37 43 45 52 54
Algorithm
• Algorithm shellSort(a[], n)
• Input: An array a containing n elements.
• Output: The elements of a get sorted in increasing order.
1. for (gap = n/2; gap > 0; gap /=2)
2. { for (i = gap; i < n; i++)
3. { temp = a[i]
4. for j = i; j >= gap && a[j - gap] > temp; j -= gap)
5. a[j] = a[j-gap]
6. a[j]= temp; }
7. }
Quick Sort
Quick Sort
• Divide and Conquer algorithm. In-place algorithm.
• Picks an element as pivot and partitions the given array
around the picked pivot, such that
– The pivot is placed at its correct position
– All elements smaller than the pivot are placed before the
pivot.
– All elements greater than the pivot are placed after the pivot.
• Several ways to pick a pivot.
– The first element.
– The last element.
– Any random element.
– The median.
Algorithm
1. PARTITION(A, p, r)
2. x = A[r]
3. i = p – 1
4. for j = p to r – 1
5. if A[j] ≤ x
6. i=i+1
7. Exchange A[i] with A[j]
8. Exchange A[i + 1] with A[r]
9. return i + 1
Partition Procedure
p r p r
2 8 7 1 3 5 6 4 2 1 3 8 7 5 6 4
i j i j
p r p r
2 8 7 1 3 5 6 4 2 1 3 8 7 5 6 4
i j i j
p r p r
2 8 7 1 3 5 6 4 2 1 3 8 7 5 6 4
i j i j
p r p r
2 8 7 1 3 5 6 4 2 1 3 4 7 5 6 8
i j i j
p r
2 1 7 8 3 5 6 4
i j
Algorithm
• QUICKSORT(A, p, r) 1. PARTITION(A, p, r)
1. if p < r 2. x = A[r]
2. q = PARTITION(A, p, r) 3. i = p – 1
4. for j = p to r – 1
3. QUICKSORT(A, p, q – 1)
5. if A[j] ≤ x
4. QUICKSORT(A, q + 1, r)
6. i=i+1
To sort an array A with n 7. Exchange A[i] with A[j]
elements, the first call to
QUICKSORT is made with p = 8. Exchange A[i + 1] with A[r]
0 and r = n – 1. 9. return i + 1
Call sequence for QUICKSORT
QUICKSORT (A, 0, 7) Partition point at index 3

QUICKSORT (A, 0, 2) Partition point at index 2

QUICKSORT (A, 0, 1) Partition point at index 0

QUICKSORT (A, 0, -1)

QUICKSORT (A, 1, 1)

QUICKSORT (A, 3, 2)

QUICKSORT (A, 4, 7) Partition point at index 7

QUICKSORT (A, 4, 6) Partition point at index 5

QUICKSORT (A, 4, 4)

QUICKSORT (A, 6, 6)

QUICKSORT (A, 8, 7)
Merge Sort
Merge Sort
• Based on the divide-and-conquer paradigm.
• To sort an array A[p .. r], (initially p = 0 and r = n-1)
1. Divide Step
– If a given array A has zero or one element, then return as it is
already sorted.
– Otherwise, split A[p…r] into two subarrays A[p…q] and A[q +
1... r], each containing about half of the elements of A[p…r].
That is, q is the halfway point of A[p...r].
2. Conquer Step
– Recursively sort the two subarrays A[p…q] and A[q + 1…r].
3. Combine Step
– Combine the elements back in A[p…r] by merging the two
sorted subarrays A[p…q] and A[q + 1…r] into a sorted sequence.
Example 5 4 6 1 2 7 3

5 4 6 1 2 7 3

5 4 6 1 2 7 3

5 4 6 1 2 7 3

4 5 1 6 2 7 3

1 4 5 6 2 3 7

1 2 3 4 5 6 7
Merge Two Sorted Arrays n1 - #Elements in L
n2 - #Elements in R
L: 1 4 5 6
i i i i i A: 1 2 3 4 5 6 7
k k k k k k k k
R: 2 3 7
j j j j
8. i = 0, j = 0, and k = p.
9. while i < n1 and j < n2 17. while i < n1
10. if L[i] ≤ R[j] 18. A[k] = L[i]
11. A[k] = L[i] 19. i++
12. i=i+1 20. k++
13. else 21. while j < n2
14. A[k] = R[j] 22. A[k] = R[j]
15. j=j+1 23. j++
16. k++ 24. k++
Algorithm
• MERGE-SORT (A, p, r)
1. if p < r
2. q = FLOOR[(p + r)/2]
3. MERGE-SORT(A, p, q)
4. MERGE-SORT(A, q + 1, r)
5. MERGE (A, p, q, r)
• To sort an array A with n elements, the first call to
MERGE-SORT is made with p = 0 and r = n – 1.
Contd…
• Algorithm MERGE (A, p, q, r)
• Input: Array A and indices p, q, r such that p ≤ q ≤ r.
Subarrays A[p…q] and A[q + 1…r] are sorted.
• Output: The two subarrays are merged into a single sorted
subarray in A[p .. r].
1. n1 = q − p + 1
2. n2 = r − q
3. Create arrays L[n1] and R[n2]
4. for i = 0 to n1 – 1
5. L[i] = A[p + i]
6. for j = 0 to n2 – 1
7. R[j] = A[q + 1 + j ]
Contd…
8. i = 0, j = 0, and k = p. 17. while i < n1
9. while i < n1 and j < n2 18. A[k] = L[i]
10. if L[i] ≤ R[j] 19. i++
11. A[k] = L[i] 20. k++
12. i=i+1 21. while j < n2
13. else 22. A[k] = R[j];
14. A[k] = R[j] 23. j++;
15. j=j+1 24. k++;
16. k++
Call sequence for an array with size 8
5 4 6 1 2 7 3 8
MERGE-SORT (A, 0, 7)
5 4 6 1 2 7 3 8
MERGE-SORT (A, 0, 3) 5 4 6 1 2 7 3 8
MERGE-SORT (A, 0, 1) 5 4 6 1 2 7 3 8
MERGE-SORT (A, 0, 0) 5 4 6 1 2 7 3 8
MERGE-SORT (A, 1, 1) 4 5 6 1 2 7 3 8
MERGE (A, 0, 0, 1)
MERGE-SORT (A, 2, 3) 4 5 6 1 2 7 3 8
4 5 6 1 2 7 3 8
MERGE-SORT (A, 2, 2)
MERGE-SORT (A, 3, 3) 4 5 6 1 2 7 3 8
MERGE (A, 2, 2, 3) 4 5 1 6 2 7 3 8
MERGE (A, 0, 1, 3) 1 4 5 6 2 7 3 8
MERGE-SORT (A, 4, 7) 1 4 5 6 2 7 3 8
MERGE-SORT (A, 4, 5) 1 4 5 6 2 7 3 8
MERGE-SORT (A, 4, 4) 1 4 5 6 2 7 3 8
MERGE-SORT (A, 5, 5) 1 4 5 6 2 7 3 8
MERGE (A, 4, 4, 5) 1 4 5 6 2 7 3 8
MERGE-SORT (A, 6, 7) 1 4 5 6 2 7 3 8
MERGE-SORT (A, 6, 6) 1 4 5 6 2 7 3 8
MERGE-SORT (A, 7, 7)
1 4 5 6 2 7 3 8
MERGE (A, 6, 6, 7)
MERGE (A, 4, 5, 7) 1 4 5 6 2 7 3 8
1 4 5 6 2 3 7 8
MERGE (A, 0, 3, 7)
1 2 3 4 5 6 7 8
Counting Sort
Counting Sort
• Assumes that the input consists of integers in a
small range 1 to k, for some integer k.
• Runs in O(n + k) time.
– k = O(n), the sort runs in q(n) time.
• For each element x, the algorithm
– First determines the number of elements less
than x.
– Then directly place the element into its correct
position.
Example
0 1 2 3 4 5 6 7 k=6
A[] 3 6 4 1 3 4 1 4 n=8

• Compute frequency of k elements, i.e. array C.

0 1 2 3 4 5 6
C[] 0 2 0 2 3 0 1

• Update C to store cumulative frequency.

0 1 2 3 4 5 6
C[] 0 2 2 4 7 7 8
 A[] C[] B[]
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 2 2 4 7 7 8 4

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 2 2 4 6 7 8 1 4
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 1 2 4 6 7 8 1 4 4
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 1 2 4 5 7 8 1 3 4 4
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 1 2 3 5 7 8 1 1 3 4 4
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 0 2 3 5 7 8 1 1 3 4 4 4
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 0 2 3 4 7 8 1 1 3 4 4 4 6
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 1 2 3 4 5 6 7 8
3 6 4 1 3 4 1 4 0 0 2 2 4 7 7 1 1 3 3 4 4 4 6
Algorithm
• Algorithm countingSort(A,n,k)
• Input: Array A, its size n, and the maximum integer k in
the list.
• Output: The elements of A get sorted in increasing
order.
1. for i = 0 to k
2. C[i] = 0 7. for i = n – 1 to 0
3. for i = 0 to n – 1 8. B[C[A[i]]] = A[i]
4. C[A[i]] = C[A[i]] + 1 9. C[A[i]] = C[A[i]] - 1
5. for i = 1 to k 10.for i = 0 to n – 1
6. C[i] = C[i] + C[i-1] 11. A[i] = B[i+1]
Radix Sort
• Similar to alphabetizing a large list of names.
– List of names is first sorted according to the first letter
of each names, that is, the names are arranged in 26
classes.
– Then sort on the next most significant letter, and so on.
• Radix sort do counter-intuitively by sorting on the
least significant digits first.
– First pass sorts entire list on the least significant digit.
– Second pass sorts entire list again on the second least-
significant digits and so on.
Example: Least Significant Digit

INPUT 1st pass 2nd pass 3rd pass

329 720 720 329
457 355 329 355
657 436 436 436
839 457 839 457
436 657 355 657
720 329 457 720
355 839 657 839
Algorithm
• Assumption: Each element in the n-element
array A has d digits, where digit 1 is the least-
significant digit and d is the most-significant digit.

• radixSort(A, d)
1. for i = 1 to d
2. use a stable sort to sort A on digit i
// counting sort will do the job