Sorting

Why Sorting?

Practical application
People by last name
Countries by population
Search engine results by relevance

Fundamental to other algorithms

Different algorithms have different asymptotic and

constant-factor trade-offs
No single ‘best’ sort for all scenarios
Knowing one way to sort just isn’t enough

Many to approaches to sorting which can be used for

2 other problems
 Problem statement

There are n comparable elements in an array and we want

to rearrange them to be in increasing order
Pre:
An array A of data records
A value in each data record
A comparison function
<, =, >, compareTo
Post:
For each distinct position i and j of A, if i<j then A[i] 
A[j]
A has all the same data it started with and in specified order
3
 Sorting Classification

Externa
In memory sorting l
sorting
Comparison sorting Specialized
(N log N) Sorting
O(N log
O(N2) O(N)
N)
• Bubble • Merge • Bucket Sort • Simple
Sort Sort • Radix Sort External
4
• Selection • Quick Sort Merge
Sort • Heap Sort Sort
 Comparison Sorting

Determine order through comparisons on the input

data
Bubble Sorting
 Bubble sort
bubble sort: orders a list of values by repetitively
comparing neighboring elements and swapping
their positions if necessary
more specifically:
scan the list, exchanging adjacent elements if they are
not in relative order; this bubbles the highest value to
the top
scan the list again, bubbling up the second highest
value
repeat until all elements have been placed in their
7
proper order
 "Bubbling" largest element

Traverse a collection of elements

Move from the front to the end
"Bubble" the largest value to the end using pair-wise
comparisons and swapping

0 1 2 3 4 5
42 Swap77
42 35 12 101 5
77

8
 "Bubbling" largest element

Traverse a collection of elements

Move from the front to the end
"Bubble" the largest value to the end using pair-wise
comparisons and swapping

0 1 2 3 4 5

42 35Swap35
77 77 12 101 5

9
 "Bubbling" largest element

Traverse a collection of elements

Move from the front to the end
"Bubble" the largest value to the end using pair-wise
comparisons and swapping

0 1 2 3 4 5

42 35 12Swap12
77 77 101 5

10
 "Bubbling" largest element

Traverse a collection of elements

Move from the front to the end
"Bubble" the largest value to the end using pair-wise
comparisons and swapping

0 1 2 3 4 5

42 35 12 77 101 5

No need to swap

11
 "Bubbling" largest element

Traverse a collection of elements

Move from the front to the end
"Bubble" the largest value to the end using pair-wise
comparisons and swapping

0 1 2 3 4 5

42 35 12 77 5 Swap101
101 5

12
 "Bubbling" largest element

Traverse a collection of elements

Move from the front to the end
"Bubble" the largest value to the end using pair-wise
comparisons and swapping

0 1 2 3 4 5

42 35 12 77 5 101

Largest value correctly placed

13
 Bubble sort code

public static void bubbleSort(int[] a) {

for (int i = 0; i < a.length ; i++) {
for (int j = 1; j < a.length - i; j++) {
// swap adjacent out-of-order elements
if (a[j-1] > a[j]) {
swap(a, j-1, j);
}
}
}
}

14
Selection Sort
 Selection sort

selection sort: orders a list of values by

repetitively putting a particular value into its final
position

more specifically:
find the smallest value in the list
switch it with the value in the first position
find the next smallest value in the list
switch it with the value in the second position
16
repeat until all values are in their proper places
 Selection sort example

17
 Selection sort example 2

Index
0 1 2 3 4 5 6 7

Value
27 63 1 72 64 58 14 9

1st pass
1 63 27 72 64 58 14 9

2nd pass
1 9 27 72 64 58 14 63

3rd pass
1 9 14 72 64 58 27 63

18 …
 Selection sort code

public static void selectionSort(int[] a) {

for (int i = 0; i < a.length; i++) {
// find index of smallest element
int minIndex = i;
for (int j = i + 1; j < a.length; j++) {
if (a[j] < a[minIndex]) {
minIndex = j;
}
}

// swap smallest element with a[i]

swap(a, i, minIndex);
}
19 }
Insertion Sort
 Insertion sort

insertion sort: orders a list of values by

repetitively inserting a particular value into a
sorted subset of the list

more specifically:
consider the first item to be a sorted sublist of length 1
insert the second item into the sorted sublist, shifting the first item if
needed
insert the third item into the sorted sublist, shifting the other items
as needed
repeat until all values have been inserted into their proper positions
21
 Insertion sort

Simple sorting algorithm.

n-1 passes over the array
At the end of pass i, the elements that occupied A[0]…A[i]
originally are still in those spots and in sorted order.
2 15 8 1 17 10 12 5
after
pass 2 0 1 2 3 4 5 6 7

2 8 15 1 17 10 12 5
after
0 1 2 3 4 5 6 7
pass 3
1 2 8 15 17 10 12 5
22
0 1 2 3 4 5 6 7
 Insertion sort example

23
 Insertion sort code
public static void insertionSort(int[] a) {
for (int i = 1; i < a.length; i++) {
int temp = a[i];

// slide elements down to make room for a[i]

int j = i;
while (j > 0 && a[j - 1] > temp) {
a[j] = a[j - 1];
j--;
}

a[j] = temp;
24 }
MergeSort
 MergeSort

 MergeSort is a divide and conquer method of

sorting

26
 MergeSort Algorithm
 MergeSort is a recursive sorting procedure that
uses at most O(n lg(n)) comparisons.
 To sort an array of n elements, we perform the
following steps in sequence:
 If n < 2 then the array is already sorted.
 Otherwise, n > 1, and we perform the following
three steps in sequence:
1. Sort the left half of the the array using MergeSort.
2. Sort the right half of the the array using MergeSort.
3. Merge the sorted left and right halves.
27
 How to Merge
Here are two lists to be merged:
First: (12, 16, 17, 20, 21, 27)
Second: (9, 10, 11, 12, 19)
Compare12 and 9
First: (12, 16, 17, 20, 21, 27)
Second: (10, 11, 12, 19)
New: (9)
Compare 12 and 10
First: (12, 16, 17, 20, 21, 27)
Second: (11, 12, 19)
New: (9, 10)
28
 Merge Example

Compare 12 and 11
First: (12, 16, 17, 20, 21, 27)
Second: (12, 19)
New: (9, 10, 11)
Compare 12 and 12
First: (16, 17, 20, 21, 27)
Second: (12, 19)
New: (9, 10, 11, 12)

29
 Merge Example

Compare 16 and 12
First: (16, 17, 20, 21, 27)
Second: (19)
New: (9, 10, 11, 12, 12)
Compare 16 and 19
First: (17, 20, 21, 27)
Second: (19)
New: (9, 10, 11, 12, 12, 16)

30
 Merge Example

Compare 17 and 19
First: (20, 21, 27)
Second: (19)
New: (9, 10, 11, 12, 12, 16, 17)

Compare 20 and 19
First: (20, 21, 27)
Second: ( )
New: (9, 10, 11, 12, 12, 16, 17, 19)

31
 Merge Example

Checkout 20 and empty list

First: ()
Second: ( )
New: (9, 10, 11, 12, 12, 16, 17, 19, 20,
21, 27)

32
 MergeSort

Original 24 13 26 1 12 27 38 15
Divide in 2 24 13 26 1 12 27 38 15
Divide in 4 24 13 26 1 12 27 38 15
Divide in 8 24 13 26 1 12 27 38 15
Merge 2 13 24 1 26 12 27 15 38
Merge 4 1 13 24 26 12 15 27 38
Merge 8 1 12 13 15 24 26 27 38

33
 Merge-Sort Tree
 An execution of merge-sort is depicted by a binary tree
• each node represents a recursive call of merge-sort and stores
 unsorted sequence before the execution and its partition
 sorted sequence at the end of the execution
• the root is the initial call
• the leaves are calls on subsequences of size 0 or 1

7 29 4  2 4 7 9

72  2 7 94  4 9

77 22 99 44

34
 Execution Example
 Partition

7 2 9 43 8 6 1

35
 Execution Example (cont.)

 Recursive call, partition

7 2 9 43 8 6 1

7 29 4

36
 Execution Example (cont.)
 Recursive call, partition
7 2 9 43 8 6 1

7 29 4

72

37
 Execution Example (cont.)
 Recursive call, base case
7 2 9 43 8 6 1

7 29 4

72

77

38
 Execution Example (cont.)
 Recursive call, base case
7 2 9 43 8 6 1

7 29 4

72

77 22

39
 Execution Example (cont.)

 Merge

7 2 9 43 8 6 1

7 29 4

722 7

77 22

40
 Execution Example (cont.)
 Recursive call, …, base case, merge
7 2 9 43 8 6 1

7 29 4

722 7 9 4  4 9

77 22 99 44

41
 Execution Example (cont.)
 Merge

7 2 9 43 8 6 1

7 29 4 2 4 7 9

722 7 9 4  4 9

77 22 99 44

42
 Execution Example (cont.)
 Recursive call, …, merge, merge
7 2 9 43 8 6 1

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

43
 Execution Example (cont.)

 Merge

7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

44
// A : Array that needs to be sorted
MergeSort(A)
{
n = length(A)
if n<2 return
mid = n/2
left = new_array_of_size(mid) // Creating temporary array for left
right = new_array_of_size(n-mid) // and right sub arrays
for(int i=0 ; i<=mid-1 ; ++i)
{
left[i] = A[i] // Copying elements from A to left
}
for(int i=mid ; i<=n-1 ; ++i)
{
right[i-mid] = A[i] // Copying elements from A to right
}
MergeSort(left) // Recursively solving for left sub array
MergeSort(right) // Recursively solving for right sub array
merge(left, right, A) // Merging two sorted left/right sub array to final array
}
 Bitonic Sort

• Bitonic Sort is a classic parallel algorithm for sorting.

• It is an Comparison based sorting algorithm.
• The number of comparisons done by Bitonic sort is more.
• But Bitonic sort is better for parallel implementation.
• Because we always compare elements in a predefined
sequence and the sequence of comparison doesn’t
depend on data.
• Therefore it is suitable for implementation in hardware
and parallel processor array.
Constraint

• Bitonic Sort can only be done if the number of

elements to sort is 2^n.
• The procedure of bitonic sequence fails if the
number of elements is not in the
forementioned quantity precisely.
Bitonic Sequence

• A sequence is called Bitonic if it is first

increasing, then decreasing.
• In other words, an array arr[0..n-i] is Bitonic
• if there exists an index I, where 0<=i<=n-1
such that
• x0 <= x1 …..<= xi
• and
• xi >= xi+1….. >= xn-1
Bitonic Sort

• A sequence, sorted in increasing order is

considered Bitonic with the decreasing part as
empty.
• Similarly, decreasing order sequence is
considered Bitonic with the increasing part as
empty.
• A rotation of the Bitonic Sequence is also
bitonic.
 Bitonic Sequence from a random input

• We start by forming 4-element bitonic

sequences the from consecutive 2-element
sequences.
• Consider 4-element in sequence x0, x1, x2,
x3.
• We sort x0 and x1 in ascending order and x2
and x3 in descending order.
• We then concatenate the two pairs to form a 4
element bitonic sequence.
• Next, we take two 4-element bitonic
sequences, sorting one in ascending order,
the other in descending order and so on, until
we obtain the bitonic sequence.
Example

• Convert the following sequence to a bitonic

sequence: 3, 7, 4, 8, 6, 2, 1, 5
• Step 1: Consider each 2-consecutive element as a
bitonic sequence.
• And apply bitonic sort on each 2- pair element.
• In the next step, take 4-element bitonic sequences
and so on.
• x0 and x1 are sorted
in ascending order
• x2 and x3 in
descending order
and so on
Step 2: Two 4 element bitonic sequences:
A(3,7,8,4) and B(2,6,5,1) with comparator
length as 2

After this step, we’ll get a

Bitonic sequence of
length 8.

3, 4, 7, 8, 6, 5, 2, 1
Bitonic sort network
Algorithm
 Complexity of MergeSort
Pass Number of Merge list # of comps /
Number merges length moves per
merge
1 2k-1 or n/2 1 or n/2k  21
2 2k-2 or n/4 2 or n/2k-1  22
3 2k-3 or n/8 4 or n/2k-2  23
. . . .
. . . .
. . . .
k = log n
k–1 21 or n/2k-1 2k-2 or n/4  2k-1
k 20 or n/2k 2k-1 or n/2  2k
58
 Complexity of MergeSort

Multiplying the number of merges by the maximum

number of comparisons per merge, we get:
(2k-1)21 = 2k
k passes each require
(2 )2
k-2 2
=2 k


2 k
comparisons (and
 moves). But k = lg n
 and hence, we get
(21)2k-1 = 2k lg(n)  n comparisons
(20)2k = 2k or O(n lgn)

59