Sorting Algorithms

Professor Yeali S. Sun

National Taiwan University

1
 Sorting Algorithms: Two
Categories
Internal Sort
For data sets to be sorted that can fit entirely
in main memory
External Sort
For data set to be sorted that cannot fit in
main memory all at once but must reside
in secondary storage (e.g., disk)

2
 Sorting Algorithms
Selection Sort O(N2)
Bubble Sort - O(N2)
Insertion Sort O(N2) (worst-case)
Mergesort - O(N*log2N)
Quicksort - O(N*log2N)
Heapsort
Treesort
3
 Sorting: Introduction
Arrangement of objects according to some
ordering criteria.
Assume we have a collection of information
concerning some set of objects.
Assume this collection of information is organized
in records.
Within a record, information is structured into a
number of units called fields.

4
Sorting: Introduction (contd)
The data structure of a record depends on the
application.
e.g.,
collection of objects: telephone directory
objects: companies/stores
information about an object: company name,
products, phone number, address, price, etc
one record for one company/store
e.g.,
student data file for a university
objects: students
information about an object: name, id, address,
department, etc. 5
 There are MANY sorting
algorithms!
No single sorting technique is the
"best" for all applications.
Size of the problem, e.q., N
Time complexity in search, insert, and
delete
Space complexity data store and
auxiliary space for sorting

6
 Formal description of the
sorting problem
Given a list of records in which each record has a
key value. There exists an ordering relation on the
keys (>=, =. <=)
Note that ordering relations are transitive.

The sorting problems are to find a permutation

such that if the ordering relation is > then Key(i-1)
< Key(i).

7
 Formal description of the
sorting problem
If values of key are not unique, then
consider two cases:
sorted: only obeys ordering relation
stable: if Key (i) = Key (j), element i
precedes element j then in the sorted list
element i also precedes element j.

8
Application #1 - SEARCHING!!

How to IDENTIFY a record (or an object)?

Use information about a record: one or
more fields, e.g.,
student data file: id
telephone directory: company name &
product type or telephone number
"Key(s)": to uniquely identify a record

9
 "Efficiency" of searching?

Depends on how records are arranged!

random, sorted, ...
e.g., sequential search O (n) , binary
search O (log 2 n )

10
 Application #2 -
How to know if two sets of information are
identical?
e.g., tax reports from employees and employer

11
 Selection Sort
Input: a list of records: R0 , R1 ,..., Rn 1
Output: an ordered list of records:
Rk0 , Rk1 ,..., Rk N 1 , where k0 k1 ... k N 1
Algorithm:
step 1: i 0
step 2: find the largest item R j from list
R0 ,..., RN 1i , 0 j N 1 i
step 3: swap R jand RN 1ito produce a sequence of
ordered records RN 1i , RN i ,..., RN 1
step 4: increment i; repeat step 2 until i = N 1
Ineach
In eachround,
round,select
selectthe
thelargest
largestone
oneand
andplace
placeititto
tothe
theend.
end.12
Selection Sort (contd)

N-1
N-2
.
.
+ 1
N (N 1)
2
O(N 2 )

13
 Bubble Sort

+ + + = N ( N 1) 2
( N 1) ( N 2) ... 1 O( N )
2
O(N ) best case
Original data is sorted
14
 Insertion Sort
Partitioned the list into two regions: sorted
(front region) and unsorted (rear region).
At each step, takes the first item of the
unsorted region and places it into its
correct position in the sorted region

15
 Insertion Sort (contd)
Input: a list of records: R0 , R1,..., Rn1
Output: an ordered list of records:
Rk0 , Rk1 ,..., Rkn1 , where k0 k1 ... kn1
Algorithm:
step 1: i 0
step 2: insert Ri into a sequence of ordered records
R0 , R1 ,..., Ri 1 to produce a sequence of ordered
R0 , R1 ,..., Ri records
step 3: increment i; repeat step 2 until i n

16
Insertion Sort an example

17
18
 Divide-and-Conquer
Mergesort and Quicksort
recursive formulations
Efficient O(N*log2N)
Regardless the initial order of the items in the
data set

19
 Mergesort
Algorithm
Divide the array into halves
Sort each half
Merge the sorted halves into one sorted list
The merge step
Compare an item in one half with an item in the other
half.
Move the smaller item to a temporary array until no
more items to consider on e half.
Move the remaining items to the temporary array.
Copy the temporary array back into the original array.
20
Merge Sort an example

21
 Merge Sort - algorithm
mergesort(A, F, L)
// Sorts A[F..L] by
// 1. sorting the first half of the array
// 2. sorting the second half of the array
// 3. merging the two sorted halves
if (F < L)
{ Mid = (f + L)/2 // get midpoint
mergesort(A, F, Mid) // sort A[F..Mid]
mergesort(A, Mid + 1, L) // sort A[Mid+1..L]

// merge sorted halves A[F..Mid] and A[Mid+1..L]

merge(A, F, Mid, L)
}// end if
else quit 22
Merge Sort

23
Merge Sort an example
(contd)

24
Merge Sort an example
(contd)

25
 Complexity
Time - O(N log2N)
Space twice the size of the sorted data

26
 Quicksort
Time complexity analysis
Average-case - O(N*log2N)
Worse-case O(N2)

Often used to sort large arrays

Usually extremely fast in practice
The original arrangement of data is random
Has the best "average" behavior among all
the sorting methods.
27
 Quick Sort
quicksort(inout theArray:ItemArray, in first:integer, in
last:integer)
// Sorts theArrray[first..last].
if (first < last)
{
Choose a pivot item p from theArrray[first..last]
Partition the items of theArrray[first..last] about p
// the partition is theArrray[first..pivotIndex..last]
quicksort(theArrray, first, pivotIndex-1) // sort S1
quicksort(theArray, pivotIndex+1, last) // sort S2
}
// if first >= last, there is nothing to do
28
Quick Sort: invariant of the
partition algorithm

29
Quick Sort: initial state of the
array

30
Quick Sort: swapping <

31
Quick Sort: swapping >

32
Partition(contd)

33
 Partition (page 467)
void partition(DataType theArray[], int first, int last, int
& pivotIndex)
// ---------------------------------------------------------
// Partitions an array for quicksort.
// Precondition: theArray[first..last] is an array; first <=
last.
// Postcondition: Partitions theArray[first..last] such that:
// S1 = theArray[first..pivotIndex-1] < pivot
// theArray[pivotIndex] == pivot
// S2 = theArray[pivotIndex+1..last] >= Pivot
// Calls: choosePivot and swap.
// --------------------------------------------------------- 34
 Partition(contd)
{ choosePivot(theArray, first, last);

dataType pivot = theArray [first]; // copy pivot

// index of last item in S1
int lastS1 = first;
// index of first item in unknown
int firstUnknown = first+1;

35
 Partition(contd)
// move one item at a time until unknown region is empty
for (; firstUnknown <= last; ++firstUnknown)
{ // Invariant: theArray[first..LastS1] < Pivot
// theArray[LastS1+1..firstUnknown-1] >= Pivot
// move item from unknown to proper region
if (theArray[firstUnknown] < pivot)
{ // item from unknown belongs in S1
++lastS1;
swap(theArray[firstUnknown], theArray[lastS1]);
} // end if
// else item from unknown belongs in S2
}// end for
36
 Partition(contd)
// place pivot in proper position and mark its
location
swap (theArray[first], (theArray[lastS1]);
pivotIndex = lastS1;
}// end Partition

37
Partition - Example
lastS1=0, firstUnknown=1

38
Partition - Example
lastS1=0

39
 Quick Sort (page 468)
quicksort(DataType theArray, int first, int last)
{ // Sorts theArrray[first..last].
if (first < last)
{ // create the partition: S1, pivot, S2
partition[theArray, first, last, pivotIndex);

// sort region S1 and S2

quicksort(theArrray, first, pivotIndex-1);
quicksort(theArray, pivotIndex+1, last);
} // end if
} // end quicksort
40
 Quick Sort - example
Analysis
Worst-case: reverse sorted order O( n 2 )
average case: O ( n log 2 n )
Variation
QuickSort using a median of three
median (left, right, middle)

41
 Heapsort (p.550)
Use a heap to sort an array of items
Algorithm
Transform the array into a heap - use HeapInsert to
insert the items into the heap one by one
Or,
Image the array as a complete binary tree
Transform the tree into a heap use RebuildHeap
Call Rebuildheap on the leaves from right to left
Move up the tree
Until reach the root
42
 Building a Heap from an Array
of Items
for(Index = N-1 down to 0)
// Assertion: the tree rooted at Index is a semiheap
RebuildHeap (A, Index, N)

// Assertion: the tree rooted at Index is a heap

43
 Heap Sort: Example
Consider
Considerthe
thearray
arrayas
asaacomplete
completebinary
binarytree
tree

44
Heap Sort: Transform an Array
into a Heap

45
 Heap Sort: Invariant
After step k, the Sorted region
contains the k largest values in A in
sorted order, i.e.,
A[N-1] is the largest, A[N-2] is the second
largest, and so on.
The items in the Heap region form a
heap.

46
Heap Sort: Partition an
Array into Two Regions

47
Heap Sort: Example

48
Heap Sort: Example (contd)

49
Heap Sort: Example (contd)

50
 Heap Sort: Algorithm
HeapSort(A, N)
// Sorts A[0..N-1]
// build initial heap
for (index = N - 1 down to 0)
{ // Invariant: the tree rooted at Index is a semiheap
RebuildHeap(A, index, N)
// Assertion: the tree rooted at index is a heap
}
// Assertion: A[0] is largest element in heap A[0..N-1]

// initialize the regions

Last = N - 1 51
Heap Sort: Algorithm (contd)
// Invariant: A[0..Last] is a heap, A[Last+1..N-1] is
// sorted and contains the largest elements of A

for (Step = 1 through N)

{ // move the largest item A[0] in the Heap region
// to the beginning of the Sorted region by swapping items
Swap A[0] and A[Last]

// expand the Sorted region, shrink the Heap region

--Last

// make the Heap region a heap again

RebuildHeap(A, 0, Last)
} // end for
52
53
 Treesort
Algorithm
Use a binary search tree to sort an array of
data items into searchkey order
Traverse the tree inorder
Analysis
Each items insertion takes O(log2N) or log(N)
N times - O(N*log2N) or O(N2)
Traversal O(N)

54
 Treesort
treesort(A, N)
// Sorts the N integers in an array A into ascending order.
Insert A's elements into a binary search tree T

Traverse T inorder. As you visit T's nodes, copy

their data portions into successive locations of A

Treesort uses a binary search tree

Treesort.
Average case: O ( N log N )
worst case: O ( N 2 )

55
Homeworks

56
 Quick Sort Algorithm(contd)
Developed by C.A.R. Hoar
Has the best average behavior among all the
sorting methods
Input: a list of records: Rleft , R(left +1) ,..., Rright , left < right
Assumptions:
K left K ( right+1)
let pivot key be Rright
Output of each step:
j , left j i - 1, k j < ki
j , i + 1 j right, k j ki
ki = k pivot _ key (i.e. initial kleft )
57
 Quick Sort Algorithm
Algorithm
step 1:
step 2: search from i and stop at a record whose key
value is greater than or equal to ;
let the position pointed by i
step 3: search from j and stop at a record whose key
value is less than ;
let the position pointed by j
step 4: If i < j SWAP and and go to step 2
step 5: otherwise SWAP and
step 6: quicksort (list, left, j-1) /* all records with
keys */
step 7: quicksort (list, j+1, right) /* all records with
58
keys */
 Quick Sort Algorithm
1. start from the first element and let it be the pivot
element
2. scan from left to right and find an element greater than
the pivot element
3. scan from right to left and find an element smaller than
the pivot element
4. swap them
5. continue until two scans "cross
6. swap pivot and the "cross" element
7. The resulting list has the left sublist smaller than the
cross element(i.e. pivot element) and the right sublist
greater than the cross element
8. repeat the QuickSort for left- and right-sublist 59
60
61
62
63
64
65
 Quick Sort - example
Example: 26, 5, 37, 1, 61, 11, 59, 15, 48, 19
Analysis
worse case: reverse sorted order O( n 2 )
average case: O ( n log 2 n )
Variation
QuickSort using a median of three
median (left, right, middle)

66