CSE 325 Design and Analysis of

Algorithms

Sorting
 The Sorting Problem

• Input:

– A sequence of n numbers a1, a2, . . . , an

• Output:

– A permutation (reordering) a1’, a2’, . . . , an’ of the input

sequence such that a1’ ≤ a2’ ≤ · · · ≤ an’

2
Structure of data

3
 Why Study Sorting Algorithms?
• There are a variety of situations that we can
encounter
– Do we have randomly ordered keys?
– Are all keys distinct?
– How large is the set of keys to be ordered?
– Need guaranteed performance?

• Various algorithms are better suited to some of

these situations

4
 Some Definitions
• Internal Sort
– The data to be sorted is all stored in the computer’s
main memory.
• External Sort
– Some of the data to be sorted might be stored in
some external, slower, device.
• In Place Sort
– The amount of extra space required to sort the data is
constant with the input size.

5
 Some Definition
• Stable Sorting: A sorting algorithm for which the
order of appearance of equal elements is same
their order of appearance in the input, is called a
stable sorting algorithm
• Comparison based Sorting algorithm: The
sorting algorithm that uses only compare, swap
and assignment.

6
 Insertion Sort

input array

5 2 4 6 1 3

at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted

7
Insertion Sort

8
 INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

for j ← 2 to n a 1 a 2 a 3 a 4 a 5 a 6 a 7 a8

do key ← A[ j ] key
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
• Insertion sort – sorts the elements in place
9
 Loop Invariant for Insertion Sort

Alg.: INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key

10
 Proving Loop Invariants
• Proving loop invariants works like induction
• Initialization (base case):
– It is true prior to the first iteration of the loop
• Maintenance (inductive step):
– If it is true before an iteration of the loop, it remains true before
the next iteration
• Termination:
– When the loop terminates, the invariant gives us a useful
property that helps show that the algorithm is correct
– Stop the induction when the loop terminates

11
Loop Invariant for Insertion Sort
Invariant: at the start of the jth iteration of the j for loop the
elements in A[1 . . j-1] contains the elements originally in
A[1 . . j-1] but are in sorted order

• Initialization:
– Just before the first iteration, j = 2:
the subarray A[1 . . j-1] = A[1],
(the element originally in A[1]) – is
sorted

12
Loop Invariant for Insertion Sort
– Maintenance:
– the while inner loop moves A[j -1], A[j -2], A[j -3],
and so on, by one position to the right until the proper
position for key (which has the value that started out in
A[j]) is found
– At that point, the value of key is placed into this
position.

13
 Loop Invariant for Insertion Sort
• Termination:
– The outer for loop ends when j = n + 1  j-1 = n
– Replace n with j-1 in the loop invariant:
• the subarray A[1 . . n] consists of the elements originally in
A[1 . . n], but in sorted order
j-1 j

• The entire array is sorted!

Invariant: at the start of the jth iteration of the for loop the
elements in A[1 . . j-1] contains the elements originally in
A[1 . . j-1] but are in sorted
14
 Analysis of Insertion Sort
INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
tj: # of times the while statement is executed at iteration j

15
 Complexity Analysis
• Worst case: T(n)= σ𝑛𝑖=1 𝑡𝑗 + 𝑐 = σ𝑛𝑖=1 𝑖 + 𝑐 = 𝜃(𝑛2)
• Best Case: T(n)= 𝜃(𝑛)

16
 Insertion Sort - Summary
• Advantages
– Good running time for “almost sorted” arrays (n)
• Disadvantages
– (n2) running time in worst case
–  n2/2 comparisons and exchanges

17
 Bubble Sort
• Idea:
– Repeatedly pass through the array
– Swaps adjacent elements that are out of order

i
1 2 3 n

8 4 6 9 2 3 1
j

• Easier to implement, but slower than Insertion

sort

18
 Example
8 4 6 9 2 3 1 1 8 4 6 9 2 3
i=1 j i=2 j

8 4 6 9 2 1 3 1 2 8 4 6 9 3
i=1 j i=3 j

8 4 6 9 1 2 3 1 2 3 8 4 6 9
i=1 j i=4 j

8 4 6 1 9 2 3 1 2 3 4 8 6 9
i=1 j i=5 j

8 4 1 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=6 j

8 1 4 6 9 2 3 1 2 3 4 6 8 9
i=1 j i=7
j
1 8 4 6 9 2 3
19
i=1 j
 Bubble Sort
Alg.: BUBBLESORT(A)
for i  1 to length[A]
do for j  length[A] downto i + 1
do if A[j] < A[j -1]
then exchange A[j]  A[j-1]
i
8 4 6 9 2 3 1
i=1 j

20
 Bubble-Sort Running Time
Alg.: BUBBLESORT(A)
for i  1 to length[A] c1
do for j  length[A] downto i + 1 c2
Comparisons:  n2/2 do if A[j] < A[j -1] c3
Exchanges:  n2/2
then exchange A[j]  A[j-1] c4
n

 (n − i )
n n
T(n) = c1(n+1) + c2  (n − i + 1) + c3  (n − i ) + c4
i =1 i =1 i =1
n

= (n) + (c2 + c2 + c4)  (n − i )

i =1
n n n
n ( n + 1) n 2
n
where  (n − i ) = n −  i = n −
2
= −
i =1 i =1 i =1 2 2 2
Thus,T(n) = (n2) 21
Proof of Correctness of Bubble sort
• Loop invariant for j-loop:
• Before the start of jth iteration of the j-loop, the minimum
value in the sub array A[i,n] occurs within A[i,…,j] ( at least
one of them does, incase of a tie)
• Initialization: We must show the loop invariant holds before the
first iteration of the loop. Before the first iteration of the loop, j = n.
Thus, we must show that the minimum value in the subarray A[i...n]
occurs within A[i...n]. This is trivially true.

22
• Maintenance: Suppose that before a given iteration of the inner for loop,
the minimum value in the subarray A[i...n] is within A[i...j]. We must show that before
the beginning of the next iteration, the loop invariant still holds. There are three cases
to consider:
• Case 1: The minimum value in the subarray A[i...n] occurs within A[i...j-2]. Then the
exchange step does not impact the minimum value, so the minimum value still occurs
within A[i...j-2] after the iteration.
• Case 2: The minimum value in the subarray A[i...n] is at A[j-1]. Then the A[j-1] ≤ A[j],
since A[j-1] is the minimum, so the exchange step doesn’t happen. Thus the minimum
value still occurs within A[i...j-1] after the iteration.
• Case 3: The minimum value in the subarray A[i...n] is at A[j]. Then A[j] ≤ A[j-1], since
A[j] is the minimum, so an exchange occurs and the minimum is now within A[i...j-1].
If A[j] == A[j-1], then an exchange does NOT occur, but they’re the same value so it
doesn’t matter; there is still at least one occurrence of the minimum value of A[ i...n]
within A[i...j-1]. Since under all of these cases, the minimum value of A[i...n] occurs
within A[i...j-1] after the iteration, when j decrements at the beginning of the next
iteration, the minimum value of A[i...n] occurs within A[i...j]. So, the loop invariant
holds for the next iteration.

23
• Termination: The inner loop terminates just
before the iteration where j = i. So, by the loop
• invariant, the minimum value of A[i...A.length]
occurs within A[i...i], which means that the

• minimum value of A[i...A.length] is A[i].

24
 Loop invariant for i-loop
• Loop invariant: At the start of ith iteration, the
smallest i-1 elements are in the first i-1 places in
the array, in sorted order.
• Initialization: Before the first iteration, i = 1. “The
smallest 0 elements are in the first 0 places in
the array, in sorted order” is vacuously true.

25
• Maintenance: Suppose that before a given iteration of the loop, the
smallest i-1 elements were in the first i-1 places in the array, in sorted order.
The inner for loop cannot alter any elements in the subarray A[1...i-1]
because it only ever alters elements A[j] and A[j-1], and during all iterations
of the inner for loop, j ≥ i+1 by the loop bounds. Thus, the smallest i-1
elements are still in the first i-1 places in the array, in sorted order, after the
iteration completes. So, because A must be a permutation of the original
array at all times, the smallest element in the subarray A[i...n] is the ith
smallest element overall. By the inner for loop invariant, after the iteration,
A[i] is the smallest element of the subarray A[i...n]. This means that the
smallest i elements are in the first i places in the array, in sorted order. At
the start of the next iteration, i increments, so smallest i-1 elements are in
the first i-1 places in the array, in sorted order. Thus, the loop invariant holds
before the next iteration.

26
• Termination: After the end of the loop, i = n+1.
So, the smallest n elements of the array are in
the first n places, in sorted order. This means
that the whole array is in sorted order.

• Maintenance: Homework

27
Selection Sort
• Idea:
– Find the smallest element in the array
– Exchange it with the element in the first position
– Find the second smallest element and exchange it with
the element in the second position
– Continue until the array is sorted
• Disadvantage:
– Running time depends only slightly on the amount of
order in the file

28
 Example
8 4 6 9 2 3 1 1 2 3 4 9 6 8

1 4 6 9 2 3 8 1 2 3 4 6 9 8

1 2 6 9 4 3 8 1 2 3 4 6 8 9

1 2 3 9 4 6 8 1 2 3 4 6 8 9

29
 Selection Sort
Alg.: SELECTION-SORT(A)
n ← length[A] 8 4 6 9 2 3 1

for j ← 1 to n - 1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
exchange A[j] A[smallest]

30
 Analysis of Selection Sort
Alg.: SELECTION-SORT(A) cost times
n ← length[A] c1 1
for j ← 1 to n - 1 c2 n
do smallest ← j c3 n-1
n2/2 for i ← j + 1 to n c4 nj=−11 (n − j + 1)
comparisons
do if A[i] < A[smallest] c5 
n −1
j =1
(n − j )
n
exchanges then smallest ← i c6 
n −1
j =1
(n − j )

exchange A[j] A[smallest] c7 n-1

n −1 n −1 n −1
T ( n) = c1 + c2 n + c3 (n − 1) + c4  (n − j + 1) + c5  ( n − j ) + c6  ( n − j ) + c7 (n − 1) = (n 2 ) 31
j =1 j =1 j =2
 Selection Sort: :Loop Invariant
• Invariant: j- loop
– At the start of jth iteration of j-loop, A[min_index] is the
smallest element from among A[i,…,j-1]
• Intitialization: j=i+1
– So, A[min_index] is the smallest Element of A[i], which
is trivially true.
• Termination: j=n+1
– So, A[min_index] is the smallest element of A[i,…,n].
– So, j-loop finds the min element and swaps with A[i]

32
• i-loop invariant:
• Loop invariant: At the start of ith iteration, the
smallest i-1 elements are in the first i-1 places in
the array, in sorted order.
• Initialization: Before the first iteration, i = 1. “The
smallest 0 elements are in the first 0 places in
the array, in sorted order” is vacuously true.
• Termination: After the end of the loop, i = n+1.
So, the smallest n elements of the array are in
the first n places, in sorted order. This means
that the whole array is in sorted order.
33
• Maintenance: Home work

34