Design and Analysis

of Algorithms
Lecture 7
Instructor: Yusra Mansoor
Email: yusra_mansoor@outlook.com
Heap Sort
Like merge sort, it’s running time is O(nlogn)
It sorts in place.
QuickSort
Uses Divide and Conquer approach
Best case is O(nlogn)
Worst case is O(n²)
On average, it is faster than the heapsort, due to a highly optimized inner loop.
QuickSort
Quicksort sorts by employing a divide and conquer strategy to
divide a list into two sub-lists.
The steps are:
◦ Pick an element, called a pivot, from the list.
◦ Reorder the list so that all elements which are less than the pivot come
before the pivot and all elements greater than the pivot come after it (equal
values can go either way). After this partitioning, the pivot is in its final
position. This is called the partition operation.
◦ Recursively sort the sub-list of lesser elements and the sub-list of greater
elements.
◦ The base case of the recursion are lists of size zero or one, which are always
sorted.
QuickSort:
More formally: for an array A[p .. r]:
QuickSort
Some points:
◦ Base case includes the possibility that the set A might be
empty.
◦ Algorithm allows any number to be a pivot.
◦ The pivot divides the set into two subsets:
◦ The left subset contains values smaller than the pivot
◦ The right subset contains values greater than the pivot.
◦ Obviously some choice for pivot are better than others.
What does it do?
A[r] is called the pivot
Partitions the elements A[p…r-1] in to two sets, those ≤ pivot and those > pivot
Operates in place
Final result:
p pivo r
t

≤ pivot > pivot

…5 7 1 2 8 4 3 6…

p r
i

…5 7 1 2 8 4 3 6…

p r
i j

…5 7 1 2 8 4 3 6…

p r
i j

…5 7 1 2 8 4 3 6…

p r
 ij

…5 7 1 2 8 4 3 6…

p r
 ij

…5 7 1 2 8 4 3 6…

p r
 i j

…5 7 1 2 8 4 3 6…

p r
 i j

…5 7 1 2 8 4 3 6…

p r
 i j

…5 7 1 2 8 4 3 6…

p r
 i j

…5 1 7 2 8 4 3 6…

p r
 i j

…5 1 7 2 8 4 3 6…

p r
 i j

…5 1 7 2 8 4 3 6…

p r
 i j

…5 1 2 7 8 4 3 6…

p r
 i j

…5 1 2 7 8 4 3 6…

p r
 i j

…5 1 2 7 8 4 3 6…

p r

What’s happening?
 i j

…5 1 2 7 8 4 3 6…

p r

≤ pivot > pivot unprocessed

 i j

…5 1 2 7 8 4 3 6…

p r
 i j

…5 1 2 4 8 7 3 6…

p r
 i j

…5 1 2 4 3 7 8 6…

p r
 i j

…5 1 2 4 3 6 8 7…

p r
1 2 3 4 5 6
i j

…5 1 2 4 3 6 8 7…

p r
Is Partition correct?
Partitions the elements A[p…r-1] in to two sets, those ≤ pivot and those
> pivot?
Loop Invariant:
Partition running time?
Θ(n)
Quicksort
8 5 1 3 6 2 7 4
8 5 1 3 6 2 7 4
1 3 2 4 6 8 7 5
1 3 2 4 6 8 7 5
1 3 2 4 6 8 7 5
1 2 3 4 6 8 7 5
1 2 3 4 6 8 7 5
1 2 3 4 6 8 7 5
1 2 3 4 6 8 7 5
1 2 3 4 6 8 7 5
1 2 3 4 5 8 7 6

What happens here?

1 2 3 4 5 8 7 6
1 2 3 4 5 8 7 6
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
Some observations
Divide and conquer: different than MergeSort – do the work before
recursing
How many times is/can an element selected for as a pivot?
What happens after an element is selected as a pivot?

1 3 2 4 6 8 7 5
Analysis of quicksort—best case
Suppose each partition operation divides the array almost exactly in half
Then the depth of the recursion in log2n
◦ Because that’s how many times we can halve n

However, there are many recursions!

◦ How can we figure this out?
◦ We note that
◦ Each partition is linear over its subarray
◦ All the partitions at one level cover the array

49
Partitioning at various levels

50
Best case II
We cut the array size in half each time
So the depth of the recursion in log2n
At each level of the recursion, all the partitions at that level do work that is
linear in n
O(log2n) * O(n) = O(n log2n)
Hence in the average case, quicksort has time complexity O(n log2n)
What about the worst case?

51
Worst case
In the worst case, partitioning always divides the size n array into these three
parts:
◦ A length one part, containing the pivot itself
◦ A length zero part, and
◦ A length n-1 part, containing everything else

We don’t recur on the zero-length part

Recurring on the length n-1 part requires (in the worst case) recurring to
depth n-1

52
Worst case partitioning

53
Example: Assume in this and the next slide that first
element is considered as the pivot. Worst Case Call
Tree (N=4)

S =[ 1,3,5,7 ] Q(1,4)
Left=1, pivotitem = 1, Right =4

Q(1,0) Q(2,4) S =[ 3,5,7 ]

Left =2,pivotItem=3

Q(2,1) Q(3,4) S =[ 5,7 ]

pivotItem = 5, Left = 3

Q(4,4) S =[ 7 ]
Q(3,2)
QuickSort 54 Q(4,3) Q(5,4)
Worst Case n-1
t(n) = n-1
Intuition
0 n-2 n-2

0 n-3 n-3

0 n-4 n-4
.
.
.

0 1 1

0 0 0
QuickSort 55
Worst case for quicksort
In the worst case, recursion may be n levels deep (for an array of
size n)
But the partitioning work done at each level is still n
O(n) * O(n) = O(n2)
So worst case for Quicksort is O(n2)
When does this happen?
◦ There are many arrangements that could make this happen
◦ Here are two common cases:
◦ When the array is already sorted
◦ When the array is inversely sorted

56
Typical case for quicksort
If the array is sorted to begin with, Quicksort is terrible: O(n2)
It is possible to construct other bad cases
However, Quicksort is usually O(n log2n)
The constants are so good that Quicksort is generally the fastest algorithm
known
Most real-world sorting is done by Quicksort

57
Picking a better pivot
Before, we picked the last element of the subarray to use as a pivot
◦ If the array is already sorted, this results in O(n2) behavior
◦ It’s no better if we pick the first element

We could do an optimal quicksort (guaranteed

O(n log n)) if we always picked a pivot value that exactly cuts the array in half
◦ Such a value is called a median: half of the values in the array are larger, half are smaller
◦ The easiest way to find the median is to sort the array and pick the value in the middle (!)

58
Median of three
Obviously, it doesn’t make sense to sort the array in order to find the median to use
as a pivot
Instead, compare just three elements of our (sub)array—the first, the last, and the
middle
◦ Take the median (middle value) of these three as pivot
◦ It’s possible (but not easy) to construct cases which will make this technique O(n2)

59
Example: Median-of-three Partitioning
◦ Let input S = {6, 1, 4, 9, 0, 3, 5, 2, 7, 8}

◦ left=1 and S[left] = 6

◦ right=10 and S[right] = 8

◦ center = (left+right)/2 = 5 and S[center] = 0

◦ Pivot
◦ = Median of S[left], S[right], and S[center]
◦ = median of 6, 8, and 0
◦ = S[left] = 6
 Another way to avoid the worst case?
Inject randomness into the data
What is the running time of randomized
Quicksort?
Worst case?

O(n2)
Still could get very unlucky and pick “bad” partitions at every step
Final comments
Quicksort is the fastest known sorting algorithm
It is NOT stable
For optimum efficiency, the pivot must be chosen carefully
“Median of three” is a good technique for choosing the pivot
However, no matter what you do, there will be some cases where Quicksort runs in
O(n2) time

63
Comparison Sorting