CS 332: Algorithms: Introduction To Heapsort

CS 332: Algorithms
Introduction to heapsort
Review: The Master Theorem
Given: a divide and conquer algorithm
An algorithm that divides the problem of size n

into a subproblems, each of size n/b
Let the cost of each stage (i.e., the work to divide
the problem + combine solved subproblems) be
described by the function f(n)
Then, the Master Theorem gives us a

cookbook for the algorithms running time:
Review: The Master Theorem
if T(n) = aT(n/b) + f(n) then
log b a
T ( n) n
log n
f ( n )
log b a
f (n) O n log b a
f ( n) n
log b a
c 1
f (n) n logb a AND
af (n / b) cf (n) for large n
Sorting Revisited
So far weve talked about two algorithms to sort

an array of numbers
What is the advantage of merge sort?
Answer: O(n lg n) worst-case running time
What is the advantage of insertion sort?

Answer: sorts in place
Also: When array nearly sorted, runs fast in practice
Next on the agenda: Heapsort
Combines advantages of both previous algorithms
Heaps
A heap can be seen as a complete binary tree:

16
14
10
What makes a binary tree complete?

Is the example above complete?
Heaps
A heap can be seen as a complete binary tree:

16
14
10
The book calls them nearly complete binary trees;

can think of unfilled slots as null pointers
Heaps
In practice, heaps are usually implemented as

arrays:
16
14
A = 16 14 10 8
10
1 =
2
7
4
Heaps
To represent a complete binary tree as an array:
The root node is A[1]

Node i is A[i]
The parent of node i is A[i/2] (note: integer divide)
The left child of node i is A[2i]
The right child of node i is A[2i + 1]
16
14
A = 16 14 10 8
10
1 =
2
7
4
Referencing Heap Elements
So
Parent(i) { return i/2 ; }
Left(i) { return 2*i; }
right(i) { return 2*i + 1; }
An aside: How would you implement this

most efficiently?
Trick question, I was looking for i << 1, etc.

But, any modern compiler is smart enough to do this
for you (and it makes the code hard to follow)
The Heap Property
Heaps also satisfy the heap property:

A[Parent(i)] A[i]
for all nodes i > 1
In other words, the value of a node is at most the

value of its parent
Where is the largest element in a heap stored?
Heap Height
Definitions:
The height of a node in the tree = the number of

edges on the longest downward path to a leaf
The height of a tree = the height of its root
What is the height of an n-element heap?

Why?
This is nice: basic heap operations take at most
time proportional to the height of the heap
Heap Operations: Heapify()
Heapify(): maintain the heap property
Given: a node i in the heap with children l and r

Given: two subtrees rooted at l and r, assumed to be
heaps
Problem: The subtree rooted at i may violate the heap
property (How?)
Action: let the value of the parent node float down so
subtree at i satisfies the heap property
What do you suppose will be the basic operation between i, l,

and r?
Heap Operations: Heapify()

Heapify(A, i)
{
l = Left(i); r = Right(i);
if (l <= heap_size(A) && A[l] > A[i])
largest = l;
else
largest = i;
if (r <= heap_size(A) && A[r] > A[largest])
largest = r;
if (largest != i)
Swap(A, i, largest);
Heapify(A, largest);
}
Heapify() Example
16
4
10
14
2
7
8
A = 16 4 10 14 7
Heapify() Example
16
4
10
14
2
7
8
A = 16 4 10 14 7
Heapify() Example
16
4
10
14
2
7
8
A = 16 4 10 14 7
Heapify() Example
16
14
10
4
2
7
8
A = 16 14 10 4
Heapify() Example
16
14
10
4
2
7
8
A = 16 14 10 4
Heapify() Example
16
14
10
4
2
7
8
A = 16 14 10 4
Heapify() Example
16
14
10
8
2
7
4
A = 16 14 10 8
Heapify() Example
16
14
10
8
2
7
4
A = 16 14 10 8
Heapify() Example
16
14
10
8
2
7
4
A = 16 14 10 8
Analyzing Heapify(): Informal

Aside from the recursive call, what is the
running time of Heapify()?
How many times can Heapify() recursively
call itself?
What is the worst-case running time of
Heapify() on a heap of size n?
Analyzing Heapify(): Formal

Fixing up relationships between i, l, and r takes
(1) time
If the heap at i has n elements, how many
elements can the subtrees at l or r have?
Draw it
Answer: 2n/3 (worst case: bottom row 1/2 full)

So time taken by Heapify() is given by
T(n) T(2n/3) + (1)
Analyzing Heapify(): Formal

So we have
T(n) T(2n/3) + (1)
By case 2 of the Master Theorem,
T(n) = O(lg n)
Thus, Heapify() takes logarithmic time
Heap Operations: BuildHeap()
We can build a heap in a bottom-up manner by

running Heapify() on successive subarrays
Fact: for array of length n, all elements in range

A[ n/2 + 1 .. n] are heaps (Why?)
So:
Walk backwards through the array from n/2 to 1, calling
Heapify() on each node.
Order of processing guarantees that the children of node
i are heaps when i is processed
BuildHeap()
// given an unsorted array A, make A a heap
BuildHeap(A)
{
heap_size(A) = length(A);
for (i = length[A]/2 downto 1)
Heapify(A, i);
}
BuildHeap() Example
Work through example

A = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7}
4
1
2
14
16
8
10
Analyzing BuildHeap()
Each call to Heapify() takes O(lg n) time
There are O(n) such calls (specifically, n/2 )
Thus the running time is O(n lg n)
Is this a correct asymptotic upper bound?

Is this an asymptotically tight bound?
A tighter bound is O(n)
How can this be? Is there a flaw in the above

reasoning?
Analyzing BuildHeap(): Tight
To Heapify() a subtree takes O(h) time

where h is the height of the subtree
h = O(lg m), m = # nodes in subtree

The height of most subtrees is small
Fact: an n-element heap has at most n/2h+1

nodes of height h
CLR 7.3 uses this fact to prove that
BuildHeap() takes O(n) time
Heapsort
Given BuildHeap(), an in-place sorting

algorithm is easily constructed:
Maximum element is at A[1]

Discard by swapping with element at A[n]
Decrement heap_size[A]
A[n] now contains correct value
Restore heap property at A[1] by calling

Heapify()
Repeat, always swapping A[1] for A[heap_size(A)]
Heapsort
Heapsort(A)
{
BuildHeap(A);
for (i = length(A) downto 2)
{
Swap(A[1], A[i]);
heap_size(A) -= 1;
Heapify(A, 1);
}
}
Analyzing Heapsort
The call to BuildHeap() takes O(n) time
Each of the n - 1 calls to Heapify() takes
O(lg n) time
Thus the total time taken by HeapSort()
= O(n) + (n - 1) O(lg n)
= O(n) + O(n lg n)
= O(n lg n)
Priority Queues
Heapsort is a nice algorithm, but in practice
Quicksort (coming up) usually wins
But the heap data structure is incredibly useful
for implementing priority queues
A data structure for maintaining a set S of elements,

each with an associated value or key
Supports the operations Insert(), Maximum(),
and ExtractMax()
What might a priority queue be useful for?
Priority Queue Operations

Insert(S, x) inserts the element x into set S
Maximum(S) returns the element of S with
the maximum key
ExtractMax(S) removes and returns the
element of S with the maximum key
How could we implement these operations
using a heap?

CS 332: Algorithms: Introduction To Heapsort

Uploaded by

Copyright:

Available Formats

CS 332: Algorithms: Introduction To Heapsort

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CS 332: Algorithms: Introduction To Heapsort

Uploaded by

Copyright:

Available Formats

CS 332: Algorithms

Review: The Master Theorem

Given: a divide and conquer algorithm

An algorithm that divides the problem of size n

Then, the Master Theorem gives us a

Review: The Master Theorem

if T(n) = aT(n/b) + f(n) then

f (n) n logb a AND

af (n / b) cf (n) for large n

So far weve talked about two algorithms to sort

What is the advantage of merge sort?

Answer: O(n lg n) worst-case running time

What is the advantage of insertion sort?

Next on the agenda: Heapsort

Combines advantages of both previous algorithms

A heap can be seen as a complete binary tree:

What makes a binary tree complete?

A heap can be seen as a complete binary tree:

The book calls them nearly complete binary trees;

In practice, heaps are usually implemented as

To represent a complete binary tree as an array:

The root node is A[1]

Referencing Heap Elements

An aside: How would you implement this

Trick question, I was looking for i << 1, etc.

The Heap Property

Heaps also satisfy the heap property:

In other words, the value of a node is at most the

The height of a node in the tree = the number of

What is the height of an n-element heap?

Heap Operations: Heapify()

Heapify(): maintain the heap property

Given: a node i in the heap with children l and r

What do you suppose will be the basic operation between i, l,

Heap Operations: Heapify()

Analyzing Heapify(): Informal

Analyzing Heapify(): Formal

Answer: 2n/3 (worst case: bottom row 1/2 full)

Analyzing Heapify(): Formal

Heap Operations: BuildHeap()

We can build a heap in a bottom-up manner by

Fact: for array of length n, all elements in range

Work through example

Is this a correct asymptotic upper bound?

A tighter bound is O(n)

How can this be? Is there a flaw in the above

Analyzing BuildHeap(): Tight

To Heapify() a subtree takes O(h) time

h = O(lg m), m = # nodes in subtree

Fact: an n-element heap has at most n/2h+1

Given BuildHeap(), an in-place sorting

Maximum element is at A[1]

Restore heap property at A[1] by calling

A data structure for maintaining a set S of elements,

Priority Queue Operations

You might also like