Data Structures and Algorithms

with Object-Oriented Design Patterns in C++

Bruno R. Preiss
B.A.Sc., M.A.Sc., Ph.D., P.Eng.
Associate Professor
Department of Electrical and Computer Engineering
University of Waterloo, Waterloo, Canada

Contents
Preface

Chapter 0 Introduction

0.1 What This Book Is About

1.2 Object-Oriented Design
0.2.1 Abstraction
0.2.2 Encapsulation
0.3 Object Hierarchies and Design Patterns
0.3.1 Containers
0.3.2 Iterators
0.3.3 Visitors
o Adapters
o Singletons
o The Features of C++ You Need to Know
Variables
Parameter Passing
Pointers
Classes and Objects
Inheritance
Other Features
o How This Book Is Organized
Models and Asymptotic Analysis
Foundational Data Structures
Abstract Data Types and the Class Hierarchy
Data Structures
Algorithms
Algorithm Analysis
o A Detailed Model of the Computer
The Basic Axioms
A Simple Example-Arithmetic Series Summation
Array Subscripting Operations
Another Example-Horner's Rule

 Analyzing Recursive Functions

Solving Recurrence Relations-Repeated Substitution
Yet Another Example-Finding the Largest Element of an Array
Average Running Times
About Harmonic Numbers
Best-Case and Worst-Case Running Times
The Last Axiom
o A Simplified Model of the Computer
An Example-Geometric Series Summation
About Arithmetic Series Summation
Example-Geometric Series Summation Again
About Geometric Series Summation
Example-Computing Powers
Example-Geometric Series Summation Yet Again
o Exercises
o Projects
Asymptotic Notation
o An Asymptotic Upper Bound-Big Oh
A Simple Example
Big Oh Fallacies and Pitfalls
Properties of Big Oh
About Polynomials
About Logarithms
Tight Big Oh Bounds
More Big Oh Fallacies and Pitfalls
Conventions for Writing Big Oh Expressions
o An Asymptotic Lower Bound-Omega

 A Simple Example
About Polynomials Again
o More Notation-Theta and Little Oh
o Asymptotic Analysis of Algorithms
Rules For Big Oh Analysis of Running Time
Example-Prefix Sums
Example-Fibonacci Numbers
Example-Bucket Sort
Reality Check
Checking Your Analysis
o Exercises
o Projects
Foundational Data Structures
o Dynamic Arrays
Default Constructor
Array Constructor
Copy Constructor
Destructor
Array Member Functions
Array Subscripting Operator
Resizing an Array
o Singly-Linked Lists
An Implementation
List Elements
Default Constructor
Destructor and Purge Member Function
Accessors

 First and Last Functions

Prepend
Append
Copy Constructor and Assignment Operator
Extract
InsertAfter and InsertBefore
o Multi-Dimensional Arrays
Array Subscript Calculations
Two-Dimensional Array Implementation
Multi-Dimensional Subscripting in C++
Canonical Matrix Multiplication
o Exercises
o Projects
Data Types and Abstraction
o Abstract Data Types
o Design Patterns
Class Hierarchy
Objects
Implementation
The NullObject Singleton Class
Implementation
Object Wrappers for the Built-In Types
Implementation
Containers
Visitors
The IsDone Member Function
Container Class Default Put Member Function

 Iterators
The NullIterator Class
Direct vs. Indirect Containment
Ownership of Contained Objects
Associations
Implementation
Searchable Containers
o Exercises
o Projects
Stacks, Queues and Deques
o Stacks
Array Implementation
Member Variables
Constructor and Destructor
Push, Pop, and Top Member Functions
The Accept Member Function
Iterator
Linked List Implementation
Member Variables
Constructor and Destructor
Push, Pop, and Top Member Functions
The Accept Member Function
Iterator
Applications
Evaluating Postfix Expressions
Implementation

o Queues
Array Implementation
Member Variables
Constructor and Destructor
Head, Enqueue, and Dequeue Member Functions
Linked List Implementation
Member Variables
Constructor and Destructor
Head, Enqueue and Dequeue Member Functions
Applications
Implementation
o Deques
Array Implementation
Tail, EnqueueHead, and DequeueTail Member Functions
Linked List Implementation
Tail, EnqueueHead, and DequeueTail Member Functions
Doubly-Linked and Circular Lists
o Exercises
o Projects
Ordered Lists and Sorted Lists
o Ordered Lists
Array Implementation
Member Variables
Inserting and Accessing Items in a List
Finding Items in a List
Removing Items from a List

 Positions of Items in a List

Finding the Position of an Item and Accessing by Position
Inserting an Item at an Arbitrary Position
Removing Arbitrary Items by Position
Linked List Implementation
Member Variables
Inserting and Accessing Items in a List
Finding Items in a List
Removing Items from a List
Positions of Items in a List
Finding the Position of an Item and Accessing by Position
Inserting an Item at an Arbitrary Position
Removing Arbitrary Items by Position
Performance Comparison: ListAsArray vs. ListAsLinkedList
Applications
o Sorted Lists
Array Implementation
Inserting Items in a Sorted List
Locating Items in an Array-Binary Search
Finding Items in a Sorted List
Removing Items from a List
Linked List Implementation
Inserting Items in a Sorted List
Other Operations on Sorted Lists
Performance
Comparison: SortedListAsArray vs. SortedListAsList
Applications

 Implementation
Analysis
o Exercises
o Projects
Hashing, Hash Tables and Scatter Tables
o Hashing-The Basic Idea
Example
Keys and Hash Functions
Avoiding Collisions
Spreading Keys Evenly
Ease of Computation
o Hashing Methods
Division Method
Middle Square Method
Multiplication Method
Fibonacci Hashing
o Hash Function Implementations
Integral Keys
Floating-Point Keys
Character String Keys
Hashing Objects
Hashing Containers
Using Associations
o Hash Tables
Separate Chaining
Implementation
Constructor and Destructor

 Inserting and Removing Items

Finding an Item
Average Case Analysis
o Scatter Tables
Chained Scatter Table
Implementation
Constructors and Destructor
Inserting and Finding an Item
Removing Items
Worst-Case Running Time
Average Case Analysis
o Scatter Table using Open Addressing
Linear Probing
Quadratic Probing
Double Hashing
Implementation
Constructors and Destructor
Inserting Items
Finding Items
Removing Items
Average Case Analysis
o Applications
o Exercises
o Projects
Trees
o Basics

 Terminology
More Terminology
Alternate Representations for Trees
o N-ary Trees
o Binary Trees
o Tree Traversals
Preorder Traversal
Postorder Traversal
Inorder Traversal
Breadth-First Traversal
o Expression Trees
Infix Notation
Prefix Notation
Postfix Notation
o Implementing Trees
Tree Traversals
Depth-First Traversal
Preorder, Inorder and Postorder Traversals
Breadth-First Traversal
Accept Member Function
Tree Iterators
Member Variables
Constructor and Reset Member Function
Operator Member Functions
General Trees
Member Variables

 Member Functions
Constructor, Destructor, and Purge Member Function
Key and Subtree Member Functions
AttachSubtree and DetachSubtree Member Functions
N-ary Trees
Member Variables
Member Functions
Constructors
IsEmpty Member Function
Key, AttachKey and DetachKey Member Functions
Subtree, AttachSubtree and DetachSubtree Member
Functions
Binary Trees
Member Variables
Constructors
Destructor and Purge Member Functions
Binary Tree Traversals
Comparing Trees
Applications
Implementation
o Exercises
o Projects
Search Trees
o Basics
M-Way Search Trees
Binary Search Trees
o Searching a Search Tree

 Searching an M-way Tree

Searching a Binary Tree
o Average Case Analysis
Successful Search
Solving The Recurrence-Telescoping
Unsuccessful Search
Traversing a Search Tree
o Implementing Search Trees
Binary Search Trees
Member Variables
Find Member Function
FindMin Member Function
Inserting Items in a Binary Search Tree
Insert and AttachKey Member Functions
Removing Items from a Binary Search Tree
Withdraw and DetachKey Member Functions
o AVL Search Trees
Implementing AVL Trees
Constructor
Height, AdjustHeight and BalanceFactor Member
Functions
Inserting Items into an AVL Tree
Balancing AVL Trees
Single Rotations
Double Rotations
Implementation
Removing Items from an AVL Tree

o M-Way Search Trees

Implementing M-Way Search Trees
Implementation
Member Functions
Inorder Traversal
Finding Items in an M-Way Search Tree
Linear Search
Binary Search
Inserting Items into an M-Way Search Tree
Removing Items from an M-Way Search Tree
o B-Trees
Implementing B-Trees
Member Variables
Constructors
Private Member Functions
Inserting Items into a B-Tree
Implementation
Running Time Analysis
Removing Items from a B-Tree
o Applications
o Exercises
o Projects
Heaps and Priority Queues
o Basics
o Binary Heaps
Complete Trees

 Complete N-ary Trees

Implementation
Member Variables
Constructor, Destructor and Purge Member Functions
Putting Items into a Binary Heap
Removing Items from a Binary Heap
o Leftist Heaps
Leftist Trees
Implementation
Member Variables
SwapContents Member Function
Merging Leftist Heaps
Putting Items into a Leftist Heap
Removing Items from a Leftist Heap
o Binomial Queues
Binomial Trees
Binomial Queues
Implementation
Heap-Ordered Binomial Trees
Binomial Queues
Member Variables
AddTree and RemoveTree
FindMinTree and FindMin Member Functions
Merging Binomial Queues
Putting Items into a Binomial Queue
Removing an Item from a Binomial Queue

o Applications
Discrete Event Simulation
Implementation
o Exercises
o Projects
Sets, Multisets and Partitions
o Basics
Implementing Sets
o Array and Bit-Vector Sets
Basic Operations
Union, Intersection and Difference
Comparing Sets
Bit-Vector Sets
Basic Operations
Union, Intersection and Difference
o Multisets
Array Implementation
Basic Operations
Union, Intersection and Difference
Linked List Implementation
Union
Intersection
o Partitions
Representing Partitions
Implementing a Partition using a Forest
Implementation
Constructors and Destructor

 Find and Join Member Functions

Collapsing Find
Union by Size
Union by Height or Rank
o Applications
o Exercises
o Projects
Dynamic Storage Allocation: The Other Kind of Heap
o Basics
C++ Magic
Working with Multiple Storage Pools
The Heap
o Singly Linked Free Storage
Implementation
Constructor and Destructor
Acquiring an Area
Releasing an Area
o Doubly Linked Free Storage
Implementation
Constructor and Destructor
Releasing an Area
Acquiring an Area
o Buddy System for Storage Management
Implementation
Constructor and Destructor
Acquiring an Area

 Releasing an Area
o Applications
Implementation
o Exercises
o Projects
Algorithmic Patterns and Problem Solvers
o Brute-Force and Greedy Algorithms
Example-Counting Change
Brute-Force Algorithm
Greedy Algorithm
Example-0/1 Knapsack Problem
o Backtracking Algorithms
Example-Balancing Scales
Representing the Solution Space
Abstract Backtracking Solvers
Depth-First Solver
Breadth-First Solver
Branch-and-Bound Solvers
Depth-First, Branch-and-Bound Solver
Example-0/1 Knapsack Problem Again
o Top-Down Algorithms: Divide-and-Conquer
Example-Binary Search
Example-Computing Fibonacci Numbers
Example-Merge Sorting
Running Time of Divide-and-Conquer Algorithms
Case 1 (

 Case 2 (

Case 3 (

Summary
Example-Matrix Multiplication
o Bottom-Up Algorithms: Dynamic
Programming
Example-Generalized Fibonacci Numbers
Example-Computing Binomial Coefficients
Application: Typesetting Problem
Example
Implementation
o Randomized Algorithms
Generating Random Numbers
The Minimal Standard Random Number Generator
Implementation
Random Variables
Implementation
Monte Carlo Methods
Example-Computing
Simulated Annealing
Example-Balancing Scales
o Exercises
o Projects
Sorting Algorithms and Sorters
o Basics
o Sorting and Sorters
Sorter Class Hierarchy

o Insertion Sorting
Straight Insertion Sort
Implementation
Average Running Time
Binary Insertion Sort
o Exchange Sorting
Bubble Sort
Quicksort
Implementation
Running Time Analysis
Worst-Case Running Time
Best-Case Running Time
Average Running Time
Selecting the Pivot
o Selection Sorting
Straight Selection Sorting
Implementation
Sorting with a Heap
Implementation
Building the Heap
Running Time Analysis
The Sorting Phase
o Merge Sorting
Implementation
Merging
Two-Way Merge Sorting
Running Time Analysis

o A Lower Bound on Sorting

o Distribution Sorting
Bucket Sort
Implementation
Radix Sort
Implementation
o Performance Data
o Exercises
o Projects
Graphs and Graph Algorithms
o Basics
Directed Graphs
Terminology
More Terminology
Directed Acyclic Graphs
Undirected Graphs
Terminology
Labeled Graphs
Representing Graphs
Adjacency Matrices
Sparse vs. Dense Graphs
Adjacency Lists
o Implementing Graphs
Implementing Vertices
Implementing Edges
Abstract Graphs and Digraphs
Accessors and Mutators

 Iterators
Graph Traversals
Implementing Undirected Graphs
Using Adjacency Matrices
Using Adjacency Lists
Edge-Weighted and Vertex-Weighted Graphs
Comparison of Graph Representations
Space Comparison
Time Comparison
o Graph Traversals
Depth-First Traversal
Implementation
Running Time Analysis
Breadth-First Traversal
Implementation
Running Time Analysis
Topological Sort
Implementation
Running Time Analysis
Graph Traversal Applications:
Testing for Cycles and Connectedness
Connectedness of an Undirected Graph
Connectedness of a Directed Graph
Testing for Cycles in a Directed Graph
o Shortest-Path Algorithms
Single-Source Shortest Path
Dijkstra's Algorithm

 Data Structures for Dijkstra's Algorithm

Implementation
Running Time Analysis
All-Pairs Source Shortest Path
Floyd's Algorithm
Implementation
Running Time Analysis
o Minimum-Cost Spanning Trees
Constructing Spanning Trees
Minimum-Cost Spanning Trees
Prim's Algorithm
Implementation
Kruskal's Algorithm
Implementation
Running Time Analysis
o Application: Critical Path Analysis
Implementation
o Exercises
o Projects
C++ and Object-Oriented Programming
o Variables, Pointers and References
Pointers Are Variables
Dereferencing Pointers
References are Not Variables
o Parameter Passing
Pass By Value

 Pass By Reference
The Trade-off
Constant Parameters
o Objects and Classes
Member Variables and Member Functions
Constructors and Destructors
Default Constructor
Copy Constructor
The Copy Constructor, Parameter Passing and Function Return
Values
Destructors
Accessors and Mutators
Mutators
Member Access Control
o Inheritance and Polymorphism
Derivation and Inheritance
Derivation and Access Control
Polymorphism
Virtual Member Functions
Abstract Classes and Concrete Classes
Algorithmic Abstraction
Multiple Inheritance
Run-Time Type Information and Casts
o Templates
o Exceptions
Class Hierarchy Diagrams
Character Codes

 References
Index

Preface
This book was motivated by my experience in teaching the
course E&CE 250: Algorithms and Data Structures in the Computer
Engineering program at the University of Waterloo. I have observed that
the advent of object-oriented methods and the emergence of objectoriented design patterns has lead to a profound change in the pedagogy
of data structures and algorithms. The successful application of these
techniques gives rise to a kind of cognitive unification: Ideas that are
disparate and apparently unrelated seem to come together when the
appropriate design patterns and abstractions are used.
This paradigm shift is both evolutionary and revolutionary. On the one
hand, the knowledge base grows incrementally as programmers and
researchers invent new algorithms and data structures. On the other
hand, the proper use of object-oriented techniques requires a
fundamental change in the way the programs are designed and
implemented. Programmers who are well schooled in the procedural ways
often find the leap to objects to be a difficult one.
Goals
The primary goal of this book is to promote object-oriented design using
C++ and to illustrate the use of the emerging object-oriented design
patterns. Experienced object-oriented programmers find that certain
ways of doing things work best and that these ways occur over and over
again. The book shows how these patterns are used to create good
software designs. In particular, the following design patterns are used
throughout the text: singleton, container, iterator, adapter and visitor.
Virtually all of the data structures are presented in the context of
a single, unified, polymorphic class hierarchy. This framework clearly
shows the relationships between data structures and it illustrates how
polymorphism and inheritance can be used effectively. In
addition, algorithmic abstraction is used extensively when presenting
classes of algorithms. By using algorithmic abstraction, it is possible to
describe a generic algorithm without having to worry about the details of
a particular concrete realization of that algorithm.

A secondary goal of the book is to present mathematical tools just in

time. Analysis techniques and proofs are presented as needed and in the
proper context. In the past when the topics in this book were taught at
the graduate level, an author could rely on students having the needed
background in mathematics. However, because the book is targeted for
second- and third-year students, it is necessary to fill in the background
as needed. To the extent possible without compromising correctness, the
presentation fosters intuitive understanding of the concepts rather than
mathematical rigor.

Approach
One cannot learn to program just by reading a book. It is a skill that
must be developed by practice. Nevertheless, the best practitioners study
the works of others and incorporate their observations into their own
practice. I firmly believe that after learning the rudiments of program
writing, students should be exposed to examples of complex, yet welldesigned program artifacts so that they can learn about the designing
good software.
Consequently, this book presents the various data structures and
algorithms as complete C++ program fragments. All the program
fragments presented in this book have been extracted automatically from
the source code files of working and tested programs.
The full functionality of the proposed draft ANSI standard C++ language
is used in the examples--including templates, exceptions and run-time
type information[3]. It has been my experience that by developing the
proper abstractions, it is possible to present the concepts as fully
functional programs without resorting to pseudo-code or to hand-waving.
Outline
This book presents material identified in the Computing
Curricula 1991 report of the ACM/IEEE-CS Joint Curriculum Task
Force[38]. The book specifically addresses the following knowledge units:
AL1: Basic Data structures, AL2: Abstract Data Types, AL3: Recursive
Algorithms, AL4: Complexity Analysis, AL6: Sorting and Searching, and
AL8: Problem-Solving Strategies. The breadth and depth of coverage is
typical of what should appear in the second or third year of an
undergraduate program in computer science/computer engineering.
In order to analyze a program, it is necessary to develop a model of the
computer. Chapter develops several models and illustrates with

examples how these models predict performance. Both average-case and

worst-case analyses of running time are considered. Recursive algorithms
are discussed and it is shown how to solve a recurrence using repeated
substitution. This chapter also reviews arithmetic and geometric series
summations, Horner's rule and the properties of harmonic numbers.
Chapter introduces asymptotic (big-oh) notation and shows by
comparing with Chapter that the results of asymptotic analysis are
consistent with models of higher fidelity. In addition to

, this chapter

also covers other asymptotic notations (

,
and
) and develops
the asymptotic properties of polynomials and logarithms.
Chapter introduces the foundational data structures--the array and the
linked list. Virtually all the data structures in the rest of the book can be
implemented using either one of these foundational structures. This
chapter also covers multi-dimensional arrays and matrices.
Chapter deals with abstraction and data types. It presents the recurring
design patterns used throughout the text as well a unifying framework for
the data structures presented in the subsequent chapters. In particular,
all of the data structures are viewed as abstract containers.
Chapter discusses stacks, queues and deques. This chapter presents
implementations based on both foundational data structures (arrays and
linked lists). Applications for stacks and queues and queues are
presented.
Chapter covers ordered lists, but sorted and unsorted. In this chapter, a
list is viewed as a searchable container. Again several applications of lists
are presented.
Chapter introduces hashing and the notion of a hash table. This chapter
addresses the design of hashing functions for the various basic data types
as well as for the abstract data types described in Chapter . Both scatter
tables and hash tables are covered in depth and analytical performance
results are derived.
Chapter introduces trees and describes their many forms. Both depthfirst and breadth-first tree traversals are presented. Completely generic
traversal algorithms based on the use of the visitor design pattern are
presented, thereby illustrating the power of algorithmic abstraction. This
chapter also shows how trees are used to represent mathematical
expressions and illustrates the relationships between traversals and the
various expression notations (prefix, infix and postfix).

Chapter addresses trees as searchable containers. Again, the power

of algorithmic abstraction is demonstrated by showing the relationships
between simple algorithms and balancing algorithms. This chapter also
presents average case performance analyses and illustrates the solution
of recurrences by telescoping.
Chapter presents several priority queue implementations, including
binary heaps, leftist heaps and binomial queues. In particular this chapter
illustrates how a more complicated data structure (leftist heap) extends
an existing one (tree). Discrete-event simulation is presented as an
application of priority queues.
Chapter covers sets and multisets. Also covered are partitions and
disjoint set algorithms. The latter topic illustrates again the use of
algorithmic abstraction.
Techniques for dynamic storage management are presented in Chapter
This is a topic that is not found often in texts of this sort. However, the
features of C++ which allow the user to redefine
the new and delete operators make this topic approachable.

Chapter surveys a number of algorithm design techniques. Included are

brute-force and greedy algorithms, backtracking algorithms (including
branch-and-bound), divide-and-conquer algorithms and dynamic
programming. An object-oriented approach based on the notion of
an abstract solution space and an abstract solver unifies much of the
discussion. This chapter also covers briefly random number generators,
Monte Carlo methods, and simulated annealing.
Chapter covers the major sorting algorithms in an object-oriented style
based on the notion of an abstract sorter. Using the abstract sorter
illustrates the relationships between the various classes of sorting
algorithm and demonstrates the use of algorithmic abstractions.
Finally, Chapter presents an overview of graphs and graph algorithms.
Both depth-first and breadth-first graph traversals are presented.
Topological sort is viewed as yet another special kind of traversal. Generic
traversal algorithms based on the visitor design pattern are presented,
once more illustrating algorithmic abstraction. This chapter also covers
various shortest path algorithms and minimum-spanning-tree algorithms.
At the end of each chapter is a set of exercises and a set of programming
projects. The exercises are designed to consolidate the concepts
presented in the text. The programming projects generally require the
student to extend the implementation given in the text.

Online Course Materials

Additional material supporting this book can be found on the world-wide
web at the URL:
http://www.pads.uwaterloo.ca/Bruno.Preiss/books/opus4
In particular, you will find there the source code for all the program
fragments in this book as well as an errata list.