---& Algorithms

Data Structures
 What The Course Is About
 Data structures is concerned with the
representation and manipulation of
data.
 All programs manipulate data.
 Data manipulation requires an
algorithm.
 What The Course Is About
• Algorithm design methods are needed
to develop programs that do the data
manipulation.
• The study of data structures and
algorithms is fundamental to Computer
Science.
 Sorting
 Rearrange a[0], a[1], …, a[n-1] into
ascending order. When done, a[0] <=
a[1] <= … <= a[n-1]
 8, 6, 9, 4, 3 => 3, 4, 6, 8, 9
 Sort Methods
 Insertion Sort
 Bubble Sort
 Selection Sort
 Count Sort
 Shaker Sort
 Shell Sort
 Heap Sort
 Merge Sort
 Quick Sort
 Insert An Element
 Given a sorted list/sequence, insert a
new element
 Given 3, 6, 9, 14
 Insert 5
 Result 3, 5, 6, 9, 14
 Insert an Element
 3, 6, 9, 14 insert 5
 Compare new element (5) and last one
(14)
 Shift 14 right to get 3, 6, 9, , 14
 Shift 9 right to get 3, 6, , 9, 14
 Shift 6 right to get 3, , 6, 9, 14
 Insert 5 to get 3, 5, 6, 9, 14
 Insert An Element
// insert t into a[0:i-1]
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t;
 Insertion Sort
 Start
with a sequence of size 1
 Repeatedly insert remaining elements
 Insertion Sort
 Sort 7, 3, 5, 6, 1
 Start with 7 and insert 3 => 3, 7
 Insert 5 => 3, 5, 7
 Insert 6 => 3, 5, 6, 7
 Insert 1 => 1, 3, 5, 6, 7
 Insertion Sort
for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
// code to insert comes here
}
 Insertion Sort
for (int i = 1; i < a.length; i++)
{// insert a[i] into a[0:i-1]
int t = a[i];
int j;
for (j = i - 1; j >= 0 && t < a[j]; j--)
a[j + 1] = a[j];
a[j + 1] = t;
}
ALGORITHMS
 Definition
 What is an Algorithm?
 Informally an algorithm is any well-defined
computational procedure that takes some value
or set of values as input and produces some
value or set of values as output.
 An algorithm is thus a sequence of
computational steps that transform the input
into the output.
 You can also view an algorithm as a tool for
solving a well-specified computational problem.
 Definition
 An algorithm is a set of well-defined
steps required to accomplish some
task.
 Algorithms also usually involve taking a
system from one state to another,
possibly transitioning through a series
of intermediate states along the way.
 Algorithms
 Look at the problem : we need to sort a sequence
of numbers into non deceasing order.

 Input: A sequence of n numbers (a1, a2, …….an)

 Output: A permutation (reordering ) {a11, a12,
a13…… a1n} of the input sequence such that
a11<= a12 <= a13……<= a1n
 For example given
– the input sequence {31, 41, 59, 26, 41 58}
– a sorting algorithm returns
– as output the sequence {26, 31, 41, 58, 59}
 Algorithms
 An algorithm is said to be correct if for every
input instance it halts with the correct output
 An algorithm can be specified in
– English
– as a computer program
– or even as a hardware design
 The only requirement is that the specification
must provide a precise description of the
computational procedure to be followed .
 Algorithms
Practical applications
1. The Internet
 The internet enables people around the
world to quickly access and retrieve large
amounts of information .
 With the aid of clever algorithms, sites on
the internet are able to manage and
manipulate this large volume of data
 Algorithms
Practical applications
 Finding good routes on which the data
will travel (the shortest route/path) and
using search engines to quickly find
pages which particular information
resides
 This is all done with the aid of “string
matching” algorithm
 Algorithms
Practical applications
2. Electronic Commerce
 Electronic commerce enables goods and
services to be negotiated and exchanged
electronically, it depends on the privacy of
personal information such as credit card
numbers, passwords and bank statements .
 The core technologies used in electronic
commerce include public-key cryptography
and digital signatures. These are based on
numerical algorithms and number theory.
 Algorithms
Other Practical applications
 Manufacturing and other commercial enterprises
need to allocate scarce resources in the most
beneficial way
 A political candidate may want to determine where
to spend money buying campaign advertising in
order to maximize the chances of winning an
election
 An internet provider may wish to determine where
to place additional resources in order to serve its
customers more effectively.
 All these problems above can be solved using
linear programming algorithms
 Algorithm
 An algorithm is defined or measured in
terms of efficiency and for the purpose of
this course how long an algorithm takes to
produce its results.
 Different algorithms devised to solve the
same problem often differ dramatically in
their efficiency and these difference can
be more significant than differences due to
hardware and software.
 Algorithm Comparisons
 Take
two sorting algorithms insertion sort and
merge sort:
– insertion sort takes n2 times to sort
– merge sort takes nlogn times

 Take two computers A & B

– A is 1,000 times faster than B .
 Therefore suppose
– A computer executes 1010 (ten billion )
instructions per second and
– B computer executes 107 (ten million )
instructions per second
 Algorithm Comparisons
 Tosort 10 million numbers on computer A
using insertion sort Algorithm:
This will take
– A = (107)2 / 1010 = 20,000 seconds
( more than 5.5 hours )
 Tosort 10 million numbers on computer B
using merge sort Algorithm 2 This will take
– B = 107 log 107 / 107 = 1163 seconds (less
than 20 minutes)
 Algorithm Comparisons
 Using an Algorithm 2 whose running
time grows more slowly, even with a
slower machine B runs more than 17
times faster than Algorithm 1 on
machine A which is 1,000 times faster.
 This is noticed even more when the
number that needs to be sorted
increases.
 Algorithm Comparisons
 To sort 100 million numbers,
– insertion sort (Algorithm 1) takes more than 23
days
– merge sort (Algorithm 2) takes under 4 hours
 Thiscan be summed up by saying in general
as the input/problem size increases so does
the relative advantage of merge sort.
 Algorithm Comparisons
 The demonstration in the previous slide
of the different sorting algorithms
example shows that we should consider
algorithms like computer hardware as a
technology.
 Total system performance depends on
choosing efficient algorithms as much
as on choosing fast hardware.
 Algorithm Comparisons
A good algorithm on a slow computer can
outperform a bad algorithm on a super
computer.
 How do you determine the efficiency of the
algorithm you have defined
 Its important to ascertain how a given
algorithm behaves as the size of the
problem increases. In most cases, the
input size is the reference of the problem.
 Order of Growth
 Itis traditional to describe the running
time of an algorithm as a function of the
input size.
 The running time of an algorithm on a
particular input is the number of
primitive operations or steps executed.
 Order of growth
 The worst-case running time is the longest
running time for any input of size n
1. The worst case gives us an upper-bound on
the run time for any input and guarantees that
the algorithm will never take any longer
2. For some algorithms the worst case occurs
very frequently. Example: search a database
for a particular piece of information. Worst case
will occur when information is not there. In
some applications searching for information
that is not there can be frequent.
 Order of growth
 We are usually interested in the rate of
growth or order of growth of the running
time.
 We say one algorithm A is more efficient
than another say B if its worst case
running time has a lower order of growth
 Properties of an Algorithm
 Not all processes are algorithms.
 To be an algorithm, a process must have the
following properties:
1. It must be unambiguous. In other words, it must be
possible to describe every step of the process in
enough detail that anyone (even a machine) can
carry out the algorithm in the way intended by its
designer.
– This requires not only knowing exactly how to
perform each step, but also the exact order in
which the steps should be performed.
2. It must always solve the problem.
– In other words, a person (or machine) who
starts carrying out the algorithm in order to
solve an instance of the problem must be
able to stop with the correct answer after
performing a finite number of steps.
– You must eventually reach a correct answer
no matter what instance of the problem you
started with.
 The two properties defined in the previous
slide lead to the following concise
definition:

 analgorithm is a finite, ordered sequence of

unambiguous steps that leads to a solution
to a problem.
 Algorithm
 The study of algorithms also involves the
study of the data that algorithms process,
because the nature of an algorithm often
follows closely from the nature of the data
on which the algorithm works.
 Description of algorithms:
 Pseudo code
 Programs
 Natural language e.g. English, French, etc.
 Expressions
 Computer hardware (electronic circuits in a
computer)
Description of algorithms:
Pseudo code
 Pseudo code is any notation intended to describe
algorithms clearly and unambiguously to humans.
 Pseudo code is a popular alternative to actual
programs when describing algorithms to people.
 Pseudo code typically combine programming
languages' precision about steps and their ordering
with natural language's flexibility of syntax and
wording.
 There is no standard pseudo code that you must
learn
 Pseudo code
 We use pseudo code to describe algorithms'
general outlines, particularly when we
begin to present an algorithm whose details
are not fully developed.
 We use programming language like Java
when we want to describe an algorithm with
enough detail for a computer to understand
and execute it.
 Pseudo code
 The really important aspects of the science
of computing can be expressed just as well
in pseudo code as in a programming
language

 In Java the methods, more often than not,

are the algorithms
 In order to be solve a problem by an
algorithm, the problem must be defined
precisely enough for a person to be able to
tell whether a proposed answer is right or
wrong.
 Pre & post conditions
A precondition is a requirement that must be
met before you start solving a problem. For
example, "I know the traffic laws" is a
precondition for receiving a driver's license.
 A post condition is a statement about
conditions that exist after solving the problem.
For example, "I can legally drive a car" is a
post condition of receiving a driver's license.
 Pre & post conditions
 Like all algorithms, expressions have preconditions
and post conditions. For example, the expression:
 x / y has a precondition that y ≠ 0.
 Similarly, the expression: Math.sqrt(x)
 has a precondition that x ≥ 0.
 In Java, this expression also has a post condition
that its result is non-negative.
 This post condition is important, because
mathematically any non-zero number has both a
positive and a negative square root.
 ALGORITHMS THAT
PRODUCE EFFECTS
 Algorithmsproduce their results by changing
something e.g.
– changing the contents of a file or the image
displayed on a monitor,
– changing a robot's position, etc.
 The things that are changed are external to the
algorithms (that is, not defined within the
algorithms themselves), and the changes
persist after the algorithms finish. Such
changes are called side effects
 ALGORITHMS THAT
PRODUCE VALUES
 Consider an algorithm that calculates the
area of a square that is n meters on a side.
In English, this algorithm might be stated
as:
– Take the value n, and multiply it by itself.
• In Java, this algorithm corresponds to the
expression: n*n
Example (producing by effect and
by value)
 Two fragments of code that place a value in variable
“result”
 In class ValueExample...
– private static int x;
– public static void normal( )
– { x = 1; int result = add(1, 2); }
 In class SideEffectExample...
– private static int x;
– public static void weird( )
– { x = 1; add(1, 2); int result = x; }
 Drink chilling time problem
 Some people chill bottles or cans of soft drinks
or fruit juice by putting them in a freezer for a
short while before drinking them.
 This is a nice way to get an extra-cold drink,
but it risks disaster: a drink left too long in the
freezer begins to freeze, at which point it starts
to expand, ultimately bursting its container and
spilling whatever liquid isn't already frozen all
over the freezer.
 Drink chilling time problem
 Now different freezers may chill drinks at
different speeds, and larger drinks will
generally have longer safe chilling times than
smaller drinks.
 Furthermore, there will be some margin of error
on chilling times, within which more or less
chilling really doesn't matter—for example,
chilling a drink for a second more or a second
less than planned is unlikely to change it from
unacceptably warm to messily frozen
 Algorithm 1 to calculate longest safe
chilling time for a drink
 solving the longest safe chilling time problem :
 means finding the longest time that a given drink
can be chilled in a given freezer without starting to
freeze:
In pseudo code
 Set chilling time to 0 minutes.
 Repeat until drink starts to freeze:
– Add margin of error to chilling time.
– Chill a drink for the chilling time.
 (End of repeated section)
 Previous chilling time is the answer.
Algorithm 2 to calculate longest safe
chilling time for a drink
 Another algorithm could be written like this in
pseudo code:
 Set "too warm" time to 0 minutes.
 Set "too cold" time to a very long time.
– Repeat until "too cold" time is within margin of error of
"too warm" time:
 Set "middle time" to be halfway between "too warm" and "too
cold" times. Chill a drink for "middle time."
 If the drink started to freeze,
– Set "too cold" time to "middle time."
– Otherwise Set "too warm" time to "middle time." (End of repeated
section) "Too warm" time is the answer.
 Algorithm to give change from a
vending machine
 Algorithm MakeChange
– Give as many 50p as you can without
exceeding the amount owed.
– Give as many 20p as you can without
exceeding the amount still owed after the 50p.
– Give as many 10p as you can without
exceeding the amount still owed after the 20p
– Give as many 5p as you can without
exceeding the amount still owed after the 10p
– Give whatever is left in 1p.
 Algorithms
 Algorithms can also take the form of computer
hardware.
 The electronic circuits inside a computer
implement algorithms for such operations as
adding or multiplying numbers, sending
information to or receiving it from external
devices, and so forth.
 Algorithms thus pervade all of computing, not
just software and programming, and they
appear in many forms.
 Sequence of steps:
Order does not matter
 For some problems, the order in which you
perform steps doesn't matter.
 For example, if setting a table involves
putting plates and glasses on it, then the
problem “setting a table “ the table will get
set regardless of whether you put the plates
on first, or the glasses.
 Sequence of steps
order does not matter
 Every algorithm specifies some order
(which may be "simultaneously") for
executing steps
 Problems in which order doesn't matter can
simply be solved by several (minimally)
different algorithms that use different
orders.
 Sequence of steps
order does not matter
 In the “setting the table algorithm” Several people
can help solve the problem at the same time If
several people are helping, one person can even put
on the plates while another puts on the glasses.
 This last possibility is particularly interesting,
because it suggests that "simultaneously" can
sometimes be a valid order in which to do things.
 The subfield of computer science known as parallel
computing studies algorithms that take advantage of
this.
 Algorithm Design Types
 Algorithm types include:
– Simple recursive algorithms
– Backtracking algorithms
– Divide and conquer algorithms
– Dynamic programming algorithms
– Greedy algorithms
– Branch and bound algorithms
– Brute force algorithms
– Randomized algorithms
Problem Solving: Iteration and Recursion
Many problems can be solved using either iteration or
recursion.

Iteration: Iteration is simply the repetition of processing

steps.
The number of repetitions/iterations required to solve a
problem can be determined by many different factors. E.g. To
calculate the total of your stock portfolio, you would iterate over
your stock holdings, keeping a running total until each holding
has been processed.
 In this case, the number of repetitions is determined by how
many stock holdings you happen to have.
 In this case, the number of repetitions is determined by how
many stock holdings you happen to have.
 Iteration lends itself more readily to solving some problems
while for others recursion can seem more natural.
 Iteration is a very simple, straightforward approach to
solving many common problems such as performing
calculations and processing arrays.
Recursion: A recursive algorithm involves a
method or function calling itself. It does this by
breaking a problem into smaller and smaller parts,
each looking very similar to the larger part yet finer
grained.
Recursion uses a divide-and-conquer approach
whereby a method makes repeated, nested calls to
itself. It is often a better choice for processing nested
data structures.
Recursion can often, though not always, be a more
natural way of expressing an algorithm than iteration.
 Exercise 1
 Write, as expressions, algorithms for
calculating the following:
1. The area of a circle whose radius is r.
2. The perimeter of a square whose side is n units
long.
3. The amount of paint needed to paint a wall h
feet high by w feet wide. Assume that 1 gallon
of paint paints 400 square feet of wall.
 Exercise 2
 Describe, in pseudo code, algorithms for
solving each of the following problems (you
can devise your own pseudo code syntax).
1. Counting the number of lines in a text file.
2. Raising a number to an integer power.
3. Finding the largest number in an array of
numbers.
4. Given two words, finding the one that would
come first in alphabetical order.
 Exercise 3
Explain why each of the following is
or is not an algorithm:
1. The following directions for becoming rich:
"Invent something that everyone wants. Then
sell it for lots of money."
2. The following procedure for baking a fish:
"Preheat the oven to 450 degrees. Place the
fish in a baking dish. Place the dish (with
fish) in the oven, and bake for 10 to 12
minutes per inch of thickness. If the fish is not
flaky when removed, return it to the oven for
a few more minutes and test again."
 Exercise 3 contd.

3. The following way to find the square root of a

number, n: "Pick a number at random, and
multiply it by itself. If the product equals n, stop,
you have found the square root. Otherwise, repeat
the process until you do find the square root."
4. How to check a lost-and-found for a missing
belonging: "Go through the items in the lost-and-
found one by one. Look carefully at each, to see if
you recognize it as yours. If you do, stop, you have
found your lost possession