LECTURE 3

ALGORITHM ANALYSIS
LECTURER: M: SOHAIB AJMAL
Algorithm

• Algorithm is step by step procedure to solve

any problem.

Analyzing / Judgment of the Algorithm

• An algorithm can be written in different ways

for solving a single problem.
• So by analyzing the algorithms we can find the
best solution (algorithm) to the problem.
Order of Growth

• Any algorithm is expected to work fast for any

input size.

• For smaller input size our algorithm will work fine

but for higher input size the execution time is
much higher

• By increasing the size of n (input size) we can

analyze how well our algorithm works.
• Let input size, n=5 and we have to sort the list of
elements for e.g. 25,29,10,15,2

• So for n=5 our algorithm will work fine but what

if n=5000?

• So our algorithm will take much longer time to

sort the elements
or cause small delays to give the result

• So how the behaviour of algorithm changes with

the no. of inputs will give the analysis of the
algorithm and is called the Order of Growth.
• For calculating the order of growth we have to
go for higher value of n, because

i. as the input size grow higher algorithm

makes delays.

ii.for all real time applications we have higher

values for n.
Efficiencies of the Algorithm
There are three cases
i. Best case
ii. Worst case
iii. Average case

• Let we have 5 nos.(n=5) 25,31,42,71,105 and

we have to find any element in the list.
Best case efficiency
• let we have to find 25 in List =>25,31,42,71,105
• k=25
• 25 is present at the first position
• Since only single comparison is made to search
the element so we say that it is best case
efficiency
• CBest (n)=1
Worst case efficiency
• If we want to search the element which is present at
the last of the list or not present at all in the list then
such cases are called the worst case efficiency.
• let we have to find 105 in List =>25,31,42,71,105
• k=105
• Therefore we have to make 5 (=n) comparisons to
search the element
• CWorst (n)=n
• And if we have to find 110
• k=110
• Since the element is not in the list even then we have
to make 5 (=n) comparisons to search the element
• CWorst (n)=n
Average case efficiency
• Let the element is not present at the first or the
last position

• Let it is present somewhere in middle of the list

• We know that probability of a successful search =

p where 0≤p≤1.

• And probability of unsuccessful search = 1-p

• Let the element we are searching for is present at
position 'i' in the list
• Therefore probability of the element to be found
is given by p/n.
• Therefore CAvg (n)
=[1*p/n + 2*p/n + ... + i*p/n + ... + n*p/n] + n(1-p)
=p/n[1+2+...+i+...+n] + n(1-p)
=p/n[n*(n+1)/2] + n(1-p)
=p[(n+1)/2] + n(1-p)
I II
• case 1. If element is available
therefore p=1 (for successful search)
Now substituting p=1 in above eqn
• CAvg (n) = 1[(n+1)/2] + n(1-1)
= (n+1)/2

• case 2. If element is unavailable

therefore p=0 (for unsuccessful search)
Now substituting p=0 in above eqn
• CAvg (n) = 0[(n+1)/2] + n(1-0)
=n
Therefore on average half of list will be searched to find
the element in the list
Operations in Order of Growth
• i.Big 'O‘ (Oh)
• ii.Big Ω (Omega)
• iii.Big θ (Theta)

• Consider an algorithm representing two

functions f(n) and g(n)
BIG O
• It represents the maximum time that the
algorithm takes for its execution.
g(n)
f(n)
Time

No. of inputs

• Growth rate of f(n) is not more than growth

rate of g(n).
• f(n)≤O[g(n)]
BIG Ω
• It represents the minimum time that the
algorithm takes for its execution.
f(n)
g(n)
Time

No. of inputs

• Growth rate of f(n) is not less than growth rate

of g(n).
• f(n)≥ Ω[g(n)]
BIG Θ
• It represents the average time an algorithm takes for its
execution.

C1.g(n)
f(n)
Time

C2.g(n)

No. of inputs

• Let C1*g(n) and C2*g(n) are two constant multiples of g(n)

• f(n) is bound above and below by two constant multiples of
g(n). f(n)<=O[g(n)]
• The growth rate of f(n) will not be less than
the growth rate of C2*g(n) and not more than
c1*g(n).
• C2*g(n)<=f(n)<=C1*g(n)
ALGORITHMIC PERFORMANCE
There are two aspects of algorithmic performance:
• Time
• Instructions take time.
• How fast does the algorithm perform?
• What affects its runtime?
• Space
• Data structures take space
• What kind of data structures can be used?
• How does choice of data structure affect the runtime?
➢ We will focus on time:
– How to estimate the time required for an algorithm
– How to reduce the time required

CENG 213 Data Structures 17

 ANALYSIS OF ALGORITHMS
• Analysis of Algorithms is the area of computer science that
provides tools to analyze the efficiency of different methods of
solutions.
• How do we compare the time efficiency of two algorithms that
solve the same problem?
Naïve Approach: implement these algorithms in a programming
language (C++), and run them to compare their time
requirements. Comparing the programs (instead of algorithms)
has difficulties.
– How are the algorithms coded?
• Comparing running times means comparing the implementations.
• We should not compare implementations, because they are sensitive to programming
style that may cloud the issue of which algorithm is inherently more efficient.
– What computer should we use?
• We should compare the efficiency of the algorithms independently of a particular
computer.
– What data should the program use?
• Any analysis must be independent of specific data.
CENG 213 Data Structures 18
 ANALYSIS OF
ALGORITHMS
• When we analyze algorithms, we should employ
mathematical techniques that analyze algorithms
independently of specific
implementations, computers, or data.

• To analyze algorithms:
– First, we start to count the number of significant
operations in a particular solution to assess its
efficiency.
– Then, we will express the efficiency of algorithms
using growth functions.

CENG 213 Data Structures 19

 THE EXECUTION TIME OF
ALGORITHMS
• Each operation in an algorithm (or a program) has a cost.
➔ Each operation takes a certain of time.

count = count + 1; ➔ take a certain amount of time, but it is constant

A sequence of operations:

count = count + 1; Cost: c1

sum = sum + count; Cost: c2

➔ Total Cost = c1 + c2

CENG 213 Data Structures 20

 THE EXECUTION TIME OF
ALGORITHMS (CONT.)
Example: Simple If-Statement
Cost Times
if (n < 0) c1 1
absval = -n c2 1
else
absval = n; c3 1

Total Cost <= c1 + max(c2,c3)

CENG 213 Data Structures 21

 THE EXECUTION TIME OF ALGORITHMS (CONT.
Example: Simple Loop
Cost Times
i = 1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n
sum = sum + i; c5 n
}

Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*c5

➔ The time required for this algorithm is proportional to n

CENG 213 Data Structures 22

THE EXECUTION TIME OF ALGORITHMS (CONT.)
Example: Nested Loop
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8
➔ The time required for this algorithm is proportional to n2

CENG 213 Data Structures 23

 GENERAL RULES FOR
ESTIMATION
• Loops: The running time of a loop is at most the running time
of the statements inside of that loop times the number of
iterations.
• Nested Loops: Running time of a nested loop containing a
statement in the inner most loop is the running time of statement
multiplied by the product of the sized of all loops.
• Consecutive Statements: Just add the running times of those
consecutive statements.
• If/Else: Never more than the running time of the test plus the
larger of running times of S1 and S2.

CENG 213 Data Structures 24

 ALGORITHM GROWTH
RATES
• We measure an algorithm’s time requirement as a function of the
problem size.
– Problem size depends on the application: e.g. number of elements in a list for a
sorting algorithm, the number disks for towers of hanoi.
• So, for instance, we say that (if the problem size is n)
– Algorithm A requires 5*n2 time units to solve a problem of size n.
– Algorithm B requires 7*n time units to solve a problem of size n.
• The most important thing to learn is how quickly the algorithm’s
time requirement grows as a function of the problem size.
– Algorithm A requires time proportional to n2.
– Algorithm B requires time proportional to n.
• An algorithm’s proportional time requirement is known as
growth rate.
• We can compare the efficiency of two algorithms by comparing
their growth rates.
CENG 213 DataStructures 25
ALGORITHM GROWTH RATES
(CONT.)

Time requirements as a function

of the problem size n

CENG 213 DataStructures 26

 COMMON GROWTH
Function
RATES Growth Rate Name
c Constant
log N Logarithmic
log2N Log-squared
N Linear
N log N
N2 Quadratic
N3 Cubic
2N Exponential
CENG 213 Data Structures 27
FIGURE 6.1
RUNNING TIMES FOR
SMALL INPUTS

CENG 213 Data Structures 28

FIGURE 6.2
RUNNING TIMES FOR
MODERATE INPUTS

CENG 213 Data Structures 29

 ORDER-OF-MAGNITUDE ANALYSIS
AND BIG O
NOTATION
• If Algorithm A requires time proportional to f(n), Algorithm A is
said to be order f(n), and it is denoted as O(f(n)).
• The function f(n) is called the algorithm’s growth-rate
function.
• Since the capital O is used in the notation, this notation is called
the Big O notation.
• If Algorithm A requires time proportional to n2, it is O(n2).
• If Algorithm A requires time proportional to n, it is O(n).

CENG 213 DataStructures 30

 DEFINITION OF THE ORDER OF AN
ALGORITHM
Definition:
Algorithm A is order f(n) – denoted as O(f(n)) –
if constants k and n0 exist such that A requires
no more than k*f(n) time units to solve a problem
of size n  n0.

• The requirement of n  n0 in the definition of O(f(n)) formalizes

the notion of sufficiently large problems.
– In general, many values of k and n can satisfy this definition.

CENG 213 DataStructures 31

ORDER OF AN ALGORITHM
(CONT.)

CENG 213 DataStructures 32

A COMPARISON OF GROWTH-RATE
FUNCTIONS

CENG 213 DataStructures 33

A COMPARISON OF GROWTH-RATE
FUNCTIONS (CONT.)

CENG 213 DataStructures 34

 GROWTH-RATE
FUNCTIONS
O(1) Time requirement is constant, and it is independent of the problem’s size.
O(log2n) Time requirement for a logarithmic algorithm increases increases slowly
as the problem size increases.
O(n) Time requirement for a linear algorithm increases directly with the size
of the problem.
O(n*log2n) Time requirement for a n*log2n algorithm increases more rapidly than
a linear algorithm.
O(n2) Time requirement for a quadratic algorithm increases rapidly with the
size of the problem.
O(n3) Time requirement for a cubic algorithm increases more rapidly with the
size of the problem than the time requirement for a quadratic algorithm.
O(2n) As the size of the problem increases, the time requirement for an
exponential algorithm increases too rapidly to be practical.

CENG 213 DataStructures 35

 GROWTH-RATE
FUNCTIONS
• If an algorithm takes 1 second to run with the problem size
8, what is the time requirement (approximately) for that algorithm
with the problem size 16?
• If its order is:
O(1) ➔ T(n) = 1 second
O(log2n) ➔ T(n) = (1*log216) / log28 = 4/3 seconds
O(n) ➔ T(n) = (1*16) / 8 = 2 seconds
O(n*log2n) ➔ T(n) = (1*16*log216) / 8*log28 = 8/3 seconds
O(n2) ➔ T(n) = (1*162) / 82 = 4 seconds
O(n3) ➔ T(n) = (1*163) / 83 = 8 seconds
O(2n) ➔ T(n) = (1*216) / 28 = 28 seconds = 256 seconds

CENG 213 DataStructures 36

 PROPERTIES OF GROWTH-RATE
FUNCTIONS
1. We can ignore low-order terms in an algorithm’s growth-rate
function.
– If an algorithm is O(n3+4n2+3n), it is also O(n3).
– We only use the higher-order term as algorithm’s growth-rate function.

2. We can ignore a multiplicative constant in the higher-order term

of an algorithm’s growth-rate function.
– If an algorithm is O(5n3), it is also O(n3).

3. O(f(n)) + O(g(n)) = O(f(n)+g(n))

– We can combine growth-rate functions.
– If an algorithm is O(n3) + O(4n), it is also O(n3 +4n2) ➔ So, it is O(n3).
– Similar rules hold for multiplication.

CENG 213 DataStructures 37

 GROWTH-RATE FUNCTIONS –
Cost Times
i =
EXAMPLE1
1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n
sum = sum + i; c5 n
}

T(n) = c1 + c2 + (n+1)*c3 + n*c4 + n*c5

= (c3+c4+c5)*n + (c1+c2+c3)
= a*n + b
➔ So, the growth-rate function for this algorithm is O(n)

CENG 213 DataStructures 38

 GROWTH-RATE FUNCTIONS –
Cost Times
i=1; EXAMPLE2 c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
T(n) = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8
= (c5+c6+c7)*n2 + (c3+c4+c5+c8)*n + (c1+c2+c3)
= a*n2 + b*n + c
➔ So, the growth-rate function for this algorithm is O(n2)
CENG 213 DataStructures 39