Analysis of

Algorithms

26-Aug-22
 Time and • To analyze an algorithm means:
space • developing a formula for predicting how fast
an algorithm is, based on the size of the
input (time complexity)
• developing a formula for predicting how
much memory an algorithm requires, based
on the size of the input (space complexity)
2
• Usually, time is our biggest concern
• Most algorithms require a fixed amount of
space
 • In a higher-level language (such as Java), we
do not know how long each operation takes
• Which is faster, x < 10 or x <= 9 ?
• We don’t know exactly what the compiler does
with this
• The compiler probably optimizes the test
Higher-level anyway (replacing the slower version with the
faster one)
languages • In a higher-level language, we cannot do an
exact analysis
• Our timing analyses will use major
oversimplifications
• Nevertheless, we can get some very useful
results

3
 • A technique used to characterize the execution
behavior of algorithms in a manner independent
of a particular platform, compiler, or language.

Algorithmic • Abstract away the minor variations and describe

the performance of algorithms in a more
Analysis theoretical, processor independent fashion.

• A method to compare speed of algorithms

against one another depending on size of the
input.

4
 What does “size of the input” mean?

If we are searching an array, the “size” of the input could be the size of the array

If we are merging two arrays, the “size” could be the sum of the two array sizes

If we are computing the nth Fibonacci number, or the nth factorial, the “size” is n

We choose the “size” to be the parameter that most influences the actual time/space
required
• Sometimes we need two or more parameters

5
Operations to count
• In computing time complexity, one good approach is to count
characteristic operations
• What a “characteristic operation” is depends on the particular problem
• If searching, it might be comparing two values
• If sorting an array, it might be:
• comparing two values
• swapping the contents of two array locations
• both of the above
• Sometimes we just look at how many times the innermost loop is executed

6
Exact values
• It is sometimes possible, in assembly language, to compute exact
time and space requirements
• We know exactly how many bytes and how many cycles each machine
instruction takes
• For a problem with a known sequence of steps (factorial, Fibonacci), we can
determine how many instructions of each type are required
• However, often the exact sequence of steps cannot be known in
advance
• The steps required to sort an array depend on the actual numbers in the array
(which we do not know in advance)

7
 Resource (running time, memory, etc.) usage
Average, in
best, and • Best Case: at least
• Average Case: at average
worst cases
• Worst Case: at most

8
9
 • Constant time means there is some constant
k such that this operation always takes k
nanoseconds
• A Java statement takes constant time if:
Constant • It does not include a loop
time • It does not include calling a method whose time
is unknown or is not a constant
complexity • If a statement involves a choice (if or
switch) among operations, each of which
takes constant time, we consider the
statement to take constant time

10
 Asymptotic Notations are languages that allow
us to analyse an algorithm’s running time by
identifying its behaviour as the input size for
the algorithm increases.
Types of Asymptotic Notations (Algorithm
Asymptotic Growth rate):
Notations • Big O
• Big Ω
• Theta
• Small o
• Small ℧

11
• The most common method and notation for
discussing the execution time of algorithms
is "Big O”.
• Operations involving constant time are said
to require order 1 steps, that is O(1) steps.
• For an alphabetized dictionary the algorithm
Big O -- Big
requires O(log N) steps to search for an item. Oh
• For an unsorted list, the linear search
algorithm requires O(N) steps.

12
 • Normally a loop operates on a data set
which can vary in size.
int minimum(int values[], int n)
{
int minValue = values[0];
Loops for(int i = 1; i < n; i++)
if(values[i] < minValue)
That Work minValue = values[i];
return minValue;
}
on a Data Set
• The number of executions of the loop
depends on the number of elements in the
array .
• The run time is O(n).
 boolean member(int x, int[] a) {
int n = a.length;
for (int i = 0; i < n; i++) {
if (x == a[i])
return true;
}
return false;
}
Linear Search • If x is not in a, the loop executes n times,
where n is number of elements in the array
• This is the worst case
• If x is in a, the loop executes n/2 times on
average
• Either way, it is order of n, O(n).

14
 • for (i = 0, j = 1; i < n; i++) {
j = j * i;
}
• This loop takes time k*n + c, for some
constants k and c
Linear time k : How long it takes to go through the loop
once
algorithm (the time for j = j * i, plus loop overhead)
n : The number of times through the loop
(we can use this as the “size” of the problem)
c : The time it takes to initialize the loop
• The total time k*n + c is linear in n
• Execution time O(n).

15
 • Suppose we have two algorithms to solve a task:
• Algorithm A takes 5000 time units
• Algorithm B takes 100*n time units
• Which is better?
Constant time is • Clearly, algorithm B is better if our problem
size is small, that is, if n < 50
(usually) better • Algorithm A is better for larger problems, with
n > 50
than linear time • So, B is better on small problems that are
quick anyway.
• But A is better for large problems, where it
matters more.
• We usually care most about very large problems.

16
Nested Loops
void bubbleSort(int data[], int n)
{
for(int i = n - 1; i > 0; i--)
for(int j = 0; j < i; j++)
if(data[j] > data[j+1])
{
double temp = data[j];
data[j] = data[j + 1];
data[j + 1] = temp;
}
}
• Number of executions?

17
18
 • Suppose you have two sets, represented as unsorted arrays:
• int[] sub = { 7, 1, 3, 2, 5 };
int[] super = { 8, 4, 7, 1, 2, 3, 9 };
• and you want to test whether every element of the first set
(sub) also occurs in the second set (super)
• If there are m elements in first set, and n elements in the
The array subset second set, an algorithm will select one element from first
set and go through n elements of the second set.
problem • Thus for m elements of first set, total number of operations
are going to be order of m*n.
• We can say that the array subset problem has time
complexity of O(mn), along with assorted constants.
• If m and n are similar in value, this is roughly quadratic
time complexity.

19
 • Forget the constants!

• An added constant, f(n)+c, becomes less

and less important as n gets larger
What about
the • Suppose an algorithm takes
12n3+4n2+15 steps and another takes
constants? 24n2+8n+35 how do we compare the
two algorithms?

• We need to simplify the formulae for a quick

comparison.
20
 • Consider a jungle with number of animals in it.
• Let there be n elephants, m tigers, p foxes, k
squirrels, and g ants in the jungle. Is there a simple
formula to represent the net weight of the animals?
• Consider 3 elephants, 5 tigers, 10 foxes, 200
Simplifying squirrels, and 10,000 ants. The net weight is going
to be governed mainly by the weight of the
the formula elephants.
• In another jungle with no elephants, the weight of
the tigers would be most prominent.
• Thus, it makes sense to keep only the highest order
terms of the formula for calculating time complexity.

21
 • Throwing out the constants is one of two
things we do in analysis of algorithms

• By throwing out constants, we simplify 12n2 +

35 to just n2
Simplifying
the formulae • Our timing formula is a polynomial, and may
have terms of various orders (constant,
linear, quadratic, cubic, etc.)

• We usually discard all but the highest-order term

We simplify n2 + 3n + 5 to just n2

22
 • When we have a polynomial that describes
the time requirements of an algorithm, we
simplify it by:
• Throwing out all but the highest-order term
• Throwing out all the constants
Big O
• If an algorithm takes 12n3+4n2+8n+35
notation time, we simplify this formula to just n3

• We say the algorithm requires O(n3) time

• We call this Big O notation

23
Formal Definition of Big O-notation
Let f(n) denote the expression denoting the number of operations for an
algorithm.
If there are positive constants c and n0,
such that for all n  n0,
f(n)  cg(n)
then we say that f(n) is of order O(g(n))

This covers set of all functions whose rate of growth is the same as or lower than
that of g(n).
24
Big O-notation

c g(n) is an asymptotic upper bound for f(n).

Comp 122
More on Big - O
• There is a point n0 such that for all values of n that
are past this point, f(n) is bounded by some
multiple of g(n).

• Thus, if f(n) of the algorithm is O( n2 ) then, ignoring

constants, at some point we can bound the running
time by a quadratic function of the input size.

• Given a linear algorithm, it is technically correct to

say the running time is O(n2). O(n) is a more precise
answer as to the Big O bound of a linear algorithm.

26
 Can we justify Big O notation?

Big O notation is a huge simplification; can we justify it?

• It only makes sense for large problem sizes
• For sufficiently large problem sizes, the highest-order term swamps all the
rest!

Consider R = x2 + 3x + 5 as x varies:
•x=0 x2 = 0 3n = 0 5=5 R=5
• x = 10 x2 = 100 3x = 30 5=5 R = 135
• x = 100 x2 = 10000 3x = 300 5=5 R = 10,305
• x = 1000 x2 = 1000000 3x = 3000 5=5 R = 1,003,005
• x = 10,000 x2 = 108 3x = 3*104 5=5 R = 100,030,005
• x = 100,000 x2 = 1010 3x = 3*105 5=5 R = 10,000,300,005 27
 • 3n3 = O(n3)
• 3n3 + 8 = O(n3)
• 8n2 + 10n * log(n) + 100n + 1020 = O(n2)
Big O • 3 log(n) + 2n1/2 = O(n1/2)
Examples • 25 = O(1)
• TlinearSearch(n) = O(n)
• TbinarySearch(n) = O(log(n))

28
y = x2 + 3x + 5, for x=1..20

29
 • f(n) is the actual growth rate of the algorithm.

What it all • g(n) is the function that bounds the growth rate.
• may be upper or lower bound

means • g(n) may not equal f(n).

• constants and lesser terms ignored because it
is a bounding function

30
The function log n

31
k represents log n

32
log n when n is very small

33
log n
graph

34
Common time complexities
BETTER • O(1) constant time
• O(log n) log time
• O(n) linear time
• O(n log n) log linear time
• O(n2) quadratic time
• O(n3) cubic time
• O(2n) exponential time
WORSE

35
Ranking of Algorithmic Behaviors
Function Common Name
N! factorial
2N Exponential
N d, d > 3 Polynomial
N3 Cubic
N2 Quadratic
N N
N log N
N Linear
N Root - n
log N Logarithmic
1 Constant

36
Plot of Time
Complexities
 Running Times

• Assume N = 100,000 and a computer with processor

speed of 1,000,000 operations per second
Function Running Time
2N over 100 years
N3 31.7 years
N2 2.8 hours
N N 31.6 seconds
N log N 1.2 seconds
N 0.1 seconds
N 3.2 x 10-4 seconds
log N 1.2 x 10-5 seconds

38
Big Omega (Ω)
• Use for Best Case/ floor growth rate/ lower bound.
• Let f(n) denote the expression denoting the number of operations for
an algorithm.
• If there are positive constants c>0 and n0>0,
such that for all n > n0,
f(n) ≥ cg(n)
then we say that f(n) is of order Ω(g(n))

39
Big Omega (Ω)

40
Theta (θ)
• Theta is used for Tight Bound.
• Let f(n) denote the expression denoting the number of
operations for an algorithm.
• If there are positive constants c1>0 and c2>0 and n0>0,
such that for all n ≥ n0,
c1g(n) ≤ f(n) ≤ c2g(n)
then we say that f(n) is of order θ(g(n))
• It implies that f(n) = O(g(n)) and f(n)=Ω(g(n))

41
Theta (θ)
Small Oh (o)
• Use for upper bound but not tight.
• Let f(n) denote the expression denoting the number of operations for
an algorithm.
• If there are positive constants c>0 and n0>0,
such that for all n > n0,
f(n) < cg(n)
then we say that f(n) is of order o(g(n))

43
Small Omega (℧)
• Use for lower bound but not tight.
• Let f(n) denote the expression denoting the number of operations for
an algorithm.
• If there are positive constants c>0 and n0>0,
such that for all n > n0,
f(n) > cg(n)
then we say that f(n) is of order ℧(g(n))

44
Notations

45
Bubble, Selection and Insertion Sort

46
 Algorithm Best Average Worst

Selection
Ω(n2) θ(n2) O(n2)
Time Sort

Complexity Bubble Sort Ω(n) θ(n2) O(n2)

Insertion
Ω(n) θ(n2) O(n2)
Sort

47
Thank you
gauravsingal789@gmail.com
gaurav.singal@nsut.ac.in
www.gauravsingal.in

LinkedIn: in/gauravsingal789/