lOMoARcPSD|34251814

Asymptotic notation and Basic Efficiency Classes

Analysis and Design of Algorithms (Jain (Deemed-to-be University))

Studocu is not sponsored or endorsed by any college or university

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)
 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

Asymptotic notation and Basic Efficiency Classes

Asymptotic Notation:

Asymptotic notations are the mathematical notations used to describe the running time of an
algorithm when the input tends towards a particular value or a limiting value. Execution time
of an algorithm depends on the instruction set, processor speed, disk I/O speed, etc. Hence, we
estimate the efficiency of an algorithm asymptotically.
Let t(n) and g(n) can be any nonnegative functions defined on the set of natural numbers. The
algorithm’s running time t(n) usually indicated by its basic operation count C(n), and g(n),
some simple function to compare with the count.
Different types of asymptotic notations are used to represent the complexity of an algorithm.
Following asymptotic notations are used to calculate the running time complexity of an
algorithm.
 O − Big Oh
 Ω − Big omega
 θ − Big theta

Big Oh (O): Asymptotic Upper Bound

‘O’ (Big Oh) is the most commonly used notation. A function f(n) can be represented is the
order of g(n) that is O(g(n)), if there exists a value of positive integer n as n0 and a positive
constant c such that −

f(n)⩽ c.g(n) for n>n0 in all case

Hence, function g(n) is an upper bound for function f(n), as g(n) grows faster than f(n).
Example
Let us consider a given function, f(n)=4.n3+10. 4.n2+5.n+1
Considering g(n)=n3,
f(n)⩽ 5.g(n) for all the values of n>2

Hence, the complexity of f(n) can be represented as O(g(n)), i.e. O(4.n3)

V Sem, Dept of BCA 21

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

Omega (Ω): Asymptotic Lower Bound

We say that f(n)=Ω(g(n)) when there exists constant c that f(n)⩾ c.g(n) for all
sufficiently large value of n. Here n is a positive integer. It means function g is a lower bound
for function f; after a certain value of n, f will never go below g.

Example
Let us consider a given function, f(n)= 4.n3+10. 4.n2+5.n+1
3
Considering g(n)= , f(n)⩾ 4.g(n) for all the values of n>0

Hence, the complexity of f(n) can be represented as Ω(g(n)), i.e. Ω( 3)

Theta (θ): Asymptotic Tight Bound

We say that f(n)=θ(g(n)) when there exist constants c1 and c2 that 1.g(n)⩽ f(n)⩽ 2.g(n)

for all sufficiently large value of n. Here n is a positive integer. This means function g is a
tight bound for function f.

Example
Let us consider a given function, f(n)= 4.n3+10. 4.n2+5.n+1

3
Considering g(n)= , 4.g(n)⩽ f(n)⩽ 5.g(n) for all the large values of n.

Hence, the complexity of f(n) can be represented as θ(g(n)), i.e. θ(n3).

V Sem, Dept of BCA 22

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

Asymptotic Analysis

Solving Recurrence Equations

A recurrence is an equation or inequality that describes a function in terms of its value on

smaller inputs. Recurrences are generally used in divide-and-conquer paradigm.
A recurrence relation can be solved using the following methods −
 Substitution Method − In this method, we guess a bound and using mathematical
induction we prove that our assumption was correct.
 Recursion Tree Method − In this method, a recurrence tree is formed where each
node represents the cost.
 Master’s Theorem − This is another important technique to find the complexity of a
recurrence relation.

General Plan for Analyzing the Time Efficiency of Recursive Algorithms

1. Decide on a parameter (or parameters) indicating an input’s size.

2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation is executed can vary on
different inputs of the same size; if it can, the worst-case, average-case, and best-case
efficiencies must be investigated separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number of
times the basic operation is executed.
5. Solve the recurrence or, at least, ascertain the order of growth of its solution.

V Sem, Dept of BCA 23

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

EXAMPLE 1: Compute the factorial function F(n) = n! for an arbitrary non-negative integer
n. Since n! = 1•. . . . • (n − 1) • n = (n − 1)! • n, for n ≥ 1 and 0!= 1 by definition, we can
compute F(n) = F(n − 1) • n with the following recursive algorithm.

ALGORITHM Factorial(n)
//Computes n! Recursively
//Input: A nonnegative integer n
//Output: The value of n!
if n = 0 return 1
else return Factorial(n − 1) * n

Algorithm analysis

 For simplicity, we consider n itself as an indicator of this algorithm’s input size.

i.e. 1.
 The basic operation of the algorithm is multiplication, whose number of
executions we denote M(n). Since the function F(n) is computed
according to the formula F(n) = F(n
−1)*n for n > 0.
 The number of multiplications M(n) needed to compute it must satisfy the
equality

 M(n − 1) multiplications are spent to compute F(n − 1), and one

more multiplication is needed to multiply the result by n.

Recurrence relations
The last equation defines the sequence M(n) that we need to find. This equation

V Sem, Dept of BCA 24

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

defines M(n) not explicitly, i.e., as a function of n, but implicitly as a function

of its value at another point, namely n − 1. Such equations are called recurrence
relations or recurrences.

Solve the recurrence relation M(n)=M(n-1)+1 , i.e., to find an explicit formula

for M(n) in terms of n only. To determine a solution uniquely, we need an initial
condition that tells us the value with which the sequence starts. We can obtain
this value by inspecting the condition that makes the algorithm stop its recursive
calls:
if n = 0 return 1.

This tells us two things. First, since the calls stop when n = 0, the smallest value
of n for which this algorithm is executed and hence M(n) defined is 0. Second, by
inspecting the pseudo code’s exiting line, we can see that when n = 0, the
algorithm performs no multiplications.

V Sem, Dept of BCA 25

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

General Plan for Analyzing the Time Efficiency of Non-recursive Algorithms

1. Decide on a parameter (or parameters) indicating an input’s size.

2. Identify the algorithm’s basic operation (in the innermost loop).

3. Check whether the number of times the basic operation is executed

depends only on the size of an input. If it also depends on some

additional property, the worst-case, average-case, and, if necessary,
best-case efficiencies have to be investigated separately.
4. Set up a sum expressing the number of times the algorithm’s basic

operation is executed.
5. Using standard formulas and rules of sum manipulation either find a

closed form formula for the count or at the least, establish its order
of growth.

V Sem, Dept of BCA 26

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

EXAMPLE 1: Consider the problem of finding the value of the largest element in a
list of n numbers. Assume that the list is implemented as an array for simplicity.

ALGORITHM MaxElement(A[0..n − 1])

//Determines the value of the largest element in a given array
//Input: An array A[0..n − 1] of real numbers
//Output: The value of the largest element in
A maxval ←A[0]
for i ←1 to n − 1 do
if A[i]>maxval
maxval←A[i]
End for
return maxval

Algorithm Analysis
 The measure of an input’s size here is the number of elements in the array, i.e.,
n.
 There are two operations in the for loop’s body: o The comparison A[i]>
maxval and o The assignment maxval←A[i].
 The comparison operation is considered as the algorithm’s basic operation,
because the comparison is executed on each repetition of the loop and not
the assignment.
 The number of comparisons will be the same for all arrays of size n;
therefore, there is no need to distinguish among the worst, average, and
best cases here.
 Let C(n) denotes the number of times this comparison is executed. The
algorithm makes one comparison on each execution of the loop, which is
repeated for each value of the loop’s variable i within the bounds 1 and n

− 1, inclusive. Therefore, the sum for C(n) is calculated as follows:

V Sem, Dept of BCA 27

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

EXAMPLE 2:
Consider the element uniqueness problem: check whether all the Elements in a
given array of n elements are distinct.

ALGORITHM UniqueElements(A[0..n − 1])

//Determines whether all the elements in a given array are distinct
//Input: An array A[0..n − 1]
//Output: Returns “true” if all the elements in A are distinct and
“false” otherwise
for i ←0 to n − 2 do
for j ←i + 1 to n − 1 do
if A[i]= A[j ]
return 0;
end for
end for

Algorithm Analysis:
 The measure of an input’s size here is the number of elements in the array, i.e.,
n.
 There are two operations in the for loop’s body: o The comparison A[i]>
maxval and o The assignment maxval←A[i].
 The comparison operation is considered as the algorithm’s basic operation,
because the comparison is executed on each repetition of the loop and not
the assignment.
 The number of comparisons will be the same for all arrays of size n;
therefore, there is no need to distinguish among the worst, average, and
best cases here.
 Let C(n) denotes the number of times this comparison is executed. The
algorithm makes one comparison on each execution of the loop, which is
repeated for each value of the loop’s variable i within the bounds 1 and n

− 1, inclusive. Therefore, the sum for C(n) is calculated as follows:

V Sem, Dept of BCA 28

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

Algorithm Analysis:
The natural measure of the input’s size here is again n (the number of elements in the
array).
Since the innermost loop contains a single operation (the comparison of two elements),
we should consider it as the algorithm’s basic operation.

The number of element comparisons depends not only on n but also on whether there
are equal elements in the array and, if there are, which array positions they occupy. We
will limit our investigation to the worst case only.
One comparison is made for each repetition of the innermost loop, i.e., for each value
of the loop variable j between its limits i + 1 and n − 1; this is repeated for each value of
the outer loop, i.e., for each value of the loop variable i between its limits 0 and n −2

Algorithm Design :
The important aspects of algorithm design include creating an efficient algorithm to
solve a problem in an efficient way using minimum time and space.
To solve a problem, different approaches can be followed. Some of them can be efficient
with respect to time consumption, whereas other approaches may be memory efficient.
However, one has to keep in mind that both time consumption and memory usage
cannot be optimized simultaneously. If we require an algorithm to run in lesser time, we
have to invest in more memory and if we require an algorithm to run with lesser memory,
we need to have more time.

V Sem, Dept of BCA 29

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)

 lOMoARcPSD|34251814

16BCA5D11 ANALYSIS AND DESIGN OF ALGORITHMS

Problem Development Steps

The following steps are involved in solving computational problems.
Problem definition
Development of a model
Specification of an Algorithm
Designing an Algorithm
Checking the correctness of an Algorithm
Analysis of an Algorithm
Implementation of an Algorithm
Program testing
Documentation

V Sem, Dept of BCA 30

Downloaded by Sudhir Varma (sudhirvarma406@gmail.com)