Chapter Two

Algorithm Analysis
What is an Algorithm?
• An algorithm is a process or a set of rules required to perform calculations
or some other problem-solving operations especially by a computer.

• The formal definition of an algorithm is that it contains the finite set of

instructions which are being carried in a specific order to perform the
specific task.

• It is not the complete program or code; it is just a solution (logic) of a

problem, which can be represented either as an informal description
using a Flowchart or Pseudocode.
Characteristics of an Algorithm
• The following are the characteristics of an algorithm:
• Input: An algorithm has some input values. We can pass 0 or some input value to
an algorithm.
• Output: We will get 1 or more output at the end of an algorithm.
• Unambiguity: An algorithm should be unambiguous which means that the
instructions in an algorithm should be clear and simple.
• Finiteness: An algorithm should have finiteness. Here, finiteness means that the
algorithm should contain a limited number of instructions, i.e., the instructions
should be countable.
• Effectiveness: An algorithm should be effective as each instruction in an algorithm
affects the overall process.
• Language independent: An algorithm must be language-independent so that the
instructions in an algorithm can be implemented in any of the languages with the
same output.
 Dataflow of an Algorithm
• Problem: A problem can be a real-world problem or any instance from the
real-world problem for which we need to create a program or the set of
instructions. The set of instructions is known as an algorithm.
• Algorithm: An algorithm will be designed for a problem which is a step by
step procedure.
• Input: After designing an algorithm, the required and the desired inputs are
provided to the algorithm.
• Processing unit: The input will be given to the processing unit, and the
processing unit will produce the desired output.
• Output: The output is the outcome or the result of the program.
 Why do we need Algorithms?
• We need algorithms because of the following reasons:

• Scalability: It helps us to understand the scalability. When we have a big

real-world problem, we need to scale it down into small-small steps to
easily analyze the problem.

• Performance: The real-world is not easily broken down into smaller steps.
If the problem can be easily broken into smaller steps means that the
problem is feasible.
Algorithm - a real-world example
• Suppose we want to make a lemon juice, so following are the steps required to make a lemon
juice:
• Step 1: First, we will cut the lemon into half.
• Step 2: Squeeze the lemon as much you can and take out its juice in a container.
• Step 3: Add two tablespoon sugar in it.
• Step 4: Stir the container until the sugar gets dissolved.
• Step 5: When sugar gets dissolved, add some water and ice in it.
• Step 6: Store the juice in a fridge for 5 to minutes.
• Step 7: Now, it's ready to drink
Factors of an Algorithm
• The following are the factors that we need to consider for designing an algorithm:
• Modularity: If any problem is given and we can break that problem into small-small modules or small-
small steps, which is a basic definition of an algorithm, it means that this feature has been perfectly
designed for the algorithm.

• Correctness: The correctness of an algorithm is defined as when the given inputs produce the desired
output, which means that the algorithm has been designed algorithm. The analysis of an algorithm has
been done correctly.

• Maintainability: Here, maintainability means that the algorithm should be designed in a very simple
structured way so that when we redefine the algorithm, no major change will be done in the algorithm.

• Functionality: It considers various logical steps to solve the real-world problem.

Factors of an Algorithm

• The following are the factors that we need to consider for designing an
algorithm:
• Robustness: Robustness means that how an algorithm can clearly define our problem.

• User-friendly: If the algorithm is not user-friendly, then the designer will not be able to

explain it to the programmer.

• Simplicity: If the algorithm is simple then it is easy to understand.

• Extensibility: If any other algorithm designer or programmer wants to use your algorithm

then it should be extensible.

Importance of Algorithms
• Theoretical importance: When any real-world problem is given to us
and we break the problem into small-small modules. To break down
the problem, we should know all the theoretical aspects.

• Practical importance: As we know that theory cannot be completed

without the practical implementation. So, the importance of
algorithm can be considered as both theoretical and practical.
Issues of Algorithms
• The following are the issues that come while designing an algorithm:

• How to design algorithms: As we know that an algorithm is a step-by-

step procedure so we must follow some steps to design an algorithm.

• How to analyze algorithm efficiency

•
Classification of Data Structures Algorithms
• The major categories of algorithms are given below:
• Sort: Algorithm developed for sorting the items in a certain order.

• Search: Algorithm developed for searching the items inside a data structure.

• Delete: Algorithm developed for deleting the existing element from the data structure.

• Insert: Algorithm developed for inserting an item inside a data structure.

• Update: Algorithm developed for updating the existing element inside a data structure.
Algorithm Analysis
• The algorithm can be analyzed in two levels, i.e., first is before
creating the algorithm, and second is after creating the algorithm.
• The following are the two analysis of an algorithm:
1. Priori Analysis: Here, priori analysis is the theoretical analysis of an
algorithm which is done before implementing the algorithm. Various factors
can be considered before implementing the algorithm like processor speed,
which has no effect on the implementation part.

2. Posterior Analysis: Here, posterior analysis is a practical analysis of an

algorithm. The practical analysis is achieved by implementing the algorithm
using any programming language. This analysis basically evaluate that how
much running time and space taken by the algorithm.
Algorithm Complexity
• The performance of the algorithm  Let's understand the time
can be measured in two factors: complexity through an example.
1. Time complexity: • sum=0;
 The time complexity of an algorithm • // Suppose we have to calculate the s
is the amount of time required to um of n numbers.
complete the execution. • for i=1 to n
 The time complexity of an algorithm • sum=sum+i;
is denoted by the big O notation. • // when the loop ends then sum holds
 Here, big O notation is the asymptotic the sum of the n numbers
notation to represent the time • return sum;
complexity.
 The time complexity is mainly  Time complexity of the loop
calculated by counting the number of statement will be atleast n, and if
steps to finish the execution. the value of n increases, then the
time complexity also increases.
Algorithm Complexity
2. Space complexity: An algorithm's space complexity is the amount of space required to
solve a problem and produce an output. Similar to the time complexity, space
complexity is also expressed in big O notation.
• For an algorithm, the space is required for the following purposes:
• To store program instructions
• To store constant values
• To store variable values
• To track the function calls, jumping statements, etc.
• Auxiliary space: is the extra space required by the algorithm, excluding the input size.
• The space complexity considers both the spaces, i.e., auxiliary space, and space used by
the input. So,
• Space complexity = Auxiliary space + Input size.
Types of Algorithms
• The following are the types of algorithm: Search Algorithm and Sort
Algorithm
1. Search Algorithm
• On each day, we search for something in our day to day life.
• Similarly, with the case of computer, huge data is stored in a computer that
whenever the user asks for any data then the computer searches for that data in
the memory and provides that data to the user.
• There are mainly two techniques available to search the data in an array:
a) Linear search
b) Binary search
Types off Algorithm : Search Algorithm
1. Linear Search
• Linear search is a very simple algorithm that starts searching for an element or a value
from the beginning of an array until the required element is not found.
• It compares the element to be searched with all the elements in an array, if the match is
found, then it returns the index of the element else it returns -1.
• This algorithm can be implemented on the unsorted list.
2. Binary Search
• A Binary algorithm is the simplest algorithm that searches the element very quickly.
• It is used to search the element from the sorted list.
• The elements must be stored in sequential order or the sorted manner to implement the
binary algorithm.
• Binary search cannot be implemented if the elements are stored in a random manner.
• It is used to find the middle element of the list.
Types of Linear Data Structures
2. Sorting Algorithms
• Sorting algorithms are used to rearrange the elements in an array or a given
data structure either in an ascending or descending order.
• The comparison operator decides the new order of the elements.
• Why do we need a sorting algorithm?
• An efficient sorting algorithm is required for optimizing the efficiency of other
algorithms like binary search algorithm as a binary search algorithm requires an
array to be sorted in a particular order, mainly in ascending order.
• It produces information in a sorted order, which is a human-readable format.
• Searching a particular element in a sorted list is faster than the unsorted list.
Asymptotic Analysis
• Data structure is a way of organizing the data efficiently and that efficiency is
measured either in terms of time or space.
• So, the ideal data structure is a structure that occupies the least possible time to
perform all its operation and the memory space.
• Our focus would be on finding the time complexity rather than space complexity,
and by finding the time complexity, we can decide which data structure is the best
for an algorithm.
• The main question arises in our mind that on what basis should we compare the
time complexity of data structures?.
• The time complexity can be compared based on operations performed on them.
How to find the Time Complexity or running time for
performing the operations?
• The measuring of the actual running time is not practical at all.
• The running time to perform any operation depends on the size of the
input.
• Suppose we have an array of five elements, and we want to add a new
element at the beginning of the array.
• To achieve this, we need to shift each element towards right, and suppose
each element takes one unit of time. There are five elements, so five units of
time would be taken.
• Suppose there are 1000 elements in an array, then it takes 1000 units of time
to shift. It concludes that time complexity depends upon the input size.
• Therefore, if the input size is n, then f(n) is a function of n that
denotes the time complexity.
How to calculate f(n)?
• Calculating the value of f(n) for smaller programs is easy but for bigger programs,
it's not that easy.
• We can compare the data structures by comparing their f(n) values.
• We will find the growth rate of f(n) because there might be a possibility that one
data structure for a smaller input size is better than the other one but not for the
larger sizes.
• Now, how to find f(n)
How to calculate f(n)?
• Let's look at a simple example, f(n) = 5n2 + 6n + 12, where n is the number of instructions executed, and it
depends on the size of the input.

• From the above calculation, it is observed that most of the time is taken by 12.

• But, we have to find the growth rate of f(n), we cannot say that the maximum amount of time is taken by 12.
How to calculate f(n)?
• Let's assume the different values of n to find the growth rate of f(n).

• Therefore, f(n) = 5n2

 Time complexity types
• Usually, the time required by an algorithm comes under three types:
• Worst case: defines the input for which the algorithm takes a huge time.
• Average case: takes average time for the program execution.
• Best case: defines the input for which the algorithm takes the lowest
time
Asymptotic Notations
• The commonly used asymptotic notations used for calculating the
running time complexity of an algorithm is given below:
• Big oh Notation (O)

• Omega Notation (Ω)

• Theta Notation (θ)

Big oh Notation (O)
• Big O notation is an asymptotic notation that
measures the performance of an algorithm by
simply providing the order of growth of the
function.
• This notation provides an upper bound on a
function which ensures that the function never
grows faster than the upper bound.
• So, it gives the least upper bound on a function so
that the function never grows faster than this
upper bound.
• It is the formal way to express the upper boundary
of an algorithm running time. It measures the
worst case of time complexity or the algorithm's
longest amount of time to complete its operation.
It is represented as shown below:
Big oh Notation (O) - Example
• f(n)=2n+3 , g(n)=n • We can say that f(n) grows linearly.
• Now, we have to find Is f(n)=O(g(n))? • Therefore, it concludes that c.g(n) is
• To check f(n)=O(g(n)), it must satisfy the given the upper bound of the f(n).
condition: f(n)<=c.g(n) • It can be represented graphically as:
• First, we will replace f(n) by 2n+3 and g(n) by n :
2n+3 <= c.n
• Let's assume c=5, n=1 then, 2*1+3<=5*1 -> 5<=5
• For n=1, the above condition is true.
• If n=2, 2*2+3<=5*2, 7<=10, For n=2, the above
condition is true.
• Since for some constants c and for some constants
n, it will always satisfy 2n+3<=c.n.
• Therefore, f(n) is big oh of g(n)
Big oh Notation (O)
• If f(n) and g(n) are the two functions defined for positive integers,
then
• f(n) = O(g(n)) as f(n) is big oh of g(n) or f(n) is on the order of g(n)) if there
exists constants c and no such that:
• f(n)≤c.g(n) for all n≥no
• This implies that f(n) does not grow faster than g(n), or g(n) is an
upper bound on the function f(n).
• In this case, we are calculating the growth rate of the function which
eventually calculates the worst time complexity of a function, i.e.,
how worst an algorithm can perform.
Omega Notation (Ω)
• It basically describes the best-case scenario which is opposite to the big o
notation.
• It is the formal way to represent the lower bound of an algorithm's running
time.
• It measures the best amount of time an algorithm can possibly take to
complete or the best-case time complexity.
• It determines what is the fastest time that an algorithm can run.
• If we required that an algorithm takes at least certain amount of time
without using an upper bound, we use big- Ω notation i.e. the Greek letter
"omega".
• It is used to bound the growth of running time for large input size.
Omega Notation (Ω) - Example
• If f(n) = 2n+3, g(n) = n, Is f(n)= Ω (g(n))?
• It must satisfy the condition:
• f(n)>=c.g(n)
• To check the above condition, we first replace
f(n) by 2n+3 and g(n) by n.
• 2n+3>=c*n
• Suppose c=1, 2n+3>=n (This equation will be
true for any value of n starting from 1).
• Therefore, it is proved that g(n) is big omega of
2n+3 function.
Theta Notation (θ)
• The theta notation mainly describes the average case scenarios.
• It represents the realistic time complexity of an algorithm.
• Every time, an algorithm does not perform worst or best, in real-
world problems, algorithms mainly fluctuate between the worst-case
and best-case, and this gives us the average case of the algorithm.
• Big theta is mainly used when the value of worst-case and the best-
case is same.
• It is the formal way to express both the upper bound and lower
bound of an algorithm running time.
Theta Notation (θ)
• Let's understand the big theta notation The graphical representation of theta
mathematically: notation is given below:
• Let f(n) and g(n) be the functions of n where n
is the steps required to execute the program
then:
f(n)= θg(n)
• The above condition is satisfied only if when
c1.g(n)<=f(n)<=c2.g(n)
where the function is bounded by two limits,
i.e., upper and lower limit, and f(n) comes in
between.
The condition f(n)= θg(n) will be true if and only
if c1.g(n) is less than or equal to f(n) and c2.g(n)
is greater than or equal to f(n).
Theta Notation (θ) - Example
• Let's consider the same example where f(n)=2n+3 and g(n)=n
• c1.g(n)<=f(n)<=c2.g(n)
• Replace g(n) by n and f(n) by 2n+3
• c1.n <=2n+3<=c2.n
if c1=1, c2=2, n=1, then 1*1 <=2*1+3 <=2*1, 1 <= 5 <= 2 // for n=1, it satisfies
the condition c1.g(n)<=f(n)<=c2.g(n)
If n=2, 1*2<=2*2+3<=2*2, 2<=7<=4 // for n=2, it satisfies the condition
c1.g(n)<=f(n)<=c2.g(n)
• Therefore, we can say that for any value of n, it satisfies the condition
c1.g(n)<=f(n)<=c2.g(n).
• Hence, it is proved that f(n) is big theta of g(n).
• So, this is the average-case scenario which provides the realistic time
complexity.