1947

www.1042notes.blogspot.com

[WHAT IS ALGORITHM]
Outline:
Algorithm
Why study Algorithm
Prosperities of Algorithm
Need of Algorithm
Algorithm VS. Program
Complexity of Algorithm
 Time Complexity
 Space Complexity
Types of Analysis
Algorithm Design Techniques

`
 August 14, 1947 [WHAT IS ALGORITHM]

Algorithm
The term "analysis of algorithms" was coined by Donald Knuth.

An algorithm can be defined as a well-defined computational procedure that

takes some values, or the set of values, as an input and produces some value, or
the set of values, as an output.

An algorithm is a sequence of computational steps that transform the input into

output.
It describes specific computational procedures for achieving the input-output
relationship.

 Algorithm analysis is an important part of computational complexity theory,

which provides theoretical estimation for the required resources of an
algorithm to solve a specific computational problem.
 Most algorithms are designed to work with inputs of arbitrary length.
 Analysis of algorithms is the determination of the amount of time and space
resources required to execute it.
 Usually, the efficiency or running time of an algorithm is stated as a function
relating the input length to the number of steps, known as time complexity, or
volume of memory, known as space complexity.

1 www.1042notes.blogspot.com
 August 14, 1947 [WHAT IS ALGORITHM]

Why study Algorithm?

As the speed of processor increases, performance is frequently said to be less
central than other software quality characteristics (e.g. security, extensibility,
reusability etc.). However, large problem sizes are commonplace in the area of
computational science, which makes performance a very important factor. This is
because longer computation time, to name a few mean slower results, less
through research and higher cost of computation (if buying CPU Hours from an
external party).

The study of Algorithm, therefore, gives us a language to express

performance as a function of problem size.
An algorithm must have the following properties:
Correctness: It should produce the output according to the requirement of the
algorithm
Finiteness: Algorithm must complete after a finite number of instructions have
been executed.
An Absence of Ambiguity: Each step must be defined, having only one
interpretation.
Definition of Sequence: Each step must have a unique defined preceding
and succeeding step. The first step and the last step must be noted.
Input/output: Number and classification of needed inputs and results must be
stated.
Feasibility: It must be feasible to execute each instruction.
Flexibility: It should also be possible to make changes in the algorithm without
putting so much effort on it.
Efficient - Efficiency is always measured in terms of time and space requires
implementing the algorithm, so the algorithm uses a little running time and
memory space as possible within the limits of acceptable development time.
Independent: An algorithm should focus on what are inputs, outputs and how
to derive output without knowing the language it is defined. Therefore, we can
say that the algorithm is independent of language.

Need of Algorithm:
1. To understand the basic idea of the problem.
2. To find an approach to solve the problem.

2 www.1042notes.blogspot.com
 August 14, 1947 [WHAT IS ALGORITHM]

3. To improve the efficiency of existing techniques.

4. To understand the basic principles of designing the algorithms.
5. To compare the performance of the algorithm with respect to other
techniques.
6. It is the best method of description without describing the implementation
detail.
7. The Algorithm gives a clear description of requirements and goal of the
problem to the designer.
8. A good design can produce a good solution.
9. To understand the flow of the problem.
10. To measure the behavior (or performance) of the methods in all cases (best
cases, worst cases, average cases)
11. With the help of an algorithm, we can also identify the resources (memory,
input-output) cycles required by the algorithm.
12. With the help of algorithm, we convert art into a science.
13. To understand the principle of designing.
14. We can measure and analyze the complexity (time and space) of the problems
concerning input size without implementing and running it; it will reduce the cost
of design.

Algorithm vs Program:
 A finite set of instructions that specifies a sequence of operations to
be carried out to solve a specific problem of a class of problem is
called an algorithm.
 On the other hand, the Program doesn't have to satisfy the
finiteness condition.

For example, we can think of an operating system that continues in a
"wait" loop until more jobs are entered. Such a program doesn't
terminate unless the system crashes.

3 www.1042notes.blogspot.com
 August 14, 1947 [WHAT IS ALGORITHM]

Given a Problem to solve:

 Design Phase produces an algorithm.
 Implementation phase then generates a program that expresses the
designed algorithm.

So, the concrete expression of an algorithm in a particular programming

language is called a program.

Complexity of Algorithm:
It is very convenient to classify algorithm based on the relative amount of time or
relative amount of space they required and specify the growth of time/space
requirement as a function of input size.
Time Complexity: Running time of a program as a function of the size of the
input.
Space Complexity: Some forms of analysis could be done based on how
much space an algorithm needs to complete its task.
This space complexity analysis was critical in the early days of computing when
storage space on the computer was limited. When considering this algorithm are
divided into those that need extra space to do their work and those that work in
place.

But now a day's problem of space rarely occurs because space on the computer
(internal or external) is enough.

4 www.1042notes.blogspot.com
 August 14, 1947 [WHAT IS ALGORITHM]

Broadly, we achieve the following types of analysis -

Worst-case: f (n) defined by the maximum number of steps taken on any instance
of size n.
Best-case: f (n) defined by the minimum number of steps taken on any instance
of size n.
Average case: f (n) defined by the average number of steps taken on any instance
of size n.

To solve a problem:
we need to consider time as well as space complexity as the program may run on
a system where memory is limited but adequate space is available or may be vice-
versa.
In this context, if we compare bubble sort and merge sort. Bubble sort does not
require additional memory, but merge sort requires additional space. Though
time complexity of bubble sort is higher compared to merge sort, we may need to
apply bubble sort if the program needs to run in an environment, where memory
is very limited.

Algorithm Design Techniques

The following is a list of several popular design approaches:
 Divide and conquer Approach
 Greedy Technique
 Dynamic Programming
 Branch and Bound
 Randomized Algorithms
 Backtracking Algorithm
 Randomized Algorithm

1. Divide and Conquer Approach:

It is a top-down approach.
The algorithms which follow the divide & conquer techniques involve three steps:
 Divide the original problem into a set of sub problems.
 Solve every sub problem individually, recursively.
 Combine the solution of the sub problems (top level) into a solution of the
whole original problem.

5 www.1042notes.blogspot.com
 August 14, 1947 [WHAT IS ALGORITHM]

2. Greedy Technique:
Greedy method is used to solve the optimization problem.
An optimization problem is one in which we are given a set of input values, which
are required either to be maximized or minimized (known as objective), i.e. some
constraints or conditions.
Greedy Algorithm always makes the choice (greedy criteria) looks best at the
moment, to optimize a given objective.
The greedy algorithm doesn't always guarantee the optimal solution however it
generally produces a solution that is very close in value to the optimal.

3. Dynamic Programming:
Dynamic Programming is a bottom-up approach.
we solve all possible small problems and then combine them to obtain solutions
for bigger problems.
This is particularly helpful when the number of copying sub problems is
exponentially large.
Dynamic Programming is frequently related to Optimization Problems.

4. Branch and Bound:

In Branch & Bound algorithm a given sub problem, which cannot be bounded, has
to be divided into at least two new restricted sub problems.
Branch and Bound algorithm are methods for global optimization in non-convex
problems.
Branch and Bound algorithms can be slow, however in the worst case they require
effort that grows exponentially with problem size, but in some cases we are lucky,
and the method coverage with much less effort.
5. Randomized Algorithms: A randomized algorithm is defined as an
algorithm that is allowed to access a source of independent, unbiased random
bits, and it is then allowed to use these random bits to influence its computation.
6. Backtracking Algorithm: Backtracking Algorithm tries each
possibility until they find the right one. It is a depth-first search of the set of
possible solution. During the search, if an alternative doesn't work, then backtrack
to the choice point, the place which presented different alternatives, and tries the
next alternative.
7. Randomized Algorithm: A randomized algorithm uses a random
number at least once during the computation make a decision.

6 www.1042notes.blogspot.com
 August 14, 1947 [WHAT IS ALGORITHM]

Example 1: In Quick Sort, using a random number to choose a pivot.

Example 2: Trying to factor a large number by choosing a random number as
possible divisors.

7 www.1042notes.blogspot.com