What is Algorithm | Introduction to

Algorithms



Definition of Algorithm
The word Algorithm means ” A set of finite rules or instructions to be followed in
calculations or other problem-solving operations ”
Or
” A procedure for solving a mathematical problem in a finite number of steps that
frequently involves recursive operations”.
Therefore Algorithm refers to a sequence of finite steps to solve a particular problem.

Use of the Algorithms:

Algorithms play a crucial role in various fields and have many applications. Some of
the key areas where algorithms are used include:
1. Computer Science: Algorithms form the basis of computer programming
and are used to solve problems ranging from simple sorting and searching
to complex tasks such as artificial intelligence and machine learning.
2. Mathematics: Algorithms are used to solve mathematical problems, such
as finding the optimal solution to a system of linear equations or finding the
shortest path in a graph.
3. Operations Research: Algorithms are used to optimize and make decisions
in fields such as transportation, logistics, and resource allocation.
 4. Artificial Intelligence: Algorithms are the foundation of artificial
intelligence and machine learning, and are used to develop intelligent
systems that can perform tasks such as image recognition, natural language
processing, and decision-making.
5. Data Science: Algorithms are used to analyze, process, and extract insights
from large amounts of data in fields such as marketing, finance, and
healthcare.
These are just a few examples of the many applications of algorithms. The use of
algorithms is continually expanding as new technologies and fields emerge, making it
a vital component of modern society.
Algorithms can be simple and complex depending on what you want to achieve.
It can be understood by taking the example of cooking a new recipe. To cook a new
recipe, one reads the instructions and steps and executes them one by one, in the given
sequence. The result thus obtained is the new dish is cooked perfectly. Every time you
use your phone, computer, laptop, or calculator you are using Algorithms. Similarly,
algorithms help to do a task in programming to get the expected output.
The Algorithm designed are language-independent, i.e. they are just plain instructions
that can be implemented in any language, and yet the output will be the same, as
expected.
What is the need for algorithms?
1. Algorithms are necessary for solving complex problems efficiently and
effectively.
2. They help to automate processes and make them more reliable, faster, and
easier to perform.
3. Algorithms also enable computers to perform tasks that would be difficult
or impossible for humans to do manually.
4. They are used in various fields such as mathematics, computer science,
engineering, finance, and many others to optimize processes, analyze data,
make predictions, and provide solutions to problems.
What are the Characteristics of an Algorithm?

As one would not follow any written instructions to cook the recipe, but only the
standard one. Similarly, not all written instructions for programming are an algorithm.
For some instructions to be an algorithm, it must have the following characteristics:
 Clear and Unambiguous: The algorithm should be unambiguous. Each of
its steps should be clear in all aspects and must lead to only one meaning.
 Well-Defined Inputs: If an algorithm says to take inputs, it should be well-
defined inputs. It may or may not take input.
  Well-Defined Outputs: The algorithm must clearly define what output will
be yielded and it should be well-defined as well. It should produce at least 1
output.
 Finite-ness: The algorithm must be finite, i.e. it should terminate after a
finite time.
 Feasible: The algorithm must be simple, generic, and practical, such that it
can be executed with the available resources. It must not contain some
future technology or anything.
 Language Independent: The Algorithm designed must be language-
independent, i.e. it must be just plain instructions that can be implemented
in any language, and yet the output will be the same, as expected.
 Input: An algorithm has zero or more inputs. Each that contains a
fundamental operator must accept zero or more inputs.
 Output: An algorithm produces at least one output. Every instruction that
contains a fundamental operator must accept zero or more inputs.
 Definiteness: All instructions in an algorithm must be unambiguous,
precise, and easy to interpret. By referring to any of the instructions in an
algorithm one can clearly understand what is to be done. Every fundamental
operator in instruction must be defined without any ambiguity.
 Finiteness: An algorithm must terminate after a finite number of steps in all
test cases. Every instruction which contains a fundamental operator must be
terminated within a finite amount of time. Infinite loops or recursive
functions without base conditions do not possess finiteness.
 Effectiveness: An algorithm must be developed by using very basic,
simple, and feasible operations so that one can trace it out by using just
paper and pencil.
Properties of Algorithm:
 It should terminate after a finite time.
 It should produce at least one output.
 It should take zero or more input.
 It should be deterministic means giving the same output for the same input
case.
 Every step in the algorithm must be effective i.e. every step should do some
work.
Types of Algorithms:
There are several types of algorithms available. Some important algorithms are:
1. Brute Force Algorithm:
It is the simplest approach to a problem. A brute force algorithm is the first approach
that comes to finding when we see a problem.
2. Recursive Algorithm:
A recursive algorithm is based on recursion. In this case, a problem is broken into
several sub-parts and called the same function again and again.
3. Backtracking Algorithm:
The backtracking algorithm builds the solution by searching among all possible
solutions. Using this algorithm, we keep on building the solution following criteria.
Whenever a solution fails we trace back to the failure point build on the next solution
and continue this process till we find the solution or all possible solutions are looked
after.
4. Searching Algorithm:
Searching algorithms are the ones that are used for searching elements or groups of
elements from a particular data structure. They can be of different types based on their
approach or the data structure in which the element should be found.
5. Sorting Algorithm:
Sorting is arranging a group of data in a particular manner according to the
requirement. The algorithms which help in performing this function are called sorting
algorithms. Generally sorting algorithms are used to sort groups of data in an
increasing or decreasing manner.
6. Hashing Algorithm:
Hashing algorithms work similarly to the searching algorithm. But they contain an
index with a key ID. In hashing, a key is assigned to specific data.
7. Divide and Conquer Algorithm:
This algorithm breaks a problem into sub-problems, solves a single sub-problem, and
merges the solutions to get the final solution. It consists of the following three steps:
 Divide
 Solve
 Combine
8. Greedy Algorithm:
In this type of algorithm, the solution is built part by part. The solution for the next
part is built based on the immediate benefit of the next part. The one solution that
gives the most benefit will be chosen as the solution for the next part.
9. Dynamic Programming Algorithm:
This algorithm uses the concept of using the already found solution to avoid repetitive
calculation of the same part of the problem. It divides the problem into smaller
overlapping subproblems and solves them.
10. Randomized Algorithm:
In the randomized algorithm, we use a random number so it gives immediate benefit.
The random number helps in deciding the expected outcome.
To learn more about the types of algorithms refer to the article about “Types of
Algorithms“.
Advantages of Algorithms:
 It is easy to understand.
 An algorithm is a step-wise representation of a solution to a given problem.
 In an Algorithm the problem is broken down into smaller pieces or steps
hence, it is easier for the programmer to convert it into an actual program.
Disadvantages of Algorithms:
 Writing an algorithm takes a long time so it is time-consuming.
 Understanding complex logic through algorithms can be very difficult.
 Branching and Looping statements are difficult to show in
Algorithms(imp).
An algorithm is defined as a finite set of instructions that, if followed,
performs a particular task. All algorithms must satisfy the following criteria

Input. An algorithm has zero or more inputs, taken or collected from a

specified set of objects.

Output. An algorithm has one or more outputs having a specific relation to

the inputs.

Definiteness. Each step must be clearly defined; Each instruction must be

clear and unambiguous.

Finiteness. The algorithm must always finish or terminate after a finite

number of steps.

Effectiveness. All operations to be accomplished must be sufficiently basic

that they can be done exactly and in finite length.

We can depict an algorithm in many ways.

 Natural language: implement a natural language like English

 Flow charts: Graphic representations denoted flowcharts, only if the algorithm is small
and simple.
 Pseudo code: This pseudo code skips most issues of ambiguity; no particularity regarding
syntax programming language.

Example 1: Algorithm for calculating factorial value of a number

Step 1: a number n is inputted

Step 2: variable final is set as 1
Step 3: final<= final * n
Step 4: decrease n
Step 5: verify if n is equal to 0
Step 6: if n is equal to zero, goto step 8 (break out of loop)
Step 7: else goto step 3
Step 8: the result final is printed

Recursive Algorithms
A recursive algorithm calls itself which generally passes the return value as a
parameter to the algorithm again. This parameter indicates the input while
the return value indicates the output.

Recursive algorithm is defined as a method of simplification that divides the

problem into sub-problems of the same nature. The result of one recursion is
treated as the input for the next recursion. The repletion is in the self-similar
fashion manner. The algorithm calls itself with smaller input values and
obtains the results by simply accomplishing the operations on these smaller
values. Generation of factorial, Fibonacci number series are denoted as the
examples of recursive algorithms.

Example: Writing factorial function using recursion

What Is Time Complexity?

Time complexity

Time complexity is defined in terms of how many times it takes to run a given
algorithm, based on the length of the input. Time complexity is not a measurement
of how much time it takes to execute a particular algorithm because such factors as
programming language, operating system, and processing power are also considered
.Time complexity is a type of computational complexity that describes the time
required to execute an algorithm. The time complexity of an algorithm is the amount
of time it takes for each statement to complete.

The time required by the algorithm to solve given problem is called time
complexity of the algorithm. Time complexity is very useful measure in algorithm
analysis.
It is the time needed for the completion of an algorithm. To estimate the time
complexity, we need to consider the cost of each fundamental instruction and the
number of times the instruction is executed.
Example 1: Addition of two scalar variables.
Algorithm ADD SCALAR(A, B)
//Description: Perform arithmetic addition of two numbers
//Input: Two scalar variables A and B
//Output: variable C, which holds the addition of A and B
C <- A + B
return C
The addition of two scalar numbers requires one addition operation. the time
complexity of this algorithm is constant, so T(n) = O(1) .

What Is Space Complexity?

Space Complexity

When an algorithm is run on a computer, it necessitates a certain amount of memory

space. The amount of memory used by a program to execute it is represented by its
space complexity. Because a program requires memory to store input data and
temporal values while running, the space complexity is auxiliary and input space.

The amount of memory required by the algorithm to solve given problem is

called space complexity of the algorithm.
The space complexity of an algorithm quantifies the amount of space taken by an
algorithm to run as a function of the length of the input. Consider an example:
Suppose a problem to find the frequency of array elements .
It is the amount of memory needed for the completion of an algorithm.
To estimate the memory requirement we need to focus on two parts:
(1) A fixed part: It is independent of the input size. It includes memory for
instructions (code), constants, variables, etc.
(2) A variable part: It is dependent on the input size. It includes memory for
recursion stack, referenced variables, etc.
Example : Addition of two scalar variables
Algorithm ADD SCALAR(A, B)
//Description: Perform arithmetic addition of two numbers
//Input: Two scalar variables A and B
//Output: variable C, which holds the addition of A and B
C <— A+B
return C
The addition of two scalar numbers requires one extra memory location to hold the
result. Thus the space complexity of this algorithm is constant, hence S(n) = O(1).
Types of Asymptotic Notations in Complexity
Analysis of Algorithms



We have discussed Asymptotic Analysis, and Worst, Average, and Best Cases of
Algorithms. The main idea of asymptotic analysis is to have a measure of the
efficiency of algorithms that don’t depend on machine-specific constants and don’t
require algorithms to be implemented and time taken by programs to be compared.
Asymptotic notations are mathematical tools to represent the time complexity of
algorithms for asymptotic analysis.
Asymptotic Notations:
.Asymptotic Notations are mathematical tools that allow you to analyze an algorithm’s
running time by identifying its behavior as its input size grows.
There are mainly three asymptotic notations:
1. Big-O Notation (O-notation)
2. Omega Notation (Ω-notation)
3. Theta Notation (Θ-notation)

Asymptotic Notation
Asymptotic Notation is used to describe the running time of an algorithm -
how much time an algorithm takes with a given input, n. There are three
different notations: big O, big Theta (Θ), and big Omega (Ω). big-Θ is used
when the running time is the same for all cases, big-O for the worst case
running time, and big-Ω for the best case running time.

Big-Θ Notation
We compute the big-Θ of an algorithm by counting the number of
iterations the algorithm always takes with an input of n. For instance, the
loop in the pseudo code below will always iterate N times for a list size of
N. The runtime can be described as Θ(N).

Big-O Notation
The Big-O notation describes the worst-case running time of a program.
We compute the Big-O of an algorithm by counting how many iterations an
algorithm will take in the worst-case scenario with an input of N. We
typically consult the Big-O because we must always plan for the worst case.
For example, O(log n) describes the Big-O of a binary search algorithm.

Big-Ω Notation
Big-Ω (Omega) describes the best running time of a program. We compute
the big-Ω by counting how many iterations an algorithm will take in the
best-case scenario based on an input of N. For example, a Bubble Sort
algorithm has a running time of Ω(N) because in the best case scenario the
list is already sorted, and the bubble sort will terminate after the first
iteration.

This article was published as a part of the Data Science Blogathon.

Machine Learning is a very interesting branch of Artificial Intelligence where the
machine is made to learn from the data with the help of models created by us, and
then identify patterns and thus make predictions based on that. If you’re
considering taking your future ahead in the field of Data Science, then one of the
most important things you should be knowing is “Probability Theory‘. It plays a
very crucial role in Data Science and you should be a master of this theory. Have
you ever came across the word ‘Probability’? Don’t Worry! We’ll discuss this, and
at the end of this lecture, you will be confident about this theory. We’ll discuss all
the important things that one should know without wasting any time.

Introduction to Basics of Probability Theory

Probability simply talks about how likely is the event to occur, and its value
always lies between 0 and 1 (inclusive of 0 and 1). For example: consider that you
have two bags, named A and B, each containing 10 red balls and 10 black balls. If
you randomly pick up the ball from any bag (without looking in the bag), you
surely don’t know which ball you’re going to pick up. So here is the need of
probability where we find how likely you’re going to pick up either a black or a
red ball. Note that we’ll be denoting probability as P from now on. P(X) means the
probability for an event X to occur.

P(Red ball)= P(Bag A). P(Red ball | Bag A) + P(Bag B). P(Red ball | Bag B),
this equation finds the probability of the red ball. Here I have introduced the
concept of conditional probability ( which finds probability when we’re provided
with the condition). P(Bag A) = 1/2 because we’ve 2 bags out which we’ve to
select Bag A. P(Red ball | Bag A) should read as “probability of drawing a red
ball given the bag A” here “given” word specifies the condition which is Bag A in
this case, so it is 10 red balls out of 20 balls i.e. 10/20. So let’s solve:
P(Red Ball)= 1/2. 10/20 + 1/2. 10/20 = 1/2

Similarly, you can try to find the probability of drawing a black ball? Also, find the
probability of drawing two consecutive red balls from the bag after transferring one
black ball from bag A to Bag B?

Now if you look at the image above, you must be thinking what is it? I haven’t
introduced the “intersection” in Set Theory. I have already discussed the concept
above, there is nothing new in the image given above. Here we’re finding the
probability for an event A to occur given that event B has already occurred. The
numerator of the right-hand side of the equation is the probability for both events
to occur, divided by the probability for an event B to occur. The numerator has an
inverted shape symbol between A and B which we call “Intersection” in set theory.

Basic concepts of probability theory

There are a few key concepts that are important to understand in probability
theory. These include:

 Sample space: The sample space is the collection of all potential outcomes
of an experiment. For example, the sample space of flipping a coin is
{heads, tails}.
  Event: An event is a collection of outcomes within the sample space. For
example, the event of flipping a head is {heads}.
 Probability: The probability of an event is a number between 0 and 1 that
represents the likelihood of the event occurring. A chance of 0 means that
the event is impossible, and a probability of 1 means that the event is
specific.

Importance of Randomized Algorithms




Introduction:
 Randomization is an important concept, and hence
randomization algorithms are used in a variety of fields, such as number
theory, computational geometry, graph theory, and distributed computing.
 The inputs for a randomized algorithm are similar to those of deterministic
algorithms, along with a sequence of random bits that can be used by the
algorithm for making random choices.
 In other words, a randomized algorithm is one whose behavior depends on
the inputs, similar to a deterministic algorithm, and the random choices are
made as part of its logic.
 As a result, the algorithm gives different outputs even for the same input.
 In other words, the algorithm exhibits randomness; hence its run-time is
often explained in terms of a random variable.
Advantages:
 Randomized algorithms are known for their simplicity.
 Any deterministic algorithm can easily be converted to a randomized
algorithm. These algorithms are very simple to understand and implement.
 Randomized algorithms are very efficient.
 They utilize little execution time and space compared to any deterministic
algorithms.
 Randomized algorithms exhibit superior asymptotic bounds compared to
deterministic algorithms.
 In other words, the algorithm complexity of randomized algorithms is better
than that of most deterministic algorithms.
 Reliability is an important issue in many critical applications, as not all
randomized algorithms always give correct answers.
 In addition, many randomized algorithms may not terminate.
 Hence, reliability is an important concern that needs to be dealt with.
 The quality of randomized algorithms is dependent on the quality of the
random number generator used as part of the algorithm.
 Unlike other design paradigms, a randomized algorithm does not use a
single design principle.
 Instead, one should view randomized algorithms as those designed using a
set of principles.
 Instead, one should view randomized algorithms as those designed using a
set of principles.