Chapter 1

Introduction
1.1. C++ Review
C++ is an object oriented programming language that is derived from a language called C. It has
many language constructs that are available to the programmer. In this course we will use two of
them: pointers and structure.
Pointer
A pointer is a variable that holds the address of other variables.
Syntax to declare a pointer
type *pointer_name;
To get the address of a variable we use the & (ampersand) operator.
e.g.
int a = 20;
int *p1 = &a;
One of the usages of pointers is for dynamic memory allocation. Dynamic memory allocation
allows us to allocate memory at run time. Allocating memory dynamically increases efficiency
because only the required memory will be allocated. There is no memory shortage or wastage.
To allocate memory dynamically, we use new operator.
Exercise
1. A C++ program that accepts age of students and displays it in sorted order. Use a
dynamic array. Your program should not allow negative values.
2. A C++ program that accepts score of n students and calculates the minimum, maximum
and average of the score.
Structure
A structure is a means of grouping different variables so that they can be used as a single entity
in our program. By using structures, the programmer can define his own data type.
Syntax to define a structure
struct structure_Name
{
//list of variables
};
e.g.
struct book
{
char title[200];
char author[200];
double price;
};
Once the structure is defined we can have variables whose data type the structure.
book b1;
book *bp1;
To access the members, we use dot operator for normal variables and we use arrow operator for
pointers.
b1.price = 100.50;
bp1-> = 100.50;
Exercise
1. Write a C++ program that accepts student data and sorts the data in one of the following
attributes of student.
a. Name
b. Age
c. Department
1.2. Algorithms
An algorithm is a well-defined computational procedure that takes some value or a set of values
as input and produces some value or a set of values as output. Data structures model the static
part of the world. They are unchanging while the world is changing. In order to model the
dynamic part of the world we need to work with algorithms. Algorithms are the dynamic part of
a program’s world model.
An algorithm transforms data structures from one state to another state in two ways:
 An algorithm may change the value held by a data structure
 An algorithm may change the data structure itself
The quality of a data structure is related to its ability to successfully model the characteristics of
the world. Similarly, the quality of an algorithm is related to its ability to successfully simulate
the changes in the world.
However, independent of any particular world model, the quality of data structure and algorithms
is determined by their ability to work together well. Generally speaking, correct data structures
lead to simple and efficient algorithms and correct algorithms lead to accurate and efficient data
structures.
Properties of an algorithm
• Finiteness: Algorithm must complete after a finite number of steps.
• Definiteness: Each step must be clearly defined, having one and only one
interpretation. At each point in computation, one should be able to tell exactly what
happens next.
• Sequence: Each step must have a unique defined preceding and succeeding step. The
first step (start step) and last step (halt step) must be clearly noted.
• Feasibility: It must be possible to perform each instruction.
• Correctness: It must compute correct answer all possible legal inputs.
• Language Independence: It must not depend on any one programming language.
• Completeness: It must solve the problem completely.
• Effectiveness: It must be possible to perform each step exactly and in a finite amount
of time.
• Efficiency: It must solve with the least amount of computational resources such as
time and space.
• Generality: Algorithm should be valid on all possible inputs.
• Input/Output: There must be a specified number of input values, and one or more
result values.
Usually algorithms are written using pseudo code. In pseudo code we use arithmetic operations,
assignment, if, while and other loop statement like high level languages. But also some
operations are written in natural language. Since it is a combination of natural language and
elements of high level languages, it is a false code, hence the name pseudo code.
One possible format for pseudo code is
 Title
Input
Output
List of Steps
Example
An algorithm that adds the first n +ve integers
Title: Algorithm that adds the first n +ve integers
Input: +ve integer n
Output: Sum of first n +ve integers
Steps
1. Read n
2. sum = 0
3. i = 0
4. while i<=n
sum = sum +i
i = i+1
loop
5.Display sum
Note: The above algorithm fulfills all the characteristics listed above. When we write algorithms,
they should fulfill all the characteristics listed above.
Exercise
For the problems given below develop an algorithm
A. checking weather a number is prime number or not.
B. listing all the prime numbers less or equal to +ve integer n.
1.3. Data Structures
Computer is a programmable electronic device. To use the computer we need to write programs.
A program is written in order to solve a problem. A solution to a problem actually consists of
two things:
 A way to organize the data
 Sequence of steps to solve the problem(Algorithms discussed above)
The way data are organized in a computer’s memory is said to be Data Structure and the
sequence of computational steps to solve a problem is said to be an algorithm. Therefore, a
program is nothing but data structures plus algorithms.
Given a problem, the first step to solve the problem is obtaining one’s own abstract view, or
model, of the problem. This process of modeling is called abstraction.
The model defines an abstract view to the problem. This implies that the model focuses only on
problem related stuff and that a programmer tries to define the properties of the problem.
These properties include:
 The data which are affected and
 The operations that are involved in the problem.
With abstraction you create a well-defined entity that can be properly handled. These entities
define the data structure of the program.
An entity with the properties just described is called an abstract data type (ADT).
An ADT consists of an abstract data structure and operations. Put in other terms, an ADT is an
abstraction of a data structure.
The ADT specifies:
1. What can be stored in the Abstract Data Type
2. What operations can be done on/by the Abstract Data Type.
For example, if we are going to model employees of an organization:
 This ADT stores employees with their relevant attributes and discarding irrelevant
attributes.
 This ADT supports hiring, firing, retiring, operations.
A data structure is a language construct that the programmer has defined in order to implement
an abstract data type.
There are lots of formalized and standard Abstract data types such as Stacks, Queues, Trees, etc.
Do all characteristics need to be modeled?
Not at all
 It depends on the scope of the model
 It depends on the reason for developing the model
Abstraction is a process of classifying characteristics as relevant and irrelevant for the particular
purpose at hand and ignoring the irrelevant ones.
Applying abstraction correctly is the essence of successful programming
How do data structures model the world or some part of the world?
 The value held by a data structure represents some specific characteristic of the world
 The characteristic being modeled restricts the possible values held by a data structure
 The characteristic being modeled restricts the possible operations to be performed on the
data structure.
Exercise
Arrays and linked lists are basic data structures that are used as a building block for other
complex data structures like stack and queue. List the advantages and disadvantages of these two
basic data structures.
 Chapter 2
Complexity Analysis
Algorithm analysis refers to the process of determining how much computing time and storage
that algorithms will require. In other words, it’s a process of predicting the resource requirement
of algorithms in a given environment.
In order to solve a problem, there are many possible algorithms. One has to be able to choose the
best algorithm for the problem at hand using some scientific method. To classify some data
structures and algorithms as good, we need precise ways of analyzing them in terms of resource
requirement. The main resources are:
 Running Time
 Memory Usage
 Communication Bandwidth
Running time is usually treated as the most important since computational time is the most
precious resource in most problem domains.
There are two approaches to measure the efficiency of algorithms:
• Empirical: Programming competing algorithms and trying them on different
instances.
• Theoretical: Determining the quantity of resources required mathematically
(Execution time, memory space, etc.) needed by each algorithm.
However, it is difficult to use actual clock-time as a consistent measure of an algorithm’s
efficiency, because clock-time can vary based on many things. For example,
 Specific processor speed
 Current processor load
 Specific data for a particular run of the program
o Input Size
o Input Properties
 Operating Environment
Accordingly, we can analyze an algorithm according to the number of operations required, rather
than according to an absolute amount of time involved. This can show how an algorithm’s
efficiency changes according to the size of the input.
2.1. Complexity Analysis
Complexity Analysis is the systematic study of the cost of computation, measured either in time
units or in operations performed, or in the amount of storage space required.
The goal is to have a meaningful measure that permits comparison of algorithms independent of
operating platform.
There are two things to consider:
 Time Complexity: Determine the approximate number of operations required to solve a
problem of size n.
 Space Complexity: Determine the approximate memory required to solve a problem of
size n.
The factor of time is more important than space.
Complexity analysis involves two distinct phases:
 Algorithm Analysis: Analysis of the algorithm or data structure to produce a function T
(n) that describes the algorithm in terms of the operations performed in order to measure
the complexity of the algorithm.
  Order of Magnitude Analysis: Analysis of the function T(n) to determine the general
complexity category to which it belongs.
There is no generally accepted set of rules for algorithm analysis. However, an exact count of
operations is commonly used.
2.2. Asymptotic Complexity
Asymptotic analysis is concerned with how the running time of an algorithm increases with the
size of the input in the limit, as the size of the input increases without bound.
Usually a function expressing the relationship between t (running time) and n (data size) involves
many terms. Since we are concerned with large values of n, we can ignore terms whose value is
insignificant for large values of n. Such measure of complexity is called asymptotic complexity.
e.g.
f(n) = n3 + 1000n
f(n)  n3
There are five notations used to describe asymptotic complexity. These are:
 Big-Oh Notation (O)
 Big-Omega Notation ()
 Theta Notation ()
 Little-o Notation (o)
 Little-Omega Notation ()
The Big-Oh Notation
Big-Oh notation is a way of comparing algorithms and is used for computing the complexity of
algorithms; i.e., the amount of time that it takes for computer program to run. It’s only concerned
with what happens for very a large value of n. Therefore only the largest term in the expression
(function) is needed. For example, if the number of operations in an algorithm is n2 – n (from n2
to n), n is insignificant compared to n2 for large values of n. Hence the n term is ignored. Of
course, for small values of n, it may be important. However, Big-Oh is mainly concerned with
large values of n.

Formal Definition: f (n) = O (g (n)) if there exist c, k ∊ℛ+ such that for all n≥ k, f (n) ≤ c.g (n).

Examples: The following points are facts that you can use for Big-Oh problems:
 1<=n for all n>=1
 n<=n2 for all n>=1
 2n<=n! for all n>=4
 log2n<=n for all n>=2
 n<=nlog2n for all n>=2
1. f(n)=10n+5 and g(n)=n. Show that f(n) is O(g(n)).
To show that f(n) is O(g(n)) we must show that constants c and k such that
f(n) <=c.g(n) for all n>=k
Or 10n+5<=c.n for all n>=k
Try c=15. Then we need to show that 10n+5<=15n
Solving for n we get: 5<5n or 1<=n.
So f(n) =10n+5 <=15.g(n) for all n>=1.
(c=15, k=1).
2. f(n) = 3n2 +4n+1. Show that f(n)=O(n2).
 4n <=4n2 for all n>=1 and 1<=n2 for all n>=1
3n2 +4n+1<=3n2+4n2+n2 for all n>=1
<=8n2 for all n>=1
So we have shown that f(n)<=8n2 for all n>=1
Therefore, f (n) is O(n2) (c=8,k=1)

Typical Orders
Here is a table of some typical cases. This uses logarithms to base 2, but these are simply
proportional to logarithms in other base.

N O(1) O(log n) O(n) O(n log n) O(n2) O(n3)

1 1 1 2 1 1 1
2 1 1 2 2 4 8
4 1 2 4 8 16 64
8 1 3 8 24 64 512
16 1 4 16 64 256 4,096
1024 1 10 1,024 10,240 1,048,576 1,073,741,824
Demonstrating that a function f(n) is big-O of a function g(n) requires that we find specific
constants c and k for which the inequality holds (and show that the inequality does in fact hold).
Big-O expresses an upper bound on the growth rate of a function, for sufficiently large values of
n.
An upper bound is the best algorithmic solution that has been found for a problem.
“What is the best that we know we can do?”
Exercise:
f(n) = (3/2)n2+(5/2)n-3
Show that f(n)= O(n2)
In simple words, f (n) =O(g(n)) means that the growth rate of f(n) is less than or equal to g(n).
Big-O Theorems
For all the following theorems, assume that f(n) is a function of n and that k is an arbitrary
constant.
Theorem 1: k is O(1)
Theorem 2: A polynomial is O(the term containing the highest power of n).
Polynomial’s growth rate is determined by the leading term
 If f(n) is a polynomial of degree d, then f(n) is O(nd)
In general, f(n) is big-O of the dominant term of f(n).
Theorem 3: k*f(n) is O(f(n))
Constant factors may be ignored
E.g. f(n) =7n4+3n2+5n+1000 is O(n4)
Theorem 4(Transitivity): If f(n) is O(g(n))and g(n) is O(h(n)), then f(n) is O(h(n)).
Theorem 5: For any base b, logb(n) is O(logn).
All logarithms grow at the same rate
logbn is O(logdn). b, d > 1
Theorem 6: Each of the following functions is big-O of its successors:
k
logbn
n
nlogbn
n2
n to higher powers
2n
3n
larger constants to the nth power
n!
nn
f(n)= 3nlogbn + 4 logbn+2 is O(nlogbn) and2n+n2 is O(2n)
Properties of the O Notation
Higher powers grow faster
• nr is O( ns) if 0 <= r <= s
Fastest growing term dominates a sum
• If f(n) is O(g(n)), then f(n) + g(n) is O(g)
E.g 5n4 + 6n3 is O (n4)
Exponential functions grow faster than powers, i.e. is O( bn ) b > 1 and k >= 0
E.g. n20 is O( 1.05n)
Logarithms grow more slowly than powers
• logbn isO( nk) b > 1 and k >= 0
E.g. log2n is O( n0.5)
Big-Omega Notation
Just as O-notation provides an asymptotic upper bound on a function,  notation provides an
asymptotic lower bound.
Formal Definition: A function f(n) is ( g (n)) if there exist constants c and k ∊ ℛ+ such that
f(n) >=c. g(n) for all n>=k.
f(n)=( g (n)) means that f(n) is greater than or equal to some constant multiple of g(n) for all
values of n greater than or equal to some k.
Example: If f(n) =n2, then f(n)= ( n)
In simple terms, f(n)=( g (n)) means that the growth rate of f(n) is greater that or equal to g(n).
Theta Notation
A function f (n) belongs to the set of  (g(n)) if there exist positive constants c1 and c2 such that
it can be sandwiched between c1.g(n) and c2.g(n), for sufficiently large values of n.
Formal Definition: A function f (n) is  (g(n)) if it isboth O( g(n)) and  ( g(n)). In other words,
there exist constants c1, c2, and k >0 such that c1.g (n)<=f(n)<=c2. g(n) for all n >= k
If f(n)=  (g(n)), then g(n) is an asymptotically tight bound for f(n).
In simple terms, f(n)=  (g(n)) means that f(n) and g(n) have the same rate of growth.
Example:
1. If f(n)=2n+1, then f(n) =  (n)
2. f(n) =2n2 then
f(n)=O(n4)
f(n)=O(n3)
 f(n)=O(n2)
All these are technically correct, but the last expression is the best and tight one. Since 2n2 and n2
have the same growth rate, it can be written as f(n)= (n2).
Little-o Notation
Big-Oh notation may or may not be asymptotically tight, for example:
2n2 = O(n2)
=O(n3)
f(n)=o(g(n)) means for all c>0 there exists some k>0 such that f(n)<c.g(n) for all n>=k.
Informally, f(n)=o(g(n)) means f(n) becomes insignificant relative to g(n) as n approaches
infinity.
Example: f(n)=3n+4 is o(n2)
In simple terms, f(n) has less growth rate compared to g(n).
g(n)= 2n2 g(n) =o(n3), O(n2), g(n) is not o(n2).
Little-Omega ( notation)
Little-omega () notation is to big-omega () notation as little-o notation is to Big-Oh notation.
We use  notation to denote a lower bound that is not asymptotically tight.
Formal Definition: f(n)=  (g(n)) if there exists a constant no>0 such that 0<= c. g(n)<f(n) for
all n>=k.
Example: 2n2=(n) but it’s not (n).
Relational Properties of the Asymptotic Notations
Transitivity
• if f(n)=(g(n)) and g(n)= (h(n)) then f(n)=(h(n)),
• if f(n)=O(g(n)) and g(n)= O(h(n)) then f(n)=O(h(n)),
• if f(n)=(g(n)) and g(n)= (h(n)) then f(n)= (h(n)),
• if f(n)=o(g(n)) and g(n)= o(h(n)) then f(n)=o(h(n)), and
• if f(n)= (g(n)) and g(n)= (h(n)) then f(n)= (h(n)).
Symmetry
• f(n)=(g(n)) if and only if g(n)=(f(n)).
Transpose symmetry
• f(n)=O(g(n)) if and only if g(n)=(g(n),
• f(n)=o(g(n)) if and only if g(n)=(g(n)).
Reflexivity
• f(n)=(f(n)),
• f(n)=O(f(n)),
• f(n)=(f(n)).
2.3. Best, Average and Worst Cases
In order to determine the running time of an algorithm it is possible to define three functions
Tbest(n), Tavg(n) and Tworst(n) as the best, the average and the worst case running time of the
algorithm respectively.
Average Case (Tavg): The amount of time the algorithm takes on an "average" set of inputs.
Worst Case (Tworst): The amount of time the algorithm takes on the worst possible set of inputs.
Best Case (Tbest): The amount of time the algorithm takes on the smallest possible set of inputs.
We are interested in the worst-case time, since it provides a bound for all input
2.4 Running time calculation examples
Analysis Rules:
1. We assume an arbitrary time unit.
2. Execution of one of the following operations takes time 1:
 Assignment Operation
 Single Input/output Operation
 Single Boolean Operations
 Single Arithmetic Operations
 Function Return
3. Running time of a selection statement (if, switch) is the time for the condition evaluation
+ the maximum of the running times for the individual clauses in the selection.
4. Loops: Running time for a loop is equal to the running time for the statements inside the
loop * number of iterations.
The total running time of a statement inside a group of nested loops is the running time of
the statements multiplied by the product of the sizes of all the loops.
For nested loops, analyze inside out.
 Always assume that the loop executes the maximum number of iterations possible.
5. Running time of a function call is 1 for setup + the time for any parameter calculations +
the time required for the execution of the function body.
Examples:
1. int count()
{
int k=0;
cout<< “Enter an integer”;
cin>>n;
for (i=0;i<n;i++)
k=k+1;
return 0;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int k=0
1 for the output statement.
1 for the input statement.
In the for loop:
1 assignment, n+1 tests, and n increments.
n loops of 2 units for an assignment, and an addition.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n)
2. int total(int n)
{
int sum=0;
for (int i=1;i<=n;i++)
 sum=sum+1;
return sum;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int sum=0
In the for loop:
1 assignment, n+1 tests, and n increments.
n loops of 2 units for an assignment, and an addition.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+ (1+n+1+n)+2n+1 = 4n+4 = O(n)
3. void func()
{
int x=0;
int i=0;
int j=1;
cout<< “Enter an Integer value”;
cin>>n;
while (i<n){
x++;
i++;
}
while (j<n)
{
j++;
}
}

Time Units to Compute

-------------------------------------------------
1 for the first assignment statement: x=0;
1 for the second assignment statement: i=0;
1 for the third assignment statement: j=1;
1 for the output statement.
1 for the input statement.
In the first while loop:
n+1 tests
n loops of 2 units for the two increment (addition) operations
In the second while loop:
n tests
n-1 increments
-------------------------------------------------------------------
T (n)= 1+1+1+1+1+n+1+2n+n+n-1 = 5n+5 = O(n)

4. int sum (int n)

{
int partial_sum = 0;
for (int i = 1; i <= n; i++)
partial_sum = partial_sum +(i * i * i);
return partial_sum;
}

Time Units to Compute

-------------------------------------------------
1 for the assignment.
1 assignment, n+1 tests, and n increments.
n loops of 4 units for an assignment, an addition, and two multiplications.
1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+(1+n+1+n)+4n+1 = 6n+4 = O(n)
Formal Approach to Analysis
In the above examples we have seen that analysis so complex. However, it can be simplified by
using some formal approach in which case we can ignore initializations, loop control, and book
keeping.
For Loops: Formally
• In general, a for loop translates to a summation. The index and bounds of the summation
are the same as the index and bounds of the for loop.
N

1 
for (int i = 1; i <= N; i++) {
sum = sum+i; N
}
i 1
• Suppose we count the number of additions that are done. There is 1 addition per iteration of
the loop, hence N additions in total.

Nested Loops: Formally

• Nested for loops translate into multiple summations, one for each for loop.

for (int i = 1; i <= N; i++) {

for (int j = 1; j <= M; j++) { N M N

}
sum = sum+i+j;   2   2M
i 1 j 1 i 1
 2 MN
}
• Again, count the number of additions. The outer summation is for the outer for loop.

Consecutive Statements: Formally

• Add the running times of the separate blocks of your code
for (int i = 1; i <= N; i++) {
sum = sum+i;
}  N   N N 
for (int i = 1; i <= N; i++) { 
 1       N  2N 2
 i 1   i 1 j 1
2
for (int j = 1; j <= N; j++) { 
sum = sum+i+j;
}
}

Conditionals: Formally
• If (test) s1 else s2: Compute the maximum of the running time for s1 and s2.
 if (test == 1) {
for (int i = 1; i <= N; i++) {  N N N 
}}
sum = sum+i; max
  1,   2 

 i 1 i 1 j 1 
 
else for (int i = 1; i <= N; i++) {
for (int j = 1; j <= N; j++) { max N , 2 N 2  2 N 2
sum = sum+i+j;
}}

Example:
Suppose we have hardware capable of executing 106 instructions per second. How long would it
take to execute an algorithm whose complexity function was:
T (n) = 2n2 on an input size of n=108?
The total number of operations to be performed would be T (108):
T(108) = 2*(108)2 =2*1016
The required number of seconds required would be given by
T(108)/106 so:
Running time =2*1016/106 = 2*1010 = 20,000,000,000 sec
The number of seconds per day is 86,400. So, this is about 231,481 days (634 years).
Exercises
Determine the run time equation and complexity of each of the following code segments.
1. for (i=0;i<n;i++)
for (j=0;j<n; j++)
sum=sum+i+j;
What is the value of sum if n=100?
2. for(int i=1; i<=n; i++)
for (int j=1; j<=i; j++)
sum++;
What is the value of the sum if n=20?
3. int k=0;
for (int i=0; i<n; i++)
for (int j=i; j<n; j++)
k++;
What is the value of k when n is equal to 20?
4. int k=0;
for (int i=1; i<n; i*=2)
for(int j=1; j<n; j++)
k++;
What is the value of k when n is equal to 20?
5. int x=0;
for(int i=1;i<n;i=i+5)
x++;
What is the value of x when n=25?
6. int x=0;
for(int k=n;k>=n/3;k=k-5)
x++;
What is the value of x when n=25?
7. int x=0;
for (int i=1; i<n;i=i+5)
for (int k=n;k>=n/3;k=k-5)
x++;
What is the value of x when n=25?
8. int x=0;
for(int i=1;i<n;i=i+5)
for(int j=0;j<i;j++)
for(int k=n;k>=n/2;k=k-3)
x++;
What is the correct big-Oh Notation for the above code segment?
 Chapter 3
Sorting and Searching Algorithms
3.1. Optimal Sorting Time
Sorting is one of the most important operations performed by computers. Sorting is a process of
reordering a list of items in either increasing or decreasing order. There are different types of
sorting algorithms. In this section we will see the optimal time for sorting.
Let us take three variables a, b and c, which hold some values. To arrange these values in
ascending order, we need at most three comparisons. See figure below.

a<b No
Yes
a<c
No
b<c Yes
No
Yes b<c
b,a,c
No
a<c Yes
a,b,c
No
Yes c,b,a
b,c,a
c,a,b
a,c,b

The above tree is called decision tree. The internal nodes of the tree represent the element
comparison, and the external nodes represent the sorted sequence. For three variables there are 6
permutations possible, out of which one will be the correct sequence. Depending on the
comparison at level i , further comparison at level i+1 will be carries out if the answer is not
reached. Only one comparison will be carried out at each level. The maximum number of
comparisons required therefore is one less than the height of the tree.
Given n numbers there are n! possible permutations. These n! permutations become the leaf
nodes of the decision tree. Moreover, any binary tree of height k has at most 2 k-1 leaves.
Therefore the height of the decision tree with n! leaf nodes must be log 2n! + 1 and the maximum
number of comparisons required to sort n elements is log 2n!. However, we know that n! 
(n/2)n/2 , which leads to
log2n!  O (n log 2n)
The optimal time therefore for sorting n numbers using comparison and swapping operations is
O (n log 2n). This shows that in the worst case the best comparison based algorithm takes O (n
log 2n).
The following sections we will see the following sorting algorithms.
• Insertion Sort
• Selection Sort
• Bubble Sort
• Quick
• Merge
Insertion, selection and bubble sorts are called simple sorting algorithms whereas quick and
merge sorts are called efficient sorting algorithms.
3.2. Insertion Sort
The insertion sort works just like its name suggests - it inserts each item into its proper place in
the final list. The simplest implementation of this requires two list structures - the source list and
the list into which sorted items are inserted. To save memory, most implementations use an in-
place sort that works by moving the current item past the already sorted items and repeatedly
swapping it with the preceding item until it is in place.
It's the most instinctive type of sorting algorithm. The approach is the same approach that you
use for sorting a set of cards in your hand. While playing cards, you pick up a card, start at the
beginning of your hand and find the place to insert the new card, insert it and move all the others
up one place.
Basic Idea:
Find the location for an element and move all others up, and insert the element.
The process involved in insertion sort is as follows:
1. The left most value can be said to be sorted relative to itself. Thus, we don’t need to do
anything.
2. Check to see if the second value is smaller than the first one. If it is, swap these two
values. The first two values are now relatively sorted.
3. Next, we need to insert the third value in to the relatively sorted portion so that after
insertion, the portion will still be relatively sorted.
4. Remove the third value first. Slide the second value to make room for insertion. Insert the
value in the appropriate position.
5. Now the first three are relatively sorted.
6. Do the same for the remaining items in the list.
Implementation
void insertion_sort(int list[]){
int temp;
for(int i=1;i<n;i++){
temp=list[i];
for(int j=i; j>0 && temp<list[j-1];j--)
{ // work backwards through the array finding where temp should go
list[j]=list[j-1];
list[j-1]=temp;
}//end of inner loop
}//end of outer loop
}//end of insertion_sort
Analysis
How many comparisons?
1+2+3+…+(n-1)= O(n2)
How many swaps?
1+2+3+…+(n-1)= O(n2)
How much space?
In-place algorithm
3.3. Selection Sort
Basic Idea:
 Loop through the array from i=0 to n-1.
 Select the smallest element in the array from i to n
 Swap this value with value at position i.
Implementation:
void selection_sort(int list[])
{
int i,j, smallest;
for(i=0;i<n;i++){
smallest=i;
for(j=i+1;j<n;j++){
if(list[j]<list[smallest])
smallest=j;
}//end of inner loop
temp=list[smallest];
list[smallest]=list[i];
list[i]=temp;
} //end of outer loop
}//end of selection_sort
Analysis
How many comparisons?
(n-1)+(n-2)+…+1= O(n2)
How many swaps?
n=O(n)
How much space?
In-place algorithm
3.4. Bubble Sort
Bubble sort is the simplest algorithm to implement and the slowest algorithm on very large
inputs.
Basic Idea:
 Loop through array from i=0 to n and swap adjacent elements if they are out of order.
Implementation:
void bubble_sort(list[])
{
int i,j,temp;
for(i=0;i<n; i++){
for(j=n-1;j>i; j--){
 if(list[j]<list[j-1]){
temp=list[j];
list[j]=list[j-1];
list[j-1]=temp;
}//swap adjacent elements
}//end of inner loop
}//end of outer loop
}//end of bubble_sort
Analysis of Bubble Sort
How many comparisons?
(n-1)+(n-2)+…+1= O(n2)
How many swaps?
(n-1)+(n-2)+…+1= O(n2)
Space?
In-place algorithm.
General Comments
Each of these algorithms requires n-1 passes: each pass places one item in its correct place. The
ith pass makes either i or n - i comparisons and moves. So:

or O(n2). Thus these algorithms are only suitable for small problems where their simple code
makes them faster than the more complex code of the O(n logn) algorithm. As a rule of thumb,
expect to find an O(n logn) algorithm faster for n>10 - but the exact value depends very much on
individual machines!.
Empirically it’s known that Insertion sort is over twice as fast as the bubble sort and is just as
easy to implement as the selection sort. In short, there really isn't any reason to use the selection
sort - use the insertion sort instead.
If you really want to use the selection sort for some reason, try to avoid sorting lists of more than
a 1000 items with it or repetitively sorting lists of more than a couple hundred items.
3.5. Quick Sort
Quick sort is the fastest known algorithm. It uses divide and conquer strategy and in the worst
case its complexity is O (n2). But its expected complexity is O(nlogn).
Algorithm:
1. Choose a pivot value (mostly the first element is taken as the pivot value)
2. Position the pivot element and partition the list so that:
 the left part has items less than or equal to the pivot value
 the right part has items greater than or equal to the pivot value
3. Recursively sort the left part
4. Recursively sort the right part
The following algorithm can be used to position a pivot value and create partition.
Left=0;
Right=n-1; // n is the total number of elements in the list
 PivotPos=Left;
while(Left<Right)
{
if(PivotPos==Left)
{
if(Data[Left]>Data[Right])
{
swap(data[Left], Data[Right]);
PivotPos=Right;
]
Left++;}
else
Right--; }
else
{
if(Data[Left]>Data[Right])
{
swap(data[Left], Data[Right]);
PivotPos=Left;
Right--;}
else
Left++;}
Example: Sort }the following list using quick sort algorithm. 0 3 2 4 1 5 9 7 6 8

5 8 2 4 1 3 9 7 6 0 LeftRight
Pivot
5 8 2 4 1 3 9 7 6 0 0 3 2 4 1 5 9 7 6 8

Left Right Left Right Left Right

Pivot Pivot Pivot
0 8 2 4 1 3 9 7 6 5 0 3 2 4 1 5 8 7 6 9

Left Right Left Right Left

Right
Pivot Pivot Pivot
0 5 2 4 1 3 9 7 6 8 0 3 2 4 1 5 8 7 6 9

Left Right Left Right LeftRight

Pivot Pivot Pivot
0 5 2 4 1 3 9 7 6 8 0 3 2 4 1 5 8 7 6 9

Left Right LeftRight Left Right

Pivot Pivot Pivot
0 5 2 4 1 3 9 7 6 8 0 3 2 4 1 5 6 7 8 9

Left Right Left Right LeftRight

Pivot Pivot Pivot
0 5 2 4 1 3 9 7 6 8 0 1 2 4 3 5 6 7 8 9

Left Right Left Right LeftRight

Pivot Pivot Pivot
0 5 2 4 1 3 9 7 6 8 0 1 2 4 3 5 6 7 8 9
 Left Right Left Right
Pivot Pivot
0 5 2 4 1 3 9 7 6 8 0 1 2 3 4 5 6 7 8 9

Left Right LeftRight

Pivot Pivot
0 3 2 4 1 5 9 7 6 8 0 1 2 3 4 5 6 7 8 9

LeftRight LeftRight
Pivot Pivot
0 3 2 4 1 5 9 7 6 8 0 1 2 3 4 5 6 7 8 9

Left Right
Pivot
3.6. Merge Sort
Like quick sort, merge sort uses divide and conquer strategy and its time complexity is O(nlogn).
Algorithm:
1. Divide the array in to two halves.
2. Recursively sort the first n/2 items.
3. Recursively sort the last n/2 items.
4. Merge sorted items (using an auxiliary array).
Example: Sort the following list using merge sort algorithm.

5 8 2 4 1 3 9 7 6 0

5 8 2 4 1 3 9 7 6 0

5 8 2 4 1 3 9 7 6 0
Division phase
5 8 2 4 1 3 9 7 6 0

5 8 2 4 1 3 9 7 6 0

4 1 6 0

1 4 0 6

5 8 1 2 4 3 9 0 6 7
Sorting and merging phase

1 2 4 5 8 0 3 6 7 9

0 1 2 3 4 5 6 7 8 9

3.7. Linear Search (Sequential Search)

Searching is a process of looking for a specific element in a list of items or determining that the
item is not in the list. There are two simple searching algorithms:
• Sequential Search, and
• Binary Search
Linear Search
Pseudo code
Loop through the array starting at the first element until the value of target matches one
of the array elements.
If a match is not found, return –1.
Time is proportional to the size of input (n) and
we call this time complexity O(n)
.
Example Implementation:
int Linear_Search(int list[], int key)
{
int index=0;
int found=0;
do{
if(key==list[index])
found=1;
else
index++;
}while(found==0&&index<n);
if(found==0)
index=-1;
return index;
}
3.8 Binary Search
This searching algorithms works only on an ordered list.
The basic idea is:
• Locate midpoint of array to search
• Determine if target is in lower half or upper half of an array.
o If in lower half, make this half the array to search
o If in the upper half, make this half the array to search
• Loop back to step 1 until the size of the array to search is one, and this element does not
match, in which case return –1.
The computational time for this algorithm is proportional to log2 n
. Therefore the time complexity is O(log n)

Example Implementation:
int Binary_Search(int list[],int k)
{
int left=0;
int right=n-1;
int found=0;
do{
mid=(left+right)/2;
if(key==list[mid])
found=1;
else{
if(key<list[mid])
right=mid-1;
else
left=mid+1;
}
}while(found==0&&left<=right);
if(found==0)
index=-1;
else
index=mid;
return index;}