Data Structures

RM.Periakaruppan
Associate Professor
Dept. of Applied Mathematics and Computational Sciences
PSG College of Technology
Introduction to Data Structures
• Data
- Data represents fact
eg. Temperature recorded in a city on specific day
• Information
- Processed data
eg. Average temperature of a city in a month
• Knowledge
- based on information
eg. A City is very hot during a specific period
Introduction to Data Structures
• Data Structure
- Data Structure is a way of storing and organizing data in such a
way that we can perform operations on these data in an effective
way.

eg. Files to be printed by printer are arranged in computer’s

memory in sequential order. The file which is submitted first will
be printed first.
Linear Vs Non-linear Data Structure
• Linear
- When the elements stored in data structure form a sequence, it is
called linear data structure. Here the relationship between the
elements is linear. Each element has a unique predecessor and
successor element. (Except the first element which doesn’t have a
predecessor and the last element doesn’t have a successor.)
Eg.
array

Linked list
Linear Vs Non-Linear
• Non-linear
- When the elements stored in data structure do not form a
sequence, it is called non linear data structure. Here the relationship
between the elements is not linear. (ie. not one-one. It can be one-
many or many-many)
Eg.
Tree
Linear Vs Non-Linear Data Structures

Basic Data Structures

Linear Data Structures Non-Linear Data Structures

Arrays Linked Lists Stacks Queues Trees Graphs

Algorithm, Program and Data Structures
• An algorithm is the step-by-step instructions to solve the given
problem.
• A program is an implementation of algorithm in a particular
programming language.
• An appropriate choice of data structure is the key for designing an
efficient algorithm.
Efficiency of algorithm
• Running time (Time Complexity)
• Memory (Space Complexity)
Time Complexity
• Empirical or Posteriori approach (dependent on machine,
programming language etc)
• Theoretical or apriori approach(independent of hardware and
software used)
- running time is expressed as a function of input size
Why Time Complexity is important?
• To analyze the running time of algorithm as the input size increases.
• For larger input size, it is important to analyze the running time of
algorithm.
Understanding growth rate – Rice Grains and
Chessboard
•
Growth rate of algorithm
• Algorithm1 for finding Prime Number
• bool isPrime1(int n)
// Check from 2 to n-1
• for (int i = 2; i < n; i++)
• if (n % i == 0) Total Number of divisions = ?
• return false; (in case of prime number)
• return true;
•}
•
Growth rate of algorithm
• Algorithm1 for finding Prime Number
• bool isPrime1(int n)
// Check from 2 to n-1
• for (int i = 2; i < n; i++)
• if (n % i == 0) Total Number of divisions = n-2
• return false; (in case of prime number)
• return true;
•}
•
Growth rate of algorithm
• Algorithm2 for finding Prime Number
• bool isPrime2(int n)
// Check from 2 to sqroot(n)
• for (int i = 2; i <= sqroot(n); i++)
• if (n % i == 0) Total Number of divisions = ?
• return false;
• return true;
•}
•
Growth rate of algorithm
• Algorithm2 for finding Prime Number
• bool isPrime2(int n)
// Check from 2 to sqroot(n)
• for (int i = 2; i < sqroot(n); i++)
• if (n % i == 0) Total Number of divisions = sqroot(n)-1
• return false;
• return true;
•}
•
Growth rate of algorithm
•
Frequency Count
• We can express the running time of algorithm as function of input size
N.
• Example
• sum() Frequency Count
•{
• S=0 1
• for(i=0;i<n;i++) n+1
• S= S+ A[i] n
• return S 1
•} Total 2n + 3 expressed as O(n)
Asymptotic notations
• The following notations are used to express the time complexity of an
algorithm
• Big Oh (O) – gives an upper bound of an algorithm

• Big Omega (Ω) – gives a lower bound of an algorithm

• Big Theta (Θ) – gives tight bound, where the upper bound and lower
bound are same for an algorithm
Big Oh notation
Definition
f(n) = O(g(n)) means there are positive constants c and no, such that
0 ≤ f(n) ≤ cg(n) for all n ≥ no
• (ie) as n increases f(n) doesn’t grows faster than g(n)
•
Big Oh Example
• Consider the function f(n) = 2n + 3
• 2 n + 3 < = 4 n for every n>=2
• Here f(n) = 2 n + 3, g(n) = n, c = 4, n0=2
Here g(n)=n is an upper bound of f(n) and hence f(n)=O(g(n))= O(n)
Since O(n)<O(nlogn)<O(n^2)..<O(n^k)<O(2^n)<(3^n)..<O(k^n)
It is correct to say f(n) = O(n^2),...O(2^n).
However the closest upper bound is used. Hence f(n)=O(n) is more
meaningful.
Since log n < n it is incorrect to say f(n)=O(log n)
Big Oh Example -How to find Upper bound?
Consider the function f(n) = 2n + 3

2 n + 3 < = 2n + 3n (Promote all lower terms )

2 n + 3 <= 5n for every n>=1

Here f(n) <= C g(n) for every n>=1

Hence C=5, g(n)=n and n0 = 1

Therefore f(n)=O(g(n)) = O(n)

Big Oh Example (Continued)
Input n Frequency f(n) = 2n+3 C g(n)=5n

0 3 0

1 5 5

2 7 10

3 9 15

4 11 20

5 13 25
Big Oh Example (Continued)
Chart Title
30

25
25

20
20
F(N) AND G(N)

15
15
13
11
10
10 9
7
5
5
3

0
0
0 1 2 3 4 5
INPUT N

f(n) c g(n)
Big Omega notation
Definition
f(n) = Ω(g(n)) means there are positive constants c and no, such that
0 ≤ f(n) >= cg(n) for all n ≥ no
• (ie) as n increases, f(n) doesn’t grows slower than g(n)
•
Big Omega Example
• Consider the function f(n) = 2n + 3
• 2 n + 3 >= n for every n>=1
• Here f(n) = 2 n + 3, g(n) = n, c = 1, n0=1
Here g(n)=n is an lower bound of f(n) and hence f(n)=Ω(g(n))= Ω(n)
Since 1<log n < n
It is correct to say f(n) = Ω(log n),Ω(1)
Since n^2 > n it is incorrect to say f(n)=Ω(n^2)
Big Omega Example -How to find Lower
bound?
Consider the function f(n) = n^2 + n

n^2 + n >= 1 . n ^2 for all n >=1

(Drop all lower terms and leading coefficients)

Here f(n)=n^2+n, g(n)=n ^2, C=1 , n0=1

(ie) f(n)>=c g(n) for all n>=1
Big Omega Example (Continued)
Input n Frequency f(n) = n^2+n C g(n)=n^2

0 0 0

1 2 1

2 6 4

3 12 9

4 20 16

5 30 25
Big Omega Example (Continued)
Chart Title
35

30
30

25
25

20
F(N) AND G(N)

20
16
15
12

10 9

6
5 4
2
1
0
0
0 1 2 3 4 5
INPUT N

f(n) C g(n)
Theta notation
Definition
f(n) = Θ(g(n)) means there are positive constants c1, c2 and no, such
that c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ no
• (ie) as n increases f(n) grows at the same rate as g(n)
Theta Example
• Consider the function f(n) = 2n + 3
• n<=2 n + 3 < = 5 n for every n>=1
• Here f(n) = 2 n + 3, g(n) = n, c1 = 1, c2=5,n0=1
Here g(n)=n is an tight bound of f(n) and hence f(n)=Θ(g(n))= Θ(n)
Since log n # n or n^2 # n
It is incorrect to say f(n) = Θ(log n) or f(n)=Θ(n^2)
Big Theta Example (Continued)
Input n f(n)=2n+3 C1 g(n) = 1. n C2.g(n)=5.n

0 3 0 0

1 5 1 5

2 7 2 10

3 9 3 15

4 11 4 20

5 13 5 25
Theta Example (Continued)
Chart Title
30

25
25

20
20
F(N) AND G(N)

15
15 f(n)
13
C1 g(n)
11
10 C2(g(n)
10 9
7
5 5
5 4
3 3
2
1
0
0
0 1 2 3 4 5
INPUT N
 Example 2
• Consider the function f(n) = n^2 log n + n
• Upper Bound
• n^2 log n + n <= n^2 log n + n^2 log n
• n^2 log n + n < = 2 n^2 log n for all n>=1
• Hence upper bound is O(n^2 log n).
• Lower Bound
• n^2 log n + n >= n^2 log n for all n>=1
Hence lower bound is Ω(n^2 log n).
Tight Bound
Since n^2 log n <= n^2 log n + n <= 2 n^2 log n for all n>=1
We have Θ(n^2 log n)
Asymptotic Notations Example 3
• Find upper bound for
• f(n) = 2 n^2 + 3 n + 4

2 n^2 + 3 n + 4 <= 2 n^2 + 3 n^2 + 4 n^2

2 n^2 + 3 n + 4 <= 9 n^2 , n>=1
f(n) = O(n^2)
Asymptotic Notations Example 3
• Find lower bound for
• f(n) = 2 n^2 + 3 n + 4

2 n^2 + 3 n + 4 >= 1 . n^2

2 n^2 + 3 n + 4 >= n^2 , n>=1
f(n) = Ω(n^2)
Asymptotic Notations Example 3
• Find theta bound for
• f(n) = 2 n^2 + 3 n + 4

n^2 <= 2 n^2 + 3 n + 4 <=9 n^2

f(n)= Ɵ(n^2)
Asymptotic Notations Example 4
• Find upper bound for
• f(n) = n^2 log n + n

• n^2 log n + n <= n^2 log n + n^2 log n

n^2 log n + n <= 2 n^2 log n , for all n >= 2
f(n) = O(n^2 log n)
Asymptotic Notations Example 4
• Find lower bound for
• f(n) = n^2 log n + n

• n^2 log n + n >= n^2 log n for all n >= 1

f(n) = Ω(n^2 log n)
Asymptotic Notations Example 4
• Find theta bound for
• f(n) = n^2 log n + n

• n^2 log n <= n^2 log n + n <= 2 n^2 log n

f(n) = Ɵ(n^2 log n)
Time Complexity Examples
Example 1
• for(i=1;i<=n;i++)
{
statements;
}
Time complexity is
Example 1
• for(i=1;i<=n;i++)
{
statements;
}
Time complexity is O(n)
Example 2
• for(i=n;i>=1;i--)
{
statements;
}
Time complexity is
Example 2
• for(i=n;i>=1;i--)
{
statements;
}
Time complexity is O(n)
Example 3
• for(i=1;i<=n;i=i+2)
{
statements;
}
Number of times statements executed = ?
Time complexity is ?
Example 3
• for(i=1;i<=n;i=i+2)
{
statements;
}
Number of times statements executed = n/2
Time complexity is O(n)
Example 4
• for(i=1;i<=n;i++)
• for(j=1;j<=n;j++)
{
statements;
}
Time complexity is ?
Example 4
• for(i=1;i<=n;i++)
• for(j=1;j<=n;j++)
{
statements;
}
Time complexity is O(n^2)
Example 5
• for(i=1;i<=n;i++)
• for(j=1;j<=i; j++)
{
statements;
}
Time complexity is ?
Example 5
• for(i=1;i<=n;i++)
• for(j=1;j<=i; j++)
{
statements;
}
Time complexity is O(n^2)
(ie. When i=1, j=1
i=2 , j=1+2
i=n, j=1+2+...n
No of times j executes=n(n+1)/2 approximated as n^2)
Example 6
• p=0
• for(i=1;p<=n;i++)
{
p=p+i
}
Time complexity is ?
Example 6
• p=0
• for(i=1;p<=n;i++)
{
p=p+i
}
Time complexity is O(sqrt(n))
Example 6
• i p
• 1 0+1=1
• 2 1+2 = 3
• 3 1+2+3=5
• .
• k 1+2+3...+k = k(k+1)/2
• Loop will terminate, when p>n
• k(k+1)/2 >n or k^2 > n => k = sqrt(n)
Example 7

• for(i=1;i<=n;i=i*2)
{
Stmt;
}
Time complexity is ?
Example 7

• for(i=1;i<=n;i=i*2)
{
Stmt;
}
Time complexity is O(log n)
Example 7
‘i’ takes values 1,2,2^2,....2^k
Loop will terminate, when i>n
2^k > n
(i.e) 2 ^ k =n
=> k = log n
• Time complexity is O(log n)
Example 8

• for(i=n;i>=1;i=i/2)
{
Stmt;
}
Time complexity is ?
Example 8

• for(i=n;i>=1;i=i/2)
{
Stmt;
}
Time complexity is O(log n)
( i takes values n, n/2, n/(2^2),...n/(2^k)
Loop will terminate when i<1 (ie) n/(2^k) <1
2^k > n => k= log n)
Example 9

• for(i=0;i*i<n;i++)
{
Stmt;
}
Time complexity is ?
Example 9

• for(i=0;i*i<n;i++)
{
Stmt;
}
Time complexity is O(sqrt(n))
( loop will terminate when i^2 >=n
i = sqrt(n) )
 Example 10
• for(i=1;i<=n;i++)
•{
• for(j=1;j<=n;j++)
{
Stmt;
}
}
for(k=1;k<=n; k++)
{
Stmt;
}
Time complexity is ?
 Example 10
• for(i=1;i<=n;i++)
•{
• for(j=1;j<=n;j++)
{
Stmt;
}
}
for(k=1;k<=n; k++)
{
Stmt;
}
Time complexity is O(n^2) ( we have f(n)= n^2 + n which is n^2)
Example 11
• p=0
• for(i=1;i<=n;i=i*2)
•{
• p++;
•}
• for(j=1; j<=p;j=j*2)
•{
• Stmt;
•}
Time complexity is ?
Example 11
p=0
for(i=1;i<=n;i=i*2)
p++;
for(j=1; j<=p;j=j*2)
Stmt;
After first for loop, the value of p=log n
and hence first for loop runs for O(log n) times.
The second loop runs for log p, substituting p, we have
log log n and hence second for loop runs for O(log log n).
The overall time complexity is max(logn, log log n)= O(log n)
Example 12

for(i=0;i<n;i++)
{
for(j=1;j<n;j=j*2)
{
Stmt;
}
}
Time complexity is ?
Example 12

for(i=0;i<n;i++)
{
for(j=1;j<n;j=j*2)
{
Stmt;
}
}
Time complexity is O(nlog n)
Inner for loop runs for log n times and outer for loop runs for n times,
=> total number of times is n log n
Commonly used time complexities
• O(1) – Constant
• O(log n) - logarithmic
• O(n) – linear
• O(n logn) – linear logarithmic
• O(n2)- Quadratic
• O(n3)- Cubic
• O(nk) – Polynomial (in general)
• O(2n ) – Exponential time complexity ( in general O(kn)
Comparison of time complexities

• O(1)<O(logn)<O(n)<O(nlogn)<O(n^2)..<O(n^k)<O(2^n)<(3^n)..<O(k^n)
Analysis of Recursive function
• A function which call itself is called recursive function.
• Terminates when a base case is reached
• Each recursive call uses stack memory
• Used in solving problems based on divide and conquer strategy
• (eg) int fact(int n)
• { if (n==0 || n ==1) return 1; //base case
• else
• return n*fact(n-1); // recursive call to function fact
• }
• Mathematically can be expressed as recurrence relation
• T(n) = T(n-1) + a , n>1
= 1 , n=0,1
Analysis of Recursive function
• Solving the following Recurrence Relation
• T(n) = T(n-1) + a , n>1
= 1 , n=0,1

T(n) = T(n-1) + a
= [T(n-2)+a] + a
= T(n-2)+2a
= T(n-3)+3a
Analysis of Recursive function
•
T(n) = T(n-k)+ka {k times}
When k=n
T(n) = T(n-n)+na
= T(0) + na
=1+an
= O(n)
Worst, best and average case time complexity
• Based on the input instances of the problem, the best, worst and
average case time complexities can be found.
• The input instances for which the algorithm takes maximum possible
time is called worst case.
• The input instances for which the algorithm takes minimum possible
time is called best case.
• The input instances for which the algorithm is neither behaves as best
case nor worst case is called average case
Worst, best and average case time complexity
Example Linear Search
• Worst Case happens when the element to be searched is not present
or the last element in the array. Hence worst case time complexity is
O(n) [or Ɵ(n) or Ω(n)].

• Best case happens when the element to be searched when the

element to be searched is found in first position. Hence worst case
time complexity is O(1) [or Ɵ(1) or Ω(1)].

•
Worst, best and average case time
complexity
Example Linear Search
In average case analysis, we take all possible inputs and calculate computing
time for all of the inputs. Sum all the calculated values and divide the sum by total
number of inputs. We must know (or predict) distribution of cases. For the linear
search problem, let us assume that all cases are uniformly distributed.

The probability of element occurring at ‘i’ th position is 1/n. Hence we have

Position Probability No. of Comparisions

1 1/n 1
2 1/n 2
..
n 1/n n

Average case = 1. 1/n + 2. 2/n + 3. 3/n

= 1/n (n (n+1)/2)
= n+1/2
= O(n) [or Ɵ(n) or Ω(n)]