Algorithms and Problem Solving (15B17CI411)

EVEN 2021

Module 1: Lecture 1

Jaypee Institute of Information Technology (JIIT)

A-10, Sector 62, Noida
Algorithms and Problem Solving (15B17CI411)
Course Outline
Module Count of
Topics
No. Lectures
Introduction to problem solving approach; Asymptotic Analysis: Growth of Functions and Solving Recurrences;
1 Notations- Big O, big omega, big theta, little o; Empirical analysis of sorting and searching algorithms – Merge sort, 6
Quick sort, Heap sort, Radix sort, Count sort, Binary search, and Median search
2 Search Trees: Segment tree, Interval Tree, and RB Tree; Priority queue using Binomial and Fibonacci Heap 6
Fundamentals of Divide and Conquer (D&C) approach using Binary search, Quick sort, and Merge sort; Strassen’s
3 matrix multiplication; and Closest pair, etc.
2

Introduction to greedy based solution approach; Minimum Spanning Trees (Prim’s and Kruskal algorithms); Shortest
Path using Dijkstra’s algorithm; Fractional and 0/1 Knapsack; Coinage problem; Bin packing; Job scheduling –
4 Shortest job first, Shortest remaining job first, etc.; Graph coloring; and Text compression using Huffman coding and
6
Shannon-Fano coding, etc.

Review of backtracking based solution approach using N queen, and Rat in a maze; M-coloring problem; Hamiltonian
5 Cycle detection; Travelling salesman problem; Network flow
4

Fundamentals of Dynamic programming based solution approach; 0/1 Knapsack ; Shortest path using Floyd Warshall;
6 Coinage problem; Matrix Chain Multiplication; Longest common subsequence; Longest increasing sequence, String 6
editing
Naïve String Matching, Finite Automata Matcher, Rabin Karp matching algorithm, Knuth Morris Pratt, Tries; Suffix
7 Tree; and Suffix Array
6

Problem Spaces: States, goals and operators, Factored representation (factoring state into variables) Uninformed
8 search (BFS, DFS, DFS with iterative deepening), Heuristics and informed search (hill-climbing, generic best-first, A*)
4

9 Efficiency and Tractability, P, NP, NP-Complete, NP- Hard problems 2

Algorithms and Problem Solving (15B17CI411)
Course Outcome

Course Outcomes (CO) Cognitive Levels

Analyze the complexity of different algorithms using asymptotic

CO1 Analyze Level (Level 4)
analysis.

Select an appropriate data structure and apply related operations

CO2 Apply Level (Level 3)
for a given problem.

CO3 Apply algorithmic principles for solving a given problem. Apply Level (Level 3)

Identify, formulate and design an efficient solution to a given

CO4 problem using appropriate data structure and algorithm design Create Level (Level 6)
technique.
Algorithms and Problem Solving (15B17CI411)
Evaluation Criteria

S.
Evaluative Components Marks
No.
1 T1 20

2 T2 20

3 End Semester Examination 35

Teachers Assessment:
4 25
Attendance / PBL / Assignments

Total 100
 Algorithms and Problem Solving (15B17CI411)
Relevant Resources - I
Recommended Reading material: Author(s), Title, Edition, Publisher, Year of Publication etc. (Reference Books, Journals, Reports,
Websites etc. in the IEEE format)
1. Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein , Introduction to Algorithms, MIT Press, 3rd Edi tion, 2009
2. Steven Skiena ,The Algorithm Design Manual, Springer; 2nd edition , 2008
3. Knuth, The art of Computer Programming Volume 1, Fundamental Algorithms, Addison-Wesley Professional; 3 edition,1997
4. Horowitz and Sahni, Fundamentals of Computer Algorithms, Computer Science Press, 2008
5. Sedgewick, Algorithms in C, 3rd edition. Addison Wesley, 2002
Alfred V. Aho, J.E. Hopcroft, Jeffrey D. Ullman, Data Structures and Algorithms, Addison-Wesley Series in Computer Science and Information
6.
Processing, 1983
7. ACM Transactions on Algorithms (TALG)
8. Algorithmica Journal, Springer
9. Graphs and Combinatorics, Journal, Springer
10. The ACM Journal of Experimental Algorithmics
Recommended Reading material: Author(s), Title, Edition, Publisher, Year of Publication etc. ( Text books)
1. Tim Roughgarden, Algorithms Illuminated: Part 1: The Basics, Soundlikeyourself Publishing, September 27, 2017
2. Tim Roughgarden, Algorithms Illuminated:Part 2: Graph Algorithms and DataStructures ,Soundlikeyourself Publishing, First Edition, 2018.
Tim Roughgarden, Algorithms Illuminated :Part3:Greedy Algorithms and Dynamic Programming,Soundlikeyourself Publishing, First Edition,
3.
2019.
4. Weiss, Data Structures and Algorithm Analysis in C++, 4th Edition, Pearson, 2014
Algorithms and Problem Solving (15B17CI411)
Relevant Resources - II

S. No. E-contents and Platforms

NPTEL:
Design and Analysis of Algorithms
Introduction to Algorithms and Analysis
1
Data Structures and Algorithms
Fundamental Algorithms: Design and Analysis
Programing, Data Structures and Algorithms

2 Resources from MIT, Stanford, Cornell, etc

3 GeekforGeeks, etc
Virtual Lab
https://www.vlab.co.in/
Module 1

• Introduction to problem solving approach;

• Asymptotic Analysis: Growth of Functions and Solving Recurrences;
Notations- Big O, big omega, big theta, little o;
• Empirical analysis of sorting and searching algorithms – Merge sort,
Quick sort, Heap sort, Radix sort, Count sort, Binary search, and
Median search
Algorithms
• What is an Algorithm?
Algorithms
• What is an Algorithm?

⮚ It is a combination or a sequence of finite steps required to solve a given

problem
Algorithms
• What is an Algorithm?
• What are the properties of an Algorithm?
 Loop Invariant Example:
Algorithms Given A={7, 5, 3, 10, 2, 6}, find maximum element
max in the array.
max = -INF (minus infinite)
for (i = 0 to n-1)
if (A[i] > max) max = A[i]
• What is an Algorithm? In the above example after the 3rd iteration of the
loop max value is 7, which holds true for the first 3
elements of array A.
• What are the properties of an Algorithm? Here, the loop invariant condition is that max is
always maximum among the first i elements of
⮚ It should produce at-least one output. array A.

⮚ It should not have any ambiguous statements.

⮚ It should be loop invariant.
⮚ It should produce an output within a finite amount of time.
⮚ It should take a finite number of inputs.
⮚ It should be language independent.
❖ Correctness: It should produce the same output for the given same input at
any given time.
Algorithms
• What is an Algorithm?
• What are the properties of an Algorithm?
• Why “Analysis of Algorithm”?
Algorithms
• What is an Algorithm?
• What are the properties of an Algorithm?
• Why “Analysis of Algorithm”?
• What is the goal of “Analysis of Algorithm”?
Algorithms
• What is an Algorithm?
• What are the properties of an Algorithm?
• Why “Analysis of Algorithm”?
• What is the goal of “Analysis of Algorithm”?

Analysis
o Posteriori (experimental analysis)
o Priori (theoritical analysis)
Algorithms
• What is an Algorithm?
• What are the properties of an Algorithm?
• Why “Analysis of Algorithm”?
• What is the goal of “Analysis of Algorithm”?

Time Complexity: Compile-time + Run-time

Space Complexity: Extra memory space needed by an algorithm
Algorithms
• Expectation from an algorithm
Algorithms
• Expectation from an algorithm
• Correctness
• Less resource usage
Algorithms
• Expectation from an algorithm
❖ Correctness
⮚ Correct: Algorithms must produce correct result.
⮚ Produce an incorrect answer: Even if it fails to give correct results all the time still
there is a control on how often it gives wrong result.
⮚ Approximation algorithm: Exact solution is not found, but near optimal solution can be
found out.
❖ Less resource usage
Algorithms
• Time taken by an algorithm
o Performance measurement or Apostoriori Analysis: Implementing the
algorithm in a machine and then calculating the time taken by the system to
execute the program successfully.
o Performance Evaluation or Apriori Analysis. Before implementing the
algorithm in a system.
❑ How long the algorithm takes :-will be represented as a function of the size of the
input. f(n)→how long it takes if ‘n’ is the size of input.
❑ How fast the function that characterizes the running time grows with the input size.
“Rate of growth of running time”.
The algorithm with less rate of growth of running time is considered better.
 Algorithms Let Ci be the cost of ith line.
Cost No. Of times Executed
• Linear search algorithm C1 t1

for i = 0 to (n - 1) -----------------C1 C2 t2

if (A[i] == item) ----------------C2 C3 1/ 0

loc = i ------------------C3 C4 1/ 0
Exit ---------------------C4 C5 1/ 0
set loc = -1 -------------------------C5 Total time taken:
e.g. A = {1,2,3,4,5}
Search item 1  T(5) = C1+C2+C3+C4
Search item 3  T(5) = C1*3+C2*3+C3+C4
Search item 5  T(5) = C1*5+C2*5+C3+C4
Search item 6  T(5) = C1*5+C2*5+C5
 Algorithms Let Ci be the cost of ith line.
Cost No. Of times Executed
• Max and Min algorithm C1 1
C2 1
max = A[0] ------------------C1 C3 n-1
min = A[0] -------------------C2 C4 n-1
for ( i = 1 to n-1 ) -----------C3 C5 t1
if ( A[i] > max ) ----------C4 C6 t2
max = A[i] -------C5 C7 t3
else if ( A[i] < min ) -----C6 Total time taken:
min = A[i] -------C7 e.g. A = {1,2,3,4,5}  T(5) = C1+C2+4C3+4C4+4C5
(n-1) comparisons
e.g. A = {5,4,3,2,1}  T(5) = C1+C2+4C3+4C4+4C6+4C7
2(n-1) comparisons
e.g. A = {1}  T(1) = C1+C2
How to analyze an algorithm??
Why consider worst-case running time???
• The worst-case running time gives a guaranteed upper bound on the running
time for any input.
• For some algorithms, the worst case occurs often. For example, when searching,
the worst case often occurs when the item being searched for is not present,
and searches for absent items may be frequent.
How to analyze an algorithm??
Why consider worst-case running time???
• The worst-case running time gives a guaranteed upper bound on the running time
for any input.
• For some algorithms, the worst case occurs often. For example, when searching,
the worst case often occurs when the item being searched for is not present, and
searches for absent items may be frequent.

Why not analyze the average case?

Because it’s often about as bad as the worst

case.