What is an

Algorithm?
Design?
Analysis?
 Course Objectives

 Students will be able to do the following:

 Analyze the asymptotic performance of algorithms.
 Demonstrate a familiarity with paradigm of algorithm
 Applyalgorithmic design paradigms and methods of
analysis.
 Synthesize efficient algorithms in engineering design
situations.
Outline ……
 Learn
general approaches to algorith
m design
 Divide and conquer
 Greedy method
 Dynamic Programming
 Basic Search and Traversal Technique
 Graph Theory
 Linear Programming
 Approximation Algorithm
 NP Problem
Some Application
 Sorting, Searching
 data retrieval
 network routing
 Games
 Other graph applications
 Machine Learning
 AI
 everywhere
 Algorithms
 Informally,
A tool for solving a well-specified computational
problem.
 sequence of computational steps that transform
the input into the output.

 Example: sorting
input: A sequence of numbers.
output: An ordered permutation of the input.
issues: correctness, efficiency, storage, etc.

Input Algorithm Output

Algorithms
 No Subjective Decision ( salt as per taste)
 No Creativity / Intuition (Until tender)

 Exception
 Randomization (uniform distribution of
choice)
 Approximation (Square root of 2)
 Strengthening the
Informal Definiton
 Analgorithm is a finite sequence of
unambiguous instructions for solving a
well-specified computational problem.
Important Features:

Finiteness.[algo should end in finite steps]

Definiteness.[each instruction should be
clear]
Correctness [Generate correct output]
Input.[valid input clearly specified ]
Output.[single/multiple valid output]
Effectiveness.[ steps are sufficiently simple
and basic]
Questions

Is a problem solvable? (Does an

algorithm exist?)
Complexity classes of problems. (Is
an efficient algorithm possble?)
Is it computationaly feasible?
How you can express algorithm ?

Algorithm by 3 ways
1. Using Natural language
2. Pseudocode;
3. flowchart.
Pseudocode
Natural language + programming language
construct
For e.g.to perform addition of two numbers we
write in pseudo-code
//this algo perform addition of two integers
//input two integers a and b
// output of two integers
Read a,b
 C=a+b
 Write c
Flow Chart
Is it recent, to build better algorithm?
Is it recent, to build better algorithm?

Algorithm Existed since Ancient Time

Addition, subtraction, multiplication etc.
Euclid’s algo - GCD (ancient)
Gauss (various Efficient Algorithm)

Recent Times – Utilization of efficient

Algorithm
 Faster computing power
 More Memory
 Connected Computers

Algorithmic: Study of the design and Analysis of

Algorithms.
What is a program?
 Expression of an algorithm in a programming langu
age
a set of instructions - computer will follow
 Solves a problem
 Takes input (0/More) , Returns output
 Important
 Solution for need
 Aesthetics
 Marketing
Why - Algorithm analysis ?

 Working program is not enough

 May be inefficient!
 large
data set, then the running time
becomes an issue

 Tower of Hanoi – for 10,20,100000

disks
 Algorithm Analysis
 Determining performance characteristics.
 (Predicting the resource requirements.)
 Computation time (running time) is of primary concern.
 Memory
 Communication bandwidth etc.

 Resources to build ??
 Ease of Implementation ??
Analysis of an
Algorithm

There two types of analysis:

 Posterior
Analysis done after
implementation and running on target
 Priori
Analysis - theoretical estimation of
resources required.
Complexity of an
algorithm
Time complexity:
 Amount of time required by an algorithm to run.
 Order of magnitude / degree
 Space complexity:
 The amount of space required by an algorithm
to run.
 Order of magnitude.
 constant size + variable size
Types of Algorithms
 Iterative
 Recursive
Why - Algorithm analysis ?

 Choose the most efficient from

several
 Is the best possible running time for a
problem reasonably finite for practical
purposes?
 Is the algorithm optimal (best in some
sense)? – Is something better possible?
Few Classical Examples
Classical Multiplication
Algorithms
English

American

A la russe

Divide and Conquer

Classic Multiplication
Multiplication
The Selection Problem

Which algorithm is bett

er?
Which algorithm is better?
The algorithms are correct, but
which is the best?
 Measure the running time (number of
operations needed).
 Measure the amount of memory
used.
 Running time of the algorithms
increase as the size of the input
increases.
Experimental Studies
Practical 1

 Write a program 9000

implementing the 8000

algorithm 7000

6000

Time (ms)
 Run the program with 5000

inputs of varying size 4000

3000
and composition 2000

 Use chrono for C++ to 1000

0
get an accurate 0 50 100

measure of the actual I nput Size

running time
 Plot the results
Limitations of Experiments

 Need to implement the algorithm

 Difficult, Resource constraint
 Results may not be indicative of the
running time on other inputs not
included in the experiment.
 In order to compare two algorithms, the
same hardware and software
environments must be used
C++ random

 std::random_device rd;
 std::mt19937 generator(rd());
std::uniform_int_distribution<int> distribution(1, 1000);
const int SIZE = 10000;
// Adjust this value to change the size of the array
std::vector<int> data(SIZE);
for (int i = 0; i < SIZE; ++i) { data[i] =
distribution(generator); }
 C++ chrono

auto startTime =
std::chrono::high_resolution_clock::now();
selectionSort(data);
auto endTime =
std::chrono::high_resolution_clock::now();
auto duration =
std::chrono::duration_cast<std::chrono::microseconds
>(endTime - startTime).count();
Python time library

start_time = time.perf_counter() selection_sort(data)

end_time = time.perf_counter()
duration = (end_time - start_time) * 1e6 # Convert to microseconds
 Efficiency comparison of two algorithms

 Suppose n=106 numbers:

 Insertion sort: c1n2
 Best programmer (c1=2), machine language, one
billion/second computer A.
 2 (106)2 instructions/109 instructions per second =
2000 seconds.

34
Bad Programmer and Fast
Computer
Suppose n=106 numbers:
Merge sort: c n (lg n)
2
Bad programmer (c =50), high-language, ten
2
million/second computer B.
50 (106 lg 106) instructions/107 instructions
per second  100 seconds.
Thus, merge sort on B is 20 times faster than
insertion sort on A!
If sorting ten million numbers, 2.3 days VS.
20 minutes.
 Conclusions:
 Algorithms for solving the same problem
can differ dramatically in their efficiency.
 much more significant than the
differences due to hardware and
software.
Theoretical Analysis
 Uses a high-level description of the algorithm instead of an
implementation
 Characterizes running time as a function of the input size, n.
 Takes into account all (maximum) possible inputs
 Evaluate the speed of an algorithm independent of the
hardware/software environment
 Evaluation before implementation
Running Time

 Run time expression should be machine-

independent.
 Use a model of computation or “hypothetical”
computer.
 Our choice – RAM model (most commonly-used).
 Model should be
 Simple.
 Applicable.
 RAM Model
 Generic single-processor model.
 Supports simple constant-time
instructions found in real computers.
 Arithmetic (+, –, *, /, %, floor, ceiling).
 Data Movement (load, store, copy).
 Control (branch, subroutine call).
 Run time (cost) is uniform (1 time unit) for all
simple instructions.
 Memory is unlimited.
 Flat memory model – no hierarchy.
 Access to a word of memory takes 1 time
unit.
 Sequential execution – no concurrent
operations.
Running Time – Definition

 Call each simple instruction and

access to a word of memory a
“primitive operation” or “step.”
 Running time of an algorithm for a
given input is
 The number of steps executed by the
algorithm on that input.
 Often referred to as the complexity
of the algorithm.
Counting Primitive
Operations
 By inspecting the pseudo code, we can determine the
maximum number of primitive/basic operations executed
by an algorithm, as a function of the input size

Algorithm arrayMax(A, n) # operations

currentMax  A[0] 2
for (i =1; i<n; i++) 2n
(i=1 once, i<n n times, i++ (n-1) times: i.e 1+ n + n -1 = 2n )
if A[i]  currentMax then 2(n  1)
currentMax  A[i] 2(n  1)
return currentMax 1
Total 6n 
Estimating Running Time

 Algorithm arrayMax executes 6n  1 primitive

operations in the worst case.
Define:
a = Time taken by the fastest primitive operation
b = Time taken by the slowest primitive operation
 Let T(n) be worst-case time of arrayMax. Then
a (6n  1)  T(n)  b(6n  1 )
 Hence, the running time T(n) is bounded by two
linear functions
Growth Rate of Running
Time
 Changing the hardware/ software
environment
 Affects T(n) by a constant factor, but
 Does not alter the growth rate of T(n)
 The linear growth rate of the running
time T(n) is an essential property of
algorithm arrayMax
Function of Growth rate
A Simple Example – Linear Search
INPUT: a sequence of n numbers, key to search for.
OUTPUT: true if key occurs in the sequence, false
otherwise.
LinearSearch(A, key) cost times
1 i1 c1 
n
i 2
1 1

2 while i ≤ n and A[i] != key c2 x

3 do i++ c3 x-1
4 if i  n c4 1
5 then return true c5 1
6 else return false c6 1
x ranges between 1 and n+1.
So, the running time ranges between
c1+ c2+ c4 + c5 – best case
and
c1+ c2(n+1)+ c3n + c4 + c6 – worst case
A Simple Example – Linear Search
INPUT: a sequence of n numbers, key to search for.
OUTPUT: true if key occurs in the sequence, false
otherwise.
LinearSearch(A, key) cost times
1 i1 1 
n
i 2
1
1

2 while i ≤ n and A[i] != key 1 x

3 do i++ 1 x-1
4 if i  n 1 1
5 then return true 1 1
else return false 1 1
Assign a cost of 1 to all statement executions.
Now, the running time ranges between
1+ 1+ 1 + 1 = 4 – best case
and
1+ (n+1)+ n + 1 + 1 = 2n+4 – worst case
A Simple Example – Linear Search
INPUT: a sequence of n numbers, key to search for.
OUTPUT: true if key occurs in the sequence, false
otherwise.
LinearSearch(A, key) cost times
1 i1 1 
n
i 2
11
2 while i ≤ n and A[i] != key 1 x
3 do i++ 1 x-1
4 if i  n 1 1
5 then return true 1 1
6 else return false 1 1
If we assume that we search for a random item in the list,
on an average, Statements 2 and 3 will be executed n/2 times.
Running times of other statements are independent of input.
Hence, average-case complexity is
1+ n/2+ n/2 + 1 + 1 = n+3
Complexity and Input

 Complexity of an algorithm generally

depends on
 Size of input.
 Input size depends on the problem.
 Examples: No. of items to be sorted.
 No. of vertices and edges in a graph.
 Other characteristics of the input data.
 Are the items already sorted?
 Are there cycles in the graph?
Input Size

 Input size (number of elements in the input)

 size of an array

# of elements in a matrix

# of bits in the binary

representation of the input

 vertices and edges in a graph

How do we compare
algorithms?
We need to define a number of objective measures.

(1)Compare execution times?

Not good: times are specific to a
particular computer !!

(2) Count the absolute number of statements

executed?
Not good: number of statements vary with
the
programming language as well as the style
of the individual programmer.
Worst, Average, and
Best-case Complexity
 Worst-case Complexity
 Maximum steps the algorithm takes for any possible input.
 Most tractable measure.
 Average-case Complexity
 Average of the running times of all possible inputs.
 Demands a definition of probability of each input, which is usually
difficult to provide and to analyze.
 Best-case Complexity
 Minimum number of steps for any possible input.
 Not a useful measure. Why?
 Order of growth
 Principal interest is to determine
 how running time grows with input size – Order of
growth.
 the running time for large inputs – Asymptotic
complexity.
Order of growth

 In determining the above,

 Lower-order terms and coefficient of the
highest-order term are insignificant.
 Ex: In 7n5+6n3+n+10, which term
dominates the running time for very large
n?
 Complexity of an algorithm is denoted by
the highest-order term in the expression
for running time.
 Ex: Ο(n), Θ(1), Ω(n2), etc.
 Constant complexity when running time is
independent of the input size – denoted Ο(1).
 Linear Search: Best case Θ(1), Worst and
Average cases: Θ(n).
 Types of Analysis
 Worst case (at most BIG O)
 Provides an upper bound on running time
 An absolute guarantee that the algorithm would not run
longer, no matter what the inputs are
 Best case (at least Omega Ω )
 Provides a lower bound on running time
 Input is the one for which the algorithm runs the fastest

Lower Bound Running Time Upper Bound

 Average case (Theta Θ )

 Provides a prediction about the running time
 Assumes that the input is random
Asymptotic Analysis
 Mathematical and computational technique
 to analyze the behavior of algorithms or functions
as the input size approaches infinity.
 Helps us understand how the algorithm or function's
performance scales with larger inputs.
 predictions about its efficiency.
 The term "asymptotic" refers to the behavior of a
function or algorithm as it approaches a
particular limit or infinity.
 Provide a simplified representation of an algorithm's
efficiency
 It abstracts away the precise details of the
algorithm's performance and instead focuses on
the dominant factors
Asymptotic Analysis
 Can categorize algorithms into different complexity
classes, such as constant time, logarithmic time, linear
time, quadratic time, etc.
 Helps us compare different algorithms to determine which
one is more efficient for large inputs.
 Provides a high-level of understanding for scalability.
 To compare two algorithms with running times f(n) and
g(n), we need a rough measure that characterizes how
fast each function grows.
 Hint: use rate of growth
 Compare functions in the limit, that is, asymptotically!
(i.e., for large values of n)
Asymptotic Notation
 O notation :Big-O is the formal method
of expressing the upper bound of an
algorithm's running time.
 It's a measure of the longest amount of
time it could possibly take for the
algorithm to complete.
 Formally, for non-negative functions, f(n)
and g(n), if there exists an integer n0 and a
constant c > 0 such that for all integers n
> n0, f(n) ≤ cg(n), then f(n) is Big O of g(n).
Asymptotic notations
 O-notation
Asymptotic Notation
 Big-Omega Notation Ω
 This is almost the same definition as Big
Oh, except that "f(n) ≥ cg(n)”
 This makes g(n) a lower bound function,
instead of an upper bound function.
 It describes the best that can happen
for a given data size.
 For non-negative functions, f(n) and g(n),
if there exists an integer n0 and a
constant c > 0 such that for all integers n
> n0, f(n) ≥ cg(n), then f(n) is omega of
g(n). This is denoted as "f(n) = Ω(g(n))".
Asymptotic notations (cont.)

  - notation

(g(n)) is the set of

functions with larger or same
order of growth as g(n)
Asymptotic Notation
 Theta Notation Θ
 Theta Notation For non-negative functions,
f(n) and g(n), f(n) is theta of g(n) if and only
if f(n) = O(g(n)) and f(n) = Ω(g(n)). This is
denoted as "f(n) = Θ(g(n))".

This is basically saying that the function,

f(n) is bounded both from the top and
bottom by the same function, g(n).
Asymptotic notations
(cont.)
-notation

(g(n)) is the set of

functions with the same
order of growth as g(n)
Asymptotic notations
(cont.)
 Little-O Notation

 For non-negative functions, f(n) and g(n), f(n) is

little o of g(n) if and only if f(n) = O(g(n)), but f(n)
≠ Θ(g(n)). This is denoted as "f(n) = o(g(n))".
 Represents a loose bounding version of Big O.
 Little Omega Notation
 For non-negative functions, f(n) and g(n), f(n) is
little omega of g(n) if and only if f(n) = Ω(g(n)),
but f(n) ≠ Θ(g(n)). This is denoted as "f(n) =
ω(g(n))".

 It bounds from the bottom, but not from the top.

 There exist positive constants c
There exist positive constants c1 and c2
such that there is a positive constant n0
such that there is a positive constant n0
such that … cg(n)
such that …
c2g(n)
f(n) f(n)
c1g(n)

n n
n0 n0
f(n) = ( g(n)) f(n) = O( g(n))
There exist positive constants c
such that there is a positive constant n0
such that …
f(n)
cg(n)

n0 n
f(n) = ( g(n)) 64
Notes on o-notation

 O-notation may or may not be asymptotically

tight for upper bound.
 2n2 = O(n2) is tight, but 2n = O(n2) is not tight.
 o-notition is used to denote an upper bound
that is not tight.
 2n = o(n2), but 2n2  o(n2).
 Difference: for some positive constant c in O-
notation, but all positive constants c in o-
notation.
 In o-notation, f(n) becomes insignificant relative
to g(n)f(n)
as n approaches infinitely: i.e.,
lim g(n) = 0.
n
65
 Example 1.
1 2 2
Show that f ( n )  n  3n  ( n )
2

We must find c1 and c2 such that

2 1 2 2
c1n  n  3n c2 n
2
Dividing bothsides by n2 yields
1 3
c1   c2
2 n
1 2
For n0 7 , n  3n (n 2 )
2
 Theorem

• For any two functions f (n) and g (n) , we have

f (n) ( g (n))

if and only if
f (n) O( g (n)) and f (n) ( g (n)).
 Example 2.

f (n) 3n 2  2n  5 (n 2 )

Because :

3n 2  2n  5 (n 2 )

3n 2  2n  5 O(n 2 )
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6 O(n) since for any c, cn  3n 2 when n  c
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6 O(n) since for any c, cn  3n 2 when n  c
3n 2  100n  6 (n 2 ) since for c 2, 2n 2  3n 2  100n  6 when n  100
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6 O(n) since for any c, cn  3n 2 when n  c
3n 2  100n  6 (n 2 ) since for c 2, 2n 2  3n 2  100n  6 when n  100
3n 2  100n  6 (n 3 ) since for c 3, 3n 2  100n  6  n 3 when n  3
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6 O(n) since for any c, cn  3n 2 when n  c
3n 2  100n  6 (n 2 ) since for c 2, 2n 2  3n 2  100n  6 when n  100
3n 2  100n  6 (n 3 ) since for c 3, 3n 2  100n  6  n 3 when n  3
3n 2  100n  6 (n) since for any c, cn  3n 2  100n  6 when n  100
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6 O(n) since for any c, cn  3n 2 when n  c
3n 2  100n  6 (n 2 ) since for c 2, 2n 2  3n 2  100n  6 when n  100
3n 2  100n  6 (n 3 ) since for c 3, 3n 2  100n  6  n 3 when n  3
3n 2  100n  6 (n) since for any c, cn  3n 2  100n  6 when n  100
3n 2  100n  6 (n 2 ) since both O and  apply.
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6 O(n) since for any c, cn  3n 2 when n  c
3n 2  100n  6 (n 2 ) since for c 2, 2n 2  3n 2  100n  6 when n  100
3n 2  100n  6 (n 3 ) since for c 3, 3n 2  100n  6  n 3 when n  3
3n 2  100n  6 (n) since for any c, cn  3n 2  100n  6 when n  100
3n 2  100n  6 (n 2 ) since both O and  apply.
3n 2  100n  6 (n 3 ) since only O applies.
 Example 3.

3n 2  100n  6 O(n 2 ) since for c 3, 3n 2  3n 2  100n  6

3n 2  100n  6 O(n 3 ) since for c 1, n 3  3n 2  100n  6 when n  3
3n 2  100n  6 O(n) since for any c, cn  3n 2 when n  c
3n 2  100n  6 (n 2 ) since for c 2, 2n 2  3n 2  100n  6 when n  100
3n 2  100n  6 (n 3 ) since for c 3, 3n 2  100n  6  n 3 when n  3
3n 2  100n  6 (n) since for any c, cn  3n 2  100n  6 when n  100
3n 2  100n  6 (n 2 ) since both O and  apply.
3n 2  100n  6 (n 3 ) since only O applies.
3n 2  100n  6 (n) since only  applies.
Example
 Associate a "cost" with each statement.
 Find the "total cost“ by finding the total number
of times each statement is executed.

Algorithm 1 Algorithm 2

Cost Cost
arr[0] = 0; c1 for(i=0; i<N; i++) c2
arr[1] = 0; c1 arr[i] = 0; c1
arr[2] = 0; c1
... ...
arr[N-1] = 0; c1
----------- -------------
c1+c1+...+c1 = c1 x N (N+1) x c2 + N x c1
=
78
(c2 + c1) x N + c2
 Another Example

Algorithm 3 Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N2

79
Another Example
 Rate of Growth

 Considerthe example of buying elephants

and goldfish:
Cost: cost_of_elephants +
cost_of_goldfish
Cost ~ cost_of_elephants (approximation)
 The low order terms in a function are
relatively insignificant for large n
n4 + 100n2 + 10n + 50 ~ n4

i.e., we say that n4 + 100n2 + 10n + 50 and n4

have the same rate of 81growth
Comparison of Algorithms

 Complexity function can be used to compare

the performance of algorithms.
 Algorithm A is more efficient than Algorithm B
for solving a problem, if the complexity function
of A is of lower order than that of B.
 Examples:
 Linear Search – (n) vs. Binary Search – (lg n)
 Insertion Sort – (n2) vs. Quick Sort – (n lg n)
Comparisons of
Algorithms
 Multiplication
 classicaltechnique: O(nm)
 divide-and-conquer: O(nmln1.5) ~ O(nm0.59)
For operands of size 1000.
 Sorting
 insertionsort: (n2)
 merge sort: (n lg n)
For 106 numbers
Why Order of Growth Matters?

 Computer speeds double every two

years, so why worry about algorithm
speed?
 When speed doubles, what happens to
the amount of work you can do?
 What about the demands of
applications?
 Effect of Faster Machines
No. of items sorted
H/W Speed 1 M* 2M Gain
Comp. of Alg.

Ο(n2) 1000 1414 1.414

O(n lgn) 62700 118600 1.9

*
Million operations per second.

• Higher gain with faster hardware for more efficient algorithm.

• Results are more dramatic for more higher speeds.
Problem Type

 Decision
 Only one solution
 Answer in Yes / No
 Optimization
 Optimal
 Multiple solutions
 Maximization or Minimization problem
 Finding Optimal
 Conversion of optimal problem to
decision problem by bound
 Applications: Resource allocation, scheduling, route
planning, etc.
 Constraints: Real-world limitations that define feasible
solutions.
Optimality Problem
 Maximality:
 Concept: Finding the largest or most
desirable outcome.
 Examples: Maximizing profit, efficiency, or
output.
 Challenges:Avoiding local maxima, dealing
with multiple objectives.
Minimization

 Minimality:
 Concept: Finding the smallest or least costly
outcome.
 Examples: Minimizing cost, time, or resource
usage.
 Challenges: Balancing minimization with other
constraints or objectives.
Techniques for Solving
Optimization Problems:

 Linear Programming
 Dynamic Programming
 Greedy Algorithms
 Genetic Algorithms
 Simulated Annealing
Important Points

 Trade-offs:
 Balancing multiple objectives (e.g., cost vs.
quality).
 Pareto optimality: Solutions where improving one
objective worsens another.
 Global vs. Local Optima:
 Global optimum: Best solution across entire
solution space.
 Local optimum: Best solution within a neighboring
set of solutions.
 Challenge: Avoiding getting stuck in local optima.
Feasibility

 Computationally feasible
 For relative Large input, will algorithm stops in
reasonable time
 Prime factorization
 Tower of Hanoi
 Slow problem
 NP-hard problems: Many optimization problems are
computationally intensive.
 How to solve them?
 Chess
Cont..

 Convex optimization problems are generally

easier to solve.
 Many real-world problems can be formulated as
convex optimization.

 Optimization in Machine Learning:

 Training models often involves minimizing loss

functions.
 Gradient descent and its variants are common
optimization techniques in ML.
Slow Problem

 Distributed Algorithm
 Quntum computing
 Approximation algorithms: Finding near-optimal
solutions efficiently.
 Randomization