Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
58 views

1.algorithm and Complexity

This document discusses algorithm analysis and asymptotic notation. It explains that algorithms can be evaluated based on their time and space complexity. Asymptotic notation like Big-O, Ω, and Θ are introduced to describe an algorithm's worst-case, best-case, and tight asymptotic running time bounds in terms of input size. Common complexity classes like logarithmic, linear, quadratic, and exponential are covered. Asymptotic notation provides a simplified way to analyze how an algorithm's running time increases with larger inputs.

Uploaded by

Ankit Agrawal
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
58 views

1.algorithm and Complexity

This document discusses algorithm analysis and asymptotic notation. It explains that algorithms can be evaluated based on their time and space complexity. Asymptotic notation like Big-O, Ω, and Θ are introduced to describe an algorithm's worst-case, best-case, and tight asymptotic running time bounds in terms of input size. Common complexity classes like logarithmic, linear, quadratic, and exponential are covered. Asymptotic notation provides a simplified way to analyze how an algorithm's running time increases with larger inputs.

Uploaded by

Ankit Agrawal
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 26

Department of Computational and DataSciences

Asymptotic notation
& Complexity
A l g o r i t h m A n a l ys i s
• Algorithms can be evaluated on two
performance measures
• Time taken to run an algorithm
• Memory space required to run an algorithm
• …for a given input size

• Why are these important?


S p a c e Complexity
• Space complexity = Estimate of the amount
of memory required by an algorithm to run
to completion, for a given input size
– Core dumps = the most often encountered
cause is “memory leaks” – the amount of
memory required larger than the memory
available on a given system
• Some algorithms may be more efficient if
data completely loaded into memory
– Need to look also at system limitations
S p a c e Complexity
• Fixed part: The size required to store certain
data/variables, that is independent of the size
of the problem:
– e.g., given the definition for every word in a given
book
– Constant size for storing “dictionary”, e.g., 50MB
• Variable part: Space needed by variables,
whose size is dependent on the size of the
problem:
– e.g., actual book size may vary, 1MB … 1GB
Analyzing Running T i m e
• Write program
• Run Program
• Measure actual running time with some methods
like System.currentTimeMillis(), gettimeofday()

• Is that good enough as a programmer?


Limitation of Experimental Running
Time Measurements
• Need to implement the algorithm.
• Cannot exhaust all possible inputs
– Experiments can be done only on a limited to
set of inputs, and may not be indicative of the
running time for other inputs.
• Harder to compare two algorithms
– Same hardware/environments to be used
Running
Time

• Suppose the program includes an if-then statement


that may execute or not  variable running time
• Typically algorithms are measured by their worst
case
Develop General Methodology for
Analysis
• Uses High Level Description instead of
implementation
• Takes into account for all possible inputs
• Allows one to evaluate the efficiency
independent of hardware/software environment.
Pseudo Code
• Mix of natural language and high level
programming concepts that describes the main
idea behind algorithm

arrayMax(A,n)
Max=A[0]
for i=1 to n-1 do
If Max < A[i] then Max = A[i]
Return Max
Pseudo Code
• More structured and less formal

• Expressions
– Standard Math symbols (numeric/boolean)

• Method Declarations
– Algorithm name(param1,param2)
Pseudo Code
• Programming Constructs:
– If … then …else
– While-loop
– for-loop
– Array : A[i]; A[I,j]
• Methods
– Calls: method(args)
– Returns: return value
Analysis of Algorithms
• Analyze time taken by Primitive Operations
• Low level operations independent of
programming language
– Data movement (assign..)
– Control (branch, subroutine call, return…)
– Arithmetic/logical operations (add, compare..)

• By inspecting the pseudo-code, we can count


the number of primitive operations executed by
an algorithm
Example: A r ray Tra n s p o s e
function Transpose(A[][], n)
• for i = 0 to n-1 do for j = i+1 to n-1 do
• tmp = A[i][j] A[i][j] = A[j][i] j=0 j=3
• A[j][i] = tmp end i=0 0,0 0,1 0,2
0,3
• end 1,
0
1,1 1,2 1,3

2,1 2,2 2,3


2,
i=3 0 3,1 3,2 3,3

3,
0

end
Estimated time for A[n][n] = (n(n-1)/2).(3+2) + 2.n
Is this constant for a given ‘n’?
Asymptotic Analysis
• Goal: to simplify analysis of running time
by getting rid of ‘details’ which may be
affected by specific implementation and
hardware.
– Like ‘rounding’: 1001 = 1000
– 3n2=n2
• How the running time of an algorithm
increases with the size of input in the limit.
– Asymptotically more efficient algorithms are
best for all but small inputs.
Asymptotic Notation: “Big O”

TB, Sahni

• O Notation
– Asymptotic upper bound
– f(n)=O(g(n)), if there exists
constants c and n0, s.t. c.g(n)

 f(n) ≤ c.g(n) for n ≥ n0 f(n)


– f(n) and g(n) are functions over
non negative intergers
• Used for worst-case analysis
• g(n) is the asymptotic upper boundof
actual time taken
Input Size
Asymptotic Notation
• Simple Rule: Drop lower order terms
and constant factors
– (n(n-1)/2).(3+2) + 2.n is O(n2)
– 23.n.log(n) is O(n.log(n))
– 9n-6 is O(n)
– 6n2.log(n) + 3n2 + n is O(n2.log(n))
• Note: It is expected that the approximation
should be as small an order as possible
Asymptotic Analysis of
Running Time
• Use O notation to express number of primitive
operations executed as a function of input size.
• Hierarchy of functions
1 < log n < n < n2 < n3 < 2n
Better

• Warning! Beware of large constants (say 1M).


• This might be less efficient than one running in time 2n2,
which is O(n2)
E xa m p l e of Asymptotic Analysis
• Input: An array X[n] of numbers.
• Output: An array A[n] of numbers s.t A[k]=mean(X[0]+X[1]+…+X[k-1])
for i=0 to (n-1) do
a=0
for j=0 to i do
a = a + X[j]
end
A[i] = a/(i+1)
end
return A
 Analysis: running time is O(n2)
CDS.IISc.ac.in | Department of Computational and DataSciences

A Better A l go r i t h m
s=0
for i=0 to n do
s = s + X[i]
A[i] = s/(i+1)
end
return A

 Analysis: running time is O(n)


Asymptotic Notation
• Special Cases of algorithms
– Logarithmic O(log n)
– Linear O(n)
– Quadratic O(n2)
– Polynomial O(nk), k >1
– Exponential O(an), a>1
Comparison
CDS.IISc.ac.in | Department of Computational and DataSciences

Asymptotic Notation: Lower


Bound
• The “big-Omega” Ω
notation
– asymptotic lower bound
f(n)
– f(n) = Ω(g(n)) if there exists const. c c.g(n)
and n0 s.t.
 c.g(n) ≤ f(n) for n ≥ n0
– Used to describe best-case
asymptotic running times
Input Size
• E.g., lower-bound of searching an
unsorted array; lower bound for
sorting an array
Asymptotic Notation: Tight
Bound
• The “big-Theta” θ- c2.g(n)
Notation
– Asymptotically tight bound f(n) c1.g(n)

– f(n) = θ(g(n)) if there exists


consts. c1, c2 and n0 s.t. c1 g(n)
≤ f(n) ≤ c2 g(n) for n ≥ n0
• f(n) = θ (g(n)) iff Input Size

f(n)=O(g(n)) and f(n)=Ω(g(n))


S m a l l “o”
• o Notation
– Asymptotic strict upper bound
– f(n)=O(g(n)), if there exists constants c and
n0, s.t.
 f(n) < c.g(n) for n ≥ n0

Similarly small omega, ω, is strict lower bound


Asymptotic
Notation
• Analogy with real numbers
 f(n) =O(g(n))  f≤g
 f(n) =Ω(g(n))  f≥g
 f(n) =θ(g(n))  f=g
 f(n) =o(g(n))  f<g
 f(n) =ω(g(n))  f>g
Polynomial and Intractable
A l go r i t h ms
• Polynomial Time complexity
– An algorithm is said to be polynomial if it is O( nd )
for some integer d
– Polynomial algorithms are said to be efficient
• They solve problems in reasonable times!
• Intractable Algorithms
– Algorithms for which there is no known
polynomial time algorithm.

You might also like