Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
22 views

2-CST3011 - Algorithms and Data Structures-Algorithm Analysis

Lecture 2 for algorithms course

Uploaded by

Ben Aiah
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
22 views

2-CST3011 - Algorithms and Data Structures-Algorithm Analysis

Lecture 2 for algorithms course

Uploaded by

Ben Aiah
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 34

CST3011 – Algorithms and Data

Structures
Lecture II: Algorithm Analysis
Objectives
 At the end of this lesson, students are
expected to be able to
 Use the mathematical framework that more
rigorously describe the running time of an
algorithm

 Estimate the time required for a program for a


given input size N

 Apply techniques that drastically reduce the


running time of an algorithm
Outline
 Introduction
 Why Analyse Algorithms?
 Experimental studies
 Asymptotic Analysis
 Model
 Expressing Algorithms
 Running Time Calculations
 Asymptotic Notation
Introduction
 An algorithm is a clearly specified set of simple
instructions to be followed to solve a problem

 Criteria for judging an algorithm are


 Correctness: testing, verification
 Efficiency: usage of resources, the main subject of the
course
 Solves the problem but requires a year
 Solves the problem takes over the entire resources
 Simplicity: design and implementation

 Once the algorithm is determined to be correct, we


determine the amount of resources required:
Algorithm Analysis
Algorithm Analysis
 The running time of an algorithm, is a function of the
input
 Sorting 10000 elements require more time than sorting 10

 The exact values of these functions depend on


several factors
 Speed of the host machine
 Quality of the compiler
 Quality of the program etc.

 Analysis can be done by


 experimental studies
 Theoretical analysis
Experimental Studies
 Write a program implementing the algorithm

 Run the program with inputs of varying size

 Get the actual running time


 Use System.currentTimeMillis() for instance

 Plot the results

 Limitations
 The necessity to implement the algorithm
 Results may not be indicative of the running time on other
inputs not included in the experiment
 To compare two algorithms, the same hardware must be used
Theoretical Analysis
 Use a high level description of the algorithm
instead of implementation

 Takes into account all possible inputs

 Allows evaluation of the speed of the


algorithm independent of the
hardware/software environment
Why Analyse?
 To discover characteristics for evaluation or comparing
 In this course, we care most about asymptotic performance
 How does the algorithm behave as the problem size gets very large?
 *Running time
 Other characteristics
 Memory/storage requirements
 Bandwidth/power requirements/logic gates/etc.

 We define two functions


 Tavg(N) as average running time and
 Tworst(N) as the worst running time
 Clearly Tavg(N) ≤ Tworst(N)

 We focus on the worst case


 Easier to analyse
 Provides an upper bound
 An absolute guarantee, crucial to applications like games, finance and robotics
Model
 Analysis is performed with respect to a
computational model

 We will usually use a generic uniprocessor


random-access machine (RAM)
 All memory equally expensive to access
 No concurrent operations
 All reasonable instructions take unit time
 Except, of course, function calls
Algorithm description
 Algorithms can be expressed in one of the following medium,
depending on the level of expressiveness that is intended
 Plain English: most natural but least precise language
 Pseudocode: programming language which never complains about
syntax errors
 Programming language e.g. C, Java: most precise but difficult

 Pseudocode is a high-level description of an algorithm


 More structure than English prose
 Less detailed than programming language
 Preferred notation for describing algorithms, hides program design
issues
 There is no single standard for Pseudocode.

 We, for consistency sake, are going to define our own


pseudocode, CSC3011CODE based on Pascal/Basic syntax
CSC3011CODE Definition in BNF
<algorithm> ::= Algorithm <id> (<arglist>) <body> End.
<arg_list>::= <letter> , <arg_list> | <empty>
<body> ::= Input : <input_desc>
Output: <output desc>
<stmt_list>

<stmt_list> ::= <var_decl>; <stmt_list>


|<assign>; <stmt_list>
|<empty>
<var_decl>::=<letter> <letters>:<type>;
<letters>::=, <letter><letters>|<empty>
<assign>::=<letter> := <exp>
<exp>::=<letter>+<exp>|<letter>*<exp>|<empty> ,
(<exp>)etc
CST3011CODE Definition
 Loops
 For . . Do . . Next statement
 Repeat . . Until . .
 Selection
 If . . Then . . Else statement
 Case statement
 Array indexing starts at 0
 Indentation represents the block structures
 N2 superscripts and other mathematical
formatting allowed
 Other operators
  for assignment,  not equal,  comparison etc
Example
Algorithm sumOfSquares(M, N)
Input: integers M and N
Output: the sum of M2 and N2

P := M*M; 2

Q:= N*N; 2
Return P + Q; 2
End.
6 operations
Primitive operations
 Basic computation performed by an algorithm
 Identifiable in pseudocode
 Largely independent from the programming
language
 Exact definition not important
 Examples:
 Evaluating an expression
 Assigning a value to a variable
 Indexing into an array
 *Calling a method
 Return from a method
Rules for counting operators
 Rule 1 – For loops:
 The running time of the for loop is at most the running time of the
statements inside the for loop (including tests) times the number of
iterations. E.g:
For i  1 to n – 1 Do 1+ n – 1 = n
s  s + A[i] 3(n – 1) = 3n – 3,
Next n-1 total 5n – 4 ops
 Rule 2 – Nested loops:
 Analyse these inside out. The total running time of a statement
inside a group of nested loops is the running time of the
statement multiplied by the product of the sizes of all the
loops: E.g.
For i  0 to n – 1 1+n
For j  0 to m – 1 (1 + m)(n)
A[i][j]  i * j 4 (m)(n)
Next (m)(n)
Next (n)
Rules for counting operators cont’d
 Rule 3: Consecutive statements
 These just add

 Rule 4: If . . Then . . Else


 Consider If <cond> Then <S1> Else <S2>
 The running time of this statement is never more than
the running time of the test plus the larger of the
running times of S1 and s2: E.g.
Algorithm f(n)
If n  1 Then
Return 1
Else
Return n * f(n-1)
End.
Running time is O(n) (Recurrence problem)
Counting Primitive Operations
 Inspect the Pseudocode to determine the
maximum number of primitive operations executed
by an algorithm, as a function of the input size
 Example: # of operations
Algorithm arrayMax(A, n)
Input: A, an array of size n
Output: x, the largest value in A
current Max  A[0] 2
For i  1 to n – 1 do n-1+1 = n
If A[i]> currentMax then 2(n – 1)
currentMax  A[i] 2(n – 1)
End For
Return currentMax 1
End.
Total operations = 5n-1
Estimating Running Time
 Algorithm arrayMax executes 5n-1 primitive
operations in the worst case scenario

 Define:
 T(N) worst case time function dependant on N the
input
 A, Time taken by the fastest operation
 B ,Time taken by the slowest operation

 We have
 A(6N – 3) T(N)B(6N – 3)
 Therefore, T(N) is bound by two linear
functions
Growth Rate of Running Time
 Changing the hardware/software environment
 Affects T(N) by a constant factor
 Does not alter the growth rate of T(N)

 The Linear growth rate of running time T(N) is


a intrinsic property of the algorithm arrayMax
Growth Rates
 Popular growth rates
of functions in order: 5000

 Logarithmic 4000
 Linear f(n) = n
f(n) = log(n)
3000
 Quadratic f(n) = n log(n)
f(n) = n^2
 Cubic 2000 f(n) = n^3
f(n) = 2^n

1000
 Others include
0
 LogLogN 1 3 5 7 9 11 13 15 17 19

 Nlog(N)
 Exponential
Asymptotic Notation
 The analysis required to estimate resource
use is a theoretical issue

 We use the asymptotic (big-O) notation:


 What does O(n) running time mean? O(n 2)? O(n lg
n)?

 Our first task is to define this notation more


formally and completely
Asymptotic Notation
 We will use the following four definitions

 T(N) =  (f(N)) if  c,n0  +,  T(N)  cf(N)  N  n0

 T(N) =  (f(N)) if  c,n0  +,  T(N)  cf(N)  N  n0

 T(N) =  (f(N)) T(N) =  (f(N)) & T(N) =  (f(N))

 T(N) =  (f(N)) if  c,n0  +,  T(N)  cf(N)  N  n0

 These definitions establish a relation order among


functions: i.e. the relative rates of growth of
functions
Example
 1000N = (N2) i.e. N2 grows more rapidly than 1000N and
will eventually be larger than 1000N.

 If g(N) = 100N2 + 20N + 200 then g(N) = (N2) = (N3) =


(N4)

 2N + 10 = (N)

 Since:
 2N + 10  cN
 (c – 2)N  10
 N  10/(c – 2) for c = 3 and n0 = 10, it is true
 Exercise:
 Show that g(N) =  (f(N))  f(N) =  (g(N))
Big-Oh Rules and Growth Rates
 The Big-Oh notation gives a upper bound on
the growth rate of the function

 The statement “f(n) is O(g(n))”, simply means


that the growth rate of f(n) is no more than
the growth rate of g(n)

 We can use the Big-Oh notation to rank


functions according to their growth rate
Big-Oh Rules
 Rule 1:
 If T1(n) = O(f(n)) and T2(N) = O(g(n)), then
 T1(n) + T2(n) = max(O(f(n), O(g(n))

 Rule II
 If f(n) is a polynomial of order k, then f(n) is (nk) i.e.
 Rule III
 Logk(n) = O(n) for any constant k

 Rule of Thumb: Drop the lower-order terms


 Drop constant factors
 Say 2n + 4 is O(n) not (O(2n) or O(2n + 4)

 Use the smallest possible class of functions


 Say “2n is O(n)” instead of “2n is O(n 2)
Asymptotic Algorithm Analysis
 The Asymptotic analysis of an algorithm determines
the running time in Big-Oh notation
 To perform the Asymptotic analysis:
 We find the worst-case number of primitive operations
executed as a function of the input size
 We express this function with the Big-Oh notation
 Example:
 We determine that arrayMax executes at most n – 1
operations
 We say arrayMax runs O(n) time
 Since the constant factors and lower-order terms are
eventually dropped anyway, we can disregard them
when counting primitive operations – but don’t!!!
Example: Prefix averages
 We illustrate asymptotic analysis with two
algorithms for prefix averages

 The i-th prefix average is the average of the


array X is the average of the first i elements of
X
 A[i] = [X[0]+X[1]+ . . +X[i]]/(i+1)

 Computing the array A of prefix averages of


another array X has applications to financial
analysis
Solution I
 Pseudo code
Algorithm prefixAverages(A, n)
Input: an array A of size n
Output: X, an array of prefix averages of X

X  new array of size n n


For i  0 to n – 1 do 1+n
S  A[0] 2n
For j = 1 to i do 1 + 2 + .. + (n-1)
S  S + A[j] 3(1 + 2 + .. + (n-1))
End For
X[i]  S/(i + 1) 4n
End For
Return X 1
End.
Solution I: Digest
 Arithmetic Progression
 Quadratic order
Solution II
 Pseudocode
Algorithm prefixAverages2(A, n)
Input
Output

X  new array of size n


s0
For i  0 to n – 1 do
s  s + A[i]
X[i]  s/(i + 1)
Next
Return A
End.
Growth rate by limit
 To determine the growth rate of two function
f(N) and g(N), compute

 This limit has four possible values


 Limit = 0  f(N) = (g(N)
 Limit = c  0  f(N) = (g(N))
 Limit =   g(N) = (f(N))
 Limit oscillates: There is no relation
L’Hôpitals Rule
 In determining the limit of a quotient of f(N)
and g(N) as n tends to infinity, you can use
the L’Hôpitals Rule, which states

 Exercise: Show that log(N) = o(N), using the


L’Hôpitals Rule.
Example
 Show that log(n) = o(N)

 Soln:
??QUESTIONS??

You might also like