Algorithm Analysis & Design Module I Intoduction to Algorithms

1. Algorithms
An algorithm is a type of effective method in which a definite list of well-defined
instructions for completing a task; that given an initial state, will proceed through a well-
defined series of successive states, eventually terminating in an end-state !he concept of
an algorithm originated as a means of recording procedures for solving mathematical
pro"lems such as finding the common divisor of two num"ers or multiplying two
num"ers
Algorithms are named for the #
th
century $ersian mathematician Al-%howari&mi 'e
wrote a treatise in Ara"ic in ()* AD, On Calculation with Hindu Numerals. It was
translated into +atin in the ,)th century as Algoritmi de numero Indorum, which title was
likely intended to mean -./ook "y0 Algoritmus on the num"ers of the Indians-, where
-Algoritmi- was the translator1s rendition of the author1s name in the genitive case; "ut
people misunderstanding the title treated Algoritmi as a +atin plural and this led to the
word -algorithm- 2+atin algorismus3 coming to mean -calculation method-
,, Algorithm 4pecification
!he criteria for any set of instruction for an algorithm is as follows5
Input 5 6ero of more 7uantities that are e8ternally applied
9utput 5 At least one 7uantity is produced
Definiteness 5 :ach instruction should "e clear and unam"iguous
;initeness 5 Algorithm terminates after finite num"er of steps for all test
cases
:ffectiveness 5 :ach instruction is "asic enough for a person to carried out
using a pen and paper !hat means ensure not only definite "ut also check whether
feasi"le or not
,) Algorithm <lassification
!here are various ways to classify algorithms !hey are as follows
,), Classification by implementation
Recursion or iteration5 A recursive algorithm is one that invokes 2makes reference
to3 itself repeatedly until a certain condition
matches, which is a method common to functional programming Iterative algorithms
use repetitive constructs like loops and sometimes additional data structures like
stacks to solve the given pro"lems 4ome pro"lems are naturally suited for one
implementation or the other ;or e8ample, towers of hanoi is well understood in
recursive implementation :very recursive version has an e7uivalent 2"ut possi"ly
more or less comple83 iterative version, and vice versa
Logical5 An algorithm may "e viewed as controlled logical deduction !his notion
may "e e8pressed as5
Algorithm = logic + control
!he logic component e8presses the a8ioms that may "e used in the computation and
the control component determines the way in which deduction is applied to the
a8ioms !his is the "asis for the logic programming paradigm In pure logic
Prepared by Sminu Izudheen, DCE, RASET
1
Algorithm Analysis & Design Module I Intoduction to Algorithms
programming languages the control component is fi8ed and algorithms are specified
"y supplying only the logic component !he appeal of this approach is the elegant
semantics5 a change in the a8ioms has a well defined change in the algorithm
Serial or parallel or distributed5 Algorithms are usually discussed with the
assumption that computers e8ecute one instruction of an algorithm at a time !hose
computers are sometimes called serial computers An algorithm designed for such an
environment is called a serial algorithm, as opposed to parallel algorithms or
distri"uted algorithms $arallel algorithms take advantage of computer architectures
where several processors can work on a pro"lem at the same time, whereas distri"uted
algorithms utilise multiple machines connected with a network $arallel or distri"uted
algorithms divide the pro"lem into more symmetrical or asymmetrical su"pro"lems
and collect the results "ack together !he resource consumption in such algorithms is
not only processor cycles on each processor "ut also the communication overhead
"etween the processors 4orting algorithms can "e paralleli&ed efficiently, "ut their
communication overhead is e8pensive Iterative algorithms are generally
paralleli&a"le 4ome pro"lems have no parallel algorithms, and are called inherently
serial pro"lems
Deterministic or non-deterministic5 Deterministic algorithms solve the pro"lem with
e8act decision at every step of the algorithm whereas non-deterministic algorithm
solves pro"lems via guessing although typical guesses are made more accurate
through the use of heuristics
!act or appro!imate5 =hile many algorithms reach an e8act solution,
appro8imation algorithms seek an appro8imation that is close to the true solution
Appro8imation may use either a deterministic or a random strategy 4uch algorithms
have practical value for many hard pro"lems
,)) Classification by Design "aradigm
Di#ide and con$uer A divide and con7uer algorithm repeatedly reduces an instance
of a pro"lem to one or more smaller instances of the same pro"lem 2usually
recursively3, until the instances are small enough to solve easily 9ne such e8ample of
divide and con7uer is merge sorting 4orting can "e done on each segment of data
after dividing data into segments and sorting of entire data can "e o"tained in con7uer
phase "y merging them A simpler variant of divide and con7uer is called decrease
and con7uer algorithm, that solves an identical su" pro"lem and uses the solution of
this su" pro"lem to solve the "igger pro"lem Divide and con7uer divides the pro"lem
into multiple su" pro"lems and so con7uer stage will "e more comple8 than decrease
and con7uer algorithms An e8ample of decrease and con7uer algorithm is "inary
search algorithm
Dynamic programming =hen a pro"lem shows optimal su"structure, meaning the
optimal solution to a pro"lem can "e constructed from optimal solutions to su"
pro"lems, and overlapping su" pro"lems, meaning the same su" pro"lems are used to
solve many different pro"lem instances, a 7uicker approach called dynamic
programming avoids recomputing solutions that have already "een computed ;or
e8ample, the shortest path to a goal from a verte8 in a weighted graph can "e found "y
using the shortest path to the goal from all ad>acent vertices Dynamic programming
and memoi&ation go together !he main difference "etween dynamic programming
and divide and con7uer is that su" pro"lems are more or less independent in divide
and con7uer, whereas su" pro"lems overlap in dynamic programming !he difference
Prepared by Sminu Izudheen, DCE, RASET
2
Algorithm Analysis & Design Module I Intoduction to Algorithms
"etween dynamic programming and straightforward recursion is in caching or
memoi&ation of recursive calls =hen su" pro"lems are independent and there is no
repetition, memoi&ation does not help; hence dynamic programming is not a solution
for all comple8 pro"lems /y using memoi&ation or maintaining a ta"le of su"
pro"lems already solved, dynamic programming reduces the e8ponential nature of
many pro"lems to polynomial comple8ity
%he greedy method A greedy algorithm is similar to a dynamic programming
algorithm, "ut the difference is that solutions to the su" pro"lems do not have to "e
known at each stage; instead a -greedy- choice can "e made of what looks "est for the
moment !he greedy method e8tends the solution with the "est possi"le decision 2not
all feasi"le decisions3 at an algorithmic stage "ased on the current local optimum and
the "est decision 2not all possi"le decisions3 made in previous stage It is not
e8haustive, and does not give accurate answer to many pro"lems /ut when it works,
it will "e the fastest method !he most popular greedy algorithm is finding the
minimal spanning tree as given "y %ruskal
Linear programming =hen solving a pro"lem using linear programming, specific
ine7ualities involving the inputs are found and then an attempt is made to ma8imi&e
2or minimi&e3 some linear function of the inputs Many pro"lems 2such as the
ma8imum flow for directed graphs3 can "e stated in a linear programming way, and
then "e solved "y a 1generic1 algorithm such as the simple8 algorithm
Reduction !his techni7ue involves solving a difficult pro"lem "y transforming it
into a "etter known pro"lem for which we have 2hopefully3 asymptotically optimal
algorithms !he goal is to find a reducing algorithm whose comple8ity is not
dominated "y the resulting reduced algorithm1s ;or e8ample, one selection algorithm
for finding the median in an unsorted list involves first sorting the list 2the e8pensive
portion3 and then pulling out the middle element in the sorted list 2the cheap portion3
!his techni7ue is also known as transform and conquer
Search and enumeration Many pro"lems 2such as playing chess3 can "e modeled as
pro"lems on graphs A graph e8ploration algorithm specifies rules for moving around
a graph and is useful for such pro"lems !his category also includes search
algorithms, "ranch and "ound enumeration and "acktracking
%he probabilistic and heuristic paradigm Algorithms "elonging to this class fit the
definition of an algorithm more loosely
Probabilistic algorithms are those that make some choices randomly 2or pseudo-
randomly3; for some pro"lems, it can in fact "e proven that the fastest solutions
must involve some randomness
Genetic algorithms attempt to find solutions to pro"lems "y mimicking "iological
evolutionary processes, with a cycle of random mutations yielding successive
generations of -solutions- !hus, they emulate reproduction and -survival of the
fittest- In genetic programming, this approach is e8tended to algorithms, "y
regarding the algorithm itself as a -solution- to a pro"lem
Heuristic algorithms, whose general purpose is not to find an optimal solution, "ut
an appro8imate solution where the time or resources are limited !hey are not
practical to find perfect solutions An e8ample of this would "e local search, or
simulated annealing
Prepared by Sminu Izudheen, DCE, RASET
3
Algorithm Analysis & Design Module I Intoduction to Algorithms
1.& Recursi#e Algorithms
A recursive algorithm is an algorithm which calls itself with -smaller 2or simpler3-
input values, and which o"tains the result for the current input "y applying simple
operations to the returned value for the smaller 2or simpler3 input More generally if a
pro"lem can "e solved utili&ing solutions to smaller versions of the same pro"lem, and
the smaller versions reduce to easily solva"le cases, then one can use a recursive
algorithm to solve that pro"lem ;or e8ample, the elements of a recursively defined set, or
the value of a recursively defined function can "e o"tained "y a recursive algorithm
?ecursive computer programs re7uire more memory and computation compared with
iterative algorithms, "ut they are simpler and for many cases a natural way of thinking
a"out the pro"lem

;or e8ample consider factorial of a num"er, n
n@ A nB2n-,3B2n-)3BB)B,, and that C@ A ,
In other words,
;unction to calculate the factorial can "e written as
int factorial2int n3
D
if 2n AA C3
return ,;
else
return 2n B factorial2n-,33;
E
factorial2C3AF ,
factorial2G3
G B factorial2)3
G B ) B factorial2,3
G B ) B , B factorial2C3
G B ) B , B ,
AF H
!his corresponds very closely to what actually happens on the e8ecution stack in the
computer1s memory
Prepared by Sminu Izudheen, DCE, RASET
4
Algorithm Analysis & Design Module I Intoduction to Algorithms
1.' Space and %ime Comple!ity
!wo important ways to characteri&e the effectiveness of an algorithm are its space
comple8ity and time comple8ity
1.'.1 Space Comple!ity
4pace comple8ity of an algorithm is the amount to memory needed "y the program
for its completion 4pace needed "y a program has the following components5
1. (nstruction Space
4pace needed to store the compiled version of program It depends on
i <ompiler used
ii 9ptions specified at the time of compilation
eg, whether optimi&ation specified, Is there any overlay option etc
iii !arget computer
eg, ;or performing floating point arithmetic, if hardware present or not
). Data Space
4pace needed to store constant and varia"le values It has two components5
i 4pace for constants5
eg, value IGJ in program ,,
4pace for simple varia"les5
eg, varia"les a,",c in program ,,
"rogram 1.1
int add 2int a, int ", int c3
D
return 2aK"Kc3LG;
E

ii 4pace for component varia"les like arrays, structures, dynamically
allocated memory
eg, varia"les a in program ,)
"rogram 1.)
int ?add 2int a.0, int n3
, D
) If 2nFC3
G return ?add 2a, n-,3 K a.n-,0;
M else
* return C;
H E
Prepared by Sminu Izudheen, DCE, RASET
5
Algorithm Analysis & Design Module I Intoduction to Algorithms
&. n#ironment stac* space
:nvironment stack is used to store information to resume e8ecution of partially
completed functions =hen a function is invoked, following data are stored in
:nvironment stack
i ?eturn address
ii Nalue of local and formal varia"les
iii /inding of all reference and constant reference parameters
4pace needed "y the program can "e divided into two parts
i ;i8ed part independent of instance characteristics :g, code space, simple
varia"les, fi8ed si&e component varia"les etc
ii Naria"le part 4pace for component varia"les with space depends on
particular instance Nalue of local and formal varia"les
'ence we can write the space comple8ity as
42$3 A c K 4
p
2instance characteristics3
!ample 1.1
?efer $rogram ,,
9ne word for varia"les a,",c Oo instance characteristics 'ence 4
p
2!<3 A C
!ample 1.)
"rogram 1.&
int Aadd 2int Ba, int n3
, D
) int sAC;
G for 2iAC; iPn; iKK3
M sK A a.i0;
* return s;
H E
9ne word for varia"les n and i 4pace for a.0 is address of a.C0 'ence it re7uires one
word Oo instance characteristics 'ence 4
p
2!<3 A C
!ample 1.&
?efer $rogram ,)
Instance characteristics depend on values of n ?ecursive stack space includes space
for formal parameters, local varia"les and return address 4o one word each for a.0,n,
return address and return varia"les 'ence for each pass it needs M words !otal recursive
stack space needed is M2n3
'ence 4
p
2!<3 A M2n3
1.'.) %ime Comple!ity
!ime comple8ity of an algorithm is the amount of time needed "y the program for its
completion !ime taken is the sum of the compile time and the e8ecution time <ompile
time does not depend on instantaneous characteristics 'ence we can ignore it
Prepared by Sminu Izudheen, DCE, RASET
6
Algorithm Analysis & Design Module I Intoduction to Algorithms
$rogram step5 A program step is syntactically or semantically meaningful segment of a
program whose e8ecution time is independent of instantaneous characteristics =e can
calculate comple8ity in terms of
, <omments5
Oo e8ecuta"les, hence step count A C
) Declarative 4tatements5
Define or characteri&e varia"les and constants like 2int , long, enum, Q3
4tatement ena"ling data types 2class struct union template3
Determine access statements 2 public pri!ate protected friend 3
<haracter functions 2 !oid !irtual 3
All the a"ove are non e8ecuta"les, hence step count A C
G :8pressions and Assignment 4tatements5
4imple e8pressions 5 4tep count A , /ut if e8pressions contain function call, step
count is the cost of the invoking functions !his will "e large if parameters are passed
as call "y value, "ecause value of the actual parameters must assigned to formal
parameters
Assignment statements 5 Reneral form is Pvaria"leF A Pe8prF 4tep count A e8pr,
unless si&e of Pvaria"leF is a function of instance characteristics eg, a A ", where a
and " are structures In that case, 4tep count A si&e of Pvaria"leF K si&e of P e8pr F
M Iterative 4tatements5
+hile Pe8prF do
Do .. +hile ,e8pr-
4tep count A Oum"er of step count assigna"le to Pe8prF
.or 2Pinit-stmtF; Pe8pr,F; Pe8pr)F3
4tep count A ,, unless the Pinit-stmtF, Pe8pr,F,Pe8pr)F are function of instance
characteristics If so, first e8ecution of control part has step count as sum of count of
Pinit-stmtF and Pe8pr,F ;or remaining e8ecutions, control part has step count as
sum of count of Pe8pr,F and Pe8pr)F
* 4witch 4tatements5
S/itch 2Pe8prF3 D
Case cond, 5 Pstatement,F
Case cond) 5 Pstatement)F


Default 5 PstatementF
E
S/itch 2Pe8prF3 has step count A cost of Pe8prF
<ost of Cond statements is its cost plus cost of all preceding statements
H If-else 4tatements5
(f 2Pe8prF3 Pstatement,F;
Prepared by Sminu Izudheen, DCE, RASET
7
Algorithm Analysis & Design Module I Intoduction to Algorithms
lse Pstatement)F;
4tep count of (f and lse is the cost of Pe8prF
S ;unction invocation5
All function invocation has 4tep count A ,, unless it has parameters passed as called
"y value which depend s on instance characteristics If so, 4tep count is the sum of the
si&e of these values
If function "eing invoked is recursive, consider the local varia"les also
( Memory management 4tatements5
ne/ o">ect, delete o">ect, si0eof2o">ect3, 4tep count A,
# ;unction 4tatements5
4tep count A C, cost is already assigned to invoking statements
,C Tump 4tatements5
continue1 brea*1 goto has 4tep count A,
return Pe8prF5 4tep count A,, if no e"pr which is a function of instance
characteristics If there is, consider its cost also
!ample 1.'
?efer $rogram ,)
Introducing a counter for each e8ecuta"le line we can rewrite the program as
int ?add 2int a.0, int n3
D
countKK LL if
If 2nFC3
D
countKK LL return
return ?add 2a, n-,3 K a.n-,0;
E
else
D
countKK LL return
return C;
E
E
<ase ,5 nAC
t
?add
A )
<ase )5 nFC
) K t
?add
2n-,3
A ) K ) K t
?add
2n-)3
A ) B ) K t
?add
2n-)3

A )n K t
?add
2C3
A )n K )
Prepared by Sminu Izudheen, DCE, RASET

Algorithm Analysis & Design Module I Intoduction to Algorithms

!ample 1.2
"rogram 1.'
int Madd 2int a.0.0, int ".0.0, int c.0.0, int n3
, D
) ;or 2int iAC; iPm; iKK3
G ;or 2int >AC; >Pn; >KK3
M c.i0.>0 A a.i0.>0 K ".i0.>0;
* E
Introducing a counter for each e8ecuta"le line we can rewrite the program as
int Madd 2int a.0.0, int ".0.0, int c.0.0, int n3
D
;or 2int iAC; iPm; iKK3
D
countKK LLfor i
;or 2int >AC; >Pn; >KK3
D
countKK LLfor >
c.i0.>0 A a.i0.>0 K ".i0.>0;
countKK LLfor assignment
E
countKK LLfor last >
E
countKK LLfor last i
E
4tep count is )mn K )m K,
4tep count does not reflect the comple8ity of statement It is reflected in step per
e8ecution 2sLe3
?efer $rogram ,)
Line s3e .re$uency %otal Steps
n=4 n-4 n=4 n-4
, C , , C C
) , , , , ,
G , K t
?add
2n-,3 C , C , K t
?add
2n-,3
M C , C C C
* , , C , C
%otal no. of steps ) ) + t
Radd
5n-16
?efer $rogram ,G
Line s3e .re$uency %otal Steps
, C , C
) , , ,
G , nK, nK,
M , n n
* , , ,
H C , C
%otal no. of steps )n + &
Prepared by Sminu Izudheen, DCE, RASET
!
Algorithm Analysis & Design Module I Intoduction to Algorithms
?efer $rogram ,M
Line s3e .re$uency %otal Steps
, C , C
) , mK, mK,
G , m2nK,3 m2nK,3
M , mn mn
* C , C
%otal no. of steps )mn + )m + 1
1.2 Asymptotic 7otations
4tep count is to compare time comple8ity of two programs that compute same
function and also to predict the growth in run time as instance characteristics changes
Determining e8act step count is difficult and not necessary also 4ince the values are not
e8act 7uantities we need only comparative statements like c
,
n
)
U t
p
2n3 U c
)
n
)

;or e8ample, consider two programs with comple8ities c

,
n
)
K c
)
n and c
G
n respectively
;or small values of n, comple8ity depend upon values of c
,
, c
)
and c
G
/ut there will also
"e an n "eyond which comple8ity of c
G
n is "etter than that of c
,
n
)
K c
)
n!his value of n is
called "reak-even point If this point is &ero, c
G
n is always faster 2or at least as fast3
<ommon asymptotic functions are given "elow
.unction 7ame
, <onstant
log n +ogarithmic
n +inear
n log n n log n
n
)
Vuadratic
n
G
<u"ic
)
n
:8ponential
n@ ;actorial
1.2.1 8ig 9:h; 7otation 5:6
92g2n33 A D f2n3 5 there e8ist positive constants c and n
C
such that C U f2n3 U cg2n3 for all n
W n
C
E
It is the upper "ound of any function 'ence it denotes the worse case comple8ity of any
algorithm =e can represent it graphically as
Prepared by Sminu Izudheen, DCE, RASET
1"
Algorithm Analysis & Design Module I Intoduction to Algorithms
.ig 1.1
;ind the /ig I9hJ for the following functions5
Linear .unctions
!ample 1.<
f2n3 A Gn K )
Reneral form is f2n3 U cg2n3
=hen n W ), Gn K ) U Gn K n A Mn
'ence f2n3 A 92n3, here c A M and n
C
A )
=hen n W ,, Gn K ) U Gn K )n A *n
'ence f2n3 A 92n3, here c A * and n
C
A ,
'ence we can have different c,n
C
pairs satisfying for a given function
!ample 1.=
f2n3 A Gn K G
=hen n W G, Gn K G U Gn K n A Mn
'ence f2n3 A 92n3, here c A M and n
C
A G
!ample 1.>
f2n3 A ,CCn K H
=hen n W H, ,CCn K H U ,CCn K n A ,C,n
'ence f2n3 A 92n3, here c A ,C, and n
C
A H
?uadratic .unctions
!ample 1.@
f2n3 A ,Cn
)
K Mn K )
=hen n W ), ,Cn
)
K Mn K ) U ,Cn
)
K *n
=hen n W *, *n U n
)
, ,Cn
)
K Mn K ) U ,Cn
)
K n
)
A ,,n
)
'ence f2n3 A 92n
)
3, here c A ,, and n
C
A *
!ample 1.14
f2n3 A ,CCCn
)
K ,CCn - H
Prepared by Sminu Izudheen, DCE, RASET
11
Algorithm Analysis & Design Module I Intoduction to Algorithms
f2n3 U ,CCCn
)
K ,CCn for all values of n
=hen n W ,CC, *n U n
)
, f2n3 U ,CCCn
)
K n
)
A ,CC,n
)
'ence f2n3 A 92n
)
3, here c A ,CC, and n
C
A ,CC
!ponential .unctions
!ample 1.11
f2n3 A HB)
n
K n
)
=hen n W M, n
)
U )
n

4o f2n3 U HB)
n
K )
n
A SB)
n

'ence f2n3 A 92)
n
3, here c A S and n
C
A M
Constant .unctions
!ample 1.1)
f2n3 A ,C
f2n3 A 92,3, "ecause f2n3 U ,CB,
1.2.) :mega 7otation 5A6
X 2g2n33 A D f2n3 5 there e8ist positive constants c and n
C
such that C U cg2n3 U f2n3 for all n
W n
C
E
It is the lower "ound of any function 'ence it denotes the "est case comple8ity of any
algorithm =e can represent it graphically as
.ig 1.)
!ample 1.1&
f2n3 A Gn K )
Gn K ) F Gn for all n
'ence f2n3 A X2n3
4imilarly we can solve all the e8amples specified under /ig I9hJ
1.2.& %heta 7otation 5B6
Y2g2n33 A Df2n3 5 there e8ist positive constants c
,
,c
)
and n
C
such that c
,
g2n3 Uf2n3 Uc
)
g2n3
for all n W n
C
E
If f2n3 A Y2g2n33, all values of n right to n
C
f2n3 lies on or a"ove c
,
g2n3 and on or "elow
c
)
g2n3 'ence it is asymptotic tight "ound for f2n3
Prepared by Sminu Izudheen, DCE, RASET
12
Algorithm Analysis & Design Module I Intoduction to Algorithms
.ig 1.&
!ample 1.1'
f2n3 A Gn K )
f2n3 A Y2n3 "ecause f2n3 A 92n3 , n W )
4imilarly we can solve all e8amples specified under /igJ9hJ
1.2.' Little 9:h; 7otation 5o6
o2g2n33 A D f2n3 5 for any positive constants c F C, there e8ists n
C
FC, such that C U f2n3 P
cg2n3 for all n W n
C
E
It defines the asymptotic tight upper "ound Main difference with /ig 9h is that /ig 9h
defines for some constants c "y +ittle 9h defines for all constants
1.2.2 Little :mega 5C6
Z2g2n33 A D f2n3 5 for any positive constants cFC and n
C
FC such that C U cg2n3 P f2n3 for all
n W n
C
E
It defines the asymptotic tight lower "ound Main difference with X is that, Z defines for
some constants c "y Z defines for all constants
1.< Recurrence Relations
?ecurrence is an e7uation or ine7uality that descri"es a function in terms of its value on
smaller inputs, and one or more "ase cases
eg, recurrence for Merge-4ort
Prepared by Sminu Izudheen, DCE, RASET
13
Algorithm Analysis & Design Module I Intoduction to Algorithms
[ \seful for analy&ing recurrent algorithms
[ Make it easier to compare the comple8ity of two algorithms
[ Methods for solving recurrences
4u"stitution method
?ecursion tree method
Master method
Iteration method
1.<.1 Substitution Dethod
\se mathematical induction to derive an answer
Derive a function of n 2or other varia"les used to e8press the si&e of the pro"lem3
that is not a recurrence so we can esta"lish an upper andLor lower "ound on the
recurrence
May get an e8act solution or may >ust get upper or lower "ounds on the solution
Steps
Ruess the form of the solution
\se mathematical induction to find constants or show that they can "e found and
to prove that the answer is correct
!ample 1.12
;ind the upper "ound for the recurrence relation
!2n3 A ) !2 nL) 3 K n
Ruess the solution as !2n3 A 92nlg2n33
!hen !2n3 A cnlgn
4u"stituting in !2n3, we get
!2n3 A
A c n lg2nL)3 K n
A cn lg2n3 cnlg2)3 K n
A cn lg2n3 cn K n
A cn lg2n3, c FA,
!o prove using mathematical induction, we have to show that the solution holds for
"oundary condition also =e select "oundary condition as nFA) 2/ecause for n A ,,
!2,3 A c,lg2,3 A C which is false according to the definition of !2n33
!ample 1.1<
;ind the worse case comple8ity of /inary 4earch
!2n3 A c K !2nL)3
[ Ruess5 !2n3 A 92lgn3
Prepared by Sminu Izudheen, DCE, RASET
14

#2n3
2,3 if n A ,
)#2nL)3 K 2n3 if n F ,

'



Algorithm Analysis & Design Module I Intoduction to Algorithms
Induction goal5 !2n3 U d lgn, for some d and n W nC
Induction hypothesis5 !2nL)3 U d lg2nL)3
[ $roof of induction goal5
!2n3 A !2nL)3 K c U d lg2nL)3 K c
A d lgn d K c U d lgn
if5 d K c U C, d W c
!ample 1.1=
!2n3 A !2n-,3 K n
[ Ruess5 !2n3 A 92n)3
Induction goal5 !2n3 U c n), for some c and n W nC
Induction hypothesis5 !2n-,3 U c2n-,3) for all k P n
[ $roof of induction goal5
!2n3 A !2n-,3 K n U c 2n-,3) K n
A cn) 2)cn c - n3 U cn)
if5 )cn c n W C c W nL2)n-,3 c W ,L2) ,Ln3
;or n W , ) ,Ln W , any c W , will work
!ample 1.1=
!2n3 A )!2nL)3 K n
[ Ruess5 !2n3 A 92nlgn3
Induction goal5 !2n3 U cn lgn, for some c and n W nC
Induction hypothesis5 !2nL)3 U cnL) lg2nL)3
[ $roof of induction goal5
!2n3 A )!2nL)3 K n U )c 2nL)3lg2nL)3 K n
A cn lgn cn K n U cn lgn
if5 - cn K n U C c W ,
1.<.) Recursion tree Dethod
Main disadvantage of 4u"stitution method is that it is always difficult to come up
with a good guess
?ecursion tree method allows you make a good guess for the su"stitution method
Allows to visuali&e the process of iterating the recurrence
Steps
<onvert the recurrence into a tree
:ach node represents the cost of a single su" pro"lem somewhere in the set of
recursive function invocations
4um the costs within each level of the tree to o"tain a set of per-level costs
4um all the per-level costs to determine the total cost of all levels of the recursion
!ample 1.1>
!2n3 A G!2nLM3 K 2n
)
3
!2n3 A G!2nLM3 K cn
)
Prepared by Sminu Izudheen, DCE, RASET
15
Algorithm Analysis & Design Module I Intoduction to Algorithms
[ !he su" pro"lem si&e for a node at depth i is nLMi
=hen the su" pro"lem si&e is ,, nLMi A ,, iAlog
M
n
!he tree has logMnK, levels 2C, ,, ),, log
M
n3
[ !he cost at each level of the tree 2C, ,, ),, log
M
n-,3
Oum"er of nodes at depth i is G
i
:ach node at depth i has a cost of c2nLMi3
)
!he total cost over all nodes at depth i is G
i
c2nLMi3
)
A2GL,H3
i
cn
)
[ !he cost at depth log
M
n
Oum"er of nodes is
:ach contri"uting cost !2,3
!he total cost
$rove !2n3A92n)3 is an upper "ound "y the su"stitution method
!2n3 dn) for some constant d F C
Prepared by Sminu Izudheen, DCE, RASET
16
3 2
3 2
,G
,H
3 2
,H
G
,
,
3 2 3
,H
G
2
3 2 3
,H
G
2
3 2 3
,H
G
2 3
,H
G
2
,H
G
3 2
)
G log ) G log )
G log
C
)
G log
, log
C
)
G log ) , log ) ) ) )
M M
M
M
M
M M
n O
n cn n cn
n cn
n cn
n cn cn cn cn n #
i
i
n
i
i
n

+ +

+ <
+
+ + + + +

]
]
)
) )
) )
) )
)
,H
G
3 M L 2 G
M L G
3 M L 2 G 3 2
dn
cn dn
cn n d
cn n d
cn n # n #

+
+
+
+
G log log
M M
G n
n

3 2 3 , 2
G log G log
M M
n # n
Algorithm Analysis & Design Module I Intoduction to Algorithms
!ample 1.1@
=2n3 A )=2nL)3 K 2n
)
3
[ 4u"pro"lem si&e at level i is5 nL)i
[ 4u"pro"lem si&e hits , when , A nL)i i A lgn
[ <ost of the pro"lem at level i A 2nL)i3) Oo of nodes at level i A )i
[ !otal cost5
=2n3 A 92n)3
!ample 1.)4
=2n3 A =2nLG3 K =2)nLG3 K 92n3
[ !he longest path from the root to a leaf is5 n 2)LG3n 2)LG3) n Q ,
[ 4u"pro"lem si&e hits , when , A 2)LG3in iAlogGL)n
[ <ost of the pro"lem at level i A n
[ !otal cost5
=2n3 A 92nlgn3
Prepared by Sminu Izudheen, DCE, RASET
17
) )
C
)
, lg
C
) lg
, lg
C
)
) 3 2
)
,
,
,
3 2
)
,
)
,
3 , 2 )
)
3 2 n n O n n O n n n $
n
n $
i
i
n
i
i
n
n
i
i
+

,
_

,
_

n n
n
n n n n n n n
n
i
n
i
lg
) L G lg
,
) L G lg
lg
log ,
) L G ) L G
log
C
) L G
log
C


< + +
Algorithm Analysis & Design Module I Intoduction to Algorithms
!ample 1.)1
!2n3 A !2nLM3 K !2nL)3 K n)
Prepared by Sminu Izudheen, DCE, RASET
1
Algorithm Analysis & Design Module I Intoduction to Algorithms
1.<.& Daster Dethod
!he master method applies to recurrences of the form #2n3 A a #2nLb3 K f 2n3 ,
where a ,, b F ,, and f is asymptotically positive
Descri"e the running time of an algorithm that divides a pro"lem of si&e n into a
su" pro"lems, each of si&e n%b
!here are three common cases
Prepared by Sminu Izudheen, DCE, RASET
1!
Algorithm Analysis & Design Module I Intoduction to Algorithms
Case 1
<ompare f 2n3 with n
logba
If f 2n3 A O2n
logba e
3 for some constant e F C
ie, f 2n3 grows polynomially slower than n
logba

ie, f&n' is asymptotically smaller "y an n
e
factor
!hen Solution: #2n3 A Y2n
logba
3
Case )
<ompare f 2n3 with n
logba
If f 2n3 A Y 2n
logba
3
ie, f 2n3 and n
logba
grow at similar rates
!hen Solution: #2n3 A Y2n
logba
lgn3
Prepared by Sminu Izudheen, DCE, RASET
2"
Algorithm Analysis & Design Module I Intoduction to Algorithms
Case &
<ompare f 2n3 with n
logba
If f 2n3 A X2n
logba(e
3 for some constant e F C
ie, f 2n3 grows polynomially faster than n
logba

ie, f&n' is asymptotically larger "y an n
e
factor
!hen Solution: #2n3 A Y2f&n'3
!ample 1.))
!2n3A#!2nLG3 K n
aA#, "AG, f2n3 A n
<A4: ,5
!2n3 A 2n)3
!ample 1.)&
Prepared by Sminu Izudheen, DCE, RASET
21
3 2 3 2 3, 2
, # log ) # log log
G G

n O n f n n n
a
b
Algorithm Analysis & Design Module I Intoduction to Algorithms
#2n3 A M#2nL)3 K n
a ) M, b A ) nlogba A n); f 2n3 A n.
<A4: ,5 f 2n3 A O2n
) e
3 for e A ,
#2n3 A Y2n)3
!ample 1.)'
!2n3 A !2)nLG3 K ,
aA,, "AGL), f2n3A,
<ase )5
!2n3 A 2lg n3
!ample 1.)2
#2n3 A M#2nL)3 K n
)
a ) M, b A ) nlogba A n); f 2n3 A n).
<A4: )5 f 2n3 A V2n)lgCn3, that is, * A C
#2n3 A Y2n
)
lg n3
!ample 1.)<
!2n3 A G!2nLM3 K n lg n 2<ase G3
aAG, "AM, f2n3 A n lg n
<aseG5 e A C)
;or sufficiently large n, af2nL"3 A G2nLM3lg2nLM3 2GLM3n lg nAcf2n3 for cAGLM
!2n3 A 2n lg n3
!ample 1.)=
#2n3 A M#2nL)3 K n
G
a ) M, b A ) n
logba
A n
)
; f 2n3 A n
G
.
<A4: G5 f 2n3 A X2n) K e3 for e A , and M2cnL)3G ] cnG 2reg cond3 for c A ,L)
#2n3 A Y2n
G
3
Prepared by Sminu Izudheen, DCE, RASET
22
3 , 2 , 3 2 , ,
, log log
) L G
n f n n
a
b
3 2
S#G C G log log
M
n O n n
a
b
3 2 3 2
G log
M
+
n n f