Lecture 3: Recurrences.

Dr. Khaled Alburaihi

Assistant Prof & Head of the
Circle of Academic Quality and
Accreditation in SCC.
Head of the Dept. of MIS In QAU.
 Name
 Scientific Background
 Practical Background
General Rules
Syllabus
 Course Texts
 Design and Analysis Algorithms Texts Books :
 Introduction to Algorithms, 2nd edition by Corman, Rivest, Leiserson,
and Stein.
 Computer Algorithms: Introduction to Design and Analysis By Sara
Baase, 3rd Edition, Addison-Wesley Publishing Company
 Using internet Research.
Recurrences
Recurrences
 Definition: A recurrence is an equation or inequality
that describes a function in terms of its value on
smaller inputs.
 The worst-case running time T(n) of an algorithm

could be described as by the recurrence

 T(n) = aT(n/b) + f(n)

 where a≥1, b>1, and f(n) is a given function

Recurrences
 Two methods will be discussed to solve the recurrences
obtaining asymptotic bounds on the solution
 Substitution method:

 We guess a bound and then use mathematical induction to

prove that our guess is correct.

 Recursion-tree method:

 It converts the recurrence into a tree whose nodes

represent the cost incurred at various levels of the recursion

 It’s possible to uses techniques for bounding summations

to solve the recurrence

Induction
Definition of Proof: The process or an instance of
establishing the validity of a statement especially by
derivation from other statements in accordance with
principles of reasoning.
 Proof by Induction: Let Pn be a statement for all the

positive integers (n = 1, 2, 3, . . .). If the following two

properties hold:
 P1 is true.
 Pk+1 is true if Pk is true for each positive integer k. Then Pn is
true for all n.
Why Induction Works?
 Let S be the set of all numbers n for which Pn is
false.
 Let k be the minimum number in S.

 k > 1 since by the first property of the induction

definition, P1 is true.
 By the minimality of k, Pk−1 is true and Pk is true.
Substitution
Method
A Summation Problem
 Prove that for any integer n ≥ 1:
 1 + 2 + 3 + · · · + n = n(n+1)/2 .
 Define:
 L(n) = 1 + 2 + 3 + · · · + n.
 R(n) = n(n+1)/2 .

 Prove that L(n) = R(n) for n ≥ 1.

A Summation Problem
 Verifying the Claim
 First check for small values of n
A Direct Proof
 Idea: Compute the value of 2L(n).
 Example:
 2(1 + 2 + 3) = (1 + 2 + 3) + (3 + 2 + 1)
= (1 + 3) + (2 + 2) + (3 + 1)
=3·4
= 12
= 2(3·4/2)
A Direct Proof
 Hence:
 2·L(n) = (1 + · · · + n) + (n + · · · + 1)
= (1 + n) + (2 + (n − 1)) + · · · + (n + 1)
= (n + 1) + (n + 1) + · · · + (n + 1)
= n(n + 1)
 This implies that: L(n) = n(n+1)/2 = R(n).
A Proof by Induction
 For n=1, L(1) = 1 and R(1) = (1.2)/2 = 1
So, for n=1, it is true.

 Let assume that it is true for n=k,

that is, L(k) = R(k)
=>1+2+3+…..+k = k(k+1)/2
 Now, if we can prove that it is true for n=k+1 also, then it can be said that it is true for all n.
A Proof by Induction
 For n=k+1,
L(k + 1) = 1 + 2 + · · · + k + (k + 1)
= L(k) + (k + 1)
= R(k) + (k + 1)
= k(k + 1)/2 + (k + 1)
= (k + 1)(k/2 + 1)
= (k + 1)(k + 2)/2
= R(k + 1)
 Hence L(n) = R(n), for all integer n.
Recursion-tree
Method
Recursion-tree Method
 Although the substitution method can provide
a concise proof that a solution to a recurrence
is correct, it is sometimes difficult to come up
with a good guess.
 Drawing out a recursion-tree is a straight

forward way to devise a good guess.

Recursion-tree Method
 In a recursion-tree:
 each node represents the cost of a single sub-
problem somewhere in the set of recursive
function invocations.
 sum the costs within each level of the tree to

obtain a set of per-level costs.

 Then sum all the per-level cost to determine the
total cost of all levels of the recursion.
Recursion-tree Method
 Recursion trees are particularly useful when the
recurrence describes the running time of a divide-
and-conquer algorithm.
 A recursion tree is best used to generate a
good guess, which is then verified by the
substitution method.
 In this section, we will use recursion-trees to

generate good guess.

 Example
Recursion-tree
Method
Recursion-tree Example
 Let see a recursion-tree would provide a good guess
for the recurrence:
 T(n) = 3T(n/4) + cn2
 Start by focusing on finding an upper bound for the
solution. Following figure shows the derivation of the
recursion-tree, assuming n as exact power of 4.
 Create a recursion tree for T(n) = 3T(n/4) + cn2, c>0.
Explanation
 Part (a) of the figure shows T(n), which is expanded in
 Part (b) into an equivalent tree representing the recurrences.
The cn2 term at the root represents the cost at the top level of
recursion, and the three subtrees of the root represent the
cost incurred by the subproblems of size n/4.
 Part (c) shows this process carried one step further by
expanding each node with cost T(n/4) from part (b). The cost
for each of the three children of the root is c(n/4)2.
Explanation (Cont.)
 We continue expanding each node in the tree by
breaking it into its constituent parts as determined by
the recurrence.
 Since the subproblem sizes decrease as we get
further from the root, we eventually must reach a
boundary condition.
 How far from the root do we reach one?
Explanation (Cont.)
 The sub problem size for a node at depth i is n/4i.
Thus the sub problem size hits n=1, when n/4i=1, that
is, i=log4 n.
 Thus the tree has levels (0, 1, 2, ……, log4 n).
 Now, we determine the cost at each level of the tree.
 Each level has three times more nodes than the level
above, So the number of nodes at depth i is 3i.
Explanation (Cont.)
 Each node at depth i has a cost of c(n/4i), for i=0, 1, 2, ….., log4n
-1.
 Multiplying, we get that total cost over all nodes at depth i is
3ic(n/4i)2 = (3/16)i cn2.
 The last level at, depth log4n, has 3log4n = nlog43 nodes, each
contributing cost T(1), for a total cost of nlog43 .T(1) , which is
Θ(nlog43).
 Now add up the costs over all levels to determine the cost for
the entire tree.
Explanation (Cont.)
2 log n  1 log 3
2 3 2  3 2  3 4 2 4 )
T ( n) cn  cn    cn  ....    cn  ( n
16  16   16 
log n  1 i
4  3 2  log 4 3 
    cn   n 

i 0  16   
  3 i 2  log 4 3 
    cn   n 
 
i 0  16   
1 2  log 4 3 
 cn   n 
1  (3 / 16)  
 
 n 2 .
 
Explanation (Cont.)
 Now, we can use the substitution method to verify that
our guess was correct, that is T(n)=O(n2) is an upper
bound for the recurrence
T ( n) 3T (  n / 4  )  ( n 2 )

 We want to show that T(n) ≤ dn2, for some d>0.

 Using the same constant c>0 as before, we have
Explanation (Cont.)
 Using the same constant c>0 as before, we have
T ( n) 3T (  n / 4  )  cn 2
2
3d  n / 4   cn 2
3d ( n / 4) 2  cn 2
3
 dn 2  cn 2
16
dn 2 , where d (16 / 3)c.
Hence, T ( n) ( n 2 ).
Explanation (Cont.)
 Let us see a recursion-tree would provide a good guess
for the recurrence:
 T(n) = T(n/3) + T(2n/3) + O(n).
Explanation (Cont.)
 Now, when we add the values across the levels of the
recursion-tree, we get a value of cn for every level.
 The longest path from root to a leaf is
 n→(2/3)n →(2/3)2n →….. →1.
 Since (2/3)kn =1, when k = log3/2n,
 The height of the tree is log3/2n.
 Intuitively, we expect the solution to the recurrence be
O(cn.log3/2n) = O(n.lg n)
Explanation (Cont.)
 We can use the substitution method to verify that our
guess was correct, that is T(n)=O(n.lg n) is an upper
bound for the given recurrence .
 We want to show that T(n) ≤ dn.lg n, for some d>0.
 Using the same constant c>0 as before, we have
 Explanation (Cont.)
T (n) T (n / 3)  T (2n / 3)  cn
d (n / 3) lg(n / 3)  d (2n / 3) lg(2n / 3)  cn
(d (n / 3) lg n  d (n / 3) lg 3)
 (d (2n / 3) lg n  d (2n / 3) lg(3 / 2))  cn
dn lg n  d (( n / 3) lg 3  (2n / 3) lg(3 / 2))  cn
dn lg n  d (( n / 3) lg 3  (2n / 3) lg 3  (2n / 3) lg 2)  cn
dn lg n  dn(lg 3  2 / 3)  cn
dn lg n.
Hence, T (n) (n lg n).
References
 Introduction to Algorithms, 2nd edition by Corman, Rivest,
Leiserson, and Stein.

 Computer Algorithms: Introduction to Design and Analysis

By Sara Baase, 3rd Edition, Addison-Wesley Publishing
Company

 Using internet Research.

‫خالد البريهي‪/‬‬
‫د‬