Solving Recurrences

SUBSTITUTION METHOD
ITERATION METHOD
RECURSION TREE
CHANGE OF VARIABLE
MASTERS THEOREM
• A recurrence is an equation that describes a
function in terms of itself by using smaller
inputs
 Substitution: Guess-and-Test
• Guess the form of the solution
• (Using mathematical induction) find the
constants and show that the solution works
 1. T(n)=T(n/2)+1
Guess: T(n)=0(log n)

T(n)  c log n
T(n/2)=c log(n/2)
T(n)=c log(n/2) +1
=c log n-clog 2+1
=clog n-c+1
if c>=2
T(n)  c log n
i.e. T(n)=O(log n)
 2. T(n) = 2T(n/2) + n = O(n lg n)

• T(n)  c n lg n
• assume
T(n/2)  c (n/2) lg (n/2)
T(n) = 2 T(n/2) + n
 2 (c (n/2) lg (n/2)) + n
= cn lg(n/2) + n
= cn lg n – cn lg 2 + n
= cn lg n – cn + n
 cn lg n for c  1
= O(n lg n)
 Example-3

 b if n  2
T(n)  
2T(n / 2)  bnlogn if n  2
Guess : T(n) =0(n log2 n)
T(n) <= c n log2 n.

T ( n )  2T ( n / 2)  bn log n
 2(c ( n / 2) log 2 ( n / 2))  bn log n
2
 cn (log n  log 2)  bn log n
2
 cn log n  2cn log n  cn  bn log n
2
 cn log n
If c>b

So, T(n) is O(n log2 n).

 ITERATION METHOD
– Expand the recurrence k times

– Work some algebra to express as a summation

– Evaluate the summation

(1)
 0 n0
T (n)  
c  T (n 1) n  0
• T(n) = c + T(n-1) T(0)=0
c + c + T(n-2)
2c + T(n-2)
2c + c + T(n-3)
3c + T(n-3)
…
k c + T(n-k) = ck + T(n-k)
If k=n
T(n) = c n + T(0) = c n O(n)
(2)

T(n)=T(n-1)+1
T(1)=Θ(1)
(3)
T(n)=T(n/2)+n
T(1)=Θ(1)
(4)

 0 n0
T(n)  
n T(n 1) n  0
  0 n 0
T(n)  
n T(n 1) n  0
• T(n) = n + T(n-1)
= n + n-1 + T(n-2)
= n + n-1 + n-2 + T(n-3)
= n + n-1 + n-2 + n-3 + T(n-4)
=…
= n + n-1 + n-2 + n-3 + … + (n-k+1) + T(n-k)

= for n  k

• To stop the recursion, we should have n - k = 0  k = n

n n
n 1
i 1
i  T (0)   i  0  n
i 1 2
n 1
T ( n)  n  O(n 2 )
2
(5)

 c n 1
 n
T ( n )   2T
 c n 1
  2 
• T(n) = 2 T(n/2) + c 1
= 2(2 T(n/2/2) + c) + c 2
= 22 T(n/22) + 2c + c
= 22(2 T(n/22/2) + c) + (22-1)c 3
= 23 T(n/23) + 4c + 3c
= 23 T(n/23) + (23-1)c
= 23(2 T(n/23/2) + c) + 7c 4
= 24 T(n/24) + (24-1)c
= …
= 2k T(n/2k) + (2k - 1)c k

• To stop the recursion, we should have

– n/2k = 1  n = 2k  k = lg n
– T(n) = n T(n/n) + (n - 1)c
= n T(1) + (n-1)c
= nc + (n-1)c
= nc + nc – c = 2cn – c
T(n) = 2cn – c = O(n)
(6)

T(n) = n + 2T(n/2)
T(n) = n + 2T(n/2)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
… = in + 2iT(n/2i)
= k n + 2kT(1)
= n lgn + n T(1) = Θ(n lgn)
(7)

T(n)=T(n/3) +c T(1)=1
T(n)= T(n/3) +c
T(n)= (T(n/9)+c)+c
T(n)= T(n/9)+2c
T(n)= T(n/27)+3c
……
……
T(n)=T(n/3k)+k c
Let n=3k  log3n=k
= T(n/n)+k c
= T(1)+log3nc
= c*log3n+1
=O(log3n)
(8)

T(n)= T(n-1)+n4 T(0)=0

T(n) =(T(n-2)+(n-1)4)+n4
=T(n-2)+(n-1)4+n4
=T(n-3)+(n-2)4+ (n-1)4+n4
=T(n-4)+(n-3)4+(n-2)4+ (n-1)4+n4
=T(n-k)+(n-k+1)4+(n-k+2)4+(n-k+3)4+……+n4
If k=n
=T( 0)+14+24+34+…..+n4
n 4
=0+ ∑ i
i=1

=
0 +n(n4)
=O(n5)
(9)

T(n)=4T(n/3)+n2
 T(n) =4(4T(n/32)+(n/3)2)+n2

= 42T(n/32)+4(n/3)2+n2
= 43T(n/33)+42(n/32)2+4(n/3)2+n2
=4kT(n/3k)+4k-1(n/3k-1)2+4k-2(n/3k-2)2+…+n2
Let n=3k
=4kT(1)+4k-1(3)2+4k-2(3)4+…+32k
= 4k+4k-1(3)2+4k-2(3)4+…+32k
=4k[ (3/2)0+(3/2)2+(3/2)4+…+(3/2)2k]
= 4k[32k/4k -1]
= 32k-4k
=32log3n-4log3n
=n2-nlog34
=n2-n0.79
=O(n2)
 (10)

T(n)=k T(n/k)+n2 T(1)=1 and k is any

constant.
Let k=2
=2(2T(n/22)+(n/2)2)+n2
= 22T(n/22)+2(n/2)2+n2
= 23T(n/23)+22T(n/22)2+2(n/2)2+n2
=2kT(n/2k)+2k-1T(n/2k-1)2+2k-2(n/2k-2)2+…+n2
Let n=2k
=2kT(1)+2k-1(2)2+2k-2(2)4+…+22k
= 2k[20+(2)2/2+(2)4/22+…+2k]
=2k[20+21+22+…+2k]
= 2k[(2k-1)/(2-1)]
=22k-2k
=n2-n
=O(n2)
(11)

T(n)=2T(n/2)+n logn
 =2[2T(n/22)+(n/2)log(n/2)]+n logn
=22T(n/22)+n log(n/2)+n logn
=22[2T(n/23)+n/22log(n/22)] +n log(n/2)+n logn
=23T(n/23)+n log(n/22)+n log(n/2)+n logn
=2kT(n/2k)+n[log2+log22+log23+…+log2k-1]
Let n=2k.
=n+nlog2[1+2+3+…+(k-1)]
= n+nlog2[((k)*(k-1))/2]
substituting k=log2n in above equation.
=n + c*n*[(log2n)(log2n-1)/2] [c=Log2]
=n+ c*n*(log2n)2
=O(n(log2n)2)
 (12)

1. T(n)=8T(n/2)+n2 T(1)=1
2. T(n)=3T(n/4)+n T(1)=1
32
33
RECURSION TREE METHOD
• Recursion-tree method works in those cases
– Write down the recurrence as a tree with recursive calls as the children
– Expand the children
– Add up each level
– Sum up the levels
• Useful for analyzing divide-and-conquer algorithms
• Also useful for generating good guesses to be used by substitution method
1. T(n)=3T(n/4)+cn 2
 T(n)=cn2+(3/16) cn2 +(3/16)2 cn2 +---------+Θ(nlog43)

log4 n1 i i
 3 2 
 3 2 1
 
T(n)     cn  n    cn  n 
log4 3
 
log4 3
3
 
cn2  nlog4 3  O(n2)
i0 16 i0 16
1
16
2. W(n) = 2W(n/2) + n2
 Sub problem size at level i is: n/2i
Sub problem size hits 1 when
n/2i=1  i = logn
No. of nodes at level i = 2i

lgn1 2 i
lgn1 i
n 1 
1 1
W (n)   i  2 W (1)  n    n  n    O(n) n
lgn 2 2 2
 O(n)  2n2
i 0 2 i 0  2  i 0  2  1 1
2

W(n) = O(n2)
3. W(n) = W(n/3) + W(2n/3) + n
The longest path from the root to a leaf is:
n  (2/3)n  (2/3)2 n  …  1

• Sub problem size hits 1 when 1 = (2/3)in  i=log3/2n

• Cost of the problem at level i = n

• Total cost:

lg n
W ( n )  n  n  ...  n (lo g 3 / 2 n)  n  O ( n lg n )
3
lg
2

W(n) = O(n log n)

4. T(n)=T(αn)+T((1-α)n)+n

where 0<α<1 and c>0

MASTERS THEOREM
 Let a>=1 and b>1be constants, let f (n) be a function, and let T(n) be
defined on non-negative integers by the recurrence:

T(n) = a T(n/b) + f(n) then

 

 
 n 
log b a
 
if f ( n )  O n log b a   

 
   0

T ( n )    n log b a log n  
if f ( n )   n 
log b a

  c 1
 
  f (n )  
if f ( n )   n 
log b a  
AND 
 
 af ( n / b )  cf ( n ) for large n 
1. if f (n) is O(nlogb a ), then T (n) is (nlogb a )
2. if f (n) is (nlogb a logk n), then T (n) is (nlogb a logk 1 n)
3. if f (n) is (nlogb a ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some  1.
 The master theorem – intuition

Compares two terms: O nlogb a and f (n) 
 
1. When O nlogb a dominates, the complexity is
T (n)  n logb a

2. When f (n) dominates, the complexity is
T (n)   f (n)
3. When they are comparable

 
T (n)   nlogb a lg n   f (n) lg n

Data Structures, Spring see book for formal proof!

2006 © L. Joskowicz 51
Master Method – Three Cases

Analysis of Algorithms 52
Master Method – Three Cases

Analysis of Algorithms 53
T ( n )  4T ( n / 2 )  n

a=4 b=2 f(n)=n

So, by Case 1 of the Master Method,

 T (n)  2T (n / 2)  n log n

• Solution: Let a=2, b = 2, k =1, and f(n) = nlog n.

•

• and, clearly, f(n) is Θ(n logn). Thus, by Case 2

of the Master Method, T(n) is Θ(n log2 n).
 T ( n )  T ( n / 3)  n

• Solution: Let a=1, b=3, ε =1, and f(n) =n

and f(n) = n is clearly in Ω(n). Moreover, a f(n/3) = 1*n/3 =

(1/3)n=(1/3)*f(n). .For c=1/3 the condition gets satisfied. Thus, the second
condition is met. By the 3rd case of the Master Method, T(n) is Θ(n).
. The master method cannot solve every recurrence of this form; there is a gap between cases 1 and 2, as well as
cases 2 and 3

You cannot use the Master Theorem if

1. f(n) is smaller than function, but NOT asymptotically smaller

2. f(n) is greater than function, but NOT asymptotically greater

3. a is not constant.

4. CASE 3 condition does not hold

• T(n) = 2T(n/2) + nlgn
– a=2, b=2, f(n) = nlgn
– n logba = n
– Since f(n) = nlgn is asymptotically larger than n logba = n
– CASE 3 applies:
– But f(n) = nlgn is NOT polynomially larger than n logba = n
– The ratio f(n) / n logba = (nlgn) / n is asymptotically less than n
For any positive constant 
– So the recurrence falls the gap between case 2 and case 3
What if the master theorem cannot be applied?
Extend the recurrence and compute the sum of the terms.
To cover the gap between 1-2 and 2-3, corollary below..
Corollary if f(n) is θ(nlogb(a) lgk(n)), then
T(n) = θ(nlogb(a) lgk+1(n) )
Changing Variables Method
 T(n) = 2T( √n ) + lgn

Rename: m = lgn  n = 2m

T (2m) = 2T(2m/2) + m
Rename: S(m) = T(2m)
S(m) = 2S(m/2) + m  S(m) = O(mlgm)
(demonstrated before)

T(n) = T(2m) = S(m) = O(mlgm)=O(lgnlglgn)

Idea: transform the recurrence to one that

you have seen before
 T(n)=2T(√n)+1

Let m=log n
N=2m
√n=2m/2

T(2m)=2T(2m/2)+1

Let s(m)=T(2m)

S(m)=s(m/2)+1
S(m)=0(log m)

i.e. T(n)=0(log log n)

 Recurrence equations to remember

• T(n) = T(n – 1) + O(1)  O(n)

• T(n) = T(n – 1) + O(n)  O(n2)
• T(n) = 2T(n – 1) + O(1)  O(2n)
• T(n) = T(n/2) + O(1)  O(lg n)
• T(n) = 2T(n/2) + O(1)  O(n)
• T(n) = 2T(n/2) + O(n)  O(n lg n)

65
Q.1. Solve the following recurrence

T(n)=T(n-1)+T(n-2)

T(0)=0 and T(1)=1