Recurrences 2

Recurrences
Recurrences /Dr.A.Sattar / 1
Recurrences
Topics
Introduction to recurrences
Applications of recurrences
Solving recurrences
-Iterations method
-Substitution method
-Recursion tree method
-Characteristic Equation
- Induction method
Introduction
Recurrences
Definition
A recurrence is a relation or an equation that describes a function in terms of its lower
order arguments. It has the following characteristics
(i) The function is defined over a set of natural numbers
(ii) The definition includes a base value for the function, called boundary condition
Example(1): The factorial function f(n)=n! can be expressed in terms of

recurrence
f(n) = n.f(n – 1)
f(0)=1 (boundary condition)
Example(2) The Fibonacci Sequence (0,1, 1, 2, 3, 5, 8, 13, 21….) is defined

recursively by the relation
f(n) = f(n-1) + f(n-2)
f(0) = 0,
f(1)=1 (boundary conditions)
Recurrences
Taxonomy
Recurrences are frequently used to specify running times (complexity) for a broad
categories of problems. Common examples are decrease-and-conquer , divide-and-conquer,
probabilistic analysis, and dynamic programming algorithms.
• In a decrease-and-conquer algorithm the size of a problem is reduced by a constant amount

in each iteration. The running time can be specified as
T(n)=T(n-c) + f(n),
where c is the reduction constant, and f(n) is the cost of reduction in each iteration.
• In a divide-and-conquer algorithm a problem is split (divided) into subproblems , which are then
solved (conquered) and combined to obtain solution to the main problem. In general the running time is
expressed as
T(n)=aT(n/b) + f(n)
where a is the number of subproblems and b is the size of each subproblem. The function f(n)
is the cost of dividing and combing the subproblem solutions.
• In probabilistic analysis algorithm, the average running time is typically written as

T(n) = an-1T(n-1)+an-2T(n-2)+……..+a1T(1) +f(n)
where an-1,an-2, …….a1 are the probabilities
• In dynamic programming, optimal cost is typically given by c(i,j)=max ( c(i,j-1), c(i-1,j))
Recurrences
Examples
The following examples demonstrate the use of recurrence for the running times of
common algorithms. Here T(n) denotes the running time for a problem of size n.
Example(1): Here is an example of recurrence relation for decrease- and-conquer problem.
T(n) = T(n-1 ) + cn
Subproblem size Cost of decreasing
Example(2) This example illustrates the recurrence relation for divide-and-conquer problem.
T(n) = 8 T(n / 4 ) + c n2
Number of Subproblem Cost of dividing

subproblems size and combining
Recurrences
Applications
Some typical applications of recurrences are as follows;
Linear Search: T(n)= T(n-1) + c
Selection Sort: T(n)=T(n-1) + n
Binary tree traversal: T(n)=T( k )+T(n - k) +c
Binary Search: T(n)= T(n/2) + c
Quick /Merge Sort: T(n)= 2T(n/2) + n
Finding Maximum : T(n)= 2T(n/2) +c
Multiplying Integers : T(n)= 3T(n/2) +n
Multiplying Matrices : T(n) =8T(n/2)+ n2

n −1
Quick Sort (Expected): T (n) = ( ∑ T (n − k )) / n + n

k =1
Linear Programming (Shortest Paths) dij(k) = min( dij(k-1) , dik(k-1) + dkj(k-1) )

Solving Recurrences
Solution of Recurrence
Closed Form
In some cases exact solution to a recurrence can be expressed in a closed form , or as a
formula. For example, the exact solution to the following recurrence
1, n =2
T(n) =
2T(n/2) + 2 n > 2
is given by the following formula :
T(n)=(3/2)n-2.
Solution of Recurrence
Asymptotic Notation
For most problems the exact solution to a recurrence cannot be determined, or it may not exist
in a closed form (formula). Since analysis of algorithm is primarily concerned with large inputs,
for analysis purposes, it is sufficient to express the solution in terms of asymptotic notation
For example, the solution to the recurrence
c, n =1
T(n) =
3T(n/4) + cn2 n>1
is expressed, using asymptotic notation, as follows:
T(n) = θ(n2)
Solving Recurrences
Methods
Common methods for the solution are of recurrences are:
The Iterative Method
The Substitution Method
The Recursion Tree Method
Using Theorems
Solving Characteristic Equation
Using induction method
The Iteration Method
Procedure
In the iteration method the recurrence is solved by following the top-down approach.
It involves following steps:
(1) Using definition, equations are set up for arguments n, n-1, n-2…..
(2) On reaching the bottom level the boundary condition is applied.
(3)The equations are summed up.
(4) Finally, the solution is obtained by canceling out identical terms on the left-hand
and right- hand sides of the iterated equations
The iteration method is particularly useful for solving decrease-and-conquer problems.

In other cases additional efforts are required to cancel out the terms appearing on both
sides of the final equation
Example(1)
Here is a recurrence for reduce-and-conquer algorithm :
T(0)=0
T(n)= T(n-1) +c for n>0
Iterating the recurrence:.
T(n) = T(n-1 ) + c
T(n-1) = T(n-2) + c
T(n-2) = T(n-3) + c
……………………
T(3) = T(2) + c
T(2) = T(1) + c
T(1) = T(0) + c
Adding both sides of the equations, and canceling equal terms:
T(n) = T(0) + c+ c +………+ c (n terms )
Or, T(n) = n.c
It follows that the recurrence has the solution

T(n) ∈ θ(n)
Example(2)
Here is another recurrence for reduce-and-conquer algorithm with
linear cost function
T(0)=0
T(n) = T(n-1) + c.n
Iterating the recurrence:
T(n) = T(n-1 ) + c.n
T(n-1) = T(n-2) + c.(n-1)
T(n-2) = T(n-3) + c.(c-2)
……………………
T(3) = T(2) + c.3
T(2) = T(1) + c.2
T(1) = T(0) + c.1
Adding both sides of the equations, and canceling out the equal terms:
T(n) = T(0)+ c(1 + 2+ 3 +………+n)
Using boundary condition for T(0), and summing the arithmetic series:
T(n)=c.n(n+1)/2
It follows that recurrence has the solution
∈
T(n) θ(n2)
Procedure
The substitution method provides a systematic procedure for solving recursive equation.
It is similar to the iteration method because it also follows the top-down approach.
It is, however, useful for solving recurrences for divide-and-conquer problems.
The procedure involves following steps:
(1) In the recurrence, values are plugged in repeatedly on the right-hand side
of the equation.
(2) The procedure is repeated until the base case is reached
(3) The iterative steps generate some kind of pattern or a series.
(4) The summation for the series is analyzed to determine the asymptotic behavior.
To facilitate the substitutions, a copy of the recurrence is used. .
Example(1)
Here is recurrence for a divide-and-conquer algorithm, with fixed cost of splitting/combining.
T(1) = c
T(n)=T(n/2) + c, n > 1
Initially, T(n) = T(n/2) + c = T(n/21) + c ………..(1)
Substituting for T(n/2),

T(n) = T(n/4) + 2c
=T(n/22) + 2.c ……………………….(2)
Again substituting for T(n/4),
T(n) = T(n/8) + 3c
=T(n/23) + 3.c …………………..(3)
Continuing, after kth step
T(n) = T(n/2k) + k.c …………………….(4)
It will be seen that the base case, T(1)=c, will be reached when n / 2k =1, or n=2k .
Taking binary logarithm, we have k = lg n
Substituting for k in equation (4), we get
T(n) = T(1)+ lg n. c
= c + lg n. c
Thus, T(n) ∈θ( lg n)
Example(2)
Here is a recurrence for divide-and-conquer algorithm with fixed cost function. The problem is split into
two sub-problems of equal sizes.
T(1) = c
T(n)=2T(n/2) + c, n > 1
Initially: T(n) = 2.T(n/2) + c

= 2.T(n/21) +20 c ………………….……………..(1)
Substituting for T(n/2), T(n) = 2.[2.T(n/4)+c] +c = 4.T(n/4) + 3c

= 22T(n/22)+ (20+ 21).c …………………………….(2)
Substituting for T(n/4), T(n) = 4.[2 T(n/8)+c] + 3c =8.T(n/8) + 7.c

= 23T(n/23) + (20+21+22).c ………………………(3)
Continuing, after kth substitution,
T(n) = 2kT(n/2k) + (20+21+22+…..2k-1).c ………(4)
Summing the geometric series,
T(n) = 2kT(n/2k) + (2k - 1)c ……. ………………(5)
It will follow that the base case , T(1)=c ,will be reached when n / 2k =1, or n=2k ,
Substituting for 2k in (5) , we get
T(n) = n.T(1) +( n-1).c = n.c +n.c - c = c.(2n-1)
Ignoring constants, we have,
∈
T(n) θ(n)
Example(3)
This is recurrence for a divide-and-conquer algorithm with linear cost.
T(1)= c
T(n)= 2T(n/2) + cn, n > 1
Initially, T(n)= 2T(n/2) + cn ………………………………………………….……(1)
Substituting for T(n/2), T(n)= 2[ 2T(n/4)+ cn/2] +cn=4T(n/4) + 2.cn

=22T(n/22)+2cn …………………..…………….…….….(2)
Again substituting for T(n/4), T(n)=22[2T(n/8) +cn/4]+2.cn=23T(n/8)+3.cn

=23T(n/23) + 3.cn ……………………………..…..(3)
Continuing, after kth step,
T(n) = 2kT(n/2k)+ kcn ……………………………………….……(4)
The base case, T(1)=c, will be reached when n/2k = 1, i.e 2k =n, or k=lg(n)
T(n) = n.T(1) + cn.lg n =cn+ cn.lg n …….……………………….(5)
Here, cnlg n is the dominant term, compared to cn. Therefore, it follows,

T(n) ∈ θ(n lgn)
Example (4)
Here is a recurrence for divide-and-conquer algorithm which splits the problem into three sub-problems
of sizes one fourth of the size of the original problem. The cost of splitting/combining is linear.
T(1)= c
T(n)= 3T(n/4) + cn, n > 1
Initially, T(n)= 3T(n/4)+ cn ……………………………………………………………………. (1)
Substituting for T(n/4),
T(n)= 3[3T(n/16) + cn/4] +cn = 9T(n/16)+ cn+cn.3/4 …………………..…….(2)
=32T(n/42)+cn[(3/4)0 + (3/4)1]
Again. substituting for T(n/16) in (2) ,
T(n)=9[ 3T(n/64)+cn/16] +cn+3cn/4 =27T(n/64)+ cn+cn.3/4+ cn.916]
=33T(n/43) + cn[(3/4)0 + (3/4)1+ (3/4)2] …………………………………..………...(3)
Continuing, after kth step, it follows
T(n)= 3kT(n/4k)+cn[(3/4)0+ (3/4)1+(3/4) 2+ ……..+(3/4) k-1] ……………………….……..(4)
The base case, T(1)=c, will be reached when n/4k = 1. Taking log to base 4, k=log4 n. There fore,
T(n)=c.3 log4 n +cn[ (3/4)0+ (3/4)1+(3/4)2+………..+(3/4) log 4 n-1 ] …………….. …………...(5)
In relation (5), the geometric series has geometric ratio 3/4, which is less than 1. Therefore,
(3/4)0 + (3/4)1+ (3/4)2+………..(3/4)log4 n-1 = θ(1) …………………………………………(6)
Thus, using relations (5) and (6) we have
T(n) = c. 3 log 4 n + cn.θ(1) ……………………………………………………………………….(7)
Now, using property of logarithm, 3 log 4 n = n log 4 3, and ignoring constant term θ(1), we have
T(n) = cn log 4 3 + cn. …………………………………………………………….……………….(8)
Since, log 4 3 < 1, the term n log 4 3 is smaller compared to n. The term c n log 4 3 in (8) can be
discarded in favor of n. Therefore,
T(n)∈ θ(n)
Procedure
The recursion tree provides a visual tool for solving recursive equation. It involves
following steps
Step # 1 The recurrence is expressed in a hierarchical way using a tree structure, such that
each node contains two fields: the size field and cost field . The number of child
nodes in the tree equals the number of subproblems
size
cost
size size size

size
cost
cost cost cost
Step #2: The size field of a node is set by plugging in the the size of parent node
into the relation
Step # 3 :The cost field is set by substituting node size into cost function
of the relation
Step #4: The solution is found by summing the costs over all nodes of the tree
Example(1)
T(n)= 2T(n/2) + cn, n>1, T(1)=c
Step #1: Constructing tree structure
The fully expanded recursion tree is shown below. It has 2d nodes at the bottom level (called leaves) where
d is the tree depth. Since at the bottom level T(n/2d)=T(1), it follows that n/2d=1, or 2d= n. i.e d= lg n.
Thus, tree depth = lg n. , and number of leaves is 2d=n
Nodes
20
21
Tree with size fields

lg(n) 22
23
2d
n
The Recursion Tree Method Cont’d-1
Example(1)
T(n)= 2T(n/2) + cn, n>1, T(1)=c
Step#2: Inserting costs
The root has associated size n and cost cn. Each child of root has size n/2 and associated cost cn/2
At the next level the costs are reduced by a factor of 2. This reduction is continued up to the bottom level.
Each leaf has associated cost of c.
lg(n)
n
Example(1)
T(n)= 2T(n/2) + cn, n>1, T(1)=c
Step #3: Summing up rows and leaves costs
Each row contributes total cost cn. Since there are lg n+1 levels, including the last level, the total cost
associated with all nodes excluding leaves is cn.lg n. There are n leaves, each having cost c. Thus, total
contribution of leaves is c.n Hence, the recurrence has the solution T(n)=cn.lg n + cn =θ(n lg n)
rows costs
cn
2xcn/2 = cn
4xcn/4 = cn
lg(n)
8xcn/8 = cn
leaves cost=c.n Recurrences /Dr.A.Sattar / 26

Example(2)
T(n) = 2T(n/2) + cn2 , n>1, T(1)=c
The fully expanded binary recursion tree is shown below. Tree has depth lg n, and n leaves.
20
21
22
lg(n)
23
2d
Example(2)
T(n) = 2T(n/2) + cn2 , n>1, T(1)=c
The root has associated size of n and cost of cn2. Each child of root has size n/2 and associated cost cn2/4
lg(n)
leaves= n
Example(2)
T(n) = 2T(n/2) + cn2 , n>1, T(1)=c
Step #3: Summing up rows and leaves costs
Summing the costs associated with the internal nodes and leaves:
T(n) = cn2. [1/20+ 1/21+ 1/23 + ……..1/2lgn-1] + cn
The asymptotic behavior of the series is determined by the largest term, which is 1.Thus,
1/20+ 1/21+ 1/23 + ……..1/2lg n-1 = θ(1).
Therefore, T(n)=cn2.θ(1)+cn= θ(n2) ( n2 being the dominant term in the sum)
rows cost
cn2.[1/20]
2 x cn2/4 = cn2 .[1/21]
lg(n) 4 x cn2/16 = cn2 .[1/22]
8 x cn2/64 = cn2.[1/23]
leaves cost = c.n Recurrences /Dr.A.Sattar / 29

Example(3)
T(n) = 3T(n/4) + cn2 , n>1, T(1)=c
The recursion tree is tertiary, as shown below. It has 3d nodes at the bottom level, where d is the tree depth.
Since at the bottom level T(n/4d)=T(1), it follows that n/4 d=1, or 4d= n. i.e d= log4 n.
Thus, tree depth = log 4 n, and number of leaves is 3d= 3 log 4 n= n log4 3
nodes
30
31
log4n
32
3d
leaves= 3log4n= nlog 34
Example(3)
T(n) = 3T(n/4) + cn2 , n>1, T(1)=c

The root has associated size of n and cost of cn2. Each child of root has size n/4 and associated cost cn2/16
cn2(1)
cn2(1/2)
log4n cn2(1/4)
cn2(1/8)
leaves= 3log4n= nlog 34

Example(3)
T(n) = 3T(n/4) + cn2 , n>1, T(1)=c
Step #3: Summing over the rows and leaves costs
There are log 4 n -1 rows of internal nodes and one root node, each contributing a cost of cn2. The total
contribution by all leaves is c.nlog 4 3.Summing over all nodes of the recursion tree:
T(n)= cn2. [1+(3/16)1+(3/16)2 + …….. +(3/16)log4 n-1 ]+ c.n log 4 3
The summation in brackets is geometric series with ratio 3/16 <1.Asymtotically the summation is θ(1) Thus,
1+(3/16)1+(3/16)2 + …….. +(3/16)log4 n-1 = θ(1)
Therefore, T(n) = cn2.θ(1)+ cn log4 3 =θ(n2), since log43<1, n log43 < n2

rows costs
cn2=c.(3/16)0 n2
c x 3 x (n/4)2=c.(3/16)1 n2
log4n
c x 9 x (n/16)2= c.(3/16)2n2
leaves= nlog 3
4
Using Theorems
Theorem #1
Statement
If a is the number of sub-problems and b the size of each sub-problem, then the solution to
the divide-and-conquer recurrence
T(1) = c
T(n) = a T(n/b) + cn, for n >1
is given by
θ(n), when a<b

T(n) = θ(n logb n), when a= b
θ(nlog a),
b when a> b
The theorem provides solution to problems in which cost of dividing and combining sub-
problems is linear.
Theorem #1
Usage
According to Theorem#1, the solution of recurrence, T(n)= aT(n/b) + cn ,is given by
θ(n) when a<b (case 1)
T(n) = θ(n log b n) when a=b (case 2)
θ(nlogba) when a >b (case 3)
Example(1): T(n)=2T(n/4) +cn
Here, a=2, b=4 .
Since a < b, case 1 applies
Therefore, T(n)=θ(n)
Theorem #1
Usage
a=2, b=2
Since a = b, case 2 applies
Therefore, T(n)=θ(n lg n)
Theorem #1
Usage
a=3, b=2
Since a > b, case 3 applies
Therefore, T(n)=θ(n log 2 3)
=θ(n lg 3)
Theorem #1
Proof
The method of substitution can be used to obtain solution to the recurrence
T(n)=aT(n)+cn .
Substituting for T(n/b):
T(n)=a[ aT(n/b2) + c (n/b)] + cn = a2T(n/b2) + cn[ 1+(a/b)]
Substituting for T(n/b2)

T(n)=a2[aT(n/b3) +c (n/b2)] + cn[1+(a/b)] =a3T(n/b3) + cn[1+(a/b) +(a/b)2]
Again substituting for T(n/b3):

T(n)= a3[ aT(n/b4)+ c(n/b3)] + cn[1+(a/b)+(a/b)2] = a4T(n/b4) + cn[1+(a/b) + (a/b)2 + (a/b)3]
Continuing, after kth step,

T(n)= akT(n/bk) +cn[1+(a/b)+(a/b)2+(a/b)3+…..+(a/b)k-1]
The term T(n/bk) would reduce to base case T(1) when n / bk =1, or bk = n.
Or, taking logarithm with base b, k = log b n.
Substituting for k in the summation:

T(n)= alogb n.T(1) + cn[1+(a/b)+(a/b)2+(a/b)3+……+(a/b)log b n - 1]
=c. alogb n. + cn[1+(a/b)+(a/b)2+(a/b)3+……+(a/b)log b n - 1] ………(1)

Cont’d
Theorem#1 Cont’d
Proof
T(n) =c. alogb n. + cn[1+(a/b)+(a/b)2+(a/b)3+……+(a/b)log b n - 1] ……….(1)
Now, consider alogb n. Multiplying and dividing with . blogb n,.

alogb n.blogb n,
alogb n= = ( a/b) logb n. blogb n
blogb n
Since, blogb n = n log b b =n, it follows, alogb n = n( a/b) logb n
Substituting into the above equation (1), and rearranging
T(n) =cn[1+(a/b)+(a/b)2+(a/b)3+……+(a/b)log b n ] ………………….(2)
The right side is geometric series with geometric ratio a/b. The asymptotic behavior is determined by the
larges term of the series. Three cases need to be considered.
Case a < b:The geometric series (2) has decreasing terms Its first term, (a/b)0 = 1, makes the main
contribution asymptotically, which is θ(1) Therefore, recurrence has the solution
T(n)=cn.θ(1).=θ(n)
Case a =b :All terms of the series make equal contribution of 1.There being log b n +1 terms,
T(n) =cn.log b n +cn = θ(n log b n)+θ(n) =θ(n log b n) !
Case a > b: The geometric series has increasing terms .The last term, (a/b)log b n , makes main
contribution asymptotically
Since, (a/b)log b n = a log b n / b log b n =n log b a / n log b b =n log b a / n
Therefore, the recurrence has the solution
T(n)= cn.nlog b a / n = c n log b a =θ(n log b a )
Theorem #2
Statement
If a is the number of subproblems and b the size of each subproblem, then the
solution to the divide-and-conquer recurrence
T(1) = c
T(n) = a T(n/b) + cnx , for n >1, where x is some constant,
is given by
θ(nx), when a < bx

T(n) = θ(nx log n), when a= bx
θ(nlog a),
b when a > bx
The theorem provides solution to problems in which cost of dividing and combining
subproblems is some polynomial term..
The recurrence can be solved by substitution method.
Theorem #2
Usage
According to Theorem#2, the solution of recurrence T(n)=aT(n/b) + cnx, where x is some
positive constant is given by
θ(nx) when a < bx (case 1)

T(n) = θ(nx lg n) when a = bx (case 2)
θ(nlogba) when a > bx (case 3)
Example(1): T(n)=2T(n/2) +n3

Here x=3, a=2, b=2 , b3.=8
Since a < b3, case 1 applies
Therefore, T(n)=θ(n3)
Theorem #2
Usage

x=2, a=4, b=2, b2=4
Since a = b2, case 2 applies
Therefore, T(n)=θ(n2 lg n)
Theorem #2
Usage

x=2, a=7, b=2 , b2=4
Since a > b2, case 3 applies
Therefore, T(n)=θ(n log 2 7)=θ(n lg 7)
Theorem #2
Proof
The following is an outline of the proof of theorem #2
The recurrence
T(1) = c
T(n) = a T(n/b) + cnx , for n >1
can be solved by substitution method.
It can be shown that after k substitutions the result is given
T(n)= akT(n/bk) +cnx[1+(a/bx)+(a/bx)2+(a/bx)3+…..+(a/bx)k-1]
The term T(n/bk) would reduce to base case T(1) when n / bk =1, or bk = n.
Taking logarithm with base b, k = log b n. Substituting for k in the summation:
T(n)= alogb n.T(1) + cnx[1+(a/bx)+(a/bx)2+(a/bx)3+……+(a/bx)log b n - 1]
The geometric series has geometric ratio a/bx By considering the asymptotic behavior
of the series it can be shown that
T(n) = θ(nx), when a < bx
T(n) = θ(nx log n), when a= bx
T(n) = θ(nlog a),
b when a > bx
Master Theorem
Statement
Let a ≥ 1 and b>1 be constants, then the recurrence
T(n)= aT(n/b) +f(n)
has the solution:
(1) T(n)= θ(n log b a ) when f(n)=O( n log b a-ε) for some ε>0
(2) T(n)= θ(n log b a lg n) when f(n)=θ(n log b a)
( 3) T(n)= θ( f(n) ) when f(n)= Ω(n log b a + ε) for some ε>0

provided also that af(n/b) ≤ c.f(n) for some c<1 and large n
The Master Theorem provides a generalized solution for the divide-and-conquer algorithms.
The cost function can be a polynomial or logarithmic function
Master Theorem
Constraints
In Master Theorem, the cost function f(n) is compared with the function nlog b a
Depending on the outcome, the larger of the two functions provides the solution, subject to
some additional constraints.
The constraint is that the function f(n) and n log b a should not be simply larger or smaller
asymptotically, but should grow faster or slower by a polynomial factor nε , where ε is some
arbitrary small positive constant. Accordingly, in using Master Theorem the following
conditions are further examined:
Case 1: If f(n)= O( nlog b a – ε) f(n) grows slower than nlog b a by a factor of nε, then
solution of the recurrence is
T(n) = θ(nlog b a)
Case 2: If f(n)= θ( nlog b a), i. e. f(n) grows as fast as nlog b a, then solution of the
recurrence is
T(n) = θ(nlog b a lg n)
Case 3: If f(n)= Ω( nlog b a + ε) , i. e. f(n) grows faster than nlog b a by a factor of nε,
and in addition f(n/b) ≤ c.f(n) for some c<1,then solution of the recurrence is
T(n)= θ( f(n) )
Master Theorem
Usage
Example(1): T(n)=4T(n/2) +n
Here a=4, b=2, f(n)=n
Consider n log b a -ε = n log 2 4 -ε= n2-ε .
Take ε = 0.5 ( ε can be some arbitrary small constant)
Since f(n) = n grows slower than = nlog b a-ε =n1.5, it follows that f(n)= O(nlog b a-ε)
This is case I of Master Theorem. Thus,
T(n) =θ(nlog b a)’

=θ( n log 2 4)
= θ(n2)
Master Theorem
Usage
Example(2): T(n)=T(n/2) +1
Here a=1, b=2, f(n)=1
Consider n log b a = n log 2 1 = n0 =1
Since f(n) = 1 grows as fast as nlog b a = 1, it follows that f(n)= θ(nlog b a)
This is case 2 of Master Theorem. Therefore ,
T(n)=θ(nlog b a .lg n)
=θ( n log 2 1 .lg n)
= θ(lg n)
Master Theorem
Usage
Example(3): T(n)=T(n/3) +n
Here a=1, b=3, f(n)=n
Consider n log b a+ε = n log 3 1+ε = nε. .
Take ε=0.5
Since f(n) = n grows faster than nlog b a+ε = n0.5, it follows that f(n)= Ω(nlog b a +ε )
Further , af(n/b)<c. f(n) if 1(n/3)<c.n

i.e, n/3 <c.n for some c<1.
Or c>1/3
This is true if c=1/2 !

T(n)=θ(f(n))
=θ(n)
Master Theorem
Usage
Example(4): T(n)=3T(n/4) +n lg n
Here a=3, b=4, f(n)=n lg n
Consider n log b a+ε = n log 3 4+ε = n0.793+ε. .
Take ε=0.207
Since f(n) = n log n grows faster than nlog b a+ε = n, it follows that f(n)= Ω(nlog b a +ε )
Further , we check if af(n/b)<c. f(n) .
Now 3(n/4)lg (n/4) < (3n/4). nlg n =3/4.f(n) for large n
Thus, af(n/b)<c.n lg n where c=3/4.

!
T(n)=θ(f(n))
=θ(n lg n)
Master Theorem
Usage
Example(5): The Master Theorem cannot be used to obtain solution to certain
problems.
Consider, for example, the recurrence
T(n)=4T(n/4) +n lg n
In this case a=4, b=4, f(n)=n lg n. nlogb a = n
Consider Lim n lg n /n = lg n = ∞ i.e n lg n =Ω(n)

n→∞
Now, n log b a +ε = n log 4 4+ε = n1+ε
Consider Lim n lg n / n1+ε = lg n/ nε = 0 , for ε >0

n→∞
It follows that although nlg n =Ω(n) but n lgn n ≠Ω(n1+ε) !
Therefore, the case 3 of Master Theorem does not apply. Thus, this example cannot
be solved by applying Master Theorem
Master Theorem
Proof
Here is brief outline of the proof. For details consult the text book.
The Master theorem can be proved by substitution method. An outline of the proof is as follows
T(n)= aT(n/b) + f(n)
= a((aT(n/b2) + f(n/b) )+ f(n)
= a2(T(n/b2) + af(n/b) + f(n)
= a3 T(n/b3) + a2f(n/b2)+ af(n/b) + f(n)
log b n -1
=a
log b n
T(1)+ ∑ak f(n/b )
k =1
k
log a log b n - 1
= n b T(1) +
∑ ak f(n/b )
k =1
k
By plugging in the asymptotic forms of f(n), in three cases; and examining

behavior of the series we can prove the solution of the recurrence.
The Characteristic Equation
TheCharacteristic Equation
Definition
The algorithms for some problems can be expressed in terms of linear homogenous
relation of the type
T(n)=an-1T(n-1)+an-2T(n-2)+……+a1T(1).
In order to solve the recurrence, the relation is usually converted into an algebraic
equation . This equation is called characteristic equation.
A simpler form of homogenous recursion is

T(n)=aT(n-1) +bT(n-2)
where a and b are constants . The boundary conditions for the recurrence might be
T(0)=0, T(1)=1
The above recurrence can be solved in a closed form by converting into an

algebraic quadratic equation.
Characteristic Equation
Solution
The solution to the recurrence
T(n) = aT(n-1)+ bT(n-2), T(0)=0, T(1)=1
is obtained in a closed form by the following procedure:
Step #1: Let T(n) = xn, , where x is some unknown variable

Plugging into the recurrence relation
xn = a.xn-1 + bxn-2
Dividing with xn-2 and simplifying
x2 – a. x – b =0
Step #2: The quadratic equations has two roots. If x1 and x2 are the distinct roots then
x1 = ( a+√(a2 + 4b) ) / 2
x2 = ( a - √(a2 + 4b) ) / 2
Step #3: A general solution of recurrence is linear combination of x1 n and x2 n, as follows:

T(n)= c1.x1 n + c2. x2n
where c1 and c2 are constants, which can be obtained by applying the boundary conditions,
T(0)=0 and T(1)=1
Characteristic Equation
Usage
The Fibonacci Sequence is a homogenous recurrence, which can be solved by using the method of
characteristic equation. The recurrence has the form:
T(n) = T(n-1)+ T(n-2) for >1 , T(0)=0, T(1)=1
Let T(n) = xn
Plugging into the relation and simplifying

x2 – x – 1 =0
Solving the equation
x1 = ( 1+√(5 ) / 2
x2 = ( 1 - √(5 ) / 2
A general solution of recurrence is
T(n)= c1. x1n + c2.x2n
Using the boundary conditions, T(0)=0 and T(1)=1
n=0, c1 + c2 = T(0)= 0
n=1, c1.x1 + c2 .x2 = T(1)=1
Solving for c1 and c2
c1 = 1/√5, c2= -1/√5
The recurrence has the solution
T(n)= ( x1n - x2 n) / √5
n
=[( ( 1+√(5 ) / 2 ) + ( (1 - √(5 ) / 2 )n ] / √5
The Induction Method
The Induction Method
Statement
The induction method is commonly used to establish the validity of
some known recurrence. It involves following steps
• First it is proved that the recurrence T(n) is valid for boundary

conditions.
• Next, it is assumed that T(n) is true for some value k < n
• Finally, using the basic definition and assumption made at the previous step , it is proved
that relation T(n) also holds true for k+1 or 2k
If the above conditions are satisfied then recurrence holds true for all n
Induction Method
Usage
Using induction method it can be proved that recurrence
1, n =2
T(n) =
2T(n/2) + 2 n > 2
has the solutions T(n)= (3/2)n – 2.
Step #1: T(2) =(3/2)2 -2 = 1. True for base case
Step #2: Assume the solution is true for n=k ,

T(k) =(3/2)k – 2 ………………(1)
Step #3: Consider T(2k),

T(2k) = 2T(2k / 2) + 2 by definition
= 2T(k)+ 2
= 2.((3/2)k – 2) +2 by assumption at (1)
=(3/2)(2k) – 2
It follows that the relation is true for n=2k. Therefore, it is true for all n

Recurrences 2

Uploaded by

Copyright:

Available Formats

Recurrences 2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Recurrences 2

Uploaded by

Copyright:

Available Formats

Recurrences

-Recursion tree method

(i) The function is defined over a set of natural numbers

Example(1): The factorial function f(n)=n! can be expressed in terms of

Example(2) The Fibonacci Sequence (0,1, 1, 2, 3, 5, 8, 13, 21….) is defined

• In a decrease-and-conquer algorithm the size of a problem is reduced by a constant amount

• In probabilistic analysis algorithm, the average running time is typically written as

• In dynamic programming, optimal cost is typically given by c(i,j)=max ( c(i,j-1), c(i-1,j))

Example(1): Here is an example of recurrence relation for decrease- and-conquer problem.

Subproblem size Cost of decreasing

Number of Subproblem Cost of dividing

 Selection Sort: T(n)=T(n-1) + n

 Binary tree traversal: T(n)=T( k )+T(n - k) +c

 Binary Search: T(n)= T(n/2) + c

 Quick /Merge Sort: T(n)= 2T(n/2) + n

 Finding Maximum : T(n)= 2T(n/2) +c

 Multiplying Integers : T(n)= 3T(n/2) +n

 Multiplying Matrices : T(n) =8T(n/2)+ n2

 Quick Sort (Expected): T (n) = ( ∑ T (n − k )) / n + n

 Linear Programming (Shortest Paths) dij(k) = min( dij(k-1) , dik(k-1) + dkj(k-1) )

is given by the following formula :

is expressed, using asymptotic notation, as follows:

 The Iterative Method

 The Substitution Method

 The Recursion Tree Method

 Solving Characteristic Equation

Using induction method

(2) On reaching the bottom level the boundary condition is applied.

(3)The equations are summed up.

 The iteration method is particularly useful for solving decrease-and-conquer problems.

 It follows that the recurrence has the solution

(2) The procedure is repeated until the base case is reached

(3) The iterative steps generate some kind of pattern or a series.

 To facilitate the substitutions, a copy of the recurrence is used. .

Substituting for T(n/2),

 Initially: T(n) = 2.T(n/2) + c

Substituting for T(n/2), T(n) = 2.[2.T(n/4)+c] +c = 4.T(n/4) + 3c

Substituting for T(n/4), T(n) = 4.[2 T(n/8)+c] + 3c =8.T(n/8) + 7.c

Initially, T(n)= 2T(n/2) + cn ………………………………………………….……(1)

Substituting for T(n/2), T(n)= 2[ 2T(n/4)+ cn/2] +cn=4T(n/4) + 2.cn

Again substituting for T(n/4), T(n)=22[2T(n/8) +cn/4]+2.cn=23T(n/8)+3.cn

T(n) = 2kT(n/2k)+ kcn ……………………………………….……(4)

Here, cnlg n is the dominant term, compared to cn. Therefore, it follows,

size size size

Tree with size fields

leaves cost=c.n Recurrences /Dr.A.Sattar / 26

Step #1: Constructing tree structure

2 x cn2/4 = cn2 .[1/21]

lg(n) 4 x cn2/16 = cn2 .[1/22]

leaves cost = c.n Recurrences /Dr.A.Sattar / 29

leaves= 3log4n= nlog 34

Step#2: Inserting costs

leaves= 3log4n= nlog 34

Therefore, T(n) = cn2.θ(1)+ cn log4 3 =θ(n2), since log43<1, n log43 < n2

θ(n), when a<b

Example(1): T(n)=2T(n/4) +cn

Here, a=2, b=4 .

Since a < b, case 1 applies

Example(2): T(n)=2T(n/2) +cn

Selection Sort: T(n)=T(n-1) + n

Binary tree traversal: T(n)=T( k )+T(n - k) +c

Binary Search: T(n)= T(n/2) + c

Quick /Merge Sort: T(n)= 2T(n/2) + n

Finding Maximum : T(n)= 2T(n/2) +c

Multiplying Integers : T(n)= 3T(n/2) +n

Multiplying Matrices : T(n) =8T(n/2)+ n2

Quick Sort (Expected): T (n) = ( ∑ T (n − k )) / n + n

Linear Programming (Shortest Paths) dij(k) = min( dij(k-1) , dik(k-1) + dkj(k-1) )

The Iterative Method

The Substitution Method

The Recursion Tree Method

Solving Characteristic Equation

Using induction method

The iteration method is particularly useful for solving decrease-and-conquer problems.

It follows that the recurrence has the solution

To facilitate the substitutions, a copy of the recurrence is used. .

Initially: T(n) = 2.T(n/2) + c

Initially, T(n)= 2T(n/2) + cn ………………………………………………….……(1)

The recurrence can be solved by substitution method.

By plugging in the asymptotic forms of f(n), in three cases; and examining

A simpler form of homogenous recursion is

The above recurrence can be solved in a closed form by converting into an

Plugging into the relation and simplifying