Recurrences

CS1006T Data Structures
Analysis of Recursive Algorithms
Chandrabose Aravindan
<AravindanC@ssn.edu.in>
Professor of Information Technology

SSN College of Engineering
March 12, 2024
C. Aravindan (SSN Institutions) Data Structures March 12, 2024 1 / 14

Simple Recurrence
Consider a simple case of divide-conquer-merge, where the input of

size n is divided in to 1 and (n − 1)
def fact (n ) :
i f (n < 2):
return 1
else :
r e t u r n n ∗ f a c t ( n−1)

Simple Recurrence

def fact (n ) :
i f (n < 2):
return 1
else :
Here, the merge cost is constant

Simple Recurrence

def fact (n ) :
i f (n < 2):
return 1
else :

How do we obtain T (n) in this case?

Simple Recurrence

def fact (n ) :
i f (n < 2):
return 1
else :

How do we obtain T (n) in this case?
(
d n≤1
T (n) =
T (n − 1) + c n>1
This is a very simple recurrence equation.

Solving recurrences: Expansion Method
Recurrences may be solved by a simple expansion process

T (n) = T (n − 1) + c n>1

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3
..
.

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3
..
.
= T (n − i) + ic n>i

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3
..
.
= T (n − i) + ic n>i
The objective is to represent T (n) directly in terms of the base case

T (1).

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3
..
.
= T (n − i) + ic n>i

T (1). This is achieved when i = n − 1
T (n) = T (n − (n − 1)) + (n − 1)c

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3
..
.
= T (n − i) + ic n>i

T (n) = T (n − (n − 1)) + (n − 1)c
= T (1) + (n − 1)c

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3
..
.
= T (n − i) + ic n>i

T (n) = T (n − (n − 1)) + (n − 1)c
= T (1) + (n − 1)c
= d + (n − 1)c

T (n) = T (n − 1) + c n>1
= T (n − 2) + c + c n>2
= T (n − 2) + 2c n>2
= T (n − 3) + 3c n>3
..
.
= T (n − i) + ic n>i

T (n) = T (n − (n − 1)) + (n − 1)c
= T (1) + (n − 1)c
= d + (n − 1)c
It is obvious that this T (n) is Θ(n)

Solving Recurrences: Telescopic Sum
T (n) = T (n − 1) + c

T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c

T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c
..
.
T (2) = T (1) + c

T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c
..
.
T (2) = T (1) + c

T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c
..
.
T (2) = T (1) + c
T (n) + T (n − 1) + T (n − 2) + · · · + T (2) =
T (n − 1) + T (n − 2) + · · · + T (2) + T (1) + (n − 1)c

T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c
..
.
T (2) = T (1) + c
T (n) + T (n − 1) + T (n − 2) + · · · + T (2) =
T (n − 1) + T (n − 2) + · · · + T (2) + T (1) + (n − 1)c
T (n) +
T (n 1) +
− T (n −2) + · · · +
T (2)
=

T (n
− 1) +
T (n
− 2) + · · · + T (2)
+ T (1) + (n − 1)c


T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c
..
.
T (2) = T (1) + c
T (n) + T (n − 1) + T (n − 2) + · · · + T (2) =
T (n − 1) + T (n − 2) + · · · + T (2) + T (1) + (n − 1)c
T (n) +
T (n 1) +
− T (n −2) + · · · +
T (2)
=

T (n
− 1) +
T (n
− 2) + · · · + T (2)
+ T (1) + (n − 1)c

T (n) = T (1) + (n − 1)c

T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c
..
.
T (2) = T (1) + c
T (n) + T (n − 1) + T (n − 2) + · · · + T (2) =
T (n − 1) + T (n − 2) + · · · + T (2) + T (1) + (n − 1)c
T (n) +
T (n 1) +
− T (n −2) + · · · +
T (2)
=

T (n
− 1) +
T (n
− 2) + · · · + T (2)
+ T (1) + (n − 1)c

T (n) = T (1) + (n − 1)c

= d + (n − 1)c

T (n) = T (n − 1) + c
T (n − 1) = T (n − 2) + c
..
.
T (2) = T (1) + c
T (n) + T (n − 1) + T (n − 2) + · · · + T (2) =
T (n − 1) + T (n − 2) + · · · + T (2) + T (1) + (n − 1)c
T (n) +
T (n 1) +
− T (n −2) + · · · +
T (2)
=

T (n
− 1) +
T (n
− 2) + · · · + T (2)
+ T (1) + (n − 1)c

T (n) = T (1) + (n − 1)c

= d + (n − 1)c
It is obvious that this T (n) is Θ(n)

Another Example
Consider another simple case of divide-conquer-merge, where the
merge cost is not constant (for example, insertion sort)

Another Example
The recurrence equation may be obtained as:
(
d n≤1
T (n) =
T (n − 1) + cn n>1

Another Example
(
d n≤1
T (n) =
T (n − 1) + cn n>1
Let us solve this by telescopic sum method:
T (n) = T (n − 1) + cn

Another Example
(
d n≤1
T (n) =
T (n − 1) + cn n>1
T (n) = T (n − 1) + cn
T (n − 1) = T (n − 2) + c(n − 1)

Another Example
(
d n≤1
T (n) =
T (n − 1) + cn n>1
T (n) = T (n − 1) + cn
T (n − 1) = T (n − 2) + c(n − 1)
..
.
T (2) = T (1) + 2c

Another Example
(
d n≤1
T (n) =
T (n − 1) + cn n>1
T (n) = T (n − 1) + cn
T (n − 1) = T (n − 2) + c(n − 1)
..
.
T (2) = T (1) + 2c

Another Example
(
d n≤1
T (n) =
T (n − 1) + cn n>1
T (n) = T (n − 1) + cn
T (n − 1) = T (n − 2) + c(n − 1) T (n) = T (1) + c(2 + 3 + · · · + n)
..
.
T (2) = T (1) + 2c

Another Example
(
d n≤1
T (n) =
T (n − 1) + cn n>1
T (n) = T (n − 1) + cn
T (n − 1) = T (n − 2) + c(n − 1) T (n) = T (1) + c(2 + 3 + · · · + n)
..
. = T (1) + c( n(n+1) − 1) 2
T (2) = T (1) + 2c

Another Example
(
d n≤1
T (n) =
T (n − 1) + cn n>1
T (n) = T (n − 1) + cn
T (n − 1) = T (n − 2) + c(n − 1) T (n) = T (1) + c(2 + 3 + · · · + n)
..
. = T (1) + c( n(n+1) − 1) 2
T (2) = T (1) + 2c
It is obvious that this T (n) is Θ(n2 )

Yet Another Example
Let us consider the following recurrence equation:

(
d n≤1
T (n) =
2T (n/2) + n n>1

Yet Another Example
Let us consider the following recurrence equation:

(
d n≤1
T (n) =
2T (n/2) + n n>1
This arises, for example, in the divide and conquer algorithms such as
mergesort
The input is divided into two halves and both the halves are solved
independently
Then both the solutions are merged in linear time to get the overall
solution

Yet Another Example — Expansion
Let’s first try the expansion method
T (n) = 2T (n/2) + n n>1

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2
= 22 T (n/22 ) + 2n n>2

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2
= 22 T (n/22 ) + 2n n>2
= 22 [2T (n/23 ) + n/22 ] + 2n n > 22

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2
= 22 T (n/22 ) + 2n n>2
= 22 [2T (n/23 ) + n/22 ] + 2n n > 22
= 23 T (n/23 ) + 3n n > 22

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2
= 22 T (n/22 ) + 2n n>2
= 22 [2T (n/23 ) + n/22 ] + 2n n > 22
= 23 T (n/23 ) + 3n n > 22
..
.

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2
= 22 T (n/22 ) + 2n n>2
= 22 [2T (n/23 ) + n/22 ] + 2n n > 22
= 23 T (n/23 ) + 3n n > 22
..
.
= 2i T (n/2i ) + in n > 2i−1

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2
= 22 T (n/22 ) + 2n n>2
= 22 [2T (n/23 ) + n/22 ] + 2n n > 22
= 23 T (n/23 ) + 3n n > 22
..
.
= 2i T (n/2i ) + in n > 2i−1
When i = log n
T (n) = nT (1) + n log n

T (n) = 2T (n/2) + n n>1
= 2[2T (n/4) + n/2] + n n>2
= 22 T (n/22 ) + 2n n>2
= 22 [2T (n/23 ) + n/22 ] + 2n n > 22
= 23 T (n/23 ) + 3n n > 22
..
.
= 2i T (n/2i ) + in n > 2i−1
When i = log n
T (n) = nT (1) + n log n
nO(1) + O(n log n)

O(n) + O(n log n)
O(n log n)
Yet Another Example — Telescoping
T (n) = 2T (n/2) + n

T (n) = 2T (n/2) + n
T (n) 2T (n/2)
n = n +1

T (n) = 2T (n/2) + n
T (n) 2T (n/2)
n = n +1
T (n) T (n/2)
n = n/2 +1

T (n) = 2T (n/2) + n
T (n) 2T (n/2)
n = n +1
T (n) T (n/2)
n = n/2 +1
T (n/2) T (n/4)
n/2 = n/4 +1
T (n/4) T (n/8)
n/4 = n/8 +1
..
.
T (2) T (1)
2 = 1 +1

T (n) = 2T (n/2) + n
T (n) 2T (n/2)
n = n +1
T (n) T (n/2)
n = n/2 +1
T (n/2) T (n/4)
n/2 = n/4 +1
T (n/4) T (n/8)
n/4 = n/8 +1
..
.
T (2) T (1)
2 = 1 +1

T (n) = 2T (n/2) + n
T (n) 2T (n/2)
n = n +1
T (n) T (n/2)
n = n/2 +1
T (n/2) T (n/4)
n/2 = n/4 +1
T (n/4) T (n/8)
n/4 = n/8 +1
..
.
T (2)
2 = T 1(1) + 1
T (n)
n = T (1) + log n
T (n) = cn + n log n

T (n) = 2T (n/2) + n
T (n) 2T (n/2)
n = n +1
T (n) T (n/2)
n = n/2 +1
T (n/2) T (n/4)
n/2 = n/4 +1
T (n/4) T (n/8)
n/4 = n/8 +1
..
.
T (2)
2 = T 1(1) + 1
T (n)
n = T (1) + log n
T (n) = cn + n log n
O(n log n)
Slightly Complicated Example
Let us consider the following recurrence equation (where T (1) is a

constant):  
2 n−1
T (n) = T (j) + cn
X
n j=0

Let us consider the following recurrence equation (where T (1) is a

constant):  
2 n−1
T (n) = T (j) + cn
X
n j=0
Any idea how we may arrive at such a recurrence!?

 
2 n−1
T (n) =  T (j) + cn
X
n j=0

 
2 n−1
T (n) =  T (j) + cn
X
n j=0
 
n−1
nT (n) = 2  T (j) + cn2
X
j=0

 
2 n−1
T (n) =  T (j) + cn
X
n j=0
 
n−1
nT (n) = 2  T (j) + cn2
X
j=0
 
n−2
(n − 1)T (n − 1) = 2  T (j) + c(n − 1)2
X
j=0

 
2 n−1
T (n) =  T (j) + cn
X
n j=0
 
n−1
nT (n) = 2  T (j) + cn2
X
j=0
 
n−2
(n − 1)T (n − 1) = 2  T (j) + c(n − 1)2
X
j=0
— — — — — — — — — — — — — —Subtract

 
2 n−1
T (n) =  T (j) + cn
X
n j=0
 
n−1
nT (n) = 2  T (j) + cn2
X
j=0
 
n−2
(n − 1)T (n − 1) = 2  T (j) + c(n − 1)2
X
j=0
— — — — — — — — — — — — — —Subtract
nT (n) − (n − 1)T (n − 1) = 2T (n − 1) + 2cn − c

 
2 n−1
T (n) =  T (j) + cn
X
n j=0
 
n−1
nT (n) = 2  T (j) + cn2
X
j=0
 
n−2
(n − 1)T (n − 1) = 2  T (j) + c(n − 1)2
X
j=0
— — — — — — — — — — — — — —Subtract
nT (n) − (n − 1)T (n − 1) = 2T (n − 1) + 2cn − c
nT (n) = (n + 1)T (n − 1) + 2cn

nT (n) = (n + 1)T (n − 1) + 2cn

nT (n) = (n + 1)T (n − 1) + 2cn

Now, to telescope divide by n(n + 1)
T (n) T (n − 1) 2c
= +
n+1 n n+1


Master Theorem
Let T (n) = aT (n/b) + f (n), where a ≥ 1 and b > 1

Master Theorem

If f (n) is O(nlogb (a−ϵ) ) with ϵ > 0, then T (n) is Θ(nlogb a )

Master Theorem

When a = b, if f (n) is sub-linear time, then T (n) is Θ(n)

Master Theorem

If f (n) is Θ(nlogb a ), then T (n) is Θ(nlogb a lg n)

Master Theorem

When a = b, if f (n) is linear time, then T (n) is Θ(n log n)

Master Theorem

If f (n) is Ω(nlogb (a+ϵ) ) with ϵ > 0 and if af (n/b) ≤ cf (n) for some
c < 1, then T (n) is Θ(f (n))

Master Theorem

If f (n) is Ω(nlogb (a+ϵ) ) with ϵ > 0 and if af (n/b) ≤ cf (n) for some
c < 1, then T (n) is Θ(f (n))
When a = b, if f (n) is more than linear time, then T (n) is Θ(f (n))

Summary
Time / Space complexity of an algorithm are expressed in notations

such as Big-Oh and Big-Theta
These notations bring out the growth rate of time / space wrt the
size of the input
These notations enable us to avoid exact calculations of number of
“basic steps” or memory space required — overall growth rate can be
estimated based on growth rates of components
Complexity of recursive algorithms can be analyzed through the
corresponding recurrence equations
Recurrences may be solved by expansion method, telescopic sum
method, or by solving corresponding characteristic equations
It is also possible to perform empirical ratio analysis to determine /
verify time complexity of an algorithm

Recurrences

Uploaded by

Copyright:

Available Formats

Recurrences

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Recurrences

Uploaded by

Copyright:

Available Formats

CS1006T Data Structures

Analysis of Recursive Algorithms

Professor of Information Technology

March 12, 2024

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 1 / 14

Consider a simple case of divide-conquer-merge, where the input of

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 2 / 14

Consider a simple case of divide-conquer-merge, where the input of

Here, the merge cost is constant

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 2 / 14

Consider a simple case of divide-conquer-merge, where the input of

Here, the merge cost is constant

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 2 / 14

Consider a simple case of divide-conquer-merge, where the input of

Here, the merge cost is constant

This is a very simple recurrence equation.

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 2 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

The objective is to represent T (n) directly in terms of the base case

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

The objective is to represent T (n) directly in terms of the base case

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

The objective is to represent T (n) directly in terms of the base case

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

The objective is to represent T (n) directly in terms of the base case

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 3 / 14

The objective is to represent T (n) directly in terms of the base case

It is obvious that this T (n) is Θ(n)

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

T (n) = T (1) + (n − 1)c

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

T (n) = T (1) + (n − 1)c

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

T (n) = T (1) + (n − 1)c

It is obvious that this T (n) is Θ(n)

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 4 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14

Let us solve this by telescopic sum method:

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14

Let us solve this by telescopic sum method:

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14

Let us solve this by telescopic sum method:

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14

Let us solve this by telescopic sum method:

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14

Let us solve this by telescopic sum method:

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14

Let us solve this by telescopic sum method:

C. Aravindan (SSN Institutions) Data Structures March 12, 2024 5 / 14