5.

Induction and Recursion

5.1 Mathematical Induction

Consider the sum of the first n positive odd numbers:

1=1, 1+3=4, 1+3+5=9, 1+3+5+7=16, 1+3+5+7+9 = 25

Is it n2?

Induction is a powerful tool to prove assertions of this type.

 Example:
P(n): The sum of the first n positive
Mathematical Induction: odd numbers is n2

Prove the theorem: P(n) is true n Z+

Proof by induction:
1. Basis step P(1) is shown to be true
2. Inductive step P(n) → P(n+1) is shown to be true nZ + Assume P(n) is true,
show P(n+1) is also true

Then we apply the following rule of inference,

[P(1)  n (P(n) → P(n+1))] → n P(n)

to conclude that P(n) is true nZ +

Z+ denotes the set of all positive integers

e.g.
P(n) : The sum of the first n positive odd integers is n2.
Prove P(n) is true nZ + .
Basis step:
P(1): 1 = 12 (True)

Inductive step:

? P(n) → P(n+1) nZ +

Suppose, for a fixed arbitrary n  1, P(n) is T, i.e., 1 + 3 +…+ (2n1) = n2

Then, show that P(n+1) is also T.
1 + 3 + … + (2n1) + (2n+1) = (n+1)2 ?

1 + 3 + … + (2n1) = n2 (by inductive hypothesis)

 1 + 3 + … + (2n1) + (2n+1) = n2 + (2n+1)
= (n+1)2  P(n+1) is T

 By induction, P(n) is T nZ +

Remark: Here, "P(n) is true for a fixed arbitrary n” is our inductive hypothesis.
e.g.
Prove n < 2n nZ +

Basis step:

P(1): 1 < 21 = 2 (T)

Inductive step:

Show P(n) → P(n+1) nZ +

n < 2n → n+1 < 2(n+1) ?

Start with inductive hypothesis n < 2n and add 1 to both sides

 n+1 < 2n +1
 n+1 < 2n +1 < 2n + 2n = 2n+1 since n  1
  P(n+1) is T
 By induction, n < 2n  n  Z +
e.g. Inequality for Harmonic Numbers:

Harmonic number:
Hk = 1 + 1/2 + 1/3 + ∙∙∙ +1/k, k =1, 2, 3…

Show that H2n  1 + n/2 n, n is a nonnegativeinteger.

Basis step:
P(0) is true, since H20 = H1 = 1  1 + 0/2 = 1.

Inductive step:
Assume P(n) is true  H2n  1 + n/2

H2n+1 = H2n + 1/(2n + 1) + … + 1/(2n+1)

 (1 + n/2) + 1/(2n + 1) + …. + 1 /(2n+1)
 (1 + n/2) + 2n (1/2n+1) = 1 + (n+1) / 2
 P(n+1) is true.
Hence by induction H2n  1 + n/2 n
e.g.
Prove that 2n < n! for n = 4, 5, 6, …

Let P(n): 2n < n!

Basis step:

P(4) is true since 24 = 16 < 4! = 24

Inductive step:

Assume P(n) is true: 2n < n! for a fixed arbitrary n  4

 2n+1 < 2n!
< (n+1) n!
= (n+1)!  P(n+1) is true.

Hence by induction 2n < n! for n = 4, 5, 6, ….

5.2 Strong Induction
1. Basis step:
Show that P(1) is true.

2. Inductive step:
Show that [P(1)  P(2)  …  P(n)] → P(n+1) is true nZ + .

Hence, the inductive hypothesis here is stronger than the previous

version and assumes that the statement is true for all integers up to n.
 Note that in the following example,
the starting case is n=2, not n=1.
1. Basis step: Show that P(1) is true.
2. Inductive step: Show that [P(1)  P(2)  …  P(n)] → P(n+1) is true nZ + .

e.g. Show that if n > 1 integer, then n is either prime or can be written as a product of primes.
P(n): n is either prime or can be written as product of primes.
Basis step: P(2) is true since 2 is prime itself. This means the
Inductive step: Assume P(k) is true for all 2  k  n, for a fixed arbitrary n  2. statement is true for
all integers up to n.
Show P(n+1) is also true.
There are two cases to consider: (hence, use proof by cases)

i) If (n+1) is prime then P(n+1) is already true.

ii) If (n+1) is composite then n+1 = a·b s.t. 1< a  b < n+1.
We know that P(a) and P(b) are true by inductive hypothesis since a  b  n.
 Each of a and b is either prime or can be written as product of primes.
 a∙b can also be written as product of primes.
 P(n+1) is also true.

 P(n) is T n >1 by strong induction

This proves the existence part of the Fundamental Theorem of Arithmetic (see previous lectures, Ch. 4).
5.3 Recursive Definitions

A function can often be defined also recursively:

1. Specify the value of the function at the beginning, e.g., at zero.

2. Give a rule for finding its value based on its previous values.

generates a sequence
e.g. of numbers a0 a1 a2 a3 a4
an = 2n n = 0, 1, 2,… 1, 2, 4, 8, 16,…

We can also generate the same sequence via recursion:

 an+1 = 2an a0 =1, n = 0, 1, 2, 3,….

e.g.
F(n) = n! can be defined recursively:
F(0) = 1
F(n+1) = F(n)(n+1) n = 0, 1, 2, 3,….
e.g.
Fibonacci numbers: fn1  1 5
f0 = 0, f1 = 1 lim  Golden Ratio
n fn 2
fn +1 = fn + fn-1 n = 1, 2, 3, …

A sequence of numbers that can be generated via recursion: 0, 1, 1, 2, 3, 5, 8, 13, …

Golden ratio, found in nature, used in art and architecture

 Fibonacci numbers:
e.g. Show that fn > αn–2 n ≥ 3, where α = (1+ 5)/2. f0 = 0, f1 = 1
Note that α is a solution of x2 – x – 1 = 0. fn+1 = fn + fn-1 n = 1, 2,…
Use strong induction.

Let P(n): fn > αn–2

Basis step: n = 3  α < 2 = ƒ3  P(3) is true
n = 4  α2 = (3+ 5)/2 < 3 = ƒ4  P(4) is true

Inductive step: Assume ƒk > αk–2 k, 3  k  n, where n  4.

ƒn+1 > αn–1 ?
Since α is a solution of x2 – x – 1 = 0, α2 = α+1
 αn–1 = α2 ∙αn–3 = (α+1) αn–3 = αn–2 + αn–3

By inductive hypothesis, ƒn > αn–2 and ƒn–1 > αn–3

 ƒn+1 = ƒn + ƒn–1 > αn–2 + αn–3 = αn–1
 fn > αn–2 n ≥ 3 by strong induction
Remark: In this example, the basis step involves checking the statement for two initial cases, n = 3 and n = 4,
since the proof at the inductive step is based on the assumption that the statement is true for two previous cases.

(so that we can generalize the statement to all n by strong induction starting from n = 3 and n = 4)
 5.4 Recursive Algorithms

Definition:
An algorithm is called recursive if it solves a problem by reducing it to an instance of
the same problem with smaller input.

e.g. Compute gcd(a,b), a < b

Example:
b a
procedure gcd(a, b: nonnegative integers with a < b) 155 = 125× 1 + 30
if a = 0 then return b 125 = 30× 4 +5
else return gcd(b mod a, a) 30 = 5× 6
{output is gcd(a, b)}  gcd (125, 155) = gcd (30,125)
= gcd (5,30) = gcd (0,5) = 5

Recall from Number Theory:

Recursive implementation of Euclidean algorithm 155 = 125× 1 + 30

if c | 155 and c | 125  c | 30

 gcd (125, 155) = gcd (30,125)
e.g. Compute an, aR and nN
a0 = 1, an+1 = aan

procedure power (a: nonzero real number, n: nonnegative integer)

if n = 0 then return 1
else return a ⋅ power(a, n − 1)
{output is an}
e.g. Linear search x in {a1, a2, …, an}

The input to the procedure is the triplet (1, n, x)

procedure search(i, j, x: integers, 1 ≤ i ≤ j ≤ n)

if ai = x then
return i
else if i = j then
return 0
else
return search(i + 1, j, x)
{output is the location of x in a1, a2,…, an if it appears; otherwise it is 0}
e.g. Binary search

procedure binary search(i, j, x: integers, 1 ≤ i ≤ j ≤ n)

m := ⌊(i + j)/2⌋
if x = am then
return m
else if (x < am and i < m) then
return binary search(i, m − 1, x) (search in the first half)
else if (x > am and j > m) then
return binary search(m + 1, j, x) (search in the second half)
else return 0
{output is location of x in a1, a2, ... , an if it appears; otherwise it is 0}.

Initially, i = 1 and j = n.
e.g. Compute n!, nN
0! = 1, (n+1)! = (n+1)n!

procedure factorial(n: nonnegative integer)

if n = 0 then return 1
else return n ⋅ factorial(n − 1)
{output is n!}
Recursion vs. Iteration:

e.g. Computing factorial

Recursive:

procedure factorial(n: nonnegative integer)

if n = 0 then return 1
else return n ⋅ factorial(n − 1)
{output is n!}

Iterative:

procedure factorial(n: nonnegative integer)

fact := 1
for i := 1 to n
fact = fact ⋅ i
return fact
{output is n!}

All recursive algorithms have an iterative equivalent, and vice versa.

Recursion vs. Iteration:

e.g. Computing factorial Let T(n) be the time complexity of the recursive
solution (in terms of comparison and arithmetic
Recursive: operations).

T(n) = T(n 1) + 3 for n ≥ 1 with T(0) = 1.

procedure factorial(n: nonnegative integer)
if n = 0 then return 1
Then T(n) = T(n 2) + 3 + 3
else return n ⋅ factorial(n − 1)
{output is n!} = T(n 3) + 3 + 3 + 3
.
.
.
Iterative: = T(0) + 3 + 3 + … + 3
= 3n+1
procedure factorial(n: nonnegative integer)
fact := 1 Hence T(n) is O(n).
for i := 1 to n
fact = fact ⋅ i
return fact
{output is n!}

Iterative solutions are usually faster than recursive solutions, though in this example
both algorithms are of O(n) complexity.
e.g. Computing Fibonacci numbers

Recursive:
procedure fibonacci(n: nonnegative integer)
if n = 0 then return 0
else if n = 1 then return 1 ?
else return fibonacci(n − 1) + fibonacci(n − 2)
{output is fibonacci(n)}

Iterative:
procedure iterative fibonacci(n: nonnegative integer)
if n = 0 then return 0
else
x := 0
y := 1
for i := 1 to n − 1 O(n) complexity
z := x + y
x := y
y := z
return y
{output is the nth Fibonacci number}
Complexity analysis for recursive algorithms:
Recursive solution for computing Fibonacci numbers seems more clever and compact;
however the iterative solution is much more efficient!!!

Let T(n) be the time complexity of the recursive solution (in terms of comparison
and arithmetic operations).
Then T(n) = T(n1) + T(n2) + 5 for n ≥ 2 with T(0) = 1 and T(1) = 2.
(The term 5 counts for 2 comparisons plus 1 addition plus 2 subtractions).

procedure fibonacci(n: nonnegative integer)

if n = 0 then return 0
else if n = 1 then return 1
else return fibonacci(n − 1) + fibonacci(n − 2)
{output is fibonacci(n)}
Complexity analysis for recursive algorithms:
Recursive solution for computing Fibonacci numbers seems more clever and compact;
however the iterative solution is much more efficient!!!

Let T(n) be the time complexity of the recursive solution (in terms of comparison
and arithmetic operations).
Then T(n) = T(n1) + T(n2) + 5 for n ≥ 2 with T(0) = 1 and T(1) = 2.
(The term 5 counts for 2 comparisons plus 1 addition plus 2 subtractions).
Since ƒn = ƒn-1 + ƒn-2 with ƒ0 = 0 and ƒ1 = 1, n T(n)  ƒn
Also ƒn > αn-2 n ≥ 3, α = (1+ 5)/2 (see previous slides)
 T(n) > αn-2 n ≥ 3
 Exponential complexity (αn) or worse.
(Or you can simply use big-Omega notation: T(n) is (αn).)

Note that complexity of the iterative solution is (n) (big-Theta).

Moral of the story: Don’t compute anything more than once!

e.g. What is the complexity of this algorithm?

procedure compute(n: nonnegative integer)

if n = 0 then return 1
else if n = 1 then return 2
else return compute(n − 1) ⋅ compute(n − 2)
{output is compute(n)}

The same as before:

T(n) = T(n1) + T(n2) + 5 for n ≥ 2 with T(0) = 1 and T(1) = 2.

Correctness of recursive algorithms:
We can use mathematical induction also to show that a given recursive algorithm
produces the correct (i.e., the desired) output.

Consider the recursive factorial example:

procedure factorial(n: nonnegative integer)
if n = 0 then return 1 Is it correct?
else return n ⋅ factorial(n − 1)
{output is n!}

We have to show that the recursive code works correctly for all non-negative n, i.e.,
P(n) is true n, where P(n): “The output of the algorithm is n!”

Basis step:
P(0): “ The output is 1” (True) 0! = 1 by definition of factorial

Inductive step:
Show P(n) → P(n+1) n ≥ 0, or equivalently show P(n1) → P(n) n ≥1.
We assume factorial(n1) correctly returns (n1)! and show factorial(n) returns (n!).
For n, the output is, as seen from the code, n ⋅ factorial(n − 1) = n·(n1)! by inductive hypothesis
= n!  P(n) is T
 By induction P(n) is true n ≥0
e.g. Show that the following recursive function correctly computes the n-th Fibonacci
number (Exercise: use strong induction).

procedure fibonacci(n: nonnegative integer)

if n = 0 then return 0
else if n = 1 then return 1
else return fibonacci(n − 1) + fibonacci(n − 2)
{output is fibonacci(n)}
e.g. Show that the following recursive function correctly returns gcd(a,n), a < n, for all
non-negative integers a and n.
procedure GCD(a, n: nonnegative integers with a < n) Here, gcd denotes greatest common
if a = 0 then return n divisor; GCD is the function that
else return GCD(n mod a, a) supposedly computes gcd.
{output is gcd(a, n)}

Let P(n): The function GCD(a, n) correctly computes gcd(a, n) whenever a < n, n > 0.
Basis step: P(1) is true because the only possible case for a is a = 0 (note a < n). Looking
into the code, we see that the algorithm returns 1 in this case, as expected.
Inductive step: Assume P(k) for all 1≤ k ≤ n and check whether GCD(a, n+1) returns
gcd(a, n+1) for any a < n+1.
We see from the code that if a = 0, the function returns n+1, which is correct.
Else, the function calls itself with input ((n+1) mod a, a), and this call generates gcd((n+1)
mod a, a) correctly because P(a) is true by inductive hypothesis for a < n+1.
Since gcd(a, n+1) = gcd((n+1) mod a, a) from what we’ve learned in number theory,
GCD(a, n+1) returns gcd(a, n+1).
Hence, by strong induction, we conclude that P(n) is true for all n; that is, GCD(a, n)
returns gcd(a, n) whenever a < n, n>0.
8.1 & 8.2 Recurrence Relations

Definition:
A recurrence relation for the sequence {an} is an equation that expresses an in terms of one or more
previous terms of the sequence a0, a1, ..., an-1. A sequence is called a solution of a recurrence
relation if its terms satisfy the recurrence relation.

The recurrence relation together with the initial conditions uniquely determine a sequence, i.e., a
solution.

e.g. Consider the recurrence relation (Fibonacci numbers):

fn = fn-1 + fn-2 , n ≥ 2, n  Z
where f0 = 0 and f1 = 1.

Then {0,1,1,2,3,5,8,…} is the solution.

Q: Is it possible to find an explicit formula for an?

In some conditions yes!
e.g., Given recurrence relation: an = 2an-1 – an-2 for n:2,3,4,…
Determine if {an} with an=3n is a solution.

an = 2an-1 – an-2 = 2(3(n-1)) – 3(n-2) = 6n – 6 – 3n + 6 = 3n = an

e.g., For the recurrence relation above, is an=2n a solution? an=5?

e.g., Suppose you deposit $10000 to a savings account at a bank yielding 11% interest per year,
compounded annually. How much money will you get back after 30 years?

an = 1.11 an-1
a0 = 10000

If we iterate:
a1 = 1.11 a0
a2 = 1.11 a1 = (1.11)2 a0
…
an = (1.11)n a0

a30 = (1.11)30 10000 = $228922,97

(What if interest is not compounded?)
e.g., Rabbit Population
A pair of baby rabbits, one male one female, was placed on an island. A rabbit does not breed
until it is an adult (2 months old). Then, every pair produces a new pair (if there were n adult
pairs, n new baby rabbit pairs will be produced).
The recurrence relation for the rabbit population?

a0 = 1 pair
a1 = 1 pair
a2 = 2 pairs (1 adult, 1 baby)
a3 = 3 pairs (1 adult, 2 baby)
a4 = 5 pairs (2 adult, 3 baby) WHY??
an = an-1 + an-2 Fibonacci numbers

Rabbits from last month continue to live, and rabbits from 2 months ago breed and bring that
many new rabbits.
e.g., Towers of Hanoi
See example 2 in Section 8.1.2 of your book.
This is a good example of a very common and important problem type.
Your computers won’t even be able to solve it for n = 64.
8.5 Inclusion-Exclusion

Consider set union. We have:

|A  B| = |A| + |B| – |A  B|
since we count the intersection twice when we count the elements in A and in B.
The Principle of Inclusion-Exclusion is a generalization of this.

e.g., How many positive integers not exceeding 1000 are divisible by 7 OR 11 ?
Divisible by 7: |A| = 1000/7 = 142
Divisible by 11: |B| = 1000/11 = 90
Divisible by 7 AND 11: |A  B| = 1000/(7*11) = 12
Divisible by 7 OR 11: |A  B| = 142 + 90 – 12 = 220

e.g., There are 951 freshman at KU.

158 freshmen are taking COMP courses
643 freshmen are taking MATH courses
65 freshmen are taking COMP and MATH courses.
How many freshmen are NOT taking any COMP or MATH courses?
Answer: 215 (WHY??)
Theorem: The Principle of Inclusion-Exclusion
Let A1, A2, A3, …, An be finite sets. Then,

Remarks: