COT 4400

Algorithms - Proof review

Oguzhan Topsakal

Lecture 1
Outline
• Algorithms
• Desirable Properties for an Algorithm
• Proof review
– Induction
– Strong induction

2
 Story

What
could be
the story
of this
image?

3
 Story - Algorithms
In a small village, a wise old skillful
carpenter named Elias lived and
started to tell the secret of his work to
Leo, his apprentice.
"let me show you the blueprint. This is "In life, we face many
like a recipe, a set of steps that will challenges, and having
guide you through the process. a good algorithm helps
This, my young friend, is called an us, - whether it's for
algorithm." building furniture,
cooking a meal, or
"An algorithm?" Leo asked, puzzled.
solving a puzzle.
"What does that mean?”
Helps to achieve our
"An algorithm is simply a step-by- goals efficiently and
step procedure for solving a correctly. The better
problem. It’s like a map that leads your algorithm, the
you from where you are to where faster and more
you want to be. In our case, it will reliably you can reach
4
guide us from a pile of wood to a
What is an algorithm?
• algorithm:
– a well-defined, step-by-step procedure for solving a
problem
– can be more than just computer programs

• problem:
– General task in which we are given some input and need
to compute some corresponding output.
– An algorithmic problem is specified by
• describing the set of instances (inputs) it must work on
• what desired properties the output must have.

• many algorithms possible for a given problem

– E.g., the sorting problem can be solved using bubblesort,
insertion sort, selection sort, quicksort, mergesort, heap
sort, radix sort, etc, algorithms.
5
Algorithm Example
The minimum problem: given a set of values that can be
compared, output the smallest value in that set
• instance: a particular input for a problem
• solution: the corresponding output for a problem instance
– Examples:
• min(5, 3, 14) -> 3
• min(-1, -1, -1, -1, -1) -> -1
• min("Gallup", "Trot", "Canter") -> “Canter”

The sorting problem:

• Input: A sequence of N keys a1...aN
• Output: the permutation (reordering) of the input sequence such
that a1l ≤ a2l ≤ . . . ≤ aNl .
– Examples:
• sort( [7, 6, 11, −3, 56] ) -> [-3, 6, 7, 11, 56]
• sort( [bill, mike, bob] ) -> [bill, bob, mike]
6
Let's look at a concrete example
• Problem: sorting a list of numbers

Formal
definition of
problem

Pseudocode

7
How can we express an algorithm?

We need some way to express the sequence of steps

comprising an algorithm.

Our options, in increasing order of precision

1. English (natural language)
2. Pseudocode (a programming language that doesn’t
complain about syntax errors)
• Algorithm description between code and
English
• Code-like to eliminate ambiguity
• English-like for simplicity
3. Real programming languages.
Desirable Properties for an Algorithm

• Correct

• Efficient

• Easy to Implement

Some other properties that might be desired:

• Scalability,
• Robustness,
• Determinism,
• Optimality,
• Parallelizability
Correctness
For any algorithm, we must prove that it always returns
the desired output for all legal instances of the problem.
For sorting, this means even if the input is already
sorted, or it contains repeated elements.

How can we determine correctness?

- Formal Verification (Proofs)
- Software Testing

September 30,
Proofs vs. software testing
• Why don't we just test algorithms?
1. Testing does not guarantee correctness
2. Algorithms ≠ computer programs
3. Coding an algorithm is time-consuming and error-prone

• Why don't we use proofs for all programs?

1. Problem not always easy to define mathematically
2. Realistic programs are large and complex

11
 Ask a question to your AI Companion
• Ask your course/syllabus related questions
anonymously to:

• https://topsakal.com/usf/cot4400/

12
 How do we determine correctness?
• Correct
– For every input, algorithm must terminate with solution
• Proof of correctness
1. Start with arbitrary input
2. Describe what the algorithm does
• If statements: proof by cases
• For/do/while loops: often need loop invariants
• Recursive function calls: induction or strong induction
• Other function calls: assume achieves the desired subgoals
or subtasks
3. Prove that output meets problem criteria (and
terminates)
• Sometimes you can use contradiction
– Assume claim is false
– Show something impossible or logically inconsistent must
be true
– Often used for optimization algorithms (max/min, 13
Proof by Induction
• Proof technique for statements of the form "P(i), for all i ≥
b."
• Base case
– Prove claim at b
• Induction hypothesis (weak)
– Suppose claim is true for some k ≥ b
• Induction goal
– Prove claim for k+1

• Strong induction changes the induction hypothesis

– Suppose claim is true for all n from b up to k, for some
k≥b

• Multiple base cases

– Needed when assuming claim for some strong induction
proofs
14
– Suppose claim is true from min base case to k, k ≥ max
Induction exercise

P(i), for all i ≥ b

Base case:
prove P(b)
Inductive step:
assume P(k) for some k ≥ b
prove P(k+1)

15
Induction exercise

• Base case:
5! = 120 and 25+1 = 26 = 64
Since 120 > 64, Base case is satisfied for n! > 2n+1
for n>=5
• Inductive step:
Assume that k! > 2k+1 , for some k>= 5 and consider
(k+1)!
(k + 1) ! = (k+1) k!
(k+1) k! > (k+1) 2k+1
Can replace k+1 with 2 since k+ 1 > 2, because
k>=5
> 2 (2k+1) (= 2k+2)
16
 Strong Induction exercise
• Consider a two-player game where
stones are arranged into piles.
• On their turn, a player chooses a
pile with two or more stones and
divides it into two piles, each with
at least one stone.
• The game ends when all of the piles
onlythe
Prove have one stone.
following:
A pile with n stones will always take n – 1 turns to
split up into n piles of one stone, for all n ≥ 1.

P(i), for all i ≥ b

Base case: prove P(b)
Inductive step: assume P(x) for all x from b up to
17
k
prove P(k+1)
 Strong Induction exercise
Prove the following:
A pile with n stones will always take n – 1 turns to
split up into n piles of one stone, for all n ≥ 1.
• Base case:
n=1, will take no moves to split it into piles
of 1, 0=n-1 moves
Assume: that all piles of x stones take x-1
moves to divide into piles of size 1, for any
x<=k

Pile with k+1 stones • Divide this pile into piles of size a
and b are both at least one and a+b
= k+1
• a or b can not be any larger than k.
Takes a-1 and b-1 moves to fully
split.
• Total moves: 1 + (a-1) + (b-1)18
• =a+b–1
 Strong Induction exercise
A pile with n stones will always take n – 1 turns to split up into
n piles of one stone, for all n ≥ 1
• Base case:
n=1, will take no moves to split it into piles of 1, 0=n-1
moves
• Inductive step:
Assume: that all piles of x stones take x-1 moves to divide
into piles of size 1, for any x<=k
Consider: a pile with k+1 stones
– On the first move, a player will divide this pile into piles
of size a and b are both at least one and a+b = k+1
– a or b can not be any larger than k. Hence, the pile with
a number of stones will take a-1 moves to fully split, and
the pile with b number of stones will take b-1 moves to
fully split. Total number of moves to split:
1 + (a-1) + (b-1) = a + b – 1 = (k + 1) – 1 = k 19
Proof review
• Induction
– Proof technique for statements of the form "P(i), for all i ≥
b."
– Two parts (both required)
• Base case: prove P(b)
• Inductive step: assume P(k) for some k ≥ b
• Strong induction: assume P(x) for all x from b up to k
• Prove P(k+1)

0 1 2 3 4 • • •

• Contradiction
– Assume claim is false
– Show something impossible or logically inconsistent must
be true
• Example/counterexample
20
– Used to prove existence/nonexistence
 Ask a question to your AI Companion
• Ask your course/syllabus related questions
anonymously to:

• https://topsakal.com/usf/cot4400/

21
 Questions?
Thanks for your attention…
Please make your best effort to learn!
References:
• Credit: Some of the slides were adapted from Dr. Hendrix’s
and Dr. Kouri’s presentations (thanks to them!).
• Books are great resources:

Share your wisdom/feedback for AI Companion:

- AI Companion: https://topsakal.com/usf/cot4400/
- Feedback form: https://forms.gle/F6MQCYqx9Be63Fp79
22