‫ساختمان داده و ا لگور یتم ها‬

‫مبحث سوم‪:‬‬
‫زما ا لگور ا‬
‫ﺳﺠﺎد ﺷﯿﺮﻋیل ﴲﺮﺿﺎ‬
‫ﲠﺎر ‪1402‬‬
‫ﯾﮑﺸﻨﺒﻪ‪ 23 ،‬ﲠﻤﻦ ‪1401‬‬ ‫‪1‬‬
‫‪Based on Stanford CS161 Slides from Summer 2020 by Karey Shi‬‬
 ‫﮳ﺤ﮲ ﮶ﺴﻬﺎی ﻣﺮ ﮴ ﮳ﻂ در ﮐ ﮴ﺎب‬
‫(‪ 2.3 :‬و ‪4.4‬‬ ‫مو‬ ‫ب ) بو‬ ‫●‬
‫زما (‪3 :‬‬ ‫)‬ ‫● ا‬
‫(‪:‬‬ ‫) وس ای ‪ 3‬و ‪ 4‬اب د‬ ‫ا‬ ‫ار و ار‬ ‫● واژه ام ی ا‬
‫‪http://sharif.edu/~ghodsi/books/ds-algf-dics-both.pdf‬‬

‫‪2‬‬
FROM LAST WEEK
THE POINT OF ASYMPTOTIC NOTATION
suppress constant factors and lower-order terms
too system dependent irrelevant for large inputs

● Some guiding principles: we care about how the running time/number of

operations scales with the size of the input (i.e. the runtime’s rate of growth), and
we want some measure of runtime that’s independent of hardware,
programming language, memory layout, etc.
○ We want to reason about high-level algorithmic approaches rather than lower-level details
3
A NOTE ON RUNTIME ANALYSIS
There are a few different ways to analyze the runtime of an algorithm:

Worst-case analysis:
What is the runtime of the algorithm on the worst possible input?

Best-case analysis:
What is the runtime of the algorithm on the best possible input?

Average-case analysis:
What is the runtime of the algorithm on the average input?

4
 A NOTE ON RUNTIME ANALYSIS
There are a few different ways to analyze the runtime of an algorithm:

We’ll mainly Worst-case analysis:

focus on worst
What is the runtime of the algorithm on the worst possible input?
case analysis
since it tells us
how fast the
algorithm is on Best-case analysis:
any kind of What is the runtime of the algorithm on the best possible input?
input
We’ll also work
Average-case analysis: on this in some
What is the runtime of the algorithm on the average input? cases.

5
BIG-O NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is O(f(n))”?

English Pictorial Mathematical

Definition Definition Definition

6
BIG-O NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is O(f(n))”?

In English

T(n) = O(f(n)) if and only if

T(n) is eventually upper
Pictorial Mathematical
bounded by a constant Definition Definition
multiple of f(n)

7
BIG-O NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is O(f(n))”?

In English In Pictures
n0 c·f(n)

T(n) = O(f(n)) if and only if

T(n) is eventually upper
Runtime (ms)
T(n) Mathematical
bounded by a constant
f(n)
Definition
multiple of f(n)

n (input size)
8
BIG-O NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is O(f(n))”?

In English In Pictures In Math

n0 c·f(n)

T(n) = O(f(n)) if and only if T(n) = O(f(n)) if and only if

Runtime (ms)
T(n) is eventually upper
T(n) there exists positive constants
bounded by a constant c and n0 such that for all n ≥ n0
f(n)
multiple of f(n) T(n) ≤ c · f(n)

n (input size)
9
BIG-O NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is O(f(n))”?

In English In Pictures In Math

n0 c·f(n)

T(n) = O(f(n)) if and only if Runtime (ms)

T(n) = O(f(n))
T(n) ⇔
T(n) is eventually upper
bounded by a constant ∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
f(n)
multiple of f(n) T(n) ≤ c · f(n)
n (input size)
10
BIG-O NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is O(f(n))”?

In English In Pictures In Math

n0 c·f(n)

T(n) = O(f(n)) if and only if T(n) = O(f(n))

Runtime (ms)
T(n) “for all”
T(n) is eventually upper
“if and only if” ⇔
bounded by a constant ∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
f(n)
multiple of f(n) T(n) ≤ c · f(n) “such that”
“there exists”
n (input size)
11
PROVING BIG-O BOUNDS
If you’re ever asked to formally prove that T(n) is O(f(n)), use the MATH definition:
T(n) = O(f(n))
⇔
∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
must be constants!
T(n) ≤ c · f(n) i.e. c & n0 cannot
depend on n!

● To prove T(n) = O(f(n)), you need to announce your c & n0 up front!

○ Play around with the expressions to find appropriate choices of c & n0 (positive constants)
○ Then you can write the proof! Here how to structure the start of the proof:

12
PROVING BIG-O BOUNDS
If you’re ever asked to formally prove that T(n) is O(f(n)), use the MATH definition:
T(n) = O(f(n))
⇔
∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
must be constants!
T(n) ≤ c · f(n) i.e. c & n0 cannot
depend on n!

● To prove T(n) = O(f(n)), you need to announce your c & n0 up front!

○ Play around with the expressions to find appropriate choices of c & n0 (positive constants)
○ Then you can write the proof! Here how to structure the start of the proof:

“Let c = __ and n0 = __. We will show that T(n) ≤ c・f(n) for all n ≥ n0.”
13
PROVING BIG-O BOUNDS: EXAMPLE
T(n) = O(f(n))
⇔
∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
T(n) ≤ c · f(n)

Prove that 3n2 + 5n = O(n2).

Let c = 4 and n0 = 5. We will now show that 3n2 + 5n ≤ c・n2 for all n ≥ n0. We know
that for any n ≥ n0, we have:
5≤n
5n ≤ n2
3n2 + 5n ≤ 4n2
Using our choice of c and n0, we have successfully shown that 3n2 + 5n ≤ c・n2 for
all n ≥ n0. From the definition of Big-O, this proves that 3n2 + 5n = O(n2).
14
PROVING BIG-O BOUNDS: EXAMPLE
T(n) = O(f(n))
⇔
∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
T(n) ≤ c · f(n)

Prove that 3n2 + 5n = O(n2).

Let c = 4 and n0 = 5. We will now show that 3n2 + 5n ≤ c・n2 for all n ≥ n0. We know
that for any n ≥ n0, we have:
5≤n
5n ≤ n2
3n2 + 5n ≤ 4n2
Using our choice of c and n0, we have successfully shown that 3n2 + 5n ≤ c・n2 for
all n ≥ n0. From the definition of Big-O, this proves that 3n2 + 5n = O(n2).
15
PROVING BIG-O BOUNDS: EXAMPLE
T(n) = O(f(n))
⇔
∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
T(n) ≤ c · f(n)

Prove that 3n2 + 5n = O(n2).

Let c = 4 and n0 = 5. We will now show that 3n2 + 5n ≤ c・n2 for all n ≥ n0.
We know that for any n ≥ n0=5, we have:
5≤n
5n ≤ n2
3n2 + 5n ≤ 4n2
Using our choice of c and n0, we have successfully shown that 3n2 + 5n ≤ c・n2 for
all n ≥ n0. From the definition of Big-O, this proves that 3n2 + 5n = O(n2).
16
DISPROVING BIG-O BOUNDS
If you’re ever asked to formally disprove that T(n) is O(f(n)), use proof by contradiction!

17
DISPROVING BIG-O BOUNDS
If you’re ever asked to formally disprove that T(n) is O(f(n)), use proof by contradiction!

For sake of contradiction, assume that T(n) is O(f(n)). In other words,

assume there does indeed exist a choice of c & n0 s.t. ∀ n ≥ n0 , T(n) ≤ c · f(n)
pretend you have a friend that comes up and says “I have a c & n0 that will prove T(n) = O(f(n))!!!”,
and you say “ok fine, let’s assume your c & n0 does prove T(n) = O(f(n))”

18
DISPROVING BIG-O BOUNDS
If you’re ever asked to formally disprove that T(n) is O(f(n)), use proof by contradiction!

For sake of contradiction, assume that T(n) is O(f(n)). In other words,

assume there does indeed exist a choice of c & n0 s.t. ∀ n ≥ n0 , T(n) ≤ c · f(n)
pretend you have a friend that comes up and says “I have a c & n0 that will prove T(n) = O(f(n))!!!”,
and you say “ok fine, let’s assume your c & n0 does prove T(n) = O(f(n))”

Treating c & n0 as variables, derive a contradiction!

although you are skeptical, you’ll entertain your friend by saying: “let’s see what happens. [some math work... and then...]
AHA! regardless of what your constants c & n0, trusting you has led me to something impossible!!!”

19
DISPROVING BIG-O BOUNDS
If you’re ever asked to formally disprove that T(n) is O(f(n)), use proof by contradiction!

For sake of contradiction, assume that T(n) is O(f(n)). In other words,

assume there does indeed exist a choice of c & n0 s.t. ∀ n ≥ n0 , T(n) ≤ c · f(n)
pretend you have a friend that comes up and says “I have a c & n0 that will prove T(n) = O(f(n))!!!”,
and you say “ok fine, let’s assume your c & n0 does prove T(n) = O(f(n))”

Treating c & n0 as variables, derive a contradiction!

although you are skeptical, you’ll entertain your friend by saying: “let’s see what happens. [some math work... and then...]
AHA! regardless of what your constants c & n0, trusting you has led me to something impossible!!!”

Conclude that the original assumption must be false, so T(n) is not O(f(n)).
you have triumphantly proven your silly (or lying) friend wrong. 20
DISPROVING BIG-O: EXAMPLE
T(n) = O(f(n))
⇔
Prove that 3n2 + 5n is not O(n). ∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
T(n) ≤ c · f(n)
For sake of contradiction, assume that 3n2 + 5n is O(n). This means that there exists
positive constants c & n0 such that 3n2 + 5n ≤ c・n for all n ≥ n0. Then, we would have
the following:
3n2 + 5n ≤ c・n
3n + 5 ≤ c
n ≤ (c - 5)/3

However, since (c - 5)/3 is a constant, we’ve arrived at a contradiction since n cannot

be bounded above by a constant for all n ≥ n0. For instance, consider n = n0 + c: we see
that n ≥ n0, but n > (c - 5)/3. Thus, our original assumption was incorrect, which means
that 3n2 + 5n is not O(n).

21
DISPROVING BIG-O: EXAMPLE
T(n) = O(f(n))
⇔
Prove that 3n2 + 5n is not O(n). ∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
T(n) ≤ c · f(n)
For sake of contradiction, assume that 3n2 + 5n is O(n). This means that there exists
positive constants c & n0 such that 3n2 + 5n ≤ c・n for all n ≥ n0.
Then, we would have the following:
3n2 + 5n ≤ c・n
3n + 5 ≤ c
n ≤ (c - 5)/3

However, since (c - 5)/3 is a constant, we’ve arrived at a contradiction since n cannot

be bounded above by a constant for all n ≥ n0. For instance, consider n = n0 + c: we see
that n ≥ n0, but n > (c - 5)/3. Thus, our original assumption was incorrect, which means
that 3n2 + 5n is not O(n).

22
DISPROVING BIG-O: EXAMPLE
T(n) = O(f(n))
⇔
Prove that 3n2 + 5n is not O(n). ∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
T(n) ≤ c · f(n)
For sake of contradiction, assume that 3n2 + 5n is O(n). This means that there exists
positive constants c & n0 such that 3n2 + 5n ≤ c・n for all n ≥ n0.
Then, we would have the following:
3n2 + 5n ≤ c・n
3n + 5 ≤ c
n ≤ (c - 5)/3

However, since (c - 5)/3 is a constant, we’ve arrived at a contradiction since n cannot

be bounded above by a constant for all n ≥ n0. For instance, consider n = n0 + c: we see
that n ≥ n0, but n > (c - 5)/3. Thus, our original assumption was incorrect, which means
that 3n2 + 5n is not O(n).

23
DISPROVING BIG-O: EXAMPLE
T(n) = O(f(n))
⇔
Prove that 3n2 + 5n is not O(n). ∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
T(n) ≤ c · f(n)
For sake of contradiction, assume that 3n2 + 5n is O(n). This means that there exists
positive constants c & n0 such that 3n2 + 5n ≤ c・n for all n ≥ n0.
Then, we would have the following:
3n2 + 5n ≤ c・n
3n + 5 ≤ c
n ≤ (c - 5)/3

However, since (c - 5)/3 is a constant, we’ve arrived at a contradiction since n cannot

be bounded above by a constant for all n ≥ n0. For instance, consider n = n0 + c: we see
that n ≥ n0, but n > (c - 5)/3. Thus, our original assumption was incorrect, which means
that 3n2 + 5n is not O(n).

24
BIG-O EXAMPLES

log2n + 15 = O(log2n) 3n = O(4n)

Polynomials
k k-1
Say p(n) = akn + ak-1n + ··· + a1n + a0 is a 6n3 + n log2n= O(n3)
polynomial of degree k ≥ 1.
Then:
i. p(n) = O(nk) 25 = O(1)
ii. p(n) is not O(nk-1)
[any constant] = O(1)
25
BIG-O EXAMPLES
lower order terms remember, big-O
don’t matter! is upper bound!

log2n + 15 = O(log2n) 3n = O(4n)

constant multipliers & lower

Polynomials order terms don’t matter

k k-1
Say p(n) = akn + ak-1n + ··· + a1n + a0 is a 6n3 + n log2n= O(n3)
polynomial of degree k ≥ 1.
Then:
i. p(n) = O(nk) 25 = O(1)
ii. p(n) is not O(nk-1)
[any constant] = O(1)
26
‫ﺳﻮال؟‬
‫‪27‬‬
BIG-Ω NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is Ω(f(n))”?

English Pictorial Mathematical

Definition Definition Definition

28
BIG-Ω NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is Ω(f(n))”?

In English

T(n) = Ω(f(n)) if and only if

T(n) is eventually lower
Pictorial Mathematical
bounded by a constant Definition Definition
multiple of f(n)

29
BIG-Ω NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is Ω(f(n))”?

In English In Pictures
n0 f(n)
T(n) = Ω(f(n)) if and only if
T(n) is eventually lower
Runtime (ms) T(n) Mathematical
bounded by a constant
c·f(n)
Definition
multiple of f(n)

n (input size)
30
BIG-Ω NOTATION
Let T(n) & f(n) be functions defined on the positive integers.
(In this class, we’ll typically write T(n) to denote the worst case runtime of an algorithm)

What do we mean when we say “T(n) is Ω(f(n))”?

In English In Pictures In Math

n0 f(n)
T(n) = Ω(f(n)) if and only if T(n) = Ω(f(n))
Runtime (ms) T(n)
T(n) is eventually lower ⇔
bounded by a constant ∃ c , n0 > 0 s.t. ∀ n ≥ n0 ,
c·f(n)
multiple of f(n) T(n) ≥ c · f(n)
n (input size) inequality switched directions!
31
BIG-Ө NOTATION
We say “T(n) is Ө(f(n))” if and only if both
T(n) = O(f(n))
and
T(n) = Ω(f(n))
T(n) = Ө(f(n))
⇔
∃ c1 , c2 , n0 > 0 s.t. ∀ n ≥ n0 ,
c1· f(n) ≤ T(n) ≤ c2· f(n)
32
ASYMPTOTIC NOTATION CHEAT SHEET
BOUND DEFINITION (HOW TO PROVE) WHAT IT REPRESENTS

T(n) = O(f(n))
∃ c > 0, ∃ n0 > 0 s.t. ∀ n ≥ n0, T(n) ≤ c ᐧ upper bound
f(n)

T(n) = Ω(f(n))
∃ c > 0, ∃ n0 > 0 s.t. ∀ n ≥ n0, T(n) ≥ c ᐧ lower bound
f(n)

T(n) = Ө(f(n)) T(n) = O(f(n)) and T(n) = Ω(f(n)) tight bound

33
‫ﺳﻮال؟‬
‫‪34‬‬