2 Notes

Lecture Outline (CSCI3160-18F, 2nd week)
CAI Leizhen
CSE-CUHK-HK-CHN
September 13, 2018
Keywords: Time complexity, divide-and-conquer, master theorem, integer multiplication,

matrix multiplication, skyline and closet pairs.
1 Complexity
1. Length of input (input size): Usually some natural measures, such as O(lg n) for integer
n, the number of items in a set, and O(m + n) for a graph with n vertices and m edges.
2. Basic step: We assume that each elementary operation (such as addition, subtraction,
multiplication, division, Boolean operations, comparisons, assignment and build-in func-
tions) takes a constant amount of time.
For this assumption, operands must be of reasonable size.
3. Time complexity T (n) of algorithm A: Among all legal inputs i of length n, the maximum
amount of time (in big-O notation in terms of the number of basic steps as a function of
n) that A takes to produce the required output f Π (i).
4. The algorithm Fibonacci(n) uses Θ(n) elementary operations, but the operands i and j
have length Θ(k), therefore each subtraction and addition takes Θ(k), instead of O(1)
time, and the algorithm takes Θ(n2 ) time.
2 Divide-and-conquer
1. Three major steps —
Divide: Decompose an instance I of a problem Π into small subinstances of Π.

Conquer: Solve each subinstance recursively and independently.
Combine: Use solutions of subinstances to solve the original instance I.
2. Real-world examples: Sport tournaments, and Miss World pageants.
1
CSCI3160-18F Design and Analysis of Algorithms 2
3. Mergesort: O(n log n).

Evenly divide a list L of n numbers into two lists L 1 and L2 . Recursively mergesort L1
and L2 , and then merge the two sorted lists into one. Merging two sorted lists takes
O(n) time and thus T (n) = 2T (n/2) + O(n), which yields T (n) = O(n log n).
4. Typically, the running time T (n) of a divide-and-conquer algorithm obeys
T (n) ≤ aT (n/b) + f (n),
where a is the number of subinstances, n/b the size of subinstances and f (n) the time
for Divide and Combine.
5. Master Theorem for T (n) ≤ aT (n/b) + O(n d ) with constants a ≥ 1, b > 1 and d ≥ 0:

d
 O(n )
 if d > log b a
T (n) = O(nd log n) if d = log b a
 O(nlogb a )

if d < log b a.
Note that the master theorem covers most common cases for T (n) ≤ aT (n/b) + f (n).
6. Efficiency of divide-and-conquer method.

Reduce log b a:
3T (n/2) a/b = 1.5

5T (n/3) a/b = 1.67
but log3 5 ≈ 1.47 < log 2 3 ≈ 1.59.

Balance partition:
T (n) = 2T (n/2) + O(n) =⇒ O(n log n)

T (n) = T (n − 5) + T (5) + O(n) =⇒ O(n2 )
T (n) = T (4n/5) + T (n/5) + O(n) =⇒ O(n log n)
7. Multiplication (Karatsuba and Ofman (1962)): O(n 1.59 ) to compute the product of two
n-digit integers.
Let x and y be two n-digit integers (for convenience, we may assume that n is a power
of 2). We split x (y respectively) into left and right halves x 0 and x1 (y0 and y1 ). Then
xy = (x0 10n/2 + x1 )(y0 10n/2 + y1 ) = x0 y0 10n + (x0 y1 + x1 y0 )10n/2 + x1 y1 .
Since x0 y1 + x1 y0 = (x0 + x1 )(y0 + y1 ) − x0 y0 − x1 y1 , three n/2-digit multiplications are

sufficient to compute xy, and these three multiplications are computed recursively using
the same method.
function product(x,y): return the product of n-digit integers x and y.
if n = 1 then return xy.

x0 , x1 = leftmost n/2, rightmost n/2 digits of x;
y0 , y1 = leftmost n/2, rightmost n/2 digits of y;
p1 = product(x0 , y0 );
p2 = product(x1 , y1 );
p3 = product(x0 + x1 , y0 + y1 );
return p1 10n + (p3 − p1 − p2 )10n/2 + p2 .
Time: T (n) = 3T (n/2) + O(n) which yields T (n) = O(n 1.59 ).

The following table illustrates the difference between n 1.59 and n2 , which gets bigger as
n grows.
n n1.59 n2 /n1.59
10 39 2.5
102 1514 6.6
103 58884 17.0
104 2290868 43.7
8. Strassen’s matrix multiplication (1969): O(n 2.81 ).

To compute the product of two n!× n matrices A and!B, we decompose A and B into
A11 A12 B11 B12
n/2 × n/2 matrices and and recursively compute their
A21 A22 B21 B22
products.
We need only 7 multiplications to compute the product of two 2 × 2 matrices.
! ! !
a b e f p5 + p 4 − p 2 + p 6 p1 + p 2
= ,
c d g h p3 + p 4 p1 + p 5 − p 3 − p 7
where p1 = a(f − h), p2 = (a + b)h, p3 = (c + d)e, p4 = d(g − e), p5 = (a + d)(e + h), p6 =
(b − d)(g + h) and p7 = (a − c)(e + f ).
This results in a divide-and-conquer algorithm satisfying T (n) = 7T (n/2)+O(n 2 ), which
yields T (n) = O(n2.81 ). The current fastest algorithm takes O(n 2.376 ) time (Coppersmith
and Winograd, 1986).
9. Fibonacci number revisited: O(n 1.59 ).
! !n !
fn 0 1 0
=
fn+1 1 1 1
n n
We can use “repeated squaring” to compute M n , i.e., M n = M 2 M 2 , by O(log n) ma-
trix multiplications. Since integers inside matrices have length Θ(n) and each integer
multiplication takes O(n1.59 ) time, the algorithm takes O(n1.59 log n) time. In fact, we
can tighten the bound of running time to O(n 1.59 ) by the Master Theorem.
10. Find the skyline of a set of n rectangular buildings sharing a common horizon.
Building Bi is represented by a triple (li , ri , hi ), where li and ri , resp., are left and right
x-coordinates of the building, and h i the height of the building.
The skyline is represented by a sequence of pairs (x i , hi ), arranged with increasing x-
coordinators xi , such that the height in [xi , xi+1 ] is hi .
Divide: Evenly divide all buildings into two groups B and B 0 .
Conquer: Recursively find skylines S and S 0 of B and B 0 respectively.
Combine: Merge S and S 0 into a single skyline.
The Combine step is similar to merge sort and takes O(n) time. Total time T (n) =
2T (n/2) + O(n), which yields T (n) = O(n log n).
11. Find a closest pair of points among n points P in the plane: O(n log n).
Divide: Use a vertical line L to divide P “evenly” into two sets P l and Pr .
Conquer: Recursively find closest pairs p l and pr , respectively, in Pl and Pr and let
δ = min{dist(pl ), dist(pr )}.
Combine: Find a closet pair pL that consists of one point in the δ-wide strip S l just to
the left of L and one point in the δ-wide strip S r just to the right of L. One of pl , pr and
pL is a solution. Why does this step take O(n) time?
Consider an arbitrary point p in Sl . With the fact that pairwise distance for points inside
Sr is at least δ, how many points are there inside S r that are not below p and at distance
less than δ from p? The answer is at most 4. Therefore we need only consider 4 points
in Sr for each point in Sl , and similar for each point in Sr , which gives us O(n) time for
the Combine step. The recurrence T (n) = 2T (n/2) + O(n) yields T (n) = O(n log n).

2 Notes

Uploaded by

Copyright:

Available Formats

2 Notes

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2 Notes

Uploaded by

Copyright:

Available Formats

Lecture Outline (CSCI3160-18F, 2nd week)

September 13, 2018

Keywords: Time complexity, divide-and-conquer, master theorem, integer multiplication,

1. Three major steps —

Divide: Decompose an instance I of a problem Π into small subinstances of Π.

2. Real-world examples: Sport tournaments, and Miss World pageants.

3. Mergesort: O(n log n).

4. Typically, the running time T (n) of a divide-and-conquer algorithm obeys

T (n) ≤ aT (n/b) + f (n),

6. Efficiency of divide-and-conquer method.

3T (n/2) a/b = 1.5

but log3 5 ≈ 1.47 < log 2 3 ≈ 1.59.

T (n) = 2T (n/2) + O(n) =⇒ O(n log n)

xy = (x0 10n/2 + x1 )(y0 10n/2 + y1 ) = x0 y0 10n + (x0 y1 + x1 y0 )10n/2 + x1 y1 .

Since x0 y1 + x1 y0 = (x0 + x1 )(y0 + y1 ) − x0 y0 − x1 y1 , three n/2-digit multiplications are

function product(x,y): return the product of n-digit integers x and y.

if n = 1 then return xy.

Time: T (n) = 3T (n/2) + O(n) which yields T (n) = O(n 1.59 ).

8. Strassen’s matrix multiplication (1969): O(n 2.81 ).

You might also like