Divide-and-Conquer

CSE 780

Reading Assignment: Sections 2.3, 4.1, 4.2, 4.3, 28.2, 33.4.

1 Introduction
Given an instance x of a problem, the divide-and-conquer method works as follows.

function DAQ(x)
if x is sufficiently small or simple then
solve it directly
else
divide x into smaller subinstances x1 , . . . , xk ;
for i ← 1 to k do yi ← DAQ(xi );
combine the yi ’s to obtain a solution y for x;
return(y)
endif

It is often that x1 , . . . , xk are of the same size, say bn/bc. In that case, the time com-
plexity of DAQ can be expressed as a recurrence:

 c, if n ≤ n0
T (n) =
 kT (bn/bc) + f (n), if n > n0

1
2 Merge Sort
Sort an array:

procedure mergesort(A[1..n], i, j)
// Sort A[i..j]//
if i ≥ j then return
m ← (i + j) div 2
mergesort(A, i, m)
mergesort(A, m + 1, j)
merge(A, i, m, j)
end

Sort a linked list

function mergesort(i, j)
// Sort A[i..j]. Initially, Link[k] = 0, 1 ≤ k ≤ n.//
global A[1..n], Link[1..n]
if i ≥ j then return(i)
m ← (i + j) div 2
ptr1 ← mergesort(i, m)
ptr2 ← mergesort(m + 1, j)
ptr ← merge(ptr1, ptr2)
return(ptr)
end

2
3 Solving Recurrences
A function T (n) satisfies the following recurrence:

 c, if n ≤ 1
T (n) =
 3T (bn/4c) + n, if n > 1

where c ia a positive constant. Obtain a function g(n) such that T (n) = O(g(n)).

3.1 Iteration Method and Recurrence Tree

Assume n to be a power of 4, say, n = 4m .

T (n) = n + 3T (n/4)
h i
= n + 3 n/4 + 3T (n/16)
h i
= n + 3(n/4) + 9 (n/16) + 3T (n/64)

= n + n(3/4) + n(3/4)2 + 33 T (n/43 )

h i
= n 1 + 3/4 + (3/4)2 + (3/4)3 + . . . + (3/4)m−1 + 3m T (n/4m )
h i
= n 1 + 3/4 + (3/4)2 + (3/4)3 + . . . + (3/4)m−1 + c3m

= n(1 − (3/4)m )/(1 − 3/4) + c3m

≤ 4n + c4m

= 4n + cn

= O(n)

So, T (n) = O(n|n a power of 4). The function f (n) = n is smooth and T (n) can be
verified to be asymptotically nondecreasing. Thus, T (n) = O(n).

3
3.2 Guess and Prove
Solve 
 c, if n ≤ 1
T (n) =
 3T (bn/4c) + n, if n > 1

• First, we guess that T (n) = O(n).

• Then we prove the guess is right by showing T (n) ≤ bn for some constant b.

• It suffices to consider n = 4m , m = 0, 1, 2, . . .. Thus, we prove T (4m ) ≤ b4m by

induction on m.
Induction Base: When m = 0, T (1) = c ≤ b if b is chosen to be large enough.
Induction Hypothesis: Assume T (4m−1 ) ≤ b4m−1 .
Induction Step: Show T (4m ) ≤ b4m as follows.

T (4m ) = 4m + 3T (4m−1 )

≤ 4m + 3b4m−1 (by I.H.)

≤ (1 + 3b/4)4m

≤ b4m , if 1 ≤ b/4

• Letting b ≥ 4 + c, we have T (n) ≤ bn for all n. This establishes T (n) = O(n).

4
3.3 Master’s Theorem
Definition 1 f (n) is polynomially smaller than g(n), denoted as f (n)  g(n), if f (n) =
O(g(n)n− ), or f (n)n = O(g(n)), for some  > 0.
Theorem 1 If T (n) = aT (n/b) + f (n), then T (n) is bounded asymptotically as follows.
1. If f (n) = O(nlogb a n− ), then T (n) = Θ(nlogb a ).

2. If f (n) = Ω(nlogb a n ), then T (n) = Θ(f (n)).

3. If f (n) = Θ(nlogb a ), then T (n) = Θ(nlogb a log n).

4. If f (n) = Θ(nlogb a logk n), then T (n) = Θ(nlogb a logk+1 n).

In case 2, it is required that af (n/b) ≤ cf (n) for some constant c < 1, which is satisfied
by most polynomialy bounded functions that we shall encounter.
In the theorem, n/b should be interpreted as bn/bc or dn/be.

Master’s theorem means:

1. If f (n)  nlogb a , then T (n) = Θ(nlogb a ).

2. If f (n)  nlogb a , then T (n) = Θ(f (n)).

3. If f (n) ≈ nlogb a , then T (n) = Θ(f (n) log n) = Θ(nlogb a log n).

4. If f (n) ≈ nlogb a logk n, then T (n) = Θ(f (n) log n) = Θ(nlogb a logk+1 n).
Applying the Master theorem to

 c, if n ≤ 1
T (n) = 
3T (bn/4c) + n, if n > 1
we immediately obtain T (n) = Θ(n), as nlogb a = nlog4 3 < n.

3.4 More Examples

• T (n) = 9T (n/3) + n.

• T (n) = T (2n/3) + 1

• T (n) = 3T (n/4) + n log n.

• T (n) = 7T (n/2) + Θ(n2 ).

• T (n) = 2T (n/2) + n log n.

• T (n) = T (n/3) + T (2n/3) + n.

5
4 Strassen’s Algorithm for Matrix Multiplication
• Problem: Compute C = AB, where A and B are n × n matrices.

• The straightforward method requires Θ(n3 ) time, using the formula cij = aik bkj .
P
1≤k≤n

• Toward the end of 1960s, Strassen showed how to multiply matrices in O(nlog 7 ) =
O(n2.81 ) time. (For n = 100, n2.81 ≈ 416869.)

• The time complexity was reduced to O(n2.521813 ) in 1979, to O(n2.521801 ) in 1980, and
to O(n2.376 ) in 1986.

• In the following discussion, assume n to be a power of 2.

6
• Write ! ! !
A11 A12 B11 B12 C11 C12
A= , B= , C=
A21 A22 B21 B22 C21 C22
where each Aij , Bij , Cij is a n/2 × n/2 matrix. Then
! ! !
C11 C12 A11 A12 B11 B12
= .
C21 C22 A21 A22 B21 B22

• Multiplication of two 2 × 2 matrices can be done using only 7 scalar multiplications

as follows. Let

M1 = (A21 + A22 − A11 )(B22 − B12 + B11 )

M2 = A11 B11
M3 = A12 B21
M4 = (A11 − A21 )(B22 − B12 )
M5 = (A21 + A22 )(B12 − B11 )
M6 = (A12 − A21 + A11 − A22 )B22
M7 = A22 (B11 + B22 − B12 − B21 )

Then !
M2 + M3 M1 + M2 + M5 + M6
C= .
M1 + M2 + M4 − M7 M1 + M2 + M4 + M5

• Time complexity:
T (n) = 7T (n/2) + Θ(n2 ).

Solving the recurrence yields T (n) = Θ(nlog7 ) = O(n2.81 ).

7
5 The Closest Pair Problem
5.1 Problem Statement
Given a set of n points A = {(xi , yi ) : 1 ≤ i ≤ n} in the plane, find two points in A with
smallest distance.

5.2 Basic Idea

Straightforward method: O(n2 )

Divide and conquer: O(n log n)

Basic idea:

1. Break up A into A = B ∪ C.

2. Find a closest pair (p1 , q1 ) in B.

3. Find a closest pair (p2 , q2 ) in C.

4. Let δ = min{dist(p1 , q1 ), dist(p2 , q2 )}.

5. Find a closest pair (p3 , q3 ) between B and C with distance less than δ, if such a pair
exists.

6. Return the pair (p, q) such that dist(p, q) = min{dist(pk , qk ) : 1 ≤ k ≤ 3} is smallest.

8
5.3 Main Program
Assume that the coordinates of the n points are stored in X[1..n] and Y [1..n], with (X[i], Y [i])
the ith point. For simplicity, let A[i] = (X[i], Y [i]).

Assume that the points have been sorted such that X[1] ≤ X[2] ≤ · · · ≤ X[n]

First Attempt

procedure Closest-Pair(A[i..j], (p, q))

{*This procedure returns a closest pair (p, q) in A[i..j].*}
{*global X[0..n + 1], Y [0..n + 1]*}
case
j − i = 0: (p, q) ← (0, n + 1);
j − i = 1: (p, q) ← (i, j);
j − i > 1: m ← (i + j) div 2
Closest-Pair(A[i..m], (p1 , q1 ))
Closest-Pair(A[m + 1..j], (p2 , q2 ))
ptr1 ← Mergesort(A[i..m])
ptr2 ← Mergesort(A[m + 1..j])
ptr ← Merge(ptr1, ptr2)
δ ← min{dist(p1 , q1 ), dist(p2 , q2 )}
Closest-Pair-Between-Two-Sets(A[i..j], ptr, δ, (p3 , q3 ))
(p, q) ← the pair (pk , qk ), 1 ≤ k ≤ 3 with shortest dist.
endcase

9
Refined Version

procedure Closest-Pair(A[i..j], (p, q), ptr)

{*This procedure performs two functions.
First, it finds a closest pair (p, q) in A[i..j].
Second, it sorts A[i..j] on y-coordinate, and yields a linked list ptr.*}
{*global X[0..n + 1], Y [0..n + 1], Link[1..n]
Initially, X[0] ← Y [0] ← −∞, X[n + 1] ← Y [n + 1] ← ∞,
and Link[k] ← 0, 1 ≤ k ≤ n.*}
case
j − i = 0: (p, q) ← (0, n + 1); ptr ← i
j − i = 1: (p, q) ← (i, j)
if Y [i] ≤ Y [j] then {ptr ← i; Link[i] ← j}
else {ptr ← j; Link[j] ← i}
j − i > 1: m ← (i + j) div 2
Closest-Pair(A[i..m], (p1 , q1 ), ptr1)
Closest-Pair(A[m + 1..j], (p2 , q2 ), ptr2)
Merge(ptr1, ptr2, ptr)
δ ← min{dist(p1 , q1 ), dist(p2 , q2 )}
Closest-Pair-Between-Two-Sets(A[i..j], ptr, δ, (p3 , q3 ))
(p, q) ← the pair (pk , qk ), 1 ≤ k ≤ 3 with shortest distance
endcase

10
5.4 Subroutine
To find a closest pair between A[i..m] and A[m+1..j] with distance < δ, where m = (i+j)
div 2.

L0 : vertical line passing through the point A[m]

L1 and L2 : as shown.

We observe that:

1. We need to consider only points between L1 and L2 .

2. For a point r in A[i..m], we need to consider only the points in A[m + 1..j] that are
inside the square of δ × δ. There are at most three such points.

3. Similarly, for a point r in A[m + 1..j], we need to consider at most three points in
A[i..m].

11
procedure Closest-Pair-Between-Two-Sets(A[i..j], ptr, δ, (p3 , q3 ))
// Find the closest pair between A[i..m] and A[m + 1..j] with dist < δ.
If there exists no such a pair, then return the dummy pair (0, n + 1)//
global X[0..n + 1], Y [0..n + 1], Link[1..n]
(p3 , q3 ) ← (0, n + 1)
b1 ← b2 ← b3 ← 0
c1 ← c2 ← c3 ← n + 1
m ← (i + j) div 2
k ← ptr
while k 6= 0 do
if |X[k] − X[m]| < δ then
case
k ≤ m: compute d ← min{dist(k, ci ) : 1 ≤ i ≤ 3};
if d < δ then update δ and (p3 , q3 );
b3 ← b2 ; b2 ← b1 ; b1 ← k
k > m: compute d ← min{dist(k, bi ) : 1 ≤ i ≤ 3};
if d < δ then update δ and (p3 , q3 );
c3 ← c2 ; c2 ← c1 ; c1 ← k
endcase
endif
k ← Link[k]
endwhile

12
6 Convex Hull
6.1 Problem Statement
• Given a set A of n points in the plane, we want to find the convex hull of A.

• The convex hull of A is the smallest convex polygon that contains all the points in A.

6.2 Sketch of Algorithm

• Let A = {p1 , p2 , . . . , pn }. Denote the conex hull of A by CH(A).

• An observation: the segment pi pj is an edge of CH(A) if all points of A are on the

same side of pi pj .

• Straightforward method: Ω(n2 )

• Divide and conquer: O(n log n)

• Basic ideas:

1. Let A be sorted by the x-coordinates of the points.

2. If |A| ≤ 3, solve the problem directly. Otherwise, apply divide-and-conquer as
follows.
3. Break up A into A = B ∪ C.
4. Find the convex hull of B.
5. Find the convex hull of C.
6. Combine the two convex hulls.

13
6.3 Combine CH(B) and CH(C) to get CH(A)
1. We need to find the “upper bridge” and the “lower bridge” that connect the two
convex hulls.

2. Find the upper bridge as follows:

(a) v := the rightmost point in CH(B);

w:= the leftmost point in CH(C).
(b) Loop
if counterclockwise neighbor(v) lies above the line vw then
v := counterclockwise neighbor(v)
else if clockwise neighbor(w) lies above the line vw then
w := clockwise neighbor(w)
else
exit from the loop
End of loop
(c) vw is the upper bridge.

3. Find the lower bridge similarly.

14
7 Supplementary Problems
1. Why do we need to sort the given points by x at the beginning of the closest-pair
algorithm?

2. Consider the closest-pair algorithm. Suppose we do not sort A[i..j] by y-coordinate

in Closest-Pair(A[i..j], (p, q), ptr), but instead we sort the n points A[1..n] by y into a
linked list in the beginning of the algorithm, immediately after they are sorted by x.

(a) Does the modified algorithm work correctly?

(b) What is its time complexity?

3. Let X[1..n] and Y [1..n] be two sets of integers, each sorted in nondecreasing order.
Write a divide-and-conquer algorithm that finds the nth smallest of the 2n combined
elements. Your algorithm must run in O(log n) time. You may assume that all the 2n
elements are distinct.

4. The time complexity of an algorithm satisfies the following recurrence equation:

( √
2T (b nc) + 1 if n > 2
T (n) =
1 otherwise

Solve the equation and express T (n) in terms of the Θ notation.

15