Divide and Conquer

Outline and Reading Divide-and-Conquer
7 29 4 2 4 7 9 72 2 7 77 22 94 4 9 99 44
Divide-and-conquer paradigm (10.1.1) Review Merge-sort (10.1.3) Recurrence Equations (10.1.3)

Iterative substitution Recursion trees Guess-and-test The master method
Integer Multiplication
Divide-and-Conquer
Divide-and-Conquer
Divide-and-Conquer
Divide-and conquer is a general algorithm design paradigm:
Divide: divide the input data S in two or more disjoint subsets S1, S2, Recur: solve the subproblems recursively Conquer: combine the solutions for S1, S2, , into a solution for S
Merge-Sort Review
Merge-sort on an input sequence S with n elements consists of three steps:
Divide: partition S into two sequences S1 and S2 of about n/2 elements each Recur: recursively sort S1 and S2 Conquer: merge S1 and S2 into a unique sorted sequence Algorithm mergeSort(S, C) Input sequence S with n elements, comparator C Output sequence S sorted according to C if S.size() > 1 (S1, S2) partition(S, n/2) mergeSort(S1, C) mergeSort(S2, C) S merge(S1, S2)
The base case for the recursion are subproblems of constant size Analysis can be done using recurrence equations
Divide-and-Conquer
Divide-and-Conquer
Recurrence Equation Analysis

The conquer step of merge-sort consists of merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b. Likewise, the basis case (n < 2) will take at b most steps. Therefore, if we let T(n) denote the running time of merge-sort:
Iterative Substitution
In the iterative substitution, or plug-and-chug, technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern: T ( n ) = 2T ( n / 2) + bn
= 2(2T ( n / 22 )) + b(n / 2)) + bn = 22 T (n / 22 ) + 2bn = 23 T (n / 23 ) + 3bn = 24 T (n / 24 ) + 4bn = ... = 2i T ( n / 2i ) + ibn

Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n. So, T (n ) = bn + bn log n Thus, T(n) is O(n log n).
b T (n) = 2T ( n / 2) + bn
if n < 2 if n 2
We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation.
That is, a solution that has T(n) only on the left-hand side.
Divide-and-Conquer
Divide-and-Conquer
The Recursion Tree

Draw the recursion tree for the recurrence relation and look for a pattern:
b T (n) = 2T (n / 2) + bn if n < 2 if n 2
Guess-and-Test Method
In the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction:
b T (n) = 2T (n / 2) + bn log n if n < 2 if n 2
depth 0 1 i
Ts 1 2 2i
size n n/2 n/2i
time bn bn bn
Guess: T(n) < cn log n.
T (n ) = 2T (n / 2) + bn log n = 2( c( n / 2) log(n / 2)) + bn log n = cn (log n log 2) + bn log n = cn log n cn + bn log n

Wrong: we cannot make this last line be less than cn log n
Total time = bn + bn log n

(last level plus all previous levels)
Divide-and-Conquer 7
Divide-and-Conquer
Guess-and-Test Method, Part 2

Recall the recurrence equation:
b T (n) = 2T (n / 2) + bn log n if n < 2 if n 2
Master Method
Many divide-and-conquer recurrence equations have the form:
Guess #2: T(n) < cn log2 n.
T (n) = 2T (n / 2) + bn log n = 2(c(n / 2) log 2 (n / 2)) + bn log n = cn(log n log 2) 2 + bn log n = cn log n 2cn log n + cn + bn log n
2
c T (n ) = aT (n / b) + f ( n )
if n < d
if n d
The Master Theorem:
1. if f (n) is O(n logb a ), then T (n) is (n logb a ) 2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k +1 n) 3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.
9 Divide-and-Conquer 10
if c > b.
cn log 2 n
So, T(n) is O(n log2 n). In general, to use this method, you need to have a good guess and you need to be good at induction proofs.
Divide-and-Conquer
Master Method, Example 1

The form:
c T (n) = aT ( n / b) + f ( n ) if n < d if n d

The form: T (n ) =
c aT ( n / b) + f ( n ) if n < d if n d
The Master Theorem:

1. if f (n) is O(n logb a ), then T (n) is (n logb a ) 2. if f ( n) is (n
log b a
The Master Theorem:

1. if f (n) is O(n logb a ), then T (n) is (n logb a )
log b a k
log n), then T (n) is (n
log
k +1
n)
2. if f ( n) is (n logb a log k n), then T (n) is (n logb a log k +1 n) 3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.
3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.
Example:
T ( n) = 4T ( n / 2) + n
Example:
T ( n) = 2T (n / 2) + n log n
Solution: logba=2, so case 1 says T(n) is O(n2).

Solution: logba=1, so case 2 says T(n) is O(n log2 n).


The form: T (n ) =

The form: T (n ) =
The Master Theorem:

1. if f (n) is O(n logb a ), then T (n) is (n logb a ) 2. if f ( n) is (n logb a log k n), then T (n) is (n logb a log k +1 n) 3. if f (n) is (n
log b a +
The Master Theorem:

1. if f (n) is O(n logb a ), then T (n) is (n logb a ) 2. if f ( n) is (n logb a log k n), then T (n) is (n logb a log k +1 n) 3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.
), then T (n) is ( f (n)),
provided af (n / b) f (n) for some < 1.
Example:
T ( n) = T (n / 3) + n log n
Example:
T (n) = 8T (n / 2) + n 2
Solution: logba=0, so case 3 says T(n) is O(n log n).

Solution: logba=3, so case 1 says T(n) is O(n3).


The form: T (n ) =

The form: T (n ) =
The Master Theorem:

The Master Theorem:

Example:
T ( n) = 9T (n / 3) + n 3
O(n3).
15
Example:
T ( n ) = T ( n / 2) + 1
(binary search)
Solution: logba=2, so case 3 says T(n) is

Divide-and-Conquer
Solution: logba=0, so case 2 says T(n) is O(log n).


The form: T (n ) =
1. if f (n) is O(n
Iterative Proof of the Master Theorem

Using iterative substitution, let us see if we can find a pattern: T (n) = aT (n / b) + f (n)
= a(aT (n / b 2 )) + f (n / b)) + bn = a 2T (n / b 2 ) + af (n / b) + f (n)
c aT ( n / b) + f ( n )
if n < d if n d
log b a
The Master Theorem:

log b a log b a
), then T (n) is (n
k
)
log b a
2. if f ( n) is (n
log n), then T (n) is (n
log
k +1
n)
3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.
= a 3T (n / b3 ) + a 2 f (n / b 2 ) + af (n / b) + f (n) = ... = a logb nT (1) +

(log b n ) 1 i
a
i =0 i=0
f (n / bi ) f (n / b i )
Example:
T (n) = 2T (n / 2) + log n
(heap construction)
= n logb aT (1) +
(log b n ) 1 i
Solution: logba=1, so case 1 says T(n) is O(n).

We then distinguish the three cases as

The first term is dominant Each part of the summation is equally dominant The summation is a geometric series
Divide-and-Conquer
18
Integer Multiplication
Algorithm: Multiply two n-bit integers I and J.
Divide step: Split I and J into high-order and low-order bits
An Improved Integer Multiplication Algorithm

Algorithm: Multiply two n-bit integers I and J.
Divide step: Split I and J into high-order and low-order bits
I = I h 2n / 2 + I l J = J h 2n / 2 + J l
We can then define I*J by multiplying the parts and adding:
I = I h 2n / 2 + I l
Observe that there is a different way to multiply parts:
J = J h 2n / 2 + J l
I * J = ( I h 2n / 2 + I l ) * ( J h 2n / 2 + J l ) = I h J h 2n + I h J l 2n / 2 + I l J h 2n / 2 + I l J l
So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). But that is no better than the algorithm we learned in grade school.
I * J = I h J h 2 n + [( I h I l )( J l J h ) + I h J h + I l J l ]2 n / 2 + I l J l = I h J h 2 n + [( I h J l I l J l I h J h + I l J h ) + I h J h + I l J l ]2 n / 2 + I l J l = I h J h 2 n + ( I h J l + I l J h )2 n / 2 + I l J l
So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by the Master Theorem. Thus, T(n) is O(n1.585).

Divide and Conquer

Uploaded by

Copyright:

Available Formats

Divide and Conquer

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Divide and Conquer

Uploaded by

Copyright:

Available Formats

Outline and Reading Divide-and-Conquer

Divide-and-conquer paradigm (10.1.1) Review Merge-sort (10.1.3) Recurrence Equations (10.1.3)

Recurrence Equation Analysis

= 2(2T ( n / 22 )) + b(n / 2)) + bn = 22 T (n / 22 ) + 2bn = 23 T (n / 23 ) + 3bn = 24 T (n / 24 ) + 4bn = ... = 2i T ( n / 2i ) + ibn

The Recursion Tree

size n n/2 n/2i

Guess: T(n) < cn log n.

T (n ) = 2T (n / 2) + bn log n = 2( c( n / 2) log(n / 2)) + bn log n = cn (log n log 2) + bn log n = cn log n cn + bn log n

Total time = bn + bn log n

Guess-and-Test Method, Part 2

Guess #2: T(n) < cn log2 n.

The Master Theorem:

Master Method, Example 1

Master Method, Example 2

The Master Theorem:

The Master Theorem:

log n), then T (n) is (n

Solution: logba=2, so case 1 says T(n) is O(n2).

Solution: logba=1, so case 2 says T(n) is O(n log2 n).

Master Method, Example 3

Master Method, Example 4

The Master Theorem:

The Master Theorem:

), then T (n) is ( f (n)),

provided af (n / b) f (n) for some < 1.

Solution: logba=0, so case 3 says T(n) is O(n log n).

Solution: logba=3, so case 1 says T(n) is O(n3).

Master Method, Example 5

Master Method, Example 6

The Master Theorem:

The Master Theorem:

Solution: logba=2, so case 3 says T(n) is

Solution: logba=0, so case 2 says T(n) is O(log n).

Master Method, Example 7

Iterative Proof of the Master Theorem

The Master Theorem:

log n), then T (n) is (n

= a 3T (n / b3 ) + a 2 f (n / b 2 ) + af (n / b) + f (n) = ... = a logb nT (1) +

Solution: logba=1, so case 1 says T(n) is O(n).

We then distinguish the three cases as

An Improved Integer Multiplication Algorithm

Observe that there is a different way to multiply parts:

You might also like