Algorithms and Problem Solving (15B17CI411)

Design Technique: Divide and Conquer

Jaypee Institute of Information Technology (JIIT)

A-10, Sector 62, Noida
Divide-and-Conquer
•
Divide-and conquer is a general
algorithm design paradigm:
• Divide: divide the input data S in two
or more disjoint subsets S1, S2, …
• Recur: solve the subproblems
recursively
• Conquer: combine the solutions for
S1, S2, …, into a solution for S

•
The base case for the recursion are
subproblems of constant size
•
Analysis can be done using
recurrence equations
Divide-and-Conquer 2
Approach
DAC(P)
{
if (small (P))
// P is small and solution is obvious
return solution (P);

k= divide(P); // divide the problem into k sub problems

return combine ( DAC(P1), DAC(P2), DAC(P3).. DAC(PK))
 Divide-and-Conquer Examples
•
Sorting: mergesort and quicksort

•
Binary search

•
Binary tree traversals

•
Matrix multiplication: Strassen’s algorithm
General Divide-and-Conquer Recurrence

Solved using master’s theorem

Divide and Conquer Algorithms which you
already know:
1. Merge Sort
2. Quick Sort
3. Binary Search

•
Homework: Revise these algorithms and try to understand why these
are divide and conquer algorithms.
8

Integer Arithmetic 1 1 1 1 1 1 0 1
1 1 0 1 0 1 0 1
Add
+ 0 1 1 1 1 1 0 1
•
Add. Given two n-digit integers a and b, 1 0 1 0 1 0 0 1 0

•
compute a + b.
1 1 0 1 0 1 0 1
• O(n) bit operations. * 0 1 1 1 1 1 0 1
1 1 0 1 0 1 0 1 0
Multiply
0 0 0 0 0 0 0 0 0
•
Multiply. Given two n-digit integers a and b, 1 1 0 1 0 1 0 1 0

compute a × b. 1 1 0 1 0 1 0 1 0
1 1 0 1 0 1 0 1 0
• Brute force solution: (n2) bit operations.
Can we optimize this in any way? 1 1 0 1 0 1 0 1 0
1 1 0 1 0 1 0 1 0
0 0 0 0 0 0 0 0 0
0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 0
Integer Multiplication
•
So we haven’t really improved that much,
• Since we went from a O(n2) solution to a O(n2) solution
•
Can we optimize this in any way?
Karatsuba multiplication
Exercise

Compute 2101 * 1130 by applying the divide-and-conquer algorithm outlined.

Solution:
 Matrix Multiplication Algorithm
•
A fundamental numerical operation is the multiplication of 2 matrices.
• The standard method of matrix multiplication of two n x n matrices takes T(n) =
O(n3).

X =

The following algorithm multiples n x n matrices A and B:

// Initialize C.
for i = 1 to n
for j = 1 to n
for k = 1 to n
C [i, j] += A[i, k] * B[k, j];
 Divide and Conquer approach

Idea - Block Matrix Multiplication The idea behind Strassen's algorithm is in the formulation
of matrix multiplication as a recursive problem.

A and B as block matrices,

where block matrices Aij are of size n/2 x n/2 (same with respect to block entries of B and C).
The matrices Aij and Bkl are smaller matrices, hence we have broken down our initial problem of multiplying two n x n
matrices into a problem requiring 8 matrix multiplies between matrices of size n/2 x n/2 , as well as a total of 4 matrix
additions.
T(n) = 8T(n/2) + O(n2)

Strassen's algorithm
Reduce the number of sub-calls to matrix-multiplies to 7, using just a bit of algebra.
We write down Cij 's in terms of block matrices Mk's. Each Mk may be calculated simply from products and
sums of sub-blocks of A and B. That is, we let

Source :https://stanford.edu/~rezab/classes/cme323/S16/notes/Lecture03/cme323_lec3.pdf
It can be verified that

Hence, the work is given by a recurrence of the form

T(N) = 7T(N/2) + O(N2)

From Master's Theorem, time complexity of above method is O(NLog7) which is

approximately

O(N2.8074)
Strassen’s Algorithm

Mult Add Recurrence Relation Runtime

Regular 8 4 T(n) = 8T(n/2) + O(n2) O(n3)

Strassen 7 18 T(n) = 7T(n/2) + O(n2) O(n log27) = O(n2.81)
Exercise
Solution
Solution
 Maximum Single-Sell Profit

> Problem: Given a list of prices on each day, find the days on which to buy & sell that
would have achieved max profit – maximize (sell price – buy price)

Approach#1: Brute-Force
 Stock Pricing Problem -Maximum Single-Sell Profit

> Problem: Given a list of prices on each day, find the days on which to buy & sell that
would have achieved max profit – maximize (sell price – buy price)

> Apply divide & conquer

1. Divide prices into first half and second half
2. Recursively solve each sub-problem
> gives best with both buy & sell on same half
3. Combine by considering best buy on one half and sell on
other half
> only need to consider buying in first half and selling in second half
because we cannot buy after we sell

Apply divide & conquer

1. Combine by considering best (buy in first half, sell in second
half)
> Q: what is the best price to buy at in first half?
> A: minimum price
> Q: what is the best price to sell at in second half?
> A: maximum price
Apply divide & conquer
1. Divide prices into first half and second half
2. Recursively solve each sub-problem
> gives best with both buy & sell on same half
3. Combine by considering best buy on one half and sell on
other half
> find (min(first half), max(second half)) in O(n) time

Running time is O(n log n) by master

theorem
maximum-subarray: using divide and conquer approach
Given array A, find the nonempty, contiguous subarray of A whose values have the largest sum.

Task is to find out subarray which has the largest sum. So we need to find out the 2 date indexes (left index and right
index) between which the profit is maximum.
Question: If the array is split into two subarrays, where could the maximum-subarray be located?
Answer: There are three possibilities:
• In the left subarray (Buy date index and sell date index both are in left half of the array)
• In the right subarray (Buy date index and sell date index both are in right half of the array).
• Across the two subarrays (Buy date index in the and left half of the array and sell date index in right half of the
array).
 Example

Instead of looking at daily prices. Consider daily change in the price, where change on day i is the difference
between the prices after day (i-1) and after day i.

Given array A, find the nonempty, contiguous subarray of A whose values have the largest sum.
Buy the stock just before day 8(i.e. after day 7) and sell it after day 11, earning a profit of $43 per share.
Closest Pair Problem
Finding closest pair of points

Given a set of points {p1,..pn} find the pair of points{pi,pj} that are
closest together.

•
Brute force gives an algorithm: just check every pair of
points.
•
Can we do it faster?
•
we can find the closest pair in O(n log n) time.
Divide-and-conquer for closest pair

find vertical line that splits P in half

• let P1, P2 = set of points to the left/right of line

Source : https://www.bowdoin.edu/~ltoma/teaching/cs3250-CompGeom/spring17/Lectures/cg-closestPair.pdf
find vertical line that splits P in half
• let P1, P2 = set of points to the left/right of line
• recursively find closest pair in P1
• recursively find closest pair in P2

How can you find closest pair in P?

 • find vertical line that splits P in half
• let P1, P2 = set of points to the left/right of line
• d1 = find closest pair in P1
• d2 = find closest pair in P2
• for each p in P1, for each q in P2
• compute distance d(p,q)
• mindist = min{d1, d2, d(p,q)}

The closest pair is either:

• both points are in P1, and then it is found by the recursive call on P1
• both points are in P2, and then it is found by the recursive call on P2
• one point is in P1 and one in P2, and then it is found in the merge phase, because the merge phase consider
all such pairs

Running time?
• T(n) = 2T(n/2) + O(n2) => solves to
Refining the merge
Do we need to examine all pairs (p,q), with p in P 1, q in P2?

Can (p,q) be the closest pair?

Notation: d = min {d1, d2}
In order for dist(p,q) to be smaller than d, it must be that both the horizontal and the vertical distance between p and q
must be smaller than d.
In order to be candidates for closest pair, points p, q must lie in the d-by-d strip centered at the median.
Consider a point p in the stripe. How many points below it, at most, could be candidates for the closest
pair (p,q)?
Claim:
A point p needs to check at most 5 points following p in y-order
Closest-Pair(p1, …, pn) {
Compute separation line L such that half the points are on one side and half on the O(n log n)
other side.
2T(n / 2)
d1 =Closest-Pair(left half)
d2= Closest-Pair(right half)
d = min(d1, d2)

Delete all points further than d from separation line L O(n)

Sort remaining points by y-coordinate. O(n log n)

Scan points in y-order and compare distance between O(n)

each point and next 5 neighbors.
If any of these
distances is less than d, update d .
return d.
Closest Pair of Points: Analysis
Running time.

Can we achieve O(n log n)?

Yes. Don't sort points in strip from scratch each time.

Lecture slides adapted from:
·
Chapter 4 of A. Levitin “Introduction to design and analysis of algorithms “, Second edition.
·
https://www.geeksforgeeks.org/closest-pair-of-points-onlogn-implementation/
·
https://www.geeksforgeeks.org/strassens-matrix-multiplication/
·
https://www.bowdoin.edu/~ltoma/teaching/cs3250-CompGeom/spring17/Lectures/cg-
closestPair.pdf