Unit II - DIVIDE-AND-CONQUER

General Method
Divide and conquer is a design strategy which is well known to breaking down efficiency
barriers. When the method applies, it often leads to a large improvement in time complexity.
Divide-and-conquer algorithms work according to the following general plan.

1. Divide: Divide the problem into a number of smaller sub-problems ideally of about the same size.

2. Conquer: The smaller sub-problems are solved, typically recursively. If the sub-problem sizes are
small enough, just solve the sub-problems in a straight forward manner.

3. Combine: If necessary, the solution obtained the smaller problems are connected to get the solution
to the original problem. The following figure shows

Given a function to compute on n inputs the divide-and-conquer strategy suggests splitting the
inputs into k distinct subsets, 1< k < n, yielding k sub problems. These sub problems must be
solved, and then a method must be found to combine sub solutions into a solution of the whole. If
the sub problems are still relatively large, then the divide-and-conquer strategy can possibly be
reapplied. Often the sub problems resulting from a divide-and conquer design are of the same type
as the original problem. For those cases the reapplication of the divide-and-conquer principle is
naturally expressed by a recursive algorithm.

Example : Detecting Counterfeit Coin

A bag with n 16 coins and one of these coins may be counterfeit. Counterfeit coins are lighter than
genuine ones. The task is to determine whether the bag contains a counterfeit coin. It has a
machine that compares the weights of two sets of coins and tells which set is lighter or whether
both sets have the same weight.
We can compare the weights of coins 1 and 2. If coin 1 is lighter than coin 2, then coin 1 is
counterfeit and we are done with our task. If coin 2 is lighter than coin 1, then coin 2 is
counterfeit. If both coins have the same weight, we compare coins 3 and 4. Proceeding in the way,
we can determine whether the bag contains a counterfeit coin by making at most eight weight
comparisons. This process also identifies the counterfeit coin.
Another approach is to use the divide-and-conquer methodology. Suppose that our 16-coin
instance is considered a large instance. In step 1, we divide the original instance into two or more
smaller instances. Let us divide the 16-coin instance into two 8-coin instances by arbitrarily
selecting 8 coins for the first instance (say A) and the remaining 8 coins eight coins for the second
instance B. In step 2 we need to determine whether A or B has a counterfeit coin. For this step we
use our machine to compare the weights of the coin sets A and B. If both sets have the same
weight, a counterfeit coin is not present in the 16-coin set. If A and B have different weights, a
counterfeit coin is present and it is in the lighter set.

To be more precise, suppose we consider the divide-and-conquer strategy when it splits the input
into two sub problems of the same kind as the original problem. This splitting is typical of many
of the problems we examine here. We can write a control abstraction that mirrors the way an
algorithm based on divide-and-conquer will look .By a control abstraction we mean a procedure
whose flow of control is clear but whose primary operations are specified by other procedures
whose precise meanings are left undefined. DAndC (algorithm below) is initially invoked as
DAndC (P),where P is the problem to be solved.

Type DAndC(P)
{
if Small(P)
return S(P);
else
{
divide P into smallerinstancesPi,P2,..Pk, k ≥ 1;

Apply DAndC to each of these sub problems;

return Combine(DAndC(P1),DAndC(P2), .... , DAndC(Pk));

Small(P) is a Boolean-valued function that determines whether the input Size is small enough that
the answer can be computed without splitting. If this is so, the function S is invoked. Otherwise
the problem P is divided into smaller sub problems. These sub problems P1, P2 .... Pk are solved by
recursive applications of DAndC. Combine is a function that determines the Solution to P using
the solutions to the k sub problems. If the size of P is n and the sizes of the k sub problems are n1,
n2, ..., nk respectively, then the computing time of DAndC is described by the recurrence relation,

Where, T(n) is the time for DAndC on any input of size n

g(n) is the time to compute the answer directly for small inputs.
The function f(n) is the time for dividing P and combining the solutions to sub problems.

For divide-and-conquer-based algorithms that produce sub problems of the same type as the
original problem, it is very natural to first describe such algorithms using recursion.
The complexity of many divide-and-conquer algorithms is given by recurrences of the form,

Where a and b are known constants. We assume that T(1)is known and n is a power of b (i.e. ,
n= bk). One of the methods for solving any such recurrence relation is called the substitution
method.

Binary Search
Binary search is an efficient searching technique that works with only sorted lists. So the list must
be sorted before using the binary search method. Binary search is based on divide-and-conquer
technique.
The method starts with looking at the middle element of the list. If it matches with the key
element, then search is complete. Otherwise, the key element may be in the first half or second
half of the list. If the key element is less than the middle element, then the search continues with
the first half of the list. If the key element is greater than the middle element, then the search
continues with the second half of the list. This process continues until the key element is found or
the search fails indicating that the key is not there in the list.

Let ai, 1<i <n, be a list of elements that are sorted in non-decreasing order. Consider the problem
of determining whether a given element x is present in the list. If x is present, we are to determine
a value j such that aj = x. If x is not in the list, then j is to be set to zero. Let P = (n, ai .. -
al ,x) denote an arbitrary instance of this search problem (n is the number of elements in the list,
ai, ... , al is the list of elements, and x is the element searched for).

Divide-and-conquer can be used to solve this problem. Let Small (P) be true if n = 1. In this case,
S(P)will take the value i if x = ai, otherwise it will take the value 0. If P has more than one
element, it can be divided (or reduced) into a new sub problem as follows. Pick an index q (in the
range [i,l] and compare x with aq. If q is always chosen such that aq is the middle element (that
is, q = Ḻ(n+1)/2˩), then the resulting search algorithm is known as binary search. There are three
possibilities:

(1) x = aq : In this case the problem P is immediately solved.

(2) x < aq: In this case x has to be searched for only in the sub list a2, aj+i, ... , a q-1.
Therefore, P reduces to (q – i, ai, ... , aq-1,x).
(3) x > aq: In this case the sub list to be searched is aq+1, ... , al, x).

Example 1:
Consider the list of elements: -4, -1, 0, 5, 10, 18, 32, 33, 98, 147, 154, 198, 250, 500. Trace the
binary search algorithm searching for the element -1.
Example 2

List of elements are,

-15, -6, 0, 7, 9, 23, 54, 82, 101, 112, 125, 131, 142, 151

placed in a[l:14], and simulate the steps that BinSearch goes through as it searches for different
values of x. Only the variables low, high, and mid need to be traced as we simulate the algorithm.
We try the following values for x: 151, -14, and 9 for two successful searches and one
unsuccessful search. The following entry shows the traces of BinSearch on these three inputs.

Advantages of Binary Search

The main advantage of binary search is that it is faster than sequential (linear) search. Because it
takes fewer comparisons, to determine whether the given key is in the list, then the linear search
method.

Disadvantages of Binary Search

The disadvantage of binary search is that can be applied to only a sorted list of elements. The
binary search is unsuccessful if the list is unsorted.

Efficiency of Binary Search

To evaluate binary search, count the number of comparisons in the best case, average case, and
worst case.
Program - Recursive Binary Search
int BinSrch ( Type a[], int i, int l, Type x)
// Given an array a[i :l] of elements in non-decreasing
// order, 1<=i <=l, determine whether x is present, and
// if so, return j such that x = = a[j]; else return 0.
{
if (l = = i)
{// if Small(P)
if (x = a[i]) return i;
else return 0;
}
else
{// ReduceP into a smaller sub-problem.
Int mid = (i+l)/2;
if (x = = a[mid]) return mid;
else if (x < a[mid]) return BinSrch(a, i, mid-1,x);
else return BinSrch(a,mid+1,l,x);
}

Program – Iterative Binary Search

int BinSrch ( Type a[], int n, Type x)
// Given an array a[i :n] of elements in non-decreasing
// order, n>=0, determine whether x is present, and
// if so, return j such that x = = a[j]; else return 0.
{
int low = 1, high = n;
while (low <= high) {
{
int mid = (low + high)/2;
if (x < a[mid]) high = mid - 1;
else if (x > a[mid]) low =mid + 1;
else return (mid);
}
return 0;
}
 Finding the Maximum and Minimum
The problem is to find the maximum and minimum items in a set of n elements. In analyzing the
time complexity of this algorithm, concentrate on the number of element comparisons. The
justification for this is that, the frequency count for other operations in this algorithm is of the
same order as that for element comparisons. More importantly, when the elements in a[1:n] are
polynomials, vectors, very large numbers, or strings of characters, the cost of an element
comparison is much higher than the cost of the other operation.

Straight forward Minimum and Maximum Algorithm

void StraightMaxMin(Type a[], int n, Type & max, Type & min)
// Set max to the maximum and min to the minimum of a[l:n]
{
max = min = a[l];
for( int i = 2; i <= n; i++)
{
if (a[i]>= max) max =a[i];
if (a[i]<= min) min =a[i];
}
}

StraightMaxMin requires 2(n - 1) element comparisons in the best, average, and worst cases. An
immediate improvement is possible by realizing that the comparison a[i] < min is necessary only
when a[i] > max is false. Hence we can replace the contents of the for loop by

if( a[i] > max max = a[i];

else if ( a[i] < min) min = a[i];

Now the best case occurs when the elements are in increasing order. The number of element
comparisons is n - 1. The worst case occurs when the elements are in decreasing order. In this case
the number of element comparison is s 2(n - 1).The average number of element comparisons is
less than 2(n - 1). On the average, a[i] is greater than max half the time, and so the average
number of comparisons is 3n/2 – 1.

A divide-and-conquer algorithm for this problem would proceed as follows: Let P = (n,a[i], ...,
a[j]) denote an arbitrary instance of the problem. Here n is the number of elements in the list a[i],
..., a[j] and we are interested in finding the maximum and minimum of this list. Let Small(P) be
true when n ≤ 2. In this case, the maximum and minimum are a[i] if n = 1. If n = 2, the problem
can be solved by making one comparison.

If the list has more than two elements, P has to be divided into smaller instances. For example, we
might divide P into the two instances P1 = (∟n/2˩, a[l],...,a[∟n/2˩) and P2 = (n - ∟n/2˩,
a[∟n/2˩ + 1], ..., a[n). After having divided P into two smaller sub problems it can solve by
recursively invoking the same divide-and-conquer algorithm. If MAX(P) and MIN(P) are the
maximum and minimum of the elements in P, then MAX(P) is the larger of MAX(P1) and
MAX(P2). Also, MIN(P) is the smaller of MIN(P1)and MIN(P2).
The following algorithm is recursively finding the maximum and minimum.

1 void MaxMin (int i, int j, Type & max, Type & min)
2 // a[1:n] is a global array. Parameters i and j are integers,
3 // 1 <= i <= j <=n. The effect is to set max and min to the
4 // largest and smallest values in a[i :j],respectively.
5 {
6 if (i = = j) max = min = a[i]; // Small(P)
7 else if (i = = j - 1) { // Another case of Small(P)
8 if (a[i] < a[j]) { max = a[j]; min =a[i]; }
9 else { max = a[i]; min:=a[j]; }
10 }
11 else { // if Pis not small, divide P into sub-problems.
12 // Find where to split the set.
13 int mid = (i+j)/2; Tpe max1, min1;
14 // Solve the sub-problems.
15 MaxMin (i, mid, max, min);
16 MaxMin (mid+l, j, maxl, minl);
17 // Combine the solutions.
18 if (max < max1) max = maxl;
19 if (min > min1) min = mini;
20 }
21 }

Trees of Recursive Calls of MaxMin

 Merge Sort
Merge sort is based on divide-and-conquer technique. Merge sort method is a two phase process,

1. Dividing

2. Merging

Dividing Phase: During the dividing phase, each time the given list of elements is divided into
two parts. This division process continues until the list is small enough to divide.

Merging Phase: Merging is the process of combining two sorted lists, so that, the resultant list is
also the sorted one. Suppose A is a sorted list with n1 element and B is a sorted list with n2
elements. The operation that combines the elements of A and B into a single sorted list C with
n=n1 + n2, elements is called merging.

A sorting algorithm that has the nice property that in the worst case its complexity is O (n log n).
This algorithm is called merge sort. We assume throughout that the elements are to be sorted in
non-decreasing order. Given a sequence of n elements (also called keys) a[1], ..., a[n], the general
idea is to imagine them split into two sets a[1], ... , a[∟n/2˩] and a[∟n/2˩+1], ... , a[n]. Each set is
individually sorted, and the resulting sorted sequences are merged to produce a single sorted
sequence of n elements. Thus we have another ideal Example of the divide-and-conquer strategy
in which the splitting is into two equal-sized sets and the combining operation is the merging of
two sorted sets into one.

Merge Sort - Algorithm

void MergeSort (int low, int high)

// a[low :high] is a global array to be sorted.
// Small(P) is true if there is only one element
// to sort. In this case the list is already sorted.
{
if (low < high) { //If there are more than one element
// Divide P into sub-problems.
// Find where to split the set.
int mid = (low + high)/2;
// Solve the sub-problems.
MergeSort (low, mid);
MergeSort (mid + 1, high);
// Combinethe solutions.
Merge(low, mid, high);

}
Merging two Sorted Sub-arrays using Auxiliary Storage

void Merge( int low, int mid, int high)

// a[low :high] is a global array containing two sorted
// subsets in a [low :mid] and in a[mid+ 1: high]. The goal
//is to merge these two sets into a single set residing in a[low : high].
// b[ ] is an auxiliary global array.
{
int h =low; i = low; j = mid + 1, k;
while ((h <= mid) && (j <= high)) {
if (a[h] <= a[j]) {b[i] = =a[h]; h = h + 1; }
else {b[i] = a[j]j; j++;} i++;

} //while

if (h > mid) for( k = j; k <= high; k++) {

b[i] =a[k]; i++;
}
else for( k = h; k <= mid; k++) {
b[i] =a[k]; i++;
}
for( k = low; k <= high; k++ ) a[k] = b[k];
}

Example:
Tree of Calls of MergeSort(1l0, )
 Quick Sort

The function Partition in the algorithm accomplishes an in-place partitioning of the elements of
a[m:p - 1]. It is assumed that a[p] ≥ a[m] and that a[m] is the partitioning element. If m = 1 and p -
1 = n, then a[n + 1] must be defined and must be greater than or equal to all elements in a[1:n].
The assumption that a[m] is the partition element is merely for convenience; other choices for the
partitioning element than the first item in the set are better in practice. The function Interchange
(a, i, j) exchanges a[i] with a[j].

int Partion (Type a[], int m, int p)

// Within a[m], a[m+1], ... , a[p-1] the elements are
// rearranged in such a manner that if initially t = = a[m],
// then after completion a[q] = = t for some q between m and p - 1, a[k] <= t
//for m <= k < q, and a[k] >= t for q <k <p. q is returned.
{
Type v = a[m]; int i = m; j=p;
do {
do i++
while (a[i] < v);
do j--;
while (a[j] > v);
if (i < j) Interchange (a, i, j);
} while (i < j);
a[m] = a[j]; a[j] =v; return(j);
}

inline void Interchange(Type a[], int i, int j)

// Exchange a[i] with a[j].
{
Type p = a[i];
a[i] = a[j; a[j] = p;
}
Example:

to the elements in S2. Hence S1 and S2 can be sorted independently. Each set is sorted by reusing
the function Partition. The following algorithm describes the complete process.

void QuickSort (int p, int q)

// Sorts the elements alp], ..., a[q] which reside in the global
// array a[1:n] into ascending order; a[n+ 1] is considered to
// be defined and must be >= all the elements in a[l:n].
{
if (p < q) { // If there are more than one element
// divide P into two sub-problems.
int j = Partition (a, p, q+ 1);
// j is the position of the partitioning element.
// Solve the sub-problems.
QuickSort (p, j-1);
QuickSort(j+ l, q);
// There is no need for combining solutions.
}
}
Performance Measurement

Table3.5 Average computing times for two sorting algorithms on random inputs

Table3.6 Worst-case computing times for two sorting algorithms on random inputs

Scanning the tables, we immediately see that QuickSort is faster than MergeSort for all values.
Even though both algorithms require O(n log n) time on the average, QuickSort usually performs
well in practice.

Selection
The Partition algorithm can also be used to obtain an efficient Solution for the selection problem.
In this problem, we are given n elements a[1 : n] and are required to determine the kth-smallest
element. If the Partitioning element v is positioned at a[j], then j-1 elements are less than or equal
to a[j] and n- j elements are greater than or equal to a[j]. Hence if k < j, then the kth-smallest
element is in a[l: j - 1]; if k = j, then a[j] is the kth-smallest element; and if k >j, then the kth-
smallest element is the (k - j)th-smallest element in a[j + 1: n]. The resulting algorithm is function
Selectl below. This function places the kth-smallest element into position a[k] and partitions the
remaining elements so that a[i] ≤ a[k], 1≤ i < k, and a[i] ≥a[k], k < i ≤ n.

void Selectl (Type a[], int a, int k)

// Selects the kth-smallest element in a[l:n] and places it in the kth position of a[ ]
// The remaining elements are rearranged such that a[m] <= a[k] for 1<= m < k, and
// a[m] >= a[k] for k <m <=n.
{
int low =1; up = n + 1;
 a[n+ 1]= INFTY; // a[n+ 1]is set to infinity.
do { // Each time the loop is entered, 1<= low <=k <=up <= n+ 1.
int j :=Partition(a, low, up);
//j is such that a[j] is the jth-smallest value in a[ ].
if (k = = j) return;
else if (k < j) up =j; // j is the new upper limit.
else low = j + 1; // j + 1is the new lower limit.
}while(TRUE);
}

Example

The array has the nine elements 65, 70, 75, 80, 85, 60, 55, 50, and 45, with a[10] = ∞. If k = 5,
then the first call of Partition will be sufficient since 65 is placed into a[5]. Instead, assume that
we are looking for the seventh-smallest element of a, that is, k = 7. The next invocation of
Partition is Partition (6, 10).
Strassen's Matrix Multiplication
Where a and b are constants.

∞∞∞-∞∞∞-∞∞∞
Big O notation

Big O notation tells the number of operations an algorithm will make. It gets its name from the
literal "Big O" in front of the estimated number of operations. Big-O notation represents the upper
bound of the running time of an algorithm. Thus, it gives the worst-case complexity of an
algorithm.

O(g(n)) = { f(n): there exist positive constants c and n0

such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }

This expression can be described as a function f(n) belongs to the set O(g(n)) if there exists a
positive constant c such that it lies between 0 and cg(n), for sufficiently large n.

For any value of n, the running time of an algorithm does not cross the time provided by O(g(n)).

Since it gives the worst-case running time of an algorithm, it is widely used to analyze an
algorithm in the worst-case scenario.

Here are some common algorithms and their run times in Big O notation:

BIG O EXAMPLE
NOTATION ALGORITHM
O(log n) Binary search
O(n) Simple search
O(n * log n) Quick sort
O(n2) Selection sort
Travelling
O(n!) salesperson
Omega Notation (Ω-notation)

Omega notation represents the lower bound of the running time of an algorithm. Thus, it provides
the best case complexity of an algorithm.

Ω(g(n)) = { f(n): there exist positive constants c and n0

such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }

This expression can be described as a function f(n) belongs to the set Ω(g(n)) if there exists a
positive constant c such that it lies above cg(n), for sufficiently large n.

For any value of n, the minimum time required by the algorithm is given by Omega Ω(g(n))

Theta Notation (Θ-notation)

Theta notation encloses the function from above and below. Since it represents the upper and the
lower bound of the running time of an algorithm, it is used for analyzing the average-case
complexity of an algorithm.
For a function g(n), Θ(g(n)) is given by the relation:

Θ(g(n)) = { f(n): there exist positive constants c1, c2 and n0

such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0 }

This expression can be described as a function f(n) belongs to the set Θ(g(n)) if there exist positive
constants c1 and c2 such that it can be sandwiched between c1g(n) and c2g(n), for sufficiently
large n.

If a function f(n) lies anywhere in between c1g(n) and c2g(n) for all n ≥ n0, then f(n) is said to be
asymptotically tight bound.