Algorithm Design

Techniques

Dr M Muzammal
Divide and Conquer
The most-well known algorithm design strategy:
 Divide instance of problem into two or more smaller
instances

 Solve smaller instances recursively

 Obtain solution to original (larger) instance by combining

these solutions
Divide and Conquer
 Divide the problem into sub-problems that are similar to
the original but smaller in size

 Conquer the sub-problems by solving them recursively. If

they are small enough, just solve them in a straightforward
manner.

 Combine the solutions to create a solution to the original

problem
Analysis:
Divide and Conquer Algorithm
D&C algorithms result into following general recurrence relation:
T(n) = aT(n/b) + f (n) where f(n)  (nd), d  0
Master Theorem: If a < bd, T(n)  (nd)
If a = bd, T(n)  (nd log n)
If a > bd, T(n)  (nlog b a )

Examples:
T(n) = 4T(n/2) + n  T(n)  ?
Let’s Solve it
Analysis:
Divide and Conquer Algorithm
Examples
T(n) = 9T(n/3) + n
T(n) = 8T(n/2) + n2
Divide and Conquer:
Merge Sort
 Divide: Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each

 Conquer: Sort the two subsequences recursively using merge

sort.

 Combine: Merge the two sorted subsequences to produce

the sorted answer
Divide and Conquer:
Merge Sort
8 3 2 9 7 1 5 4

8 3 2 9 7 1 5 4

8 3 2 9 71 5 4

8 3 2 9 7 1 5 4

3 8 2 9 1 7 4 5

2 3 8 9 1 4 5 7

1 2 3 4 5 7 8 9
Divide and Conquer:
Analysis of Merge Sort
 Divide: Computing the middle takes (1)
– copying two n/2 elements arrays take 2*n/2 = (n) time
 Conquer: Solving 2 sub-problems takes 2T(n/2)
 Combine: Merging n elements takes (n)

Total Execution Time:

T(n) = (1) if n = 1
T(n) = 2T(n/2) + (n) if n > 1
Divide and Conquer:
Quick Sort
 Select a pivot (partitioning element) – here, the first element
 Rearrange the list so that all the elements in the first s
position are smaller than or equal to the pivot and all the
elements in the remaining n-s position are larger than or
equal to the pivot
p 

A[i]p A[i]p
Divide and Conquer:
Quick Sort
 Exchange the pivot with the last element in the first (i.e., )
subarray — the pivot is now in its final position
 Sort the two subarrays recursively
Efficiency:
Best case: split in the middle — Θ(n log n)
Worst case: sorted array! — Θ(n2)
Average case: random arrays — Θ(n log n)
Divide and Conquer:
Binary Tree Traversal
 Graph algorithms: binary tree traversals
 Inorder traversal:
 traverse left subtree of current vertex
 visit current vertex
 traverse right subtree of current vertex
 Preorder traversal similar, but visit current vertex first
 Postorder traversal similar, but visit current vertex last
 All three take O(n) time, where n is number of nodes in tree
 Note difference from searching in a binary tree
Divide and Conquer:
Closest Pair Problem
 Recall the problem: Given n points in the plane, find two that
are the minimum distance apart.
 Brute force algorithm took Θ(n2) time.
 Try to do better with divide and conquer:
― divide points into two disjoint subsets
― recursively find closest pairs in the two subsets
― somehow combine results to get final answer
Divide and Conquer:
Closest Pair Problem
 Separate points into two equal-sized groups on either side of a
vertical line
 Recursively compute closest pair for left group and for right
group
– what should base of the recursion be?
 Check if there is a smaller distance between two points on
opposite sides of the vertical line
– This is the tricky part
Divide and Conquer:
Closest Pair Problem
 d is min. of min. distance on right and min.
distance on left
 any pair with distance < d must be in this
strip of width 2d centered around dividing
line
 consider points in strip from bottom to top
 for each such point, compare it against
other points in the strip that could possibly
be closer
 there are only a constant number of these
other points!
Divide and Conquer:
Closest Pair Problem
Each box is d/2 by d/2

No point in comparing p
against points in red area –
more than d away

d Just need to worry about

the six blue boxes
p

Each box contains at most one

d d point, since maximum distance
in a box is d/√2, which is < d 15
Divide and Conquer:
Closest Pair Problem
 ClosestPairDist(P):
– if n is small then return result of brute force algorithm
– Pl := left half of P w.r.t. x-coordinate
– Pr := right half of P w.r.t. x-coordinate
– dl := ClosestPairDist(Pl)
– dr := ClosestPairDist(Pr)
– d := min(dl,dr)
– for each point p in S (2d-wide center strip) do
o for each point q in one of the six boxes do
• d := min(dist(p,q),d)
– return d
Divide and Conquer:
Closest Pair Problem
 Before calling recursive code, preprocess:
– sort P into array PX by increasing x-coordinate
– sort P into array PY by increasing y-coordinate
 Use PX to efficiently divide P into half w.r.t. x-coordinates
 Use PY to efficiently scan up the 2d-wide center strip
Divide and Conquer:
Closest Pair Problem Analysis
 Preprocessing takes O(n log n) time
 Recursive code, if implemented carefully, has running time
described by this recurrence:
T(n) = 2T(n/2) + O(n)
– I.e., two recursive calls (left half and right half)
– rest of the work takes time linear in the number of points
being handled
– Solution is T(n) = O(n log n)
 Total time is O(n log n); beats brute force
Divide and Conquer:
Multiplication of Large Integers
 Consider the problem of multiplying two (large) n-digit
integers represented by arrays of their digits such as:
A = 12345678901357986429 B = 87654321284820912836
The grade-school algorithm:
a1 a2 … an
b1 b2 … bn
(d10) d11d12 … d1n
(d20) d21d22 … d2n
…………………
(dn0) dn1dn2 … dnn

Efficiency: Θ(n2) single-digit multiplications

Divide and Conquer:
Multiplication of Large Integers
Divide And Conquer Solution

Let X and Y be two n-digit numbers

X= a b
Y= c d

X  a10n / 2  b, Y  c10n / 2  d
X *Y  a * c10n  (a * d  b * c)10n / 2  b * d
Divide and Conquer:
Multiplication of Large Integers

 1 if n = 1
T ( n)  
4T (n / 2)  (n) if n > 1

Solution by Master Theorem:

T (n)   n  2
Divide and Conquer:
Large Matrix Multiplication
Input:
Output:
Divide and Conquer:
Large Matrix Multiplication
Idea:
n×n matrix = 2×2 matrix of (n/2)×(n/2) submatrices

8 mults of (n/2)×(n/2) submatrices

4 adds of (n/2)×(n/2) submatrices s submatrices
Divide and Conquer:
Large Matrix Multiplication
 8 multiplications of (n/2)×(n/2) sub-matrices
 4 additions of (n/2)×(n/2) sub-matrices

Recurrence:
T(n) = 8 T(n/2) + 4n2

O(n3) solution
Divide and Conquer:
Strassen’s Matrix Multiplication
 Reduce Multiplications
Divide and Conquer:
Strassen’s Matrix Multiplication
Recurrence:
T(n) = 7 T(n/2) + 18n2

T(n) = Θ(n 2.81)

Best to date: Θ(n2.376)

Algorithm Design Technique

Transform and Conquer

Transform and Conquer

A group of design methods based on the idea of transformation

 Two stage procedure:
― Transformation: problem instance is modified (transformed)
― Conquer: modified problem instance is solved
 Variations:
– Instance Simplification
– Representation Change
– Problem Reduction
Transform and Conquer:
Instance Simplification
 Solve a problem’s instance by transforming it into another
simpler/easier instance of the same problem
Presorting
Many problems involving lists are easier when list is sorted.
― Searching
― Computing the median (selection problem)
― Checking if all elements are distinct (element uniqueness)
Instance Simplification:
Searching With Presorting
Problem: Search for a given K in A[0..n-1]

 Presorting-based algorithm:
― Stage 1 Sort the array by an efficient sorting algorithm
― Stage 2 Apply binary search

Efficiency: Θ(nlog n) + O(log n) = Θ(nlog n)

Transform and Conquer:
Representation Change
 Solve the problem by transforming into a different
representation of the same problem

Example:
Polynomial Evaluation
Transform and Conquer:
Representation Change
 Polynomial Evaluation:

Given a polynomial of degree n

p(x) = anxn + an-1xn-1 + … + a1x + a0
and a specific value of x, find the value of p at that point.

Two brute-force algorithms:

p0 p  a0; power  1
for i  n downto 0 do for i  1 to n do
power  1 power  power * x
for j  1 to i do p  p + ai * power
power  power * x return p
p  p + ai * power
return p
Representation Change:
Polynomial Evaluation
Horner’s rule:
Example: p(x) = 2x4 - x3 + 3x2 + x - 5
= x(2x3 - x2 + 3x + 1) - 5
= x(x(2x2 - x + 3) + 1) - 5
= x(x(x(2x - 1) + 3) + 1) - 5
 Substitution into the last formula leads to a faster
algorithm
Representation Change:
Polynomial Evaluation
 Same sequence of computations are obtained by simply
arranging the coefficient in a table and proceeding as follows:
coefficients 2 -1 3 1 -5
x=3 2 3*2+(-1)=5 3*5+3=18 3*18+1=55 3*55+(-5)=160
Transform and Conquer:
Problem Reduction
 Solve a problem by reducing it to another problem whose
solution is known to you.

To be of practical value, the combined time of the

transformation and solving the other problem should be
smaller than solving the problem as given by another method.
Problem Reduction:
Find LCM
Problem Reduction:
Circuit Reduction to Graph
Problem Reduction:
Maps Reduction Graph
END