Chapter 11

Approximation
Algorithms

Slides by Kevin Wayne.

Copyright @ 2005 Pearson-Addison Wesley.
All rights reserved.

1
 Approximation Algorithms

Q. Suppose I need to solve an NP-hard problem. What should I do?

A. Theory says you're unlikely to find a poly-time algorithm.

Must sacrifice one of three desired features.

Solve problem to optimality.
Solve problem in poly-time.
Solve arbitrary instances of the problem.

-approximation algorithm.
Guaranteed to run in poly-time.
Guaranteed to solve arbitrary instance of the problem
Guaranteed to find solution within ratio of true optimum.

Challenge. Need to prove a solution's value is close to optimum, without

even knowing what optimum value is!

2
11.1 Load Balancing
 Load Balancing

Input. m identical machines; n jobs, job j has processing time tj.

Job j must run contiguously on one machine.
A machine can process at most one job at a time.

Def. Let J(i) be the subset of jobs assigned to machine i. The

load of machine i is Li = j J(i) tj.

Def. The makespan is the maximum load on any machine L = maxi Li.

Load balancing. Assign each job to a machine to minimize makespan.

4
 Load Balancing: List Scheduling

List-scheduling algorithm (Greedy based algorithm).

Consider n jobs in some fixed order.
Assign job j to machine whose load is smallest so far.

List-Scheduling(m, n, t1,t2,,tn) {
for i = 1 to m {
Li 0 load on machine i
J(i) jobs assigned to machine i
}

for j = 1 to n {
i = argmink Lk machine i has smallest load
J(i) J(i) {j} assign job j to machine i
Li Li + tj update load of machine i
}
return J(1), , J(m)
}

Implementation. O(n log m) using a priority queue.

5
 Load Balancing: List Scheduling Analysis

Theorem. [Graham, 1966] Greedy algorithm is a 2-approximation.

First worst-case analysis of an approximation algorithm.
Need to compare resulting solution with optimal makespan L*.

Lemma 1. The optimal makespan L* maxj tj.

Pf. Some machine must process the most time-consuming job.

Lemma 2. The optimal makespan L * m1 j t j .

Pf.
The total processing time is j tj .
One of m machines must
do at least a 1/m fraction of total work.

6
 Load Balancing: List Scheduling Analysis

Theorem. Greedy algorithm is a 2-approximation.

Pf. Consider load Li of bottleneck machine i.
Let j be last job scheduled on machine i.
When job j assigned to machine i, i had smallest load. Its load
before assignment is Li - tj Li - tj Lk for all 1 k m.

blue jobs scheduled before j

machine i j

0
Li - tj L = Li
7
 Load Balancing: List Scheduling Analysis

Theorem. Greedy algorithm is a 2-approximation.

Pf. Consider load Li of bottleneck machine i.
Let j be last job scheduled on machine i.
When job j assigned to machine i, i had smallest load. Its load
before assignment is Li - tj Li - tj Lk for all 1 k m.
Sum inequalities over all k and divide by m:

Li tj m1 k Lk k=1,,m
m1 k t k k=1,,n
Lemma 1 L*

Now Li (Li t j ) t j 2L *.

L* L*

Lemma 2

Q. Is our analysis tight?

8
 Load Balancing: List Scheduling Analysis

Q. Is our analysis tight?

A. Essentially yes.

Ex: m machines, m(m-1) jobs length 1 jobs, one job of length m

machine 2 idle
machine 3 idle
machine 4 idle
m = 10 machine 5 idle
machine 6 idle
machine 7 idle
machine 8 idle
machine 9 idle
machine 10 idle

list scheduling makespan = 19

9
 Load Balancing: List Scheduling Analysis

Q. Is our analysis tight?

A. Essentially yes.

Ex: m machines, m(m-1) jobs length 1 jobs, one job of length m

m = 10

optimal makespan = 10

10
 Load Balancing: LPT Rule

Longest processing time (LPT). Sort n jobs in descending order of

processing time, and then run list scheduling algorithm.
(intuition: We want to add the last job not poking out that much)

LPT-List-Scheduling(m, n, t1,t2,,tn) {
Sort jobs so that t1 t2 tn

for i = 1 to m {
Li 0 load on machine i

J(i) jobs assigned to machine i

for j = 1 to n {
i = argmink Lk machine i has smallest load
J(i) J(i) {j} assign job j to machine i

L i Li + tj update load of machine i

}
return J(1), , J(m)
}

11
 Load Balancing: LPT Rule

Observation. If at most m jobs, then list-scheduling is optimal.

Pf. Each job put on its own machine.

Lemma 3. If there are more than m jobs, L* 2 tm+1.

Pf.
Consider first m+1 jobs t1, , tm+1.
Since the ti's are in descending order, each takes at least tm+1 time.
There are m+1 jobs and m machines, so by pigeonhole principle, at
least one machine gets two jobs.

Theorem. LPT rule is a 3/2 approximation algorithm.

Pf. Same basic approach as for list scheduling.

L i (Li t j ) t j 3 L *.
2
L* 12 L*

Lemma 3
( by observation, can assume number of jobs > m )
12
 Load Balancing: LPT Rule

Q. Is our 3/2 analysis tight?

A. No.

Theorem. [Graham, 1969] LPT rule is a 4/3-approximation.

Pf. More sophisticated analysis of same algorithm.

Q. Is Graham's 4/3 analysis tight?

A. Essentially yes.

Ex: m machines, n = 2m+1 jobs, 2 jobs of length m+1, m+2, , 2m-1 and
one job of length m.

13
11.2 Center Selection
 Center Selection Problem

Input. Set of n sites s1, , sn and integer k > 0.

Center selection problem. Select k centers C so that maximum

distance from a site to nearest center is minimized.

k=4

r(C)

center
site

15
 Center Selection Problem

Input. Set of n sites s1, , sn and integer k > 0.

Center selection problem. Select k centers C so that maximum

distance from a site to nearest center is minimized.

Notation.
dist(x, y) = distance between x and y.
dist(si, C) = min c C dist(si, c) = distance from si to closest center.
r(C) = maxi dist(si, C) = smallest covering radius.

Goal. Find set of centers C that minimizes r(C), subject to |C| = k.

Distance function properties.

dist(x, x) = 0 (identity)
dist(x, y) = dist(y, x) (symmetry)
dist(x, y) dist(x, z) + dist(z, y) (triangle inequality)

16
 Center Selection Example

Ex: each site is a point in the plane, a center can be any point in the
plane, dist(x, y) = Euclidean distance.

Remark: search can be infinite!

r(C)

center
site

17
 Greedy Algorithm: A False Start

Greedy algorithm. Put the first center at the best possible location
for a single center, and then keep adding centers so as to reduce the
covering radius each time by as much as possible.

Remark: arbitrarily bad!

greedy center 1

center
k = 2 centers site

18
 Center Selection: Greedy Algorithm

Greedy algorithm. Repeatedly choose the next center to be the site

farthest from any existing center.

Greedy-Center-Selection(k, n, s1,s2,,sn) {

C =
repeat k times {
Select a site si with maximum dist(si, C)
Add si to C
} site farthest from any center
return C
}

Observation. Upon termination all centers in C are pairwise at least r(C)

apart.
Pf. By construction of algorithm (considering the last center site sk; it
is the site that is furthest away from C - sk).

19
 Center Selection: Analysis of Greedy Algorithm

Theorem. Let C* be an optimal set of centers. Then r(C) 2r(C*).

Pf. (by contradiction) Assume r(C*) < r(C).
For each site ci in C, consider ball of radius r(C) around it.
Exactly one ci* in each ball; let ci be the site paired with ci*.
Consider any site s and its closest center ci* in C*.
dist(s, C) dist(s, ci) dist(s, ci*) + dist(ci*, ci) 2r(C*).
Thus r(C) 2r(C*).
-inequality r(C*) since ci* is closest center

r(C) r(C)

ci
r(C)
C*
ci*
sites s

20
 Center Selection

Theorem. Let C* be an optimal set of centers. Then r(C) 2r(C*).

Theorem. Greedy algorithm is a 2-approximation for center selection

problem.

Remark. Greedy algorithm always places centers at sites, but is still

within a factor of 2 of best solution that is allowed to place centers
anywhere.
e.g., points in the plane

Question. Is there hope of a 3/2-approximation? 4/3?

Theorem. Unless P = NP, there no -approximation for center-selection

problem for any < 2 (not proved here).

21
11.4 The Pricing Method: Vertex Cover
 Weighted Vertex Cover

Definition. Given a graph G = (V, E), a vertex cover is a set S V such

that each edge in E has at least one end in S.

Weighted vertex cover. Given a graph G with vertex weights, find a

vertex cover of minimum weight. (NP hard problem)

all nodes with weight of 1 reduces the problem to standard vertex

cover problem.
2 4 2 4

2 9 2 9

weight = 2 + 2 + 4 weight = 11

23
 Pricing Method

Pricing method. Set prices and find vertex cover simultaneously.

Weighted-Vertex-Cover-Approx(G, w) {
foreach e in E
pe = 0

while ( edge e=(i,j) such that neither i nor j are

tight)
select such an edge e
increase pe as much as possible until i or j tight
}

S set of all tight nodes

pe wi
e (i , j )
return S
}

Why S is a vertex cover set? (use contradiction to prove)

24
 Approximation method: Pricing Method

Pricing method. Each edge must be covered by some vertex.

Edge e = (i, j) pays price pe 0 to use vertex i and j.

Fairness. Edges incident to vertex i should pay wi in total.

2 4

for each vertex i : pe wi

e(i, j)

2 9

Lemma. For any vertex cover S and any fair prices pe: e pe w(S).
Pf.

pe pe wi w( S ).
e E i S e ( i , j ) iS

each edge e covered by sum fairness inequalities

at least one node in S for each node in S

25
 Pricing Method

price of edge a-b

vertex weight

Figure 11.8

Example shows the pricing method does not provide the optimal
weighted vertex cover solution

26
 Pricing Method: Analysis

Theorem. Pricing method is a 2-approximation.

Pf.
Algorithm terminates since at least one new node becomes tight
after each iteration of while loop.

Let S = set of all tight nodes upon termination of algorithm. S is a

vertex cover: if some edge i-j is uncovered, then neither i nor j is
tight. But then while loop would not terminate.

Let S* be optimal vertex cover. We show w(S) 2w(S*).

w(S) wi pe pe 2 pe 2w(S*).
i S i S e(i, j) iV e(i, j) e E

all nodes in S are tight S V, each edge counted twice fairness lemma
prices 0

27
11.6 LP Rounding: Vertex Cover
 Weighted Vertex Cover

Weighted vertex cover. Given an undirected graph G = (V, E) with

vertex weights wi 0, find a minimum weight subset of nodes S such
that every edge is incident to at least one vertex in S.

10 A F
6 9

16 B G
7 10

6 C
3 H 9

23 D I 33

7 E J
10 32

total weight = 55

29
 Weighted Vertex Cover: IP Formulation

Weighted vertex cover. Given an undirected graph G = (V, E) with

vertex weights wi 0, find a minimum weight subset of nodes S such
that every edge is incident to at least one vertex in S.

Integer programming formulation.

Model inclusion of each vertex i using a 0/1 variable xi.

0 if vertex i is not in vertex cover

xi
1 if vertex i is in vertex cover

Vertex covers in 1-1 correspondence with 0/1 assignments:

S = {i V : xi = 1}

Objective function: minimize i wi xi.

Constraints:..
Must take either i or j: xi + xj 1.

30
 Weighted Vertex Cover: IP Formulation

Weighted vertex cover. Integer programming formulation.

( ILP) min wi xi
i V
s. t. xi x j 1 (i, j) E
xi {0,1} i V

Observation. If x* is optimal solution to (ILP), then S = {i V : x*i = 1}
is a min weight vertex cover.

31
 Integer Programming

INTEGER-PROGRAMMING. Given integers aij and bi, find integers xj that

satisfy:

n
t
max c x aij x j bi 1 i m
j1
s. t. Ax b
xj 0 1 j n
x integral
xj integral 1 j n

Observation. Vertex cover formulation proves that integer
programming is NP-hard search problem.

even if all coefficients are 0/1 and

at most two variables per inequality

32
 Linear Programming

Linear programming. Max/min linear objective function subject to

linear inequalities.
Input: integers cj, bi, aij .
Output: real numbers xj.
n
(P) max cj xj
j1
t
(P) max c x n
s. t. aij x j bi 1 i m
s. t. Ax b j1
x 0 xj 0 1 j n

Linear.
No x2, xy, arccos(x), x(1-x), etc.

Simplex algorithm. [Dantzig 1947] Can solve LP in practice.

Ellipsoid algorithm. [Khachian 1979] Can solve LP in poly-time.

33
 LP Feasible Region

LP geometry in 2D.

x1 = 0

x2 = 0

x1 + 2x2 = 6
2x1 + x2 = 6
34
 LP Feasible Region

LP geometry in 3D.

Graph is from Wikipiedia.com

http://en.wikipedia.org/wiki/Linear_programming

35
 Weighted Vertex Cover: LP Relaxation

Weighted vertex cover. Linear programming formulation.

( LP) min wi xi
i V
s. t. xi x j 1 (i, j) E
xi 0 i V

Observation.
Optimal value of (LP) is optimal value of (ILP).
Pf. LP has fewer constraints.

Note. LP is not equivalent to vertex cover.

Q. How can solving LP help us find a small vertex cover?

A. Solve LP and round fractional values.

36
 Weighted Vertex Cover

Theorem. If x* is optimal solution to (LP), then S = {i V : x*i } is a

vertex cover whose weight is at most twice the min possible weight.

Pf. [S is a vertex cover]

Consider an edge (i, j) E.
Since x*i + x*j 1, either x*i or x*j (i, j) covered.

Pf. [S has desired cost]

Let S* be optimal vertex cover. Then

wi wi xi* 1
2 wi
i S* iS iS

LP is a relaxation x*i

37
 Weighted Vertex Cover

Theorem. 2-approximation algorithm for weighted vertex cover.

Theorem. [Dinur-Safra 2001] If P NP, then no -approximation

for < 1.3607, even with unit weights.

10 5 - 21

Open research problem. Close the gap.

38
11.8 Knapsack Problem
 Polynomial Time Approximation Scheme

PTAS. (1 + )-approximation algorithm for any constant > 0.

Load balancing. [Hochbaum-Shmoys 1987]
Euclidean TSP (travel salesman problem). [Arora 1996]

Consequence. PTAS produces arbitrarily high quality solution, but trades

off accuracy for time.

This section. PTAS for knapsack problem via rounding and scaling.

40
 Knapsack Problem

Knapsack problem.
Given n objects and a "knapsack."
Item i has value vi > 0 and weighs wi > 0. we'll assume wi W

Knapsack can carry weight up to W.

Goal: fill knapsack so as to maximize total value.

Ex: { 3, 4 } has value 40. Item Value Weight

1 1 1
2 6 2
W = 11
3 18 5
4 22 6
5 28 7

41
 Knapsack is NP-Complete

KNAPSACK: Given a finite set X, nonnegative weights wi, nonnegative

values vi, a weight limit W, and a target value V, is there a subset S X
such that:
wi W
iS

vi V
iS

SUBSET-SUM: Given a finite set X, nonnegative values ui, and an integer

U, is there a subset S X whose elements sum to exactly U?

Claim. SUBSET-SUM P KNAPSACK.

Pf. Given instance (u1, , un, U) of SUBSET-SUM, create KNAPSACK
instance:
vi wi ui ui U
iS
V W U ui U
iS

42
 Knapsack Problem: Dynamic Programming 1

Def. OPT(i, w) = max value subset of items 1,..., i with weight limit w.
Case 1: OPT does not select item i.
OPT selects best of 1, , i1 using up to weight limit w
Case 2: OPT selects item i.
new weight limit = w wi
OPT selects best of 1, , i1 using up to weight limit w wi

0 if i 0

OPT(i, w) OPT(i 1, w) if w i w
max OPT(i 1, w), v OPT(i 1, w w ) otherwise
i i

Running time. O(n W). (introduced in 06dynamic-programming.ppt

lecture notes)

W = weight limit.
Not polynomial in input size!

43
 Knapsack Problem: Dynamic Programming II

Def. OPT(i, v) = min weight subset of items 1, , i that yields value

exactly v.
Case 1: OPT does not select item i.
OPT selects best of 1, , i-1 that achieves exactly value v
Case 2: OPT selects item i.
consumes weight wi, add new value of vi, which means:
OPT selects best of 1, , i-1 that achieves exactly value v- vi.

0 if v 0

if i 0, v > 0
OPT (i, v)
OPT (i 1, v) if v i v

min OPT (i 1, v), wi OPT (i 1, v vi ) otherwise

V* n vmax

Running time. O(n V*) = O(n2 vmax).

V* = optimal value = maximum v such that OPT(n, v) W.
Not polynomial in input size!
44
 Knapsack: FPTAS

Intuition for approximation algorithm.

Round all values up to lie in smaller range.
Run dynamic programming algorithm on rounded instance.
Return optimal items in rounded instance.
FPTAS: Fully Polynomial Time. Approximation Schemes

Item Value Weight Item Value Weight

1 934,221 1 1 1 1
2 5,956,342 2 2 6 2
3 17,810,013 5 3 18 5
4 21,217,800 6 4 22 6
5 27,343,199 7 5 28 7

W = 11 W = 11

original instance rounded instance

45
 Knapsack: FPTAS

Knapsack FPTAS. Round up all values:

vmax = largest value in original instance

= precision parameter
= scaling factor = vmax / n

Observation. Optimal solution to problems with v or v are equivalent.

Intuition. v close to v so optimal solution using v is nearly optimal;
small and integral so dynamic programming algorithm is fast.

Running time. O(n3 / ).
Dynamic program II running timeis
, where

46
 Knapsack: FPTAS

Knapsack FPTAS. Round up all values:

Theorem. If S is solution found by our algorithm and S* is any other

feasible solution then (1 ) vi vi
iS i S*

Pf. Let S* be any feasible solution satisfying weight constraint.

vi vi
always round up

i S* i S*

solve rounded instance optimally

vi
iS

never round up by more than

(vi )
iS

vi n |S| n
i S DP alg can take vmax

(1 ) vi n = vmax, vmax iS vi
i S
Original problem assumes no individual item has weight w_n that exceeds weight limit W all by itself
47