Approximation Algorithms

P, NP, NP-Complete, NP-Hard

Need of approximation algorithms
TSP
Introduction
All the algorithms studied so far have been
polynomial-time algorithms on inputs of size n.
The worst-case running time is O(nk) for some
constant k.

That mean all problems can be solved in

polynomial time. NO.
Introduction
There are problems that
Can be solved, but not in time O(nk) for any
constant k.
Cannot be solved by any computer. (Turing's
famous "Halting Problem")
Problems solvable by polynomial-time
algorithms are tractable, or easy.
Problems that require superpolynomial time
are intractable, or hard.
Decision vs. Optimization Problems
Many problems of interest are optimization
problems.
Each feasible solution has an associated value, and
objective is to find a feasible solution with the best
value.
Example, a SHORTEST-PATH problem states that given
an undirected graph G and vertices u and v, find a
path from u to v with the fewest edges.
Npcompleteness is not directly applicable to such
problems.
Contd
NP completeness applies directly and also
confined to decision problems
The answer is simply "yes (1)" or "no (0)".
An optimization problem can be casted as a
related decision problem by imposing a bound on
the value to be optimized.
For example, a decision problem related to SHORTEST-
PATH is PATH: given a directed graph G, vertices u and
v, and an integer k, does a path exist from u to v
consisting of at most k edges?
Contd
If an optimization problem is easy, its related
decision problem is easy as well.
Talking in relevance to NP-completeness, if we
can provide evidence that a decision problem is
hard, we also provide evidence that its related
optimization problem is hard.
Thus, even though it restricts attention to
decision problems, the theory of NP-
completeness often has implications for
optimization problems as well.
The Class P
The complexity class P is the set of all decision
problems that can be solved with worst-case
polynomial time-complexity.
More specifically, they are problems that can
be solved in time O(nk) for some constant k,
where n is the size of the input to the
problem.
P is just the set of tractable decision problems.
Example
Determining if a graph can be colored with 2
colors, i.e. determining whether or not the
graph is bipartite?

Find the shortest path from a single source in

a directed graph.
The Class NP
NP is not the same as non-polynomial complexity /
running time.
NP does not stand for not polynomial.
NP = Non-Deterministic polynomial time
NP means verifiable in polynomial time
Verifiable?
If we are somehow given a certificate of a solution
we can verify the legitimacy (correct or incorrect) in
polynomial time in the size of the input to the
problem.
All P problems are in NP, i.e. P NP
Example
Hamiltonian cycle problem: A cycle passing
through all the vertices of a graph is called a
Hamiltonian cycle.
Given a directed graph G = (V,E), a certificate would be
a sequence <v1, v2, v3, v|V|> of |V| vertices.
It can easily be checked in polynomial time that (vi, vi +
1) E for i = 1, 2, 3, |V| - 1 and that (v|V|, v1) E as
well.
As another example, for 3-CNF satisfiability, a
certificate would be an assignment of values to
variables. We could check in polynomial time that
this assignment satisfies the boolean formula.
P is a subset of NP
Since it takes polynomial time to run the
program, just run the program and get a
solution.
But is P a proper subset of NP?
AN OPEN QUESTION
No one knows if P = NP or not.
NP - hard
What are the hardest problems in NP?

That notation means that L1 is reducible in

polynomial time to L2 .
The less than symbol basically means that the
time taken to solve L1 is not more than a
polynomial factor harder than L2.
The Class NP-hard
A problem is said to NP-hard if every problem
in NP can be reduced to it in polynomial time.

Here L is NP-hard.
Example
The halting problem.
This is the problem that given a program P and
input I, will it halt?
This is a decision problem but it is not in NP.
Any NP-complete problem can be reduced to
this one.
The Class NP-complete
If a problem is in NP and also in NP-Hard, then it is
an NP complete problem.
If we know a single problem in NP-Complete that
helps when we are asked to prove some other
problem is NP-Complete
Assume problem P is NP Complete, i.e. all NP
problems are reducible to this problem P.
Now given a different problem P, if we show P
reducible to P, then by transitivity all NP problems
are reducible to P.
The "first" NP-complete Problem
For the technique of reduction, it is required
to have a problem already known to be NP-
complete.
The problem is the circuit-satisfiability
problem.
Given a boolean combinational circuit composed
of AND, OR, and NOT gates, determine whether
there exists some set of boolean inputs to this
circuit that causes its output to be 1.
Important Note
If any NP-complete problem is polynomial-time
solvable, then P = NP. Equivalently, if any problem
in NP is not polynomial-time solvable, then no
NP-complete problem is polynomial-time
solvable.
No polynomial-time algorithm has yet been
discovered for an NP-complete problem, nor has
anyone yet been able to prove that no
polynomial-time algorithm can exist for any one
of them.
Euler diagram
Approximation Algorithms
Many problems of practical significance are NP-
complete and can be solved in following ways:
1. If the actual inputs are small, an algorithm
with exponential running time may be
perfectly satisfactory.
2. Identify important special cases that can be
solved in polynomial time.
3. Approximation Algorithm
Algorithms that runs in polynomial time and always
produce a solution close to the optimal.
 Performance Ratios
An algorithm for a problem has an approximation ratio of
(n) if, for any input of size n, the cost C of the solution
produced by the algorithm is within a factor of (n) of the
cost C* of an optimal solution:
C C *
max * , (n)
C C
If an algorithm achieves an approximation ratio of (n), it
is called as a (n)-approximation algorithm.
(n) 1.
1-approximation algorithm produces an optimal solution.
Contd
Definitions of approximation ratio and (n)-
approximation algorithm can be applied to both
minimization and maximization problems.
For a maximization problem,
0 < C C*, and the ratio C*/C gives the factor by which
the cost of an optimal solution is larger than the cost
of the approximate solution.
For a minimization problem,
0 < C* C, and the ratio C/C* gives the factor by which
the cost of the approximate solution is larger than the
cost of an optimal solution.
The Traveling Salesman Problem
Given a complete undirected graph G = (V, E)
with a nonnegative integer cost c(u,v) associated
with each edge (u,v) E, find a hamiltonian cycle
(a tour) of G with minimum cost.
Consider two cases:
with and without triangle inequality.
c satisfies triangle inequality, if for all vertices u, v, w
V, c(u,w) c(u,v) + c(v,w)
Finding an optimal solution is NP-complete in
both cases.
TSP with Triangle Inequality
Compute a minimum spanning tree, whose
weight gives a lower bound on the length of
an optimal traveling-salesman tour.

Use the minimum spanning tree to create a

tour whose cost is no more than twice that of
the minimum spanning trees weight, as long
as the cost function satisfies the triangle
inequality.
APPROX-TSP-TOUR(G,c)
1. Select a vertex r G.V to be a "root" vertex.
2. Compute a minimum spanning tree T for G
from root r using MST-PRIM (G, c, r).
3. Let H be a list of vertices, ordered according
to when they are first visited in a preorder
tree walk of T.
4. Return the hamiltonian cycle H.
Example

1. A complete undirected graph. Vertices

lie on intersections of integer grid lines. 2. A minimum spanning tree T of the
For example, f is one unit to the right complete graph, as computed by MST-
and two units up from h. The cost PRIM. Vertex a is the root vertex. Only
function between two points is the
ordinary euclidean distance. edges in the minimum spanning tree are
shown. The vertices happen to be
labeled in such a way that they are
added to the main tree by MST-PRIM in
alphabetical order.
 Contd

3. A walk of T, starting at a. A full walk of

the tree visits the vertices in the order
4. A tour obtained by visiting the vertices
a b c b h b a d e f e g e d a. A preorder
in the order given by the preorder
walk of T lists a vertex just when it is
walk, which is the tour H returned by
first encountered, as indicated by the
APPROX-TSP-TOUR. Its total cost is
dot next to each vertex, yielding the
approximately 19.074.
ordering a b c h d e f g.
 It is known that APPROX-TSP-TOUR is a
polynomial-time 2-approximation algorithm, i.e.
Contd 19.074 <= 2 * 14.715. The relation is clearly
maintained for the considered example.

Tour H obtained An optimal tour H* for

using APPROX-TSP- the original complete
TOUR. Cost = 19.074 graph. Cost = 14.715
Theorem:
APPROX-TSP-TOUR is a polynomial-time 2-
approximation algorithm for the traveling-
salesman problem with the triangle
inequality.
Polynomial running time obvious, simple
MSTPrim takes (|V|2 ), computing pre-order
walk takes no longer.
Correctness obvious, pre-order walk is always
a tour.
 Contd
Let H denotes an optimal tour for given set of
vertices.
Deleting any edge from H gives a spanning tree.
Thus, weight of minimum spanning tree (T) is lower
bound on cost of optimal tour:
c(T) c(H)
A full walk of T lists vertices when they are first
visited, and also when they are returned to, after
visiting a subtree.
The full walk (W) of our example gives the order
a, b, c, b, h, b, a, d, e, f, e, g, e, d, a
Contd

a, b, c, b, h, b, a, d, e, f, e, g, e, d, a
Contd
Full walk W traverses every edge exactly twice, thus
c(W) = 2c(T).
Together with c(T) c(H), this gives
c(W) = 2c(T) 2c(H)
Find a connection between cost of W and cost of "our"
tour.
Problem: W is in general not a proper tour, since vertices
may be visited more than once.
But: using the triangle inequality, we can delete a visit to
any vertex from W and cost does not increase.
Deleting a vertex v from walk W between visits to u and w
means going from u directly to w, without visiting v.
Contd
We can consecutively
remove all multiple
visits to any vertex.

Example:
full walk a,b,c,b,h,b,a,d,e,f,e,g,e,d,a
becomes a,b,c,h,d,e,f,g.
Contd
This ordering (with multiple visits deleted) is
identical to that obtained by pre-order walk of T
(with each vertex visited only once).
It certainly is a Hamiltonian cycle. Lets call it H.
H is just what is computed by Approx-TSP-Tour.
H is obtained by deleting vertices from W , thus
c(H) c(W )
Conclusion:
c(H) c(W) 2c(H)
Contd
Although factor 2 looks nice, there are better
algorithms.
Theres a 3/2 approximation algorithm by
Christofedes (with triangle inequality).
In The general TSP Cost function c does not
satisfy triangle inequality.
Theorem:
If P NP, then for any constant 1, there is no
polynomial time approximation algorithm with
approximation ratio for the general traveling-
salesman problem.
Example
Compute a tour for a given graph satisfying
the triangle inequality. d

Solution:
15
10
c 20
e
Get MST using prims, 25
35
22
let root be a. 40 10 25

Get preorder traversal to b

30
a
order vertices in the tour.
Compute cost.
 Contd
d d d
15 15 15
10 10 10
c 20
c 20
c 20

25
e 25
e 25
e
35 35 35
22 22 22
40 10 25 40 10 25 40 10 25

30 30 30
b b b
a a a
A B C

d d
a
15 15
10 10
c 20
c 20
10
25
e c 25
e
35 35
22 22
40 10 25 15 25 40 10 25
d b
30 10 30
b b
a e a
D E: MST F: Obtained tour
 Contd
Minimum spanning tree cost = 10 + 15 + 10 + 25 = 60
Full walk of MST starting from a: a c d e d c b c a.
Cost of full walk = 2 * 60 = 120. OR
ac + cd + de + ed + dc + cb + bc + ca
= 10 + 15 + 10 + 10 + 15 + 25 + 25 + 10 = 120
Preorder walk starting from a: a c d e b.
Cost of preorder walk = ac + cd + de + eb + ba
= 10 + 15 + 10 + 40 + 30
= 105 < 120. Hence proved.