Complexity Classes

Reducibility and NP
Completeness
Computational Complexity
Theory
 Computational Complexity Theory
is the study of how much of a given
resource (such as time, space,
parallelism, algebraic operations,
communication) is required to solve
important problems.

2
Classification of Problems
 Q. Which problems will we be able to solve in
practice?
 A working definition. [Cobham 1964, Edmonds
1965, Rabin 1966] Those with polynomial-time
algorithms.

Yes Probably no
Shortest path Longest path
Matching 3D-matching
Min cut Max cut
2-SAT 3-SAT
Planar 4-color Planar 3-color
Bipartite vertex cover Vertex cover

Primality testing Factoring

3
 Tractable Problems
 We have generally studied tractable problems
(solvable in polynomial time).

• Algorithm design patterns. Examples.

– Greedy O(n log n) activity scheduling.
– Divide-and-conquer. O(n log n) merge sort.
– Dynamic programming. O(nm) Longest Common Subsequence.
– Duality. O(n3) Max-Flow Min-Cut.

4
Intractable Problems

 There are other problems that provably require

exponential-time.
 Examples:
 Given a Turing machine, does it halt in at most k steps
on any finite input?
 Given a board position in an n-by-n generalization of
chess, can black guarantee a win?

5
Impossible Problems

 There are other problems that cannot be

solved by any algorithm.

6
The Halting Problem

 The halting problem is a particular decision

problem:
 Given a description of a program and a finite input,
decide whether the program will halt or run forever
on that input.
 A general algorithm to solve the halting
problem for all possible program-input pairs
cannot exist: The halting problem is
undecidable.

7
Complexity Classes

 Bad news. Huge number of fundamental

problems have defied classification for
decades.

 Some good news. Using the technique of

reduction, we can show that these
fundamental problems are "computationally
equivalent" and appear to be different
manifestations of one really hard problem.
8
Different versions of problems

Focus on
 Decision problems decision problems
 Answer is yes or no
 Optimization problems
 Answer is a number
 Construction problems
 Answer is some object (set of vertices, function,
…)

9
Decision problems versus
optimization problems
 Finding the maximum sized clique is an
optimization problem
 But we can reduce it to a series of decision
problems
 Can we find a clique of size 3 (why start at 3??)
 Can we find a clique of size 4
 Etc
 In general in our study of complexity classes,
we will focus on decision problems
P Vs NP
Complexity Class P

FORMAL INFORMAL
 Class of languages  Class of decision
L, for which there problems that
exists a have polynomial
deterministic
Turing Machine time algorithms
deciding whether i solving them
 L, using running
time O(p(|i|)) for
some polynomial p

12
Complexity Class P

 An algorithm solves a problem

 Time: deterministic, worst case
 Algorithm uses polynomial time, if
there is a polynomial p such that on
inputs of length n the algorithm uses
at most p(n) time.
 Size of input x is denoted |x|.
 P is the class of concrete decision
problems that have an algorithm that
solves it, and that uses polynomial
time

13
Verification algorithm

 Verification algorithm has two arguments:

 Problem input
 Certificate (“solution”)
 Answers “yes” or “no”
 Checks if 2nd argument is certificate for
first argument for studied problem
 The language verified by the verification
algorithm A is
 {i | there is a c with A(i,c)= true}

14
Complexity Class NP

Two equivalent definitions of NP

 Class of languages L, for which there
exists a Non-Deterministic Turing
Machine deciding whether i  L+,
using running time O(p(|i|))
 Class of languages L, for which there
exists a Deterministic Turing Machine
verifying whether i  L+, using a
polynomial sized certificate c, and
using running time O(p(|i|))

15
Complexity Class NP
 Problems with polynomial time verification
algorithm and polynomial size certificates
 Problem L belongs to the class NP, if there
exists a 2-argument algorithm A, with
 A runs in polynomial time
 There is a constant d such that for each x, there is a
certificate y with
 A(i,c) = true
 |c| = O(|i|d)

16
Many problems are in NP
 Examples: Hamiltonian Path, Maximum
Independent Set, Satisfiability, Vertex
Cover, …
 Al of these have trivial certificates (set of
vertices, truth assignment, …)

17
P  NP

 If A decides L in polynomial time, then as

verification algorithm, compute
 B(i,c) = A(i)
 “We do not need a certificate”.
 Famous open problem: P = NP ?? Or not??

18
Reducibility
Reducibility

 Language L1 is polynomial time reducible

to language L2 (or: L1 P L2 ), if there
exists a polynomial time computable
function f : {0,1}*  {0,1}* such that
 For all x  {0,1}*:
x  L1 if and only if f(x)  L2

20
Lemma

Lemma: If L1 P L2 then if L2  P, then L1  P.

Proof-idea: run an algorithm for L2 on f(i) for
input i to problem L1.
Also: If L1 P L2 then if L2  NP, then L1  NP.

21
K-COLORING
 A k-coloring of a graph is an assignment of
one color to each vertex such that:
 No more than k colors are used
 No two adjacent vertices receive the same
color

 A graph is called k-colorable iff it has a k-

coloring

22
Course Scheduling Problem

Given the courses students want to take

and the time slots available, schedule
courses to minimize number of conflicts
(Avoid scheduling two courses at the
same time if a student wants to take
both).

23
K-Coloring Problem:

Colour each node.

Nodes with lines between them
must have different colours.

 Given a graph G and a k,

find a way to colour G with k colours.

24
 Two Different Problems

Schedule each course. Colour each node.

Courses that conflict
can’t be at same time.
 Nodes with lines
between them must
have different colours.

Two problems that are cosmetically different,

but substantially the same
25
 Problems are the Same!

Schedule each course. Colour each node.

Courses that conflict can’t Nodes with lines between
be at same time.  them must have different
colours.
course node
can’t be scheduled
 line between them
at same time
scheduled time colour

Science Math
1pm Mon 3pm Fri

1pm Mon English 26

A coloring Question!
 Is Graph 2-colorable?

No: it contains a triangle! 27

A Coloring Question!
 Is Graph 3-colorable?

Yes!
28
2 Colorable
Given a graph G, how do we
decide if it can be 2-colored?

PERSPIRATION; BRUTE FORCE:

Try out all 2n ways of 2 coloring G.
29
3 Colorable
Given a graph G, what is a fast
algorithm to decide if it can be
3-colored?

? ? ? ? ? ?? ?
30
Let’s consider a completely different
problem.
k-CLIQUES
 A k-clique is a set of k nodes with all
k(k-1)/2 possible edges between them.

32
This graph contains a 4-clique

33
Given an n-node graph G and a
number k, how can you decide if G
contains a k-clique?

 PERSPIRATION: Try out all n-choose-k

possible locations for the k clique
n n!
   possibilities e.g., k  3  (n 3 )
 k  k !(n  k )! In general, (n k )
 INSPIRATION:

? ? ? ? ? ?? ?
34
OK, how about a slightly different
problem?
INDEPENDENT SET

 An independent set is a set of

vertices with no edges between
them.

This graph
contains an
independent
set of size 3.

36
Given an n-node graph G and a number
k, how can you decide if G contains an
independent set of size k?

 PERSPIRATION: Try out all n-choose-k

possible locations for independent set

 INSPIRATION:

? ? ? ? ? ?? ?
37
One more completely different problem
Combinational Circuits

 AND, OR, NOT, gates wired together

with no feedback allowed (acyclic).

39
Logic Gates

Not And Or

40
Example Circuit

41
CIRCUIT-SATISFIABILITY
(decision version)

 Given a circuit with n-inputs

and one output, is there a way
to assign 0-1 values to the
input wires so that the output
value is 1 (true)?

42
CIRCUIT-SATISFIABILITY
(search version)

 Givena circuit with n-inputs

and one output, find an
assignment of 0-1 values to the
input wires so that the output
value is 1 (true), or determine
that no such assignment exists.

43
Satisfiable Circuit Example

44
Satisfiable?

No!

45
Given a circuit, is it
satisfiable?
 PERSPIRATION: Try out all 2n
assignments

 INSPIRATION:

? ? ? ? ? ?? ?
46
We have seen 4 problems:
coloring, clique,
independent set, and
circuit SAT.

They all have a common

story: A large space of
possibilities only a tiny
fraction of which satisfy
the constraints. Brute
force takes too long, and
no feasible algorithm is
known. 47
CLIQUE / INDEPENDENT SET

Two problems that are

cosmetically different,
but substantially the
same

48
Complement Of G

 Givena graph G, let G*, the

complement of G, be the graph
obtained by the rule that two
nodes in G* are connected if
and only if the corresponding
nodes of G are not connected

49
Example

G G*

3-Clique
Independent set of size 3

50
Reduction

 Suppose you have a method for solving the

k-clique problem.
 How could it be used to solve the
independent set problem?

51
Or what if you have an
Oracle?
 or·a·cle

 1 a : a person (as a priestess of ancient

Greece) through whom a deity is believed
to speak

 2 a : a person giving wise or authoritative

decisions or opinions

52
 Let G be an n-node graph.

<G,k>

<G*, k>

BUILD:
Indep. GIVEN:
Set Clique
Oracle Oracle
COSC 3101B, PROF. J. ELDER
53
 Let G be an n-node graph.

<G,k>

<G*, k>

BUILD:
GIVEN:
Clique
Indep.
Oracle
COSC 3101B, PROF. J. ELDER
54 Set
Oracle
 Thus, we can quickly
reduce clique problem
to an independent set
problem and vice versa.

There is a fast method

for one if and only if
there is a fast method
for the other.
55
 NP-completeness and
the Cook-Levin theorem
NP-completeness

A language L is NP-complete, if
1. L  NP
2. For every L’ NP: L’ P L
A language L is NP-hard, if
1. For every L’ NP: L’ P L
 NP-hardness sometimes also used as term for
problems that are not a decision problem, and
for problems that are ‘harder than NP’

57
What does it mean to be
NP-complete?
 Evidence that it is (very probably) hard to
find an algorithm that solves the problem
 Always
 Exact
 In polynomial time

58
CNF-Satisfiability

 Given: expression over Boolean variables in

conjunctive normal form
 Question: Is the expression satisfiable? (Can we
give each variable a value true or false such
that the expression becomes true).
 CNF: “and” of clauses; each clause “or” of variables
or negations (xi or not(xj))

59
Cook-Levin theorem

 Satisfiability is NP-complete
 Most well known is Cook’s proof, using Turing
machine characterization of NP.

60
Proving that problems are
NP-complete
Proving problems NP-
complete
Lemma
1. Let L’ P L and let L’ be NP-complete. Then L is
NP-hard.
2. Let L’ P L and let L’ be NP-complete, and L  NP.
Then L is NP-complete.

62
3-Sat
 3-Sat is CNF-Satisfiability, but each clause has
exactly three literals
 Lemma: CNF-Satisfiability P 3-Sat
 Clauses with one or two literals:
 Use two extra variables p and q
 Replace 2-literal clause (x or y) by (x or y or p) and (x or
y or not(p))
 Similarly, replace 1-literal clause by 4 clauses
 Clauses with more than three literals:
 Repeat until no such clauses
 For (l1 or l2 or … lr) add new variable t and take as
replacement clauses (l1 or l2 or t) and (not(t) or l3 or … or
lr)

63
3-Sat is NP-complete

 Membership in NP
 Reduction
 3-Sat is important starting problem for many NP-
completeness proofs

64
Clique

 Given: graph G=(V,E), integer k

 Question: does G have a clique with at least k vertices?
Clique is NP-complete.
In NP … easy!
NP-hardness: using 3-sat.

65
Reduction for Clique

Clause: {x1,not(x2),x3}
 One vertex per
x1 not(x2) x3
literal per clause
 Edges between
vertices in different x1
clauses, except …
edges between xi x2
and not(xi) not(x3)
 If m clauses, look
Clause: {x1,x2,not(x3)}
for clique of size m
66
Correctness

 There is a satisfying truth assignment, if and

only if there is a clique with m vertices
 =>: Select from each clause the true literal.
The corresponding vertices form a clique with
m vertices.
 <=: Set variable xi to true, if a vertex
representing xi is in the clique, otherwise set
it to false. This is a satisfying truth
assignment:
 The clique must contain one vertex from each 3
vertices representing a clause.
 It cannot contain a vertex representing xi and a
vertex representing not(xi).

67
Independent set

 Independent set: set of vertices W  V, such that for all

v,w  W: {v,w}  E.
 Independent set problem:
 Given: graph G, integer k
 Question: Does G have an independent set of size at least
k?
 Independent set is NP-complete

68
Independent set is NP-
complete

 In NP.
 NP-hard: transform from
Clique.
 W is a clique in G, if and
only if W is an independent
set in the complement of G
(there is an edge in Gc iff. there is no edge in G).

69
How do I write down this
proof?
 Theorem. Independent Set is NP-complete.
 Proof: The problem belongs to NP: as
certificates, we use sets of vertices; we can
check in polynomial time for a set that it is a
clique, and that its size is at least k.
To show NP-hardness, we use a reduction
from Clique. Let (G,k) be an input to the
clique problem. Transform this to (Gc,k) with
Gc the complement of G. As G has a clique
with at least k vertices, if and only if Gc has
an independent set with k vertices, this is a
correct transformation. The transformation
can clearly be carried out in polynomial time.
QED

70
Vertex Cover

 Set of vertices W  V with for all {x,y}  E: x  W or y 

W.
 Vertex Cover problem:
 Given G, find vertex cover of minimum size

= vertex cover

71
Vertex cover is NP-complete

 In NP.
 NP-hard: transform from independent set.
 W is a vertex cover in G, if and only if V-W is an
independent set in G.

72
P=NP?

 Is there an NP-complete problem that can

be solved in polynomial time?
 If Yes (P=NP)

73