Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Com c3&4

Download as docx, pdf, or txt
Download as docx, pdf, or txt
You are on page 1of 43

Chapter three

1. Computability
After the completion of this chapter students will be able to:
 Define what recursive is and recursively enumerable languages are.
 Differentiate recursively enumerable and non-enumerable languages
 How to represent machines as a string.
 Know what language enumerators are.
What is Recursive?
Definition: A language L is recursive if there exists a TM M such that L=L(M) and M halts on
every w2 Σ+.
1.1. Recursive and Recursively Enumerable Languages

Remember that there are three possible outcomes of executing a Turing machine over a
given input. The Turing machine may

 Halt and accept the input;

 Halt and reject the input; or

 Never halt.

A language is recursive if there exists a Turing machine that accepts every string of the
language and rejects every string (over the same alphabet) that is not in the language.

Note that, if a language L is recursive, then its complement must also be recursive.

A language is recursively enumerable if there exists a Turing machine that accepts every
string of the language, and does not accept strings that are not in the language. (Strings
that are not in the language may be rejected or may cause the Turing machine to go into an
infinite loop.)
A language, L is recursively enumerable if it is the language accepted by some Turing
machine M. That is, w € L if and only if M halts on input w in an accepting state. Note that if
we process some word v not element of L with M then M might halt in a non-accepting
state, might crash, or might simply run forever. This situation is rather unsatisfactory – we
know that if a ‘yes’ answer is coming it will arrive eventually, but if the answer is ‘no’ we
may never be sure.
That makes it natural to define a (possibly) stronger notion: a language, L, is recursive if
it is the language accepted by some Turing machine M that halts on all inputs. Now we are
guaranteed a yes or no answer.

Theorem 2.1. If both L and Σ∗ \ L are recursively enumerable, then L is recursive.

Proof. Let Min accept L and Mout accept Σ∗\L. We use these two machines to put together
a multi tape machine. Given an input w we first copy w onto another tape. Then we
run Min on one tape in parallel with Mout on the other (the states of this new machine
correspond to pairs of states of the two given machines, so there are still only finitely
many of them). As soon as one of the two simulations halts, we can halt and announce
the status of w. Since one of the two simulations is guaranteed to halt (Min if w € L and
Mout otherwise) this is sufficient.
1.2. Language-enumerators
There are certainly circumstances in which we might be more interested in making a list of
the elements of a language L rather than recognizing membership in L (for instance, a list
of primes, a list of possible solutions to some optimization problem, . . . ). A machine M is
said to enumerate L if it produces on some output tape (which always moves to the right,
though not necessarily at every computation step) a sequence:

where # is an extra symbol (not from the alphabet of L) and w € L if and only if w = ui for
some i. In other words, the machine just lists the elements of L in some order, and possibly
with repetitions, but every element gets listed eventually.
Proposition 3.1. If there is a TM that enumerates L, then there is a TM that enumerates L
without repetitions.
Proof. Take a TM enumerating L and augment it with another output tape – the ‘real’
output tape. Each time M writes a word uk on its output tape, pause M and use a separate
control module to check whether uk = uj for some j < k. If so, just restart M. If not, write uk
to the ‘real’ output tape (followed by a #) and then restart M.
Of course the main result (and the reason for the name ‘recursively enumerable’) is:
Theorem 3.2. A language L is enumerated by some TM if and only if it is recursively
enumerable.
Proof. Suppose that L is enumerated by M. To build a machine T that accepts M simply add
an extra input tape on which an input word w is written. Then start M. Each time M
produces a new word ui check whether ui = w. If so, halt and accept. If not, carry on.
Thus, if w 2 L we halt and accept, and otherwise we do not.
Now suppose that L is recursively enumerable, accepted by some machine T. To build an
enumerator M we use the ‘simulate for a fixed number of steps on each possible input of a
fixed length’ idea. That is, we build a machine M that successively runs T for k steps on all
input sequences of length at most k for k from 1 to · · · . Each time we spot an acceptance
we add it to the output tape. Thus, we eventually add all the words that T accepts to the
output tape and no others.
The problem with language enumerators (as with recursively enumerable languages) is
that if we haven’t seen a word yet we have no indication whether we ever will. If we could
enumerate the strings of a language in order of their length, then that would be a different
story – once we saw a string longer than the one we were interested in we would know
that waiting any longer would be a forlorn hope.
Say that L can be enumerated by length if there is an enumerator for L which first lists all
the strings of length 0 in L, then all the strings of length 1, then of length 2, and so on. Not
surprisingly
Theorem 3.3. A language L can be enumerated by length if and only if it is recursive.
Proof. If L is recursive, take a machine M that accepts it and halts on all inputs. Run it on
each string of length 0, then each string of length 1 and so on. Each time a string is
accepted, add it to an output tape. The resulting machine enumerates L by length.
Conversely, suppose that L can be enumerated by length. If L is finite then it is regular and
so certainly recursive, and there is nothing to prove. Otherwise, take the enumerator by
length and a desired input word w. Wait until either w or some longer word appears, and
accept or reject accordingly.
1.3. Representing machines as strings
In several of the preceding arguments we have been piecing together new machines from
old in an ad hoc fashion as required. There’s nothing wrong with that, but effectively we’re
treating machines as modules here – and it would seem sensible to perhaps represent
them in some uniform fashion (then we might feel more comfortable about adding certain
operations ‘erase the working tape and write the first string of length 6 on it, then run M
for 6 steps’).
A first step in that process is to come up with a representation of (basic) TMs as strings
– and there’s nothing particularly special about the representation we might choose – in
practice, anything vaguely sensible will do. Since we’re worried more about theoretical
rather than practical arguments we’ll just take a very simple representation.
We’ll use a set alphabet of f0; 1g. The 0 symbol will be used only as punctuation, so we’ll
effectively encode everything else in unary. What do we need to encode?
• An actual alphabet Γ. Call the elements of Γ, a1 through an and encode ai as 1i (a
string of i 1’s).
• A set of states. Call the states q1 through qm and encode qj as 1j
• The directions. Call L, 1 and R, 11.
• The transitions. Transitions can be thought of as 5-tuples (qstart; aread; awrite;
dmove; q finish). So, simply encode these as the encodings of the individual parts
separated from one another by single 0’s.
• The complete set of transitions. Take each transition, encode it, and separate it
from
the next by 00.
• The start/finish of the encoding. Add a 000 at the beginning and at the end.
For convenience, having done all this to a Turing machine M we’ll call the resulting string
the representation of M and denote it by R(M).
1.4. Recursively Enumerable Languages
Definition: A language L is recursively enumerable if there exists a TM M such
that L=L(M).

Recursive Languages Clearly, every recursive language is also recursively enumerable. It is


not obvious whether every recursively enumerable language is also recursive.
Examples
 The set of C program-input pairs that do not run into an infinite loop is recursively
enumerable.
Just run its binary code in a simulator environment.
 The set of C programs that contain an infinite loop is not recursively enumerable
1.4.1. Recursively Enumerable Languages: Examples
• The set of C program-input pairs that do not run into an infinite loop is recursively
enumerable.
Just run its binary code in a simulator environment.
• The set of C programs that contain an infinite loop is not recursively enumerable
Turing-Computable Functions
• Let f : (Σ - {⊔})∗ → Σ∗.
Optimization problems, root finding problems, etc.
• Let M be a TM with alphabet Σ.
• M computes f if for any string x ∈ (Σ - {⊔})∗,
M(x) = f(x).
• We call f a recursive function if such an M exists.
Enumerating Strings in a Language
To enumerate a set is to place the elements of the set in a one-to-one correspondence with
the natural numbers.
The set of all strings over an alphabet is denumerable (think of a string as a number in an |
∑|-ary number system). The strings in a language form a subset of the set of all strings
over . But can we enumerate the strings in a language?
If a language is recursive, then there exists a Turing machine for it that is guaranteed to
halt.
We can generate the strings of * in a shortest-first order (to guarantee that every finite
string will be generated), test the string with the Turing machine, and if the Turing
machine accepts the string, assign that string the next available natural number.
We can also enumerate the recursively enumerable languages. We have a Turing machine
that will halt and accept any string that belongs to the language; the trick is to avoid
getting hung up on strings that cause the Turing machine to go into an infinite loop. We do
this by "time sharing." Here's how:
1.5. Non-Recursively Enumerable Languages

A language is a subset of *. A language is any subset of *.


We have shown that Turing machines are enumerable. Since recursively enumerable
languages are those whose strings are accepted by a Turing machine, the set of recursively
enumerable languages is also enumerable.
We have shown that the power set of an infinite set is not enumerable -- that it has more
than 0 subsets. Each of these subsets represents a language. Therefore, there must be
languages that are not computable by a Turing machine.
According to Turing's thesis, a Turing machine can compute any effective procedure.
Therefore, there are languages that cannot be defined by any effective procedure. We can
find a non-recursively enumerable language X by diagonalization. Using the enumerations
described earlier, let string i belong to language X if and only if it does not belong to
language i.
When Recursively Enumerable Implies Recursive?
Suppose a language L is recursively enumerable. That means there exists a Turing
machine T1 that, given any string of the language, halts and accepts that string. (We don't
know what it will do for strings not in the language -- it could reject them, or it could
simply never halt.)

Now let's also suppose that the complement of L, -L = {w: w L}, is recursively enumerable.
That means there is some other Turing machine T2 that, given any string of -L, halts and
accepts that string.
Clearly, any string (over the appropriate alphabet) belongs to either L or -L. Hence, any
string will cause either T1 or T2 (or both) to halt. We construct a new Turing machine that
emulates both T1 and T2, alternating moves between them. When either one stops, we can
tell (by whether it accepted or rejected the string) to which language the string belongs.
Thus, we have constructed a Turing machine that, for each input, halts with an answer
whether or not the string belongs to L. Therefore L and -L are recursive languages.
Recursively Enumerable But Not Recursive
We know that the recursive languages are a subset of the recursively enumerable
languages, We will now show that they are a proper subset.
We have shown how to enumerate strings for a given alphabet, w1, w2, w3, .... We have
also shown how to enumerate Turing machines, T1, T2, T3, .... (Recall that each Turing
machine defines a recursively enumerable language.) Consider the language
L = {wi: wi L(Ti)}
A little thought will show that L is itself recursively enumerable. But now consider its
complement:
-L = {wi: wi L(Ti)}
If -L is recursively enumerable, then there must exist a Turing machine that recognizes it.
This Turing machine must be in the enumeration somewhere -- call it Tk.
Does wk belong to L?
 If wk belongs to L then (by the way we have defined L) Tk accepts this string. But Tk
accepts only strings that do not belong to L, so we have a contradiction.
 If wk does not belong to L, then it belongs to -L and is accepted by Tk. But since Tk
accepts wk, wk must belong to L. Again, a contradiction.
We have now defined a recursively enumerable language L and shown by contradiction
that - L is not recursively enumerable.

We mentioned earlier that if a language is recursive, its complement must also be


recursive. If language L above were recursive, then -L would also be recursive, hence
recursively enumerable. But -L is not recursively enumerable; there fore L must not be
recursive.
We have therefore shown that L is recursively enumerable but not recursive, therefore the
set of recursive languages is a proper subset of the set of recursively enumerable
languages.
3.6. Chapter Summary
1. A language L is said to be recursively enumerable (RE), if some Turing Machine accepts
the language L.
2. A language L is said to be recursive (decidable), if some Turing Machine accepts the
language L, and Halts on all the inputs.
3. A function f is said to be partial recursive, if some Turing Machine computes the function
(it halts on all the inputs on which f is defined, and it does not halt on inputs on which f is
not defined).
4. A function f is said to be recursive, if some Turing Machine computes the function, and f
is defined on all elements of Σ∗.
CHAPTER FOUR
2. Computational Complexity
2.1. What-is-computation?
2.2. computation is the process by which to produce an answer to a question: To
ensure that all parts of this process (the question, the computation, and the
answer) are finite, it must be possible to present the question in finite time, to do
a computation in finite time, and to read the answer in finite time. Using human
standards, all questions that can be presented in finite time must be pairwise
distinguishable by a human in finite time. Or in other words, a human must be
able to determine in finite time whether two questions are different

or the same. Since the limits to human cognition are finite, it can be argued that the set of
all possible questions consists of finite, enumerable elements. The same holds for the
answers.
Furthermore, it can be shown that any infinite set of enumerable elements is equal to the
set of natural numbers (in a sense that there is a bijection between the set of enumerable
elements and the set of natural numbers). Thus, a computation can be viewed as a
transformation from a natural number q (the question) to a natural number a (the answer).
Computation always requires a computer", i.e., a system that performs the computation.
A configuration of a computer is a complete description of the state of such a system at
some time point. The initial configuration of the computer is the question plus the initial
part of its state that is independent of the question (this usually consists of the
computational rules or algorithms the computer will apply in order to answer the
question). The final configuration of the computer contains the answer. Note that a
computer may not necessarily always reach a final configuration (that is, there may be
questions for which it runs forever). Mathematically, a computer can be viewed as a
(partial) mapping that maps the set of questions to the set of answers, i.e. a function of the
form (or, using our arguments above, simply
A fundamental problem has been: is there a “universal" model for computation? Or in
other words, is there a model of a computer that ensures that for every function f that is
“computable" there is a computer with an initial configuration (independent of the
question) so that for every q 2 Q it outputs f(q) in finite time (if f(q) exists)? The main di–
culty in answering this problem is that we simply do not know how powerful computation
can be.
Obviously, any realizable computer must be a finite physical system. The operation of any
such
system is limited by the physical laws. However, as long as we do not have a complete
picture of
the physical laws, we do not know what kind of computations, or changes in state, are
possible.
All computational models developed so far (including Quantum computing models) have
been shown to be computationally equal to a very simple model, the so-called Turing
machine.
This lead Church and Turing in 1936 to the conjecture that the limitations on what can be
computed do not depend on the understanding of physics, but are universal. This is called
the
Church-Turing hypothesis:
Every computable function can be computed by a Turing machine.
One can formulate a more restrictive version of this hypothesis:
Every e–ciently computable function can be e–ciently computed by a Turing machine.
Whereas this was shown to be true for all classical computational models, it is most likely
not true for the Quantum computational models, since there is strong evidence that
Quantum
effects cannot be simulated e–ciently by a (classical) Turing machine. However, since we
are still far away from building a Quantum computer, the hypothesis still holds for all
existing systems.
Definition 1 Big-O Notation

Introduction

How efficient is an algorithm or piece of code? Efficiency covers lots of resources,


including:

 CPU (time) usage


 Memory usage
 Disk usage
 Network usage

All are important but we will mostly talk about time complexity (CPU usage).

Be careful to differentiate between:

1. Performance: how much time/memory/disk/... is actually used when a


program is run.

This depends on the machine, compiler, etc. as well as the code.

2. Complexity: how do the resource requirements of a program or algorithm


scale, i.e.,

What happens as the size of the problem being solved gets larger?

Complexity affects performance but not the other way around.

The time required by a function/procedure is proportional to the number of "basic


operations"

that it performs. Here are some examples of basic operations:

One arithmetic operation (e.g., +, *).

one assignment (e.g. x := 0)

one test (e.g., x = 0)

one read (of a primitive type: integer, float, character, Boolean)


one write (of a primitive type: integer, float, character, Boolean)

Some functions/procedures perform the same number of operations every time


they are

called. For example,

StackSize in the Stack implementation always returns the number of elements


currently in the stack or states that the stack is empty, then we say that StackSize
takes constant time.

Other functions/ procedures may perform different numbers of operations, depending on


the value of a parameter. For example, in the BubbleSort algorithm, the number of elements
in the array, determines the number of operations performed by the algorithm. This
parameter (number of elements) is called the problem size/ input size.

When we are trying to find the complexity of the function/ procedure/ algorithm/
program, we are not interested in the exact number of operations that are being
performed. Instead, we are interested in the relation of the number of operations to the
problem size.

Typically, we are usually interested in the worst case: what is the maximum number of
operations that might be performed for a given problem size. For example, inserting an
element into an array, we have to move the current element and all of the elements that
come after it one place to the right in the array. In the worst case, inserting at the beginning
of the array, all of the elements in the array must be moved. Therefore, in the worst case,
the time for insertion is proportional to the number of elements in the array, and we say
that the worst- case time for the insertion operation is linear in the number of elements in
the array. For a linear-time algorithm, if the problem size doubles, the number of
operations also doubles.

Definition 2 Big-O Notation


We express complexity using big-O notation.
For a problem of size N:

 a constant-time algorithm is "order 1": O(1)


 a linear-time algorithm is "order N": O(N)
 A quadratic time algorithm order N square O(N)2

Note that the big-O expressions do not have constants or low-order terms. This is because,
when N gets large enough, constants and low-order terms don't matter (a constant-time
algorithm will be faster than a linear-time algorithm, which will be faster than a quadratic-
time algorithm).

A function T(N) is O(F(N)) if for some constant c and for values of N greater than some

value n0:

T(N) <= c * F(N)

The idea is that T(N) is exact complexity of the procedure/function algorithm as a function
of the problem size N, and that F(N) is an upper bound on that complexity(i.e. the actual
time/space or whatever for a problem of size N will be no worse than F(N)).

In practice, we want the smallest F(N) -- the least upper bound on the actual complexity.

How to determine Complexities?

In general, how can you determine the running time of a piece of code? The answer is that
it depends on what kinds of statements are used.

Sequence of statements
statement 1;

statement 2;

...

statement k;

The total time is found by adding the times for all statements:
Total time = time (statement 1) + time (statement 2) + ... + time (statement k)

If each statement is "simple" (only involves basic operations) then the time
for each statement is constant and the total time is also constant: O(1).

If-Then-Else

if (condition)
{
----------
----------
}
else if (condition)
{
----------
----------
}
else if (condition)
{
----------
----------
}
else
{
----------
----------
}
Example 1

#include<iostream.h>

void main()
{
int num;

cout << "Enter any number : ";


cin >> num;

if (num>0)
cout << num <<" is Positive.";
else
cout << num <<" is Negative.";
}

Output :

Enter any number : 24


24 is Positive

Example 2

#include<iostream.h>

void main()
{
int amt,discount;
cout << "Enter amount : ";
cin >> amt;

if (amt>20000)
discount=15;

else if (amt>15000)
discount=10;

else if (amt>10000)
discount=5;

else
discount=0;

cout << "You will get " << discount << "% discount.";
}

Output :

Enter amount : 14000


You will get 5% discount.

Here, either block 1 will execute, or block 2 will execute. Therefore, the worst-case time is
the slower of the two possibilities:

max(time(block 1), time(block 2))

If block 1 takes O(1) and block 2 takes O(N), the if-then-else statement would be O(N).

Loops
initialization;

while (condition)
{
----------
----------

inc/dec;
}

Example 1

#include<iostream.h>

void main()
{
int a=1,num;

cout << "Enter any number : ";


cin >> num;

while (a<=num)
{
cout << "\nHello...!!";
a++;
}
}

Output :
Enter any number : 5
Hello...!!
Hello...!!
Hello...!!
Hello...!!
Hello...!!

Syntax of Do-While Loop

Initialization;

do
{
----------
----------

inc/dec;
} while (condition);

Example 2

#include<iostream.h>

void main()
{
int a=1,num;

cout << "Enter any number : ";


cin >> num;

do
{
cout << "\nHello...!!";
a++;
} while (a<=num);
}

Output :

Enter any number : 5


Hello...!!
Hello...!!
Hello...!!
Hello...!!
Hello...!!

The loop executes N times, so the sequence of statements also executes N times. If we
assume the statements are O(1), the total time for the for loop is N * O(1), which is O(N)
overall.

Syntax of For Loop

for (initialization;condition;inc/dec)
{
----------
----------
}

Example of For Loop


#include<iostream.h>

void main()
{
int a,num;

cout << "Enter any number : ";


cin >> num;

for (a=1;a<=num;a++)
cout << "\nHello...!!";
}

Output :

Enter any number : 24


Hello...!!
Hello...!!
Hello...!!
Hello...!!
Hello...!!

Nested loops
…………………………………………………………………..

…………………………………………………………………..
Definition - 3 Big-Oh notation
If f, g are two functions from N to N, then we (1) say that f = O(g) if there exists a constant c
such that f(n) ≤ c · g(n) for every sufficiently large n, (2) say that f = Ω(g) if g = O(f), (3) say
that f = Θ(g) is f = O(g) and g = O(f), (4) say that f = o(g) if for every > 0, f(n) ≤ · g(n) for
every sufficiently large n, and (5) say that f = ω(g) if g = o(f).
To emphasize the input parameter, we often write f(n) = O(g(n)) instead of f = O(g), and use
similar notation for o, Ω, ω, Θ.
Example 1.3
Here are some examples for use of big-Oh notation:
1. If f(n) = 100n log n and g(n) = n2 then we have the relations f = O(g), g = Ω(f), f = o(g), g =
ω(f).
2. If f(n) = 100n2 + 24n + 2logn and g(n) = n2 then f = O(g). We will often write this relation
as f(n) = O(n2). Note that we also have the relation g = O(f) and hence f = Θ(g) and
g = Θ(f).
3. If f(n) = min{n, 106} and g(n) = 1 for every n then f = O(g). We use the notation f = O(1)
to denote this. Similarly, if h is a function that tends to infinity with n (i.e., for every c it
holds that h(n) > c for n sufficiently large) then we write h = ω(1).
4. If f(n) = 2n then for every number c ∈ N, if g(n) = nc then g = o(f). We sometimes write
this as 2n = nω(1). Similarly, we also write h(n) = nO(1) to denote the fact that h is bounded
from above by some polynomial. That is, there exist a number c > 0 such that for sufficiently
large n, h(n) ≤ nc.
Definition-4-Big-O-notation
In complexity theory, one is mostly interested in the “bigger picture”. Therefore, constant
factors
are usually ignored when considering the resource requirements of computations,
although they
are certainly important in practice. For this, the O-notation has been introduced. For any
function
, the set

1) O(f(n)) denotes the set of all functions with the property that
there are

constants c > 0 and n0 > 0 so that for all

2) denotes the set of all functions with the property


that there are constants c > 0 and n0 > 0 so that for all and

3) denotes the set of all functions with the property

that and

For example, and

Also stricter variants of O and are known. For any

function , the set

4) o(f(n)) denotes the set of all functions with the property that

and

5) denotes the set of all functions with the property that

For example, it is also true that

is neither in o(n) nor in


Although it is not correct from a formal point of view, one can often find statements like

in the literature instead of using

the correct relation, but it has been used so often that people just got stuck with it.
2.3. NP and mathematical proofs
By definition, NP is the set of languages where membership has a short certificate. This is
reminiscent of another familiar notion, that of a mathematical proof. As noticed in the past
century, in principle all of mathematics can be axiomatized, so that proofs are merely
formal manipulations of axioms. Thus the correctness of a proof is rather easy to verify —
just check that each line follows from the previous lines by applying the axioms. In fact, for
most known axiomatic systems (e.g., Peano arithmetic or Zermelo-Fraenkel Set Theory)
this verification runs in time polynomial in the length of the proof. Thus the following
problem is in NP for any of the usual axiomatic systems A:
THEOREMS = {(ϕ, 1n) : ϕ has a formal proof of length ≤ n in system A}
In fact, the exercises ask you to prove that this problem is NP-complete. Hence the P versus
NP question is a rephrasing of G¨odel’s question (see quote at the beginning of the chapter),
which asks whether or not there is a algorithm that finds mathematical proofs in time
polynomial in the length of the proof.
Of course, all our students know in their guts that finding correct proofs is far harder than
verifying their correctness. So presumably, they believe at an intuitive level that P is not
equal NP.

2.4. Complexity Classes P and NP

Intuitively, the complexity classes P and NP refer to decision problems that can be solved
efficiently and those for which yes-instances can be verified efficiently, respectively. If we 3
insisted on formal correctness here, we would define these classes in terms of a specific
computer model called Turing machines. However, this is beyond the scope of this course
and we therefore take the freedom to introduce these classes using a more high-level (but
essentially equivalent) point of view.
We define the complexity class P (which stands for polynomial-time).
Definition 5.3. A decision problem Π belongs to the complexity class P if there exists an
algorithm that for every instance I ∈ I determines in polynomial time whether I is a yes
instance or a no-instance.
All problems that we have treated so far in this course belong to this class. But also the
LINEAR PROGRAMMING PROBLEM (LP) belongs to this class, even though the simplex
algorithm is not a polynomial-time algorithm for LP.
The simplex algorithm works almost always very fast in practice for any LP of whatever
size, but as mentioned before the running time of an algorithm is determined by its worst-
case running time. For most pivoting rules devised for the simplex algorithm, there have
been constructed instances on which the algorithm has to visit an exponential number of
basic feasible solutions in order to arrive at an optimal one. A polynomial-time algorithm
for LP is the ellipsoid method. This algorithm is an example where the time bound is
polynomial in the logarithm of the largest coefficient in the instance next to the number of
variables and number of restrictions. One of the most interesting open research questions
in Operations Research is whether there exists an algorithm for LP whose running time is a
polynomial in the number of variables and the number of restrictions only.
Next, we define the complexity class NP. NP does not stand for “non-polynomial-time” as
one might guess, but for “non-deterministic polynomial-time” because this class is formally
defined in terms of non-deterministic Turing machines.
Given a yes-instance I ∈ I of a decision problem Π, we say that S is a certificate for I if S ∈ F
and c(S) ≤ K. Note that every yes-instance I must have a certificate. The specialty of a
problem in NP is that yes-instances admit certificates that can be verified in polynomial
time.
Definition 5.4. A decision problem Π belongs to the complexity class NP if every yes
instance I ∈ I admits a certificate whose validity can be verified in polynomial time.
Note that the polynomial-time verifiability of S implies that the size of S must be
polynomially bounded in |I| (because we need to look at S to verify its validity). That is, the
definition above also states that yes-instances of problems in NP have short, i.e.,
polynomialy bounded, certificates.
We consider some examples:
Example 5.3. HAMILTONIAN CYCLE is in NP: A certificate for a yes-instance corresponds to
a set of edges S ⊆ E. One can verify in O(n) time whether S constitutes a cycle in G that visits
all vertices exactly once.
Example 5.4. Consider the decision variant of LINEAR PROGRAMMING:

2.4.1. LINEAR PROGRAMMING (LP):

Instance: A set F of feasible solutions x = (x1,...,xn) defined by m linear constraints


F = (x1,...,xn) ∈ Rn ≥0 : i∑ = n1aijxi ≥ bj for every j = 1,...,m together with an objective
function c(x) =∑n i=1cixi and a parameter K.
Goal: Determine whether there exists a feasible solution x ∈ F that satisfies c(x) ≤ K.
LP is in NP: A certificate for a yes-instance corresponds to a solution x = (x 1,...,xn). We need
O(n) time to verify each of the m constraints and O(n) time to compute the objective
function value c(x). The total time needed to check whether x ∈ F and c(x) ≤ K is thus
O(nm).
Exercise 5.4. Show that the following problems are in NP: MINIMUM SPANNING TREE,
STEINER TREE, TSP.
Exercise 5.5. Try to identify a problem for which you believe that it is not in NP.

2.5. Polynomial-time Reductions and NP-completeness

After thinking for a little while, we conclude that P ⊆ NP. Several decades of intensive
research seem to suggest that there are problems in NP that are intrinsically more difficult
than the ones in P and thus P = NP: Despite the many research efforts, no polynomial time
algorithms have been found for problems in NP such as TSP, HAMILTONIAN CYCLE,
STEINER TREE, etc.
On the other hand, all attempts to show that these problems are in fact harder than the
ones in P failed as well. The question whether P = NP is one of the biggest mysteries in
mathematics to date and constitutes one of the seven millennium-prize
problems; see http://www.claymath.org/millennium for more information.
Complexity theory attempts to give theoretical evidence to the conjecture that P = NP. It
defines within the complexity class NP a subclass of most difficult problems, the so-called
NP-complete problems. This subclass is defined in such a way that if for any of the NP
complete problems there will ever be found a polynomial-time algorithm then this implies
that for every problem in NP there exists a polynomial-time algorithm, and thus P = NP.
The definition of this class crucially relies on the notion of polynomial-time reductions:
Definition 5.5. A polynomial-time reduction from a decision problem Π1 to a decision
problem Π2 is a function ϕ : I1 → I2 that maps every instance I1 ∈ I1 of Π1 to an instance
I2 = ϕ(I1) ∈ I2 of Π2 such that:
1. the mapping can be done in time that is polynomially bounded in the size of I 1;
2. I1 is a yes-instance of Π1 if and only if I2 is a yes-instance of Π2.
If there exist such a polynomial-time reduction from Π1 to Π2 then we say that Π1 can be
reduced to Π2, and we will write Π1 Π2.
Let us think about some consequences of the above definition in terms of polynomial-time
computability.
Suppose Π1 Π2. Then Π2 is more difficult to solve than Π1 (which also justifies the use of
the symbol).
To see this, note that every polynomial-time algorithm ALG2 for Π2 can be used to derive a
polynomial-time algorithm ALG1 for Π1 as follows:
1. Transform the instance I1 of Π1 to a corresponding instance I2 = ϕ(I1) of Π2.
2. Run ALG2 on I2 and report that I1 is a yes-instance if and only if ALG2 concluded that I2
is a yes-instance.
By the first property of Definition 5.5, the transformation in Step 1 above takes time
polynomial in the size n1 = |I1| of I1. As a consequence, the size n2 = |I2| of I2 is
polynomially bounded in n1. (Think about it!) In Step 2, ALG2 solves I2 in time polynomial
in the size n2 of I2, which is polynomial in the size n1.2 The overall time needed by ALG1 to
output a solution for I1 is thus bounded by a polynomial in n1. Note that the second
property of
Definition 5.5 ensures that ALG1 correctly identifies whether I1 is a yes-instance or not.
Observe the existence of a polynomial-time algorithm for Π1 has in general no implications
for the existence of a polynomial-time algorithm for Π2, even if we assume that we can
compute the inverse of ϕ efficiently. The reason for that is that ϕ is not necessarily a one-
to-one mapping and may thus map the instances of Π1 to a subset of the instances of Π2
which correspond to easy instances of Π2. Thus, being able to efficiently solve every
instance of Π1 reveals nothing about the problem of solving Π2.
It is not hard to show that polynomial-time reductions are transitive:
Lemma 5.1. If Π1 Π2 and Π2 Π3 then Π1 Π3.
We can now define the class of NP-complete problems.
Definition 5.6. A decision problem Π is NP-complete if
1. Π belongs to NP;
2. every problem in NP is polynomial-time reducible to Π.
Intuitively, the above definition states that an NP-complete problem is as difficult as any
other problem in NP. The above definition may not seem very helpful at first sight: How do
we prove that every problem in NP is polynomial-time reducible to the problem Π we are
interested in? Let us assume for the time being that there are some problems that are
known to be NP-complete. In order to prove NP-completeness of Π it is then sufficient to
show that Π is in NP and that some NP-complete problem is polynomial-time reducible to
Π. (Think about it!) That is, showing NP-completeness of a problem becomes much easier
now because we “just” need to find an appropriate NP-complete problem that can be
reduced to it. Nevertheless, we remark that the reductions of many NP-completeness
proofs are highly non-trivial and often require a deep understanding of the structural
properties of the problem.
The class NP has a very precise definition in terms of executions of non-deterministic
Turning machines (which we skipped and persist in skipping), which enabled Steven Cook
in 1974 to prove that any such execution can be reduced to an instance of a famous
problem in Boolean logic called the SATISFIABILITY problem (stated below). Thus, Cook
provided us with a problem that is NP-complete. Starting from this, many other problems
were proven to be NP-complete.

In a way, proving that a problem is NP-complete is a beautiful way of stating:


Exercise 5.6. Indicate for each of the four statements below whether it is true or false. Let
Π1 and Π2 be two problems that are in NP.
1. If Π1 ∈ P and Π1 Π2 then Π2 ∈ P.
2. If Π1 is NP-complete and Π1 Π2 then Π2 is NP-complete.
3. If Π1 ∈ P and Π2 Π1 then Π2 ∈ P.
4. If Π1 is NP-complete and Π2 Π1 then Π2 is NP-complete.
5.5 Examples of NP-completeness Proofs
We introduce some more problems and show that they are NP-complete. However, most of
the reductions are technically involved and will be omitted here because the intention is to
gain some basic understanding of the proof methodology rather than diving into technical
details.
We first introduce the SATISFIABILITY problem for which Cook established NP
completeness. The basic ingredients are variables. A variable reflects an expression which
can be TRUE or FALSE. For example,
x1 = Koen is taller than Michael and x2 = Soup is always eaten with a fork.
A variable can also occur negated. For example, we write ¬x1 to express that Koen is not
taller than Michael. A literal refers to a negated or un negated variable. We compose more
Reductions to proof NP-completeness of the example problems considered here;
proofs are given for the ones marked with (*) complicated expressions, called clauses, from
literals. An example of a clause is
C1 = (x1 ∨ ¬x2 ∨ x3 ∨ x4).
The interpretation is that clause C1 is TRUE if and only if x1 is TRUE or (indicated by ∨)
not-x2 is TRUE or x3 is TRUE or x4 is TRUE. That is, a clause is TRUE if at least one of its
literals is TRUE. An instance of the SAT problem is a Boolean formula F in so-called
conjunctive normal form (CNF):
F = C1 ∧C2 ∧ ...∧Cm,
where each Ci is a clause. F is TRUE if C1 is TRUE and (indicated by ∧) C2 is TRUE and ...
and Cm is TRUE, i.e., if all its clauses are TRUE.
SATISFIABILITY (SAT):
Instance: A Boolean formula F in CNF.
Goal: Determine whether there is a TRUE/FALSE-assignment to the variables such that F is
TRUE.
Theorem 5.1. proof that SAT is NP-complete.
The following restriction of SATISFIABILITY is also NP-complete.
3-SATISFIABILITY (3-SAT):
Instance: A Boolean formula F in CNF with each clause consisting of 3 literals.
Goal: Determine whether there is a TRUE/FALSE-assignment to the variables such that F is
TRUE.
Theorem 5.2. 3-SAT is NP-complete.
The proof reduces SAT to 3-SAT. We refer the reader to [PS82, Theorem 15.2].

Illustration of the construction in the proof of Theorem5.3 for the formula F =


(x1 ∨ x2 ∨ ¬x3)∧(¬x1 ∨ x2 ∨ x4)∧(¬x2 ∨ x3 ∨ x4). The red vertices constitute a vertex cover
of size K = n+ 2m = 10.
We introduce some more problems and give some examples of NP-completeness proofs.
Let G = (V,E) be an undirected graph. We need the following definitions: A clique of G is a
subset V of the vertices that induces a complete sub graph, i.e., for every two vertices u,v ∈
V , {u,v} ∈ E. An independent set of G is a subset V of vertices such that no two of them are
incident to the same edge, i.e., for every two vertices u,v ∈ V , {u,v} ∈ / E. A vertex cover of G
is a subset V of vertices such that every edge has at least one of its two incident vertices in
V , i.e., for every edge {u,v} ∈ E, {u,v} ∩V = 0. /
CLIQUE:
Instance: An undirected graph G = (V,E) and an integer K.
Goal: Determine whether G contains a clique of size at least K.
INDEPENDENT SET:
Instance: An undirected graph G = (V,E) and an integer K.
Goal: Determine whether G contains an independent set of size at least K.
VERTEX COVER:
Instance: An undirected graph G = (V,E) and an integer K.
Goal: Determine whether G contains a vertex cover of size at most K.
CLIQUE is NP-complete by a rather complicated reduction from 3-SAT. We first prove that
VERTEX COVER is NP-complete, again reducing from 3-SAT, but with a much easier
reduction.
Theorem 5.3. VERTEX COVER is NP-complete.
Proof. We first argue that VERTEX COVER is in NP. A certificate of a yes-instance is a subset
V ⊆ V of vertices with |V | ≤ K that forms a vertex cover of G = (V,E). This can be verified in
time at most O(n+ m) by checking whether each edge {u,v} ∈ E has at least one of its
incident vertices in V .
In order to prove that VERTEX COVER is NP-complete, we will show that 3-SAT VERTEX
COVER. Note that this is sufficient because 3-SAT is NP-complete.
We transform an instance of 3-SAT to an instance of VERTEX COVER as follows: Consider a
Boolean formula F in CNF with each clause having three literals. Let n and m denote the
number of variables and clauses of F, respectively. We create a variable-gadget for each
variable x consisting of two vertices x and ¬x that are connected by an edge. Moreover, we
create a clause-gadget for each clause C = (l1 ∨l2 ∨l3) consisting of three vertices l1,l2,l3
that are connected by a triangle. Finally, we connect each vertex representing a literal in a
clause-gadget to the corresponding vertex representing the same literal in the variable
gadget.
Let G = (V,E) be the resulting graph; see Figure 2 for an example.
Note that this transformation can be done in polynomial time.
We show that F is satisfiable if and only if G has a vertex cover of size at most K = n+ 2m.
First note that every assignment satisfying F can be turned into a vertex cover of size K:
For each variable-gadget we pick the vertex that corresponds to the literal which is TRUE.
This covers all edges in the variable-gadgets and their respective connections to the clause
gadgets. For each clause-gadget we choose two additional vertices so as to ensure that all
remaining edges are covered. The resulting vertex cover has size K = n+ 2m as claimed.
Next suppose that we are given a vertex cover V of G of size at most K. Note that every
vertex cover has to pick at least one vertex for every variable-gadget and two vertices for
each clause-gadget just to cover all edges inside these gadgets. Thus, V contains exactly K
vertices. The vertices in V now naturally induce an assignment as described above that
satisfies F. We conclude that yes-instances correspond under the above reduction, which
completes the proof.
Theorem 5.4. INDEPENDENT SET is NP-complete.
Proof. We first argue that INDEPENDENT SET is in NP. A certificate for a yes-instance is a
subset V of vertices that forms an independent set. To verify this, we just need to check that
there is no edge between every pair of vertices in V . This can be done in O(n + m) time.
We prove that VERTEX COVER INDEPENDENT SET in order to establish NP completeness
of INDEPENDENT SET. Given an instance G = (V,E) with parameter K of VERTEX COVER, we
define an instance of INDEPENDENT SET by taking the same graph G with parameter n − K.
Note that this mapping can be done in polynomial time, in fact constant time.
It remains to show that yes-instances correspond. We claim that V is a vertex cover in G if
and only if V \V is an independent set in G. If V is a vertex cover in G then for every pair u,v
∈V \V we have that edge {u,v} ∈ / E. Hence V \V is an independent set. Vice versa, if V \V is
an independent set, then for every edge {u,v} ∈ E at not both u and v are in V \V ,
i.e. at least one of u and v is in V . Hence V is a vertex cover.
We conclude that V is a vertex cover of size K in G if and only if V \V is an independent set
of size n −K in G ¯.
Theorem 5.5. CLIQUE is NP-complete.
Proof. We first argue that CLIQUE is in NP. A certificate for a yes-instance is a subset V of
vertices that forms a clique. To verify this, we just need to check that there is an edge
between every pair of vertices in V . This can be done in O(n+m) time.
We prove that INDEPENDENT SET CLIQUE in order to establish NP-completeness of
CLIQUE. We need the notion of a complement graph for this reduction. Given a graph G =
(V,E), the complement graph of G is defined as the graph G ¯ = (V,E ¯) with {u,v} ∈ E ¯ if and
only if {u,v} ∈ / E.
Given an instance G = (V,E) with parameter K of INDEPENDENT SET, we create the
complement graph of G and let G ¯ with parameter K be the respective instance of CLIQUE.
Note that this mapping can be done in polynomial time by adding an edge {u,v} toE ¯ for
every pair of vertices u,v ∈ V with {u,v} ∈ / E. This takes at most O(n2) time.
It remains to show that yes-instances correspond. We claim that V is an independent set of
G if and only if V is a clique of G ¯. Note that V is an independent set of G if and only if for
each pair of vertices in V there is no edge in E. The latter is true if and only if for each pair
of vertices in V there is an edge in E ¯, which is equivalent to V being a clique of G ¯. This
proves the claim. We conclude that V is an independent set of size K in G if and only if V is a
clique of size K in G ¯.
Theorem 5.6. HAMILTONIAN CYCLE is NP-complete.
The proof follows by reducing 3-SAT to HAMILTONIAN CYCLE. The reader is referred to
[PS82, Theorem 15.6]. The proof is very ingenious. See it as a challenge to understand it. Of
course you won’t be asked to reproduce it at the exam.
Once we know that HAMILTONIAN CYCLE is NP-complete, it becomes rather easy to prove
NP-completeness of two other problems: HAMILTONIAN PATH, which is the problem that
asks for the existence of a path i.o. a cycle, in a graph that contains each vertex, and the
Travelling Salesman Problem. The proof of NP-completeness of HAMILTONIAN PATH I
leave to study by yourself. The proof of NP-completeness of TSP follows.
Theorem 5.7. TSP is NP-complete.
Proof. We argued before that TSP is in NP. The proof now follows trivially because
HAMILTONIAN CYCLE is a special case of TSP: Given an instance G=(V,E) of HAMILTONIAN
CYCLE we construct an instance of TSP as follows: Let G = (V,E ) be the complete graph on V
and define de = 1 if e ∈ E and de = 2 otherwise. Now a tour in G of length at most K = n
relates to a Hamiltonian cycle in G and vice versa.
The above proof actually shows that the restriction of TSP in which all distances are either
1 or 2 is NP-complete. Because TSP is in NP and it is a generalization of this problem, NP
completeness of TSP follows immediately. This is a proof by restriction: we showed by a
reduction from HAMILTONIAN CYCLE that TSP with distances 1 and 2 is NP-complete, and
because this is a restriction (special case) of TSP, it follows automatically that TSP is NP
complete (given hat it is in NP). Also TSP with distances 1 and 2 is restriction of METRIC
TSP, i.e., distances satisfy the triangle inequality. Hence, METRIC TSP is NP-complete as
well.
The same holds true for SATISFIABILITY: If we would not know that it is NP-complete but
we would know that 3-SAT is NP-complete, then NP-completeness of SAT followed
automatically using the fact that 3-SAT is a special case of SAT. While restrictions can
create an easier subclass of problem instances, generalizations always create more difficult
problems. This gives sometimes easy ways to show NP-completeness of problems
Hopefully it is clear that the other way round does not work. A special case of a NP-
complete problem is not necessarily NP-complete. In the above some examples of easy
special cases of NP-complete problems are given. Nowadays it is a hot research topic in
combinatorial optimization to find conditions on NP-complete problems under which the
problem becomes polynomial time solvable. This research is referred to in the literature as
Fixed Parameter Tractability.
Exercise 4.1. Show that 0/1-ILP is NP-complete. Hint: Reduce SAT to this problem.
Exercise 4.2. Show that ILP is NP-complete. Hint: This is trivial.
Exercise 4.3 Show that CLIQUE is NP-complete by a reduction from VERTEX COVER.
Exercise 4.4. Show that the following problem is NP-complete:
LONGEST PATH:
Instance: An undirected graph G = (V,E) and a parameter K.
Goal: Determine whether there exists a (simple) path in G of length at least K.

2.6. Some Other NP-complete Problems

We list some more NP-complete problems (without proof) that are often used in
reductions.
2-PARTITION:
Instance: Integers s1,...,sn.
Goal: Decide whether there is a set S ⊆ {1,...,n} such that∑i∈S si = 1 2 ∑n i=1 si.
3-PARTITION:
Instance: Rational numbers s1,...,s3n with 1 4 < si < 1 2 for every i = 1,...,3n.
Goal: Determine whether the set {1,...,3n} can be partitioned into n triplets S1,...,Sn such
that ∑i∈Sk ai = 1 for every k = 1,...,n.
SET COVER.:
Instance: A universe U = {1,... ,n} of n elements, a family of m subsets S1,...,Sm ⊆ U and an
integer K.
Goal: Determine whether there is a selection of at most K subsets such that their union is U.
For example NP-completeness of 2-PARTITION allows an easy proof of the following basic
scheduling problem.
MAKESPAN SCHEDULING:
Instance: m machines and n jobs; each job j has a processing time pj, j = 1,...,n, a constant K.
Goal: Determine whether there is a schedule of the jobs on the machines, in which each
machine cannot process two jobs simultaneously, each job is processed uninterruptedly by
exactly one machine, and all jobs complete processing before time K.
The MAKESPAN SCHEDULING can be interpreted as a load balancing problem: divide the
total work as equally as possible over the machines, such that the load of the machine with
heaviest load is minimized.
Exercise . Show that MAKESPAN SCHEDULING is NP-complete by a reduction from 2-
PARTITION.
2.7. NP-hard Problems
Sometimes we may be unable to prove that a problem Π is in NP but nevertheless can show
that all problems in NP are reducible to Π. According to previous Definition, Π does not
qualify to be an NP-complete problem because it is not in NP. Yet, Π is as hard as any other
problem in NP and thus probably a difficult problem. We call such problem NP-hard. An
example of such a problem is the KTH HEAVIEST SUBSET problem:
KTH HEAVIEST SUBSET:
Instance: Integers w1,...,wn,t and a parameter K.
Goal: Determine whether the weight of the tth heaviest subset of {1,... ,n}
is at least K. (Formally, determine whether there are t distinct subsets
S1,...,St ⊆ {1,...,n} such that w(Si) = ∑j∈Si wj ≥ K for every i = 1,...,t.)
It can be proven that all problems in NP are polynomial-time reducible to the KTH
HEAVIEST SUBSET problem (see [PS82, Theorem 16.8]). However, a proof that short
certificates exist for yes-instance is non-existent. How else could we provide a certificate
for a yes instance other than explicitly listing t subsets that are heavier than K? (Note that
this is not a short certificate because t can be exponential in n.)
We use NP-hardness also for the complexity of optimisation problems whose recognition
version is NP-complete. It is clear that these problems are at least as hard as their decision
versions. Formally they may be excluded from NP. because we cannot speak of certificates
of a yes-instance. But even if polynomial verifiability of a certificate for optimality would
permit membership to the class NP, it is not so clear that they are in NP.
A natural certificate for an optimal solution would be the solution itself, but how would one
verify that this is indeed the optimal solution. In LP it is easy, we could also give (or
compute) the corresponding dual solution, and verification of feasibility of both solutions
and equality of their objective values can be done in polynomial time. In Max Flow, an
optimal flow and a corresponding minimum cut have the same easy verifiability. But for
TSP of VERTEX COVER we do not have such characterizations of optimal solutions and
Verifications of claimed optimality of a given solution is much less obvious.
Usually we do not bother about this and any problem that is not in NP, but for which
polynomial solvability would imply that all problems in the class NP can be solved in
polynomial time is called an NP-Hard problem. In the same way the term NP-Easy is
sometimes used for problems that can be solved in polynomial time using the solution of a
problem in NP.
Officially, NP-hard problems may be in a higher complexity class.

2.8. Complexity Class co-NP

Another complexity class that is related to NP is the class co-NP (which stands for
complement of NP). Here one considers complements of decision problems. As an example,
the complement of HAMILTONIAN CYCLE reads as follows:
HAMILTON CYCLE COMPLEMENT:
Instance: An undirected graph G = (V,E).
Goal: Determine whether G does not contain a Hamiltonian cycle.
There are no short certificates known for yes-instances of this problem.
Definition 5.7. A decision problem Π belongs to the class co-NP if and only if its complement
belongs to the class NP. Said differently, a decision problem belongs to co-NP if every no-
instance I ∈ I admits a certificate whose validity can be verified in polynomial time.
It is not hard to see that every problem in P also belongs to co-NP. Thus, P ⊆ NP∩ co-NP.
Similar to the P = NP conjecture, it is widely believed that NP = co-NP.
Theorem 5.8. If the complement of an NP-complete problem is in NP, then NP = co-NP.
Proof. Assume that the complement Π ¯ 2 of an NP-complete problem Π2 is is in NP. We will
show that the complement Π ¯ 1 of an arbitrary problem Π1 ∈ NP is also in NP thus showing
that NP = co-NP.
Because Π2 is NP-complete, we know that Π1 is polynomial-time reducible to Π2. Note that
the reduction ϕ from Π1 to Π2 is also a polynomial-time reduction from Π ¯ 1 to Π ¯ 2. We
can therefore exhibit a short certificate for every yes-instanceI¯1 of Π ¯ 1 as follows: We first
transform I¯1 to I¯2 = ϕ(I¯1) and then use the short certificate for the yes-instance I¯2
(which must exist because Π ¯ 2 ∈ NP). We conclude that Π ¯ 1 is in NP which finishes the
proof.
Note that the above theorem also implies that if the complement of a problem in NP is also
in NP then (unless NP = co-NP) this problem is not NP-complete. Said differently, a problem
that belongs to NP ∩ co-NP is unlikely to be NP-complete. As an example, consider the
linear programming problem LP. Using duality theory, it is not hard to see that LP ∈ NP∩
co-NP. Before LP was known to be polynomial-time solvable, it was in fact the above
observation that gave strong evidence to the conjecture that LP ∈ P.
Exercise 4.5. Show that LP ∈ NP∩ co-NP.
2.9. Pseudo-polynomiality and Strong NP-completeness
Sometimes the running time of an algorithm is polynomial in the size of the instance and
the largest number in the input. As an example, consider the INTEGER KNAPSACK problem.
INTEGER KNAPSACK:
Instance: Integers c1,...,cn and a parameter K.
Goal: Determine whether there exist integers x1,...,xn such that ∑n k=1 ckxk = K.
This problem can be solved as follows: Create a directed graph G = (V,A) with K + 1
vertices V = {0,1,... ,K} and O(nK) arcs:
A = {(i, j) : 0 ≤ i < j ≤ K and j = i+ ck for some k}.
It is not hard to prove that an instance I of the INTEGER KNAPSACK problem is a yes
instance if and only if there exists a path from 0 to K in G. The latter problem can be solved
in time O(n+m) = O(nK). This running time is not polynomial in the size of I. To see this,
recall that we defined the size |I| of I to be the number of bits that are needed to represent I
in binary. Making the (reasonable) assumption that c1,...,cn < K, the size of I is therefore at
most O(nlogK). The running time of the algorithm is therefore not polynomially bounded in
general. However, if K is bounded by a polynomial function of n then the algorithm would
be a polynomial-time algorithm. That is, depending on the application the above algorithm
might indeed be considered to be reasonably efficient, despite the fact that the problem is
NP-complete (which it is).
The above observation gives rise to the following definition. Given an instance I, let num(I)
refer to the largest integer appearing in I.
Definition 5.8. An algorithm ALG for a problem Π is pseudo-polynomial if it solves every
instance I ∈ I of Π in time bounded by a polynomial function in |I| and num(I).
Problems that remain NP-complete even if the largest integer appearing in its description is
bounded polynomially in the size of the instance is called strongly NP-complete.
Definition 5.9. A problem Π is strongly NP-complete if the restriction of Π to instances I ∈ I
satisfying that num(I) is polynomially bounded in |I| is NP-complete.
Note that many problems that we showed to be NP-complete do not involve any numerical
data that is larger than the input size itself. For example, all graph problems such as
HAMILTONIAN CYCLE, CLIQUE, INDEPENDENT SET, VERTEX COVER, etc. satisfy num(I) =
O(n) and are therefore even strongly NP-complete by definition. In fact, also TSP is strongly
NP-complete because we established NP-completeness even for instances with distances 1
or 2.
As the theorem below shows, we cannot expect to find a pseudo-polynomial algorithm for a
strongly NP-complete problem (unless P = NP).
Theorem 5.9. There does not exist a pseudo-polynomial algorithm for a strongly
NPcomplete problem, unless P = NP.
15Proof. Let Π be a strongly NP-complete problem and suppose that ALG is a
pseudopolynomial algorithm for Π. Consider the restriction Π ¯ of Π to instances I ∈ I that
satisfy that num(I) is polynomially bounded in |I|. By Definition 5.9, Π ¯ is NP-complete. But
ALG can solve every instance I¯ of Π ¯ in time polynomial in |I¯| and num(I¯), which is
polynomial in |I¯|. This is impossible unless P = NP.
2.10. Complexity Class PSPACE
The common criterion that we used to define the complexity classes P and NP was time: P
refers to the set of problems that are solvable in polynomial time; NP contains all problems
for which yes-instances can be verified in polynomial time. There are other complexity
classes that focus on the criterion space instead: The complexity class PSPACE refers to the
set of problems for which algorithms exist that only require a polynomial amount of space
(in the size of the input).
Definition 5.10. A decision problem Π belongs to the complexity class PSPACE if there exists
an algorithm that for every instance I ∈ I determines whether I is a yes-instance or a no-
instance using space that is polynomially bounded in the size of I.
Clearly, every polynomial-time algorithm cannot consume more than polynomial space and
thus P ⊆ PSPACE. However, even exponential-time algorithms are feasible as long as they
only require polynomial space. We can use this observation to see that NP ⊆ PSPACE:
Consider an arbitrary problem Π in NP. We know that every yes-instances of Π admits a
short certificate. We can therefore generate all potential short certificates one after another
and verify the validity of each one. If we encounter a valid certificate throughout this
procedure then we report that the instance is a yes-instance; otherwise, we report that it is
a no-instance. The algorithm may take exponential time because the number of certificates
to be checked might be exponential. However, it can be implemented to use only
polynomial space by deleting the previously generated certificate each time.
What if P = NP?
If P = NP —specifically, if an NP-complete problem like 3SAT had a very efficient algorithm
running in say O(n2) time— then the world would be mostly a Utopia. Mathematicians
could be replaced by efficient theorem-discovering programs (a fact pointed out in Kurt
G¨odel’s 1956 letter and discovered three decades later). In general for every search
problem whose answer can be efficiently verified (or has a short certificate of correctness),
we will be able to find the correct answer or the short certificate in polynomial time. AI
software would be perfect since we could easily do exhaustive searches in a large tree of
possibilities. Inventors and engineers would be greatly aided by software packages that can
design the perfect part or gizmo for the job at hand.
VLSI designers will be able to whip up optimum circuits, with minimum power
requirements. Whenever a scientist has some experimental data, she would be able to
automatically obtain the simplest theory (under any reasonable measure of simplicity we
choose) that best explains these measurements; by the principle of Occam’s Razor the
simplest explanation is likely to be the right one.
Of course, in some cases it took scientists centuries to come up with the simplest theories
explaining the known data. This approach can be used to solve also non-scientific
problems: one could find the simplest theory that explains, say, the list of books from the
New York Times’ bestseller list.
Somewhat intriguingly, this Utopia would have no need for randomness. If P = NP then
randomized algorithms would buy essentially no efficiency gains over deterministic
algorithms; (Philosophers should ponder this one.)
This Utopia would also come at one price: there would be no privacy in the digital domain.
Any encryption scheme would have a trivial decoding algorithm. There would be no digital
cash,
no SSL, RSA or PGP. We would just have to learn to get along better without
these, folks.
This utopian world may seem ridiculous, but the fact that we can’t rule it out shows how
little
we know about computation. Taking the half-full cup point of view, it shows how many
wonderful
things are still waiting to be discovered.
What if NP = coNP?
If NP = coNP, the consequences still seem dramatic. Mostly, they have to do with existence of
short certificates for statements that do not seem to have any. To give an example, remember the
NP-complete problem of finding whether or not a set of multivariate polynomials has a common
root,
In other words, deciding whether a system of equations of the following type has a solution:
f1(x1, . . . , xn) = 0
f2(x1, . . . , xn) = 0
...
fm (x1, . . . , xn) = 0
Where each fi is a quadratic polynomial.
If a solution exists, then that solution serves as a certificate to this effect (of course, we have
to
also show that the solution can be described using a polynomial number of bits, which we omit).
The problem of deciding that the system does not have a solution is of course in coNP. Can we give
a certificate to the effect that the system does not have a solution? Hilbert’s Nullstellensatz
Theorem seems to do that: it says that the system is infeasible iff there is a sequence of polynomials
g1, g2, . . . , gm such that Pi figi = 1, where 1 on the right hand side denotes the constant polynomial 1.
What is happening?
Does the Nullstellensatz prove coNP = NP? No, because the degrees of the gi’s —and hence the
number of bits used to represent them— could be exponential in n, m.
(And it is simple to construct fi’s for which this is necessary.)
However, if NP = coNP then there would be some other notion of a short certificate to the effect that
the system is infeasible. The effect of such a result on mathematics would probably be even greater
than the effect of Hilbert’s Nullstellensatz. Of course, one can replace Nullstellensatz with any other
coNP problem in the above discussion.

2.11. Chapter summary


 The complexity classes P and NP refer to decision problems that can be
solved efficiently and those for which yes-instances can be verified
efficiently, respectively.
 The common criterion that we used to define the complexity classes P and NP
was time: P refers to the set of problems that are solvable in polynomial time;
NP contains all problems for which yes-instances can be verified in
polynomial time. There are other complexity classes that focus on the
criterion space instead: The complexity class PSPACE refers to the set of
problems for which algorithms exist that only require a polynomial amount
of space (in the size of the input).
 After thinking for a little while, we conclude that P ⊆ NP. Several decades of
intensive research seem to suggest that there are problems in NP that are
intrinsically more difficult than the ones in P and thus P = NP: Despite the
many research efforts, no polynomial time algorithms have been found for
problems in NP such as TSP, HAMILTONIAN CYCLE, STEINER TREE, etc.

You might also like