P, NP, NP-Hard, NP-Complete

NP complete problems are problems whose status is unknown. No polynomial time algorithm has yet
been discovered for any NP complete problem, nor has anybody yet been able to prove that no
polynomial-time algorithm exist for any of them. The interesting part is, if any one of the NP complete
problems can be solved in polynomial time, then all of them can be solved.

P- Polynomial time solving . Problems which can be solved in polynomial time, which take time like O(n),
O(n2), O(n3). Eg: finding maximum element in an array or to check whether a string is palindrome or not.
so there are many problems which can be solved in polynomial time.

NP- Non deterministic Polynomial time solving. Problem which can't be solved in polynomial time like
TSP( travelling salesman problem) or An easy example of this is subset sum: given a set of numbers, does
there exist a subset whose sum is zero?. These problems can’t be solved in polynomial time by
deterministic algorithms but they can be solved in polynomial time by nondeterministic algorithms.

but NP problems are checkable in polynomial time means that given a solution of a problem , we can
check that whether the solution is correct or not in polynomial time.

So NP is the set of decision problems that can be verified in polynomial time.

Examples of algorithms which can be solved by algorithms that take polynomial time and exponential
time are:

Polynomial Time Exponential time

Linear Search – O(n) Travelling Salesman Problem – O(n22n)
Binary Search –O(logn) Knapsack Problem – O(2n/2)
Selection Sort –O(n2) Sum of Subsets –O(2n)
Merge Sort –O(nlogn) Graph coloring – O(nmn) where n is the no. of
vertices, m is the no. of colours
Matrix multiplication – O(n3) Hamiltanion Cycle – O(2n)
Ordered Searching – O(logn)
Polynomial Evaluation –O(n)(when Horners Rule is
used)
Huffman Coding, Optimal merge patterns –O(n2)

Many of the problems which do not have polynomial time algorithms are computationally related.
A problem that is NP-complete has the property that it can be solved in polynomial time if and only if all
other NP-complete problems can also be solved in polynomial time. A problem is said to be NP-hard if
every problem in NP-complete set is reducible to it.

Reducibility- If we can convert one instance of a problem A into problem B (NP problem) then it means
that A is reducible to B.

A problem is said to be NP-hard if everything in NP can be transformed in polynomial time into it, and a
problem is NP-complete if it is both in NP and NP-hard. The NP-complete problems represent the
hardest problems in NP. If any NP-complete problem has a polynomial time algorithm, all problems in
NP do. Although a solution to an NP-complete problem can be verified "quickly", there is no known way
to find a solution quickly. That is, the time required to solve the problem using any currently
known algorithm increases rapidly as the size of the problem grows.

The above table shows the time taken by different algorithms with different time complexities.

Nondeterministic algorithms

Algorithms whose every operation is uniquely defined are called deterministic algorithms. Such
algorithms agree with the way programs are executed on a computer. In theory we can remove this
restriction on the outcome of every operation. We can allow algorithms to contain operations whose
outcomes are not uniquely defined but are limited to a specified set of possibilities. The machine
executing such operations is allowed to choose any one of these outcomes subject to a termination
condition. This leads to the concept of a non deterministic algorithm. To specify such algorithms, three
functions are used.

1. Choice(S): arbitrarily chooses one of the elements of set S.

2. Failure() signals an unsuccessful completion

3. Success() signals a successful completion.

The assignment statement x:= Choice(1,n) could result in x being assigned one of the integers in the
range [1,n]. There is no rule specifying how this choice is to be made. The Failure() and Success() signals
are used to define a computation of the algorithm. Whenever there is a set of choices that leads to a
successful completion, then one such set of choices is always made and the algorithm terminates
successfully. A nondeterministic algorithm terminates unsuccessfully if and only if there exists no set
of choices leading to a success signal.

The computing times of Choice, Success and Failure are taken as O(1). A machine capable of executing
a nondeterministic algorithm in this way is called a nondeterministic machine (which does not exist in
practice).

Nondeterministic algorithm for search

The above algorithm outputs 0 if and only if there is no j such that A[j]=x; If ‘x’ is in A, Choice will
somehow find its index (how it finds is not known now but some researcher might find it in future). So
only if ‘x’ is not in A the algorithm terminates unsuccessfully.

The time complexity of this nondeterministic algorithm is O(1). It should be observed that if A is
unordered, every deterministic search algorithm (like linear search) has a time complexity of Ω(𝑛)

Nondeterministic algorithm for Sorting:

The nondeterministic sorting algorithm for sorting somehow determines the right position where every
element should be there after sorting ( this is done by the Choice function). The element is then placed
in second array B in that position. Finally in line 11 and 12 it is verified if every element in in B is less
than its next consecutive element in B if not the algorithm terminates unsuccessfully otherwise the
algorithm terminates successfully.
Definition: Any problem for which the answer is either zero or one is called a decision problem. An
algorithm for a decision problem is termed a decision algorithm. Any problem that involves the
identification of an optimal (either minimum or maximum) value of a given cost function is known as an
optimization problem. An optimization algorithm is used to solve an optimization problem.

We concentrate on nondeterministic decision algorithms. Although we are restricting ourselves to

decision problems, many optimization problems can be recast into decision problems with the
property that the decision problem can be solved in polynomial time if and only if the corresponding
optimization problem can. In other cases we can at least make a statement that if the decision
problem cannot be solved in polynomial time, then the optimization problem cannot either.

Converting Maximum Clique Optimization problem to corresponding Decision Problem

A maximal complete subgraph of a graph G= (V,E) is a clique. The size of the clique is the number of
vertices in it. The max clique problem is an optimization problem that has to determine the size of the
largest clique in G. The corresponding decision problem is to determine whether G has a clique of size
atleast ‘k’ for some given ‘k’. The optimization problem can be solved using this decision problem by
calling the decision problem for different values of k=n,n-1,n-2,…1.

Line 4 selects ‘k’ vertices from the graph. Lines 6 to 9 are used to find the vertices (distinct vertices) that
define the clique (again the choice function some how finds these vertices). Lines 11 and 12 are used to
check if there is an edge between every pair of vertices in S. If not the algorithm terminates
unsuccessfully otherwise the algorithm terminates successfully. The time complexity of this non
deterministic algorithm is determined by lines 11 and 12 in the algorithm which is O(n2) when adjacency
matrix is used.

0/1 Knapsack Optimization Problem

A thief is robbing a store and can carry a maximal weight of m into his knapsack. There are n items and
weight of ith item is wi and the profit of selecting this item is pi. What items should the thief take to make
maximum profit?

0/1 Knapsack Decision Problem

// n is the number of items

// w is an array containing the weights of n items
// p is an array containing the profits of n items
// m is maximum capacity of knapsack
// r is minimum profit that is to be earned
// x is a Boolean array containing 0s or 1s that specifies which item is selected and which item is not
selected
The for loop in line 4 is used to check if ith item has to be placed in the Knapsack or not. Choice function
in line 6 some how correctly determines if ith item has to be placed in Knapsack or not and accordinly the
weight W and profit P are updated. In line 9 we check if the total weight of all items selected into
knapsack W doesnot exceed the capacity of the knapsack ‘m’ and the total profit earned is atleast ‘r’, if
not the algorithm terminates unsuccessfully otherwise the algorithm terminates successfully. The time
complexity of the above nondeterministic algorithm is O(n).

Satisfiability Problem (SAT):

The deterministic algorithm for satisfiability takes O(2n) time because it has to check all possible values
of given ‘n’ Boolean variables. In the non deterministic algorithm described above lines 5 nd 6 assigns
either 0 or 1 (false or true) to each of the variables x1,x2,x3,x4…. The above nondtermininstic algorithm
time depends on line 7 which is used to check if the values assigned to each variable (either 0 or 1)
makes the formula true or not.
If satisfiability (SAT) problem can be solved in polynomial time all exponential time algorithms can be
solved in polynomial time.

SAT problem: Boolean satisfiability

Input is a Boolean formula. It has variables like x1,x2,x3 …. If a Boolean formula has only 3 unique
variables it is called 3-SAT. In Boolean formula x1,x2,x3… can take 0 or 1.

The allowed operations are not 𝑥̅̅̅1 , ̅̅̅

𝑥2 …

And – x1^ x2
Or – x1 V x2

So, the satisfiability problem is for what values of X1,X2,X3 a formula like the following is true

((𝑋1 ∨ 𝑋2 ) ∨ ̅̅̅ ̅̅̅1 ∨ 𝑋2 )

𝑋3 ) ∧ (𝑋

X1 X2 X3
3
Since there are 3 variables there are 2 possible combinations of values 0 0 0
0 0 1
If there are n variables we need to check 2n possible combinations of values. 0 1 0
0 1 1
So, finding the answer takes exponential time. 1 0 0
1 0 1
1 1 0
1 1 1

The above diagram shows the state space tree for 3-SAT problem. If satisfiability can be solved in
polynomial time all others can also be solved in polynomial time. If any one of them can be solved in
polynomial all can be solved in polynomial time.

0/1 Knapsack problem

Let n=3, m=8, p={10,8,12}, w={5,4,3}

Solution can be expressed as {x1, x2, x3} i.e. which can be one of the following as solution set i.e. either
{0,0,0} or {0,0,1} or {0,1,0} or {0,1,1}… -----------------------------------------------------------------------------(1)
If there are n objects there 2n combinations. This can be solved using state space tree using techniques
like branch and bound or backtracking. ------------------------------------------------------------- (2)

Prove that 0/1 Knapsack is NP-Hard

Since Satisfiability is NP-hard and if Satifiability reduces (symbol ∝) to 0/1 Knapsack then 0/1 Knapsack
will be NP-Hard.

Take an instance of satisfiabilty problem and convert it to 0/1 Knapsack problem. Conversion takes
polynomial time. If 0/1 Knapsack problem can be solved in polynomial time, then satisfiability problem
can also be solved in polynomial time. From (1) satisfiability can be reduced to 0/1 Knapsack . So, 0/1
Knapsack is also NP-hard.

Satisfiability is not only NP-hard but also has a non-deterministic polynomial time algorithm so it is NP-
complete.

Cooks theorem:

To find how satisfiability reduces to clique decision problem go through Abdul Bari Youtube channel.