ALGORITHM DESINGH

UNIT 5
Introduction to Backtracking – Data Structure and Algorithm Tutorials

Backtracking is an algorithmic technique for solving problems recursively by trying to build

a solution incrementally, one piece at a time, removing those solutions that fail to satisfy the
constraints of the problem at any point in time (by time, here, is referred to the time elapsed
till reaching any level of the search tree). Backtracking can also be said as an improvement
to the brute force approach. So basically, the idea behind the backtracking technique is that it
searches for a solution to a problem among all the available options. Initially, we start the
backtracking from one possible option and if the problem is solved with that selected option
then we return the solution else we backtrack and select another option from the remaining
available options. There also might be a case where none of the options will give you the
solution and hence we understand that backtracking won’t give any solution to that particular
problem. We can also say that backtracking is a form of recursion. This is because the
process of finding the solution from the various option available is repeated recursively until
we don’t find the solution or we reach the final state. So we can conclude that backtracking at
every step eliminates those choices that cannot give us the solution and proceeds to those
choices that have the potential of taking us to the solution.
According to the wiki definition,
Backtracking can be defined as a general algorithmic technique that considers searching
every possible combination in order to solve a computational problem.

There are three types of problems in backtracking –

1. Decision Problem – In this, we search for a feasible solution.
2. Optimization Problem – In this, we search for the best solution.
3. Enumeration Problem – In this, we find all feasible solutions.
How to determine if a problem can be solved using Backtracking?
Generally, every constraint satisfaction problem which has clear and well-defined constraints
on any objective solution, that incrementally builds candidate to the solution and abandons a
candidate (“backtracks”) as soon as it determines that the candidate cannot possibly be
completed to a valid solution, can be solved by Backtracking. However, most of the problems
that are discussed, can be solved using other known algorithms like Dynamic
Programming or Greedy Algorithms in logarithmic, linear, linear-logarithmic time
complexity in order of input size, and therefore, outshine the backtracking algorithm in every
respect (since backtracking algorithms are generally exponential in both time and space).
However, a few problems still remain, that only have backtracking algorithms to solve them
until now.
Consider a situation that you have three boxes in front of you and only one of them has a gold
coin in it but you do not know which one. So, in order to get the coin, you will have to open
all of the boxes one by one. You will first check the first box, if it does not contain the coin,
you will have to close it and check the second box and so on until you find the coin. This is
what backtracking is, that is solving all sub-problems one by one in order to reach the best
possible solution.
Consider the below example to understand the Backtracking approach more formally,
Given an instance of any computational problem and data corresponding to the
instance, all the constraints that need to be satisfied in order to solve the problem are
represented by . A backtracking algorithm will then work as follows:
The Algorithm begins to build up a solution, starting with an empty solution set . S = {}
1. Add to Backtracking | Set 1Backtracking | Set 1the first move that is still left (All
possible moves are added to one by one). This now creates a new sub-tree in
the search tree of the algorithm.
2. Check if satisfies each of the constraints in .
 If Yes, then the sub-tree is “eligible” to add more “children”.
 Else, the entire sub-tree is useless, so recurs back to step 1 using
argument .
3. In the event of “eligibility” of the newly formed sub-tree , recurs back to step 1,
using argument .
4. If the check for returns that it is a solution for the entire data . Output
and terminate the program.
If not, then return that no solution is possible with the current and hence discard
it.

Basic terminologies:

 Solution vector:
The desired solution X to a problem instance P of input size n is as a vector of candidate
solutions that are selected from some finite set of possible solutions S.
Thus, a solution can be represented as an n-tuple (X1, X2, …, Xn) and its partial solution is
given as (X1, X2,…, Xi) where i<n.
E.g. for a 4-queens problem X = {2,4,1,3) is a solution vector.
 Constraints:
Constraints are the rules to confine the solution vector (X1, X2…… Xa).
They determine the values of candidate solutions and their relationship with each other. There
are two types of constraints:
N-Queens Problem

N - Queens problem is to place n - queens in such a manner on an n x n chessboard that no

queens attack each other by being in the same row, column or diagonal.

It can be seen that for n =1, the problem has a trivial solution, and no solution exists for n =2
and n =3. So first we will consider the 4 queens problem and then generate it to n - queens
problem.

Given a 4 x 4 chessboard and number the rows and column of the chessboard 1 through 4.
Since, we have to place 4 queens such as q 1 q2 q3 and q4 on the chessboard, such that no two
queens attack each other. In such a conditional each queen must be placed on a different row,
i.e., we put queen "i" on row "i."

Backward Skip 10sPlay VideoForward Skip 10s

Now, we place queen q1 in the very first acceptable position (1, 1). Next, we put queen q 2 so
that both these queens do not attack each other. We find that if we place q 2 in column 1 and 2,
then the dead end is encountered. Thus the first acceptable position for q 2 in column 3, i.e. (2,
3) but then no position is left for placing queen 'q 3' safely. So we backtrack one step and place
the queen 'q2' in (2, 4), the next best possible solution. Then we obtain the position for placing
'q3' which is (3, 2). But later this position also leads to a dead end, and no place is found
where 'q4' can be placed safely. Then we have to backtrack till 'q 1' and place it to (1, 2) and
then all other queens are placed safely by moving q 2 to (2, 4), q3 to (3, 1) and q4 to (4, 3). That
is, we get the solution (2, 4, 1, 3). This is one possible solution for the 4-queens problem. For
another possible solution, the whole method is repeated for all partial solutions. The other
solutions for 4 - queens problems is (3, 1, 4, 2) i.e.
Job Assignment Problem using Branch And Bound
Let there be N workers and N jobs. Any worker can be assigned to perform any job, incurring
some cost that may vary depending on the work-job assignment. It is required to perform all
jobs by assigning exactly one worker to each job and exactly one job to each agent in such a
way that the total cost of the assignment is minimized.

Let us explore all approaches for this problem.

Solution 1: Brute Force
We generate n! possible job assignments and for each such assignment, we compute its total
cost and return the less expensive assignment. Since the solution is a permutation of the n
jobs, its complexity is O(n!).
Solution 2: Hungarian Algorithm
The optimal assignment can be found using the Hungarian algorithm. The Hungarian
algorithm has worst case run-time complexity of O(n^3).
Solution 3: DFS/BFS on state space tree
A state space tree is a N-ary tree with property that any path from root to leaf node holds one
of many solutions to given problem. We can perform depth-first search on state space tree
and but successive moves can take us away from the goal rather than bringing closer. The
search of state space tree follows leftmost path from the root regardless of initial state. An
answer node may never be found in this approach. We can also perform a Breadth-first search
on state space tree. But no matter what the initial state is, the algorithm attempts the same
sequence of moves like DFS.

Types of Complexity Classes | P, NP, CoNP, NP hard and NP complete

umng01

 Read
 Discuss
 Courses
 Practice
In computer science, there exist some problems whose solutions are not yet found, the
problems are divided into classes known as Complexity Classes. In complexity theory, a
Complexity Class is a set of problems with related complexity. These classes help scientists
to group problems based on how much time and space they require to solve problems and
verify the solutions. It is the branch of the theory of computation that deals with the resources
required to solve a problem.
The common resources are time and space, meaning how much time the algorithm takes to
solve a problem and the corresponding memory usage.
The time complexity of an algorithm is used to describe the number of steps required to solve
a problem, but it can also be used to describe how long it takes to verify the answer.
The space complexity of an algorithm describes how much memory is required for the
algorithm to operate.
Complexity classes are useful in organizing similar types of problems.
Types of Complexity Classes
This article discusses the following complexity classes:
1. P Class
2. NP Class
3. CoNP Class
4. NP-hard
5. NP-complete
P Class
The P in the P class stands for Polynomial Time. It is the collection of decision
problems(problems with a “yes” or “no” answer) that can be solved by a deterministic
machine in polynomial time.
Features:
1. The solution to P problems is easy to find.
2. P is often a class of computational problems that are solvable and tractable.
Tractable means that the problems can be solved in theory as well as in practice.
But the problems that can be solved in theory but not in practice are known as
intractable.
This class contains many natural problems:
1. Calculating the greatest common divisor.
2. Finding a maximum matching.
3. Decision versions of linear programming.
NP Class
The NP in NP class stands for Non-deterministic Polynomial Time. It is the collection of
decision problems that can be solved by a non-deterministic machine in polynomial time.
Features:
1. The solutions of the NP class are hard to find since they are being solved by a
non-deterministic machine but the solutions are easy to verify.
2. Problems of NP can be verified by a Turing machine in polynomial time.
Example:
Let us consider an example to better understand the NP class. Suppose there is a company
having a total of 1000 employees having unique employee IDs. Assume that there are 200
rooms available for them. A selection of 200 employees must be paired together, but the CEO
of the company has the data of some employees who can’t work in the same room due to
personal reasons.
This is an example of an NP problem. Since it is easy to check if the given choice of 200
employees proposed by a coworker is satisfactory or not i.e. no pair taken from the coworker
list appears on the list given by the CEO. But generating such a list from scratch seems to be
so hard as to be completely impractical.
It indicates that if someone can provide us with the solution to the problem, we can find the
correct and incorrect pair in polynomial time. Thus for the NP class problem, the answer is
possible, which can be calculated in polynomial time.
This class contains many problems that one would like to be able to solve effectively:
1. Boolean Satisfiability Problem (SAT).
2. Hamiltonian Path Problem.
3. Graph coloring.
Introduction to NP-Completeness
4. We have been writing about efficient algorithms to solve complex problems,
like shortest path, Euler graph, minimum spanning tree, etc. Those were all success
stories of algorithm designers. In this post, failure stories of computer science are
discussed.
5. Can all computational problems be solved by a computer? There are
computational problems that can not be solved by algorithms even with unlimited
time. For example Turing Halting problem (Given a program and an input, whether
the program will eventually halt when run with that input, or will run forever). Alan
Turing proved that a general algorithm to solve the halting problem for all possible
program-input pairs cannot exist. A key part of the proof is, that the Turing machine
was used as a mathematical definition of a computer and program (Source Halting
Problem).
Status of NP-Complete problems is another failure story, NP-complete problems are
problems whose status is unknown. No polynomial-time algorithm has yet been
discovered for any NP-complete problem, nor has anybody yet been able to prove that
no polynomial-time algorithm exists for any of them. The interesting part is, that if
any one of the NP-complete problems can be solved in polynomial time, then all of
them can be solved.
6. NP-complete problems are the hardest problems in the NP set. A decision problem
L is NP-complete if:
1) L is in NP (Any given solution for NP-complete problems can be verified
quickly, but there is no efficient known solution).
2) Every problem in NP is reducible to L in polynomial time (Reduction is defined
below).
7. A problem is NP-Hard if it follows property 2 mentioned above, and doesn’t need to
follow property 1. Therefore, the NP-Complete set is also a subset of the NP-Hard
set.
approximation Algorithms for NP -Hard Problems

In this section, we discuss a different approach to handling difficult problems of

combinatorial optimization, such as the traveling salesman problem and the knapsack
problem. As we pointed out in Section 11.3, the decision versions of these problems are NP-
complete. Their optimization versions fall in the class of NP-hard problems—problems that
are at least as hard as NP-complete problems.2 Hence, there are no known polynomial-time
algorithms for these problems, and there are serious theoretical reasons to believe that such
algorithms do not exist. What then are our options for handling such problems, many of
which are of significant practical importance?
If an instance of the problem in question is very small, we might be able to solve it by an
exhaustive-search algorithm (Section 3.4). Some such problems can be solved by the
dynamic programming technique we demonstrated in Section 8.2. But even when this
approach works in principle, its practicality is limited by dependence on the instance
parameters being relatively small. The discovery of the branch-and-bound technique has
proved to be an important breakthrough, because this technique makes it possible to solve
many large instances of difficult optimization problems in an acceptable amount of time.
However, such good performance cannot usually be guaranteed.

Travelling Salesman Problem

The travelling salesman problem is a graph computational problem where the salesman needs
to visit all cities (represented using nodes in a graph) in a list just once and the distances
(represented using edges in the graph) between all these cities are known. The solution that is
needed to be found for this problem is the shortest possible route in which the salesman visits
all the cities and returns to the origin city.
If you look at the graph below, considering that the salesman starts from the vertex ‘a’, they
need to travel through all the remaining vertices b, c, d, e, f and get back to ‘a’ while making
sure that the cost taken is minimum.
There are various approaches to find the solution to the travelling salesman problem: naïve
approach, greedy approach, dynamic programming approach, etc. In this tutorial we will be
learning about solving travelling salesman problem using greedy approach.
Travelling Salesperson Algorithm
As the definition for greedy approach states, we need to find the best optimal solution locally
to figure out the global optimal solution. The inputs taken by the algorithm are the graph G
{V, E}, where V is the set of vertices and E is the set of edges. The shortest path of graph G
starting from one vertex returning to the same vertex is obtained as the output.
Algorithm
 Travelling salesman problem takes a graph G {V, E} as an input and declare
another graph as the output (say G’) which will record the path the salesman is
going to take from one node to another.
 The algorithm begins by sorting all the edges in the input graph G from the
least distance to the largest distance.
 The first edge selected is the edge with least distance, and one of the two
vertices (say A and B) being the origin node (say A).
 Then among the adjacent edges of the node other than the origin node (B), find
the least cost edge and add it onto the output graph.
 Continue the process with further nodes making sure there are no cycles in the
output graph and the path reaches back to the origin node A.
 However, if the origin is mentioned in the given problem, then the solution
must always start from that node only. Let us look at some example problems
to understand this better.

Introduction to Amortized Analysis

 Amortized Analysis is used for algorithms where an occasional operation is very
slow, but most of the other operations are faster. In Amortized Analysis, we analyze a
sequence of operations and guarantee a worst-case average time that is lower than the
worst-case time of a particularly expensive operation.
The example data structures whose operations are analyzed using Amortized Analysis
are Hash Tables, Disjoint Sets, and Splay Trees.
 Amortized analysis is a technique used in computer science to analyze the average-
case time complexity of algorithms that perform a sequence of operations, where
some operations may be more expensive than others. The idea is to spread the cost of
these expensive operations over multiple operations, so that the average cost of each
operation is constant or less.
 For example, consider the dynamic array data structure that can grow or shrink
dynamically as elements are added or removed. The cost of growing the array is
proportional to the size of the array, which can be expensive. However, if we amortize
the cost of growing the array over several insertions, the average cost of each
insertion becomes constant or less.
 Amortized analysis provides a useful way to analyze algorithms that perform a
sequence of operations where some operations are more expensive than others, as it
provides a guaranteed upper bound on the average time complexity of each operation,
rather than the worst-case time complexity.
 Amortized analysis is a method used in computer science to analyze the average
performance of an algorithm over multiple operations. Instead of analyzing the
worst-case time complexity of an algorithm, which gives an upper bound on the
 running time of a single operation, amortized analysis provides an average-case
analysis of the algorithm by considering the cost of several operations performed over
time.
 The key idea behind amortized analysis is to spread the cost of an expensive
operation over several operations. For example, consider a dynamic array data
structure that is resized when it runs out of space. The cost of resizing the array is
expensive, but it can be amortized over several insertions into the array, so that the
average time complexity of an insertion operation is constant.
 Amortized analysis is useful for designing efficient algorithms for data structures
such as dynamic arrays, priority queues, and disjoint-set data structures. It provides a
guarantee that the average-case time complexity of an operation is constant, even if
some operations may be expensive.