DAA Unit -5

Branch and Bound

Branch and Bound is another method to systematically search a solution space. Just like backtracking, we
will use bounding functions to avoid generating subtrees that do not contain an answer node. However
branch and Bound differs from backtracking in two important manners:
 It has a branching function, which can be a depth first search, breadth first search or based on
bounding function.
 It has a bounding function, which goes far beyond the feasibility test as a mean to prune efficiently
the search tree.
The term branch-and-bound refers to all state space search methods in which all children of the E-node
are generated before any other live node can become the E-node.
In branch-and bound terminology,
 A BFS-like state space search will be called FIFO (First in First Out) search as the list of live node sis
a first-in-first-out list (or queue).
 A D-search-like state space search will be called LIFO (Last in First Out) search as the list of live
nodes is a last-in-first-out list (or stack)
Live node is a node that has been generated but whose children have not yet been generated.
E-node is a live node whose children are currently being explored. In other words, an E-node is a node
currently being expanded.
Dead node is a generated node that is not to be expanded or explored any further. All children of a dead
node have already been expanded.
Least Cost (LC) search:
 In both LIFO and FIFO Branch and Bound the selection rule for the next E-node in rigid and blind.
The selection rule for the next E-node does not give any preference to a node that has a very good
chance of getting the search to an answer node quickly.
 The search for an answer node can be speeded by using an “intelligent” ranking function Ĉ( .) for live
nodes. The next E-node is selected on the basis of this ranking function. The node x is assigned a rank
using:

where,
Ĉ( x ) is the cost of x.
h(x) is the cost of reaching x from the root and f(.) is any non-decreasing function.
g ( x ) is an estimate of the additional effort needed to reach an answer node from x.
 A search strategy that uses a cost function to select the next E-node would
always choose for its next E-node a live node with least LC–search (Least Cost search).
 BFS and D-search are special cases of LC-search. If g( x ) = 0 and f(h(x)) = level of node x, then an
LC search generates nodes by levels. This is eventually the same as a BFS. If f(h(x))=0 and essentially
a D-search. g( x ) > g( y ) whenever y is a child of x, then the search is essentially a D-search.
 An LC-search coupled with bounding functions is called an LC-branch and bound search
Control Abstractions for LC-Search

----------------------------------------------------------------------------------------------------------------------------- -
0/1 Knapsack Problem (LC Branch and Bound)
 We are given n number of objects with weights Wi and profits Pi where i varies from 1 to n and also
a knapsack capacity M.
 The problem is to fill the knapsack with the help of n objects and the resulting profits has to be
maximum.
 This problem is similar to ordinary knapsack problem but we may not take a fraction of an object.
 The problem can be stated as
The terminologies used in this method are,
 Branching: Covering the feasible region by several smaller feasible sub-regions.
 Bounding: It is fast way of finding upper and lower bounds.
Procedure:
1. Draw a state space tree and upper = ∞
2. Compute Ĉ(x), u(x) for each node.
3. U(x) = -∑pi

𝑀 − 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑤𝑒𝑖𝑔ℎ𝑡
Ĉ(x) = 𝑈 (𝑥) − ∗ actual profit of remaining object
𝐴𝑐𝑡𝑢𝑎𝑙 𝑤𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑟𝑒𝑚𝑎𝑖𝑛𝑖𝑛𝑔 𝑜𝑏𝑗𝑒𝑐𝑡

4. If U(x) is minimum than upper, then upper will be set to u(x)

5. If Ĉ(x) >upper, kill node x.
6. Next live node becomes E-node and generate children for E-node.
7. Repeat steps 2 to 6 until all the nodes gets covered.
8. The minimum cost Ĉ(x) becomes the answer node. Trace the path backward direction towards x to
root for solution subset.

Example:
 Consider the knapsack instance n = 4, (p1,p2,p3,p4) = (10,10,12,18), (w1,w2,w3,w4) = (2,4,6,9),
and m = 15.
 Let us trace the working of an FIFO branch-and-bound search using Ĉ(x) and u(x) as defined
previously
 The search begins with the root as the E-node. For this node1, we have Ĉ(1)=-38 and
u(1)=-32 and queue of live nodes is empty.
 After the completion of state space tree the only remaining live node is node 8. It has no children and
the search terminates.
 The value of upper and the path from node 8 to the root is the output.
 So, the objects can be filled into the knapsack are, (1, 1, 0, 1) and the profit is 38.
----------------------------------------------------------------------------------------------------------------------------- -
0/1 Knapsack Problem (FIFO Branch and Bound)
 We are given n number of objects with weights Wi and profits Pi where i varies from 1 to n and also
a knapsack capacity M.
 The problem is to fill the knapsack with the help of n objects and the resulting profits has to be
maximum.
 This problem is similar to ordinary knapsack problem but we may not take a fraction of an object.
 The problem can be stated as
Procedure:
1. Draw a state space tree and upper = ∞
2. Compute Ĉ(x), u(x) for each node.
3. U(x) = -∑pi

𝑀 − 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑤𝑒𝑖𝑔ℎ𝑡
Ĉ(x) = 𝑈 (𝑥) − ∗ actual profit of remaining object
𝐴𝑐𝑡𝑢𝑎𝑙 𝑤𝑒𝑖𝑔ℎ𝑡 𝑜𝑓 𝑟𝑒𝑚𝑎𝑖𝑛𝑖𝑛𝑔 𝑜𝑏𝑗𝑒𝑐𝑡

4. If U(x) is minimum than upper, then upper will be set to u(x)

5. If Ĉ(x) >upper, kill node x.
6. Next live node becomes E-node and generate children for E-node.
7. Repeat steps 2 to 6 until all the nodes gets covered.
8. The minimum cost Ĉ(x) becomes the answer node. Trace the path backward direction towards x to
root for solution subset.

Example:
 Consider the knapsack instance n = 4, (p1,p2,p3,p4) = (10,10,12,18), (w1,w2,w3,w4) = (2,4,6,9),
and m = 15.
 Let us trace the working of an FIFO branch-and-bound search using Ĉ(x) and u(x) as defined
previously
 The search begins with the root as the E-node. For this node1, we have Ĉ(1)=-38 and
u(1)=-32 and queue of live nodes is empty.
 After the completion of state space tree the only remaining live node is node 12. It has no children
and the search terminates.
 The value of upper and the path from node 12 to the root is the output.
 So, the objects can be filled into the knapsack are, (1, 1, 0, 1) and the profit is 38.
----------------------------------------------------------------------------------------------------------------------------- -
Travelling salesperson Problem (Branch and Bound)
 Let G =(V,E) be a directed graph with edge costs cij. The variables cij is defined such that cij >0 for all
I and j and cij = ∞ if (i,j) Є E. Let |V| = n and assume n>1. A tour of G is a directed simple cycle that
includes every vertex in V.
 The cost of tour is the sum of the cost of the edges on the tour. The travelling salesperson problem is
to find a tour of minimum cost.
 The problem asks to find the shortest tour through a given set of n cities that visits each city exactly
once before returning to the city where it started.
 The time complexity of traveling sale person problem using LC branch and bound is O(n22n).
Procedure for solving traveling sale person problem:
 Reduce the given cost matrix. A matrix is reduced if every row and column is reduced. A row
(column) is said to be reduced if it contain at least one zero and all-remaining entries are non-
negative. This can be done as follows:
 a) Row reduction: Take the minimum element from first row, subtract it from all elements of first
row, next take minimum element from the second row and subtract it from second row. Similarly
apply the same procedure for all rows.
b) Find the sum of elements, which were subtracted from rows.
c) Apply column reductions for the matrix obtained after row reduction.
Column reduction: Take the minimum element from first column, subtract it from all elements of
first column, next take minimum element from the second column and subtract it from second
column. Similarly apply the same procedure for all columns.
d) Find the sum of elements, which were subtracted from columns.
e) Obtain the cumulative sum of row wise reduction and column wise reduction.
Cumulative reduced sum = Row wise reduction sum + column wise reduction sum.
Associate the cumulative reduced sum to the starting state as lower bound and ∞ as upper bound
Example:
Consider the following cost matrix of a graph. The matrix shows the cost matrix and reduced cost
matrix.

State space tree generated by procedure LCBB is

The Reduced cost matrices for the corresponding nodes are

The solution node, node11 is generated. The tour length for this node is c(ll)= 28 and upper is updated to
28. The cost of the remaining leaf nodes are greater than upper. Hence, LCBB terminates with 14
2 5 3 1 as the shortest length tour with the cost 28.
----------------------------------------------------------------------------------------------------------------------------- -
Deterministic and Non - deterministic Algorithms
 The algorithms in which the result of every algorithm is uniquely defined are known as
the deterministic algorithm. In the theoretical framework, we can remove this restriction on the
outcome of every operation. We can allow algorithms to contain operations whose outcomes are not
uniquely defined but are limited to specified sets of possibilities.
 Deterministic algorithms can be defined in terms of a state machine: a state describes what a machine
is doing at a particular instant in time. State machines pass in a discrete manner from one state to
another. Just after we enter the input, the machine is in its initial state or start state. If the machine is
deterministic, this means that from this point onwards, its current state determines what its next state
will be; its course through the set of states is predetermined.

 The machine executing each operation is allowed to choose any one of these outcomes subjects to a
determination condition to be defined later. This leads to the concept of a Non-deterministic algorithm.
 The algorithm to contain operations whose outcomes are not uniquely defined but are limited to
specified set of possibilities are non-deterministic algorithms.
 There are three new functions which specify such types of algorithms are:
o Choice(S) arbitrarily chooses one of the elements of the set S.
o Failure() signals an unsuccessful completion.
o Success() signals a successful completion.
 The assignments statement x: Choice (1, n) could result in x being assigned any one of the integers in
the range [1, n]. There is no rule specifying how this choice is to be made.
 The Failure () and the Success() signals are used to define a computation of the algorithm. These
statements cannot be used to effect a return.
 Whenever there is a set of the choices that lead to a successful completion, then one such set of the
choices is always made and the algorithm terminates successfully.
 A non - deterministic algorithm terminates unsuccessfully if and only if there exists no set of the
choices leading to a success signal. The computing times for the Choices, the Success, and the Failure
are taken to be O (1).
 A machine capable of executing a non - deterministic algorithm in this way is called a non –
deterministic machine.
Example1:
Consider the problem of searching for an element x in a given set of elements A[1:n],n ≥ 1.We are required
to determine an index j such that A[j] = x or j = 0 if x is not in A. A non-deterministic algorithm for this
is

Example2:
Let A[i], 1≤i ≤n, be an unsorted array of positive integers. The non-deterministic algorithm NSort(A,n)
sorts the numbers into non-decreasing order.
The algorithm is,

----------------------------------------------------------------------------------------------------------------------------- -
P, NP, NP-hard and NP-complete class problems
P Problems: P is the set of all the decision problems solvable by deterministic algorithms in polynomial
time.
NP Problems: NP is the set of all the decision problems that are solvable by non - deterministic algorithms
in polynomial time.
Since deterministic algorithms are just the special case of non - deterministic ones, so we can conclude
that P is the subset of NP.
 Relation between P and NP

NP Hard Problem
 A problem L is the NP hard if and only if satisfiability reduces to L. A problem is NP complete if and
only if L is the NP hard and L belongs to NP.
 Only a decision problem can be NP complete. However, an optimization problem may be the NP hard.
Furthermore if L1 is a decision problem and L2 an optimization problem, then it is possible that L1 α
L2. One can trivially show that the knapsack decision problem reduces to knapsack optimization
problem. In fact, one can also show that these optimization problems reduce to their corresponding
decision problems.
 Yet, optimization problems cannot be NP-complete whereas decision problems can. There also exist
NP-hard decision problems that are not NP-complete.

NP Completeness Problem
 Polynomial time reductions provide a formal means for showing that one problem is at least as hard
as another, within a polynomial time factor. This means, if L1 <= L2, then L1 is not more than a
polynomial factor harder than L2. Which is why the “less than or equal to” notation for reduction is
mnemonic. NP complete are the problems whose status are unknown.
 Some of the examples of NP complete problems are:
1. Travelling Salesman Problem:
Given n cities, the distance between them and a number D, does exist a tor programme for a salesman
to visit all the cities so that the total distance travelled is at most D.
 2. Zero One Programming Problem:
Given m simultaneous equations,

3. Satisfiability Problem:
Given a formula that involves propositional variables and logical connectives.
A language L is the subset [0, 1]* is NP complete if,
L belongs to NP and
L' ← L for every L' belongs to NP
All NP complete problems are NP hard, but some NP hard problems are not known to be NP
complete.
If NP hard problems can be solved in polynomial time, then all the NP complete problems can be
solved in polynomial time.
----------------------------------------------------------------------------------------------------------------------------- -
Cook’s theorem
Cook’s theorem states that satisfiability is in P if and only if P=NP.
Hence if P=NP, then satisfiability is in P. It remains to be shown that if satisfiability is in P, then P=NP.
----------------------------------------------------------------------------------------------------------------------------- -