DESIGN AND ANALYSIS OF ALGORITHMS

UNIT V BRANCH AND BOUND

BRANCH AND BOUND

A B&B algorithm searches the complete space of solutions for a given problem for the best solution.
Ø Branch and bound is a systematic method for solving optimization problems
Ø B&B is a rather general optimization technique that applies where the greedy method and
dynamic programming fail.
Ø However, it is much slower. Indeed, it often leads to exponential time complexities in the
worst case.
Ø On the other hand, if applied carefully, it can lead to algorithms that run reasonably fast on
average.
Ø The general idea of B&B is a BFS-like search for the optimal solution, but not all nodes get
expanded (i.e., their children generated). Rather, a carefully selected criterion determines
which node to expand and when, and another criterion tells the algorithm when an optimal
solution has been found.

1. O/I Knapsack problem

Consider of size K and we want to select from a set of n objects , where the ith object has size si and
value vi, a subset of these objects to maximize the value contained in the knapsack with the contents
of the knapsack less than or equal to K.

Suppose that K = 16 and n = 4, and we have the following set of objects ordered by their value
density.

i vi si vi/si
1 $45 3 $15
2 $30 5 $ 6
3 $45 9 $ 5
4 $10 5 $ 2

We will construct the state space where each node contains the total current value in the knapsack,
the total current size of the contents of the knapsack, and maximum potential value that the
knapsack can hold. In the algorithm, we will also keep a record of the maximum value of any node
(partially or completely filled knapsack) found so far. When we perform the depth-first traversal of
the state-space tree, a node is "promising" if its maximum potential value is greater than this current
best value.

We begin the state space tree with the root consisting of the empty knapsack. The current weight
and value are obviously 0. To find the maximum potential value we treat the problem as if it were
the fractional knapsack problem and we were using the greedy algorithmic solution to that
problem. We have shown that the greedy approach to the fractional knapsack problem yields an

Design and Analysis of Algorithms-P.Vasantha Kumari 1

optimal solution. We place each of the remaining objects, in turn, into the knapsack until the next
selected object is too big to fit into the knapsack. We then use the fractional amount of that object
that could be placed in the knapsack to determine the maximum potential value.

totalSize = currentSize + size of remaining objects that can be fully placed

bound (maximum potential value) = currentValue + value of remaining objects

fully placed + (K - totalSize) * (value density of item that is partially
placed)

In general, for a node at level i in the state space tree (the first i items have been considered for
selection) and for the kth object as the one that will not completely fit into the remaining space in
the knapsack, these formulae can be written:

k-1
totalSize = currentSize + S sj
j=i+1
k-1
bound = currentValue + S vj + (K - totalSize) * (vk/sk)
j=i+1

For the root node, currentSize = 0, currentValue = 0

totalSize = 0 + s1 + s2 = 0 + 3 + 5 = 8

bound = 0 + v1 + v2 + (K - totalSize) * (v3/s3) = 0 + $45 + $30 + (16 - 8) * ($5) = $75 +

$40 = $115

The computation of the bound and the selection criteria for promising nodes is the same as
before. We must replace the depth-first traversal of the state space tree with a breadth first
traversal. In the depth-first traversal the auxiliary data structure used to store Nodes was
(implicitly) the stack. In breath-first traversal, the auxiliary data structure is explicitly the queue.

Suppose n = 4, W= 16, and we have the following:

i pi wi pi / wi
1 $40 2 $20
2 $30 5 $6
3 $50 10 $5
4 $10 5 $2

Set maxprofit to $0 (Remember n = 4, W = 16 )

2) Visit node (0,0)
a) Compute its profit and weight : profit = $ 0, weight = 0
b) Compute its bound.
Since w1 + w2+ w3 = 2 + 5 + 10 = 17 > 16. Therefore k = 3 ( to keep the sum of the weights <=
16).totweight = weight + w1 + w2 = 0 + 2 + 5 = 7
bound = profit + p1 + p2 + (W - totweight )
p3 / w3= $0 + $40 + $30 + (16 -7) $50/10 =$115

Design and Analysis of Algorithms-P.Vasantha Kumari 2

c) It is promising because its weight =0 < W = 16 and its bound $115 > 0 the value of maxprofit.

2. Traveling Salesperson Problem

Traveling salesperson problem is to find the shortest path in directed graph that starts at a given
vertex, visits each vertex in the graph exactly once. Such a path is called an optimal tour.
Construct the state-space tree:
Ø A node = a vertex: a vertex in the graph.
Ø A node that is not a leaf represents all the tours that start with the path stored at that node;
each leaf represents a tour (or non-promising node).
Ø Branch-and-bound: we need to determine a lower bound for each node
Ø Expand each promising node, and stop when all the promising nodes have been expanded.
During this procedure, prune all the nonpromising nodes.
– Promising node: the node’s lower bound is less than current minimum tour
length.
– Non-promising node: the node’s lower bound is NO less than current
minimum tour length.

Ø Because a tour must leave every vertex exactly once, a lower bound on the length of a tour is
b (lower bound) minimum cost of leaving every vertex.
o The lower bound on the cost of leaving vertex v1 is given by the minimum of all the
nonzero entries in row 1 of the adjacency matrix.
o …

Design and Analysis of Algorithms-P.Vasantha Kumari 3

 o The lower bound on the cost of leaving vertex vn is given by the minimum of all the
nonzero entries in row n of the adjacency matrix.
Ø Note: This is not to say that there is a tour with this length. Rather, it says that there can be
no shorter tour.
Ø Assume that the tour starts with v1.
Ø Because every vertex must be entered and exited exactly once, a lower bound on the length
of a tour is the sum of the minimum cost of entering and leaving every vertex.
o For a given edge (u, v), think of half of its weight as the exiting cost of u, and half of
its weight as the entering cost of v.
o The total length of a tour = the total cost of visiting( entering and exiting) every
vertex exactly once.
o The lower bound of the length of a tour = the lower bound of the total cost of visiting
(entering and exiting ) every vertex exactly once.
Calculation:
v for each vertex, pick top two shortest adjacent edges (their sum
divided by 2 is the lower bound of the total cost of entering and
exiting the vertex);
v add up these summations for all the vertices.
Ø Assume that the tour starts with vertex a and that b is visited before c.

Algorithm TSP

1. First, find out all (n -1)! Possible solutions, where n is the number of cities.
2. Next, determine the minimum cost by finding out the cost of everyone of these
(n -1)! Solutions.
3. Finally, keep the one with the minimum cost.

Design and Analysis of Algorithms-P.Vasantha Kumari 4

Example :

Input
Number of cities n
Cost of traveling between the cities.
c (i, j) i, j = 1, . . , n.

Design and Analysis of Algorithms-P.Vasantha Kumari 5

 Start with city 1

Output
Vector of cities and total cost.

Main Steps
Initialization
c← 0
Cost ← 0
visits ← 0
e = 1 /* pointer of the visited city */
For 1 ≤ r ≤ n
Do {
Choose pointer j with
minimum = c (e, j) = min{c (e, k); visits (k) = 0 and 1 ≤ k ≤ n }
cost ← cost + minimum - cost
e=j
C(r) ← j
C(n) = 1
cost = cost + c (e, 1)

3. NP Hard and NP Complete problems

NP stands for Non-deterministic Polynomial time. This means that the problem can be solved in
Polynomial time using a Non-deterministic Turing machine (like a regular Turing machine but also
including a non-deterministic "choice" function). Basically, a solution has to be testable in poly
time. If that's the case, and a known NP problem can be solved using the given problem with
modified input (an NP problem can be reduced to the given problem) then the problem is NP
complete.

Design and Analysis of Algorithms-P.Vasantha Kumari 6

The main thing to take away from an NP-complete problem is that it cannot be solved in polynomial
time in any known way. NP-Hard/NP-Complete is a way of showing that certain classes of
problems are not solvable in realistic time.

NP-Complete means something very specific and you have to be careful or you will get the
definition wrong. First, an NP problem is a yes/no problem such that

1. There is polynomial-time proof for every instance of the problem with a "yes" answer that
the answer is "yes", or (equivalently)
2. There exists a polynomial-time algorithm (possibly using random variables) that has a non-
zero probability of answering "yes" if the answer to an instance of the problem is "yes" and
will say "no" 100% of the time if the answer is "no." In other words, the algorithm must
have a false-negative rate less than 100% and no false positives.

A problem X is NP-Complete if

1. X is in NP, and
2. For any problem Y in NP, there is a "reduction" from Y to X: a polynomial-time algorithm
that transforms any instance of Y into an instance of X such that the answer to the Y-
instance is "yes" if and only if the answer X-instance is "yes".

What is NP?

NP is the set of all decision problems (question with yes-or-no answer) for which the 'yes'-answers
can be verified in polynomial time (O(n^k) where n is the problem size, and k is a constant) by a
deterministic Turing machine. Polynomial time is sometimes used as the definition of fast or quickly.

What is P?

P is the set of all decision problems which can be solved in polynomial time by a deterministic
Turing machine. Since it can solve in polynomial time, it can also be verified in polynomial time.
Therefore P is a subset of NP.

What is NP-Complete?

A problem x that is in NP is also in NP-Complete if and only if every other problem in NP can be
quickly (ie. in polynomial time) transformed into x. In other words:

1. x is in NP, and
2. Every problem in NP is reducible to x

What is NP-Hard?

NP-Hard are problems that are at least as hard as the hardest problems in NP. Note that NP-
Complete problems are also NP-hard. However not all NP-hard problems are NP (or even a
decision problem), despite having 'NP' as a prefix. That is the NP in NP-hard does not mean 'non-

Design and Analysis of Algorithms-P.Vasantha Kumari 7

deterministic polynomial time'. Yes this is confusing but its usage is entrenched and unlikely to
change.

Basic concepts

• Solvability of algorithms
– There are algorithms for which there is no known solution, for example, Turing’s Halting
Problem
Decision problem
Given an arbitrary deterministic algorithm A and a finite input I
Will A with input I ever terminate, or enter an infinite loop?
– Halting problem cannot be solved by any computer, no matter how much time is provided
In algorithmic terms, there is no algorithm of any complexity to solve this problem

• Efficient algorithms
– Efficiency measured in terms of speed
– For some problems, there is no known efficient solution
– Distinction between problems that can be solved in polynomial time and problems for
which no polynomial time algorithm is known

• Problems classified to belong to one of the two groups

1. Problems with solution times bound by a polynomial of a small degree
– Most searching and sorting algorithms
– Also called tractable algorithms
2. Problems with best known algorithms not bound by a polynomial
– Hard, or intractable, problems
– Traveling salesperson (O(n22n)), knapsack (O(2n/2))
– None of the problems in this group has been solved by any polynomial time algorithm
– NP-complete problems

• Theory of NP-completeness
– Show that many of the problems with no polynomial time algorithms are computationally related
– The group of problems is further subdivided into two classes
NP-complete. A problem that is NP-complete can be solved in polynomial time iff all other
NP-complete problems can also be solved in polynomial time
NP-hard. If an NP-hard problem can be solved in polynomial time then all NP-complete
problems can also be solved in polynomial time
– All NP-complete problems are NP-hard but some NP-hard problems are known not to be NP-
complete

Summary

Ø Branch and bound is a systematic method for solving optimization problems. B&B is a
rather general optimization technique that applies where the greedy method and dynamic
programming fail.

Design and Analysis of Algorithms-P.Vasantha Kumari 8

 Ø Traveling salesperson problem is to find the shortest path in directed graph that starts at a
given vertex, visits each vertex in the graph exactly once. Such a path is called an optimal
tour.
Ø NP stands for Non-deterministic Polynomial time. This means that the problem can be
solved in Polynomial time using a Non-deterministic.
Ø NP is the set of all decision problems (question with yes-or-no answer) for which the 'yes'-
answers can be verified in polynomial time (O(n^k) where n is the problem size, and k is a
constant) by a deterministic Turing machine. Polynomial time is sometimes used as the
definition of fast or quickly.
Ø P is the set of all decision problems which can be solved in polynomial time by a
deterministic Turing machine. Since it can solve in polynomial time, it can also be verified
in polynomial time. Therefore P is a subset of NP.
Ø A problem x that is in NP is also in NP-Complete if and only if every other problem in NP
can be quickly (ie. in polynomial time) transformed into x.
Ø NP-Hard are problems that are at least as hard as the hardest problems in NP. Note that NP-
Complete problems are also NP-hard. However not all NP-hard problems are NP (or even a
decision problem), despite having 'NP' as a prefix. That is the NP in NP-hard does not mean
'non-deterministic polynomial time'. Yes this is confusing but its usage is entrenched and
unlikely to change.

Key Terms
>> Branch and Bound >> 0/1 Knapsack
>> Traveling Salesman Problem >> NP-completeness
>> NP-hard

Key Term Quiz

1. -------------------- is a systematic method for solving optimization problems. B&B is a rather
general optimization technique that applies where the greedy method and dynamic
programming fail.
2. -------------------- problem is to find the shortest path in directed graph that starts at a given
vertex, visits each vertex in the graph exactly once. Such a path is called an optimal tour.
3. ------------------ can be solved in Polynomial time using a Non-deterministic

Objective Type Questions

1. Traveling salesman problem
(a)All the vertices have to be traversed only once with maximum cost
(b)The vertices have to be traversed only once with minimum cost and return to the starting
vertex.
(c)The vertices have to be covered only once without considering the cost
(d)All are true

2. Methods that are used to solve TSP

(a)Approximation algorithms (b) Branch and bound
(c)Dynamic programming (d) All the above

Design and Analysis of Algorithms-P.Vasantha Kumari 9

 3. NP hard problems

(a)Difficult combinatorial problems for salvation

(b)Easiest problems that can be solved
(c)The optimization version of the combinatorial problems
(d)None of the above

4. NP complete problems
(a)Belongs to class NP and is polynomial reducible
(b)Belongs to class P and is polynomial reducible
(c)Not belongs to class NP and polynomial reducible
(d)None of the above

5. Class NP problems
(a)Decision problems that are solved by deterministic polynomial algorithm
(b)Decision problems that are solved by nondeterministic polynomial algorithm
(c)Both are true
(d)None of the above

6. Class P problems
(a)Decision problems that are solved by nondeterministic polynomial algorithm
(b)Comes under NP category
(c)Problems that are solved by deterministic algorithms
(d)None of the above

7. Knapsack problem can be solved by using

(a)Greedy technique (b) Dynamic Programming
(c)Backtracking (d)All the above

8. The two types of knapsack problems are

(a)Discrete and continuous (b) Non discrete and non continuous
(c) Non Discrete and continuous (d) None of the above

9. The minimum number of queens need to solve the n queen problem properly is
(a)1 (b)2 (c)3 (d)4 (Ans)4

10. N queen problem is solved by

(a)Greedy technique (b) Dynamic Programming
(c)Backtracking (d)All the above

Review Questions
Two Mark Questions

1. Define branch and bound problem

2. What do you mean by 0/1 knapsack problem

Design and Analysis of Algorithms-P.Vasantha Kumari 10

 3. Define NP-completeness
4. Define TSP problem
5. Compare branch and bound and backtracking
6. Define NP-hard problem

Big Questions
1. Explain briefly branch and bound technique for solving problems.
2. job 1 job 2 job 3 job 4
a 9 2 7 8
b 6 4 3 7
c 5 8 1 8
d 7 6 9 4
Consider the above matrix for assignment problem involving persons and jobs. Explain in
detail how branch and bound technique is useful is solving assignment problems
3. Write an algorithm for 0/1-knapsack problem.
4. Write the three reasons to terminate a search path at the current node in a state-space tree of
a branch-and-bound algorithm.
5. Apply the branch-and-bound technique in solving the Traveling Salesman Problem.
6. Solve the following instance of the knapsack problem by the branch-and-bound algorithm,
with W = 16. (8)
Item Weight Value
1 10 100
2 7 63
3 8 56
4 4 12
7. Give short notes on decision problems, undecidable problem, and NP-Complete problem.

Lesson Lab
1. Apply TSP and calculate the shortest path for the following graph

-------------- END OF FIFTH UNIT ---------------

Design and Analysis of Algorithms-P.Vasantha Kumari 11