Branch & Bound

A new Algorithm Paradigm

Review of Backtracking
1. Construct the state-space tree

• nodes: partial solutions

• edges: choices in extending partial solutions

2. Explore the state space tree using depth-first search

3. “Prune” nonpromising nodes

• dfs stops exploring subtrees rooted at nodes that cannot
lead to a solution and backtracks to such a node’s
parent to continue the search.
 Introduction
 The branch-and-bound design strategy is very similar to
backtracking in that a state space tree is used to solve a
problem.

 The differences are that the branch-and-bound method

 1) does not limit us to any particular way of traversing the tree,
 2) and is used only for optimization problems.

 A branch-and-bound algorithm computes a number (bound)

at a node to determine whether the node is promising.
 Introduction …
 The number is a bound on the value of the solution
that could be obtained by expanding beyond the node.

 If that bound is no better than the value of the best

solution found so far, the node is nonpromising.
Otherwise, it is promising.

 The backtracking algorithm for the 0-1 Knapsack

problem is actually a branch-and-bound algorithm.

 A backtracking algorithm, however, does not exploit

the real advantage of using branch-and-bound.
 Introduction …

 Besides using the bound to determine whether a node

is promising, we can compare the bounds of promising
nodes and visit the children of the one with the best
bound.

 This approach is called best-first search with branch-

and-bound pruning..
0/1 KNAPSACK PROBLEM

Solve by Branch and Bound

 The Knapsack Problem

 Input
 Capacity K
 n items with weights wi and values vi
 Goal
 Output a set of items S such that
 the sum of weights of items in S is at most K
 and the sum of values of items in S is maximized
 Greedy Does Not Work

 What are possible greedy strategies?

 Highest Density First
 Highest Value First
 Lowest Weight First
 Prove these are not optimal for the 0-1 knapsack
problem.
Situation where dynamic programming
does not work
What if our problem can’t be described with integers?

wA = 2 vA = $40
wB =  vB = $50
wC = 1.98 vC = $100
wD = 5 vD = $95
wE = 3 vE = $30

We have to resort to brute force….

 Brute Force

 Generate all possible solutions

 With n items, there are 2n solutions to be generated.
 Check each to see if they satisfy the constraint.
 Save maximum solution that satisfies constraint.
 Can be represented as a tree.
 Brute Force: Branching
In A Out

B B

C C C C

D D D D D D D D

E E E E E E E E E E E E E E E E

Weight = 15.12 Weight = 8.98 Weight = 9.98

Value = $315 Value = $235 Value = $225
 Backtracking
 In the tree representation, we can think of the
previous algorithm doing a DFS in the tree.

 If we reach a point where a solution no longer is

feasible, there is no need to continue exploring.

 We can “backtrack” from this point and

potentially cut off much of the tree and many of
the solutions.

 In the given example, backtracking would be

much more effective if we had even more items
or a smaller knapsack capacity
 2, $40
, $50
Backtracking 1.98, $100
5, $95
In A 3, $30
Out

B B

C C C C

D D D D D D D D

E E E E E E E E E E E E E E E E

Weight = 8.98 Weight = 9.98

Value = $235 Value = $225
LC ALGORITHM(LEAST COST)

 This is a generic form/generalization of both the BFS and DFS

algorithms
 In LC algorithm a traversal is performed on the state space tree
 Each node is given a rank based on a minimization rule(also
known as best first rule) f(x)
(in case of the Knapsack problem the minimization rule can be
stated as f(x)=--g(x),where g(x) is the maximization rule for the
Knapsack problem)
 Once all nodes have been ranked eliminate/prune the nodes
with the poorest ranks.
 Repeat till a feasible solution is obtained
LC ALGORITHM for 0-1 knapsack problem
Example:

We have n=4 items. Their respective profits are (p1, p2, p3 p4 )={10, 10,
12, 18} and associated weights are (w1,w2,w3,w4) = {2,4,6,9}. Given
knapsack capacity is M=15 units.

We have to calculate cost(rank) and upper bound for each node of

state space tree.

While calculating cost we follow fractional knapsack and to calculate

upper bound we follow 0-1 knapsack problem.

The resultant state space tree is as follows:

So solution vector for given knapsack problem is X(k) = {1,1,0,1}
 Summary: Branch and bound

 Branch and Bound is:

 a general search method.
 minimize a function f(x), where x is restricted to some feasible
region.
 To apply branch and bound, one must have:
 a means of computing a lower bound on an instance of the
optimization problem.
 a way to compute an upper bound (feasible solution) for at least
some instances.
 Summary: Branch and bound
 The method starts by:
 considering the original problem with the complete feasible region (called the
root problem).

 The lower-bounding and upper-bounding procedures are applied to the root

problem.

 If the bounds match, then an optimal solution has been found. Otherwise, the
feasible region is divided into two or more regions, which together cover the
whole feasible region.

 These subproblems become children of the root search node.

 The algorithm is applied recursively to the subproblems, generating a tree of

subproblems.
 Summary: Branch and bound

 If an optimal solution is found to a subproblem,

 it is a feasible solution to the full problem, but not necessarily
globally optimal.
 Since it is feasible, it can be used to prune the rest of the tree:
 if the lower bound for a node exceeds the best known feasible solution,
no globally optimal solution can exist in the subspace of the feasible
region represented by the node.
 Therefore, the node can be removed from consideration.
 The search proceeds until all nodes have been solved or pruned, or
until some specified threshold is meet between the best solution
found and the lower bounds on all unsolved subproblems.
THANK YOU!!!!!