CS 306 DAA, IIITDM Jabalpur

11/10/2012

Combinatorial Search and Optimization Backtracking and Branch & Bound

Combinatorial Search Problems

By default, they are large problems whose brute-force solutions take exponential time
Can be solved by generating a search space (search tree) and finding the solution

Goal: Find a solution node (object) such that

It satisfies certain constraints It maximize (or minimize) certain objective function

CS 306 DAA, IIITDM Jabalpur

11/10/2012

CSP: Examples
8-Queens Knapsack (0/1) TSP LCS (Longest Common Subsequence) Edit Distances LIS (Longest Increasing Subsequence) Shortest Paths Matrix Chain Multiplications

Solving Hard CSP

Intelligent exponential search
Backtracking (Backtrack Search)
Brute force approach Faces Combinatorial explosion

Branch & Bound

Improvement on Backtrack Search by pruning the search tree

Greedy (Ex: Vertex Cover) Dynamic Programming (Ex: TSP) Linear Programming

CS 306 DAA, IIITDM Jabalpur

11/10/2012

Backtracking
Idea: it is often possible to reject a solution by looking at just a small portion of it
Ex: SAT, if (x1 V x2) is a clause, then x1=x2=0 (i.e. false) can be instantly eliminated In other word, of the search space can straight be eliminated

A promising direction, but can it be systematically exploited?

SAT

CS 306 DAA, IIITDM Jabalpur

11/10/2012

SAT

8-Queens
Placing 8 queens on the chessboard such that no two queens do not attack each other

A SOLUTION ?

CS 306 DAA, IIITDM Jabalpur

11/10/2012

Backtracking: 4-Queens
*

* *

* *

This amount of further search can be avoided Further, so we backtrack to other branches

Backtracking vs. B&B

Backtracking is a general search strategy and hence align nicely with search problems For optimization problems, we can further improve the backtrack search with a (minimum) bound value Branch & Bound

CS 306 DAA, IIITDM Jabalpur

11/10/2012

B&B: TSP
2
10 2 1

10 1

15

1 2 2 15 1 2 3 1

No Solution better than 7 is possible here. So backtrack

A Solution here is = 7
3

B&B on 0/1 Knapsack An Example

Items 1 2 3 4 Values 20 3 7 5 Weights 5 2 4 1 C=8

0/1 Knapsack is an optimization problem However, its a maximization problem (NOT MINIMIZATION) So, how to use B&B to prune the search?
The Idea: Convert maximization problem into minimization problem!

CS 306 DAA, IIITDM Jabalpur

11/10/2012

B&B on 0/1 Knapsack An Example

Items 1 2 3 4 Values 20 3 7 5 Weights 5 2 4 1 C = Knapsack size = 8 OF- Objective Function C = Knapsack size to be filled { }, {1,2,3,4}, C=8, OF = 0
1st Item included 1st Item NOT included

{ 1}, {2,3,4}, C=3, OF = 0

2nd Item NOT included

{ }, {2,3,4}, C=8, OF = 20

2nd Item included

{ 1,2}, {3,4}, C=1, OF = 0

3rd Item included

{ 1}, {3,4}, C=3

3rd Item NOT included

{ 1,2, 3}, {4}, C= -3, OF = 7

4th Item included

{ 1,2}, {3,4}, C=1

{ 1,2, 4}, { }, C= 0, OF = 7

Objective Function = the cost of the items rejected so far Hence, (minimize) objective function OF = 7 for this solution

TSP: Another Solution using B&B

Given a graph G(V, E) with starting node a Consider a path from a to another node b with some intermediate nodes S The cost of completing a partial tour [a, S, b] will be sum of the following:
1. The lightest edge from a to V - S. 2. The lightest edge from b to V - S. 3. The minimum spanning tree of V - S

CS 306 DAA, IIITDM Jabalpur

11/10/2012

TSP

Branch & Bound

Some important considerations
How to choose a suitable Objective Function?
A clever choice can results in substantial pruning of the solution space to look for

The order of selection of the objects to be included in the solution (?)

Ex: choosing an item with highest value, or value/weight ratio, etc.\

At a given node, which edge to explore first?