DESIGN AND ANALYSIS OF ALGORITHMS (III - CSE) – I SEM

UNIT IV

Backtracking: The General Method, The 8-Queens problem, sum of subsets, Graph coloring,
Hamiltonian cycles, knapsack problem.

Branch and Bound: FIFO Branch-and-Bound, LC Branch-and-Bound, 0/1 Knapsack problem,

Traveling salesperson problem.

Backtracking:

Backtracking is a technique based on algorithm to solve problem. It uses recursive calling to

find the solution by building a solution step by step increasing values with time. It removes the
solutions that doesn't give rise to the solution of the problem based on the constraints given to
solve the problem.

Backtracking algorithm is applied to some specific types of problems,

 Decision problem used to find a feasible solution of the problem.

 Optimisation problem used to find the best solution that can be applied.
 Enumeration problem used to find the set of all feasible solutions of the problem.
A backtracking algorithm need not actually create a tree. Rather, it only needs to keep track of
the values in the current branch being investigated. This is the way we implement backtracking
algorithm.

State space is the set of paths from root node to other nodes. State space tree is the tree
organization of the solution space. The state space trees are called static trees. This terminology
follows from the observation that the tree organizations are independent of the problem instance
being solved.

Terminology:
Problem state is each node in the depth first search tree.
Solution states are the problem states „S‟ for which the path from the root node to „S‟ defines a
tuple in the solution space.

UNIT-4 [1] K.PRASANTHI

Answer states are those solution states for which the path from root node to s defines a tuple that
is a member of the set of solutions.
Live node is a node that has been generated but whose children have not yet been generated.
E-node is a live node whose children are currently being explored. In other words, an E-node is a
node currently being expanded.
Dead node is a generated node that is not to be expanded or explored any further. All children of
a dead node have already been expanded.
Branch and Bound refers to all state space search methods in which all children of an E-node
are generated before any other live node can become the E-node.
Depth first node generation with bounding functions is called backtracking. State generation
methods in which the E-node remains the E-node until it is dead, lead to branch and bound
methods.

N-Queens Problem:

Let us consider, N = 8. Then 8-Queens Problem is to place eight queens on an 8 x 8 chessboard

so that no two “attack”, that is, no two of them are on the same row, column, or diagonal.

All solutions to the 8-queens problem can be represented as 8-tuples (x1, ......... , x8), where xi is
the column of the ith row where the ith queen is placed.

The promising function must check whether two queens are in the same column or diagonal:
Suppose two queens are placed at positions (i, j) and (k, l) Then:

Example:
Suppose we start with the feasible sequence 7, 5, 3, 1.

UNIT-4 [2] K.PRASANTHI

Step 1:
Add to the sequence the next number in the sequence 1, 2, . . . , 8 not yet used.
Step 2:
If this new sequence is feasible and has length 8 then STOP with a solution. If the new sequence
is feasible and has length less then 8, repeat Step 1.
Step 3:
If the sequence is not feasible, then backtrack through the sequence until we find the most recent
place at which we can exchange a value. Go back to Step 1.

UNIT-4 [3] K.PRASANTHI

4- Queens Problem using State space tree:

UNIT-4 [4] K.PRASANTHI

UNIT-4 [5] K.PRASANTHI
 SUM OF SUBSETS
Subset sum problem is to find subset of elements that are selected from a given set whose sum
adds up to a given number K. We are considering the set contains non-negative values. It is
assumed that the input set is unique (no duplicates are presented).
Given positive numbers wi, 1 ≤ i ≤ n, and m, this problem requires finding all subsets of wi
whose sums are „m‟.

All solutions are k-tuples, 1 ≤ k ≤ n.

{ 2, 9, 10, 1, 99, 3}
We need to find if there is a subset for a given sum say 4: { 1, 3 }

Steps:
1. Start with an empty set
2. Add the next element from the list to the set
3. If the subset is having sum M, then stop with that subset as solution.
4. If the subset is not feasible or if we have reached the end of the set, then backtrack
through the subset until we find the most suitable value.
5. If the subset is feasible (sum of seubset < M) then go to step 2.
6. If we have visited all the elements without finding a suitable subset and if no
backtracking is possible then stop without solution.
Example:
Consider the sum-of-subset problem, n = 4, Sum = 13, and w = (3, 4, 5, 6). Find a
solution to the problem using backtracking. Show the state-space tree leading to the solution.
Also, number the nodes in the tree in the order of recursion calls.

Set X = [0, 0, 0, 0]

Set Sum = 0. Sum indicates summation of selected numbers from W.

Step 1 : i = 1, Adding item wi

Sum = Sum + wi = Sum + w1 = 0 + 3 = 3

Sum ≤ M, so add item i to solution set.

UNIT-4 [6] K.PRASANTHI

X[i] = X[1] = 1 ⇒ X =[1, 0, 0, 0]

Step 2 : i = 2, Adding item w2

Sum = Sum + wi = Sum + w2 = 3 + 4 = 7

Sum ≤ M, so add item i to solution set.

X[i] = X[2] = 1 ⇒ X =[1, 1, 0, 0]

Step 3 : i = 3, Adding item w3

Sum = Sum + wi = Sum + w3 = 7 + 5 = 12

Sum ≤ M, so add item i to solution set.

X[i] = X[3] = 1 ⇒ X =[1, 1, 1, 0]

Step 4 : i = 4, Adding item w4

Sum = Sum + wi = Sum + w4 = 12 + 6 = 18

Sum > M, so backtrack and remove the previously added item from the solution set.

X[i] = X[3] = 0 ⇒ X =[1, 1, 0, 0].

Update Sum accordingly. So, Sum = Sum – w3 = 12 – 5 = 7

And don‟t increment i.

Step 5 : i = 4, Adding item w4

Sum = Sum + wi = Sum + w4 = 7 + 6 = 13

Sum = M, so solution is found and add item i to solution set.

X[i] = X[4] = 1 ⇒ X =[1, 1, 0, 1] A complete state space tree for given data is shown in Fig. (b)

UNIT-4 [7] K.PRASANTHI

At level i, the left branch corresponds to the inclusion of number wi and the right branch corresponds to
exclusion of number wi. The recursive call number for the node is stated below the node. Node 8 is the
solution node. The bold solid line shows the path to the output node.

Graph Coloring

 Graph Coloring is a process of assigning colors to the vertices of a graph.

 It ensures that no two adjacent vertices of the graph are colored with the same color.
 Chromatic Number is the minimum number of colors required to properly color any graph.
Example:

UNIT-4 [8] K.PRASANTHI

 Hamiltonian cycles
A Hamiltonian cycle (or Hamiltonian circuit) is a cycle that visits each vertex exactly once. A
Hamiltonian path that starts and ends at adjacent vertices can be completed by adding one more
edge to form a Hamiltonian cycle, and removing any edge from a Hamiltonian cycle produces a
Hamiltonian path.

UNIT-4 [9] K.PRASANTHI

Example: Consider a graph G = (V, E) shown in fig. we have to find a Hamiltonian circuit using
Backtracking method.

Solution: Firstly, we start our search with vertex 'a.' this vertex 'a' becomes the root of our
implicit tree.

UNIT-4 [10] K.PRASANTHI

knapsack problem using Back tracking:
Example: Consider knapsack problem : n = 8. (W1, W2, W3, W4, W5, W6, W2, W8) = (1, 11, 21,
23, 33, 43, 45, 55), P = (11, 21, 31, 33, 43, 53, 55, 65), m = 110. Solve the problem using
backtracking approach.

UNIT-4 [11] K.PRASANTHI

 Branch and Bound
Branch and bound is one of the techniques used for problem solving. It is similar to the
backtracking since it also uses the state space tree. It is used for solving the optimization
problems and minimization problems. If we have given a maximization problem then we can
convert it using the Branch and bound technique by simply converting the problem into a
maximization problem.

Branch and Bound refers to all state space search methods in which all children of the E-node are
generated before any other live node becomes the E-node

Branch and Bound is the generalization of both graph search strategies, BFS and D- search.

 A BFS like state space search is called as FIFO (First in first out) search as the list of live
nodes in a first in first out list (or queue).

 A D search like state space search is called as LIFO (Last in first out) search as the list of
live nodes in a last in first out (or stack).

 Live node is a node that has been generated but whose children have not yet been
generated.

 E-node is a live node whose children are currently being explored. In other words, an E-
node is a node currently being expanded.

 Dead node is a generated node that is not to be expanded or explored any further. All
children of a dead node have already been expanded.

 Branch-an-bound refers to all state space search methods in which all children of an E-
node are generated before any other live node can become the E-node.

 The adjective "heuristic", means" related to improving problem solving performance".

As a noun it is also used in regard to "any method or trick used to improve the efficiency
of a problem solving problem". But imperfect methods are not necessarily heuristic or
vice versa.

1. FIFO branch and bound.

2. LIFO branch and bound.
3. Least Cost branch and bound.

UNIT-4 [12] K.PRASANTHI

1. FIFO branch and bound.
A FIFO branch-and-bound algorithm for the job sequencing problem can begin with upper =
as an upper bound on the cost of a minimum-cost answer node.

For this we will use a data structure called Queue. Initially Queue is empty

In FIFO search, first we will take E-node as a node 1. Next we generate the children of node 1.
We will place all these live nodes in a queue.

2 3 4 5

Now one level is completed. Once I take first job, then we can consider either j2, j3 or j4. If we
follow the route then it says that we are doing jobs j1 and j4 so we will not consider jobs j2 and
j3.

Now we will consider the node 3. In this case, we are doing job j2 so we can consider either job
j3 or j4. Here, we have discarded the job j1.

UNIT-4 [13] K.PRASANTHI

 3 4 5 6 7 8

4 5 6 7 8 9 10

Finally,

UNIT-4 [14] K.PRASANTHI

LIFO Branch and Bound :

For this we will use a data structure called stack. Initially stack is empty.

On each expansion, the node will be pushed into the stack shown as below:

Now the expansion would be based on the node that appears on the top of the stack. Since the
node 5 appears on the top of the stack, so we will expand the node 5. We will pop out the node 5
from the stack. Since the node 5 is in the last job, i.e., j4 so there is no further scope of
expansion.

LC (Least Cost) Branch and Bound:

There is one more method that can be used to find the solution and that method is Least cost
branch and bound. In this technique, nodes are explored based on the cost of the node. The cost
of the node can be defined using the problem and with the help of the given problem, we can
define the cost function. Once the cost function is defined, we can define the cost of the node.

UNIT-4 [15] K.PRASANTHI

where,
c( x ) is the cost of x.
h(x) is the cost of reaching x from the root and f(.) is any non-decreasing function.
g ( x ) is an estimate of the additional effort needed to reach an answer node from x.

Now we will expand the node 1. The node 1 will be expanded into four nodes named as 2, 3, 4
and 5 shown as below:

Let's assume that cost of the nodes 2, 3, 4, and 5 are 25, 12, 19 and 30 respectively.

Since it is the least cost branch n bound, so we will explore the node which is having the least
cost. In the above figure, we can observe that the node with a minimum cost is node 3. So, we
will explore the node 3 having cost 12.

Since the node 3 works on the job j2 so it will be expanded into two nodes named as 6 and 7
shown as below:

UNIT-4 [16] K.PRASANTHI

 0/1 Knapsack problem using Branch and Bound
Consider the instance: M = 15, n = 4, (P1, P2, P3, P4) = (10, 10, 12, 18) and
(w1, w2, w3, w4) = ( 2, 4, 6, 9).
0/1 knapsack problem can be solved by using branch and bound technique. In this problem we
will calculate lower bound and upper bound for each node.
 Place first item in knapsack. Remaining weight of knapsack is 15 – 2 = 13. Place next
item w2 in knapsack and the remaining weight of knapsack is 13 – 4 = 9. Place next item
w3 in knapsack then the remaining weight of knapsack is 9 – 6 = 3. No fractions are
allowed in calculation of upper bound so w4 cannot be placed in knapsack.

Knapsack problem is maximization problem but branch and bound technique is applicable for
only minimization problems. In order to convert maximization problem into minimization
problem we have to take negative sign for upper bound and lower bound.
Therefore,
Upper bound (U) = -32
Lower bound (L) = -38

We choose the path, which has minimum difference of upper bound and lower bound. If the
difference is equal then we choose the path by comparing upper bounds and we discard node
with maximum upper bound.

UNIT-4 [17] K.PRASANTHI

Now we will calculate upper bound and lower bound for nodes 2, 3. For node 2, x1= 1, means
we should place first item in the knapsack.

UNIT-4 [18] K.PRASANTHI

UNIT-4 [19] K.PRASANTHI
 Traveling salesperson problem using Branch and Bound

Travelling Salesman Problem states-

 A salesman has to visit every city exactly once.

 He has to come back to the city from where he starts his journey.
 What is the shortest possible route that the salesman must follow to complete his tour?

Let G = (V', E) be a directed graph defining an instance of the traveling salesperson problem. Let
Cij equal the cost of edge (i, j), Cij = ∞ if (i, j) != E, and let IVI = n, without loss of generality,
we can assume that every tour starts and ends at vertex 1.

UNIT-4 [20] K.PRASANTHI

If salesman starting city is A, then a TSP tour in the graph is-
A→B→D→C→A
Cost of the tour
= 10 + 25 + 30 + 15
= 80 units

Procedure for solving travelling sales person problem

1. Reduce the given cost matrix. A matrix is reduced if every row and column is reduced. This
can be done as follows:

Row Reduction:
a) Take the minimum element from first row, subtract it from all elements of first row, next
take minimum element from the second row and subtract it from second row. Similarly apply the
same procedure for all rows.
b) Find the sum of elements, which were subtracted from rows.
c) Apply column reductions for the matrix obtained after row reduction.

Column Reduction:
d) Take the minimum element from first column, subtract it from all elements of first column,
next take minimum element from the second column and subtract it from second column.
Similarly apply the same procedure for all columns.

e) Find the sum of elements, which were subtracted from columns.

f) Obtain the cumulative sum of row wise reduction and column wise reduction.
Cumulative reduced sum=Row wise reduction sum + Column wise reduction sum.

Associate the cumulative reduced sum to the starting state as lower bound and α as upper bound.

UNIT-4 [21] K.PRASANTHI

2. Calculate the reduced cost matrix for every node.
a) If path (i,j) is considered then change all entries in row i and column j of A to α.
b) Set A(j,1) to α.
c) Apply row reduction and column reduction except for rows and columns containing only α.
Let r is the total amount subtracted to reduce the matrix.
d) Find ĉ(S)= ĉ(R)+A(i,j)+r.
Repeat step 2 until all nodes are visited

Travelling Salesman Problem Using Branch and Bound Approach-

UNIT-4 [22] K.PRASANTHI

UNIT-4 [23] K.PRASANTHI
UNIT-4 [24] K.PRASANTHI
UNIT-4 [25] K.PRASANTHI
UNIT-4 [26] K.PRASANTHI
UNIT-4 [27] K.PRASANTHI
UNIT-4 [28] K.PRASANTHI
UNIT-4 [29] K.PRASANTHI
UNIT-4 [30] K.PRASANTHI
UNIT-4 [31] K.PRASANTHI