Professional Documents
Culture Documents
Unit V
Unit V
Traversing Trees
Depth first search
Breadth first search
Back tracking
The Knapsack Problem(3)
The Eight Queens problem
The General Templates
Branch and Bound
The assignment problem
The knapsack problpme(4)
General Consideration
The Minimax principle
Introduction to NP Completeness
Classes P and NP
Polynomial reductions
NP-Complete Problems
NP-Hard Problems
Non Deterministic algorithms.
Part-I
Exploring Graphs
Breadth First Search
(BFS)(G)
{
list L = empty
tree T = empty
choose a starting vertex x
Search (x) while (L nonempty)
remove edge (v, w) from start of L
if w not yet visited
{
add (v, w) to T
Search (w)
}
}
04/12/17 3
Depth First Search
DFS(G)
{
list L = empty
tree T = empty
choose a starting vertex x
Search (x) while (L nonempty)
remove edge (v, w) from end of L
if w not yet visited
{
add (v, w) to T
Search (w)
}
}
04/12/17 4
Relation between
Both of these BFS and
search algorithms DFS
now keep a list of edges to
explore;
the only difference between the two is that, while both
algorithms adds items to the end of L,
BFS removes them from the beginning, which results in
maintaining the list as a queue,
while DFS removes them from the end, maintaining the list
as a stack.
04/12/17 5
BackTracking
04/12/17 6
Backtracking
Suppose you have to make a series of decisions, among
various choices, where
You dont have enough information to know what to
choose
Each decision leads to a new set of choices
04/12/17 7
Backtracking (animation)
dead end
?
dead end
dead end
?
start ? ? dead end
dead end
?
success!
04/12/17 8
The backtracking algorithm
04/12/17 9
Backtracking
The principal idea is to construct solutions one component
at a time and evaluate such partially constructed candidates
as follows.
04/12/17 10
10
Backtracking
This kind of processing is often implemented by constructing a tree
of choices being made, called the state-space tree.
Its root represents an initial state before the search for a solution
begins.
The nodes of the first level in the tree represent the choices
made for the first component of a solution,
the nodes of the second level represent the choices for the second
component, and so on.
04/12/17 11
11
Backtracking
04/12/17 12
Backtracking
Construct the state-space tree
nodes: partial solutions
04/12/17 13
The Knapsack Problem(3)
Consider of size K and we want to select from a set of n
objects , where the ith object has size si and value vi, a subset
of these objects to maximize the value contained in the
knapsack with the contents of the knapsack less than or equal
to K.
Suppose that K = 16 and n = 4, and we have the following set
of objects ordered by their value density.
i vi si vi/si
1 $45 3 $15
2 $30 5 $ 6
3 $45 9 $ 5
4 $10 5 $ 2
04/12/17 14
We will construct the state space where each
node contains the total current value in the
knapsack, the total current size of the
contents of the knapsack, and maximum
potential value that the knapsack can hold. In
the algorithm, we will also keep a record of
the maximum value of any node (partially or
completely filled knapsack) found so far.
When we perform the depth-first traversal of
the state-space tree, a node is "promising" if
its maximum potential value is greater than
this current best value.
04/12/17 15
We begin the state space tree with the root
consisting of the empty knapsack. The current
weight and value are obviously 0. To find the
maximum potential value we treat the problem as
if it were the fractional knapsack problem and we
were using the greedy algorithmic solution to that
problem. We have shown that the greedy
approach to the fractional knapsack problem
yields an optimal solution. We place each of the
remaining objects, in turn, into the knapsack until
the next selected object is too big to fit into the
knapsack. We then use the fractional amount of
that object that could be placed in the knapsack to
determine the maximum potential value.
04/12/17 16
totalSize = currentSize + size of
remaining objects that can be fully
placed
bound (maximum potential value) =
current Value + value of remaining
objects fully placed +
(K - totalSize) * (value density of item
that is partially placed)
04/12/17 17
In general, for a node at level i in the state space tree (the
first i items have been considered for selection) and for
the kth object as the one that will not completely fit into
the remaining space in the knapsack, these formulae can
be written:
k-1
totalSize = currentSize + sj
j=i+1
k-1
bound = currentValue + vj + (K - totalSize) *
(vk/sk)
j=i+1
04/12/17 18
For the root node,
currentSize = 0, currentValue = 0
totalSize = 0 + s1 + s2 = 0 + 3 + 5 = 8
bound = 0 + v1 + v2 + (K - totalSize) * (v3/s3)
= 0 + $45 + $30 + (16 - 8) * ($5)
= $75 + $40
= $115
04/12/17 19
From the root, we add two children at level 1
-- the node where the first item is included in
the knapsack and the node where it is not.
For the child where the first item is not
included in the knapsack, the calculation for
the bound proceeds as follows:
totalSize = 0 + s2 + s3 = 5 + 9 = 14
04/12/17 20
The state spaced traversed by the backtracking algorithm is
displayed below. When the bound of a node is less than or
equal to the current maximum value, or adding the current item
to the node causes the size of the contents to exceed the
capacity of the knapsack, the sub-trees rooted at that node are
pruned, and the traversal backtracks to the previous parent in
the state space tree.
04/12/17 21
The Eight Queens problem
04/12/17 22
04/12/17 23
04/12/17 24
Example: the 8 queens problem
04/12/17 25
The 8 queens problem: representation
Phenotype:
a board configuration
04/12/17 26
Algorithm
This heuristic solves n queens for any n n 4 or n = 1:
1) Divide n by 12.
2) Remember the remainder (n is 8 for the eight queens
puzzle).
3) Write a list of the even numbers from 2 to n in order.
4) If the remainder is 3 or 9, move 2 to the end of the list.
5) Append the odd numbers from 1 to n in order, but, if the
remainder is 8, switch pairs (i.e. 3, 1, 7, 5, 11, 9, ).
6) If the remainder is 2, switch the places of 1 and 3, then
move 5 to the end of the list.
7) If the remainder is 3 or 9, move 1 and 3 to the end of
the list.
8) Place the first-column queen in the row with the
first number in the list, place the second-column
queen in the row with the second number in the list, etc.
04/12/17 27
Hamiltonian Circuit Problem
we make vertex a the root of the
state-space tree. The first
component of our future solution,
if it exists, is a first intermediate
vertex of a Hamiltonian cycle to
be constructed.
Using the alphabet order to
break the three-way tie among
the vertices adjacent to a, we
select vertex b.
From b, the algorithm proceeds
to c, then to d, then to e, and
finally to f , which proves to be a
dead end.
28
04/12/17 28
Hamiltonian Circuit Problem
So the algorithm backtracks from f to e, then to
d, and then to c, which provides the first
alternative for the algorithm to pursue. Going
from c to e eventually proves useless, and the
algorithm has to backtrack from e to c and then
to b.
04/12/17 29
29
04/12/17 30
The General Templates
04/12/17 31
Branch and Bound
General Consideration
04/12/17 32
Introduction
Branch and Bound is a general search
method.
Starting by considering the root problem (the
original problem with the complete feasible
region), the lower-bounding and upper-
bounding procedures are applied to the root
problem.
If the bounds match, then an optimal solution
has been found and the procedure
terminates.
04/12/17 33
Introduction
Otherwise, the feasible region is
divided into two or more regions,
these sub problems partition the
feasible region.
The algorithm is applied recursively
04/12/17 34
Introduction
If the lower bound for a node exceeds the best
known feasible solution, no globally optimal
solution can exist in the subspace of the feasible
region represented by the node. Therefore, the
node can be removed from consideration.
The search proceeds until all nodes have been
solved or pruned, or until some specified threshold
is met between the best solution found and the
lower bounds on all unsolved sub problems.
04/12/17 35
The assignment problem
Same as Operation Research
04/12/17 36
The knapsack problem(4)
Solve
this algorithm using branch and
bound techniques.
04/12/17 37
General Consideration
Branch and bound (BB) is a general algorithm
for finding optimal solutions of various
optimization problems, especially in discrete
and combinatorial optimization.
04/12/17 38
The Minimax Principle
Same as Game Theory
04/12/17 39
Part-II
Introduction to NP-completeness
Problems
Some problems cannot be solved by any algorithms
04/12/17 41
Problems
04/12/17 42
42
Polynomial time
Definition:
04/12/17 43
43
P and NP Problems
04/12/17 44
44
Class P
Examples:
searching
element uniqueness
graph connectivity
graph acyclicity
04/12/17 45
45
Class NP
NP (Nondeterministic Polynomial):
class of decision problems whose
proposed solutions can be verified in
polynomial time = solvable by a
nondeterministic polynomial
algorithm
04/12/17 46
46
Non-deterministic polynomial
time
Deterministic Polynomial Time: The TM
takes at most O(nc) steps to accept a
string of length n
Non-deterministic Polynomial Time: The
TM takes at most O(nc) steps on each
computation path to accept a string of
length n
04/12/17 47
Nondeterministic polynomial algorithm
04/12/17 48
48
What problems are in NP?
Knapsack problem
04/12/17 49
49
P NP
P NP
NP-Complete
P
NP
04/12/17 51
Definition of NP-Complete
Q is an NP-Complete problem if:
1) Q is in NP
2) every other NP problem polynomial
time reducible to Q
04/12/17 52
P = NP?
No one knows if this is true
How can we make progress on this
problem?
04/12/17 53
Progress
P = NP if every NP problem has a
deterministic polynomial algorithm
We could find an algorithm for every NP
problem
Seems hard
04/12/17 55