Professional Documents
Culture Documents
CH 12
CH 12
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 1
Exact Solution Strategies
exhaustive search (brute force)
• useful only for small instances
dynamic programming
• applicable to some problems (e.g., the knapsack problem)
backtracking
• eliminates some unnecessary cases from consideration
• yields solutions in reasonable time for many instances but
worst case is still exponential
branch-and-bound
• further refines the backtracking idea for optimization
problems
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 2
Backtracking
Construct the state-space tree
• nodes: partial solutions
• edges: choices in extending partial solutions
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 3
Example: n-Queens Problem
1 2 3 4
1 queen 1
2 queen 2
3 queen 3
4 queen 4
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 4
State-Space Tree of the 4-Queens Problem
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 5
Example: Hamiltonian Circuit Problem
a b
c f
d e 0
with 3 w/o 3
3 0
with 5 w/o 5 with 5 w/o 5
8 3 5 0
with 6 w/o 6 with 6 w/o 6 with 6 w/o 6 X
0+13<15
14 8 9 3 11 5
X with 7 w/o 7 X X X X
14+7>15 9+7>15 3+7<15 11+7>14 5+7<15
15 8
solution X
8<15
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 6
Branch-and-Bound
An enhancement of backtracking
Example
Job 1 Job 2 Job 3 Job 4
Person a 9 2 7 8
Person b 6 4 3 7
Person c 5 8 1 8
Person d 7 6 9 4
Lower bound: Any solution to this problem will have total cost
at least: 2 + 3 + 1 + 4 (or 5 + 2 + 1 + 4)
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 8
Example: First two levels of the state-space tree
12.5
12.5
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 9
Example (cont.)
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 10
Example: Complete state-space tree
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 11
Example: Traveling Salesman Problem
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 12
Approximation Approach
Apply a fast (i.e., a polynomial-time) approximation algorithm
to get a solution that is not necessarily optimal but hopefully
close to it
Accuracy measures:
accuracy ratio of an approximate solution sa
r(sa) = f(sa) / f(s*) for minimization problems
r(sa) = f(s*) / f(sa) for maximization problems
where f(sa) and f(s*) are values of the objective function f for
the approximate solution sa and actual optimal solution s*
sa : A – B – C – D – A of length 10
6 2
3 3
s* : A – B – D – C – A of length 8
D C
1
Stage 2: Add next edge on the sorted list to the tour, skipping
those whose addition would’ve created a vertex of
degree 3 or a cycle of length less than n. Repeat
this step until a tour of length n is obtained
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 15
Twice-Around-the-Tree Algorithm
Stage 1: Construct a minimum spanning tree of the graph
(e.g., by Prim’s or Kruskal’s algorithm)
9 9
8 4 7 11
b d b d
8
6 10
c c
Walk: a – b – c – b – d – e – d – b – a Tour: a – b – c – d – e – a
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 17
Christofides Algorithm
Stage 1: Construct a minimum spanning tree of the graph
9 9
8 4 7 11
b d
8
6 10
c
a e a e
4 4 7 11
4 9 7 11
b d b
8 d
6 6
c c
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 19
Euclidean Instances
Theorem If P ≠ NP, there exists no approximation algorithm
for TSP with a finite performance ratio.
Definition An instance of TSP is called Euclidean, if its
distances satisfy two conditions:
1. symmetry d[i, j] = d[j, i] for any pair of cities i and j
2. triangle inequality d[i, j] ≤ d[i, k] + d[k, j] for any cities i, j, k
Example of a 2-change
C4 C3 C4 C3
C1 C2 C1 C2
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 21
Example of a 3-change
C5 C4
C6 C3
C1 C2
C5 C4 C5 C4
C6 C3 C6 C3
C1 C2 C1 C2
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 22
Empirical Data for Euclidean Instances
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 23
Greedy Algorithm for Knapsack Problem
Step 1: Order the items in decreasing order of relative values:
v1/w1 ⋯ vn/wn
Step 2: Select the items in this order skipping those that don’t
fit into the knapsack
Accuracy
R is unbounded (e.g., n = 2, C = m, w =1, v =2, w =m, v =m)
A 1 1 2 2
yields exact solutions for the continuous version
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 24
Approximation Scheme for Knapsack Problem
Accuracy
Number of extra bins never exceeds optimal by more than
70% (i.e., RA ≤ 1.7)
Empirical average-case behavior is much better. (In one
experiment with 128,000 bins, the relative error was found
to A.be no more than 2%.)
Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 26
Bin Packing: First-Fit Decreasing Algorithm
Accuracy
Number of extra bins never exceeds optimal by more than
50% (i.e., RA ≤ 1.5)
Empirical average-case behavior is much better, too
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 27
Numerical Algorithms
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 28
Principal Accuracy Metrics
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 29
Two Types of Errors
truncation errors
• Taylor’s polynomial approximation
ex 1 + x + x2/2! + ⋯ + xn/n!
round-off errors
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 30
Solving Quadratic Equation
subtractive cancellation
use alternative formulas (see p. 411)
use double precision for D = b2 - 4ac
Useful:
• sketch graph of f(x)
• A.separate roots
Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 32
Three Classic Methods
Three classic methods for solving nonlinear equation
f(x) = 0
in one unknown:
bisection method
Newton’s method
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 33
Bisection Method
Based on
Theorem: If f(x) is continuous on ax b and f(a) and f(b) have
opposite signs, then f(x) = 0 has a root on a < x < b
binary search idea
f(x)
x
a x1 b
an
. . xn
. bn
x
. f(xn )
. .
xn+1 x n
x
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 37
Notes on Newton’s Method
Normally, approximations xn converge to root very fast but
can diverge with a bad choice of initial approximation x0
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 12 ©2012 Pearson
Education, Inc. Upper Saddle River, NJ. All Rights Reserved. 38