Divide and Conquer (Closest Pair, Convex Hull, Strassen Matrix Multiply) Demo

Algorithm Design and Analysis
LECTURE 13
Divide and Conquer
• Closest Pair of Points
• Convex Hull
• Strassen Matrix Mult.
Adam Smith
9/24/2008
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Midterm Exam #1
•  Willard Building Room 76
•  Tuesday night, 8:15pm
•  You may bring: one (1)

double-sided,
hand-written 8.5” x 11” sheet of notes
on colored paper
–  Hint: use its preparation as a study aid
9/24/2008
Review questions
•  Find the solution to the recurrence using MT:
T(n)=8T(n/2)+cn.
(Answer: logb(a)=3>1, solution is Θ(n3).)
•  Draw the recursion tree for this recurrence.
a. What is its height?
(Answer: h=log n.)
b. What is the number of leaves in the tree?
(Answer: 8h = 8log n = nlog 8= n3.)
9/24/2008
Review questions: recursion tree
Solve T(n)=8T(n/2)+cn:
cn cn
8
c(n/2) c(n/2) 4cn
8 8
c(n/4) c(n/4) c(n/4) (n/4) 16cn
…
Θ(1) Total = cn(1+4+42+43+…+n2)
= Θ(n3) geometric series
9/24/2008
Appendix: geometric series
for x ≠ 1
for |x| < 1
Return to last
slide viewed.
9/24/2008
Closest Pair of Points
Closest pair. Given n points in the plane, find a pair with smallest
Euclidean distance between them.
Fundamental geometric primitive.

 Graphics, computer vision, geographic information systems,
molecular modeling, air traffic control.
 Special case of nearest neighbor, Euclidean MST, Voronoi.
fast closest pair inspired fast algorithms for these problems
Brute force. Check all pairs of points p and q with Θ(n2) comparisons.
1-D version (points on a line): O(n log n) easy via sorting
Assumption. No two points have same x coordinate.
to make presentation cleaner

Closest Pair of Points: First Attempt
Divide. Sub-divide region into 4 quadrants.
L
Closest Pair of Points: First Attempt
Divide. Sub-divide region into 4 quadrants.

Obstacle. Impossible to ensure n/4 points in each piece.
L
Algorithm.
 Divide: draw vertical line L so that roughly ½n points on each side.
L
Algorithm.
 Conquer: find closest pair in each side recursively.
21
16
Algorithm.
 Conquer: find closest pair in each side recursively.
 Combine: find closest pair with one point in each side. seems like Θ(n2)
 Return best of 3 solutions.
8
21
16
Find closest pair with one point in each side, assuming that distance < δ.
21
δ = min(16, 21)
16
 Observation: only need to consider points within δ of line L.
21
δ = min(16, 21)
16
δ
 Sort points in 2δ-strip by their y coordinate.
L
7
5
4 21
δ = min(16, 21)
16 3
δ
 Sort points in 2δ-strip by their y coordinate.
 Theorem: Only need to check distances of those
within 11 positions in sorted list!
L
7
5
4 21
δ = min(16, 21)
16 3
δ
Def. Let si be the point in the 2δ-strip, with

the ith smallest y-coordinate.
Claim. If |i – j| ≥ 12, then the distance between 39 j

si and sj is at least δ.
31
Proof.
 No two points lie in same ½δ-by-½δ box. ½δ
 Two points at least 2 rows apart 2 rows
30
½δ
have distance ≥ 2(½δ). ▪ 29
i 27
28 ½δ
Fact. Still true if we replace 12 with 7.

26
25
δ δ
Closest Pair Algorithm
Closest-Pair(p1, …, pn) {
Compute separation line L such that half the points O(n log n)
are on one side and half on the other side.
δ1 = Closest-Pair(left half) 2T(n / 2)

δ2 = Closest-Pair(right half)
δ = min(δ1, δ2)
Delete all points further than δ from separation line L O(n)
Sort remaining points by y-coordinate. O(n log n)
Scan points in y-order and compare distance between

each point and next 11 neighbors. If any of these O(n)
distances is less than δ, update δ.
return δ.
}
Closest Pair of Points: Analysis
Running time.
Q.  Can we achieve O(n log n)?
A. Yes. Don't sort points in strip from scratch each time.

Sort entire point set by x-coordinate only once
 
Each recursive call takes as input a set of points sorted by

 
x coordinates and returns the same points sorted by y coordinate

(together with the closest pair)
Create new y-sorted list by merging two output from recursive calls
 
Totaltime(n) = O(n log(n)) + T(n)
T(n) ≤ 2T ( n /2) + O(n) ⇒ T(n) = O(n log n)

€
€
Divide and Conquer in low-dimensional geometry
  Powerful technique for low-dimensional geometric problems

  Intuition:
points in different parts of the plane don’t
interfere too much
  Example: convex hull in O(n log (n)) time a la MergeSort
1.  Convex-Hull(left-half) T(n/2)
2.  Convex-Hull(right-half) T(n/2)
3.  Merge (see Cormen et al., Chap 33) Θ(n)
Matrix multiplication
Input: A = [aij], B = [bij].
i, j = 1, 2,… , n.
Output: C = [cij] = A⋅ B.
9/24/2008
Standard algorithm
for i ← 1 to n
do for j ← 1 to n
do cij ← 0
for k ← 1 to n
do cij ← cij + aik⋅ bkj
Running time = Θ(n3)
9/24/2008
Divide-and-conquer algorithm
IDEA:
n£n matrix = 2£2 matrix of (n/2)£(n/2) submatrices:
C = A ⋅ B
r = ae + bg recursive
s = af + bh 8 mults of (n/2) £(n/2) submatrices
t = ce + dh ^
4 adds of (n/2) £(n/2) submatrices
u = cf + dg
9/24/2008
Analysis of D&C algorithm
T(n) = 8 T(n/2) + Θ(n2)
# submatrices work adding

submatrices
submatrix size
nlogba = nlog28 = n3 ⇒ CASE 1 ⇒ T(n) = Θ(n3).
No better than the ordinary algorithm.
9/24/2008
Strassen’s idea
• Multiply 2£2 matrices with only 7 recursive mults.
P1 = a ⋅ ( f – h) r = P5 + P4 – P2 + P6
P2 = (a + b) ⋅ h s = P1 + P2
P3 = (c + d) ⋅ e t = P3 + P4
P4 = d ⋅ (g – e) u = P5 + P1 – P3 – P7
P5 = (a + d) ⋅ (e + h) 7 mults, 18 adds/subs.
P6 = (b – d) ⋅ (g + h) Note: No reliance on
P7 = (a – c) ⋅ (e + f ) commutativity of
multiplication!
9/24/2008
Strassen’s idea
• Multiply 2£2 matrices with only 7 recursive mults.
P1 = a ⋅ ( f – h) r = P5 + P4 – P2 + P6
P2 = (a + b) ⋅ h = (a + d) (e + h)
P3 = (c + d) ⋅ e + d (g – e) – (a + b) h
P4 = d ⋅ (g – e) + (b – d) (g + h)
P5 = (a + d) ⋅ (e + h) = ae + ah + de + dh
P6 = (b – d) ⋅ (g + h) + dg –de – ah – bh
P7 = (a – c) ⋅ (e + f ) + bg + bh – dg – dh
= ae + bg
9/24/2008
Strassen’s algorithm
1. Divide: Partition A and B into (n/2) x (n/2)
submatrices. Form terms to be multiplied
using + and – .
2. Conquer: Perform 7 multiplications of (n/2) x
(n/2) submatrices recursively.
3. Combine: Form product matrix C using + and –
on (n/2) x (n/2) submatrices.
T(n) = 7 T(n/2) + Θ(n2)
9/24/2008
Analysis of Strassen
T(n) = 7 T(n/2) + Θ(n2)
nlogba = nlog27 ≈ n2.81 ⇒ CASE 1 ⇒ T(n) = Θ(nlg 7).
• Number 2.81 may not seem much smaller than 3.
• But the difference is in the exponent.
• The impact on running time is significant.
• Strassen’s algorithm beats the ordinary algorithm
on today’s machines for n ≥ 32 or so.
Best to date (of theoretical interest only): Θ(n2.376).
9/24/2008

Divide and Conquer (Closest Pair, Convex Hull, Strassen Matrix Multiply) Demo

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Divide and Conquer (Closest Pair, Convex Hull, Strassen Matrix Multiply) Demo

Uploaded by

Copyright:

Available Formats

Algorithm Design and Analysis

• You may bring: one (1)

for |x| < 1

Fundamental geometric primitive.

1-D version (points on a line): O(n log n) easy via sorting

Assumption. No two points have same x coordinate.

to make presentation cleaner

Divide. Sub-divide region into 4 quadrants.

Divide. Sub-divide region into 4 quadrants.

Def. Let si be the point in the 2δ-strip, with

Claim. If |i – j| ≥ 12, then the distance between 39 j

Fact. Still true if we replace 12 with 7.

δ1 = Closest-Pair(left half) 2T(n / 2)

Delete all points further than δ from separation line L O(n)

Sort remaining points by y-coordinate. O(n log n)

Scan points in y-order and compare distance between

Q. Can we achieve O(n log n)?

A. Yes. Don't sort points in strip from scratch each time.

Each recursive call takes as input a set of points sorted by

x coordinates and returns the same points sorted by y coordinate

Totaltime(n) = O(n log(n)) + T(n)

T(n) ≤ 2T ( n /2) + O(n) ⇒ T(n) = O(n log n)

 Powerful technique for low-dimensional geometric problems

Running time = Θ(n3)

# submatrices work adding

nlogba = nlog28 = n3 ⇒ CASE 1 ⇒ T(n) = Θ(n3).

No better than the ordinary algorithm.

T(n) = 7 T(n/2) + Θ(n2)

You might also like

•  You may bring: one (1)

Q.  Can we achieve O(n log n)?

  Powerful technique for low-dimensional geometric problems