Professional Documents
Culture Documents
Solutions of RDBMS
Solutions of RDBMS
Solutions of RDBMS
4.If S contains such an element z, then STOP, since we have found y and z such that x = y + z.
Otherwise, repeat Step 2.
5.If S is empty, then no two elements in S sum to x.
Notice that when we consider an element yi of S during ith iteration, we dont need to look at the
elements that have already been considered in previous iterations. Suppose there exists y j S,
such that x = yi + yj . If j < i, i.e. if yj has been reached prior to yi , then we would have found yi
when we were searching for x yj during jth iteration and the algorithm would have terminated
then.
Step 1 takes (n lg n) time. Step 2 takes O(1) time. Step 3 requires at most lg n time. Steps 24
are repeated at most n times. Thus, the total running time of this algorithm is (n lg n). We can do
a more precise analysis if we notice that Step 3 actually requires (lg(n i)) time at ith iteration.
However, if we evaluate n1
i=1 lg(n i), we get lg(n 1)!, which is (n lg n). So the total running
time is still (n lg n).
Exercise 1-2. Do Exercise 3.1-3 on page 50 in CLRS.
Exercise 1-3. Do Exercise 3.2-6 on page 57 in CLRS.
Exercise 1-4. Do Problem 3-2 on page 58 of CLRS.
(a) f (n) = O(g(n)) and g(n) = O(f (n)) implies that f (n) = (g(n)).
Solution:
Since f (n) = O(g(n)), then there exists an n0 and a c such that for all n n0 , f (n)
cg(n). Similarly, since g(n) = O(f (n)), there exists an n0 and a c such that for all
n n0 , g (n) c
f (n). Therefore, for all n max(n0 , n0 ), c1 g(n) f (n) cg(n).
Hence, f (n) = (g(n)).
(b) f (n) + g(n) = (max(f (n), g(n))).
Solution:
For all n 1, f (n) max(f (n), g(n)) and g(n) max(f (n), g(n)). Therefore:
f (n) + g(n) max(f (n), g(n)) + max(f (n), g(n)) 2 max(f (n), g(n))
and so f (n) + g(n) = O(max(f (n), g(n))). Additionally, for each n, either f (n)
max(f (n), g(n)) or else g(n) max(f (n), g(n)). Therefore, for all n 1, f (n) +
g(n) max(f (n), g(n)) and so f (n) + g(n) = (max(f (n), g(n))). Thus, f (n) +
g(n) = (max(f (n), g(n))).
(c) Transitivity: f (n) = O(g(n)) and g(n) = O(h(n)) implies that f (n) = O(h(n)).
Solution:
Since f (n) = O(g(n)), then there exists an n0 and a c such that for all n n0 ,
f (n) cg(n). Similarly, since g(n) = O(h(n)), there exists an n 0 and a c such that
for all n n0 , g (n) c h(n). Therefore, for all n max(n0 , n0 ), f (n) cc
h(n).
Hence, f (n) = O(h(n)).
(d) f (n) = O(g(n)) implies that h(f (n)) = O(h(g(n)).
Solution:
Let h(n) = o(f (n)). We prove that f (n) + o(f (n)) = (f (n)). Since for all n 1,
Since h(n) = o(f (n)), then there exists an n0 such that for all n > n0 , h(n) f (n).
Therefore, for all n > n0 , f (n) + h(n) 2f (n) and so f (n) + h(n) = O(f (n)).
(f) f (n) = o(g(n)) and g(n) = o(f (n)) implies f (n) = (g(n)).
Solution:
F1 = 1 ,
Fn = Fn1 + Fn2
for n 2 .
In Exercise 1-3, of this problem set, you showed that the nth Fibonacci number is
Fn =
n
n
,
5
A fellow 6.046 student comes to you with the following simple recursive algorithm for computing
the nth Fibonacci number.
F IB(n)
1 if n = 0
2
then return 0
3 elseif n = 1
4
then return 1
5 return F IB(n 1) + F IB(n 2)
This algorithm is correct, since it directly implements the denition of the Fibonacci numbers.
Lets analyze its running time. Let T (n) be the worst-case running time of F IB(n). 1
(a) Give a recurrence for T (n), and use the substitution method to show that T (n) =
O(Fn ).
In this problem, please assume that all operations take unit time. In reality, the time it takes to add two num
bers depends on the number of bits in the numbers being added (more precisely, on the number of memory words).
However, for the purpose of this problem, the approximation of unit time addition will sufce.
Professor Grigori Potemkin has recently published an improved algorithm for computing the nth
Fibonacci number which uses a cleverly constructed loop to get rid of one of the recursive calls.
Professor Potemkin has staked his reputation on this new algorithm, and his tenure committee has
asked you to review his algorithm.
F IB (n)
1 if n = 0
2
then return 0
3 elseif n = 1
4
then return 1
5 sum 1
6 for k 1 to n 2
7
do sum sum + F IB (k)
8 return sum
Since it is not at all clear that this algorithm actually computes the nth Fibonacci number, lets
prove that the algorithm is correct. Well prove this by induction over n, using a loop invariant in
the inductive step of the proof.
(c) State the induction hypothesis and the base case of your correctness proof.
Solution: To prove the algorithm is correct, we are inducting on n. Our induction
hypothesis is that for all n < m, Fib (n) returns Fn , the nth Fibonacci number.
Our base case is m = 2. We observe that the rst four lines of Potemkin guarantee
that Fib (n) returns the correct value when n < 2.
(d) State a loop invariant for the loop in lines 6-7. Prove, using induction over k, that your
Solution: Our loop invariant is that after the k = i iteration of the loop,
sum = Fi+2 .
We prove this induction using induction over k. We assume that after the k = (i 1)
iteration of the loop, sum = Fi+1 . Our base case is i = 1. We observe that after the
rst pass through the loop, sum = 2 which is the 3rd Fibonacci number.
To complete the induction step we observe that if sum = Fi+1 after the k = (i1) and
if the call to F ib (i) on Line 7 correctly returns Fi (by the induction hypothesis of our
correctness proof in the previous part of the problem) then after the k = i iteration of
the loop sum = Fi+2 . This follows immediately form the fact that Fi + Fi+1 = Fi+2 .
(e) Use your loop invariant to complete the inductive step of your correctness proof.
Solution: To complete the inductive step of our correctness proof, we must show that
if F ib (n) returns Fn for all n < m then F ib (m) returns m. From the previous part
we know that if F ib (n) returns Fn for all n < m, then at the end of the k = i iteration
of the loop sum = Fi+2 . We can thus conclude that after the k = m 2 iteration of
the loop, sum = Fm which completes our correctness proof.
(f) What is the asymptotic running time, T (n), of F IB (n)? Would you recommend
Solution: We will argue that T (n) = (Fn ) and thus that Potemkins algorithm,
F ib does not improve upon the assymptotic performance of the simple recurrsive
algorithm, F ib. Therefore we would not recommend tenure for Professor Potemkin.
One way to see that T (n) = (Fn ) is to observe that the only constant in the program
is the 1 (in lines 5 and 4). That is, in order for the program to return F n lines 5 and 4
must be executed a total of Fn times.
Another way to see that T (n) = (Fn ) is to use the substitution method with the
=
=
=
=
a1 b1 ,
a1 b0 ,
a0 b1 ,
a0 b0 ,
Let p(x) and q(x) be polynomials of degree n 1, and divide each into the upper n/2
Solution:
We can use the following 3 multiplications:
m1 = (a + b)(c + d) = ac + ad + bc + bd ,
m2 = ac ,
m3 = bd ,
Solution:
The algorithm is the same as in part (a), except for the fact that we need only compute
three products of polynomials of degree n/2 to get the polynomial product.
(e) Give and solve a recurrence for the worst-case running time of your algorithm.
Solution:
Similar to part (b):
Introduction to Algorithms
Massachusetts Institute of Technology
Professors Piotr Indyk and Charles E. Leiserson
October 1, 2004
6.046J/18.410J
Handout 9
2
Consider a polynomial in the eld Z/p:
a(x) =
ai xi mod p
(1)
i=0
A root or zero of a polynomial is a value of x for which a(x) = 0. The following theorem describes
the number of zeros for a polynomial of degree n.
Theorem 1 A polynomial a(x) of degree n has at most n distinct zeros.
Polly the Parrot is a very smart bird that likes to play math games. Today, Polly is thinking of a
polynomial a(x) over the eld Z/p. Though Polly will not tell you the coefcients of a(x), she
will happily evaluate a(x) for any x of your choosing. She challenges you to gure out whether or
not a is equivalent to zero (that is, whether x {0, . . . , p 1} : a(x) 0 mod p).
Throughout this problem, assume that a(x) has degree n, where n < p.
(a) Using a randomized query, describe how much information you can obtain after a
single interaction with Polly. That is, if a is not equivalent to zero, then for a query x
chosen uniformly at random from {0, . . . , p1}, what is the probability that a(x) = 0?
Solution:
If a is not equivalent to zero, then the probability that a(x) = 0 is at most n/p. By
Theorem 1, a has at most n distinct zeros in {0, . . . , p 1}. As we are choosing x
uniformly at random from {0, . . . , p 1}, the probability of choosing any given x in
the eld is 1/p. Thus, the probability that the chosen x is a zero of a is at most n/p.
If a is equivalent to zero, then the probability that a(x) = 0 is 1. By denition, if a is
equivalent to zero, then a(x) = 0 for all x {0, . . . , p 1}.
(b) If n = 10 and p = 101, how many interactions with Polly do you need to be 99%
Solution:
Our strategy is to send k queries to Polly, each time evaluating a(x) for a random
x as described in Part (a). If each query evaluates to zero, then we report that a is
equivalent to zero. Otherwise, we report that a is not equivalent to zero.
In the event that a is equivalent to zero, then our report will always be correct (even
for k = 1). Thus, we focus on the case when a is not equivalent to zero. If we nd
that any query does not evaluate to zero, then our report will also be correct (since a
is not equivalent to zero for a certain x).
The problem thus becomes: if a is not equivalent to zero, choose k such that the
probability that all k queries evaluate to zero is no more than 1%. Let denote the
margin of error in the general case ( = 1% in this part), and let Q i be a random
variable indicating the result of the ith query. The constraint is as follows:
Pr[Q1 = 0 and Q2 = 0 and . . . and Qk = 0]
= Pr[Q1 = 0]Pr[Q2 = 0] . . . Pr[Qk = 0]
(n/p)k
The rst step follows from the fact that all of the queries are independent. The second
step utilizes the bound from Part (a). Solving for k, we have:
(n/p)k
k lg(n/p) lg
k lg / lg(n/p)
The last step above utilizes the assumption that n < p, and thus lg(n/p) < 0.
Substituting = 0.01, n = 10 and p = 101 gives k 1.991. Thus, k = 2 ensures that
we are 99% certain whether or not a is equivalent to zero.
Later, you are given three polynomials: a(x), b(x), and c(x). The degree of a(x) is n, while b(x)
and c(x) are of degree n/2. You are interested in determining whether or not a(x) is equivalent to
b(x) c(x); that is, whether x {0, . . . , p 1} : a(x) b(x) c(x) mod p.
Professor Potemkin recalls that in Problem Set 1, you showed how to multiply polynomials in
(nlg2 (3) ) time. Potemkin suggests using this procedure to directly compare the polynomials.
However, recalling your fun times with Polly, you convince Potemkin that there might be an even
more efcient procedure, if some margin of error is tolerated.
(c) Give a randomized algorithm that decides with probability 1 whether or not a(x)
is equivalent to b(x) c(x). Analyze its running time and compare to Potemkins
proposal.
Solution:
The algorithm checks the equivalence of a(x) and b(x) c(x) by testing if a(x)
b(x) c(x) is equivalent to zero. The testing is done using the randomized procedure
developed in Part (b), choosing k to ensure the desired level of certainty. Pseudocode
for the algorithm is as follows:
(2)
That is, the running time of our algorithm is bounded above by the running time of
Potemkins algorithm so long as p and satisfy Equation 2 for sufciently large n.
For example, consider that p = c n for some c > 1 and is a xed constant. Then
lg / lg(n/p) = lg / lg(1/c ) = (1). Thus, the loop executes (1) times and
the algorithm is has a running time of (n), which is asymptotically faster than
Potemkins proposal.
On the other hand, if p = n + 1 while remains xed, then lg / lg(n/p) =
lg / lg(n/(n + 1)) = lg /(lg n lg(n + 1))). Intuitively, one can see that this
d
(lg n) = 1/n, which means that lg(n + 1) lg n 1/n and
is (n) because dn
lg /(lg n lg(n + 1))) lg /(1/n) = (n). Thus, the loop executes (n)
times and the algorithm has a running time of (n2 ), which is asymptotically slower
than Potemkins proposal.
We can prove more rigorously that lg /(lg n lg(n + 1))) = (n) by appealing to
the following identity [CLRS, p.53]:
lim 1 +
n
1
n
=e
Then, using the denition of limit, there exist positive constants c 1 , c2 , and n0 such
that for all n > n0 :
1 n
c1 e 1 +
c2 e
n
1
ln(c1 e) n ln 1 +
ln(c2 e)
n
n+1
ln(c1 e) n ln
ln(c2 e)
n
ln(c1 e) n(ln(n + 1) ln(n)) ln(c2 e)
ln(c1 e)/n ln(n + 1) ln(n) ln(c2 e)/n
n/ ln(c1 e) 1/(ln(n + 1) ln(n)) n/ ln(c2 e)
n/ ln(c1 e) 1/(ln(n) ln(n + 1)) n/ ln(c2 e)
Thus, 1/(ln(n) ln(n + 1)) = (n) because it is bounded above and below by
a constant factor times n. By adjusting the constants, this implies that lg /(lg n
lg(n + 1))) = (n).
Finally, we point out the desirable property that the algorithm is logarithmic in for
xed values of p and n. Decreasing the error margin by a given factor results in only
an additive increase in the runtime.
Problem 2-2. Distributed Median
Alice has an array A[1..n], and Bob has an array B[1..n]. All elements in A and B are distinct.
Alice and Bob are interested in nding the median element of their combined arrays. That is, they
want to determine which element m satises the following property:
|{i [1, n] : A[i] m}| + |{i [1, n] : B[i] m}| = n
(3)
This equation says that there are a total of n elements in both A and B that are less than or equal to
m. Note that m might be drawn from either A or B.
Because Alice and Bob live in different cities, they have limited communication bandwidth. They
can send each other one integer at a time, where the value either falls within {0, . . . , n} or is drawn
from the original A or B arrays. Each numeric transmission counts as a communication between
Alice and Bob. One goal of this problem is to minimize the number of communications needed to
compute the median.
(a) Give a deterministic algorithm for computing the combined median of A and B. Your
algorithm should run in O(n log n) time and use O(log n) communications. (Hint:
consider sorting.)
Solution:
The algorithm works as follows. Alice and Bob begin by sorting their arrays using
a deterministic (n log n) algorithm such as HeapSort or MergeSort. Then, Alice
assumes the role of the master and Bob the role of the slave. Alice considers an
element A[i] and sends n i to Bob, who returns two elements: B[n i] and B[n
i + 1]. Because A is sorted, A[i] is the combined median if and only if there are
exactly n i elements in B that are less than A[i]. Because B is sorted, this condition
is reduced to checking whether or not B[n i] < A[i] < B[n i + 1]. Note that
when i = n, Bob needs to return B[0], which is out of range; in this case, he returns
B[0] = to satisfy the above condition.
Alice checks whether B[n i] < A[i] < B[n i + 1] and returns A[i] as the median
if the condition holds. If the condition fails, then she proceeds to do a binary search
within her array. The search is on i, with an initial range of [1, n]. On each step, she
descends into the top half of the range if A[i] is too small (i.e., A[i] < B[n i]); she
descends into the bottom half if A[i] is too big (i.e., A[i] > B[n i + 1]). If the
combined median is stored within A, the search is guaranteed to terminate at the right
spot, with B[n i] < A[i] < B[n i + 1], because A[i] B[n i] is a monotonically
increasing function of i while B[ni+1]A[i] is a monotonically decreasing function
of i.
If the combined median is not held in A, then Alices binary search terminates after
1 + lg n steps and returns a value of NIL. In this case, Alice and Bob swap roles,
with Bob becoming the master and Alice the slave. The procedure is repeated, and this
time the binary search returns the combined median because it must be stored within
B.
For clarity, pseudocode for this algorithm1 is given below.
1
Note that the DONE value used on line 15 of M ASTER and line 3 of S LAVE can be implemented by preceding
each communication with a 0 or 1 to indicate whether or not the following value indicates DONE. This does not impact
the asymptotic running time or communication cost.
A LICE(A[1 . . . n])
1 H EAP S ORT(A)
2 median M ASTER(A)
3 if median = NIL
4
then median S LAVE(A)
5 return median
B OB(B[1 . . . n])
1 H EAP S ORT(B)
2 median S LAVE(B)
3 if median = NIL
4
then median M ASTER(B)
5 return median
M ASTER(M [1 . . . n])
1 lower 1
2 upper n
3 median NIL
4 while lower upper and median = NIL
5
do i lower + (upper lower)/2
6
send n i
7
receive b1
8
receive b2
9
cur M [i]
10
if b1 < cur < b2
11
then median= cur
12
elseif cur < b1
13
then lower i +1
14
else upper i 1
15 send DONE
16 send median
17 return median
S LAVE(S[1 . . . n])
1 while true
2
do receive j
3
if j = DONE
4
then receive median
5
return median
6
if j = 0
7
then send
8
else send S[j]
9
send S[j + 1]
Running Time. Each individual statement is (1) except for the calls to HeapSort
(line 1 of A LICE and B OB), which requires (n lg n) time. The loop in M ASTER
performs a binary search, which (as we saw in lecture) requires (lg n) iterations.
Each iteration does (1) work, so the total running time for the loop is (lg n). The
loop in S LAVE terminates when it receives a DONE value, which happens exactly
when the loop in M ASTER terminates; thus, S LAVE is also (lg n). Alice and Bob
each execute M ASTER, S LAVE and HeapSort; HeapSort dominates, yielding a nal
running time of (n lg n).
Communication cost. Most of the communication is in the loop of M ASTER, in
which three items are relayed between Alice and Bob per each iteration. Since this
loop executes (lg n) times, it contributes (lg n) communications. The items sent
and received at the end of M ASTER contribute (1) communications, leaving the total
at (lg n).
Alternate Solution:
An alternate solution is simpler in some ways, but cannot be adapted to solve Part (b).
As before, Alice and Bob begin by sorting their arrays using a deterministic (n log n)
algorithm such as HeapSort or MergeSort. Then, Alice assumes the role of the master
and Bob the role of the slave. When Alice sends a value A[i] to Bob, Bob returns the
number of elements, count(A[i]), in his array that are less than A[i]. Because A is
sorted, the element A[i] is the combined median if and only if i + count(A[i]) = n.
Alice checks this condition and returns A[i] as the median if the condition holds. If the
condition fails, then she proceeds to do a binary search within her array. The search
is on i, with an initial range of [1, n]. On each step, she descends into the top half of
the range if i + count(A[i]) < n; she descends into the bottom half if i + count > n.
Because the quantity i+count (A[i]) is a monotonic function of i, the search terminates
with A[i] = i + count(A[i]) if the combined median is stored within A.
If the combined median is not held in A, then Alices binary search terminates after
1 + lg n steps and returns a value of NIL. In this case, Alice and Bob swap roles,
with Bob becoming the master and Alice the slave. The procedure is repeated, and this
time the binary search returns the combined median because it must be stored within
B.
For clarity, pseudocode for this algorithm is given below.
A LICE(A[1 . . . n])
1 H EAP S ORT(A)
2 median M ASTER(A)
3 if median = NIL
4
then median S LAVE(A)
5 return median
B OB(B[1 . . . n])
1 H EAP S ORT(B)
2 median S LAVE(B)
3 if median = NIL
4
then median M ASTER(B)
5 return median
M ASTER(M [1 . . . n])
1 lower 1
2 upper n
3 median NIL
4 while lower upper and median = NIL
5
do i lower + (upper lower)/2
6
send A[i]
7
receive count
8
if i + count = n
9
then median= M [i]
10
elseif i + count < n
11
then lower i +1
12
else upper i 1
13 send DONE
14 send median
15 return median
S LAVE(S[1 . . . n])
1 while true
2
do receive val
3
if val = DONE
4
then receive median
5
return median
6
else send |i [1, n] : S[i] val|
Running Time. All but three statements are (1) time. Both Alice and Bob call
HeapSort, which is (n lg n). Line 6 of S LAVE counts how many elements in S are
at most val. This can be implemented in (n) time with a brute-force comparison or,
because the array is sorted, in (lg n) time using a binary search. The last statements
of interest are lines 6-7 of M ASTER, which wait for one iteration of S LAVE. Since the
slave executes (lg n) operations between a receive and send statement, lines 6-7 of
M ASTER are also (lg n).
It remains to account for the loops. The loop in M ASTER is performing a binary
search, which (as we saw in lecture) requires (lg n) iterations. Each iteration does
(lg n) work, so the total running time for the loop is (lg 2 n). The loop in S LAVE
terminates when it receives a DONE value, which happens exactly when the loop
in M ASTER terminates; thus, S LAVE is also (lg 2 n). Alice and Bob each execute
M ASTER, S LAVE and HeapSort; HeapSort dominates, yielding a nal running time of
(n lg n).
Communication cost. Most of the communication is in the loop of M ASTER, in
which two items are relayed between Alice and Bob per each iteration. Since this
loop executes (lg n) times, it contributes (lg n) communications. The items sent
and received at the end of M ASTER contribute (1) communications, leaving the total
at (lg n).
(b) Give a randomized algorithm for computing the combined median of A and B. Your
algorithm should run in expected O(n) time and use expected O(log n) communica
Solution:
The algorithm is almost identical to Part (a). As before, Alice starts as the master
and conducts a binary search through Bobs elements, looking for A[i] such that
B[n i] < A[i] < B[n i + 1]. The only difference is that, instead of nding
the ith largest element by sorting the arrays and referencing A[i] directly, we use
R ANDOMIZED -S ELECT to nd the ith largest element in expected linear time. As
the algorithm is so close to Part (a), we describe only the differences below; in the
pseudocode, the modied lines are denoted by .
A LICE(A[1 . . . n])
1 H EAP S ORT(A)
2 median M ASTER(A)
3 if median = NIL
4
then median S LAVE(A)
5 return median
B OB(B[1 . . . n])
1 H EAP S ORT(B)
2 median S LAVE(B)
3 if median = NIL
4
then median M ASTER(B)
5 return median
10
2 upper n
3 median NIL
5
do i lower + (upper lower)/2
6
send n i
7
send n upper
8
send n lower
9
receive b1
10
receive b2
11
cur R ANDOMIZED -S ELECT(M, lower, upper, i lower + 1)
12
if b1 < cur < b2
13
then median= cur
14
elseif cur < b1
15
then lower i +1
16
else upper i 1
17 send DONE
18 send median
19 return median
S LAVE(S[1 . . . n])
1 while true
2
do receive j
3
if j = DONE
4
then receive median
5
return median
6
receive bottom
7
receive top
8
if j = 0
9
then send
10
else send R ANDOMIZED -S ELECT(S, bottom, top, j bot + 1)
11
send R ANDOMIZED -S ELECT(S, bottom, top, j bot + 2)
Correctness. We need to show that all calls to R ANDOMIZED -S ELECT return the cor
responding values from Part (a). On line 11 of M ASTER, we have replaced M [i], the
ith smallest value of the sorted array M , with R ANDOMIZED -S ELECT(M, lower, upper, i
lower+1). To show that these values are equivalent, we need the following loop invari
ant I: before each iteration of the loop, M [lower . . . upper] contains the ith smallest
element in M for all i {lower, . . . , upper}. This invariant is true at the beginning,
since lower = 1, upper = n and all of the elements are within the range.
11
For the inductive step, assume I is true on the current iteration. Then the call to
R ANDOMIZED -S ELECT will partition around the ith smallest element in M , because
1) (by the inductive hypothesis) the smallest element in M [lower . . . upper] is the
lowerth smallest element in M , 2) by our call to R ANDOMIZED -S ELECT, we are
selecting for the i lowerth smallest element in M [lower . . . upper] (we also add 1
to compensate for the 1-based array indexing) and 3) lower + i lower = i. Thus, the
hypothesis will be satised for the range {lower, . . . , i} and {i, . . . , upper} because
the PARTITION subroutine of R ANDOMIZED -S ELECT will place elements on the ap
propriate side of the pivot i. Finally, the inductive hypothesis will hold on the next
iteration because we assign either lower or upper to be adjacent to i (but excluding i
from the next range).
Using the invariant, we conclude that the call to R ANDOMIZED -PARTITION on line
11 of M ASTER returns the ith smallest element in M , which is equivalent to the ex
pression M [i] from Part (a).
It remains to show the equivalent property for lines 10 and 11 of S LAVE. This is done
using the same loop invariant, but translating j = n i, bottom = n upper, and
top = n lower across the call between M ASTER and S LAVE. In this way, we can
show that lines 10 and 11 of S LAVE return the nith and ni+ 1th smallest elements
of S, respectively.
We have shown that our changes from Part (a) preserve the behavior of the algorithm,
and thus the algorithm remains correct.
Running Time. We can write a recurrence to model the running time of the main loop
in M ASTER. Let m = upperlower. On each iteration, m decreases to at most m/2
and R ANDOMIZED -S ELECT runs three times (once in M ASTER, twice in S LAVE)
over a segment of size m, with expected running time (m). Thus E[T (m)] =
E[T (m/2)] + (m), and by Case 3 of the Master Theorem, E[T (m)] = (m).
Finally, noting that m = n at the beginning of the procedure, we have that the ex
pected running time is (n).
Communication cost. The communication cost is identical to Part (a), as the loop
in M ASTER still executes (lg n) iterations and sends (1) items on each iteration.
Thus, the total number of communications is (lg n). (Note that this algorithm gives
a deterministic bound on the number of communications.)
12
match. If the contestant and gladiator have equal strength, then they are perfect equals and a
tie is declared. We assume that each contestant is the perfect equal of exactly one gladiator, and
each gladiator is the perfect equal of exactly one contestant. However, as the gladiators sometimes
change from one show to another, we do not know the ordering of strength among the gladiators.
The game show currently uses a round-robin format in which (n 2 ) matches are played and con
testants are ranked according to their number of victories. Since few contestants can happily endure
(n) gladiator confrontations, you are called in to optimize the procedure.
(a) Give a randomized algorithm for ranking the contestants. Using your algorithm, the
Solution:
The problem statement does not describe exactly how the contestants and gladiators
are specied, so we rst need to come up with a reasonable representation for the
input. Lets assume the contestants and gladiators are provided to us in two arrays
C[1 . . . n] and G[1 . . . n], where we are allowed to compare elements across, but not
within, these two arrays.
We use a divide-and-conquer algorithm very similar to randomized quicksort. The al
gorithm rst performs a partition operation as follows: pick a random contestant C[i].
Using this contestant, rearrange the array of gladiators into three groups of elements:
rst the gladiators weaker than C[i], then the gladiator that is the perfect equal of C[i],
and nally the gladiators stronger than C[i]. Next, using the gladiator that is the per
fect equal of C[i] we perform a similar partition of the array of contestants. This pair
of partitioning operations can easily be implemented in (n) time, and it leaves the
contestants and gladiators nicely partitioned so that the pivot contestant and glad
iator are aligned with each other and all other contestants and gladiators are on the
correct side of these pivots weaker contestants and gladiators precede the pivots,
and stronger contestants and gladiators follow the pivots. Our algorithm then nishes
by recursively applying itself to the subarrays to the left and right of the pivot position
to sort these remaining contestants and gladiators. We can assume by induction on n
that these recursive calls will properly sort the remaining contestants.
To analyse the running time of our algorithm, we can use the same analysis as that
of randomized quicksort. We are performing a partition operation in (n) time that
splits our problem into two subproblems whose sizes are randomly distributed ex
actly as would be the subproblems resulting from a partition in randomized quicksort.
Therefore, applying the analysis from quicksort, the expected running time of our
algorithm is (n log n).
Interesting side note: Although devising an efcient randomized algorithm for this
problem is not too difcult, it appears to be very difcult to come up with a deter
ministic algorithm with running time better than the trivial bound of O(n 2 ). This
13
remained an open research question until the mid-to-late 90s, when a very compli
cated deterministic algorithm with (n log n) running time was nally discovered.
This problem provides a striking example of how randomization can help simplify the
task of algorithm design.
(b) Prove that any algorithm that solves part (a) must use (n log n) matches in the worst
case. That is, you need to show a lower bound for any deterministic algorithm solving
this problem.
Solution:
Lets use a proof based on decision trees, as we did for comparison-based sorting.
Note that we can model any algorithm for sorting contestants and gladiators as a de
cision tree. The tree will be a ternary tree, since every comparison has three possible
outcomes: weaker, equal, or stronger. The height of such a tree corresponds to the
worst-case number of comparisons made by the algorithm it represents, which in turn
is a lower bound on the running time of that algorithm. We therefore want a lower
bound of (n log n) on the height, H, of any decision tree that solves part (a). To
begin with, note that the number of leaves L in any ternary tree must satisfy
L 3H .
Next, consider the following class of inputs. Let the input array of gladiators G be
xed and consist of n gladiators sorted in order of increasing strength, and consider
one potential input for every permutation of the contestants. Our algorithm must in
this case essentially sort the array of contestants. In our decision tree, if two different
inputs of this type were mapped to the same leaf node, our algorithm would attempt
to apply to both of these the same permutation of contestants, and it follows that the
algorithm could not compute a ranking correctly for both of these inputs. Therefore,
we must map every one of these n! different inputs to a distinct leaf node, so
L
3H
H
H =
n!
n!
log3 n!
(n log n) [Using Stirlings approximation]
Introduction to Algorithms
Massachusetts Institute of Technology
Professors Piotr Indyk and Charles E. Leiserson
Prof. Potemkin realizes that this packing problem is NP-hard, which means that the research
community has not yet found a polynomial time algorithm 1 that solves this problem exactly.
He thinks of the heuristic approach called BEST-PACK:
1.Take the books in the order in which they appear on his shelves.
2.For each book, scan the boxes in increasing order of the remaining capacity and place the
book in the rst box in which it ts.
(a) Describe a data structure that supports efcient implementation of BEST-PACK. Show
Solution: BEST-PACK can be implemented using any data structure that supports the
following three operations:
1. Insert(x), where x is an element and key[x] is a number
2. Delete(x)
3. Successor(x), which reports the smallest x such that key[x] k
There are several ways to obtain such a data structure. For example, one can use redblack trees or 2 3 trees. Because they are balanced, they support Insert, Delete and
Successor operations in O(log n) time. Even though the Successor operation was not
explained for 2 3 trees, they can be implemented by modifying search.
Our implementation is as follows: We use the remaining capacity of the boxes as the
key in the binary tree. Suppose that the elements weigh w1 , . . . , wn . Then, for a given
book with weight wi , if there are no boxes that are already used and whose remaining
capacity is greater than wi (i.e., the successor of wi ), then we assign wi to a new box.
(b) Analyze the running time of your implementation.
Solution: The BEST-PACK implementation performs O(n) operations on the data
structure which implies that the total running time is O(n log n)
That is, an algorithm with running time O(nk ) for some xed k.
Soon, Prof. Potemkin comes up with another heuristic WORST-PACK, which is as follows:
1.Take the books in the order in which they appear on his shelves.
2.For each book, nd a partially used box which has the maximum remaining capacity. If
possible, place the book in that box. Otherwise, put the book into a new box.
(c) Describe a data structure that supports an efcient implementation of WORST-PACK.
Solution: WORST-PACK can be implemented using any priority queue data structure.
We learned in recitation that a heap implements this data structure in O(log n) time.
You can also use a balanced search tree to implement these operations.
Our implementation is as follows: Pick a book. Delete the maximum from the priority
queue. If the capacity is greater than the weight of the book, insert the book and reduce
the capacity of the box. Reinsert the box in the priority queue. Otherwise pick a new
box and insert the book.
(d) Analyze the running time of your implementation.
Solution: Our implementation performs O(n) operations. This means that the total
running time is O(n log n).
4
Problem 3-2. AVL Trees
An AVL tree is a binary search tree with one additional structural constraint: For any of its internal
nodes, the height difference between its left and right subtree is at most one. We call this property
balance. Remember that the height is the maximum length of a path to the root.
For example, the following binary search tree is an AVL tree:
In this case, we can rebalance the tree by doing a simple operation, called a rotation, as follows:
=
Rotation
1
Unbalanced
Balanced
Though the original tree was balanced, more than one rotation is needed to restore
balance following the insertion. This can be seen by an exhaustive enumeration of the
rotation possibilities.
The problem asks for a tree of height 4, so we can extend the above example into a
larger tree:
(b) Denote the minimum number of nodes of an AVL tree of height h by M (h). A tree
of height 0 has one node, so M (0) = 1. What is M (1)? Give a recurrence for M (h).
Show that M (h) is at least Fh , where Fh is the hth Fibonacci number.
Solution: M (1) = 2. For h 2, the tree will consist of a root plus two subtrees.
Since the tree is of height h, one of the subtrees must be of height h 1. The minimum
number of nodes in this subtree is M (h1). Since the height of the subtrees can differ
by at most 1, the minimum number of nodes in the other subtree is M (h 2). Thus
the total number of nodes is M (h) = M (h 1) + M (h 2) + 1.
Note that M (h) is remarkably similar to the Fibonacci numbers and that the recursion
holds for the worse case AVL trees, which are called Fibonacci trees. It is easy to
h+3
h
1+ 5
h+3
Introduction to Algorithms
Massachusetts Institute of Technology
Professors Piotr Indyk and Charles E. Leiserson
stack S1 . To remove from our queue we rst check if S2 is empty, and if so, we dump S1 into S2
(that is, we pop each element from S1 and push it immediately onto S2 ). Then we pop from S2 .
For instance, if we execute I NSERT(a), I NSERT(b), D ELETE(), the results are:
S1
I NSERT(a) S1
I NSERT(b) S1
D ELETE() S1
S1
=[]
S2 =[]
=[a]
S2 =[]
=[b a] S2 =[]
=[]
S2 =[a b] dump
=[]
S2 =[b]
pop (returns a)
Suppose each push and pop costs 1 unit of work, so that performing a dump when S 1 has n elements
costs 2n units (since we do n pushes and n pops).
(a) Suppose that (starting from an empty queue) we do 3 insertions, then 2 removals,
then 3 more insertions, and then 2 more removals. What is the total cost of these 10
operations, and how many elements are in each stack at the end?
running time of one of the operations be (give an exact, non-asymptotic answer)? Give
a sequence of operations that induces this behavior, and indicate which operation has
Solution: An insertion always takes 1 unit, so our worst-case cost must be caused by
a removal. No more that n elements can ever be in S1 , and no fewer than 0 elements
can be in S2 . Therefore the worst-case cost is 2n + 1: 2n units to dump, and one extra
to pop from S2 . This bound is tight, as seen by the following sequence: perform n
insertions, then n removals. The rst removal will cause a dump of n elements plus a
pop, for 2n + 1 work.
(c) Suppose we perform an arbitrary sequence of insertions and removals, starting from
an empty queue. What is the amortized cost of each operation? Give as tight (i.e.,
non-asymptotic) of an upper bound as you can. Use the accounting method to prove
your answer. That is, charge $x for insertion and $y for deletion. What are x and y?
Solution: The tightest amortized upper bounds are 3 units per insertion, and 1 unit per
removal. We will prove this 2 ways (using the accounting and potential methods; the
aggregate method seems too weak to employ elegantly in this case). (We would also
accept valid proofs of 4 units per insertion and 0 per removal, although this answer is
looser than the one we give here.)
Here is an analysis using the accounting method: with every insertion we pay $3: $1
is used to push onto S1 , and the remaining $2 remain attached to the element just
inserted. Therefore every element in S1 has $2 attached to it. With every removal we
pay $1, which will (eventually) be used to pop the desired element off of S 2 . Before
that, however, we may need to dump S1 into S2 ; this involves popping each element
off of S1 and pushing it onto S2 . We can pay for these pairs of operations with the $2
attached to each element in S1 .
(d) Now well analyze the structure using the potential method. For a queue Q imple
Now we compute the amortized costs: for an insertion, we have S 1i+1 = S1i + 1, and
the actual cost ci = 1, so
ci = ci + (Qi+1 ) (Qi ) = 1 + 2(S1i + 1) 2(S1i ) = 3.
For a removal, we have two cases. First, when there is no dump from S 1 to S2 , the
actual cost is 1, and S1
i+1 = S1i . Therefore ci = 1. When there is a dump, the actual
cost is 2|S1 i | + 1, and we have S1i+1 = 0. Therefore we get
ci = (2|S1i | + 1) + 0 2|S1i | = 1
as desired.
Problem 4-2. David Digs Donuts
Your TA David has two loves in life: (1) roaming around Massachusetts on his forest-green Can
nondale R300 road bike, and (2) eating Boston Kreme donuts. One Sunday afternoon, he is biking
along Main Street in Acton, and suddenly turns the corner onto Mass Ave. (Yes, that Mass Ave.)
His growling stomach announces that it is time for a donut. Because Mass Ave has so many donut
shops along it, David decides to nd a shop somewhere along that street. He faces two obstacles in
his quest to satisfy his hunger: rst, he does not know whether the nearest donut shop is to his left
or to his right (or how far away the nearest shop is); and second, when he goes riding his contact
lenses dry out dramatically, blurring his vision, and he cant see a donut shop until he is directly in
front of it.
You may assume that all donut shops are at an integral distance (in feet) from the starting location.
(a) Give an efcient (deterministic) algorithm for David to locate a donut shop on Mass
Ave as quickly as possible. Your algorithm will be online in the sense that the location
of the nearest donut shop is unknown until you actually nd the shop. The algorithm
should be O(1)-competitive: if the nearest donut shop is distance d away from Davids
starting point, the total distance that David has to bike before he gets his donut should
be O(d). (The optimal ofine algorithm would require David to bike only distance d.)
Solution: WLOG, lets call the two directions of Mass Ave east and west.
1.
2.
3.
4.
Notice that you are back at the origin after every iteration of the loop.
Suppose that the nearest donut shop is d feet away from the origin. Let k be such that
2k < d 2k+1 . Observe that in the ith iteration of the loop, we explore the stretch of
road between the origin and 2i , so after k + 2 iterations we are guaranteed to have
The total distance that we travel in the ith iteration is 22i , so the total distance traveled
i+1
in these k + 2 iterations is k+2
2 2k+3 = 16 2k < 16d. Thus the algorithm
i=0 2
is 16-competitive.
(b) Optimize the competitive ratio for your algorithmthat is, minimize the constant hid
Solution: The only tweak to the above is to more tightly analyze the last iteration.
In the (k + 2)nd iteration, we are done after we travel distance d, since we were
at the origin and had to travel only distance d. Thus the total distance we travel is
k+1 i+1
+ d 8 2k + d 9d. (One can also try to optimize the base of the
i=0 2
exponential search, but it turns out that two is optimal.)
(c) Suppose you ip a coin to decide whether to start moving to the left or to the right ini
tially. Show that incorporating this step into your algorithm results in an improvement
Solution: Then, using the above notation, with probability 1/2 we will nd the shop
in the (k + 1)st iteration, and with probability 1/2 we will nd it in the (k + 2)nd
k+1
i=0
k+2
2
k + d)/2 + (
i=0
Introduction to Algorithms
Massachusetts Institute of Technology
Professors Piotr Indyk and Charles E. Leiserson
Exercise 4-3. Do exercise 15.4-4 on page 356 in CLRS and show how to reconstruct the actual
You have an input text consisting of a sequence of n words of lengths 1 , 2 , . . . , n , where the
length of a word is the number of characters it contains. Your printer can only print with its built-in
Courier 10-point xed-width font set that allows a maximum of M characters per line. (Assume
that i M for all i = 1, . . . , n.) When printing words i and i + 1 on the same line, one space
character (blank) must be printed between the two words. Thus, if words i through j are printed
on a line, the number of extra space characters at the end of the linethat is, after word jis
M j + i jk=i k .
To produce nice-looking output, the heuristic of setting the cost to the square of the number of
extra space characters at the end of the line has empirically shown itself to be effective. To avoid
the unnecessary penalty for extra spaces on the last line, however, the cost of the last line is 0. In
other words, the cost linecost(i, j) for printing words i through j on a line is given by
linecost(i, j) =
M j + i j
k
k=i
The total cost for typesetting a paragraph is the sum over all lines in the paragraph of the cost of
each line. An optimal solution is an arrangement of the n words into lines in such a way that the
total cost is minimized.
(a) Argue that this problem exhibits optimal substructure.
Solution: First, notice that linecost(i, j) is dened to be if the words i through j
do not t on a line to guarantee that no lines in the optimal solution overow. (This
relies on the assumption that the length of each word is not more than M .) Second,
notice that linecost(i, j) is dened to be 0 when j = n, where n is the total number
of words; only the actual last line has zero cost, not the recursive last lines of subprob
lems, which, since they are not the last line overall, have the same cost formula as any
other line.
Consider an optimal solution of printing words 1 through n. Let i be the index of the
rst word printed on the last line of this solution. Then typesetting of words 1, . . . , i1
must be optimal. Otherwise, we could paste in an optimal typesetting of these words
and improve the total cost of solution, a contradiction. Please notice that the same
cut-and-paste argument can be applied if we take i to be the index of the rst word
printed on the kth line, where 2 k n. Therefore this problem displays optimal
substructure.
(b) Dene recursively the value of an optimal solution.
Solution: Let c(j) be the optimal cost of printing words 1 through j. From part (a),
we see that given the optimal i (i.e., the index of the rst word printed on the last line
of an optimal solution), we have c(j) = c(i 1) + linecost(i, j). But since we do
not know what i is optimal, we need to consider every possible i, so our recursive
denition of the optimal cost is
c(j) = min {c(i 1) + linecost(i, j)} .
1ij
k=1
Filling in this array takes O(n) time using recursion. Using the auxillary array we
linecost(i, j) = 0
This algorithm uses (n) space for the arrays and runs in O(n2 ) time, since each value
(M + 1)/2 words can t on a single line, we can reduce running time to O(nM )a
(d) Write code (in any language you wisheven Visual Java ++ :-) 1 ) to print an optimal
arrangement of the words into lines. For simplicity, assume that a word is any se
quence of characters not including blanksso a word is everything included between
two space characters (blanks).
1
(d) requires 5 parts: you should turn in the code you have written, and the output of your program
on the two input samples using two values of M (the maximum number of characters per line),
namely M = 72 and M = 40, on each input sample.
Sample 1 is from A Capsule History of Typesetting by Brown, R.J. Sample 2 is from Out of Their
Minds, by Shasha, Lazere. Remember that collaboration, as usual, is allowed to solve problems,
but you must write your program by yourself.
/* NOTE: This is an implementation of the O(nM) algorithm. */
#include <stdio.h>
#include <limits.h>
/* macros */
int auxL[WORD_NUM+1];
/* auxillary array for computing lengths
of lines - MM*/
/* function prototypes */
/* verify arguments */
exit(1);
exit(2);
exit(3);
n = 1;
while(!feof(ifile)) {
strcpy(words[n++], read_word);
if(n == WORD_NUM)
n--;
auxL[0] = 0;
l = 0;
do {
l++;
strcat(lines[l], words[i]);
int i, j;
long c[WORD_NUM+1];
c[j] = LONG_MAX;
int k;
if(extras < 0)
return 0;
else
Solutions:
sample1 72
COST = 160
1:[67] The first practical mechanized type casting machine was invented in
2:[69] 1884 by Ottmar Mergenthaler. His invention was called the "Linotype".
3:[72] It produced solid lines of text cast from rows of matrices. Each matrice
5:[69] letter had been engraved or stamped. The line-composing operation was
11:[70] the punched code into electrical signals that could be sent by wire to
12:[71] tape-punching units in many cities simultaneously. The first major news
sample1 40
COST = 360
sample2 72
COST = 229
1:[65] Throughout his life, Knuth had been intrigued by the mechanics of
2:[70] printing and graphics. As a boy at Wisconsin summer camp in the 1940s,
3:[71] he wrote a guide to plants and illustrated the flowers with a stylus on
4:[69] the blue ditto paper that was commonly used in printing at that time.
5:[71] In college, he recalls admiring the typeface used in his math texbooks.
6:[71] But he was content to leave the mechanics of designing and setting type
7:[72] to the experts. "I never thought I would have any control over printing.
8:[71] Printing was done by typographers, hot lead, scary stuff. Then in 1977,
9:[71] I learned about new printing machines that print characters made out of
10:[69] zeros and ones, just bits, no lead. Suddenly, printing was a computer
12:[66] tools using the new technology with which to write my next books."
13:[67] Knuth designed and implemented TeX, a computer language for digital
15:[68] thoroughness. For example, he wrote a paper called "The letter S" in
16:[67] which he dissects the mathematical shape of that letter through the
17:[67] ages, and explains his several day effort to find the equation that
sample2 40
COST = 413
33:[8] outline
Here is what Sample 1 should look like when typeset with M = 50. Feel free to use this output
to debug your code.
The first practical mechanized type casting
(e) Suppose now that the cost of a line is dened as the number of extra spaces. That is,
when words i through j are put into a line, the cost of that line is
M j + i k=i k otherwise;
and that the total cost is still the sum over all lines in the paragraph of the cost of each
line. Describe an efcient algorithm that nds an optimal solution in this case.
Solution: We use a straightforward greedy algorithm, which puts as many words as
possible on each line before going to the next line. Such an algorithm runs in linear
time.
Now we show that any optimal solution has the same cost as the solution obtained by
this greedy algorithm. Consider some optimal solution. If this solution is the same as
the greedy solution, then we are done. If it is different, then there is some line i which
has enough space left over for the rst word of the next line. In this case, we move
the rst word of line i + 1 to the end of line i. This does not change the total cost,
since if the length of the word moved is l, then the reduction to the cost of line i will
10
be l + 1, for the word and the space before it, and the increase of the cost of line i + 1
will also be l + 1, for the word and the space after it. (If the moved word was the only
word on line i + 1, then by moving it to the previous line the total cost is reduced, a
contradiction to the supposition that we have an optimal solution.) As long as there
are lines with enough extra space, we can keep moving the rst words of the next lines
back without changing the total cost. When there are no longer any such lines, we will
have changed our optimal solution into the greedy solution without affecting the total
cost. Therefore, the greedy solution is an optimal solution.
Problem 4-2. Manhattan Channel Routing
A problem that arises during the design of integrated-circuit chips is to hook components together
with wires. In this problem, well investigate a simple such problem.
In Manhattan routing, wires run on one of two layers of an integrated circuit: vertical wires run
on layer 1, and horizontal wires run on layer 2. The height h is the number of horizontal tracks
used. Wherever a horizontal wire needs to be connected to a vertical wire, a via connects them.
Figure 1 illustrates several pins (electrical terminals) that are connected in this fashion. As can be
seen in the gure, all wires run on an underlying grid, and all the pins are collinear.
In our problem, the goal is to connect up a given set of pairs of pins using the minimum number
of horizontal tracks. For example, the number of horizontal tracks used in the routing channel of
Figure 1 is 3 but fewer might be sufcient.
Let L = {(p1 , q1 ), (p2 , q2 ), . . . , (pn , qn )} be a list of pairs of pins, where no pin appears more than
once. The problem is to nd the fewest number of horizontal tracks to connect each pair. For exam
ple, the routing problem corresponding to Figure 1 can be specied as the set {(1, 3), (2, 5), (4, 6), (8, 9)}.
(a) What is the minimum number of horizontal tracks needed to solve the routing problem
in Figure 1?
Solution: You can verify that the wire connecting pins 4 and 6 could be at the same
height as the wire connecting pins 1 and 3, making the number of horizontal track
needed 2. Note that this is the minimum possible. Otherwise the wire connecting pins
2 and 3 and the wire connecting 1 and 3 would be on the same track, violating the
problem specications.
(b) Give an efcient algorithm to solve a given routing problem having n pairs of pins us
ing the minimum possible number of horizontal tracks. As always, argue correctness
(your algorithm indeed minimizes the number of horizontal tracks), and analyze the
running time.
Algorithm description
The following algorithm routes pin pairs greedily into available horizontal tracks in
order of the smaller pin of a pair.
11
h=3
Figure 1: Pins are shown as circles. Vertical wires are shown as solid. Horizontal wires are dashed.
Vias are shown as squares.
1. Go through L and if qi > pi swap them. For each pair, we call the smaller pin,
start, and the larger, end. Sort the start and end pins of the pairs in an increasing
order. The resulting list contains 2n values.
2. Place all available horizontal tracks in a stack S.
3. Go through the list in sorted order.
If the current pin is a start pin, pop the rst available horizontal track from S
and route it in that horizontal track. If S is empty, then it is not possible to
route all of the pin pairs using the given number of horizontal tracks. Report
an error in this case.
If the current pin is an end pin, look up in which horizontal track it has been
routed and push that horizontal track back onto S.
Correctness
Suppose that the algorithm terminates with a routing requiring m horizontal tracks.
Let k denote the rst pair of pins routed in the mth horizontal track. Let s k denote
the start pin and fk denote the end pin of this pair. Let fl be the earliest nish pin
appearing in the sorted list after sk . Necessarily, fl > sk . The closest routing in each
of the m 1 horizontal tracks already in use starts before sk . Each routing terminates
after fl . Thus there are m routings in between [sk , fl ], i.e., any routing must use at
least m vertical tracks. Thus the routing returned by the algorithm is optimal.
The above argument shows that given an innite supply of horizontal tracks, our algo
rithm will always produce a routing that uses the fewest number of horizontal tracks.
Thus, if the algorithm terminates with an error, it means that a given number of hor
izontal tracks is less than the number of horizontal tracks in an optimal routing, and
12
Introduction to Algorithms
Massachusetts Institute of Technology
Professors Piotr Indyk and Charles E. Leiserson
2
Problem 6-1. Truckin
Professor Almanac is consulting for a trucking company. Highways are modeled as a directed
graph G = (V, E) in which vertices represent cities and edges represent roads. The company is
planning new routes from San Diego (vertex s) to Toledo (vertex t).
(a) It is very costly when a shipment is delayed en route. The company has calculated the
probability p(e) [0, 1] that a given road e E will close without warning. Give an
efcient algorithm for nding a route with the minimum probability of encountering
a closed road. You should assume that all road closings are independent.
Solution:
To simplify the solution, we use the probability q(e) = 1 p(e) that a road will be
open. Further, we remove from the graph roads with p(e) = 1, as they are guaranteed
to be closed and will never be included in a meaningful solution. (Following this
transformation, we can use depth rst search to ensure that some path from s to t
has a positive probability of being open.) By eliminating p(e) = 1, we now have
0 < q(e) 1 for all edges e E. It is important to have eliminated the possibility of
q(e) = 0, because we will be taking the logarithm of this quantity later.
Because the road closings are independent, the probability that a given path will be
open is the product of the probabilities of the edges being open. That is, for each path
r = e1 , e2 , . . . , en , the probability Q(r) of the path being open is:
Q(r) =
q(ei )
i=1
Our goal is to nd the path r, beginning at s and ending at t, that maximizes Q(r).
Taking the negative logarithm of both sides yields:
lg Q(r) = lg
q(ei )
i=1
lg q(ei )
i=1
w(ei )
i=1
The right hand side is a sum of edge weights w(e) along the path from s to t. We
can minimize this quantity using Dijkstras algorithm for single-source shortest paths.
Doing so will yield the path r that minimizes lg Q(r), thereby maximizing Q(r).
This path will have the maximum probability of being open, and thus the minimum
probability of being closed.
Alternate Solution:
It is also possible to modify Dijkstras algorithm to directly compute the path with the
highest probability of being open. As above, let q(e) = 1p(e) denote the probability
that a given road e E will be open, and let Q(r) denote the probability that a given
path r will be open. For a given vertex v, let o[v] denote the maximum value of Q(r)
over all paths r from s to v. Then, make the following modications to Dijkstras
algorithm:
1. Change I NITIALIZE -S INGLE -S OURCE to assign o[s] = 1 for the source vertex
and o[v] = 0 for all other vertices:
I NITIALIZE -S INGLE -S OURCE (G, s)
1 for each vertex v V [G]
2
do o[v] 0
3
[v] NIL
4 o[s] 1
That is, we can reach the source vertex with probability 1, and the probability of
reaching all other vertices will increase monotonically from 0 using R ELAX.
2. Instead of E XTRACT-M IN, use E XTRACT-M AX (and a supporting data structure)
to see which vertex to visit. That is, rst explore paths with the highest probability
of being open.
3. Rewrite the R ELAX step as follows:
R ELAX (u, v, q)
1 if o[v] < o[u] q(e) where e = (u, v)
2
then o[v] o[u] q(e)
3
[v] u
That is, if a vertex v can be reached with a higher probability than before along
the edge under consideration, then increase the probability o[v]. Because the
probabilities of roads being open are independent, the probability of a path being
open is the product of the probabilities of each edge being open.
The argument for correctness parallels that of Dijkstras algorithm, as presented in
lecture. The proof relies on the following properties:
1. Optimal substructure. A sub-path of a path with the highest probability of being
open must also be a path with the highest probability of being open. Otherwise we
could increase the overall probability of being open by increasing the probability
along this sub-path (cut-and-paste).
2. Triangle inequality. Let (u, v) denote the highest probability of a path from u
to v being open. Then for all u, v, x V , we have (u, v) (u, x) (x, v).
Otherwise (u, v) could be increased if we chose the path through x.
3. Well-denedness of shortest paths. Since q(e) [0, 1], the probability of a path
being open can only decrease as extra edges are added to a path. Since we are
exploring the path with the highest probability of being open, this ensures that
there are no analogues of negative-weight cycles.
Properties (1) and (2) are true whenever an associative operator is used to combine
edge weights into a path weight. In Dijkstras shortest path algorithm, the operator is
addition; here it is multiplication.
The running time is O(E + V lg V ). We spend (E) to calculate q(e) = 1 p(e),
and Dijkstras algorithm runs in O(E + V lg V ).
(b) Many highways are off-limits for trucks that weigh more than a given threshold. For a
given highway e E, let w(e) + denote the weight limit and let l(e) + denote
the highways length. Give an efcient algorithm that calculates: 1) the heaviest truck
that can be sent from s to t, and 2) the shortest path this truck can take.
Solution:
First, we modify Dijkstras algorithm to nd the heaviest truck that can be sent from
s to t. The weight limit w(e) is used as the edge weight for e. There are three modi
cations to the algorithm:
1. In I NITIALIZE -S INGLE -S OURCE, assign a value of to the source vertex and a
value of 0 to all other vertices.
2. Instead of E XTRACT-M IN, use E XTRACT-M AX (and a supporting data structure)
to see which vertex to visit. That is, rst explore those paths which support the
heaviest trucks.
3. In the R ELAX step, use min in place of addition. That is, maintain the minimum
weight limit encountered on a given path instead of the total path length from the
source.
As in Part (a), the proof of correctness follows that of Dijkstras algorithm. Since the
min operator is associative, the optimal paths exhibit optimal substructure and support
the triangle inequality. There are no analogues of negative-weight cycles because the
weight supported by a path can only decrease as the path becomes longer (and we are
searching for the heaviest weight possible).
Given the weight of the heaviest truck that can pass from s to t, we can nd the shortest
path as follows. Simply remove all edges from the graph that are less than the weight
of the heaviest truck. Then, run Dijkstras algorithm (unmodied) to nd the shortest
path.
The overall runtime of our algorithms is that of Dijkstras algorithm: O(E + V lg V ).
(c) Consider a variant of (b) in which trucks must make strictly eastward progress with
each city they visit. Adjust your algorithm to exploit this property and analyze the
runtime.
Solution:
Remove from the graph any edges that do not make eastward progress. Because
we always go eastward, there are no cycles in this graph. Thus, we can use DAG S HORTEST-PATHS to solve the problem in (V + E) time. We need only modify the
I NITIALIZE -S INGLE -S OURCE and R ELAX procedures as in (b).
Problem 6-2. Constructing Construction Schedules
Consider a set of n jobs to be completed during the construction of a new ofce building. For
each i {1, 2, . . . , n}, a schedule assigns a time xi 0 for job i to be started. There are some
constraints on the schedule:
1. For each i, j {1, 2, . . . , n}, we denote by A[i, j] the minimum latency from the start
of job i to the start of job j. For example, since it takes a day for concrete to dry, construction
of the walls must begin at least one day after pouring the foundation. The constraint on the
schedule is:
i, j {1, 2, . . . , n} : xi + A[i, j] xj
(1)
2. For each i, j {1, 2, . . . , n}, we denote by B[i, j] the maximum latency from the start
of job i to the start of job j. For example, weatherproong must be added no later than one
week after an exterior wall is erected. The constraint on the schedule is:
i, j {1, 2, . . . , n} : xi + B[i, j] xj
(2)
is, given A[1 . . n, 1 . . n] and B[1 . . n, 1 . . n], construct a matrix C[1 . . n, 1 . . n] such
that the following constraints are equivalent to Equations (1) and (2):
i, j {1, 2, . . . n} : xi xj C[i, j]
(3)
Solution:
Re-arranging Equation (1) yields:
i, j {1, 2, . . . , n} : xi xj A[i, j]
(4)
6
Re-arranging Equation (2) yields:
i, j {1, 2, . . . , n} : xi xj B[i, j]
i, j {1, 2, . . . , n} : xj xi B[i, j]
i, j {1, 2, . . . , n} : xi xj B[j, i]
(5)
(b) Show that the Bellman-Ford algorithm, when run on the constraint graph correspond
ing to Equation (3), minimizes the quantity (max{xi } min{xi }) subject to Equation
xk xk1
0
C[1, 2]
C[2, 3]
C[k 1, k]
k1
C[i, i + 1]
i=1
= w(p)
That is, in any solution that satises the constraints, xk cannot be greater than w(p),
the weight of the shortest path from s to vk . As Bellman-Ford sets xk to the shortest
path value, this implies that xk is as large as possible.
(c) Give an efcient algorithm for minimizing the overall duration of the construction
schedule. That is, given A[1 . . n, 1 . . n] and B[1 . . n, 1 . . n], choose {x1 , x2 , . . . , xn }
for all xi . Assume that an unlimited number of jobs can be performed in parallel.
Solution:
The algorithm is as follows:
1. Construct a constraint graph from the constraints in Equation (3). However, if
C[i, j] = , then do not add the edge (vj , vi ) to the graph.
2. Run Bellman-Ford on the constraint graph to obtain a solution {x1 , x2 , . . . , xn }.
3. Set y = min{xi } and calculate a new solution xi = xi y for all xi . Output
{x1 , x2 , . . . , xn }.
This algorithm differs from Part (b) in two ways. First, to incorporate the constraint
xi 0, we simply shift the solution xi (in Step 3) so that all values are non-negative.
This linear shift will not affect the feasibility of the difference constraints, as the
difference between each pair of variables remains unchanged. Also, min{xi } =
min{xi } y = 0, which (in combination with Theorem 2) implies that the algorithm
minimizes max{xi } subject to the constraints.
The second difference between the algorithm and Part (b) is in the construction of the
constraint graph. In order to improve the runtime, we omit edges from the constraint
graph that correspond to an innite weight C[i, j]. Because these edge weights are
innite, they will not impact the shortest path found by Bellman-Ford.
The running time is (V 2 ) for Step 1, (V E) for Step 2, and (V ) for Step 3; the
overall runtime is (V 2 + V E). In terms of the scheduling problem, the running time
is (n2 + nk), where k represents the number of (non-innite) constraints between
jobs. If each job has at least one constraint, then the runtime is (nk). Though
k = (n2 ) in the worst case, we would expect the number of constraints to be sparse
in practice. Thus, the algorithm above is a signicant improvement over the (n3 )
algorithm that would result from a naive construction of the constraint graph.
(d) If the constraints are infeasible, wed like to supply the user with information to help in
diagnosing the problem. Extend your algorithm from (c) so that, if the constraints are
infeasible, your algorithm prints out a set S of conicting constraints that is minimal
that is, if any constraint is dropped from S, the remaining constraints in S would be
feasible.
Solution:
A simple algorithm is to run depth-rst search on the constraint graph, starting from
vertex s. Upon encountering a back edge, construct a cycle in the graph by tracing
back through the predecessor matrix to the target of the back edge. Then print out the
constraint corresponding to each edge in the cycle; these constraints form a set S as
specied.
A set of linear difference constraints are conicting if and only if they form a cycle
in the constraint graph. Thus, identifying a single cycle in the constraint graph will
indicate a set of conicting constraints S. This set is minimal in that, with the removal
of any edge, the cycle is broken and the remaining constraints in S are feasible.
The running time is (n + k), where k is the number of (non-innite) constraints
between jobs. This runtime follows directly from the (V + E) runtime of depth-rst
search on the constraint graph.
Alternate Solution:
It is also possible to utilize the predecessor matrix that is constructed as part of the
Bellman-Ford algorithm. If the last pass of Bellman-Ford is modied to update the
predecessor matrix for the vertices under consideration, then the infeasible constraints
will be manifested as cycles in the predecessor graph. A cycle can be detected in the
predecessor graph using depth rst search. In this case, the runtime for the search is
(V ) = (n), since each vertex has at most one incoming edge (E = (V )).
While this is arguably a more elegant solution, the asymptotic runtime is no better than
the rst solution when considered as an extension of Bellman-Ford. Bellman-Ford is
(nk), which dominates the overall runtime.
Note:
It is not a correct solution to simply trace back in the predecessor graph from the rst
edge that fails the test in Bellman-Ford. If there are multiple cycles in the graph, the
rst edge found might not be part of a cycle in the predecessor graph.
Problem 6-3. Honeymoon Hiking
Alice and Bob (after years of communicating in private) decide to get married and go hiking
for their honeymoon. They obtain a map, which they naturally regard as an undirected graph
G = (V, E); vertices represent locations and edges represent trails. Some of the locations are bus
stops, denoted by S V .
Let w(e) + denote the length of a trail e E, and let h(v) denote the elevation (height)
of a location v V . You may assume that no two locations have exactly the same elevation.
(a) Give an efcient algorithm to nd the shortest romantic hike for Alice and Bob (if any
Solution:
The algorithm is as follows:
1. Construct a directed acyclic graph GU = (VU , EU ) that represents the uphill trail
segments. The vertices in this graph are the same as the original: VU = V . For
each undirected edge in G, there is an edge in GU that is directed uphill:
EU = {(u, v) : (u, v) E and h(u) < h(v)}
Because the elevation of each vertex is distinct, one direction of each trail segment
must be strictly uphill. This implies that GU is acyclic, as it is impossible to form
a loop while hiking strictly uphill.
2. For each bus stop s S, compute the shortest uphill path to all other vertices
v V . As GU is acyclic, this can be done by running DAG -S HORTEST-PATHS
from each s S over the graph GU .
3. For each vertex v VU , compute the two closest bus stops s1 [v] and s2 [v] from
which one can reach v by hiking strictly uphill. This can be done by taking the
two minima, over all vertices s S, of the shortest uphill path from s to v (as
computed in (2)). (Note that if v is a bus stop, then one of these paths has zero
lengththat is, if v S then s1 [v] = v or s2 [v] = v.)
4. Observe that for every vertex v, there is a romantic hike s1 [v] v s2 [v].
Further, this is the shortest romantic hike that has v as its high point (i.e., where
the hike transitions from uphill to downhill). Thus, the shortest romantic hike
overall can be found by taking the minimum over all high points v V . Using
l[v] to denote the length of the path s1 [v] v s2 [v] in G, the shortest hike is:
Shortest = min l[v]
vV
After all, what is more frustrating than going uphill after you have started going downhill?
10
(b) Give an efcient algorithm to nd the longest romantic hike for Alice and Bob (if any
Solution:
The same algorithm as in Part (a) applies, with minor modications to compute the
longest path instead of the shortest:
1. Each min operation from Part (a) is replaced with max.
2. DAG -S HORTEST-PATHS is modied to compute longest paths instead of shortest
paths. This involves modifying I NITIALIZE -S INGLE -S OURCE to invert the roles
of 0 and and modifying R ELAX to use max instead of min.
Correctness follows the same argument as in Part (a). The running time is identical.
Note that while this problem is very similar to (a), it does introduce the possibility of
traversing an edge twice in the undirected graph. Some other approaches to (a) break
down given this possibility.
Alternate Solution:
Use the same algorithm as in Part (a), but invert all of the edge weights on the original
graph. That is, use edge weights w (e) = w(e) for all edges e E.
As the algorithm in Part (a) breaks the graph into DAGs, there cannot be any negativeweight cycles (there are no cycles at all). Thus, DAG -S HORTEST-PATHS still gives the
correct answer. Steps 3 and 4 remain unchanged in selecting the shortest composition
of paths from GD and GU .
The runtime is identical to Part (a).
Introduction to Algorithms
Massachusetts Institute of Technology
Professors Piotr Indyk and Charles E. Leiserson
December 1, 2004
6.046J/18.410J
Handout 31
paths. Any legal set of invitations should correspond to a feasible setting of the vari
ables for your LP, and any feasible integer setting of the variables in your LP should
correspond to a legal set of invitations. What objective function maximizes the number
of students invited?
pP,gG
xp,g
p P
x
gG p,g
g G
pP xp,g
g G, p P
xp,g
g G, p P
xp,g
size(p)
residents(g)
1
0
If we have a legal set of invitations, then it is easy to see that all of these constraints are
satised, since the set of invitations must satisfy the stated conditions. Similarly, any
feasible integral setting of the variables yields xp,g {0, 1}, and it is straightforward
to verify that sending an invitation to party P to some resident of group g if and only
if xp,g = 1 will satisfy the requirements. (We do not permit the administration to send
more than one invitation to a party p to the same student; thus we can send only one
invitation to party p to group g.) Because the objective function p,g xp,g measures
the number of invitations sent (and thus the number of students invited), an optimal
setting of the variables for the LP therefore corresponds to a maximum number of
invited students.
(b) Show how this problem can be solved using a maximum-ow algorithm. Your algo
rithm should return a set of legal invitations, if one exists, and return FAIL if none
exists.
Solution: Dene the graph G = (V, E), and capacities c on the edges, where
V = {s, t} G P , where s and t are brand new source and sink nodes, respec
tively.
(g, p), (s, g), (p, t) E for every g G, p P .
c(g, p) = 1, c(s, g) = residents(g), and c(p, t) = size(p).
of f is less than gG residents(g).) For every party p, send an invitation to party p
to an uninvited member of group g if and only if f (g, p) = 1.
The constraints on the graph guarantee that no two residents of the same house can get
an invitation to the same party and that no party has more invitations sent to it than its
size. Thus if the algorithm returns a set of invitations, then they are valid. Conversely,
any legal set of i invitations can be expressed as a ow of value i.
(c) (Optional.) Can this problem can be solved more efciently than with a maximum-
ow algorithm?
Solution: It seems like some version of a greedy algorithm can be used to solve
this problem more efciently, but after an hour-long course-staff meeting, we werent
convinced either way.
Problem 8-2. Zippity-doo-dah day
On Interstate 93 south of Boston, an ingenious device for controlling trafc has been installed.
A lane of trafc can be switched so that during morning rush hour, trafc ows northward to
Boston, and during evening rush hour, if ows southward away from Boston. The clever engi
neering behind this design is that the reversible lane is surrounded by movable barriers that can be
Let (u, v) E be a particular edge in G such that f (u, v) > 0 and c(v, u) 1. That is, there is
positive ow on this edge already, and there is positive capacity in the reverse direction. Suppose
that zipper technology increases the capacity of the edge (u, v) by 1 while decreasing the capacity
of its transpose edge (v, u) by 1. That is, the zipper moves 1 unit of capacity from (v, u) to (u, v).
(a) Give an O(V + E)-time algorithm to update the maximum ow in the modied graph.
Solution: The algorithm is simple: increment the capacity of (u, v), and search for a
single augmenting path in the residual graph in O(E + V ) time. Augment along that
path if one is found. Decrement the capacity of (v, u), and return the resulting ow.
If there is a minimum cut in G that (u, v) does not cross, then the same ow f is a
maximum ow in the modied graph: it is feasible because no capacities have de
creased, and it is maximum because the same unmodied minimum cut still gives an
upper bound on the size of the ow. If (u, v) does cross all minimum cuts, then the
maximum ow will increase by 1. Thus, the new maximum ow has value either |f |
or |f |+1, and a single augmenting path will sufce to nd the updated maximum ow
(since the residual capacities are all integral).
Finally, we observe that decrementing the capacity of (v, u) does not affect the max
imum ow: because there is positive ow on (u, v) in f , there is net negative ow
on (v, u), and the soon-to-be-deleted unit of capacity on (v, u) is unused. Eliminating
any capacity unused by the max ow cannot make the ow infeasible, and it therefore
must remain a maximum ow in the modied graph.
Zap 86 years into the future! Zipper lanes are commonplace on many more roads in the Boston
area, allowing one lane of trafc to be moved from one direction to the other. You are once again
given the directed graph G = (V, E) and integer capacities c : E + on the edges. You
also have a zipper function z : E {0, 1} that tells whether an additional unit of capacity can be
moved from (v, u) to (u, v). For each (u, v) E, if z(u, v) = 1, then you may now choose to move
1 unit of capacity from the transpose edge (v, u) to (u, v). (You may assume that if z(u, v) = 1,
then the edge (v, u) exists and has capacity c(v, u) 1. Again, you are given a source node s V ,
a sink node t V , and a maximum ow f . Governor Mitt IV asks you to congure all the zippered
lanes so that the maximum ow from s to t in the congured graph is maximized.
(b) Describe an algorithm that employs a maximum-ow computation to determine the
following:
1. the maximum amount that the ow can be increased in this graph after your cho
sen zippered lanes are opened; and
2. a conguration of zippered lanes that allows this ow to be achieved.
Solution: One can solve this problem by extending the denition of an augmenting
path to include the possibility of zipping an edge to increase the capacity of a min cut,
but theres an easier way.
Note that in a net ow, there is never positive ow in both directions (u, v) and (v, u).
The idea of the algorithm is this: since we can compute a maximum ow that only
uses an edge in one direction, well add the zippered capacity in both directions, and
compute max ow. The above note guarantees that the resulting ow satises the
zippered capacity constraintsi.e., does not use the extra zipped capacity along both
(u, v) and (v, u).
Given G = (V, E), nodes s, t, capacity function c, and zipper function z:
1. For every pair {u, v} E, set c (u, v) := z(u, v) + c(u, v).
2. Run the Edmonds/Karp max ow algorithm on G with capacities c to get a max
ow f .
3. For every pair (u, v) with z(u, v) = 1, if f (u, v) > 0, then set the direction of the
zippered lane on (u, v) so that the unit of capacity is moved from the transpose
edge (v, u) to the edge (u, v). If f (u, v) = f (v, u) = 0, set the direction of the
zipper lane for (u, v) arbitrarily.
4. Return f and the direction settings above.
The running time of this algorithm is then O(V E 2 ).
To prove correctness, well show that (i) the value of f is at least the max ow f for
the best zippered setting, and (ii) the value of f is at most f . For condition (i), observe
the following: the capacity of each edge in the best zippered setting is upper bounded
by the capacities c , which immediately implies that the value of f is at most f . For
condition (ii), note that f is a feasible ow in some zippered graph (in particular, in
the graph with edges oriented as per the returned directions).
Because the graph G is actually a network of roads, it is nearly planar, and thus |E| = O(V ).
(c) Give an algorithm that runs in time O(V 2 ) to solve the graph conguration problem
under the assumption that |E| = O(V ). You should assume that the original ow f
Introduction to Algorithms
Massachusetts Institute of Technology
Professors Piotr Indyk and Charles E. Leiserson
December 8, 2004
6.046J/18.410J
Handout 34
time, in the order T [1], T [2], . . .. You are interested in checking if the text seen so far
contains a pattern P , where P has length m. Every time you see the next letter of the
Design an algorithm that solves this problem efciently. Your algorithm should use
no more than (m) time on preprocessing P . In addition it should do only constant
amount of work per letter received. Your algorithm can be randomized, with constant
probability of correctness.
Solution:
For both parts we are going to use Karp-Rabin (KR) algorithm seen in the class (the
nite state machine / Knuth-Morris-Pratt algorithms could be used as well). As in the
class, we will assume that the alphabet is {0, 1} (the algorithm can be easily modied
to handle arbitrary alphabet). Recall that KR used log O(1) n time to nd a random
prime q, and O(m) time to hash the pattern P ; we will refer to these two steps as
preprocessing.
After the preprocessing, the KR algorithm computes the hash values for the m-length
substrings of T in an incremental way. Whenever a new symbol T [i] is given, KR
computes the hash value of the substring T [i m + 1 . . . i] in constant time. Thus, KR
works in our setting without any modications.
(b) Now say that you have the same pattern P , but the text T [1 . . . n] is being broadcast in
reverse. That is, in the order T [n], T [n 1], . . . Modify your algorithm so that it still
Solution:
For an array A[1 . . . n], let AR be the reverse of A, i.e., AR [i] = A[n i + 1].
Searching for an occurrence of P in T is equivalent to searching for an occurrence of
P R in T R ; thus, we can focus on the latter task. Since the symbols of T are given
in the reverse order, it means that the symbols of T R are given in the proper order
(i.e., T R [1], T R [2] . . .). Thus, we can nd the occurrences of P R in T R by using the
algorithm from the part (a).
Exercise 9-2. Some Summations
Assume you are given two sets A, B {0 . . . m}. Your goal is to compute the set C = {x + y :
x A, y B}. Note that the set of values in C could be in the range 0 . . . 2m.
Your solution should run in time O(m log m) (the sizes of |A| and |B| do not matter).
Example:
A = {1, 4}
B = {1, 3}
C = {2, 4, 5, 7}
Solution:
The key realization is that when two polynomials multiply, their exponents are added in every
possible pairing. So we take our set A, and do the following: Turn the set into a polynomial
of maximum degree m. The coefcient of x0 is the number of times 0 appears in the set. The
coefcient of x1 is the number of times 1 appears in the set, and on. Do the same with B. The
Example:
A = x 4 + x1
B = x 3 + x1
C = x 7 + x 5 + x 4 + x2
C = A B. Recall that the coefcient of xi in C is given by:
ci =
aj bij
j =0
Note that in this case, all terms of the sum are non-negative. We can compute C in O(m log m)
For a simple argument of correctness, we observe that every coefcient in the target polynomial
(ci xi ) represents the number of ways we can add a number from A and from B to get to i. First, we
note that all coefcients in A and B are either zero or one (no multisets were indicated). Clearly
if there is no a A, b B that sum to i, then for all j {0..i}, at least one of a j , bij is zero.
Similarly, if ci = z, then there are z pairs aj , bij since each such pair will multiply out to xi , and
then the sum of all of those pairs will yield z xi .
The running time of this algorithm is the time to convert A and B into polynomials. If A is a set (not
a multiset) {0..m} then this can be done in O(m). Same is true for B. To multiply them takes
O(m log m). Finally, to take the resulting polynomial and output the set C takes O(2m) = O(m).
As an aside, we also note that if we allowed multisets, everything still works. If there are two ones
in one set, and two threes in the other, there are indeed four different ways to get a target value of
3.
Exercise 9-3. Do Problem 35-5, on page 1051 of CLRS.
Solution:
(a) In the best case, each job has its own machine. Since a job cannot be broken down into smaller
pieces, the makespan cannot be shorter than the longest job. That is, C max
max pk .
1kn
(b) In the best case, the jobs are evenly distributed over the machines, such that each machine n
ishes at exactly the same time, Ceven . (Starting from such a distribution, any re-allocation
would clearly increase the makespan, as the maximum nish time would increase.) The
makespan of an even distribution
is the sum of the job lengths divided by the number of pro
(c) For each machine Mi , we maintain a set ai of the jobs that are allocated to Mi , as well as a
eld fi indicating the time at which Mi will become idle (under the current allocation of jobs).
The G REEDY-S CHEDULE algorithm assigns each job to the machine that will become idle rst.
The machines are organized in a priority queue (implemented as a M IN -H EAP) using the next
nish time fi as the key for machine Mi .
G REEDY-S CHEDULE(J1 . . . Jn , p1 . . . pn , M1 . . . Mm )
1 Q empty M IN -H EAP
2 for i 1 to m
3
fi 0
// initialize nish time
4
ai {} // initialize job allocation
5
I NSERT(Q, Mi , ai , fi ), using fi as key
6 for j 1 to n
7
do Mk , ak , fk E XTRACT-M IN(Q)
8
ak ak Jj
9
fk fk + pj
10
I NSERT(Q, Mk , ak , fk )
11 return {a1 , a2 , . . . , am }
This algorithm directly implements the greedy strategy. It iterates over the jobs, assigning
each job to the machine that will be idle next. Each call to I NSERT and E XTRACT-M IN takes
O(lg m) time, as there are m machines in the heap. These calls are enclosed in two loops; the
rst executes m times and the second executes n times. Assuming n m (as otherwise the
problem is trivial), the overall runtime is O(n lg m).
(d) Let Jj denote the job that has the largest completion time as scheduled by the greedy algorithm
min fjk
+ pj
k[1,m]
min fmk
+ pj
k[1,m]
1/m
pk + p j
1kn
pk + max pk
1kn
1kn
Cmax
+ Cmax
2 Cmax
1/m
The rst line is a direct consequence of the greedy algorithm: on iteration j, the algorithm
selects the minimum nish time and schedules job Jj to execute next on the corresponding
machine. The second line follows because the nish times are monotonically increasing with
successive iterations of the j loop. By the same argument as in part (b), the minimum nish
time is maximized when all nish times are the same; thus, the average load is used as an
upper bound in the third line. The fourth line recognizes that p j is at most max pk . Finally,
1kk
we substitute the results from (a) and (b) to conclude that C j 2Cmax
, thereby showing that
the greedy algorithm is a 2-approximation algorithm.