GREEDY ALGORITHMS

Optimization problems
• An optimization problem is one in which you
want to find, not just a solution, but the best
solution
• A “greedy algorithm” sometimes works well for
optimization problems
• A greedy algorithm works in phases. At each
phase:
– You take the best you can get right now, without
regard for future consequences
– You hope that by choosing a local optimum at each
step, you will end up at a global optimum 2
 Example: Counting money
• Suppose you want to count out a certain amount
of money, using the fewest possible bills and coins
• A greedy algorithm would do this would be:
At each step, take the largest possible bill or coin
that does not overshoot
– Example: To make $6.39, you can choose:
• a $5 bill
• a $1 bill, to make $6
• a 25¢ coin, to make $6.25
• A 10¢ coin, to make $6.35
• four 1¢ coins, to make $6.39
• For US money, the greedy algorithm always gives
the optimum solution 3
 A failure of the greedy algorithm
• In some (fictional) monetary system, “krons”
come in 1 kron, 7 kron, and 10 kron coins
• Using a greedy algorithm to count out 15 krons,
you would get
– A 10 kron piece
– Five 1 kron pieces, for a total of 15 krons
– This requires six coins
• A better solution would be to use two 7 kron
pieces and one 1 kron piece
– This only requires three coins
• The greedy algorithm results in a solution, but
not in an optimal solution 4
 SUBSET PARADIGM

• Feasible checks if x can be included in solution vector.

• Select, feasible, and union needs to be properly implemented.
 Some common Problems
• Activity Selection/Interval scheduling
• Huffman coding
• Knapsack Algorithm
• Tree Vertex Splitting Problem
• Container Loading Problem
• MST
– Boruvka
– Kruskal
– Prim
• Shortest Path in a graph
Interval Scheduling Problem
 Problem
• We have a set of requests [1,2,…,n]; the ith
request corresponds to an interval of time
starting at s(i) and finishing at f(i).

• Objective: maximize the number of non-

overlapping intervals.
 Natural rule 1

• Rule 1: Select the available request that starts

earliest.
Failure!
 Natural rule 2

• Rule 2: Choose the smallest interval.

Failure!
 Natural rule 3

• Rule 3: Select the interval with the fewest

conflicts with other remaining intervals (but
still not overlapping the already chosen
intervals)
Failure!
 Natural rule 4

• Rule 4: Select the interval which ends first (but

still not overlapping the already chosen
intervals)
• Verify it works in all cases
 Why ?
• Algorithm gives non-overlapping intervals:
obvious, since we always choose an interval
which does not overlap the previously chosen
intervals
The solution is exact: Let A be the set of
intervals obtained by the algorithm, and O be
the optimal largest set of pairwise non-
overlapping intervals. We show that A must be
as large as Opt.
 Prove it!
• Let i1, i2 , …, ik be the set of request in A in the
order they were added to A. Note that |A| = k.
• Let the request in O be j1, j2 ,…, jm .
• Prove that k = m.
 Lemma 1
• Prove that for all indices r ≤ k we have f(ir) ≤f(jr).
• Proof: Proof is by induction.
• Base case: When r = 1, the greedy algorithm chooses the interval with earliest
finish time and hence f(i1) ≤f(j1).
• Induction hypothesis: Let the statement is true for r-1 for r > 1.
• Induction step. According to induction hypothesis f(ir-1) ≤f(jr-1).
ir-1 ir
Can it
jr-1 jr happen?

• But s(jr)≥ f(jr-1 ) ≥ f(ir-1). Thus the interval jr is in the set R of available intervals at
the time when the greedy algorithm selects ir.
• The greedy algorithm selects the available interval with smallest finish time, since
interval jr is one of these available intervals, we have f(ir) ≤ f(jr). Thus completes
the induction step.
 Theorem
• The greedy algorithm returns an optimal set A.
• Proof: Let say A is not optimal then optimal set will
have more intervals so m > k. From last lemma we
have f(ik) ≤f(jk). Since m > k, there is a request jk+1 in O.
This requests start after jk ends, and hence after ik
ends. So after deleting all requests that are
overlapping with requests i1, i2 , …, ik , the set of
possible requests still contains jk+1 . But then greedy
algorithm stopped because R was empty – a
contradiction.