Greedy Algorithms

Greed Some possibly familiar examples:

"Greed is good. Greed is right. Greed
works. Greed cuts through, clarifies,
and captures the essence of the
evolutionary spirit."
■ Gale-Shapley stable matching algorithm.
■ Dijkstra’s shortest path algorithm.
■ Prim and Kruskal MST algorithms.
■ Huffman codes.
■ . . .

Gordon Gecko
(Michael Douglas)

Selecting Breakpoints Selecting Breakpoints: Greedy Algorithm

Minimizing breakpoints.
■ Truck driver going from Princeton to Palo Alto along
predetermined route. Greedy Breakpoint Selection Algorithm

■ Refueling stations at certain points along the way. Sort breakpoints by increasing value:
0 = b0 < b1 < b2 < ... < bn.
■ Truck fuel capacity = C.
S ← {0} S = breakpoints selected.
Greedy algorithm. x = 0
while (x ≠ bn)
Go as far as you can before refueling.
let p be largest integer such that bp ≤ x + C
■

if (bp = x)
C C C C return "no solution"
x ← bp
1 2 3 4 5 6 7 S ← S ∪ {p}
return S
Princeton Palo Alto
C C C

3 4
 Selecting Breakpoints Selecting Breakpoints
Theorem: greedy algorithm is optimal. Theorem: greedy algorithm is optimal.

Proof (by contradiction): Proof (by contradiction):

■ Let 0 = g0 < g1 < . . . < gp = L denote set of breakpoints chosen by
greedy and assume it is not optimal.
■ Let 0 = f0 < f1 < . . . < fq = L denote set of breakpoints in optimal
solution with f0 = g0, f1= g1 , . . . , fr = gr for largest possible value of r.
■ Note: q < p.
■ Let 0 = g0 < g1 < . . . < gp = L denote set of breakpoints chosen by
greedy and assume it is not optimal.
■ Let 0 = f0 < f1 < . . . < fq = L denote set of breakpoints in optimal
solution with f0 = g0, f1= g1 , . . . , fr = gr for largest possible value of r.
■ Note: q < p.

g0 g1 g2 gr gp g0 g1 g2 gp
Greedy: 1 2 3 4 5 6 7 8 9 Greedy: 1 2 3 4 5 6 7 8 9

OPT: 1 2 3 4 5 6 7 OPT: 1 2 3 4 55 6 7
f0 f1 f2 fr fq f0 f1 f2 fq
r=4 r=4 r=5

5 6

Selecting Breakpoints Activity Selection

Theorem: greedy algorithm is optimal. Activity selection problem (CLR 17.1).
■ Job requests 1, 2, … , n.
Proof (by contradiction): ■ Job j starts at sj and finishes at fj.
■ Let 0 = g0 < g1 < . . . < gp = L denote set of breakpoints chosen by ■ Two jobs compatible if they don't overlap.
greedy and assume it is not optimal. ■ Goal: find maximum subset of mutually compatible jobs.
■ Let 0 = f0 < f1 < . . . < fq = L denote set of breakpoints in optimal
solution with f0 = g0, f1= g1 , . . . , fr = gr for largest possible value of r. A
■ Note: q < p. B
■ Thus, f0 = g0, f1= g1 , . . . , fq = gq
C

gq D
g0 g1 g2 gp
Greedy: 1 2 3 4 5 6 7 E

F
OPT: 1 2 3 4 5
G
f0 f1 f2 fq
H Time
r=q=5
7
0 1 2 3 4 5 6 7 8 9 10 11 8
 Activity Selection: Greedy Algorithm Activity Selection
Theorem: greedy algorithm is optimal.
Greedy Activity Selection Algorithm
Sort jobs by increasing finish times so that
Proof (by contradiction):
f1 ≤ f2 ≤ ... ≤ fn.
■ Let g1, g2, . . . gp denote set of jobs selected by greedy and assume
S = φ S = jobs selected. it is not optimal.
for j = 1 to n ■ Let f1, f2, . . . fq denote set of jobs selected by optimal solution with
if (job j compatible with A) f1 = g1, f2= g2, . . . , fr = gr for largest possible value of r.
S ← S ∪ {j}
return S ■ Note: r < q.

p=6

Greedy: 1 5 8 9 13 21
f1 = g 1 f2 = g 2 f3 = g 3
OPT: 1 5 8 11 15 17 21

r=3 q=7

9 10

Activity Selection Activity Selection

Theorem: greedy algorithm is optimal. Theorem: greedy algorithm is optimal.

Proof (by contradiction): Proof (by contradiction):

■ Let g1, g2, . . . gp denote set of jobs selected by greedy and assume
it is not optimal.
■ Let f1, f2, . . . fq denote set of jobs selected by optimal solution with
f1 = g1, f2= g2, . . . , fr = gr for largest possible value of r.
■ Note: r < q.
■ Let g1, g2, . . . gp denote set of jobs selected by greedy and assume
it is not optimal.
■ Let f1, f2, . . . fq denote set of jobs selected by optimal solution with
f1 = g1, f2= g2, . . . , fr = gr for largest possible value of r.
■ Note: r < q.

p=6 p=6

Greedy: 1 5 8 9 13 21 Greedy: 1 5 8 9 13 21
f1 = g 1 f2 = g 2 f3 = g 3 f1 = g 1 f2 = g 2 f3 = g 3
OPT: 1 5 8 9 11 15 17 21 OPT: 1 5 8 9 15 17 21

r=3 q=7 r=3 q=7

Replace 11 with 9 Replace 11 with 9

11 12
 Activity Selection Making Change
Theorem: greedy algorithm is optimal. Given currency denominations: 1, 5, 10, 25, 100, devise a method to
pay amount to customer using fewest number of coins.
Proof (by contradiction): ■ Ex. 34¢.
■ Let g1, g2, . . . gp denote set of jobs selected by greedy and assume
it is not optimal.
■ Let f1, f2, . . . fq denote set of jobs selected by optimal solution with
f1 = g1, f2= g2, . . . , fr = gr for largest possible value of r.
■ Note: r < q. Greedy algorithm.
■ At each iteration, add coin of the largest value that does not take
p=6
us past the amount to be paid.
Greedy: 1 5 8 9 13 21 ■ Ex. $2.89.
f1 = g 1 f2 = g 2 f3 = g 3
OPT: 1 5 8 9 15 17 21

r=3 r=4 q=7

13 14

Coin-Changing: Greedy Algorithm Is Greedy Optimal for Coin-Changing Problem?

Yes, for U.S. coinage: {c1, c2, c3, c4, c5 } = {1, 5, 10, 25, 100}.

Greedy Coin-Changing Algorithm Ad hoc proof.

Sort coins denominations by increasing value:
■ Consider optimal way to change amount ck ≤ x < ck+1 .
c1 < c2 < ... < cn.
■ Greedy takes coin k.
S ← φ S = coins selected. ■ Suppose optimal solution does not take coin k.
while (x ≠ 0) – it must take enough coins of type c1, c2, . . . , ck-1 to add up to x.
let p be largest integer such that cp ≤ x
if (p = 0)
return "no solution found" Max # taken by Max value of coins
k ck
x ← x - cp optimal solution 1, 2, . . . , k in any OPT
S ← S ∪ {p}
return S 1 1 4 4
2 5 1 4+5=9
3 10 2 20 + 4 = 24
2 dimes ⇒
4 25 3 75 + 24 = 99 no nickels
5 100 no limit no limit

15 16
 Does Greedy Always Work? Characteristics of Greedy Algorithms
US postal denominations: 1, 10, 21, 34, 70, 100, 350, 1225, 1500. Greedy choice property.
■ Ex. 140¢. ■ Globally optimal solution can be arrived at by making locally
■ Greedy: 100, 34, 1, 1, 1, 1, 1, 1. optimal (greedy) choice.
■ Optimal: 70, 70. ■ At each step, choose most "promising" candidate, without
worrying whether it will prove to be a sound decision in long run.

Optimal substructure property.

■ Optimal solution to the problem contains optimal solutions to sub-
problems.
– if best way to change 34¢ is {25, 5, 1, 1, 1, 1} then best way to
change 29¢ is {25, 1, 1, 1, 1}.

Objective function does not explicitly appear in greedy algorithm!

Hard, if not impossible, to precisely define "greedy algorithm."

■ See matroids (CLR 17.4), greedoids for very general frameworks.

17 18

Minimizing Lateness Minimizing Lateness: Greedy Algorithm

Minimizing lateness problem. Greedy Activity Selection Algorithm
■ Single resource can process one job at a time. Sort jobs by increasing deadline so that
■ n jobs to be processed. d1 ≤ d2 ≤ … ≤ dn.
– job j requires pj units of processing time.
t = 0
– job j has due date dj.
for j = 1 to n
■ If we assign job j to start at time sj, it finishes at time fj = sj + pj. Assign job j to interval [t, t + pj]
■ Lateness: lj = max { 0, fj - dj }. sj ← t, fj ← t + pj
t ← t + pj
■ Goal: schedule all jobs to minimize maximum lateness L = max lj.
output intervals [sj, fj]

d=9 d = 15 d = 11
Lateness = 3 max lateness = 2
d=8 d=6 d=9

d=9 d=8 d=2 d=6 d = 11 d=9 d1 = 6 d2 = 8 d3 = 9 d4 = 9 d5 = 11 d6 = 15

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

19 20
 Minimizing Lateness: No Idle Time Minimizing Lateness: Inversions
Fact 1: there exists an optimal schedule with no idle time. An inversion in schedule S is a pair of jobs i and j such that:
■ i<j
d=4 d=6 d = 12 ■ j scheduled before i inversion
0 1 2 3 4 5 6 7 8 9 10 11
d5 = 8 d2 = 4
d=4 d=6 d = 12
0 1 2 3 4 5 6 7 8 9 10 11
Fact 3: greedy schedule ⇔ no inversions.

Fact 2: the greedy schedule has no idle time. Fact 4: if a schedule (with no idle time) has an inversion, it has one
whose with a pair of inverted jobs scheduled consecutively.

21 22

Minimizing Lateness: Inversions Minimizing Lateness: Proof of Fact 5

An inversion in schedule S is a pair of jobs i and j such that: An inversion in schedule S is a pair of jobs i and j such that:
■ i<j ■ i<j
■ j scheduled before i inversion ■ j scheduled before i
fi
d5 = 8 d2 = 4 j i

d2 = 4 d5 = 8 i j
f’j
Fact 3: greedy schedule ⇔ no inversions.
Swapping two adjacent, inverted jobs does not increase max lateness.
Fact 4: if a schedule (with no idle time) has an inversion, it has one ■ l’k = lk for all k ≠ i, j
whose with a pair of inverted jobs scheduled consecutively. ■ l’i ≤ li
If job j is late:
Fact 5: swapping two adjacent, inverted jobs:
■
l′j = f j′ − d j (definitio n)
■ Reduces the number of inversions by one. = fi − d j ( j finishes at time f i )
■ Does not increase the maximum lateness. ≤ fi − di (i < j)
≤ li (definitio n)
Theorem: greedy schedule is optimal.
23 24