2020/2021(2)

IF184401 Design & Analysis of Algorithms

Greedy Algorithms
IRFAN SUBAKTI
Misbakhul Munir

司馬伊凡
Мисбакхул Мунир Ирфан Субакти
Greedy Algorithms: Why?
• Optimisation problems → sequence of steps of algorithms, a
set of choices at each steps
• Dynamic programming (DP) → overkill
• Simpler, more efficient algorithms will do
• Greedy algorithm → always makes the choice that looks best
at the moment
• Locally optimal choice → hopefully it will lead to a globally optimal
solution

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

2
Irfan Subakti
Greedy Algorithms: Discussion
• Do not always yield optimal solutions → but works well for a wide range of problems
• But, for many problems they do
• DP vs greedy algorithms
• E.g., the activity-selection problem → DP approach first, while greedy choices always
can be used to arrive at an optimal solution
• Basic elements of greedy approach → direct approach for proving greedy algorithms
correct
• Basic application: designing data-compression (Huffman) codes
• Investigating “matroids” → greedy algorithm always produces an optimal solution
• Solving a problem of scheduling unit-time tasks with deadlines and penalties
• Advance application: minimum-spanning tree (Ch.23), Dijkstra’s alg. for shortest paths
from a single sources (Ch.24), etc.

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

3
Irfan Subakti
Activity-Selection Problem
• Scheduling several competing activities, require exclusive use of a
common resource
• S = {a1, a2, …, an}, n proposed activities
• ai has a start time si and a finish time fi, 0 ≤ si < fi < ∞
• If selected, activity ai takes place during the half-open time
interval [si, fi)
• Activity ai and aj are compatible if the intervals [si, fi) and [sj, fj)
do not overlap → ai and aj are compatible if si ≥ fj or sj ≥ fi
• Goal: max subset of mutually compatible activities
• f1 ≤ f2 ≤ f3 ≤ … ≤ fn-1 ≤ fn (monotonically increasing order)
2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM
4
Irfan Subakti
Activity-Selection Problem (continued)

• {a3, a9, a11} consists of mutually compatible activities → not max set
• {a1, a4, a8, a11} and {a2, a4, a9, a11} → largest subset, max set
• Solving the problem: relationship between greedy alg. & DP
• DP → several choices which subproblems to use in an optimal solution
• Only one choice: the greedy choice → when we make greedy choice, only one
subproblem remains
• Recursive greedy algorithm for the activity-selection problem
• Complete the process of developing a greedy solution by converting the
recursive algorithm to an iterative one
2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM
5
Irfan Subakti
Activity-Selection Problem: Optimal
Substructure
• The activity-selection problem exhibits optimal substructure
• Sij the set of activities that start after activity ai finishes and that finish before activity aj starts
• Goal: max set of mutually compatible activities in Sij, suppose that such a max set is Aij, which
includes some activities ak
• By including ak in an optimal solution → left with 2 subproblems
• Finding mutually compatible activities in Sik
• Activities that start after activity ai finishes & that finish before activity ak starts
• Finding mutually compatible activities in Skj
• Activities that start after activity ak finishes & that finish before activity aj starts
• Aik = Aij ∩ Sik and Akj = Aij ∩ Skj, so that Aik contains the activities in Aij that finish before ak starts
and Akj contains the activities in Aij that start after ak finishes → Aij = Aik ∪ {ak} ∪ Akj, and so the
max-size set Aij of mutually compatible activities in Sij consists of |Aij| = |Aik| + |Akj| + 1 activities
• Cut-and-paste argument shows that the optimal solution Aij must also include optimal solutions
to the 2 subproblems for Sik and Skj
2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM
6
Irfan Subakti
Activity-Selection Problem: Optimal
Substructure (continued)
• Contradiction
• If we could find a set A’kj of mutually compatible activities in Skj where |A’kj| > |Akj| → we
could use A’kj, rather than Akj, in a solution to the subproblem for Sij
• Constructed |Aik| + |A’kj| + 1 > |Aik| + |Akj| + 1 = |Aij| mutually compatible activities →
which contradicts the assumption that Aij is an optimal solution. A symmetric argument
applies to the activities in Sik
• DP: if the size of an optimal solution for the set Sij by c[i, j] → recurrence: c[i, j] =
c[i, k] + c[k, j] + 1
0 𝑖𝑓 𝑆𝑖𝑗 = ∅
• c[i, j] = ቐ max {𝑐 𝑖, 𝑘 + 𝑐 𝑘, 𝑗 + 1} 𝑖𝑓𝑆 ≠ ∅
𝑖𝑗
𝑎𝑘 ∈𝑆𝑖𝑗

• Can continue by developing a recursive algorithm and memoize it, or by using

bottom-up and fill in table entries as we go along
2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM
7
Irfan Subakti
Activity-Selection Problem: Making the Greedy
Choice
• If we could choose an activity to add to our optimal solution without having to first solve all the
subproblems → save us from having to consider all the choices inherent in recurrence
• For the activity-selection problem has only ONE CHOICE: the greedy choice
• Intuition
• Should choose an activity that leaves the resources available for as many other activities as possible
• Of the activities we end up choosing → one of them must be the first one to finish
• Choose the activity in S with the earliest finish time → would leave the resource available for as many of the
activities that follow it as possible
• Since the activities are sorted in monotonically increasing order by finish time → the greedy choice is activity
a1
• Other possibilities aside choosing the first activity to finish of making a greedy choice?
• Made the greedy choice → only one remaining subproblem to solve: finding the activities that
start after a1 finishes
• No need to consider activities that finish before a1 start
• s1 < f1. f1 is the earliest finish time of any activity → no activity can have a finish time ≤ s1 → all activities that
are compatible with activity a1 must start after a1 finishes

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

8
Irfan Subakti
Activity-Selection Problem: Making the Greedy
Choice (continued)
• The activity-selection problem exhibits optimal substructure
• Sk = {ai ∈ S : si ≥ fk} be the set of activities that start after activity ak finishes
• Made the greedy choice of activity a1 → S1 remains as the only subproblem to
solve
• Optimal structure told that if a1 is in the optimal solution → an optimal solution to the original
problem consists of activity a1 and all the activities in an optimal solution to the subproblem S1
• Intuition to theorems → read the book!

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

9
Irfan Subakti
Activity-Selection Problem: A Recursive Greedy
Algorithm (1)

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

10
Irfan Subakti
Activity-Selection
Problem: A Recursive
Greedy Algorithm (2)

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

11
Irfan Subakti
Activity-Selection Problem: An Iterative Greedy
Algorithm

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

12
Irfan Subakti
Greedy Strategy: The Elements
• Greedy alg. obtains an optimal solution by making a
sequence of choices
• At each decision point → makes the choice that seems best at the
moment
• This heuristic strategy does NOT always produces an optimal
solution
• As in the activity-selection problem, sometimes it DOES

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

13
Irfan Subakti
Greedy Strategy: The Elements (continued)
• Greedy strategy:
1. Determine the optimal substructure of the problem
2. Develop a recursive solution
3. Show that if we make the greedy choice, then only one
subproblem remains
4. Prove that it is always safe to make the greedy choice. (Steps 3
and 4 can occur in either order.)
5. Develop a recursive algorithm that implements the greedy
strategy
6. Convert the recursive algorithm to an iterative algorithm

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

14
Irfan Subakti
Greedy Algorithm: The Steps
1. Cast the optimisation problem as one in which we make a choice
and are left with one subproblem to solve
2. Prove that there is always an optimal solution to the original
problem that makes the greedy choice → the greedy choice is
always safe
3. Demonstrate optimal substructure by showing that, having made
the greedy choice, what remains is a subproblem with the property
that if we combine an optimal solution to the subproblem with the
greedy choice we have made, we arrive at an optimal solution to
the original problem
2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM
15
Irfan Subakti
Greedy-Choice Property
• Assembling a globally optimal solution by making locally optimal (greedy) choices
• Which choice to make? The choice that looks best in the current problem, without
considering results from subproblems
• DP and Greedy alg.
• DP: make a choice at each step, but the choice usually depends on the solutions to
subproblems → bottom-up manner (or top down with memoizing → the code works
top down, solve the subproblem before making a choice)
• Greedy alg.: make whatever choice seems best at the moment → solve the
subproblem that remains
• The choice may depend on choices so far, but it cannot depend on any future choices or on the solution
to subproblems
• DP: solves the subproblems before making the first choice
• Greedy alg.: makes its first choice before solving any subproblems.
• Greedy alg.: usually progresses in a top-down fashion → making one greedy choice
after another, reducing each given problem instance to a smaller one

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

16
Irfan Subakti
Optimal Substructure
• A problem exhibits optimal substructure
• If an optimal solution to the problem contains within it optimal
solutions to subproblems
• Key in accessing the applicability of DP and greedy alg.
• Greedy alg.: assuming that we arrived at a subproblem by having
made the greedy choice in the original problem
• Argue that an optimal solution to the subproblem, combined with the
greedy choice already made, yields an optimal solution to the original
problem
• Implicitly uses induction on the subproblems to prove that making the
greedy choice at every steps produces an optimal solution
2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM
17
Irfan Subakti
Greedy vs DP
• Both strategies exploit optimal substructure → generate a
DP solution to a problem when a greedy solution suffices or,
conversely, think that a greedy solution works when in fact a
DP is required
• Example
• The 0-1 knapsack problem → greedy cannot, only DP
• The fractional knapsack problem → greedy can

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

18
Irfan Subakti
Huffman Codes
• Compress data very effectively → 20% to 90%

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

19
Irfan Subakti
Huffman Codes (continued)

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

20
Irfan Subakti
Huffman Codes (continued)

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

21
Irfan Subakti
Huffman Codes
(continued)

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

22
Irfan Subakti
Matroids and Greedy Methods
• Combinatorial structures known as “matroids”
• This theory does NOT cover all cases for which a greedy method
applies
• The activity-selection problem & Huffman-coding problem cannot be covered
• It does cover many cases of practical interest

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

23
Irfan Subakti
Matroids and Greedy Methods (continued)

• Greedy alg. on a weighted matroid

• Minimum-spanning-tree problem
• A task-scheduling problem as a matroid

2020/2021(2) - IF184401 Design & Analysis of Algorithms - MM

24
Irfan Subakti