Greedy Algorithm - I

Activity Selection Problem

Click to add text

Instructor: Ashok Singh Sairam

Lecture Plan
• Greedy algorithm definition
• Activity selection problem
 DP solution
 Greedy solution
Click
Clicktotoadd
addtext
text

MA512: Data Structures and Algorithms

2
Optimization problems
• Algorithms for optimization problems
 Sequence of steps, with a set of choices at each step
• Types of algorithms we can consider
 Brute force algorithms
 Simple recursive algorithms
 Divide and conquer algorithms
 Dynamic programming algorithms
 Greedy algorithms
 Branch and bound algorithms
 Randomized algorithms

3
Greedy Algorithms
• A greedy algorithm works in steps. At each step:
 You take the best choice, without regard for future
consequences
 You hope that by choosing a local optimum at each step,
you will end up at a global optimum

MA512: Data Structures and Algorithms

4
Activity-selection problem
• Consider a set of activities S={a1, a2,…, an}
 They use resources, such as lecture hall, one lecture at a
time
 Each ai, has a start time si, and finish time fi,
with 0 si< fi<.
 ai and aj are compatible if [si, fi) and [sj, fj) do not overlap
• Goal: select maximum-size subset of mutually
compatible activities.
• Start from dynamic programming, then greedy
algorithm, see the relation between the two

MA512: Data Structures and Algorithms

5
Activity-selection problem
• The activity-selection problem is to select a maximum-size
subset of mutually compatible activities.
Example:
i 1 2 3 4 5 6 7 8 9 10 11
si 1 3 0 5 3 5 6 8 8 2 12
fi 4 5 6 7 9 9 10 11 12 14 16

Some mutually compatible activities: 1 4 9

{a3 , a9 , a11} --- not the largest 2 7 11
{a1 , a4 , a8 , a11} 3 8
{ a2, a4 , a9 , a11} 5 10
6
time
MA512: Data Structures and Algorithms
6
DP solution: Notations
• Optimal substructure of activity-selection problem
Sij={akS : fi  sk < fk  sj},
is the subset of activities in S that can start after activity ai
finishes and finish before activity aj starts.
• Problem:
 Find maximum set of mutually compatible activities in Sij

Aij: Maximum set of mutually compatible activities in Sij

MA512: Data Structures and Algorithms

7
Example
Sij

S
si fi sj
Aij

i 1 2 3 4 5 6 7 8 9 10 11
si 1 3 0 5 3 5 6 8 8 2 12
fi 4 5 6 7 9 9 10 11 12 14 16

• S1,11= ??
• A1,11= ??

MA512: Data Structures and Algorithms

8
Illustration:
DP solution – Optimal Substructure
Sij

si fi sj
Sik ak Skj

𝐴𝑖𝑗 = 𝐴𝑖𝑘 ∪ 𝑎𝑘 ∪ 𝐴𝑘𝑗

MA512: Data Structures and Algorithms

9
DP solution –step 1
• Optimal substructure
• Suppose an optimal solution 𝐴𝑖𝑗 includes ak
• We are left with two subproblems - Sik and Skj
• Let 𝐴𝑖𝑘 = 𝐴𝑖𝑗 ∩ 𝑆𝑖𝑘
 Contain the activities in 𝐴𝑖𝑗 that finish before ak
• Let 𝐴𝑘𝑗 = 𝐴𝑖𝑗 ∩ 𝑆𝑘𝑗
 Contain the activities in 𝐴𝑖𝑗 that start after ak
• 𝐴𝑖𝑗 = 𝐴𝑖𝑘 ∪ {𝑎𝑘 } ∪ 𝐴𝑘𝑗
|𝐴𝑖𝑗 = |𝐴𝑖𝑘 + 𝐴𝑘𝑗 + 1

MA512: Data Structures and Algorithms

10
DP Solution – step 2
• Recursive cost
c[i,j]: size of optimal solution of Sij
c[i, j ]  c[i, k ]  c[k , j ]  1

 0 if Sij  0
c[i, j ]  
max {c[i, k ]  c[k , j ]  1} if Sij  0
ik  j

• Need to solve all sub-problems

MA512: Data Structures and Algorithms

11
Greedy Choice
• Choose an activity to add to the optimal solution
without having to solve all the subproblems
• What is a greedy choice?
 One possibility: Choose an activity such that it will leave
maximum resources available for others
• Choose an activity a1 with the earliest finish time
• Making a greedy choice leaves us with only one
subproblem to choose
 Find activities that start after a1

MA512: Data Structures and Algorithms

12
Greedy choice (2)
• Set of activities that start after ak finishes
𝑺𝒌 = {𝑎𝑖 ∈ 𝑆: 𝑠𝑖 ≥ 𝑓𝑘 }
 Ex: If a1 is greedy choice, only subproblem is 𝑆1
𝑨𝒌 is optimal solution of Sk
• We have shown that Activity-Selection problem
exhibit optimal substructure
 That is , if 𝑎1 is in the optimal solution, then
Aij = 𝑎1 ∪ 𝐴1
• We need to prove 𝑎1 in the optimal solution
MA512: Data Structures and Algorithms
13
Illustration: Greedy choice
Sij

si fi sj
ak Sk
No subproblem in Sik
𝐴𝑖𝑗 = 𝑎𝑘 ∪ 𝐴𝑘
ak: Activity with earliest finish time
Sk: Activities that start after ak finishes

MA512: Data Structures and Algorithms

14
Illustration of Theorem

i 1 2 3 4 5 6 7 8 9 10 11
si 1 3 0 5 3 5 6 8 8 2 12
fi 4 5 6 7 9 9 10 11 12 14 16

• Consider S0 [all activities that start after a0, f0=0]

The activity with earliest finish time is a1
 a1 must be included in some optimal solution

• Optimal solutions: {a1 , a4 , a8 , a11}, { a2, a4 , a9 , a11}

MA512: Data Structures and Algorithms
15
Theorem

• Proof: Let 𝑎𝑗 ∈ 𝐴𝑘 has the earliest finish time

If 𝑎𝑗 = 𝑎𝑚 , we are done
If 𝑎𝑗 ≠ 𝑎𝑚 , let 𝐴′𝑘 = 𝐴𝑘 − 𝑎𝑗 ∪ 𝑎𝑚
The activities in 𝐴′𝑘 are disjoint
 Activities in 𝐴𝑘 are disjoint and 𝑓𝑚 ≤ 𝑓𝑗
That is 𝐴′𝑘 is an optimal solution, since |𝐴′𝑘 | = |𝐴𝑘 |

MA512: Data Structures and Algorithms

16
Recursive Greedy Algorithm
• Assume that n input activities are already ordered
by monotonically increasing finish times
f1  f2  …  fn

• Add fictitious activities a0 with f0=0

 Subproblem S0 is the entire set of activities

MA512: Data Structures and Algorithms

17
RECURSIVE-ACTIVITY-SELECTOR:
Illustration on the 11 activities given earlier

MA512: Data Structures and Algorithms

18
Recursive Greedy Algorithm
• Input: s[]: start time of ai; f[]: finish time of ai
• Initial call: Recursive-Activity-Selector(s,f,0,n)

• While loop returns first activity am compatible

with ak Running time: Θ 𝑛

MA512: Data Structures and Algorithms

19
MA512: Data Structures and Algorithms
20
MA512: Data Structures and Algorithms
21
Iterative Greedy Algorithm

MA512: Data Structures and Algorithms

22
Exercise

MA512: Data Structures and Algorithms

23