The Greedy method

Dr. Dwiti Krishna Bebarta

Outline

The Greedy method

– The general method
– Minimum cost spanning trees
– Single-source shortest paths
– Knapsack problem
– Optimal merge patterns
Greedy algorithms build a solution part by part, choosing the n
ext part in such a way, that it gives an immediate benefit. This
approach never reconsiders the choices taken previously. This
approach is mainly used to solve optimization problems.
Components of Greedy Algorithm
• A candidate set − A solution is created from this set.
• A selection function − Used to choose the best candidate
to be added to the solution.
• A feasibility function − Used to determine whether a
candidate can be used to contribute to the solution.
• An objective function − Used to assign a value to a
solution or a partial solution.
• A solution function − Used to indicate whether a complete
solution has been reached.
General Method
• Greedy method control abstraction for subset
paradigm
Minimum-cost Spanning Trees
• Definition 4.1 Let G=(V, E) be at undirected connected graph. A
sub-graph t=(V, E’) of G is a spanning tree of G iff t is a tree.

• Example 4.5

Spanning trees

• Applications
– Obtaining an independent set of circuit equations for an electric
network
– network designs (i.e. telephone or cable networks)
– Computer Network Routing Protocol
Minimum-cost Spanning Trees
• Example of MCST
– Finding a spanning tree of G with minimum cost

1 1
28
10 2 10 2
14 16 14 16

6 7 3 6 7 3
24
25 18 12 25 12
5 5
22 4 22 4

(a) (b)
Prim’s Algorithm

1 1 1
10 10 2 10 2
2

6 7 3 6 7 3 6 7 3

25 25
5 5 5
4 4 22 4
(a) (b) (c)
1 1 1
10 10 2 10 2
2 16
16 14
3 6 7 3 6 7 3
6 7
25 25 12 25 12
12
5 5 5
22 4 22 4 22 4
(d) (e) (f)
 Prim’s Algorithm
Prim’s algorithm
GENERIC-MST(G, w)
1. A=Φ
2. while A does not form a spanning tree
3. find an edge (u, v) that is safe for A
4. A ∪ {(u, v)}
5. return A

• Initialization: After line 1, the set A trivially satisfies the loop in

variant.
• Maintenance: The loop in lines 2–4 maintains the invariant by
adding only safe edges.
• Termination: All edges added to A are in a minimum spanning
tree, and so the set A returned in line 5 must be a minimum
spanning tree.
MST-PRIM (G, w, r)
1 for each u ∈ G[V]
2 u.cost = ∞
3 u.parent = NIL
4 r.cost = 0
5 Q = G[V]
6 mincost = 0
7 while Q ≠ Φ
8 u = EXTRACT-MIN(Q)
9 mincost = mincost+ u.cost
9 for each v ∈ G.Adj(u)
10 if v ∈ Q and w(u, v) < v.cost
11 v.parent = u
12 v.cost = w(u, v)
Prim’s Algorithm
• Time complexity
– Line 1-11: Constant time
– Line 12-14: (n)
– Line 15: Constant time
– Outer for loop of line 16: (n)
– Inner for loop of line 23: (n)
• n iterations for outer loop: O(n)
• Each iteration for loop of inside outer loop: O(n)
– So, Prim’s algorithm: O(n2)
• More efficient implementation using red-black tree
– Using red-black tree
• Lines 16 take O(log n)
• Line 27: O(|E|)
– So total time: O((n+|E|) log n)
 a is extracted
for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
a b h i c g d f e v.parent = u
0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ ∞ v.cost = w(u, v)

∞ ∞ ∞
Nodes Parent (v) Cost (v)
a Nil 0
0 ∞
b a 4
∞
h a 8
i
∞ ∞ ∞ c
g
ab<cost of b ah<cost of h d
4<∞ T 8<∞T f
e
‘ab’ and ‘ah’ edges are considered sin
ce they are adjacent to the node ‘a’
 a b h i c g d f e
b is extracted
0 4 8 ∞ ∞ ∞ ∞ ∞ ∞ for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
v.parent = u
4 ∞ ∞ v.cost = w(u, v)

0 ∞
∞
Nodes Parent (v) Cost (v)
a Nil 0
8 ∞ ∞
b a 4
a∈ Q F h a 8
bh<cost of h
Not considered s
11 < 8 F i
ince ‘a’ is not ele
ment of Q c b 8
bc < cost of c
g
8<∞ T
d
a b h i c g d f e f
0 4 8 ∞ 8 ∞ ∞ ∞ ∞ e
 a b h i c g d f e Now c is extracted
0 4 8 ∞ 8 ∞ ∞ ∞ ∞
for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
v.parent = u
4 8 ∞ v.cost = w(u, v)

b c
0 ∞
∞
Node (v) Parent (v) Cost (v)

8 ∞ ∞ a Nil 0
b a 4
b∈ Q F cd<cost of d h a 8
Not cosidered 7<∞T i c 2
c b 8
cf < cost of f ci < cost of i
g
4<∞ T 2<∞ T
d c 7
a b h i c g d f e
f c 4
0 4 8 2 8 ∞ 7 4 ∞ e
 a b h i c g d f e
0 4 8 2 8 ∞ 7 4 ∞
i is extracted
4 8
7 for each v ∈ G.Adj(u)
b if v ∈ Q and w(u, v) < v.cost
c
v.parent = u
0 2 v.cost = w(u, v)
i ∞

4 Node (v) Parent (v) Cost (v)

8 ∞
a Nil 0
c∈ Q F ih<cost of h b a 4
Not considered 7<8T h a, i 8, 7
i c 2
ig < cost of g
c b 8
6<∞ T
g i 6
d c 7
a b h i c g d f e
f c 4
0 4 7 2 2 6 7 4 ∞ e
 a b h i c g d f e Now f is extracted
0 4 7 2 2 6 7 4 ∞ for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
v.parent = u
4 2 v.cost = w(u, v)
7
b c
0 2
i ∞
Node (v) Parent (v) Cost (v)
7 4
6 a Nil 0
fg<cost of g fd<cost of d b a 4
2<6 T 14 < 7 F h a, i 8, 7
i c 2
fe < cost of e c∈ Q F c b 8
10 < ∞ T Not cosidered g i, f 6, 2
d c 7
a b h i c g d f e
f c 4
0 4 7 2 2 2 7 4 10 e f 10
 a b h i c g d f e Now g is extracted
0 4 7 2 2 2 7 4 10
for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
v.parent = u
4 2 v.cost = w(u, v)
7
b c
0 2
i 10
Node (v) Parent (v) Cost (v)
7 4
6 a Nil 0
i∈ Q F gh<cost of h b a 4
Not considered 1<7T h a, i, g 8, 7, 1
i c 2
f∈ Q F
c b 8
Not considered
g i, f 6, 2
d c 7
a b h i c g d f e
f c 4
0 4 1 2 2 2 7 2 10 e f 10
 a b h i c g d f e Now h is extracted
0 4 1 2 2 2 7 4 10
for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
v.parent = u
4 2 v.cost = w(u, v)
7
b c
0 2
i 10
Node (v) Parent (v) Cost (v)
7 4
6 a Nil 0
a∈ Q F b∈ Q F b a 4
Not considered Not considered h a, i, g 8, 7, 1
i c 2
i∈ Q F g∈ Q F
c b 8
Not considered Not considered
g i, f 6, 2
d c 7
a b h i c g d f e
f c 4
0 4 1 2 2 2 7 2 10 e f 10
 a b h i c g d f e Now d is extracted
0 4 1 2 2 2 7 4 10
for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
v.parent = u
4 2 v.cost = w(u, v)
7
b c
0 2
i 10
Node (v) Parent (v) Cost (v)
7 4
6 a Nil 0
c∈ Q F f∈ Q F b a 4
Not considered Not considered h a, i, g 8, 7, 1
i c 2
de < cost of e
c b 8
9 < 10 T
g i, f 6, 2
d c 7
a b h i c g d f e
f c 4
0 4 1 2 2 2 7 2 10 e f, d 10, 9
 a b h i c g d f e Now d is extracted
0 4 1 2 2 2 7 4 10
for each v ∈ G.Adj(u)
if v ∈ Q and w(u, v) < v.cost
v.parent = u
4 2 v.cost = w(u, v)
7
b c
0 2
i 10
Node (v) Parent (v) Cost (v)
7 4
6 a Nil 0
d∈ Q F f∈ Q F b a 4
Not considered Not considered h a, i, g 8, 7, 1
i c 2
a b h i c g d f e
c b 8
0 4 1 2 2 2 7 2 10
g i, f 6, 2
d c 7
f c 4
e f, d 10, 9
 a b h i c g d f e
0 4 1 2 2 2 7 2 10

b c

Node (v) Parent (v) Cost (v)

a Nil 0
b a 4
h a, i, g 8, 7, 1
i c 2
c b 8
g i, f 6, 2
d c 7
f c 4
e f, d 10, 9
Disjoint-set operations
Disjoint-Set
• A disjoint-set is a collection i.e. S={S1, S2,…, Sk} of di
stinct dynamic sets.
• Each set is identified by a member of the set, called re
presentative.
Disjoint Set Operations：
– MAKE-SET(x)：Create new set {x} with representative
x.
– UNION(x,y)：x and y are elements of two sets. Re
move these sets and add their union. Choose a repr
esentative for it.
– FIND-SET(x)：return the representative of the set c
ontaining x.
Disjoint-set Representations using linked lists
Linked-List Implementation
• Each set as a linked-list, with head and tail, and each node co
ntains value, next node pointer and back-to-representative point
er.
Example:
• MAKE-SET costs O(1): just create a single element list.
• FIND-SET costs O(1): just return back-to-representative pointer.

UNION(x,y)
Applications
• Disjoint-set data structures model the partitioning of a set, for ex
ample to keep track of the connected components of an undire
cted graph. This model can then be used to determine whether t
wo vertices belong to the same component, or whether adding an e
dge between them would result in a cycle.
• This data structure is used Graph Library to implement its Inc
remental Connected Components functionality. It is also used for i
mplementing Kruskal's algorithm to find the minimum spanning tre
e of a graph.
• Note that the implementation as disjoint-set forests doesn't allow d
eletion of edges — even without path compression or the rank heu
ristic, this is not as easy, although more complex schemes have be
en designed that can deal with this type of incremental update.
 Kruskal's Algorithm
MST-Kruskal(G,w)
1. Sort the edges E in non-decreasing order by weight
2. A  
3. for each vertex v V[G]
4. do make-set(v)
5. for each edge (u,v)  E, taken in non-decreasing
order weight.
6. do if find-set(u)  find-set(v)
7. then A = A ∪ {(u,v)}
8. union ( u,v )
9. return A
It uses a disjoint-set data structure to maintain several disjoint sets of elements. Eac
h set contains the vertices in one tree of the current forest. The operation FIND-SE
T (u) returns a representative element from the set that contains u. Thus, we can deter
mine whether two vertices u and belong to the same tree by testing whether FIND-
SET (u) equals FIND-SET (v). To combine trees, Kruskal’s algorithm calls the UNION
procedure.
 Kruskal's Algorithm: Time Analysis

MST-Kruskal(G,w)
1. Sort the edges E in non-decreasing order ( E lg E )
by weight
2. A   Count2= (1)
3. for each vertex v V[G] Count3= ( V )
4. do make-set(v)
5. for each edge (u,v)  E, taken in Count5 = O( E )
non-decreasing order by weight
6. do if find-set(u)  find-set(v)
Sorting dominates the runtime.
7. then A = A ∪ {(u,v)} We get T( V,E ) = ( E lg E),
8. union ( u,v )
9. return A
Example:

a 4
6

5
b u

14 2
10
c v

3
8 15
d