Greedy algorithm

A greedy algorithm is any algorit hm t hat follows t he problem-solving heurist ic of making t he

locally opt imal choice at each st age.[1] In many problems, a greedy st rat egy does not produce an
opt imal solut ion, but a greedy heurist ic can yield locally opt imal solut ions t hat approximat e a
globally opt imal solut ion in a reasonable amount of t ime.

Greedy algorithms determine the minimum number of coins to give while making change. These are the steps most
people would take to emulate a greedy algorithm to represent 36 cents using only coins with values {1, 5, 10, 20}. The coin
of the highest value, less than the remaining change owed, is the local optimum. (In general, the change-making problem
requires dynamic programming to find an optimal solution; however, most currency systems, including the Euro and US
Dollar, are special cases where the greedy strategy does find an optimal solution.)

For example, a greedy st rat egy for t he t ravelling salesman problem (which is of high
comput at ional complexit y) is t he following heurist ic: "At each st ep of t he journey, visit t he
nearest unvisit ed cit y." This heurist ic does not int end t o find t he best solut ion, but it t erminat es
in a reasonable number of st eps; finding an opt imal solut ion t o such a complex problem t ypically
requires unreasonably many st eps. In mat hemat ical opt imizat ion, greedy algorit hms opt imally
solve combinat orial problems having t he propert ies of mat roids and give const ant -fact or
approximat ions t o opt imizat ion problems wit h t he submodular st ruct ure.

Specifics

Greedy algorit hms produce good solut ions on some mat hemat ical problems, but not on ot hers.
Most problems for which t hey work will have t wo propert ies:

Greedy choice property

We can make what ever choice seems best at t he moment and t hen solve t he subproblems
t hat arise lat er. The choice made by a greedy algorit hm may depend on choices made so far,
but not on fut ure choices or all t he solut ions t o t he subproblem. It it erat ively makes one
greedy choice aft er anot her, reducing each given problem int o a smaller one. In ot her words, a
greedy algorit hm never reconsiders it s choices. This is t he main difference from dynamic
programming, which is exhaust ive and is guarant eed t o find t he solut ion. Aft er every st age,
dynamic programming makes decisions based on all t he decisions made in t he previous st age
and may reconsider t he previous st age's algorit hmic pat h t o t he solut ion.
Optimal substructure
"A problem exhibit s opt imal subst ruct ure if an opt imal solut ion t o t he problem cont ains
opt imal solut ions t o t he sub-problems."[2]

Cases of failure

Examples on how a greedy algorithm may fail to achieve the optimal solution.
 Starting from A, a greedy algorithm that tries to find
the maximum by following the greatest slope will find
the local maximum at "m", oblivious to the global
maximum at "M".

To reach the largest sum, at each step, the greedy

algorithm will choose what appears to be the optimal
immediate choice, so it will choose 12 instead of 3 at

the second step, and will not reach the best solution,
which contains 99.

Greedy algorit hms fail t o produce t he opt imal solut ion for many ot her problems and may even
produce t he unique worst possible solut ion. One example is t he t ravelling salesman problem
ment ioned above: for each number of cit ies, t here is an assignment of dist ances bet ween t he
cit ies for which t he nearest -neighbour heurist ic produces t he unique worst possible t our.[3]
For
ot her possible examples, see horizon effect .
 Types

Greedy algorit hms can be charact erized as being 'short sight ed', and also as 'non-recoverable'.
They are ideal only for problems t hat have an 'opt imal subst ruct ure'. Despit e t his, for many
simple problems, t he best -suit ed algorit hms are greedy. It is import ant , however, t o not e t hat
t he greedy algorit hm can be used as a select ion algorit hm t o priorit ize opt ions wit hin a search, or
branch-and-bound algorit hm. There are a few variat ions t o t he greedy algorit hm:

Pure greedy algorit hms

Ort hogonal greedy algorit hms

Relaxed greedy algorit hms

Theory

Greedy algorit hms have a long hist ory of st udy in combinat orial opt imizat ion and t heoret ical
comput er science. Greedy heurist ics are known t o produce subopt imal result s on many
problems,[4] and so nat ural quest ions are:

For which problems do greedy algorit hms perform opt imally?

For which problems do greedy algorit hms guarant ee an approximat ely opt imal solut ion?

For which problems are t he greedy algorit hm guarant eed not t o produce an opt imal solut ion?

A large body of lit erat ure exist s answering t hese quest ions for general classes of problems, such
as mat roids, as well as for specific problems, such as set cover.

Matroids

A mat roid is a mat hemat ical st ruct ure t hat generalizes t he not ion of linear independence from
vect or spaces t o arbit rary set s. If an opt imizat ion problem has t he st ruct ure of a mat roid, t hen
t he appropriat e greedy algorit hm will solve it opt imally.[5]

Submodular functions

A funct ion defined on subset s of a set is called submodular if for every we have
t hat .
Suppose one want s t o find a set which maximizes . The greedy algorit hm, which builds up a
set by increment ally adding t he element which increases t he most at each st ep, produces
as out put a set t hat is at least .[6] That is, greedy performs wit hin a

const ant fact or of as good as t he opt imal solut ion.

Similar guarant ees are provable when addit ional const raint s, such as cardinalit y const raint s,[7] are
imposed on t he out put , t hough oft en slight variat ions on t he greedy algorit hm are required. See
[8] for an overview.

Other problems with guarantees

Ot her problems for which t he greedy algorit hm gives a st rong guarant ee, but not an opt imal
solut ion, include

Set cover

The St einer t ree problem

Load balancing[9]

Independent set

Many of t hese problems have mat ching lower bounds; i.e., t he greedy algorit hm does not perform
bet t er t han t he guarant ee in t he worst case.

Applications

Greedy algorit hms t ypically (but not always) fail t o find t he globally opt imal solut ion because
t hey usually do not operat e exhaust ively on all t he dat a. They can make commit ment s t o cert ain
choices t oo early, prevent ing t hem from finding t he best overall solut ion lat er. For example, all
known greedy coloring algorit hms for t he graph coloring problem and all ot her NP-complet e
problems do not consist ent ly find opt imum solut ions. Nevert heless, t hey are useful because
t hey are quick t o t hink up and oft en give good approximat ions t o t he opt imum.

If a greedy algorit hm can be proven t o yield t he global opt imum for a given problem class, it
t ypically becomes t he met hod of choice because it is fast er t han ot her opt imizat ion met hods
like dynamic programming. Examples of such greedy algorit hms are Kruskal's algorit hm and Prim's
algorit hm for finding minimum spanning t rees and t he algorit hm for finding opt imum Huffman
t rees.
Greedy algorit hms appear in t he net work rout ing as well. Using greedy rout ing, a message is
forwarded t o t he neighbouring node which is "closest " t o t he dest inat ion. The not ion of a node's
locat ion (and hence "closeness") may be det ermined by it s physical locat ion, as in geographic
rout ing used by ad hoc net works. Locat ion may also be an ent irely art ificial const ruct as in small
world rout ing and dist ribut ed hash t able.

Examples

The act ivit y select ion problem is charact erist ic of t his class of problems, where t he goal is t o
pick t he maximum number of act ivit ies t hat do not clash wit h each ot her.

In t he Macint osh comput er game Crystal Quest t he object ive is t o collect cryst als, in a fashion
similar t o t he t ravelling salesman problem. The game has a demo mode, where t he game uses a
greedy algorit hm t o go t o every cryst al. The art ificial int elligence does not account for
obst acles, so t he demo mode oft en ends quickly.

The mat ching pursuit is an example of a greedy algorit hm applied on signal approximat ion.

A greedy algorit hm finds t he opt imal solut ion t o Malfat t i's problem of finding t hree disjoint
circles wit hin a given t riangle t hat maximize t he t ot al area of t he circles; it is conject ured t hat
t he same greedy algorit hm is opt imal for any number of circles.

A greedy algorit hm is used t o const ruct a Huffman t ree during Huffman coding where it finds
an opt imal solut ion.

In decision t ree learning, greedy algorit hms are commonly used, however t hey are not
guarant eed t o find t he opt imal solut ion.
One popular such algorit hm is t he ID3 algorit hm for decision t ree const ruct ion.

Dijkst ra's algorit hm and t he relat ed A* search algorit hm are verifiably opt imal greedy algorit hms
for graph search and short est pat h finding.
A* search is condit ionally opt imal, requiring an "admissible heurist ic" t hat will not
overest imat e pat h cost s.

Kruskal's algorit hm and Prim's algorit hm are greedy algorit hms for const ruct ing minimum
spanning t rees of a given connect ed graph. They always find an opt imal solut ion, which may
not be unique in general.

See also

Best-first search Epsilon-greedy strategy

 Greedy algorithm for Egyptian fractions Horizon effect

Greedy source Matroid

References

1. Black, Paul E. (2 February 2005). "greedy algorithm" (http://xlinux.nist.gov/dads//HTML/greedyalgo.h

tml) . Dictionary of Algorithms and Data Structures. U.S. National Institute of Standards and
Technology (NIST). Retrieved 17 August 2012.

2. Cormen et al. 2001, Ch. 16

3. Gutin, Gregory; Yeo, Anders; Zverovich, Alexey (2002). "Traveling salesman should not be greedy:
Domination analysis of greedy-type heuristics for the TSP" (https://doi.org/10.1016%2FS0166-218X%28
01%2900195-0) . Discrete Applied Mathematics. 117 (1–3): 81–86. doi:10.1016/S0166-
218X(01)00195-0 (https://doi.org/10.1016%2FS0166-218X%2801%2900195-0) .

4. Feige 1998

5. Papadimitriou & Steiglitz 1998

6. Nemhauser, Wolsey & Fisher 1978

7. Buchbinder et al. 2014

8. Krause & Golovin 2014

9. "Lecture 5: Introduction to Approximation Algorithms" (http://www.win.tue.nl/~mdberg/Onderwijs/AdvAl

g_Material/Course%20Notes/lecture5.pdf) (PDF). Advanced Algorithms (2IL45) — Course Notes. TU
Eindhoven.

Sources
Cormen, Thomas H.; Leiserson, Charles E.; Rivest, Ronald L.; Stein, Clifford (2001). "16 Greedy
Algorithms" (https://books.google.com/books?id=NLngYyWFl_ YC&pg=PA370) . Introduction To
Algorithms. MIT Press. pp. 370–. ISBN 978-0-262-03293-3.

Gutin, Gregory; Yeo, Anders; Zverovich, Alexey (2002). "Traveling salesman should not be greedy:
Domination analysis of greedy-type heuristics for the TSP" (https://doi.org/10.1016%2FS0166-218X%
2801%2900195-0) . Discrete Applied Mathematics. 117 (1–3): 81–86. doi:10.1016/S0166-
218X(01)00195-0 (https://doi.org/10.1016%2FS0166-218X%2801%2900195-0) .

Bang-Jensen, Jørgen; Gutin, Gregory; Yeo, Anders (2004). "When the greedy algorithm fails" (https://d
oi.org/10.1016%2Fj.disopt.2004.03.007) . Discrete Optimization. 1 (2): 121–127.
doi:10.1016/j.disopt.2004.03.007 (https://doi.org/10.1016%2Fj.disopt.2004.03.007) .

Bendall, Gareth; Margot, François (2006). "Greedy-type resistance of combinatorial problems" (http
s://doi.org/10.1016%2Fj.disopt.2006.03.001) . Discrete Optimization. 3 (4): 288–298.
doi:10.1016/j.disopt.2006.03.001 (https://doi.org/10.1016%2Fj.disopt.2006.03.001) .

Feige, U. (1998). "A threshold of ln n for approximating set cover" (https://www.cs.duke.edu/courses/

cps296.2/spring07/papers/p634-feige.pdf) (PDF). Journal of the ACM. 45 (4): 634–652.
doi:10.1145/285055.285059 (https://doi.org/10.1145%2F285055.285059) . S2CID 52827488 (https://a
pi.semanticscholar.org/CorpusID:52827488) .

Nemhauser, G.; Wolsey, L.A.; Fisher, M.L. (1978). "An analysis of approximations for maximizing
submodular set functions—I" (https://www.researchgate.net/publication/242914003) . Mathematical
Programming. 14 (1): 265–294. doi:10.1007/BF01588971 (https://doi.org/10.1007%2FBF01588971) .
S2CID 206800425 (https://api.semanticscholar.org/CorpusID:206800425) .

Buchbinder, Niv; Feldman, Moran; Naor, Joseph (Seffi); Schwartz, Roy (2014). "Submodular
maximization with cardinality constraints" (http://theory.epfl.ch/moranfe/Publications/SODA2014.p
df) (PDF). Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms. Society
for Industrial and Applied Mathematics. doi:10.1137/1.9781611973402.106 (https://doi.org/10.1137%2
F1.9781611973402.106) . ISBN 978-1-61197-340-2.

Krause, A.; Golovin, D. (2014). "Submodular Function Maximization" (https://books.google.com/books?

id=YxliAgAAQBAJ&pg=PA71) . In Bordeaux, L.; Hamadi, Y.; Kohli, P. (eds.). Tractability: Practical
Approaches to Hard Problems. Cambridge University Press. pp. 71–104.
doi:10.1017/CBO9781139177801.004 (https://doi.org/10.1017%2FCBO9781139177801.004) .
ISBN 9781139177801.

External links

Wikimedia Commons has media relat ed t o Greedy algorithms.

"Greedy algorit hm" (ht t ps://www.encyclopediaofmat h.org/index.php?t it le=Greedy_ algorit hm) ,

Encyclopedia of Mathematics, EMS Press, 2001 [1994]

Gift , Noah. "Pyt hon greedy coin example" (ht t p://www.oreillynet .com/onlamp/blog/2008/04/p
yt hon_ greedy_ coin_ changer_ alg.ht ml) .
Retrieved from
"https://en.wikipedia.org/w/index.php?
title=Greedy_algorithm&oldid=1052706695"

Last edited 15 days ago by 163.118.51.53

Wikipedia

Content is available under CC BY-SA 3.0 unless

otherwise noted.