Graph Traversals and Minimum Spanning Trees: 15-211: Fundamental Data Structures and Algorithms Rose Hoberman

Graph Traversals
and
Minimum Spanning Trees
15-211: Fundamental Data Structures

and Algorithms
Rose Hoberman
April 8, 2003
Announcements
Announcements
Readings:
• Chapter 14
HW5:
• Due in less than one week!
• Monday, April 14, 2003, 11:59pm
15-211: Fundamental Data 3 Rose Hoberman

Structures and Algorithms April 8, 2003
Today
• More Graph Terminology (some review)

• Topological sort
• Graph Traversals (BFS and DFS)
• Minimal Spanning Trees
• After Class... Before Recitation

Graph
Terminology
Paths and cycles
• A path is a sequence of nodes

v1, v2, …, vN such that (vi,vi+1)∈E for 0<i<N
– The length of the path is N-1.

– Simple path: all vi are distinct, 0<i<N
• A cycle is a path such that v1=vN

– An acyclic graph has no cycles

Cycles
BOS
DTW
SFO
PIT
JFK
LAX

More useful definitions
• In a directed graph:
• The indegree of a node v is the number of

distinct edges (w,v)∈E.
• The outdegree of a node v is the number of
distinct edges (v,w)∈E.
• A node with indegree 0 is a root.

Trees are graphs
• A dag is a directed acyclic graph.
• A tree is a connected acyclic undirected

graph.
• A forest is an acyclic undirected graph (not

necessarily connected), i.e., each connected
component is a tree.

Example DAG
Undershorts
Socks
Watch
Pants Shoes
Shirt
a DAG implies an
Belt Tie ordering on events
Jacket

Example DAG
Undershorts
Socks
Watch
Pants Shoes
Shirt
In a complex DAG, it
Belt Tie can be hard to find a
schedule that obeys
Jacket all the constraints.
Topological Sort
Topological Sort
• For a directed acyclic graph G = (V,E)

• A topological sort is an ordering of all of G’s
vertices v1, v2, …, vn such that...
Formally: for every edge (vi,vk) in E, i<k.

Visually: all arrows are pointing to the right

Topological sort
• There are often many possible topological

sorts of a given DAG
• Topological orders for this DAG :
1 2
• 1,2,5,4,3,6,7
• 2,1,5,4,7,3,6 3 4 5
• 2,5,1,4,7,3,6
• Etc. 6 7
• Each topological order is a feasible schedule.

Topological Sorts for Cyclic
Graphs?
1 2
Impossible!
3
• If v and w are two vertices on a cycle, there

exist paths from v to w and from w to v.
• Any ordering will contradict one of these paths

Topological sort algorithm
• Algorithm
– Assume indegree is stored with each node.
– Repeat until no nodes remain:
• Choose a root and output it.
• Remove the root and all its edges.
• Performance
– O(V2 + E), if linear search is used to find a root.

Better topological sort
• Algorithm:
– Scan all nodes, pushing roots onto a stack.
– Repeat until stack is empty:
• Pop a root r from the stack and output it.
• For all nodes n such that (r,n) is an edge, decrement n’s
indegree. If 0 then push onto the stack.
• O( V + E ), so still O(V2) in worst case, but better
for sparse graphs.
• Q: Why is this algorithm correct?

Correctness
• Clearly any ordering produced by this algorithm is

a topological order
But...
• Does every DAG have a topological order, and if

so, is this algorithm guaranteed to find one?

Quiz Break
Quiz
• Prove:
– This algorithm never gets stuck, i.e. if there
are unvisited nodes then at least one of
them has an indegree of zero.
• Hint:
– Prove that if at any point there are unseen
vertices but none of them have an indegree
of 0, a cycle must exist, contradicting our
assumption of a DAG.

Proof
• See Weiss page 476.

Graph Traversals
Graph Traversals
•Both take time: O(V+E)

Use of a stack
• It is very common to use a stack to keep track

of:
– nodes to be visited next, or
– nodes that we have already visited.
• Typically, use of a stack leads to a depth-first
visit order.
• Depth-first visit order is “aggressive” in the
sense that it examines complete paths.

Topological Sort as DFS
• do a DFS of graph G
• as each vertex v is “finished” (all of it’s
children processed), insert it onto the front of
a linked list
• return the linked list of vertices
• why is this correct?

Use of a queue
• It is very common to use a queue to keep

track of:
– nodes to be visited next, or
– nodes that we have already visited.
• Typically, use of a queue leads to a breadth-
first visit order.
• Breadth-first visit order is “cautious” in the
sense that it examines every path of length i
before going on to paths of length i+1.

Graph Searching ???
• Graph as state space (node = state, edge = action)

• For example, game trees, mazes, ...
• BFS and DFS each search the state space for a best
move. If the search is exhaustive they will find the same
solution, but if there is a time limit and the search space is
large...
• DFS explores a few possible moves, looking at the effects
far in the future
• BFS explores many solutions but only sees effects in the
near future (often finds shorter solutions)

Minimum Spanning Trees

Problem: Laying Telephone Wire
Central office

Wiring: Naïve Approach
Central office
Expensive!

Wiring: Better Approach
Central office
Minimize the total length of wire connecting the customers

Minimum Spanning Tree (MST)
(see Weiss, Section 24.2.2)
A minimum spanning tree is a subgraph of an

undirected weighted graph G, such that
• it is a tree (i.e., it is acyclic)

• it covers all the vertices V
– contains |V| - 1 edges
• the total cost associated with tree edges is the
minimum among all possible spanning trees
• not necessarily unique

Applications of MST
• Any time you want to visit all vertices in a graph at
minimum cost (e.g., wire routing on printed circuit boards,
sewer pipe layout, road planning…)
• Internet content distribution

– $$$, also a hot research topic
– Idea: publisher produces web pages, content distribution network
replicates web pages to many locations so consumers can access
at higher speed
– MST may not be good enough!
• content distribution on minimum cost tree may take a long time!
• Provides a heuristic for traveling salesman problems. The

optimum traveling salesman tour is at most twice the
length of the minimum spanning tree (why??)
How Can We Generate a MST?
9 9
b b
a 2 6 a 2 6
d d
4 5 4 5
5 4 5 4
5 e 5 e
c c

Prim’s Algorithm
Initialization
a. Pick a vertex r to be the root
b. Set D(r) = 0, parent(r) = null
c. For all vertices v ∈ V, v ≠ r, set D(v) = ∞
d. Insert all vertices into priority queue P,
using distances as the keys
9 Vertex Parent
b e -
a 2 6 e a b c d
d
4 5
5 4 0 ∞ ∞ ∞ ∞
5 e
c
Prim’s Algorithm
While P is not empty:
1. Select the next vertex u to add to the tree

u = P.deleteMin()
2. Update the weight of each vertex w adjacent to

u which is not in the tree (i.e., w ∈ P)
If weight(u,w) < D(w),
a. parent(w) = u
b. D(w) = weight(u,w)
c. Update the priority queue to reflect
new distance for w

Prim’s algorithm
Vertex Parent
e -
e d b c a b -
0 ∞ ∞ ∞ ∞ c -
9 b d -
a 2 6
d Vertex Parent
4 5
5 4 e -
5 e d b c a b e
c 4 5 5 ∞ c e
d e
The MST initially consists of the vertex e, and we update

the distances and parent for its adjacent vertices

Prim’s algorithm
Vertex Parent
e -
d b c a b e
4 5 5 ∞ c e
9 b d e
a 2 6
d
4 5
5 4
Vertex Parent
5 e e -
c a c b b e
2 4 5 c d
d e
a d

Prim’s algorithm
Vertex Parent
e -
a c b b e
2 4 5 c d
9 d e
b
a d
a 2 6
d
4 5
5 4
5 e
c Vertex Parent
e -
c b b e
4 5 c d
d e
a d

Prim’s algorithm
Vertex Parent
e -
c b b e
9 4 5 c d
b d e
a 2 6
a d
d
4 5
5 4
5 e Vertex Parent
c e -
b b e
5 c d
d e
a d

Prim’s algorithm
Vertex Parent
e -
b b e
5 c d
9 b d e
a 2 6 a d
d
4 5
5 4
5 e
c Vertex Parent
e -
b e
The final minimum spanning tree c d
d e
a d

Running time of Prim’s algorithm
(without heaps)
Initialization of priority queue (array): O(|V|)
Update loop: |V| calls

• Choosing vertex with minimum cost edge: O(|V|)
• Updating distance values of unconnected
vertices: each edge is considered only once
during entire execution, for a total of O(|E|)
updates
Overall cost without heaps: O(|E| + |V| 2)
When heaps are used, apply same analysis as for

Dijkstra’s algorithm (p.469) (good exercise)

Prim’s Algorithm Invariant
• At each step, we add the edge (u,v) s.t. the

weight of (u,v) is minimum among all edges
where u is in the tree and v is not in the tree
• Each step maintains a minimum spanning tree of

the vertices that have been included thus far
• When all vertices have been included, we have a

MST for the graph!

Correctness of Prim’s
• This algorithm adds n-1 edges without creating a cycle,
so clearly it creates a spanning tree of any connected
graph (you should be able to prove this).
But is this a minimum spanning tree?

Suppose it wasn't.
• There must be point at which it fails, and in particular

there must a single edge whose insertion first
prevented the spanning tree from being a minimum
spanning tree.

• Let G be a connected,
undirected graph
• Let S be the set of
edges chosen by Prim’s x
algorithm before y
choosing an errorful
edge (x,y)
• Let V' be the vertices incident with edges in S
• Let T be a MST of G containing all edges in S, but not (x,y).
w
v
• Edge (x,y) is not in T, so
there must be a path in
T from x to y since T is x
connected. y
• Inserting edge (x,y) into
T will create a cycle
• There is exactly one edge on this cycle with exactly one
vertex in V’, call this edge (v,w)
• Since Prim’s chose (x,y) over (v,w), w(v,w) >= w(x,y).

• We could form a new spanning tree T’ by swapping (x,y)
for (v,w) in T (prove this is a spanning tree).
• w(T’) is clearly no greater than w(T)
• But that means T’ is a MST
• And yet it contains all the edges in S, and also (x,y)
...Contradiction

Another Approach
• Create a forest of trees from the vertices
• Repeatedly merge trees by adding “safe edges”
until only one tree remains
• A “safe edge” is an edge of minimum weight which
does not create a cycle
9 b
a 2 6
d forest: {a}, {b}, {c}, {d}, {e}
4 5
5 4
5 e
c

Kruskal’s algorithm
Initialization
a. Create a set for each vertex v ∈ V
b. Initialize the set of “safe edges” A
comprising the MST to the empty set
c. Sort edges by increasing weight
9 b
a 2 6 F = {a}, {b}, {c}, {d}, {e}
d A=∅
4 5
5 4
E = {(a,d), (c,d), (d,e), (a,c),
5 e (b,e), (c,e), (b,d), (a,b)}
c

For each edge (u,v) ∈ E in increasing order
while more than one set remains:
If u and v, belong to different sets U and V
a. add edge (u,v) to the safe edge set
A = A ∪ {(u,v)}
b. merge the sets U and V
F = F - U - V + (U ∪ V)
Return A
• Running time bounded by sorting (or findMin)

• O(|E|log|E|), or equivalently, O(|E|log|V|) (why???)

9 b
a 2 6
E= {(a,d), (c,d), (d,e), (a,c),
d
5 4
4 5 (b,e), (c,e), (b,d), (a,b)}
5 e
c
Forest A
{a}, {b}, {c}, {d}, {e} ∅
{a,d}, {b}, {c}, {e} {(a,d)}
{a,d,c}, {b}, {e} {(a,d), (c,d)}
{a,d,c,e}, {b} {(a,d), (c,d), (d,e)}
{a,d,c,e,b} {(a,d), (c,d), (d,e), (b,e)}

Kruskal’s Algorithm Invariant
• After each iteration, every tree in the forest is a MST

of the vertices it connects
• Algorithm terminates when all vertices are connected

into one tree

Correctness of Kruskal’s
• This algorithm adds n-1 edges without creating a
cycle, so clearly it creates a spanning tree of any
connected graph (you should be able to prove this).
But is this a minimum spanning tree?

Suppose it wasn't.
• There must be point at which it fails, and in particular

there must a single edge whose insertion first
prevented the spanning tree from being a minimum
spanning tree.

K T
S
e
• Let e be this first errorful edge.
• Let K be the Kruskal spanning tree
• Let S be the set of edges chosen by Kruskal’s algorithm before choosing e
• Let T be a MST containing all edges in S, but not e.

Lemma: w(e’) >= w(e) for all edges e’ in T - S
Proof (by contradiction):

• Assume there exists some K T
edge e’ in T - S, w(e’) < S
w(e)
• Kruskal’s must have e
considered e’ before e
• However, since e’ is not in K (why??), it must have
been discarded because it caused a cycle with some of
the other edges in S.
• But e’ + S is a subgraph of T, which means it cannot
form a cycle ...Contradiction
• Inserting edge e into T will create a cycle

• There must be an edge on this cycle which is not in K (why??).
Call this edge e’
• e’ must be in T - S, so (by our lemma) w(e’) >= w(e)
• We could form a new spanning tree T’ by swapping e for e’ in T
(prove this is a spanning tree).
• w(T’) is clearly no greater than w(T)
• But that means T’ is a MST
• And yet it contains all the edges in S, and also e
...Contradiction

Greedy Approach
• Like Dijkstra’s algorithm, both Prim’s and Kruskal’s

algorithms are greedy algorithms
• The greedy approach works for the MST problem;

however, it does not work for many other
problems!

That’s All!

Graph Traversals and Minimum Spanning Trees: 15-211: Fundamental Data Structures and Algorithms Rose Hoberman

Uploaded by

Copyright:

Available Formats

You might also like

Graph Traversals and Minimum Spanning Trees: 15-211: Fundamental Data Structures and Algorithms Rose Hoberman

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Graph Traversals and Minimum Spanning Trees: 15-211: Fundamental Data Structures and Algorithms Rose Hoberman

Uploaded by

Copyright:

Available Formats

Graph Traversals

15-211: Fundamental Data Structures

15-211: Fundamental Data 3 Rose Hoberman

• More Graph Terminology (some review)

15-211: Fundamental Data 4 Rose Hoberman

• A path is a sequence of nodes

– The length of the path is N-1.

• A cycle is a path such that v1=vN

15-211: Fundamental Data 6 Rose Hoberman

15-211: Fundamental Data 7 Rose Hoberman

• The indegree of a node v is the number of

• A node with indegree 0 is a root.

15-211: Fundamental Data 8 Rose Hoberman

• A dag is a directed acyclic graph.

• A tree is a connected acyclic undirected

• A forest is an acyclic undirected graph (not

15-211: Fundamental Data 9 Rose Hoberman

15-211: Fundamental Data 10 Rose Hoberman

• For a directed acyclic graph G = (V,E)

Formally: for every edge (vi,vk) in E, i<k.

15-211: Fundamental Data 13 Rose Hoberman

• There are often many possible topological

• Each topological order is a feasible schedule.

15-211: Fundamental Data 14 Rose Hoberman

• If v and w are two vertices on a cycle, there

15-211: Fundamental Data 15 Rose Hoberman

15-211: Fundamental Data 16 Rose Hoberman

• Q: Why is this algorithm correct?

15-211: Fundamental Data 17 Rose Hoberman

• Clearly any ordering produced by this algorithm is

• Does every DAG have a topological order, and if

15-211: Fundamental Data 18 Rose Hoberman

15-211: Fundamental Data 20 Rose Hoberman

• See Weiss page 476.

15-211: Fundamental Data 21 Rose Hoberman

•Both take time: O(V+E)

15-211: Fundamental Data 23 Rose Hoberman

• It is very common to use a stack to keep track

15-211: Fundamental Data 24 Rose Hoberman

• why is this correct?

15-211: Fundamental Data 25 Rose Hoberman

• It is very common to use a queue to keep

15-211: Fundamental Data 26 Rose Hoberman

• Graph as state space (node = state, edge = action)

15-211: Fundamental Data 27 Rose Hoberman

15-211: Fundamental Data 28 Rose Hoberman

15-211: Fundamental Data 29 Rose Hoberman

15-211: Fundamental Data 30 Rose Hoberman

Minimize the total length of wire connecting the customers

15-211: Fundamental Data 31 Rose Hoberman

A minimum spanning tree is a subgraph of an

• it is a tree (i.e., it is acyclic)

15-211: Fundamental Data 32 Rose Hoberman

• Internet content distribution

• Provides a heuristic for traveling salesman problems. The

15-211: Fundamental Data 34 Rose Hoberman

1. Select the next vertex u to add to the tree