Spectral Graph Theory and its Applications

Lillian Dai
6.454
Oct. 20, 2004

1
Outline

• Basic spectral graph theory

• Graph partitioning using spectral methods

D. Spielman and S. Teng, “Spectral Partitioning Works: Planar

Graphs and Finite Element Meshes,” 1996

2
 Graph and Associated Matrices
2
G = (V , E ) Laplacian matrix
1 3
V =n=4 LG = DG − AG

4
= BG BG T
E =m=5

Adjacency matrix Degree matrix Incidency matrix

0 1 1 1 3 0 0 0 1 1 1 0 0
1 0 0 1  0 2 0 0   −1 0 0 1 0 
AG =  DG =  BG =  
1 0 0 1 0 0 2 0  0 −1 0 0 1 
     
1 1 1 0 0 0 0 3  0 0 −1 −1 −1
3
Properties of the Laplacian Matrix
2

1 3

λ = {0, 2, 4, 4}
4

 3 −1 −1 −1 1 0  −1  −2 

 −1 2 0 −1 1  −1  −1 1
LG =         
 −1 0 2 −1 1 1  −1 1
        
 −1 −1 −1 3  1 0 3 0
• Symmetric -> real eigenvalues; eigenspaces are mutually
orthogonal

• Orthogonally diagonalizable -> an eigenvalue with multiplicity k

4
has k-dimensional eigenspace
More Properties of the Laplacian Matrix
• Positive semidefinite -> non-
λ = {0, 2, 4, 4} negative eigenvalues

• Row sum = 0 -> singular -> at

1 0  −1  −2 
1  −1  −1 1 least one eigenvalue = 0, unity
       eigenvector (since row sum = 1)
1 1  −1 1
       • Orthogonal eigenspaces
1 0 3 0
u = eigenvector of non-zero
n
eigenvalue u =0 ∑i =1
i

x = ( x1 , x2 ,..., xn ) ∈ \ n

( )( x ) ∑ (x − x )
T 2
T T T T T
x LG x = x BG BG x = x BG BG = i j ≥0
5
1× m m ×1 ( i , j )∈E
 Spectrum of Some Graphs
Which graphs are
determined by their
Eigenvalues spectrum?
,
Complete {
0, n(
n −1)
} • Complete Graphs
• Graphs with one edge
Line 2 − 2cos (π k n )
• Graphs missing 1 edge
k = 1,..., n • Regular graphs with
Ring 2 − 2cos ( 2π k n ) degree 2
k = 1,..., n 2 • Regular graphs of
degree n - 3
Star
{ 0,1(
n− 2 )
,2 }
6
Graph Connectedness λ1 ≤ λ2 ≤ ... ≤ λn

λ = {0, 2, 4, 4} For connected graphs, λ2 > 0

∑ (x − x )
KT K 2
1 0  −1  −2  Recall LG x =
x i j
1  −1  −1 1 ( i , j )∈E
       K
1 1  −1 1 If is eigenvector for eigenvalue 0
x
       K xi = x j ( i , j )∈E
1 0 3 0 LG x = 0

Multiplicity of the 0 eigenvalue

indicates # of connected components
λ2 Fiedler Value
K
v Fiedler Vector
7
Onto Graph Partitioning …

8
Graph Partitioning
• Remove as little of the graph as possible
to separate out a subset of vertices of some
desired “size”
• “Size” may mean the number of vertices,
number of edges, etc.
• Typical case is to remove as few edges as
possible to disconnect the graph into two
parts of almost equal size Diagram from Berkeley CS 267 lecture notes

Isoperimetric problem
One of the earliest problems in geometry –
considered by the ancient Greeks: Find, among all
closed curves of a given length, the one which
encloses the maximum area Stein, 1841
9
Applications

• Load balancing while minimizing communication

• Sparse matrix-vector multiplication

• Optimizing VLSI layout

• Communication network design

10
Bisection and Ratio-Partition
• Divide vertices into two disjoint subsets S and S

• Cut Size (
E S, S )

• Cut Ratio φG ( S ) =
(
E S, S )
(
min S , S )
• Isoperimetric Number φG = min φG ( S )
S ⊂V

Bisection Minimize E ( S , S ) subject to # of nodes in each partition

differ by at most 1.
NP-Complete

Ratio-Partition Minimize φG ( S ) 11
Spectral Partitioning

• Find Fiedler vector of the Laplacian matrix – map to vertices

• Choose some real number s
• Partition vertices given by VL = {i : vi ≤ s}

VL = {i : vi > s}

• Bisection, s = median of {v1 ,...vn }

• Ratio partition, s is chosen to give the best cut ratio

12
Example

1 4

2 5

3 6

Fiedler vector [-1 -2 -1 1 2 1]

13
Spectral Partitioning For Planar Graphs

• Guattery and Miller – Performance of Spectral Graph

Partitioning, 1995

• Spielman and Teng, Spectral Partitioning Works on Planar

Graphs, 1996

• Kelner, Spectral Partitioning Works on Graphs with Bounded

Genus, 2004

14
Simple Spectral Bisection May Fail
(Guattery & Miller)

The simple spectral bisection method produces cut size of Θ ( n )

for Gk , for any k
15
Optimal Bisector for Graphs with Bounded Genus
(Kelner)

Genus g of a graph G: smallest integer such that G can be

embedded on a surface of genus g without any of its edges
crossing one another. Eg. Planar graphs have genus 0

Sphere, disc, and annulus has genus 0

Torus has genus 1

There is a spectral algorithm that produces bisector of size O ( gn )

For every g, there is a class of bounded degree graphs that have
(
no bisectors smaller than O gn )
16
Improved Bisection Algorithm on Planar Graphs
(Spielman and Teng)

Bisector of size O ( n)
Why does the spectral method work?
Why does it work well on planar graphs?
Why does simple bisection fail even on
planar graphs?

17
Another Look at Fiedler Value
K
T
Recall x LG x = ∑ (x − x ) i j
2
where x = (1 2
x , x ,..., xn) ∈ \ n

( i , j )∈E

∑( (x − x )
2
K K
x T LG x i , j )∈E i j
Rayleigh quotient: φx = KT K =
x x ∑x i
2

λ = min φ
Fiedler value satisfies 2 xK ⊥(1,...,1) x with the minimum
occurring only when xK is a Fiedler vector.

K K K K
x T LG x x T λ2 x
φx = KT K = KT K = λ2
x x x x

18
Connection Between Fiedler Value and
Isoperimetric Number

Recall Isoperimetric Number

(
E S, S )
φG = min
is the best ratio-partition possible
S ⊂V
min ( S , S )
Theorem 1 (Mihail ‘89) Let G be a graph on n nodes of
n
maximum degree ∆ . For any vector xK ∈ \ n such that ∑x i =1
i =0

K K
x T LG x φG 2 φG 2 Good ratio-partition
KT K ≥ λ2 ≥ can be achieved if
x x 2∆ 2∆ Fiedler value is small

Moreover, there is an s so that the cut {i : vi ≤ s} {i : vi > s} has

ratio at most φG 2 ( 2∆ )
19
Upperbound on the Fiedler Value for Planar Graphs

Theorem 2 (Spielman & Teng ‘96) For all planar graphs G

with n vertices and maximum degree ∆
8∆ 1
λ2 ≤ O 
n n

φG 2 8∆ 4∆  1 
≤ λ2 ≤ φG ≤ O 
2∆ n n  n

By bounding Fiedler value of planar graphs, ratio-partitioning

method is shown to work well

What about bisection? 20

Relationship Between Ratio-Partitioning and Bisection

Lemma 3 Given an algorithm that will find a cut ratio of at most

φ ( k ) in every k-node subgraph of G , for some monotonically
decreasing function φ . Then repeated application of this
algorithm can be used to find a bisection of G of size at most
n
∫ φ ( x )dx
x =1

1
φ ( x) =
∫ φ ( x )dx = 2 ( ) ( n)
n
n −1 O
x x =1

Bisection can be obtained by repeated application of ratio-

partitioning

21
 K K
x T LG x φG 2 n

Theorem 1 KT K ≥
x x 2∆
∑x
i =1
i =0

2
(
E S, S ) Map graph vertices to a line 1 3
φG = min
S ⊂V
(
min S , S ) 4

x1 ≤ x2 ≤ ... ≤ xn x1 x2 xn

) (
∑( x −x )
2
K K
sum ( length of edge )
2
x T LG x i , j ∈E i j
KT K = =
x x ∑x i
2
sum ( length away from 0 )
2

If i ≤ n 2 At least φG i edges must cross over xi

22
 8∆
Proof of Theorem 2 λ2 ≤
n

Theorem 4 (Koebe-Andreev-Thurston). Let G

be a planar graph. Then, there exist a set of disks {D1 ,..., Dn }
in the plane with disjoint interiors such that Di touches D j
iff ( i, j ) ∈ E

Kissing disks
.

23
Proof of Theorem 2 cont.

Stereographic Projection
{π ( D1 ) ,...,π ( Dn )}
Circles in the plane ->
circular caps on the
sphere

24
Proof of Theorem 2 cont.
K
Let i be the center of π ( Di ) on the sphere.
x
n
K K 2
xi = 1 ∑ i = n
x
i =1

Let ri be the radius of the cap π ( Di )

K K
( ) ( )
2 2
xi − x j ≤ ri + rj ≤ 2 ri 2 + rj 2 ( i, j ) ∈ E

∑π r i
2
≤ 4π

∑ (r ) ∑d r
K K
∑
2 2
xi − x j ≤2 i + rj 2 ≤ 2 i i
2
≤ 8∆
( i , j )∈E ( i , j )∈E i

K K
∑(
2
i , j )∈E
xi − x j 8∆
λ2 ≤ K 2 ≤
∑ xi n
25
Conclusion

Why does the spectral method work?

φG 2
- close relationship between Fiedler value and Isoperimetric number λ2 ≥
2∆

Why does it work well on planar graphs? 8∆

- planar graphs have nice collection of spherical cap embeddings λ2 ≤
n

Why does simple bisection fail even on planar graphs?

- even though good ratio-partitions can be found, the result may be
unbalanced in the size of the partitions

26