In my lectures everything is finite and real or even integral valued!

Topics: Graphs, Matrix representations of Graphs, in particular the graph Laplacian, some
applications and results in spectral graph theory

1 Introduction to Graphs and Matrix Representations

Simple graph G = (N, E) (without loops or multiple edges)

of nodes/vertices, e.g., N = {1, . . . , n}
N . . . set
N
E ⊆ 2 := {{i, j} : i, j ∈ N, i 6= j} . . . set of edges
Instead of {i, j} ∈ E we often write ij ∈ E.
In a multigraph there may be several copies of ij as well as loops ii.
Usually we don’t care about the names of the nodes, mostly we are only interested in
properties, that are independent on the names of the nodes but just deal with their pairwise
relations, the edges:
G1 = (N1 , E1 ) and G2 (N2 , E2 ) are isomorphic (G1 ∼ G2 ), if there is a bijective map
ϕ : N1 → N2 so that ij ∈ E1 ⇔ ϕ(i)ϕ(j) ∈ E2 .
Some special graphs (w.l.o.g. N = {1, . . . , n}, ):
Complete Graph/clique (of size n): Kn = (N, N2 )

Cycle (of length n): Cn = (N, {12, 23, . . . , (n − 1)n, n1})

(a cycle with n = 1 requires a loop, with n = 2 multiple edges)
Path (of length n − 1): Pn = (N, {12, 23, . . . , (n − 1)n}), 1 and n are its end nodes
Some terminology for graphs: G = (N, E)
Nodes i, j ∈ N are adjacent or neighbors (in G) if ij ∈ E,
Node i ∈ N and edge e ∈ E are incident (in G) if i ∈ e
Edges e, f ∈ E are incident (in G) if e ∩ f 6= ∅
The number of incident edges in a vertex v ∈ N is its degree dG (v) = |{vj ∈ E}|
Questions: dKn (i) =?, dCn (i) =?
Some operations on a graph G = (N, E):
delete an edge e ∈ E: G − e := (N, E \ {e})
delete a node v ∈ N : G − v := (N \ {v}, E \ {vj ∈ E})
Likewise define deletion of subsets of edges (G − E 0 ) or nodes (G − N 0 ).
The contraction G/N 0 of a node subset N 0 ⊆ N (e. g., an edge) replaces N 0 by a new node
v∈/ N and for e ∈ E any i ∈ e ∩ N 0 is replaced by v as well. This may give rise to multiple
edges and loops, thus to a multigraph. If the result is to be a simple graph, multiple edges
are replaced by a single copy and loops are deleted.
The complement of a graph: Ḡ := (N, N2 \ E)


[K̄n ?, C̄n ?]
0 0 0 0 0
A graph G = (N , E ) is a subgraph of G if N ⊆ N and E ⊆ E.

1
A subgraph G0 = (N 0 , E 0 ) of G is induced if E 0 = {ij ∈ E : i, j ∈ N 0 }.
A walk (of length k ∈ N0 ) in G (between u ∈ N and w ∈ N ) is a sequence of edges
({u = v0 , v1 }, v1 v2 , . . . , {vk−1 , vk = w}) with vi vi+1 ∈ E and the vi not nec. distinct. A walk
with u = w is closed .

Exercise 1.1 For G = (N, E) and u, w ∈ N put dG (u, w) to be the minmum length of a
walk between u and w if one exists and ∞ otherwise. Show that d : N × N → N0 ∪ {∞}
defines a metric on G and that any existing shortest walk is a path (as a subgraph in G).
This d is the shortest path distance.

n
o
Attention! We will need to be careful not to mix up the degree dG (i) of a node and the
distance dG (i, j) between nodes i and j. 4

u, w ∈ N are connected in G if there is a walk in G between them.

Connectedness is an equivalence relation. The subgraphs induced by the corresponding
equivalence classes are the connected components of G. A graph G = (N, E) is connected if
it has only one connected component, i. e., if all u, v ∈ N are connected (in G).
The adjacency matrix of a graph is the matrix A ∈ RN ×N with

Aij = |{ij ∈ E}| (also works for multigraphs)

Exercise 1.2 Show that for k ∈ N0 the ij entry of Ak = A

| · .{z
. . · A} gives the number of
k times
walks of length k.

n
o
0
Proof: By induction:X A = I is ok.X
(Ak−1 )i,• · A•,j = (Ak−1 )ih ahj = (append hj to each k − 1 walk from i to h). 
h∈N hj∈E
k
In particular (A )ii gives the number of closed walks of length k starting and P
ending in i.
Hence, the trace (the sum of diagonal elements of a square matrix) tr(A ) = i∈N (Ak )ii
k

yields the sum of all closed walks of length k over all starting nodes in G.
Remember the spectral theorem for symmetric matrices:
Theorem 1.3 (Spectral decomposition for symmetric matrices)
Let A = A> be a symmetric matrix of order n. All its eigenvalues are real and it has an or-
thonormal basis of eigenvectors, i. e., there is a diagonal matrix ΛA = Diag(λ1 (A), . . . , λn (A))
and a matrix P = [v1 , . . . , vn ] with P > P = In = P P > so that
n
X
>
A = P ΛA P = λi (A)vi vi> .
i=1

Exercise 1.4 Show that for matrices U ∈ Rn×m , V ∈ Rm×n the trace is invariant under
rotations,
P tr U V = tr V U . Use this to show tr A = tr ΛA . In consequence, for a simple
graph ni=1 λi (A)3 /3 gives the number of triangles (C3 subgraphs) of G.
n
o

2
For the extremal eigenvalues the following characterization is useful.

Theorem 1.5 (Rayleigh-Ritz characterization of extremal eigenvalues)

For a symmetrix matrix A

v > Av
λmax (A) = max v > Av = max ,
kvk=1 v6=0 v>v

v > Av
λmin (A) = min v > Av = min ,
kvk=1 v6=0 v>v

Proof: Consider the eigenvalue decomposition A = P ΛA P > with ΛA = Diag(λ1 ≥ · · · ≥

λn ), and some v with kvk = 1, then w = P > v satisfies kwk = 1 and
X X
v > Av = (P > v)> ΛA (P > v) = λi wi2 ≤ λ1 wi2 = λ1 ,
i∈N i∈N
X X
and v > Av = λi wi2 ≥ λn wi2 = λn .
i∈N i∈N

Now v = P•,1 yields v > Av = λ1 and v = P•,n gives v > Av = λn . 

In order to learn more about the maximum eigenvalue of the adjacency matrix A we will
exploit that A is nonnegative, symmetric and irreducible.

Definition 1.6 A symmetric matrix A is reducible if there is a permutation matrix P (an

ordering of the nodes) so that
 
−1 X 0
P AP = (with X and Y square matrices),
0 Y

otherwise it is irreducible.

A reducible adjacency matrix has at least two connected components, so for a connected
graph it is irreducible.

Theorem 1.7 (Symmetric Perron-Frobenius)

Let A be an irrducible symmetric nonnegative matrix. The largest eigenvalue µ1 of A has a
unique strictly positive eigenvector and µ1 ≥ kµi k for eigenvalues i = 2, . . . , n,

Proof: Put E = {ij : Aij > 0, i = 6 j} and let x with

P kxk = 1 be P an eigenvector to µ1 . Put
> >
yi = |xi |, i ∈ N , then kyk = 1 and 0 ≤ y Ay = Aij yi yj ≥ Aij xi xj = x Ax = µ1 , so
by Rayleigh-Ritz y is eigenvctor to µ1 and satisfies the eigenvector equations
X
µ1 yi = Aii yi + Aij yj , i = 1 . . . n.
ij∈E

3
Suppose yi = 0, then yj = 0 for all ij ∈ E and by connectednedss y = 0, thus y > 0
componentwise.
Suppose µ1 is not simple, then there is an eigenvector z with z > y = 0, thus with negative
entries. Choose α so that y + αz ≥ 0 with at least one zero entry to see the contradiction
as before.
Let Az = µz with
>
Pkzk = 1 andPsome eignevalue µ, then
|µ| = |z Az| = | Aij zi zj | ≤ Aij |zi ||zj | ≤ µ1 . 
So every connected component gives rise to a block submatrix of A with a corresponding
strictly positive eigenvector on its support to its largest eigenvalue.
Under which circumstances is there another eigenvalue µ with |µ| = µ1 ?
˙ 2 , E ⊆ {ij : i ∈ N1 , j ∈ N : 2}) is called bipartite. Graph theory tells
A graph G = (N1 ∪N
us
Theorem 1.8 A graph is bipartite if and only if it contains no cycle of odd length.

Proof: reading exercise. 

       
0 B x x x
For bipartite graphs A = . Thus if A =µ , then A =
  B> 0 y y −y
x
(−µ) .
−y
Theorem 1.9 A graph is bipartite iff the spectrum of its adjacency matrix is symmetric
with respect to 0.

Proof: For “⇒” see above.

“⇐”: Symmetry of the spectrum implies tr Ak = µki = 0 for k odd, so G has no odd walks
P
and therefore no odd cycles. 
If G is connected, it suffices to check the largest eigenvalue.
Theorem 1.10 A connected graph is bipartite iff µ1 (A) = −µn (A).

Proof: By the previous Theorme it is enough to prove “µ1 = −µn ⇒ bipartite”.

For µ1 = −µn A2 has µ21 = µ2n as maximal eigenvalue of multiplicity at least two. By Perron-
Frobenius A2 is reducible with some bipartition N1 ∪N ˙ 2 = N , N1 6= ∅ =
6 N2 . Therefore G has
no N1 -N2 walks of length two and hence no edge ij ∈ E with ij ⊆ N1 or ij ⊆ N2 . Indeed,
suppose w.l.o.g. ij ∈ N1 , pick some u ∈ N2 , let ({i = w0 , w1 }, w1 w2 , . . . , {wk−1 , wk = u})
be a shortest walk fomr i to u and let h be the smallest index with wh+1 ∈ N2 .
If h > 0, then (wh − 1wh , wh wh+1 ) is an N1 -N2 walk of length two,
If h = 0, then ({j, i = w0 }, w0 w1 ) is an N1 -N2 walk of length two. 
The Laplace Matrix or Laplacian of a graph G is the matrix LG = Diag(dG (·)) − A, where
Diag(dG (·)) denotes the diagonal matrix holding the degrees.

4
 P
Equivalently, using the canonical unit vectors ei (i-th column of IN ), LG = ij∈E Eij where
Eij = (ei − ej )(ei − ej )> (value 1 in ii, jj, value −1 in ij, ji and 0 otherwise). Both versions
also work for multigraphs.
For x ∈ Rn we have
X X
x> Lx = x> (ei − ej )(ei − ej )> x = (xi − xj )2 ≥ 0.
ij∈E ij

Thus, L is positive semidefinite and has a smallest eigenvalue λ1 (L) = 0 with eigenvector 1.
Recall the various characterizations of positive semidefinite matrices:

Theorem 1.11 (Characterizations of positive semidefinte matrices)

For A = A> the following are equivalent:
i) A is positive semidefinite (x> Ax ≥ 0 for all x ∈ Rn ).
ii) λi (A) ≥ 0 for all i.
iii) Gram representation: A = V > V for some V ∈ Rh×n (automatically rank V = rank A).

Proof: Exercise 
A tree is a connected graph that contains no cycle. A not nec. connected graph containing
no cycles is a forest, so a tree is a connected forest.
A subgraph (N 0 , E 0 ) of G = (N, E) is spanning if N 0 = N .
Exercise 1.12 Every connected graph contains a spanning tree.

n
o
Theorem 1.13 (Matrix-Tree Theorem)
Given a multigraph G = (N, E) and i, j ∈ N , the number of its spanning trees is
n
(i,j) 1Y
det LG = λj (LG ),
n j=2

where L(i,j) is obtained by replacing in L row i by e>

j and column j by ei ( i. e., det L
(i,j)
is
the (i, j)-cofactor of L).

Proof: We first prove for i ∈ N that det L(i,i) gives the number of spanning trees in G
by induction on n and for each n > 1 on the number of edges. For n = 1 the cofactor
is 1, so let n > 1. If i has degree d(i) = 0 then row i and column i are both zero in LG
(i,i)
and det LG = det LG−i = 0. So assume there is an edge ij ∈ E. Then by induction
(LG − ej eTj )(i,i) gives the number of spanning trees of G − ij and increasing element jj by
one increases this number by (L(i,i) )(j,j) which is by induction the number of spanning trees
of G/{i, j} where ij is contracted, i. e., where ij is part of the spanning tree.
For the Laplacian det L(i,i) = det L(i,j) for i, j ∈ N because in L any column is the negative
sum of the others.

5
For proving det L(i,i) = n1 nj=2 λj (L) note that in the polynomial det(tI −L) = t nj=2 (t−λj )
Q Q

the coefficient of t is (−1)n−1 nj=2 λj (L) = ddt det(tI − L)|t=0 . By the Leibniz-formula
Q
d
det(tI − L)|t=0 = i∈N (−1)n−1 det(L(i,i) ).
P
dt

Exercise 1.14 Prove that det(L + n1 11> ) also gives the number of spanning trees.

n
o
Corollary 1.15 The number of connnected components of a graph is equal to the multi-
plicity of the zero eigenvalue of its Laplacian.

2 Laplacian Eigenvalues and Cuts

For a graph G = (N, E) and some S ⊆ N , the edges running between S and N \ S form
the cut
E(S, N \ S) = {ij ∈ E : i ∈ S, j ∈ N \ S}.
If the edges have weights w ∈ RE , the weight of the cut is
X
w(S, N \ S) = we .
e∈E(S,N \S)

Classical problems related to (weighted) cuts are finding

a maximum cut max{w(S, N \ S) : S ⊆ N }
a minimum equipartition min{w(S, N \ S) : S ⊆ N, 2|S| = |N |} (|N | even)
\S)
a sparsest cut min{ w(S,N
|S|
: S ⊆ N, 2|S| ≤ |N |}
Given weights w ∈ RE
+ (sometimes even general ones), the weighted Laplacian is
X
Lw = wij (ei − ej )(ei − ej )> .
ij∈E

By Rayleigh-Ritz it gives rise to the following bounds for sparsest cut.

Observation 2.1
|N \ S| w(S, N \ S) |N \ S|
λ2 (Lw ) ≤ ≤ λmax (Lw )
|N | |S| |N |
(
1
− |S|
i∈S
Proof: The vector ui = 1 satisfies 1T u = 0, thus
i∈/S
|N \S|
X
λ2 kuk2 ≤ uT Lw u = wij (ui − uj )2 ≤ λmax kuk2
ij∈E X
1 1 1 1 2 1 1
λ2 ( |S| + |N \S|
) ≤ ( |S| + |N \S|
) wij ≤ λmax ( |S| + |N \S|
)
ij∈E(S,N \S)

6
Max-Cut
Partition G into two subgraphs connected by as many edges as possible.
For x ∈ {−1, 1}N , X X
x> L w x = wij (xi − xj )2 = wij 4,
ij∈E ij∈E,xi =−xj

Put S = {i ∈ N : xi = 1}, then w(S, N \ S) = 14 x> Lx

1 > 1 >
max 4
x Lx = min max 4
x (L − Diag(y))x
x∈{−1,1}N y∈RN x∈{−1,1}N
1 > y=0

n
≤ min λ (L
4 max
− Diag(y))
y∈RN
1> y=0
= min nλmax ( 14 L − Diag(y)) + 1> y
y∈RN
= min 1> y
y∈RN
1
L−Diag(y)0
4

This is a semidefinite program, use SDP duality

min 1> y max 41 L, X



s.t. Diag(y) + Z = 14 L s.t. diag(X) = 1

y ∈ RN , Z  0 X0