The Eigenvalue Problem

Homogeneous Systems
•  A homogeneous linear system

Ax=0
•  If det(A)≠0, the unique solution is the trivial
solution x = 0.
•  If det(A)=0, there exist nontrivial solutions.
 Example 1
Find the nontrivial solutions to the
homogeneous system

x1 + 2 x2 − x3 = 0
2 x1 + x2 + x3 = 0
5 x1 + 4 x2 + x3 = 0
 Solution 1
The system reduces to:
x1 + 2x2 − x3 = 0
x1 + x2 =0
2x1 + x2 + x3 = 0
− x 2 + x3 = 0
5x1 + 4x2 + x3 = 0
Letting x3=t, the solution can be expressed as
x1 = −t ⎧− 1⎫
x2 = t ⎪ ⎪
X = t ⎨ 1 ⎬
x3 = t ⎪ 1 ⎪
⎩ ⎭
 Applications
•  Quantum mechanics, geology, mathematics,
image processing, etc.

•  Mechanics
Principal stresses and the orientation of the
principal planes

•  Structural Dynamics
Natural frequencies and mode shapes of
structures
 The Eigenvalue Problem
Let A be a square matrix of dimension n x n, and
let x be a vector of dimension n. The problem is
to find scalars λ for which there exists a
nonzero vector x such that

Ax = λ x
Eigenvector Eigenvalue
(characteristic vector) (characteristic value)
 Solution Methods
•  Analytical Solution
–  Solution of the characteristic equation

•  Numerical Solution
–  Power method
–  Inverse power method
–  Jacobi’s method, QR method
 Analytical Method
•  The eigenvalues of A are the solutions of the
(
characteristic equation A − λI x = 0 )
a11 − λ a12 ! a1n
a21 a22 − λ ! a2 n
det ( A − λI ) = =0
" " # "
an1 an 2 ! ann − λ

•  When the determinant is expanded, it becomes

a polynomial of degree n, called the
characteristic polynomial.
 Eigenvalues
If A is an n x n real matrix, then its n eigenvalues
λ1, λ2, …, λn. are the real and complex roots of
the characteristic polynomial

p(λ ) = det ( A − λI )
Properties:
n
p(λ ) = (− 1) (λ − λ1 )(λ − λ2 )!(λ − λn )
n n n
trace( A) = ∑a = ∑λii k det ( A) = ∏λ k
i =1 k =1 k =1
 Eigenvector
If λ is an eigenvalue of A and the nonzero vector v
has the property that

Av = λ v
Then v is called the eigenvector of A
corresponding to the eigenvalue λ. v is obtained
by solving the homogeneous system (A - λ I) v =
0.
 Analytical Method
Hand computations used to solve the eigenvalue
problem when the dimension n is small.

1.  Find the coefficients of the characteristic

polynomial p(λ)= det(A - λ I)

2.  Find its roots (eigenvalues).

3.  Find the nonzero solutions (eigenvectors) of

the homogeneous linear system (A - λ I) v = 0.
 Linear Dependence
Definition 1. The vectors v1, v2, …, vr are said to
be linearly independent if

c1v1 + c2v2 + … + cnvn = 0

implies that c1=0, c2=0, …, cn=0. The vectors

are linearly dependent if there exists a set of
numbers {c1, c2, …, cn} not all zero, such that the
equation above holds.
 Theorems
Theorem 1. For each distinct eigenvalue λ there
exists at least one eigenvector v corresponding
to λ. If λ has multiplicity r, then there exists at
most r linearly independent eigenvectors v1, v2,
…, vr that correspond to λ.

Theorem 2. Suppose that A is a square matrix and

λ1, λ2, …, λk are distinct eigenvalues of A, with
associated eigenvectors v1, v2, …, vk,
respectively; then {v1, v2, …, vk} is a set of
linearly independent vectors.
 Example 2
Find the eigenpairs λj, vj for the matrix

⎡ 3 − 1 0 ⎤
⎢
A = ⎢− 1 2 − 1⎥⎥
⎢⎣ 0 − 1 3 ⎥⎦
 Solution 2
Eigenvalues. The characteristic equation det(A - λ I)=0 is

3−λ −1 0
3 2
−1 2−λ − 1 = −λ + 8λ − 19λ + 12 = 0
0 −1 3 − λ − (λ − 1)(λ − 3)(λ − 4) = 0

Therefore, the three eigenvalues are

λ1 = 1, λ2 = 3, and λ3 = 4.
 Solution 2
Eigenvectors.
For λ1 = 1,
⎡ 2 − 1 0 ⎤ ⎧ x1 ⎫ ⎧0⎫ x2 = 2a ⎧1 ⎫
⎢− 12 x11− x2− 1⎥ ⎪ x ⎪==0⎪0⎪ ⎪ ⎪
⎢ ⎥ ⎨ 2 ⎬ ⎨ ⎬ x1 = a v1 = a ⎨2⎬
− x + 2x = 0
⎢⎣ 0 − 1 22 ⎥⎦ ⎪⎩ x33⎪⎭ ⎪⎩0⎪⎭ x3 = a ⎪1 ⎪
⎩ ⎭
For λ2 = 3,
⎡ 0 − 1 0 ⎤ ⎧ x1 ⎫ ⎧0⎫ x2 = 0 ⎧ 1 ⎫
⎢− 1 x1 +⎥x⎪3 =⎪0 ⎪ ⎪ ⎪ ⎪
⎢ − 1 − 1⎥ ⎨ x2 ⎬ = ⎨0⎬ x1 = b v 2 = b ⎨ 0 ⎬
x2 ⎪ = ⎪0 ⎪ ⎪ ⎪− 1⎪
⎢⎣ 0 − 1 0 ⎦ ⎩ x3 ⎭ ⎩0⎭
⎥ x3 = − b ⎩ ⎭
 Solution 2
Eigenvectors.
For λ3 = 4,
⎡− 1 − 1 0 ⎤ ⎧ x1 ⎫ ⎧0⎫ x3 = c ⎧ 1 ⎫
⎢− 1 x1 + x2 ⎥ ⎪ = ⎪0 ⎪ ⎪ ⎪ ⎪
⎢ − 2 − 1⎥ ⎨ x2 ⎬ = ⎨0⎬ x2 = −c v 2 = c ⎨− 1⎬
x2 + ⎪x3 =⎪0 ⎪ ⎪ ⎪ 1 ⎪
⎢⎣ 0 − 1 − 1⎥⎦ ⎩ x3 ⎭ ⎩0⎭ x1 = c ⎩ ⎭
 Numerical Methods

1.  Power Method

2.  Shifted-inverse Power Method
 Power Method
Assume that the n x n matrix A has n distinct
eigenvalues λ1, λ2, …, λn and that they are
ordered in decreasing magnitude; that is,

|λ1| > |λ2| ≥ |λ3| ≥ … ≥ |λn|

eigenvector v1 corresponding to λ1 is called a

dominant eigenvector.
 Power Method
If x0 is chosen appropriately, then the sequences
{xk = [x1(k) x2(k) … xn(k) ]’} and {ck} generated
recursively by
yk = A xk
and
xk+1 = (1/ck+1) yk,
where
ck+1 = max{|xi(k)|
will converge to the dominant eigenvector v1 and
eigenvalue λ1, respectively.
 Example 3
Use the power method to find the dominant
eigenpair for the matrix

⎡ 0 11 − 5 ⎤
⎢
A = ⎢− 2 17 − 7 ⎥⎥
⎢⎣− 4 26 − 10⎥⎦
 Solution 3
Start with x0 = {1 1 1}T and use to generate the
sequence of vectors {xk} and constants {ck}.
The first iteration produces

⎡ 0 11 − 5 ⎤ ⎧1⎫ ⎧ 6 ⎫ ⎧ 12 ⎫
⎢− 2 17 − 7 ⎥ ⎪1⎪ = ⎪ 8 ⎪ = 12⎪ 2 ⎪ = c x
⎢ ⎥ ⎨ ⎬ ⎨ ⎬ ⎨ 3 ⎬ 1 1
⎢⎣− 4 26 − 10⎥⎦ ⎪⎩1⎪⎭ ⎪⎩12⎪⎭ ⎪1 ⎪
⎩ ⎭
 Solution 3
The second iteration produces
⎡ 0 11 − 5 ⎤ ⎧ 12 ⎫ ⎧ 73 ⎫ ⎧167 ⎫
⎢− 2 17 − 7 ⎥ ⎪ 2 ⎪ = ⎪10 ⎪ = 16 ⎪ 5 ⎪ = c x
⎢ ⎥ ⎨ 3 ⎬ ⎨ 3 ⎬ 3 ⎨ 8 ⎬ 2 2
⎢⎣− 4 26 − 10⎥⎦ ⎪⎩1 ⎪⎭ ⎪⎩163 ⎪⎭ ⎪ 1 ⎪
⎩ ⎭
Iteration generates the sequence
⎧ 12 ⎫ ⎧167 ⎫ ⎧125 ⎫ 31
⎧ 76 ⎫ 21
⎧ 52 ⎫
⎪ 2 ⎪ 16 ⎪ 5 ⎪ 9 ⎪ 11 ⎪ 38 ⎪ 23 ⎪ 78 ⎪ 47 ⎪
12⎨ 3 ⎬, ⎨ 8 ⎬, ⎨18 ⎬, ⎨ 38 ⎬, ⎨ 78 ⎬,!
⎪1 ⎪ 3 ⎪ 1 ⎪ 2 ⎪ ⎪ 9 ⎪ ⎪ 19 ⎪ ⎪
⎩ ⎭ ⎩ ⎭ ⎩ 1 ⎭ ⎩ 1 ⎭ ⎩ 1 ⎭
The sequence converges to v = {2/5 3/5 1}T, λ=4.
 C-Implementation
Vector PowerMethod(
Matrix A, /* nxn matrix A */
Vector x0, /* nx1 starting vector */
double e, /* tolerance */
int N, /* maximum iterations */
double * ev /* eigenvalue */
);

Returns: The dominant eigenvector

Shifted-inverse Power Method
Shifting Eigenvalues. Suppose that λ, v is an
eigenpair of A. If α is any constant, then λ - α, v
is an eigenpair of the matrix A - αI.

Inverse Eigenvalues. Suppose that λ, v is an

eigenpair of A. If λ≠0, then 1/λ, v is an
eigenpair of the matrix A-1.

Suppose that λ, v is an eigenpair of A. If α≠λ, then

1/(λ - α), v is an eigenpair of the matrix (A-αI)-1.
Shifted-inverse Power Method
A constant α can be chosen so that µ1 = 1/(λi - α)
is the dominant eigenvalue of (A - αI)-1.
Furthermore, if x0 is chosen appropriately, then
the sequences {xk = [x1(k) x2(k) … xn(k) ]’} and {ck}
generated recursively by
yk = (A - αI)-1 xk
and
xk+1 = (1/ck+1) yk,
where
ck+1 = max{|xi(k)|}
Shifted-inverse Power Method
will converge to the dominant eigenpair of the
matrix (A - αI)-1. Finally, the corresponding
eigenvalue for the matrix A is given by the
calculation
1
λj = +α
µ1
 Example 4
Employ the shifted-inverse power method to find the
eigenpairs of the matrix

⎡ 0 11 − 5 ⎤
A = ⎢⎢− 2 17 − 7 ⎥⎥
⎢⎣− 4 26 − 10⎥⎦

Note: The eigenvalues of A are λ1 = 4,λ2 = 2 and λ3 = 1,

and select and appropriate α and starting vector for
each case
 Solution 4
Case (1): for the eigenvalue λ1 = 4, we select
α = 4.2 and the starting vector x0 = {1 1 1}T. First
form the matrix (A – 4.2I)-1, iteration produces

⎡−4.2 11 −5 ⎤ ⎧1⎫
⎢ − 2 12.8 −7 ⎥ y 0 = ⎨1⎬ = x 0
⎣ − 4 26 −14.2 ⎦ ⎩1⎭
⎧−9.545454545 ⎫ ⎧0.4117647059⎫
y0 = ⎨ −14.09090909 ⎬ = −23.18181818 ⎨0.6078431373⎬ = c1 x1
⎩− 23.18181818⎭ ⎩ 1 ⎭
Solution 4
 Solution 4
the sequence converges to µ1 = -5 which is
the dominant eigenvalue of (A – 4.2I)-1 ,
and xk converges to v1 = [2/5, 3/5,1]T.

the eigenvalue λ1 of A is given by

λ1 = (1/ µ1 ) + α = 1/(-5) + 4.2 = -0.2+4.2 = 4
 Solution 4
Case (2): for the eigenvalue λ2 = 2, we select α =
2.1 and the starting vector x0 = {1 1 1}T then form
the matrix (A – 2.1I)-1.

the sequence converges to µ1 = -10 and the

eigenpair of the matrix is

λ2 = (1/ µ2 ) + α = (1/-10) + 2.1 = -0.1+2.1 =2

v1 = [1/4, 1/2,1]T.
Solution 4
 Solution 4
Case (3): for the eigenvalue λ3 = 1, we select α =
0.875 and the starting vector x0 = {0 1 1}T then
form the matrix (A – 0.875I)-1.

the sequence converges to µ1 = 8 and the

eigenpair of the matrix is

λ2 = (1/ µ1 ) + α = (1/8) + 0.875 = 0.125+0.875 = 1

v3 = [1/2, 1/2,1]T.
Solution 4