The Inverse Power Method

Assume that A Cnn has a n linearly independent eigenvectors v1 , v2 , . . . , vn , and associated eigenvalues 1 , 2 , . . . , n , with |1 | > |2 | |i |, i = 3, 4, . . . , n. Then (1 , v1 ) is a dominant eigenpair of A, and for almost all x Cn , xk Ak x v1 as k .
n

How fast does such an iteration converge? Write x =

i=1

ci vi . Then
n

2 Ak x = c1 v1 + c2 ( )k v2 + 1 k 1

ci (
i=3

i k ) vi 1

(1)

and it is clear (yes?) that the error gets multiplied by about |2 /1 | at each step (we say that the convergence is linear with asymptotic error constant |2 /1 |). So the smaller the ratio |2 /1 |, the better, and if |1 | |2 | then we expect very slow convergence. Let B Cnn have eigenvalues i labeled so that |i | |i+1 |. The power method applied to B converges to the dominant eigenvector of B (if one exists) with a speed that depends on |2 /1 |. If B = A1 , then the dominant eigenpair of B is (1/n , vn ), and the power method applied to B converges to vn with a speed that depends on |n /n1 |. This iteration is fast if |n | is small compared to the other eigenvalues. The power method applied to A1 is called the inverse power method. We now have methods to compute the |largest| and |smallest| eigenvectors of a matrix A Cnn . How fast these methods go depends upon how relatively large or small the |largest| and |smallest| eigenvalues are. We can turn these ideas into a very powerful method with one more observation. That is, if we take B = (A sI)1 , then 1 = r s, where r is the eigenvalue of A that is closest to s. The dominant eigenpair of B is (1/(r s), vr ), and the power method applied to B converges to vr with a speed, , that depends on how close s is to r relative to the next closest eigenvalue of A: = |r s| . min |i s|
i=r

(2)

The power method applied to (A sI)1 is called the inverse power method with shift; it is a very powerful method, and is closely related to many state-of-the-art methods. The inverse power method should probably be called the inverse power kernel, for there are many decisions yet to be made about its implementation. For example, suppose we have an approximation s to r . Then we might use Gaussian Elimination with partial pivoting to compute the factorization P (A sI) = LU (which requires about 2n3 /3 ops if A doesnt have some special structure). This is done once, and then each iteration of the inverse power method requires only about 2n2 ops (a forward and backward substitution). Do we recompute the LU factorization as our approximation to r gets better? For the general eigenproblem, it is probably not ecient, but for a matrix with some special structure it may be fast. For symmetric matrices, a reduction to tridiagonal form, followed by an inverse power method with shift, using a dierent shift (a Rayleigh quotient) at each iteration, is a standard technique. In fact, all of our best methods for computing eigenstu are ultimately based on this kernel.