Norm and condition number of a matrix

If A.x = b then A(x+x) = b+b. Since we are dealing with a set of linear equations,
A. x = b.

If b is an error term then the corresponding error in x, x, is given by x = A-1.b.

The magnitude and direction of this error obviously depend on the inverse of A and
will be large (for some b) when A is nearly singular. Suppose that A is symmetric
with positive eigenvalues then it is possible to expand the inverse of A in terms of the
eigenvectors and eigenvalues of A as follows:

A.xi = i xi

A -1 c i x i x iT
i

ci are coefficients to be determined.

A.A -1 c i A.x i x Ti 1
i

c i i x i x iT 1
i

xx
i
i
T
i 1

The last equation holds provided the vectors xi are complete and orthonormal and so ci
i =1 for all i and ci =1/i.

1
A -1 x i x iT
i i
Now consider x = A-1.b. Rewrite this as
1
x x i x iT .b and we see that the error term, b, is projected onto xi with
i i
coefficient 1/i xiT.b. The largest contribution to x will come from eigenvectors
with the smallest eigenvalues and those for which b and xi are nearly parallel.
Nearly singular matrices will be most sensitive since they have the smallest
eigenvalues.

Relative error

The error x will depend on both the direction and magnitude of b. It makes more
sense therefore to compare the relative change ||b||/||b|| with the relative error ||x ||/||
x||. For a positive definite matrix (all eigenvalues positive) the solution vector and
error term always satisfy:

||x|| ||b||/max and ||x|| ||b||/min

Therefore the relative error is bounded by ||x||/||x|| ||b||/||b|| (max/min)
The ratio (max/min) is known as the condition number of A.
Norm of A
For a symmetric matrix the condition number can be expressed in terms of its
maximum and minimum eigenvalues. When A is not symmetric then we make use of
the norm of the matrix defined by:

||A|| = max (x0) ||A.x||/||x||.

This is a measure of the amplifying power of A and replaces ||x||/||x|| when A is not
symmetric. In this case the condition number is

c = ||A|| ||A-1||

To see why this is an appropriate definition for the condition number, replace A-1 by
its expansion in eigenvectors and eigenvalues above. The relative error satisfies

||x||/||x|| c ||b||/||b||

as before.

If we consider the effect of an error in the elements of A instead of the vector b then
we find, perhaps surprisingly, that the same definition of condition number satisfies

||x||/||x|| c ||A||/||A||

(A+A).(x+x) = b
A.x+A.x+A.x = 0
x = -A-1.A.(x+.x) = 0

Multiplying by A amplifies a vector by no more than the norm, ||A||, and

multiplying by A-1amplifies by no more than ||A-1||. Therefore

||x|| = ||A-1||.||A||.||(x+.x)||
||x||/||x+x || ||A-1||.||A|| = c ||A||/||A||

Round off error therefore comes from: natural sensitivity, c, and the actual errors in A
and b.

For the case of an eigenvalue problem the relevant condition number is that of the
eigenvector matrix, S, not A. For the eigenvalue problem A.S = S. with an error
A in A

(A+A).S = S.(+ )
A.S = S.
= S-1.A.S
|||| = ||S-1||.||A||.||S|| = c ||A||

NB When S is an orthogonal matrix (A real symmetric, unitary if A Hermitian) then c

= 1 since there is no amplification by an orthogonal matrix.
 IX Special Functions

Special functions such as the gamma function, Legendre polynomials or Bessel

functions appear in many aspects of computational physics. For example, the
incomplete gamma function arises in methods for rapidly and absolutely converging
lattice sums in crystal physics. They are available in many scientific computing
libraries such as the Gnu Scientific library (GSL) www.gnu.org/software/gsl/

Gamma function

See any text in mathematical physics, e.g. Mathematical methods for physicists,
Arfken and Weber.

Euler definition (Infinite limit)

lim 1.2.3. n
(z) n z z 0, - 1, - 2 , - 3
n z(z 1)(z 2) (z n)
lim 1.2.3. n
(z 1) n z 1 z - 1, - 2 , - 3
n (z 1)(z 2)(z 3) (z 1 n)
lim nz 1.2.3. n
n z z 0, - 1, - 2 , - 3
n z n 1 z(z 1)(z 2) (z n)

(z+1) = z (z)

(1) = 1.(0) = 1

(2) = 1.(1) = 1

(3) = 2.(2) = 2.1 = 2

(4) = 3.(3) = 3.2.1 = 6

(5) = 4.(4) = 4.3.2.1 = 24

(n+1) = n.(n) = (n-1)!

Euler definition (Definite Integral)

(z) =
0
e - t t z -1 dt Re(z) > 0

With the change of variable = t1/2 this becomes

(z) = 2 e - t t 2z -1 dt Re(z) > 0
2

0

(1/2) = 2 e - t dt 2 erf()
2

0

erf(x) is known as the error function.

Incomplete gamma function
The incomplete gamma function is defined by generalising the Euler definite integral
definition of the gamma function to
x
(z, x) = e - t t z -1 dt Re(z) > 0
0

(z, x) = x
e - t t z -1 dt Re(z) > 0
(z) (z, x) (z, x)
(z, x) (z, x)
1
(z) (z)

The utility of the incomplete gamma function lies in the fact that multiplying an
expression by ((z,x)+(z,x))/(z) leaves it unchanged and the two parts generated in
this way may be, rapidly convergent, e.g., when the original expression is not.
Functions such as the gamma functions are generally computed numerically using
either asymptotic series (which may converge rapidly for large or small arguments of
the function) and recursion relations.

By expanding the exponential in the incomplete gamma function and integrating term
by term we obtain
x
(z, x) = e - t t z -1 dt Re(z) > 0
0
z 1
xs
(z 1)!(1 e x )
s 0 s!

(z, x) = e - t t z -1 dt Re(z) > 0
x
z 1
xs
(z 1)!e x
s 0 s!

For the particular case z=1/2

x
(1/2, x) = 2 e - t dt Re(z) > 0
2

2 x 1

2
erf(x) e - t dt (1/2, x 2 )
0

2 1

2
erfc(x) e -t dt (1/2, x 2 )
x

erfc(x) is known as the complementary error function and for all arguments

erf(x) + erfc(x) = 1
 2

1.5

0.5

-4 -2 2 4
-0.5

-1

Plot of erf(x) (rising from -1 to 1), erfc(x) (falling from 2 to zero) and erf(x) +
erfc(x) (constant value of 1).

The series expansions for erf(x) and erfc(x) are

2 x3 x5 x 2n 1
x ... 1 ...
n
erf(x)
3 5.2! (2n 1)n!
2
e -x 1 1.3 1.3.5 n (2n - 1)!!
erf(x) 1 - 1 2
2 4 3 6 ... 1 n 2n
...
x 2x 2 x 2 x 2 x

The first expansion is obtained simply by expanding e -t^2 and integrating erf(x) term
by term. The second is obtained by writing

2 1 2 1 d 1 -t 2
e - t te - t - e
t t dt 2

and integrating by parts. The first expansion is good for small x and the second for
large x.

Recurrence relations for the incomplete gamma function

The incomplete gamma functions satisfy the following upwards and downwards
recursion relations

(z, x) (z 1)(z 1, x) e x x z 1
(z, x) (z 1) (z 1, x) e x x z 1
(z 1, x) e x x z
(z, x)
z
(z 1, x) e x x z
(z, x)
z
The first two are upwards and the last two downwards and the relations can be
obtained using integration by parts. The starting points for the downwards
recurrences are
 e x
for z 0
(1 z, x)
( 1) m x 1 z m
m!(1 z m) for 0 z 1
m 0

1 e x for z 0
(1 z, x)
(1 z, x) - (1 z, x) for 0 z 1

See paper by Guseinov and Mamedov for further details.

Use of gamma incomplete functions for calculating lattice sums

The Madelung constant is a familiar concept from crystal physics.

A real space lattice is defined by

r i a1 j a 2 k a 3 denotes ijk

and the corresponding reciprocal lattice is defined by

G h b1 k b 2 b 3 denotes hk

The Madelung constant is defined by

(1)
' i j k

where the primed sum indicates that the self term i = j = k = 0 is omitted. Write this
formally as an integral over all space. Define

'
w(r )
( 1) i j k (r r )
ik 1/2 .r 1

'
w(r ) e (r r ) k 1/2
(b 1 , b 2 , b 3 )

2
using a i .b j 2 ij (definition of reciprocal lattice)
1
(1,0,0) k 1/2 .r (b 1 .a1 ) e i - 1
2
w(r )dr w(r )erf( r) dr w(r )erfc( r) dr
r
r

r
 w(r )dr w(r )erf( r) dr w(r )erfc( r) dr
r
r

r

F(h)G (h)dh f(x)g

* *
Parseval' s Relation (x)dx
- -

choose f(r ) w(r ) F(h)

1

'

(h h k 1/2 ) - 1
e - h
2
erf( r)
g(r ) G(h)
r h 2
- h 2

w(r )erf( r) dr
r

1

(h h
'
k 1/2 ) - 1 e h 2
dh

2
- h k 1/2
1 e
'

2
h k 1/2
2

2
i j k - h k 1/2
(-1) 1 e
' '
erfc( r )
2
r h k 1/2
2

is the unit cell volume h is a vector in reciprocal space

 Sturm-Liouville theory and orthogonal functions

The Sturm-Liouville equation is a second order ordinary differential equation (ode) of

the form

d du
p(x) q(x)u(x) w(x)u(x)
dx dx

w(x) is a weight (or density) function. Many physical problems described by an

eigenvalue equation belong to the class of Sturm-Liouville equations. Some examples
are

equation p(x) q(x) w(x)

Legendre 1-x2 0 (+1) 1
Bessel x -n2/x a2 x
Hermite e-x2 0 2a e-x2
SHO 1 0 n2 1

We will look at Bessels equation illustrated by vibration of a drumhead and

Legendres equation illustrated by Laplaces equation in 3-D. In particular we will
look at recursive methods for calculating Bessel functions and Legendre polynomials.
For further details on the Sturm-Liouville equation and properties its solutions see e.g.
Arfken and Weber.

Bessels equation and the wave equation in 2-D

1 2
2 (r, , t) 0
c 2 t 2
1 1 2
(r, , t) R(r)( )T(t) 2 r 2
r r r r 2
1 1 1
R(r)( )T' ' (t) R' ' (r)( )T(t) R' (r)( )T(t) 2 R(r)' ' ( )T(t)
c2 r r
1 T' ' (t) R' ' (r) 1 R' (r) 1 ' ' ( )
2 -2
c2 T(t) R(r) r R(r) r ( )
T' ' (t) 2 c 2 T(t) 0
R' ' (r) R' (r) ' ' ( )
r2 r 2 r 2 - n 2
R(r) R(r) ( )
' ' ( ) n 2 ( ) 0
r 2 R' ' (r) r R' (r) 2 r 2 n 2 R(r) 0 or

r
d
r R' (r) 2 r 2 n 2 R(r) 0 Bessel' s equation
dr
R(r) A J n (r) B Yn (r)
Recurrence relations and zeroes of Bessel functions

Jn and Yn are Bessel functions of the first and second kinds, respectively. Jn is finite at
the origin (Jn(0) = 1) while Yn is divergent. For a vibrating drumhead, we impose the
boundary conditions, R(0) = finite, R(1) = 0 (assume drumhead has unit radius). Then
B = 0 and the solution to the ode is a linear combination of Bessel functions of the
first kind. The allowed solutions will have Bessel function zeroes at r = 1. Let the m th
zero of the nth Bessel function be denoted kmn.

The first three Bessel functions of the first kind are plotted below. The function
which takes the value one at the origin is J0[x]. J1(x) and J2(x) have value zero at the
origin and J1 is more tightly curved than J2.

Bessel Jn x
1

0.8

0.6

0.4

0.2

x
2.5 5 7.5 10 12.5 15
-0.2

-0.4

Zeroes of the first two Bessel functions of the first kind

m/n 0 1
1 2.4048 3.8318
2 5.5201 7.0516
3 8.6537 10.1735

Radial and angular nodes for modes tabulated above.

n=0 n=1
 Radial nodal positions for corresponding modes on drumhead

m/n 0 1
1 1.0000 1.0000
2 0.4356 0.5461
3 0.2778, 0.6378 0.3766, 0.6895

The generating function for Bessel functions of the first kind is

x 1
(h )
e2 h m J (x)h m
m
m

set h 1 and obtain

m
e0 1 J m (x)
m

From the property that J -m (x) (-1) m J m (x) for integral m

J 0 (x) 2m 1 J m (x) 1
m

Differentiating the generating function wrt h results in a recurrence relation

x 1 x 1
d 2 (h h ) x 1 2 (h h ) d m m
e 1 2 e J m (x)h m J m (x)mh m-1
dh 2 h dh m m

x 1 m m
1 2 J m (x)h m J m (x)mh m-1
2 h m m

Everything is an infinite sum so rearrange and collect coefficents of h m

2m m 1 m
J m (x)h m-1 1 2 J m (x)h m
x m h m

2m 1
J m (x)h m-1 1 2 J m (x)h m J m (x)h m J m (x)h m-2
x h
2m
J m (x) J m 1 (x) J m-1 (x)
x
2m
J m-1 (x) J m (x) J m 1 (x) Downwards recurrence relation
x
m
lim 1 ex
Initialise recursive process with J m (x) for m x
m 2m 2m
To obtain zeroes of Bessel functions use Newton - Raphson
J (x ) m d
x n 1 x n - 'm n J 'm (x n ) J m (x) - J m 1 (x) also from generating function
J m (x n ) x dx

2m
J m-1 (x) J m (x) J m 1 (x) Downwards recurrence relation
x
m
J 'm (x n ) J m (x) - J m 1 (x) For calculating zeroes from Newton - Raphson
x
Legendre polynomials and Laplaces equation in spherical polar coordinates

The Laplacian operator in spherical polar coordinates is

 1 2 1 1 2
r sin (r, , ) 0
r r
2
r 2 sin r 2 sin 2 2

For an azimuthally symmetric problem this reduces to
1 2 1
r 2 sin R(r)( ) 0
r r
2
r sin
2 R(r) ( )
( )r 2 rR(r) sin 0
r sin
2 1 ( )
r 2 rR(r) sin
r sin n(n 1)
R(r) ( )
2
r rR(r) - n(n 1)R(r) 0 soln. R(r) c1 r n c 2 r -(n 1)
r 2

1 ( )
sin n(n 1)( ) 0
sin
Let x cos sin 1 - x 2 equation in becomes
d d
(1 x 2 ) P n (x) n(n 1)P n (x) 0 in the standard Sturm - Liouville form
dx dx
Solutions are the Legendre polynomials given by Rodrigues formula
n
1 d
P n (x) n ( x 2 1) n
2 n! dx

Recurrence relations for Legendre polynomials

The generating function for Legendre polynomials is

(1 2xt t 2 ) -1/2 n 0 Pn (x)t n

n

A recurrence relation for Pn (x) may be obtained by differentiating wrt t

d (x t )
n 0 Pn (x)nt n -1
n
(1 2xt t 2 ) -1/2
dt (1 2xt t ) 2 3/2

(x t )
( x t ) n 0 Pn (x)t n (1 2xt t 2 )n 0 Pn (x)nt n -1
n n

(1 2xt t )2 1/2

x Pn (x)t n - Pn (x)t n 1 Pn (x)nt n -1 2xPn (x)nt n Pn (x)nt n 1

Collect coefficients of t n
x Pn (x) - Pn -1 (x) ( n 1)Pn 1 (x) - 2nxPn (x) ( n - 1)Pn -1 (x)
(2n 1)x n
Pn 1 (x) Pn (x) - P (x)
(n 1) n 1 n -1
A recurrence relation for Pn' (x) may be obtained by differentiating wrt x
d t
n 0 Pn' (x)t n
n
(1 2xt t 2 ) -1/2
dx (1 2xt t 2 ) 3/2
t
n 0 Pn (x)t n 1 (1 2xt t 2 )n 0 Pn' (x)t n
n n

(1 2xt t )
2 1/2

Pn (x)t n 1 Pn' (x)t n 2xPn' (x)t n 1 Pn' (x)t n 2

Collect coefficients of t n
Pn -1 (x) Pn' (x) - 2xPn' -1 (x) Pn' - 2 (x)
Pn' 1 (x) Pn' -1 (x) 2xPn' (x) Pn (x) (index has been shifted up by 1)

(2n 1)x n
Pn 1 (x) Pn (x) - P (x)
(n 1) n 1 n -1
Pn' 1 (x) Pn' -1 (x) 2xPn' (x) Pn (x)

The first two Legendre polynomials are P0(x) = 1 and P1(x) =x so these may be used
to calculate higher Legendre polynomials using the recurrence relation.

Example of a recursive function

double ftuvn(int t, int u, int v, int n, double *f, VECTOR_DOUBLE x)

double result ;

if (t < 0 || u < 0 || v < 0 ) return 0.0 ;

if (t >= u && t >= v) {

if (t == 0 && u == 0 && v == 0) return f[n] ;
result = (t > 0) ? x.comp1*ftuvn(t-1,u,v,n+1,f,x) + (t-1)*ftuvn(t-2,u,v,n+1,f,x) : 0.0 ;
}
else if (u > t && u >= v) {
result = (u > 0) ? x.comp2*ftuvn(t,u-1,v,n+1,f,x) + (u-1)*ftuvn(t,u-2,v,n+1,f,x) : 0.0 ;
}
else {
result = (v > 0) ? x.comp3*ftuvn(t,u,v-1,n+1,f,x) + (v-1)*ftuvn(t,u,v-2,n+1,f,x) : 0.0 ;
}
return result ;