1.

DOOLITTLES METHOD WITH ROW INTERCHANGES

We have seen that Doolittle factorization of a matrix A may fail the moment
at stage i we encounter a uii which is zero. This occurrence corresponds to the
occurrence of zero pivot at the ith stage of simple Gaussian elimination method.
Just as we avoided this problem in the Gaussian elimination method by
introducing partial pivoting we can adopt this procedure in the modified Doolittles
procedure. The Doolittles method which is used to factorize A as LU is used
from the point of view of reducing the system
Ax = y
to two triangular systems
Lz = y
Ux = z
as already mentioned on page 19.
Thus instead of actually looking for a factorization A = LU we shall be
looking for a system,
A*x = y*
and for which A* has LU decomposition.
We illustrate this by the following example: The basic idea is at each stage
calculate all the uii that one can get by the permutation of rows of the matrix and
choose that matrix which gives the maximum absolute value for uii.
As an example consider the system
Ax = y
where

3 1 2 1

3
2 2 2
A=
1 5 4 1

3 1

2
3

3

8
y=
3

1

We want LU decomposition for some matrix that is obtained from A by row

interchanges.
We keep lii = 1 for all i .
Stage 1:
1st diagonal of U. By Doolittle decomposition,
u11 = a11 = 3
If we interchange 2nd or 3rd or 4th rows with 1st row and then find the u11 for the
new matrix we get respectively u11 = 2 or 1 or 3. Thus interchange of rows does
42

not give any advantage at this stage as we have already got 3, without row
interchange, for u11.
So we keep the matrix as it is and calculate 1st row of U, by Doolittles method.
u11 = 3; u12 = a12 = 1; u13 = a13 = 2; u14 = 1
The first column of L:

l11 = 1; l 21 =

a
a 21 2
a
1
3
=
; l31 = 31 = ; l 41 = 41 = = 1.
3
u11
u11 3
u11 3

Thus

1
2
3
L is of the form
13

l 32

l 42

l43

3 1
0 u
22
U is of the form
0 0

0 0

0
; and
0

l 44

2 1
u23 u24
; A and Y remaining unchanged.
u33 u34

0 u44

Stage 2
We now calculate the second diagonal of U: By Doolittles method we have

u 22 = a22 l 21u12
8
2
= 2 (1) =
3
3

Suppose we interchange 2nd row with 3rd row of A and calculate u22 : our new a22
is 5.
But note that l21 and l31 get interchanged. Therefore new l21 is 1/3.
Suppose instead of above we interchange 2nd row with 4th row of A:
New a22 = 1 and

new l21 = 1 and

therefore new u22 = 1 (1) (1) = 0

43

Of these 14/3 has largest absolute value. So we prefer this. Therefore we

interchange 2nd and 3rd row.

1
NewA =
2

13
NewL =
23

1
5
2
1

2
4

2
2

1
3

1
3
; Newy =
8
3

1
3

0 0 0
1
3

1 0 0
0 14 3
;
NewU
=

* 1 0
0
0

0
0

* * 1

2 1

*
*

*
*
0
*

Now we do the Doolittle calculation for this new matrix to get 2nd row of U and 2nd
column of L.

u 23 = a 23 l 21u13
10
1
= ( 4 ) ( 2 ) =
3
3

u 24 = a 24 l 21u14
2
1
= ( 1) ( 1) =
3
3

2nd column of L:

l 32 = [a 32 l 31u12 ] u 22

2 14
= ( 2 ) (1 )
3
3

4
7

l42 = [a42 l41u12 ] u11

= [3 (1 )(1 )]

14
3

=0

44

Therefore new L has form

1
3

0 14 3
New U has form
0
0
0
0

1
1
3
2
3
1

2
10

*
0

1
*

2
3

*
*

This completes the 2nd stage of our computation.

Note: We had three choices of u22 to be calculated, namely 8/3, 14/3, 0 before
we chose 14/3. It appears that we are doing more work than Doolittle. But this is
not really so. For, observe, that the rejected u22 namely 8/3 and 0 when divided
by the chosen u22 namely 14/3 give the entries of L below the second diagonal.
3rd Stage:
3rd diagonal of U:

u 33 = a 33 l 31u13 l 32 u 23
2
4 10
= 2 ( 2)
7 3
3
=

10
7

Suppose we interchange 3rd row and 4th row of new A obtained in 2nd stage. We
get new a33 = 2.
But in L also the second column gets 3rd and 4th row interchanges
Therefore new l31 = 1 and new l32 = 0
Therefore new u33 = a33 l31 u13 l32 u23

10
= 2 (1)( 2) + (0 )
3
= 4.

45

Of these two choices of u33 we have 4 has the largest magnitude.

interchange 3rd and 4th rows of the matrix of 2nd stage to get

So we

3 1 2 1
3

1 5 4 1
3
NewA =
NewY =

3 1
2
3
1

2 2 2
8
3

13
NewL =
1
2
3

0
1
3

0
0 14 3
NewU
=
;

0
0
0

1
0
0

2
10

2
3

*
*

4
0

Now for this set up we calculate the 3rd stage entries as in Doolittles method:

u 34 = a34 l 31u14 l32 u 24

2
= 3 (1)( 1) (0 ) = 4
3

l 43 = (a 43 l 41u13 l 42 u 23 ) u 33
2
4 10
= 2 ( 2) 4
7 3
3
= 5/14.

13
NewL =
1
2
3

0
1
0
4

0
1
3

0
0
0 14 3
;
NewU
=

1
0
0
0

5
1
0
0
14

10
4
0

2
3

4
*

Note: The rejected u33 divided by chosen u33 gives l43.

46

4th Stage

u 44 = [a44 l 41u14 l 42 u 24 l 43u 34 ]

2
4 2 5
= 3 ( 1) (4)
3
7 3 14
= 13/7.

1
NewA = A * =
3

1
5

2
4

1
2

2
2

1
3

1
3
*
=
=
NewY
Y
1
3

8
3

New L = L* , New U = U*

13
*
L =
1
2
3

0
1
0
4

1
0
3

0
0
0 14 3
*
;U =
1
0
0
0
5
0
1
0
14

2
10

4
0

2
3
,
4
13
7

and A* = L*U*
The given system Ax=y is equivalent to the system
A*x=y*
and hence can be split into the triangular systems,
L * z = y*
U*x = z
Now L*z = y* gives by forward substitution:

47

Z1 =3

1
z1 + z 2 = 3 z 2 = 3 1 = 2
3
z1 + z 3 = 1 z 3 = 1 z 1 = 4
2
4
5
z1 z 2 +
z 3 + z 4 = 8
3
7
14
2
4
5
(3 ) (2 ) + ( 4 ) + z 4 = 8
3
7
14

52
z4 =
7
3

2
z = 4
52

7
Therefore U*x = z gives by back-substitution;
13
52
x4 =
7
7

therefore x4 = -4.

4x3 + 4x4 = 4 x3 + x4 = 1 x3 = 1 x4 = 3
therefore x3 = 3

14
10
2
x2
x3 x 4 = 2
3
3
3

48

14
10 2
x 2 (3) ( 4 ) = 2
3
3 3

x2 = 2

3 x1 + x 2 2 x3 x 4 = 3
3x1 + 2 6 + 4 3 x1 = 1
Therefore the solution of the given system is

2
x =
3

Some Remarks:
The factorization of a matrix A as the product of lower and upper triangular
matrices is by no means unique. In fact, the diagonal elements of one or the
other factor can be chosen arbitrarily; all the remaining elements of the upper
and lower triangular matrices may then be uniquely determined as in Doolittles
method; which is the case when we choose all the diagonal entries of L as 1.
The name of Crout is often associated with triangular decomposition methods,
and in Crouts method the diagonal elements of U are all chosen as unity. Apart
from this, there is little distinction, as regards procedure or accuracy, between the
two methods.
As already mentioned, Wilkinsons (see Page 25), suggestion is to get a LU
decomposition in which l ii = u ii ;1 i n .

We finally look at the Cholesky decomposition for a symmetric matrix:

Let A be a symmetric matrix.

49

Let A = LU be its LU decomposition

Then AT = UT LT where the superscript T denotes the Transpose.
We have
UT is lower triangular and LT is upper triangular
Therefore UTLT is a decomposition of AT as product of lower and upper triangular
matrices. But AT = A since A is symmetric.
Therefore LU = UTLT
We ask the question whether we can choose L as UT; so that
A = UTU

(or same as LLT)

In that case, determining U automatically gets L = UT

We now do the Doolittle method for this. Note that it is enough to determine the
rows of U.
Stage 1: 1st row of U:
n

a11 = l1k uk1 = ukl 2

k =1

since l1k = uk1

k =1

= u112 since
u 11 =

Q L = UT

u k 1 = 0 for k>1 as U is upper triangular.

a 11

k =1

k =1

a1i = l1k u ki = u k1u ki

= u 11 u 1 i since uk 1 = 0 for k > 1

u11 = a11

u1i = a1i / u11

determines the first row of U.

and hence the first column of L.

Having determined the 1st i-1 rows of U; we determine the ith row of U as follows:

a ii =

k =1

l ik u k i =

k =1

u k l 2 s i n c e l ik = u k i
50

k =1

i 1

u
k =1

2
ki

s in c e u

ki

= 0 fo r k > i

+ u ii 2

u ii 2 = a ii

u ii =

2
ki

a ii

i 1

k =1

u ki 2

i 1

k =1

u ki 2

( Note that uki are known for k i -1,because 1st i-1 rows have already been
obtained).

a ij =

ik

u kj =

k =1

ki

u kj

k =1

Now we need uij for j > i

= u ki u kj

Because uki = 0 for k > i

k =1

i 1

ki

u kj + u ii u ij

k =1

Therefore

u ij

= a ij

u
u

ki
kj u ii
k =1

i 1

51

i 1

2
u ii = a ii u ki

k =1

i 1

u = a
u
u
ki kj u ij
ij ij

=
1
k

determines the ith row of U in terms of the previous rows. Thus we get U and L is
U1. This is called CHOLESKY decomposition.
Example:

1
1 1 1

1 5 3 3
Let A =
1 3 3
1

1
3
1
10

This is a symmetric matrix. Let us find the Cholesky decomposition.

1st row of U

u11

u12

u13
u
14

a11 = 1

= a12 u11 = 1
= a13 u11 = 1
= a14 u11 = 1

2nd row of U

u = a u 212 = 5 1 = 2
22
22
u 23 = (a 23 u12 u13 ) u 22 = ( 3 ( 1)(1)) 2 = 1
u = (a u u ) u = (3 ( 1)(1)) 2 = 2
24
12 14
22
24
3rd row of U

52

u = a u 213 u 2 23 = 3 1 1 = 1
33
33

u 34 = (a34 u13u14 u 23u 24 ) u 33 = (1 (1)(1) ( 1)(2 )) 1 = 2

4th row of U

u 44 = a44 u 214 u 2 24 u 2 34 = 10 1 4 4 = 1
1 1 1

0 2 1
U =
0 0
1

0 0
0

2
2

0
1

1 2
U 1 = L =
1 1

1
2

0 0

0 0
and
1 0

2 1

A = LU
= LLT
= UTU

53