42

1.7 DOOLITTLES METHOD WITH ROW INTERCHANGES

We have seen that Doolittle factorization of a matrix A may fail the moment
at stage i we encounter a
ii
u which is zero. This occurrence corresponds to the
occurrence of zero pivot at the i
th
stage of simple Gaussian elimination method.
Just as we avoided this problem in the Gaussian elimination method by
introducing partial pivoting we can adopt this procedure in the modified Doolittles
procedure. The Doolittles method which is used to factorize A as LU is used
from the point of view of reducing the system
Ax = y
to two triangular systems
Lz = y
Ux = z
as already mentioned on page 19.
Thus instead of actually looking for a factorization A = LU we shall be
looking for a system,
A*x = y*
and for which A* has LU decomposition.
We illustrate this by the following example: The basic idea is at each stage
calculate all the u
ii
that one can get by the permutation of rows of the matrix and
choose that matrix which gives the maximum absolute value for u
ii
.
As an example consider the system
Ax = y
where
|
|
|
|
|
.
|

\
|

=
3 2 1 3
1 4 5 1
3 2 2 2
1 2 1 3
A
y =
|
|
|
|
|
.
|

\
|

1
3
8
3

We want LU decomposition for some matrix that is obtained from A by row
interchanges.
We keep 1
ii
l = for all i .
Stage 1:
1
st
diagonal of U. By Doolittle decomposition,
u
11
= a
11
= 3
If we interchange 2
nd
or 3
rd
or 4
th
rows with 1
st
row and then find the u
11
for the
new matrix we get respectively u
11
= 2 or 1 or 3. Thus interchange of rows does

43
not give any advantage at this stage as we have already got 3, without row
interchange, for u
11
.

So we keep the matrix as it is and calculate 1
st
row of U, by Doolittles method.
11 12 12 13 13 14
3; 1; 2; 1 u u a u a u = = = = = =
The first column of L:
. 1
3
3
;
3
1
;
3
2
; 1
11
41
41
11
31
31
11
21
21 11
= = = = = = = =
u
a
l
u
a
l
u
a
l l
Thus
L is of the form
32
42 43 44
1 0 0 0
2
1 0 0
3
1
1 0
3
1
l
l l l
| |
|
|
|
|
|
|
\ .
; and

U is of the form
22 23 24
33 34
44
3 1 2 1
0
0 0
0 0 0
u u u
u u
u
| |
|
|
|
|
\ .
; A and Y remaining unchanged.

Stage 2

We now calculate the second diagonal of U: By Doolittles method we have

12 21 22 22
u l a u =

( )
3
8
1
3
2
2 = |
.
|

\
|
=
Suppose we interchange 2
nd
row with 3
rd
row of A and calculate u
22
: our new a
22

is 5.
But note that
21
l and
31
l get interchanged. Therefore new
21
l is 1/3.
Suppose instead of above we interchange 2
nd
row with 4
th
row of A:
New a
22
= 1 and new l
21
= 1 and therefore new u
22
= 1 (1) (1) = 0

44
Of these 14/3 has largest absolute value. So we prefer this. Therefore we
interchange 2
nd
and 3
rd
row.

|
|
|
|
|
.
|

\
|

=
|
|
|
|
|
.
|

\
|

=
1
8
3
3
;
3 2 1 3
3 2 2 2
1 4 5 1
1 2 1 3
Newy NewA
1 0 0 0
3 1 2 1
1
1 0 0 14
0 * *
3
3
;
2
* 1 0 0 0 * *
3
0 0 0 *
1 * * 1
NewL NewU
| |
| |
|
|
|
|
= =
|
|
|
|
|
|
|
\ .
\ .


Now we do the Doolittle calculation for this new matrix to get 2
nd
row of U and 2
nd

column of L.

13 21 23 23
u l a u =

( ) ( )
3
10
2
3
1
4 = |
.
|

\
|
=

14 21 24 24
u l a u =

( ) ( )
3
2
1
3
1
1 = |
.
|

\
|
=

2
nd
column of L:
[ ]
22 12 31 32 32
u u l a l =

( ) ( )
3
14
1
3
2
2
(

|
.
|

\
|
=

7
4
=
[ ]
11 12 41 42 42
u u l a l =
( )( ) [ ]
3
14
1 1 3 =
= 0

45
Therefore new L has form
1 0 0 0
1
1 0 0
3
2 4
1 0
3 7
1 0 * 1
| |
|
|
|
|
|
|
\ .


New U has form
3 1 2 1
10 14 2
0
3 3 3
0 0 * *
0 0 0 *
| |
|

|
|
|
|
\ .


This completes the 2
nd
stage of our computation.

Note: We had three choices of u
22

to be calculated, namely 8/3, 14/3, 0 before
we chose 14/3. It appears that we are doing more work than Doolittle. But this is
not really so. For, observe, that the rejected u
22
namely 8/3 and 0 when divided
by the chosen u
22
namely 14/3 give the entries of L below the second diagonal.

3
rd
Stage:
3
rd
diagonal of U:
23 32 13 31 33 33
u l u l a u =

( )
|
.
|

\
|
|
.
|

\
|
|
.
|

\
|
=
3
10
7
4
2
3
2
2

7
10
=
Suppose we interchange 3
rd
row and 4
th
row of new A obtained in 2
nd
stage. We
get new a
33
= 2.
But in L also the second column gets 3
rd
and 4
th
row interchanges
Therefore new l
31
= 1 and new l
32
= 0
Therefore new u
33
= a
33
l
31
u
13
l
32
u
23

( )( ) ( )
|
.
|

\
|
+ =
3
10
0 2 1 2
= 4.

46
Of these two choices of u
33
we have 4 has the largest magnitude. So we
interchange 3
rd
and 4
th
rows of the matrix of 2
nd
stage to get

|
|
|
|
|
.
|

\
|

=
|
|
|
|
|
.
|

\
|

=
8
1
3
3
3 2 2 2
3 2 1 3
1 4 5 1
1 2 1 3
NewY NewA



1 0 0 0
3 1 2 1
1
1 0 0 10 14 2
0
3
3 3 3
;
1 0 1 0
0 0 4 *
2 4
* 1 0 0 0 *
3 7
NewL NewU
| |
| |
|
|
|
|
= =
|
|
|
|
|
|
|

\ .
\ .


Now for this set up we calculate the 3
rd
stage entries as in Doolittles method:
24 32 14 31 34 34
u l u l a u =
( )( ) ( ) 4
3
2
0 1 1 3 = |
.
|

\
|
=
( )
33 23 42 13 41 43 43
u u l u l a l =

( ) 4
3
10
7
4
2
3
2
2
(

|
.
|

\
|
|
.
|

\
|
|
.
|

\
|
=
= 5/14.
1 0 0 0
3 1 2 1
1
1 0 0 10 14 2
0
3
3 3 3
;
1 0 1 0
0 0 4 4
5 2 4
1 0 0 0 *
3 7 14
NewL NewU
| |
| |
|
|
|

|
= =
|
|
|
|
|
|
|

\ .
\ .


Note: The rejected u
33
divided by chosen u
33
gives l
43.



47
4th Stage
[ ]
34 43 24 42 14 41 44 44
u l u l u l a u =
( ) ( ) 4
14
5
3
2
7
4
1
3
2
3 |
.
|

\
|
|
.
|

\
|
|
.
|

\
|
|
.
|

\
|
=
= 13/7.

|
|
|
|
|
.
|

\
|

= =
|
|
|
|
|
.
|

\
|

= =
8
1
3
3
3 2 2 2
3 2 1 3
1 4 5 1
1 2 1 3
* *
Y NewY A NewA


New L = L* , New U = U*

* *
3 1 2 1 1 0 0 0
10 1 14 2
1 0 0 0
3 3 3 3
; ,
1 0 1 0 0 0 4 4
5 13 2 4
1 0 0 0
3 7 14 7
L U
| | | |
| |

| |
= =
| |
| |
| |
| |

\ . \ .


and A
*
= L
*
U
*


The given system Ax=y is equivalent to the system

A
*
x=y
*


and hence can be split into the triangular systems,

L
*
z = y
*


U
*
x = z

Now L
*
z = y
*
gives by forward substitution:

48

Z
1
=3

2 1 3 3
3
1
2 2 1
= = = + z z z

4 1 1
1 3 3 1
= = = + z z z z

8
14
5
7
4
3
2
4 3 2 1
= + + z z z z

( ) ( ) ( ) 8 4
14
5
2
7
4
3
3
2
4
= + |
.
|

\
|
+ |
.
|

\
|
|
.
|

\
|
z

7
5
2
4
= z

|
|
|
|
|
.
|

\
|

=
7
52
4
2
3
z

Therefore U*x = z gives by back-substitution;

7
52
7
13
4
= x therefore x
4
= -4.

3 1 1 4 4 4
4 3 4 3 4 3
= = = + = + x x x x x x


therefore x
3
= 3
2
3
2
3
10
3
14
4 3 2
= x x x


49
( ) ( ) 2 4
3
2
3
3
10
3
14
2
= |
.
|

\
|
|
.
|

\
|
x

2
2
= x

3 2 3
4 3 2 1
= + x x x x


1 3 4 6 2 3
1 1
= + + x x

Therefore the solution of the given system is

|
|
|
|
|
.
|

\
|

=
4
3
2
1
x

Some Remarks:

The factorization of a matrix A as the product of lower and upper triangular
matrices is by no means unique. In fact, the diagonal elements of one or the
other factor can be chosen arbitrarily; all the remaining elements of the upper
and lower triangular matrices may then be uniquely determined as in Doolittles
method; which is the case when we choose all the diagonal entries of L as 1.
The name of Crout is often associated with triangular decomposition methods,
and in Crouts method the diagonal elements of U are all chosen as unity. Apart
from this, there is little distinction, as regards procedure or accuracy, between the
two methods.

As already mentioned, Wilkinsons (see Page 25), suggestion is to get a LU
decomposition in which n i u l
ii ii
= 1 ; .

We finally look at the Cholesky decomposition for a symmetric matrix:

Let A be a symmetric matrix.

50
Let A = LU be its LU decomposition

Then A
T
= U
T
L
T
where the superscript T denotes the Transpose.
We have


U
T
is lower triangular and L
T
is upper triangular
Therefore U
T
L
T
is a decomposition of A
T
as product of lower and upper triangular
matrices. But A
T
= A since A is symmetric.
Therefore LU = U
T
L
T

We ask the question whether we can choose L as U
T
; so that
A = U
T
U (or same as LL
T
)
In that case, determining U automatically gets L = U
T

We now do the Doolittle method for this. Note that it is enough to determine the
rows of U.
Stage 1: 1
st
row of U:
2
11 1 1
1 1
n n
k k kl
k k
a l u u
= =
= =
since
1 1 k k
l u =
T
L U = Q

2
11
u =
since
1
0
k
u = for k>1 as U is upper triangular.
11 11
a u =


= =
= =
n
k
n
k
ki k ki k i
u u u l a
1 1
1 1 1

i
u u
1 11
=
since
1
0 for 1
k
u k = >


11 11
a u =
11 1 1
/ u a u
i i
= determines the first row of U.
and hence the first column of L.

Having determined the 1
st
i-1 rows of U; we determine the i
th
row of U as follows:

2
1 1
s i n c e
n n
i i i k k i k l i k k i
k k
a l u u l u
= =
= = =


51

2
1
s i n c e 0 f o r
i
k i k i
k
u u k i
=
= = >

1
2 2
1
i
ki ii
k
u u

=
= +


1
2 2
1
i
i i i i k i
k
u a u

=
=


1
2
1
i
i i i i k i
k
u a u

=
=
;
( Note that u
ki
are known for k i -1,because 1
st
i-1 rows have already been
obtained).


= =
= =
n
k
n
k
kj ki kj ik ij
u u u l a
1 1
Now we need u
ij
for j > i


=
=
i
k
kj ki
u u
1
Because u
ki
= 0 for k > i

=
+ =
1
1
i
k
ij ii kj ki
u u u u



Therefore


ii
i
k
kj ki ij ij
u u u a u
(

=
1
1




52
1
2
1
1
1
i
ii ii ki
k
i
ij ij ki kj ij
k
u a u
u a u u u

=
(

determines the i
th
row of U in terms of the previous rows. Thus we get U and L is
U
1
. This is called CHOLESKY decomposition.

Example:

Let
|
|
|
|
|
.
|

\
|

=
10 1 3 1
1 3 3 1
3 3 5 1
1 1 1 1
A


This is a symmetric matrix. Let us find the Cholesky decomposition.

1
st
row of U

= =
= =
= =
= =
1
1
1
1
11 14 14
11 13 13
11 12 12
11 11
u a u
u a u
u a u
a u



2
nd
row of U

( ) ( )( ) ( )
( ) ( )( ) ( )

= = =
= = =
= = =
2 2 1 1 3
1 2 1 1 3
2 1 5
22 14 12 24 24
22 13 12 23 23
12
2
22 22
u u u a u
u u u a u
u a u


3
rd
row of U


53
( ) ( )( ) ( )( ) ( )

= = =
= = =
2 1 2 1 1 1 1
1 1 1 3
33 24 23 14 13 34 34
23
2
13
2
33 33
u u u u u a u
u u a u



4
th
row of U

1 4 4 1 10 34
2
24
2
14
2
44 44
= = = u u u a u


|
|
|
|
|
.
|

\
|

=
1 0 0 0
2 1 0 0
2 1 2 0
1 1 1 1
U

|
|
|
|
|
.
|

\
|

= =
1 2 2 1
0 1 1 1
0 0 2 1
0 0 0 1
1
L U
and

A = LU
= LL
T

= U
T
U