PY Decomposition: Factoring E œ P † Y

Motivation

To solve EB œ ,, we can row reduce the augmented matrix

Ò +" +# ÞÞÞ +3 ÞÞÞ +8 l , Ó µ ÞÞÞ µ Ò -" -# ÞÞÞ -3 ÞÞÞ -8 | . Ó

until Ò -" -# ÞÞÞ -3 ÞÞÞ -8 Ó is in an echelon form (the “forward” part of the row reduction process)
and then use “back substitution” to complete the solution. A computer might proceed this way
(see the first paragraph about “back substitution” in Sec. 1.2 (p. 19) of the textbook.)

Or, we could continue the row reduction (the “backward” part of the row reduction process)
until we arrive at the row reduced echelon form, from which we can easily read off the solutions.

Either way, in some applications it is necessary to solve the system EB œ , many times (perhaps
hundreds or thousands of times), always keeping E the same but changing ,Þ
Moreover, in these applications, E may also be a very large matrix. In such a situation, row
reducing Ò +" +# ÞÞÞ +3 ÞÞÞ +8 l , Ó each time is too inefficient, even with a computer doing the
work. (See the airplane design example discussed in the introduction for Chapter 2.)

One way to avoid these repeated row reductions is to try to factor E as

E œ P † Y where 
Y is an echelon form of E, and
P is a square lower triangular matrix with "'s on the diagonal

(Y might not be square, but it has only ! entries below leading entries in each row, so Y
resembles an upper triangular matrix. )

A factorization E œ PY cannot always be found but, if it can, the work involved does not take
many more steps than it takes to solve EB œ , just once by row reduction. And if we get this
factorization E œ PY , then it takes relatively few additional steps to find B each time a new , is
chosen. More specifically:

We start with EB œ ,Þ so if E œ PY . then EB œ PY B

Then make a substitution: Y B œ Cß so that PC œ ,

So we have replaced one matrix equation EB œ , with two new ones.


PC œ ,
(**)
C œ YB

This is an improvement because:

i) solving PC œ , for C is simple since P is lower triangular; and

ii) after finding C ß it is easy to solve Y B œ C because Y is in echelon
form.
iii) Finally, if we change , , P and Y remain the same.
 " " #

! #
Example 1 Find an PY decomposition of E œ # " ! and use it to solve the matrix
%
" " #  #
!   "
equation # " ! Bœ " Þ
% #

First, we reduce E to an echelon form. (We need to be careful about how we do this: see
comments later, but for now, just follow along.) Each elementary row operation on E
corresponds to left multiplication of E by an elementary matrix, and we record those
matrices for each step.

" # "
" " #  "  " # 

! # !   ## 
Eœ # "
! µ ! $ % µ ! $  % œ Y (an echelon form)
% % # ! ! $
Å Å
 " ! !  " ! !

 ! ! " !  % "
I" œ # " ! I# œ ! " !
$

If we use the elementary matrices, the row reduction looks like

Y œ I# I" Eß so

ÐI# I" Ñ" Y œ I"" I#" Y œ E

Æ Æ Æ
 " ! !  "  " #  "  " #

 ! % "  ! ##  ! #
œ # " ! ! $  % œ # " !
$
! $
%
Å
I"" I#" is lower triangular with "'s on the diagonal, so we let P œ I"" I#" and we have the
factorization we wanted: E œ PY Þ

Note: I"" I#" turned out (to be lower triangular because of the particular row
operations that we used: see later comments.

"  " #  #
!   "
To solve # " ! Bœ " .
% #
|___________|
Å Å
E œ PY ,

Let Y B œ C and substitute. We get two equations

 


 " ! !  #


 PC ! % "  "
œ # " ! Cœ "

 C œ YB 
PC œ ,

  #
$


(**) that is,

"  "

C œ YB
 ! ## 
œ ! $ % B
! $

First solve the top equation for CÞ To do this, we can row reduce ÒP , Ó to row reduced echelon
form, or we can use “forward substitution”  just like back substitution but starting with the top
equation. Either way we need only a few steps to get C because P is lower triangular. We
illustrate using forward substitution:
 C"   #

 C$   $
C" œ #à C# œ "  #C" œ  $à and C$ œ  "  %$ C# œ 3: so C œ C# œ  $ Þ

"  " #  #
! ##   $
Now the second equation becomes ! $  % B œ  $ which, again, we can quickly
! $

solve with back substitution or further row reduction. We will use back substitution:
$ * "& &
B$ œ $Ð ## Ñœ ## ; $B# œ  $  %ÐB$ Ñ œ  "" , so B# œ  "" ;

 B"   
and B" œ #  B#  #ÐB$ Ñ œ "" : so B œ B# œ   
)
""

 B$  
) &


""
*
##

Example 1, continued: same coefficient matrix E but different ,

 "
!
Now that we have factored E œ PY we can solve EB œ # easily:

See above: here only , has changed !



Æ

    "

" ! !

 ! % " !
# " ! C œ #


  # 
$



" "

 Cœ
  
! $  % B
##
! ! $

"
!
By forward substitution C" œ "; C# œ #  2Ð"Ñ œ 0; C$ œ  %$ Ð0Ñ œ 0: so C œ ! Þ
 " " #  "
! ##  !
The second equation becomes ! $  % B œ ! and back substitution gives
! $

"
!
B$ œ !; $B# œ !  %Ð!Ñ so B# œ !; and B" œ "  "Ð!Ñ  #Ð!Ñ œ ": so B œ ! Þ

Comments and Explanations based on Example 1

1) As mentioned above, we could have solved PC œ , and C œ Y B by further row

reduction instead of forward/backward substitution. It takes relatively few steps, either
way, because of the form of P and Y Þ See the comments about efficiency in Example 1,
pp, 124-125 of the textbook.

2) In Example 1, E was a square matrix, but that's not necessary. If there is a

factorization E œ PY of the kind we described, then

i) P (by definition) will always be a square lower triangular matrix, with "'s on
its diagonal. If E is 7 ‚ 8, then P will be 7 ‚ 7Þ P is also invertible because
we get it as a product of invertible(elementary) matrices.

ii) Y will be 7 ‚ 8 because Y is just an echelon form of E.

And because Y is in echelon form, it has only !'s below leading entries in each
row  so Y resembles an upper triangular matrix.

3) Does the method in Example 1 always work? CAUTION: You can always perform
the steps illustrated in Example 1 and get a factorization E œ ^ * Y , where Y is an
echelon form and where ^ is a product of elementary matrices Ðin Example 1, ^ œ
I"" I#" Ñ. But ^ might not be a lower triangular matrix (so it might not deserve the
name “P”).

In Example 1,

i) I"" I#" turned out to be a lower triangular matrix because the I3 's, the
inverses I3" were all lower triangular (convince yourself that inverses and products of
lower triangular matrices are always lower triangular). And the I3 's were lower
triangular because the only kind of ERO we used was “add a multiple of row 3 to a lower
row 4” (used to create !'s in rows below the pivot)Þ Such an ERO corresponds to an
elementary matrix that is lower triangular.
When you're doing a row reduction following the standard steps, you never
add a multiple of a row to a higher row anyway. So only adding multiples of rows to
lower rows is “standard procedure.”

ii) I"" I#" turned out to have only "'s on the diagonal because the I3 's had only
"'s on the diagonal. And that was true because no row rescalings were used during the
reduction of E.
 The method in Example 1 to get a factorization E œ PY will always work if it is
possible to row reduce E to an echelon form using only one kind of ERO: “add a
multiple of some row 3 to a lower row 4 in the matrix.” ÐIn particular, no row
interchanges and no row rescaling allowed.) This is not always possible: for example,
you cannot get E œ 
2 !
! 2
into echelon form without a row interchange, and the

elementary matrix I3 œ 
!
! "
matrix that swaps rows is not lower triangular. So the
"
product that gives ^ may not produce a lower triangular matrix.

Side comment: What if we also allowed row rescaling operations? For row
rescaling, the corresponding elementary matrices I3 and I3" would be lower
triangular and P œ I"" † ÞÞÞ † I:" would be lower triangular. But P would no
longer have only "'s on its diagonal. That's perhaps not very important as a
practical computational matter, but the textbook requires "'s along the diagonal
38 P as part of the definition of PY decomposition, so we follow that
convention  and therefore allow no row rescaling operations while row
reducing E.

In any case, “no rescaling rows” is at most an inconvenience: although rescaling

can be convenient, rescaling is never necessary to reduce E to echelon form; the
restriction never prevents us from reducing E to echelon form. (Row rescaling
does become necessary to go from echelon to row reduced echelon form: we
need it then to convert all leading entries into "'s.)

Example 2 Suppose I" ß ÞÞÞÞß I: are elementary matrices such that I: † ÞÞÞ † I" E œ Y is in
echelon form, and that each I3 corresponds to a row operation of type “add a multiple of a row to
a lower row.” Then (just as in Example 1)

E œ ÐI: † ÞÞÞ † I" Ñ" Y and

P œ ÐI: † ÞÞÞ † I" Ñ" œ I"" † ÞÞÞ † I:" is lower triangular with "'s on its
diagonal.

If E is large, there are likely to be many such matrices I3 and finding P with this formula is too
much work. Fortunately, as E is row reduced, we can write down P automatically as we go. .
Example 2 illustrates the technique. (The text offers a slightly different presentation, and an
explanation of why this works.)

If E is 7 ‚ 8, then P will be 7 ‚ 7Þ Create P using these steps:

i) put "'s on the diagonal of P and !'s everywhere above the diagonal
ii) whenever a row operation “add - times row 3 to a lower row 4” is used, use  -
as the P43 entry in P Ðnote: L43 , not P34 Ñ
iii) when E has been reduced to echelon form, fill in any remaining empty entries in P
with !'s.
In Example "ß E was $ ‚ $Þ The steps outlined to construct P work out like this:

" ! !

* "
i) Start with P œ * " !
*

ii) The row operations used in Example 1 were:

add  #Ðrow 1Ñ to row #, so P#" œ #

add  %$ Ðrow #Ñ to row 3, so P$# œ %$

" ! !

* "
and these contribute two more entries in P, so we have P œ # " !
%
$

iii) Fill in the rest of P with !'sß so

" ! !

! "
Pœ # " ! ß just as in Example 1.
%
$

Example 3 There's not really anything new in Example 3: just another example that's a little
more complicated.

" " ! # " $

# !
If possible, find an PY factorization for E œ " # " ! # " and use it to solve
" ! " !
 "
 $ 
EB œ , œ #

It depends on whether we can reduce E to echelon form only by adding multiples of rows to
lower rows. (It turns out that we can; follow the steps)

" " ! # " $

# !
Eœ " # " ! # "
" ! " !

" " ! # " $

# !
µ ! $ " # " # Ðadd  "Ðrow ") to row #Ñ
" ! " !

" " ! # " $

! '
µ ! $ " # " # Ðadd  #Ðrow ") to row 3Ñ
$ ! $ #
 " " ! # " $

! %
µ ! $ " # " # Ðadd  "Ðrow #) to row 3Ñ
! " " $

œ Y (an echelon form)

" ! ! " ! !

* " # "
The steps above give P œ * " ! Ä ... Ä " " !
* "

Check:

" " ! # " $ " ! !  " " ! # " $

# ! # "  ! %
Eœ " # " ! # " œ PY œ " " ! ! $ " # " #
" ! " ! " ! " " $
  "
 $
We can solve EB œ , œ # by writing




 " !  C"   "


!


 #    $
PC œ ,, that is " " ! C# œ , œ #




" " C$
 B" 
 B 

 " $  #   C" 

 B 
  #  $  œ C#
" ! # "

 ! B 
  %  %   C$ 
C œ Y Bß that is ! $ " # "




! " " $
  B' 
B&

From the first equation : C" œ  "ß C# œ #  Ð  "Ñ œ $ß C$ œ $  #Ð  "Ñ  Ð$Ñ œ #

 C"    " 
 C$   #
so C œ C# œ $ Þ The second equation becomes

 B" 
B 
" $  #    " 
B 
" #  $œ
" ! # "

!  B%  
$ %   #
! $ " # $ , so
! " "
 B' 
B&

B$ œ #  B%  $B&  %B'

B# œ "$ Ð$  B$  #B%  B&  #B' Ñ œ "$ Ð$  Ð#  B%  $B&  %B' Ñ  #B%  B&  #B' Ñ

œ "$ Ð&  $B%  #B&  'B' Ñ œ &

$  B%  #$ B&  #B'

B" œ  "  B#  #B%  B&  $B' œ  "  Ð &$  B%  #$ B&  #B' Ñ  #B%  B&  $B'

#
œ $  B%  "$ B&  B'

 B"   $  B%  $ B&  B'   $   "   "$   "

 B#    B%  #$ B&  #B'  &  # 
# " #

     "   $ 
 # 
           
B  #  B%  $B&  %B'  œ  #   B  "  $  B  % 
&

so B œ  $  œ  %   B&   ß
$ $

 B%  
 !
    "  
 !    ! 
          
" 
'
B %

 B'    !  !   !   " 

B& B& ! ! !
B'

where B% ß B& ß B' are free.

What if E cannot be reduced Y without row interchanges? You can get a factorization that's very
similar to “PY ” and just as useful.

1) Look at the steps required to row reduce E to echelon form Y and see what row interchanges
were needed. Note what rows will need to be interchanged in the process. Perform those
interchanges on E first.

Suppose the row reduction will require 5 row interchanges represented by an elementary
matrix I" ß ÞÞÞß I5 Þ Performing all these on E gives I5 † ÞÞÞ † I" † E œ T EÞ
T œ I5 † ÞÞÞ † I" is just the result of 5 row interchanges performed on M , so T looks like
some version of M with its rows scrambled. We could describe T as some square matrix with
exactly one " in each row and exactly one " in each column. Such a T is called a permutation
matrix because T E permutes (“scrambles”) the rows of E in just the same way.
Of course every permutation matrix is invertible, because it's a product of elementary
matrices: T " T E œ Eß that is T " “unscrambles” the rows of T E and get back to the original E.
And it's just as easy to write down T " and it is to write down T .

2) Now, T E can be row reduced to echelon form without any row interchanges (and we never
really need row rescaling, as mentioned earlier). So use the earlier process to find P and Y so
that

T E œ PY

If the original goal was to solve EB œ ,, then use Pß Y to solve T EB œ T , instead.

If B makes T EB œ T , is true, then multiplying both sides by T " shows that EB œ , also
true. ÐIf you thing in terms of linear systems of equations: T rearranges the left side of the
system corresponding to the coefficient matrix E. To solve, you need to rearrange the order of
the constants on the right in the same way.Ñ

3) Note that if you get T E œ PY ß then E œ ÐT " PÑ Y Þ So E is factored into ^Y

where Y is an echelon form of E and ^ œ T " P is a lower triangular matrix P with its rows
permuted. (MATLAB gives T " P the cute name “psychologically lower diagonal matrix” a term
never seen anywhere else to my knowledge).

For those who use MATLAB:

If you define a square matrix E, and then enter the MATLAB command

ÒPß Y ß T Ó œ lu (A) ( “ lu” is lowercase in MATLAB )

MATLAB returns to you

1) a square lower triangular P with 1's on the diagonal,

2) a square Y in echelon form, and
3) a permutation matrix T

for which T E œ PY
 If its possible to factor E œ PY ß then MATLAB just gives T œ M œ the identity matrix

ÐOur remarks, above, didn't require a square E; but that's all MATLAB is set up for.Ñ

" #
" "
"
Example Suppose E œ  Þ
"

! "
" % %
"

1) On scratch paper, do enough steps in row reducing E to echelon form (no row
rescaling allowed, and, following standard procedure, only adding multiples of rows to
lower rows)) to see what (if any) row interchanges are necessary. It turned out (the way I
did it) that just one, interchanging rows 2 and 4 at a certain stage, was necessary.

2) So go back to the start and perform that row interchange first, creating

" # " !
! " ! "
" ! !
TE œ  , where T œ  
" ! !

" " ! !
" % % ! ! " !
" " !

3) Row reduce T E to an echelon form Y , keeping track of each EROs used:

" # " " # " #  " " # 

! "  Ð"Ñ  ! " "  Ð#Ñ  ! "  Ð$Ñ  ! " "
" "
  µ   µ   µ  
" "

" " " " " !  " ! !  "

" % % ! $ 2 ! $ 2 ! ! 1
" !

" #
Ð%Ñ  ! "
"
µ   œY
"

! !
! ! 1
!

The row operations were

(1) add  "‡row(1) to row 3

(2) add  "‡row(1) to row 4
(3) add  3‡row(2) to row $
 Ð%Ñ add  1*row(3) to row 4

We could write down the corresponding elementary matrices:

I% I$ I# I" ÐT EÑ œ Y , so that T E œ ÐI% I$ I# I" Ñ" Y

and then compute P œ ÐI% I$ I# I" Ñ" œ I"" I#" I$" I%"

or we could use the mechanical algorithm described above write down PÞ In steps

1 0 1 0 1 0 1 0
* 0 * 0 * 0 * 0
0 0 0 0 0 0 0 0
 Ä Ä Ä 
1 0 1 0 1 0 1 0

*       1
* * 1 0 " * 1 0 " * 1 0 " $ 1 0
* * 1 * * * 1 " * * 1 " ‡ ‡

" ! " !
‡ ! ! !
! ! ! !
Ä  Ä œP
" ! " !

"   "
" $ " ! " $ " !
‡ " " " ! "

Then TE œ P Y

" # " ! " #

! " ! ! ! "
" ! ! "
  œ   
" " ! "

"   " ! !
" % % " $ " ! ! ! 1
" " " ! " !

#
"
To solve EB œ  ß write
(
$

#
(
T EB œ PY B œ T , œ   Ðrows 2,4 of , interchanged, as forE earlierÑ
"
$

#
(
We then solve PY B œ   as before.
"
$