CSC475: Numerical Computing

Tanveer Ahmed Siddiqui

Department of Computer Science

COMSATS University, Islamabad
 Lecture No 8

Solution of
System of Linear equation

Department of Computer Science 2

The LU decomposition method:Why

 The elimination procedure requires many

mathematical operations and significantly more
computing time than the back-substitution
calculations.
 During the elimination procedure, the matrix of
coefficients [a] and the vector [b] are both
changed.

Department of Computer Science 3

 Background

 This means that if there is a need to solve

systems of equations that have the same left-
hand-side terms (same coefficient matrix [a])
but different right-hand-side constants (different
vectors [b] ), the elimination procedure has to
be carried out for each [b] again.

Department of Computer Science 4

 Background

 Ideally, it would be better if the operations on

the matrix of coefficients [a] are separated from
those on the vector of constants [b].
 In this way, the elimination procedure with [a] is
done only once and then is used for solving
systems of equations with different vectors [b].
 One option for solving various systems of
equations [a][x] = [b] that have the same
coefficient matrices [a] but different constant
vectors [b] is to first calculate the inverse of the
matrix [a] .
Department of Computer Science 5
 Background

 Once the inverse matrix [a]-1 is known, the

solution can be calculated by: [x] = [a]-1[b]
 Calculating the inverse of a matrix, however,
requires many mathematical operations, and is
computationally inefficient.
 Then what is other solution?
 A more efficient method of solution for this case
is the LU decomposition method.

Department of Computer Science 6

 Background

 In the LU decomposition method, the operations

with the matrix [a] are done without using, or
changing, the vector [b] , which is used only in
the substitution part of the solution.
 The LU decomposition method can be used for
solving a single system of linear equations, but it
is especially advantageous for solving systems
that have the same coefficient matrices [a] but
different constant vectors [b] .
 So, Let us discussed LU decomposition method.

Department of Computer Science 7

 The LU decomposition method

 The LU decomposition method is a method for

solving a system of linear equations
[a] [x] = [b].
 In this method the matrix of coefficients [a] is
decomposed (factored) into a product of two
matrices [L] and [U]: [a] = [L][U] (4.20)

where the matrix [L] is a lower triangular matrix

and [U] is an upper triangular matrix.

Department of Computer Science 8

 How it works

 With this decomposition, the system of equations

to be solved has the form:
[L][U][x] = [b] (4.21)
 To solve this equation, the product [U][x] is
defined as:
[U][x] = [y] (4.22)
 and is substituted in Eq. (4.21) to give:
 [L][y] = [b] (4.23)

Department of Computer Science 9

 How it works

 Now, the solution [x] is obtained in two steps.

 First, Eq. (4.23) i.e. [L][y] = [b] is solved for [y].
 Since the matrix [L] is a lower triangular matrix, the
solution [y] in Eq. (4.23) is obtained by using the
forward substitution method.
 Then, the solution [y] is substituted in Eq. (4.22)
i.e. [U][x] = [y], and that equation is solved for
[x].
 Once [y] is known and is substituted in Eq. (4.22),
this equation is solved by using back substitution,
since [U] is an upper triangular matrix.

Department of Computer Science 10

 Upshot

Department of Computer Science 11

 LU Decomposition Using Crout's Method

Department of Computer Science 12

 LU Decomposition Using Crout's Method

Department of Computer Science 13

 LU Decomposition Using Crout's Method

Department of Computer Science 14

 LU Decomposition Using Crout's Method

Department of Computer Science 15

 Example

Department of Computer Science 16

 Solution

Department of Computer Science 17

 Solution

Department of Computer Science 18

 Solution

Department of Computer Science 19

 Solution

Department of Computer Science 20

 Solution

Department of Computer Science 21

 Solution

Department of Computer Science 22

 Solution

Department of Computer Science 23

 Solution

Department of Computer Science 24

 Example

 Solve the following system of equations by

Crout’s reduction method

5 x1  2 x2  x3  4
7 x1  x2  5 x3  8
3 x1  7 x2  4 x3  10
Department of Computer Science
 Solution

 Let the coefficient matrix [A] be written as [L]

[U]. Thus,

 l11 0 0  1 u12 u13   5 2 1 

l l  
0  0 1  
u23   7 1 5
 21 22

l31 l32 l33  0 0 1   3 7 4 

Department of Computer Science

 Solution

 Step I: Multiply all the rows of [L] by the first

column of [U], we get

l11  5, l21  7, l31  3

Department of Computer Science

 Solution

Step II: Multiply the first row of [L] by the

second and third columns of [U], we have

l11u12  2, l11u13  1

2 1
u12   , u13 
5 5
Department of Computer Science
 STEP III: Multiply the 2nd and 3rd rows of [L]
by the 2nd column of [U], we get

14 19 
l21u12  l22  1 or l22  1   
5 5

6 41 
l31u12  l32  7 or l32  7  
5 5 
Department of Computer Science
 STEP IV: Multiply the 2nd row of [L] by the 3rd
column of [U]

l21u13  l22u23  5
19 7 32
u23  5  u23  
5 5 19
Department of Computer Science
STEP V: Finally, multiply the 3rd row of [L] with
the 3rd column of [U], we obtain

l31u13  l32u23  l33  4

327
l33 
19
Department of Computer Science
Thus, the given system of equations takes the
form [L][U][X] = (B).

  1  2 1 x   4 
0  5    
1
5 0 5
     
7 19 0  0 1 
32 
 x2    8 
 5  19     
      
41 327 0 0 1 
3   x3  10 
 5 19   

Department of Computer Science

  That is,

   z1  4
5 0 0    
    
7 19
0   z2    8 
 5    
 41 327     
3  z  10 
 5 19   3   

Department of Computer Science

 Let [U](X) = (Z), then
[ L]( Z )  (4 8 10) T
 or
   z1   4 
5 0 0    
    
 7 19 0   z2    8 
 5    
 41 327     
3   z  10 
 5 19   3   

Department of Computer Science

 Which gives utilizing these values of z, the Eq.
becomes

 2 1 x   4 
 1 
5    5 
1
5
    
0 1 32   12 
  x2  
 19     19 
    
 0 0 1  46
 x3   
   327 
Department of Computer Science
By back substitution method, we obtain
This is the required solution.

46 284 366
x3  , x2  , x1 
327 327 327

Department of Computer Science

 Your Turn

Department of Computer Science 37

 Your Turn

Department of Computer Science 38

 Do matrices always have an LU
decomposition?
 No. Sometimes it is impossible to write a matrix in the
form “lower triangular” × “upper triangular”.
 Why NO
 An invertible matrix A has an LU decomposition provided
that all its leading submatrices have non-zero
determinants.
 Can we get around this problem?
 Yes. It is always possible to re-order the rows of an
invertible matrix so that all of the submatrices have non-
zero determinants.

Department of Computer Science 39