Professional Documents
Culture Documents
Chapert 5
Chapert 5
6 Linear Equations
I. Gaussian Elimination
I.1 Introduction of the method
25 5 1 x1 106.8
0 − 4.8 − 1.56 x = − 96.21
2
0 0 0.7 x3 0.735
Forward Elimination
A set of n equations and n unknowns
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
a21 x1 + a22 x2 + a23 x3 + ... + a2 n xn = b2
. .
. .
. .
an1x1 + an 2 x2 + an3 x3 + ... + ann xn = bn
a21
a (a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1 )
11
a21 a21 a21
a21 x1 + a12 x2 + ... + a1n xn = b1
a11 a11 a11
Forward Elimination
Subtract the result from Equation 2.
a21 x1 + a22 x2 + a23 x3 + ... + a2 n xn = b2
a21 a21 a21
− a21 x1 + a a12 x2 + ... + a a1n xn = a b1
_________________________________________________
11 11 11
or a x + ... + a x = b
'
22 2
'
2n n
'
2
Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations as
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
'
a32 x2 + a33
'
x3 + ... + a3' n xn = b3'
. . .
. . .
. . .
End of Step 1
Forward Elimination
Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
"
a33 x3 + ... + a3" n xn = b3"
. .
. .
. .
( n −1) (n −1 )
ann xn = bn
25 5 1 x1 106.8
0 − 4.8 − 1.56 x = − 96.21
2
0 0 0.7 x3 0.735
( n −1) (n −1 )
ann xn = bn
Back Substitution
Start with the last equation because it has only one unknown
( n −1)
b
xn = n
( n −1)
a nn
Back Substitution
( n −1)
b
xn = n
( n −1)
a nn
(i −1)
− aij(i −1) x j
n
bi
j =i +1
xi = (i −1) for i = n − 1,...,1
a ii
Example 1
The upward velocity of a rocket is given at three
different times
t12 t1 1 a1 v1
2 a = v
t 2 t2 1 2 2
t32 t3 1
a3
v3
1. Forward Elimination
2. Back Substitution
Forward Elimination
Number of Steps of Forward
Elimination
Number of steps of forward elimination is
(n−1)=(3−1)=2
Forward Elimination: Step 1
25 5 1 106.8 Divide Equation 1 by 25 and
64 8 1 177.2
64
multiply it by 64, = 2.56 .
144 12 1 279.2 25
25 5 1 106.8 2.56 = 64 12.8 2.56 273.408
. 64 8 1 177.2
Subtract the result from − 64 12.8 2.56 273.408
Equation 2
0 − 4.8 − 1.56 − 96.208
25 5 1 106.8
Substitute new equation for 0 − 4.8 − 1.56 − 96.208
Equation 2
144 12 1 279.2
Forward Elimination: Step 1 (cont.)
25 5 1 106.8
Divide Equation 1 by 25 and
0 − 4.8 − 1.56 − 96.208
144
144 12 1 279.2 multiply it by 144, 25 = 5.76 .
25 5 1 106.8
Substitute new equation for 0 − 4.8 − 1.56 − 96.208
Equation 3
0 0 0.7 0.76
Back Substitution
Back Substitution
25 5 1 106.8 25 5 1 a1 106.8
0 − 4.8 − 1.56 − 96.2 0 − 4.8 − 1.56 a = − 96.208
2
0 0 0.7 0.7 0 0 0.7 a3 0.76
Solving for a3
0.7a3 = 0.76
0.76
a3 =
0.7
a3 = 1.08571
Back Substitution (cont.)
25 5 1 a1 106.8
0 − 4.8 − 1.56 a = − 96.208
2
0 0 0.7 a3 0.76
Solving for a2
− 4.8a2 − 1.56a3 = −96.208
− 96.208 + 1.56a3
a2 =
− 4.8
− 96.208 + 1.56 1.08571
a2 =
− 4.8
a2 = 19.6905
Back Substitution (cont.)
25 5 1 a1 106.8
0 − 4.8 − 1.56 a = − 96.2
2
0 0 0.7 a3 0.76
Solving for a1
25a1 + 5a2 + a3 = 106.8
106.8 − 5a2 − a3
a1 =
25
106.8 − 5 19.6905 − 1.08571
=
25
= 0.290472
Gaussian Elimination Solution
25 5 1 a1 106.8
64 8 1 a2 = 177.2
144 12 1 a3 279.2
a1 0.290472
a = 19.6905
2
a3 1.08571
Example 1 Cont.
Solution a1 0.290472
a = 19.6905
The solution vector is
2
a3 1.08571
The polynomial that passes through the three data points is then:
= 129.686 m/s.
Be careful!! Division by zero
10 x2 − 7 x3 = 3
6 x1 + 2 x2 + 3x3 = 11
5 x1 − x2 + 5 x3 = 9
0 10 − 7 x1 3
6 2 3 x2 = 11
5 − 1 5 x3 9
Is division by zero an issue here?
12 x1 + 10 x2 − 7 x3 = 15
6 x1 + 5 x2 + 3x3 = 14
5 x1 − x2 + 5 x3 = 9
12 10 − 7 x1 15
6 5 3 x2 = 14
5 − 1 5 x3 9
Is division by zero an issue here?
YES
12 x1 + 10 x2 − 7 x3 = 15
6 x1 + 5 x2 + 3x3 = 14
24 x1 − x2 + 5 x3 = 28
Exact Solution
x1 1
x = 1
2
x3 1
20 15 10 x1 45
− 3 − 2.249 7 x = 1.751
2
5 1 3 x3 9
6 14 5.1 3.7 6 x1 5
0 − 7 6 1 2 x2 − 6
0 4 12 1 11 x3 = 8
0 9 23 6 8 x4 9
0 − 17 12 11 43 x5 3
6 14 5.1 3.7 6 x1 5
0 − 17 12 11 43 x 3
2
0 4 12 1 11 x3 = 8
0 9 23 6 8 x 4 9
0 − 7 6 1 2 x5 − 6
Switched Rows
Gaussian Elimination
with Partial Pivoting
A method to solve simultaneous linear
equations of the form [A][X]=[C]
Two steps
1. Forward Elimination
2. Back Substitution
Forward Elimination
Same as naïve Gauss elimination method
except that we switch rows before each
of the (n-1) steps of forward elimination.
Example: Matrix Form at Beginning
of 2nd Step of Forward Elimination
a11 a12 a13 a1n x1 b1
0 a' a '
'
a 2 n x2 '
b2
22 23
0 a32'
a '
33 a3n x3 = b3
' '
0 a'n 2 a '
n3 a '
n4
'
ann xn bn
'
Matrix Form at End of Forward
Elimination
a11 a12 a13 a1n x1 b1
0 a' a ' '
a 2 n x2 b2'
22 23
0 0 a "
33 a3n x3 = b3
" "
0 0 0 (n −1 )
0 ann xn bn
(n-1 )
Back Substitution Starting Eqns
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
'
a22 x2 + a23
'
x3 + ... + a2' n xn = b2'
"
a33 x3 + ... + an" xn = b3"
. .
. .
. .
( n −1) (n −1 )
ann xn = bn
Back Substitution
( n −1)
b
xn = n
( n −1)
a nn
(i −1) n
(i −1)
bi − aij x j
j =i +1
xi = (i −1) for i = n − 1,...,1
a ii
Example 2
Solve the following set of equations
by Gaussian elimination with partial
pivoting
25 5 1 a1 106.8
64
8 1 a 2 = 177.2
144 12 1 a 3 279.2
Example 2 Cont.
25 5 1 a1 106.8 25 5 1 106.8
64 8 1 a = 177.2
2 64 8 1 177. 2
144 12 1 a3 279.2 144 12 1 279.2
1. Forward Elimination
2. Back Substitution
Forward Elimination
Number of Steps of Forward
Elimination
.
0 2.667 0.5556 53.10
Subtract the result from − 0 2.667 0.7556 53.33
Equation 3
0 0 − 0.2 − 0.23
144 12 1 279.2
Substitute new equation for 0 2.917 0.8264 58.33
Equation 3
0 0 − 0.2 − 0.23
Back Substitution
Back Substitution
144 12 1 279.2 144 12 1 a1 279.2
0 2.917 0.8264 58.33 0 2.917 0.8264 a = 58.33
2
0 0 − 0.2 − 0.23 0 0 − 0.2 a3 − 0.23
Solving for a3
− 0.2a3 = −0.23
− 0.23
a3 =
− 0.2
= 1.15
Back Substitution (cont.)
144 12 1 a1 279.2
0 2.917 0.8264 a = 58.33
2
0 0 − 0.2 a3 − 0.23
Solving for a2
2.917a2 + 0.8264a3 = 58.33
58.33 − 0.8264a3
a2 =
2.917
58.33 − 0.8264 1.15
=
2.917
= 19.67
Back Substitution (cont.)
144 12 1 a1 279.2
0 2.917 0.8264 a = 58.33
2
0 0 − 0.2 a3 − 0.23
Solving for a1
144a1 + 12a2 + a3 = 279.2
279.2 − 12a2 − a3
a1 =
144
279.2 − 12 19.67 − 1.15
=
144
= 0.2917
Gaussian Elimination with Partial
Pivoting Solution
25 5 1 a1 106.8
64 8 1 a = 177.2
2
144 12 1 a3 279.2
a1 0.2917
a = 19.67
2
a3 1.15
Gauss Elimination with
Partial Pivoting
Another Example
Partial Pivoting: Example
Consider the system of equations
10 x1 − 7 x2 = 7
− 3x1 + 2.099 x2 + 6 x3 = 3.901
5 x1 − x2 + 5 x3 = 6
In matrix form
10 − 7 0 x1 7
− 3 2.099 6 x 3.901
2 =
5 − 1 5 x3 6
10 − 7 0 x1 7
0 2.5 5 x = 2.5
2
0 0 6.002 x3 6.002
Partial Pivoting: Example
Back Substitution
Solving the equations through back substitution
10 − 7
6.002
0 x1 7 x3 = =1
0 2.5 5 x = 2.5 6.002
2
0 0 6.002 x3 6.002 2.5 − 5 x3
x2 = = −1
2.5
7 + 7 x 2 − 0 x3
x1 = =0
10
Partial Pivoting: Example
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting
x1 0 x1 0
X calculated = x2 = − 1 X exact = x 2 = − 1
x3 1 x3 1
II. LU Decomposition
LU Decomposition is another method to solve a set of
simultaneous linear equations
http://numericalmethods.eng.usf.edu
LU Decomposition
Method
For most non-singular matrix [A] that one could conduct Naïve Gauss
Elimination forward elimination steps, one can always write it as
[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix
http://numericalmethods.eng.usf.edu
How does LU Decomposition work?
If solving a set of linear equations [A][X] = [C]
If [A] = [L][U] then [L][U][X] = [C]
Multiply by [L]-1
Which gives [L]-1[L][U][X] = [L]-1[C]
Remember [L]-1[L] = [I] which leads to [I][U][X] = [L]-1[C]
Now, if [I][U] = [U] then [U][X] = [L]-1[C]
Now, let [L]-1[C]=[Z]
Which ends with [L][Z] = [C] (1)
and [U][X] = [Z] (2)
http://numericalmethods.eng.usf.edu
LU Decomposition
How can this be used?
http://numericalmethods.eng.usf.edu
Method: [A] Decomposes to [L] and [U]
1 0 0 u11 u12 u13
A = LU = 21 1 0 0 u 22 u 23
31 32 1 0 0 u 33
[U] is the same as the coefficient matrix at the end of the forward elimination step.
[L] is obtained using the multipliers that were used in the forward elimination process
http://numericalmethods.eng.usf.edu
Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination
25 5 1
64 8 1
144 12 1
25 5 1
= 2.56; Row2 − Row1(2.56) = 0 − 4.8 − 1.56
64
Step 1:
25
144 12 1
25 5 1
= 5.76; Row3 − Row1(5.76) = 0 − 4.8 − 1.56
144
25
0 − 16.8 − 4.76
http://numericalmethods.eng.usf.edu
Finding the [U] Matrix
25 5 1
Matrix after Step 1: 0 − 4.8 − 1.56
0 − 16.8 − 4.76
25 5 1
− 16.8
Step 2: = 3.5; Row3 − Row2(3.5) = 0 − 4.8 − 1.56
− 4.8
0 0 0.7
25 5 1
U = 0 − 4.8 − 1.56
0 0 0.7
http://numericalmethods.eng.usf.edu
Finding the [L] matrix
1 0 0
1 0
21
31 32 1
http://numericalmethods.eng.usf.edu
Finding the [L] Matrix
From the second 25 5 1
step of forward 0 − 4.8 − 1.56 32 = a32 = − 16.8 = 3.5
elimination a 22 − 4.8
0 − 16.8 − 4.76
1 0 0
L = 2.56 1 0
5.76 3.5 1
http://numericalmethods.eng.usf.edu
Does [L][U] = [A]?
1 0 0 25 5 1
LU = 2.56 1 0 0 − 4.8 − 1.56 =
5.76 3.5 1 0 0 0.7
?
http://numericalmethods.eng.usf.edu
Example
Set [L][Z] = [C] 1 0 0 z1 106.8
2.56 1 0 z = 177.2
2
5.76 3.5 1 z 3 279.2
http://numericalmethods.eng.usf.edu
Example
Complete the forward substitution to solve for [Z]
z1 = 106.8
z 2 = 177.2 − 2.56 z1 z1 106.8
= 177.2 − 2.56(106.8)
= −96.2
Z = z2 = − 96.21
z3 = 279.2 − 5.76 z1 − 3.5 z 2 z3 0.735
= 279.2 − 5.76(106.8) − 3.5(− 96.21)
= 0.735
http://numericalmethods.eng.usf.edu
Example
Set [U][X] = [Z]
25 5 1 x1 106.8
0 − 4.8 − 1.56 x = − 96.21
2
0 0 0.7 x3 0.735
http://numericalmethods.eng.usf.edu
Example
Substituting in a3 and using the
From the 3rd equation second equation
a3 =
0.735 − 96.21 + 1.56a3
a2 =
0.7 − 4.8
a3 = 1.050 − 96.21 + 1.56(1.050)
a2 =
− 4.8
a2 = 19.70
http://numericalmethods.eng.usf.edu
Example
Substituting in a3 and a2 using Hence the Solution Vector is:
the first equation
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
How can LU Decomposition be used to find the inverse?
Assume the first column of [B] to be [b11 b12 … bn1]T
Using this and the definition of matrix multiplication
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Find the inverse of a square matrix [A]
25 5 1
A = 64 8 1
144 12 1
Using the decomposition procedure, the [L] and [U] matrices are found to be
1 0 0 25 5 1
A = LU = 2.56 1 0 0 − 4.8 − 1.56
5.76 3.5 1 0 0 0.7
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving for the each column of [B] requires two steps
1) Solve [L] [Z] = [C] for [Z]
2) Solve [U] [X] = [Z] for [X]
1 0 0 z1 1
Step 1: LZ = C → 2.56 1 0 z2 = 0
5.76 3.5 1 z3 0
This generates the equations:
z1 = 1
2.56 z1 + z 2 = 0
5.76 z1 + 3.5 z 2 + z3 = 0
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving for [Z]
z1 = 1
z1 1
z 2 = 0 − 2.56 z1
= 0 − 2.56(1)
Z = z2 = − 2.56
= −2.56 z3 3.2
z3 = 0 − 5.76 z1 − 3.5 z 2
= 0 − 5.76(1) − 3.5(− 2.56 )
= 3.2
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
25 5 1 b11 1
Solving [U][X] = [Z] for [X] 0 − 4.8 − 1.56 b = − 2.56
21
0 0 0.7 b31 3.2
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Using Backward Substitution
3.2
b31 = = 4.571 So the first column of
0.7
the inverse of [A] is:
−2.56 + 1.560b31
b21 =
−4.8 b11 0.04762
−2.56 + 1.560(4.571) b = − 0.9524
= = −0.9524
−4.8 21
b11 =
1 − 5b21 − b31 b31 4.571
25
1 − 5(− 0.9524) − 4.571
= = 0.04762
25
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Repeating for the second and third columns of the inverse
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
The inverse of [A] is
http://numericalmethods.eng.usf.edu
Is LU Decomposition better than
Gaussian Elimination?
Solve [A][X] = [B]
8n 3 4n 8n 3 4n
T + 12n 2 + T + 12n 2 +
3 3 3 3
http://numericalmethods.eng.usf.edu
To find inverse of [A]
Time taken by Gaussian Elimination Time taken by LU Decomposition
= n(CT |FE +CT |BS ) = CT |DE + n CT |FS + n CT |BS
8n 4 4n 2 32n 3 20n
= T + 12n +
3
= T + 12n 2 −
3 3 3 3
http://numericalmethods.eng.usf.edu
To find inverse of [A]
Time taken by Gaussian Elimination Time taken by LU Decomposition
8n 4 4n 2 32n 3 20n
T + 12n +
3
T + 12n 2 −
3 3 3 3
http://numericalmethods.eng.usf.edu
Using LU Decomposition to solve SLEs
Solve the following set of 25 5 1 x1 106.8
linear equations using LU 64 8 1 x = 177.2
Decomposition 2
144 12 1 x3 279.2
Using the procedure for finding the [L] and [U] matrices
1 0 0 25 5 1
A = LU = 2.56 1 0 0 − 4.8 − 1.56
5.76 3.5 1 0 0 0.7
http://numericalmethods.eng.usf.edu