Linear Systems – Iterative methods

Prepared by
Rasel Ahmmed
Lecturer, Dept. of EEE
BSMRSTU, Gopalganj
10/20/22 1
 SYSTEMS OF EQUATIONS
To this point, we have focused on the determination of the roots of a single equation. A related problem is to locate the roots of a set
of simultaneous equations,

(1)

The solution of this system consists of a set of x values that simultaneously result in all the equations equaling zero.
Present methods for the case where the simultaneous equations are linear—that is, they can be expressed in the general form;

(2)
where the b and the a’s are constants. Algebraic and transcendental equations that do not fit this format are called nonlinear
equations. For example,

are two simultaneous nonlinear equations with two unknowns, x and y. They can be expressed in the form of Eq. (1) as

(3.a)
(3.b)
Thus, the solution would be the values of x and y that make the functions u(x, y) and v(x, y) equal to zero. Most approaches for
determining such solutions are extensions of the open methods for solving single equations. In this section, we will investigate two
of these: fixed-point iteration and Newton-Raphson.

Fixed-Point Iteration
The fixed-point-iteration approach can be modified to solve two simultaneous, nonlinear equations. This approach will be
illustrated in the following example.

Problem Statement: Use fixed-point iteration to determine the roots of Eq. (3). Note that a correct pair of roots is x = 2 and y = 3.
Initiate the computation with guesses of x = 1.5 and y = 3.5.

Solution: Equation (3.a) can be solved for

(4)

and Eq. (3.b) can be solved for

(5)

Note that we will drop the subscripts for the remainder of the example. On the basis of the initial guesses, Eq. (4) can be used to
determine a new value of x:

This result and the initial value of y = 3.5 can be substituted into Eq. (5) to determine a new value of y:
Thus, the approach seems to be diverging. This behavior is even more pronounced on the second iteration:

Obviously, the approach is deteriorating.

Now we will repeat the computation but with the original equations set up in a different format. For example, an alternative
formulation of Eq. (3.a) is

and of Eq. (3.b) is

Now the results are more satisfactory:

Thus, the approach is converging on the true values of x = 2 and y = 3.

Example 1: Use fixed-point iteration to determine the roots of x2 -x – 1=0. That a correct of root is x = 2.
Solution:
1 x1
2 x2
3 x3
4 x4
5 x5
6 x6
7 x7
8 x8
9 x9
10 x10
f(x) x2 -x – 1=0
g(x) x=1+1/x
=2
= 1+1/x1=1+1/2=1.5
=1+1/x2=1+1/1.5=1.6666
=1+1/x3=1+1/1.6666=1.6
=1+1/x4=1+1/1.6=1.625
=1+1/x5=1+1/1.625=1.615385
=1+1/x6=1+1/1.615385=1.619
=1+1/x7=1+1/1.619=1.6176
=1+1/x8=1+1/1.6176=1.61819
=1+1/x9=1+1/1.61819=1.6179
x1=g1(y0) ln(1-0)=ln(1) 0

x2=g2(y1) ln(1-(-2))=ln(3) 1,0986

x3=g3(y2) ln(1-(-1,671))=ln(1) 0,9824

x4=g4(y3) ln(1-(-1,7421))=ln(2,7421) 1,008724

The solution is converging to the exact solution of x=1.004169 , y= - 1.729637

 Linear Systems – Iterative methods
1. Jacobi Method
2. Gauss-Siedel Method

7
 Jacobi iteration
a11 x1  a12 x2    a1n xn  b1  x10 
a21 x1  a22 x2    a2 n xn  b2  0
0  x2 
x 
  
 0
an1 x1  an 2 x2    ann xn  bn  xn 

1
x11  (b1  a12 x20    a1n xn0 ) 1  i 1 n 
a11
k 1
xi   bi 
a ii 
j 1 a ij x  j
k
j
k
a ij x 
j

1
 i 1 
x12  (b2  a21 x10  a23 x30    a2 n xn0 )
a22
1
x1n  (bn  an1 x10  an 2 x20    ann 1 xn01 )
ann
8
 Gauss-Seidel (GS) iteration
a11 x1  a12 x2    a1n xn  b1  x10 
Use the latest a21 x1  a22 x2    a2 n xn  b2  0
0  x2 
update x 
  
 0
an1 x1  an 2 x2    ann xn  bn  xn 

1
x11  (b1  a12 x20    a1n xn0 ) 1  i 1 n 
a11
k 1
xi   bi 
a ii 
a ij x k 1
j   a ij x  k
j

1 1 1 0 0
j 1 j  i 1 
x2  (b2  a21 x1  a23 x3    a2 n xn )
a22
1
x1n  (bn  an1 x11  an 2 x12    ann 1 x1n 1 )
ann
9
 Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
If: the diagonal elements are
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1 non-zero
a21 x1  a22 x2  a23 x3  ...  a2n xn  b2 Rewrite each equation solving
. .
. . for the corresponding unknown
. .
ex:
an1 x1  an 2 x2  an 3 x3  ...  ann xn  bn
First equation, solve for x1
Second equation, solve for x2

10
 Gauss-Seidel Method
Algorithm
Rewriting each equation
c1  a12 x 2  a13 x3   a1n x n From Equation 1
x1 
a11

c2  a21 x1  a23 x3   a2 n xn
x2  From equation 2
a22
  
cn 1  an 1,1 x1  an 1, 2 x2   an 1, n  2 xn  2  an 1,n xn From equation n-1
xn 1 
an 1,n 1
cn  an1 x1  an 2 x2    an ,n 1 xn 1
xn  From equation n
ann

11
 Gauss-Seidel Method
Algorithm
General Form of each equation
n n

c1   a1 j x j cn 1  a
j 1
n 1, j xj
j 1
j  n 1
x1 
j 1 xn 1 
a11 an 1,n 1
n
c n   a nj x j
n
c2   a2 j x j
j 1 j 1
j n
x2 
j2
xn 
a 22 a nn

12
 Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci   aij x j
j 1
j i
xi  , i  1,2, , n.
aii

How or where can this equation be used?

13
 Graphical depiction of the difference between (a) the Gauss-Seidel and (b) the
Jacobi iterative methods for solving simultaneous linear algebraic equations.

14
 Jacobi Iterative Technique

Consider the following set of

equations.
10 x1  x2  2 x3 6
 x1  11 x2  x3  3x4  25
2 x1  x2  10 x3  x4  11
3 x2  x3  8 x4  15

15
Convert the set Ax = b in the form of x = Tx
+ c.
1 1 3
x1  x2  x3 
10 5 5
1 1 3 25
x2  x1  x3  x4 
11 11 11 11
1 1 1 11
x3   x1  x2  x4 
5 10 10 10
3 1 15
x4   x2  x3 
8 8 8

16
 Start with an initial approximation of:
(0) (0) (0) (0)
x1  0, x 2  0, x3  0 and x 4  0.

(1) 1 (0) 1 (0) 3

x1  x2  x3 
10 5 5
( 1) 1 (0) 1 (0) 3 (0) 25
x2  x1  x3  x4 
11 11 11 11
( 1) 1 (0) 1 (0) 1 (0) 11
x3   x1  x2  x4 
5 10 10 10
(1) 3 (0) 1 (0) 15
x4   x2  x3 
8 8 8

17
 (1) 1 1 3
x1  (0)  (0) 
10 5 5
(1) 1 1 3 25
x2  (0)  (0)  (0) 
11 11 11 11
(1) 1 1 1 11
x3   (0)  (0)  (0) 
5 10 10 10
( 1) 3 1 15
x4   (0)  (0) 
8 8 8

(1) (1)
x1  0.6000, x 2  2.2727,
(1) (1)
x3  1.1000 and x 4  1.8750
18
 (2) 1 (1) 1 (1) 3
x1  x2  x3 
10 5 5
( 2) 1 (1) 1 (1) 3 (1) 25
x2  x1  x3  x4 
11 11 11 11
( 2) 1 (1) 1 (1) 1 (1) 11
x3   x1  x2  x4 
5 10 10 10
( 2) 3 (1) 1 (1) 15
x4   x2  x3 
8 8 8

19
 (k) 1 ( k 1 ) 1 ( k 1 ) 3
x1  x2  x3 
10 5 5
(k) 1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
x2  x1  x3  x4 
11 11 11 11
(k) 1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
x3   x1  x2  x4 
5 10 10 10
(k) 3 ( k 1 ) 1 ( k 1 ) 15
x4   x2  x3 
8 8 8

20
 Results of Jacobi Iteration:

k 0 1 2 3
(k ) 0.0000 0.6000 1.0473 0.9326
x1
(k ) 0.0000 2.2727 1.7159 2.0530
x2
(k ) 0.0000 -1.1000 -0.8052 -1.0493
x3
(k ) 0.0000 1.8750 0.8852 1.1309
x4

21
Gauss-Seidel Iterative Technique

Consider the following set of

equations.
10 x1  x2  2 x3 6
 x1  11 x2  x3  3x4  25
2 x1  x2  10 x3  x4  11
3 x2  x3  8 x4  15

22
 (k) 1 ( k 1 ) 1 ( k 1 ) 3
x1  x2  x3 
10 5 5
(k ) 1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
x2  x1  x3  x4 
11 11 11 11
(k ) 1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
x3   x1  x2  x4 
5 10 10 10
(k ) 3 ( k 1 ) 1 ( k 1 ) 15
x4   x2  x3 
8 8 8

23
 (k) 1 ( k 1 ) 1 ( k 1 ) 3
x1  x2  x3 
10 5 5
(k) 1 (k) 1 ( k 1 ) 3 ( k 1 ) 25
x2  x1  x3  x4 
11 11 11 11
(k) 1 (k) 1 (k) 1 ( k 1 ) 11
x3   x1  x2  x4 
5 10 10 10
(k) 3 (k) 1 (k) 15
x4   x2  x3 
8 8 8

24
 Results of Gauss-Seidel Iteration:
(Blue numbers are for Jacobi iterations.)
k 0 1 2 3
(k ) 0.0000 0.6000 1.0300 1.0065
x1
0.6000 1.0473 0.9326
(k ) 0.0000 2.3272 2.0370 2.0036
x2
2.2727 1.7159 2.0530
(k ) 0.0000 -0.9873 -1.0140 -1.0025
x3
-1.1000 -0.8052 -1.0493
(k ) 0.0000 0.8789 0.9844 0.9983
x4
1.8750 0.8852 1.1309

25
 The solution is: x1= 1, x2 = 2, x3 = -1, x4 = 1

It required 15 iterations for Jacobi

method and 7 iterations for Gauss-Seidel
method to arrive at the solution with a
tolerance of 0.00001.

While Jacobi would usually be the

slowest of the iterative methods, it
is well suited to illustrate an
algorithm that is well suited for
parallel processing!!!

26
 EXAMPLE Gauss-Seidel Method
Problem Statement. Use the Gauss-Seidel method to
obtain the solution for
3x1  0.1x2  0.2 x3  7.85
0.1x1  7 x2  0.3x3  19.3
0.3x1  0.2 x2  10 x3  71.4

Note that the solution is x

T
 3  2.5 7 

Solution. First, solve each of the equations for its

unknown on the diagonal:

27
 7.85  0.1x2  0.2 x3
x1  (E11.1.1)
3
 19.3  0.1x1  0.3 x3
x2  (E11.1.2)
7
71.4  0.3x1  0.2 x2
x3  (E11.1.3)
10
By assuming that x2 and x3 are zero
7.85  0.10   0.20
x1   2.616667
3
This value, along with the assumed value of x3 =0,
can be substituted into Eq.(E11.1.2) to calculate
 19.3  0.12.616667   0.30
x2   2.794524
7

28
 The first iteration is completed by substituting the
calculated values for x1 and x2 into Eq.(E11.1.3) to yield
71.4  0.32.616667   0.2 2.794524
x3   7.005610
10
For the second iteration, the same process is repeated to
compute
7.85  0.1 2.794524  0.27.005610
x1   2.990557
3
 19.3  0.12.990557  0.37.005610
x2   2.499625
7
71.4  0.32.990557  0.2 2.499625
x3   7.000291
10

29
 Gauss-Seidel Method: Example 2
Given the system of equations The coefficient matrix is:
12 x1  3 x2 - 5 x3  1 12 3  5
x1  5 x2  3x3  28 A   1 5 3 
3 x1  7 x2  13x3  76  3 7 13 

With an initial guess of Will the solution converge using the

Gauss-Siedel method?
 x1  1
 x   0 
 2  
 x3  1

30
 Gauss-Seidel Method: Example 2
Checking if the coefficient matrix is diagonally dominant
a11  12  12  a12  a13  3   5  8
12 3  5
A   1 5 3  a 22  5  5  a 21  a 23  1  3  4
 3 7 13 
a33  13  13  a31  a32  3  7  10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

31
 Gauss-Seidel Method: Example 2
Rewriting each equation With an initial guess of
12 3  5  a1   1   x1  1
 1 5 3  a   28  x   0 
   2    2  
 3 7 13   a3  76  x3  1

1  3 x 2  5 x3 1  30   51
x1  x1   0.50000
12 12

28  x1  3 x3 28  0.5  31
x2  x2   4.9000
5 5

76  3x1  7 x 2 76  30.50000  74.9000

x3  x3   3.0923
13 13

32
 Gauss-Seidel Method: Example 2
After Iteration #1
 x1  0.5000
 x   4.9000
 2  
 x3  3.0923

Substituting the x values into the After Iteration #2

equations
 x1  0.14679
1  34.9000   53.0923  x    3.7153 
x1   0.14679
12  2  
 x3   3.8118 
28  0.14679  33.0923
x2   3.7153
5

76  30.14679  74.900
x3   3.8118
13
33
 Gauss-Seidel Method: Example 3
Given the system of equations

3x1  7 x2  13x3  76 Rewriting the equations

x1  5 x2  3x3  28 76  7 x2  13x3
x1 
12 x1  3x2  5 x3  1 3
28  x1  3x3
With an initial guess of x2 
5
 x1  1
 x   0  1  12 x1  3 x 2
 2   x3 
 x3  1 5

34
 Gauss-Seidel Method
The Gauss-Seidel Method can still be used
 3 7 13 
The coefficient matrix is not
diagonally dominant
A   1 5 3 
12 3  5
But this is the same set of 12 3  5
equations used in example #2,
which did converge.
A   1 5 3 
 3 7 13 

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equations can form a diagonally dominant matrix.

35
 Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations

x1  x 2  x3  3
2 x1  3x 2  4 x3  9
x1  7 x 2  x3  9

Which equation(s) prevents this set of equation from having a

diagonally dominant coefficient matrix?

36
Naïve Gauss Elimination
 Naïve Gaussian Elimination
A method to solve simultaneous linear
equations of the form [A][X]=[C]

Two steps
1. Forward Elimination
2. Back Substitution
 Forward Elimination
The goal of forward elimination is to transform the
coefficient matrix into an upper triangular matrix

 25 5 1  x1  106.8 
 64 8 1  x   177.2 
   2  
144 12 1  x3  279.2

25 5 1   x1   106.8 
 0  4.8  1.56  x     96.21
   2  
 0 0 0.7   x3   0.735 
 Forward Elimination
A set of n equations and n unknowns
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
a21 x1  a22 x2  a23 x3  ...  a2 n xn  b2
. .
. .
. .
an1 x1  an 2 x2  an 3 x3  ...  ann xn  bn

(n-1) steps of forward elimination

Step 1
Forward Elimination
For Equation 2, divide Equation 1 by a11 and
multiply by a21 .

 a21 
 a ( a11 x1  a12 x2  a13 x3  ...  a1n xn  b1 )
 11 
a21 a21 a21
a21x1  a12 x2  ...  a1n xn  b1
a11 a11 a11
Subtract theForward Elimination
result from Equation 2.
a21 x1  a22 x2  a23 x3  ...  a2 n xn  b2
a21 a21 a21
− a21 x1  a a12 x2  ...  a a1n xn  a b1
11 11 11
_________________________________________________
 a21   a21  a21
 a22  a12  x2  ...   a2 n  a1n  xn  b2  b1
 a11   a11  a11

' ' '

or a x  ...  a x  b
22 2 2n n 2
 Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations as
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
' '
a22 x2  a23 x3  ...  a2' n xn  b2'
' '
a32 x2  a33 x3  ...  a3' n xn  b3'
. . .
. . .
. . .

an' 2 x2  an' 3 x3  ...  ann

'
xn  bn'

End of Step 1
Step 2 Forward Elimination
Repeat the same procedure for the 3rd term of
Equation 3.
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
' '
a22 x2  a23 x3  ...  a2' n xn  b2'
"
a33 x3  ...  a3" n xn  b3"
. .
. .
. .

an" 3 x3  ...  ann

"
xn  bn"
End of Step 2
 Forward Elimination
At the end of (n-1) Forward Elimination steps, the
system of equations will look like
a11 x1  a12 x 2  a13 x3  ...  a1n x n  b1
' '
a22 x2  a23 x3  ...  a2' n xn  b2'
" " "
a x  ...  a x  b
33 3 3n n 3
. .
. .
. .

n 1 n 1 
ann xn  bn

End of Step (n-1)

 Matrix Form at End of Forward
Elimination
a11 a12 a13  a1n   x1   b1 
0 '
a22 '
a23  '
a 2 n x2    b2' 
    
" " "
0 0 a
33  a3n   x3    b3 
    
          
 0 0 0 0 ann   xn  bn 
(n 1 )
    (n-1 )
 Back Substitution
Solve each equation starting from the last equation

25 5 1   x1   106.8 
 0  4.8  1.56  x    96.21
   2  
 0 0 0.7   x3   0.735 

Example of a system of 3 equations

Back Substitution Starting
Eqns
a11 x1  a12 x 2  a13 x3  ...  a1n x n  b1
' '
a22 x2  a23 x3  ...  a2' n xn  b2'
"
a33 x3  ...  an" xn  b3"
. .
. .
. .

n 1 n 1 
ann xn  bn
 Back Substitution
Start with the last equation because it has only one unknown
( n 1)
b n
xn  ( n 1)
a nn
 Back bSubstitution
( n 1)
n
xn  ( n 1)
a nn

bii 1  ai,ii11 xi 1  ai,ii12 xi  2  ...  ai,in1 xn

xi  i 1 for i  n  1,...,1
aii
n
i 1
bi   aiji 1 x j
j i 1
xi  i 1 for i  n  1,...,1
a ii
Naïve Gauss Elimination
Pitfalls
Pitfall#1. Division by zero
10 x2  7 x3  3
6 x1  2 x2  3x3  11
5 x1  x2  5 x3  9

0 10  7  x1   3 
6 2 3   x2   11
    
5  1 5   x3   9 
Is division by zero an issue here?
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
5 x1  x2  5 x3  9

12 10  7  x1  15
6 5 3   x2   14
    
 5  1 5   x3   9 
 Is division by zero an issue here?
YES
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
24 x1  x2  5 x3  28

12 10  7  x1  15 12 10  7  x1   15 

6 5 3   x2   14  0
     0 6.5  x2    6.5
    
24  1 5   x3  28 12  21 19   x3    2

Division by zero is a possibility at any step of

forward elimination
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
  3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Exact Solution

 x1  1
 x   1
 2  
 x3  1
Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
  3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 6 significant digits with chopping

 x1   0.9625 
 x    1.05 
 2  
 x3  0.999995
 Pitfall#2. Large Round-off Errors
 20 15 10  x1   45 
  3  2.249 7   x   1.751
   2  
 5 1 3   x3   9 

Solve it on a computer using 5 significant digits with chopping

 x1   0.625 
 x    1.5 
 2  
 x3  0.99995

Is there a way to reduce the round off error?

 Avoiding Pitfalls
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero
 Avoiding Pitfalls
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error
Gauss Elimination with
Partial Pivoting
 What is Different About Partial
Pivoting?
At the beginning of the kth step of forward elimination,
find the maximum of

akk , ak 1,k ,................, ank

If the maximum of the values is a pk
in the p th
row, k  p  n, then switch rows p and
k.
 Matrix Form at Beginning of 2nd
Step of Forward Elimination
a11 a12 a13  a1n   x1   b1 
0 '
a22 '
a23  ' 
a 2 n x2   ' 
b2
    
' ' ' '
0 a 32 a 33  a3n   x3    b3 
    
           
 0 '
an 2 ' '
an 3 an 4 ann   xn  bn 
'
    '
 Example (2nd step of FE)
6 14 5.1 3.7 6   x1   5 
0  7 6 1 2   x2    6
    
0 4 12 1 11   x3    8 
    
 0 9 23 6 8   x 4  9
 
0  17 12 11 43  x5   3 

Which two rows would you switch?

 Example (2nd step of FE)
6 14 5.1 3.7 6   x1   5 
0  17 12 11 43  x   3 
  2   
0 4 12 1 11   x3    8 
    
 0 9 23 6 8 x
   
4 9
0  7 6 1 2   x5    6

Switched Rows
 Gaussian Elimination
with Partial Pivoting
A method to solve simultaneous linear
equations of the form [A][X]=[C]

Two steps
1. Forward Elimination
2. Back Substitution
 Forward Elimination
Same as naïve Gauss elimination method
except that we switch rows before
each of the (n-1) steps of forward
elimination.
Example: Matrix Form at Beginning
of 2nd Step of Forward Elimination

a11 a12 a13  a1n   x1   b1 

0 a '
a '
 ' 
a 2 n x2   ' 
b2
 22 23    
' ' ' '
0 a 32 a 33  a3n   x3    b3 
    
          
 0 a '
n2 a '
n3 a '
n4 ann   xn  bn 
'
    '
Matrix Form at End of Forward
Elimination

a11 a12 a13  a1n   x1   b1 

0 '
a22 '
a23  '
a 2 n x2    b2' 
    
" " "
0 0 a
33  a3n   x3    b3 
    
         
 0 0 0 0 ann   xn  bn 
(n 1 )
    (n-1 )
Back Substitution Starting
Eqns
a11 x1  a12 x 2  a13 x3  ...  a1n x n  b1
' '
a22 x2  a23 x3  ...  a2' n xn  b2'
"
a33 x3  ...  an" xn  b3"
. .
. .
. .

n 1 n 1 
ann xn  bn
 Back Substitution
( n 1)
bn
x n  ( n 1)
a nn

n
i 1 i 1
bi   aij x j
j i 1
xi  i 1 for i  n  1,...,1
a ii
Gauss Elimination with
Partial Pivoting
Example
 Example 2

Solve the following set of equations

by Gaussian elimination with partial
pivoting

 25 5 1  a1  106.8 
 64   
8 1 a 2  177.2  
     
144 12 1  a 3  279.2
 Example 2 Cont.

 25 5 1  a1  106.8   25 5 1  106.8 
 64 8 1 a   177.2    
    2   64 8 1  177. 2 
144 12 1  a 3  279.2 144 12 1  279.2

1. Forward Elimination
2. Back Substitution
Forward Elimination
 Number of Steps of Forward
Elimination

Number of steps of forward elimination is

(n1)=(31)=2
 Forward Elimination: Step 1
 Examine absolute values of first column, first row
and below.
25 , 64 , 144
• Largest absolute value is 144 and exists in row 3.
• Switch row 1 and row 3.

 25 5 1  106.8  144 12 1  279.2

 64 8 1  177.2    64 8 1  177.2 
   
144 12 1  279.2  25 5 1  106.8 
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 64 8 1  177.2  64
  multiply it by 64,  0.4444 .
 25 5 1  106.8  144

144 12 1  279.2 0.4444  63.99 5.333 0.4444  124.1

.
Subtract the result from 64 8 1  177.2
Equation 2  63.99 5.333 0.4444  124.1
0 2.667 0.5556  53.10

Substitute new equation for 144 12 1  279.2

Equation 2  0 2.667 0.5556  53.10
 
 25 5 1  106.8 
Forward Elimination: Step 1 (cont.)
144 12 1  279.2 Divide Equation 1 by 144 and
 0 2.667 0.5556  53.10 25
  multiply it by 25,  0.1736 .
 25 5 1  106.8  144

144 12 1  279.2 0.1736  25.00 2.083 0.1736  48.47

.
25 5 1  106.8
Subtract the result from
Equation 3  25 2.083 0.1736  48.47
0 2.917 0.8264  58.33

Substitute new equation for 144 12 1  279.2

Equation 3  0 2.667 0.5556  53.10
 
 0 2.917 0.8264  58.33

Forward Elimination: Step 2
Examine absolute values of second column, second row
and below.
2.667 , 2.917
• Largest absolute value is 2.917 and exists in row 3.
• Switch row 2 and row 3.

144 12 1  279.2 144 12 1  279.2

 0 2.667 0.5556  53.10   0 2.917 0.8264  58.33
   
 0 2.917 0.8264  58.33  0 2.667 0.5556  53.10
 Forward Elimination: Step 2 (cont.)
Divide Equation 2 by 2.917 and
144 12 1  279.2
 0 2.917 0.8264  58.33 multiply it by 2.667,
  2.667
 0 2.667 0.5556  53.10  0.9143.
2.917
0 2.917 0.8264  58.33 0.9143  0 2.667 0.7556  53.33

.
0 2.667 0.5556  53.10
Subtract the result from  0 2.667 0.7556  53.33
Equation 3
0 0  0.2   0.23

144 12 1  279.2 
Substitute new equation for  0 2.917 0.8264  58.33 
Equation 3  
 0 0  0.2   0.23
Back Substitution
144 12 1
Back
 279.2 
Substitution
144 12 1  a   279.2 
1
 0 2.917 0.8264  58.33    0 2.917 0.8264 a    58.33 
    2  
 0 0  0.2   0.23  0 0  0.2   a3   0.23

Solving for a3
 0.2a3  0.23
 0.23
a3 
 0.2
 1.15
 Back Substitution
144 12 1   a   279(cont.)
.2 1
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2   a3    0.23

Solving for a2
2.917a2  0.8264a3  58.33
58.33  0.8264a3
a2 
2.917
58.33  0.8264 1.15

2.917
 19.67
 Back Substitution
144 12 1   a   279(cont.)
.2 1
 0 2.917 0.8264 a    58.33 
   2  
 0 0  0.2   a3    0.23

Solving for a1
144a1  12a2  a3  279.2
279.2  12a2  a3
a1 
144
279.2  12 19.67  1.15

144
 0.2917
Gaussian Elimination with Partial
Pivoting Solution

 25 5 1  a1  106.8 
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2

 a1  0.2917 
a    19.67 
 2  
 a3   1.15 
Gauss Elimination with
Partial Pivoting
Another Example
 Partial
Consider thePivoting: Example
system of equations
10 x1  7 x2  7
 3 x1  2.099 x2  6 x3  3.901
5 x1  x2  5 x3  6

In matrix form
 10  7 0  x1   7 
 3 2.099 6  x  3.901
   2 =  
 5  1 5  x3   6 

Solve using Gaussian Elimination with Partial Pivoting using five

significant digits with chopping
 Partial Pivoting: Example
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch
row1 with row1.

Performing Forward Elimination

 10  7 0  x1   7  10 7 0  x1   7 
 3 2.099 6  x   3.901

 5
 2   
 1 5  x3   6 
  0  0.001 6  x   6.001

 0 2.5
 2   
5  x3   2.5 
 Partial Pivoting: Example
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is
switched with row 3

Performing the row swap

10 7 0  x1   7  10 7 0  x1   7 
 0  0.001 6  x   6.001

 0 2.5
 2   
5  x3   2.5 
 0
 2.5 5   x    2.5 
 2   
 0  0.001 6  x3  6.001
 Partial Pivoting: Example
Forward Elimination: Step 2

Performing the Forward Elimination results in:

10  7 0   x1   7 
 0 2.5 5   x    2. 5 
  2   
 0 0 6.002  x3  6.002
 Partial
Back Substitution
Pivoting: Example
Solving the equations through back substitution

10  7 0   x1   7  6.002
x3  1
 0 2 .5 5   x    2.5  6.002
  2   
 0 0 6.002  x3  6.002 2.5  5 x3
x2   1
2.5

7  7 x 2  0 x3
x1  0
10
 Partial Pivoting: Example
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting

 x1   0   x1   0 
X  calculated   x2    1 X  exact   x 2    1
 x3   1   x3   1 
THE END