Numerical Methods

Solving Linear Algebraic Equations:

Iterative Methods
by

Norhayati Rosli, Rozieana Khairuddin & Nurfatihah Mohd Hanafi

Faculty of Industrial Sciences & Technology
norhayati@ump.edu.my, rozieana@ump.edu.my

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 Description

AIMS

This chapter is aimed to solve small numbers of linear algebraic equations by using
iterative methods involving Jacobi method and Gauss-Seidel method

EXPECTED OUTCOMES

1. Students should be able to solve linear algebraic equations by using Jacobi and
Gauss-Seidel methods

REFERENCES

1. Norhayati Rosli, Nadirah Mohd Nasir, Mohd Zuki Salleh, Rozieana

Khairuddin, Nurfatihah Mohamad Hanafi, Noraziah Adzhar. Numerical
Methods, Second Edition, UMP, 2017 (Internal use)
2. Chapra, C. S. & Canale, R. P. Numerical Methods for Engineers, Sixth
Edition, McGraw–Hill, 2010.
Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 Content

1 Introduction
2 Jacobi Method
3 Gauss-Seidel Method

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 INTRODUCTION

Solving Linear Algebraic Equations: Iterative Methods

Two Types
of Methods

Gauss-Seidel Method 2 1 Jacobi Method

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD
Jacobi Methods Procedures

Rearrange the equation to make the system diagonally

STEP 1
dominant

STEP 2 Write the equation in an explicit form

The calculation process starts by assuming initial values for

STEP 3
the unknown for the first iteration

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Jacobi Methods Procedures (Cont.)

Rearrange the equation to make the system diagonally

Step 1
dominant. For a system of 𝑛 equations, 𝐀𝐱 = 𝐛 a sufficient
condition for convergence is that for every row of matrix, the
absolute of the diagonal element is greater than or equal to
the sum of the absolute values of the diagonal elements in
that row

n
aii  
j 1, j  i
aij

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Jacobi Methods Procedures (Cont.)

Write the equation in an explicit form

Step 2
Each unknown is written in terms of the other unknowns.
For 3 𝑥 3 matrix
k  k 
 k 1 b  a x  a x
x1 1

12 2 13 3

a11
 a11 a12 a13   x1   b1 
b2  a21 x1   a23 x3
k
a a23   x2   b2 
k
x2
k 1
 21 a22 
a22
 a31 a32 a33   x3   b3 
b3  a31 x1   a32 x2
k k
x3
k 1

a33
Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Jacobi Methods Procedures (Cont.)

The calculation process starts by assuming initial values for the

Step 3 unknowns for the first iteration. Start the calculation by using the
given initial condition.

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Example 1
Use the Jacobi method to obtain the solution for.
8 x1  x2  x3  10
2 x1  x2  9 x3  2
x1  7 x2  2 x3  4
Use 4 decimal places in your computation and let xi     0, 0, 0  . Compute up to two
0 T

iterations and calculate the approximate percent relative error for each iteration.
Solution

Rearrange the equation to make the system diagonally dominant.

Step 1
8 1 1   x1  10  8 1 1   x1  10 
 2 1 9   x    2  R2  R3 1 7 2   x    4 
  2     2  
1 7 2   x3   4   2 1 9   x3   2 
Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Solution (Cont.)

Step 2 Write the equations in explicit form.

10  x2   x3
k k
 k 1
x1 
8
4  x1   2 x3
k k
 k 1
x2 
7
2  2 x1   x2
k k
 k 1
x3 
9

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Solution (Cont.)

Start calculation with the given initial condition.

Step 3
iteration, k  0, xi     0, 0, 0 
0 T
1st

10  x2   x3  10  0  0
0 0
1
x1    1.25
8 8
4  x1   2 x3  4  0  0
0 0
1
x2    0.5714
7 7
2  2 x1   x2  2  0  0
0 0
1
x3    0.2222
9 9

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Solution (Cont.)
 1 1 1 
The approximate percent relative error for x1 , x2 , and x3 are
computed as follows:

1.25  0
a   100%  100%
x1
1.25
0.5714  0
a   100%  100%
x2
0.5714
0.2222  0
a   100%  100%
x3
0.2222

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Solution (Cont.)

2nd iteration, k  1, xi    1.2500, 0.5714, 0.2222 

1 T

10  x2   x3  10   0.5714    0.2222 

1 1
x1
2
   1.3492
8 8
4  x1   2 x3  4  1.25  0.4444
1 1
x2
2
   0.4563
7 7
2  2 x1   x2  2  2.5  0.5714
1 1
x3
2
   0.4365
9 9

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 JACOBI METHOD (Cont.)
Solution (Cont.)

 2  2  2
The approximate percent relative error for x1 , x2 , and x3 are computed as
follows:

1.3492  1.25
a   100%  7.35%
x1
1.3492
0.4563  0.5714
a   100%  25.22%
x2
0.4563
0.4365  0.2222
a   100%  49.10%
x3
0.4365

Therefore after two iterations, 𝑥1 = 1.3492, 𝑥2 = −0.4563, 𝑥3 = −0.4365.

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
GAUSS-SEIDEL METHOD (Cont.)
Gauss Seidel Methods Procedures
Rearrange the equation to make the system diagonally
STEP 1
dominant

STEP 2 Write the equation in an explicit form

b1  a12 x2   a13 x3
k k
 k 1
x 1 
a11
b2  a21 x1
k 1
 a23 x3
k
x2
k 1

a22
b3  a31 x1
k 1
 a32 x2
k 1
x3
k 1

a33
The calculation process starts by assuming initial values for
STEP 3
the unknown for the first iteration
Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 GAUSS-SEIDEL METHOD (Cont.)
Example 2
Use the Gauss-Seidel method to obtain the solution for.
8 x1  x2  x3  10
2 x1  x2  9 x3  2
x1  7 x2  2 x3  4
Use 4 decimal places in your computation and let xi     0, 0, 0  . Compute up to two
0 T

iterations and calculate the approximate percent relative error for each iteration and
compare the solution with the results obtain in Example 1.
Solution
Rearrange the equation to make the system diagonally dominant.
Step 1
8 1 1   x1  10  8 1 1   x1  10 
 2 1 9   x    2  R2  R3 1 7 2   x    4 
  2     2  
1 7 2   x3   4   2 1 9   x3   2 
Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 GAUSS-SEIDEL METHOD (Cont.)
Solution (Cont.)

Step 2 Write the equations in explicit form.

10  x2   x3
k k
x1
k 1

8
4  x1
k 1
 2 x3
k
x2
k 1

7
2  2 x1
k 1
 x2
k 1
x3
k 1

9

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 GAUSS-SEIDEL METHOD (Cont.)
Solution (Cont.)

Start calculation with the given initial condition.

Step 3
1st iteration, k  0, xi     0, 0, 0 
0 T

10  x2   x3  10  0  0
0 0
1
x1    1.25
8 8
4  x1   2 x3  4  1.25  0
1 0
1
x2    0.3929
7 7
2  2 x1   x2  2  2 1.25   0.3929
1 1
x3     0.4563
1

9 9

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 GAUSS-SEIDEL METHOD (Cont.)
Solution (Cont.)

1 1 1 
The approximate percent relative error for x1 , x2 , and x3 are
computed as follows:

1.25  0
a   100%  100%
x1
1.25
0.3929  0
a   100%  100%
x2
0.3929
0.4563  0
a   100%  100%
x3
0.4563

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 GAUSS-SEIDEL METHOD (Cont.)
Solution (Cont.)

2nd iteration, k  1, xi    1.2500, 0.3929, 0.4563 

1 T


 2  10  x2  x3
1 1 
10   0.3929    0.4563
x1    1.3562
8 8
4  x  2
 2 x 1
4  1.3562  2  0.4563
x2     0.5081
2 1 3

7 7
 2  2 x  2
 x  2 2  2 1.3562   0.5081
x3     0.4671
2 1 2

9 9

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 GAUSS-SEIDEL METHOD (Cont.)
Solution (Cont.)

 2  2  2
The approximate percent relative error for x1 , x2 , and x3 are computed as
follows:

1.3562  1.25
a   100%  7.83%
x1
1.3562
0.5081  0.3929
a   100%  22.67%
x2
0.5081
0.4671  0.4563
a   100%  2.31%
x3
0.4671

Therefore after two iterations, 𝑥1 = 1.3562, 𝑥2 = −0.5081, 𝑥3 = −0.4671.

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 GAUSS-SEIDEL METHOD (Cont.)
Solution (Cont.)
Comparison between Gauss-Seidel and Jacobi methods of this example is
presented below:
Vari Jacobi 𝜀𝑎𝐽𝑎𝑐𝑜𝑏𝑖 (%) Gauss-Seidel 𝜀𝑎𝐺𝑎𝑢𝑠𝑠−𝑆𝑒𝑑𝑒𝑙 (%) Exact
able
Methods Method Solution
𝑥1 1.3492 7.35 1.3562 7.83 1.3725
𝑥2 −0.4563 25.22 -0.5081 22.67 -0.5098
𝑥3 -0.4365 49.10 -0.4671 2.31 -0.4706

The solution that is obtained via Gauss-Seidel method converge faster compare
to Jacobi Method.

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 Conclusion

Method Formula
b1  a12 x2   a13 x3
k
Jacobi  k 1
k
x1 
a11
b2  a21 x1   a23 x3
k k
 k 1
x2 
a22
b3  a31 x1   a32 x2
k k
 k 1
x3 
a33

b1  a12 x2   a13 x3

Gauss-Seidel k k
 k 1
x1 
a11
b2  a21 x1
k 1
 a23 x3
k
 k 1
x2 
a22
b3  a31 x1
k 1
 a32 x2
k 1
 k 1
x3 
a33
Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449
 Author Information
Rozieana Binti Khairuddin,
Norhayati Binti Rosli, Lecturer,
Senior Lecturer, Faculty of Industrial Sciences & Technology (FIST),
Applied & Industrial Mathematics Research Universiti Malaysia Pahang,
Group, 26300 Gambang, Pahang.
Faculty of Industrial Sciences & Technology UMPIR ID: 3481
(FIST), Google
Universiti Malaysia Pahang, Scholars: https://scholar.google.com/citations?user
26300 Gambang, Pahang. =o_8-p0UAAAAJ&hl=en
SCOPUS ID: 36603244300 e-mail: rozieana@ump.edu.my
UMPIR ID: 3449
Google
Scholars: https://scholar.google.com/citations Nurfatihah Binti Mohd Hanafi
?user=SLoPW9oAAAAJ&hl=en e-mail: nurfatihah_hanafi@yahoo.com
e-mail: norhayati@ump.edu.my

Numerical Methods
by Norhayati Rosli
http://ocw.ump.edu.my/course/view.php?id=449