Indirect/Iterative Methods

The Jacobi Method

Gauss-Siedel Method
Successive Over Relaxation Method
The Jacobi Iterative Technique
The Jacobi Method:
Procedure
In this section we describe the general procedure for Jacobi iterative method

Step 01 Choose Initial Guess

𝑥10 = 𝑥20 = 𝑥30 =. . . . . . . . . = 𝑥𝑛0 = 𝟎
(if no better initial guess is available)

Step 02 Compute
𝑛
1
𝑥𝑖𝑟+1 = 𝑏𝑖 − ෍ 𝑎𝑖𝑗 𝑥𝑗𝑟 𝑖 = 1, 2,3, . . . 𝑛
𝑎𝑖𝑖
𝑗=1
𝑗≠𝑖

Step 03 Process (step 1 and 2) repeats till accurate result is achieved

Termination of the Processes:
i) Use a fixed number of iteration
ii) Use a predefined accuracy as
𝑥 𝑟+1 − 𝑥 𝑟 < 𝜀
The Jacobi Method

In this section we describe the Jacobi iterative method with an example

We want to solve the following  10 -1 2 0   x1   6 

linear system  x   25 
 -1 11 -1 3  2    
 2 -1 10 -1   x3    11
  
𝐴𝑥 = 𝑏  0 3 -1 8   4 
x

 15


𝑨 𝒙 𝒃
Example
solve the linear system

10 x1  x2  2 x3 6
 x1  11x2  x3  3x4  25
2 x1  x2  10 x3  x4  11
3 x2  x3  8 x4  15
The Jacobi Iterative Techniques

We want to solve the following linear system

Example
Keep the diagonal on the left hand side
solve the linear system

10 x1  x2  2 x3 6 10 x1  x2  2 x3 6
 x1  11x2  x3  3x4  25 11x2  x1  x3  3x4  25
2 x1  x2  10 x3  x4  11 10 x3  2 x1  x2  x4  11
3 x2  x3  8 x4  15  8 x4   3x2  x3  15
Jacobi Iterative Method
From the initial approx. 𝒙(𝟎) = (𝟎, 𝟎, 𝟎, 𝟎)𝑻 we 𝒙(𝟏) Change the coefficient of LHS to be one by
division

x1(1)  ( x2( 0 )  2 x3( 0 )  6) / 10 x1  ( x2  2 x3  6) / 10

x2  ( x1  x3  3x4  25) / 11
x (1)
( x ( 0)
x ( 0)
 3x ( 0)
 25) / 11
2 1 3 4
x3  (2 x1  x2  x4  11) / 10
x(1)
 ( 2 x
( 0)
 x ( 0)
 x ( 0)
 11) / 10
3 1 2 4
x4  (  3x2  x3  15) /( 8)
x (1)
4 (  3x ( 0)
2 x( 0)
3  15) /( 8)
The Jacobi Iterative Technique

From the initial approx. 𝒙(𝟎) = (𝟎, 𝟎, 𝟎, 𝟎)𝑻 we 𝒙(𝟏)

x1(1)  ( x2( 0 )  2 x3( 0 )  6) / 10  0  6 / 10   0.600

 0  25 / 11   2.272
x2(1)  ( x1( 0 )  x3( 0 )  3x4( 0 )  25) / 11 x ( 0)   x (1)    
 0  11 / 10   1.100 
x3(1)  ( 2 x1( 0 )  x2( 0 )  x4( 0 )  11) / 10       1.875 
 0   15 / 8   
x4(1)  (  3x2( 0 )  x3( 0 )  15) /( 8)

Jacobi Iterative Method

From the k-th approx. 𝒙(𝒌) = (𝟎, 𝟎, 𝟎, 𝟎)𝑻 we 𝒙(𝒌+𝟏)

x1( k 1)  ( x2( k )  2 x3( k )  6) / 10  0.600  1.047 

 2.272  1.715
( k 1) x (1)   x(2)  
x 2 ( x (k )
1 x (k )
3  3x (k )
4  25) / 11   1.100   0.805
   
x3( k 1)  ( 2 x1( k )  x2( k )  x4( k )  11) / 10   1.875   0.885
x4( k 1)  (  3x2( k )  x3( k )  15) /( 8)
The Jacobi Iterative Techniques

Jacobi Iterative Method

 0
x1( k 1)  ( x2( k )  2 x3( k )  6) / 10  0
x ( 0)  
x2( k 1)  ( x1( k )  x3( k )  3x4( k )  25) / 11  0
 
x3( k 1)  ( 2 x1( k )  x2( k )  x4( k )  11) / 10  0
x4( k 1)  (  3x2( k )  x3( k )  15) /( 8)

𝒌=𝟏 𝒌=𝟐 𝒌=𝟑 𝒌=𝟒 𝒌=𝟓

(𝒌)
𝒙𝟏
0.6000 1.0473 0.9326 1.0152 0.9890
(𝒌)
𝒙𝟐 2.2727 1.7159 2.0533 1.9537 2.0114
𝒙𝟑
(𝒌)
-1.1000 -0.8052 -1.0493 -0.9681 -1.0103
(𝒌)
𝒙𝟒 1.8750 0.8852 1.1309 0.9738 1.0214

𝒌=6 𝒌=7 𝒌=8 𝒌=9 𝒌 = 10

(𝒌)  1
𝒙𝟏 1.0032 0.9981 1.0006 0.9997 1.0001  2
x  
(𝒌)
𝒙𝟐 1.9922 2.0023 1.9987 2.0004 1.9998 *

𝒙𝟑
(𝒌)
-0.9945 -1.0020 -0.9990 -1.0004 -0.9998   1
 
(𝒌)
𝒙𝟒 0.9944 1.0036 0.9989 1.0006 0.9998  1
The Gauss-Siedel Iterative Technique
The Gauss Sediel Method:
Procedure
In this section we describe the general procedure for iterative method

Step 01 Choose Initial Guess

𝑥10 = 𝑥20 = 𝑥30 =. . . . . . . . . = 𝑥𝑛0 = 𝟎
(if no better initial guess is available)

Step 02 Compute
𝑖−1 𝑛
1
𝑥𝑖𝑟+1 = 𝑏𝑖 − ෍ 𝑎𝑖𝑗 𝑥𝑗𝑟+1 − ෍ 𝑎𝑖𝑗 𝑥𝑗𝑟 𝑖 = 1, 2,3, . . . 𝑛
𝑎𝑖𝑖
𝑗=1 𝑗=𝑖+1

Step 03 Process (step 1 and 2) repeats till accurate result is achieved

Termination of the Processes:
i) Use a fixed number of iteration
ii) Use a predefined accuracy as
𝑥 𝑟+1 − 𝑥 𝑟 < 𝜀
Gauss-Siedel Iterative Techniques

Gauss Seidel Method

From the k-th approx. 𝒙(𝒌) = (𝟎, 𝟎, 𝟎, 𝟎)𝑻 we 𝒙(𝒌+𝟏)
x1( k 1)  ( x2( k )  2 x3( k )  6) / 10
Note that in the Jacobi iteration one
does not use the most recently x2( k 1)  ( x1( k )  x3( k )  3x4( k )  25) / 11
available information.
x3( k 1)  ( 2 x1( k )  x2( k )  x4( k )  11) / 10
x4( k 1)  (  3x2( k )  x3( k )  15) /( 8)

From the k-th approx. 𝒙(𝒌) = (𝟎, 𝟎, 𝟎, 𝟎)𝑻 we 𝒙(𝒌+𝟏)

x1( k 1)  ( x2( k )  2 x3( k )  6) / 10

x2( k 1)  ( x1( k 1)  x3( k )  3x4( k )  25) / 11
x3( k 1)  ( 2 x1( k 1)  x2( k 1)  x4( k )  11) / 10
x4( k 1)  (  3x2( k 1)  x3( k 1)  15) /( 8)
𝒌=𝟏 𝒌=𝟐 𝒌=𝟑 𝒌=𝟒 𝒌=𝟓
(𝒌)
𝒙𝟏
Note that Jacobi’s method in this 0.6000 1.0302 1.0066 1.0009 1.0001
(𝒌)
𝒙𝟐
example required twice as many 2.3273 2.0369 2.0036 2.0003 2.0000
(𝒌)
iterations for the same accuracy. 𝒙𝟑 -0.9873 -1.0145 -1.0025 -1.0003 -1.0000
(𝒌)
𝒙𝟒 0.8789 0.9843 0.9984 0.9998 1.0000
The Jacobi and Gauss-Siedel Iterative Techniques

 a11  a1n   x1   b1 
Jacobi iteration for general n:           
    
for i  1 : n an1  ann   xn  bn 
 i 1 n 
( k 1)
x   bi   aij x j   aij x j 
(k ) (k )
i a
 j 1 j i 1  ii
end

Gauss-Seidel iteration for general n:

for i  1 : n
 i 1 n 
( k 1)
x   bi   aij x j   aij x j 
 ( k 1) (k )
i a
 j 1 j i 1  ii
end
The Successive Over Relaxation (SOR)
Method
The SOR Method:
Procedure
In this section we describe the general procedure for Jacobi iterative method

Step 01 Choose Initial Guess

𝑥10 = 𝑥20 = 𝑥30 =. . . . . . . . . = 𝑥𝑛0 = 𝟎
(if no better initial guess is available)

Step 02 Compute

Step 03 Process (step 1 and 2) repeats till accurate result is achieved

Termination of the Processes:
i) Use a fixed number of iteration
ii) Use a predefined accuracy as
𝑥 𝑟+1 − 𝑥 𝑟 < 𝜀
The Successive Over Relaxation Iterative Techniques
The Successive Over Relaxation Iterative Techniques
The Successive Over Relaxation Iterative Techniques
The Successive Over Relaxation Iterative Techniques
H.W The Jacobi and Gauss-Siedel Iterative Techniques
 Thank You !!!!

Have a Nice Weekend. . .