ITERATIVE TECHNIQUES

FOR SOLVING
NON-LINEAR SYSTEMS
(AND LINEAR SYSTEMS)
 Jacobi Iterative Technique

Consider the following set of equations.

10 x1  x2  2 x3 6
 x1  11 x2  x3  3x4  25
2 x1  x2  10 x3  x4  11
3 x2  x3  8 x4  15
Convert the set Ax = b in the form of x = Tx + c.

1 1 3
x1  x2  x3 
10 5 5
1 1 3 25
x2  x1  x3  x4 
11 11 11 11
1 1 1 11
x3   x1  x2  x4 
5 10 10 10
3 1 15
x4   x2  x3 
8 8 8
Start with an initial approximation of:
(0) (0) (0) (0)
x1  0, x 2  0, x3  0 and x 4  0.

(1) 1 (0) 1 (0) 3

x1  x2  x3 
10 5 5
(1) 1 (0) 1 (0) 3 (0) 25
x2  x1  x3  x4 
11 11 11 11
(1) 1 (0) 1 (0) 1 (0) 11
x3   x1  x2  x4 
5 10 10 10
(1) 3 (0) 1 (0) 15
x4   x2  x3 
8 8 8
 (1) 1 1 3
x1  (0)  (0) 
10 5 5
(1) 1 1 3 25
x2  (0)  (0)  (0) 
11 11 11 11
(1) 1 1 1 11
x3   (0)  (0)  (0) 
5 10 10 10
(1) 3 1 15
x4   (0)  (0) 
8 8 8

(1) (1)
x1  0.6000, x 2  2.2727,
(1) (1)
x3  1.1000 and x 4  1.8750
 (2) 1 (1) 1 (1) 3
x1  x2  x3 
10 5 5
( 2) 1 (1) 1 (1) 3 (1) 25
x2  x1  x3  x4 
11 11 11 11
( 2) 1 (1) 1 (1) 1 (1) 11
x3   x1  x2  x4 
5 10 10 10
( 2) 3 (1) 1 (1) 15
x4   x2  x3 
8 8 8
 (k) 1 ( k 1 ) 1 ( k 1 ) 3
x1  x2  x3 
10 5 5
(k) 1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
x2  x1  x3  x4 
11 11 11 11
(k) 1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
x3   x1  x2  x4 
5 10 10 10
(k) 3 ( k 1 ) 1 ( k 1 ) 15
x4   x2  x3 
8 8 8
Results of Jacobi Iteration:

k 0 1 2 3
(k ) 0.0000 0.6000 1.0473 0.9326
x1
(k ) 0.0000 2.2727 1.7159 2.0530
x2
(k ) 0.0000 -1.1000 -0.8052 -1.0493
x3
(k ) 0.0000 1.8750 0.8852 1.1309
x4
 Gauss-Seidel Iterative Technique

Consider the following set of equations.

10 x1  x2  2 x3 6
 x1  11 x2  x3  3x4  25
2 x1  x2  10 x3  x4  11
3 x2  x3  8 x4  15
 (k) 1 ( k 1 ) 1 ( k 1 ) 3
x1  x2  x3 
10 5 5
(k ) 1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
x2  x1  x3  x4 
11 11 11 11
(k ) 1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
x3   x1  x2  x4 
5 10 10 10
(k ) 3 ( k 1 ) 1 ( k 1 ) 15
x4   x2  x3 
8 8 8
 (k) 1 ( k 1 ) 1 ( k 1 ) 3
x1  x2  x3 
10 5 5
(k) 1 (k) 1 ( k 1 ) 3 ( k 1 ) 25
x2  x1  x3  x4 
11 11 11 11
(k) 1 (k) 1 (k) 1 ( k 1 ) 11
x3   x1  x2  x4 
5 10 10 10
(k) 3 (k) 1 (k) 15
x4   x2  x3 
8 8 8
Results of Gauss-Seidel Iteration:
(Blue numbers are for Jacobi iterations.)
k 0 1 2 3
(k ) 0.0000 0.6000 1.0300 1.0065
x1
0.6000 1.0473 0.9326
(k ) 0.0000 2.3272 2.0370 2.0036
x2 2.2727 1.7159 2.0530
(k ) 0.0000 -0.9873 -1.0140 -1.0025
x3
-1.1000 -0.8052 -1.0493
(k ) 0.0000 0.8789 0.9844 0.9983
x4
1.8750 0.8852 1.1309
The solution is: x1= 1, x2 = 2, x3 = -1, x4 = 1

It required 15 iterations for Jacobi method

and 7 iterations for Gauss-Seidel method
to arrive at the solution with a tolerance of
0.00001.
 Newton’s Iterative Technique
Given:
f1 x1 , x 2 , x3 ,......, x n   0
f 2 x1 , x 2 , x3 ,......, x n   0
f 3 x1 , x 2 , x3 ,......, x n   0
....
f n x1 , x 2 , x3 ,......, x n   0

   
1
( k 1) ( k 1) ( k 1)
X (k )
X  J(X ) F X
  f1 x1 , x 2 , x3 ,......, x n 
 f x , x , x ,......, x 
 2 1 2 3 n 

F ( X )   f 3 x1 , x 2 , x3 ,......, x n 
 
 .... 
 f n x1 , x 2 , x3 ,......, x n 
Jacobian Matrix:

  f1  f1  f1 
 x ...
x 2 x n 
 1 
 f 2 f 2
...
f 2 
J ( X )   x1 x 2 x n 
 ... ... ... ... 
 f f n f n 
 n ... 
 x1 x 2 x n 
Consider the following set of non-linear equation:

x  x  x 1  0
2
1
2
2
2
3

1
x 2
1 x  2
3
4
x1  x 2  4 x3  0
2 2

f1 x1 , x 2 , x3   x12  x 22  x32  1

1
f 2 x1 , x 2 , x3   x 2
1 x 
2
3
4
f 3 x1 , x 2 , x3   x12  x 22  4 x3
  f1 f1  f1 
 
 x1 x 2 x 3   2 x
1 2 x2 2 x3 
f 2 f 2 f 2  
J(X )     2 x1 0 2 x3 
 x x 2 x 3 
 f 1   2 x1 2 x2  4 
 3 f 3 f 3 
 x1 x 2 x3 

Make an initial guess:

1
X (0) 
 1 

1
  2 2 2 2
 
F ( X )  1.75 J ( X ( 0 ) )   2 0 2
(0)

  2  2 2  4

   
1
X (1)
X (0)
 J(X (0)
) F X (0)

1
1  2 2 2   2 
X (1)   
 1   2 0 2   1.75
 
1  2 2  4   2 
 0.79167
X (1)   0.87500
 0.33333

   
1
X ( 2)
X (1)
 J(X ) F X
(1) (1)

0.50348
F ( X (1) )  0.48785
0.05905
 1.58334 1.75000 0.66666

J ( X )  1.58334
(1)
0 0.66666 
1.58334 1.75000  4

   
1
X ( 2)
X (1)
 J(X ) F X
(1) (1)

1
0.79167  1.58334 1.75000 0.66666  0.50348
X ( 2)  0.87500   1.58334 0 0.66666  0.48785
 0.33333 1.58334 1.75000  4 0.05905
  0.52365  0.44733
X ( 2)  0.86607 X ( 3)   0.86603
 0.23810 0.23607

 0.44081
X ( 4) 
  0.86603
0.23607
Example of Gauss-Seidel Iteration

x13  x12 x 2  x1 x3  6  0
e  e  x3  0
x1 x2

x  2 x1 x3  4
2
2

x 1 
3
 2
x 1  x 2 
 x 1 x 3  6
2
x2 4

x 2  ln x 3  e
x1
 x 3 
2 x 1
x 1  1 x 2  1 x 3  1
x 1  1.81712 x 2  0.17733 x 3  1.09199
x 1  1.94859 x 2  0.0518 x 3  1.0257
x 1  1.98336 x 2  0.11868 x 3  1.00484
x 1  1.9597 x 2  0.14625 x 3  1.01511
x 1  1.95112 x 2  0.13583 x 3  1.02033