2.

GAUSS SEIDEL METHOD Once again we consider the system

Ax = y .. (I)
(k ) (k ) (k ) In the Jacobi scheme we used the values of x2 , x3 ,L , xn obtained in the kth iteration, in place of x2, x3, .., xn in the first equation,

a11 x1 + a12 x2 + ..... + a1n xn = y1

to calculate x1( k +1) from

(k ) (k ) (k ) a11 x1( k +1) = a12 x2 a13 x3 ..... a1n xn + y1

Similarly,
(k ) 1 (k ) 2

in
(k ) i 1

the
(k ) i +1

ith
(k ) n

equation,

( i = 2,3,L , n ) ,

we

used

the

values,

x , x ,L , x , x ,L , x from

, in place of x1 , x2 ,L , xi 1 , xi +1 ,L , xn to calculate xi( k +1)

(k ) k) (k ) (k ) aii xi( k +1) = ai1 x1( k ) ai 2 x2 ...... ai ,i 1 xi( 1 ai , i +1 xi +1 ..... ain xn + yi LL (*)

What Gauss Seidel suggests is that having obtained x1( k +1) from the first
( k +1) equation use this value for x1 in the second equation to calculate x2 from

( k +1) (k ) (k ) a22 x2 = a21 x1( k +1) a23 x3 ...... a2 n xn + y2

( k +1) ( k +1) and use these values of x1( k +1) , x2 , in the 3rd equation to calculate x3 , and so k +1) (k ) k) on. Thus in the equation (*) use x1( k +1) , L , xi( in place of x1( k ) , x2 , L , xi( 1 1 to get

the following modification of the i -th equation to calculate xi( k +1) :

( k +1) k +1) k) (k ) (k ) aii xi( k +1) = ai1 x1( k +1) ai 2 x2 ...... ai ,i 1 xi( ai ,i +1 xi(+ 1 1 ai ,i + 2 xi + 2 ..... ain xn + yi

In matrix notation we can write this as,

Dx ( k +1) = Lx ( k +1) Ux ( k ) + y
which can be rewritten as,

(D + L )x (k +1) = Ux (k ) + y , and hence

x (k +1 ) = (D + L ) Ux k + (D + L ) y
1 1

65

Thus we get the Gauss Seidel iteration scheme as, x(0) initial guess

x (k +1) = Gx ( k ) + y
where, G = -(D+L)-1U

..(II)

is the Gauss Seidel iteration matrix, and

= ( D + L ) 1 y y
The scheme converges if and only if G
sp

< 1.

Of course, the scheme will

converge if G < 1 in some matrix norm. But some matrix norm, G 1 does not mean that the scheme will diverge. The acid test for convergence is G We shall now consider some examples. Example 3: Let us consider the system x1 + 2x2 2x3 = 1 x1 + x2 + x3 = 0 2x1 + 2x2 + x3 = 0
sp

< 1.

considered in example 1 on page 59; and for which the Jacobi scheme gave the exact solution in the 3rd iteration. (see page 60). We shall now try to apply the Gauss Seidel scheme for this system. We have,

1 A = 1 2

2 1 2

2 1 ; 1
0 1 2 0 0 ; 1

1 y = 0 0

1 D + L = 1 2

0 U = 0 0

2 0 0

2 1 0
66

(D + L )
Thus,

0 1 = 1 1 0 2

0 0 1

G = ( D + L)

0 0 0 2 2 1 U = 1 1 0 0 0 1 0 2 1 0 0 0

From above we get the Gauss Seidel iteration matrix as,

0 G = 0 0

2 2 0

2 3 2

Since G is triangular we get its eigenvalues immediately, as its diagonal entries. Thus 1 = 0, 2 = 2, 3 = 2 are the three eigenvalues. Therefore,

sp

= 2 >1

Hence the Gauss Seidel scheme for this system will not converge. Thus for this system the Jacobi scheme converges so rapidly giving the exact solution in the third iteration itself whereas the Gauss Seidel scheme does not converge. Example 4: Consider the system

1 1 x2 x3 = 1 2 2 x1 + x 2 + x 3 = 0 x1 1 1 x1 x 2 + x 3 = 0 2 2

Let us apply the Gauss Seidel scheme to this system. We have,

67

 1 A= 1 1 2

1 1 1

2
0 1 1

1 2 1 ; 1
0 0 1

1 y = 0 0

1 D+L= 1 1 2

(D + L)

1 = 1 0

0 1 1 2

0 0 , 1

0 U = 0 0
Thus,

2 0 0

1 2 1 . 0

1 1 G = ( D + L ) U = 1 0

0 1 1 2

0 0 00 10

2 0

1 2 1 0

0 G = 0 0

2 12 0

2 3 2 ............(*) 1 2 1

is the Gauss Seidel matrix for this sytem. The Gauss Seidel scheme is

x (k +1 ) = Gx ( k ) + y x (0 ) 0 = 0 0

68

where

= (D + L) y

1 y = 1 0

0 1 1 2

01 1 0 0 = 1 ; 10 0

and where G is given (*). Notice that G is upper triangular and hence we readily get the eigenvalues of G as its diagonal entries. Thus the eigenvalues of G are, 1 = 0, 2 = -1/2, 3 = 1/2. Hence G will converge. Let us now carry out a few steps of the Gauss Seidel iteration, since we have now been assured of convergence. (We shall first do some exact calculations).
sp

1 < 1 . Hence in this example the Gauss Seidel scheme 2

x (1 )

0 1 1 = G 0 + 1 = 1 = Gx (0 ) + y 0 0 0 1 1 = G 1 + 1 +y 0 0
0 = 0 0 2 1 2 0

(2 )

= Gx

(1 )

2 3 2 1 2

1 1 1 + 1 0 0

1 1 2 = 1 1 2 0

69

x (3 ) = Gx ( 2 )

1 1 + 1 2 2 2 1 1 + y = 1 2 + 22 0

If we continue this process we get

k 1 ( ) 1 1 1 1 2 + 2 ..... + k 1 2 2 k 1 ( ) 1 1 1 ..... + = 1 2 + 22 2 k 1 0

x (k )

Clearly,

x(

k)

1 1 + 1 2 1 3 + 1 4 ..... 2 2 2 2 1 1 2 + 1 2 1 3 ..... 2 2 0

and by summing up the geometric series we get,

(k )

2 3 2 3 0

which is the exact solution. Of course, here we knew a priori that the sequence is going to sum up neatly for each component and so we did exact calculation. If we had not noticed this we still would have carried out the computations as follows:

x (1 ) = Gx (0 )

1 = 1 +y 0

as before

70

(2 )

= Gx

(1)

0 .5 = 0 .5 +y 0 0 .625 = 0 .625 +y 0 0 .6875 = 0 .6875 +y 0

(3 )

= Gx

(2 )

(4 )

= Gx

(3 )

x (5 )

0 .65625 = 0 .65625 ; = Gx (4 ) + y 0

x (5 ) x ( 4 )

= 0 .03125

x (6 )

0 .671875 = 0 .671875 ; x (6 ) x (5 ) = Gx (5 ) + y 0 0 . 664062 = 0 . 664062 0 0 . 667969 = 0 . 667969 0 0 . 666016 = 0 . 666016 0 0 . 666504 = 0 . 666504 0 ; ; ; ;

= 0 .025625

(7 )

x ( 7 ) x (6 )

= 0 .007813

x (8 )

x (8 ) x (7 )

= 0 .003907

(9 )

x (9 ) x (8 )

= 0 .001953

(10 )

x (10 ) x (9 )

= 0 .000488

71

Since now error is < 10-3 we may stop here and take x(10) as our solution for the system. Or we may improve our accuracy by doing more iterations, to get,

x (11 )

0 . 666748 = 0 . 666748 0 0 . 666656 = 0 . 666656 0

0 . 666626 (12 ) = 0 . 666626 ; x 0

0 . 666687 (13 ) = 0 . 666687 ; x 0

(14 )

x (14 ) x (13 )

= 0 .000031 < 10 4

and hence we can take x(14) as our solution within error 10-4. Let us now try to apply the Jacobi scheme for this system. We have

1 A= 1 1 2 0 J = 1 1 2

1 1 1 1

1 2 1 ; 1 1 2 1 0

and therefore,

2 0 1 2

We have the characteristic polynomial of J as

I J = + 1
1 2

2 2

1 2 1 + 1 = + 2 + 1 2 2

Thus the eigenvalues of J are

1 = ; 2 =

1 2

1 1 + i 15 ; 3 = i 15 4 4 2 2
1 15 + = 16 = 2 4 4 4

1 1 = ; 2 = 3 = 2

72

sp

= 2 which is >1. Thus the Jacobi scheme for this system will not

converge. Thus, in example 3 we had a system for which the Jacobi scheme converged but Gauss Seidel scheme did not converge; whereas in example 4 above we have a system for which the Jacobi scheme does not converge, but the Gauss Seidel scheme converges. Thus, these two examples demonstrate that, in general, it is not correct to say that one scheme is better than the other. Let us now consider another example. Example 5: 2x1 x2 =y1 -x1 + 2x2 x3 = y2 -x2 + 2x3 x4 =y3 -x3 + 2x4 = y4 Here

0 2 1 0 1 2 1 0 A= , 0 1 2 1 0 0 1 2
is a symmetric tridiagonal matrix. The Jacobi matrix for this scheme is

J =

0 1 0 0 2

1 0 1 0

0 1 2

0 1 2

0 0 1 2 0

The characteristic equation is, 16 4 - 12 2 + 1 = 0 Set 2 = (CJ)

73

Therefore 162 - 12 + 1 = 0 (CJ1)

is the square root of the roots of (CJ1).

Thus the eigenvalues of J are 0.3090; 0.8090. Hence

sp

= 0.8090 ; and the Jacobi scheme will converge.

The Gauss Seidel matrix for the system is found as follows:

0 0 2 1 2 0 (D + L ) = 0 1 2 0 0 1

0 0 0 2

0 0 U = 0 0

1 0 0 0
=

0 1 0 0
1 1 1 1 2 4 8

0 0 1 0
0 1 1 1 2 4 8 0 0 1 1 2 4 0 0 0 1 2

(D + L )

16

74

 1 G = (D + L ) U = 1 0 0 0 2 0 1 1 0 4 2 = 0 1 1 1 8 4 2 1 1 1 0 16 8 4

1 1 1 1

2 4 8

0 1 1 1 2 4 8

0 0 1 1 2 4

16

0 0 0 1 2

0 0 0 0

1 0 0 0

0 1 0 0

0 0 1 0

The characteristic equation of G is

I G = 0 , which becomes in this case

164 123 + 2 = 0.......... .......... (C G )

This can be factored as

2 (162 12 + 1) = 0
Thus the eigenvalues of G are roots of

2 = 0 ; and
162 - 12 + 1 = 0 .(CG1) Thus one of the eigenvalues of G is 0 (repeated twice), and two eigenvalues of G are roots of (CG1). Notice that roots of (CG1) are same as those of (CJ1). Thus nonzero eigenvalues of G are squares of eigenvalues of J. the nonzero eigenvalues of G are, 0.0955, 0.6545. Thus,
G
sp

= 0.6545 < 1

Thus the Gauss Seidel scheme also converges. Observe that

75

sp

= J

2 sp

sp

< J

sp

Thus the Gauss Seidel scheme converges faster than the Jacobi scheme. In many class of problems where both schemes converge it is the Gauss Seidel scheme that converges faster. We shall not go into any further details of this aspect.

76