The Islamic University of Gaza

Faculty of Engineering
Civil Engineering Department

Numerical Analysis
ECIV 3306

Chapter 11

Special Matrices and Gauss-Siedel

 Symmetric Matrices
– Certain matrices have particular structures
that can be exploited to develop efficient
solution schemes.

• Symmetric matrix:
5 1 2 16 
If aij = aji  [A]nxn is a 1 3 
7 39 
symmetric matrix [ A]  
2 7 9 6
 
16 39 6 88 
 Cholesky Decomposition
 This method is suitable for only symmetric systems
where:
 l11 0 0
a ij  a ji and A A T [L ]  l 21 l 22 0
 
 l 31 l 32 l 33 

A  L * LT

a11 a12 a13   l11 0 0  l11 l 21 l 31 

a a a   l l 0  *  0 l 22 l 32 
 21 22 23   21 22   
a31 a32 a33   l 31 l 32 l 33   0 0 l 33 

3
 Cholesky Decomposition
i 1

l ijl ij lkjl kj

i 1
aakiki 
j 11
ll kiki   1,2, , k ,k1 1
j for i i 1,2,
for
lliiii
k k11

l kjkj
l kkkk  aakkkk  22
l
j j 11

L11  a11 L22  a22  L21

2

For 3x3 a21 a32  L21L31

matrix, L21  L32 
L11 L22
a31
L31  L33  a33  L232  L231
L11
Once the matrix L has been calculated, LT can easily be found and the
forward , backward substitution steps can be followed to find the
solution {X} for any right hand side {B}.
 Pseudocode for Cholesky’s LU
Decomposition algorithm (cont’d)
 Gauss-Siedel

• Iterative or approximate methods provide an

alternative to the elimination methods. The Gauss-
Seidel method is the most commonly used iterative
method.
• The system [A]{X}={B} is reshaped by solving the
first equation for x1, the second equation for x2,
and the third for x3, …and nth equation for xn. We
will limit ourselves to a 3x3 set of equations.
 Gauss-Siedel
a11x1  a12 x2  a13 x3  b1 b1  a12 x2  a13 x3
x1 
a11

a21x1  a22 x2  a23 x3  b2  x2 

b2  a21x1  a23 x3
a22
b3  a31x1  a32 x2
x1 
a31 x1  a32 x2  a33 x3  b3 a33

Now we can start the solution process by choosing

guesses for the x’s. A simple way to obtain initial
guesses is to assume that they are zero. These
zeros can be substituted into x1 equation to
calculate a new x1=b1/a11.
 Gauss-Siedel
• New x1 is substituted to calculate x2 and x3. The
procedure is repeated until the convergence
criterion is satisfied:

new
x x old
 a ,i  i
new
i
100%   s
x i
 Gauss Seidal-
Graphical Interpretation
 Jacobi iteration Method

An alternative approach, called Jacobi iteration,

utilizes a somewhat different technique. This
technique includes computing a set of new x’s on the
basis of a set of old x’s. Thus, as the new values are
generated, they are not immediately used but are
retained for the next iteration.
 Gauss-Siedel

The Gauss-Seidel method The Jacobi iteration method

Convergence Criterion for Gauss-Seidel Method
• The gauss-siedel method is similar to the technique of
fixed-point iteration.
• The Gauss-Seidel method has two fundamental
problems as any iterative method:
1. It is sometimes non-convergent, and
2. If it converges, converges very slowly.
• Sufficient conditions for convergence of two linear
equations, u(x,y) and v(x,y) are:
u u
 1
x y
v v
 1
x y
 Convergence Criterion for Gauss-Seidel
Method (cont’d)
• Similarly, in case of two simultaneous equations, the
Gauss-Seidel algorithm can be expressed as:
b1 a12
u (x 1 , x 2 )   x2
a11 a11
b 2 a21
v (x 1 , x 2 )   x1
a22 a22
u u a12
0 
x 1 x 2 a11
v a21 v
 0
x 1 a22 x 2
 Convergence Criterion for Gauss-Seidel
Method (cont’d)
Substitution into convergence criterion of two linear
equations yield: a12 a21
 1, 1
a11 a22
In other words, the absolute values of the slopes must be
less than unity for convergence:
a11  a12 For n equations
n

a22  a21 aii  a

j 1
i, j

j i

That is, the diagonal element must be greater than the sum
of off-diagonal element for each row.
 Gauss-Siedel Method- Example 1
7.85  0.1x2  0.2 x3
 3  0.1 0.2   x1   7.85  x1  3
0.1       19.3  0.1x1  0.3 x3
 7  0.3  x2    19.3  x2 
7
0.3  0.2 10   x3   71.4  x3  71.4  0.3 x1  0.2 x2
10
• Guess x1, x2, x3= zero for the first guess
Iter. x1 x2 x3 |a,1|(%) |a,2| (%) |a,3| (%)
0 0 0 0 - - -
1 2.6167 -2.7945 7.005610 100 100 100
2 2.990557 -2.499625 7.000291 12.5 11.8 0.076
 Improvement of Convergence Using
Relaxation
x i
new
  x i
new
 1     x old
i

• Where  is a weighting factor that is assigned a

value between [0, 2]
• If  = 1 the method is unmodified.
• If  is between 0 and 1 (under relaxation) this is
employed to make a non convergent system to
converge.
• If  is between 1 and 2 (over relaxation) this is
employed to accelerate the convergence.
 Gauss-Siedel Method- Example 2

The system can be re-arranged to the following

6x1-x2-x3 = 3
6x1+9x2+x3=40
-3x1+x2+12x3=50
 Gauss-Siedel Method- Example 2
3  x2  x3
x1 
6
40  6 x1  x3
x2 
9
50  3 x1  x2
x3 
12
Without Relaxation
Gauss-Siedel Method- Example 2