ES-361

Computing Methods in
Engineering

by Murat Buyuk, PhD

 Part 1

SOLUTION OF LINEAR
EQUATIONS

1.1 Scope
The equation in which the degree of the variable is one (largest degree) is called
a linear equation, and the equation with more than one degree of the variables
(the largest degree) is called a system of equations or a set of equations. For
example, containing unknowns ( x1 , x 2 ,…, x n ), the system of linear equations
consisting of n linear equations can be written in general form as.
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
a21 x1 + a22 x2 + a23 x3 + ... + a2 n xn = b2
a31 x1 + a32 x2 + a33 x3 + ... + a3n xn = b3 (1.1)
...
an1 x1 + an 2 x2 + an3 x3 + ... + ann xn = bn

Where, a ij constants represent the coefficients of the variables in the linear

equations. In order to solve the linear system of equations, there must be as many
independent equations as there are unknowns.
In real applications, the expression of very few engineering problems (such as the
extension of springs connected in series or parallel under force, the static balance
of truss systems) is expressed with a system of direct linear equations. We
encounter systems of linear equations in the numerical solution of engineering
problems, which are usually expressed in differential or integral equations.
Problems that are difficult or difficult to solve analytically can be defined as Finite
Elements, Finite Differences etc. as shown in Figure 1.1. They are solved by
transforming them into linear or nonlinear algebraic equation system by numerical

Methods
Êngineering Problems Numerical- Equation System
Computational Methods
-Differential Equation -Finite Difference -Linear Equation System
-Finete Element -Non-linear Equation
-Finite Volume System
-etc.
-Interal Equation
Figure 1.1: Solution of differential and integral equations by converting them to
linear or nonlinear system of equations.

Below are the differential equations of some problems in the field of mechanics
and heat conduction and their equivalent forms expressed as a system of linear
equations for solution.
Beam Bending:
The bending differential equation and boundary conditions (both ends simply
supported) can be expressed as follows for a beam of width b, height h, and length
L, which has a modulus of elasticity E shown in Figure 1.2, under the distributed
d 4 w( x) bh 3
load q:Differential Equation: EI = q ( x) (I = )
dx 4 12
d 2w
Boundary Conditions: w(0) = 0 , M (0) = EI 2 (0) = 0 , w( L) = 0 ,
dx
d 2w
M ( L) = EI ( L) = 0
dx 2

Figure 1.2: Beam Bending.

The deflection values of the beam can be obtained for the nodal points in Figure
1.3 with the Finite Difference method, one of the numerical methods oftenly
utilized.

Figure 1.3: Discretization of the beam for the solution with the Finite Difference
method.
The unknown deflection values of the discretized beam as above can be expressed
in matrix form as a system of linear equations as follows
 q3 (x )4 
 
1 − 4 6 − 4 1 0 0  w1   EI 
0 1 − 4 6 − 4 1 0 w   q 4 (x )4 
  2   
0 0 1 − 4 6 − 4 1  w3   EI 4 
     q5 (x ) 
 0 1 0 0 0 0 0  w4  =  EI 
1 − 2 1 0 0 0 0  w5   0 
    
0 0 0 0 0 1 0 w6   0 
0 0    
 0 0 1 − 2 1 w7   0 
 0 

Heat Transfer:
The heat transfer (conduction) of a one-dimensional rod shown in Figure 1.4 can
d 2T
be expressed as the differential equation − k =q.
dx 2

Figure 1.4: One dimensional heat transfer problem.

The temperature solution of the bar can be searched by the Finite Difference
method at the nodal points in Figure 1.5.

Figure 1.5: Discretization of one-dimensional heat conduction problem for solution

by Finite Difference method.

In this case, the unknown temperature values of the bar can be expressed in
matrix form as a system of linear equations as follows
− 2 1 0 0  T1  − Ta 
 
 1 −2 1
 0  T2  0 
 = 
0 1 − 2 1  T3  0 
 
0 0 1 − 2 T4  − Tb 

The above examples can be easily replicated for other fields of engineering.
Systems of linear equations are of great importance in engineering, since problems
that are difficult or difficult to solve analytically are solved by transforming them
into linear or nonlinear equation systems. The solution efficiency and effectiveness
of the linear equation system determines the solution efficiency and effectiveness
of the engineering problem.

1.2 CONDITION TO EXIST A SOLUTION IN LINEAR EQUATIONS

The solution of a linear system of equations means finding a solution that satisfies
all the equations in the system. Therefore, with the solution of the linear system
of equations, a single solution point corresponding to the intersection point of all
equations is obtained. But the solution of the linear equation system may not
always be. Therefore, before starting the solution, it should be ensured whether
there is a solution or not. Two methods or approaches can be used for this.
1.2.1 Graphical Method
This method is applied to systems of linear equations consisting of three or fewer
equations, for which graphs can be drawn. All equations in the equation system
are graphed and the intersection point of these equations is determined from the
graph as the solution point. In this method, the graphics must be drawn precisely
for the correct solution. For example, the solution of linear equations with two
unknowns can be shown graphically as in Figure 1.6.

2 x1 + 3x2 = 4 , x1 − x2 = 1

Figure 1.6: Graphical solution of a system of linear equations with two unknowns.
It is seen from Figure 1.6 that the solution of the equation with two unknowns is
7 2
around x1 = = 1.4, x2 = = 0.4 . In this method, since it is not possible to draw the
5 5
graph of more than three systems of equations and show the intersection point
graphically, the graphical method cannot be used for solving larger systems of
equations.

1.2.2 Determinant Method

Since it is not possible to draw the graphs of the equations in the solution of large
systems of equations, it cannot be understood by the graphical method whether
there is a solution to these equations or what their solution is. For this reason, it
is necessary to develop some mathematical methods in order to understand the
existence of its solution in large equation systems. For example, before starting
the solution of large equations, whether there is a solution or not can be
understood by the determinant method. For this, the set of linear equations is
written in matrix form and the determinant of the coefficients matrix is calculated.
According to the determinant value, it is said that there is a solution or there is no
solution.
To explain the determinant method, let's write a general system of linear equations
in matrix form as follows.

a11 x1 + a12 x 2 + a13 x3 + ... + a1n x n = b1   a11 a12 a13 ... a1n   x1  b1 
a 21 x1 + a 22 x 2 + a 23 x3 + ... + a 2 n x n = b2  a 21 a 22 a 23 ... a 2 n   x 2  b2 
     
a31 x1 + a32 x 2 + a33 x3 + ... + a3n x n = b3   a31 a32 a33 ... a3n   x3  = b3  (1.2)
...   ... ... ... ... ...  ...  ... 
     

a n1 x1 + a n 2 x 2 + a n 3 x3 + ... + a nn x n = b  a n1 a n 2 a n 3 ... a nn   x n  bn 
  
A X B

The linear equation system in Equation 1.2 can also be written as AX=B in matrix
form. Here, A is the coefficients matrix, X is the unknowns and B is the known
constants.
Example: A set of linear equations with two unknowns can be written in matrix
form as follows:

2 x1 + x2 = 12 2 1   x1  12
4 x1 − 3x2 = 4
➔ 4 − 3  x  = 4 

  2 
A X B

In the linear equation system expressed in matrix form as above, it can be easily
understood whether the equations have a common intersection point (solution) by
calculating the determinant (detA) of the coefficients matrix A. There are 3 cases
for the determinant value of A:

a) case of detA≠0 (determinant nonzero):

In this case, it is said that the equations have a common intersection
(solution). In other words, there are as many independent equations as
there are unknowns in mathematics, and the set of linear equations is said
to be solvable.

b) detA=0 (determinant being zero):

In this case, some of the equations are parallel to each other and it is said
that the equations do not have a common intersection (solution). In other
words, there are no independent equations as much as the number of
unknowns in mathematics. In the graphical representation, it is seen that
some equations are shifted on the axes of each other. In this case, the
coefficients matrix A is called the Singular matrix.

Example: Let, 2 x1 + 3x2 = 4 , 2 x1 + 3x2 = 2 . It can be seen from Figure 1.7 that
the equations do not have a common solution (intersection). The
determinant of the coefficients matrix, for which these equations do not
have a solution, can be calculated and shown as follows.
2 x1 + 3x2 = 4 2 3  x1  4 2 3
➔ 2 3  x  = 2 ➔ A=  ➔ det A = 0
2 x1 + 3x2 = 2   2  2 3
A X B

Figure 1.7: The case of detA=0 (parallel equations) in the system of linear
equations with two unknowns.
 c) detA≅ 0 (determinant is very close to zero) condition:
In this case, some of the equations intersect each other at infinity. The
solution of the equations depends on the sensitivity of the computer to which
the solution is made. While a common intersection point of equations can
be obtained in sensitive computers, some equations may be perceived as
parallel to each other in non-precision computers and a common intersection
point may not be obtained. In other words, it depends on the sensitivity of
the computer whether independent equations as much as the unknown
number of mathematical unknowns will appear. In this case, the coefficients
matrix A is called the ill-conditioned matrix.

1.3 SOLUTION OF LINEAR EQUATIONS

Two different types of (group) solution methods were developed according to the
size of the linear equation system and the efficiency of the solution. These methods
are generally called Direct and Iterative (step-by-step , correction-based) solution
methods. Direct solution methods are preferred in solving small equation sets and
if there is no problem in the equation system, they will definitely find the solution.
The number of operations to be performed to obtain the solution and hence the
duration of the solution can be estimated in Direct methods. Iterative solution
methods, on the other hand, are mostly preferred in solving large equation sets
and they are not guaranteed to find the solution. Iterative methods need initial
values, also called estimated solutions, to start the solution, and they arrive at the
real solution by correcting the solution step by step (iteratively). Therefore, in
iterative methods, the efficiency of the solution and the achievement of the
solution in a short time depend on the good selection of the initial value (close to
the real solution). Initial values chosen too far from the real solution may cause
the solution to diverge (diverge) from the real value and the solution cannot be
obtained at all.

1.4 DIRECT METHODS

1.4.1 Gauss Elimination
The solution methodology of direct methods is generally based on solving the
system of equations by transforming them into independent equations with one
unknown. The Gaussian elimination method is also based on such a solution
philosophy. In the Gaussian elimination method, the linear equation system is
written in matrix form for the solution AX = B . Then, by multiplying/dividing both
sides of the expression AX = B with the same coefficients and adding the lines
together, the AX = B system is made into equations that can be solved
independently and one after the other. In this new form, the matrix is usually
upper triangular A (values above the diagonal are nonzero). The fact that the
coefficients matrix A is upper triangle means that the equations in the linear
equation system are independent of each other and can be solved separately. For
this reason, the Gaussian Elimination method consists of zeroing the coefficients
under the diagonal (this step is called Elimination or Elimination step) and then
solving the obtained equations one by one starting from the end (Substitution
step). Now let's explain the Gauss-elimination method on an example.

Example: We can obtain the solution of a general system of linear equations with
three unknowns, by writing the equations in matrix form, using the Gaussian
elimination method as follows.

a11 x1 + a12 x2 + a13 x3 = b1  a11 a12 a13   x1  b1 

   
a21 x1 + a22 x2 + a23 x3 = b2 ➔ Matrix form ➔ a21 a22 a23   x2  = b2 
a31 x1 + a32 x2 + a33 x3 = b3 a31 a32 a33   x3  b3 
  
A X B

To obtain the solution, let's apply the elimination and substitution steps to the
above system of equations, respectively.

a) Elemination Step
In the elimination step, all the column elements below the diagonal in the
coefficients matrix A are set to zero. The diagonal term under which all columns
will be reset is called pivot. A systematic is adopted for the reset process. The
terms under the diagonal in row 1 are reset first. Then the terms under the
diagonal in rows 2 and 3, respectively, are reset. For example, to reset the term
a21
a21 below the pivot a11 in the first (1st) row, it is multiplied by row 1 and
a11
subtracted from row 2. To reset the term a31 under the pivot a11 , it is multiplied by
a31
the 1st row and subtracted from the 3rd row. Considering that a ij denotes the
a11
i -th line and the j -th element, the zeroing of the subdiagonal terms can be written
as follows.

   
 a11 a12 a13   
 pivot   x1   b1 
 a21   a  a   a       a  
a2 j −  a1 j → a21 −  21 a11 a22 −  21 a12 a23 −  21 a13   x2  = b2 −  21 b1 
 a11    a11   a11   a11    x    a11  
a   a  a   a     3
  
a3 j −  31 a1 j →  a31 −  31 a11 a32 −  31 a12 a33 −  31 a13  b3 −  a31 b1 
 a11    a11   a11   a11    a  
 11  
 a11 a12 a13   x1  b1 
'    '
➔ 0 '
a22 a23  x  = b2 
  2

 0 '
a32 ' 
a33   x3   '
 b3 

The term under its pivot a22 in row 2 can be reset by applying the elimination
'

process again to the new matrix above. It is subtracted from row 3 by multiplying
'
a32
by row 2 to reset the term a32 under its pivot a22 .
' '
'
a22

 
   
a   
  x1   1
a12 a13 b 
 11 
0 a'
22
'
a23   x2  = b2 '

    
  x3   '  a32  ' 
pivot

'
 a '
  a32
'
 '  '
 b − b
a3' j −  32  '
→ 0 
a32 −  ' a22
a
−  32 a '   3  a22
'  2
 
' 
a ' '
a33 '  23
2 j
 a22    a22   22  
a 

Note: During elimination, the pivot may become zero even though it was not
initially zero. If the pivot has changed to zero, divisions by zero (indefiniteness)
will occur during elimination (zero). In order to prevent this, in the column where
the pivot is zero before the reset, the absolute value (maximum coefficient) is
determined from the values below the pivot. Then the rows with the zero pivot and
the maximum value are swapped. Thus, the value of the pivot is made non-zero
and then the elimination (reset) process continues.
After the reset process (elimination step) is finished, a matrix form as below is
obtained. In this new matrix form, all terms below the diagonal of the matrix A
are zero.

a11 a12 a13   x1  b1 

0 '    '
  x2  = b2 
'
a 22 a 23 (1.3)

 0 0 ''
a33     '' 
 x3  b3 

The matrix form above corresponds to the elimination of x1 from equation 2 and
x1 and x 2 from equation 3.

b) Substitution Step
If the matrix form in Equation 1.3 is written in plain symbolic form as below, it can
easily be seen that the system of equations corresponds to equations that can be
solved one after the other.
𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 = 𝑏1
′ ′
𝑎22 𝑥2 + 𝑎23 𝑥3 = 𝑏2′ (𝑥1 eleminated)
′′ ′′
𝑎33 𝑥3 = 𝑏3 (𝑥1 ve x2 eleminated)
Starting from the end in the above equations and substituting the result in the
previous equation, all equations can be solved as follows, respectively.
''
a33 x3 = b3'' (equation with one unknown) ➔ x3
'
a22 x2 + a23
'
x3 = b2' (equation with one unknown after substitution x3 ) ➔ x 2
a11 x1 + a12 x2 + a13 x3 = b1 (equation with one unknown after substitution x 2 and x3 ) ➔
x1

Example 1.1: Obtain the solution of the following system of linear equations using
the Gaussian elimination method?
18 x1 + 4 x2 − 3x3 + 5 x4 = 91
− 2 x1 − 24 x2 + 7 x3 − 3x4 = −79
4 x1 − x2 + 32 x3 + 5 x4 = 55
− 9 x1 − 7 x2 − 3x3 + 40 x4 = 20
The above set of equations can be written in AX=B matrix form as follows.

 18 4 − 3 5   x1  91 
− 2 − 24 7 − 3  x  − 79
  2
 =
 

4 − 1 32 5   x3  55 
 
 − 9 − 7 − 3 40  
 x4 
 
20 

Elimination step:
By applying line-by-line elimination to matrix A and vector B at the same time,
matrix A can be turned into an upper triangle (only the above-diagonal terms are
nonzero). For this, first the terms under the diagonal (pivot) in the first row in
matrix A can be reset as follows:

 18
 4 − 3 5   x1  91 
 pivot    
 − 2 − 24 7 − 3  x 2  = − 79
   x  55  ➔
 4 − 1 32 5  3   
 
 − 9 − 7 − 3 40   4  
x  20 
 91 
 18 4 −3 5   
 −2 −2 −2 −2  x  −2 
 − 2 −  18 − 24 −   4 7 −   ( −3) − 3 −  5 
 1  − 79 −  91
  18   18   18   18    x    18  
 2  
 4 −  4 18 4
− 1 −  4
4
32 −  (−3)
4
5 −  5   x  = 55 −  4 91 
  18   18   18   18    3   
      18  
−9 −9 −9  − 9   4 
 − 9 − 
x
18 − 7 −  4 − 3 −  (−3) 40 −  5  −9 
  18   18   18   18   20 −  18 91 
   

Then the terms under the diagonal (pivot) in the second row in matrix A can be
reset as follows:

18 4 −3 5   x1  91 
 0 − 23.5556 6.6667 − 2.4444    
     x 2  − 68.8889
 pivot
  x  = 34.7778  ➔
 0 − 1.8889 32.6667 3.8889  3   
 0 −5 − 4.5   
42.5   4  
x 65.5 

18 4 −3 5 
0 − 23.5556 6.6667 − 2.4444 
 − 1.8889 − 1.8889 − 1.8889 
 0 − 1.8889 − ( )(−23.5556) 32.6667 − ( )6.6667 3.8889 − ( )(−2.4444).
 − 23.5556 − 23.5556 − 23.5556 
 −5 −5 −5 
 0 −5−( )(−23.5556) − 4.5 − ( )6.6667 42.5 − ( )(−2.4444) 
− 23.5556 − 23.5556 − 23.5556 
91 
 x1   − 68.8889 
x   
 2  − 1.8889 
  = 34.7778 − ( )(−68.8889)
 x3   − 23.5556 
 x 4   −5 
65.5 − ( )(−68.8889) 
 − 23.5556 

Then the terms under the diagonal (pivot) in the third row in matrix A can be reset
as follows:

18 4 −3 5   x1  91 
 0 − 23.5556 6.6667 − 2.4444    
   x 2  − 68.8889
0  =  ➔
0 32
.  4.0849   x3  40.3019 
1321

 
− 5.9151 43.0189   x 4  80.1226 
pivot
 0 0
18 4 −3 5   x1 
 0 − 23.5556 6.6667 − 2.4444  
   x2 
0 0 32.1321 4.0849 x 
 − 5.9151 − 5.9151  3 
 0 0 − 5.9151 − ( )32.1321 43.0189 − ( ) 4.0849  x 4 
 32.1321 32.1321 
91 
− 68.8889 
 
= 40.3019 
 
80.1226 − ( − 5.9151)40.3019
 32.1321 

Thus, at the end of the elimination process, the following equations are obtained,
in which matrix A turns into an upper triangle.

18 4 −3 5   x1  91 
 0 − 23.5556 6.6667 − 2.4444  x  − 68.8889
  2
 =
 

0 0 32.1321 4.0849   x3  40.3019 
 
0 0 0 43.7708  
 x4 
 
87.5417 


Substitution step:

The above system of equations can be written in standard form as:

18 x1 + 4 x2 − 3 x3 + 5 x4 = 91
− 23.5556 x2 + 6.6667 x3 − 2.4444 x4 = −68.8889
32.1321x3 + 4.0849 x4 = 40.3019
43.7708x4 = 87.5417

It can be solved immediately x 4 from equation 4 above: 43.7708x4 = 87.5417 ➔

x4 = 2

It can be solved x 3 by substituting the x4 = 2 value obtained from the 4th equation
in the 3rd equation. 4.: 32.1321x3 + 4.0849x4 = 40.3019 ➔ x3 = 1

It can be solved x 2 by substituting the x3 = 1 and x4 = 2 values obtained from the

3th and 4th equation in the 2nd equation:
− 23.5556x2 + 6.6667x3 − 2.4444x4 = −68.8889 ➔ x2 = 3

It can be solved x1 by substituting the x2 = 3 , x3 = 1 ve x4 = 2 values obtained from

the 2nd, 3th and 4th equation in the 1st equation: 18x1 + 4 x2 − 3x3 + 5x4 = 91 ➔
x1 = 4.
1.4.2 Kramer Method
The system of linear equations can be written in matrix form for 3 equations as
follows:

 a11 a12 a13   x1  b1 

a   x  = b 
 21 a 22 a 23  2   2 
a31 a32 a33   x3  b3 
  
A X B

From the above equation expression, matrix A and vector B can be rewritten as
 a11 a12 a13  b1 
   
follows. A = a21 a22
 a23  , B = b2 
a31 a32 a33  b 
 3
In the Kramer method, unknowns are calculated by using the determinant ratios
of the A' matrix obtained by replacing the A matrix and the A matrix. The matrix
A' is obtained by replacing the matrix A, and the expression of the matrix A' varies
according to which unknown to be solved. Whichever unknown is to be solved, the
A' matrix is obtained by replacing that column in the A matrix with the B vector.
The relevant unknown is calculated by proportioning the determinants of the
matrix A' and A'. For example, if the solution x1 is sought, A' matrix is obtained by
replacing the 1st column in the A matrix with the B vector. Below, the solution of
the system of equations with 3 unknowns by Kramer's method is expressed in
symbolic form.

' '
 
A

  A 

b1 a12 a13 a11 a12 b1
𝐴′
b2 a22 a23 ⏞ a21 a22 b2
|𝑎11 𝑏1 𝑎13 |
b3 a32 a33 𝑎21 𝑏2 𝑎23 a31 a32 b3
|𝑎 𝑏3 𝑎33 |
31
det A' 𝑑𝑒𝑡 𝐴′ det A'
x1 = = , 𝑥2 = =
𝑑𝑒𝑡 𝐴
, x3 = = (1.4)
det A 𝑎11 𝑎12 𝑎13 det A
||𝑎21 𝑎22 𝑎23 ||
a11 a12 a13 a11 a12 a13
𝑎31
⏟ 𝑎32 𝑎33
a21 a22 a23 𝐴 a21 a22 a23
a31 a32 a33 a31 a32 a33
    
A A
Determinant calculation requires a lot of time for large matrices. Therefore, the
Kramer method is mostly used for solving small systems of equations. Some
engineering software consider systems of equations less than 100,000 (one
hundred thousand) variables as small equation systems.

Example 1.2: Solve the following system of linear equations using Kramer's
method.
9 x1 + 10 x2 + 7 x3 = 5
− x1 + 5 x2 + 6 x3 = 9
2 x2 + 3x3 = 12
For the solution with Kramer's method, let's first write the above set of equations
in matrix form.

 9 10 7  x1  5 
− 1 5 6  x  = 9 
  2   
 0 2 3  x3  12
   
 
A X B

Using the matrix and vector above, the unknowns from the determinant ratios can
be obtained as follows

5 10 7 9 5 7
9 5 6 −1 9 6
12 2 3 − 474 0 12 3 488
x1 = = = -2.7719 , x2 = = = 2.8538 ,
9 10 7 171 9 10 7 171
−1 5 6 −1 5 6
0 2 3 0 2 3
9 10 5
−1 5 9
0 2 12 43
x3 = = = 0.2515
9 10 7 171
−1 5 6
0 2 3

1.4.3 LU Decomposition Method

The decomposition method, like the Gaussian elimination, is based on the principle
of transforming the system of equations into equations with one unknown that can
be solved consecutively. However, the Decomposition method performs the
elimination process in Gaussian Elimination more efficiently. For this, matrix A is
decomposed as the product of two matrices, one in the form of a lower triangle (L
matrix) and the other in the form of an upper triangle (U matrix). The and matrices
in special forms are shown below.

 L11 0 0  1 U12 U13 

L = L 21 L 22 0  , U = 0 1
 U 23 
L 31 L 32 L 33  0 0 1 

A = LU The decomposition can be written using matrix multiplication as follows:

 a11 a12 a13   L11 0 0  1 U12 U13 
a a23  = L21 L22 0  0 1 U 23  (1.5)
 21 a22
a31 a32 a33  L31 L32 L33  0 0 1 

L and U The elements of the matrix can be found by being equal to the left and
right sides of the matrix product as follows:
1.row:

a11 = L11 ➔ L11 found.

a12 = L11U12 ➔ L11 substituted U12 found.
a13 = L11U13 ➔ L11 substituted U13 found.

2.row:

a21 = L 21 ➔ L 21 found.

a22 = L 21U12 + L22 ➔ L 21 ve U12 substituted L 22 found.

a23 = L 21U13 + L 22U 23 ➔ L 21 , U13 ve L22 substituted U 23 found.

3.row:
a31 = L 31 ➔ L31 found.

a32 = L31U12 + L32 ➔ L31 ve U12 substituted L32 found.

a33 = L31U13 + L32U 23 + L33 ➔ L 21 , U13 , L32 ve U 23 substituted L33 found.

After it is decomposed into A = LU a matrix A , its solution AX = B can be

converted to the solution of two consecutive systems of linear equations. It can be
expressed as LUX = B a set of linear equations AX = B using the decomposed form
of A . If the product UX is called Y (temporary unknown UX = Y ), it can be
written as a set of principal linear equations LY = B . From the solution of the
LY = B system of equations, temporary unknowns Y are solved. Then, the main
unknown X is obtained by substituting the solution Y in the expression UX = Y .
The and equations LY = B and UX = Y in the decomposition method correspond to
equations with one unknown that can be solved independently of each other. The
steps of the decomposition method can be explained as follows.
1. step: LY = B

 L11 0 0   y1  b1  L11 y1 = b1  y1

L     
 21 L 22 0   y 2  = b2  ➔ L 21 y1 + L 22 y 2 = b2  y 2
L 31 L 32 L 33   y3  b3  L 31 y1 + L 32 y 2 + L 33 y 3 = b3  y 3
  
L Y B

Y is obtained from the above solution.

2. step: UX = Y

1 U12 U13   x1   y1  x1 + U12 x 2 + U13 x3 = y1  x1

0 1 U   x  =  y  ➔ x 2 + U 23 x3 = y 2  x 2
 23   2   2
0 0  
1  x3   y3  x3 = y 3  x3
  
U X Y

X is obtained from the above solution.

In the solution of some engineering problems, special cases of the A matrix are
sometimes encountered. For example, sometimes matrix A is band-shaped,
sometimes matrix A is symmetrical. In such cases, the Decomposition method can
be made more efficient by using the special cases of the A matrix as follows.
The state of the matrix [A] being a band matrix (triple band matrix):
 a11 a12 0
A = a21 a22 a23 
 0 a32 a33 

The band form above is mostly encountered in the solution of engineering

problems such as heat transfer and strength, which are expressed in differential
equations, by numerical methods such as the Finite Difference method. In this
case, the coefficients matrix A, which is in band form, can be decomposed into the
product of the special L (upper triangle) and U (lower triangle) matrices, which are
also in band form, as follows.

 a11 a12 0   1 0 0  f1 g1 0 
a    
 21 a22 a23  = e2 1 0  0 f 2 g 2  (1.6)
 0 a32 a33   0 e3 1  0 0 f 3 
   
A L U

Again, by using the equality of the left and right sides in matrix multiplication, the
elements of the L and U matrix are obtained as follows.

1.row: a11 = f1  f1
a12 = g1  g1
 a21 = e2 f1  e2
2.row:
a22 = e2 g1 + f 2  f 2
a23 = g 2  g 2

a32 = e3 f 2  e3
3.row:
a33 = e3 g 2 + f 3  f 3

After obtaining the special L and U matrices, the system of linear equations will
again be solved one after the other and converted into systems of equations and
solved. From the equation, the temporary unknowns are obtained as follows:

1 0 0  y1  b1  y1 = b1  y1
e    
 2 1 0  y2  = b2  ➔ e2 y1 + y 2 = b2  y 2
 0 e3 1    
 y3  b3  e3 y 2 + y 3 = b3  y 3

By substituting Y into UX = Y , the main unknowns X are obtained as follows.

 f1 g1 0   x1   y1 
 0 f g x  =  y 
 2 2  2   2
 
 0 0 f 3   x3   y3 
  
U X
f1 x1 + g1 x 2 = y1  x1
f 2 x 2 + g 2 x3 = y 2  x 2
➔
f 3 x3 = y 3  x3
Y

The state of the [A] matrix being symmetrical:

 a11 a12 a13 ... a1n   x1  b1   a11 a12 a13 ... a1n 
a   x  b  a
 12 a22 a23 ... a2 n   2   2   12 a22 a23 ... a2 n 
 a13 a23 a33 ... a3n   x3  = b3  ➔ A = a13 a23 a33 ... a3 n 
      
 ... ... ... ... ...  ...  ...   ... ... ... ... ... 
a1n a2 n a3n ... ann   xn  bn  a1n a2 n a3 n ... ann 
  
A X B

The above form of symmetrical equations is mostly encountered in the solution of

engineering problems such as heat transfer and strength, which are expressed in
differential equations, by numerical methods such as the Finite Element method.
In this case, the coefficients matrix A in symmetrical form can be decomposed into
the product of the special matrix L of the form [A]=[L][L]T:

 L11 0 0  L11 L 21 L 31 
A = LL = L 21
T
L 22 0   0 L 22 L 32  (1.7)
L 31 L 32 L 33   0 0 L 33 
This decomposition method is also called the Cholesky Decomposition method. For
the Cholesky decomposition to be applied, the coefficients matrix A must be
positive definite. Whether [A] is positive definite can be understood in two ways.

i) For [A] to be positive definite, the absolute value of the element on the
diagonal in any row must be greater than the sum of the absolute values
of the other elements on that row. This can be written mathematically
as:
n
aii   aij ( i = 1,..., n )
j =1
j n

ii) For [A] to be positive definite, eigenvalues  of A should   0 . If thre

are three equations, eigenvalues  = 1 , 2 , 3 of A can be calculated
from the solution of detA=0 equation as:

[ A]{x} = I {x} a11 −  a12 a13

[ A − I ]{x} = 0 ➔ a 21 a 22 −  a 23 = 0 ➔ c1 + c 2  + c3  + c 4  = 0  1 ,  2 , 3
2 3

det[ A − I ] = 0 a31 a32 a33 − 

Note: The decomposition method is particularly efficient in solving problems

where matrix A is the same pattern and vector B changes. In such problems, once
the matrix A is decomposed as A=LU, the obtained L and U matrices can be used
continuously, even if the B vector changes.

Example: The bending calculation of a cantilever beam with a modulus of elasticity

E, length L, width b, height h under different F forces (F1, F2, F3,…) is an example
of the above situation (matrix A remains the same, vector B changes).

Figure 1.8: Example of the case where only B changes in the AX=B linear equation
set.

Example 1.3: Find the solution of the following system of linear equations using
the Cholesky Decomposition method?
18 x1 + 4 x2 − 3x3 + 5 x4 = 91
− 2 x1 − 24 x2 + 7 x3 − 3x4 = −79
4 x1 − x2 + 32 x3 + 5 x4 = 55
− 9 x1 − 7 x2 − 3x3 + 40 x4 = 20

The above set of linear equations can be written in matrix form AX=B as follows
 18 4 − 3 5   x1  91 
− 2 − 24 7 − 3  x  − 79
  2
 =
 

4 − 1 32 5   x3  55 
 
 − 9 − 7 − 3 40  
 x4 
 
20 

In the Cholesky Decomposition method, matrix A is decomposed as the product of
two special matrices, one in the form of a lower triangle (L matrix) and the other
in the form of an upper triangle (U matrix).

 a11 a12 a13 a14   L11 0 0 0  1 U 12 U 13 U 14 

a a 22 a 23 a 24   L21 L22 0 0  0 1 U 23 U 24 
 21 =  
a31 a32 a33 a34   L31 L32 L33 0  0 0 1 U 34 
    
a 41 a 42 a 43 a 44   L41 L42 L43 L44  0 0 0 1 

By using the equality of the left and right sides in the matrix multiplication above,
the elements of the matrix can be obtained as follows.

Using the 1st row elements of matrix A:

a11 = L11 ➔ L11 = 18
a12 = L11U12 ➔ 4 = 18U12 ➔ U12 = 0.2222
a13 = L11U 13 ➔ − 3 = 18U 13 ➔ U13 = −0.1667
a14 = L11U14 ➔ 5 = 18U14 ➔ U14 = 0.2778

Using the 2nd row elements of matrix A:

a21 = L21 ➔ L21 = −2
a22 = L21U12 + L22 ➔ − 24 = −2(0.2222) + L22 ➔ L22 = −23.5556
a23 = L21U13 + L22U 23 ➔ 7 = −2(−0.1667) + (−23.5556)U 23 ➔ U 23 = −0.2830
a24 = L21U14 + L22U 24 ➔ − 3 = −2(0.2778) + (−23.5556)U 24 ➔ U 24 = 0.1038

Using the 3rd row elements of matrix A:

a 31 = L31 ➔ L31 = 4
a32 = L31U12 + L32 ➔ − 1 = 4(0.2222) + L32 ➔ L32 = −1.8889
a33 = L31U13 + L32U 23 + L33 ➔ 32 = 4(−0.1667) + (−1.8889)(−0.2830) + L33 ➔ L33 = 32.1321
a34 = L31U14 + L32U 24 + L33U 34 ➔ 5 = 4(0.2778) + (−1.8889)(0.1038) + 32.1321U 34 ➔
U 34 = 0.1271
Using the 4tt row elements of matrix A:
a41 = L41 ➔ L41 = −9
a42 = L41U12 + L42 ➔ − 7 = −9(0.2222) + L42 ➔ L42 = −5
a43 = L41U13 + L42U 23 + L43 ➔ − 3 = −9(−0.1667) + (−5.0002)(−0.2830) + L43 ➔
L43 = −5.9151
a44 = L41U14 + L42U 24 + L43U 34 + L44 ➔
40 = −9(0.2778) + (−5.0002)(0.1038) + (−5.9151)(0.1271) + L44 ➔ L44 = 43.7708

Thus, the L and U matrices are obtained as follows:

 18 0 0 0  1 0.2222 − 0.1667 0.2778

− 2 − 23.5556 0 0   0 1 − 0.2830 0.1038
L=  , U = 
4 − 1.8889 32.1321 0  0 0 1 0.1271
   
− 9 −5 − 5.9151 43.7708 0 0 0 1 

From the solution LY=B, the temporary unknowns (Y) are obtained as follows

 18 0 0 0   y1  91 
− 2 − 23.5556  
 0 0   y 2   
− 79 ➔
 = 
 4 − 1.8889 32.1321 0   y 3  55 
 
− 9 −5 − 5.9151 43.7708 
 y4 
 
20 

18 y1 = 91 ➔ y1 = 5.0556
− 2 y1 + (−23.5556) y2 = −79 ➔ y2 = 2.9245
4 y1 + (−1.8889) y2 + 32.1321y3 = 55 ➔ y 3 = 1.2543
− 9 y1 + (−5) y2 − 5.9151y3 + 43.7708 y4 = 20 ➔ y 4 = 2

From the UX=Y solution, the main unknowns (X) are obtained as follows

1 0.2222 − 0.1667 0.2778  x1  5.0556

0  
 1 − 0.2830 0.1038  x 2   
2.9245 ➔
 = 
0 0 1 0.1271  x3  1.2543 
 
0 0 0 1   x4  
2 


x4 = 2
x3 + (0.1271) x4 = 1.2543 ➔ x3 = 1
x2 + (−0.2830) x3 + (0.1038) x4 = 2.9245 ➔ x2 = 3
x1 + 0.2222x2 + (−0.1667) x3 + (0.2778) x4 = 5.0556 ➔ x1 = 4
1.4.4 Matrix Inverse Method
In this method, the solution of the system of equations is obtained by multiplying
the inverse of the matrix A with the vector B as follows.

AX = B  X =  A B
−1
(1.8)

Although the method is simple, the solution time can be large depending on the
dimensions of A. In short, how quickly and easily the inverse (  A
−1
) of A can be
calculated determines the difficulty and time of the solution. Calculating the
inverse of large matrices is often time consuming. Adjoint (Additional matrix),
Jacobi and LU decomposition methods are widely used to calculate the inverse of
matrices.

a) Adjoint (Additional matrix) Methods

When calculating the inverse of the matrix, the determinants of the sub-matrices
of that matrix are used and therefore the calculation cost is high. The adjoint
method is mostly used to calculate the inverse of small matrices.
The inverse of the matrix A is obtained by dividing the adjoint of A by the
determinant of A, and is expressed as:
adj ( A) C T
A −1 = = .
A A

The adjoint adjA of matrix A can be obtained by transposing the cofactor matrix (
C matrix) corresponding to matrix A. The elements of the C cofactor matrix are
obtained by using the determinants of the sub-matrices to be formed from matrix
A ( adj( A) = C T ). For example, it is calculated as the cofactor term cij = ( −1) i + j Ar
corresponding to the a ij element in the matrix A . Here, Ar the sub (reduced) matrix
represents the matrix obtained by deleting the i -th row and j -th column
elements from matrix A.

Example 1.4: Find the inverse of the following matrix using the Adjoint method?

4 2 1
A = 3 6 2
1 4 8 

The elements of the Cofactor matrix corresponding to matrix A can be calculated

as follows.
6 2 3 2 3 6
c11 = (−1)1+1 = 40 , c12 = (−1)1+ 2 = −22 , c13 = (−1)1+ 3 = 6,
4 8 1 8 1 4
 2 1 4 1 4 2
c21 = (−1) 2 +1 = −12 , c22 = (−1) 2 + 2 = 31 , c23 = (−1) 2 + 3 = −14 ,
4 8 1 8 1 4
2 1 4 1 4 2
c31 = (−1) 3+1 = −2 , c32 = (−1) 3+ 2 = −5 , c33 = (−1) 3+ 3 = 18 ,
6 2 3 2 3 6

So the Cofactor matrix C is obtained as follows

 c11 c12 c13   40 − 22 6 

C = c21 c22 c23  = − 12 31 − 14
c31 c32 c33   − 2 − 5 18 
The Adjoint matrix can be written as the transpose of the Cofactor matrix as
follows.

 40 − 22  40 − 12 − 2
T
6 
 
adj ( A) = C T = − 12 31 − 14 = − 22 31 − 5
 − 2 − 5 18   6 − 14 18 

adj ( A)
In this case, A is the inverse of the matrix. A −1 = It can be obtained as
A
follows, with the expression.

 40 − 12 − 2  40 − 12 − 2
− 22 31 − 5 − 22 31 − 5
    0.3279 - 0.0984 - 0.0164
adj ( A)  6 − 14 18   6 − 14 18  
A −1 = = = = - 0.1803 0.2541 - 0.0410
A 4 2 1 122
 0.0492 - 0.1148 0.1475 
3 6 2
1 4 8

b) Gauss-Jordan Elimination Method

Compared to the adjoint method, it is a more economical method for calculating

the matrix inverse. The Gauss-Jordan elimination method uses the Gaussian
elimination method to calculate the inverse of a matrix. In this method, the A
matrix to be found inverted and the Unit matrix (I matrix) of the same size are
written side by side. By applying the Gaussian elimination to the A and I matrices
at the same time, the A matrix is converted to the Unit matrix, and the Unit matrix
is converted to the inverse of A ( A−1 matrix) as follows.
   
 a11 a12 a13 1 0 0 1 0 0d11 d12 d13 
   
 a 21 a 22 a 23 0 1 0    0 1 0 d 21 d 22 d 23  (1.9)
a31 a32 a33 0 0 1 0 0 1 d 31 d 32 d 33 
         
A I   I A −1 

Example 1.5: Find the inverse of the following matrix using the Gauss-Jordan
elimination method.

4 2 1 
A = 3 6 2
1 4 8 
With the Gauss-Jordan elimination method, the inverse of the A matrix can be
obtained as follows.

   
4 2 11 0 0 1 0 0 0.3279 - 0.0984 - 0.0164 
   
 3 6 2 0 1 0    0 1 0 − 0.1803 0.2541 − 0.0410
1 4 80 0 1 0 0 1 0.0492 − 0.1148 0.1475 
  
       
A I   I A −1 

Note: Like the decomposition method, the inverse of the matrix method is also
effective in solving the problems where the matrix A is the same pattern and the
vector B changes. In such problems, even if the B vector changes, it is sufficient
to calculate the inverse of the A matrix once.

i) LU Decomposition Method:
This method is based on the idea that the product of a matrix A and its inverse is
equal to the unit matrix ( AA −1 = I ). The method can be described using the 3x3
matrix A below.
 a11 a12 a13 
A = a21 a22 a23 
a31 a32 a33 
Let's express the inverse of the above matrix A as follows.

 d11 d12 d13 

A = d 21
−1
d 22 d 23 
d 31 d 32 d 33 
In this case, the AA −1 = I equation can be written explicitly as:
 a11 a12 a13   d11 d12 d13  1 0 0
a a23  d 21 d 22 d 23  = 0 1 0
 21 a22
a31 a32 a33  d 31 d 32 d 33  0 0 1

By taking one column at a time from the matrix multiplication above, 3 different
linear equation systems can be obtained, which can be solved by the LU
decomposition method as follows.

linear system of equations 1:

 a11 a12 a13  d11  1   L11 0 0  1 U12 U13  d11  1 
a        
 21 a22 a23  d 21  = 0 ➔ L
 21 L 22 0  0 1 U 23  d 21  = 0
a31 a32 a33     
d 31  0 L 31 L 32 L 33  0 0 1  d 31  0

linear system of equations 2:

 a11 a12 a13  d12  0  L11 0 0  1 U12 U13  d12  0
a        
 21 a22 a23  d 22  = 1  ➔ L
 21 L 22 0  0 1 U 23  d 22  = 1 
a31 a32 a33     
d 32  0 L 31 L 32 L 33  0 0 1     
d 32  0

linear system of equations 3:

 a11 a12 a13  d13  0  L11 0 0  1 U12 U13  d13  0
a        
 21 a22 a23  d 23  = 0 ➔ L
 21 L 22 0  0 1 U 23  d 23  = 0
a31 a32 a33     
d 33  1  L 31 L 32 L 33  0 0 1     
d 33  1 

The inverse of the matrix A can be obtained by solving the above 3 linear system
of equations by LU decomposition method. Returns the elements in the 1st column
of the matrix A−1 which is the inverse of the solution of the 1st system of linear
equations, the elements in the 2nd column of the 2nd column of the solution of
the 2nd linear equation system A−1 , and the 3rd column of the 3rd column of the
solution A−1 of the 3rd linear equation system.

Note: As can be seen from the above, in order to calculate the inverse of a matrix
A of size nxn , it is necessary to solve a system of n linear equations (each
system of equations consists of n unknowns). This explains why calculating the
inverse of large matrices requires a long time and computational cost.

Example 1.6: Find the solution of the following system of linear equations using
the Inverse of the Matrix method?
18 x1 + 4 x2 − 3x3 + 5 x4 = 91
− 2 x1 − 24 x2 + 7 x3 − 3x4 = −79
4 x1 − x2 + 32 x3 + 5 x4 = 55
− 9 x1 − 7 x2 − 3x3 + 40 x4 = 20
With the inverse of the matrix method, the solution of the system of linear
equations in the form of AX=B is expressed as 𝑋 = 𝐴−1 𝐵 as the product of the
inverse of the matrix A (𝐴−1 ) and the vector B.

The inverse of the matrix A is expressed by the Cofactor method as

adj ( A) C T
A −1 = = .
A A

 c11 c12 c13 c14 

c c c 23 c 24 
C Cofactor matrix C =  21 22 Its elements can be calculated as
c 31 c 32 c 33 c 34 
 
c 41 c 42 c 43 c 44 
follows.

−24 7 −3
𝑐11 = (−1)1+1 | −1 32 5 | = −24(32𝑥40 + 15) − 7(−40 + 35) − 3(3 + 32𝑥7)
−7 −3 40
= −31726
−2 7 −3
( )1+2
𝑐12 = −1 |4 32 5 | = −(−2(32𝑥40 + 15) − 7(160 + 45) − 3(−12 + 32𝑥9))
−9 −3 40
= 4853
−2 −24 −3
( )1+3
𝑐13 = −1 |4 −1 5 | = (−2(−40 + 35) + 24(160 + 45) − 3(−28 − 9))
−9 −7 40
= 5041
−2 −24 7
𝑐14 = (−1)1+4 | 4 −1 32 | = −(−2(−3 + 32𝑥7) + 24(−12 + 32𝑥9) + 7(−28 − 9))
−9 −7 −3
= −5911

4 −3 5
𝑐21 = (−1)2+1 |−1 32 5 | = −(4(32𝑥40 + 15) + 3(−40 + 35) + 5(3 + 32𝑥7))
−7 −3 40
= −6300
18 −3 5
𝑐22 = (−1)2+2 |−1 32 5 | = (18(32𝑥40 + 15) + 3(160 + 45) + 5(−12 + 32𝑥9))
−9 −3 40
= 25305
18 4 5
𝑐23 = (−1)2+3 | 4 −1 5 | = −(18(−40 + 35) − 4(160 + 45) + 5(−28 + 9))
−9 −7 40
= 1095
18 4 −3
𝑐24 = (−1)2+4 | 4 −1 32 | = (18(3 + 32𝑥7) − 4(−12 + 32𝑥9) − 3(−28 + 9))
−9 −7 −3
= 3093

4 −3 5
𝑐31 = (−1)3+1 |−24 7 −3| = (4(280 − 9) + 3(−24𝑥40 − 21) + 5(72 + 49))
−7 −3 40
 = −1254

18 −3 5 18 4 5
𝑐32 = (−1)3+2 |−2 7 −3| = −4902, 𝑐33 = (−1)3+3 |−2 −24 −3| = −18240
−9 −3 40 −9 −7 40
18 4 −3
𝑐34 = (−1)3+4 |−2 −24 7 | = −2508
−9 −7 −3

4 −3 5 18 −3 5
𝑐41 = (−1)4+1 |−24 7 −3| = 3650, 𝑐42 = (−1)4+2 |−2 7 −3| = 1904
−1 32 5 4 32 5

18 4 5 18 4 −3
𝑐43 = (−1)4+3 |−2 −24 −3| = 1732, 𝑐44 = (−1)4+4 |−2 −24 7 | = −13624
4 −1 5 4 −1 32

−31726 4853 5041 −5911

−6300 25305 1095 3093
C=[ ] ➔
−1254 −4902 −18240 −2508
3650 1904 1732 −13624

In this case, the Adjoint matrix adj(A) and the inverse of A can be calculated as
follows
−31726 −6300 −1254 3650
4853 25305 −4902 1904
adj(A)=𝐶 𝑇 = [ ]➔
5041 1095 −18240 1732
−5911 3093 −2508 −13624

0.0532 0.0106 0.0021 −0.0061

𝑎𝑑𝑗(𝐴) −0.0081 −0.0424 0.0082 −0.0032
𝐴−1 = ➔ 𝐴−1 =[ ]
|𝐴| −0.0085 −0.0018 0.0306 −0.0029
0.0099 −0.0052 0.0042 0.0228

After calculating the inverse of A, the solution of the set of linear equations is
obtained as follows.
0.0532 0.0106 0.0021 −0.0061 91
−1 −0.0081 −0.0424 0.0082 −0.0032 −79
𝑋=𝐴 𝐵 ➔ 𝑋=[ ]{ }➔
−0.0085 −0.0018 0.0306 −0.0029 55
0.0099 −0.0052 0.0042 0.0228 20

𝑥1 = 4 , 𝑥2 = 3, 𝑥3 = 1, 𝑥4 = 2
1.5 İTERATİVE METHODS
In iterative methods, solution equations are obtained by leaving an unknown from
each equation in the equation system. The solution equations are solved iteratively
and the real solution is tried to be obtained. In iterative methods, the solution
cannot start by itself. Therefore, the initial values are used in the first iteration.
Iterative methods do not guarantee a solution. Whether the solution can be
obtained or not depends on the iterative method, the structure of the system of
equations (the dominance of diagonal values over other values), and the chosen
initial values. Iterative methods are generally more efficient than direct methods
and are therefore preferred for solving large sets of equations. Some commonly
used iterative methods are described in detail below.

1.5.1 Jacobi Method

In this method, the linear equation system expressed as
a11 x1 + a12 x2 + a13 x3 = b1
a21 x1 + a22 x2 + a23 x3 = b2
a31 x1 + a32 x2 + a33 x3 = b3

Leaving x1 , x 2 , x 3 alone, respectively, equations with one unknown are obtained

for the solution
b1 − (a12 x2 + a13 x3 )
x1 =
a11
b2 − (a21 x1 + a23 x3 )
x2 = (1.10)
a22
b3 − (a31 x1 + a32 x2 )
x3 =
a33
From the above expressions, it can easily be seen that the values in the
denominator, that is, the diagonal elements of the matrix A, must be different than
zero ( a11 , a22 , a33  0 ) in order to have a solution.

Before starting the solution with the Jacobi method, the estimated solution (initial
values) is chosen as x1 , x2 , x3 . By replacing the initial values on the right side
(0)
(0) (0)

of the equations, corrected or renewed solution values are obtained on the left
side. By creating a loop in this way, this loop is continued until the unknown
variable values change slightly. The steps of this cycle are called iterations. Below
is a symbolic explanation of how the unknowns are calculated in the iterations in
the loop.
İteration 1:

b1 − (a12 x2( 0 ) + a13 x3( 0 ) )

x(1)
1 =
a11
b2 − (a21 x1( 0 ) + a23 x3( 0 ) )
x2(1) =
a22
b3 − (a31 x1( 0 ) + a32 x2( 0 ) )
x3(1) =
a33
İteration 2:

b1 − (a12 x2(1) + a13 x3(1) )

x1( 2 ) =
a11
b2 − (a21 x1(1) + a23 x3(1) )
x2( 2 ) =
a22
b3 − (a31 x1(1) + a32 x2(1) )
x ( 2)
3 =
a33

İteration k:

b1 − (a12 x2( k −1) + a13 x3( k −1) )

x(k )
1 =
a11
b2 − (a21 x1( k −1) + a23 x3( k −1) )
x2( k ) =
a22
b3 − (a31 x1( k −1) + a32 x2( k −1) )
x3( k ) =
a33
In the iterative solution above, when to stop the loop is an important issue and
various convergence criteria are used for this. Convergence criteria are mostly
based on comparing the solution in the new step (k-th iteration) with the solution
in the previous step ((k-1)-th iteration).
Convergence criterion:

x1( k ) − x1( k −1) x2( k ) − x2( k −1) x3( k ) − x3( k −1)

  ,   ,  (1.11)
x1( k −1) x2( k −1) x3( k −1)

If the change in the last two iterations is small enough (eg  =0.001), the loop is
stopped and the values in the last iteration are considered the solution to the
system of equations. The Jacobi method has no guarantee of a solution. For a
solution to exist, the terms on the diagonal must be more dominant (greater than
the sum) of the sum of the other terms in that row.

a11  a12 + a13

a22  a21 + a23 (1.12)
a33  a31 + a32
The convergence condition can generally be expressed as
n
aii   aij , i = 1,..., n .
j =1
j i

Most engineering problems satisfy the above convergence condition. Although it is

not always possible, sometimes changing the place or order of the equations may
cause the convergence condition to be met and the solution to be possible.
Changing the place or order of equations is often called pivoting.
For example, consider the following system of equations.
Before pivoting:

2 x2 + 3x3 = 12  0 2 3
− x1 + 5 x2 + 6 x3 = 9 ➔ A = − 1 5 6
9 x1 + 10 x2 + 7 x3 = 5  9 10 7

The above system of equations cannot be solved by the Jacobi method since the
in coefficient x1 in the first equation is 0. However, if the position of the equations
is changed and the number 0 is prevented from appearing on the diagonal, the
system of equations can be made solvable by the Jacobi method.
After pivoting:

9 x1 + 10 x2 + 7 x3 = 5  9 10 7 
− x1 + 5 x2 + 6 x3 = 9 ➔ A = − 1 5 6
2 x2 + 3x3 = 12  0 2 3

1.5.2 Gauss-Siedel Method

The solution of the linear equation system with the Gauss-Siedel method is very
similar to the Jacobi method. The difference of the Gauss-Siedel method from the
Jacobi method is that the latest corrected values are used in each iteration, as
shown below.
İteration 1:

b1 − (a12 x2( 0 ) + a13 x3( 0 ) )

x1(1) =
a11
b2 − (a21 x1(1) + a23 x3( 0 ) )
x (1)
2 =
a22
b3 − (a31 x1(1) + a32 x2(1) )
x (1)
3 =
a33
İteration 2:

b1 − (a12 x2(1) + a13 x3(1) )

x( 2)
1 =
a11
b2 − (a21 x1( 2 ) + a23 x3(1) )
x2( 2 ) =
a22
b3 − (a31 x1( 2 ) + a32 x2( 2 ) )
x3( 2 ) =
a33
İteration k:

b1 − (a12 x2( k −1) + a13 x3( k −1) )

x1( k ) =
a11
b2 − (a21 x1( k ) + a23 x3( k −1) )
x2( k ) =
a22
b3 − (a31 x1( k ) + a32 x2( k ) )
x (k )
3 =
a33

In the iterative solution with Gauss-Siedel, the stopping criterion is the same as in
the Jacobi method.

Since the Gauss-Siedel method uses the newest values of the variables in each
iteration, it converges faster than the Jacobi method. Therefore, it is more efficient
than the Jacobi method. The convergence condition in the Gaus-Siedel method is
the same as in the Jacobi method. For convergence, the diagonal terms must be
greater than the sum of the other terms in that row.
n
( aii  a
j =1
ij , i = 1,..., n ).
j i

Example 1.7: Find the solution of the following system of linear equations using
the Gauss-Seidel Method?
18 x1 + 4 x2 − 3x3 + 5 x4 = 91
− 2 x1 − 24 x2 + 7 x3 − 3x4 = −79
4 x1 − x2 + 32 x3 + 5 x4 = 55
− 9 x1 − 7 x2 − 3x3 + 40 x4 = 20
It can be shown as follows that the above system of equations satisfies the
convergence condition in the Gauss-Siedel method and therefore can be solved by
the Gauss-Siedel method.

a11  a12 + a13 + a14 ➔ 18  4 + − 3 + 5 ,

a22  a21 + a23 + a24 ➔ − 24  − 2 + 7 + − 3 ,
a33  a31 + a32 + a34 ➔ 32  4 + − 1 + 5 ,
a44  a41 + a42 + a43 ➔ 40  − 9 + − 7 + − 3 ,
The solution of the above system of linear equations with the Gauss-Seidel method
can be written in iterative form as follows.

b1 − (a12 x 2( k −1) + a13 x3( k −1) + a14 x 4( k −1) )

x(k )
1 =
a11
b2 − (a 21 x1( k ) + a 23 x3( k −1) + a 24 x 4( k −1) )
x 2( k ) =
a 22
b3 − (a31 x1( k ) + a32 x 2( k ) + a34 x3( k −1) )
x3( k ) =
a33
b4 − (a 41 x1( k ) + a 42 x 2( k ) + a 43 x3( k ) )
x (k )
4 =
a 44

If 𝑥10 = 0, 𝑥20 = 0, 𝑥30 = 0, 𝑥40 = 0 is selected as initial values, the iterative solution is
obtained as follows.
İteration 1:
b1 − (a12 x 2( 0) + a13 x3( 0) + a14 x 4( 0) ) 91
x1(1) = = = 5.0556
a11 18
b2 − (a 21 x1(1) + a 23 x3( 0) + a 24 x 4( 0) ) - 79 - (-2)5.0556
x 2(1) = = = 2.8704
a 22 - 24
b3 − (a31 x1(1) + a32 x 2(1) + a34 x3( 0) ) 55 - 4(5.0556) + (-1)2.8704
x3(1) = = = 1.1765
a33 32
b4 − (a 41 x1(1) + a 42 x 2(1) + a 43 x3(1) ) 20 - (-9)5.0556 + (-7)2.8704 + (-3)1.1765
x 4(1) = = = 2.2281
a 44 40

İteration 2:
b1 − (a12 x 2(1) + a13 x3(1) + a14 x 4(1) ) 91 − 4(2.2281) + (-3)1.1765 + 5(2.2281)
x1( 2) = = = 3.9949
a11 18
b2 − (a 21 x1( 2) + a 23 x3(1) + a 24 x 4(1) ) - 79 - (-2)3.9949 + 7(1.1765) + (-3)2.2281
x 2( 2) = = = 3.0234
a 22 - 24
b3 − (a31 x1( 2) + a32 x 2( 2) + a34 x3(1) ) 55 - 4(3.9949) + (-1)3.0234 + 5(2.2281)
x3( 2) = = = 0.9657
a33 32
b4 − (a 41 x1( 2) + a 42 x 2( 2) + a 43 x3( 2) ) 20 - (-9)3.9949 + (-7)3.0234 + (-3)0.9657
x 4( 2) = = = 2.0004
a 44 40

Similarly, iterations are continued until the values in successive iterations are very
close to each other. After the fourth iteration, the convergence (Convergence
tolerance:  = 0.005 ) condition is satisfied and the following solution values are
obtained.
𝑥14 = 4.0033, 𝑥24 = 3.0005, 𝑥34 = 1.0002, 𝑥44 = 2.0009
1.5.3 Optimization Method
Optimization methods are iterative methods that solve minimum (smallest) or
maximum (largest) type problems. If the system of linear equations is transformed
into an optimization problem, it can be solved using optimization methods. For
this, all equations in the linear equation system are written as equal to zero and
then the expressions to the left of the equation are represented by functions such
as f1 , f 2 ,…, f n

a11 x1 + a12 x 2 + a13 x3 − b1 = 0 f1 ( x1 , x2 , x3 ) = a11 x1 + a12 x2 + a13 x3 − b1

a 21 x1 + a 22 x 2 + a 23 x3 − b2 = 0 ➔ f 2 ( x1 , x2 , x3 ) = a21 x1 + a22 x2 + a23 x3 − b2
a31 x1 + a32 x 2 + a33 x3 − b3 = 0 f 3 ( x1 , x2 , x3 ) = a31 x1 + a32 x2 + a33 x3 − b3

Then a new single function F = f12 + f 22 + f 32 is created by adding the squares of

functions such as f1 , f 2 ,…, f n . The minimum point of this new function F gives
the solution of the system of linear equations as follows.

F = f12 + f 22 + f 32  min (1.13)

Since F function is expressed by the sum of the squares of functions such as f1 ,

f 2 ,…, f n , The minimum value of the function F is zero. Meaning that F is zero, it
means f1 = 0 , f 2 = 0 ,…, f n = 0 , that is, the linear equations are obtained again.
This shows that the optimization solution and the linear equation system solution
are equivalent. Engineering software such as ANSYS use PCG (Preconditioned
Conjugate Gradient) optimization method to solve large linear equation systems.
Optimization methods can be divided into two main groups as follows
1- Local optimization methods using derivative information
2- Global optimization methods that use only function information (not
derivatives)
Local optimization methods (methods that search for the optimum near the
starting point) are fast, but they do not always find the global optimum (best of
local optima). Global optimization methods, on the other hand, are slower
(requiring more processing), but they are more successful in obtaining the global
optimum than local methods.
The Steepeset-Descent method, which is the simplest of the local optimization
methods, solves the optimization problem iteratively (step by step), starting from
an initial value. The iterative solution with the Steepest descent method can be
written as follows.

k k k
 F   F   F 
xk +1
= x −  
k
 , x2k +1 = x2k −    , x3k +1 = x3k −    (1.14)
 x1   x2   x3 
1 1
It shows the number of iterations k and  the step length parameter above.
Parameter  is usually determined from the solution of a second optimization
problem. In another approach that does not use optimization, either a value is
given  in the same iteration and it is checked whether the function value
k +1
decreases ( f ( x )  f ( x k ) ) at the new point ( xik +1 ). If the function value at the
new point is smaller, this point is accepted as the optimum point for this iteration.
If not, the value of the function at the new point is recalculated by reducing its
value (for example, by dividing it in half) and compared with the previous iteration
value. Its value continues to be decreased until the function value at the new point
is less than the function value  at the old point.

Example 1.8: 2 x1 + 3x2 = 4 , − x1 + x2 = 9 Express the solution of the system of linear

equations in the form of an optimization problem?

After the equations are written as equal to zero, the non-zero side f1 ( x1 , x2 ) is
renamed as f 2 ( x1 , x2 ) and

2 x1 + 3x2 − 4 = 0 f1 ( x1 , x 2 ) = 2 x1 + 3x 2 − 4
➔
− x1 + x2 − 9 = 0 f 2 ( x1 , x 2 ) = − x1 + x 2 − 9

A new function F is obtained by adding the squares of the and functions f1 ( x1 , x2 )

and f 2 ( x1 , x2 ) . The solution of the system of linear equations is obtained by
minimizing the function as follows.

F = f1 ( x1 , x2 ) 2 + f 2 ( x1 , x2 ) 2 = (2 x1 + 3x2 − 4) 2 + (− x1 + x2 − 9) 2 ➔ min
 SUMMARY
- Numerical solutions of engineering problems are usually expressed directly
or indirectly with linear equations.
- Linear sets of equations can be expressed in matrix form AX=B.
- In order to solve linear equations, independent equations are required as
much as the number of unknowns. Whether this condition is met or not can
be checked by calculating the determinant of the matrix obtained from the
coefficients of the linear equations. The solution should be detA≠0
- Methods for solving linear equations are divided into two groups as Direct
and Iterative methods.
- Direct methods (such as Gauss-Elimination, Decomposition, Inverse of
Matrix, Kramer) and detA≠0 are methods that definitely find the solution.
- Since the number of operations is high in direct methods, they are generally
preferred in solving small equation sets.
- Iterative methods (such as Jacobi, Gauss-Seidel, Optimization methods) are
the methods that find the solution step by step (iteratively) starting from an
initial value (solution estimation).
- Iterative methods do not always find the solution. Finding the solution is
conditional. For the solution, the diagonal terms of the coefficients matrix A
should be dominant compared to the other terms.
- Iterative methods can find the solution faster than direct methods if a good
starting value is chosen.
- Iterative methods are generally preferred for solving large sets of equations.
 MATLAB PROGRAMS
Gauss Eliminasyon
function LinearEquation_Solution_GaussElimination

% SOLUTION OF LINEAR EQUATION SYSTEM WITH GAUSS ELIMINATION METHOD

A=[18 4 -3 5 ;
-2 -24 7 -3 ;
4 -1 32 5 ;
-9 -7 -3 40 ] ;

B=[91 ; -79 ; 55 ; 20 ] ;

n=length(B) ;

% 1- ELIMINATION STEP
for ii=1:n-1
for i=ii+1:n

PivotCoeff=(A(i,ii)/A(ii,ii)) ;

for j=ii:n
A(i,j)=A(i,j)-PivotCoeff*A(ii,j) ;
end

B(i)=B(i)-PivotCoeff*B(ii) ;
end
end

% 2- SUBSTITUTION STEP
X=zeros(n,1) ;
for i=n:-1:1
sum=0 ;
for j=1:n
if j~=i
sum=sum+A(i,j)*X(j) ;
end
end
X(i)=(B(i)-sum)/A(i,i) ;
end

A
B
X
LU Decomposition
function LinearEquation_Solution_LU_Decomposition

% SOLUTION OF LINEAR EQUATION SYSTEM WITH CHOLESKY L*U DECOMPOSITION

METHOD

A=[18 4 -3 5 ;
-2 -24 7 -3 ;
4 -1 32 5 ;
-9 -7 -3 40 ] ;

B=[91 ; -79 ; 55 ; 20 ] ;

n=length(B) ;

L=zeros(n,n) ;
U=zeros(n,n) ;

for i=1:n
U(i,i)=1 ;
end

% A=L*U Decomposition
for i=1:n
for j=1:n
if j<=i
sum=0 ;
for k=1:j-1
sum=sum+L(i,k)*U(k,j) ;
end

L(i,j)=A(i,j)-sum ;
else
sum=0 ;
for l=1:i-1
sum=sum+L(i,l)*U(l,j) ;
end

U(i,j)=(A(i,j)-sum)/L(i,i) ;
end
end
end

% L*Y=B Solution
Y=zeros(n,1) ;
for i=1:n
sum=0 ;
for j=1:n
if j~=i
sum=sum+L(i,j)*Y(j) ;
end
end
Y(i)=(B(i)-sum)/L(i,i) ;
end

% U*X=Y Solution
X=zeros(n,1) ;
for i=n:-1:1
sum=0 ;
for j=1:n
 if j~=i
sum=sum+U(i,j)*X(j) ;
end
end
X(i)=Y(i)-sum ;
end

Y
X
Gauss Siedel
function LinearEquation_Solution_Gauss_Siedel

% SOLUTION OF LINEAR EQUATION SYSTEM WITH GAUSS-SIEDEL ITERATION METHOD

A=[18 4 -3 5 ;
-2 -24 7 -3 ;
4 -1 32 5 ;
-9 -7 -3 40 ] ;

B=[91 ; -79 ; 55 ; 20 ] ;

n=length(B) ;

ConvTol=0.005 ;
MaxIter=1000 ;

% Initial Values
X=zeros(n,1) ;
Conv=1e10 ;
it=0 ;

X0=X ;

while (Conv>=ConvTol) && (it<=MaxIter)

it=it+1 ;

for i=1:n
sum=0 ;
for j=1:n
if j~=i
sum=sum+A(i,j)*X(j) ;
end
end
X(i)=(B(i)-sum)/A(i,i) ;
end

% Convergence Check
for i=1:n
if abs(X0(i))<=1e-10
ConvVec(i)=abs(X(i)-X0(i)) ;
else
ConvVec(i)=abs((X(i)-X0(i))/X0(i)) ;
end
end

Conv=max(ConvVec) ;
X0=X ;
end

it
X