Solution of the linear algebraic system

Ax = b

 A : coefficient matrix is m*m

 b : vector of constant is m*1

 X:unknown vector is m*1

Methods of Solution

1-Direct methods: solve the system in a fixed and predetermined

number of steps.
Examples are Gaussian elimination, LU factorization and
Cholesky decomposition.

2-Indirect methods: are iterative techniques that approach the

solution.
Examples are Gauss-Seidel, Gauss-Jacobi, Conjugate Gradient,
etc.
 Gaussian elimination
The elimination steps are as follows:
1.Normalize the first equation with respect to the first coefficient

2. Add the normalized first equation to the second equation to

eliminate its leading coefficient.

3. Normalize the new equation 2.

4. Add the new, normalized equation 2 to the third equation to

eliminate its leading coefficient.

5- repeat the above steps for more equations

Gaussian elimination
Example:
Solve the following nodal equations using Gaussian Elimination
2v1 - v2 = 1
- v1 + 3v2 - v3 = 0
- v2 + 3v3 - v4 = 0
-v3 +2v4 = 1
Gaussian elimination

The above steps are summarized in matrix form as following.

Steps
2 −1 0 0 𝑣1 1
−1 3 −1 0 𝑣2 = 0
0 −1 3 −1 𝑣3 0
0 0 −1 2 𝑣4 1
A b

2 −1 0 0 1 Divide row1 by 2

−1 3 −1 0 0
0 −1 3 −1 0
0 0 −1 2 1
Gaussian elimination

step 1)
1 −1/2 0 0 1/2
−1 3 −1 0 0 row2=row1 +row2
0 −1 3 −1 0
0 0 −1 2 1

Step 2)
1 −1/2 0 0 1/2
row2*(2/5)
0 5/2 −1 0 1/2
0 −1 3 −1 0
0 0 −1 2 1
Gaussian elimination

Step 3)
1 −1/2 0 0 1/2 row3=row2 +row3
0 1 − 2/5 0 1/5
0 −1 3 −1 0
0 0 −1 2 1

Step 4)
row3 *(5/13)
1 −1/2 0 0 1/2
0 1 − 2/5 0 1/5
0 0 13/5 −1 1/5
0 0 −1 2 1
Gaussian elimination

Step 5)
1 −1/2 0 0 1/2
0 1 − 2/5 0 1/5 row4=row3 +row4
0 0 1 −5/13 1/13
0 0 −1 2 1
Step 6)

1 −1/2 0 0 1/2
row4 *(13/21)
0 1 − 2/5 0 1/5
0 0 1 −5/13 1/13
0 0 0 21/13 14/13
Gaussian elimination

Step 7

1 −1/2 0 0 1/2
0 1 − 2/5 0 1/5
0 0 1 −5/13 1/13
0 0 0 1 2/3

Matrix A transformed into normalized upper triangular matrix

 Gaussian elimination
We can now proceed to the solution of the upper triangular equations
by means of Back Substitution(BS):
v4 =2/3
v3-5/13 v4=1/13
V3=1/13+5/13 v4=1/3
V2-2/5v3=1/5
V2=1/5+2/5v3=1/3
V1-1/2 v2=1/2
V1=1/2+1/2 v2=2/3
 LU Factorization

Ax=b Factorize A into L and U

A=LU L:Lower triangular matrix
U:Upper triangular matrix
LU x=b
Let Ux=y Ly=b

we first solve the lower triangular system Ly=b by Forward

Substitution to obtain y and we then solve the upper triangular system
Ux= y by Back Substitution to obtain the final result x
 LU Factorization

Gaussian Elimination provides the combination of LU factorization.

We can obtain the L matrix from the values that appeared on and
below the main diagonal immediately before their normalization (for
the main diagonal elements).
 LU Factorization

For the previous Gaussian Elimination example We will calculate LU

Factorization

A U L
2 −1 0 0 2
−1 3 −1 0 −1
0 −1 3 −1 0
0 0 −1 2 0
Step 1) and 2)
1 −1/2 0 0 2
R1/2
0 5/2 −1 0 −1 5/2
0 −1 3 −1 R2=R2+R1 0 −1
0 0 −1 2 0 0
 LU Factorization

Step 3) and 4)
1 −1/2 0 0 R2*(2/5) 2
0 1 − 2/5 0 R3=R3+R2 −1 5/2
0 0 13/5 −1 0 −1 13/5
0 0 −1 2 0 0 −1
Step 5) and 6)
𝟐
1 −1/2 0 0 −𝟏
𝟎
𝟓/𝟐
−𝟏 𝟏𝟑/𝟓
𝟐𝟏/𝟏𝟑
0 1 − 2/5 0 𝟎 𝟎 −𝟏

0 0 1 −5/13 R3*(5/13)

21/13 R4=R4+R3
0 0 0
 LU Factorization
Step 7
1 −1/2 0 0 2 0 0 0
−1 5/2 0 0
0 1 − 2/5 0 0 −1 13/5 0
0 0 1 −5/13 0 0 −1 21/13
0 0 0 1 R4*(13/21)
U L
Returning once again to the original problem, perform first the Forward
Substitution:

Ly=b
𝟐 𝟎 𝟎 𝟎 𝑦1 1
−𝟏 𝟓/𝟐 𝟎 𝟎 𝑦2 0
𝟎 −𝟏 𝟏𝟑/𝟓 𝟎 = 0
𝟎 𝟎 −𝟏 𝟐𝟏/𝟏𝟑
𝑦3
𝑦4 1
 LU Factorization

2 y1=1 y1=1/2
-y1+5/2y2=0 y2=1/5
-y2+13/5y3=0 y3=1/13
-y3+21/13y4=1 y4=2/3
Perform the Back Substitution.
Uv=y

1 −1/2 0 0 𝑣1 1/2
0 1 − 2/5 0 𝑣2 1/5
=
0 0 1 −5/13 𝑣3 1/13
0 0 0 1 𝑣4 2/3
 LU Factorization

We thus obtain equations identical to Gaussian Elimination

v4 =2/3
v3-5/13 v4=1/13
V3=1/13+5/13 v4=1/3
V2-2/5v3=1/5
V2=1/5+2/5v3=1/3
V1-1/2 v2=1/2
V1=1/2+1/2 v2=2/3