Solution of Linear Systems

𝑨𝑿 = 𝑩
For Example:
𝟏𝟎𝒙𝟏 + 𝟑𝒙𝟐 + 𝟏𝒙𝟑 = 𝟏𝟗
𝟑𝒙𝟏 + 𝟏𝟎𝒙𝟐 + 𝟐𝒙𝟑 = 𝟐𝟗
𝟏𝒙𝟏 + 𝟐𝒙𝟐 + 𝟏𝟎𝒙𝟑 = 𝟑𝟓

𝟏𝟎 𝟑 𝟏 𝒙𝟏 𝟏𝟗
𝑨=[𝟑 𝟏𝟎 𝟐 ] , 𝑿 = [𝒙𝟐 ] , 𝑩 = [𝟐𝟗]
𝟏 𝟐 𝟏𝟎 𝒙𝟑 𝟑𝟓
 Direct Methods for Linear systems
3. LU Decomposition:
Decompose / factorize the coefficient matrix A into a lower triangular matrix L
and upper triangular matrix U.
𝐴𝑋 = 𝐵 … (1)
Take 𝐴 = 𝐿𝑈
Put 𝐴 in (1)
⟹ 𝐿𝑈𝑋 = 𝐵
Take 𝑈𝑋 = 𝑌
⟹ 𝐿𝑌 = 𝐵

Solve 𝐿𝑌 = 𝐵 first and then use 𝑌 to solve 𝑈𝑋 = 𝑌

Methods to decompose A:

𝑎11 𝑎12 𝑎13 𝑙11 0 0 𝑢11 𝑢12 𝑢13

[𝑎21 𝑎22 𝑎23 ] = [𝑙21 𝑙22 0][ 0 𝑢22 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑙31 𝑙32 𝑙33 0 0 𝑢33

• Dolittle’s Method
• Crout’s Method
• Cholesky Method
1. Dolittle’s Method:
Consider a general 3 x 3 coefficient matrix
𝑎11 𝑎12 𝑎13 𝑙11 0 0 𝑢11 𝑢12 𝑢13
[𝑎21 𝑎22 𝑎23 ] = [𝑙21 𝑙22 0 ][ 0 𝑢22 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑙31 𝑙32 𝑙33 0 0 𝑢33
Set 𝑙𝑖𝑖 = 1
𝑎11 𝑎12 𝑎13 1 0 0 𝑢11 𝑢12 𝑢13
[𝑎21 𝑎22 𝑎23 ] = [𝑙21 1 0] [ 0 𝑢22 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑙31 𝑙32 1 0 0 𝑢33

𝑎11 𝑎12 𝑎13 𝑢11 𝑢12 𝑢13

[𝑎21 𝑎22 𝑎23 ] = [𝑙21 𝑢11 𝑙21 𝑢12 + 𝑢22 𝑙21 𝑢13 + 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑙31 𝑢11 𝑙31 𝑢12 + 𝑙32 𝑢22 𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑢33
Example: Solve the following system using LU decomposition Method.
Decompose A into LU using Dolittle’s Method.
2𝑥1 + 4𝑥2 + −6𝑥3 = −4
𝑥1 + 5𝑥2 + 3𝑥3 = 10
𝑥1 + 3𝑥2 + 2𝑥3 = 5

2 4 −6 𝑥1 −4
𝐴 = [1 5 3 ] , 𝑋 = [𝑥2 ] , 𝐵 = [ 10 ]
1 3 2 𝑥3 5

Step 1: Decompose A into LU using Dolittle’s Method:

2 4 −6 1 0 0 𝑢11 𝑢12 𝑢13

[1 5 3 ] = [𝑙21 1 0] [ 0 𝑢22 𝑢23 ]
1 3 2 𝑙31 𝑙32 1 0 0 𝑢33
 2 4 −6 𝑢11 𝑢12 𝑢13
[1 5 3 ] = [𝑙21 𝑢11 𝑙21 𝑢12 + 𝑢22 𝑙21 𝑢13 + 𝑢23 ]
1 3 2 𝑙31 𝑢11 𝑙31 𝑢12 + 𝑙32 𝑢22 𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑢33
From Row 1:
𝒖𝟏𝟏 = 2 , 𝒖𝟏𝟐 = 4 , 𝒖𝟏𝟑 = −6
From Column 1:
𝑎21 1 𝑎31 1
𝒍𝟐𝟏 = = 𝒍𝟑𝟏 = =
𝑢11 2 𝑢11 2
From Row 2:
𝒖𝟐𝟐 = 𝑎22 − 𝑙21 𝑢12 = 3 , 𝒖𝟐𝟑 = 𝑎23 − 𝑙21 𝑢12 = 6

From Column 2:
𝑎32 − 𝑙31 𝑢12 1
𝒍𝟑𝟐 = =
𝑢22 3
From Row 3:
𝒖𝟑𝟑 = 𝑎32 − 𝑙31 𝑢13 − 𝑙32 𝑢23 = 3
 𝑨 = 𝑳𝑼

𝟐 𝟒 −𝟔 𝟏 𝟎 𝟎 𝟐 𝟒 −𝟔
[𝟏 𝟓 𝟑 ] = [𝟏/𝟐 𝟏 𝟎] [𝟎 𝟑 𝟔]
𝟏 𝟑 𝟐 𝟏/𝟐 𝟏/𝟑 𝟏 𝟎 𝟎 𝟑

Step 2: Solving 𝐿𝑌 = 𝐵
1 0 0 𝑦1 −4
[1/2 1 0] [𝑦2 ] = [ 10 ]
1/2 1/3 1 𝑦3 5

𝑦1 + 3𝑥2 − 1𝑥3 = −4
1
𝑦 + 𝑦2 + 0.5𝑥3 = +10
2 1
1 1
𝑦1 + 𝑦2 + 𝑦3 = 5
2 3
By applying forward substitution
10 − 0.5(−4)
⟹ 𝑦1 = −4 , 𝑦2 = = 12
1
1
5 − 0.5(−4) − ( )(12)
⟹ 𝑦3 = 3 =3
1

𝑦1 −4
[𝑦2 ] = [ 12 ]
𝑦3 3

Step 3: Solving 𝑈𝑋 = 𝑌 to find solution 𝑋.

2 4 −6 𝑥1 −4
[0 3 6 ] [𝑥2 ] = [ 12 ]
0 0 3 𝑥3 3
 2𝑥1 + 4𝑥2 − 6𝑥3 = −4
3𝑥2 + 6𝑥3 = 12
3𝑥3 = 3
By applying back substitution
12 − 6(1)
⟹ 𝑥3 = 1 , 𝑥2 = =2
3
−4 + 6(1) − 4(2)
⟹ 𝑥1 = = −3
2
So, Solution of given system of linear equations is:

𝑥1 −3
[𝑥2 ] = [ 2 ]
𝑥3 1
2. Crout’s Method:
Consider a general 3 x 3 coefficient matrix
𝑎11 𝑎12 𝑎13 𝑙11 0 0 𝑢11 𝑢12 𝑢13
[𝑎21 𝑎22 𝑎23 ] = [𝑙21 𝑙22 0 ][ 0 𝑢22 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑙31 𝑙32 𝑙33 0 0 𝑢33

Set 𝑢𝑖𝑖 = 1

𝑎11 𝑎12 𝑎13 𝑙11 0 0 1 𝑢12 𝑢13

[𝑎21 𝑎22 𝑎23 ] = [𝑙21 𝑙22 0 ] [0 1 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑙31 𝑙32 𝑙33 0 0 1

𝑎11 𝑎12 𝑎13 𝑙11 𝑙11 𝑢12 𝑙11 𝑢13

[𝑎21 𝑎22 𝑎23 ] = [𝑙21 𝑙21 𝑢12 + 𝑙22 𝑙21 𝑢13 + 𝑙22 𝑢23 ]
𝑎31 𝑎32 𝑎33 𝑙31 𝑙31 𝑢12 + 𝑙32 𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑙33
Example: Solve the following system using LU decomposition Method.
Decompose A into LU using Crout’s Method.
2𝑥1 + 4𝑥2 + −6𝑥3 = −4
𝑥1 + 5𝑥2 + 3𝑥3 = 10
𝑥1 + 3𝑥2 + 2𝑥3 = 5

2 4 −6 𝑥1 −4
𝐴 = [1 5 3 ] , 𝑋 = [𝑥2 ] , 𝐵 = [ 10 ]
1 3 2 𝑥3 5

Step 1: Decompose A into LU using Crout’s Method:

2 4 −6 𝑙11 0 0 1 𝑢12 𝑢13

[1 5 3 ] = [𝑙21 𝑙22 0 ] [0 1 𝑢23 ]
1 3 2 𝑙31 𝑙32 𝑙33 0 0 1
 2 4 −6 𝑙11 𝑙11 𝑢12 𝑙11 𝑢13
[1 5 3 ] = [𝑙21 𝑙21 𝑢12 + 𝑙22 𝑙21 𝑢13 + 𝑙22 𝑢23 ]
1 3 2 𝑙31 𝑙31 𝑢12 + 𝑙32 𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑙33
From Column 1:
𝒍𝟏𝟏 = 2 , 𝒍𝟐𝟏 = 1 , 𝒍𝟑𝟏 = 1
From Row 1:
𝑎12 𝑎13
𝒖𝟏𝟐 = = 2 𝒖𝟏𝟑 = = −3
𝑙11 𝑙11
From Column 2:
𝒍𝟐𝟐 = 𝑎22 − 𝑙21 𝑢12 = 3 , 𝒍𝟑𝟐 = 𝑎32 − 𝑙31 𝑢12 = 1

From Row 2:
𝑎23 − 𝑙21 𝑢13
𝒖𝟐𝟑 = =2
𝑙22
From Column 3:
𝒍𝟑𝟑 = 𝑎33 − 𝑙31 𝑢13 + 𝑙32 𝑢23 = 3
 𝑨 = 𝑳𝑼

𝟐 𝟒 −𝟔 𝟐 𝟎 𝟎 𝟏 𝟐 −𝟑
[𝟏 𝟓 𝟑 ] = [𝟏 𝟑 𝟎] [𝟎 𝟏 𝟐]
𝟏 𝟑 𝟐 𝟏 𝟏 𝟑 𝟎 𝟎 𝟏

Step 2: Solving 𝐿𝑌 = 𝐵
2 0 0 𝑦1 −4
[1 3 0] [𝑦2 ] = [ 10 ]
1 1 3 𝑦3 5

2𝑦1 + 3𝑥2 1𝑥3 = −4

𝑦1 + 3𝑦2 + 0.5𝑥3 = +10
𝑦1 + 𝑦2 + 3𝑦3 = 5

By applying forward substitution

 10 − 1(−2)
⟹ 𝑦1 = −2 , 𝑦2 = =4
3
5 − 1(−2) − 1(4)
⟹ 𝑦3 = =1
3

By applying forward substitution

𝑦1 −2
[𝑦2 ] = [ 4 ]
𝑦3 1

Step 3: Solving 𝑈𝑋 = 𝑌 to find solution 𝑋.

1 2 −3 𝑥1 −2
[0 1 2 ] [𝑥2 ] = [ 4 ]
0 0 1 𝑥3 1
 𝑥1 + 2𝑥2 − 3𝑥3 = −2
𝑥2 + 2𝑥3 = 4
𝑥3 = 1
By applying back substitution
4 − 2(1)
⟹ 𝑥3 = 1 , 𝑥2 = =2
1
−2 + 3(1) − 2(2)
⟹ 𝑥1 = = −3
1
So, Solution of given system of linear equations is:

𝑥1 −3
[𝑥2 ] = [ 2 ]
𝑥3 1
Computational Cost:
• Computational cost of Gauss-Elimination Method is 𝑂(𝑛3 ). Since it must
2𝑛3
perform steps to solve 𝑛𝑥𝑛 system.
3

• Computational cost of solving triangular system is 𝑂(𝑛2 ).

Book Burden and Faires