Linear Algebra and Applications

EEF281E
Week #4

Linear Algebra and Applications 1

EXAMPLE-2: 𝐹𝐸 −1
= 𝐸 −1 𝐹 −1 In general ;
Now, consider the elimination matrices :
1 0 0 1 0 0
1 0 0 1 0 0 𝐸 −1 −1
= 5 1 0 , 𝐹 = 0 1 0
1
𝐸 = −5 1 0 , 𝐹 = 0 1 0 0 0 1 0 4 1 𝑃𝑖𝑗 = ⋱
0 0 1 0 −4 1 0 1
Find 𝐹𝐸 and 𝐹𝐸 −1
=? 1 0 0 1 0
−1 −1
𝐸 𝐹 = 5 1 0
0 4 1
For example;
1 0 0 Subtract 5 times row1 Here, we see that row3 feels no effect 𝑃23
𝐸 = −5 1 0 from row2 from row1.
1 0 0 𝑥 𝑥
0 0 1
Permutation matrices : 0 0 1 𝑦 = 𝑧
1 0 0 Subtract 4 times row2 𝑦
𝐹= 0 1 0 Sometimes we need row exchanges, 0 1 0 𝑧
from row3
0 −4 1 We perform these exchanges by
Row2 and row3 exchanged.
permutation matrices. For example :
1 0 0 1 0 0 1 0 0
𝐹𝐸 = 0 1 0 −5 1 0 = −5 1 0 0 1 𝑥 𝑦 1 0 0 2 4 1 2 4 1
= 0 0 1 0 0 3 = 0 6 5
0 −4 1 0 0 1 20 −4 1 1 0 𝑦 𝑥
0 1 0 0 6 5 0 0 3
0 1
Here, we see that row3 feels an effect from row1. 𝑃12 =
1 0

Linear Algebra and Applications 2

 Augmented matrix Calculating 𝑨−𝟏 by Gauss-Jordan elimination
The idea is to solve 𝐴𝐴−1 = 𝐼 and find each column of 𝐴−1 .
Let’s come back to 𝐴𝒙 = 𝒃 → 𝐸𝐴𝒙 = 𝐸𝒃
Elimination performs the same row 𝐴−1 𝐼
operations to 𝐵 as it does to 𝐴. 𝐴 𝑥റ1 𝑥റ2 𝑥റ3 = 𝑒റ1 𝑒റ2 𝑒റ3
Therefore, 𝐴 can be enlarged or augmented
by an extra column 𝒃. Here, 𝑥റ1 𝑥റ2 𝑥റ3 are the column vectors of 𝐴−1 and 𝑒റ1 𝑒റ2 𝑒റ3
are the column vectors of 𝐼.
For example consider ;
2 4 −2 𝑥 2
𝑥11 𝑥21 𝑥31 1 0 0
𝐴𝒙 = 𝒃; 4 9 −3 𝑦 = 8 𝑥12 𝑥22 𝑥32
0 1 0
−2 −3 7 𝑧 10 𝐴 =
𝑥
ด 3
ด23
𝑥 𝑥
ด3
3 ณ
0 ณ
0 ณ
1
1
𝑒റ1 𝑒റ2 𝑒റ3
2 4 −2 2 𝐴𝑛𝑦 𝑥റ1 𝑥റ2 𝑥റ3
Augmented
𝐴 𝑏 = 4 9 −3 8 𝑚𝑎𝑡𝑟𝑖𝑥
matrix 𝑡𝑜 𝑏𝑒
−2 −3 7 10 𝑖𝑛𝑣𝑒𝑟𝑡𝑒𝑑
1 0 0 2 4 −2 2
This can be done by
𝐸21 𝐴 𝑏 = −2 1 0 4 9 −3 8
𝑐𝑜𝑛𝑣𝑒𝑟𝑡
0 0 1 −2 −3 7 10
𝐴 𝐼 𝐼 𝐴−1 by GJ elimination steps
2 4 −2 2 𝐴𝑢𝑔𝑚𝑒𝑛𝑡𝑒𝑑
= 0 1 1 4 𝑚𝑎𝑡𝑟𝑖𝑥
−2 −3 7 10
Linear Algebra and Applications 3
 3 1 1
EXAMPLE-3: 2 −1 0 1 0 0
3 3 3 3 3 4 2 4
𝑅𝑜𝑤3+𝑅𝑜𝑤2
Invert K by using GJ 4 0 0 1 1
2 4 2 4 𝐾 −1 = 1 obtained by G-J
elimination methods 4 1 2 2 2 elimination.
0 0 1 1 1 3
2 −1 0 3 3 3
4 2 4
𝐾 = −1 2 −1
0 −1 2 3 1
2 0 0 1
2 3 2 2 Here, the product of the pivots
𝑅𝑜𝑤2+𝑅𝑜𝑤1
3 0 0 3 3 3 3 4
is 2 × × = 4.
2 2 3
2 −1 0 1 0 0 4 4 2 4
𝐾 𝐼 = −1 2 −1 0 1 0 0 0 1 2
3 1
0 −1 2 0 0 1 3 3 −1
1 3 2 1
𝐾 = 2 4 2
Now, K is diagonal. 4
1 2 −1 0 1 0 0 1 2 3
𝑅𝑜𝑤1+𝑅𝑜𝑤2
2 3 1
0 −1 1 0 3 1 1
2 2 4 2 4
0 −1 2 0 0 1 1 2 3
𝑅𝑜𝑤1, 𝑅𝑜𝑤2, 𝑅𝑜𝑤3 1 0 0 Another notable thing is that
2 3 4 1 1
0 1 0 1 K is symmetric across its
2 2
0 0 1 1 1 3 main diagonal and 𝐾 −1 is
2 −1 0 1 0 0 4 2 4 also symmetric.
2 3 1
𝑅𝑜𝑤2+𝑅𝑜𝑤3
3 0 −1 1 0
2 2 = 𝐼 𝐾 −1
4 1 2
0 0 1
3 3 3
Now, triangular form of K
is obtained : U . Linear Algebra and Applications 4
Invertible matrices 𝐸 −1 𝐸𝐴 = 𝐸 −1 𝑈 1 0 2 1 2 1
=
−3 1 6 8 0 5
𝐸21 𝐴 𝑈
An invertible 𝑛 × 𝑛 matrix exists exactly Here let’s denote
when A has a full set of 𝑛 pivots.
(Row changes are allowed. )
𝐿 = 𝐸 −1 𝐴 = 𝐸21 −1 𝑈
(Backward protocol ≡ Factorization )
If matrix 𝐴𝑛×𝑛 does not have 𝑛 pivots, 𝐴 = 𝐿𝑈
elimination will produce a zero row. 1 0
𝐸21 −1 =
This means that 3 1
An invertible matrix can’t have a zero row.
Elimination ≡ Factorization
If it has, that is called a singular matrix.
1 0 2 1 2 1
Factorization is very important for =
3 1 0 5 6 8
Note: Remember that, matrices due to some reasons.
Factorization
𝐸 EXAMPLE-4:
(𝐸𝑖𝑗 … 𝐸31 𝐸21 )𝐴𝒙 = 𝐸𝒃 2 1 For 3 × 3 matrices,
Factorize 𝐴 = by using
6 8
Here, 𝐸𝐴 = 𝑈 elimination matrices. 𝐸32 𝐸31 𝐸21 𝐴 = 𝑈
𝑈 𝑖𝑠 𝑢𝑝𝑝𝑒𝑟 𝑡𝑟𝑖𝑎𝑛𝑔𝑢𝑙𝑎𝑟 1 0 𝐿
𝐸 𝑖𝑠 𝑙𝑜𝑤𝑒𝑟 𝑡𝑟𝑖𝑎𝑛𝑔𝑢𝑙𝑎𝑟 𝐸21 = −1 −1
−3 1 ⇒ 𝐴 = 𝐸21 𝐸31 𝐸32 −1 𝑈
𝐸21 𝐴 = 𝑈 (Forward protocol)

Linear Algebra and Applications 5

EXAMPLE-5:
1 0 0 1 0 0
2 1 0 1 0 1 0
𝐸21 −1 = 1 0 𝐸32 −1 = 2
Factorize 𝐴 = 1 2 1 by
2 0 1
0 1 2 0 0 1 3
using elimination matrices.

𝐸 = 𝐸32 𝐸21 𝐸 −1 = 𝐸21 −1 𝐸32 −1

2 1 0 1
𝐸21 :Subtract
2 1 0 𝐸21 3 2
𝐴= 1 2 1 0 1 times row1 from 1 0 0
2
0 1 2
0 1 2 row2 1
1 0
𝐸 −1 = 2 =𝐿
2 1 0 2 2
3 𝐸32 : Subtract
3 0 1
𝐸32 0 1 times row2 from 3
2
4 row3. 𝐿 𝑈
0 0
3 1 0 0 2 1 0
𝑈
2 1 0 1 3
1 0 0 1
𝐴= 1 2 1 = 2 2
0 1 2 2 4
1 0 0 1 0 0 0 1 0 0
1 0 1 0 3 3
𝐸21 = − 1 0 𝐸32 = 2
2 0 − 1
0 0 1 3
Linear Algebra and Applications 6
LDU form
Sometimes it is useful to split U as ;

𝑑1 𝑢12 … 𝑢1𝑛 𝑑1 1 𝑢12 /𝑑1 𝑢13 /𝑑1 ..

𝑑2 𝑢23 𝑑2 1 𝑢23 /𝑑2 ..
=
⋱ ⋮ ⋱ 1
𝑑𝑛 𝑑𝑛 1

Here, we call this decomposition as 𝑈 ′ = 𝐷𝑈

𝐷: Pivot matrix

Then finally we have A = 𝐿𝐷𝑈 form.

Linear Algebra and Applications 7

 1 0
Transpose matrix : 𝐴+𝐵 𝑇
= 𝐴𝑇 + 𝐵𝑇 EXAMPLE: 𝐴=
6 1
𝐴𝐵 𝑇 = 𝐵𝑇 𝐴𝑇
𝑇
𝐴 : Transpose of 𝐴. 1 0 1 −6
𝐴−1 = ⟶ 𝐴−1 𝑇 =
−6 1 0 1
𝑇 𝐴−1 𝑇
= 𝐴𝑇 −1
Same
𝐴𝑖𝑗 = 𝐴𝑗𝑖 1 6 1 −6
𝐴𝑇 = ⟶ 𝐴𝑇 −1 =
0 1 0 1
1 2 3 If 𝐴 = 𝐿𝐷𝑈,
𝐴= 𝐴−1 𝑇 = 𝐴𝑇 −1 confirmed
0 0 4 2×3 𝐴𝑇 = 𝑈 𝑇 𝐷𝑇 𝐿𝑇
This is a flip operation Inner product revisited
over main diagonal. 𝐴−1 𝐴 = 𝐼 𝐴−1 𝐴 𝑇 = 𝐼𝑇 Remember that 𝒙. 𝒚 = σ𝒊 𝑥𝑖 𝑦𝑖
1 0
⇒ 𝐴𝑇 𝐴−1 𝑇
=𝐼 Consider
𝐴𝑇 = 2 0 𝑥1 𝑦1
3 4 3×2
Vice versa. 𝒙= ⋮ 𝒚= ⋮
𝑎 𝑥𝑛 𝑦𝑛
𝐴= 𝑏 , This means that 𝐴𝑇 is 𝑦1
𝐴𝑇 = 𝑎 𝑏 𝑐,
𝑐 invertible when 𝐴 is 𝒙. 𝒚 = 𝑥1 … 𝑥𝑛 ⋮ = 𝒙𝑇 𝒚
invertible. 𝑦𝑛
𝒙. 𝒚 = 𝒙𝑇 𝒚

Linear Algebra and Applications 8

Symmetric matrix : More on permutation matrices:
A symmetric matrix has A permutation matrix 𝑃 has the rows of
the property; the identity I in any order.
There are 𝑛! Permutation matrices of order 𝑛.
𝑆𝑇 =𝑆
1 2 1 2 1 0 0 1
𝑆= = 𝑆𝑇 = 𝒏=𝟐: 𝐼 = , 𝑃21 =
2 5 2 5 0 1 1 0
1 0 0 0 1 0 0 0 1
1 0
𝐷= = 𝐷𝑇 𝒏=𝟑: 𝐼 = 0 1 0 , 𝑃21 = 1 0 0 , 𝑃31 = 0 1 0
0 5 0 0 1 0 0 1 1 0 0
5 −2
𝑆 −1 = 1 0 0 0 1 0 0 0 1
−2 1
𝑃32 = 0 0 1 , 𝑃32 𝑃21 = 0 0 1 , 𝑃21 𝑃32 = 1 0 0
= 𝑆 −1 𝑇 = 𝑆 𝑇 −1 0 1 0 1 0 0 0 1 0

For permutation matrices 𝑃−1 = 𝑃𝑇 .

The inverse of a symmetric
matrix is also symmetric.

Linear Algebra and Applications 9

 𝑈

Note: 𝑙31 =2 1 2 1 1 2 1
𝑙32 =3
Check that 𝑃21 −1 = 𝑃21 = 𝑃21 𝑇 . 0 1 1 0 1 1
0 3 7 0 0 4
𝑃32 −1 = 𝑃32 𝑇 = 𝑃32
Upper triangular matrix 𝑈 obtained.
𝑃31 −1 = 𝑃31 𝑇 = 𝑃31
We can easily obtain
But, 𝐿 = 𝐸32 𝐸31 −1
𝑃 𝐴 𝐿 𝑈
𝑃32 𝑃21 −1 = 𝑃21 −1 𝑃32 −1
0 1 0 0 1 1 1 0 0 1 2 1
= 𝑃21 𝑇 𝑃32 𝑇 1 0 0 1 2 1 = 0 1 0 0 1 1
0 0 1 2 7 9 2 3 1 0 0 4
= 𝑃21 𝑃32
Note: 𝑃𝐴 = 𝐿𝑈 factorization is
possible with row exchanges.
0 1 1 1 2 1
𝑃𝐴
1 2 1 0 1 1
2 7 9 2 7 9
𝐴

Linear Algebra and Applications 10