Professional Documents
Culture Documents
Gaussian Elimination, LU - Factorization, Cholesky Factorization, Reduced Row Echelon Form
Gaussian Elimination, LU - Factorization, Cholesky Factorization, Reduced Row Echelon Form
Gaussian Elimination,
LU -Factorization, Cholesky
Factorization, Reduced Row Echelon
Form
237
238CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
Clearly, C(0) = b0, C(1) = b3, and for t 2 [0, 1], the
point C(t) belongs to the convex hull of the control points
b0, b1, b2, b3.
The polynomials
(1 t)3, 3(1 t)2t, 3(1 t)t2, t3
are the Bernstein polynomials of degree 3.
b1
b2
b0 b3
b1
b3
b0
b2
b2 b1
b0 b3
d1
x2
x1 d7
d3
x3
d0 d6
x6
x4
x5
d4
d5
x0 = d 1 x7 = d8
Figure 4.4: A C 2 cubic interpolation spline curve passing through the points x0 , x1 , x2 , x3 ,
x 4 , x 5 , x 6 , x7
244CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
This yields
xn = an n1 bn
xn 1 = an 1 1 n 1(bn 1 an 1 nxn)
..
x1 = a1 11(b1 a1 2x2 · · · a1 nxn).
4.2. GAUSSIAN ELIMINATION AND LU -FACTORIZATION 249
2x + y + z = 5
4x 6y = 2
2x + 7y + 2z = 9.
2x + y + z = 5
8y 2z = 12
8y + 3z = 14.
2x + y + z = 5
8y 2z = 12
z = 2.
This last system is upper-triangular.
4.2. GAUSSIAN ELIMINATION AND LU -FACTORIZATION 251
x + y + z =1
x + y + 3z = 1
2x + 5y + 8z = 1.
x + y + z =1
2z = 0
3y + 6z = 1.
x + y + z =1
3y + 6z = 1
2z = 0,
z = 1
2x + 7y + 2z = 1
4x 6y = 1.
2x + 7y + 2z = 1
8y + 4z = 1
z = 1,
an upper-triangular system.
(k)
If we let A1 = A and Ak = (ai j ) be the matrix obtained
after k 1 elimination steps (2 k n), then the kth
elimination step is applied to the matrix Ak of the form
0 (k) (k) (k)
1
a1 1 a1 2 · · · · · · · · · a1 n
B (k) (k) C
B a2 2 · · · · · · · · · a2 n C
B . . . .. .. C
B C
Ak = B (k) (k) C .
B a · · · a C
B k
..
k k
.. C
n
@ A
(k) (k)
an k · · · an n
Actually, note
(k) (i)
ai j = ai j
for all i, j with 1 i k 2 and i j n, since
the first k 1 rows remain unchanged after the (k 1)th
step.
4.2. GAUSSIAN ELIMINATION AND LU -FACTORIZATION 257
(k)
So, one the entries ai k with k i n can be chosen
as pivot, and we permute the kth row with the ith row,
(k)
obtaining the matrix ↵(k) = (↵j l ).
(k)
The new pivot is ⇡k = ↵k k , and we zero the entries i =
(k)
k + 1, . . . , n in column k by adding ↵i k /⇡k times row
k to row i. At the end of this step, we have Ak+1.
giving the (i, j)th entry in A shows that the ith row of
A is a linear combination of the rows of B:
Ai = pi1B1 + · · · + pimBm.
Ak+1 = Ek Pk Ak ,
and then
Ak = E k 1 P k 1 · · · E1 P1 A.
We always have
det(Ak ) = ± det(A).
4.2. GAUSSIAN ELIMINATION AND LU -FACTORIZATION 263
Furthermore, since
Ak = E k 1 P k 1 · · · E 1 P1 A
An = E n 1 P n 1 · · · E 2 P2 E 1 P1 A
is upper-triangular .
At every stage, k, after the pivot has been found and piv-
oting been performed, if necessary, in addition to adding
suitable multiples of the kth row to the rows below row
k in order to zero the entries in column k for i = k +
1, . . . , n, also add suitable multiples of the kth row to
the rows above row k in order to zero the entries in col-
umn k for i = 1, . . . , k 1.
A = LDU 0,
and
0 1
1 ··· 0 0 ··· 0
B .. ... .. .. .. .. C
B C
B0 ··· 1 0 ··· 0C
Ek 1 =B
B0
C,
B. · · · `k+1k 1 ··· 0C
@. .. .. .. ... .. C
A
0 · · · `nk 0 ··· 1
namely A = LU .
276CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
L = I + ⇤.
0 0 0 1
4.2. GAUSSIAN ELIMINATION AND LU -FACTORIZATION 281
and
0 10 1
1 0 0 0 1 1 1 0
B0 0 1 0C B 1 0C
PA = B C B1 1 C
@0 1 0 0 A @1 1 0 1A
0 0 0 1 1 1 0 1
0 1
1 1 1 0
B1 1 0 1C
=B@1
C.
1 1 0A
1 1 0 1
282CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
Then,
Ejk = Pk Pk 1 · · · Pj+1Ej Pj+1 · · · Pk 1Pk
U = Enn 11 · · · E1n 1Pn 1 · · · P1A,
and if we set
P = P n 1 · · · P1
L = (E1n 1) 1 · · · (Enn 11) 1,
then
P A = LU.
4.2. GAUSSIAN ELIMINATION AND LU -FACTORIZATION 285
Furthermore,
(Ejk ) 1 = I + Ejk , 1jn 1, j k n 1,
where Ejk is a lower triangular matrix of the form
0 1
0 ··· 0 0 ··· 0
B .. . . . .. .. .. .. C
B C
B0 · · · 0 0 · · · 0 C
(k) B C
Ej = B (k) C,
B 0 · · · ` 0 · · · 0 C
B . . j+1j . . . C
@. . . . . . . .A
(k)
0 · · · `nj 0 · · · 0
we have
Ejk = I Ejk ,
and
0 1
0 0 0 0 0 ··· ··· 0
B (k) .. .. C
B 21 0 0 0 0 0C
B (k) (k) ... .. .. C
B 0 0 0C
B 31
..
32
.. ... .. .. .. C
B 0 0 C
⇤k = B
B
C
0C
(k) (k) (k)
B k+11 k+12 ··· k+1k 0 ··· ··· C
B (k) (k) (k) ... C
B k+21 k+22 ··· k+2k 0 ··· 0C
B .. .. ... .. .. .. ... .. C
@ A
(k) (k) (k)
n1 n2 ··· nk 0 ··· ··· 0
Then
I + ⇤k = (E1k ) 1 · · · (Ekk ) 1
⇤k = E1k · · · Ekk ,
L = I + ⇤n 1 .
3 1 1 4
We check that
0 10 1
0 1 0 0 1 2 3 4
B0 0 0 1C B
4 8 12 8C
PA = B
@1 0
CB C
0 0A @
2 3 2 1A
0 0 1 0 3 1 1 4
0 1
4 8 12 8
B 3 1 1 4C
=B@1
C,
2 3 4A
2 3 2 1
and that
0 10 1
1 0 0 0 4 8 12 8
B 3/4 1 0 0C B0 5 10 10C
B
LU = @ C B C
1/4 0 1 0 @0 0 6
A 6 A
1/2 1/5 1/3 1 0 0 0 1
0 1
4 8 12 8
B 3 1 1 4C
=@B C = P A.
1 2 3 4A
2 3 2 1
294CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
10 4x + y = 1
x + y = 2.
4
Since 10 is nonzero, it can be taken as pivot, and we
get
10 4x + y = 1
(1 104)y = 2 104.
x + y = 2
(1 10 4)y = 1 2 ⇥ 10 4.
0 = 1,
1 = b1 ,
k = bk k 1 ak ck 1 k 2, 2 k n.
0 1
1
B 0 C
Ba2 1 C
B C
B 1 C
B 1 C
B a3 1 C
B 2 C
A=B ... ... C
B C
B C
B an
n 3
1 C
B 1 C
B n 2 C
@ n 2 A
an 1
0 n 1 1
1
B c1 C
B 0 C
B 2 C
B c2 C
B 1 C
B C
B 3 C
B c3 C
B 2 C.
B ... ... C
B C
B C
B n 1
cn 1 C
B C
B n 2 C
@ n A
n 1
302CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
c1
z1 = ,
b1
k 1
z k = ck , 2kn 1,
k
n
zn = = bn anzn 1,
n 1
1
A = (L )( U ) is written as
4.3. GAUSSIAN ELIMINATION OF TRIDIAGONAL MATRICES 303
0c 1
1
B z1 C
B c2 C
B a2 C
B z2 C
B c C
B a3 3 C
A=B z3 C
B . . . . C
B . . C
B cn 1 C
B an 1 C
@ zn 1 A
an z n
0 1
1 z1
B C
B C
B 1 z2 C
B C
B C
B C
B 1 z3 C
B C
B C
B C
B ... ... C.
B C
B C
B C
B 1 zn 2 C
B C
B C
B C
B 1 zn 1 C
B C
@ A
1
304CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
c1
z1 = ,
b1
ck
zk = , k = 2, . . . , n 1,
bk ak zk 1
z n = b n an z n 1 ,
d1 dk ak wk 1
w1 = , wk = , k = 2, . . . , n,
b1 bk ak zk 1
xn = w n , xk = w k zk xk+1, k=n 1, n 2, . . . , 1,
1
corresponding to solving the system U x = w.
4.3. GAUSSIAN ELIMINATION OF TRIDIAGONAL MATRICES 305
j 1
!1/2
X
bj j = aj j b2j k ,
k=1
For example, if
0 1
1 1 1 1 1 1
B1 2 2 2 2 2C
B C
B1 2 3 3 3 3C
A=B
B1
C,
B 2 3 4 4 4CC
@1 2 3 4 5 5A
1 2 3 4 5 6
we find that
0 1
1 0 0 0 0 0
B1 1 0 0 0 0C
B C
B1 1 1 0 0 0C
B=B
B1
C.
B 1 1 1 0 0CC
@1 1 1 1 1 0A
1 1 1 1 1 1
310CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
0 1 0 1
1 0 2 1 5 1 0 2 1 5
A2 ! @0 2 6 3 7A ! @0 1 3 3/2 7/2A
0 1 3 1 2 0 1 3 1 2
0 1
1 0 2 1 5
! A3 = @0 1 3 3/2 7/2 A ,
0 0 0 1/2 3/2
For example if
0 1 0 1
1 0 2 1 5
A = @1 1 5 2 A and b = @ 7 A ,
1 2 8 4 12
x1 + 2x3 = 2
x2 + 3x3 = 1
x4 = 3.
320CHAPTER 4. GAUSSIAN ELIMINATION, LU, CHOLESKY, REDUCED ECHELON
x1 = 2 2x3, x2 = 1 3x3, x4 = 3,
where x3 is arbitrary.
For example:
✓ ◆ ✓ ◆
5 4 1 0 5 4 1 0
(A, I) = !
6 5 0 1 1 1 1 1
Then, because
✓ ◆✓ ◆ ✓ ◆
Ir F d d
= = b0 ,
0m r,r 0m r,n r 0n r 0m r
we see that
✓ ◆
d
x0 =
0n r
and since each row (and column) sums to the same num-
ber, this common value (the magic sum) is
n(n2 + 1)
.
2
x1 + x2 >5
x1 < 10
x2 < 10
2x1 + x2 < 20
2x1 + x2 > 10
x1 >0
x2 >0
x1 + x2 < 15,