Professional Documents
Culture Documents
4.3. The QR Reduction: QR A A QR (Q (R ( (Ax B ( ( (Ax B (Ax X
4.3. The QR Reduction: QR A A QR (Q (R ( (Ax B ( ( (Ax B (Ax X
4.3. The QR Reduction: QR A A QR (Q (R ( (Ax B ( ( (Ax B (Ax X
The QR Reduction
Reading Trefethen and Bau (1997), Lecture 7 The QR factorization of a matrix A 2 <m n is
{ Q 2 <m m is an orthogonal matrix { R 2 <m n is upper triangular { Assume (for the moment) m n
Motivation: { An alternate to Gaussian elimination when solving Ax = b More expensive than Gaussian elimination { Determining bases for orthogonal subspaces { Determining numerical rank through the SVD { Solving least squares problems min jjAx ; bjj2 x
A = QR
(1)
Householder QR Factorization
Use Householder re ections to reduce successive columns of A to zero below their main diagonals { Suppose m = 6, n = 4 2 3 a11 a12 a13 a14 6 a21 a22 a23 a24 7 6 6 a31 a32 a33 a34 7 7 6 A=6a a a a 7 6 41 42 43 44 7 7 6 7 4 5
a51 a52 a53 a54 a61 a62 a63 a64 r12 a(1) 22 a(1) 32 a(1) 42 a(1) 52 a(1) 62 r13 a(1) 23 a(1) 33 a(1) 43 a(1) 53 a(1) 63
Householder QR Factorization
Use a re ection to annihilate a(1), i = 3 : 6 i2 2 (1) 3 2 3 a 6 22 7 6 r22 7 6 a(1) 7 6 0 7 6 32 7 6 7 P2 6 a(1) 7 = 6 0 7 6 42 7 6 7 6 a(1) 7 4 0 5 4 52 5 0 a(1) 62
1 0 H2 = 0 P 2
2 r11 6 0 6 6 6 H2H1A = 6 0 6 0 6 6 0 4 0
Householder QR Factorization
Repeat the process on the third column 2 r11 r12 r13 6 0 r22 r23 6 6 0 0 r33 H3H2H1A = 6 0 0 0 6 6 6 4 0 0 0 0 0 0 One more should do it
2 r11 6 0 6 6 0 H4H3H2H1A = 6 0 6 6 6 0 4 0
Householder QR Factorization
Algorithm for a Householder QR factorization function A = houseqr(A) % houseqr: Overwrite A 2 <m n with the product % QR where Q 2 <m m is orthogonal and % R 2 <m n is upper triangular. On return, % the essential part of Q is stored in the lower triangular % part of A and R is stored in the upper triangular part. for j = 1 : n v(j:m) = house(A(j:m,j)) A(j:m,j:n) = row.house(A(j:m,j:n), v(j:m)) if j < m A(j+1:m,j) = v(j+1:m) end end
Householder QR Factorization
As described in Example 2.4,
v(j)(v(j))T Hj = I ; 2 (v(j))T v(j) { v(j) is the Householder vector v used at step j { v(j) has the form ) ( v(j) = 0 0 ::: 0 1 vj(j+1 ::: vmj)]T
(2a)
(2b)
Normalize the Householder vector so that vj(j ) = 1 { This requires modi cation of the function house of Section 4.2. All components of v are divided by v(1) { The renormalization saves a storage location and allows v(j) to be stored in the lower part of A Upon completion of the algorithm 3 2 r11 r12 r13 r14 6 v(1) r22 r23 r24 7 7 6 2 6 (1) (2) 6 v3 v3 r33 r34 7 7 6 (1) (2) (3) (2c) A=6v v v r 7 7 44 6 4 4 4 (4) 7 6 v(1) v(2) v(3) v 7 4 5 5 5 5 5
(1) (2) (3) (4) v6 v6 v6 v6
Householder QR Factorization
QT = HnHn;1 H1 so Q = H1H2 Hn { Q rarely needs explicit construction
(3)
The factorization requires about 2n2(m ; n=3) FLOPs ( multiplications plus additions) { With m = n we require about 4n3=3 FLOPs { Gaussian elimination requires about 2n3=3 FLOPs With roundo , we compute ^ ^ QT (A + A) = R ^ { Q is exactly orthogonal ^ { R is upper triangular { The perturbation jj Ajj2 ujjAjj2 { This is acceptable
Givens QR Factorization
We can also use Givens rotations to construct the QR decomposition { The order of the reduction is { stands for a nonzero entry { The numbers indicate the rows that are rotated 2 3 2 3 2 3 6 7 7 26 7 6 7= 6 6 7= 6 7 A=6 7 36 7 360 7 4 5 4 5 4 5 4 0 0 2 3 2 3 2 3 1 7 7 260 7 260 6 7= 6 0 6 7= 6 A= 6 0 7 360 7 360 0 7 7 4 5 4 5 4 5 0 4 0 0 0 0 2 3 60 7 6 A= 3 6 0 0 7 7 4 5 4 0 0 0
Givens QR Factorization
Algorithm for the Givens QR factorization function A = givensqr(A) % givensqr: Reduce A 2 <m n to upper triangular % form by the Givens QR procedure where Q 2 <m m % is orthogonal and R 2 <m n is upper triangular. % On return, R is stored in the upper triangular part of A. for j = 1:n for i = m: -1: j + 1 c,s] = givens(A(i-1,j), A(i,j)) A(i-1:i,j:n) = row.rot(A(i-1:i,j:n), c, s) end end
{ The procedure does not save Q { The factorization requires about 3n2(m ; n=3) multiplications and additions Re ections require 2n2(m ; n=3) FLOPs Re ections are generally preferred { Let Gi = G(i i ; 1 ), then
st col:
nd col:
n th col:
(Gm} | {z )
Givens QR Factorization
Example 1. Obtain the QR factorization of 2 3 3 2 1 6 2 ;3 4 7 A = 6 5 1 ;1 7 6 7 4 5 7 4 2
s = 0:8137
2:0000 ;3:0000 ;3:8362 ;1:5112
3 0 0 7 7 7 ;0:8137 5 ;0:5812
10
Givens QR Factorization
Rotate the second and third rows using givens(2.0000, -8.6023)
c = 0:2265
s = 0:9740
2 1 0 6 0 0:2265 GT = 6 0 0:9740 6 3 4 0 0
0 ;0:9740 0:2265 0
11
Givens QR Factorization
Rotate the rst and second rows using givens(3.0000, 8.8318)
c = ;0:3216
s = 0:9469
2 ;0:3216 6 0:9469 GT = 6 6 2 0 4 0
;0:9469 ;0:3216
0 0 0 1 0 0
12
Rotate the third and fourth rows using givens(-3.7908, -1.5112) c = 0:9289 s = ;0:3703 { Applying row.rot to A(3)(3 : 4 2 : 3) yields 3 2 ;9:3274 ;3:5380 ;2:1442 6 0 0:9104 0:3278 7 6 7 (4) A =6 0 ;4:0809 2:6672 7 4 5 0 0 ;3:1908 { The Givens matrix2 3 1 0 0 0 60 1 0 07 T =6 G4 6 0 0 0:9289 ;0:3703 7 7 4 5 0 0 0:3703 0:9289 Rotate the second and third rows using givens(0.9104, -4.0809) c = 0:2177 s = 0:9760 { Applying row.rot to A(4)(2 : 3 2 : 3) yields 3 2 ;9:3274 ;3:5380 ;2:1442 6 0 4:1812 ;2:5318 7 7 6 (5) A =6 0 0 0:9007 7 5 4 0 0 ;3:1908 { The Givens matrix2 3 1 0 0 0 6 0 0:2177 ;0:9760 0 7 GT = 6 0 0:9760 0:2177 0 7 6 7 3 4 5 0 0 0 1 { The second column is done
13
Givens QR Factorization
c = 0:2717
s = 0:9624
4:1812 0 0
0 0 1 0 0 0:2717 0 0:9624
3 0 07 7 7 ;0:9624 5 0:2717
14
;3:5380
4:1812 0 0
2 ;0:3216 0:2062 6 ;0:2144 ;0:8989 Q = 6 ;0:5361 ;0:2144 6 4 ;0:7505 0:3216 { Check that QR = A { Check that QT Q = I
nd col:
st col:
15
Theorem 1: Let A 2 <m n have rank n with m n. Let A = QR, then spanfa1 a2 ak g = spanfq1 q2 qk g k = 1 : n
(4a)
qm ]
(4b) (4c) (4d)
The ? denotes the orthogonal complement { In addition A = QnRn Rn = R(1 : n 1 : n) Remark: R = Rn Q = Qn Qm;n 0
{ Thus,
A = QR = Qn Qm;n
Proof: From (1)
Rn = QnRn 0 qk g qk g
(5)
ak =
Thus, spanfa1
k X j =1
rjk qj 2 spanfq1 q2
a2
ak g spanfq1 q2
16
Proof: Consider
{ Comparing this with (6a) yields (6b) { The Cholesky factorization is unique { From (4d) Qn = AR;1 hence, Qn is unique 2 n
17
AT A = RT QT QnRn = RT Rn n n n
i=1:n
(7a)
{ The eigenvectors xi of AT A and the singular vectors vi of A satisfy xi = vi i = 1 : n (7b) { The Euclidean norm q jjAjj2 = (AT A) = 1 (7c) The spectral radius (AT A) is the largest eigenvalue T 1 of A A 1 is the largest singular value of A { The condition number of A in the Euclidean norm satis es (A) (7d) (A) = 1 2 (A)
n
18
We can use this as a de nition of the condition number for a rectangular matrix { An O( ) perturbation in A introduces O( 2(A)) perturbations in Qn and Rn cf. Stewart (1993), On the perturbation of LU Cholesky and QR factorizations, SIAM J. Matrix. Anal., 14, 1141-1145
19