312

APPENDIX D
Summary of Matrix Algebra

SOLUTIONS TO PROBLEMS

 0 1 6
 2 −1 7     20 − 6 12 
=D.1 (i) AB  =  1 8 0  
 −4 5 0    5 36 −24 
 3 0 0

(ii) BA does not exist, because B is 3 × 3 and A is 2 × 3.

D.2 This result is easy to visualize. If A and B are n × n diagonal matrices, then AB is an n × n
diagonal matrix with jth diagonal element a j b j . Similarly, BA is an n × n diagonal matrix with jth
diagonal element b j a j , which, of course, is the same as a j b j .

( X′X′) (=
D.3 Using the basic rules for transpose,= X′)( X′)′ X′X , which is what we wanted to
show.

D.4 (i) This follows from tr(BC) = tr(CB), when B is n × m and C is m × n. Take B = A′ and
C = A.

 2 0  4 0 −2 
   2 0 −1  
(ii) A′A =  0 3    =  0 9 0  ; therefore, tr(A′A) = 14.
 −1 0   0 3 0   −2 0 1
  

 2 0
 2 0 −1    5 0
Similarly, AA′ =    0 3 =   , and so tr(AA′) = 14.
 0 3 0   0 9
 −1 0 

D.5 (i) The n × n matrix C is the inverse of AB if and only if C(AB) = I n and (AB)C = I n . We
verify both of these equalities for C = B-1A-1. First, (B-1A-1)(AB) = B-1(A-1A)B = B-1I n B =
B-1B = I n . Similarly, (AB)(B-1A-1) = A(BB-1)A-1 = AI n A-1 = AA-1 = I n .

(ii) (ABC)-1 = (BC)-1A-1 = C-1B-1A-1.

D.6 (i) Let e j be the n × 1 vector with jth element equal to one and all other elements equal to
zero. Then straightforward matrix multiplication shows that e′j Ae j = a jj , where a jj is the jth
diagonal element. But by definition of positive definiteness, x′Ax > 0 for all x ≠ 0, including x =
e j . So a jj > 0, j = 1, 2,  , n.
© 2016 Cengage Learning®. May not be scanned, copied or duplicated, or posted to a publicly accessible website,
in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise
on a password-protected website or school-approved learning management system for classroom use.
 313

 1 −2 
(ii) The matrix A =   works because x′Ax = −2 < 0 for x′ = (1 1).
 −2 1

D.7 We must show that, for any n × 1 vector x, x ≠ 0, x′(P′AP) x > 0. But we can write this
quadratic form as (P x)′A(P x) = z′Az where z ≡ Px. Because A is positive definite by
assumption, z′Az > 0 for z ≠ 0. So, all we have to show is that x ≠ 0 implies that z ≠ 0. We do
this by showing the contrapositive, that is, if z = 0 then x = 0. If Px = 0, then, because P-1 exists,
we have P-1Px = 0 or x = 0, which completes the proof.

D.8 Let z = Ay + b. Then, by the first property of expected values, E(z) = Aµ y + b, where µ y =
E(y). By Property 3 for variances, Var(z) = E[(z – µ z )(z – µ z ) ′]. But z – µ z = Ay + b – (Aµ y +
b) = A(y – µ y ). Therefore, (z – µ z )′ = (y – µ y )′A′, and so (z – µ z )( z – µ z )′ = A(y – µ y )(y –
µ y )′A′. Now we can take the expectation and use the second property of expected value:
E[A(y – µ y )(y – µ y )′A′] = AE[(y – µ y )(y – µ y ) ′]A′ = A[Var(y)]A′.

D.9 To obtain the stated conclusion, first use the fact that tr(auu′a′) = tr(a′auu′) . Next, the
expected value passes through the trace operator because trace is a linear operator. Therefore,
E[tr(a′auu′)] = tr[ E (a′auu′)] . Now use the fact that a′a is nonrandom, and so the expected value
passes through:

n
E (a′auu=
′) a′aE (uu=
′) a′aI=
n
′a
a= ∑a ,
i =1
2
i

where we use the assumption that E (uu′) = I n . Of course the trace of a scalar is just the scalar.

D.10 There is not much to prove here; it is a matter of talking through the definitions and
previous claims. First, we know that if u  Normal(0, I n ) and A is a nonrandom, n × n
symmetric, idempotent matrix with rank k1 , then u′Αu  χ k21 from the first property of a chi-
square distribution. Similarly, u′Bu  χ k22 . From the second property of the chi-square
distribution, u′Αu and u′Bu are independent because we are given that AB = 0. The ratio
(u′Αu / k1 ) / (u′Bu / k2 ) is therefore the ratio of two independent chi-square random variables,
where each is divided by its degrees of freedom. That is the definition of an Fk1 ,k2 random
variable.

D.11 (i) X is n × k matrix partitioned as (X 1 X 2 ), where X 1 is n × k1 and X 2 is n × k2 .

𝐗𝐗 ′𝟏𝟏 𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟏𝟏 𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟐𝟐

X′X = � � (𝐗𝐗 𝟏𝟏 𝐗𝐗 𝟐𝟐 ) = � �.
𝐗𝐗 ′𝟐𝟐 𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟏𝟏 𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟐𝟐
The dimensions of each of the matrices are
𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟏𝟏 is k1 × k1
© 2016 Cengage Learning®. May not be scanned, copied or duplicated, or posted to a publicly accessible website,
in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise
on a password-protected website or school-approved learning management system for classroom use.
 314

𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟏𝟏 is k2 × k1
𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟐𝟐 is k1 × k2
𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟐𝟐 is k2 × k2

𝐛𝐛𝟏𝟏
(ii) Let b be a k ×1 vector, partitioned as 𝐛𝐛 = � �, where b 1 is k1 ×1 and b 2 is k2 ×1 .
𝐛𝐛𝟐𝟐
𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟏𝟏 𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟐𝟐 𝐛𝐛𝟏𝟏 (𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟏𝟏 )𝐛𝐛𝟏𝟏 + (𝐗𝐗 ′𝟏𝟏 𝐗𝐗 𝟐𝟐 )𝐛𝐛𝟐𝟐
(X′X)b = � � � � = � �.
𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟏𝟏 𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟐𝟐 𝐛𝐛𝟐𝟐 (𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟏𝟏 )𝐛𝐛𝟏𝟏 + (𝐗𝐗 ′𝟐𝟐 𝐗𝐗 𝟐𝟐 )𝐛𝐛𝟐𝟐

© 2016 Cengage Learning®. May not be scanned, copied or duplicated, or posted to a publicly accessible website,
in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise
on a password-protected website or school-approved learning management system for classroom use.