Professional Documents
Culture Documents
6794 24022 1 PB
6794 24022 1 PB
6794 24022 1 PB
Received: August 2, 2013 / Revised: December 14, 2013 / Accepted: January 8, 2014
ABSTRACT
In the theory and applications of linear models, idempotent matrices appear fre-
quently. It remains important to construct non-trivial quadratic forms distributed
as chi-squares. However, quick and easy methods to explicitly construct such ma-
trices with all non-zero entries are hard to come by. We include two direct methods
to explicitly exhibit n × n real symmetric idempotent matrices of rank r with all
non-zero entries. Then, we exhibit explicit non-trivial real symmetric idempotent
matrices A, B such that AB = O.
Keywords: Chi-square, Design matrix, Inverse matrix, Orthogonal vectors, Pro-
jection matrix, Quadratic forms.
1. Introduction
An n×n matrix A is called idempotent if A = A2 . Idempotent matrices are discussed
in many books including Rao (1973), Cook and Weisberg (1982), Chatterjee and
Hadi (1988), Harville (2001, chapter 10), Rao and Bhimasankaram (2000), and
Ravishanker and Dey (2002).
An n × n idempotent matrix A of real numbers with its rank ρ(A) = n is necessarily
In×n . Hence, non-trivial idempotent matrices of less than full rank play key roles in
much of the theory and practice of linear models.
We explicitly exhibit an n × n real symmetric idempotent matrix A with ρ(A) = r <
n for r and n held fixed. Initially, one may propose a partitioned matrix as follows:
!
I Or×n
A = r×r , (1.1)
Or×r Or×n
IASSL ISSN-1391-4987 57
Nitis Mukhopadhyay
which is idempotent. But, in principle, this idempotent matrix has not gone very
far from In×n .
We may require, for example, an n × n symmetric idempotent matrix A of real
numbers with ρ(A) = r < n with all non-zero entries when (i) n = 2, r = 1, (ii)
n = 3, r = 1, and (iii) n = 3, r = 2. Two simple examples for case (i) follow:
! !
1 1 1 −1
Case (i): A = 2 1
or A = 2
1
. (1.2)
1 1 −1 1
The next cases (ii)-(iii) are certainly more demanding. Some examples follow:
1 1 3
9 6
! 14 7 14
Case (i): A = 13 =
13 1 2 3
6 4 ; Case (ii): A 7 7 7 ;
13 13 3 3 9
14 7 14
61 9 3 (1.3)
70 − 35 14
Case (iii): A = − 35 .
9 17 3
3 35 7
3 9
14 7 14
We include case (i) again in (1.3) simply because this A matrix is drastically differ-
ent from those in (1.2). One may ask: Where do these examples come from? Can
such examples be found quickly and easily? At this juncture, we may turn back
and think of two specific results:
0 0
(a) Every real idempotent matrix has the form PQ for some P and Q where Q P =
I. This characterizes all idempotent matrices and it also means that P and Q have
full column ranks; and
0
(b) Every real symmetric idempotent matrix is of the form PP for some P where
0
P P = I. This characterizes all symmetric idempotent matrices and it also means
that P is of full column rank.
Khatri (1985), on the other hand, included the following result which he compared
with his previous result from Khatri (1983):
Let A be an n × n matrix, and let the trace, the conjugate transpose, and the Moore-
Penrose inverse of A be respectively
denoted by trA, A∗ , and A† . Suppose that
ρ(A) = trA. Then, tr A A A ≥ ρ(A), and the equality holds if and only if A is
2 † ∗
idempotent.
These rather well-known results, however, do not render immediate help in ex-
hibiting concrete but non-trivial idempotent matrices quickly. Instead, they assure
us to be able to claim that there exist ways to find non-trivial idempotent matrices.
But, the existence and finding concrete and quick examples are not the same.
We provide cute ideas to come up with non-trivial and concrete idempotent ma-
trices quickly and efficiently. This paper describes two such instructive techniques
explicitly.
One may argue in the context of a linear model that a projection matrix An×n (equiv-
alently, In×n − A) is idempotent. While that is true, the point is that one must then
exhibit some appropriate projection matrix An×n with a specified rank r(< n), both
quickly and explicitly. This idea makes up Section 2.
Next, we provide an alternative method involving chi-square distributions of quadratic
forms in the first part of Section 3. In Section 3, we also illustrate explicit pairs of
non-trivial, symmetric, real n × n idempotent matrices A, B such that AB = On×n
with ρ(A) = r(< n), ρ(B) = s(< n) where n, r, and s are arbitrary positive integers.
Section 4 adds some concluding thoughts.
0
must specify the X matrix in such a way that the inverse of X X may be determined
effortlessly.
that is
0
X X = Diagonal (c1 , ..., cr ) , (2.3)
0
where ci = pi pi (> 0), i = 1, ..., r. Obviously, one has:
0
−1
XX = Diagonal c−1
1 , ..., c −1 .
r (2.4)
These examples illustrate how quick and easy the proposed approach is. We will
close this section with another illustration.
203 37 37 1 3 7
− 240 − 120
240 20 80 80
− 37 203 37
− 120 1 3 7
240 240 20 80 80
− 37 37
− 120 23 1 3 7
A = 120 1 1
60
1
10
1
40
3
40
7
.
(2.9)
20 20 10 5 20 20
3 3 3 3 9 21
80 80 40 20 80 80
7 7 7 7 21 49
80 80 40 20 80 80
One may obviously check that this A6×6 matrix from (2.9) is idempotent with rank
3.
Recall that our general approach is found in (2.1)-(2.5) which puts great emphasis
0
upon finding (X X)−1 quickly, painlessly, and explicitly.
Q1
= 40 U 1 + 20 U 1 U 2 − 20 U 1 U 3 − 20 U 1 U 4 + 20 U 1 U 5 − 20 U 1 U 6 − 20 U 1 U 7
17 2 17 3 3 7 1 1
1
− 20 U1 U8 + 15 U1 U9 + 15 U1 U10 + 40 17 2
U2 − 203
U2 U3 − 20 3
U2 U4 + 20 7
U2 U5
− 20 U2 U6 − 20 U2 U7 − 20 U2 U8 + 5 U2 U9 + 5 U2 U10 + 40 U3 − 20 U3 U4
1 1 1 1 1 37 2 3
+ 20
7
U3 U5 − 1
20 U 3 U 6 − 1
20 U 3 U 7 − 1
20 U 3 U 8 + 15 U3 U9 + 15 U3 U10 + 37 2
40 U 4
+ 20
7
U4 U5 − 1
20 U 4 U 6 − 1
20 U 4 U 7 − 1
20 U 4 U 8 + 15 U4 U9 + 15 U4 U10 + 7 2
40 U 5
20 U 5 U 7 − 20 U 5 U 8 + 5 U 5 U 9 + 5 U 5 U 10 + 40 U 6 − 20 U 6 U 7
1 1 1 1 1 39 2 1
− 20 U5 U6 −
1
− 20 U6 U8 + 15 U6 U9 + 15 U6 X10 + 39 40 U 7 − 20 U 7 U 8 + 5 U 7 U 9 + 5 U 7 U 10
2 1 1 1
+ 39
40 U 8 + 5 U 8 U 9 + 5 U 8 U 10 + 10 U 9 + 5 U 9 U 10 + 10 U 10 ,
2 1 1 1 2 1 1 2
(3.1)
Q2
= 180 U 1 − 90 U 1 U 2 + 90 U 1 U 3 + 90 U 1 U 4 − 45 U 1 U 5 + 45 U 1 U 6 + 45 U 1 U 7
101 2 79 11 11 17 1 1
+ 45
1
U1 U8 + 451
U1 U9 − 15 U1 U10 + 180 U2 + 11
101 2
90 U 2 U 3 + 90 U 2 U 4 − 45 U 2 U 5
11 17
+ 45
1
U2 U6 + 1
45 U 2 U 7 + 1
45 U 2 U 8 + 45 U 2 U 9 − 5 U 2 U 10 + 180 U 3 + 90 U 3 U 4
1 1 11 2 11
17
− 45 U3 U5 + 1
45 U 3 U 6 + 1
45 U 3 U 7 + 451
U3 U8 + 45 1
U3 U9 − 15 U3 U10 + 180 11 2
U4
45 U 4 U 5 +
− 17 + + 45 U4 U8 + 45 U4 U9 − 15 U4 U10 + 73
1 1 1 1 2
45 U 4 U 6 45 U 4 U 7 90 U 5
+ 45
1
U5 U6 + 1
45 U 5 U 7 + 1
45 U 5 U 8 + 451
U5 U9 − 15 U5 U10 + 90 U6 + 45
1 2 1
U6 U7
+ 45
1
U6 U8 + 1
45 U 6 U 9 − 1
5 U 6 U 10 + 90 U7 + 45
1 2 1
U7 U8 + 45 1
U7 U9 − 15 U7 U10
+ 90
1 2
U8 + 1
45 U 8 U 9 − 1
5 U 8 U 10 + 90 U 9 − 5 U 9 U 10 + 10 U 10 ,
1 2 1 9 2
(3.2)
as illustrations, both constructed from U1 , ..., U10 which are assumed independent
N(0, 1) random variables.
With regard to the quadratic forms Q1 and Q2 from (3.1)-(3.2), one may ask the
following questions: (i) Are Q1 and Q2 distributed as central chi-squares? If so,
can we specify the respective degrees of freedom? and (ii) Are Q1 , Q2 distributed
independently of each other? Why or why not?
But, before we address these questions, we may indeed raise the following more
pertinent question: How could we envision such formidable looking quadratic
forms Q1 , Q2 in the first place? We give some directions and the main motivation
before proposing a systematic method of construction.
Let us begin with a set of n independent standard normal random variables U1 , ..., Un
with n ≥ 2. Next, let us define a new set of n random variables Y1 , ..., Yn as follows:
Y1 = √1 (U 1
n
+ ... + Un );
Y2 = √1 (U 1 − U2 );
2
Y3 = 1
√ (U 1 + U2 − 2U3 );
6
. . (3.3)
. .
. .
Yn−1 = √ 1
(n−1)(n−2)
{U1 + ... + Un−2 − (n − 2)Un−1 };
Yn = √ 1
n(n−1)
{U1 + ... + Un−1 − (n − 1)Un }.
W ≡ Yi21 + ... + Yi2r ∼ χ2r for any 1 ≤ i1 < i2 < ... < ir ≤ n, (3.4)
where
√1 √1 √1 √1 √1 √1 √1 √1 √1 √1
10 10 10 10 10 10 10 10 10 10
√1 − √1
0 0 0 0 0 0 0 0
2 2
√1 √1 − √2
0 0 0 0 0 0 0
C = 6 6 6 .
√1 √1 √1 − √3 0 0 0 0 0 0
12 12 12 12
√1 √1 √1 √1 − √4
0 0 0 0 0
20 20 20 20 20
√1 √1 √1 √1 √1 − √5 0 0 0 0
30 30 30 30 30 30
Hence, we have:
14 1 1 1 1 1 1 1 1 1
− 15 − 15 − 15 − 15 − 15
15 10 10 10 10
1 14 1 1 1 1 1 1 1 1
− 15 15 − 15 − 15 − 15 − 15 10 10 10 10
1 1 14 1 1 1 1 1 1 1
− 15 − 15 15 − 15 − 15 − 15 10 10 10 10
1 1 1 14 1 1 1 1 1 1
− 15 − 15 − 15 15 − 15 − 15 10 10 10 10
1 1 1 1 14 1 1 1 1 1
0 − 15 − 15 − 15 − 15 − 15
A = C C = ,
15 10 10 10 10
1 1 1 1 1 14 1 1 1 1 (3.5)
− 15 − 15 − 15 − 15 − 15 15 10 10 10 10
1 1 1 1 1 1 1 1 1 1
10 10 10 10 10 10 10 10 10 10
1 1 1 1 1 1 1 1 1 1
10 10 10 10 10 10 10 10 10 10
1 1 1 1 1 1 1 1 1 1
10 10 10 10 10 10 10 10 10 10
1 1 1 1 1 1 1 1 1 1
10 10 10 10 10 10 10 10 10 10
0 0
Y12 + Y32 + Y42 + Y62 + Y72 + Y92 = U AU ∼ χ26 with A = C C, (3.6)
where
√1 √1 √1 √1 √1 √1 √1 √1 √1 √1
10 10 10 10 10 10 10 10 10 10
√1 √1 − √2
0 0 0 0 0 0 0
6 6 6
√1 √1 √1 − √3
0 0 0 0 0 0
C = 12 12 12 12 .
√1 √1 √1 √1 √1 − √5 0 0 0 0
30 30 30 30 30 30
√1 √1 √1 √1 √1 √1 − √6
0 0 0
42 42 42 42 42 42 42
√1 √1 √1 √1 √1 √1 √1 − √7 0 0
56 56 56 56 56 56 56 56
Hence, we have:
17 17 3 3 7 1 1 1 1 1
− 40 − 40 − 40 − 40 − 40
40 40 40 10 10
17 17 3 3 7 1 1 1 1 1
40 40 − 40 − 40 40 − 40 − 40 − 40 10 10
3 3 37 3 7 1 1 1 1 1
− 40 − 40 40 − 40 40 − 40 − 40 − 40 10 10
3 3 3 37 7 1 1 1 1 1
− 40 − 40 − 40 40 40 − 40 − 40 − 40 10 10
7 7 7 7 7 1 1 1 1 1
0 − 40 − 40 − 40
A = C C = ,
40 40 40 40 40 10 10
1 1 1 1 1 39 1 1 1 1
− 40 − 40 − 40 − 40 − 40 40 − 40 − 40 10 10
1 1 1 1 1 1 39 1 1 1
− 40 − 40 − 40 − 40 − 40 − 40 40 − 40 10 10
1 1 1 1 1 1 1 39 1 1
− 40 − 40 − 40 − 40 − 40 − 40 − 40 40 10 10
1 1 1 1 1 1 1 1 1 1
10 10 10 10 10 10 10 10 10 10
1 1 1 1 1 1 1 1 1 1
10 10 10 10 10 10 10 10 10 10
(3.7)
again a 10 × 10 real symmetric idempotent matrix with its rank 6.
The matrix A from (3.7) is clearly different from the one shown in (3.5). It is our
hope that the inherent beauty and simplicity of this approach is now apparent.
where
√1 − √1
0 0 0 0 0 0 0 0
2 2
C = √1 √1 √1 √1 − √4 .
0 0 0 0 0
20 20 20 20 20
√1 √1 √1 √1 √1 √1 √1 √1 √1 − √9
90 90 90 90 90 90 90 90 90 90
Hence, we have
101 79 11 11
− 180 − 17 1 1 1 1 1
− 10
180 180 180 90 90 90 90 90
− 79 101 11 11
− 17 1 1 1 1 1
− 10
180 180 180 180 90 90 90 90 90
11 11 11 11
− 17 1 1 1 1 1
− 10
180 180 180 180 90 90 90 90 90
11 11 11 11
− 17 1 1 1 1 1
− 10
180 180 180 180 90 90 90 90 90
− 17 − 17 − 17 − 17 73 1 1 1 1 1
0 − 10
B = C C = 190 .
90 90 90 90 90 90 90 90
1 1 1 1 1 1 1 1 1
90 90 90 90 90 90 90 90 90 − 10
1 1 1 1 1 1 1 1 1 1
− 10
90 90 90 90 90 90 90 90 90
1 1 1 1 1 1 1 1 1 1
− 10
90 90 90 90 90 90 90 90 90
1 1 1 1 1 1 1 1 1 1
− 10
90 90 90 90 90 90 90 90 90
1 1 1 1 1 1 1 1 1 9
− 10 − 10 − 10 − 10 − 10 − 10 − 10 − 10 − 10 10
(3.9)
Obviously, the two quadratic forms from (3.6) and (3.8) are distributed indepen-
dently and hence AB must be the zero matrix, O10×10 . One may directly verify that
AB = O and that ρ(B) = 3.
One may certainly construct non-trivial 10 × 10 idempotent matrices A, B such that
AB , O10×10 . For example, consider the quadratic forms:
0
Y12 + Y22 + Y42 + Y62 + Y72 + Y92 = U AU ∼ χ26 and
0 (3.10)
Y12 + Y22 + Y52 + Y10
2 = U BU ∼ χ2 .
4
Obviously, the two matrices A, B associated with (3.10) will both be 10 × 10 idem-
potent with respective ranks 6 and 4 such that AB , O10×10 since the two associated
quadratic forms from (3.8) are surely dependent. One may alternatively check:
0
0
Cov U AU, U BU
= Cov Y12 + Y22 + Y42 + Y62 + Y72 + Y92 ,Y12 + Y22 + Y52 + Y10
2
= V Y12 + V Y22
= 4(, 0).
4. Concluding Thoughts
The approach given in Section 2 works well with any set of orthogonal non-zero
column vectors p1 , ..., pr with r < n, however its implementation is quick and
simple when n, r are both small or moderate. One is expected to begin with an
arbitrary non-zero vector p1 , and then write down a suitable non-zero vector p2 that
is orthogonal to p1 . But, continuing too much further along may not necessarily be
a simple task.
In this regard, the orthogonal set of vectors p1 , ..., pn in <n where
0 0 0
p1 = (1, 1, 1, ..., 1, 1, 1), p2 = (1, −1, 0, ..., 0, 0, 0), p3 = (1, 1, −2, ..., 0, 0, 0),
0 0
..., pn−1 = (1, 1, 1, ..., 1, −n + 2, 0), pn = (1, 1, 1, ..., 1, 1, −n + 1),
(4.1)
are helpful. One will note that Helmert’s orthogonal vectors utilized in (3.3) are
normalized versions of the vectors from (4.1).
Another idea will prove to be useful in locating numerous idempotent n × n real
matrices with rank r(< n) once we find one such matrix A. One may construct a
new matrix BAB−1 where Bn×n is arbitrary but non-singular. The matrix BAB−1 is
also an idempotent n × n real matrix with rank r(< n) (Harville 200, Exercise 1,
p. 49). But, then a major obstacle is that a non-trivial and non-singular matrix B
along with its inverse ought to be exhibited quickly and painlessly.
However, we will encounter a rather painless resolution if we use B ≡ (bi j ) =
0
(c − d)In×n + d1n×n 1n×n where c, d are different but arbitrary non-zero real numbers,
c , −(n−1)d. Then, the matrix B−1 ≡ (bi j ) is known to have exactly the same form
as that of B. Indeed, it is well-known (Rao 1973, Exercise 1.1, p. 67) that
c + (n − 2)d −d
bii = and bi j = , for all i , j. (4.2)
[c + (n − 1)d](c − d) [c + (n − 1)d](c − d)
In case (iii), that is when n = 3 and r = 2, we showed one matrix A in (2.8). Now,
let us employ the technique of pre- and post-multiplying A with B and B−1 where
0
B = 3I3×3 + 13×3 13×3 . Then, we will have:
77
− 104 221
90 315 630
BAB−1 = − 11 139 187
,
1145 315
88
315
443
90 315 630
which itself is a real idempotent matrix with rank 2, but it is not a symmetric
matrix.
In Section 3, we exploited the Helmert orthogonal matrix. One could alternatively
exploit Householder’s symmetric orthogonal matrix. One may refer to Rao and
Bhimasankaram (2000, pp. 269-270).
Acknowledgements
Nearly 44 years ago I began appreciating the beauty of linear algebra from Pro-
fessor S. K. Mitra, and from his protégé, P. Bhimasankaram. I fondly dedicate this
work to Professor Bhimasankaram for his continued enthusiasm and infinite posi-
tive energy. I also thank the reviewers for their helpful commentaries.
References
1. Chatterjee, S. and Hadi, A. S. (1988). Sensitivity Analysis in Linear Regres-
sion, New York: Wiley. DOI:10.1002/9780470316764
4. Helmert, F. R. (1876). Die Genauigkeit der Formel von Peters zur Berech-
nung des wahrscheinlichen Beobachtungsfehlers direkter Beobachtungen gle-
icher Genauigkeit, Astronomische Nachrichten 88: 113-131.
DOI:10.1002/asna.18760880802
9. Rao, C. R. (1973). Linear Statistical Inference and Its Applications, 2nd edi-
tion, New York: Wiley. DOI:10.1002/9780470316436