Professional Documents
Culture Documents
Gram Charlier and Edgeworth Expansion For Sample Variance
Gram Charlier and Edgeworth Expansion For Sample Variance
1. Introduction
Let X1 , . . . , Xn be a random sample and define the sample variance statistic as:
n n
1X 1 X
X̄n = Xi , s2n = (Xi − X̄n )2 , Xn = (X1 , . . . , Xn )T (1.1)
n i=1 n − 1 i=1
where X̄n is the empirical mean, s2n the Bessel corrected empirical variance also
called sample variance, and Xn the vector of the full history of this random
sample. We are interested in the distribution of the sample variance under very
weak conditions, namely that it admits a valid Edgeworth expansion.
1
imsart-generic ver. 2014/10/16 file: article_edgeworth.tex date: April 7, 2019
E. Benhamou, B. Guez, N. Paris/Gram Charlier and Edgeworth expansion for sample variance
2
distribution. But in this particular case, the different variables Xi − X̄n are not
independent and the result can not apply. Though they marginally have the
same variance (if we conditioned by X̄n ), they are correlated with each other.
Hence, in general, the distribution of the sample variance in the normal case is
not a known distribution. There is however one special case, where it is a χ2
distribution. Remember that when X ∼ N (0, Σ), X T BX has a χ2 distribution
of degree d the rank of B if and only if BΣB = B (see for instance page 413
remark following theorem 1 of Hogg et al. (1978)). Notice that in our case, the
B matrix is B = In − n1 1Tn 1n , where In is the identity matrix of size n and 1n
is the vector of size n filled with 1 as we can write
n
1 X
s2n = (Xi − X̄n )2 (1.2)
n − 1 i=1
1 1
= X T (In − 1Tn 1n )X (1.3)
n−1 n
we obtain that it is a χ2 distribution if and only if
1 T 1 1
(In − 1 1n )Σ(In − 1Tn 1n ) = In − 1Tn 1n
n n n n
In other cases, the sample variance is a linear combination of Gamma distri-
bution, and one has to rely on approximations as explained in Moschopoulos
(1985) and Mathai (1982).
and non identically distributed variable state that the resulting distribution is a
normal distribution (or in the most general case to truncated symmetrical and
α stable distributions 4 .
Let denote by fˆ (respectively φ̂ the characteristic function of our distribution
whose density function is f (respectively φ), and κj its cumulants (respectively
γj ). Cumulants definition state
∞ j ∞ j
X (it) X (it)
fˆ(t) = exp κj and φ̂(t) = exp γj , (2.1)
j=1
j! j=1
j!
Using Fourier transform property that say (it)j φ̂(t) is the Fourier transform
of (−1)j [Dj φ](−x), where D is the differential operator with respect to x, we
get the formal expansion:
∞ j
X (−D)
f (x) = exp (κj − γj ) φ(x) (2.3)
j=1
j!
h 2
i
If φ is chosen as the normal density φ(x) = √2πσ 1
exp − (x−µ)
2σ 2 with mean
and variance as given by f , that is, mean µ = κ1 and variance σ 2 = κ2 , then
the expansion becomes
∞ j
X (−D)
f (x) = exp κj φ(x), (2.4)
j=3
j!
since γj = 0 for all j > 2, as higher cumulants of the normal distribution are
0. By expanding the exponential and collecting terms according to the order of
the derivatives, we arrive at the Gram–Charlier A series. Such an expansion can
be written compactly in terms of Bell polynomials as
∞ j ∞
X (−D) X (−D)j
exp κj = Bj (0, 0, κ3 , . . . , κj ) . (2.5)
j=3
j! j=0
j!
(−1)j
(j) x−µ
φ (x) = Hej φ(x), (2.6)
σj σ
4 please see the extension of CLT in Gnedenko and Kolmogorov (1954)
If we include only the first two correction terms to the normal distribution,
we obtain
κ3 x−µ κ4 x−µ
f (x) = φ(x) 1 + He 3 + He 4 + R 5 , (2.8)
3!σ 3 σ 4!σ 4 σ
1
If in the above expression, the cumulant are function of a parameter n, we
can rearrange terms by power of n1 and find the Edgeworth expansion.
with the natural symmetric moment estimators whose expressions are provided
in appendix section A.1. The term R2 is the second order rest and is equal to
zero if the sample is i.i.d. For general case, this term does not cancel out and
should be taken into account. It can be rewritten as
R2 =(µ́22 )2 − (µ́2 )21 + 2((µ́2 )1 (µ́21 )1 − (µ́21 µ́2 )2 ) + (µ́41 )2 − (µ́21 )21 (2.15)
A43,0 = − 672(µ́81 )4 + 2688(µ́61 µ́2 )4 − 1216(µ́51 µ́3 )4 − 3120(µ́41 µ́22 )4 + 400(µ́41 µ́4 )4 + 2240(µ́31 µ́2 µ́3 )4
+ 960(µ́21 µ́32 )4 − 384(µ́23 µ́21 )4 − 480(µ́21 µ́2 µ́4 )4 − 624(µ́1 µ́22 µ́3 )4 + 144(µ́1 µ́3 µ́4 )4 − 6(µ́42 )4
+ 96(µ́2 µ́23 )4 − 3(µ́24 )4 + 12(µ́22 µ́4 )4 (2.25)
A42,1 =− 128(µ́31 µ́5 )4 + 96(µ́1 µ́2 µ́5 )4 − 24(µ́3 µ́5 )4 (2.26)
A41,2 =32(µ́21 µ́6 )4 − 4(µ́2 µ́6 )4 (2.27)
A40,3 =(µ́8 )4 − 8(µ́1 µ́7 )4 (2.28)
A43,1 =3792(µ́81 )4 − 15168(µ́61 µ́2 )4 + 6144(µ́51 µ́3 )4 + 18144(µ́41 µ́22 )4 − 1920(µ́41 µ́4 )4 − 11520(µ́31 µ́2 µ́3 )4
− 6336(µ́32 µ́21 )4 + 1680(µ́21 µ́23 )4 + 2520(µ́21 µ́2 µ́4 )4 + 3600(µ́1 µ́22 µ́3 )4 − 624(µ́1 µ́3 µ́4 )4 + 234(µ́42 )4
− 432(µ́2 µ́23 )4 + 33(µ́24 )4 − 252(µ́22 µ́4 )4 (2.29)
A42,2 =400(µ́31 µ́5 )4 − 336(µ́1 µ́2 µ́5 )4 + 48(µ́3 µ́5 )4 (2.30)
A41,3 =28(µ́2 µ́6 )4 − 56(µ́21 µ́6 )4 (2.31)
A43,2 =− 7440(µ́81 )4 + 29760(µ́61 µ́2 )4 − 10880(µ́51 µ́3 )4 − 36480(µ́41 µ́22 )4 + 3104(µ́41 µ́4 )4 + 20992(µ́31 µ́2 µ́3 )4
+ 13824(µ́21 µ́32 )4 − 2752(µ́21 µ́23 )4 − 4368(µ́21 µ́2 µ́4 )4 − 7248(µ́1 µ́22 µ́3 )4 + 976(µ́1 µ́3 µ́4 )4 − 738(µ́42 )4
+ 800(µ́2 µ́23 )4 − 57(µ́24 )4 + 612(µ́22 µ́4 )4 (2.32)
A42,3 =− 336(µ́31 µ́5 )4 + 336(µ́1 µ́2 µ́5 )4 − 56(µ́3 µ́5 )4 (2.33)
A43,3 =5040(µ́81 )4 − 20160(µ́61 µ́2 )4 + 6720(µ́51 µ́3 )4 + 25200(µ́41 µ́22 )4 − 1680(µ́41 µ́4 )4 − 13440(µ́31 µ́2 µ́3 )4
− 10080(µ́21 µ́32 )4 + 1680(µ́21 µ́23 )4 + 2520(µ́21 µ́2 µ́4 )4 + 5040(µ́1 µ́22 µ́3 )4 − 560(µ́1 µ́3 µ́4 )4
+ 630(µ́42 )4 − 560(µ́2 µ́23 )4 + 35(µ́24 )4 − 420(µ́22 µ́4 )4 (2.34)
3. Conclusion
In this paper, we have derived the most general formula for the Gram Charlier
and the resulting Edgeworth expansion for the sample variance under very weak
conditions. Our formula does not assume that the underlying sample is inde-
pendent neither identically distributed. This formula can therefore be applied
to strong mixing processes like sample of an auto regressive process of order 1
(AR(1)). It extends in particular the work of Mikusheva (2015)
Appendix A: Notations
Pn
E Xi2
i=1
(µ́2 )1 = (A.1)
P n
i6=j E [Xi Xj ]
(µ́21 )1 = (A.2)
n(n − 1)
• moments of order 2:
Pn
E Xi4
i=1
(µ́4 )2 = (A.3)
n
3
P
i6=j E Xi Xj
(µ́1 µ́3 )2 = (A.4)
n(n − 1)
P 2 2
i6=j E Xi Xj
(µ́22 )2 = (A.5)
n(n − 1)
P 2
i6=j6=k E Xi Xj Xk
(µ́21 µ́2 )2 = (A.6)
n(n − 1)(n − 2)
P
i6=j6=k6=l E [Xi Xj Xk Xl ]
(µ́41 )2 = (A.7)
n(n − 1)(n − 2)(n − 3)
P
i6=j6=k6=l6=m6=n E [Xi Xj Xk Xl Xm Xn ]
(µ́61 )3 = (A.8)
n(n − 1)(n − 2)(n − 3)(n − 4)(n − 5)
P 2
i6=j6=k6=l6=m E Xi Xj Xk Xl Xm
(µ́41 µ́2 )3 = (A.9)
n(n − 1)(n − 2)(n − 3)(n − 4)
P 3
i6=j6=k6=l E Xi Xj Xk Xl
(µ́31 µ́3 )3 = (A.10)
n(n − 1)(n − 2)(n − 3)
P 2 2
i6=j6=k6=l E Xi Xj Xk Xl
(µ́21 µ́22 )3 = (A.11)
n(n − 1)(n − 2)(n − 3)
P 4
i6=j6=k E Xi Xj Xk
(µ́21 µ́4 )3 = (A.12)
n(n − 1)(n − 2)
P 3 2
i6=j6=k E Xi Xj Xk
(µ́1 µ́2 µ́3 )3 = (A.13)
n(n − 1)(n − 2)
P 5
i6=j E Xi Xj
(µ́1 µ́5 )3 = (A.14)
n(n − 1)
P 2 2 2
i6=j6=k E Xi Xj Xk
(µ́32 )3 = (A.15)
n(n − 1)(n − 2)
P 4 2
i6=j E Xi Xj
(µ́2 µ́4 )3 = (A.16)
n(n − 1)
P 3 3
i6=j E Xi Xj
(µ́23 )3 = (A.17)
n(n − 1)
P 6
i E Xi
(µ́6 )3 = (A.18)
n
P
i6=j6=k6=l6=m6=n6=o6=p E [Xi Xj Xk Xl Xm Xn Xo Xp ]
(µ́81 )4 = (A.19)
n(n − 1)(n − 2)(n − 3)(n − 4)(n − 5)(n − 6)(n − 7)
P 2
i6=j6=k6=l6=m6=n6=o E Xi Xj Xk Xl Xm Xn Xo
(µ́61 µ́2 )4 = (A.20)
n(n − 1)(n − 2)(n − 3)(n − 4)(n − 5)(n − 6)
P 3
i6=j6=k6=l6=m6=n E Xi Xj Xk Xl Xm Xn
(µ́51 µ́3 )4 = (A.21)
n(n − 1)(n − 2)(n − 3)(n − 4)(n − 5)
P 2 2
i6=j6=k6=l6=m6=n E Xi Xj Xk Xl Xm Xn
(µ́41 µ́22 )4 = (A.22)
n(n − 1)(n − 2)(n − 3)(n − 4)(n − 5)
P 4
i6=j6=k6=l6=m E Xi Xj Xk Xl Xm
(µ́41 µ́4 )4 = (A.23)
n(n − 1)(n − 2)(n − 3)(n − 4)
P 5
i6=j6=k6=l E Xi Xj Xk Xl
(µ́31 µ́5 )4 = (A.24)
n(n − 1)(n − 2)(n − 3)
P 3 2
i6=j6=k6=l6=m E Xi Xj Xk Xl Xm
(µ́31 µ́2 µ́3 )4 = (A.25)
n(n − 1)(n − 2)(n − 3)(n − 4)
P 4 2
i6=j6=k6=l E Xi Xj Xk Xl
(µ́21 µ́2 µ́4 )4 = (A.26)
n(n − 1)(n − 2)(n − 3)
P 2 2 2
i6=j6=k6=l6=m E Xi Xj Xk Xl Xm
(µ́21 µ́32 )4 = (A.27)
n(n − 1)(n − 2)(n − 3)(n − 4)
P 3 3
i6=j6=k6=l E Xi Xj Xk Xl
(µ́21 µ́23 )4 = (A.28)
n(n − 1)(n − 2)(n − 3)
P 6
i6=j6=k E Xi Xj Xk
(µ́21 µ́6 )4 = (A.29)
n(n − 1)(n − 2)
P 3 2 2
i6=j6=k6=l E Xi Xj Xk Xl
(µ́1 µ́22 µ́3 )4 = (A.30)
n(n − 1)(n − 2)(n − 3)
P 5 2
i6=j6=k E Xi Xj Xk
(µ́1 µ́2 µ́5 )4 = (A.31)
n(n − 1)(n − 2)
P 4 3
i6=j6=k E Xi Xj Xk
(µ́1 µ́3 µ́4 )4 = (A.32)
n(n − 1)(n − 2)
P 7
i6=j E Xi Xj
(µ́1 µ́7 )4 = (A.33)
n(n − 1)
P 4 2 2
i6=j6=k E Xi Xj Xk
(µ́22 µ́4 )4 = (A.34)
n(n − 1)(n − 2)
P 2 2 2 2
i6=j6=k6=l E Xi Xj Xk Xl
(µ́42 )4 = (A.35)
n(n − 1)(n − 2)(n − 3)
P 6 2
i6=j E Xi Xj
(µ́2 µ́6 )4 = (A.36)
n(n − 1)
P 3 3 2
i6=j6=k E Xi Xj Xk
(µ́2 µ́23 )4 = (A.37)
n(n − 1)(n − 2)
E Xi5 Xj3
P
i6=j
(µ́3 µ́5 )4 = (A.38)
n(n − 1)
P 4 4
i6=j E Xi Xj
(µ́24 )4 = (A.39)
n(n − 1)
P 8
i E Xi
(µ́8 )4 = (A.40)
n
First of all, the first four cumulants, denoted by κi for ii = 1, . . . , 4 are ob-
tained through standard relationships with respect to moments denoted by µ0i
as follows:
κ1 = µ01 (B.1)
2
κ2 = µ02 − µ01 (B.2)
3
κ3 = µ03 − 3µ02 µ01 + 2µ01 (B.3)
2 2 4
κ4 = µ04 − 4µ01 µ03 − 3µ02 + 12µ01 µ02 − 6µ01 (B.4)
Hence we are left with computing the first four moments of the sample variance.
The first two moments of the sample variance are easy to compute and given
for instance in Benhamou (2018). For the cumulant of order 3 and 4, we first
compute the different moments and then regroup the terms.
X 5X
P
•
Pi6=j i 4j
• X X X
Pi6=j6=k 3 i 3 j k
• X X
Pi6=j i 3j 2
• X X X
Pi6=j6=k i 2j 2k
• i6=j6=k6=l Xi Xj Xk Xl
Pn
the expansion of ( i=1 Xi2 )( j6=k Xj Xk )2 , the possibilities are:
P
In
X 4X 2
P
•
Pi6=j i 4j
• X X X
Pi6=j6=k i3 j2 k
• X X X
Pi6=j6=k i 3j k
• X X X X
Pi6=j6=k6=l 2 i 2 j 2 k l
• X X X
Pi6=j6=k i 2j 2k
• X X X X
Pi6=j6=k6=l i j2 k l
• i6=j6=k6=l6=m Xi Xj Xk Xl Xm
which leads to
(n − 5)(n − 4)(n − 3)(n − 2)µ́61 3(n − 5)(n − 4)(n − 3)(n − 2)µ́2 µ́41
E[s8n ] = − +
(n − 1)2 n2 (n − 1)2 n2
3(n − 3)(n − 2) n2 − 6n + 15 µ́22 µ́21
4(n − 3)(n − 2)(3n − 5)µ́3 µ́31
+ −
(n − 1)2 n2 (n − 1)2 n2
2
2
3(n − 5)(n − 2)µ́4 µ́1 12(n − 2) n − 4n + 5 µ́2 µ́3 µ́1 6µ́5 µ́1
− − −
(n − 1)n2 (n − 1)2 n2 n2
2
2 2
2 3n − 6n + 5 µ́3 3 n − 2n + 5 µ́2 µ́4 µ́6
− + + 2
(n − 1)2 n2 (n − 1)n2 n
(n − 2) n3 − 3n2 + 9n − 15 µ́32
+ (B.8)
(n − 1)2 n2
with
3
M2,0 = − 60µ́61 + 180µ́41 µ́2 − 68µ́31 µ́3 − 129µ́21 µ́22 + 60µ́2 µ́3 µ́1 + 13µ́32 − 6µ́23 (B.10)
3
M0,0 =− µ́61 + 3µ́41 µ́2 − 3µ́21 µ́22 + µ́32 (B.11)
3
M2,2 = − 120µ́61 + 360µ́41 µ́2 − 120µ́31 µ́3 − 270µ́21 µ́22 + 120µ́1 µ́2 µ́3 + 30µ́32 − 10µ́23 (B.12)
3
M1,0 =12µ́61 − 36µ́41 µ́2 + 12µ́31 µ́3 + 27µ́21 µ́22 − 12µ́1 µ́2 µ́3 − 3µ́32 (B.13)
3
M2,1 =154µ́61 − 462µ́2 µ́41 + 172µ́31 µ́3 + 333µ́21 µ́22 − 156µ́1 µ́2 µ́3 − 33µ́32 + 12µ́23 (B.14)
3
M1,0 = − 3µ́21 µ́4 + 3µ́2 µ́4 (B.15)
3
M1,2 =15µ́2 µ́4 − 30µ́21 µ́4 (B.16)
3
M1,1 =21µ́21 µ́4 − 6µ́2 µ́4 (B.17)
3
M0,2 =µ́6 − 6µ́1 µ́5 (B.18)
Again, one needs to expand all the terms and look at all the various possibil-
ities to demonstrate the following relationship where we have regrouped against
each of the symmetric empirical moment estimator
(n − 7)(n − 6)(n − 5)(n − 4)(n − 3)(n − 2)µ́81 4(n − 7)(n − 6)(n − 5)(n − 4)(n − 3)(n − 2)µ́2 µ́61
E[s8n ] = 3 3
−
(n − 1) n (n − 1)3 n3
6(n − 5)(n − 4)(n − 3)(n − 2) n2 − 10n + 35 µ́22 µ́41
5
8(n − 5)(n − 4)(n − 3)(n − 2)(3n − 7)µ́3 µ́1
− +
(n − 1)3 n3 (n − 1)3 n3
2 4 16(n − 4)(n − 3)(n − 2) 3n2 − 20n + 35 µ́2 µ́3 µ́31
2(n − 4)(n − 3)(n − 2) 3n − 30n + 35 µ́4 µ́1
+ +
(n − 1)3 n3 (n − 1)3 n3
4(n − 4)(n − 3)(n − 2) n3 − 9n2 + 45n − 105 µ́32 µ́21
8(n − 3)(n − 2)(3n − 7)µ́5 µ́31
+ −
(n − 1)2 n3 (n − 1)3 n3
2
2 2
12(n − 3)(n − 2) n3 − 9n2 + 35n − 35 µ́2 µ́4 µ́21
8(n − 3)(n − 2) 9n − 30n + 35 µ́3 µ́1
+ −
(n − 1)3 n3 (n − 1)3 n3
24(n − 3)(n − 2) n − 7n + 25n − 35 µ́22 µ́3 µ́1
3 2
2
4(n − 7)(n − 2)µ́6 µ́1
− −
(n − 1)n3 (n − 1)3 n3
3 2 24(n − 2) n2 − 4n + 7 µ́2 µ́5 µ́1
8(n − 2) 3n − 21n + 45n − 35 µ́3 µ́4 µ́1
− −
(n − 1)3 n3 (n − 1)2 n3
4 3 2
4
8µ́7 µ́1 (n − 3)(n − 2) n − 4n + 18n − 60n + 105 µ́2
− +
n3 (n − 1)3 n3
8(n − 2) 3n3 − 15n2 + 35n − 35 µ́2 µ́23 3n4 − 12n3 + 42n2 − 60n + 35 µ́24
− +
(n − 1)3 n3 (n − 1)3 n3
6(n − 2) n4 − 4n3 + 16n2 − 40n + 35 µ́22 µ́4 8 3n2 − 6n + 7 µ́3 µ́5
+ −
(n − 1)3 n3 (n − 1)2 n3
2
4 n − 2n + 7 µ́2 µ́6 µ́8
+ + 3 (B.21)
(n − 1)n3 n
Once this is done, one needs to collect all the terms and can get the final
result.
References