Statistical inference theory Monday, August 24 2020

PhD Qualifying Exam

The exam is 4 hours long. It is closed book/notes. But you are allowed to bring up to four
pages of formulas, theorems, etc. Using the laptops for any purpose except for checking
emails or communicating with us is prohibited. In case you feel that a problem is incorrect
or you need more assumptions to solve it, please clearly explain your reasoning and the
assumptions you need, and then solve that problem.

1. (15 Marks) Suppose that X ∼ Binomial(n, θ). The loss function is L(θ, a) = (θ −
a)2 /θ(1 − θ). Calculate the Bayes rule δ ∗ (X) for the prior θ ∼ Uniform[0, 1]. Is δ ∗ (X)
minimax?

2. (20 Marks) Let X1 , . . . , Xn be n independent

Pn observations and assume a parametric
model with log-likelihood function i=1 log f (xi ; θ), where θ ∈ Θ ⊂ Rp . Instead of
carrying out inference by means of standard Plikelihood theory, we will focus on the
n
Lq -likelihood approach, which is defined by i=1 Lq (f (xi ; θ)), where Lq (u) = (u1−q −
1)/(1 − q) and 0 < q < 1 is a fixed constant.

(a) (8 Marks) Suppose that M Lq E is consistent. Show that M Lq E is asymptotically

normally distributed under standard regularity conditions on the likelihood. It is
OK to assume that likelihood is as many times differentiable as you wish. But
please carefully state what you have assumed.
(b) (10 Marks) For simplicity consider the univariate case p = 1. We would like to
test the null hypothesis H0 : θ = θ0 using the following Lq -likelihood ratio test
statistic n n
X X
Tn = 2{ Lq (f (Xi , θ̂)) − Lq (f (Xi , θ0 ))},
i=1 i=1

where θ̂ is the M Lq E. Compute the asymptotic distribution of Tn under the null

hypothesis. Again clearly state the regularity conditions you are assuming.
(c) (2 Marks) What do you think will happen as q % 1 (converging to one from
below)? No rigorous proof is required here. Intuitive arguments are sufficient.

3. (20 points) Let X1 , X2 , . . . , Xn ∼ F , where F is absolutely continuous with respect to

the Lebesgue measure. Define the empirical CDF of the samples as
n
1X
F̂n (x) = I(Xi ≤ x).
n i=1

1
 p
Prove that supx |F̂n (x) − F (x)| → 0.
Hint: You can use Hoeffding Inequality. Hoeffding Inequality implies that if Z1 , Z2 , . . . , Zn ∼
Bern(p), then
n
1X 2
P(| Zi − p| > ) ≤ 2e−2n .
n i=1

4. (20 Marks) Let X1 , X2 , . . . , Xn ∈ R denote iid samples from F which is absolutely

continuous with respect to the Lebesgue measure. Also, assume that E(X1 ) = µ, and
E(X1 − µ)2 = σ 2 . We would like to characterize the limiting distribution of
n
X
|Xi − X̄|,
i=1

where X̄ denotes the sample average. Follow these steps to find this limiting distribu-
tion:

(a) (8 Marks) Define In as the set of all indices for which Xi falls in the interval
between X̄n and µ, i.e., In = {i = 1, 2, . . . , n | Xi ∈ (min(X̄n , µ), max(X̄n , µ)]}.
Prove that
|In | p
→ 0.
n
Hint: If you need a hint, let me know. I will give you a hint and deduct 3 Marks.
(b) (10 Marks) Prove that
n n
1X 1X
|Xi −X̄n |− |Xi −µ|−(µ−X̄n )P(X1 > µ)+(µ−X̄n )P(X1 < µ) = op (|X̄n −µ|).
n i=1 n i=1

(c) (2 Marks) Characterize

Pn the right scaling and the non-degenerate limiting distri-
bution of n1 P i=1 |X i − X̄ n |. You do not have to simplify your answer. Note that
1 n
the mean of n i=1 |Xi − X̄n | is not zero. So, you should subtract the mean before
the scaling.

5. (25 marks) Let {(Yi , Xi , Ti )}ni=1 , be iid random samples with the Xi , Ti pairs distributed
according to distribution G, and Yi given Xi and Ti normally distributed with expec-
tation
α∗ (Ti ) + β ∗ (Xi − E(Xi |Ti ))
and variance σ 2 , for a function α : R → R, and unspecified scalar β and known σ 2 . You
may suppose that α has a continuous derivative bounded above and below by finite
values, and the Ti take values on the interval [0, T ], with density bounded below by a
positive finite value. Also assume that the random variable X is always in the interval
[−1, 1].

2
(a) (5 marks) Suppose that both the function α and the distribution G (hence E(Xi |Ti ))
are known. Let β̂1 be the solution of the following estimating equation:
n
X
0 = (Yi − α∗ (Ti ) − β(Xi − E(Xi |Ti )))(Xi − E(Xi |Ti )),
i=1
√
Characterize the limiting distribution of n(β̂1 − β ∗ ).
(b) Now, suppose that both G and α are unspecified, and consider the following
estimating equation for β:
n
X
0 = (Yi − Ȳj(i) − β(Xi − X̄j(i) ))(Xi − X̄j(i) ),
i=1

where j(i) is defined implicitly by

 
j(i) − 1 j(i)
Ti ∈ T √ γ , T √ γ ,
n n
where γ ∈ (0, 1) is a specified fixed number. Recall that Ti s take values on
the interval [0, T ]. Furthermore, Ȳk is the sample average of those Yi for which
j(i) = k, and X̄k is the sample average of those Xi for which j(i) = k. In other
words, if we define
Ik = {i = 1, 2, . . . , n | j(i) = k}.
 
k−1 k
∆Tk = T √ γ , T√ γ ,
n n
then
1 X
X̄k = Xi ,
|Ik | i∈I
k

where the notation |Ik | is used to denote the number of elements in Ik . We should
also emphasize that Ik depends on T1 , T2 , . . . , Tn . Answer the following questions:
i. (5 Marks) Prove that there exists a constant c, such that P(mink |Ik | <
cn1−γ/2 ) → 0 as n → ∞.
p
ii. (7 Marks) Prove that X̄k → E(Xi |Ti ∈ ∆Tk ).
iii. (8) Prove that
n n
1X 1X
| (Xi − X̄j(i) )2 − (Xi − E(Xi |Ti ∈ ∆Tj(i) ))2 | = op (1)
n i=1 n i=1

iv. (optional) Characterize the right scaling that leads to a non-degenrate distri-
bution for the following quantity:
n n
1X 1X
(Xi −X̄j(i) )(Yi −Ȳj(i) )− (Xi −E(Xi |Ti ∈ ∆Tj(i) ))(Yi −E(Yi |Ti ∈ ∆Tj(i) ))
n i=1 n i=1
Can you characterize the limiting distribution?