Statistical Theory MT 2007

Problems 1: Solution sketches

1. Suppose X
1
, X
2
, . . . , X
n
is a random sample from the Pareto distribution f(x, ) =

x
+1
with x > ,
> 0, and > 0 known. Find a minimal sucient statistic for .
Solution: The likelihood is
L(, x) =
(

)
n

n
j=1
x
+1
j
=

n

n
(

n
j=1
x
j
)
+1
,
so T =

n
j=1
X
j
is sucient.
For x, y we have
L(, x)
L(, y)
=
_

n
j=1
y
j

n
j=1
x
j
_
+1
is constant in if and only if t(x) = t(y), so T is minimal sucient.
2. Suppose X
1
, X
2
, . . . , X
n
is a random sample from the log-normal distribution with density
f(x, , ) =
1
x

2
exp
_

1
2
(log x )
2
_
with > 0 (so that log X
j
N(, )). Find a minimal sucient statistic for the parameter = (, ).
Solution:
L(, x) =
1
(

x
i
)(2)
n/2
exp
_

1
2
n

i=1
(log x
i
)
2
_
and the expression in the exponential can be written as

1
2

(log x
i
)
2
+

log x
i

n
2
2
,
so T = (

log X
i
,

(log X
i
)
2
) is sucient.
By considering the likelihood ratio we see that T is indeed minimal sucient.
3. Let X
1
, . . . , X
n
be i.i.d. uniform U
_

1
2
, +
1
2

random variables.
a) Show that
_
X
(1)
, X
(n)
_
is minimal sucient for .
b) Show that (S, A) =
_
1
2
(X
(1)
+X
(n)
), X
(n)
X
(1)
_
is minimal sucient for , and that the distribu-
tion of A is independent of (so A is an ancillary statistic).
c) Show that any value in the interval
_
x
(n)

1
2
, x
(1)
+
1
2

is a maximum-likelihood-estimator for .
1
Solution:
a) The likelihood is
L(, x) = 1
_

1
2
x
1
, . . . , x
n
+
1
2
_
= 1
_

1
2
x
(1)
, x
(n)
+
1
2
_
which is a function of (, x
(1)
, x
(n)
), so
_
X
(1)
, X
(n)
_
is sucient. For minimal suciency, note that
L(, x) = L(, y) if and only if (x
(1)
, x
(n)
) = (y
(1)
, y
(n)
), proving the rst assertion.
b) We have that
X
(1)
= S
1
2
A
X
(n)
= S +
1
2
A
so L(, x) is a function of (, s, a), hence (S, A) is sucient, and (S, A) is a function of a minimal
sucient statistic, so must be minimal sucient itself. To see that the distribution of A is independent
of , write Y
i
= X
i
, then Y
i
U(1/2, 1/2), and Y
(i)
= X
(i)
. Hence A = X
(n)
X
(1)
= Y
(n)
Y
(1)
,
the dierence of order statistics from a distribution which does not involve . So the distribution of A
does not depend on .
c) Follows directly from the likeihood - the likelihood
L(, x) = 1
_
x
(n)

1
2
x
(1)
+
1
2
_
is constant equal to 1 on the interval
_
x
(n)

1
2
, x
(1)
+
1
2

, and 0 otherwise.
4. The random variables X
1
, . . . , X
n
are independent with geometric distribution P(X
i
= x) = p(1p)
x1
for x = 1, 2, . . .. Let = p
1
.
(i) Show that

= X is the maximum likelihood estimator for .
(ii) Compute the expected Fisher information for

.
(iii) Show that

is unbiased; is

a minimum variance unbiased estimator?
Solution:
a) We have that
L(, x) =
n
(1
1
)

n
i=1
x
i
n
and so
() = nlog +
n

i=1
(x
i
1) log(1
1
);
dierentiate:

() =
n
( 1)
(x );
2
the only solution for

() = 0 is = x. The second derivative is

() =
n
(( 1))
2
{( 1) (x )(2 1)};
so that

) =
n
(

1))
2
(

1)) < 0
if

= x > 1, so if

> 1, the mle is

= x.
If x = 1, then
L(, x) =
n
,
and we know that 1, so the likelihood is maximized for = 1.
b) Calculate that E

X = , V ar

(X) =
(1)
n
, so, from

(),
I() =
n
2

2
( 1)
2
V ar

(X)
=
n
( 1)
,
which equals the inverse of the variance of X. As E

X = , the estimator is unbiased, hence it is

MVUE.
5. Let X
1
, . . . , X
n
be i.i.d. U[0, ], having density
f(x; ) =
1

, 0 x
with > 0.
(i) Estimate using both the method of moments and maximum likelihood.
(ii) Calculate the means and variances of the two estimators.
(iii) Which one should be preferred and why?
Solution: a) We have E

(X) =

2
, so the m.o.m. is

= 2X.
The likelihood is
L(, x) =
n
1(0 x
1
, . . . , x
n
)
=
n
1
_
x
(n)

_
and this is maximized for

= x
(n)
.
b) We have by construction
E

) = ,
3
and
V ar

) =
4
2
12n
=

2
3n
.
For

, calculate (see second year probability, or the book by Rice)
E

) =
n
n + 1
, V ar

) =
n
(n + 1)
2
(n + 2)

2
.
c) The m.l.e. is not unbiased but asymptotically unbiased, and has much smaller variance than the
m.o.m., so I would prefer it. You may decide otherwise if you put more emphasis on unbiasedness.
6. Suppose X
1
, . . . , X
n
are a random sample with mean and nite variance
2
. Use the delta method
to show that, in distribution,

n(X
n
2

2
) N(0, 4
2

2
).
What would you suggest if = 0?
Solution: From the central limit theorem we know that

n(X
n
) N(0,
2
). We use the function
g(s) = s
2
, so that g

(s) = 2s; now the delta method gives that

X
n
2
N(
2
, 4
2

2
/n).
Standardizing gives the result.
If = 0 then the result just gives convergence to point mass at 0, which is not informative. Instead
we could use that

n
X
n

N(0, 1), so that n

X
n
2

2

2
1
, and, in distribution,
nX
n
2

2
1
.
4