Probability and statistics May 24, 2016

Lecturer: Prof. Dr. Mete SONER

Coordinator: Yilin WANG

Solution Series 11

Q1. Suppose that X1 , · · · , Xn form a random sample from a Poisson distribution for which the
mean is unknown. Determine the maximum-likelihood estimator of the standard deviation
of the distribution.
Solution:
√
The Poisson distribution with mean λ has variance λ, thus the standard deviation is λ.
Given the parameter λ, the p.f. of the Poisson distribution is

λk
P(X = k|λ) = e−λ .
k!

Hence the M.L.E. is the value λ which maximizes

n
Y λXi (e−λ λX̄ )n
f (λ) = e−λ = .
i=1
Xi ! X1 ! . . . Xn !

Where X̄ = (X1 + · · · + Xn )/n. We need to find the λ which maximizes

g(λ) = e−λ λX̄ = exp(−λ + X̄ ln(λ)).

g 0 (λ) = (−1 + X̄/λ)g(λ).

√ maximum of g is reached when λ = X̄. Thus the M.L.E. of the standard deviation is
The
X̄.

Q2. Suppose that X1 , · · · , Xn form a random sample from a distribution for which the p.d.f.
f (x|θ) is as follows: (
θxθ−1 for 0 < x < 1,
f (x|θ) =
0 otherwise.
Also, suppose that the value of θ is unknown (θ > 0). Find the M.L.E. of θ.
Solution:
Let !θ−1
n
Y n
Y
L(θ) := θXiθ−1 = θn Xi .
i=1 i=1

1
Probability and statistics May 24, 2016

The derivative of ln(L)(θ) is:

" n
!#0 n
!
Y n Y
n ln(θ) + (θ − 1) ln Xi = + ln Xi .
i=1
θ i=1

Thus
n 1
θ0 = − Qn =− ,
ln ( i=1 Xi ) ln(X)
where ln(X) = n1 ni=1 ln(Xi ), is a critical point of L. For θ < θ0 , ln(L) is increasing and for
P
θ > θ0 , ln(L) is decreasing. Thus θ0 is the global maximum, and is the M.L.E. of θ.

Q3. In a lake we want to estimate the amount of a certain type of fish. For this we mark 5 fishes
and we let them mix with the others, when they are well mixed we fish 11, and we realize
that there are 3 marked and 8 non-marked. What is the maximum-likelihood estimator for
the amount of fishes?.
Solution:
Define X the amount of marked fishes we fished. If there are N fishes in the lake, the
probability of X = 3 is given by
5 N −5



3
PN (X = 3) = N

8 1{N ≥13}
11
5!(N − 5)!11!(N − 11)!
= 1{N ≥13} := g(N ).
3!2!8!(N − 13)!N !

We have to find Nmax ∈ N so that g(Nmax ) = supN ∈N g(N ). We have that for N ≥ 13

g(N ) (N − 12)(N + 1)
−1= −1
g(N + 1) (N − 4)(N − 10)
3(N − 17, 333 . . .)
= ,
(N − 4)(N − 10)

thus,

g(N ) ≤ 1 if N ≤ 17,
g(N + 1) ≥ 1 if N ≥ 18.

Then Nmax = 18.

Q4. A gas station estimates that it takes at least α minutes for a change of oil. The actual
time varies from costumer to costumer. However, one can assume that this time will be well
represented by an exponential random variable. The random variable X, therefore, possess
the following density funciont

f (t) = eα−t 1{t≥α} ,

2
Probability and statistics May 24, 2016

i.e. X = α + Z where Z ∼ Exp(1). The following values were recorded from 10 clients
randomly selected (the time is in minutes):

4.2, 3.1, 3.6, 4.5, 5.1, 7.6, 4.4, 3.5, 3.8, 4.3.

Estimate the parameter α using the estimator of maximum likelihood.

Solution:
We have that the likelihood function is given by:
n
Y
L(X1 , ..., Xn , α) = exp(α − Xi )1{Xi ≥α} ,
i=1
n
X
= exp(nα − Xi )1{Tn Xi ≥α} ,
i=1
i=1
Pn
we note that f (α) := exp(nα − i=1 Xi ) > 0 is increasing, so its maximum is attained at
the maximum point where 1{Tn {Xi ≥α}} 6= 0. Then the point that maximizes the likelihood
i=1
is in α = mini=1,..,n {Xi }.

Q5. Suppose that X1 , · · · , Xn form a random sample from a normal distribution for which both
the mean and the variance are unknown. Find the M.L.E. of the 0.95 quantile of the distri-
bution, that is, of the point θ such that P(X < θ) = 0.95.
The 0.95 quantile of a standard normal distribution is 1.645 =: θ0 .
Solution:
Let µ and σ 2 denote the mean and the variance of Xi , the density of Xi is

(x − µ)2
 
2 1
f (x|µ, σ ) = √ exp − .
2πσ 2 2σ 2

The 0.95 quantile of Xi is the value θ = θ(µ, σ) such that

 
X −µ θ−µ
P(X < θ) = 0.95 = P < ,
σ σ
θ−µ
thus θ0 = σ
which implies that θ = σθ0 + µ.
The ln of the product of the density of Xi s is

n X (Xi − µ)2
L(µ, σ 2 ) := − ln(2πσ 2 ) − .
2 2σ 2
We look for µ and σ 2 which maximizes L(µ, σ 2 ).

∂L X Xi − µ
= =0
∂µ σ2

3
Probability and statistics May 24, 2016

(Xi − µ)2
X 
∂L n 1
2
=− 2 + = 0.
∂σ 2σ 2 (σ 2 )2
Which give, for any σ 2 , µ 7→ L(µ, σ 2 ) is maximized when
1X
µ=X= Xi ,
n
which is the sample average of Xi . And for µ = X, σ 2 7→ L(µ, σ 2 ) is maximized when
X (Xi − X)2
σ2 = = (Xi − X)2 ,
n
the sample variance.
We substitute the values of µ and σ 2 into the expression of θ and gives the M.L.E. of 0.95-
quantile of X: q
θ = θ0 (Xi − X)2 + X.