Professional Documents
Culture Documents
(Width 1.5cm) Nitm Center National Institute of Technology Meghalaya Laitumkharah, Shillong-793003 15th May 2020
(Width 1.5cm) Nitm Center National Institute of Technology Meghalaya Laitumkharah, Shillong-793003 15th May 2020
(Width 1.5cm) Nitm Center National Institute of Technology Meghalaya Laitumkharah, Shillong-793003 15th May 2020
Definition
A point estimate is an approximate value for a parameter in the P
distribution of X obtained from a sample. A sample mean x̄ = n1 ni=1 xi
is an estimator of the mean µ ofh X, and the sample variancei
1 Pn 2 1 2 2
s 2 = n−1 i=1 (xi − x̄) = n−1 (x1 − x̄) + ... + (xn − x̄) is a sample
of the variance σ 2 of X.
Example
Suppose that, X the reaction time to a certain stimulus has a uniform
distribution on the interval from 0 to an unknown upper limit θ. It is
desired to estimate θ on the basis of a random sample X1 , X2 , ..., Xn of
reaction times. Since, θ is largest possible time in the entire population of
reaction times, consider as a estimator of the largest sample reaction time
θ̂ = max(X1 , X2 , ..., Xn ). If n = 5 and
x1 = 4.2, x2 = 1.7, x3 = 2.4, x4 = 3.9, x5 = 1.3, the point estimator of θ is
θ̂ = max(4.2, 1.7, 2.4, 3.9, 1.3)=4.2, where x1 , x2 , ..., x5 are the values of
X1 , X2 , ..., X5 respectively.
Mr. Mantu Kumar(Roll N-S18MA006) MSc(4th
A Study
Semester),
on Estimation
Department
of Coefficient
of Mathematics
of Variation
underfor
theNormal
supervision
Distribution
May
of Dr.
13, 2020
Adarsha Kumar
9 / 1 Jena National Institute of Tech
Point Estimate for Coefficient of Variation
If, X is a normal random variable with mean µ and variance σ 2 , then
the parameter
σ
κ≡ (5)
µ
is called the population coefficient of variation.
Let Xi for i = 1, ..., n be an independent random sample, with
Xi v N(µ, σ 2 ) for each i. In terms of usual sample estimates of the
normal parameters, sample mean
n
X Xi
X̄ = (6)
n
i=1
Definition
Given independent observations x1 , x2 , ..., xn from a population density
function(continuous case) or probability mass function (discrete case)
f(x;θ), the maximum likelihood estimator θ̂ is that which maximizes the
likelihood function
L(x1 , x2 , ..., xn , θ) = f (x1 , θ)f (x2 , θ)...f (xn , θ).
and
∂L(x, θ)
=0 (10)
∂θ
∂ 2 L(x, θ)
<0 (11)
∂θ2
Mr. Mantu Kumar(Roll N-S18MA006) MSc(4th
A Study
Semester),
on Estimation
Department
of Coefficient
of Mathematics
of Variation
underfor
theNormal
supervision
Distribution
May
of 13,
Dr.2020
Adarsha Kumar
12 / 1 Jena National Institute of Tech
Maximum Likelihood Estimator
then, the maximum among the values of θ for which equations (6)
and (7) are satisfied is called ’maximum likelihood estimator’.
L(θ) and logL(θ) have their maxima at the same values of θ and
hence, it is sometimes easier to find the maximum of logarithm of the
likelihood function.
Example
Suppose that 10 rats are used in biomedical study, when they are injected
with cancer cells and then given a cancer drug that is designed to increase
their survival rate . The survival times in months are
14,17,27,18,12,8,22,13,19 and 12. Assume that exponential distribution
exerts. Give a MLE of the mean survival time.
Solution:
(
1 −x/β
β
e , if x is greater than 0;
f (x, β) = (12)
0, elsewhere.
Thus, the log likelihood function for the data given n=10 is
−x1 −x2 −x10
L(x1 , x2 , ..., xn ; β) = β1 e β . β1 e β .... β1 e β
x1 +x2 +...x10
1 −
= β 10
e β
10
P
xi
1 − i=1β
= β 10
e
10
1 P
lnL = −10lnβ − β xi
i=1
10
∂lnL −10 1 P
∂β = β + β2
xi = 0.
i=1
10
P
=⇒ −β10 + xi = 0
i=1
10
1 P
=⇒ β = 10 xi
i=1
1 1
=⇒ β = 10 (14+17+27+18+12+8+22+13+19+12) = 10 (162) = 16.2.
n
∂L n 1 X
= − + (xi − µ)2 (15)
∂σ 2 2σ 2 2σ 4
i1
Putting these derivatives equal to zero and solving the resulting equations
for µ and σ 2 ,
n
1 P
σ2
(xi − µ) = 0
i=1
n
P
=⇒ xi − nµ = 0
i=1
n
1X
=⇒ µ̂ = xi (16)
n
i=1
n
n 1 X
− 2+ 4 (xi − µ)2 = 0 (17)
2σ 2σ
i=1
n n 2
1 P P
=⇒ −n + σ2
(xi − xi ) = 0 (using (16))
i=1 i=1
n
Mr. Mantu Kumar(Roll N-S18MA006) MSc(4th
A Study
Semester),
on Estimation
Department
of Coefficient
of Mathematics
of Variation
underfor
theNormal
supervision
Distribution
May
of 13,
Dr.2020
Adarsha Kumar
17 / 1 Jena National Institute of Tech
1
(xi − x̄)2 = 0
P
=⇒ −n + σ2
i=1
CV for MLE
n
=⇒ σ̂ 2 = 1
(xi − x̄)2
P
n
i=1 s
n
(xi −x̄)2
P
n
i=1
Hence, CV of maximum likelihood estimator = Pn .
xi
i=1
Definition
The rth moment of a RV ’X’ about the mean µ(also called the central
mean) is defined as
µr = E [(X − µ)r ] where r=0,1,2,....It follows that µo = 1, µ1 = 0 and
µ2 = Pσ2,
µr = R ni=1 (xi − µ)r f (x) (discrete case)
∞
µr = −∞ (x − µ)r f (x)dx (continuous case)
µ01 = µ, µ00 = 1,
µ02 = µ02 − µ2 ,
µ03 = µ03 − 3µ02 µ + 2µ3 , and so on.
Definition
Let x1 , x2 , ..., xn be a random sample from the density function f(.). Then
the rth sample moment about 0, denoted by
n
1X r
Mr0 = Xi (18)
n
1
Definition
In particular, if r=1 we Pget the sample mean, which is usually denoted by
X̄ or X¯n , i.e. X¯n = n1 ni=1 Xi ; also the rth sample moment about X¯n
denoted by Mr is defined to be
n
1X r
Mr = (Xi − X¯n ) (19)
n
i=1
Definition
Let f (.; θ1 , θ2 , ..., θn ) be a density of a random variable which has k
parameters θ1 , θ2 , ..., θk . Let µ0r denotes rth moment about 0; that is
µ0r = δ[x 0 ]. In general µ0r will be a known function of the k parameters
θ1 , θ2 , ..., θk . Denote this by writing µ0r = µ0r (θ1 , θ2 , ..., θk ). Let
X1 , X2 , ..., Xn be a random sample from the density f (.;P θ1 , θ2 , ..., θk ) and
as before let Mj0 be the jth sample moment, i.e.Mj0 = n1 ni=1 xij . From the
k equations Mj0 = µ0j (θ1 , θ2 , ..., θk ), j=1,2,...,k.
Definition
If the k variables θ1 , ..., θk and let θˆ1 , ..., θˆk be their respective solutions.
We say that the estimator (θ1 , ..., θk ), where θˆj estimates θj is the
estimator of (θ1 , θ2 , ..., θk ) obtained by method of moments. The
estimators are obtained by replacing population moments by sample
moments.
Mr. Mantu Kumar(Roll N-S18MA006) MSc(4th
A Study
Semester),
on Estimation
Department
of Coefficient
of Mathematics
of Variation
underfor
theNormal
supervision
Distribution
May
of 13,
Dr.2020
Adarsha Kumar
21 / 1 Jena National Institute of Tech
Moment Estimator
Example
Let X1 , X2 , ..., Xn be a random sample from a normal distribution with
mean µ and variance σ 2 . Let (θ1 , θ2 ) = (µ, σ). Estimate the parameters µ
and σ by the method of moments.
Definition
The bias of θ̂ relative to θ is defined as Biasθ [θ̂] = E θx [θ̂] − θ = E θx [θ̂ − θ].
An estimator is said to be unbiased if its bias is equal to zero for all values
of parameter θ.
Definition
An estimator δ(x) is an biased estimator of a function g (θ) of the
parameter θ if Eθ δ(x) = g (θ) for every possible values of θ. An estimator
that is not unbiased is called biased estimator. The difference between the
expectation of an estimator and g (θ) is called the bias of the estimator.
That is bias of δ as an estimator of g (θ) is Eθ [δ(x) − g (θ)] and S is
unbiased iff the bias is 0 for all values of θ.
In case of a sample from a normal distribution with unknown mean θ, X¯n
is an unbiased estimator of θ because Eθ (X¯n ) = θ for ∞ < θ < ∞.
Theorem
If S 2 is the variance of a random sample from an infinite population with
finite variance σ 2 , S 2 is an unbiased estimator of σ 2 .
Proof.
Let X1 , X2 , ..., Xn be its random variables with variance σ 2 < ∞. We have
Pn 2 Pn 2
(Xi − µ)2 − i ni=1 (X̄ − µ)
P
i=1 (Xi − X̄ ) = h i=1
1 Pn 2
Now, E (S 2 ) = E n−1 i=1 (Xi − X̄ )
hP i
1 n 2 2 1 Pn
= n−1 i=1 E (Xi − µ) − nE ( X̄ − µ) = n−1 ( i=1 σXi 2 − nσX̄2 )
However, σXi 2 = σ 2 , for i=1,2,...,n and σX̄2 = n1 σ 2 ,
1 2
∴ E (S 2 ) = n−1 (nσ 2 − n σn ) = σ 2 .
Hence, S 2 is an unbiased estimator of σ 2 .
Example
Let X1 , X2 , ..., Xn be a random sample from a population with finite mean
µ. Show that the sample mean X̄ and 13 X̄ + 23 X̄ are both unbiased
estimator of µ.