BITS Pilani

Session 10
(Maximum Likelihood Estimation
&
ANOVA)
17th Feb 2024
Maximum Likelihood Estimation (MLE)
Estimation is the process of estimating unknown true values of
population parameters using their corresponding best sample
statistics (good estimators) in an optimum manner.

An estimator is said to be a good if it is

o unbiased,

o consistent,

o efficient and

o sufficient while estimating its parameter.

 Maximum Likelihood Estimation (MLE)

❖ Method of Maximum Likelihood Estimation is the best and most

popular one among all methods to obtain an almost good or
best estimator for a population parameter.
❖ It is a method of obtaining an estimator which most (maximum)
likely estimates the true value of the parameter i.e., finding an
estimator that can give most likely nearer value for the unknown
true value of parameter.
❖ The corresponding estimator is called maximum likelihood
estimator (MLE).
Maximum Likelihood Estimation (MLE)

Suppose we have a random sample x1,x2, ..,xn whose assumed

probability distribution depends on some unknown parameter θ.
Ex:
1) For Binomial unknown parameters are n, p.
2) For Poisson unknown parameter is λ.
3) For Normal unknown parameters are 𝜇 𝑎𝑛𝑑 𝜎 2 .
Our goal is to find good estimate of θ(population parameter) using
sample and which can be done with the help of MLE.
Maximum Likelihood Estimation

❖ It is observed that a good estimate of unknown

parameter θ would be the value of θ that
maximizes the probability
❖ i.e. the likelihood of getting the data we observed
(this is reason, why we called as likelihood
function)
 Maximum Likelihood function

❖ Let x1,x2, ..,xn be i.i.d. random variables drawn from

some probability distribution that depends on some
unknown parameter θ.
❖ The goal of MLE to maximize likelihood function
𝐿 𝜃 = 𝑓(x1,x2, ..., xn 𝜃
= 𝑓(x1 𝜃 *𝑓(x2 𝜃 …𝑓(xn 𝜃
n
L( ) =  f ( xi /  )
i =1
Maximum Likelihood Estimation (MLE)

❖ The maximum likelihood estimate (MLE) of θ is that value of θ that

maximizes likelihood(θ).
It is defined as n
L( ) =  f ( xi /  )
i =1
n
log L( ) =  log f ( xi /  )
i =1
Maximum Likelihood Estimation (MLE)

MLEs are:
❖ Consistent
❖ Efficient
❖ Sufficient
❖ MLEs May (or may not) be unbiased
❖ MLEs are Asymptotically normally distributed
❖ Asymptotically tend to have least variance.
Maximum Likelihood for a Binomial distribution

❖ Suppose we wish to find the maximum likelihood

estimate (MLE) of θ for a Binomial distribution,

pk (k ,  ) = nCk k (1 −  ) n − k
log pk (k ,  ) = log(nCk ) + k log( ) + (n − k ) log((1 −  )
 log pk (k ,  ) k n−k
= 0 0+ − =0
  1−
k
k − k = n − k   =
n
 Example:
A Coin is tossed 50 times, resulted 30heads and 20
tails.
i. Find the Likelihood function for the number of
heads.
ii. Find the maximum likelihood estimate.
Solution:

Let p – heads , (1-p ) – tails then the Likelihood function is

L = P( X =30 / p) = 50𝐶30 𝑝30 1 − 𝑝 20 --------- (1)
Log (L) = Log [ 50𝐶30 𝑝30 1 − 𝑝 20]
= Log [50𝐶30 ] + Log [𝑝30] + Log [ 1 − 𝑝 20]
= Log [50𝐶30 ] + 30 Log [p] + 20 Log [(1 − p)]
Differentiating with respect to p and equating to 0

30 20
+ −1 = 0
𝑝 1−𝑝
𝒑 = 𝟑Τ𝟓

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

14
MLE for Poisson Distribution Parameter
which is the maximum likelihood estimate
Example:
In one area along the interstate, the number of dropped wireless phone connections
per call follows a Poisson distribution. From four calls, the number of dropped
connections is 2, 0, 3, 1
(a) Find the maximum likelihood estimate of λ.
(b) Obtain the maximum likelihood estimate that the next two calls will be completed
without any accidental drops.
2+0+3+1
Solution: a) 𝑥ҧ = λ = = 1.5
4
𝑒 −1.5 1.5 0
b) For next one call, P(x=0) = = 𝑒 −1.5
0!
For next two calls = P(X=0)P(X=0)= (𝑒 −1.5)(𝑒 −1.5 )= 𝑒 −3
 One-way ANOVA
ANOVA is a statistical technique used to determine whether differences
exist among three or more population means.

➢ In one-way ANOVA the effect of one factor on the mean is tested. It is

based on independent random samples drawn from k – different levels
of a factor, also called treatments.

The data and the notations used in one-way ANOVA are represented in
the following tabular structure.
Step 6: Critical Value
𝑓𝑒 < 𝑓 𝑘−1,𝑛−𝑘 ,𝛼

Step 7:
If 𝐹0 < 𝑓𝑒 then reject 𝐻0
Example:1
Linda is using ANOVA to measure whether there is a difference between
the average weekly sales of her 3 salespeople. The test will be at the 0.05
level of significance.
Weekly Sales(x) in Thousands of Dollars for 3Treatments(T)
Salesperson

L 7 6 7 4

M 6 8 6 6

N 9 8 7 10

Solution: 1) Null Hypothesis, 𝐻0 : 𝜇1 = 𝜇2 = 𝜇3

Alternate Hypothesis, 𝐻1 : 𝜇1 ≠ 𝜇2 ≠ 𝜇3

2) Level of Significance, α = 0.05

Solution:
Solution:
Example:2

Construct an analysis of variance table, and test the equality of mean weights with
α = 0.05. Each laboratory measures the tin-coating weights of 12 disks and that the
results are as follows:
 Solution:

4. Calculations:

The totals for the k = 4 samples all of sample size 𝑛𝑖 = 12, are 3.22, 2.72, 2.76,
and 3.00, respectively. The grand total is T = 11.70.
 Contd..

ANOVA table
 Contd..

5. Decision: Since the observed value of F exceeds 2.82 = 𝐹0.05 , the null
hypothesis of equal mean weights is rejected at the 0.05 level of significance.
We conclude that the laboratories are not obtaining consistent results.
 TWO-WAY CLASSIFICATION

➢ In two-way ANOVA, a study variable is compared over three or more groups,

controlling for another variable.

➢ The grouping is taken as one factor and the control is taken as another factor. The

grouping factor is usually known as Treatment.

➢ The control factor is usually called Block.

➢ The accuracy of the test in two-way ANOVA is considerably higher than that of one-

way ANOVA, as the additional factor, block is used to reduce the error variance.
 Example:
1. The illness caused by a virus in a city concerning some restaurant inspectors is
not consistent with their evaluations of cleanliness of restaurants. In order to
investigate this possibility, the director has five restaurant inspectors to grade
the cleanliness of three restaurants. The results are shown below.:

Carry out two-way ANOVA at 5% level of significance.

Solution:
Contd..
Contd..
Contd..
Contd..
Contd..
Contd..
 Practice Problem 1

Consider a sample 0,1,0,0,1,0 from a binomial distribution, with the

form P[X=0]=(1-p), P[X=1]=p. Find the maximum likelihood estimate
of p.
 Practice Problem 2:

A researcher wishes to see whether there is any difference in the weight gains of
athletes following one of three special diets. Athletes are randomly assigned to three groups
and placed on the diet for 6 weeks. The weight gains (in pounds) are given.
Assume weight gains are normally distributed and the variances are equal. At a 0.05
significance level, can the researcher conclude that there is a difference in the diets?
•Diet A: 3, 6, 7, 4
•Diet B: 10, 12, 11, 14, 8, 6
•Diet C: 8, 3, 2, 5
 Practice Problem 3
A turkey farmer tested three kinds of poultry feeds with the weights (in pounds)
of the grown turkeys in each sample given below. Test at α = 0.05 whether there is a
difference in the mean weights of turkeys consuming the different feeds.
If you find a difference, perform pairwise tests to determine where the difference lies.
Assume the distributions are normal and variances are equal.
IMP Note to Self
Thank You !