SPS 2349 Test of Hypothesis

Generalized Likelihood Ratio Test

Dr. Mutua Kilai

Kirinyaga University

Introduction
Suppose a random variable X has a pdf f (x1 , θ1 , θ2 , ..., θk ) depending on the unknown parameters θ1 , θ1 , ..., θk
The set of all parameters is denoted by Ω known as the parameter space.
Let Ω0 be a subset of Ω consider the hypothesis:

H0 : (θ1 , θ2 , ..., θk ) ∈ Ω0 vs Ha : (θ1 , θ2 , ..., θk ) ∈ Ω − Ω0

The above case is a composite hypothesis against a composite one. In this case we do not use Neyman
Pearson lemma but we use a more generalized form of it generalized likelihood ratio test(LRT)
We denote the likelihood function of a sample x1 , x2 , ..., xn by
n
Y
L(Ω) = f (xi , θ1 , θ2 , ..., θk )
i=1

Let L(Ω̂) = max L(Ω) be maximized L(Ω) and L(Ω0 ) = max(L(Ω0 )) be maximized L(Ω0 ) with Ω0 only
We take the likelihood ratio as a quotient

L(Ωˆ0 )
λ=
L(Ω̂)

Since λ is the quotient of non-negative functions it is greater than or equal to zero. Further since Ω0 ∈ Ω
then
L(Ω̂) ≤ L(Ω̂)
hence 0 ≤ λ ≤ 1
To test H0 : θ ∈ Ω0 against Ha : θ ∈ Ω − Ω0 the critical region for the likelihood ratio test is the set of points
in the sample for which
L(Ωˆ0 )
λ= ≤k
L(Ω̂)

Where k is a constant determined so that the level of significance is α

h i
P λ ≤ k|H0 is true = α

1
Example 1 INTRODUCTION

Example 1
X ∼ N (µ, σ 2 ) where µ&σ 2 are unknown. Test the hypothesis

H0 : µ = µ0 vs H1 : µ ̸= µ0

Solution
Let x1 , x2 , ..., xn be a random sample of size n from X then
 1  n2 n
− 2σ12
X
L(X, µ, σ 2 ) = e (xi − µ)2
2πσ 2 i=1

In
Ω = {(µ, σ) : −∞ < µ < ∞, σ > 0}
Ω0 = {(µ, σ) : µ = µ0 , σ > 0}

L(Ω̂) = maxµ,σ2 ∈Ω L(X, µ, σ 2 )

n
n n 1 X
ln L(µ, σ 2 ) = − ln 2π − ln σ 2 − 2 (xi − µ)2
2 2 2σ i=1

n
∂l X (xi − µ)
=0→2 0 → µ̂ = x̄
∂µ i=1
2σ 2

P2
∂ ln L n 1 i=1 (xi − µ)2
2
= 0 → −( ) 2 +
∂σ 2 σ 2(σ 2 )2

 1  1
Pn 2
L(Ω̂) = e− 2σ2 i=1 (xi −x̄)
ˆ
2π σ 2
Pn
− (xi −x̄)2
 1  n2 h n i n −
1
Pi=1
n
= Pn
2
e n i=1 i
(x −x̄)2 (1)
2
2π i=1 (xi − x̄)
 1  n2 h n i n2 n
= P e− 2
2π (xi − x̄)2

Under Ω0 the maximum likelihood estimates µ&σ 2 are

n
1X
µ̂ = µ0 , σˆ2 = (xi − µ0 )2
n i=1

 1  1
Pn 2
L(Ω̂) = e− 2σ2 i=1 (xi −µ0 )
2π σˆ2 Pn
− (xi −µ0 )2
 1  n2 h n i n −
1
Pi=1
n
= Pn
2 (x −µ )2
e n i=1 i 0 (2)
2
2π i=1 (xi − µ0 )
 1  n2 h n i n2 n
= P e− 2
2π (xi − µ0 )2

By Dr. Mutua Kilai 2

Example 1 INTRODUCTION

Therefore the likelihood ratio criterion is given by

L(Ωˆ0 )
λ= ≤k
L(Ω̂)
h i n2
Pn n
(xi −µ0 )2
= h i=1
i n2 ≤ k (3)
Pn n
(xi −x̄)2
i=1
h Pn (x − x̄)2 i n2
i
= Pni=1 ≤k
i=1 (x i − µ0 )2

The test is to reject H0 when λ ≤ c Where c is such that

P [λ ≤ c] = α

n
X n
X
(xi − µ0 )2 = (xi − x̄ + x̄ − µ0 )2
i=1 i=1
X X (4)
= (xi − x̄)2 + 2(x̄ − µ0 ) (xi − x̄) + n(x̄ − µ0 )2
X
= (Xi − X̄)2 + n(x̄ − µ0 )2

So " # n2
(xi − x̄)2
P
λ= P
(xi − x̄)2 + n(x̄ − µ0 )2

" # n2
1
λ= P
(x̄−µ0 )2
1 + P(x −x̄)2
i

√ √
n(x̄ − µ0 )/σ n(x̄ − µ0 )
T = hP 1 = r
(x −x̄)2
i 2
P
i 2 (xi −x̄)
σ 2 (n−1) n−1

This variable has the student t distribution with n − 1 degrees of freedom when H0 is true.
Thus

n(x̄ − µ0 )2
T2 = P 2
(xi −x̄)
n−1

" # n2
1
λ= T2
1 + n−1

λ ≤ c → T2 ≥ k

We therefore look for a constant c such that

P [λ ≤ c|H0 ] = α = P [T 2 ≥ k|H0 ]

By Dr. Mutua Kilai 3

Example 2 INTRODUCTION

It is enough to find a constant k such that

√
P [T 2 ≥ k|H0 ] = α = P [|T | > k|H0 ] = α

or

P [−k < T < k] = 1 − α

Since t distribution is symmetric about zero we usually find k such that

√ α √
P [T > k] = = P [T < − k]
2
So the test is to compute the statistic
√
n(x̄ − µ0 )
T = rP
2
(xi −x̄)
n−1

And reject H0 if |T | ≥ t1− α2 (n−1)

Example 2
Alfafa yields of six test plots are given by

1.5, 1.9, 1.2, 1.4, 2.3, 1.3

per acre. Use a critical region of 0.05 to test

H0 : µ = 1.8 vs µ ̸= 18
Assume the yields have a normal distribution of mean µ
Solution
The test is to compute
√
n(x̄ − µ0 )2
T = q
(xi −x̄)2
n−1

and reject H0 when |T | > c Where c is such that

P [|T | > c|H0 ] = α

or
P [−c < T < c|H0 ] = 1 − α = 0.05

Here

n=6
and
x̄ = 1.6

By Dr. Mutua Kilai 4

Example 3 INTRODUCTION

and X
(xi − x̄)2 = 0.88

Therefore

√ √
n(x̄ − µ0 ) 6(1.6 − 1.8)
T = rP = q = −1.17
(xi −x̄)2 0.88
5
n−1

From the tables with 5 df the value of c is 2.571.

Since
|T | = 1.17
and
1.17 < 2.571
we fail to reject H0 at 0.05 level of significance.

Example 3
X ∼ N (µ, σ 2 )

To test
H0 : µ = µ0 , σ = σ0 is known

vs

H1 : µ = µ1

Solution

Ω = {(µ, σ) : −∞ < µ < ∞, σ = σ0 }

Ω0 = {(µ, σ) : µ = µ0 , σ = σ0 }

 1  n2 − 1 P(xi −µ)2
2
L(X, µ, σ) = 2 e 2σ0
2πσ0

 1  n2 − 1 P(xi −x̄)2
2
L(Ω̂) = e 2σ0
2πσ02

 1  n2 − 1 P(xi −µ0 )2
2
L(Ω̂) = e 2σ0
2πσ02

Hence
P
− 1
(xi −µ0 )2
L(Ωˆ0 ) 2σ 2
P P
e 0 − 1
(xi −µ0 )2 + 1
(xi −x̄)2
2σ 2 2σ 2
λ= = 1
P =e 0 0
L(Ω̂) − (xi −x̄)2
2σ 2
e 0

The critical region is

By Dr. Mutua Kilai 5

 TESTING THE VARIANCE OF NORMAL POPULATION

P P
− 1
(xi −µ0 )2 + 1
(xi −x̄)2
2σ 2 2σ 2
λ=e 0 0 ≤k

Taking natural logarithm

1 X 2 1 X
− (x i − x̄) + (xi − µ0 )2 ≤ ln k
2σ02 2σ02

Simplifying further we have

1 1 X
2 n(x̄ − µ0 )2 ≤ ln k + 2 (xi − x̄)2
2σ0 2σ0

Then

n(x̄ − µ0 )2
<k
σ02
x̄−µ0
Let Z = σ
√0
∼ N (0, 1)
n

Hence
λ < c ⇒ Z2 > k

or

√
P [λ < c|H0 ] = α ⇒ P [|Z| ≥ k] = α

The test now is to compute

√
n(x̄ − µ0 )
Z=
σ0

and reject H0 when √

|Z| ≥ k
where k is such that

√ √ √
P [|Z| ≥ k] = α ⇒ P [− k ≤ Z ≤ k] = 1 − α

Testing the Variance of Normal Population

Let x1 , x2 , ..., xn be a random sample from a normal variable X.
Test the hypothesis
H0 : σ 2 = σ02 vs Ha : σ 2 ̸= σ02

µ is unknown. The parameter spaces are

Ω = {(µ, σ) : −∞ < µ < ∞, σ > 0}

Ω0 = {(µ, σ) : −∞ < µ < ∞, σ = σ0 }

By Dr. Mutua Kilai 6

 TESTING THE VARIANCE OF NORMAL POPULATION

 1  n2 P
− 2σ12 (xi −µ)2
L(X, µ, σ) = e
2πσ 2
The maximum likelihood estimates under Ω are

n
1X
µ̂ = x̄, σˆ2 = (xi − x̄)2
n i=1
Hence

 1  n2 h n i n2 n
L(Ω̂) = Pn 2
e− 2
2π i=1 (x i − x̄)

Under Ω0 the maximum likelihood estimates are

µ̂ = x̄, σˆ2 = σ02

Hence

 1  n2  1  − 1 P(xi −x̄)2
L(Ωˆ0 ) =
2
e 2σ0
2π 2σ02

Ωˆ0h P(x − x̄) i − 1 P(xi −x̄)2 + n

i 2 2
λ= = 2 e 2σ0
Ω̂ nσ 0

 U  n2 1
λ= e− 2 (U −n)
n
P
(xi −x̄)2
where U = σ02

The test is to compute Pn

i=1 (xi − x̄)2
U=
σ02
and reject H0 when 0 < U < b or U > b where the constants a and b are such that

P [0 < U < a] or U > b] = α

When H0 is true the statistic

Pn
i=1 (xi − x̄)2
U=
σ02
has χ2 distribution with n − 1 degrees of freedom
a and b are chosen such that

α
P [0 < χ2n−1 < a] = = P [χ2(n−1) > b]
2
Consider the sample variance
n
1 X
S2 = (xi − x̄)2
n − 1 i=1

By Dr. Mutua Kilai 7

Example TESTING THE VARIANCE OF NORMAL POPULATION

It is possible to write the variable U in terms of S as

Pn 2
i=1 (xi − x̄) (n − 1) (n − 1)S 2
U= 2 = ∼ χ2(n−1)
σ0 (n − 1) σ02

Example
In a random sample of the weights of 19 babies of certain age. The standard deviation was found to be
2.5kgs. Test the hypothesis that
H0 : σ = 3 vs H1 : σ ̸= 3
at α = 0.05
Solution
The statistic is
(n − 1)S 2 18 × 2.52
U= 2 = = 12.5
σ0 32

The critical value is

χ20.025,18 = 31.53 = b

and

χ20.975,18 = 8.23 = a

U = 12.5 lies in the acceptance region therefore we do not reject H0 at 5% level of significance.

By Dr. Mutua Kilai 8