2015/10/15

Detection & Estimation

Lecture 2
MVUE

Xiliang Luo

MVUE
• Minimize the variance while being unbiased
• Question: whether MVUE exists?
• unbiased estimator with minimum variance for all values
of the unknown parameter
• Example: [Example 2.3]
,1 , 0
0 ∼ ,1 1 ∼
,2 , 0

1
2 0 1
3

1
0 1
2

1
 2015/10/15

MVUE
• No known “turn‐the‐crank” procedure to produce
the MVUE
• Next, we will discuss
• Cramer‐Rao lower bound
• Rao‐Blackwell‐Lehmann‐Scheffe theorem
• best linear estimator

Efficient Estimator
• An unbiased estimator attaining the CRLB is
efficient
• efficiently uses the observed data
• CRLB can be evaluated always
• efficient estimator may not exist
• we may still find the MVUE when MVUE exits
• sufficient statistics
• Rao‐Blackwell‐Lehmann‐Scheffe

2
 2015/10/15

Sufficient Statistics
• DC level in noise
•
• ∑
• 0
• Both are unbiased estimators
• Question:
• Which data samples are pertinent to the estimation?
• Is there a set of data which is sufficient?

Sufficient Statistics
• DC level in noise
•
• Data set 1:

• Data set 2:

• Data set 3: minimal sufficient statistic

The data set that contains the least number of elements is called the minimal set!

3
 2015/10/15

Sufficient Statistics
• DC level in noise
•
• PDF of the observed data:
• ; exp ∑

• Assume the following statistic has been observed:

• ∑
• Then, we have the posteriori PDF
• | ;

Sufficient Statistics
• DC level in noise
•

• Since is sufficient, this conditional PDF should

not depend on A anymore!
• ;

4
 2015/10/15

Sufficient Statistics

Sufficient Statistics
• Example:
• verify ∑ is a sufficient statistic for
the DC level in noise:
•

, ; ;

5
 2015/10/15

Neyman‐Fisher Factorization
• Theorem: the PDF ; can be factorized as
• ; , ⋅
iff is a sufficient statistic for .

• Generalization: the PDF ; can be factorized

as
• ; ,…, , ⋅
iff ,…, are jointly sufficient statistics
• | ,…, ; does not depend on .

Neyman‐Fisher Factorization
• Example: DC level in WGN

1 1
; exp
2
2

1 1 1
exp 2 exp
2 2
2

6
 2015/10/15

Neyman‐Fisher Factorization
• Example: Power of WGN

1 1
; exp ⋅1
2
2

Proof of NFF Theorem

1. , ; ;

2. If , the pdf of y can be written as:

2 useful results!

7
 2015/10/15

Proof of NFF Theorem

• First, we prove if the factorization holds, then T(x) is
a sufficient statistic.
;

• Second, we prove if T(x) is a sufficient statistic, then

the factorization holds.
;

Find MVUE
• Example: DC Level in WGN
• Sufficient statistic: ∑
• Two ways to find the MVUE
• 1. 0, |
• 2. Find function such that is an unbiased
estimator for A

X, Y are jointly Gaussian with mean: , , and covariance matrix:

, , , we have:

8
 2015/10/15

RBLS Theorem
• Rao‐Blackwell‐Lehmann‐Scheffe
• If is an unbiased estimator of and is a
sufficient statistic for , then is
• a valid estimator
• unbiased
• of lesser or equal variance for all value
• additionally, if the sufficient statistic is complete,
then is the MVUE!
• a statistic is complete if there is only one function of the
statistic that is unbiased

RBLS Theorem  MVUE

9
 2015/10/15

Completeness
• A sufficient statistic is complete if the following
condition:
; 0, ∀

• is satisfied only by the zero function for by

0∀ .

Completeness
• Example: DC level in WGN
• ∑ is complete
• only one function such that
• assume another function such that
• 0, ∀ , ≔
• ∼ ,
• we can verify 0
• key: satisfy the condition for all A

10
 2015/10/15

Completeness
• Example: Incomplete
• 0 0
• 0 ∼ ,
• a sufficient statistic is 0 , which is the only
available data
• check the following condition

; 0, ∀

sin 2

Procedure to find MVUE

Use Neyman‐Fisher factorization
theorem to find sufficient statistic

Determine if the statistic is complete

Find function of the statistic that is

unbiased

MVUE

11
 2015/10/15

Proof of RBLS
• 1. a valid estimator
• sufficient statistic  new estimator is a function of
observation only
• 2. unbiased estimator
• property of conditional mean

Proof of RBLS
• 3. lower variance

+2 +

12
 2015/10/15

Mean of Uniform Noise

• Observation:
• , 0, … , 1
• is IID noise, uniformly distributed over 0,
• We want to estimate the mean:
• Option 1: Try to find the CRLB
• Option 2: Neyman‐Fisher factorization and RBLS
theorem

Mean of Uniform Noise

1
; max ⋅ min

A sufficient statistic: max

0, 0
1
,0

0,

13
 2015/10/15

Mean of Uniform Noise

1
• Sample mean: 12

1
• MVUE: 2
max
4 2

-1
10
sample mean
MVUE

-2
10

-3
10
1 2 3 4 5 6 7 8 9 10

HW
• Week 3:
• Chapter 5: 5.3, 5.4, 5.5, 5.14, 5.15, 5.16, 5.18

14