Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests

Estimadores Extremos
MQO No-Linear (NLS)
Cristine Campos de Xavier Pinto
CEDEPLAR/UFMG
Maio/2010
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
In Nonlinear Least Squares,
E[ Y[ X] = h (X, )
where h is a known function of X and is a Px1 vector of
parameters.
We assume that we know h (., ) up to a set of parameters.
Parameter Space (): subset of R
P
.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Examples:
1 Exponential Regression Function: h (X, ) = exp (X), X is a
row vector with one as the rst element.
2 Logistic Regression: h (X, ) =
exp(X)
1+exp(X)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
We need to assume that we have the correctly specied model
for the conditional mean,
E[ Y[ X] = h (X,
0
)
for some
0
.
In the nonlinear least squares, we are using an additive
unobservable error model with zero conditional mean:
Y = h (X,
0
) + , where E[ [ X] = 0
In this case, depending on the nature of Y, the error may
have some unusual properties.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
NSL1: For some
0
R
P
, E[ Y[ X] = h (X,
0
) .
The data is a random sample of size N from the population,
Z
i
, i = 1, ..., N, Z
i
= (X
i
, Y
i
)
In this case,
0
is the value that solves the population problem
min

E
_
(Y h (X,
0
))
2
_
. .
=Q
0
()
where the expectation is taking over the joint distribution of
(X, Y) .
Using the sample analog, the NLS estimator

maximizes

Q
N
() =
1
N
N

i =1
[Y
i
h (X
i
, )]
2
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
To get identication, we need to assume that
0
is the unique
solution for the population problem.
We need to assume that h (X, ) is not the conditional mean
for ,=
0
.
NSL2: h (X, ) ,= h (X,
0
) if ,=
0
.
Example: Linear Model
h (X, ) = X
/

Conditional mean identication hold is E[X

/
X] is nonsingular,
which implies that
X
/
,= X
/

0
for ,=
0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Theorem
Under assumptions NSL1 and NSL2, and h (X, ) is continuous
over

0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Lets try to get the asymptotic linear representation of the
NLS estimator.
For convenience, lets divide the original objective function by
2,

maximizes

Q
N
() =
1
N
N

i =1
[Y
i
h (X
i
, )]
2
2
The FOC condition of this problem is:
s
N
_
Z
i
,

_
=
1
N
N

i =1
\

h
_
X
i
,

_
/
_
Y
i
h
_
X
i
,

__
= 0
where \

h
_
X
i
.

_
is the 1xP gradient of h (X
i
.) .
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Doing the mean value expansion
1
_
N
N

i =1
\

h
_
X
i
,

_
/
_
Y
i
h
_
X
i
,

__
=
1
_
N
N

i =1
\

h (X
i
,
0
)
/
[Y
i
h (X
i
,
0
)] +
1
N
N

i =1
_
\
2

/ h
_
X
i
,

_
/
_
Y
i
h
_
X
i
,

__
\

h
_
X
i
,

_
/
\

h
_
X
i
,

_
_

_
N
_

0
_
So we can write
_
N
_

0
_
= H
1
0

_

1
_
N
N

i =1
\

h (X
i
,
0
)
/
[Y
i
h (X
i
,
0
)]
_
+o
p
(1)
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Note that
H
0
=
E
_
\
2

/ h (X
i
,
0
)
/
[Y
i
h (X
i
,
0
)] \

h (X
i
,
0
)
/
\

h (X
i
,
0
)

and
E
_
\
2

/ h (X
i
,
0
)
/
[Y
i
h (X
i
,
0
)] \

h (X
i
,
0
)
/
\

h (X
i
,
0
)

= \
2

/ h (X
i
,
0
)
/
E[ Y
i
h (X
i
,
0
)[ X]
\

h (X
i
,
0
)
/
\

h (X
i
,
0
)
= \

h (X
i
,
0
)
/
\

h (X
i
,
0
)
so
H
0
= E
_
\

h (X
i
,
0
)
/
\

h (X
i
,
0
)

When
0
is identied, E
_
\

h (X
i
,
0
)
/
\

h (X
i
,
0
)

is
positive semidenite
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Lets show that the population moment analog to this
expression has the expected value equals to 0 at =
0
,
E[s (Z
i
,
0
)] = E[E[ s (Z
i
,
0
)[ X]]
E[ s (Z
i
,
0
)[ X] = E
_
\

h (X,
0
)
/
[Y h (X,
0
)]

= \

h (X,
0
)
/
[E[ Y[ X] h (X,
0
)]
= 0
so
E[s (Z
i
,
0
)] = 0
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Note that
Var [s (Z
i
,
0
)] = E
_
s (Z
i
,
0
) s (Z
i
,
0
)
/

= E
_
(Y h (X,
0
))
2
\

h (X,
0
)
/
\

h (X,
0
)
_
= E
_

2
\

h (X,
0
)
/
\

h (X,
0
)

Cristine Campos de Xavier Pinto Institute

Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
The asymptotic linear representation of

is
_
N
_

0
_
=
1
_
N
N

i =1
H
1
0
s (Z
i
,
0
) +o
p
(1)
where E[s (Z
i
,
0
)] = 0, and E
_
s (Z
i
,
0
) s (Z
i
,
0
)
/

<
Applying the CLT,
_
N
_

0
_

d
A
_
0, A
1
0
A
1
0
_
where
A
0
= E
_
\

h (X,
0
)
/
\

h (X,
0
)

= E
_

2
\

h (X,
0
)
/
\

h (X,
0
)

Cristine Campos de Xavier Pinto Institute

Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
The estimator of A
0
is

A =
N

i =1
\

h
_
X
i
,

_
/
\

h
_
X
i
,

_
To estimate , we use the nonlinear least squares residuals

i
= Y
i
h
_
X
i
,

=
N

i =1

2
i
\

h
_
X
i
,

_
/
\

h
_
X
i
,

_
This is the heteroskedasticity-robust variance estimator
for NLS.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Suppose we assume homoskedasticity (NLS3):
Var [ [ X] =
2
In this case,
=
2
A
0
and
_
N
_

0
_

d
A
_
0,
2
A
1
0
_
and

V =
_

N
i =1

2
i
N P
_

_
N

i =1
\

h
_
X
i
,

_
/
\

h
_
X
i
,

_
_
1
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
The LM and QLR test, under homoskedasticity, for NLS
estimator are very familiar.
We want to test Q restrictions:
H
0
: c (
0
) = 0
To get the LM statistics under homoskedasticity, we need to
obtain
2
=
1
N

N
i =1

2
i
, where
i
= Y
i
m
_
X
i
,

_
and

is
the constrained estimator of
0
.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
The LM statistics under homoskedasticity is
LM =
1

2

_
N

i =1
\

h
_
X
i
,

i
_
/

_
N

i =1
\

h
_
X
i
,

_
/
\

h
_
X
i
,

_
_
1

_
N

i =1
\

h
_
X
i
,

i
_
This repression is equal to N times the R
2
of a regression of
i
on \

h
_
X
i
,

_
.
Under NS1-NSL3, LM
d
A
2
Q
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
Recall that
QLR = 2
_
N

i =1
q
_
Z
i
,

i =1
q
_
Z
i
,

_
_
In the case that
2
,= 1, we need to divide this expression by

2
.
Under assumptions NS1-NSL3,
QLR =
SSR
r
SSRur
SSR
ur
(N P) ~ A
2
Q
To get the classical F test, we just need to divide by Q.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos
Identication and Consistency Asymptotic Normality Estimating the Asymptotic Variance Tests
References
Amemya: 4
Wooldridge, 12
Newey, W. and D. McFadden (1994). "Large Sample
Estimation and Hypothesis Testing", Handbook of
Econometrics, Volume IV, chapter 36.
Cristine Campos de Xavier Pinto Institute
Estimadores Extremos