Professional Documents
Culture Documents
Extensions
Extensions
Extensions
REGRESSION MODEL
Christophe Croux
Katholieke Universiteit Leuven
christophe.croux@econ.kuleuven.ac.be
1
Session 1: The General Linear Model
Σ := Cov(ε) = σ In = .. . . .
2 .. .
0 ... σ 2
3
II The Generalized Least Squares (GLS) estimator
Properties of β̂GLS
• Unbiasedness: E[β̂GLS] = β
• Cov(β̂GLS) = (X 0Σ−1X)−1
• β̂GLS is consistent and efficient at the normal
model
4
III The Feasible GLS estimator
1
. ..
.
...
0 2
. . . σN
with log(σi2) = Zi0γ, and Zi an exogenous vari-
able. This implies that Σ depends now only on
one parameter.
5
We can work with a 2-step procedure:
• Step 1: Compute β̂OLS and the residuals ri =
Yi −Xi0β̂OLS. Regress log(ri2) on Zi to obtain
γ̂.
• Step 2: Compute σ̂i2 = exp(Ziγ̂) → Σ̂ →
β̂GLS
In principle, on could redo the 2 steps, now start-
ing with the obtained feasible GLS-estimator. As
such, we obtain an iterative procedure.
model by specifying the form of the volatility σi2. Many people find this
approach too restrictive and prefer in this case an OLS with White-
6
IV Exercise
(a) Estimate the variances σ12 and σ22 using the OLS-residuals.
7
(b) Estimate now the parameters by GLS. Write down an expres-
sion for Σ, the covariance matrix of the error terms, and show
that GLS boils down to Weighted Least Squares (WLS) here.
Create the series of weights to be used, and carry out the WLS
estimation (in Eviews, take estimation method LS with option
Weighted LS).
(c) What is the advantage of a WLS instead of OLS.
(a) Estimate the above model by OLS. Do you think there might
be interaction? (Hint: creating the variables STORE1*CARD
and STORE2*CARD might be useful)
(b) Test whether the interaction is significant or not.
8
Session 2: The Panel Data Model
I Introduction
Rgdplit=ai+b1*Popit+b2*Iit+b3*Git+b4*XRit
+b5*Openit+errorit
Write
Yit = Xit0 β + εit
and with vit a white noise with mean zero and
variance σv2
εit = αi + vit.
The error term εit contains a permanent compo-
nent (the random effect) αi and a transitory com-
ponent vit. Both components are supposed to be
17
independent and αi has mean zero and variance
σα2 .
20
We discuss now the most important cases where
the endogeneity problem arises:
21
Measurement error
The regressor is observed with a measurement error. Call
X ∗ the true value, and X the observed value of the exoge-
nous variable:
∗
Y = βX + ε1
X = X ∗ + ε2
Y = βX − βε2 + ε1 := βX + ε
22
II Instrumental variables
23
One could even have more IV for Y2; let Z1, . . . , Zk
be the list of instrumental variables. Estimation
of γ1 is then done by a two-stage least squares
approach:
1. Regress Y2 on the instrumental variables, giv-
ing the adjusted series
Ŷ2 = c + a1Z1 + . . . + ak Zk .
Note that Ŷ2 is itself an instrumental variable.
It can be seen as the optimal IV formed by a
linear combination of Z1, . . . , Zk .
2. Regress Y1 on X1 and Ŷ2, giving us
β̂1X1 + γ̂1Ŷ2.
The IV-estimators β̂1 and γ̂1 are consistent
estimators.
24
How to find instrumental variables?
Most of the time, it is very difficult to find good
IV. Two exceptions:
• One can always try lagged version of the vari-
able to instrument: Zt = Y2,t−1 or Zt =
Y2,t−2 (or both).
• In a system of equations, exogenous variables
from other equations can be taken.
25
III Exercise
3. Make a scatterplot
27
Session 4: Estimation of Systems of Equa-
tions
I Introduction
28
Y1 = a11X1 + a12X2 + . . . + a1pXp + b12Y2 + . . . + b1N YN + ε1
Y2 = a21X1 + a22X2 + . . . + a2pXp + b21Y1 + . . . + b2N YN + ε2
...
YN = aN 1X1 + aN 2X2 + . . . + aN pXp + bN 1Y1 + . . . + bN N YN + εN
29
If the coefficients of the structural form can be re-
trieved from the coefficients of the reduced form,
then the system is identified. If
• πij → aij , bij uniquely: exactly identified
• πij → aij , bij in multiple ways: overidentified
Restrictions need to be put on the coefficients
before a system becomes identifiable.
30
II Systems of regression equations
31
and the matrix
0
X11 0 ... 0
... 0 ... 0
0
X1T 0 ... 0
0
X̃ =
0 X21 ... 0
0 ... ... 0
0
0 X2T ... 0
... ... ... ...
The firms have labels GM, CH, US, GE, and WH.
1. Open the file ”grunfeld.wf1” and create a new object of the “system”-
type. Specify the system by typing
invgm=c(1)+c(2)*valgm+c(3)*capgm
invch=c(4)+c(5)*valch+c(6)*capch
invge=c(7)+c(8)*valge+c(9)*capge
invwh=c(10)+c(11)*valwh+c(12)*capwh
invus=c(13)+c(14)*valus+c(15)*capus
33
4. Estimate now using the SURE method. Which coefficients are
significant? Is your answer different if you do not iterate the GLS-
estimator? (Tip: you can use “freeze” to keep a table on the
screen.)
5. Test whether the coefficients for market value are equal across
firms.
7. It seems that many of the variables in the model are trending up-
wards. Is this a problem or not? Do the residuals look stationary?
34
III Simultaneous-Equations Models
For equation (3): Use X2 and X3 as instruments for Y1 and Y2. Take
X1 as instrument for itself. We have a list of 3 instruments for 3 RHS
variables. The equation is exactly identified.
36
Exercice: Estimation of the Klein model I
Consider the following model of the American (pre-war) economy:
CSt = α0 + α1Pt + α2Pt−1 + α3(W Gt + W Pt) + ε1 consumption
It = β0 + β1Pt + β2Pt−1 + β3Kt−1 + ε2 investment
W Pt = γ0 + γ1Xt + γ2Xt−1 + γ3T IM Et + ε3 private wages
with Xt (demand), Pt (private profits), Kt capital stock endogenous
and T (indirect taxes plus net exports), W G (government wages) ex-
ogenous.
1. Read in the data from “klein.wf1” and create a system object. Add
a list of instrumental variables that you can use:
cs=c(1)+c(2)*p+c(3)*p(-1)+c(4)*(wg+wp)
i=c(6)+c(7)*p+c(8)*p(-1)+c(9)*k1
wp=c(10)+c(11)*x+c(12)*x(-1)+c(13)*year
inst x(-1) k1 wg year p(-1)
T J ∼ χ2q .
38
Three-stage least squares (3SLS)
39
Step 2+3: the above system has no endogeneity
problem anymore. We will estimate it by SURE,
which is a 2-step GLS-estimation method.
40