In the Name of God

April 10, 2021

1 PS#4 - Solution
1.1 Q1
(i) β1 < 0: Higher pollution decreases house value (price). β1 is the elasticity of price with
respect to nox. β2 may be positive as hous size may be correlated with number of rooms. As
for β0 , if we take rooms = 0 and nox = 1, then the expected house price will be eβ0 , taking
all other things invariable. And for β2 we can say how many percent increase in price should
we expect for every room added to the house, taking all other things invariable.
(ii) Number of rooms may be correlated with polution such that high quality homes have more
rooms and are in less polluted areas. So taking that as a fact, in the simple regression of
log(price) on log(nox), β1 will have downward bias (considering β2 > 0 and corr(x1 , x2 ) < 0).
In other words, because beta1 is negative, it’s effect will be overstated.
(iii) Yes, we had expected this in part (ii). The results given are only for one sample so we cannot
say that the second model has the more accurate coefficient estimated. But taking the sample
as typical, the coefficient β1 is probably closer to the second one.

1.2 Q2
From OLS we have:
X
n
xi1 (yi − β̂0 − β̂1 xi1 − β̂2 xi2 ) = 0
i=1

X
n
(x̂i1 + r̂i1 )(yi − β̂0 − β̂1 xi1 − β̂2 xi2 ) = 0
i=1

x̂i1 is orthogonal to the second phrase (second paranthesis) and therefore its multiplication is zero
for all i. Those orthogonal to r̂i1 have also zero product with it. What remains is:

X
n X
n
r̂i1 (yi − β̂1 xi1 ) = r̂i1 (yi − β̂1 (x̂i1 + r̂i1 )) = 0
i=1 i=1

X
n
r̂il (yi − β̂1 r̂i1 ) = 0
i=1
Pn
r̂i1 yi
β̂1 = Pi=1
n 2
i=1 r̂i1

1
 Pn
i=1 r̂i1 (β0
+ β1 xi1 + β2 xi2 + β3 xi3 + ui )
Pn 2
β̂1 =
i=1 r̂i1
Pn Pn
r̂i1 xi3 r̂i1 ui
β̂1 = β1 + β3 Pn 2 + Pi=1
i=1
n 2
i=1 r̂i1 i=1 r̂i1
∑n
r̂ u
The last expression is zero ( ∑i=1 n
i1 i
2 = 0) because it is equal to the coefficient of r̂i1 when we
i=1 r̂i1
regress r̂i1 on ui . We also know that r̂i1 = xi1 − x̂i1 and x̂i1 is made of xi2 . So both xi1 and x̂i1
have zero correlation with u∑ i and therefore the coefficient of of r̂i1 when we regress r̂i1 on ui is zero;
n
r̂i1 ui
the last expression is zero ( ∑i=1
n
r̂2
= 0).
i=1 i1
Pn
i=1 r̂i1 xi3
β̂1 = β1 + β3 P n 2
i=1 r̂i1

1.3 Q3
(i) Degress of freedom for the first regression: n − k − 1 = 353 − 1 − 1 = 351 and for the second
one: n − k − 1 = 353 − 2 − 1 = 350. Another explanatory variable has been included in the
regression so the standard error (SER) and SSR have been decreased.
(ii) Apparently there is a positive correlation between year and rbisyr. The Variance Inflation
Factor: V IFyears = V IFrbisyr = 1−R12 = 1−R12 1
= 1−0.487 = 1.95. So there is little to
years rbisyr
moderate collinearity between years and rbisyr.
(iii) The standard error of the coefficient decreases because we have added another explanatory
variable. If the correlation between the newly added variable and years had been higher, the
standard error of the coefficient of years may even not have changed or decreased. (EXTRA
MATERIAL: To understand why, take the following two regressions as an example:

yi = β0 + β1 X1 + β2 X2 + ϵi

yi = α0 + α1 X1 + ui
√ √
With homoscedasticity, we can show that n(β̂ − β) and n(α̂ − α) have convergence to the
following distributions:
√ E(e2 )
n(β̂ − β) : N (0, )
V ar(X1 )(1 − RX
2
1 ,X2
)

√ E(u2 )
n(α̂ − α) : N (0, )
V ar(X1 )
So the fraction of standard error of β1 in the multiple regression and the standard error of α1
in the simple regression will be:
v
u
se(β1 ) u 1 − RY,X
2
t 1 ,X2 1
= .
se(α1 ) 1 − RY,X1 1 − RX
2 2
1 ,X2

2
with RY,X being the R2 for the multiple regression model, RY,X2 being the R2 for the
1 ,X2 1
2
simple regression model, and RX1 ,X2 being the correlation between X1 and X2 . We also
know that adding more variables to a regression will either not change the R2 of the regression
model or increase it. So when we add the explanatory variable rbisyr to the simple model, R2

2
 of the model most likely increases and therefore, based on the ratio we derived, the standard
2
error of years will decrease ((1−RY,X 2
) < (1−RY,X )). Also, we have may have correlation
1 ,X2 1
between years and rbisyr that will reduce the denominator of the ratio (1 > (1 − RX 2
1 ,X2
)).
The aggregate effect of the two explained channels will determine whether the standard error
of years will decrease or decrease when we add rbisyr to the model. You can explain similairly
when we eliminate an explanatory variable from a multiple regression model.)

1.4 Q4
The Frisch-Waugh-Lovell Theorem:

Xβ + ϵ = X1 β1 + X2 β2 + ϵ = y

X1′ X1 β1 + X1′ X2 β2 = X1′ y

β1 = (X1′ X1 )−1 X1′ y − (X1′ X1 )−1 X1′ X2 β2 = (X1′ X1 )−1 X1′ (y − X2 β2 )
X2′ X1 β1 + X2′ X2 β2 = X2′ y
X2′ X1 (X1′ X1 )−1 X1′ y − X2′ X1 (X1′ X1 )−1 X1′ X2 β2 + X2′ X2 β2 = X2′ y
β2 = [X2′ (I − X1 (X1′ X1 )−1 X1′ )X2 ]−1 [X2′ (I − X1 (X1′ X1 )−1 X1′ )y]
M1 = I − X1 (X1′ X1 )−1 X1′
β2 = (X2′ M1 X2 )−1 (X2′ M1 y)
Recap:
Xβ + ϵ = y
X ′ Xβ = X ′ y
β = (X ′ X)−1 X ′ y
ϵ = y − Xβ = y − X(X ′ X)−1 X ′ y = (I − X(X ′ X)−1 X ′ )y = M y
We know:
M′ = M
M.M = M
So we have:
β2 = ((M1 X2 )′ M1 X2 )−1 ((M1 X2 )M1 y)
′ ′
β2 = (X2∗ X2∗ )−1 X2∗ y ∗
And thats why the coefficients in a multiple regression are often called the partial regression coef-
ficients.

1.5 Q5
We expect experience and education increase wage on avergae. Also we expect experience and
education to be negatively correlated with each other. So the coefficient of education in the simple
regression of wage on education may have downward bias. The coefficient of education is positive
so the bias may underestimate the effect of education. Now, if we take the coefficient being biased as
a fact, β̂1 will not be consistent if neither the correlations of wage and experience and of education
and experience are zero (if at least one of the two correlations are non-zero, β̂1 in the new simple
regression is definitely inconsistent).

3
 1.6 Q6
For the coefficients we know:
β̂ = (X ′ X)−1 X ′ y
For the residual variances:
û = M y = (I − X(X ′ X)−1 X ′ )y
u′ u = y ′ M ′ M y = y ′ M y = y ′ (I − X(X ′ X)−1 X ′ )y = y ′ y − y ′ X β̂ = SSR
And for R2 we have:
SSR
R2 = 1 −
SST
1 ′ 1
SST = y ′ (I − 11 )y = y ′ (I − J)y
n n
1
Taking n as 0, we will have:
SSR
R2 = 1 −
y′y

1.7 Q7
Again we use M :

ûZ = MZ y = MXP y = (I−XP (P ′ X ′ XP )−1 P ′ X ′ )y = (I−XP P −1 (X ′ X)−1 P ′−1 P ′ X ′ )y = (I−X(X ′ X)−1 X ′ )y = ûX

P can be diagonal and act as a scale function.

1.8 Q8
Again, it solves the same way:
i
x∗ = x − i(i′ i)−1 i′ x = x − i(n)−1 i′ x = x − i( )′ x = x − ix̄ = M 0 x
n
′ ′
β = (X ∗ X ∗ )−1 X ∗ y ∗