Econometrı́a I

Regresión lineal simple

UPB

Semestre II - 2022

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 1 / 17

 Introducción

Regresión lineal simple

Ecuación lineal
y = a + bx

Regresión lineal simple

y = β0 + β1 x + u

y , variable dependiente (variable explicada)

x, variable Independiente (variable explicativa, covariate)
β0 + β1 , constantes
u, error

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 2 / 17

 Introducción

Supuestos, u

Mean of (random) error is 0

Variable explicativa y error no pueden estar correlacionados
No afecta a la pendiente, E (u) = α0
Zero condicional mean, E (u|x)
Cov (x, u) = 0

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 3 / 17

 OLS

Ordinary least squares (OLS)

y = β0 + β1 x + u

Estimar parámetros para una muestra

(xi , yi ), i = 1, ..., n
yi = β0 + β1 xi + ui

E (y |x)

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 4 / 17

 OLS

Regression line

Sample data points

Associated error terms
(xi , yi ), i = 1, ..., n

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 5 / 17

 OLS

Derivación estimadores OLS

E (u) = 0
Cov (x, u) = 0
Math
1 Pn
n i=1 (x) = x̄
n
xi (xi − x̄) = Pni=1 (xi − x̄)2
P P
Pi=1
n n
i=1 xi (yi − ȳ ) = i=1 (xi − x̄)(yi − ȳ )
Moment restriction

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 6 / 17

 OLS

Slope
Pn
(x − x̄)(yi − ȳ )
βˆ1 = Pn i
i=1
2
i=1 (xi − x̄)

Estimate the sample covariance between x and y divided by the

sample variance of x
The correlation determines the slope
OLS fits a line through the sample points such that the sum of
squared residuals is as small as possible, hence the term least squares
Estimate of the error, û = yi − yˆi

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 7 / 17

 OLS

Properties of OLS

Residual error is different from the error

OLS residuals add up to 0

Sample covariance (correlation) between regressor and residual is 0

Sample average is the same as the sample average of the fitted values

Sample covariance of fitted values and residuals is zero

The OLS regression line always goes through the mean of the sample

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 8 / 17

 OLS

Analysis of residuals

yi = yˆi + ûi

(yi − ȳ )2
P
Total sum of squares (SST),
Explained sum of squares (SSE), (ŷi − ȳ )2
P
P 2
Residual sum of squares (SSR), ûi
Proof:

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 9 / 17

 OLS

Goodness-of-Fit

SST = SSE + SSR

R 2 is the fraction of the total variation in yi that is explained by xi

0 ≤ R2 ≤ 1
R2 = 0
R2 = 1
Having a high R 2 is neither necessary nor sufficient to infer causality.

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 10 / 17

 OLS

Units of measurement and functional form

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 11 / 17

 OLS

Unbiasedness assumptions

Required assumptions
Linear parameters

Random sampling

Sample variation

Zero conditional mean

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 12 / 17

 OLS

Unbiasedness of OLS estimators

Pn Pn
Pni=1 (x i − x̄)(yi − ȳ ) = i=1 (xi − x̄)yi
Pi=1 (xi − x̄) = 0 P
n n 2
i=1 xi (xi − x̄) = i=1 (xi − x̄)
OLS unbiased estimators proof

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 13 / 17

 OLS

Variance of OLS estimator

Now we know that the sampling distribution of our estimate is

centered around the true parameter
Variance under an additional assumption
Homoskedasticity
Var (u|x) = σ 2
Var (u|x) = E (u 2 |x) − [E (u|x)]2 = σ 2

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 14 / 17

 OLS

Variance of OLS estimator

Homoskedasticity

Heteroskedastic

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 15 / 17

 OLS

Variance

1 2X 1 2X 2
Var (βˆ1 ) = ( ) di Var (ui ) = σ 2 ( ) di
SSTx SSTx
1 2 σ2
= σ2( ) SSTx = = Var (βˆ1 )
SSTx SSTx

The larger the error variance, σ 2 , the larger the variance of the slope
estimate
The larger the variability in the xi , the smaller the variance of the
slope estimate
A larger sample size should decrease the variance of the slope estimate

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 16 / 17

 OLS

Estimating variance error

ûi = yi − βˆ0 − βˆ1 xi

An unbiased estimator
P
ûi 2 SSR
σˆ2 = =
(n − 2) n−2

Standard error q
σ̂ = σˆ2

Regresión lineal simple (UPB) Econometrı́a I Semestre II - 2022 17 / 17