Chapter 4

Applied Econometrics 4th edition
Dimitrios Asteriou
and
Stephen G Hall
MULTIPLE REGRESSION
1. The Multiple Regression Model

2. The OLS Method of Estimation
3. The R2 and the Adjusted R2
4. Hypothesis Testing
5. How to Estimate a Simple Regression in EViews
Learning Objectives
• Derive mathematically the regression coefficients of a
multiple regression model.
• Understand the difference between the R2 and the adjusted
R2 for a multiple regression model.
• Appreciate the importance of the various selection criteria
for the best regression model.
• Conduct hypothesis testing and test linear restrictions,
omitted and redundant variables as well as the overall
significance of the explanatory variables.
Learning Objectives (2)
• Obtain the output of a multiple regression

estimation using econometric software.
• Interpret and discuss the results of a multiple
regression estimation output.
Multiple Regression Derivation of

the OLS
• The three variables case (explain on board)
• The k-variables case (explain on board)
– Requires matrix algebra and it is quite complicated
– Luckily Eviews, Mfit and Stata give results very
quickly and efficiently (always correct
calculations)
Assumptions of the Multiple

Regression Model
R2 and adjusted R2
• R2 measures goodness of fit as in Simple Regression
• However, it cannot be used for comparing two
different equations containing different
• numbers of explanatory variables.
• When adding more explanatory variables R2, will
always be increased.
• Therefore we need a different measure
(the adjusted R2)
R2 and adjusted R2
• R2 = ESS/TSS = 1 − RSS/TSS
• Adj R = 1  RSS /( n  k ) RSS ( n  1)

2
 1
TSS /( n  1) TSS (n  k )
• Similar to R2 but adjusts for degrees of

freedom
Criteria for Model Selection

• Akaike Information Criterion (AIC)
• Finite Prediction Error (FPE)
• Schwarz Bayesian Criterion (SBC)
• Hannan and Quin Criterion (HQC)

Multiple Regression in EViews

• Step 1 Open EViews.
• Step 2 Click File/New/Workfile in order to
create a new file or File/Open to open an existing
file.
• Step 3 Enter the data
• Step 4 Type in the EViews command line:
ls y c x2 x3 . . . xk (press ‘enter’)
Multiple Regression in EViews

Hypothesis Testing
• Testing Individual Coefficients (t-tests)
• Testing for Linear Restrictions (Wald Test)
– Cobb Douglas Production Function
• Testing for the Overall Significance (F-test)
• Testing for Omitted Variables (Wald Test)
• Testing for Redundant Variables (Wald Test)
– Explain all the tests on board…
Hypothesis Testing
• Testing Individual Coefficients (t-tests)
• Same as before (simple regression)

Hypothesis Testing
– Cobb Douglas Production Function (α+β=1)
Hypothesis Testing
– Impose more than one restriction
Hypothesis Testing
• Testing for the Overall Significance (F-test)
Testing Multiple Hypotheses: The F-test

• We used the t-test to test single hypotheses, i.e. hypotheses involving only
one coefficient. But what if we want to test more than one coefficient
simultaneously?
• We do this using the F-test. The F-test involves estimating 2 regressions.
• The unrestricted regression is the one in which the coefficients are freely
determined by the data, as we have done before.
• The restricted regression is the one in which the coefficients are restricted,
i.e. the restrictions are imposed on some s.
The F-test: Restricted and Unrestricted Regressions
• Example
The general regression is
yt = 1 + 2x2t + 3x3t + 4x4t + ut (1)
• We want to test the restriction that 3+4 = 1 (we have some hypothesis from
theory which suggests that this would be an interesting hypothesis to study).
The unrestricted regression is (1) above, but what is the restricted regression?
yt = 1 + 2x2t + 3x3t + 4x4t + ut s.t. 3+4 = 1
• We substitute the restriction (3+4 = 1) into the regression so that it is

automatically imposed on the data.
3+4 = 1  4 = 1- 3
The F-test: Forming the Restricted Regression

yt = 1 + 2x2t + 3x3t + (1-3)x4t + ut
yt = 1 + 2x2t + 3x3t + x4t - 3x4t + ut
• Gather terms in ’s together and rearrange

(yt - x4t) = 1 + 2x2t + 3(x3t - x4t) + ut
• This is the restricted regression. We actually estimate it by creating two new

variables, call them, say, Pt and Qt.
Pt = yt - x4t
Qt = x3t - x4t
so
Pt = 1 + 2x2t + 3Qt + ut
is the restricted regression we actually estimate.
Calculating the F-Test Statistic

• The test statistic is given by
RRSS  URSS T  k
test statistic  
URSS m
where URSS = RSS from unrestricted regression
RRSS = RSS from restricted regression
m = number of restrictions
T = number of observations
k = number of regressors in unrestricted regression including
a constant in the unrestricted regression (or the total number of parameters to
be estimated).
The F-Distribution
• The test statistic follows the F-distribution, which has 2 d.f. parameters.
• The value of the degrees of freedom parameters are m and (T-k)

respectively (the order of the d.f. parameters is important).
• The appropriate critical value will be in column m, row (T-k).
• The F-distribution has only positive values and is not symmetrical. We

therefore only reject the null if the test statistic > critical F-value.
Determining the Number of Restrictions in an F-test

• Examples :
H0: hypothesis No. of restrictions, m
1 + 2 = 2 1
2 = 1 and 3 = -1 2
2 = 0, 3 = 0 and 4 = 0 3
• If the model is yt = 1 + 2x2t + 3x3t + 4x4t + ut,
then the null hypothesis
H0: 2 = 0, and 3 = 0 and 4 = 0 is tested by the regression F-statistic. It tests
the null hypothesis that all of the coefficients except the intercept coefficient
are zero.
• Note the form of the alternative hypothesis for all tests when more than one
restriction is involved: H1: 2  0, or 3  0 or 4  0
What we Cannot Test with Either an F or a t-test
• We cannot test using this framework hypotheses which are not linear
or which are multiplicative, e.g.
H0: 2 3 = 2 or H0: 2 2 = 1
cannot be tested.

Chapter 4

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 4

Uploaded by

Copyright:

Available Formats

Applied Econometrics 4th edition

1. The Multiple Regression Model

Learning Objectives (2)

• Obtain the output of a multiple regression

Multiple Regression Derivation of

Assumptions of the Multiple

• Adj R = 1  RSS /( n  k ) RSS ( n  1)

• Similar to R2 but adjusts for degrees of

Criteria for Model Selection

• Finite Prediction Error (FPE)

• Schwarz Bayesian Criterion (SBC)

• Hannan and Quin Criterion (HQC)

Multiple Regression in EViews

Multiple Regression in EViews

• Same as before (simple regression)

Testing Multiple Hypotheses: The F-test

• We do this using the F-test. The F-test involves estimating 2 regressions.

The F-test: Restricted and Unrestricted Regressions

• We substitute the restriction (3+4 = 1) into the regression so that it is

The F-test: Forming the Restricted Regression

• Gather terms in ’s together and rearrange

• This is the restricted regression. We actually estimate it by creating two new

Calculating the F-Test Statistic

• The value of the degrees of freedom parameters are m and (T-k)

• The appropriate critical value will be in column m, row (T-k).

• The F-distribution has only positive values and is not symmetrical. We

Determining the Number of Restrictions in an F-test

What we Cannot Test with Either an F or a t-test

You might also like