Lecturing 04

THE LINEAR REGRESSION MODEL:

AN OVERVIEW

Damodar Gujarati
Econometrics by Example
 THE LINEAR REGRESSION MODEL (LPM)
The general form of the LPM model is:
Yi = B1 + B2X2i + B3X3i + + BkXki + ui

Or, as written in short form:

Yi = BX + ui

Y is the regressand, X is a vector of regressors,

and u is an error term.

Damodar Gujarati
Econometrics by Example
 POPULATION (TRUE) MODEL

Yi = B1 + B2X2i + B3X3i + + BkXki + ui

This equation is known as the population or

true model.
It consists of two components:
(1) A deterministic component, BX (the conditional
mean of Y, or E(Y|X)).
(2) A nonsystematic, or random component, ui.

Damodar Gujarati
Econometrics by Example
 REGRESSION COEFFICIENTS

B1 is the intercept
B2 to Bk are the slope coefficients
Collectively, they are the regression coefficients
or regression parameters
Each slope coefficient measures the (partial) rate
of change in the mean value of Y for a unit change
in the value of a regressor, ceteris paribus

Damodar Gujarati
Econometrics by Example
 SAMPLE REGRESSION FUNCTION
The sample counterpart is:
Yi = b1 + b2X2i + b3X3i + + bkXki + ei
Or, as written in short form:
Yi = bX + ei
where e is a residual.
The deterministic component is written as:

Yi b1 b2 X 2i b3 X 3i ... bk X ki bX

Damodar Gujarati
Econometrics by Example
 THE NATURE OF THE Y VARIABLE
Ratio Scale:
Ratio of two variables, distance between two variables, and
ordering of variables are meaningful
Interval Scale:
Distance and ordering between two variables meaningful, but
not ratio
Ordinal Scale:
Ordering of two variables meaningful (not ratio or distance)
Nominal Scale:
Categorical or dummy variables, qualitative in nature

Damodar Gujarati
Econometrics by Example
 THE NATURE OF DATA
Time Series Data
A set of observations that a variable takes at different
times, such as daily (e.g., stock prices), weekly (e.g.,
money supply), monthly (e.g., the unemployment
rate), quarterly (e.g., GDP), annually (e.g.,
government budgets), quinquenially or every five
years (e.g., the census of manufactures), or
decennially or every ten years (e.g., the census of
population).

Damodar Gujarati
Econometrics by Example
 THE NATURE OF DATA
Cross-Section Data
Data on one or more variables collected at the same
point in time.
Examples are the census of population conducted by
the Census Bureau every 10 years, opinion polls
conducted by various polling organizations, and
temperature at a given time in several places.

Damodar Gujarati
Econometrics by Example
 THE NATURE OF DATA
Panel, Longitudinal or Micro-panel Data
Combines features of both cross-section and time
series data
Same cross-sectional units are followed over time
Panel data represents a special type of pooled data
(simply time series, cross-sectional, where the same
cross-sectional units are not necessarily followed over
time)

Damodar Gujarati
Econometrics by Example
 METHOD OF ORDINARY LEAST SQUARES

Method of Ordinary Least Squares (OLS) does not

minimize the sum of the error term, but minimizes
error sum of squares (ESS):

i i 1 2 2 i 3 3i
u 2
(Y B B X B X .... Bk X ki ) 2

To obtain values of the regression coefficients, derivatives

are taken with respect to the regression coefficients and
set equal to zero.

Damodar Gujarati
Econometrics by Example
CLASSICAL LINEAR REGRESSION MODEL
Assumptions of the Classical Linear
Regression Model (CLRM):

A-1: Model is linear in the parameters.

A-2: Regressors are fixed or nonstochastic.
A-3: Given X, the expected value of the error
term is zero, or E(ui |X) = 0.

Damodar Gujarati
Econometrics by Example
CLASSICAL LINEAR REGRESSION MODEL
Assumptions of the Classical Linear Regression Model
(CLRM):

A-4: Homoscedastic, or constant, variance of ui, or

var(ui|X) = 2.
A-5: No autocorrelation, or cov(ui,uj|X) = 0, i j.
A-6: No multicollinearity, or no perfect linear
relationships among the X variables.
A-7: No specification bias.

Damodar Gujarati
Econometrics by Example
 GAUSS-MARKOV THEOREM
On the basis of assumptions A-1 to A-7, the OLS method
gives best linear unbiased estimators (BLUE):
(1) Estimators are linear functions of the dependent
variable Y.
(2) The estimators are unbiased; in repeated applications
of the method, the estimators approach their true values.
(3) In the class of linear estimators, OLS estimators have
minimum variance; i.e., they are efficient, or the best
estimators.

Damodar Gujarati
Econometrics by Example
 HYPOTHESIS TESTING: t TEST
To test the following hypothesis:
H0: Bk = 0
H1: Bk 0
we calculate the following and use the t table to obtain the
critical t value with n-k degrees of freedom for a given level of
significance (or , equal to 10%, 5%, or 1%):
bk
t
se(bk )

If this value is greater than the critical t value, we can reject H0.

Damodar Gujarati
Econometrics by Example
 HYPOTHESIS TESTING: t TEST
An alternative method is seeing whether zero lies within the
confidence interval:
[bk t / 2 se(bk )] (1 )
If zero lies in this interval, we cannot reject H0.

The p-value gives the exact level of significance, or the lowest

level of significance at which we can reject H0.

Damodar Gujarati
Econometrics by Example
 GOODNESS OF FIT, R2
R2, the coefficient of determination, is an overall measure of
goodness of fit of the estimated regression line.
Gives the percentage of the total variation in the dependent
variable that is explained by the regressors.
It is a value between 0 (no fit) and 1 (perfect fit).
Let: Explained Sum of Squares (ESS) (Y Y ) 2
Residual Sum of Squares (RSS) e 2
Total Sum of Squares (TSS) (Y Y ) 2

Then: ESS RSS

R
2
1
TSS TSS
Damodar Gujarati
Econometrics by Example
 HYPOTHESIS TESTING: F TEST
Testing the following hypothesis is equivalent to testing the
hypothesis that all the slope coefficients are 0:
H0 : R 2 = 0
H1 : R 2 0
Calculate the following and use the F table to obtain the critical F
value with k-1 degrees of freedom in the numerator and n-k degrees
of freedom in the denominator for a given level of significance:

ESS / df R 2 /( k 1)
F
RSS / df (1 R 2
) /(n k )
If this value is greater than the critical F value, reject H .
0

Damodar Gujarati
Econometrics by Example
 Lecturing 04 (Lanjutan)

FUNCTIONAL FORMS
OF REGRESSION MODELS

Damodar Gujarati
Econometrics by Example
 LOG-LINEAR, DOUBLE LOG, OR
CONSTANT ELASTICITY MODELS
The Cobb-Douglas Production Function:
Qi B1 Li K iB2 B3

can be transformed into a linear model by taking natural

logs of both sides:
ln Qi ln B1 B2 ln Li B3 ln K i
The slope coefficients can be interpreted as elasticities.
If (B2 + B3) = 1, we have constant returns to scale.
If (B2 + B3) > 1, we have increasing returns to scale.
If (B2 + B3) < 1, we have decreasing returns to scale.

Damodar Gujarati
Econometrics by Example
 Example

Damodar Gujarati

Econometrics by Example
 Interpretation

Damodar Gujarati

Econometrics by Example
 LOG-LIN OR GROWTH MODELS
The rate of growth of real GDP:
RGDPt RGDP1960 (1 r )t
can be transformed into a linear model by taking natural
logs of both sides:
ln RGDPt ln RGDP1960 t ln(1 r )
Letting B1 = ln RGDP1960 and B2 = ln (l+r), this can be
rewritten as:
ln RGDPt = B1 +B2 t
B2 is considered a semi-elasticity or an instantaneous growth rate.
The compound growth rate (r) is equal to (eB2 1).

Damodar Gujarati
Econometrics by Example
 LOG-LIN OR GROWTH MODELS

Damodar Gujarati

Econometrics by Example
 Example

Damodar Gujarati

Econometrics by Example
 Interpretation

Damodar Gujarati

Econometrics by Example
 LIN-LOG MODELS
Lin-log models follow this general form:

Yi B1 B2 ln X i ui

Note that B2 is the absolute change in Y responding to a

percentage (or relative) change in X
If X increases by 100%, predicted Y increases by B2 units
Used in Engel expenditure functions: The total expenditure
that is devoted to food tends to increase in arithmetic
progression as total expenditure increases in geometric
proportion.

Damodar Gujarati
Econometrics by Example
 LIN-LOG MODELS

Damodar Gujarati

Econometrics by Example
 Interpretation

Damodar Gujarati

Econometrics by Example
 Example

Damodar Gujarati

Econometrics by Example
 Interpretation

Damodar Gujarati

Econometrics by Example
 RECIPROCAL MODELS
Lin-log models follow this general form:
1
Yi B1 B2 ( ) ui
Xi
Note that:
1
As X increases indefinitely, the term B2 ( ) approaches zero and Y approaches
Xi
the limiting or asymptotic value B1.
The slope is: dY 1
B2 ( 2
)
dX X
Therefore, if B2 is positive, the slope is negative throughout, and if B2 is negative,
the slope is positive throughout.

Damodar Gujarati
Econometrics by Example
 POLYNOMIAL REGRESSION MODELS
The following regression predicting GDP is an example of
a quadratic function, or more generally, a second-degree
polynomial in the variable time:
RGDPt A1 A2time A3time 2 ut

The slope is nonlinear and equal to:

dRGDP
A2 2 A3time
time

Damodar Gujarati
Econometrics by Example
 SUMMARY OF FUNCTIONAL FORMS

Damodar Gujarati
Econometrics by Example
 COMPARING ON BASIS OF R2
We cannot directly compare two models that have different
dependent variables.
We can transform the models as follows and compare RSS:
Step 1: Compute the geometric mean (GM) of the dependent variable, call
it Y*.
Step 2: Divide Yi by Y* to obtain:
Yi ~
*
Yi
Y
Step 3: Estimate the equation with lnYi as the dependent variable using
in lieu of Yi as the dependent variable (i.e., use ln as the dependent
~ ~
Yi
variable). Yi
Step 4: Estimate the equation with Yi as the dependent variable using as
the dependent variable instead of Yi.
~
Yi
Damodar Gujarati
Econometrics by Example
 STANDARDIZED VARIABLES
We can avoid the problem of having variables
measured in different units by expressing them in
standardized form:
_
Yi Y Xi X
Yi *
; Xi
*

SY SX
where SY and SX are the sample standard deviations and
_ _
andY areX the sample means of Y and X, respectively
The mean value of a standardized variable is always
zero and its standard deviation value is always 1.

Damodar Gujarati
Econometrics by Example