1999 Prentice-Hall, Inc. Chap.

14 - 1
Dummy-Variable Regression
Model
1999 Prentice-Hall, Inc. Chap. 14 - 2
Multiple Regression
Models
Multiple
Regression
Models
Linear
Dummy
Variable
Linear
Non-
Linear
Inter-
action
Poly-
Nomial
Square
Root
Log Reciprocal Exponential
1999 Prentice-Hall, Inc. Chap. 14 - 3
Dummy-Variable
Regression Model
Involves categorical X variable with
2 or more levels
e.g., male-female, college-no college etc.
or firms or states or cities
Each level is coded 0 or 1
Assumes only intercept is different
Slopes are constant across categories
The number of dummy variables that are
included is 1-# of levels
1999 Prentice-Hall, Inc. Chap. 14 - 4
Dummy-Variable Regression
Model Example Coding
Gender (2 levels): Male=1; Female=0 for
variable MALE

Marital Status (3 levels - requires 2
dummies):
MARRIED: Single=0; Divorced=0; Married=1
DIVORCED: Single=0; Divorced=1; Married=0
1999 Prentice-Hall, Inc. Chap. 14 - 5
Interpreting Dummy-
Variable Model Equation
Given:
Starting s alary of c ollege gra d' s
i
if Female


f Male

Y b
b X
Y
X
i i
= +
=
=
0 2 2
2
0
1
b
0
= mean Y for men since for each man Y=b
0
+b
2
*(0)
b2= difference of means between men and women since for
women Y=b
0
+b
2
*(1).
b
0
+b
2
= mean Y for women
1999 Prentice-Hall, Inc. Chap. 14 - 6
Comparison to other
techniques
This is identical to a t-test for the
difference of means. We test b
2
=0 to test
if there is a significant difference of
means.
This is identical to a one-way ANOVA
for a difference of means.
1999 Prentice-Hall, Inc. Chap. 14 - 7
Dummy-Variable Model
Relationships
Y
X
1
0
0
Means for males and females
b
0
b
0
+ b
2
Females
Males
1999 Prentice-Hall, Inc. Chap. 14 - 8
Interpreting Dummy-
Variable Model Equation
Given:
Starting s alary of c ollege gra d' s
GPA
i
if Female
Males (


f Male


):
Y b b X b X
Y
X
X
Y b b X b b b X
i i i
i i i
X
= + +
=
=
=
= + + = +
=
0 1 1 2 2
1
2
0 1 1 2 0 1 1
0
1
0
2
0
1999 Prentice-Hall, Inc. Chap. 14 - 9
Interpreting Dummy-
Variable Model Equation
( ) ( )
i i i
X b b b b X b b Y
X
1 1
2 0
2 1 1 0
1

FEMALES
: ) 1 (
2
+ + = + + =
=
1999 Prentice-Hall, Inc. Chap. 14 - 10
Dummy-Variable Model
Relationships
Y
X
1
0
0
Same slopes b
1
b
0
b
0
+ b
2
Females
Males
1999 Prentice-Hall, Inc. Chap. 14 - 11
Dummy-Variable Model
Example
{
( )
( ) ( )
i
X
X
X
i i i
i i i
i i i
X X Y
X X Y
X
X X Y
1
2
2
5
: ) 1 ( Females
: ) 0 ( Males
Female if
Male f
: Output Computer
10 5 7 3 1 7 5 3

5 3 0 7 5 3

1
i 0
7 5 3

1 1
1 1
2
2 1
+
=
=
= + + = + + =
+ = + + =
=
+ + =
Same slopes
1999 Prentice-Hall, Inc. Chap. 14 - 12
Interpretation
The difference in mean output between men
and women is 7, holding constant GPA.
When there are more than two groups, the
interpretation of the coefficient is always
the difference of means between that
group and the EXCLUDED GROUP.
1999 Prentice-Hall, Inc. Chap. 14 - 13
How many dummy
variables do you need?
To compare union workers and
nonunion workers?
To compare whites, blacks, hispanics
and asians?
To compare months of the year?
1999 Prentice-Hall, Inc. Chap. 14 - 14
EXAMPLE
1999 Prentice-Hall, Inc. Chap. 14 - 15
EXAMPLE
1999 Prentice-Hall, Inc. Chap. 14 - 16
Interaction
Regression Model
1999 Prentice-Hall, Inc. Chap. 14 - 17
Multiple Regression
Models
Multiple
Regression
Models
Linear
Dummy
Variable
Linear
Non-
Linear
Inter-
action
Poly-
Nomial
Square
Root
Log Reciprocal Exponential
1999 Prentice-Hall, Inc. Chap. 14 - 18
Interaction
Regression Model
Hypothesizes interaction between pairs of X
variables
Response to one X variable varies at different
levels of another X variable
Contains two-way cross product terms
Can be combined with other models
e.g., dummy variable model
Y X X X X
i i i i i i
= + + + + | | | | c
0 1 1 2 2 3 1 2
1999 Prentice-Hall, Inc. Chap. 14 - 19
Effect of Interaction
Given:
Without interaction term, effect of X
1
on Y
is measured by |
1

With interaction term, effect of X
1
on
Y is measured by |
1
+ |
3
X
2
Effect changes as X
2i
increases
Y X X X X
i i i i i i
= + + + + | | | | c
0 1 1 2 2 3 1 2
1999 Prentice-Hall, Inc. Chap. 14 - 20
Interaction Example
Effect (slope) of X
1
on Y does depend on X
2
value
X
1
4
8
12
0
0 1 0.5 1.5
Y
Y = 1 + 2X
1
+ 3X
2
+ 4X
1
X
2

Y = 1 + 2X
1
+ 3(1) + 4X
1
(1) = 4 + 6X
1

Y = 1 + 2X
1
+ 3(0) + 4X
1
(0) = 1 + 2X
1

1999 Prentice-Hall, Inc. Chap. 14 - 21
Interaction Regression
Model Worksheet
Case, i Y
i
X
1i
X
2i
X
1i
X
2i
1 1 1 3 3
2 4 8 5 40
3 1 3 2 6
4 3 5 6 30
: : : : :
Multiply X
1
by X
2
to get X
1
X
2
.
Run regression with Y, X
1
, X
2
, X
1
X
2
1999 Prentice-Hall, Inc. Chap. 14 - 22
Interpretation when
there are 3+levels
DIVORCED MARRIED MALE Y
3 2 1
| | | o + + + =
a = Mean Y for a single female
(MALE,MARRIED,DIVORCED=0)
b
1
= Difference in means between males and females
(a+b1=mean Y for single males)
b
2
= Difference in means between single and married
(holding gender constant)
b
3
= Difference in means between divorced and single
b
2
-b
3
=Difference in means between married and
divorced
1999 Prentice-Hall, Inc. Chap. 14 - 23
Interpretation when
there are 3+levels
It is possible to interact the dummy variables. This can
give an identical result as a 2-way ANOVA.
In this example, this would allow the effect of marital
status to vary with gender.
1999 Prentice-Hall, Inc. Chap. 14 - 24
Interpretation when
there are 3+levels
DIVORCED * MALE MARRIED * MALE
DIVORCED MARRIED MALE Y
5 4
3 2 1
| + | +
| + | + | + o =
MALE=0 if female and 1 if male
1999 Prentice-Hall, Inc. Chap. 14 - 25
Interpretation when
there are 3+levels
DIVORCED * MALE MARRIED * MALE
DIVORCED MARRIED MALE Y
5 4
3 2 1
| + | +
| + | + | + o =
MALE=0 if female and 1 if male
MARRIED=1 if married; 0 if divorced or single
DIVORCED=1 if divorced; 0 if single or married
MALE*MARRIED=1 if male married; 0 otherwise
=(MALE times MARRIED)
MALE*DIVORCED=1 if male divorced; 0
otherwise(=MALE times DIVORCED)
1999 Prentice-Hall, Inc. Chap. 14 - 26
DIVORCED * MALE MARRIED * MALE
DIVORCED MARRIED MALE Y
5 4
3 2 1
| + | +
| + | + | + o =
SINGLE MARRIED DIVORCED
FEMALE 0 B
2
B
3
MALE B
1
B
1
+B
2
+B
4
B
1
+B
3
+B
5
1999 Prentice-Hall, Inc. Chap. 14 - 27
Interpreting Results
FEMALE
Single:
Married:
Divorced:
MALE
Single:
Married:
Divorced:

o
2
| + o
3
| + o
1
| + o
4 2 1
| + | + | + o
5 3 1
| + | + | + o
Difference
1
|
4 1
| + |
5 1
| + |
Main Effects: MALE; (MARRIED and DIVORCED)
Interaction Effects: MALE*MARRIED and
MALE*DIVORCED
1999 Prentice-Hall, Inc. Chap. 14 - 28
Interpreting results
Testing for interaction: Must do F-test of joint
hypothesis that

EXAMPLE
0
5 4
= | = |
1999 Prentice-Hall, Inc. Chap. 14 - 29
Polynomial (Curvilinear)
Regression Model
1999 Prentice-Hall, Inc. Chap. 14 - 30
Multiple Regression
Models
Multiple
Regression
Models
Linear
Dummy
Variable
Linear
Non-
Linear
Inter-
action
Poly-
Nomial
Square
Root
Log Reciprocal Exponential
1999 Prentice-Hall, Inc. Chap. 14 - 31
Curvilinear
Regression Model
Relationship between 1 response
variable and 2 or more explanatory
variable is a polynomial function
Useful when scatter diagram indicates
non-linear relationship
Curvilinear model:

The second explanatory variable is the
square of the 1st.
i i i i
X X Y c | | | + + + =
2
1 2 1 1 0
1999 Prentice-Hall, Inc. Chap. 14 - 32
Curvilinear Regression
Model
Curvilinear models may be considered when
scatter diagram takes on the following shapes:
X
1
Y
X
1
X
1
X
1
Y Y Y
|
2
> 0 |
2
> 0 |
2
< 0 |
2
< 0
|
2
= the coefficient of the quadratic term
1999 Prentice-Hall, Inc. Chap. 14 - 33
Testing for Significance:
Curvilinear Model
Testing for Overall Relationship
Similar to test for linear model
F test statistic =
Testing the Curvilinear Effect
Compare curvilinear model

with the linear model
MSE
MSR
i i i i
X X Y c | | | + + + =
2
1 2 1 1 0
i i i
X Y c | | + + =
1 1 0
1999 Prentice-Hall, Inc. Chap. 14 - 34
Testing for Significance:
Curvilinear Model
May require testing a portion of the
model (e.g. the linear and squared
terms) when there are other variables in
the model


Here we must test to test for the
significance of X
1
- an F-test for these
two variables


i i 2 3
2
i 1 2 i 1 1 0 i
X X X Y c + | + | + | + | =
0
2 1
= | = |
1999 Prentice-Hall, Inc. Chap. 14 - 35
Inherently Linear
Models
Non-linear models that can be expressed in
linear form
Can be estimated by LS in linear form
Require data transformation
Multiplicative model example
( ) ( ) ( ) ( ) ( )
i i i i
i i i i
X X Y
X X Y
c | | |
c |
| |
ln ln ln ln ln
2 2 1 1 0
2 1 0
2 1
+ + + =
=
1999 Prentice-Hall, Inc. Chap. 14 - 36
Using Transformations

Requires Data Transformation
Either or Both Independent and
Dependent Variables May be
Transformed
Can be based on theory, logic or scatter
diagrams
1999 Prentice-Hall, Inc. Chap. 14 - 37
Square Root
Transformation
Y
X
1
Y
X
1
i i i i
X X Y c | | | + + + =
2 2 1 1 0
|
1
> 0
|
1
< 0
Similarly for X
2
Transforms one of above model to one that appears linear.
Often used to overcome heteroscedasticity.
1999 Prentice-Hall, Inc. Chap. 14 - 38
Logarithmic
Transformation
Y
X
1
Y
X
1
i i i i
) X ln( ) X ln( Y c | | | + + + =
2 2 1 1 0
|
1
> 0
|
1
< 0
Similarly for X
2
1999 Prentice-Hall, Inc. Chap. 14 - 39
Exponential
Transformation
Y
X
1
Y
X
1
i
X X
i
i i
e Y c
| | |
2 2 1 1 0
+ +
= Original Model
|
1
> 0
|
1
< 0
Similarly for X
2
Transformed into:
1 2 2 1 1 0
c | | | ln X X Y ln
i i i
+ + + =
1999 Prentice-Hall, Inc. Chap. 14 - 40
Interpretation of
coefficients
The dependent variable is logged.
The coefficient on the independent variable can be
approximately interpreted as : a 1 unit change in X
leads to a b percentage change in Y.
The independent variable is logged.
The coefficient on the independent variable can be
approximately interpreted as : a 100 percent
change in X leads to a b unit change in Y.
1999 Prentice-Hall, Inc. Chap. 14 - 41
Interpretation of
coefficients
Both dependent and independent
variables are logged.
The coefficient on the independent variable
can be approximately interpreted as : a 1
percent change in X leads to a b percentage
change in Y. Therefore b is the elasticity of
Y with respect to a change in X.
1999 Prentice-Hall, Inc. Chap. 14 - 42
Income and Experience:
Scatter Plot
1999 Prentice-Hall, Inc. Chap. 14 - 43
Income and Experience:
Linear
Linear Model
1999 Prentice-Hall, Inc. Chap. 14 - 44
Income and Experience: Log
Independent Variable
Log independent variable
1999 Prentice-Hall, Inc. Chap. 14 - 45
Income and Experience:
Income Logged
Log(Y)
1999 Prentice-Hall, Inc. Chap. 14 - 46
Income and Experience:
Double Log
Double Log - Elasticity Model (Note: LFEXP is
already logged in this example)
1999 Prentice-Hall, Inc. Chap. 14 - 47
Income and Experience:
Quadratic
Quadratic
1999 Prentice-Hall, Inc. Chap. 14 - 48
Income and Experience: Log
plus Quadratic
Log(Y) + Quadratic
1999 Prentice-Hall, Inc. Chap. 14 - 49
Income and Experience: All
Specifications
Many specifications
1999 Prentice-Hall, Inc. Chap. 14 - 50
Standardized and
Unstandardized
Many disciplines report ONLY
standardized coefficients
The usual coefficients are then referred
to as unstandardized coefficients
The standardized coefficient are often
referred to as beta weights
The t-tests for significance of the slopes
are identical for either of these two.
1999 Prentice-Hall, Inc. Chap. 14 - 51
Interpretation of
coefficients
If both Y and X are measured in
standardized form,
and

|
|
.
|

\
|

=
Y
Y i
i
Y
y
o

|
|
.
|

\
|

=
o

X i
i
X
x
Then the bs are called standardized
coefficients. They indicate the number
of standard deviations Y will change
when X changes by one standard
deviation
1999 Prentice-Hall, Inc. Chap. 14 - 52
BETA Coefficients Example
1999 Prentice-Hall, Inc. Chap. 14 - 53
Comparison of coefficients
In general, we should NOT compare
coefficients unless they are measured in
the same units (e.g. dollars or inches)
Two unit free measures are sometimes
used to compare coefficients:
elasticities (percentage changes)
standardized coefficients (Stand. Dev.
Changes)