Week 1 MOD A Simple Regression Chapter 12 Berenson

Chapter 12
Simple Linear
Regression
PowerPoint to accompany:
Learning Objectives
After studying this Chapter you should have a better

understanding of:
How to use regression analysis to predict the value of a dependent
variable based on an independent variable
The meaning of the regression coefficients b0 and b1
How to evaluate the assumptions of regression analysis and what
to do if the assumptions are violated
To make inferences about the slope and correlation coefficient
To estimate mean values and predict individual values
2 Copyright 2013 Pearson Australia (a division of Pearson Australia Group Pty Ltd) 9781442549272/Berenson/Business Statistics /2e
Life Without FaceBook Regression
Source: http://www.youtube.com/watch?v=fmj_kMrpypg
Introduction to Regression Analysis
Regression analysis is used to

Predict the value of a dependent variable (Y) based on the value of
at least one independent variable (X)
Explain the impact of changes in an independent variable on the
dependent variable
Dependent variable (Y): the variable we wish to predict or explain

(response variable)
Independent variable (X): the variable used to explain the

dependent variable (explanatory variable)
Introduction to Regression Analysis
Simple linear regression:

Only one independent variable, X
Relationship between X and Y is described by a linear function
Changes in Y are assumed to be caused by changes in X
Types of Linear Relationships
Strong Weak relationships

relationships
Y Y
X X
Y Y
X X
Types of Non-linear Relationships
Curvilinear No relationship
relationships
Y Y
X X
Y Y
X X
Simple Linear Regression Model
Population Random
Population Independent error
slope
Y intercept variable term
coefficient
Yi 0 1Xi i
Linear component Random error
Dependent
variable component
Simple Linear Regression Model
Y
Yi 0 1Xi i
Observed
value of Y for
Xi
i
Slope = 1
Predicted
value of Y for Random error for
Xi this Xi value
Intercept = 0
Xi X
Simple Linear Regression Equation (Prediction Line)
The simple linear regression equation provides an estimate of the

population regression line
Estimated (or Estimate of the Estimate of the

predicted) Y value regression regression slope
for observation i intercept

Yi b0 b1X i
Value of X for
observation i
The individual random error terms ei have a mean of zero
Least Squares Method
b0 and b1 are obtained by finding the values of b0 and b1 that

minimise the sum of the squared differences between
)
actual values (Y) and predicted values ( Y
min (Yi Yi ) min (Yi (b0 b1Xi ))

2 2
b0 is the estimated average value of Y when the value of X is

zero
b1 is the estimated change in the average value of Y as a
result of a one-unit change in X
Simple Linear Regression Example
A manager of a local computer games store wishes to:

examine the relationship between weekly sales (Y) and the
number of customers making purchases (X) over a 10 week
period; and
use the results of that examination to predict future weekly
sales
Weekly sales Number of
in $1000s Customers
(Y) (X) Weekly sales model: scatter plot
245 1400
312 1600
279 1700
308 1875
199 1100
219 1550
405 2350
324 2450
319 1425
255 1700
Weekly sales model: scatter plot and regression line
Slope = 0.10977
Intercept
= 98.248
Weekly sales 98.24833 0.10977 (customers )

Weekly sales 98.24833 0.10977 (customers)

b0 is the estimated average value of Y when the value of X
is zero (if X = 0 is in the range of observed X values)
Here, for no customers, b0 = 98.2483 which appears
nonsensical. However, the intercept simply indicates that over
the sample size selected, the portion of weekly sales not
explained by number of customers is $98,248.33. Also note
that X=0 is outside the range of observed values
b1 measures the estimated change in the average value of

Y as a result of a one-unit change in X
Here, b1 = .10977 tells us that the average value of weekly
sales increases by .10977($1000) = $109.77, on average, for
each additional customer
Predict the weekly sales for the local store for 2000
customers:
Weekly sales 98.25 0.1098 (2000)

98.25 0.1098(2000)
317.85
The predicted weekly sales for the local computer games

store for 2000 customers is 317.85 ($1,000s) = $317,850
Interpolation vs. Extrapolation
When using a regression model for prediction, only predict
within the relevant range of data
Relevant range for interpolation
Do not try to
extrapolate
beyond the range
of observed Xs
Measures of Variation
Total variation is made up of two parts
SST SSR SSE

Total Sum of Regression Sum of Error Sum of
Squares Squares Squares
SST ( Yi Y )2 SSR (Yi Y ) 2 SSE i i

(Y Y ) 2
Measures the Explained variation Variation attributable to

variation of the Yi attributable to the factors other than
values around their relationship between the relationship between
mean Y X and Y X and Y
Measures of Variation
Y
Yi
SSE = (Yi - Yi )2 Y
_
SST = (Yi - Y)2

Y _
_ SSR = (Yi - Y)2
_
Y Y
Xi X
Coefficient of Determination, r2
The coefficient of determination is the portion of the total
variation in the dependent variable that is explained by
variation in the independent variable
The coefficient of determination is also called r-squared and
is denoted as r2
SSR
2 regression sum of squares
r
SST total sum of squares
note: 0 r2 1
Examples of Approximate r2 values
Y Y Perfect linear
relationship
between X and Y
100% of the
variation in Y is
X explained by
r2 = 1 r2 = 1 X variation in X
Y
No linear relationship between X
and Y
The value of Y does not depend
on X (None of the variation in
Y
is explained by variation in X)
r2 = 0 X
Examples of Approximate r2 values
0 < r2 < 1
Y
Weaker linear relationships
between X and Y:
X
Y
Some but not all of the
variation in Y is explained by
variation in X
X
Excel Output
Standard Error of the Estimate
The standard deviation of the variation of observations around
the regression line is estimated by
SSE (Yi Yi )
2
SYX i 1
n2 n2
where
SSE = error sum of squares
n = sample size
Excel Output
Comparing Standard Errors
SYX is a measure of the variation of observed Y
values from the regression line
Y Y
small s YX X large sYX X
The magnitude of SYX should always be judged relative

to the size of the Y values in the sample data
i.e. SYX = $41.33K is moderately small relative to
weekly sales in the $200 - $300K range
Assumptions of Regression
Use the acronym LINE

Linearity
The underlying relationship between X and Y is linear
Independence of errors
Error values are statistically independent
Normality of error
Error values () are normally distributed for any given value of X
Equal variance (homoscedasticity)

The probability distribution of the errors has constant variance
Residual Analysis
The residual for observation i, ei, is the difference between
its observed and predicted value
ei Yi Yi
Check the assumptions of regression by examining the
residuals,
Examine for linearity assumption
Evaluate independence assumption
Evaluate normal distribution assumption
Examine for constant variance for all levels of X
(homoscedasticity)
Graphical Analysis of Residuals
Can plot residuals vs. X
Residual Analysis for Linearity
Y Y
x x
residuals
residuals
x x
Not Linear Linear

Residual Analysis for Independence
Not Independent Independent

residuals
residuals
X
residuals
Residual Analysis for Normality
A normal probability plot of the residuals can be used to check

for normality
Percent
100
0
-3 -2 -1 0 1 2
3 Residual
Residual Analysis for Equal Variance
(Homoscedasticity)
Y
Y
x x
residuals
residuals
x x
Non-constant variance Constant variance

Excel Residual Output
RESIDUAL OUTPUT
Predicted
Weekly
Sales Residuals
1 251.92316 -6.923162
2 273.87671 38.12329
3 284.85348 -5.853484
4 304.06284 3.937162
5 218.99284 -19.99284
6 268.38832 -49.38832
7 356.20251 48.79749
8 367.17929 -43.17929
9 254.6674 64.33264
10 284.85348 -29.85348 Does not appear to violate any
regression assumptions
Measuring Autocorrelation: The Durbin-Watson
Statistic
Used when data are collected over time to detect if autocorrelation

is present
Autocorrelation exists if residuals in one time period are related

(dependent) to residuals in another period
Positive autocorrelation, which occurs much more commonly in

regression analysis, arises when there are clusters of residuals with
the same sign
In negative autocorrelation, residuals tend to alternate in sign
Autocorrelation
Autocorrelation is correlation of the errors (residuals) over time
Here, residuals show a

cyclic pattern, not
random. Cyclical
patterns are a sign of
positive autocorrelation
Violates the regression assumption that residuals are random

and independent
The Durbin-Watson Statistic
The Durbin-Watson statistic is used to test for autocorrelation
H0: residuals are not correlated

H1: positive autocorrelation is present
n The possible range is 0 D 4

i i 1
(e e )2
D should be close to 2 if H0 is true
D i 2
n
i
e 2
i 1
D approaches 0 when there is positive
autocorrelation, D approaches 4 when there is
negative autocorrelation
Testing for Positive Autocorrelation
H0: positive autocorrelation does not exist
H1: positive autocorrelation is present
Calculate the Durbin-Watson test statistic = D

(The Durbin-Watson Statistic can be found using Excel)
Find the values dL and dU from the Durbin-Watson table

(for sample size n and number of independent variables k)
Decision rule: reject H0 if D < dL
Reject Inconclusive Do not

H0 reject H0
0 dL dU 2
Suppose we have the following time series data
Is there positive autocorrelation?
Example with n = 25
Excel/PHStat
output:
Durbin-Watson Calculations
Sum of squared
difference of
residuals 3296.18 n
Sum of squared i i1
(e e ) 2
3296.18
residuals 3279.98 D i 2
n
1.0049 4
3279.98
Durbin-Watson ei 2
Statistic 1.00494 i 1
Here, n = 25 and there is Decision: reject H0 since D=
1.00494 < dL
k=1 one independent
variable
Using the Durbin-Watson and conclude that significant
positive autocorrelation exists
table, dL = 1.29 and
dU = 1.45
Therefore the linear model is not
the appropriate model to
forecast sales
Reject H0 Inconclusive Do not

reject H0
0 dL = 1.29 dU = 1.45 2
Dobs = 1.00494
Inferences About the Slope
The standard error of the regression slope coefficient (b1) is

estimated by
SYX SYX
Sb1
SSX (X i X) 2
where
Sb1= Estimate of the standard error of the least squares slope
SSE
S YX
n 2 = Standard error of the estimate
Excel Output
Comparing Standard Errors of the Slope
Sb1 is a measure of the variation in the slope of

regression lines from different possible samples
Y Y
small Sb1 X large Sb1 X
Inference about the Slope: t Test
t test for a population slope
Is there a linear relationship between X and Y?
Null and alternative hypotheses

H0: 1 = 0 (no linear relationship)
H1: 1 0 (linear relationship does exist)
Test statistic with d.f. = n-2
b1 1 where b1 = regression slope coefficient

t 1 = hypothesised slope
Sb1
Sb = standard error of the slope
45
1
Copyright 2013 Pearson Australia (a division of Pearson Australia Group Pty Ltd) 9781442549272/Berenson/Business Statistics /2e
Inferences about the Slope: t Test Example
Weekly sales 98.25 0.1098 (customers ) The slope of this
model is 0.1098
H 0: 1 = 0 Does number of
H 1: 1 0 b1 Sb1 customers affect
weekly sales?
Standard
Coefficients Error tStat P-value
Intercept 98.24833 58.03348 1.69296 0.12892
Number of
customers 0.10977 0.03297 3.32938 0.01039
b1 1 0.10977 0
t 3.32938
Sb1 0.03297
Inferences about the Slope: t Test Example
H 0: 1 = 0 Test Statistic: t = 3.329

H1: 1 0 T critical = +/- 2.3060 (from t tables)
Decision: Reject H0
d.f. = 10-2 = 8 Conclusion: There is
sufficient evidence that
/2=.025
/2=.025 Do not number of customers
reject H0 affects weekly sales
Reject H0 Reject H0
-t/2 0 t/2
-2.3060 2.3060 3.329
F Test for Significance
F Test statistic
SSR
MSR where MSR
F k
MSE SSE
MSE
n k 1
F follows an F distribution with k numerator and (n k - 1)

denominator degrees of freedom
k = the number of independent (explanatory) variables in the
regression model
Excel Output
F Test for Significance Example
H 0: 1 = 0 Test Statistic:
H 1: 1 0 MSR
= .05
F 11.08
MSE
df1= 1 df2 = 8
Conclusion:
Reject H0 at = 0.05
Critical Value:
There is sufficient evidence
F = 5.32 that number of customers
= .05 affects weekly sales
0
F
Do not
reject H0 F.05 = 5.32 Reject H0
Confidence Interval Estimate for the Slope
b1 t n2Sb1 d.f. = n - 2
Excel Printout for
Weekly sales:
Coefficien Standard Lower Upper
ts Error tStat P-value 95% 95%
Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386
Customers 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580
At 95% level of confidence, the confidence interval for the slope is

(0.03374, 0.18580) i.e. we are 95% confident that the average
impact on weekly sales is between $33.74 and $185.80 per
customer
This 95% confidence interval does not include 0.
Conclusion: There is a significant relationship between weekly
sales and number of customers at the .05 level of significance
t Test for a Correlation Coefficient
Hypotheses
no association (correlation) between

H 0: = 0
X and Y
H 1: 0 statistically significant association
(correlation) exists
Test statistic
where
r -
t r r 2 if b1 0
1 r 2
r r 2 if b1 0
n2 (with n 2 degrees of freedom)
t Test for a Correlation Coefficient Example
Is there evidence of a significant linear relationship between weekly sales
and number of customers at the .05 level of significance?
H0 : = 0 (No r .762 0
correlation) t 3.329
H1 : 0 (correlation 1 r 2
1 .762 2
exists)
=.05 , df = 10 - 2 = 8 n2 10 2
Decision: Reject Ho
Conclusion:
/2=.025 /2=.025 There is evidence of
a significant linear
association at the
Reject H0 Do not reject H0 Reject H0 5% level of
-t/2 t/2 significance
0
-2.3060 2.3060
3.329
Confidence Interval for the Mean of Y, Given X
Confidence interval estimate for the mean value of Y

given a particular Xi
Confidence interval for Y|X Xi

Y tn 2SYX hi
Size of interval varies according to distance away from
mean, X
1 (Xi X)2 1 (Xi X)2

hi
n SSX n (Xi X)2
Prediction Interval for an Individual Y, Given X
Confidence interval estimate for an Individual value of Y

given a particular Xi
Confidence interval for YX X i

Y tn 2SYX 1 hi
This extra term adds to the interval width to reflect the added
uncertainty for an individual case
Estimation of Mean Values vs Estimation of
Individual Values
1 (Xi X)2
Find the 95%
Y t n-2S YX 317.85 37.12
confidence n (Xi X) 2
interval for the

mean weekly The confidence interval endpoints
sales of 2,000 are 280.66 and 354.90, or from
customers $280,660 to $354,900
1 (Xi X)2
Find the 95%
Y t n-1S YX 1 317.85 102.28
prediction n (Xi X)2
interval for
individual The prediction interval endpoints
weekly sales are 215.50 and 420.07, or from
of 2,000 $215,500 to $420,070
customers
Pitfalls of Regression Analysis
Lacking an awareness of the assumptions underlying least-squares

regression
Not knowing how to evaluate the assumptions
Not knowing the alternatives to least-squares regression if a particular
assumption is violated
Using a regression model without knowledge of the subject matter
Extrapolating outside the relevant range
Concluding that a significant relationship in observational study is due
to a cause and effect relationship
Industry Application
Harvard economists made case for cutting taxes, spending
By Doug Ross
In a recent online article, Doug Ross quoted Harvard economists Alberto
Alesina and Silvia Ardagnas 2009 paper where they looked at 107
examples in developed countries over more than 30 years from 1970
to 2009 using simple regression analysis, and concluded:
Fiscal stimuli based upon tax cuts are more likely to increase growth
than those based upon spending increases. As for fiscal adjustments,
those based upon spending cuts and no tax increases are more likely
to reduce deficits and debt than those based upon tax increases.
Also, spending cuts adopted to reduce deficits have been associated
with economic expansion rather than recession.
Source:
http://www.nwitimes.com/news/opinion/mailbag/harvard-economists-made-case-for-cutting-taxe
s-spending/article_95214860-7a88-5771-89fc-204e1e5f7f1e.html
Chapter Summary
Introduced types of regression models

Reviewed assumptions of regression and correlation (LINE)
Discussed determining the simple linear regression equation using
Least Squares method
Described measures of variation
Discussed residual analysis and evaluation of assumptions
Addressed measuring autocorrelation
Described inference about the slope
Discussed correlation
Addressed estimation of mean values and prediction of individual
values
Discussed possible pitfalls and ethical issues in regression
Copyright 2013 Pearson Australia (a division of Pearson Australia Group Pty Ltd) 9781442548473/Berenson/Basic Business Statistics/3e

Week 1 MOD A Simple Regression Chapter 12 Berenson

Uploaded by

Copyright:

Available Formats

You might also like

Week 1 MOD A Simple Regression Chapter 12 Berenson

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Week 1 MOD A Simple Regression Chapter 12 Berenson

Uploaded by

Copyright:

Available Formats

Chapter 12

After studying this Chapter you should have a better

Regression analysis is used to

Dependent variable (Y): the variable we wish to predict or explain

Independent variable (X): the variable used to explain the

Simple linear regression:

Strong Weak relationships

The simple linear regression equation provides an estimate of the

Estimated (or Estimate of the Estimate of the

The individual random error terms ei have a mean of zero

b0 and b1 are obtained by finding the values of b0 and b1 that

min (Yi Yi ) min (Yi (b0 b1Xi ))

b0 is the estimated average value of Y when the value of X is

A manager of a local computer games store wishes to:

Weekly sales model: scatter plot and regression line

Weekly sales 98.24833 0.10977 (customers )

Weekly sales 98.24833 0.10977 (customers)

b1 measures the estimated change in the average value of

Weekly sales 98.25 0.1098 (2000)

The predicted weekly sales for the local computer games

Relevant range for interpolation

SST SSR SSE

SST ( Yi Y )2 SSR (Yi Y ) 2 SSE i i

Measures the Explained variation Variation attributable to

small s YX X large sYX X

The magnitude of SYX should always be judged relative

Use the acronym LINE

Equal variance (homoscedasticity)

Not Linear Linear

Not Independent Independent

A normal probability plot of the residuals can be used to check

Non-constant variance Constant variance

Used when data are collected over time to detect if autocorrelation

Autocorrelation exists if residuals in one time period are related

Positive autocorrelation, which occurs much more commonly in

In negative autocorrelation, residuals tend to alternate in sign

Here, residuals show a

Violates the regression assumption that residuals are random

The Durbin-Watson statistic is used to test for autocorrelation

H0: residuals are not correlated

n The possible range is 0 D 4

Calculate the Durbin-Watson test statistic = D

Find the values dL and dU from the Durbin-Watson table

Decision rule: reject H0 if D < dL

Reject Inconclusive Do not

Suppose we have the following time series data

Is there positive autocorrelation?

Reject H0 Inconclusive Do not

The standard error of the regression slope coefficient (b1) is

Sb1 is a measure of the variation in the slope of

small Sb1 X large Sb1 X

Null and alternative hypotheses

Test statistic with d.f. = n-2

b1 1 where b1 = regression slope coefficient

H 0: 1 = 0 Test Statistic: t = 3.329

F follows an F distribution with k numerator and (n k - 1)

At 95% level of confidence, the confidence interval for the slope is

no association (correlation) between