Slides+on+Section+3.2+ (Least Squares+Regression)

Chapter
3
Describing Relationships
Section 3.2
Least-Squares Regression
LEARNING TARGETS
By the end of this section, you should be able to:
ü MAKE predictions using regression lines, keeping in mind the dangers
of extrapolation.
ü CALCULATE and interpret a residual.
ü INTERPRET the slope and y intercept of a regression line.
ü DETERMINE the equation of a least-squares regression line using
technology or computer output.
ü CONSTRUCT and INTERPRET residual plots to assess whether a
regression model is appropriate.
Starnes/Tabor, The Practice of Statistics

LEARNING TARGETS
By the end of this section, you should be able to:
ü INTERPRET the standard deviation of the residuals and r2 and use these
values to assess how well a least-squares regression line models the
relationship between two variables.
ü DESCRIBE how the least-squares regression line, standard deviation of
the residuals, and r2 are influenced by outliers.
ü FIND the slope and y intercept of the least-squares regression line from
the means and standard deviations of x and y and their correlation.

Regression Lines
Linear (straight-line) relationships between two quantitative variables are
common. A regression line summarizes the relationship between two
variables, but only in a specific setting: when one variable helps explain
the other.

Regression Lines
the other.

Regression Lines
the other.
A regression line is a line that

describes how a response variable y
changes as an explanatory variable x
changes. Regression lines are
expressed in the form "# = %& + %( )
where "# (pronounced “y-hat”) is the
predicted value of y for a given
value of x.

Prediction
A random sample of 16 used Ford F-150 SuperCrew 4 × 4s was selected from
among those listed for sale at autotrader.com. The data are shown in the table. For
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
these data, the regression equation is *+,-.
Predict the price of a Ford F-150 that has been driven 100,000 miles.

Prediction
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

Prediction
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.

Prediction
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.
/ = 38257 − 0.1629 100000
*+,-.

Prediction
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.
/ = 38257 − 0.1629 100000
*+,-.
/ = $21,967
*+,-.

Extrapolation
Can we predict the price of a Ford F-150 with 300,000 miles driven?

Extrapolation
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.

Extrapolation
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.
/ = 38257 − 0.1629 300000
*+,-.

Extrapolation
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.
/ = 38257 − 0.1629 300000
*+,-.
/ = −$10,613
*+,-.

Extrapolation
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.
/ = 38257 − 0.1629 300000
*+,-. Extrapolation is the use of a
regression line for prediction far
/ = −$10,613
*+,-. outside the interval of x values used
to obtain the line. Such predictions
are often not accurate.

Extrapolation
/ = 38257 − 0.1629 ;,<.= >+,?.@

*+,-.
/ = 38257 − 0.1629 300000
*+,-. Extrapolation is the use of a
regression line for prediction far
/ = −$10,613
*+,-. outside the interval of x values used
to obtain the line. Such predictions
are often not accurate.
CAUTION:
Don’t make predictions using values of x that are much
larger or much smaller than those that actually appear in
your data.

Residuals
In most cases, no line will pass exactly through all the points in a scatterplot.
Because we use the line to predict y from x, the prediction errors we make are
errors in y, the vertical direction in the scatterplot. These vertical distances are
called residuals (the “leftover” variation in the response variable).

Residuals

Residuals
A residual is the
difference between the
actual value of y and the
value of y predicted by
the regression line.

Residuals
A residual is the
difference between the
actual value of y and the
value of y predicted by
the regression line.
+.=,>CD< = D-ECD< " − *+.>,-E.> "

= " − "#

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Calculate and interpret the residual for the truck that was driven 70,583 miles.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Find the predicted price.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-.
/ = 38257 − 0.1629 70583
*+,-.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-.
/ = 38257 − 0.1629 70583
*+,-.
/ = $26,759
*+,-.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Find the predicted price. Find the residual.

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-.
/ = 38257 − 0.1629 70583
*+,-.
/ = $26,759
*+,-.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-. /
+.=,>CD< = *+,-. − *+,-.
/ = 38257 − 0.1629 70583
*+,-.
/ = $26,759
*+,-.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-. /
+.=,>CD< = *+,-. − *+,-.
/ = 38257 − 0.1629 70583
*+,-. +.=,>CD< = FGHHI − FJKLH
/ = $26,759
*+,-.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-. /
+.=,>CD< = *+,-. − *+,-.
/ = 38257 − 0.1629 70583
/ = $26,759
*+,-. +.=,>CD< = −$4765

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-. /
+.=,>CD< = *+,-. − *+,-.
/ = 38257 − 0.1629 70583
/ = $26,759
*+,-. +.=,>CD< = −$4765
Interpret the residual.

Residuals
/ = 38257 − 0.1629 ;,<.= >+,?.@ .

/ = 38257 − 0.1629 ;,<.= >+,?.@
*+,-. /
+.=,>CD< = *+,-. − *+,-.
/ = 38257 − 0.1629 70583
/ = $26,759
*+,-. +.=,>CD< = −$4765
Interpret the residual.

The actual price of this truck is $4765 less than the cost
predicted by the regression line with x = miles driven.
Interpreting a Regression Line
A regression line is a model for the data, much like the density curves of
Chapter 2. The y intercept and slope of the regression line describe what
this model tells us about the relationship between the response variable y
and the explanatory variable x.

A regression line is a model for the data, much like the density curves of
Chapter 2. The y intercept and slope of the regression line describe what
this model tells us about the relationship between the response variable y
and the explanatory variable x.
In the regression equation "# = %& + %( ) :

• %& is the y intercept, the predicted value of y when x = 0
• %( is the slope, the amount by which the predicted value
of y changes when x increases by 1 unit

Recall that for a random sample of 16 used Ford F-150 SuperCrew 4 × 4s, the
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
regression equation is *+,-.
a) Interpret the slope of the regression line.
b) Does the value of the y intercept have meaning in this context? If so, interpret
the y intercept. If not, explain why.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Interpret the slope.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .

The predicted price of a used Ford F-150 goes down by $0.1629
(16.29 cents) for each additional mile that the truck has been driven.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .

Interpret the y intercept.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .


The predicted price (in dollars) of a used Ford F-150 that has been
driven 0 miles.
(The y intercept does have meaning in this case, as it is possible to have a
number of miles driven near 0 miles.)

/ = 38257 − 0.1629 ;,<.= >+,?.@ .


The predicted price (in dollars) of a used Ford F-150 that has been
driven 0 miles.
(The y intercept does have meaning in this case, as it is possible to have a
number of miles driven near 0 miles.)

The Least-Squares Regression Line
There are many different lines we could use to model the association in a
particular scatterplot. A good regression line makes the residuals as small as
possible. The regression line we prefer is the one that minimizes the sum of the
squared residuals.

squared residuals.

squared residuals.
The least-squares regression
line is the line that makes the
sum of the squared residuals
as small as possible.

Determining if a Linear Model Is Appropriate:
Residual Plots
One of the first principles of data analysis is to look for an overall pattern
and
for striking departures from the pattern. A regression line describes the
overall
pattern of a linear relationship between an explanatory variable and a
response
variable. We see departures from this pattern by looking at a residual plot.

Residual Plots
and
overall
A residual plot is a scatterplot that displays the residuals on the vertical axis
response
and the explanatory variable on the horizontal axis.

Residual Plots
and
overall
A residual plot is a scatterplot that displays the residuals on the vertical axis
response
and the explanatory variable on the horizontal axis.

Residual Plots
A residual plot magnifies the deviations of the points from the line,
making it easier to see unusual observations and patterns. If a regression
model is appropriate:
• The residual plot should show no obvious patterns.
• The residuals should be relatively small in size.

Residual Plots

Residual Plots
Pattern in residuals
Linear model not
appropriate

Residual Plots
How to Interpret a Residual Plot

To determine whether the regression model is appropriate, look at the
residual plot.
• If there is no leftover curved pattern in the residual plot, the regression
model is appropriate.
• If there is a leftover curved pattern in the residual plot, consider using a
regression model with a different form.

Residual Plots
Recall that for a random sample of 16 used Ford F-150 SuperCrew 4 × 4s, the least-
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
squares regression equation is *+,-.
For this model, technology produced the following residual plot.
Is a linear model appropriate for these data? Explain.

Residual Plots
Recall that for a random sample of 16 used Ford F-150 SuperCrew 4 × 4s, the least-
/ = 38257 − 0.1629 ;,<.= >+,?.@ .
squares regression equation is *+,-.
For this model, technology produced the following residual plot.
Is a linear model appropriate for these data? Explain.
Because there is no obvious

pattern left over in the
residual plot, the linear
model is appropriate.

How Well the Line Fits the Data:
The Role of s and r2 in Regression
To assess how well the line fits all the data, we need to consider the residuals for
each observation, not just one. Using these residuals, we can estimate the
“typical” prediction error when using the least-squares regression line.

To assess how well the line fits all the data, we need to consider the residuals for
each observation, not just one. Using these residuals, we can estimate the
“typical” prediction error when using the least-squares regression line.
The standard deviation of the residuals s

measures the size of a typical residual. That is,
s measures the typical distance between the
actual y values and the predicted y values.
s=
∑ residuals 2
=
∑ i
(y − ˆ
y ) 2
n −2 n −2

The standard deviation of the residuals s gives us a numerical estimate of the
average size of our prediction errors. There is another numerical quantity that tells
us how well the least-squares regression line predicts values of the response y.

The coefficient of determination r2 measures

the percent reduction in the sum of squared
residuals when using the least-squares
r 2 = 1−
∑ residuals 2
regression line to make predictions, rather
than the mean value of y. In other words, r2
∑ i
(y − y ) 2
measures the percent of the variability in the
response variable that is accounted for by the
least-squares regression line.

The coefficient of determination r2 measures

the percent reduction in the sum of squared
residuals when using the least-squares
r 2 = 1−
∑ residuals 2
regression line to make predictions, rather
than the mean value of y. In other words, r2
∑ i
(y − y ) 2
measures the percent of the variability in the
response variable that is accounted for by the
least-squares regression line.
r2 tells us how much better the LSRL does at predicting values of y than simply
guessing the mean y for each value in the dataset.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .
least-squares regression equation is *+,-.
For this model, technology gives s = $5740, and r2 = 0.66.
a) Interpret the value of s.
b) Interpret the value of r2.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Interpret s.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Interpret s.
The actual price of a Ford F-150 is typically about $5740 away from the
price predicted by the least-squares regression line with x = miles driven.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Interpret s.
Interpret r2.

/ = 38257 − 0.1629 ;,<.= >+,?.@ .
Interpret s.
Interpret r2.
About 66% of the variability in the price of a Ford F-150 is accounted for
by the least-squares regression line with x = miles driven.

Interpreting Computer Regression Output
A number of statistical software packages produce similar regression output.
Be sure you can locate
• the slope b1
• the y intercept b0
• the values of s
• the value of r2

• the slope b1
• the values of s
• the value of r2

• the slope b1
• the values of s
• the value of r2

Calculating the Regression Equation from
Summary Statistics
Using technology is often the most convenient way to find the equation of a
least-squares regression line. It is also possible to calculate the equation of the
least-squares regression line using only the means and standard deviations of the
two variables and their correlation.

Calculating the Regression Equation from
Summary Statistics
Using technology is often the most convenient way to find the equation of a
least-squares regression line. It is also possible to calculate the equation of the
least-squares regression line using only the means and standard deviations of the
two variables and their correlation.
How to Calculate the Least-squares Regression Line Using

Summary Statistics
We have data on an explanatory variable x and a response variable y for n
individuals. From the data, calculate the means )̅ and "O and the standard
deviations sx and sy of the two variables and their correlation r.
The least-squares regression line is the line "# = %& + %( )
QR
with slope %( = + P Q
S
and y intercept %& = "O − %( )̅

Regression to the Mean

The scatterplot shows height versus foot

length and the regression equation
"# = 103.34 + 2.75).
We have added four more lines to the
graph:
• a vertical line at the mean foot length x
• a vertical line at x + sx
• a horizontal line at the mean height y
• a horizontal line at y + sy


"# = 103.34 + 2.75).
graph:
For an increase of 1 standard deviation in

the value of the explanatory variable x, the
least-squares regression line predicts an
increase of r standard deviations in the
response variable y.

"# = 103.34 + 2.75).
graph:
For an increase of 1 standard deviation in This is called regression to

the value of the explanatory variable x, the the mean, because the
least-squares regression line predicts an values of y “regress” to their
increase of r standard deviations in the mean.
response variable y.
Correlation and Regression Wisdom
Correlation and regression are powerful tools for describing the relationship
between two variables. When you use these tools, you should be aware of
their
limitations.

their
limitations.
CORRELATION AND REGRESSION LINES DESCRIBE ONLY LINEAR RELATIONSHIPS

their
limitations.

their
limitations.
r = 0.816 r = 0.816
r = 0.816 r = 0.816

their
limitations.

their
limitations.
CORRELATION AND LEAST-SQUARES REGRESSION LINES ARE NOT RESISTANT

their
limitations.

their
limitations.

their
limitations.

their
limitations.
ASSOCIATION DOES NOT IMPLY CAUSATION

their
limitations.
When we study the relationship

between two variables, we often hope
to show that changes in the
explanatory variable cause changes in
the response variable.

their
limitations.
When we study the relationship

between two variables, we often hope
to show that changes in the
explanatory variable cause changes in
the response variable.
CAUTION:
A strong association between two variables is
not enough to draw conclusions about cause
and effect.

Section Summary
LEARNING TARGETS
After this section, you should be able to:
ü MAKE predictions using regression lines, keeping in mind the dangers
of extrapolation.
ü CALCULATE and interpret a residual.
ü INTERPRET the slope and y intercept of a regression line.
ü DETERMINE the equation of a least-squares regression line using
technology or computer output.
ü CONSTRUCT and INTERPRET residual plots to assess whether a
regression model is appropriate.

Section Summary
LEARNING TARGETS
After this section, you should be able to:
ü INTERPRET the standard deviation of the residuals and r2 and use these
values to assess how well a least-squares regression line models the
relationship between two variables.
ü DESCRIBE how the least-squares regression line, standard deviation of
the residuals, and r2 are influenced by outliers.
ü FIND the slope and y intercept of the least-squares regression line from
the means and standard deviations of x and y and their correlation.

Slides+on+Section+3.2+ (Least Squares+Regression)

Uploaded by

Copyright:

Available Formats

You might also like

Slides+on+Section+3.2+ (Least Squares+Regression)

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Slides+on+Section+3.2+ (Least Squares+Regression)

Uploaded by

Copyright:

Available Formats

Chapter

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

A regression line is a line that

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

/ = 38257 − 0.1629 ;,<.= >+,?.@

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

+.=,>CD< = D-ECD< " − *+.>,-E.> "

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics

Find the predicted price.

Starnes/Tabor, The Practice of Statistics

Find the predicted price.

Starnes/Tabor, The Practice of Statistics

Find the predicted price.

Starnes/Tabor, The Practice of Statistics

Find the predicted price.

Starnes/Tabor, The Practice of Statistics

Find the predicted price. Find the residual.

Starnes/Tabor, The Practice of Statistics

Find the predicted price. Find the residual.

Starnes/Tabor, The Practice of Statistics

Find the predicted price. Find the residual.

Starnes/Tabor, The Practice of Statistics

Find the predicted price. Find the residual.

Starnes/Tabor, The Practice of Statistics

Find the predicted price. Find the residual.

Interpret the residual.

Starnes/Tabor, The Practice of Statistics

Find the predicted price. Find the residual.

Interpret the residual.

Starnes/Tabor, The Practice of Statistics

In the regression equation "# = %& + %( ) :

Starnes/Tabor, The Practice of Statistics

Starnes/Tabor, The Practice of Statistics