CHAPTER 5

REGRESSION

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 4.1

Discovering Statistics Using SPSS © Andy Field 2005

 Please find out the mean, variance, standard
deviation of the two variables.

Then, calculate the covariance, and the r

and R square.

Discovering Statistics Using SPSS © Andy Field 2005

 We also talked about partial
correlation.
• Do you remember how to use SPSS to
calculate this and how to interpret this?

Discovering Statistics Using SPSS © Andy Field 2005

 Moving beyond Correlation
• Correlation is useful to tell us the
relationship about two variables, but it tells
us nothing about the predictive model to
our data and use that model to predict
values of the Dependent variable from one
or more independent variables.

Discovering Statistics Using SPSS © Andy Field 2005

 The method of least squares

• We need to find a “model” that has the

least “variances” and best fit the data.

• It means we need to find a straight line to

“describe” our data.

Discovering Statistics Using SPSS © Andy Field 2005

 A straight line…
1. The slope (or gradient) of the line; and
2. The point at which the line crosses the
vertical axis of the graph (known as the
intercept of the line).

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 7.1

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 7.2 – A Regression Line: a line that minimizes the sum of squared
differences.

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 5.3 - Goodness-of-fit: how “fit” is the line?

SSm = SSr - SSt

Discovering Statistics Using SPSS © Andy Field 2005

 SSm
• If the value of the SSm is larger, then the
regression model is very different from
using the mean to predict the dependent
variable.

• If the value of the SSm is small, then using

the regression model is little better than
using the mean as the model.
Discovering Statistics Using SPSS © Andy Field 2005
 R square
• It represents the amount of variance in the
outcome explianed by the SSm relative to
how much variation was to explain by the
SSt (mean).

• Thus, R square = SSm/SSt

Discovering Statistics Using SPSS © Andy Field 2005

 F ratio
• Is a measure of how much the model has
improved the prediction of the outcome
compared to the level of inaccuracy of the
model.
• A good model should have a large F-ratio.

Discovering Statistics Using SPSS © Andy Field 2005

 Class exercise – weekly records.

• R square = .335, which tells us that advertising expenditure can account for
33.5% of the variation in record sales.

• ANOVA test = F ratio = 99.587

• Beta = the change in the outcome associated with a unit change in the
predictor = if our independent variable is increased by 1 unit, the our model
predicts that 0.096 extra records will be sold.

• T-test = tests the null hypothesis that the value of beta is 0: therefore, if it is
significant we accept the hypothesis that the beta value is significantly
different from zero and that the predictor variable contributes significantly to
our ability to estimate values of the outcome.

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 5.4

Discovering Statistics Using SPSS © Andy Field 2005

 MULTIPLE REGRESSION

Discovering Statistics Using SPSS © Andy Field 2005

 • A logical extension of the simple
regression model to situations in which
there are several independent variables.

• We talked about regression LINE in a

simple regression model, now we are
talking about a regression PLANE

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 5.6

Discovering Statistics Using SPSS © Andy Field 2005

 Methods of regression
• Hierarchial (Blockwise Entry): based on
early research findings
• Forced Entry: all enter at once but based
on previous research
• Stepwise methods: exploratory

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 7.7 Outliers – Check Cook’s distance

Discovering Statistics Using SPSS © Andy Field 2005

 Figure 7.9

Discovering Statistics Using SPSS © Andy Field 2005