Inferential statistics for repeated measures designs: correlations

Dr Diane Dixon

Introduction
Hello, my name’s Diane Dixon and I would like to welcome you to the next session in
the research methods class. This session is on correlation. This is your first session
on inferential statistics for repeated measures designs. Now that sounds complicated,
but what does it mean? Well inferential comes from the word infer, so all inferential
statistics are, are tests that allow us to infer something from the data. They are tests
that allow us to conclude something from the data that we’ve collected, and repeated
measures is relatively self-explanatory. It just means that measurement is repeated
in the dataset. So, each person in your dataset has contributed more than one
measure to that dataset. So, if for example, we’re interested in the relationship
between depression and self-esteem, each person in that dataset will have
contributed a measure of depression, and a measure of self-esteem. So, inferential
statistics for repeated measures sounds complicated, but actually, it’s relatively
straightforward. So, in today’s session, we’re going to look at one type of inferential
test for repeated measures design, and that’s correlation. And what does correlation
allow us to infer from our data? Well, correlation simply allows us to test whether
two variables are related; that’s all it allows us to do. So, in this session on correlation
I’m going to introduce what correlation is. I’m going to spend some time speaking
quite in depth about something called a scatterplot, and what a scatterplot does is it
allows you to visualise your data; it allows you to look at your data; and it allows you
to actually see whether or how your variables are related, before you even start to
apply statistical analyses to your data. And I will be encouraging you always to start
by plotting the scatterplot if you’re interested in applying correlation to your data.
And because scatterplots are a visualisation technique, I’ll also introduce you to a
resource that’s been developed by the British Psychological Society which uses dance
to illustrate statistics, and in this case, dance to illustrate correlation. And that might
sound strange; it might not be something that you’re used to or familiar with, but it’s
a really useful resource, and students tell us in the feedback that they give that they
find it very helpful in enabling them to understand what correlation is, and how
statistics work. So, I’ll introduce you to that resource. We’ll then go onto look at
correlation causation. So, if you find that two variables that you’re interested in are
correlated – and significantly correlated – you can’t infer that there’s a causal
relationship between those variables. So, a change in one variable might not be
causally related or cause the change in another variable. So, we look at correlation
and causation, and we’ll do that in some depth. And then we’ll go to look at effect
sizes; we’ll move onto effect sizes, which allow you to get a measure of the strength of
the association of the relationship between the variables. And at that point, I’ll
introduce the equation for correlation, and we’ll do some work with that on how you
calculate effect sizes. And I know that students sometimes get quite anxious about
statistics, and I don’t want you to feel anxious. Just take your time and work through
the slides, and I’m sure you’ll be able to come to a full understanding of what
correlation is, and how you can use it to understand your data a little better. And I
hope you enjoy this session on correlation. Thank you.

Basics and scatterplots

So welcome to this PowerPoint presentation on correlation. In this session we’re
going to cover what correlation is, and we’re going to look in detail at scatterplots. So,
what is correlation? Well, correlation really is a very simple statistical test of whether
1
two variables are related. So, let’s start by taking a look at some data. So, over here
on the right-hand side of the slide you can see a dataset that lists height in
centimetres and weight in kilograms. So, that’s the height and weight of ten adults
who’re in this dataset. And I’m just going to ask you to look very simply at the data,
and to think about whether or not you would expect there to be a relationship
between height and weight. So, if you just look at the data, in general, do taller people
weigh more? So, let’s rearrange the data so we can see this more easily. So, over on
the right I’ve rearranged the dataset so it’s according to participant height, so shorter
people are listed at the top. And if we do this, it actually makes it easier to see if
there’s a relationship between height and weight. So, the very first person in this
dataset in 155cm tall, and they weigh 51kg, and if we just move down the height
column, you’ll see that the person that’s 175cm tall weighs 77kg. And the very tallest
person in the dataset, who’s 196cm tall, weighs 91kg. So, it seems to be that there’s a
relationship between height and weight; as people are taller, they weigh more. And
all that correlation does is it enables us to formally test this. So, it’s not just that there
seems to be a relationship, but it actually allows us to formally test whether two
things are related. In this case, is height related to weight? So, at this point I’d just
like to introduce you to a very unique view of correlation, and this is a resource that’s
been produced by the BPS, which is our professional organisation as psychologists.
And they’ve developed a really quite wonderful resource that demonstrates
correlation in a rather unique way, so I’d like to ask you to take a minute to watch
this YouTube film. And in the past, students have told us that they find it really
helpful in their learning, and it really helps support their understanding of what
correlation is. And you can see the link to this YouTube to this clip is given below; on
this slide, there’s a hyperlink to it.

Basics and scatterplots (part 2)

So, hopefully you will have enjoyed seeing the BPS’s YouTube dance of a correlation,
and that’s enabled you to get a really good understanding of just what correlation is.
So, in this continuing session, we’re going to continue to look at what correlation is,
and then we’re going to look in detail at scatterplots. So, correlation designs are a
type of repeated measures design, and correlation designs have two different
variables; not two levels of one variable. So, for example, height and weight; age and
theory and mind; depression and self-esteem. And each variable is measured on
either an ordinal, interval or ratio scale, and the type of data determines the type of
test that you will use to calculate the correlation between variables – and there’s a
presentation on that later. And you may talk about independent variables and
dependent variables if one variable is clearly the causal partner. So, for example, age
and theory of mind. But there is a full presentation on correlations and cause, and I
would ask you to be quite cautious about using IVs and DVs unless you’re very, very
clear that one variable is very clearly the causal partner. So, what types of correlation
are there? Well, a correlation can be positive, so as one variable increases, so too does
the other. So, the warmer the weather, the higher the sales of sun screen sales.
Students who study more get higher marks. But, correlation can also be negative, so
as one variable increases, the other decreases. So, greater stress at work is associated
with less happiness at home, and higher autistic trait scores are associated with lower
social acceptance. As well as type, correlation also has strength, so correlation can
range from -1 through 0 to +1. The letter r is used to denote the most frequently used
measure of correlation, which is the Pearson’s correlation. So, further from 0 –
positive or negative – the stronger the correlation is. So, now we’re going to start to
talk about scatterplots. So, in the example of the height and weight, we simply
2
rearranged the data table to be able to more easily judge whether there’s a
relationship between height and weight. But scatterplots are a much more effective
way of looking at the data in a correlational design. Scatterplots are the standard
descriptive statistics for correlational data. And a scatterplot is simply – it’s really
very straightforward – is a graph with one variable plotted on the X axis, and one
plotted on the Y axis. So, let’s take a look at a scatterplot for our height and weight
data. So, here’s the scatterplot for height versus weight. So, we’ve plotted height in
centimetres on the X axis, which runs from 150cm to 200cm. And weight on the Y
axis, in kilograms – which runs from 40 up to 100. And each person in our dataset
has got a point on this graph. If two people weighed exactly the same and were
exactly the same height, it would only show us a single point on the scatterplot. So,
sometimes if you count up all the spots on your scatterplots, you’ve got fewer than
you’ve got participants in your dataset. But that’s absolutely fine; it just means that
some people score exactly the same on the same variable. So, looking at our
scatterplot of height against weight, it shows a positive correlation, so as height
increases, so does weight. And we can add a trend line (i.e. the line of best fit between
height and weight). Positive correlations start at the bottom left corner of the
scatterplot and rise to the top right corner. Here’s an example of a negative
correlation. Negative correlations start at the top left of the scatterplot and fall
towards the bottom right. The scatterplot here shows a negative relationship between
self-esteem and depression, so as depression scores increase, self-esteem scores
decrease. There’s also another really quite frequently encountered type of scatterplot
and that’s when no correlation exists between the variables, so here the scatterplot is
showing data for two variables that are not correlated. Scatterplots have other useful
features. They can show non-linear relationships. So far, we’ve seen scatterplots that
show linear relationships, and no relationships between the variables. But linear
relationships simply mean you can draw a straight line through the data points.
Sometimes variables are related, but the relationship is curved; it’s more complicated
than a simple straight line. So, here we have an example of a non-linear relationship.
The scatterplot opposite shows a curved relationship between adrenaline and
performance. So, what’s this scatterplot showing us? Well it’s suggesting that, up to a
certain level, adrenaline is related to increase in performance, but if you add too
much adrenaline, then it impairs performance. So, there’s a kind of optimal level of
adrenaline in relation to performance. Now, if we tried to impose a linear correlation
on the adrenaline and performance data, it will likely show a 0 or non-significant
correlation, but there is an obvious relationship between the two. So, here’s a tip. It’s
slightly more than a tip, really; it’s a golden rule. Always draw a scatterplot of your
data, because seeing your data can be revealing. It’s always really useful to graph
your data and have a look at it before you start applying any complicated statistical
analyses. So, what else might scatterplots give us a clue about? Well, it gives us an
idea about relationship strength, so a stronger relationship (i.e. a stronger
correlation) will show up as data points placed more closely around the trend line,
which is the line of best fit. So, below are two scatterplots. The one on the left has a
correlation of .9; the one on the right has a correlation of .7. And if you look closely at
the data points on the left-hand graph, they’re generally closer to the trend line than
the data points on the graph on the right. The data points on the graph on the right
are more spread out. Always, as data points are closer to the trend line, then the
greater the strength of the correlation between the two variables. And what else
might scatterplots help us with? Well, they have the useful feature that they might
reveal the presence of outliers. And outliers are individual data points that obviously
do not find the general pattern. So, if we return to the scatterplot of height versus
3
weight, the person represented by the data point that’s circled in red, is far from the
trend line for all of the people in the study. So, this person is actually weighing more
than we would expect for a person of their height (i.e. they sit high above the trend
line). This data point is an outlier (i.e. it lies outside the general trend of the rest of
the data), and outliers can affect your correlation values, and you might consider
removing them from the dataset. Statistical software such as SPSS allows you to
identify significant outliers, and we’ve got a presentation around how do you use
SPSS to identify outliers. So, again, I will reemphasise the tip; always draw a
scatterplot of your data as it can be very helpful in you being able to analyse your
data appropriately. So, scatterplots are – for summary – they’re created by plotting
each participant’s scores for both variables. Each participant is represented by a dot
on the scatterplot. If you have an IV and a DV, put the IV on the X axis (i.e. along the
bottom), but if you don’t have an IV and DV, then the order doesn’t matter; doesn’t
matter which variable goes on each axis. This allows you to see whether a correlation
might be present, and if so, whether it’s likely to be positive or negative. It also tells
you something about how strong the relationship is likely to be between the two
variables. And importantly, it also tells us whether the relationship appears to be
linear – a straight line – and whether it’s more complicated than that, and whether
there might be outliers. So, again, always, always, always draw a scatterplot and look
at your data points before doing anything else. So, correlations; what do we know so
far? Well, they can be positive and negative. They can range from -1 through 0 to +1,
with values nearer to -1 and +1 representing the largest stronger correlations. A
correlation of 0 may indicate the absence of the linear relationship between the
variables, but the scatterplot should help you decide whether there’s some other non-
linear relationship (e.g. whether there’s a curved relationship between the variables).
And a scatterplot allows us to visually represent our correlation, and to check for
both linearity and outliers. And correlations can vary in strength; whether a
correlation is significant depends on the sample size.

Identifying outliers
So, just a little more on identifying outliers, because outliers can significantly impact
the correlation that we calculate for our data. So, how do I identify outliers? So, if you
take a look at the scatterplot below that shows a relationship between levels of
depression and productivity at work; how do you decide whether to remove the
possible outlier or not? Some people advocate making a qualitative judgement based
on the scatterplot. It’s relatively obvious that the person highlighted by the red circle
in the data shown on the slide is a probable outlier, but there is a much more
rigorous method that doesn’t just rely on our qualitative judgement. So, how do we
identify outliers more formally? Well, in SPSS you can see whether an individual case
has significant outliers or not. And to do this, you convert each participant score into
a z-score, and those which are greater than + and – 3.29 are outliers. So, why do we
use + and – 3.29 and not some other criterion? Well, if you remember back to the
session on normal distribution, about 5% of the scores on a normal distribution
should be above .196, 1% should be above 2.58, and none should be above 3.29. So,
3.29 is a really extreme score on a z-score. So, if your outlier is greater than + and –
3.29, you know that it’s a very extreme score, so you can use that as a criterion to
identify significant outliers. So, in SPSS, you go to the analyse menu, you choose
descriptive statistics, and you choose descriptive, and move over the variables that
you’re interested in, and tick the box asking SPSS to save standardised values as
variables. SPSS then creates, in this example, two new columns of data, which are the
z-scores for each participant on each measure; so, it gives a z-score for depression,
4
and a z-score for productivity. Now you simply need to look down the columns to
check for participants with a z-score that’s higher than 3.29 and lower than -3.29 for
either variable. In this case, you can see that we have one outlier that is participant
number three, who’s an outlier in relation to their depression score. So, they have a
very high depression score, relative to their productivity score. And this person could
be removed. So, we should remove the outlier before conducting our correlation (i.e.
delete that person’s data from our file). If we remove the outlier, the correlation value
increases from -.57 to -.84, so it’s worth removing that outlier in this case. It’s
important to know that outliers can actually increase a reported correlation, as well
as decreasing it. The point in removing them is to get a more accurate estimate of the
correlation at the population level. So, how do we go about removing outliers? Well,
there are two ways to do it. You can simply delete that case. In the example we just
looked at, that would be participant three. Or, you can use SPSS to select only those
cases where the z-scores are below a set level. And just one note of caution, if you’re
deleting the case, it’s really important to know that you only delete it for that
particular analysis, because if you delete it completely from your dataset, you’ll lose
all the data from that data. And it’s certainly likely that they’re only an outlier on one
particular analyses and not other analyses that you might wish to do with that
dataset.

Correlation and causation

Right, so, welcome to session two on correlation where we’re going to look in a little
more detail, and in a little bit more depth, at correlation and causation, and the kind
of relationship between the two. So, there’s a golden role of correlation. When two
variables are correlated, it does not – I repeat – mean that one variable causes the
other variable. For example, self-esteem and depression are correlated, but from a
correlation we cannot conclude that either depression causes self-esteem, or that
self-esteem causes depression. So, in big red letters, correlation does not imply
causation, and just sear that into your mind. So, students frequently make the
mistake of concluding that X causes Y, simply because X and Y are correlated, but
that’s not the case; we can’t make that conclusion. Many things are correlated that
are not causally related, and here are some examples. So, here are some quite silly,
spurious correlations. So, here’s one about the evils of ice cream. So, if you look at
the data on sales of ice cream, these sales are related to higher crime rates; so, when
ice cream is sold, crime rates are higher. Increased death rates due to drowning; so,
greater ice cream sales are related to increased death rates due to drowning. Ice
cream sales are also related to increased forest fires. So, I’ll ask you a question. Do
you think that ice cream causes forest fires, causes death by drowning and causes
crime? And if ice cream does cause these things, maybe we need to think about
banning ice cream. So, what do you think might be the explanation? Well, perhaps
there’s another factor involved, and this introduces us to what is called the third
variable problem. In this case, a third mystery variable is related to ice cream sales,
drowning, crime and forest fires. And that third variable is summer. So, children are
out of school, which might be related to crime rate, drowning and ice cream sales.
Higher outside temperatures, so forest fires, again, more ice cream, drowning; more
people tend to swim in the open in rivers and the sea when the weather is good. And
the crime rate; people are outdoors, so they are not in their houses, which make them
more vulnerable to crime. So, maybe we should ban summer? Of course, that doesn’t
apply to Glasgow because we never get one, and it’s certainly never hot enough to
cause forest fires – that’s never been known in Glasgow. So, here’s the third variable
problem in an example from psychology. So, any significant correlation has a number
5
of interpretations. We might have a Type 1 error; what does that mean? Go back and
think about what does Type 1 error mean. Or it might be that change in variable A
causes change in variable B. So, here’s an example from psychology. Parental discord
is related to child aggression, or it might be changes in variable A cause changes in
variable B. So, child aggression might cause parental discord. Or, it might be that
changes in variables A and B influence each other, so parental discord influences
child aggression, and child aggression influences parental discord. But, as we’re
emphasising in this session, correlation is not causation. So, here we have the third
variable problem. An omitted variable (variable C) accounts for the link between
variables A and B, so a father’s use of physical discipline affects parental discord and
child aggression, so that the relationship that we see between parental discord and
child aggression is really because both of these things are related to this common
factor; a father’s use of physical discipline. Another point to bear in mind when we’re
thinking about correlation: when might we be confident in saying that one variable
causes another variable to increase or decrease? So, what has to be present in our
study and our data to enable us to start thinking about whether or not we can infer
causal relationships? So, if one variable temporarily precedes the other; so, if one
variable comes before another in time. Though, even here it’s difficult to rule out a
third casual variable. A stronger feature would be if you manipulate one of the
variables in an experimental setting, so if you change one variable and then you can
see what impact that has on your second variable, so you’re using experimental
design – that’s a much stronger design if you want to infer a causal relationship
between two variables. Or, if one variable logically has to be the causal agent (e.g. age
in theory of mind scores). So, here are some resources which give you some more
examples of spurious correlations, just again to cement and underline the fact that
we can’t infer cause when we find a significant correlation between our variables.
Ionica Smeets gives a really good and very entertaining TED Talk on correlation and
cause, and I highly recommend this talk; she uses some really interesting examples of
correlation and causation, and she does it in a very sort of fun, but very intelligent,
way. And there’s a whole website given over to a weekly list of spurious correlations,
and again, the link to that is provided on the website.

Effect sizes
So, now we’re going to move on and look at effect sizes in correlations. So, just to
recap, correlations can be positive and negative. They can range from -1 through 0 to
+1, with values nearer to – and + 1 related to stronger correlations. Correlations of 0
do indicate the absence of a linear relationship between two variables, but there may
be other relationships present (e.g. curved relationships). And scatterplots can help
tell you whether there is some other non-linear relation present in your data.
Scatterplots allow us to visually represent our correlation, and to check for both
linearity and possible outliers. And there’s a need to check for statistical outliers by
converting row scores to z-scores; those beyond + and – 3.29 are problematic, and
my general advice would be to remove them from your dataset before carrying out
analyses which involve that variable. Effect sizes. Correlation coefficient r is actually
a measure of the effect size itself. However, because correlations are not on a ratio
scale, it’s difficult to easily compare them. A correlation of r = .6 is not twice as
strong as a correlation coefficient of .3. So, we need some way of being able to more
easily compare our correlation data, and effect sizes allow us to do this. So, a more
intuitive measure of effect size here is the coefficient of determination, and this
sounds important; kind of sounds difficult. But, it’s really quite straightforward; it’s
just the correlation multiplied by itself (i.e. r squared; r being the correlation value).
6
And this converts the value of r into a ratio scale, and we express this in terms of the
percentage of shared variation in scores, and I’ll tell you what that means. For
example, earlier we examined the correlation between productivity and depression,
and in that example, the correlation value r = -.84. Therefore, r squared – the
coefficient of determination – is -.84 times -.84 which = +.71, so what does that
mean? Well it means it tells us that 71% of the variation in productivity scores can be
accounted for by variation in depression scores, and vice versa. So, variability in
depression – 71% of the variation in depression – can be accounted for by the
variation in the productivity score. This leaves 29% unaccounted for, and this
variation is due to either random error and other things that we haven’t measured.
So, that’s all the coefficient determination is. It’s the proportion of variance that can
be explained by the relationship between the two variables in your correlation. So, if
we look at a graph to better illustrate why we need to calculate the coefficient of
determination and the effect size in correlation. So, what we’ve got here is a graph of
correlation values against r squared and this graph shows why a correlation of .8 is
not twice as strong as the correlation of .4. So, if you look at the correlation value
of .8 on the X axis and follow the red lines across to the corresponding value of R
squared on the Y axis, you can see that a correlation of .8 accounts for 64% of the
variation between scores. And if we look at a correlation of .4, so if you find .4 on the
X axis, and again follow the red arrows across to see the corresponding value of R
squared on the Y axis, a correlation of .4 accounts for 16% of the variation between
scores. So, a correlation of .8 accounts for four times as much variation in scores than
a correlation of .4. So, this tells us why it’s important to calculate effect size when
we’re thinking about correlations. And if you want to discuss r squared as an effect
size, there are some guidelines to help you do this. So, if you calculate r squared and
it falls between .01 and .08, then this is accepted as a small effect. If your r squared
values falls between .09 to .24, this is accepted as a medium effect. An r squared
value of .25 and above is a large effect and remember an effect size of .25 reflects a
correlation of .5.

Which correlation test to use

So, now we’ll move on to think about which correlation test to use, and we’re going to
look at two types of tests. A test for parametric data, and a test that you can use for
non-parametric data. So, which type of correlation test should you use? Well, there’s
more than one type, and the type of data that you’re working with will determine
which correlation test you’ll use. So, the two frequently used correlation tests are the
Pearson’s correlation coefficient which is used for parametric data, and there’s the
Spearman’s Rho which is used for non-parametric data. So, why are we interested in
parametric statistics? Well, they’re more powerful than non-parametric statistics,
which means they’re more likely to detect a significant effect, but your data need to
meet certain criteria. So, parametric statistics make certain assumptions about your
data, so your data need to be either interval or ratio level. They need to be normally
distributed; so, they can’t be skewed, and you have to have at least ten participants in
each level of your dependent variable. There are some more assumptions for a
between groups design called homogeneity of variance, but that will come later in the
course on research methods. So, if your data match all of the above, then you can
calculate a Pearson’s product moment correlation which, thankfully, is usually just
termed Pearson’s. When you can’t satisfy all the requirements laid out above, you
should instead use an appropriate non-parametric analysis. So, if it’s important to
have normally distributed data, how do you know if your data are skewed? Well, you
can calculate skew, and to calculate skew in a sample of interval or ratio data, you
7
calculate the mean, you calculate the standard deviation of the mean of your data,
and then you calculate the median – and you would do this for each of your variables.
And then, you find out what half a standard deviation is, so you just half the standard
deviation, and then you check whether the mean and the median are more than half
a standard deviation apart. And if they are, the data is skewed. Remember, for a
perfectly normally distributed dataset, the mean and median would be exactly the
same. In this case, you’re looking for skew, so if you’re more than half a standard
deviation apart, then they’re skewed. And if your data is skewed, you need to use
non-parametric statistics. So, to summarise, which correlation tests should be used?
So, you use the parametric Pearson’s correlation when both your variables are on
either an interview or ratio scale, you have more than ten participants and neither
variable is skewed. You use the non-parametric Spearman rank order correlation –
referred to as Rho – when one or both of your variables is on an ordinal scale, and/or
you have fewer than ten participants, and/or one of your variables skewed when your
data are on an interval or ratio scale. So, now you’ve selected which scale to use and
you’ve carried out your test, how do you report correlational data? So, this is
standard format. SPSS, for example, produces a table like this. So, here we’re looking
at the correlation between self-esteem and depression, and this data table is telling
us that the correlation between self-esteem and depression is -.875. You’ll see that it
also produces the correlation between self-esteem in itself, and depression in itself,
and these values are always one. So, on the diagonal in an SPSS correlation table, the
correlation is always one because the variable is being correlated with itself. To
report the results, if you have this correlation table in your analysis, you would say
something like this: Pearson’s correlation revealed a significant, negative correlation
between self-esteem and depression, r = -.88 – which we’ve got from rounding up
-.875 – n = 54, p is less than or equal to .001, two tailed – because you’ve carried out
a two-tailed test. We’ll talk about two-tailed tests in another presentation. And this
represents a large effect size: r squared = .77. So, what about Spearman’s Rho? Well,
Spearman’s Rho is a non-parametric correlation statistic. It requires at least ordinal
level data, but the distribution is unimportant, so if you have interval or ratio level
data which is skewed, you can analyse those data using Spearman’s Rho. The
calculations for Rho are based on ranks of scores, rather than actual scores. It
actually uses the same formula as Pearson’s. So, ranks is just you just rank the data,
so for example, if a student performs first in the class for the first assessment, we
might expect them to be ranked near the top in the second assessment. An output
from Spearman’s Rho for SPSS (lecture slides). So, here a teacher ranks ten students
according to how friendly they are, and also how popular they are. And here, we have
a correlation table. And we can see that the correlation between friendliness and
popularity is .939. So, how would we report that? It looks really similar as the output
from SPSS for Pearson’s product moment correlation, and the standard format is
also identical. So, you’re reporting our ‘s’ small subscript; ‘s’ denotes it’s Spearman’s
Rho. It’s .94, so we’ve rounded up .939; n = 10 and p is less than .001 (two-tailed).
And note that there is no effect size for Spearman’s Rho; it’s a non-parametric test –
it doesn’t have an effect size. So, just a note about reporting p values. There’s only
really two ways that you can report a p value. You can go p = .XXX, or you can put
the exact significant level that SPSS gives you, for example r = .43, n = 19, p = .032
(two-tailed). Or, you can write it in terms of p is less than .001. When the p value
reported by SPSS is .000, then you report this as r - .63, n = 37, p is less than .001.
So, when you get an output from SPSS that’s giving you a significance level of .000,
you always report that as p is less than .001.

8
One versus two-tailed tests
So, now we’re going to look at one and two-tailed tests, and what do they mean? This
relates to hypothesis testing. Hypothesis testing with correlation. So, what’s the null
hypothesis for a correlation? Very straightforward; that there’s no correlation in the
population. But, what’s the alternative hypothesis for a correlation? Well, it could be
any one of three things. That there is a significant correlation between your two
variables; that there’s a significant positive correlation between your two variables;
that there’s a significant negative correlation between your two variables. So, if you’re
saying that there’s a positive correlation or a negative correlation you’re making a
directional hypothesis in the relationship between your two variables. You’re saying
which direction the relationship will lie in. So, if you recall that an alternative
hypothesis can actually take two separate forms; it can be two-tailed – there is a
significant correlation between the two variables – or it can be one-tailed – there is a
significant positive or negative correlation between the two variables in the
population. In both instances, a significant correlation is represented by an r value,
which is likely to occur less than one time in twenty if the null hypothesis were true.
This is the same as saying the correlation we have recorded is likely to occur by
chance less than 5% of the time (i.e. p is less than .05, if the null hypothesis is true).
And this has important implications in terms of your accepting your correlation to be
significant or not. So, let’s take a look at a two-tailed hypothesis. So, this is what a
two-tailed hypothesis relates to with respect to the r test. So, here we’ve got a normal
distribution curve for correlation. So, the mean is 0 and the distribution; the
maximum is r = 1 and the minimum is r = -1. And in our value, which is large enough
to occur by chance only 5% of the time will be significant, however that 5% must be
evenly split between both ends of the distribution, because we’re saying that the
relationship between the two variables can neither be positive or negative, so we’ve
got to take both ends of the distribution. So, that’s the extreme 2.5% of each tail. In
contrast if we look at a one-tailed test, this is what it relates to with respect to the r
test. In here, the r value which is large enough to occur by chance only 5% of the time
will be significant; just the same as in the two-tailed test. However, in this case, the
5% is only located at one end of the distribution. So, in the graph in this example
(lecture slide), this will be that we have hypothesised that there will be a positive
correlation between the two variables, so our 5% sits at the end of the distribution,
where r =1. So, were testing a directional hypothesis. If we’d hypothesised that the
variables had a negative correlation, then that 5% would sit at the other extreme end
of the distribution curve. So, should you make a one or a two-tailed hypothesis? Well,
research has usually preferred to make one-tailed hypotheses because they’re testing
specific theoretical positions. However, you might make a one-tailed hypothesis if
you’ve got no a priori reason for being able to postulate in what direction your
variables are related. But if you do this, you can only accept as significant any
difference which is in the direction you expect. So, if you predict a positive
correlation, but you actually find a negative correlation, you cannot accept that as
being significant, no matter how large it is. Right, so you must decide on your
hypothesis before looking at your data. Construction of a hypothesis to fit results is
considered academic dishonesty, so you have to make an a priori hypothesis and
stick to it. So, you can just make a two-tailed hypothesis because you think it’s more
likely that you’re going to get a more significant finding, because the 5% is not
distributed at both ends of the curve. You can only make a one-railed hypothesis if
you’ve got a reason for doing so, and if you make a one-tailed hypothesis and you
don’t actually find that – you find something different that’s significant – you can’t
accept that significance. You have to stick to your hypothesis. Now students worry
9
that if they don’t get a significant result then they’ve done something wrong, but
that’s not the case. Failure to support an alternative hypothesis does not constitute
scientific failure. You’re still advancing science and non-effects are just as important
for knowledge as significant effects, so don’t chase significant effects – it’s academic
dishonesty. Make a hypothesis, test your hypothesis, and stick to your analysis; the
results that you get. Now, if you look at one-tailed and two-tailed, SPSS gives you the
option of choosing either a one-tailed or a two-tailed correlation. So, if you go to
analyse, select correlate, and select bivariate option. Move your two variables of
interest into the variables box. It actually lets you choose it so just lower down the
screen under the test of significance, you can select either a one or a two-tailed
hypothesis. And that should be in line with what you’ve actually hypothesised in
relation to your data. So, what does the output look like? Well, here what we’ve done
is I’ve applied a two-tailed and a one-tailed analysis to the same dataset, just to
illustrate what the results would look like. And here we can see there are two
correlation tables; the top is a two-tailed hypothesis and the bottom correlations are
for a one-tailed hypothesis. Now in both cases, the correlation coefficient is the same,
minus .198, but .186 – it’s the same. However, it’s only when we use the one-tailed
test that the correlation is significant, so in the two-tailed test, p is .056. In a one-
tailed test, p is .028. But, we could only accept the one-tailed test if we’d made an a
priori hypothesis that these two variables were negatively correlated to each other.
So, it’s important that you get your hypotheses set before you start looking at your
data, and you should stick with hypothesis testing.

Correlation and reliability

Correlation can be used for many things and it plays an important role in
establishing the reliability of the measures we have, so in this presentation we’re
going to look at correlation and reliability. So, correlation can be used to establish
whether the tests that we use, the measures that we use in psychology are reliable.
And correlation can be used to examine a number of different types of reliability, and
we’re going to look at three: test/retest reliability, split-half reliability and internal
reliability. So, text/retest reliability is used to establish the similarity of responses
over a set period of time (i.e. the temporal stability of the test; how stable is the test
over time; are the results of a test consistent over time). To do this, you would
administer the same test to the same sample of participants at two different times
(e.g. a day apart; a month apart) and then you correlate the scores on the two sets of
responses. If it’s a stable and reliable measure, then the two sets of scores should be
highly correlated. And this is reported using the same standard format as normal
correlation, but no need for effect size. There is a note of caution, here. This only
really pertains to measures of constructs that we would expect to be stable. So, if the
construct or psychological phenomena that you’re measuring is likely to vary over
time or in response to other factors then you might not expect the test to show
test/retest reliability. You would use another method of establishing the reliability of
your test under such circumstances. Next, we’ve got split-half reliability, and this test
simply randomly splits the measure into two. So, it adds up the test scores of a
random sample of half your test items, and the scores of the other half of your test
items, and then you correlate the two scores. So, basically you split a measure in two,
but you do it randomly. If both halves are measuring the same construct, then they
should be highly correlated. In this case, the standard reporting form is r with the
subscript sb = X, whatever your correlation value is. And the final sort of reliability
we’re going to look at is Cronbach’s alpha, which is a very frequently used test of
internal reliability. And you’re likely to be using Cronbach’s alpha during your
10
studies. So, you will frequently report the Cronbach’s alpha for the test that you use.
It represents the average degree to which each item in a measure is correlated with
all other items in a scale or measure. Ideally, Cronbach’s alpha should be above .7 in
order to indicate a reliable scale, or a scale with internal reliability. And you should
always report this information in your method section at the time you describe the
scale it relates to. So, for example, the children’s anxiety inventory consists of ten
items. Participants responded by indicating their agreement with each item on a one
to five Likert scale, where one represents totally disagree and five represents totally
agree. The Cronbach’s alpha for the scale was .78, indicating satisfactory reliability.

11