Chapter 8

EXAMINING CAUSAL RELATIONSHIPS AMONG YOUR VARIABLES

Change sample to local issues (e.g. Increase in price of Onion, glitch in aero system on
Jan. 1)

Introduction to Experimental Design

When the singer Adele won six Grammy Awards in 2012, The Wall Street Journal and
National Public Radio (NPR) ran stories about why many people seem to have such a
strong emotional reaction to her music, and her song “Someone Like You” in particular.
One theory is that the song includes a type of note, called an appoggiatura, which begins
as discordant with the melody but then changes to align with the melody. The
discordance is unexpected and is believed to cause tension to the listener, which is then
released when the note changes to match the melody. This tension and release is
theorized to cause an emotional response, including chills running up the spine and
possible even crying. As researchers, how might we test if Adele’s song causes an
emotional reaction, and that the presence of the appoggiatura is the causative factor?

uh·paa·juh·tur·uh
- a grace note performed before a note of the melody and falling on the beat.

 TESTING CAUSE AND EFFECT

If there is a causal relationship between two variables, there is also a correlational

relationship. In other words, a change in one of the variables corresponds with changes
in the other. If a musical note can cause an emotional response, then we would find a
significant correlation between songs that have these musical notes and listener’s reports
of distinct emotions. Doucleff (2012), the author of the Wall Street Journal article about
Adele’s song, cites such a study conducted by psychologist John Sloboda in the 1990s.
Participants identified song passages to which they had a strong reaction, and it turned
out that the majority of the passages contained the appoggiatura note. However, as we
discussed in Chapter 7, finding a correlation between two variables is not enough to
demonstrate the changes in one of the variables caused the changes in the other
Correlation is one requirement of causation but alone is insufficient (see Figure 8.1).

IN THIS CHAPTER, YOU WILL LEARN

 The key components necessary to test cause and effect
 How to design an experiment
 How to manipulate an independent variable (IV)
 How to measure the dependent variable (DV)
 How to balance internal and external validity in an experiment
 Limitations to experimental designs

Another requirement of causation is sequencing. If you want to demonstrate that

changes in one variable (which we will call variable A) caused changes in another
variable (variable B), then the changes in variable A must occur before the changes in
variable B. If we want to demonstrate that the musical note makes people have chills or
cry, then exposure to the note must occur prior to any emotional response. But even with
both correlation and sequencing, we do not have sufficient evidence that one variable
caused a change in another.

The final requirement of causation is ruling out alternative explanations for why
the change in variable B both correspond with and came after the changes in
variable A. It may just be a coincidence, or the change may be due to some other
variable that the researcher had not tested. Although people may report feeling emotional
after listening to Adele’s song “Someone Like You”, could it be due to a reason other
than the presence of a specific type of musical note? The co-writer of “Someone Like
You” gave such an alternative explanation. He suggested that the lyrics paint a vivid
picture allowing listeners to imagine the experiences portrayed in the song (NPR Staff,
2012). Another explanation is that they may be differences in the people who choose to
listen to that song, and those individual differences may be causing the reaction rather
than the notes or the lyrics.

Figure 8.1 A Clear Misunderstanding of Cause and Effect

Move the deer crossing to where there’s less traffic

A lot of deer get hit by cars west of Crown, Point on U.S. 231. There are
too many cars to have the deer crossing here. The deer crossing sign needs to
be moved to a road with less traffic.

Review of Key Concepts: Validity

Can you recall what type of validity is under examination when we wonder if one
variable caused the change in the other variable? In other words, this type of validity
refers to the extent to which we can rule out alternative explanations of causality.

If you answered “internal validity” you are correct! If you have forgotten about
internal validity, take a minute review Chapter 3 before reading on.

We discussed internal validity in Chapter 3, and this concept is very important in

experimental design. You might also recall from Chapter 3 that confounds are
variables that are not the focus of the study, and if not controlled might impact the
study’s results. In other words, confounds are potential alternative explanations
that limit the internal validity of a study and must be addressed in order to
demonstrate causality.

In summary, the criteria to demonstrate that variable A caused a change in

variable B is: (provide sample on each criteria)

 Correlation: There must be a relationship between A and B.

  Sequence: The change in variable A must come before the change in
variable B.
 Ruling Out Alternative Explanations: The researcher controlled for
possible confounds so that variable A must be the only factor that could
have caused the change in variable B.

PRACTICE 8.1
TESTING CAUSE AND EFFECT

A USA Today article reported that the United States saw a slight increase in
pedestrian fatalities in 2010. One reason cited for this increase was the increased
use of cell phones and mobile music devices. The chairman of the Governors
Highway Safety Association was quoted as saying, “Anyone who travels in a busy
city has seen countless pedestrians engrossed in conversation or listening to music
while crossing a busy street. Just as drivers need to focus on driving safety,
pedestrians need to focus on walking safety” (Copeland, 2011)

Is this sufficient evidence that pedestrians listening to music and talking or texting
on a cell phone increase the risk of a fatal traffic injury?

 THREATS TO INTERNAL VALIDITY

Campbell and Stanley (1963) identified eight confounds that are key threats to internal
validity. Gliner and Morgan (2000) divided these into two categories, based on whether
the threat was due to experiences or environmental factors, or if the threat was due to
participant (or subject) characteristics.

Threats to internal validity due to experiences or environmental factors:

1. History: Any event or environmental condition other than variable A caused the
change in variable B.
2. Maturation: The change in variable B was due to natural changes that occur over
time.
3. Testing: Observed changes in variable B were due to previous exposure to a test.
4. Instrumentation: Observed changes in variable B were due to inconsistency in the
measurement instrument, administrators, or scorers.

Threats to internal validity due to participant characteristics:

1. Statistical Regression: Observed changes in variable B were due to a statistical
phenomenon in which very high or low scores will regress to the mean, meaning
that extreme scores will get less extreme over time.
2. Attrition (or Mortality): The change in variable B was due to participants
withdrawing from the study.
3. Selection: When comparing groups, the change in variable B was due to pre-
existing differences.
4. Selection Interactions: The change in variable B was due to an interaction
between the pre-existing differences and another threat to internal validity.

Internal validity: The extent to which we can say that one variable caused a
change in another variable.

Confound*: A variable that varies systematically with the variables of interest in

a study and is a potential alternative explanation for causality.
*mix up (something) with something else so that the individual elements become difficult
to distinguish

Threats to internal validity. Confounds that must be controlled so that a cause-

effect relationship can be demonstrated. Campbell and Stanley (1963) identified
the threats of a (history), (b) maturation, (c) testing, (d) instrumentation, (‘e)
statistical regression, (f) selection, (g) mortality, and (h) selection interactions.

Why the One-Group Pretest-Posttest Design Does not Demonstrate Causality

When trying to demonstrate that variable A caused a change in variable B. you might
first consider the one-group pretest-posttest design. In this type of design, the
researcher gives all participants a pretest to assess variable B, then exposes them to
variable A., and then gives them a posttest to assess variable B (see Figure 8.2). The
impact of variable A is determined by comparing each participant’s posttest score to
the pretest score and then finding the central tendency (e.g. mean) of those change
scores.

For our song example, a one-group pretest-posttest design might involve first asking
participants how they feel, then having them listen to Adele’s song, and then again
asking them how they feel. If there is a difference between the participants’ mood
prior to the song (pretest) and after listing to the song (posttest), can we say that
Adele’s song caused a change in emotion?

You are right if you said that no, we have not met all the requirements for causality.
Although we have satisfied the correlation and sequence criteria, we have not ruled
out alternative explanations for the change in emotions. In fact, simply comparing
scores from pretest to posttest among participants of one group can never satisfy that
third criterion because there are many alternative explanations for why the
participants would have changed between pre- and posttest. Review Campbell and
Stanley’s (1963) threats to internal validity and see if you can identify the threats that
would or might impact the one-group pretest-posttest design.
Threats to Internal Validity in One-Group Pretest-Posttest Designs Due to
Experiences or Environmental Factors

1. History: Any event that occurs or environmental condition that is experienced

between pretest and posttest might be the reason for the change. The threat
increases the longer the time lapse between the pretest and posttest. A highly
controlled laboratory setting can decrease, although not eliminate this threat.

Example: If our participant were listening to Adele’s song and the fire alarm went
off, or someone in the room started crying, or the temperature changed, or a very
attractive came into the room, or some other unforeseen event occurred, it could
be that the unplanned event caused the change in emotion, not the musical note
under study.

Figure 8.2 One-Group Pretest-Posttest Design

All Participants: Pretest on variable B – Exposure to variable A – Posttest on

variable B

2. Maturation: Any changes within the participants between the pretest and posttest
might be the reason for the change. This threat is increased if the study takes place
over a long period of time.

Example: Changes in emotions before and after listening to Adele’s song may be
due to growing tired, getting hungry, or natural changes in emotions over time.

3. Testing: Giving a pretest might impact posttest scores.

Example: Asking participants how they feel prior to Adele’s song might make
them think about their emotional state and consequently change it. Participants
would also be clued into the purpose of the study, guess what your are expecting
to find, and might provide answers based on those expectations.

4. Instrumentation: This threat would occur if a researcher uses different measures,

administrators, or scores at pretest that at posttest. This threat is also present if the
same measure is used but it has poor test-retest reliability (it is inconsistent over
time).

Example: For pretest, we ask participants to rate how emotionally aroused they
are on a scale from 1-5 but for posttest we ask them to rate their arousal on a scale
from 1-10. Or, one person administers the pretest and a different person
administers the posttest. Such changes in instrumentation may be why we
observed a change in emotions.

Threats to Internal Validity in One-Group Pretest-Posttest Designs Due to

Participant Characteristics
 1. Statistical Regression: If a researcher recruits participants based on extreme
scores, their pretest scores will regress to the mean at posttest.

Example: If we purposely recruited very lethargic individuals to determine if

Adele’s song could stimulate them, increase emotional arousal from pre-to
posttest would likely be due to statistical regression.

The other threats to internal validity are controlled in a one-group pretest-posttest

design. As long as the researcher only analyzes the date for those with both pre-
and posttest data, attrition does not impact the internal validity of the study. The
attrition may still affect the results if those who drop out are systematically
different than those who stay, but in one-group pretest-posttest design attrition
affects external validity instead of internal validity. Likewise, selection and the
selection interactions may impact the one-group pretest-posttest design’s external
validity but is not an issue for its internal validity.

One-group pretest-posttest design: Non-experimental design in which all

participants are tested prior to exposure to a variable of interest and again after
exposure.

Group Designs

Hopefully you are now convinced that a one-group pretest-posttest design cannot
demonstrate causality. If you are thinking that we might be able to improve on this design
by using a group design in which we compare two or more groups, then you a on the
right track. We might divide participants into two groups and have one group listen to
Adele’s “Someone Like You: and have the other group sit in silence. We could
administer a pretest and posttest to both groups and compare the change scores between
the groups (a two-group pretest-posttest design). Alternatively, we could omit the pretest
and compare the posttest scores between the groups (a two-group posttest-only design)
(see Figure 8.3).

Figure 8.3 Example Group Designs

Two-Group Pretest-Posttest Design

Group 1: Pretest on variable B – Exposure to one level of variable A – Posttest on

variable B

Group 2: Pretest on variable B – Exposure to different level of variable A –

Posttest on variable B
 Two-Group Posttest-Only Design

Group 1: Exposure to one level of variable A – Posttest on variable B

Group 2: Exposure to different level of variable A – Posttest on variable B

In a group design, the researcher wants to demonstrate that the difference between
the groups is due to exposure to different levels of variable A. To do that, the
researcher must rule out that other systematic differences between the groups
could have caused the groups to differ.