lOMoARcPSD|9327020

Lab Activity #4: Experiments (KEY)

Part 1: Experiments

We encourage you to work with a partner as you answer the following questions. These questions are based on an
online game that we would like for you to explore. Before you answer the first set of questions below, please go
to our course website. In the Lab Activities section, you’ll see a link that says “Word Game.” Click on that link,
and then click on the link that says “Play the original game.” Note that you can also get to this site by going to
the direct url (http://mathstatic.co.nz/experiment-lab) and clicking the link to “Play the original game.”

Try to play the game at least one or two times so you can see what is involved. Essentially, you are given a word
(see the example below), and, as quickly as possible, you need to click on the circles (or “dots”) in the order in
which the letters appear in the word (i.e., you need to spell the word by clicking on the dots).

Now, imagine that you learn about an experiment that was conducted with this game. The experiment is
described below. Please read this description carefully and answer the following questions.

In our experiment, a group of 100 students was randomly selected from STAT 1350. All of these
students had to play the word game that you just played, and all of them were given the same word (the
word given was “relationship”). Half of these students were randomly assigned to play a version of the
game in which the background of the game was a bright red color, while the other half of the students
were randomly assigned to play a version of the game in which the background was a light grey color.
The length of time taken to spell the word (in seconds) was recorded for each student. The goal of the
experiment was to see if background color affects the time taken to play the game.

1. What is the explanatory variable in this particular experiment? How do you know?

The explanatory variable is the color of the background. We know this because it’s the variable the
experimenter manipulated. It’s the variable the experimenter believes will affect the outcome of the
experiment, which is the time to spell the word.
 lOMoARcPSD|9327020

2. What is the response variable in this particular experiment? How do you know?

Time to spell the word was the response variable since this is the outcome of the experiment. We believe
this depends on our manipulation.

3. Who were the individuals in this particular experiment?

The individuals are the students.

4. Why would the researcher want all students to spell the same word?

This is a way of controlling a possible lurking variable. If the words were different, we wouldn’t know if
it was the word that affected the time or the color of the background (or maybe both).

5. Based on the brief amount information you were given about the experiment, discuss how you believe the
experiment adheres to the three principles of experimental design.

a. Control: The word spelled was the same for each group, and our hope is that the subjects were
tested in similar conditions (e.g., same kind of background noise, using comparable computers,
with the same researcher present, etc.). We don’t know much about the testing conditions, so
here, we can only speculate. Random assignment will hopefully allow us to “mix” up possible
lurking variables within each group so that the groups are starting off being as similar as possible.

b. Randomize: The subjects were randomly selected and randomly assigned to groups.

c. How many subjects were used? 100 total (split into two groups of 50) Do you think enough
subjects were used? Please explain.

It seems like enough subjects were used. We clearly had several individuals go through the
experiment, and we know that the larger the sample size, the closer our sample gets to mirroring
what is in the population (provided the sample is representative of the population, of course).

6. Suppose you learn later on that the speed in which the dots moved around in the game (i.e.., the dots the
player has to click to spell the word) was not the same for the different colored backgrounds. The dots
actually moved much more quickly with the bright red backgrounds compared to the light grey
backgrounds. If this is the case, we would consider “dot speed” to be a lurking (or confounding)
variable.

2
 lOMoARcPSD|9327020

Part 2: Designing your Own Game

Go back to the link for the Word Game (http://mathstatic.co.nz/experiment-lab), but rather than choosing to “Play
the original game,” choose the option to “Create your version of the game.” Answer the questions below as you
explore that part of the Word Game website.

7. One of the “Measures to record and display” is Number of clicks. What type of variable would this be?
Circle your answer below.
QUANTITATIVE VARIABLE

8. You can change the size of the Ball options and choose the balls to be Very small, Small, Medium,
Large, or Vary Large. What type of variable would Ball options be? Circle your answer below.

CATEGORICAL VARIABLE

9. Imagine a new game is designed in which the explanatory variable is the Case of the word (uppercase or
lowercase) and the response variable is the Time to first click. We randomly assign students to be in
either the uppercase or the lowercase group, and, for each student, we measure the time to first click. The
students who participate in this experiment are randomly sampled from all of our current STAT 1350
students.
a. Why would this be an experiment an example of a “completely randomized design”?

We randomly assigned students to groups and we want to compare the groups.

b. Why would it be important to randomly assign participants (or subjects) to different groups?
This should create groups of individuals who are similar at the start of the study, in terms of
possible lurking variables that might affect the response. If the groups start off being as similar as
possible, our hope is that the only thing that will end up differing between the groups is the
treatment we have applied, and this can help us better assess if the treatment is having an impact
on the measured response.

c. Do you think we can generalize the results of the experiment beyond our sample? Why or why
not?

To be able to generalize, we need a sample that is representative of the population of interest. If

our population of interest is all STAT 1350 students, we can generalize because we obtained a

3
 lOMoARcPSD|9327020

random sample from this population. We know that a sample that is randomly selected from the
population of interest can allow us to make generalizations back to the population.

d. Is the experiment double-blind? Please explain.

No. It would appear that subjects would know if the words are presented in uppercase or
lowercase, and there is nothing in the problem to indicate that the experimenter would not know
who is in which group.

Part 3: What went wrong?

Each of the following problems contains some kind of error, or something that went wrong in the interpretation of
the results.

10. Suppose you conduct an experiment to see whether or not caffeine helps students on a midterm exam.
You randomly assign half of the students in your sample to drink caffeine on the morning of the exam;
the other half of the students are randomly assigned to drink a non-caffeinated coffee-flavored beverage
on the morning of the exam. One of your classmates argues that you cannot draw a cause-and-effect
conclusion because you didn’t control for what the students ate for breakfast that morning. Why is this
argument flawed?

This student has not trusted the randomization. Because the subjects were randomly assigned to
the two groups, the overall food consumption of the two groups should look similar. It may not be
the same for every individual, but both groups should have a similar distribution of students who
skipped breakfast, students who had a light breakfast and students who had a hearty breakfast.
Since this variable will look similar across the two groups the intake of caffeine should be the key
difference between them. Therefore, if you do see a difference later between the two groups, you
will have evidence that caffeine is the cause.

11. Many early studies on the relationship between smoking and lung cancer found that smokers were about
13 times more likely to die from lung cancer than non-smokers. Still, people argued against a cause-and-
effect conclusion, citing numerous possible confounding (or lurking) variables. Suppose a student argues
that these studies are not convincing evidence because the researchers did not record the diets of the
individuals. Explain why this argument is incomplete.

The student has identified a good possible confounding variable but has not yet explained why
that variable is confounded with whether or not the subjects smoke. To cite this as a confounding
variable, the student would need to argue that smokers have a much different diet than non-
smokers and that an unhealthy diet could be the cause instead of smoking.