Rizka Adelia Maharani

Introduction to Cognitive Science and Experimental Methods: Exercise 1

Before jumping into the discussion of why and how Douglas G. Mook defends
external invalidity, let’s start with the definition of external validity. According to
Campbell (1957), as cited in Brewer (2000), external validity refers to the
generalizability of the causal finding, whether it is applicable to other subjects, in
different places in the world, and operating under different research methods. In other
words, external validity is the extent to whether findings of a research can be applied to
a broader context. Scientific research offers knowledge about the real world, and
therefore, the results “ideally” should have the possibility to be applied across the globe.
And in Mook’s point of view, this idea that’s often referred to as generalization is often
not intended (Mook, 1983).
In this case, we are specifically discussing research in psychology, where data
collection processes would involve laboratory work. According to Campbell (1957), as
cited in Anderson et al (1999), a common belief is that laboratory studies are generally
high on internal validity but low on external. Therefore, it is assumed that those types of
studies do not apply to the real world. Proctor and Capaldi (2008) referred to this as a
misconception, it is believed that the real problem to focus on is to find out what can
and cannot be generalized by empirical means. Anderson et al (1999) also stated that
application-oriented scholars sometimes deride that laboratory studies are trivial, to the
point that they are deemed as pointless.
Some examples of statements that challenged Mook came from Babbie (1975) and
Bannister (1966): “The greatest weakness of laboratory experiments lies in their
artificiality. Social processes observed to occur within a laboratory setting might not
necessarily occur within more natural social settings.”
—(Babbie, 1975)

“In order to behave like scientists we must construct situations in which our subjects . . .
can behave as little like human beings as possible and we do this in order to allow
ourselves to make statements about the nature of their humanity.”
—(Bannister, 1966)

Which then were responded by Mook (1983) with the defense of external invalidity. And
to quote Campbell and Stanley (1967), "External validity asks the question of
generalizability: To what populations, settings, treatment variables, and measurement
variables can this effect be generalized?” Mook then proceeded to emphasize that external
validity asks a question, instead of it being a concept that is set in stone and inflexible. And
this raised the question of whether a research aims to generalize to the real world at all. After
all, all studies are bound to have limitations, no study can satisfy generalization’s demand for
one to be applicable to different contexts globally.
Probably one of the most interesting points that Mook (1983) made in defense of external
invalidity would be of the following cases, as cited in Anderson et al (1999). He described
four cases that support that artificial laboratory setting is, in fact, preferred in comparison to
the real-world setting.
“First, we may be asking whether something can happen, rather than whether it typically does
happen. Second, our prediction may . . . specify something that ought to happen in the lab. . . .
Third, we may demonstrate the power of a phenomenon by showing that it happens even
under unnatural conditions that ought to preclude it. Finally, we may use the lab to produce
conditions that have no counterpart in real life at all…”
- (Mook, 1983)

Mook (1983) also provided several examples that support his reasons. Let’s take one
where he touched upon how a laboratory experiment that is far from the “ideal”, as-close-to-
natural-setting-it-can-be setting can also be rather significant in real-world practices. Mook
gave an example of a human subject in a dark room, staring at a space where a red light might
appear. Then, the subject has to answer “yes” if they see the light, or “no” if they do not. This
alone sounds like an activity that is in no way close to any activity in a human’s day-to-day
life. However, this experiment can still play an important role in humans’ life. By
understanding how the visual system works.
“to flying planes at night, to the problem of reading X- ray prints on the spot, to effective
treatment of night blindness produced by vitamin deficiency…”
- (Mook, 1983)

Another example that Mook mentioned was from Henshel (1980), where it was discovered
that biofeedback could have never been discovered if the research was to mimic natural
settings.
Going back to the four cases that were mentioned above, we can learn that this gave a
rather solid argument as to why a study does not always have to fit the mentioned
“generalization mold” to be deemed a worthy study. Instead, one is free decide the goal of
their study – whether it aims to predict what would happen in a real-life setting or whether it
only aims to test a prediction. Mook ended his argument with questions one should ask
themselves when they spot a great difference between the research setting and the real-life
one: Is it going to or constrain our conclusions? Or is it going to actually strengthen them?
As a closure, I would like to address several things that can be learned from Mook’s
defense on external invalidity. (1) every study always has its limitations, therefore, (2) it
only goes to a certain extent of generalizability and it does not mean that the findings
would be trivial because (3) a researcher gets to decide the aim of their study and it does
not always the intention to generalize. And at the end of the day, every study counts as a
contribution to offering knowledge to the real world.

References

Anderson, C. A., Lindsay, J. J., & Bushman, B. J. (1999). Research in the Psychological
Laboratory Truth or Triviality?. Current directions in psychological science, 8(1), 3-9.
Brewer, M. B. (2000). Research design and issues of validity. In H. T. Reis & C. M. Judd
(Eds.), Handbook of research methods in social and personality psychology (pp. 3–16).
Cambridge University Press.
Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings.
Psychological bulletin, 54(4), 297- 312.
Henshel, R. L. The purposes of laboratory experimentation and the virtues of deliberate
artificiality. Journal of Experimental Social Psychology, 1980, 16, 466-478.
Proctor, R. W., & Capaldi, E. J. (2008). Chapter 10: Internal and External Validity, in their
Why science matters: Understanding the methods of psychological research. John Wiley
& Sons.