'Journal ol Consulting Psyc'kdlogy
1961, Vol. 25, No. 4, 324-326
RESPONSE BIAS IN QUESTIONNAIRE REPORTS1
N. H. AZRIN, W. HOLZ, AND I. GOLDIAMOND 2
Anna State Hospital, Illinois
If one defines psychology as the study of
behavior, the direct measurement of behavior
appears to be a minimal prerequisite to fur-
ther analysis. Several alternatives to a direct
measurement of behavior are commonly prac-
ticed. One such indirect method defines and
measures the behavior in terms of the effect
of that behavior upon the environment. For
example, the measurement of reaction time
typically is based upon the moment of closure
of an electric switch or push-button. The sim-
ple fact of closure of the push-button does not
guarantee that the movement of any one
finger be involved since any finger or even the
palm, wrist, arm, or leg might just as easily
have been used. Such ambiguities in inter-
pretation are easily overcome, and the experi-
menter can easily confirm his interpretation
of the switch closure by occasionally or con-
tinuously observing the behavior directly.
A second alternative to direct behavioral
observation is the interview or questionnaire
procedure. Here the experimenter typically
does not have any simple means of direct be-
havioral observation. Rather, the subject him-
self is expected to observe his own behavior
and to describe it at some future date. The
subject's reports usually cannot be evaluated
by direct observation of the behavior being
reported as was true of closing of the switch.
The problem is often enhanced by the fact
that the behavior being reported upon is
basically unobservable by the experimenter
by its very nature. This is true for the so-
called "subjective reactions" as when an indi-
viual states that he feels hostile or afraid. In
addition, it is quite likely that a report of
one's own behavior will be modified consid-
1
This investigation was supported by a grant from
the Psychiatric Training and Research Fund of the
Illinois Department of Public Welfare.
2
Now at Arizona State College, Tempe, Arizona.
324
erably by the audience or experimenter to
whom the report is being made. Other factors
such as social acceptability may also be in-
volved. The reply to the question "Did you
cheat?" would probably be different if the
interviewer were a classmate than if he were
an instructor. For whatever the ultimate rea-
sons may be, the individual being questioned
may have a pre-existing tendency or bias in
admitting to some statements and not to
others. The present study was performed in
order to study the influence of such response
biases upon the reports of behavior obtained
through a questionnaire.
A well-known study by Shaffer (1947)
deals with the reports of combat flyers of
their fears in combat. The reports had been
obtained from these flyers by means of a
questionnaire some 2 months following the
termination of their combat experiences. This
study has been widely interpreted as demon-
strating that some behavioral reactions such
as "soiling one's pants" are more indicative
of fear in combat than are behavioral reac-
tions such as "feeling nervous and tense." In
order to evaluate the validity of this inter-
pretation, 160 college freshmen and sopho-
mores, including males and females, in five
separate psychology and sociology classes
were given a questionnaire. This questionnaire
contained the same IS "symptoms" reported
in the original investigation by Shaffer. All of
the questionnaires had the following direc-
tions:
Imagine that you are a combat flyer who has flown
many missions over enemy territory. Your command-
ing officer gives you' the questionnaire below and
tells you to fill it in. Fill in the answers keeping in
mind what your commanding officer expects you to
have felt.
Two forms of the questionnaire were used,
however; half of each class of students re-
 Response Bias in Questionnaire Reports 325

ceived one form and half received the other TABLE 1

form. One form stated after the first sentence:
You have been extremely frightened on all of your
missions and have experienced each of the symptoms
below on every flight.
The other form stated:
You have never been frightened on any of your mis-
sions and have not experienced each of the symptoms
below on every flight.
One half of the students are thereby told
that they have never experienced any of the
listed symptoms, whereas the other half are
told they have experienced all of the symp-
toms. Further, all students were instructed by
the questionnaire to answer in terms of what
is expected, regardless of what behavior is
presumed to have occurred.
RESULTS AND DISCUSSION
Table 1 presents the percentage of students
who chose each symptom as occurring "often"
or "sometimes." It will be recalled that the
students had been told that all symptoms were
experienced equally often. The only basis for
checking some symptoms more than others
was the specific instruction to keep in mind
what answers are expected. Had there been
no predisposition or response bias toward
some symptoms, one should expect all symp-
toms to have been selected equally often. Cer-
tainly, no particular rank ordering of the
symptoms should have emerged. The results
of Table 1 demonstrate that the selection of
symptoms does not follow a random distribu-
tion. Some symptoms were selected as much
as six times as often as others.
Spearman rank-order correlations were per-
formed to determine the consistency of this
response bias among the students. It was
found that the rank-order of symptoms for
males correlated with that of females with
p ~ .88. Similarly, the rank-order correlation
of symptoms was .95 between those students
who were told that they had experienced all
the symptoms and those who were told that
they had not experienced the symptoms. This
high degree of similarity demonstrates that
the response bias toward certain symptoms
exists regardless of whether or not the symp-
tom was alleged to occur.
REPORTED SYMPTOMS OF COMBAT FEAR OP
160 COLLEGE STUDENTS
% of students
During combat missions stating "often"
did you feel: or "sometimes"
That your muscles were very tense 72
A pounding heart and rapid pulse 71
"Butterflies" in the stomach 67
Dryness of the throat or mouth 67
"Nervous perspiration" or "cold
sweat" 61
Sense of unreality that this couldn't
be happening to you 49
Easily irritated, angry, or "sore" 43
Need to urinate very frequently 42
Trembling 39
Unable to concentrate 36
Sick to the stomach 34
Right after a mission, unable to re-
member details of what happened 32
Confused or rattled 28
Weak or faint 25
That you have wet or soiled your
pants 11
In order to determine whether the same re-
sponse bias might have affected the reports of
the combat flyers, a Spearman rank-order cor-
relation was performed between the symp-
toms reported by the flyers and those re-
ported by the students. It was found that
the rank orders of the responses were highly
similar (p = .89) between the students and
the flyers. Nor was this relationship reduced
for those students who were told that they
had not experienced the symptoms (p = .94)
as compared with those who were told that
they had experienced all of the symptoms
(p = .90). The statistical stability of this re-
sponse bias is evidenced by the degree to
which the rank-order in each classroom of
students was correlated with the rank-order
reported by the combat flyers: p — .70, .82,
.85, .90, and .92.
The response pattern obtained from the
students by means of the questionnaire is al-
326 f. H. Azrm, W. Holz, and 1. Goldiamond
most completely predictable on the basis of
response bias. Therefore, it is quite likely that
the same type of response bias operated on
the combat flyers. Any conclusions concerning
the actual symptoms must await study by a
method that provides for a more direct and
objective measurement.
The present findings may well be consid-
ered for their implications for the use of in-
terview and questionnaire methods in general.
Unless an objective and direct means of meas-
urement is available, the questionnaire re-
sponses may be independent of the behavior
being studied. A definitive method of deter-
mining the validity of the reports is the direct
and objective measurement of the behavior
being reported. Once such a direct measure
is available, however, the very need for ques-
tionnaire reports is eliminated.
REFERENCE
SHAFFER, L. F. Fear and courage in aerial combat.
/. consult. Psychol, 1947, 11, 137-143.
(Received June 21, 1960)