CHAPTER

ExPost Facto Research

Among the fallacies used by man, one of the most dangerous to science is that.
known as post hoc, ergo propter hoc: after this, therefore caused by this. We may
joke, with a tinge of seriousness, "If I take an umbrella, it won't rain." We may
even seriously say that delinquents are delinquent because of a lack of discipline
in the schools or that religious education makes children more virtuous. It is very
easy to assume that one thing causes another simply because it occurs before the
other, and because one has such a wide choice of possitble "causes." Then, too.
many explanations often seem plausible. It is easy to believe, for instance, that the
learning of children improves because we institute a new educational practice or
teach in a certain way. We assume that the improvement in their learning was due
to the new spelling method, to the institution of group processes into the classroom
situation, to stern discipline and more homework (or little discipline and less
homework). We rarely realize that children will usually learn something if thcy
are given the opportunity to learn.
The social scientist and the educational scientist constantly face the problem
of the post hoc fallacy.The sociologist who seeks the causes of delinquency knows
that he must exercise extreme care in studying this problem. Slum conditions,
broken homes, lack of love-each, or all, of these conditions are possible causes
of delinquency. The psychologist seeking the roots of adult personality faces an
even subtler problem: hereditary traits, child-rearing practices, educational in
fluences, parental personality, and environmental circumstances are all plausible
explanations. The educational scientist, with the goal of understanding the basit
of successful school achievement, also faces a large number of reasonable pos
bilities: intelligence, aptitud, motivation, home environment, teacher personaHty
pupii personality, and teaching methods.
 EX OST FACTO RESTARCH 379

Tàe dng ofhe post boc assumpion is that it an. ad oftea does. ed o
ernns and misleading interpretations of research data the efect beirg par
iulary ris when the scientist has itle or no control over tine nd indee
dent varish bea be is eeking to explain a pbezomeron tbat es aziy
uT.be is onfrvnted with the unpieasant fact that he does ot bave rezt
e t the pssite causes Hene be mast pursue2 course cf rrsearchaction
dirterent in exeution and intepretation from that ofthe scieatist =bs erperimets

kaTe r r d or becuzse they are irherently no NANiDalable. irfererEs

Assume hat an invsigator is interestai in the relatioa betmea sEx z

siiy n chiktren. He easures the creativty of a anpie of boys nd gisis
nd tests the signi'ane of the diference bezween the neans of the SEEes
Te mean fboys is signiñcantly higher than the man of irs He concaies the
boys r mrcRaive than girs. This mey or may not beavakd canchion The
rkim eNt re With only this evideae however. the conchsn s åouhal.
The quesin `: Is the demonstratd rlation rally betwea seI n0 Creativity ?
Siace many other vaiablks are comeated with ex. Àmight have keen ooe or
Dore af these variables that pradad the difere betweea the creaivity sores
tie two exes

Basic Difference between Experimental

Research and Ex Post Facto Xesearch
Te basis of the sructure n whch the exprimental scnii operatesie
He àypoheizes: If x, then y; if ustratoa then sggNin Deeiing o i
cuMstaes and his personal pretilections in researcà desn, e uses soe
method to manipulate or masure x. He then observesy to sae t concomitant
vaiatoa, the variatin expected or pactad frm the variain in, Sii
does, this is evidene for the vadity of the ropan.I’I’ ening
"Ifr, hen v." Note that the scienist he r s im a contToa.o . To
belp him chieve control, he can use the pine o raomizaa ve
manipulation of xad can assume. thr things aqual, hyis varyng s a u
of the mnipulation ofr.
In ex stfacto esearch, on thether had,rishseni, a d n i T r
380 1YPES OF RESEARCH

concomitant to the observation of y.

r's, are also observed, either before, after, or logic: it can be shown that the argu
There is nodifference whatever in the basic
same in experimental and ex post
ment structure and its logical validity are the also the same: to establish the
facto research.? And the basic purpose of both is form: Ifp, then g. The
cmpirical validity of so-called conditional statements of the variable. In experi
essential difference is direct control of p, the independent direct "control."" When
mental research, pcan be manipulated, which is rather
massive reinforce
Clark and Walberg had teachers give one group of subjects
reinforcement, they were
ment and other teachers give another group moderate
directly manipulating or controlling the variable reinforcement. Similarly, when
another
Bandura and Menlove showed one group amovie with a single model,
'neutral" movie, they
group a movie with multiple models, and a third group a
subjects can be
were directly manipulating the variable modeling. In addition,
assigned at random to the experimental groups.
experimental
In ex post facto research, dirct control is not possible: neither
These are
manipulation nor random assignment can be used by the researcher.
two essential differences between experimental and ex post facto approaches.
Owing to lack of relative control of x and other possible x's, the "truth" of the
hypothesized relation between x and y cannot be asserted with the confidence of
the experimental situation. Basically, ex post facto research has, so to speak, an
inherent weakness: lack of control of independent variables.
post
The most important difference between experimental research and ex
facto research, then, is.control. In the experimental case, the investigator at least
has manipulative control: he has at least one active variable. If an experiment is a
*true" experiment, he can also exercise control by randomization. He can assign
In
subjects to groups at random, or he can assign trèatments to groupsat random.
the ex post facto research situation, this kind of control of the independent vari
ables is not possible. The investigator must take things as they are and try to
disentangle them.
Take a well-known case. When an experimenter paints the skins of rats with
carcinogenic substances (x), adequately controls other variables, and the rats
ultimately develop carcinoma (y), the argument is compelling becausex (and other
poss1ble x's, theoretically) is controlled and y is predicted. But when an investi
gator finds cases of lung cancer (y)and then goes back among the possible multi
plicity of causes (x,, Xy, ....x) and picks cigarette-smoking (say x) as the culprit.
he is in a more difficult and ambiguous situation. Neither situation is sure, of
course; both are probabilistic. But in the experinnental case the investigator can be
more sure- considerably more sure if he has adequately made "other things
equal'"that the statement If x, then y is empirically valid. In the ex post facto
case: however, the investigator is always on shakier ground because he cannot
say, with nearly as much assurance, "other things equal." He cannot control the
independent variables by manipulation or by randomization, In short, the proba
he basic logic is set forth in: F. Kerlinger,"Rescarch in Education." In R. Ebel, V. Noll, and R.
1969, pp. 1127-1144
iBaçe, cds., Encylopedia of Educational Research, 4th ed. New York: Macnillan,
(Pp. 1133-1134).
 EX POST FACTO RSEARH

iitythat a is "really" related to y is grcater in the experimental situation than it is

facto situation, because the control of xis grcater.
inthe ex post

Self-Selection and Ex Post Facto

Research
In an ideal social scientific research world, the drawing of random samples of
subjects, and the random assignment of subjects to groups and treatments to
ouns. would always be possible. In the real world, however, one, two, or even
lthree of these possibilities do not exist. It is possible to draw subjects at ran
dom in both experimental and ex post facto research. But it is not possible, in ex
nostfacto research, to assign subjects togroups at random or to assign treatments
to groups at random. Thus subjects can "assign themselves" to groups, can
select themselves" into the groups on the basis of characteristics other than those
in which the investigator may be interested. The subjects and the treatments
come, as it were, already assigned to the groups.
Selfselection occurs when the members of the groups being studied are in
the groups, in part, because they differentially possess traits or characteristics
extraneous to the research problem, characteristics that possibly influence or are
otherwise related to the variables of the re_earch problem. Examples of self
selection may aid understanding.
In the well-known research on cigarette-smoking and cancer, the smoking
habits of a large number of people were studied. This large group was divided into
those whohad lung cancer -or who had died of it- and those who did not have it.
The dependent variable was thus the presence or absence of cancer. Investigators
probed the subjects' backgrounds to determine whether they smoked cigarettes.
and if so, how many. Cigarette-smoking was the independent variable. The inves
tigators found that the incidence of lung cancer rose with the number of cigarettes
smoked daily. They alsofound that the incidence was lower in the casesof light
smokers and nonsmokers. They came to the conclusion that cigarette-smnoking
caused lung cancer. This conclusion may or may not be true. But the investigators
cannot come to this conclusion, although they can say that there is a statistically
significant relation between the variables.
The reason theycannot statea causal connectionis that there are a number
of other variables, any one of which, or any combination of which. may have
caused lung cancer. And they have not controlled other possible independent
Variables. They cannot control them, except by testing alternative hypotheses. a
procedure to be explained later. Even when they also study "control groups of
people who have no cancer, self-selection may be operating. Maybe tense. anxious
men are doomed to have lung cancer if they marry blonde women. for instance. It

arelulscientilic investigators will usuallynot say "cause."The word "cause" is used here to make
the POnt more emphatic and because authoritative sources so use it: see The Nrvl'ork Times. Dec. 6.
1959, p. E-l1, where the
tivg:"he Surgeon General ofthe United States Public Health Serviceiis diret: quote
tive) factor in the weightof eridence
increased incidenceatofpresent impicates smoking as the prin ipal etiologu al casa
lung cancer."
 TYPES OF RESEARCH
382

that this type of man also smokesscigarettes heavily. The cigarette-

may just happen kills himhe kills himself by being born tense and anxious-
not what
smoking is
marrying a blonde. Such men are selected into the ame
and possibly by
investigators only because theysmoke cigarettes. But such men select themselves
sample because they commonly possess a temperament that happens to
into the concomitant.
Self-selection can as
have cigarette-smoking bea a subtle business. There are two kinds: self-selection
The latter occurs when subjects are
into samples and into comparison groups. Cancer and no cancer, college and
lected because they are in one group or another:
underachievement. That is, they are select
nocollege, underachievement and no or lesser degree. Self.
becuuse they possess the dependent variable in greater in a nonrandom fashion
selection into samples occurs when subjects are selected
into a sample.
alwavs
The crux of the matter is that when assignment is not random, there is
into groups.
a loophole for other variables to crawlthrough. When we put subjects groups, on
in the above case and in similar cases, or they "put themselves* into
variables) cor
the basis of one variable, it is possible that another variable (or
related with this variable is the "real'" basis of the relation. The usual ex post facto
variable. In some
study uses groups that exhibit differences in the dependent
longitudinal-type studies the groups are differentiated first on the basis of the
independent variable. But the two cases are basically the same, since group mem
bership on the basis of a variable always brings selection into the picture.
follow
For example, we may select college freshmen at random and then
them to determine the relation between intelligence and success in college. The
students selected themselves into college, so to speak. One or more of the charac
teristics they bring with thm to college, other than intelligence -socioeconomic
level, motivation, family background may be the principal determinants of col
lege success. That we start with the independent variable, in this case intelligence,
does not change the self-selective nature of the research situation. In the sampling
sense, the students selected themselves into colege, which would be an important
factor if we were studying college students and noncollege students. But if we are
interested only in the success and nonsuccess of college students, self-selection
into college is irrelevant, whereas self-selection into success and nonsuccess
groups is crucial. That we measure the intelligence of the students when tney
enter college and follow them through to success and nonsuCcess does not change
either the selection problemn or the ex post facto character of the research. In sum,
succeed
the students selected themselves into college and selected themselves to
or not tO succeed in college.

Large-Scale Ex Post Facto Research research

Further study of research examples will help us evaluate ex postfactocontextof
in the behavioral sciences. Several in the research.
exampl2s have been givenpost
other problems. Now, we focus on the facto because
In Parts 5 and 6, we inherent nature of ex
research, be
necessarily concentrated on experimental
 EX POST FACTO
RESEARCH 383

analysis of variance and rescarch design have bcen developcd in an

framework. This does not mean that cxperimental rcsearch is cxperimental
nccessarily more
important or even more frequent in the bechavioral sciences. Indecd, it is
exaggerationto say that alarge proportion of probably
rescarch in sociology, education,
anthropology, and political science has been ex post facto. Though psychologists
are much more inclined to be cxperimental in their
approach -many more psycho-
ogicalthan sociological variables, for example, are manipulable -it is probably
cafe to say that asubstantial proportion of psychological studies, perhaps half or
more thanhalf. are ex post facto.

The Authoritarian Personality Study

The Authoritarian Personality Study was actually a series of studies which to
gether constitute an important and influential contribution to social scientific,
narticularly psychological, research. The general hypothesis of the study was that
political, economic, and social beliefs are related to deep-seated personality
characteristics. Another hypothesis was that adult personality is derived from
early childhood experiences. In short, attitudes and beliefs were related to under
lying personality trends. The investigators, among other things, studied anti-Semi
tism as part of a generalcharacteristic called ethnocentrism. Later, the investigalors
extended their thought and work to a stilllarger construct, authoritarianism,which
they conceived to be a broad personality syndrome that determines in part
ethnocentrism, social attitudes, and certain other behaviors. The authoritarian
personality was conceived to be conventional, cynical, destructive. aggressive,
power-centered, and ethnocentric.
While this is an inadequate summary of the basic problems of a very complex
study, it is sufficient for the present purpose. The study had to be ex post facto
although there have been later cxperimental studies inwhich authoritarianism has
been manipulated and in which high authoritarianism and ow autboritarianism,
for instance, have been attribute variables -- because author arimism as defined,
is a nonmanipulable variable. One of the major results of the studv was infoma
tion on the relation between authoritarianism and prejadic t is abvjous that
when one studies such variables one is studying already existing sets of ersonal
itycharacteristics and attitudes. The subjects are ready-made authoritarians or
noneuthoritarians (with gradations between) and come to the research with al
ready well-formulated attitudes. One can conceive, somehow, of manipulating
such variables, but the manipulation, as indicated previously, chahges their na
Lure. At any rate, whenever one studies the relations between variables that "al
!Adorno et al., The Authoritarian Personality. New York: Harper & Row, 1950. An extensive
Scops and
MtOt this book has been published: R. Christie and M. Jahoda, eds., Studies in the volume
The Authoritarian Pessonaliy." New York: Free Press, 1954. Study of the latter s
Te
ng lor the intermediate or advanced student of social suientific research. See, especally, the
MPer by H. Hyman and P. Sheatslev. The Authoritarian PersonalityA Methodolog1cal Cri
dque," Pp. 50-)22. More recent research reviews and critiques are:]. Kirscht and R. Dillehay, Dimen-
K67.nolarunism: AReiew of Reseatch and Theory. Lexington: University of Kentucky Press,
Educational and
Roh Mc8SiCk and D, Jackson, "The Measurement of Authorikdrian Atitudes,""Calitoria FScale
Rrycholagical.Measurement, XVIII (1958), 241-253; H. Titus andE. Hollander, Thè
in Piychological Research: 1950-1955," Pyychological Bulletin, LIV (1957), 47-64.
Ideology, Consensus, and Operational Beliefs Studies
Political scientists have been interested in certain key questions, the anN werN to
which have deep signiticance in a democrmey, T'wo of these re: Does consensus
exiSt among Americans on basic democratic beliels? If such coIsensus exists, do
Americans subscribc to specific operational beliefs und behaviors inplied by the
basic beliefs? Three important studies addressed to these and other relted ques-
tions have been donein the last decade or so, and the answers to the questions are
notsimple.®
In one of the most recent of these studies, Free and Cantril obtuined the re
sponses of two probability samples ol more than 3000 Americans to a variety of
estions on political beliefs. They found distinct and sharp ditlerences between
what they called the ideological spectrum andthe operational spectrun of beliels.
Americans, for instance, respond quite diflerently to abstract statements of' beliof's
than they do to more specific operational statements. Evidently they are ideologi
cally conservative and operationally liberal." A somewhat diflerent conclus0on,
however, comes from two other large-scale studies,
Prothro and Grigg sampled (randomly), in a northern city and a southern
city, the responses of registered voters to abstract and specific statements, like
"Democracy is the best fornm of govenment" (abstract) und "ANegro should not
beamong
allowed torun for mayor of this city" (speciftic). They found high agreement
the voters with the abstract statements and much less agreement on the
specific statements. Here people seem to be ideologically (abstractly) liberal and
operationally conservative.
. In the third, and perhaps most important and sophisticated study, MeCkosky
administered to his respondents a wide variety of statements on personal back
ground, personality traits, and political, economic, and social values, attiludes, und
beliefs. His respondents were of two kinds: leaders or political inluentials (N
3000-+) and the general electorate in the United States (N= I500-++) selected to be
representative (in a Gallup poll), Like Prothro and Grigg, McClosky found grealer
Support for general, abstract statements of democratic beliefs than for the speci
fic application of the beliefs. For instance, 89 percent of the clectorate sample
agreed with the statement, "I believe in free specch for all no matter what their
Views might be," but 50percent agreed with "A book that contains wrong political
Views cannot be a good book and does not deserve to be published." More
OVer, McClosky found that the leaders were distinctly more denmocratic in their
outlook and rejected antidenmocratic sentiments more than the general clectorate
did.
We note two important points about these studies. One, they are clcarly ex

Free and H, Cantril, T'he Poitical Beliets of Americos, New Brnswick, N.J.: Ruigers UnivCINily
American Politcal Sienee
DS, N67; H. McClosky, "Consensus and ldeologyin American Politics,"Priniples of DemocraKy:
B of AgreementI (1964),and361-382: 1. Prothro and C. Grigg, "Fundannental
Bases Disagreement," Journal of Politics, XXIl (1960). 276-29-4.
measred
Frec and Canril's results can be questioed on lhe bsis ol the way they
"Althofoughand in the gencral
liberalism
bature the
Conservatism, this is not our OnCCTN here, We are interested nitinly
research.
 B86 TYPES 0F RENRARCH

post facto, There is no experimental manipulation whatever. While the sampling

was random- except, perhaps, in the MeClosky study -there was no possibility
of random assignment. People were asked for their responses to questions and
statements, and these responses were related to cach other, mostly in percentage
crossbreak forn. That is, the relations betwcen independent and dependent vari
ables, as reflected in the responses that people brought with themn to the studies,
were analyzed. The beliefs of people, as expressed in their responses, were usually
taken as dependent variables and related to other rrsponses or sociologicalfacts.
There was literally no intervention by the researchers.

Small-Scale Ex Post Facto Research

Regional Diferences in Prejudice
In awell-executed study of a social problem that is difficult to probe experimen
ially,Petigrew asked the question: Is anti-Negro prejudice more closely related
osocial factors and less so to personality factors in the South than in the North? In
zffect, this amounts to contrasting sociological and psychological explanations of
prejudie. To test hypotheses derived from this question, Pettigrew adm inistered
authoritarianism, anti Senmitism, and anti-Negro scales to random sar iples of
white adults in four northern and four southern towns.
One hypothesis predicted a simple difference between northern and southern
anti-Negro prejudice. and the southern sample did have a significantly higher
mean score than the northern sample. There was no significant mean difference on
the authoritarianism measure, which was used as a control test.
soned that, since the authoritarianism scale is presumed to measure Pettigrew rea
personality potential," and since the two regions did not differ on this'externalizing
scale butdid
differ on the anti-Negro scale, the hypothesis that externalizing
are of equal importance in the North and the South, and that personality factors
are more important in the South than in the North, was social-cultural factors
supported.
Freedom and Equality Study
A great.dealof theoretical and empirical work has
safely be said that because of this work we been done on prejudice. It can
understand a good deal about the
psychology and sociology of prejudice, its stereotypes, how it operates, and how 1t
is supported. Much of the research has
been ex post facto, For example, the many
studies of sterotypes of minority-group members have been largely
tx post facto. To understand and pertore
phenomena
we have to know their incidence and their
like prejudice and stereotypes, atter all
further understanding of such phenomena isrelations to other variables. Of course.
The former knowledge requires ex post factoenhanced when we can change then.
research, while the latter requires an
"T. Ptigrew, "Regional
Prycholory, LIX (1959), 28-36, Differences
in Anti-New Prejudice," asmnak of
tbngrmal nd Sa
M. Rokeach, Beliefs, Attitudas, 4.1d Valus, San
Frncisco: josseyaBam(68,p. l68-178.
 EX POST FACIO RESEARCH 387

exne inental approach. The Pettigrew study just cited is an excellent example of
ihe 'ormnic1 demand. The research we are about to examine is an ex post facto part
With
of theoretically based research that is both ex post facto and experimental.
the later
that
out the knowledge yielded by the ex post facto work, it is doubtful
experimentalwork would have been pessible.
Rokeach has hada number cf groups, including a national sample, rank order
terminal
two sets of what he calls terminal and instrumental values. Two of the
they are
values, freedom and equality, have been particularly significant becauseoutlooks.
evidently keys to fundamental differences in social and political value
Rokeach has, found, for instance, that different groups reliably and significantly1.
ank these two values (embedded among others) quite differently. In Table 22.
TABIE 22.1 COMPOSITE RANKS OF Freedom AND Equality OF DIFFERENT
CROUPS (1), AND THEIR FREQUENCY OF MENTION AND COMPARABLE RANKS
IN FOUR SAMPLES (OF POLITICAL WRITINGS (I1)

Unemployed Unemployed Calvinist

Policemnen (50) Whites (141) Negroes (28) Students (75)

10 8
Freedorn
12 1 9
Equality

I1.
Socialists Hitler Goldwater Lenin
Freq. Rank Freq. Rank Freq. Rank Freq Rank

Freedom +66 1 -48 16 +85 -47 17

+62 2 -71 17 --10 16 +88
Equality
"? is the highest rank, 12 the lowest.
Fiequency is defined as number of favorable imentions ninus number of unfavorable mentions

tne composite ranks of the two values given by different groups are given (I). The
results are dramatic. Policemen and unemployed blacks are very, very diferent in
ther social value outlook. To the policemen, freedom is highly important, while
equality is not, but to the blacks equality is all-important and freedom is not. Quite
Sn upsI of certain of our traditional ideas! In contrast, Calvinist students rank
both values rather iow.
Rokeach seems to have hit upon a fundamental difference in value outlook.
Hisresutts are evidently not spurious. Study the bottom half of Table 22.1(II). He
and a colleague (James Morrison) counted the number of times that free dom and
equality were mentioned favorably and unfavorably in the writings of socialists,
Hitler Goldwater, and Lenin. Again, there are striking contrasts, which need no
elaborafion, This is another excellent example of ex post facto research and its
SCintific usefulness in imaginative and competent hands.o
Note that Rokeach, after the above research, did experiments in which he managed to change
students' values: ibid.,pp, 175 it.
 388 TYPES OF RESEARCU

AStudy of Children's Reactions to Finger Painting'"

A study that is particularly interesting, because it combines experimental and ex
post facto approaches,is the Alper, Blane. and Adams study of the reactions of
children of different social classes to finger-painting experience. Experimental
manipulationwas certainly involved, but the independent variable was not manipu
lated. Indeed. one might almost say that the dependent variable was manipulated!
The general question the authors asked was: Do social-class differences in child
training practices result in class differences in personality? More specifically: Are
there differences in approaches to finger painting between middle- and lower-class
children?
Two groups of nursery school children, 18 middle-class children and 18
lower-class children, were introduced to two diferent finger-painting tasks. The
children's behavior was measured on 16 variables: time began painting, accep
tanceof task, requests for help, washing. and so on. The two groups differed great
Iy intheir reactions,and the differences were significant on most of the measures.
In a control experiment, " the same procedure was followed using crayons
rather than finger paints. The only differences were procedural changes necessi
tated by the use of the two media (some of the subjects were different also). The
two groups did not difer significantly in any of the l1variables measured, a rather
surprising contrast to the former results.
To call the Aper, Blane, and Adams study ex post facto may be questionable,
because acontrol experiment was included. It is possible, however, to suppose
that the two groups varied in the finger-painting tasks, not because of different
child-rearing practices between the social classes, but because of some other vari
able. The study is classified as ex post facto research because it was not possible
to manipulate the independent varible and because the subjects came to the study
with their reactions ready-made, as it were.
Another noteworthy point is the ingenuity of the control experiment. The
running of this second experiment is analogous to Pettigrew's testing of authori
tarianism. Imagine the researchers' consternation if the differences between the
two groups had been significant on the crayon tasks! Obviouly their theoretical
thinking would have had to be reviewedwith a very critical eye.

Testing Alternative or "Control"

Hypotheses
Most investigations begin with hypotheses; the empirical implications of these
hypotheses are then tested, Although we "confirm" hypotheses in the manner
described in earlier chapters, we can also confirm" and "disconfirm" hypotheses
under study by trying to showthat alternative plausible hypotheses are or are not
supported. Let us first consider alternative independent variables as antecedents
of adependent variable. The reasoning is the same. If we say alternative inde
T. Alper, H. Blane, and B. Adams. "Reations of Micddleand Iower (Class (Children to Finget
Paints as a Funcionof Class Ditlerences in(Chilkd-Training Practices," Journal of Abnormal and Socat
Psyrhology, LI (1955), 439-448.
pendent variables, " for example, we are in effect stating alternative hypotheses or
explanations of a dependent variable.
In ex post facto studies, although one cannot have the confidence in the
"truth" of aan "If x, then y' statement that one can have in experiments, it is pos-
sible to set up andtest alternative or "control" hypothescs.(Of course, alternative
hypotheses can be and are tested in experimental studies, too.) This procedure has
been formalized and explained by Platt who, influenced by Chamberlin, called it
"strong inference.""12 Chamberlin aptly called the procedure the "method of work-
ing multiple hypotheses," and he outlined how theinvestigator's own "intellectual
affections" can be guarded against. He said: The effort is to bring up into view
ery rational explanation of new phenomena, and to develop every tenable
hunothesis respecting their cause and history. The investigator thus becomes the
narent of afamily of hypotheses;and, by his parental relation to all, he is forbidden
to fasten his affections unduly upon any one. 9913
Let Xj, Xz, and x, be three alternative independent variables, and let y be the
dependent variable. the phenomenon to be 'explained" with a statement of the
form: If x,then y. Assume that X|, Xy, and x, exhaust the possibilities. This assump
tion cannot actually be made-in scientific research it is practically impossible to
exhaust all the causal possibilities. Still, it is assumed for pedagogical reasons.
Aninvestigator has evidence that x, ánd y are substantially related. Having
reason to believe thatx, is the determinative factor, he holds x, and x, constant. He
is assuming that one of the three factors is the factor, that either x, or x, Or x, is the
"true'" independent variable. (Again, note the assumption. It may be none of them
or some combination of all three.) Suppose that the investigator succeeds in elim
inating x, that is, he shows that , is not related to y. If he also succeeds in elim
inating x3, he can then conclude that x, is the influential independent variable. Since
the alternative or "control" hypotheses have not been substantiated, the original
hypothesis is strengthened.
Similarly, we can test alternative dependent variables, which imply alterna
tive hypotheses, of course. We merely shift the alternatives to the dependent
variable, as Alper, Blane, and Adams did when they set up the crayon experiment
In juxtaposition to the finger-paint experiment. Pettigrew used the same method to
test the relation between geographical region and authoritarianism and prejudices.
Inboth of these studies alternative hypotheses were tested and found wanting.
Now consider a study by Sarnoff et al. in which it was predicted that English
and American children would differ significantly in test anxiety but not in general
anxiety.l4 The hypothesis was carefully delineated: If eleven-plus examinations
are taken, then test anxiety results. (The eleven-plus examinations are given to
lau, "Srong Inference," Science. (iXLVI (1964), 347-353; T. Chamberlin,
"The Method of
. article was
Mple Working Hypotheses." Science, CXLVII (1965), 754-759. The Chamberlin testing alterna
gially published in Science in 1890(vol. 15). A clear explanation of the logic behind
enypotheses is given in: M, Cohen and E. Nagel, An Introduction to Logic and Scientiftc Method. New
ToTk: Harcour1 Brace lovanovich, 1934, pp. 265-267.
Chamberlin, op. cit., p. 756.
Sarnoff et al., "A Cross-Cultural Study ofAnxiety among American and English School Chil
dren,""Journal of Educationul Psycholog); XLIX (1958), 129-136.
 TYPES OF RESEARCH

English school children at eleven years of age to help determine their educational
futures.) Since it was possible that there might be other independent variables
causing the difference between the English and American, children on test anxiety.
the investigators evidently wished to rule out at least some of the major conten
ders. This they accomplished by carefully matching the samples: they probably
reasoned that the difference in test anxiety might be due to a difference in general
anxiety, since the measure of test anxiety obviously must reflect some general
anxiety. If this were found to be so,the major hypothesis wouldnot be supported.
Therefore Sarnoff and his colleagues, in addition to testing the relation between
examination and test anxiety, also tested the relation between examination and
general anxiety.
In this kind of ex post facto control, instead of having alternative independent
variables, say x, and x, We have alternative dependent variables, y1 and y,. We
again assume that the alternatives exhaust the possibilities. If this is so, then x is
either associated with y, (test anxiety), or with y, (general anxiety), or with both.
Toparaphrase the Sarnoffargument: Either the examination influences test anxiety
or it influences general anxiety, or both; the examination influences test anxiety
and it does not influence general anxiety. Therefore the examination influences
test anxiety.
The method of testing alternative hypotheses, though important in all re
search, is particularly important in ex post facto studies, because it is one of the
only ways to ""control" the independent variables of such research. Lacking the
possibility of randomization and manipulation, ex post facto researchers, perhaps
more so than experimentalists, must be very sensitive to alternative hypothesis
testing possibilities.

Evaluation of Ex Post Facto Research

The reader may have concluded from the preceding discussion that ex post facto
research is inferior to experimental research, but this conclusion would be un
warranted. It is easy to say that experimental research is "better" than ex post
facto research, or that experimental research tends to be "trivial," or that ex post
facto research is "merely correlational." Such statements, in and of themselves,
are oversimplifications. What the student of research needs is a balanced under
standing of the strengths and weaknesses of both kinds of research. To be com
mitted unequivocally to experimentation or to ex post facto research may be poor
policy.
The Limitations of ExPost Facto Interpretation
alreaay
Ex post facto research has three major weaknesses, two of which havevariables.
been discussed in detail: (1) the inability to manipulate independent
(2) the lack of power to randomize, and (3) the risk of improper interpretation.
other words, compared to experimentalresearch, other things being equal, ex post
facto research lackscontrol; this lack is the basis of the third weakness: the risko
improper interpretation. re-
erroneous interpretations in exX postfacto
The danger of improper and
 EX POST FACTO RESEARCH 391

search stems in part from the plausibility of many explanations of complex events.
Ttis easy for us to accept the first and most obvious interpretation of an established
relation, especially if we work without hypotheses to guide the investigation, or
proceed from the dependent variable to the independent variable. These two cir-
aumstances are closely related because research unguided by hypotheses, research
Mto fnd out things," is inost often ex post facto research. Experimental research
is more likely to be based on carefully defined hypotheses.
Hypotheses are if-then predictions. In aresearch experiment the prediction
is from a well-controlled x to ay. If the prediction holds true, we are relatively safe
in stating the conditional, Ifx, then y. In an ex post facto study under the same
conditions, however, we are considerably less safe in stating the conditional, for
reasons discussed earlier. Careful safeguards are more essential in the latter case,
especially in the selection aad testing of alternative hypotheses, such as the pre
dicted lack of relation between the eleven-plus examination and general anxiety
in the Sarnoff study.A predicted (or unpredicted) relation in ex post facto research
may be quite spurious, but its plausibility and conformity to preconception may
make it easy toaccept. This is a danger in experimental research, but it is less of a
danger than it is in ex post facto research because an experimental situation is so
DIUch easier to control.
Ex post facto research that is conducted without hypotheses, without pre
dictions, research in which data are just collected and then interpreted, is even
nore dangerous in its power to mislead. Significant differences or correlations
are located if possible and then interpreted. Assume that an educator decides to
study the factors leading to underachievement. He selects a group of under
achievers and a group of normal achievers and administers a battery of tests to
both groups. He then calculates the means of the two groups on the tests and
analyzes the differences with tests. Among, say, twelve such differences, three
are significant. The investigator concludes, then,that underachievers and normal
achievers differ on the variables measured by these three tests. Upon analysis of
the three tests, he thinks he understands what characterizes underachievers. Since
all three of the tests seem to measure insecurity, therefore the cause of under
achievement is insecurity.
Although the simplicity of this example is a bit exaggerated, studies very
similar to this hypothetical one are often undertaken. When guided by hypotheses
the results of such studies are more valid, but the results are stillweak because
they capitalize on chance relations, and above all, the explanation of the results
Seems so plausible-once a plausible explanation has been found. According to
Merton, postfactum explanations do not lend themselves to nulliñability, because
they are so flexible. Whatever the observations,he says, new interpretations can
be found to "ft the facts." 15

The Value of Ex Post Facto Research

Despite its weaknesses,much ex post facto research must be done inpsychology,
SOciology, and education simply because many research problems in the social
ia. Merton, SocialTheory and S::sial Structure. New York: Free Press, 1949, p. 9-91.
392 TYPES OF RESEARCH

sciences and education do not lend themselves to experimental inquiry. A little

reflection on some of the important variables in educational research-intelligence
aptitude, home background. parental upbringing, teacher personality, school
atmosphere will show that they are not manipulable. Controlied inquiry is pos
sible. of course, but true experimentation is not. Sociological problems of educa
tion, such as extreme deviation in group behavior and its effect on educational
achievement, and board of education decisions and their effects on teacher and
administrator performance and morale, are mostly ex post facto in nature. Even if
we would avoid ex post facto research, we cannot.
ltcan even be said that ex post facto research is more important than experi
mental research. This is,of course, not a methodological observation. It means,
rather, that themost important social scientific and educational research problems
do not lend themselves to experimentation, although many of them dolend them
selves to controlled inquiry of the ex post facto kind. Consider Piaget's studies of
children's thinking, Gross' study of boards of education and superintendents, the
authoritarianism studies of Adornoet al., the enormously important study, Equality
of Educational Opportunity. If a tally of sound and important studies in the be
havioral sciences and education were made, it is possible that ex post facto studies.
would outnumber and outrank experimental studies.

Conclusions
Some students of research believe that much behavioral research, but particularly
educational research, suffers from a serious lack of a rigorous experimental ap
proach and that it will lag as long as this situation exists. The author believes that
good experimental research is badly needed in all fields, and that large doses. of
poor ex post facto research should be avoided. Improvements in educational ex
post facto research are badly needed. Perhaps agood rule to followwould be to
ignore the results of any ex post facto study that does not test hypotheses. Excep
tions to this stricture should be few and far between. Perhaps another good rule
,would be to be highly skeptical of any ex post facto study that tests only one
hypothesis; that is, alternative "negative" hypotheses should be routinely tested.
Researchers should predict significant relations and nonsignificant relations
whenever possible.
A final piece of advice is this: always treat the results and
interpretations of
the data of ex post facto investigations with great care and caution. Where
one
must be careful with experimental results and interpretations, one must be doubly
careful with ex post facto results and interpretations.

Addendum: Causality and Scientific

Research
A great deal of work, especially in sociology, has been and is being done on
study and analysis of causal relationsin
tne
ex post facto research. One of the princl
 EX POST FACTO RESEARCH 393
l
annroaches has been what is called path analysis,
Ptstudy toand test alternative whose purpose appears to
hypotheses or alternative
Utohelp establish causal connections and inferences. independent variables
onk by Blalock which exXamines this A good reference is a
ial knowledge of multiple difficult subject in depth i6 Without substan
ubiect inthis regression analysis - beyond the treatment of the
The advanced text-it
is not possible to follow
student, however, willcertainly want Blalock's and others' expositions.
ments. to be aware of the develop
The position
andless maze. Onetaken
in this book is that the
of the difficulties is that study of cause and causation is an
and metaphysical overtones. the word "cause'" has surplus
Scientific research can be donePerhaps more important, it is not really meaningneeded.
even though the words and otherwithout invoking cause and causal explanatioES,
avoid and thus will occasionally bewords that imply cause are almost
used. impossible to
not be demonstrated empirically, butthat Blalock points out that causal laws
it is helpful to think causally."7 We can
that causal laws cannot be agree
thinking causally. There is demonstrated empirically, but we are equivocal about
little doubt that scientists do think causally and that
when they talk of a relation between pand q they hope or believe that p
But no amount of evidence causes q.
This position is not so can.demonstrate
much an
thatp does cause g.
tion that they are not necessary to objection
scientific
to causal notions as it is an affirma
bear on the empirical validity of conditional work. Evidence can be brought to
statements
alternative hypotheses can be tested, and probabilistic statements of the "Ifp, then g'" kind,
about p and q- and other p's and q's and conditions r, s, and t. can be made
word cause" and the expression "causal relation" does Invocation of the.
tive. Indeed, it can be misleading. nothing really construc
In expert hands and used with
methods can help to clarify theoreticalcircumspection, path analysis and related
and empirical relations.18 But when their
espousal and use imply that causes are sought and found,
be misleading. In sum, the elements of deductive logic in such methods can also
relation to conditional
statements, a probabilistic framework and method of work and inference, and the
testing of alternative hypotheses are sufficient aids to scientific ex post facto work
without the excess baggage of causal notions and methods presumably geared to
strengthening causal inferences. We rest the case with some apt words of Bertrand
Russell:

the word "cause" is so inextricably bound up with misleading associations as to

make its complete extrusion from the philosophical vocabulary desirable. the reason
physics has ceased to look for causes is that, in fact, there are no such things. The law of
causality... is a relic of abygone age, surviving. like the monarchy, only because it Is
erroneously supposed to do no harm.19

"H. Blalock, Causal Inferences in Nonextperimental Research. Chapel Hill, N.C.: University of North
Carolina Press, 1961. Blalock cites the following two books as part of the basis of his own thinking:
H. Wold and L. Jureen, Demand Analysis. New York: Wiley, 1953; H. Simon, Models of Men. New York:
Wiley, 1957.
1"Blalock, op. cit. p, 6.
°For an extended discussion of the value and use of path analysis and so-called commonality
analysis in studying relations, see F. Kerlinger and E. Pedhazur. Mutiple Regression in Behavioral
Kesearch. NewYork: Holt,Rinehart and Winston,1973 (in press). chap. 11.
B. Russell, "On the Notion of Cause,with Applications to the Free-WillProblem." In H. Feigl
and M. Brodbeck,eds., Readings in the PhilosopBy of `cience. New York: Appleton, 1958. p. 387.