Journal of the Experimental Analysis of Behavior 2021, 115, 115–128 NUMBER 1 (JANUARY)

Historically recontextualizing Sidman’s Tactics: How behavior

analysis avoided psychology’s methodological Ouroboros
Abdulrazaq A. Imam
John Carroll University

Psychology is undergoing major cultural changes methodologically, with efforts to redefine how psy-
chologists analyze and report their data. Davidson (2018) argued that psychology’s methodological cri-
ses stem from mechanical objectivity involving the adoption of an analytic tool as source of dependable
knowledge. This has led to institutionalization, and eventually uncritical ritualistic use, such as hap-
pened with null hypothesis statistical testing. Davidson invoked the mythological symbol of the
Ouroboros to represent the endless churning of statistical fads. Sidman (1960), in his Tactics of Scientific
Research provided a shield from these problems in terms of the premium he placed on the experience,
expertise, judgement, and decision-making of the scientist, that appear to be absent in psychology’s ritu-
alized processes.
Key words: psychological science, statistical reporting, NHST, behavior analysis, single-subject designs,
small-N designs, Sidman, history

When Tactics of Scientific Research turned Schwartzman, 2018), or single-subject (e.g.,

30 in 1990, The Behavior Analyst devoted a spe-
cial issue to commemorate its achievements
for behavior analysis. In his reply to contribu-
tions to the issue, Sidman pointed out the dif-
ference in the methodology covered in Tactics
from that of psychology, despite sharing
important affinities with other scientific disci-
plines, noting that “[b]ehavior analysts need
not act defensively because of their differences
with psychology; they can stand proud in their
connections with more advanced sciences”
(1990, p. 188). By using and advocating reli-
ance on in-depth study of the individual
organism, Sidman’s (1960) approach fits very
well with what had worked successfully in phys-
iology and medicine (Bernard, 1927/1957)
that predated the introduction and then
increasing adoption of inferential statistics in
psychology in the mid-1940s. The approach
was known to some as N = 1 research at the
time (Dukes, 1965) and now sometimes
referred to as N-of-1 (American Psychological
Association, 2020), single-case (e.g., Boswell &
An earlier version of this paper was presented at the 45tt
annual meeting of the Association for Behavior Analysis
International, in Chicago, IL. I thank Mike Perone, Kobla
Agbota, and two anonymous reviewers for helpful com-
ments on an earlier version of the paper.
Address correspondence to the author at Depart-
ment of Psychology, John Carroll University, 1 John
Carroll Blvd, University Heights, OH 44118, USA.
Email: aimam@jcu.edu
doi: 10.1002/jeab.661
Perone, 1994) research design, but is best
appropriately referred to as small-N, within-
subject design because the modern version usu-
ally involves more than just a single subject or
participant. The alternative approach in psy-
chology usually involves groups of subjects or
participants whose average data are then com-
pared using null hypothesis statistical testing
(NHST; Johnston & Pennypacker, 1993; Poling
et al., 1995; Rozeboom, 1960; Runyon
et al., 1996; Schneider, 2015). Because of the
undue dependence on NHST for data analyses
and interpretation in this approach, many have
roundly criticized it for contributing to lack of
progress in psychological and allied sciences
(Branch, 2014, 2019; Falk & Greenbaum, 1995;
Lambdin, 2012; Meehl, 1978; Morrison &
Henkel, 1970; Rozeboom, 1960).
Davidson (2018) provided a framework for
looking at the fate of methodological practices
in psychology, such as the ritualized NHST
usage, using the Ouroboros metaphor, the
mythical symbol of endless “cycle of destruc-
tion and rebirth” (Merriam-Webster, 2020).
The general tenor of the Ouroboros metaphor
in Davidson’s analysis is that new tools emerge
and are improved upon over time until they
become perceived as means for producing
trustworthy knowledge. As purveyors of trust-
worthy knowledge, the tools become institu-
tionalized as “forms of thinking,” which then
renders them ritualistic. The ritualized tools
then become subjects of widespread criticisms

© 2020 Society for the Experimental Analysis of Behavior

115
116 Abdulrazaq A. Imam
Figure 1
General Outline of Stages in Davidson’s (2018) Methodological
Ouroboros Template (A) and the Framework for Historical Ele-
ments in Psychology That Map On to the Template (B)
Note. Ouroboros. Adapted from Ouroboros by Mohamed
Ibrahim (https://www.learnreligions.com/ouroboros-
4123019). In the public domain.
as “uncritical science,” and, eventually, other
tools emerge to address the criticisms and the
process would begin anew triggering a new cycle
(Davidson, 2018, pp. 469-470; see Fig. 1A). This
is what Davidson argued happened to ritualistic
NHST use in psychology, and which he feared is
likely to become the fate of statistical effect sizes
(ESs) that include the familiar standardized
mean difference, Cohen’s d, and other ESs not
commonly recognized as such (e.g., mean, mean
difference, percentage, etc.; Cohen, 1990;
Cumming, 2014b). Currently, ESs are rec-
ommended as alternatives to the overemphasis of
NHST in psychology today (Cumming, 2014a).
Of course, ESs are not new to psychological
researchers, but they suffered from institutional
neglect until the recent reform efforts of the
Association for Psychological Science (APS) as
articulated by Cumming (2014a; see also
Cohen, 1990). The use of NHST in psychology,
of course, entails the ritualistic adoption of infer-
ential test statistics like the t- or F-test to make
decisions on null and alternative hypotheses fol-
lowing data collection (Cohen, 1990; Dienes,
2016; Runyon et al., 1996). How did behavior
analysis avoid the fate of the cyclical wrath of
psychology’s methodological Ouroboros? The
path of methodological demarcation for behavior
analysis and psychology appears to be laid by his-
torical, epistemological, philosophical, and practi-
cal considerations that set the two apart.
Sidman’s Tactics provided the essential rationales
for the behavior analytic approach, helping to
shield it from the same fate that befell the NHST
ritual in psychology.
The present paper intends to accomplish
three objectives. The first is to recontextualize
Sidman’s contributions in a broader historical
scheme of developments in methodological
innovations and adoption in psychology within
the Ouroboros framework, including the evi-
dence in support of the framework. A second
objective is to give an account of controversial
and sometimes conflicting philosophical and
epistemological considerations about the pur-
ported subject matter of psychology and how to
study it. A final objective is to illustrate how
Sidman’s positions on important themes and his
advocacy for expert judgment shielded behavior
analysis from experiencing similar cyclical fates
proffered by the Ouroboros analysis.
Historical Framework for Contextualizing
Sidman’s Tactics: Psychology’s Methodological
Ouroboros
Davidson (2018) argued that psychology
lives on mechanical objectivity, by which it
tries to arrive at truth, in the methods it uses.
These methods are widely adopted in stan-
dardized forms. According to Davidson, there
is an apparent struggle between the methods
and expert interpretation. In psychology, the
standardized method wins over expertise by
default, largely because of this mechanical
objectivity (see also Cohen, 1990), or “the cul-
tural norm of using quantitative methods and
following standardized rules when producing
scientific knowledge,” which is in sharp oppo-
sition to “trained judgement or to expertise”
(Davidson, 2018, p. 470). In other words, the
standardization did not come about by acci-
dent. It emerged from the epistemological
wheel of mechanical objectivity.
Figure 1B outlines the general historical ele-
ments of the Ouroboros metaphor in general
 Psychology’s Methodological Ouroboros
psychology’s methodology. The precursors to
the elements of the Ouroboros that David-
son (2018) described illuminate the strength of
its impacts on the prevailing methodological
practices in mainstream psychology. Histori-
cally, the predominance of NHST in psychol-
ogy today is really a byproduct of a
hybridization of two different controversial sta-
tistical approaches to inference introduced and
promoted for nearly a decade (1925-1933) by
competing voices on the subject, namely, Fish-
er’s on statistical significance testing (SST) and
Neyman-Pearson’s on statistical hypothesis test-
ing (SHT). Schneider (2015) provided a con-
cise summary of the differences between the
two positions. Schneider’s account of the con-
trasting stances in each approach indicates that
they could not have been more different from
each other (see also Lambdin, 2012). They dif-
fered in their interpretations of p-values
(as evidence against null or to determine how
much error is tolerable), in how and when
(before or after data collection) to use the sta-
tistical tools, and in their purported purpose
(inductive or deductive), among others. An
important point of contention between the two
approaches is on the meaning of p-values espe-
cially when reporting nonsignificant results:
Fisher’s SST would be silent on the null, but
Neyman-Pearson’s SHT would accept null,
which “has nothing to do with the actual truth
of H[0]” (Schneider, 2015, p. 415). Despite
their fundamental differences, the two
approaches were combined, perhaps for expe-
diency, or probably for the purpose of institu-
tionalization discussed further below. The
differences between them point to the reality
that the hybrid, by virtue of merging these
apparently incompatible positions, represents a
bastardization of each approach. Nevertheless,
it is what we have today, standardized for the
purpose of discovery of “truth” in psychology.
How did the hybrid come about? The
answer lies in the textbooks on statistics and
experimental design, which provided the rules
of the game early on. According to Hubbard
and Ryan (2000), Snedecor, who was a chief
promoter of the Fisherian perspective, publi-
shed Calculation and Interpretation of the Analysis
of Variance and Covariance and Statistical
Methods in 1934 and 1937, respectively. There-
after, textbooks like Guildford’s Fundamental
Statistics in Psychology and Education appeared
in 1942, consolidating “Fisher’s logic with the
117
accept-reject dichotomy of Neyman and Pear-
son” (Woods, 2011, p. 41), and the hybrid
NHST has reigned in psychological research
ever since. In light of Davidson’s (2018) analy-
sis, the hybrid NHST became a standard tool
in psychological research and the research rit-
uals that emerged became cemented by post-
World-War-II textbooks. The ritualization has
drawn a myriad of criticisms (e.g., Branch,
1999, 2014; Harlow et al., 1997; Lambdin,
2012; Morrison & Henkel, 1970; Rozeboom,
1960; Woods, 2011), which variously decried
the pervasive adoption of the hybridized
NHST and the conventional p-value, the “oft-
neglected role of interpretation” (Davidson,
2018, p. 470) of statistics, and the mindless rit-
uals involved. Davidson’s main point of con-
tention is that the outcome of the cycle of
mechanical objectivity has resulted in the
recent recommendations for the adoption of
effect sizes such as Cohen’s d, as “new tools”
to adopt as replacement, which would begin,
predictably, another methodological
Ouroboros that would meet the same fate to
ESs as with NHST previously in psychological
research. Beyond the advocacy of ESs, includ-
ing Cumming’s (2014a, b) so called “new” sta-
tistics (new because of the entrenchment of
NHST), Davidson’s strongly recommended
antidote is to include expert judgement in its
use to avoid the fate of NHST in psychology.
Davidson offered examples of advocacy for
expertise, noting, “[r]eintroducing expertise
as a justifiable mode of reasoning requires
ongoing fostering and rehabilitating of psycho-
logical researcher’s content knowledge and
self-confidence” (2018, p. 475). Sidman’s
(1960) anecdote in the next section is illustra-
tive. Of crucial interest in the present paper is
thus the historical circumstances of that first
Ouroboros of the NHST phenomenon in psy-
chology that justify the calls for trained expert
judgement, which in turn provides the context
in which Sidman’s (1960) antidotal contribution
in behavior analysis is to be appreciated. One
might wonder how speculative Davidson’s (2018)
Ouroboros framework is: Is there any evidence
to support the historical developments in the
introduction, adoption, institutionalization, and
then ritualization that marked the life of NHST
in psychological research? I argue in what fol-
lows that such evidence exists. Hubbard and
Ryan (2000) provided the empirical evidence
for what happened in psychology in this regard
118
Figure 2
Percentage of Articles Reporting Combined Probable Errors (PEs) and Critical Ratios (CRs) Vs. P-Values in Empirical Articles Appearing
in 12 APA Journals between 1911 and 1998
Note. No inferential statistics were reported from 1894 to 1910. Adapted from Hubbard and Ryan (2000). Copyright 2000
by Sage Publishing.
in their historical analysis of statistical reporting
practices as outlined in the next section.
Empirical Evidence for Psychology’s
Methodological Ouroboros
Hubbard and Ryan (2000) examined
12 American Psychological Association (APA)
journals, ranging from clinical, educational,
experimental, to general review journals, with
highly consistent correspondences with
Davidson’s (2018) elements of the Ouroboros
outlined in the previous section. Their ana-
lyses revealed that between 1894 and 1909,
covering some 15 years, there were no reports
of inferential statistics in these journals at all.
The first reports of inferential statistics were
two SST statistics, which appeared in 1910: 1)
the critical ratio (cr; of the mean difference to
its standard deviation), and 2) probable error
(pe) defined as the equivalent of two z scores
(Hubbard & Ryan, 2000). In a graphical pre-
sentation of the combined data of crs, pes, and
p-values for all the journals included in their
study from 1911 to 1998, the period in which
these tests were reported in the target
journals, Hubbard and Ryan recorded general
growths in spurts roughly spanning a decade
each followed by leveling off of the growths
about the middle of the 1950s, which marked
the institutionalization of NHST’s mechanical
objectivity. As interesting as these growths in
the three statistics may appear to be, however,
the more important historical development
Abdulrazaq A. Imam
was the process of the emergence of the
hybridization of the two aforementioned statis-
tical approaches (viz., SST and SHT) in the
form of NHST and p-value reporting.
Figure 2 summarizes Hubbard and
Ryan’s (2000) presentation of yearly reporting
of the three statistics showing that crs and pes
were the only inferential statistics reported
from 1911, averaging about 12%, until about
1928. After that, there was a surge in the two
statistics to about 36% on average, until 1941
when they declined to about 25%, before
finally dropping to 4% in 1948. Reporting of
the two statistics peaked through the 1930s,
specifically between 1928 and 1941 (A in
Fig. 2) just about when p-value reporting
started (B in Fig. 2). It was during this immedi-
ate period of hybridization (1940s; see
Fig. 1B), we see a gradual appearance of p-
value reporting until an intermediate phase
between 1942 and 1947 when it became as
common as the other two statistics (C in
Fig. 2) when crs and pes started to decline
(C in Fig. 2) relative to the preceding decade
(A in Fig. 2). By the time of the institutionali-
zation of NHST beginning in the early 1950s,
the other two indices went down virtually to
zero (D in Fig. 2) as p reporting rose to its
asymptote in the mid-1950s (E in Fig. 2). As
noted above, most importantly, there were
48 empirical articles published in these
journals between 1894 and 1909 that did not
report any inferential statistic at all (see Hub-
bard & Ryan, 2000, Table 1, p. 667), indicating
 Psychology’s Methodological Ouroboros
that the latter is not indispensable for empiri-
cal psychological research.
The mapping of the historical evidence
offered in Hubbard and Ryan’s (2000) data
onto Davidson’s (2018) proposition of a
methodological Ouroboros in psychology’s
epistemology proceeds as follows. Before
the1910s, psychological publications in APA
journals reported no inferential statistics at
all, despite decades of empirical research in
psychology. Figure 1B shows that from about
1925 to 1933, the development of new
research tools in psychology was already
underway in the debates between Fisher’s
and Neyman-Pearson’s respective statistical
approaches with their attendant controver-
sies. During the period, cr and pe reporting
appeared and peaked (see Fig. 2, A). By the
1940s, the hybrid NHST had emerged and
began to rise (see Fig. 2, B and C). Trustwor-
thy knowledge was already emanating from
the adoption of the hybrid as promoted by
postwar textbooks. In 1952, the first edition
of the APA publication manual appeared,
marking the beginning of the institutionali-
zation of NHST as a universal tool for
research in psychology and allied disciplines
(see Lambdin, 2012). From 1960 to the pre-
sent, the use of NHST became ritualized in
psychological research (see Fig. 2, E and
beyond) and had effectively displaced the
other two inferential statistics altogether
(see Fig. 2, D). Criticisms of NHST began
early and continued to be a central contro-
versial feature of research in psychology for
decades (e.g., Harlow et al., 1997; Morri-
son & Henkel, 1970), during and beyond the
years covered by Hubbard and Ryan’s data.
The previous (6th) edition of the APA man-
ual, for example, appeared in 2010 rec-
ommending reporting of ESs and other
measures explicitly (Fidler, 2010), rep-
resenting the introduction of an alternative
(to NHST) research tool for psychologists.
The new emphasis on ESs served as the
impetus for Davidson’s cautionary identifica-
tion of psychology’s methodological
Ouroboros, which might undermine the new
emphases on ESs. Despite their long-
standing availability to researchers, Davidson
fears that the new advocacy for use and
reporting of ESs that emerged from the sun-
dry criticisms of the null ritual in psychology,
awaits similar ritualization. Although there
119
had been limited attention to ESs in text-
books (Capraro & Capraro, 2002), recent
evidence of increased reporting of ESs in
select journals following the APS efforts to
reform statistical reporting in psychology
(Imam & Frate, 2019) suggests the initial
stages of the new cycle in the Ouroboros for
ESs may be already underway. The stage for
ritualization of ES reporting and interpreta-
tions, particularly the standardized varieties
such as Cohen’s d, is set by the appeal of
adopting “rules of thumb for characterizing
effect size as low, medium, or large”
(Davidson, 2018; Stewart, 2000, p. 687), in
the same vein as the adoption of p < .05 in
NHST reporting yielded mechanical
objectivity.
Before 1911, psychological researchers
reporting in APA journals did not employ
inferential statistics either of the non-NHST
varieties or of the hybrid NHST that emerged
in psychology textbooks in the 1940s
(Hubbard & Ryan, 2000), as noted above.
How then did researchers evaluate experi-
mental and other data in psychology? Hub-
bard and Ryan noted that before1940,
experimentation with the individual partici-
pant, for the most part, formed the basis for
“inductive claims in psychology” (2000,
p. 666). Schneider (2015) observed that part
of Neyman and Pearson’s objections to Fish-
er’s SST was due to the latter’s “inherent sub-
jective interpretation, as well as the concept
of inductive inference” (p. 415). Pre-1940
research was not conducted in a vacuum,
therefore; psychological researchers thrived,
but with a wholly different approach than was
represented by the predominant 20th century
NHST approach that emerged thence. What
foundational differences with psychology
allowed behavior analysis to evade the same
influences of inferential statistics on data eval-
uation? The apparent marriage of psychologi-
cal research and statistical inference was
wrought with a turmoil that is unnecessary for
a productive scientific endeavor. The histori-
cal elements that brought the coupling about,
however, were born out of philosophical, his-
torical, epistemological, and practical consid-
erations with which earlier generations of
researchers had to contend. The next
section explores some of these to illuminate
how behavior analysis diverged from the rest
of psychology, in principle, and in so doing,
120 Abdulrazaq A. Imam
avoided the rancor associated with the path
psychology took.
Points of Departure: Historical Themes that
Separated Behavior Analysis from Main
Stream Psychology
A confluence of historical and practical fac-
tors surrounding the declared subject matter
of psychology help explain the relationships in
the development of probability theory, treat-
ment of observations and variability, and the
attendant issues of reaching conclusions about
generality (Cohen, 1990; Cowles, 2001;
Stigler, 1992). Cowles’s (2001) historical
account places the nexus of evolution, biomet-
rics, and eugenics as providing the founda-
tional terrain for bringing statistics into
psychology. According to Cowles, “[c]uriosity
about diversity and variability leads to attempts
to classify and to measure” (2001, p. 2). In
essence, what made doing science possible in
that historical context was the marriage of
measurement and statistics. Consequently,
doing science meant classifying all the varieties
and assessing them, which required measuring
them by way of quantification and data collec-
tion, which then required statistical techniques
for data handling and evaluation. The follow-
ing subsections discuss the respective positions
of mainstream psychology and behavior analy-
sis on the historically related issues of the role
of variations, of reaching truth by way of error
estimation to deal with chance events, and of
the use of probability in estimating errors due
to chance with reliance on randomization as a
baseline.
Roots: Of Variations or Variability
There has been a long-standing interest in
the role of variations in nature from perspectives
of what is responsible for them, of their perva-
siveness and extent, of their impacts on organ-
isms individually and in whole, and of how to
control or manipulate them (Cowles, 2001). Var-
iation is ubiquitous as an essential element, in
many areas, from biology to psychology and
behavior analysis, in natural selection and selec-
tion by reinforcement (Cowles, 2001; Donahoe,
2003; Sidman, 1960). Without variation, neither
type of selection is possible (Donahoe, 2003;
Sidman, 1960). Psychology and behavior analysis
have dealt with it in different ways, however.
Whereas psychology embraced inferential statis-
tics and carried it further into the realm of indi-
vidual differences and mental testing (see
Cowles, 2001; Cronbach, 1957), behavior
analysis treated it methodologically as subject
to control (Sidman, 1960) and analysis
(e.g., Page & Neuringer, 1985). Variability is
not treated as a given (as the behavioral engi-
neer might do), but rather as something to
be analyzed for discovering its sources in
functional behavior–environment relations
(Sidman, 1960; cf. Cronbach, 1957).
Perhaps because behavior analysis was never
as preoccupied with issues associated with
mental measurements as the rest of psychol-
ogy was (Cowles, 2001; Cronbach, 1957), the
different attitudes adopted towards variability
obviated the historical necessity for inferential
statistics that was of particular interest to main-
stream psychology. Although Sidman (1960)
devoted two whole chapters of Tactics to the
multifaceted topic of variability, the first men-
tion of it occurred earlier on in the book on
the subject of representativeness, where he
admonished against reliance on the use of
groups and statistics as a solution. The behav-
ior analytic approach to variability has been,
and one hopes still remains, to subject it to
experimental manipulation, as available tech-
niques and technology would permit, in order
to decipher its role in a functional relation-
ship. In response to remarks on the desirabil-
ity of a possible allowance for inferential
statistics in behavior analysis on the aforemen-
tioned 30th anniversary of the publication of
Tactics, for example, Sidman asserted that
Statistical studies can tell us how many
subjects—but not which ones—are
likely to show a particular effect of,
say, age variation. Statistical analysis
cannot yield a functional relation
between the age of any individual and
the effect in which we are interested.
(1990, p. 191)
Positions such as this allowed behavior analysts
to dodge the implied necessity for inferential
statistics in dealing with variability that main-
stream psychology had to contend with in
practice within the context of the historical
marriage of probability theory and psychology
(Cowles, 2001) as it sought to tackle the pre-
senting problem of variations in nature.
 Psychology’s Methodological Ouroboros
Roots: Of Truth, Errors, and Chance
Concerns with observation and measurement
are predicated on the assumption of the discov-
ery of truth. Any measurement presumably
includes some index of error. According to
Stigler (1992), 19th century theory of error nat-
urally assumes that observation by necessity cap-
tures the truth along with a measure of error:
“observation = truth + error” (p. 62). In NHST,
the combination of a true measure and an
error component manifests, as the test statistic
(e.g., t or F value) expressed as a function of
effect variance and error variance, where the lat-
ter includes “other influences like measure-
ment error or effects of uncontrolled variables
(i.e., ‘chance’)” (Branch, 2014, p. 259; see also
Cowles, 2001). Hall (2007) quoted Fisher as
stating that we have “the two desiderata of the
reduction of error and of the valid estimation of
error…” (p. 298). Contradistinctively, the
behavior analytic approach to research is the
focus on controlling variables for their func-
tional effects on the behavior of interest as illus-
trated by the quote from Sidman (1990) above;
rather than incorporate error or chance in a
statistical “treatment” of observed effects, a
point decried by Cohen (1990) as well, behav-
ior analysts seek to control it experimentally.
Roots: Of Probabilities, Chance, and
Randomization
There are various and often incompatible
conceptions of probability in the history of psy-
chology and statistics. In Fisherian SST, for
example, the p-value assesses, after the fact,
the evidence against the null hypothesis in a
given experiment (Schneider, 2015). In that
context, when a p-value indicates that we reject
the null hypothesis, it does not mean that is
the probability that the null is true, nor does it
say anything about the truth of the alternative
hypothesis because the latter does not exist in
SST (Cohen, 1990). In the hybrid NHST tradi-
tion, where the alternative hypothesis does
exit, reflecting the Neyman-Pearson hypothe-
sis testing approach, the p-value represents a
decision rule for rejecting or accepting the
null, which corresponds to acceptance or
rejection of the alternative research hypothe-
sis, respectively (Runyon et al., 1996). In both
cases, probability is used in a way that has a
wholly different meaning in Bayesian statistics
where probability represents “degree of belief”
121
(Cohen, 1990, p. 1307; Dienes, 2016) in the
data. The use of probability, in this case, comes
out of Bayes Factor in terms of the relative
probability of the data given different compet-
ing theories (Dienes, 2016; Kruschke, 2010),
for example. It appears the difference between
the NHST and Bayesian approaches to proba-
bility is what prompted Rozeboom’s (1960)
retort that “the proper inferential task of the
experimental scientist is not a simple accep-
tance or rejection of the tested hypothesis, but
determination of the probability conferred
upon it by the experimental outcome,”
(p. 422) in favor of the latter. Both of the con-
ceptions of probability in the NHST and Bayes-
ian families, nevertheless, are very different
from that of behavior analysts (Skinner, 1953/
1999; Sidman, 1960).
For Skinner, for example, probability is
“simply a way of representing a frequency of
occurrence” (1953/1999, p. 102). In
addressing the role of hypothesis testing in
psychological experimentation, Sidman
appears to be using this very sense of probabil-
ity in this anecdote:
A colleague asked me what I expect
would happen to the on-going avoid-
ance behavior as a result of the
pairing of stimulus and unavoidable
shock. After some consideration I
replied that I could not conceive of
there being no change in the behav-
ior, because the experimental opera-
tion represented a radical alteration
of the subject’s environment. We did
not usually find organisms unre-
sponsive to this kind of manipulation.
Also, I could not conceive that the
probability of the avoidance response
would decline, because if such a reac-
tion were to occur under analogous
conditions outside the laboratory the
species would never have survived to
become subjects for my experiments.
This left only one more possibility.
The probability of the behavior would
have to increase. (1960, p. 5; emphasis
added)
Substantively, the problem of errors found two
distinct solutions in psychology (Cowles, 2001;
122 Abdulrazaq A. Imam
Perone, 1991, 1999) historically. Most of main-
stream psychology utilizes statistical control in
the form of NHST using randomization as base-
line, but culminating in the null ritual (see also
Sidman, 1952). Variants of the NHST statistical
approach are Bayesian (e.g., Dienes, 2016) and
estimation (e.g., Cumming, 2014a). They all
belong in the same family when contrasted with
the alternative approach, namely, experimental
control. Behavior analysis is unique in psychol-
ogy in adopting the latter approach by using
“active” manipulation of relevant variables and
“passive” observation as baseline (in accor-
dance to Bernard’s (1927/1957) distinction
between “active” and “passive” as “experimen-
tation” and “observation”, respectively [p. 6]),
and thereby avoiding the null ritual altogether.
The notion of randomization as a baseline has
its roots in the early works of Charles Peirce
and Joseph Jastrow reported in1885 in which
the former used a blind randomized experi-
ment with himself as the subject on the psycho-
physics of weights (Stigler, 1992). In contrast to
earlier Fechner’s self-experimentation on just-
noticeable difference, they used shuffled decks
of cards to determine which weights to use in
what order in their weight judgments.
According to Stigler, it marked the beginning
of randomization as “an artificial baseline”
(1992, p. 65; see also Hall, 2007) that has
become prominent in psychological research
practices to date. As Johnston and Pen-
nypacker (1993) aptly put it, and reflecting fur-
ther on the connection to the discussion of
variations above,
The problems begin with focusing the
experimental question on whether
there is a difference between control
and experimental groups, instead of
asking about the nature of the rela-
tions between behavior and environ-
mental variables. In the area of
measurement, this approach tends to
encourage grouping individual data
for analysis, which makes it difficult if
not impossible to identify the
behavior–environment relations of
interest. Experimental comparisons
are typically made by comparing data
from control and experimental
groups, which mixes treatment effects
with intersubject variability. Finally,
the interpretive rules of inferential sta-
tistics further shift attention away from
important behavioral issues by focus-
ing on the existence of a “significant”
difference between control and exper-
imental groups. Embedded in these
general problems is an approach to var-
iability that fails to encourage or accommo-
date controlling its sources so as to collect
meaningful data. (p. 324; emphasis
added)
“Baselines” in the two approaches serve very
different purposes, as is evident in the multi-
faceted use of the term in behavior analysis. In
behavior analysis, the baseline is a complex
research tool that may serve multiple functions
(Sidman, 1960). It could be the standard sta-
bility in data sought before implementation of
experimental manipulations (Perone, 1991). It
may also serve as behavioral baseline, such as
in the study of drug effects, or as a technique
for systematic replication, to study transition
states, or learning (Sidman, 1960). The pro-
found irony of the use of randomization as
baseline, of course, is that despite the statisti-
cal requirement for both random sampling
and random assignment in mainstream psy-
chology, in practice, the former is accom-
plished hardly ever, due to heavy reliance on
convenience samples obtained from
participant-pools of college students in place
of sampling randomly from a target popula-
tion (e.g., Jaffe, 2005). Overuse of convenient
samples raises serious questions about the rep-
resentativeness of psychological findings
derived from group designs, raising the fur-
ther irony of the mistaken charge usually
made against small-N designs as lacking in
generality due to small sample sizes.
These array of different philosophical and
epistemological differences between psychol-
ogy and behavior analysis meant that the his-
torical paths taken by each with regard to the
adoption and use of inferential statistics were
very different indeed. They thereby contrib-
uted in no small way to the avoidance in
behavior analysis of the fate that befell psy-
chology methodologically in terms of the place
occupied by inferential statistics in psychologi-
cal research. The next section focuses on the
role of Sidman’s Tactics in providing a formi-
dable alternative to withstand the onslaught of
 Psychology’s Methodological Ouroboros
the NHST fad in the psychology of its time
that turned out to be most productive for
behavior analysis.
Avoiding the Problem of Psychology’s
Methodological Ouroboros in Behavior
Analysis
In addition to prominent works in psychol-
ogy identified as reporting N = 1 research
appearing before the mid-1930s, Dukes (1965)
reported subsequent (1934-1963) 246 N = 1
studies published in 11 different psychology
journals, ranging from the American Journal of
Psychology (AJP), to the Journal of Comparative
and Physiological Psychology (JCPP), to the Jour-
nal of Social Psychology (JSP). Of the 246 studies,
only 27 (11%) were on learning and 7 (3%)
on motivation; the rest focused on categories
in extant psychology, suggesting that the use
and reporting of N = 1 research was not lim-
ited to the area of the experimental analysis of
behavior. The list of journals mentioned in
the Dukes study did not include the Journal of
the Experimental Analysis of Behavior (JEAB),
which premiered in 1958 devoted to publish-
ing behavioral research. Indeed, the listed
journals did not publish N = 1 research exclu-
sively, but some of them served as outlets for
behavior-analytic research. The AJP and the
JCPP, for example, were outlets for learning
and operant research before the founding of
JEAB. A search-term combination of operant
conditioning AND learning AND reinforce-
ment in these journals from 1945 to 1957
yielded 38 hits in AJP and 612 in JCPP. Of the
16 articles using mainly rats, a monkey, and
humans in the AJP search results, 75%
deployed group designs, 56% used group
designs with NHST reporting, 63% altogether
reported NHST, only 19% used small-N
designs, and 6% used small-N designs with
NHST reporting. A sample of 20 articles using
rats, pigeons, monkeys, chimpanzees, cats, and
humans in the JCPP search results revealed
60% used group designs, 50% used group
designs with NHST reporting, 60% altogether
reported NHST, only 35% used small-N
designs, and 10% used small-N designs with
NHST reporting. Incidentally, all nine of
Sidman’s publications and the lone one by
Skinner and Morse in JCPP between 1953 and
1957 used small-N designs without NHST
reporting. One can only wonder what role of
123
editorial decisions warranted the 6% and 10%
cases of small-N studies reporting NHST,
respectively, in the two journals.
These cursory surveys revealed that even
though these journals provided an avenue for
behavior analytic research to reach the rest of
psychology, the preponderance of the oppor-
tunity was against a line of research that advo-
cated small number of subjects, without
reliance on inferential statistics to evaluate
data and to aid reviews in their editorial deci-
sions. It is not surprising then that JEAB would
be founded to fill the gap in disseminating
behavior analytic research. The publication of
Sidman’s (1960) Tactics two years later pro-
vided a shield against the mainstream enthusi-
asm for NHST in psychological research.
The timeliness of Sidman’s (1960) publica-
tion of Tactics cannot be overemphasized in
the impact it was to have in behavior analysis.
It arrived on the scene in the nick of time just
as NHST reached its growth asymptote in psy-
chology (see Fig. 2). By 1960, all journals in
psychology, including the ones that reportedly
had published N = 1 research (Dukes, 1965),
had adopted the approach almost exclusively,
and a new era in mindless NHST ritual
(Cohen, 1990; Davidson, 2018) was well under-
way. The methodological rationales articulated
in Tactics were to provide a shield against the
onslaught of the NHST use pervading the rest
of psychology at the time. A few of the impor-
tant positions are provided in what follows to
illustrate the antidotal contributions of
Sidman’s thinking on a methodology that hel-
ped the field to evade the ubiquitous statistical
misunderstandings that were to engulf the rest
of psychology.
Of Replication and Generality
In psychology at large, there is an ongoing
crisis of replication and reproducibility of
results. The crisis arose in part due to 1) the
reliance on the aforementioned statistical con-
trol and the attendant NHST ritual; 2) as just
noted, the reliance on the often-violated sam-
pling randomization (Jaffe, 2005); and 3) the
rampant misconceptions by users of NHST
about what p-value signifies, specifically that it
carries any information about likely reproduc-
ibility (see Lambdin, 2012). In contrast, in
behavior analysis, there is no crisis of replica-
tion and reproducibility of results mainly due
124 Abdulrazaq A. Imam
to the reliance on experimental control and
therefore to the attendant lack of a null ritual,
and partly due to the inherently built-in repli-
cation in experimental manipulations that
ensure the repeatability and generality of the
results. In behavior analysis, there are three
lines of evidence of replication: 1) session-to-
session consistency in data within a condition;
2) reproducible differences across conditions;
and 3) reproducible results across individual
subjects (Perone, 2019), none of which is
afforded by the group designs of mainstream
psychology and their intimate ties to NHST rit-
uals (see also Johnston & Pennypacker, 1993).
Contrary to what is usually assumed about
the small-N experimental approach, namely,
that it lacks generality due to the sample size
that is usually small compared to what is typical
in the alternative group-design approaches,
generality is of paramount interest and is usu-
ally accounted for in behavior analytic research.
Replication is what affirms generality, especially
of the type sought after by mainstream psychol-
ogists. According to Sidman (1960), “If a series
of experiments is performed, each of which
yields results consistent with the other, the reli-
ability and generality of the individual experi-
ments are greatly enhanced” (p. 135). In
further noting that “[t]he number of such
experiments that must be performed cannot be
prejudged [and that] [i]t will depend upon the
same personal, subjective, pragmatic criteria
that science and individual scientists have
learned to use in evaluating all types of data,”
Sidman (1960, p. 135) draws out the relevance
of expert judgement arising from establishing
generality from replications.
On the Role of Scientific Judgement and
Rules
Recall that a major problem identified in
Davidson’s (2018) Ouroboros allegory in
psychology’s methodology is the opposition to
expert judgement in favor of mechanical objec-
tivity realized via NHST rituals. Davidson was not
the first or only one to advocate expert judge-
ment, however. Also in the context of NHST
practice, Cohen’s (1990) outline of what coun-
ted for appropriate use of statistics included
what he called expert judgement as well, even as
he downplayed the prominence of “inferential
statistics applied with informed judgement
[as] a useful tool” (p. 1311) in the larger view of
the research process. According to Davidson,
however, “…the primary approach to results
ought to be a researcher’s context-appropriate
interpretation; any standardized guidelines for
interpretation ought to be a last resort” (2018,
p. 473). In this regard, Davidson’s analysis comes
to bear on the relevance of Sidman’s (1960)
methodological approach to doing the science
of psychology in a wider historical context.
In comparing behavior analysis to the rest of
psychology, the former’s truth scale involves a
methodology that entails scientific judgement
at various points in the design and implementa-
tion of the experiment, data collection, data
evaluation, and interpretation. Sidman (1960)
expounded at length on many of these levels.
To start with, for example, on the role of rules
in developing scientific judgement, Sidman
advised “…the student [is] not to expect a set
of rules of experimental procedure, to be mem-
orized in classic textbook version,” remarking
that “[t]he pursuit of science is an intensely
personal affair” (Sidman, 1960, preface). By
“personal affair,” of course, he did not mean
doing sloppy science, as he clarified decades
later (Sidman, 1990).
Characteristically, then, evaluating data in
behavior analysis is not conducive to rote ritual
as Sidman variously emphasized. According to
Sidman, 1) there are no established “impartial”
rules for scientific evaluation of data; 2) there
are no firm, safe grounds upon which scientists
can claim a “set of rules” to evaluate data with-
out the illusion of “security”; and 3) mindless
eclecticism nevertheless is not the answer (1960,
p. 41). For Sidman, “the objectivity of science
consists not so much in set rules of procedure as
in the self-corrective nature of the scientific pro-
cess” (1960, p. 43). Whether or not behavior-
analytic research has evolved its own rituals that
hamper its progress, beyond the intrusions of
NHST reporting in behavioral journals, remains
to be examined systematically.
Sidman on Judgement and Replications
It is on the role of judgment in assessing
replication that the premium Sidman (1960)
gave to expert judgement comes to full view.
In addressing how one decides how often one
would need to replicate a finding to be con-
vinced, for example, Sidman noted the conun-
drum such an issue might pose to the
statistician, retorting:
 Psychology’s Methodological Ouroboros
The answer is likely to vary from one
experiment to another. Experi-
menters take into account such factors
as the magnitude of the observed
effect, their confidence in the ade-
quacy of their experimental control,
the consistency of their findings with
related data, the stability of their base-
line conditions, etc. (p. 87)
Sidman’s conception of the experimenter’s
considerations in this context is consistent with
Davidson’s (2018) advocacy of expert knowl-
edge and judgement quoted above as an anti-
dote to the null ritual. Of these
considerations, perhaps it is with respect to
determining the stability of the data that
recent developments in behavior analysis (see,
e.g., Ferron et al., 2019) may be moving away
from the important role of expert judgement
to heavy reliance on statistical approaches, to
the detriment of a previously valued approach
to data-handling in behavior analysis. Ferron
et al. (2019), for example, admitted that
“[a] limitation of preplanned CCDs [changing
criterion designs] is that by setting the phase
lengths prior to starting the study, the
researchers are unable to respond to the data
as they are collected” (p. 10), the latter being
a hallmark of behavior analytic methodology
championed by Sidman and demonstrably
effective and productive for the field. Such
efforts undermine the very thing that Sidman
ascribed to the scientist in pointing out,
Most scientists make such judgements
intuitively, unaware that they are con-
tinuously making complex computa-
tions involving advanced and as yet
unformulated probability theory. Such
evaluations are almost second nature
to them, carried out informally along
with the natural everyday activities of
planning experiments, watching their
progress, changing their course, and
interpreting their results. (1960,
p, 87)
Concluding Remarks and Cautionary Notes
Sidman (1960) placed a premium on the
experience, expertise, judgement, and decision-
125
making of the individual scientist in evaluating
experimental data. This, to the extent that
behavior analysts subscribe to and use it, is the
antidote behavior analysis has against
psychology’s methodological Ouroboros pro-
duced by its mechanical objectivity. Unfortu-
nately, there may be cause for caution in
behavior analysis given recent developments,
such as the one alluded to in the previous sec-
tion, in the field. Ferron et al. (2019) intro-
duced a randomization procedure that
culminated in p-value reporting for evaluating
single-participant data while cognizant of the
deviations involved in longstanding, proven
data handling and evaluation practices in
behavior analysis. To what end, however? I am
not aware of any critique in the literature of
the general or specific tenor of Sidman’s (1960)
elucidation of various facets of what has
become foundational in behavior analytic
research. Attempts to sidestep the informed
judgement required for adequate and effective
use of established principles and processes by
way of statistical options that would detract
from experimental control fundamentals are ill
advised. In fact, the potent antidote to the
problem warrants that the researcher uses
expert judgement about their research. Fur-
thermore, it needs to be used in peer review,
which in many domains involves ritualized calls
for p-values. Peer review can remind
researchers that the claims of likely reproduc-
ibility based on p-values are mistaken.
In another development of concern in
behavior analysis, Lanovaz et al. (2019)
suggested that within-subject replications
might not be necessary due to applied consid-
erations. Why? A problem might arise in situa-
tions where treatment withdrawal is ill advised.
Usually, that leaves only the option of using
the baseline-treatment (AB) design, which is
experimentally the weakest design in the
behavior analytic arsenal largely because the
role of extraneous variables during the treat-
ment (B) phase cannot be ruled out (Poling
et al., 1995). Within-subject replications come
into consideration only in the reversal design
in which the AB sequence might be repeated
in an ABAB format or other variants. When
treatment withdrawal is not possible, why sug-
gest that within-subject replication is not nec-
essary? First, by definition, applied concerns
about ethical, financial, and/or social costs
unquestionably allow for the use of the AB
126 Abdulrazaq A. Imam
design option so long as the demonstrable out-
come is acceptable to the client, stakeholders,
and/or the practitioner. Why lower the design
standards for that purpose then? Second, alter-
natives that allow for both within- and
between-subject replications are available, and
demonstrably effective, to both scientists and
practitioners alike (Gast, 2014). Such alterna-
tives include various multiple-baseline designs
that keep the AB structure in place, thereby
obviating a need for withdrawals, while
maintaining acceptable experimental or prac-
tical comparisons and demonstrating treat-
ment effectiveness in contradistinction to the
weaker simple AB design alone. The hybridiza-
tion of SST and SHT into NHST provides an
excellent illustration of the adverse conse-
quences of embracing inferential statistics
blindfolded by extracurricular considerations,
even if well-intentioned. Historic criticisms of
NHST were not based on how convenient or
cost-worthy it may be, but on fundamental log-
ical flaws in its use and interpretations (see,
e.g., Bernard, 1927/1957; Branch, 1999, 2014,
2019; Harlow et al., 1997; Lambdin, 2012; Mor-
rison & Henkel, 1970; Perone, 1999, 2019;
Rozeboom, 1960; Woods, 2011). Until such
fundamental problems arise and are articu-
lated for small-N designs, questioning the
value of replications is ill advised and could
begin a slippery slope we may not be able to
reverse in practice. Smith’s (2012) review indi-
cates there is no agreement yet on what statis-
tical analysis standards are to be brought to
bear on these kinds of designs.
The latest journal-article reporting stan-
dards of the APA now explicitly provide for
what is characterized as N-of-1 studies under
special designs category (American Psychologi-
cal Association, 2020). As mentioned in the
introduction, it is customary to describe N-of-1
as single-case designs (SCD) especially in prac-
tice quarters (e.g., Boswell & ests. We should be cognizant of the distinction
Schwartzman, 2018). Poling et al. (1995) made
a further distinction of case study that involves
implementing treatments without a baseline
condition. Although some may use these terms
interchangeably, they carry different meanings
and implications for use and practice. For
example, whereas SCDs may be appropriate
for practitioners who deliver services in pack-
ages to individual clients, it may not be appro-
priate for applied behavior analysts who
actually assess functional relationships, as
implied in the “analysis” of their title. Addi-
tionally, small-N designs cannot rely on a sin-
gle subject or participant, literally, as in case
studies or single-case designs, because of the
value of intersubject replications for esta-
blishing reliability and generality of findings
(see Sidman, 1960, pp. 74-85). As Sidman put
it, “replication of an experiment with two sub-
jects establishes greater generality for the data
among individuals of a population than does
replication with two groups of subjects whose
individual data have been combined” (1960,
p.75). Basic and applied behavior analysts have
a need of small-N designs for establishing the
reliability and generality of findings.
The distinction I am making here requires,
perhaps, a further differentiation within
behavior analysis to distinguish the concerns
of practitioners in contrast to basic/applied
research needs of the field. Service delivery
may be ritualized because it is packaged, but
that is not experimental analysis of behavior,
nor applied behavior analysis. As such, a fur-
ther distinction is suggested of those inter-
ested in applied behavior analysis from
practitioners who have no interest in contrib-
uting to the “science” but are themselves “con-
sumers” (albeit for implementation purposes)
of the products of that science, basic and
applied alike, akin to the “technological” (à la
Baer et al., 1968, p. 95), or packaged service
delivery. In the absence of such differentia-
tion, we run the risk of conflation and confu-
sion of methodological standards, for students,
researchers, and practitioners alike. The
recent developments in the professionalization
of behavior analysis have their major advan-
tages in drawing droves of interest the field
has not seen in decades. With it, though, come
tensions that are reminiscent of the earliest
history of behaviorism (O’Donnell, 1985)
between basic science and professional inter-
between “basic” and “engineering,” which as
Sidman noted, “yield different kinds of knowl-
edge” (1990, p. 191), one focused on pro-
cesses, the other concerned with outcomes in
the main (cf. Cronbach, 1957).
Nevertheless, there should be no doubt
about the value of proven methodological
practices within behavior analysis until there
are considered critiques of those practices;
when they have started to prove inadequate to
the task, then extant alternative approaches
 Psychology’s Methodological Ouroboros
may be desirable. In the meantime, we should
not stop “asking question(s) of nature”:
According to Sidman, “Data can be negative
only in terms of a prediction. When one sim-
ply asks a question of nature, the answer is
always positive. Even an experimental manipu-
lation that produces no change in the depen-
dent variable can provide useful and often
important information” (1960, p. 9). Such was
Sidman’s insights into scientific inquiry in
behavior analysis and it is not time to abandon
them. We should not run to statistical control.
References
American Psychological Association. (2020). Publication
manual of the American Psychological Association: The offi-
cial guide to APA style (7th ed.). https://doi.org/10.
1037/0000165-000
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some cur-
rent dimensions of applied behavior analysis. Journal
of Applied Behavior Analysis, 1, 91-97. https://doi.org/
10.1901/jaba.1968.1-91
Bernard, C. (1927/1957). An introduction to the study of
experimental medicine. Dover Publications.
Boswell, J. F., & Schwartzman, C. M. (2018). An explora-
tion of intervention augmentation in a single case.
Behavior Modification. Advance online publication.
https://doi.org/10.1177/0145445518796202
Branch, M. (1999). Statistical inference in behavior analy-
sis: Some things significance testing does and does
not do. The Behavior Analyst, 22, 87-92. https://doi.
org/10.1007/BF03391984
Branch, M. (2014). Malignant side effects of null-hypothesis
significance testing. Theory & Psychology, 24, 256-277.
https://doi.org/10.1177/0959354314525282
Branch, M. (2019). The “reproducibility crisis”: Might the
methods used frequently in behavior analysis help?
Perspectives on Behavior Science, 42, 77-89. https://doi.
org/10.1007/s40614-018-0158-5
Capraro, R. M., & Capraro, M. M. (2002). Treatments of
effect sizes and statistical significance tests in text-
books. Educational and Psychological Measurement, 62,
771-782. https://doi.org/10.1177/001216402236877
Cohen, J. (1990). Things I have learned (so far). American
Psychologist, 45, 1304-1312. https://doi.org/10.1037/
0003-066X.45.12.1304
Cowles, M. (2001). Statistics in psychology: An historical per-
spective. Lawrence Erlbaum.
Cronbach, L. J. (1957). The two disciplines of scientific
psychology. American Psychologist, 12, 671-684. https://
doi.org/10.1037/h0043943
Cumming, G. (2014a). The new statistic: Why and how?
Psychological Science, 25, 7-29. https://doi.org/10.1177/
0956797613504966
Cumming, G. (2014b, May 22). The new statistics: Effect
sizes and confidence intervals (Part 3: Research integ-
rity and the new statistics) [video file]. Retrieved from
https://www.psychologicalscience.org/members/new-
statistics
Davidson, I. J. (2018). The Ouroboros of psychological
methodology: The case of effect sizes (Mechanical
127
objectivity vs. expertise). Review of General Psychology,
22, 469-476. https://doi.org/10.1037/gpr0000154
Dienes, Z. (2016). How Bayes factors change scientific
practice. Journal of Mathematical Psychology, 72, 78-89.
https://doi.org/10.1016/j.jmp.2015.10.003
Donahoe, J. W. (2003). Selectionism. In K. A. Lattal &
P. N. Chase (Eds.), Behavior theory and philosophy
(pp. 103-128). Kluwer Academic.
Dukes, W. F. (1965). N = 1. Psychological Bulletin, 64, 74-79.
https://doi.org/10.1037/h0021964
Falk, R., & Greenbaum, C. W. (1995). Significance tests
die hard: The amazing persistence of a probabilistic
misconception. Theory & Philosophy, 5, 75-98. https://
doi.org/10.1177/0959354395051004
Ferron, J., Rohrer, L. L., & Levin, J. R. (2019). Randomiza-
tion procedures for changing criterion designs. Behav-
ior Modification. Advance online publication. https://
doi.org/10.1177/0145445519847627
Fidler, F. (2010). The American Psychological Association
Publication Manual Sixth edition: Implications for sta-
tistics education. In C. Reading (Ed.), Data and context
in statistics education: Towards an evidence-based society.
Proceedings of the Eighth International Conference on Teach-
ing Statistics. International Statistical Institute.
Gast, D. L. (2014). Replication. In D. L. Gast & J. R.
Ledford (Eds.), Single case research methodology: Applica-
tions in special education and behavioral sciences (pp.105-
123). Routledge.
Hall, N. S. (2007). R. A. Fisher and his advocacy of ran-
domization. Journal of the History of Biology, 40, 295-325.
doi:https://doi.org/10.1007/s10739-006-9119-z
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.) (1997).
What if there were no significance tests? Lawrence
Erlbaum.
Hubbard, R., & Ryan, P. A. (2000). The historical growth
of statistical significance testing in psychology—and
its future prospects. Educational and Psychological
Measurement, 60, 661-681. https://doi.org/10.1177/
0013164400605001
Imam, A. A., & Frate, M. (2019). A snapshot look at repli-
cation and statistical reporting practices in psychology
journals. European Journal of Behavior Analysis, 20, 204-
229. https://doi.org/10.1080/15021149.2019.1680179
Jaffe, E. (2005, September 10). How random is that?
Observer, 18(9). https://www.psychologicalscience.
org/observer/how-random-is-that
Johnston, J. M., & Pennypacker, H. S. (1993). Strategies and
tactics of behavioral research (2e). Lawrence Erlbaum.
Kruschke, J. L. (2010). What to believe: Bayesian methods
for data analysis. Trends in Cognitive Science, 14, 293-
300. https://doi.org/10.1016/j.tics.2010.05.001
Lambdin, C. (2012). Significance tests as sorcery: Science
is empirical—significant tests are not. Theory & Psy-
chology, 22, 67-90. https://doi.org/10.1177/
0959354311429854
Lanovaz, M. J., Turgeon, S., Cardinal, P., & Wheatley, T. L.
(2019). Using single case designs in practical settings:
Is within-subject replication always necessary? Perspec-
tives on Behavioral Science, 42, 153-162. https://doi.
org/10.1007/s40614-018-0138-9
Meehl, P. E. (1978). Theoretical risks and tabular asterisks:
Sir Karl, Sir Ronald, and slow progress of soft psychol-
ogy. Journal of Consulting and Clinical Psychology, 46,
806-834. https://doi.org/10.1037/0022-006X.46.4.806
128 Abdulrazaq A. Imam
Merriam-Webster (2020). Ouroboros. In Merriam-Webster Dic-
tionary (Version 5.2.0.) [Mobile app]. App Store. https://
apps.apple.com/us/app/merriam-webster-dictionary/
id399452287
Morrison, D. E., & Henkel, R. E. (Eds.) (1970). The signifi-
cance test controversy: A reader. Aldine.
O’Donnell, J. M. (1985). The origins of behaviorism: American
psychology, 1870-1920. University Press.
Page, S., & Neuringer, A. (1985). Variability is an operant.
Journal of Experimental Psychology: Animal Behavior
Processes, 11, 429-452. https://doi.org/10.1037/0097-
7403.11.3.429
Perone, M. (1991). Experimental design in the analysis of
free-operant behavior. In I. H. Iversen & K. A. Lattal
(Eds.), Experimental analysis of behavior, Part 1 (pp. 135-
171). Elsevier.
Perone, M. (1994). Single-subject designs and develop-
mental psychology. In S. H. Cohen & H. W. Reese
(Eds.), Life-span developmental psychology: Methodological
considerations (pp. 95-118). Taylor & Francis. https://
doi.org/10.4324/9781315792712-5
Perone, M. (1999). Statistical inference in behavior analy-
sis: Experimental control is better. The Behavior
Analyst, 22, 190-116. https://doi.org/10.1007/
BF03391988
Perone, M. (2019). How I learned to stop worrying and
love replication failures. Perspectives in Behavioral
Science, 42, 91-108. https://doi.org/10.1007/s40614-
018-0153-x
Poling, A., Methot, L. L., & LeSage, M. G. (1995). Funda-
mentals of behavior analytic research. Plenum Press.
Rozeboom, W. W. (1960). The fallacy of null hypothesis
significance test. Psychological Bulletin, 57, 416-428.
https://doi.org/10.1037/h0042040
Runyon, R. P., Haber, A., Pittenger, D. J., &
Coleman, K. A. (1996). Fundamentals of behavioral sta-
tistics (8e). McGraw-Hill.
Schneider, J. W. (2015). Null hypothesis significance tests.
A mix-up of two different theories: The basis for
widespread confusion and numerous misinterpreta-
tions. Scientometrics, 102, 411-432. https://doi.org/10.
1007/s11192-014-1251-5
Sidman, M. (1952). A note on functional relations
obtained from group data. Psychological Bulletin, 49,
263-269. https://doi.org/10.1037/h0063643
Sidman, M. (1960). Tactics of scientific research: Evaluating
experimental data in psychology. Authors Cooperative.
Sidman, M. (1990). Tactics; In reply… The Behavior Analyst,
13, 187-197. https://doi.org/10.1007/BF03392538
Skinner, B. F. (1999). The analysis of behavior. In B. F.
Skinner (Ed.), Cumulative record (pp. 101-107). Cop-
ley. (Reprinted from “Some contributions of an
Experimental Analysis of Behavior to psychology as a
whole,” 1953, American Psychologist, 8, 69-79, https://
doi.org/10.1037/h0054118)
Smith, J. D. (2012). Single-case experimental designs: A
systematic review of published research and current
standards. Psychological Methods, 17, 510-550. https://
doi.org/10.1037/a0029312
Stewart, D. W. (2000). Testing statistical significance test-
ing: Some observations of an agnostic. Educational and
Psychological Measurement, 60, 685-690. https://doi.
org/10.1177/00131640021970826
Stigler, S. M. (1992). A historical view of statistical con-
cepts in psychology and educational research. Ameri-
can Journal of Education, 101, 60-70. https://doi.org/
10.1086/444032
Woods, B. (2011). What’s still wrong with psychology, any-
way? Twenty slow years, three old issues, and one new
methodology for improving psychological research
[Unpublished master’s thesis]. University of
Canterbury.
Received: August 18, 2020
Final Acceptance: November 27, 2020
Editor-in-Chief: Mark Galizio
Associate Editor: Iver Iversen