Psychology is a science.

Discuss this statement in light of the “reproducibility

crisis” within Psychology

‘Psychology is a science’ is a statement that has been the subject of much debate, dating back to
when Wilhelm Wundt first opened the Institute for Experimental Psychology at the University of
Leipzig, in 1879. This was seen as the first step to make a distinction between Psychology as a
branch of Philosophy and a scientific discipline. Wundt’s focus on scientific methodology through
introspection, (the study of simple sensory processes through stimulus exposure, internal reflection
and analysis), provided the link with Psychology to the ‘natural sciences’.

This essay will discuss the statement ‘Psychology is a science’ in light of the “reproducibility crisis”
within Psychology, by firstly outlining some historical concepts which have led to an understanding
of the key features of science; and then utilising these features to review the range of literature
surrounding the reproducibility crisis, whilst also acknowledging an alternative view of psychology as
a science.

It should be noted however, that there is no singular definition of ‘science’ or ‘scientific method’.
Science has its roots in philosophy and different schools of thought have shaped the system from
ancient times until modern day.

An empirical approach has been followed since the scientific revolution in the 17 th century, when
John Locke influenced a shift in thinking away from the ‘rationalism’ movement of Plato and
Descartes, which prescribes that knowledge is innate and can be developed by deductive reasoning
(starting with a known fact and using logic to reach a certain conclusion) , to the approach that
humans are born with a blank slate and that knowledge is gained through the senses and is
experienced – ‘empiricism’. Empiricism purported ‘inductive reasoning’ (attributed to Sir Francis
Bacon) in which generalisations are made about repeated observations until there is a probable
conclusion that the generalisation is true.

Scientific process further evolved in the 20th century and whilst an empirical approach is still
followed, a mixture of deductive and inductive reasoning is used. This is known as the hypothetico-
deductive model. The hypothetico-deductive model was introduced by Karl Popper (1935) and
suggests that theories can be created by observation and inductive reasoning; deductive reasoning
should then be used to generate a hypothesis which should be tested as part of an experiment. The
differentiation is that hypothesis testing should be based on falsification e.g. the aim should be to
disprove the theory (Brysbaert & Rastle, 2013).

The combination of the empirical approach and hypothetico-deductive model make up the
commonly used scientific method undertaken by Psychology researchers today, which is one of the
main arguments why Psychology is a science. In general terms, the key features of science are,
objectivity (removal of bias); being able to control conditions to establish cause and effect; making
predictions that can be proved or disproved through hypothesis testing; predictability/reliability,
which is the degree of accuracy in which findings enable the researcher to predict future behaviour;
and finally, a fundamental view of the scientific method is that it should produce an established
result. The rigour and control around the experiment means that any published result should be
dependable and one that can be copied or ‘replicated’, however, across the scientific disciplines
there is a variety in specialised techniques used. (AAAS 1989:25-26, 29, in Gauch, 2003:7; Gilbert et
al, 2016; McLeod, S. A., 2008; Sturm, 2006; Turner, 2019).

In recent years there has been much academic and public attention around the ‘reproducibility crisis’
in Psychology, however issues with replication and reproducibility stem as far back as the end of the
19th century when Wundt’s introspective approach was challenged by the ‘imageless thought’
phenomena. This called into question the objectivity of the approach from a scientific method
perspective (B.J. Wiggins & C.J. Chrisopherson, 2019; J.R. Stevens, 2017; Open Science Collaboration,
2015; Wolff, W., Baumann, L., & Englert, C., 2018).

Reproducibility and replication are used interchangeably in the literature. Turner, describes
reproducibility/replicability as:

“…the growing realization among researchers and psychology professionals that the literature of
psychological science does not represent a catalogue of established empirical results and their
response to this realization” (2019:2).

A review of the literature provides a common set of issues in psychology which impact on
reproducibility, as well Psychology’s status as a science. Bishop (2019), refers to these as the ‘four
horsemen of the reproducibility apocalypse’. These are:

‘HARKing’ – hypothesising after the results are known. HARKing could be viewed as contradicting
Poppers’ hypothetico-deductive model fundamental to the scientific method and detrimental to the
status of psychology as a science (Kerr, 1998), however, Smith et al (2017) argue that HARKing is part
of science and if psychology is to progress as a science, results should be reviewed, and an improved
hypothesis should be created. This opinion would support the modification process or ‘sophisticated
falsificationism’ acknowledged by Popper. This would only apply if the modification does not make
the theory less falsifiable, however, it could be argued that distinguishing between an accepted
modification and ad hoc modification is complex (Brysbaert & Rastle, 2013:379).

The second horseman, closely related to the first is ‘P-Hacking’, when a researcher fails to use the p-
value for confirmatory analyses and instead uses it for exploratory research (Nosek & Lakens, 2014).
This can then lead to what Borges (1941) describes as the ‘Garden of forking paths’ where the
researcher cuts the data in a number of different ways to find a significant p<0.05 value. The
manipulation of the statistics in this way could be said to contravene the ‘predictability’ feature of
science, demonstrated by the Simmons et al, 2012 article which discussed researcher ‘degrees of
freedom’ and found a significant result with study participants being 18 months younger after
listening to The Beatles hit ‘When I’m sixty-four’. There is also a view that there is an unnecessary
over-reliance on the statistical significance of p<0.05. In his keynote speech at the British
Psychological Society’s debate about the reproducibility and replication crisis in psychology, Marcus
Munafo, commented that the null hypothesis test is over 100 years old, and that P value is entirely
arbitrary. Munafo’s view was that the ‘crisis’ could be reframed into an opportunity by focusing on
quality control and ‘applying scientific methods to the process of science itself’ (Rhodes, 2016).
Many authors contend that reproducibility will be increased when more focus is put on the power of
studies rather than the p-value (Ferguson, C.J., 2009; Ioannidis, 2005, Wagenmakers et al, 2011a).

The 3rd horseman is Low Power. This is described as studies that are conducted with sample sizes
that are too low to show the desired effect. This impacts the reliability of Psychology studies as
most of the effect sizes are small and underpowered (Bishop, 2019; Cumming, 2014; Maxwell, 2004).
In addition to this, it is argued that not enough studies are replicated (Makel et al, 2012; Tajika et al
2015). A further complexity appears to be the ability to replicate studies with accuracy. The Open
Science Collaboration (OCS) (2015) research paper estimating the reproducibility of psychological
science professed to use high powered studies in the one hundred replications conducted. When the
study was published the key headlines were fixated on the c.40% p-value replication (Handwerk, B.,
2015). Whilst the intention of the study was to amass more knowledge and draw attention to the
improvements that can be made in the scientific community methodology, the article received
criticism when published relating to the research methodology used. This included challenges
around power, as each study was only replicated once, and argued that on analysis of the data
provided and accounting for error and bias, the reproducibility rate was a lot higher than reported
(Carp, 2012; Etz & Vandekerckhove, 2016; Gilbert et al, 2016). A response to the challenge was
issued swiftly by the OCS (2016) stating the comments by Gilbert et al, (2016) were based on
“statistical misconceptions and selective interpretation of correlational data”. It could be argued
that the divided opinion on the method used in the study will only further validate public scepticism
surrounding the scientific status of Psychology (Ferguson, C.J., 2015; Lilienfeld, 2012).

Finally, the last horseman Bishop refers to is ‘Publication Bias’, the view that journals have a bias
towards publishing only positive results and those with statistically significant p-values (Masicampo,
E. J., & Lalande, D. R., 2012). It is proposed that there is bias towards new and exciting research
rather than a focus on the central tenet of replication work (Francis, 2012). This issue is not new as
illustrated with the ‘file drawer problem’ being quoted by Rosenthal in 1979. The lack of citing null
results in published works means not only that work with potential type II errors is not published but
also, the published data that is heavily relied on for meta-analysis (the cumulative knowledge
gathered) has the potential to be false. This could be seen to negatively impact all the key features in
the scientific method mentioned (Rosenburg, 2005). Calls have been made to reform the culture
within the community as the current publishing framework has not been changed for 400 years and
there are many criticisms about the effectiveness of the peer review process and the reward and
incentive structure for researchers (Munafo et al, 2017).

It should be noted at this stage that the reproducibility crisis is not limited to Psychology as a
discipline. It has been an issue of contention across several of the scientific disciplines including the
natural sciences (Stevens, 2017). Paradoxically, this supports the statement that psychology is a
science. According to Kuhn’s theory of scientific progress, the current crisis could be reflected as a
natural stage that follows a period of ‘normal science’, where increasingly the number of anomalies
and falsifications cause a confidence crisis in the current ‘paradigm’ or scientific method. Kuhn
states this will ultimately lead to a scientific revolution and a new paradigm. During the crisis stage
“a scientific discipline is more open to bold, alternative conjectures that question the core of the
paradigm” (Brysbaert & Rastle, 2013:386). The focus currently on the reproducibility crisis could be
seen as this stage in Kuhn’s theory, however, there is also a possibility that as the discipline is
relatively ‘young’ that it is still in the ‘pre-science’ stage yet to form a paradigm (Jogalekar, 2013).

There is a growing body of literature, which suggests several ways in which the scientific community
can tackle the reproducibility crisis including being more transparent with data for replications, pre-
registering hypothesis and methodology, a review of the funding structure, editors at journals being
blind to results, and many more that fall outside the scope of this essay discussion. (Chambers, et al
2019; Moshontz et al, 2018; Wagenmakers et al, 2016). A key theme emerging is that the culture in
the scientific community has been too focused on productivity (publishing papers) and statistics
rather than quality control, and there is a balance to be struck with novel research and replication to
progress science and improve the accumulative knowledge of the human mind (Munafo et al, 2010).
Others comment that the emphasis on improving the scientific methodology does not go far enough
into the root cause of the reproducibility crisis and fails to fully take account of the underpinning
philosophy of science (Wiggins & Chrisopherson, 2019).
The volume of literature focused on scientific method in relation to the ‘crisis’ could be reflected by
those with post-modernist views as social construction. The post-modernist view challenged the
elite status of the scientific community and the power base that had grown in the Western
civilisation, refuting that scientific knowledge was superior to another type of knowledge.

An alternative approach is the hermeneutic approach. This approach rejects the scientific
methodology, as it does not take account of the holistic aspects of the human experience, including
focusing more on understanding how people behave in their own social contexts. Wilhelm Dilthey
made the distinction between the natural sciences, which the fundamental scientific method is
synonymous with, and mental science, which takes account for how humans perceive and interpret
the world around them. This approach was developed further in the founding of ‘humanistic
psychology’ by Carl Rogers and Abraham Maslow, who maintained that humans had free will and
each individual was different, inherently good and self-motivated. This school of thought focused
more on qualitative methods of research such as interviews (Brysbaert & Rastle, 2013).

Many view the hermeneutic approach as not being scientific as the hypothetico-deductive model is
more difficult to apply given the subjectivity and differences in human beings and methodologies are
more complex to define. This may be viewed as a key difference between the research and
profession arms of psychology and there is a question around whether the narrow views of what
constitutes the traditional scientific method is still relevant for today’s environment (Bevilacqua et
al, 1995; McCaffrey et al, 2012).

In conclusion, a discussion on the available literature demonstrates that the statement ‘Psychology is
a science, in light of the reproducibility crisis’, ironically is down to interpretation. The current
paradigm may be too narrow and whilst a review of history demonstrates the scientific method
enabled psychology as a discipline to be recognised as a science, the continued focus on the
hypothetico-deductive model and proving significance of research findings through quantitative
means could now be seen to be having the opposite effect. In turn, attention on reproducibility
perpetuates a culture of focusing on statistics and continues to divide the research community. It
may be that Psychology is leading the way as a scientific discipline in its attempts to be transparent
and address the issue, however it appears a significant amount of discussion around the possible
resolutions continues to be focused around improving the scientific method and does not take
account of other approaches. To progress psychology as a science for the good of the human
condition perhaps a new paradigm which maximises the use of both the scientific and hermeneutic
approach involving the academic and psychology profession is required.
 References

Anderson, C. J., Bahnik, S., Barnett-Cowan, M., Bosco, F. A., Chandler, J., Chartier, C. R., Cheung, F.,
Christopherson, C. D., Cordes, A., Cremata, E, J., Della Penna, N., Estel, V., Fedor, A., Fitneva, S, A.,
Frank, M. C., Grange, J. A., Hartshorne, J. K., Hasselman, F., van der Hulst, M…Zuni, K. (2016).
Response to comment on “Estimating the reproducibility of psychological science”. Science,
351(6277), 1037c.

Bevilacqua, F., Giannetto, E. Hermeneutics and Science Education: The role of history of science. Sci
Educ 4, 115–126 (1995). https://doi.org/10.1007/BF00486579.

Bishop, D. (2019 Apr 24). Rein in the four horsemen of irreproducibility. Nature, vol 568/435
https://www.nature.com/articles/d41586-019-01307-2

Brysbaert, M. and Rastle, K. (2013). Historical and Conceptual Issues in Psychology (2nd Edition).
Prentice-Hall.

Carp J. (2012). The secret lives of experiments: methods reporting in the fMRI literature.
Neuroimage. Oct 15;63(1):289-300. doi: 10.1016/j.neuroimage.2012.07.004. Epub 2012 Jul 10.
PMID: 22796459.

Chambers, C. D., Forstmann, B. and Pruszynski, J. A. (2019). Science in flux: Registered reports and
beyond at the European Journal of Neuroscience. European Journal of Neuroscience 49(1), pp. 4-5.
(10.1111/ejn.14319)

Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25, 7–29.
http://dx.doi.org/10.1177/0956797613504966

Estes, W. K. (1997). Significance Testing in Psychological Research: Some Persisting Issues.

Psychological Science, 8(1), 18–20.

Etz A, Vandekerckhove J (2016) A Bayesian Perspective on the Reproducibility Project: Psychology.

PLoS ONE 11(2): e0149794. doi:10.1371/journal.pone.0149794

Ferguson, C.J. (2009). Is psychological research really as good as medical research? Effect size
comparisons between psychology and medicine. Review of General Psychology, 13(2), 130–136.

Ferguson, C.J. (2015). ‘Everybody knows psychology is not a real science’: Public perceptions of
psychology and how we can improve our relationship with policymakers, the scientific community,
and the general public. American Psychologist, 70, 527–542

Francis, G. (2012). Publication bias and the failure of replication in experimental psychology.
Psychonomic Bulletin and Review, 19, 975–991. http://dx.doi.org/10.3758/s13423-012-0322-y

Gauch, H.G., Jr. (2003) Scientific Method in Practice. Cambridge University Press.

Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016). Comment on “Estimating the
reproducibility of psychological science.” Science, 351(6277), 1037–1037.
https://doi.org/10.1126/science.aad7243
Handwerk, B. (2015, August 27). Scientists Replicated 100 Psychology Studies, and Fewer Than Half
Got the Same Results. Retrieved Nov 1, 2021, from Turner 2016
https://www.smithsonianmag.com/science-nature/scientists-replicated-100- psychology-studies-
and-fewer-half-got-same-results-180956426/

Hatfield, G. (1997). Wundt and psychology as science: Disciplinary transformations. Perspectives on

Science, 5,349–382.

Ioannidis JPA (2005). Why most published research findings are false. PLoS Med 2(8): e124. DOI:
10.1371/journal.pmed.0020124

Ioannidis, J. P. (2012). Why science is not necessarily self-correcting. Perspectives on Psychological

Science, 7(6), 645-654.

Jogalekar, A. (2013). Is psychology a “real” science? Does it really matter? Scientific American.
https://blogs.scientificamerican.com/the-curious-wavefunction/is-psychology-a-e2809creale2809d-
science-does-it-really-matter/

Kerr, N.L.,(1998) HARKing: Hypothesizing After the Results are Known. Personality and Social
Psychology Review, Vol. 2, No.3, 196-217

Lilienfeld, S. O. (2012). Public skepticism of psychology: Why many people perceive the study of
human behavior as unscientific. American Psychologist, 67, 111–129.

Lombrozo, T. https://www.npr.org/sections/13.7/2017/01/30/512402110/what-makes-science-
science?t=1636402649640

Makel, M., Plucker, J., Hegarty, B. (2012) Replications in Psychology Research How Often Do They
Really Occur? Perspectives on Psychological Science 7(6):537-542. DOI: 10.1177/1745691612460688

Masicampo, E. J., & Lalande, D. R. (2012). A peculiar prevalence of p values just below. 05. The
Quarterly Journal of Experimental Psychology, 65(11), 2271-2279.

Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: Causes,

consequences, and remedies. Psychological Methods, 9, 147–163. http://dx.doi.org/10.1037/1082-
989X.9.2.147

McCaffrey G, Raffin-Bouchal S, Moules NJ. (2012) Hermeneutics as Research Approach: A

Reappraisal. International Journal of Qualitative Methods. July 2012:214-229.
doi:10.1177/160940691201100303

McLeod, S. A. (2008). Psychology as a science. Simply Psychology.

www.simplypsychology.org/science-psychology.html

Moshontz, H., Campbell, L., Ebersole, C. R., IJzerman, H., Urry, H. L., Forscher, P. S.,... Chartier,
C. R. (2018). The psychological science accelerator: Advancing psychology through a distributed
collaborative network. Advances in Methods and Practices in Psychological Science, 1, 501–515.
http://dx.doi.org/10.1177/2515245918797607
Munafò, M. R., & Flint, J. (2010). How reliable are scientific studies?. The British Journal of
Psychiatry, 197(4), 257-258.
Munafò, M. R., Nosek, B. A., Bishop, D. V., Button, K. S., Chambers, C. D., du Sert, N. P., & Ioannidis, J.
P. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021. doi:
10.1038/s41562-016-0021

Nosek, B. A., & Lakens, D. (2014). Registered reports: A method to increase the credibility of
published results. Social Psychology, 45, 137-141. doi: 10.1027/1864-9335/a000192

Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science,
349(6251), 943-953.

Rhodes, E. (2016) https://thepsychologist.bps.org.uk/what-crisis-reproducibility-crisis

Rosenburg, M.N. (2005). The file drawer problem revisited: A general weighted method for
calculating fail-safe numbers in meta-analysis. Evolution, 59(2), 2005, pp. 464–468. DOI:
10.1111/j.0014-3820.2005.tb01004.x

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological
Bulletin, 86(3), 638-641.

Smith, J. K., Smith, L. F., & Smith, B. K. (2017). The reproducibility crisis in psychology: Attack of the
clones or phantom menace? In M. C. Makel & J. A. Plucker (Eds.), Toward a more perfect psychology:
Improving trust, accuracy, and transparency in research (pp. 273–287). American Psychological
Association. https://doi.org/10.1037/0000033-017

Stevens JR (2017) Replicability and Reproducibility in Comparative Psychology. Front. Psychol. 8:862.
doi: 10.3389/fpsyg.2017.00862

Sturm, T. (2006) Is there a Problem with mathematical psychology in the eighteenth century? A fresh
look at Kant’s old argument. Journal of the History of the Behavioral Sciences, Vol. 42(4), 353–377.

Tajika A, Ogawa Y, Takeshima N, Hayasaka Y, Furukawa TA. (2015). Replication and contradiction of
highly cited research papers in psychiatry: 10-year follow-up. Br J Psychiatry. Oct;207(4):357-62.
doi: 10.1192/bjp.bp.113.143701. Epub 2015 Jul 9. PMID: 26159600.

Turner, M. (2019) An Orientation to the Replication Crisis in Psychology. Conference: Southeastern

Psychological Association 65th Annual Meeting: Jacksonville, FL, USA. DOI: 10.17605/OSF.IO/6RGC7

Simmons, J.P., Nelson L.D. , & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed
Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological
Science 22(11) 1359–1366. DOI: 10.1177/0956797611417632

Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R. B., Jr.,... Zwaan,
R. A. (2016). Registered replication report: Strack, Martin, & Stepper (1988). Perspectives on
Psychological Science, 11, 917–928. http://dx.doi.org/10.1177/1745691616674458

Wagenmakers, E.-J., Wetzels, R., Borsboom, D. & van der Maas, H.L.J. (2011a). Why psychologists
must change the way they analyse their data: The case of psi: Comment on Bem (2011). Journal of
Personality and Social Psychology, 100, 426–432. doi: 10.1037/a0022790
Wiggins, B.J. & Chrisopherson, C.D. (2019). The Replication Crisis in Psychology: An Overview for
Theoretical and Philosophical Psychology. Journal of Theoretical and Philosophical Psychology. 2019,
Vol. 39, No. 4, 202–217. http://dx.doi.org/10.1037/teo0000137

Wolff, W., Baumann, L., & Englert, C. (2018). Self-reports from behind the scenes: Questionable
research practices and rates of replication in ego depletion research. PLOS ONE, 13(6),
e0199554. https://doi.org/10.1371/journal.pone.0199554