10/1/2019 I confess, I wrote the Arsenic DNA paper to expose flaws in peer-review at subscription based journals

it is NOT junk
a blog about genomes, DNA, evolution, open science, baseball and other important things

I confess, I wrote the Arsenic DNA paper to

expose flaws in peer-review at subscription
based journals Search
By MICHAEL EISEN | Published: OCTOBER 3, 2013 To search, type and hit enter

In 2011, after having read several really bad papers in the journal Science, I decided to
Michael Eisen
explore just how slipshod their peer-review process is. I knew that their business
I'm a biologist at UC
depends on publishing “sexy” papers. So I created a manuscript that claimed Berkeley and an Investigator
something extraordinary – that I’d discovered a species of bacteria that uses arsenic in of the Howard Hughes
its DNA instead of phosphorus. But I made the science so egregiously bad that no Medical Institute. I work
competent peer reviewer would accept it. The approach was deeply flawed – there were primarily on flies, and my
research encompases evolution,
poor or absent controls in every figure. I used ludicrously elaborate experiments where
development, genetics, genomics, chemical
simple ones would have done. And I failed to include a simple, obvious experiment that ecology and behavior. I am a strong
would have definitively shown that arsenic was really in the bacteria’s DNA. I then proponent of open science, and a co-founder
submitted the paper to Science, punching up the impact the work would have on our of the Public Library of Science. And most
importantly, I am a Red Sox fan. (More
understanding of extraterrestrials and the origins of life on Earth in the cover
about me here).
letter. And what do you know? They accepted it!
I can be reached at:
My sting exposed the seedy underside of “subscription-based” scholarly publishing,
mbeisen at berkeley.edu
where some journals routinely lower their standards – in this case by sending the paper and @mbeisen on Twitter
to reviewers they knew would be sympathetic – in order to pump up their impact factor
and increase subscription revenue. Maybe there are journals out there who do Recent Posts
subscription-based publishing right – but my experience should serve as a warning to The abysmal response of the Salk
people thinking about submitting their work to Science and other journals like it. Institute to accounts of gender
discrimination in its midst
OK – this isn’t exactly what happened. I didn’t actually write the paper. Far more Patents are destroying the soul of
frighteningly, it was a real paper that contained all of the flaws described above that academic science
was actually accepted, and ultimately published, by Science. Replace Francis Collins as NIH Director
Exploring the relationship between
I am dredging the arsenic DNA story up again, because today’s Science contains a story gender and author order and composition
by reporter John Bohannon describing a “sting” he conducted into the peer review in NIH-funded research
practices of open access journals. He created a deeply flawed paper about molecules What population genetics has to say
from lichens that inhibit the growth of cancer cells, submitted it to 304 open access about Olympic success of West African
sprinters
journals under assumed names, and recorded what happened. Of the 255 journals that
rendered decisions, 157 accepted the paper, most with no discernible sign of having
Categories
actually carried out peer review. (PLOS ONE, rejected the paper, and was one of the
few to flag its ethical flaws). #occupy
AAP
The story is an interesting exploration of the ways peer review is, and isn’t, academic freedom
implemented in today’s biomedical publishing industry. Sadly, but predictably, baseball
Science spins this as a problem with open access. Here is their press release: Berkeley
bioethics
Spoof Paper Reveals the “Wild West” of Open-Access
cool science
Publishing CRISPR
Darwin
A package of news stories related to this special issue of Science includes
EisenLab
a detailed description of a sting operation — orchestrated by
EisenLab preprints
contributing news correspondent John Bohannon — that exposes the
www.michaeleisen.org/blog/?p=1439 1/4
10/1/2019 I confess, I wrote the Arsenic DNA paper to expose flaws in peer-review at subscription based journals

dark side of open-access publishing. Bohannon explains how he created ENCODE

a spoof scientific report, authored by made-up researchers from epigenetics
institutions that don’t actually exist, and submitted it to 304 peer- evolution
gender discrimination
reviewed, open-access journals around the world. His hoax paper
gene regulation
claimed that a particular molecule slowed the growth of cancer cells,
genetics
and it was riddled with obvious errors and contradictions.
genome size
Unfortunately, despite the paper’s flaws, more open-access journals GMO
accepted it for publication (157) than rejected it (98). In fact, only 36 of HeLa
the journals solicited responded with substantive comments that HHMI
recognized the report’s scientificproblems. (And, according to intellectual property
Bohannon, 16 of those journals eventually accepted the spoof paper math
despite their negative reviews.) The article reveals a “Wild West” microbial manipulation of animal
behavior
landscape that’s emerging in academic publishing, where journals and
misc stuff
their editorial staffs aren’t necessarily who or what they claim to be.
My lab
With his sting operation, Bohannon exposes some of the unscrupulous
NIH
journals that are clearly not based in the countries they claim, though he NOT junk
also identifies some journals that seem to be doing open-access right. open access
PLoS
Although it comes as no surprise to anyone who is bombarded every day by
politics
solicitations from new “American” journals of such-and-such seeking papers and
public access
offering editorial positions to anyone with an email account, the formal exposure of
publishing
hucksters out there looking to make a quick buck off of scientists’ desires to get their race
work published is valuable. It is unacceptable that there are publishers – several owned science
by big players in the subscription publishing world – who claim that they are carrying science and culture
out peer review, and charging for it, but no doing it. science and politics
Uncategorized
But it’s nuts to construe this as a problem unique to open access publishing, if for no
University of California
other reason than the study, didn’t do the control of submitting the same paper to war on science
subscription-based publishers (UPDATE: The author, Bohannon emailed to say that, Wikipedia
while his original intention was to look at all journals, practical constraints limited him women in science
to OA journals, and that Science played no role in this decision). We obviously don’t
know what subscription journals would have done with this paper, but there is every open access
reason to believe that a large number of them would also have accepted the paper (it PLoS
has many features in common with the arsenic DNA paper afterall). Like OA journals, a science and politics
lot of subscription-based journals have businesses based on accepting lots of papers politics
EisenLab
with little regard to their importance or even validity. When Elsevier and other big
GMO
commercial publishers pitch their “big deal”, the main thing they push is the number of
papers they have in their collection. And one look at many of their journals shows that
they also will accept almost anything.

None of this will stop anti-open access campaigners (hello Scholarly Kitchen) from RSS Links
spinning this as a repudiation for enabling fraud. But the real story is that a fair All posts
number of journals who actually carried out peer review still accepted the paper, and All comments
the lesson people should take home from this story not that open access is bad, but that
peer review is a joke. If a nakedly bogus paper is able to get through journals that Meta
actually peer reviewed it, think about how many legitimate, but deeply flawed, papers Log in
must also get through. Any scientist can quickly point to dozens of papers – including,
and perhaps especially, in high impact journals – that are deeply, deeply flawed – the
arsenic DNA story is one of many recent examples. As you probably know there has
been a lot of smoke lately about the “reproducibility” problem in biomedical science, in
which people have found that a majority of published papers report facts that turn out
not to be true. This all adds up to showing that peer review simply doesn’t work.

And the real problem isn’t that some fly-by-night publishers hoping to make a quick
buck aren’t even doing peer review (although that is a problem). While some fringe OA
publishers are playing a short con, subscription publishers are seasoned grifters
playing a long con. They fleece the research community of billions of dollars every year
by convincing them of something manifestly false – that their journals and their “peer
review” process are an essential part of science, and that we need them to filter out the

www.michaeleisen.org/blog/?p=1439 2/4
10/1/2019 I confess, I wrote the Arsenic DNA paper to expose flaws in peer-review at subscription based journals

good science – and the good scientists – from the bad. Like all good grifters playing the
long con, they get us to believe they are doing something good for us – something we
need. While they pocket our billions, with elegant sleight of hand, then get us to ignore
the fact that crappy papers routinely get into high-profile journals simply because they
deal with sexy topics.

But unlike the fly by night OA publishers who steal a little bit of money, the
subscription publishers’ long con has far more serious consequences. Not only do they
traffic in billions rather than thousands of dollars and denying the vast majority of
people on Earth access to the findings of publicly funded research, the impact and
glamour they sell us to make us willing participants in their grift has serious
consequences. Every time they publish because it is sexy, and not because it is right,
science is distorted. It distorts research. It distorts funding. And it often distorts public
policy.

To suggest – as Science (though not Bohannon) are trying to do – that the problem
with scientific publishing is that open access enables internet scamming is like saying
that the problem with the international finance system is that it enables Nigerian wire
transfer scams.

There are deep problems with science publishing. But the way to fix this is not to
curtain open access publishing. It is to fix peer review.

First, and foremost, we need to get past the antiquated idea that the singular act of
publication – or publication in a particular journal – should signal for all eternity that a
paper is valid, let alone important. Even when people take peer review seriously, it is
still just represents the views of 2 or 3 people at a fixed point in time. To invest the
judgment of these people with so much meaning is nuts. And its far worse when the
process is distorted – as it so often is – by the desire to publish sexy papers, or to
publish more papers, or because the wrong reviewers were selected, or because they
were just too busy to do a good job. If we had, instead, a system where the review
process was transparent and persisted for the useful life of a work (as I’ve written about
previously), none of the flaws exposed in Bohannon’s piece would matter.

88 Like 7.5K 1 19

36

This entry was posted in open access, science. Bookmark the permalink. Both comments and
trackbacks are currently closed.

« NASA paywalls first papers arising from GMOs and pediatric cancer rates #GMOFAQ
Curiosity rover, I am setting them free »

One Comment

Joanne Gaudet
Posted June 26, 2014 at 6:27 am | Permalink

It is interesting to see the motivation behind the Arsenic DNA paper. Thank you for
sharing.

My doctoral research, in sociology, is on journal peer review. Thought you might be

interested in a sociological perspective on the dynamics you (and responses to your
blog) outline:

Investigating journal peer review as scientific object of study (Gaudet, 2014)

http://www.ruor.uottawa.ca/en/handle/10393/31161

How pre-publication journal peer review (re)produces ignorance at scientific and

medical journals: a case study (Gaudet, 2014)
http://www.ruor.uottawa.ca/en/handle/10393/31198

www.michaeleisen.org/blog/?p=1439 3/4
10/1/2019 I confess, I wrote the Arsenic DNA paper to expose flaws in peer-review at subscription based journals

Unless otherwise noted, all content on this site is licensed under a Creative Commons Attribution 3.0 Unported License.

www.michaeleisen.org/blog/?p=1439 4/4