4 of 9 | JNCI J Natl Cancer Inst, 2016, Vol. 108, No.
Phase 2 trial, by combining the chemo-
therapy with an experimental enzyme,
PEGPH20.
Pancreatic and other cancers, amass
high levels of hyaluronic acid (HA), a
molecule that absorbs water and created
a microenvironment that nurtures the
growth of tumor cells and inhibits the ef-
fectiveness of many anti-cancer agents.
PEGPH20 is a version of the naturally oc-
curring hyaluronidase enzyme that
breaks down HA, and which has been
modified to be more stable in the body.
NEWS
Breaking through the tumor microenvi-
ronment’s HA component lowers the
pressure inside the tumors, opens con-
stricted vessels, thus increasing blood
flow and access to the tumor.
Hingorani said to date, they have
nearly doubled the amount of time cer-
tain patients with metastatic pancreatic
cancer have before their disease pro-
gresses. He cautions that the trial must
“P < 0.05” Might Not Mean What You Think: American Statistical
Association Clarifies P Values
By Beatrice Grabowski
In 2011, the U.S. Supreme Court unani-
mously ruled in Matrixx Initiatives Inc. v.
Siracusano that investors could sue a
drug company for failing to report ad-
verse drug effects—even though they
were not statistically significant.
Describing the case in the April 2, 2011,
issue of the Wall Street Journal, Carl Bialik
wrote, “A group of mathematicians has
been trying for years to have a core statis-
tical concept debunked. Now the Supreme
Court might have done it for them.” That
conclusion may have been overly optimis-
tic, since misguided use of the P value
continued unabated. However, in 2014
concerns about misinterpretation and
misuse of P values led the American Stat-
istical Association (ASA) Board to convene
a panel of statisticians and experts from a
variety of disciplines to draft a policy
statement on the use of P values and hy-
pothesis testing. After a year of discussion,
ASA published a consensus statement in
American Statistician (doi:10.1080/00031305.
2016.1154108).
The statement consists of six princi-
ples in nontechnical language on the
proper interpretation of P values, hy-
pothesis testing, science and policy
decision-making, and the necessity for
full reporting and transparency of
be completed before any definitive con-
clusions can be drawn.
The Fred Hutchinson Cancer
Research team is not the only ones
working on T cell immunology vaccines
for pancreatic cancer. There are other T
cell vaccine approaches – administered
alone or combined with other therapies –
currently under investigation and enroll-
ing patients with pancreatic cancer in
clinical trials. For example, Carl H. June,
M.D., professor of pathology and labora-
tory medicine at the Perelman School of
Medicine and director of translational re-
search at the Abramson Cancer Center at
the University of Pennsylvania in Phila-
delphia, is investigating chimeric anti-
gen receptor T cells which have shown
success with another version of the ther-
apy given to patients with advanced
leukemia.
Jaffee said all told, there are seven
approaches to immunotherapy for
research studies. However, assembling
a short, clear statement by such a di-
verse group took longer and was more
contentious than expected. Participants
wrote supplementary commentaries,
available online with the published
statement.
The panel discussed many miscon-
ceptions about P values. Test your
knowledge: Which of the following is
true?
• P > 0.05 is the probability that the null
hypothesis is true.
• 1 minus the P value is the probability
that the alternative hypothesis is
true.
• A statistically significant test result
(P  0.05) means that the test hypoth-
esis is false or should be rejected.
• A P value greater than 0.05 means
that no effect was observed.
If you answered “none of the above,”
you may understand this slippery con-
cept better than many researchers. The
ASA panel defined the P value as “the
probability under a specified statistical
model that a statistical summary of the
data (for example, the sample mean dif-
ference between two compared groups)
pancreatic cancer that have shown
promise in early clinical trials. These
treatments can be broken into these
main categories: checkpoint inhibitors/
immune modulators, therapeutic vac-
cines, adoptive cell transfer, monoclonal
antibodies, oncolytic viruses, adjuvant
immunotherapies, and cytokines.
As for the Fred Hutchinson Cancer
Research Center approach, Hingorani,
who lost his father to pancreatic cancer,
said the team plans to have the human
version of the T cell therapy in clinical
trials by the end of the year. He is cau-
tiously optimistic. “We have vetted our
strategies as deeply as possible, and of
course, the goal is to see it work in a pa-
tient; that’s the great proving ground.”
© Oxford University Press 2016.
DOI:10.1093/jnci/djw193
would be equal to or more extreme than
its observed value.”
Why is the exact definition so impor-
tant? Many authors use statistical soft-
ware that presumably is based on the
correct definition. “It’s very easy for re-
searchers to get papers published and
survive based on knowledge of what sta-
tistical packages are out there but not
necessarily how to avoid the problems
that statistical packages can create for
you if you don’t understand their appro-
priate use,” said Barnett S. Kramer, M.D.,
M.P.H., JNCI’s former editor in chief and
now director of the National Cancer
Institute’s Division of Cancer Prevention.
(Kramer was not on the ASA panel.)
Part of the problem lies in how people
interpret P values. According to the ASA
statement, “A conclusion does not im-
mediately become ‘true’ on one side of
the divide and ‘false’ on the other.” Val-
uable information may be lost because
researchers may not pursue “insignifi-
cant” results. Conversely, small effects
with “significant” P values may be bio-
logically or clinically unimportant. At
best, such practices may slow scientific
progress and waste resources. At worst,
they may cause grievous harm when
adverse effects go unreported. The

Supreme Court case involved the drug
Zicam, which caused permanent hearing
loss in some users. Another drug, rofe-
coxib (Vioxx), was taken off the market
because of adverse cardiovascular ef-
fects. The drug companies involved did
not report those adverse effects because
of lack of statistical significance in the
original drug tests (Rev. Soc. Econ.
2016;74:83–97; doi:10.1080/
00346764.2016.1150730).
ASA panelists encouraged using alter-
native methods “that emphasize estima-
tion over testing, such as confidence,
credibility, or prediction intervals; Bayesian
methods; alternative measures of evi-
dence, such as likelihood ratios or Bayes
Factors; and other approaches such as
decision-theoretic modeling and false
discovery rates.” However, any method
can be used invalidly. “If success is defined
based on passing some magic threshold,
biases may continue to exert their influ-
ence regardless of whether the threshold
is defined by a P value, Bayes factor, false-
discovery rate, or anything else,” wrote
panelist John Ioannidis, Ph.D., professor of
medicine and of health research and policy
at Stanford University School of Medicine
in Stanford , Calif.
Some panelists argued that the P value
per se is not the problem and that it has
its proper uses. A P value can sometimes
be “more informative than an interval”—
such as when “the predictor of interest is
a multicategorical variable,” said Clarice
Weinberg, Ph.D., who was not on the
panel. “While it is true that P values are
imperfect measures of the extent of evi-
dence against the null hypothesis, confi-
dence intervals have a host of problems
of their own,” said Weinberg, deputy chief
of the Biostatistics and Computational
Biology Branch and principal investigator
of the National Institute of Environmental
Health Sciences in Research Triangle
Park, N.C.
Physical Activity Associated With Fewer Cancers
By Susan Jenks
Even moderate leisure-time physical activ-
ity may protect against 13 cancers, accord-
ing to a massive observational study that
appeared May 16 in JAMA Internal Medicine
(doi:10.1001/jamainternmed.2016.1548).
But which type of exercise brings the
most benefit is not yet clear, researchers
“If success is defined based
on passing some magic
threshold, biases may
continue to exert their
influence regardless of
whether the threshold is
defined by a P value, Bayes
factor, false-discovery rate,
or anything else.”
Beyond simple misinterpretation of
the P value and the associated loss of in-
formation, authors consciously or un-
consciously but routinely engage in data
dredging (aka fishing, P-hacking) and
selective reporting. “Any statistical
technique can be misused and it can be
manipulated especially after you see
the data generated from the study,”
Kramer said. “You can fish through a
sea of data and find one positive finding
and then convince yourself that even
before you started your study that
would have been the key hypothesis
and it has a lot of plausibility to the
investigator.”
In response to those practices and
concerns about replicability in science,
some journals have banned the P value
and inferential statistics. Others, such as
JNCI, require confidence intervals and ef-
fect sizes, which “convey what a P value
does not: the magnitude and relative im-
portance of an effect,” wrote panel mem-
ber Regina Nuzzo, Ph.D., professor of
mathematics and computer sciences at
Gallaudet University in Washington, D.C.
(Nature 2014;506:150–2).
How can practice improve? Panel
members emphasized the need for full
say, nor is exercise alone likely to ac-
count for its association with a lower
cancer risk in colon, breast, and endome-
trial cancers, among others.
“Physical activity is not a stand-alone,
magic bullet,” said William McCarthy, Ph.D.,
adjunct professor in the department of
NEWS | 5 of 9
reporting and transparency by authors
as well as changes in statistics educa-
tion. In his commentary, Don Berry,
Ph.D., professor of biostatistics at the
University of Texas M.D. Anderson Can-
cer Center in Houston, urged researchers
to report every aspect of the study. “The
specifics of data collection and curation
and even your intentions and motivation
are critical for inference. What have you
not told the statistician? Have you de-
leted some data points or experimental
units, possibly because they seemed to
NEWS
be outliers?” he wrote.
Kramer advised researchers to “con-
sult a statistician when writing a grant
application rather than after the study is
finished; limit the number of hypotheses
to be tested to a realistic number that
doesn’t increase the false discovery rate;
be conservative
in interpreting
the data; don’t
consider P ¼
0.05 as a magic
number; and
whenever pos-
sible, provide
confidence in-
tervals.” He also
suggested,
“Webinars and
symposia on
John Ioannidis, Ph.D.
this issue will
be useful to
clinical scien-
tists and bench researchers because
they’re often not trained in these princi-
ples.” As the ASA statement concludes,
“No single index should substitute for
scientific reasoning.”
© Oxford University Press 2016.
DOI:10.1093/jnci/djw194
health services in the Fielding School of Pub-
lic Health at the University of California, Los
Angeles. “The biggest bang [in risk reduction]
comes when exercise is coupled with a
Mediterranean-style diet and not smoking.”
Still, McCarthy said, the recent joint
study by researchers at the National