Curt Stump, PharmD

Turning Statistical Significance into Clinical Significance

Introduction & Definitions

 Statistical significance: probability that the difference between
groups is attributable to a specific cause rather than by chance
alone
o Relates to null hypothesis and alternative hypothesis
 Clinical significance: speaks to the practical significance or
importance of findings
o “Will these findings result in a change in practice?”

A Few Examples
 Investigators conduct a study to determine the effectiveness of a
new medication in reducing the risk of MI compared to standard therapy. It was found that the
reduction of the 5-year risk of MI is 0.2% compared to standard therapy (p=0.01).
o Statistically significant?
o Clinically significant?
 In a sample of 10,000 patients, investigators found that a calcium channel blocker reduced
blood glucose in type 1 DM patients by 2 mg/dL compared to placebo (p=0.01).
o Statistically significant?
o Clinically significant?
 Investigators created a new treatment regimen for prostate
cancer in which agents are given less frequently. When
compared to the standard regimen, no statistically significant
differences were found in survival or cancer progression;
however, the new regimen demonstrated a significant
reduction in adverse events.
o Clinically significant?

Determining Statistical Significance – REWIND

Decide
Reject or fail to
Formulate statistical Apply statistical
Collect data reject null
hypotheses significance significance test
hypothesis
cutoff (alpha)
Curt Stump, PharmD
Turning Statistical Significance into Clinical Significance

 Null hypothesis: there is not a statistically significant difference in cardiovascular outcomes

between dulaglutide and placebo when used to treat type 2 diabetes mellitus
 Alternative hypothesis: There is a statistically significant difference in cardiovascular outcomes
between dulaglutide and placebo when used to treat type 2 diabetes mellitus
 Interpret results: The primary outcome (first occurrence of the composite endpoint of non-fatal
MI, non-fatal stroke, or death from CV causes) occurred in 594 (12%) participants at an
incidence rate of 2.4 per 100 person-years in the dulaglutide group and in 663 (13.4%)
participants at an incidence rate of 2.7 per 100 person-years in the placebo group (HR 0.88, 95%
CI of 0.79-0.99, p=0.026).
 Apply statistical significance test  reject the null hypothesis

Determining Clinical Significance

1 NNT NNH Effect size

NNH ∨NNT =
Absolute Risk Reduction
Absolute Risk
Impact on
vs Relative
guidelines?
Risk
 What is the NNT for the primary outcome in
the REWIND study? How would you phrase this?

o The number needed to treat (NNT) for the primary endpoint was 72, which indicates
that we would need to treat 72 people with dulaglutide instead of placebo in order to
prevent one additional case of the primary composite endpoint over a median of 5.4
years.
 What impact did this have on clinical practice guidelines?

Important Takeaways
Curt Stump, PharmD
Turning Statistical Significance into Clinical Significance

 Statistical significance does NOT equal clinical significance

 Sometimes the lack of statistical significance can be clinically relevant
 Be cautious when interpreting relative risk
 If you do not have NNH/NNT equation memorized, DO IT BEFORE YOUR NAPLEX!