3-Non-Inferiority Studies

Non-inferiority Studies
Lecture Series – Basic Statistics in Clinical Trials

IIS China
Outline
 Introduction
 Statistical Methods
 Two Real Examples
 Issues and Challenges
 Summary and Conclusion
2 | Basic Statistics in Clinical Trials | IIS China | Non-inferiority Studies | All Rights Reserved
Learning Objectives
 Learn what a non-inferiority (NI,非劣效性) study is and

when to conduct such a study
 Issues and challenges of NI studies

 Learn how to use the fixed margin approach
 Be able to analyze a NI study and interpret the results
 Introduction
• Motivating Example
• What are NI Studies
• Main Objectives of NI Studies
• Regulatory Guidance
Motivating Example
 Warfarin (华法令阻凝剂) is an effective

anticoagulant (抗凝的) that is
approved in the United States for the
treatment of patients with non-valvular
atrial fibrillation (非瓣膜性心房颤动)
 The novel anticoagulant ximelagatran

(希美加群) has been developed as a
replacement for warfarin and
overcome its known safety issues
Motivating Example
 SPORTIF V is a non-inferiority study to compare
ximelagatran (the new drug, “Test”) with warfarin (drug
already in use, “Active Control (阳性对照)”)
 Study objectives: Show that ximelagatran is

1. better than placebo
2. “not inferior” to warfarin
 Here, “non inferior” means we allow ximelagatran to have

a slightly worse treatment effect than warfarin
 It would be unethical to include a placebo group in this

study since this is a serious disease, with an effective
treatment already available
Motivating Example
 Questions:
• What does “slightly worse” mean? How to measure NI?
• What is the statistical problem in NI studies (hypotheses, test
statistics, etc.)?
• Since there is no placebo group in this study, how do we estimate
the effect of ximelagatran compared to placebo in this NI study?
• Can we use the information from old studies of warfarin to assess
the treatment effect of warfarin against placebo?
• If we use old study data, how do we ensure that the effect of the
control treatment warfarin is consistent (一致的) across studies and
therefore reproducible (可复制的)?
General Concepts
Standard Study Design Options
 Uncontrolled Test
 1-arm (组) study with “Test” (= new drug) only
 Placebo controlled Test

Placebo
 2-arm study comparing “Test” against “Placebo”
Test
 Active controlled Control
 2-arm study comparing “Test” against (active) “Control” (= old drug)
 3-arm study with placebo Test

Control
• “Test” compared with “Placebo” and “Control”
Placebo
NI problems typically appear in studies with an active control,
such as options 3 and 4 above
General Concepts
Notation
 𝑇 = Test treatment, investigational treatment

 𝐶 = (active) Control treatment
 𝑃 = Placebo
 𝜇 𝑇 = Mean effect of the test treatment
 𝜇𝐶 = Mean effect of the active control treatment
 𝜇𝑃 = Mean effect of the placebo
General Concepts
Superiority (优效) studies
 Objective: to show that Test is more effective (= better) than Control
 Hypotheses: 𝐻0 : 𝜇𝐶 − 𝜇 𝑇 ≥ 0 (i.e. 𝜇𝑇 ≤ 𝜇𝐶 )
𝐻𝑎 : 𝜇𝐶 − 𝜇 𝑇 < 0 (i.e. 𝜇𝑇 > 𝜇𝐶 )
 A treatment effect can be established statistically by showing that the

upper bound of the 95% confidence interval (CI) for 𝜇𝐶 − 𝜇 𝑇 is < 0.
 Three possible results of a superiority study (point estimate and 95% CI)
1. Superiority is demonstrated
2. Superiority is not demonstrated
3. Superiority is not demonstrated
0 𝜇𝐶 – 𝜇 𝑇
What are NI Studies
NI Studies
 Objective: to show that the new treatment is not inferior to an

unacceptable extent, as measured by a NI margin (临界值) 𝛿 > 0
 Hypotheses: 𝐻0 : 𝜇𝐶 − 𝜇 𝑇 ≥ 𝛿
𝐻𝑎 : 𝜇𝐶 − 𝜇 𝑇 < 𝛿
Superior
Non-inferior
Inferior
0 𝛿 𝜇𝐶 − 𝜇 𝑇
 In NI studies, we allow T to be slightly worse than C
 Key questions:
• What does “slightly” mean? (as measured by NI margin 𝛿)
• How to determine the NI margin? (will be addressed later)
Main Objectives of NI Studies
Any NI study has two distinct objectives.
 Superiority objective to evaluate efficacy:

Show that T is better than P
• Hypotheses: 𝐻0 : 𝜇𝑃 − 𝜇 𝑇 ≥ 0 (i.e. 𝜇 𝑇 ≤ 𝜇𝑃 )
𝐻𝑎 : 𝜇𝑃 − 𝜇 𝑇 < 0 (i.e. 𝜇 𝑇 > 𝜇𝑃 )
 NI objective to evaluate relative efficacy:

Show that T is not much worse than C
• Hypotheses: 𝐻0 : 𝜇𝐶 − 𝜇 𝑇 ≥ 𝛿
𝐻𝑎 : 𝜇𝐶 − 𝜇 𝑇 < 𝛿
Main Objectives of NI Studies
 Option 3: Active controlled design Test
• The NI objective can be directly addressed. Control
• But how to evaluate efficacy without P? (main topic of this lecture)
 Option 4: 3-arm NI trial with placebo design Test

• Both objectives can be directly addressed: Control
- Efficacy: T vs P Placebo
- NI: T vs C
 Choice between options 3 and 4

• Option 4 should be preferred whenever possible, as both main
objectives can be addressed directly
• Sometimes it is not ethical to include Placebo
- For example, if an effective treatment is available for a life-threatening
disease, a 2-arm study design with T and C is the only option
Regulatory Guidance
 Key regulatory documents
• ICH (International Conference on Harmonisation, 国际协调会
议) E10 (Choice of Control Group, 2000)
“In cases where an available treatment is known to prevent serious harm,
such as death or irreversible morbidity (病态) in the study population, it is
generally inappropriate to use a placebo control”
• Draft FDA (Food and Drug Administration, 食品及药物管理局)
guidance (Non-Inferiority Clinical Trials, 2010)
“It would not be ethical to use a placebo, or a no treatment control, or a very
low dose of an active drug, if there is an effective treatment that provides an
important benefit (e.g., life-saving or preventing irreversible injury)”
Increased use of NI studies
 NI studies are becoming more and more popular
Source: Web of Science

TS=(noninferior* OR “non-inferior” OR "non-inferiority")
Timespan=2005-2015
Databases=SCI-EXPANDED, SSCI, A&HCI.
Summary
 NI studies test whether a test treatment is better or at
most slightly worse than the active control
 NI problems typically appear in studies with an active

control
 The two objectives of NI studies are to show:

• T is better than P
• T is not much worse than C
 The degree of inferiority is measured by NI margin
 Introduction
• Confidence Intervals
• Determination of the NI Margin
• Fixed Margin Approach
• Power and Sample Size Considerations

Confidence Intervals
 Confidence intervals are often used to analyze NI studies
 A treatment effect (in superiority) is established by showing
that the upper bound of the 95% CI for 𝜇𝐶 − 𝜇 𝑇 is < 0
 Example:
• Point estimate of mean effect 𝜇𝐶 − 𝜇 𝑇 : −0.4
• 95% confidence interval for 𝜇𝐶 − 𝜇 𝑇 : −0.7; −0.1
-0.5 0 0.5 𝜇𝐶 – 𝜇 𝑇
• Interpretation:
- The probability that the CI covers the true value 𝜇𝐶 − 𝜇 𝑇 is 95%
- The upper bound of the 95% CI for 𝜇𝐶 − 𝜇 𝑇 is < 0
- T is superior to C at a significance level of 5% for two-sided test（equivalent
to a 2.5% significance level for one-sided test)
Confidence Intervals
 Confidence interval approach to compare T and C
1. Superiority
2. Non-inferiority
Non-inferiority
3. Inferiority
0 𝛿 𝜇𝐶 – 𝜇 𝑇
 Interpretation:
1. The upper bound of the CI is < 0, superiority is established
2. The upper bound of the CI is < 𝛿, NI is established
3. The lower bound of the CI is > 𝛿, T is inferior to C
A Simple Example
 Data example from a NI study with
• Point estimate of mean effect 𝜇𝐶 − 𝜇 𝑇 : 0.03
• 95% confidence interval for 𝜇𝐶 − 𝜇 𝑇 : −0.02; 0.08
• Prespecified NI margin 𝛿 = 0.1
-0.1 0 0.1 𝜇𝐶 – 𝜇 𝑇
𝛿
 Interpretation:
• Upper bound of the 95% CI is less than the NI margin of 𝛿 = 0.1
• T is non-inferior to C (at a significance level of 5%) for 𝛿 = 0.1
• Superiority cannot be shown, since the upper bound of the CI is > 0
Determination of the NI Margin
 𝜇 𝑇 > 𝜇𝑃 Objective 1:
T is better than P
⇔ 𝜇𝐶 −𝜇 𝑇 < 𝜇𝐶 − 𝜇𝑃
Objective 2:
⇔ 𝜇𝐶 − 𝜇 𝑇 < 𝛿 ≤ 𝜇𝐶 − 𝜇𝑃 T is not much worse than C
current NI study necessary condition
⇔ 𝛿 ≤ 𝜇𝐶 − 𝜇𝑃
 NI margin 𝛿 must be smaller than or equal to the control

effect 𝜇𝐶 − 𝜇𝑃
 The NI margin is based on the control effect in order to

evaluate efficacy
Determination of the NI Margin: 𝑀1 & 𝑀2
 In the following, we show how to determine the NI margin 𝛿
based on the
• effect of the active control versus placebo (𝑀1 margin)
• largest clinically acceptable difference between the test drug and the
active control (𝑀2 margin)
 This approach follows the draft FDA guidance on non-

inferiority studies
NI Margin: 𝑀1
 NI hypothesis:
𝐻0 : 𝜇𝐶 − 𝜇 𝑇 ≥ 𝑀1 (T is inferior to C by 𝑀1 or more)
𝐻𝑎 : 𝜇𝐶 − 𝜇 𝑇 < 𝑀1 (T is inferior by less than 𝑀1 or even better than C)
 𝑀1 represents the effect of C relative to P in the NI study, and

establishing NI means that T has an effect > 0 (compared to P)
• If there is no placebo group in the NI study, 𝑀1 must be determined
based on past performance of the active control
• In practice, 𝑀1 is often set as the lower bound of the CI for 𝜇𝐶 − 𝜇𝑃 ,
using historical information
 FDA Terminology:
• 𝑀1 = the entire effect of the active control assumed to be present in the
NI study (from the historical data)
 NI is established if the upper CI bound for 𝜇𝐶 − 𝜇 𝑇 is less than 𝑀1

NI Margin: 𝑀1
Possible results of a NI study with 𝑀1 = 2 and using a 95% CI
1. NI is demonstrated
2. NI is not demonstrated
3. NI is not demonstrated
4. NI is demonstrated, but superiority is not
5. Superiority and NI are demonstrated
6. C is significantly better than T, but NI is

also demonstrated for the NI margin 𝑀1
𝜇𝐶 − 𝜇 𝑇
NI Margin: 𝑀2
 Demonstrating a difference < 𝑀1 shows that the test drug
is effective
 However, in many cases that would not be sufficient

assurance that the test drug had a clinically meaningful
effect
 Both parts need to be considered:

• Statistical aspects
• Clinical considerations
 Therefore in NI studies we additionally choose a smaller

margin 𝑀2 that reflects the largest loss of effect that would
be clinically acceptable
NI Margin: 𝑀2
 FDA Terminology (术语):
• 𝑀2 = the largest clinically acceptable difference (degree of inferiority)
of the test drug compared to the active control (from the NI study)
 Reason to use 𝑀2
• To ensure that the test drug has a clinically meaningful effect
 Demonstrate a difference < 𝑀2 shows that the new drug

preserves the desired fraction of the control effect
 It is critical to note that 𝑀2 is a judgment that is made after

𝑀1 is chosen, but 𝑀2 can never be larger than 𝑀1 , in most
cases 𝑀2 is a fraction of 𝑀1
Final NI Margin: 𝑀1 & 𝑀2
 The selection of the final margin for a NI study is a two-step
process
• Step 1: making a reasonable assumption about the effect of C in the
NI study. 𝑀1 is chosen to equal that treatment effect.
• Step 2: choosing a specified fraction of 𝑀1 whose loss by T must be
ruled out (排除). 𝑀2 is chosen to equal that fraction of 𝑀1
- E.g. if it is necessary for a test drug to preserve 75% of the effect, then 𝑀2
would be 25% of 𝑀1 which is the loss of effect that must be ruled out
• In practice the final NI margin 𝛿 (≤ 𝑀2 ) is decided taking both
statistical and clinical considerations
 The NI study design involves two comparisons: A direct

comparison of T with C; An indirect comparison of T with P
• In practice, compare the CI upper bound of point estimate with the
final margin to establish NI
NI Margin: 𝑀1 & 𝑀2
Possible results of NI study showing 𝜇𝐶 – 𝜇 𝑇 and 95% CI
1. NI demonstrated
2. Effect > 0, but unacceptable loss

of the control effect.
3. Decision could be made that
treatment is effective
4. No evidence of effectiveness for
test drug
𝜇𝐶 − 𝜇 𝑇
NI Margin: Results Interpretation
 “Non-inferiority was demonstrated …” alone means nothing
 Any conclusion that “non-inferiority was demonstrated” is not
complete without a full description of the trial objectives
 Be careful with: “T is non-inferior to C.”

Add the margin in each context and the conclusion of the test
• For example, an effect of 𝑀1 = 2 was ruled out and T is effective; that
is, T demonstrated superiority over placebo (indirectly)
Fixed Margin Approach
𝜇𝐶 – 𝜇 𝑇 with 95% CI 𝜇𝐶 – 𝜇𝑃 with 95% CI
0 𝑀2 𝑀1 𝜇𝐶 – 𝜇 𝑇
 Two confidence intervals are involved in the fixed margin approach

• One comes from historical information, using the lower bound of the CI
for 𝜇𝐶 − 𝜇𝑃 as 𝑀1
• One comes from the NI study itself to demonstrate 𝜇𝐶 − 𝜇𝑇 < 𝑀2
 When both intervals are 95% confidence intervals, the fixed margin
approach is also called the 95%-95% approach
Fixed Margin Approach vs. Synthesis Approach
 Fixed margin approach
• Recommended by FDA but quite conservative (保守的)
• Provides a statistically interpretable margin 𝑀1 , and the derived
margin 𝑀2 as a fraction of 𝑀1
• Use of the fixed margin is easily understood, particularly by non-
statisticians
 Synthesis (合成) approach

• It synthesizes the data from the historical trials and the current NI trial
and yields one confidence interval for testing the NI hypothesis
• It does not allow for a pre-specification of the NI margin 𝑀1 , so the
clinical judgment to determine 𝑀2 is difficult and is generally not made
until results are seen
• Not recommended by FDA, therefore not very often used in practice
Testing NI and Superiority in an NI Study
 When there is only one endpoint and one dose of the test
treatment, a planned NI study can be used to also test for
superiority without multiplicity adjustment
• The same 95% confidence interval employed for testing NI can be
used to test superiority
 Successfully showing NI allows one to go ahead to test

superiority
• This is actually a fixed sequence procedure, which controls the Type I
error rate for both NI and superiority tests at a significance level of 5%
See lecture “Introduction to Multiple Testing”
Power and Sample Size Considerations
 It is important to plan the sample size for a NI study to
achieve a reasonable statistical power to conclude that the
NI margin is ruled out if T is truly non-inferior to C
 With the fixed margin approach, the NI margin is specified

in the study protocol and the sample size must be based on
the need to rule out inferiority greater than 𝑀2
 If T is somewhat more effective than C, it will be easier to

rule out any given NI margin and a smaller sample size is
sufficient
 A less effective T will require a larger sample size
 The smaller the NI margin, the smaller the upper CI bound for
𝜇𝐶 − 𝜇 𝑇 , the lower the power, and the larger the sample size
Upper confidence limit for difference in proportions (simulation)
20% response rate for both T and C
Power ( % )
150 pat ient s/group

Non-inferiority margin
 The smaller the NI margin, the smaller the upper CI bound for
𝜇𝐶 − 𝜇 𝑇 , the lower the power, and the larger the sample size
20% response rate for both T and C
Power ( % )

If NI margin is changed from 200 pat ient s/group
9% to 7.5%, power is reduced
from 49% to 36%, respectively
Non-inferiority margin
 Sensitivity (敏感性) of power on the assumed response
rates
20% response rate for C, NI margin of 10%
100
90
150 patients /gr oup
80 200 patients /gr oup
70
Power ( % )
60
50 If the expected response rate
150 patients /gr oup for T
40 is 21.6% instead of 20%, power
200 patients /gr oup is
30 increased from 59% to 71%
20
10
0.15 0.17 0.19 0.21 0.23 0.25
Expected response rate for T
Summary
 Confidence intervals are often used to analyze NI studies
 NI margin must be smaller than or equal to the control
effect
 𝑀1 represents the effect of C relative to P in the NI study,

and establishing NI means that T has an effect > 0
(compared to P)
 In NI studies we additionally choose a smaller margin 𝑀2

that reflects the largest loss of effect that would be
clinically acceptable
 Fixed margin approach is the main statistical method for

NI analysis, sometimes also called 95% - 95% approach
 Introduction
• SPORTIF V Study
• Interferon Study

SPORTIF V Study
Re-visited
 Objective: establish non-inferiority test for ximelagatran against the
active control warfarin
 Primary endpoint: Relative risk (相对风险) 𝜃𝑇 ,

• That is, the ratio of the probability of the event occurring in the treatment
group versus control group
 Hypotheses: 𝐻0 : 𝜃𝑇 ≥ 𝑀 versus 𝐻𝑎 : 𝜃𝑇 < 𝑀

• Here, 𝑀 > 1 denotes the NI margin that needs to be determined
 There are historical placebo-controlled studies of warfarin in patients

with non-valvular atrial fibrillation,
• All six studies show a consistent effect of warfarin compared to placebo
• Thus we can assume that were placebo to be included in the warfarin-
controlled NI study, warfarin would have been superior to placebo
SPORTIF V Study
Historical Data
 Summary of six historical placebo-controlled studies with warfarin

Study Probability of Event Relative Risk (95% CI)
Warfarin Placebo
1 9/413 = 2.18% 21/398 = 5.28% 0.41 (0.19, 0.89)
2 3/487 = 0.62% 13/435 = 2.99% 0.21 (0.06, 0.72)
3 21/507 = 4.14% 54/405 = 13.3% 0.31 (0.19, 0.51)
4 7/237 = 2.95% 11/241 = 4.56% 0.65 (0.26, 1.64)
5 8/260 = 3.08% 20/244 = 8.20% 0.38 (0.17, 0.84)
6 9/489 = 1.84% 24/483 = 4.95% 0.37 (0.17, 0.79)
 The information from the six studies was combined using a meta
analysis (统合分析) to give a point estimate of 0.361 for the relative risk
with a confidence interval of (0.248, 0.527)
See lecture “Meta Analysis” (next semester)
SPORTIF V Study
NI Margin
 The 95% CI upper bound of 0.527 represents a 47% risk

reduction, which translates into a risk increase of about
90% = 1 0.527 − 1 100% from not being on warfarin
 𝑀1 = 1/0.527 = 1.898
 𝑀2 (clinical judgment) was set at 50% of 𝑀1 using the log
hazard risk ratios (对数风险比例)
• log 𝑀2 = log 𝑀1 /2 → 𝑀2 = 1.378
SPORTIF V Study
Results
 In SPORTIF V, the point estimate of the relative risk was

1.39 and the 95% CI was (0.91, 2.12)
 The non-inferiority of ximelegatran to warfarin is not

demonstrated because the upper limit 2.12 > 1.378 = 𝑀2
 Indeed, it does not even demonstrate that 𝑀1 = 1.898 has

been excluded.
 This example illustrates the fixed margin approach using

95% - 95% CIs
Interferon Study
Introduction
 Objective: establish non-inferiority test for new drug against

one of the existing Interferon (干扰素) drugs for multiple
sclerosis (多发性硬化)
 Primary endpoint: annualized relapse rate (ARR, 年均的复

发率)
 Some notations
• Risk ratio: 𝑅𝑅 𝑇/𝐶 = ARR T /ARR C ; 𝑅𝑅 𝑃/𝐶 = ARR P /ARR C
• Treatment effect: 𝜃𝑇 = 𝑅𝑅 𝑇/𝐶 − 1
• Control effect: 𝜃𝐶 = 𝑅𝑅 𝑃/𝐶 − 1
• Treatment effect can be interpreted as the relapse rate increase of
treatment over control; while control effect is the relapse rate increase
of placebo over control
Interferon Study
Hypothesis and Margin
 Fixed margin hypothesis:

𝐻0 : 𝜃𝑇 = 𝑅𝑅 𝑇/𝐶 − 1 ≥ 𝑀
𝐻𝑎 : 𝜃𝑇 = 𝑅𝑅 𝑇/𝐶 − 1 < 𝑀
• M is the pre-specified NI margin
 Determination of the NI margin

• 𝑀1 = Lower bound of the 95% CI of the control effect (recommended
by FDA)
• 𝑀2 (clinical judgment)
 50% of lower bound of 95% CI of the control effect
 50% of point estimate (PE) of the control effect
Interferon Study
Historical Data
 Use pooled data from previous Interferon trials as historical data

Study ARR Percentage Difference (95% CI)
Control Placebo Percent increase of Percent reduction of
Placebo from Control Control from Placebo
1 0.61 0.9 47% (RR(P/C) = 147%) 32%
CI: (14%, 88%)
2 0.865 1.28 48% (RR(P/C) = 148%) 32%
CI: (28%, 70%)
3 0.84 1.27 51% (RR(P/C) = 151%) 34%
CI: (26%, 82%)
Pool 49% (RR(P/C) = 149%) 33%
CI: (34%, 64%)
 The control effects in each of the 3 studies were pooled to give a

point estimate of 49% with a confidence interval of (34%, 64%)
Interferon Study
NI Margin
 𝑀1 = 34% (lower bound of 95% CI (LCI) of the control effect)

 𝑀2 = 17% (half of the LCI) or 𝑀2 = 24.5% (half of PE)
𝜃𝐶 – 𝜃𝑇 with 95% CI 𝜃𝐶 – 𝜃𝑃 with 95% CI
0 0.5LCI 0.5PE LCI PE

17% 24.5% 34% 49%
𝑀2 𝑀1
 In the actual study a single NI margin will have to be stated in the

study protocol before the study begins
Summary
 Fixed margin approach was used to analyze the two real
examples
 If there is no placebo group in the NI study, 𝑀1 must be

determined based on past performance of the active control
 The active control effect can be derived from the historical trials
by meta analysis
 We must be careful whether the historical trials show a

consistent and reproducible effect of the active control compared
to placebo
 𝑀2 is a specified fraction of 𝑀1 whose loss by T must be ruled

out (in the two examples the fractions are 50%)
 Introduction
• Biocreep
• Assay Sensitivity
Biocreep (生物蠕变)
 Biocreep basically refers to the phenomenon where a
slightly inferior treatment becomes the active control for
the next generation of NI trials which over time leads to
degradation of the efficacy of the investigational
treatment
Biocreep
Study Estimates Conclusion
better worse
1 T1 P superiority
2 T1 T2 non-inferiority
 Need to choose the right control

Assay Sensitivity
 Assay sensitivity (AS, 检测灵敏度) is the ability of a trial to detect a
treatment difference of a specified size, 𝑀1 , if such a difference
were present
• That is, if the NI study had included Placebo, a difference of at least 𝑀1
between C and P would have been demonstrated
• AS is essential for NI trials to be conclusive, and a 3-arm NI trial with
placebo (option 4) is considered as a “gold standard” to establish AS
• AS is related to 𝑀1 , our best estimate of the effect of C in the study,
even if the NI margin to be used could be smaller (if 𝑀2 < 𝑀1 )
Assay Sensitivity
 Assay sensitivity (AS, 检测灵敏度) is the ability of a trial to detect a
treatment difference of a specified size, 𝑀1 , if such a difference
were present
• That is, if the NI study had included Placebo, a difference of at least 𝑀1
between C and P would have been demonstrated
• AS is essential for NI trials to be conclusive, and a 3-arm NI trial with
placebo (option 4) is considered as a “gold standard” to establish AS
• AS is related to 𝑀1 , our best estimate of the effect of C in the study,
even if the NI margin to be used could be smaller (if 𝑀2 < 𝑀1 )
 Whether a trial will have AS is based on three considerations

• Historical evidence of sensitivity to drug effects (HESDE)
• Similarity of the new NI trial to the historical trials (constancy
assumption, 恒定假设)
• The quality of the new trial
Assay Sensitivity
HESDE
 HESDE means that historical trials using the same Control

as in the current NI trial regularly showed Control to be
superior to Placebo
• This gives reliable treatment effect estimates in the historical studies,
which can then be used to determine the NI margin for the current trial
 What to do?
• Assess the superiority of the control over placebo (efficacy) in
historical trials.
• To assess that the consistency of control effect across historical trials:
Evaluate if there is strong fluctuation (波动) or an observed trend of
the control effect across historical trials.
Assay Sensitivity
Constancy Assumption
 Constancy assumption:
• The current effect of the active control (in the new NI trial)
is similar to
the past effect of the same treatment (in historical trials)
 What to do?
• Design the NI trial as similar as possible to the historical trials:
- Same/similar inclusion criteria (入选标准), endpoints, doses, etc.
• The assessment of constancy is only possible after conducting the
trial. Key characteristics which may have an influence on the endpoint
should be compared descriptively with the historical trials.
Constancy Assumption in SPORTIF V Study
 Is the constancy assumption reasonable in the SPORTIF V Study?
• Compare some characteristics of the six historical studies with SPORTIF V
- Study 6 had the inclusion criterion that subjects had a prior history of stroke (中风)
or transient ischemic attack (TIA, 短暂性缺血发作). None of the other studies had
such a requirement, leading to the higher event rates
- The primary endpoint of Study 6 was broader, leading to a lower risk reduction.
- The overall results are quite consistent, despite the differences above
Characteristics Study 1 2 3 4 5 6 SPORTIF V

Age (mean) 73 69 68 65 67 71 72
Male (%) 53% 75% 76% 74% 100% 59% 70%

Stroke or TIA (%) 6% 3% 3% 8% 0% 100% 18.3%
Primary endpoint Stroke, TIA, Ischemic Ischemic Ischemic Ischemic Vascular (血管) death, Stroke and
systemic stroke stroke and stroke and stroke myocardial infarct (心 systemic
embolism (缺血性中风) systemic systemic 肌梗塞), stroke, embolism
(全身性栓塞) embolism embolism systemic embolism
Assay Sensitivity
Quality
 In a NI trial, poor quality can sometimes reduce the

observed difference 𝜇𝐶 − 𝜇 𝑇 , potentially leading to a false
conclusion of non-inferiority.
 What to do?
• The trial should be conducted with high quality (i.e., follow Good
Clinical Practice)
• Avoid protocol deviations (方案违背), lost to follow-up (随访), and
missing values; assure high compliance rate (依从率)
• Analyze data properly, e.g. using the right analysis sets (分析集) and
imputation methods (缺失值插补方法) in case of missing values
Summary
 We need to be careful in choosing the active control to
avoid Biocreep
 Assay sensitivity (AS) is the ability of the trial to have

detected a difference between treatments of a specified
size
 AS is based on three considerations

• HESDE: Control group was consistently better than placebo
• Constancy assumption: Similarity of the new NI trial to the historical

trials
• Quality of the new trial
 Introduction
Summary and Conclusion
 Most often NI is part of an active controlled study design
 Active controlled studies may address two questions
• Is T superior to P?
• Is T (clincially) non-inferior to C?
 Confidence intervals are used to analyze NI studies

 Fixed margin approach is the main statistical method for NI
analysis, sometimes also called 95% - 95% approach
 NI margin must be smaller than the control effect
Summary and Conclusion
 𝑀1 is the entire effect of the active control assumed to be
present in the NI study, which comes from historical data
 𝑀2 is the largest clinically acceptable difference of the test

drug compared to the active control, in most cases 𝑀2 is a
fraction of 𝑀1
 Any conclusion that “non-inferiority was demonstrated” is

not complete without a full description of the trial
objectives, and including the NI margin in the conclusion
 Without demonstrating assay sensitivity the results from

an active-controlled study with NI cannot be interpreted
properly
Q&A
Appendix
Regulatory Guidelines
 ICH:
• ICH E9: Statistical Principles for Clinical Trials (1998) (sections
3.3.2 / 5.2.3)
• ICH E10: Choice of Control Group and Related Issues in
Clinical Trials (2000) (sections 2.1.5)
 CPMP:
• CPMP guideline: Choice of the Non-inferiority Margin (2006)
CPMP/EWP/2158/99
• CPMP Points to Consider: Switching between superiority and
non-inferiority (2000) CPMP/EWP/482/99
Appendix
Regulatory Guidelines
 FDA:
• FDA Draft guidance: Non-Inferiority Clinical Trials (2010)
• FDA guidance: Antibacterial Drug Products: Use of Noninferiority
Studies to Support Approval (2010)
• FDA Draft guidance: Community-Acquired Bacterial Pneumonia:
Developing Drugs for Treatment (2009)
• Code of Federal Regulations(21 CFR 314.126). Adequate and well-
controlled studies (2002)
Appendix
References: General
 Altman, DG, Bland, JM. Absence of evidence is not the evidence of

absence. BMJ 1995;311:485.
 Ricci, S. What Does ’Non-inferior to‘ Really Mean? Cerebrovasular

Diseases 2010;29:607-608.
 Ellenberg, S, Temple, R. Placebo-Controlled Trials and Active-Control

Trials in the Evaluation of New Treatments. Part 2: Practical Issues and
Specific Cases. Ann Intern Med. 2000;133:464-470.
 Piaggio, G, Elbourne, DR, Altman, DG, Pocock, SJ, Evans, SJ.

Reporting of noninferiority and equivalence randomized trials: an
extension of the CONSORT statement. JAMA 2006; 295: 1152–1160.
 Rothmann, MD, Wiens, BL, and Chan, ISF, Raton, B. Design and
Analysis of Non-Inferiority Trials. FL: Chapman & Hall/CRC, 2012,
ISBN 978-1-58488-804-8, xvi + 438 pp.
Appendix
References: Ethics
 Garattini S, Bertelè V: Non-inferiority trials are unethical because they

disregard patients‘ interest. Lancet 2007;370:1875-1877.
 Temple R, Ellenberg S. Placebo-Controlled Trials and Active-Control

Trials in the Evaluation of New Treatments. Part 1: Ethical and scientific
issues. Ann Intern Med. 2000;133:455-463.
Appendix
References: Examples
 Halperin, J.L., Executive Steering Committee, SPORTIF III and V Study

Investigators (2003). “Ximelagatran Compared with Warfarin for
Prevention of Thromboembolism in Patients with Nonvalvular Atrial
Fibrillation: Rationale, Objectives, and Design of a Pair of Clinical
Studies and Baseline Patient Characteristics (SPORTIF III and V).” Am
Heart J 146, 431-8.
 See www.fda.gov and use ”non-inferior“ as search term.
Appendix
References: Statistical
 Hauck WW, Anderson S (2002). Some issues in the design and

analysis of equivalence trials. Drug Information Journal;33:109-118.
 Rothmann M. et al. Design and analysis of non-inferiority mortality trials

in oncology. Stat Meth 2003;22:239-264.
 Snapinn S, Jiang Q. Controlling the type 1 error rate in non-inferiorty

trials. Stat Meth 2008;27:371-381.
 Wang SJ, Hung HMJ, Tsong Y. Utility and pitfalls of some statistical
methods in active controlled clinical trials. Controlled Clinical Trials
2002;23:15-28.
 Wang SJ, Hung HMJ. Assessing treatment efficacy in noninferiority

trials. Controlled Clinical Trials 2003;24:147-155.
Appendix
Abbreviations I
 ARR: Annualized relapse rate

 AS: Assay sensitivity
 C: (active) Control treatment
 CI: Confidence interval
 FDA: Food and Drug Administration
 HESDE: Historical evidence of sensitivity to drug effects
 ICH: International Conference on Harmonisation
 LCI: Lower bound of 95% CI
Appendix
Abbreviations II
 NI: Non-inferiority
 P: Placebo
 PE: Point estimate
 RR: Risk ratio
 T: Test treatment, investigational treatment
 TIA: Transient ischemic attack
Appendix
Glossary I
 Active control: 阳性对照

 Analysis set: 分析集
 Annualized relapse rate: 年均的复发率
 Anticoagulant: 抗凝的
 Assay sensitivity: 检测灵敏度
 Biocreep: 生物蠕变
 Compliance rate: 依从率
 Conservative: 保守的
Appendix
Glossary II
 Consistent: 一致的
 Constancy assumption: 恒定假设
 FDA: 食品及药物管理局
 Fluctuation: 波动
 Follow-up: 随访
 ICH: 国际协调会议
 Imputation methods: 缺失值插补方法
 Inclusion criteria: 入选标准
Appendix
Glossary III
 Interferon: 干扰素
 Ischemic stroke: 缺血性中风
 Log hazard risk ratios: 对数风险比例
 Margin:临界值
 Meta analysis:统合分析
 Morbidity: 病态
 Multiple sclerosis: 多发性硬化
 Myocadial infarct: 心肌梗塞
Appendix
Glossary IV
 Non-inferiority: 非劣效性
 Non-valvular atrial fibrillation: 非瓣膜性心房颤动
 Protocol deviations: 方案违背
 Relative risk: 相对风险
 Reproducible: 可复制的
 Rule out: 排除
 Stroke: 中风
 Superiority: 优效
Appendix
Glossary V
 Synthesis: 合成
 Systemic embolism: 全身性栓塞
 Transient ischemic attack: 短暂性缺血发作
 Vascular: 血管
 Warfarin: 华法令阻凝剂
 Ximelagatran: 希美加群

3-Non-Inferiority Studies

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3-Non-Inferiority Studies

Uploaded by

Copyright:

Available Formats

Non-inferiority Studies

Lecture Series – Basic Statistics in Clinical Trials

 Learn what a non-inferiority (NI,非劣效性) study is and

 Issues and challenges of NI studies

 Warfarin (华法令阻凝剂) is an effective

 The novel anticoagulant ximelagatran

 Study objectives: Show that ximelagatran is

 Here, “non inferior” means we allow ximelagatran to have

 It would be unethical to include a placebo group in this

 Placebo controlled Test

 3-arm study with placebo Test

 𝑇 = Test treatment, investigational treatment

𝐻𝑎 : 𝜇𝐶 − 𝜇 𝑇 < 0 (i.e. 𝜇𝑇 > 𝜇𝐶 )

 A treatment effect can be established statistically by showing that the

2. Superiority is not demonstrated

3. Superiority is not demonstrated

 Objective: to show that the new treatment is not inferior to an

Any NI study has two distinct objectives.

 Superiority objective to evaluate efficacy:

 NI objective to evaluate relative efficacy:

 Option 4: 3-arm NI trial with placebo design Test

 Choice between options 3 and 4

Source: Web of Science

 NI problems typically appear in studies with an active

 The two objectives of NI studies are to show:

 The degree of inferiority is measured by NI margin

 Two Real Examples

current NI study necessary condition

 NI margin 𝛿 must be smaller than or equal to the control

 The NI margin is based on the control effect in order to

 This approach follows the draft FDA guidance on non-

 𝑀1 represents the effect of C relative to P in the NI study, and

 NI is established if the upper CI bound for 𝜇𝐶 − 𝜇 𝑇 is less than 𝑀1

Possible results of a NI study with 𝑀1 = 2 and using a 95% CI

4. NI is demonstrated, but superiority is not

5. Superiority and NI are demonstrated

6. C is significantly better than T, but NI is

 However, in many cases that would not be sufficient

 Both parts need to be considered:

 Therefore in NI studies we additionally choose a smaller

 Demonstrate a difference < 𝑀2 shows that the new drug

 It is critical to note that 𝑀2 is a judgment that is made after

 The NI study design involves two comparisons: A direct

Possible results of NI study showing 𝜇𝐶 – 𝜇 𝑇 and 95% CI

2. Effect > 0, but unacceptable loss

 Be careful with: “T is non-inferior to C.”

𝜇𝐶 – 𝜇 𝑇 with 95% CI 𝜇𝐶 – 𝜇𝑃 with 95% CI

 Two confidence intervals are involved in the fixed margin approach

• One comes from the NI study itself to demonstrate 𝜇𝐶 − 𝜇𝑇 < 𝑀2

 Synthesis (合成) approach

 Successfully showing NI allows one to go ahead to test

See lecture “Introduction to Multiple Testing”

 With the fixed margin approach, the NI margin is specified

 If T is somewhat more effective than C, it will be easier to

 A less effective T will require a larger sample size

150 pat ient s/group

150 pat ient s/group

 𝑀1 represents the effect of C relative to P in the NI study,

 In NI studies we additionally choose a smaller margin 𝑀2

 Fixed margin approach is the main statistical method for

 Issues and Challenges

 Primary endpoint: Relative risk (相对风险) 𝜃𝑇 ,

 Hypotheses: 𝐻0 : 𝜃𝑇 ≥ 𝑀 versus 𝐻𝑎 : 𝜃𝑇 < 𝑀