PHARMACEUTICAL STATISTICS

Pharmaceut. Statist. 2007; 6: 283–296

Published online 23 October 2007 in Wiley InterScience
(www.interscience.wiley.com) DOI: 10.1002/pst.316

Biomarker as a classifier in
pharmacogenomics clinical trials: a
tribute to 30th anniversary of PSIz,}
Sue-Jane Wang*,y
Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and
Research, US Food and Drug Administration, Silver Spring, MD, USA

Pharmacogenetics is one of many evolving sciences that have come to the fore since the formation of
the Statisticians in the Pharmaceutical Industry (PSI) 30 years ago. Following the completion of the
human genome project and the HapMap in the early 21st century, pharmacogenetics has gradually
focused on studies of whole-genome single-nucleotide-polymorphisms screening associating disease
pathophysiology with potential therapeutic interventions. Around this time, transcription profiling
aiming at similar objectives has also been actively pursued, known as pharmacogenomics. It has
become increasingly apparent that treatment effects between different genomic patient subsets can be
dissimilar, and the value and need for genomic biomarkers to help predict effects, particularly in
cancer clinical studies, have become issues of paramount importance. Pharmacogenomics/
pharmaogenetics has thus become intensely focused on the search for genomic biomarkers for use
as classifiers to select patients in randomized-controlled trials.
We highlight that the predictive utility of a genomic classifier has tremendous clinical appeal and
that there will be growing examples in which use of a companion diagnostic will need to be considered
and may become an integral part in the utilization of drugs in medical practice. The credible
mechanism to test the clinical utility of a genomic classifier is to employ the study results from a
prospective trial that recruits all patients. Such investigations, if well designed, will allow analysis of
all relevant performance factors in the drug and diagnostic combination including the sensitivity,
specificity, positive and negative predictive values of the diagnostic test and the efficacy of the
drug. Published in 2007 by John Wiley & Sons, Ltd.

*Correspondence to: Sue-Jane Wang, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and
Research, US FDA, HFD-700, WO 22, Room 6216, 10903 New Hampshire Ave., Silver Spring, MD 20993, USA.
y
E-mail: suejane.wang@fda.hhs.gov
z
The views expressed in this article are not necessarily those of the US Food and Drug Administration.
}
This article is a US Government work and is in the public domain in the USA.

Published in 2007 by John Wiley & Sons, Ltd.

284 S.-J. Wang
Keywords: adaptive vs guided design; companion diagnostic; prognostic-predictive; prospective/
retrospective; clinical utility
1. INTRODUCTION
In the past 30 years during which the important
role of statistics in the pharmaceutical industry has
been well recognized, developments in science have
rapidly evolved and have challenged the basic and
clinical tenets of pharmaceutical research in
medicine. Clinical studies of single genes that have
a major effect on the action of a therapeutic drug
are the study of pharmacogenetics entertained in
late 1950s early 1960s, coined by Vogel [1] and
used by several others, e.g. Kalow [2]. With the
completion of the human genome project and the
HapMap [3], pharmacogenetics as a field has
evolved into studies of whole-genome-single-nu-
cleotide polymorphisms (SNPs) for screening or
association with disease and/or therapeutics.
Studies that incorporate microarray data in
preclinical, pharmacokinetic, pharmacodynamic
or clinical studies for analysis are often viewed as
pharmacogenomic studies. Simon and Wang [4]
defined pharmacogenomics as the science of
determining how the benefits and adverse effects
of a drug vary among a target population of
patients based on genomic features of the patient’s
germ line and diseased tissue. For the purpose of
this paper, we will use the general term ‘genomic’
or ‘pharmacogenomics’ to include molecular,
genetic or pharmacogenetics where appropriate.
When there is a priori defined potential genomic
subset that is postulated to have a major impact
(efficacy or toxicity) from treatment, we consider it
the positive genomic subset, denoted by gþ :
2. GENOMIC BIOMARKER AS A
CLASSIFIER
Application of genomics in clinical trials requires
the use of genomic samples and knowledge of
genomic biomarker(s), e.g. [5]. With the avail-
Published in 2007 by John Wiley & Sons, Ltd.
ability of high-dimensional data, such as gene
expression studies [6,7] or whole-genome SNPs
scan studies, e.g. [8], development of genomic
(composite) biomarkers [9] that link to treatment
effect in drug development has become widespread
and of clinical practical interest. The goal of this
interest aims at the eventual individualization of
medical treatment [6,7]. A genomic (composite)
biomarker is generally developed by combining
many individual genes via a prediction algorithm
to form a classifier that may predict patients’
therapeutic outcome or guide drug treatment. A
genomic composite biomarker as a classifier has its
appeal over a single biomarker when it provides
much higher sensitivity and specificity or when it
adds much value to the existing clinical criteria to
classify patients before treatment intervention.
Establishment of a genomic classifier may be
pursued using the traditional clinical trial phased
approach. In this paper, we discuss a learn-vs-
confirm framework that involves flexible genomic
drug trial designs following the phases of genomic
biomarker development for drug response [10].
Such a framework can be performed either
prospectively or retrospectively depending on the
availability and quality of stored genomic biospe-
cimens.
2.1. Prospective
The prospective development of a genomic bio-
marker used as a classifier for patient selection
may begin in early phase studies. The key question
is ‘Is there a genomic biomarker present at
baseline that can serve to predict therapeutic
effect?’ Before launching a late-phase registration
trial, exploration of early phase data seeks to
identify promising genomic classifier(s) that may
predict inconsistent treatment effects between
genomic patient subsets or across tumor types.
Depending on the clinical development program
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
 Treatment effect, genomic classifier, prognostic-predictive clinical utility
and its range of study options, there could be a
number of objectives for which a genomic
biomarker can be used.
2.1.1. Enrichment objective
When there is a reasonable body of biological
evidences to support a specific hypothesis that a
therapeutic agent inhibits a specific molecular
target, an enrichment design [11] may be pursued
to address the targeted gþ hypothesis, viz., H0:
Dþ ¼ 0 vs H1: Dþ > 0: The genomic biomarker in
an enrichment design is used to exclude patients,
e.g. patients whose tumors underexpress the
HER2/Neu protein were excluded from the study
of trastuzmab (HerceptinR) effect [12]. Enrichment
design requires an available diagnostic assay. To
ensure generation of meaningful data, it is
imperative that the analytical validation of the
diagnostic assay be established before initiating an
enriched design approach. For instance, the result
of a diagnostic assay providing information on the
presence/absence of the CYP2C9 allele should
match with the DNA sequence gold standard [13].
An analytically validated diagnostic assay allows
accurate classification of patients’ genomic status.
Such a diagnostic assay, if used in the enriched
trial, permits the randomized comparison for
treatment effect in the targeted gþ genomic subset.
2.1.2. Composite objective
When there is no biological plausibility or no
established known drug target, there is generally
less confidence to exclude patients on the basis of a
genomic biomarker; thus, the study of all rando-
mized patients is preferred. Below, we discuss
prospective approaches that consider randomized
comparison. First, a composite objective, H0: D ¼
0 and Dþ ¼ 0 vs H1: D>0 or Dþ > 0; that new
treatment is effective in all randomized patients or
in the gþ genomic subset can be tested in a fixed
design or an adaptive design setting [4,14–17].
These approaches provide two opportunities to
conclude a treatment effect; thus, they are subject
Published in 2007 by John Wiley & Sons, Ltd.
285
to multiplicity adjustment to ensure proper control
of at least one false-positive conclusion.
Simon and Wang [4] adopt the alpha allocation
principle of Moye and Deswal [18] without
accounting for the correlation, whereas the
approach of Wang and Hung [14] accounts for
the correlation of the two test statistics in the
context of a fixed design. Wang [15], Freidlin and
Simon [16] and Wang et al. [17] consider a two-
stage adaptive design. The utility of the correlation
is discussed [15,17], but incorporation of the
correlation for multiplicity adjustment is not a
requirement.
In the approach by FS, Stage 1 data are used to
evaluate if there is a genomic patient subset (gþ)
much more likely to benefit from the treatment,
and Stage 2 adaptively tests the treatment effect in
the identified gþ subset (H0: Dþ ¼ 0 vs H1: Dþ > 0)
using only Stage 2 data if data from both stages
fail to demonstrate an overall treatment effect (H0:
D ¼ 0 vs H1: D>0) [16]. To prospectively test the
treatment effect in the gþ genomic subset, FS
separates Stage 1 learning data to develop the
genomic classifier from Stage 2 test data.
In contrast, the development of the genomic
classifier by the WOH approach is external to the
randomized-controlled trial; thus, the analysis uses
gþ data in both stages [17]. The rationale of
patient adaptation is not to exclude the g
patient
subset unless the g
subset is failing the treatment
due to futility or serious enough early safety
concern at the end of Stage 1. This stringent rule
often permits evaluation of the clinical utility of
the genomic classifier at the study end, H0: D ¼ 0
and Dþ ¼ 0 vs H1: D>0 or D+>0. In the rare case
when the interim data strongly indicate that the
g
subset is failing the treatment, recruitment of
the remaining patients in Stage 2 would consider
only the gþ subset. The final composite hypothesis
is thus an enriched hypothesis for either (H0: Dþ ¼
0 vs H1: Dþ > 0) or (H0: D ¼ 0 vs H1: D>0) in that
the percentage of gþ patients is higher than the
estimated prevalence of gþ obtained from rando-
mized patient inclusion. WOH showed that when
the genomic classifier is developed external to the
well-controlled trial, the power gain for the gþ
subset effect of the considered adaptive design as
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
286 S.-J. Wang
compared with that of FS ranges from 20% to
40% if the utility of the classifier is predictive and
from 10% to 35% if the utility of the classifier is
prognostic-predictive using an alpha allocation of
0.02 for D and 0.005 for Dþ ; see Figures 4 and 5(A)
of WOH [17]. The power gain for the gþ subset is
due to the availability of the use of the gþ Stage 1
data that were not used to learn, although the use
of data is subject to multiplicity adjustment.
2.1.3. Separate subset objective
A design stratified by the genomic biomarker
status that tests H0: Dþ ¼ 0 vs H1: Dþ 6¼ 0 and H0:
D ¼ 0 vs H1: D 6¼ 0 [4,19] separately, essentially
undertakes two independent clinical trials, which
may not be useful if the clinical utility of the
biomarker is predictive, viz., Dþ 6¼ 0 and D ¼ 0 in
truth, to be illustrated in Section 3, Graphs (vi)–
(viii) of Figure 1. The sample size for the null effect
in the g– subset will be very troublesome in
placebo-controlled trials and will likely be very
large if a non-inferiority objective is required to
show that the effect with the new treatment is
essentially no different from the comparator with a
tight non-inferiority margin in active-controlled
trials.
2.1.4. Interaction objective
An interaction study objective tests the interaction
hypothesis of whether treatment effect in the gþ
subset is the same as that in the g
subset: H0:
Dþ ¼ D vs that these effects differ: H1: Dþ 6¼ D :
A significant interaction test result provides
evidence that the magnitude of the treatment effect
between gþ and g
differs. But, the interaction
test does not address the overall hypothesis
(H0: D ¼ 0 vs H1: D > 0) and the gþ subset
hypothesis (H0: Dþ ¼ 0 vs H1: Dþ > 0) directly.
Often, the composite hypothesis is addressed
indirectly when one starts with the interaction
test. The test sequence to indirectly address the
composite hypothesis consists of two branches.
That is, if the interaction test result does not reach
Published in 2007 by John Wiley & Sons, Ltd.
statistical significance based on a pre-specified
significance level, one tests whether treatment
effect exists in all the patients randomized
(D>0). If, however, there is a significant treat-
ment-by-biomarker interaction effect, one then
tests the g
and gþ subset hypotheses separately.
It is not clear what significance level the
interaction test should be with the branch testing
scheme described above so as to maintain the
control of the overall type I error rate [17,20]. In
addition, the interaction test generally requires
much larger sample size as compared with a study
sized for an overall effect [21]. For this reason,
when the composite objective is of primary focus,
the interaction objective may not be preferred as
the study still needs to address whether treatment
effect can be concluded in all patients or the gþ
patient subset. On the other hand, when the
significance level for testing the interaction effect
guarantees the type I error rate control, the
conservative interaction test approach due to
larger sample size is likely to increase the power
of the study for concluding the alternative
composite hypothesis.
2.1.5. Guided objective
The biomarker-guided design, e.g. [19,22], pursues
a randomized comparison between the biomarker-
guided (or biomarker-based) arm and the non-
biomarker-guided arm to test the hypothesis that
the treatment effect based on the guided approach
(Dg) is superior to the non-guided approach (Dg0 ),
i.e. H0: D ¼ 0 vs H1: D>0, where D ¼ Dg
Dg0 .
The diagnostic assay is performed at study
randomization for the biomarker-guided arm and
at the study end for the non-biomarker-guided
arm. Thus, treatment assignment is guided by
biomarker status for those patients randomized to
the guided arm and is non-random. In contrast,
treatment assignment is random for those patients
randomized to the non-guided arm if more than
one treatment is to be studied. As shown in
Table I, we discuss three variations in the two-
arm-guided design and those used in practice.
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
 Treatment effect, genomic classifier, prognostic-predictive clinical utility 287

(i). Null-Dx-factor, Null-T-effect (ii). Null-Dx-factor, T-effect

80 80

response rate
response rate

60 60
50
g- 43 g-
40 40
36 36 36 g+ 36 g+

20 20

0 0
P T Pool P T Pool

(iii). Dx-factor, Null-T-effect (iv). Dx-factor, T-effect

80 80
response rate

response rate
60 60 60
55
50 50 50 50
g- 46 g-
40 40 41
36 36 36 g+ 36 g+
20 20

0 0
P T Pool P T Pool

(v). Dx-factor, T-effect (vi). Null-Dx-factor, T-effect in g+ only

(Differential T-effects) 80
80
75 60
response rate

response rate

60 61 50
g-
47 50 g- 40 39
44 g+
40 38
g+ 28 28 28
20 20

0 0
P T Pool P T Pool

(vii). Dx-factor, T-effect in g+ only (viii). Dx-factor, T-effect in g+ only

80 80
response rate

response rate

60 60
50 50
g- g-
40 39 40 39
34 34 34 g+ g+
28 28
20 20
16 16 16

0 0
P T Pool P T Pool

Figure 1. Utility of genomic biomarker as a classifier in therapeutic drug trial.

When preliminary data indicate that adminis- an example in the non-guided arm, the caveats to
tration of a new treatment or an (approved) (1a) include (i) the randomized balance between
standard of care (SOC) is likely to result in the SOC arm and the guided arm in the gþ subset
irreversible toxicity or futile effect in the gþ is lost, and (ii) if the prevalence of gþ is low,
genomic subset, (1a) considers exclusion of these (1
prevalence)*100% of patients receives SOC in
gþ patients in the guided arm from comparison, both arms, and a superior biomarker-guided
e.g. [23], see also Figure 2(C) of [22]. Using SOC as overall treatment-induced toxicity or efficacy

Published in 2007 by John Wiley & Sons, Ltd. Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
288 S.-J. Wang
Table I. Randomization to biomarker or non-biomarker-guided treatment assignment.
Biomarker status determined by the Biomarker-guided experimental
genomic diagnostic assay arm (assay performed at study
randomization)
(1a) Exclusion of likely toxic/ineffective patients; (1b) Placebo
gþ (1a) Excluded
(1b) Pacebon
g
SOC (or TRT)
(2) Randomization balance preserved
gþ T1
g
T2
n
SOC, standard of care; TRT, treatment.
nn
T1, T2 are two different treatments.
would rely on a highly predictive biomarker, thus,
a conservative approach.
An improvement to (1a) is feasible if the
blinding of the biomarker status is properly
controlled and not known by either the patient
or the investigator. In these circumstances, pa-
tients can be kept in the trial by receiving the
placebo, i.e. (1b), and be followed up for the
treatment duration. With this modification, one
can perform a randomized comparison of SOC vs
placebo in the gþ subset. If the SOC-related
toxicity event rate is high and is very likely to be
guided by the positive biomarker status, this
comparison can be very powerful with very few
gþ patients needed even though the prevalence of
gþ may also be low. In addition, the randomiza-
tion balance is maintained. Although it may be
argued that it is unethical to withhold treatment
from these gþ patients with other available drugs
once their biomarker status is known, the justifica-
tion of whether it is unethical to withhold
treatment should be based on the duration and
expected impact of such a therapeutic decision.
The bias issue can be handled by proper
blinding. Note that the irreversible toxicity is yet
to be tested and, thus, should not have ethical
consequence. SOC in the biomarker guided design
of (1a) or (1b) would be applicable to evaluate the
suspected biomarker associated irreversible toxi-
city. If SOC is any treatment, (1b) is essentially
Figure 2(A) of [19]. The (1b) design can be applied
to test clinical utility for efficacy or safety. Such
Published in 2007 by John Wiley & Sons, Ltd.
Non-biomarker-guided control arm
(assay performed at the study end)
All receive SOC (or TRT)n
Receive T1 or T2 through
randomizationnn
guided design may not be useful for efficacy or
safety assessment unless the biomarker is highly
predictive of efficacy or safety. The treatment
effect is essentially assessed in the gþ patient
subset alone, and the overall treatment effect will
be diluted by the equal performing g– patient
subset who receives the same treatment in the
guided and non-guided arms.
The guided design in (2) of Table I also
preserves the randomization balance. In this
guided design, in principle, if there is no enriched
selection of gþ or g
patients at study randomiza-
tion, the expected percentages of gþ vs g
patients
between the guided (g) and non-guided (g0 ) arms
should be equal in probability [19] and preserves
the randomization ratio. Thus, if one tests the
hypothesis, H0: D ¼ 0 vs H1: D>0, where
D ¼ Dg
Dg0 , and the null hypothesis is rejected,
then the treatment effect of the guided approach (a
weighted average effect of T1 in gþ and T2 in g
)
being superior to that with the non-guided
approach (a weighted average effect of T1 and T2
without necessarily treating T1 in gþ and T2 in g
)
can be determined.
Such guided design, however, is inefficient
compared with the simple two-arm-randomized
biomarker-stratified design. To illustrate, suppose
the gþ prevalence in the studied patient popula-
tion is 20%. In the guided design, the biomarker-
guided study arm will have 20% of the patients
receive T1 and 80% of the patients receive T2 and,
in the non-guided study arm, 10% receives T1 and
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
 Treatment effect, genomic classifier, prognostic-predictive clinical utility
10% receives T2 in the 20% gþ patients, while
40% receives T1 and 40% receives T2 in the 80%
g
patients. In other words, within each (positive
vs negative) biomarker stratum, half of the
patients receive the same treatment and the other
half receive opposite treatments. In such scenarios,
the treatment effect within each stratum effectively
uses only half of the available patients for
comparison. The inefficiency is compounded in
the comparison of the weighted effects between the
guided vs the non-guided arms, D ¼ Dg
Dg0 . If
the treatment effect is reversed between strata, e.g.
a positive effect in gþ and a negative effect in g

patient subsets, the testing of treatment effect
within each stratum is also held back due to the
use of half of the patients, and the overall effect
may be much smaller due to the offset by the
positive and negative effects. The usefulness of
biomarker-guided designs discussed in Table I also
relies on accurate classifiers exemplifying a receiver
operating characteristics curve with very high odds
ratio, the ratio of the odds of true-positive fraction
to the odds of false-positive fraction, e.g. OR c36
[24].
In practice, other than the example by Hughes
et al. [23], modification to the listed guided designs
of Table I is considered in the TAILORx trial [25]
and the MINDACT trial [26]. The primary
comparison in the TAILORx trial is performed
only in those patients who are in the intermediate
risk group and these patients are randomized to
either the experimental (biomarker-guided) arm or
the SOC arm. Thus, there is no loss in the
efficiency of the design for comparison in this
intermediate risk stratum. In the MINDACT trial,
the primary comparison is in the discordant pairs
between the genomic risk and the clinical risk
defined before treatment. With these two guided
designs, those patients who are at low risk
(genomic low and clinical low) or high risk
(genomic high and clinical high) in the TAILORx
(MIDACT) trial receive specific treatment not
based on randomization. We note that the
genomic diagnostic assay based on the 70-gene
expression signature has been approved by reg-
ulatory agency, known as MammaPrintR [27].
Based on the analytic validation of the approved
Published in 2007 by John Wiley & Sons, Ltd.
289
prognostic utility of MammaPrint, the predictive
clinical utility of the 70-gene signature used in the
MINDACT-controlled trial prospectively can be
further assessed with better precision.
2.2. Retrospective
An approved drug may have several competitors
on the drug market, but its benefit–risk profile may
be much less desirable than its competitors [28].
Completed late phase clinical trials may fail to
demonstrate a treatment effect. Other retrospective
situations include completed cohort clinical studies
that are not necessarily randomized and well
controlled, a nested case control study within a
completed controlled trial or an ongoing study
that investigates genomic exposure retrospectively.
These completed/ongoing clinical studies or clin-
ical trials can be very useful. For instance, the
recent wake up call of high failure rates in late-
phase drug development [29] and the ability to
bank the genomic samples taken before clinical
trial initiation has given the clinical trial commu-
nity the opportunity to perform flexible explora-
tory genomic drug trials [10,28]. In such cases,
development of a genomic biomarker using
banked genomic samples from completed well-
controlled clinical trials individually or meta-
analytically offers a logical tool for retrospective
exploration. The goal is to identify a genomic
subset showing much larger treatment benefit than
the originally intended patient population if
heterogeneity of therapeutic effect is at stake.
Retrospective development of a genomic bio-
marker is of genuine interest and often driven by
both time and cost considerations. Putting aside
the genomic sample quality issues, during the
exploration stage, there can be several genomic
biomarkers that are equally plausible and promis-
ing for their predictive clinical utility derived from
gene expression or whole-genomic SNPs scan data.
This is due to the fact that often more than
hundred-folds or thousand-folds of genes are
available than the number of patients studied.
The genomic biomarker developed using the
training data should be validated in an indepen-
dent test data [30]. Indeed, exploration using
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
290 S.-J. Wang
completed clinical trials, ongoing clinical trials or
observational retrospective studies where archived
genomic biospecimens are accessible has become
an emerging path for research and development in
drug development, e.g. [6–8,10,28].
There are several advantages of retrospective
exploration.
* One maximizes accrual through a collection of
all completed clinical studies.
* The right biomarker need not be known a
priori.
* Retrospective evaluation allows exploration
and refinement of the genomic biomarker with
much larger patient database than early pro-
spective studies.
* Treatment effect can be assessed in the gþ and
g
biomarker subsets.
There are some disadvantages.
* Not all randomized patients will consent to
participate in the pharmacogenomics sub(stu-
dy), resulting in convenience patient samples
that can range from as low as 10%–15% to as
high as 80%–85%.
* Methods of genomic sample collection and
genomic sample handling may be suboptimal.
* Time factor may impact on the sample quality
depending on the platforms available over time.
* Missing data in clinical sample and genomic
sample, and the methods of imputation of these
missing data can be challenging.
For exploratory purposes, however, these issues
may be less of concern.
3. UTILITY OF GENOMIC
BIOMARKER INVOLVING DRUG
TREATMENT
In a typical conventional clinical trial, the study
objective is to investigate whether an experimental
treatment is superior or non-inferior to its
comparator. After treatment effect is demon-
strated, the impact of important baseline covari-
ates on treatment effect is then assessed for a
Published in 2007 by John Wiley & Sons, Ltd.
consistent or an inconsistent effect detected by
treatment-by-covariate interaction. Wang et al.
[17] presented three scenarios of clinical utility of a
genomic biomarker in the setting of a randomized-
controlled trial:
Scenario A: predictive of drug effect;
Scenario B: prognostic of disease mechanism;
Scenario C: prognostic of disease state and
predictive of drug effect.
Here, we exhaust all types of treatment effect
outcomes in a two-arm placebo-controlled clinical
trial within which a genomic biomarker might be
used as a classifier to determine the treatment
effect in a specific gþ genomic subset, see Graphs
(i)–(viii) in Figure 1.
To illustrate, we use response rate as the
primary endpoint. In the following graphs, disease
state is abbreviated as Dx and treatment is
abbreviated as T. In Figure 1, (i) (neither genomic
biomarker nor drug treatment plays a role in
disease pathophysiology and drug effect) and (ii)
(genomic biomarker does not impact on treatment
effect) presents the scenarios where the genomic
biomarker used as a classifier has no disease utility
and clinical utility. Thus, there is no value of such
a genomic biomarker in drug development. When
a genomic biomarker has the ability to predict
differential clinical outcomes in a group of patients
independent of their exposure to treatment, all
graphs except (i) and (ii) would satisfy the
definition of prognostics based on the analysis
pooling across the treatment (T) and placebo (P)
groups, see the response rates under the ‘pool’ x-
axis.
To investigate if there is a treatment effect,
clinical outcomes in patients from both T and P
groups are compared. Graph (iii) shows that the
genomic biomarker is prognostic of disease me-
chanism and has no impact on treatment (also
Scenario B of [17]), and Graph (iv) depicts a
prognostic genomic biomarker for disease me-
chanism and the treatment effect is not impacted
by the biomarker status. For the purpose of a
diagnostic assay development, the prognostic
utility of a diagnostics is generally not considered
a high-risk product, e.g. [27]. For assessing
therapeutics, a prognostic biomarker can improve
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
 Treatment effect, genomic classifier, prognostic-predictive clinical utility
the precision of a treatment effect estimate in a
linear model.
Using a cancer trial as an example, if the
predictivity defines sensitivity of a tumor to a
distinct therapeutic agent, then Graphs (v)–(viii)
satisfy the definition of predictivity. For power
consideration in designing a clinical trial, Wang
et al. [17] distinguish between quantitative treat-
ment-by-biomarker interaction (Graph (v)) and
qualitative interaction (Graphs (vi)–(viii)). For
quantitative interaction, a genomic biomarker that
is prognostic of treatment effect, but with different
magnitudes between the gþ and g
biomarker
subsets, such as those shown in Graph (v), the
biomarker is prognostic of disease state and
predictive of differential treatment effects, where
Dþ > D > 0: In such cases, the genomic biomarker
results in differential treatment effects and its
clinical utility is considered prognostic-predictive
of drug effect (also Scenario C of [17]).
In contrast, as shown in Graphs (vi)–(viii), there
are two distinct treatment effects (D ¼ 0 and
Dþ > 0): a null effect in the g
genomic subset and
a positive treatment effect in the gþ genomic
subset. Such treatment-by-biomarker qualitative
interaction signifies the predictive nature of the
genomic biomarker (also Scenario A of [17]). In
the extreme case, the treatment effect can be
inferior in the g
genomic subset, i.e. D 50 and
Dþ > 0: Whether the genomic classifier is based on
a single gene or derived from multiple genes
expressed by a well-defined risk score or prediction
algorithm, the genomic (composite) biomarker for
assessing drug effect can be defined as a measur-
able characteristic serving as an indicator for
differential response or predictive of response to
therapy, e.g. between patients diagnostically clas-
sified as gþ or g
[9].
4. CONFIRMATION OF CLINICAL
UTILITY
Ultimately, for confirmatory purposes, two major
components should be required. First, one pre-
specifies a superiority hypothesis in an enriched
patient population or a priori clinical composite
Published in 2007 by John Wiley & Sons, Ltd.
291
hypothesis based on a single well-defined genomic
(composite) biomarker in the patient population
studied. Second, the analytical validity of the
genomic diagnostic assay should have been estab-
lished to accurately classify patients for entry to
the randomized clinical trial. In practice, the
interest to analytically validate the diagnostics
performance at the time of initiating a phase III
well-controlled trial for the composite hypothesis
may not be high, partly when the genomic
biomarker or the clinical utility of the genomic
biomarker is still under exploration or may not be
well established at trial initiation, e.g. [16]. Thus,
the study results yield less informative data.
When the analytical validity of the diagnostic
assay is not established and only the treatment
effect in the gþ subset is concluded, those designs
prospectively specify the study objectives that can
at best be considered a preliminary assessment of
clinical utility. The prospective intent to study the
clinical utility of a genomic classifier is congruent
with scientific principles. In the trastuzmab trial
[12], it would be useful had an analytically
validated HER2 diagnostic test been concurrently
used in the drug trial. Fortunately, the study
showed that Herceptin+chemotherapy is superior
to chemotherapy alone in all breast cancer patients
studied, the efficacy standard for a two-arm-
controlled trial. Thus, the concerns of the analy-
tical validation not established for the HER2
diagnostic test in the same trial were somewhat less
when the effect was also observed in the HER2 3+
patient subset identified by means of the home-
brew IHC kits developed in the research labora-
tory [4].
Although the specificity can be as low as 0.6 of a
diagnostic test, compared with the conventional
untargeted design of a single one-size-fits-all
hypothesis, the enrichment design generally has
sufficient power with fewer patients in terms of
number of patients randomized when the genomic
classifier is predictive or prognostic-predictive [31].
However, in terms of number of patients screened,
the untargeted design may not always be less
efficient, especially, when the genomic classifier is
only prognostic-predictive [31]. The adaptive de-
sign considered by WOH increases the power
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
292 S.-J. Wang
dramatically in selecting a sensitive patient subset
if the genomic biomarker is predictive of ther-
apeutics [17]. The biomarker-guided design that
excludes the gþ patient subset relies on highly
predictive biomarker classifiers to guide the treat-
ment for safety or for efficacy, and can be
improved to include the gþ subset in the guided
arm for randomized comparison of SOC vs
placebo within this gþ subset if the blinding of
the genomic status is feasible. Such modification
can increase the power for the gþ subset hypoth-
esis using a two-stage adaptive design with a pre-
specified composite hypothesis, e.g. [17]. The
modification can also provide a reference compar-
ison in the gþ subset when the prevalence of gþ is
low. It is worthwhile to note that improvement in
overall power and reduction in overall variability
through these designs are also a function of the
prevalence of gþ in addition to the usual design
parameters for sizing the study.
The pre-specified hypothesis should be tested in
an independent study not used for developing the
genomic classifier. Ideally, such study should be
prospectively planned and executed in a well-
controlled investigation separating early learning
data from data for independent confirmation.
Those designs described in Section 2.1, fixed,
adaptive or guided and that are pursued after the
analytical validity has been established are the best
candidates to confirm the clinical utility of the
genomic classifier prospectively.
In recent years, there has been an enthusiastic
interest in the use of a prospective/retrospective
framework. This framework allows one to pro-
spectively assess the clinical utility using clinical
data from completed randomized controlled clin-
ical trials for which the genomic classifier is
developed from yet other completed clinical
studies or clinical trials described in Section 2.2.
The major arguments of prospective/retrospective
assessment include the following:
(1) The confirmatory study is completely inde-
pendent of those studies used to develop the
genomic classifier.
(2) The classifier status of the genomic biomarker
is prospectively determined based on blinded
Published in 2007 by John Wiley & Sons, Ltd.
principle, i.e. those who determine the classi-
fication statuses are blinded to the treatment
code [28].
(3) The clinical data including treatment assign-
ment are retrospective.
(4) The genomic study on classifier’s clinical
utility is from a randomized well-controlled
clinical trial, though completed.
It seems intuitive that when the quality of the
banked genomic samples meets the standards of
archived biospecimens, the clinical data from the
prospective/retrospective study may be considered
as good as the original clinical trial conducted.
However, in clinical trial practice, two levels of
consent are required for conducting a clinical trial
and also studying the pharmacogenomics. Only
those patients who also consented to genomic
samples will qualify for the pharmacogenomic
(sub)study. Among other issues, genomic sample
quality control, standardization of the genomic
diagnostic assay, missing genomic status, it is not
clear whether randomized balance is preserved in
these convenient genomic samples. The various
bias issues and potential confounding effects well-
known from convenient cohort study cannot be
ignored. In addition, such completed clinical trials
often are those that have failed their primary
clinical effect prospectively investigated and are
known to be negative studies. Such studies would
be primarily exploratory for hypothesis genera-
tion, as the experimentwise type I error rate has
already been spent and the negative study evidence
has already been concluded. The genomic biomar-
ker only identified/defined at the time of the
interim analysis that embarked upon the compo-
site objective at trial initiation suffers these same
set of issues.
If the clinical hypothesis can be completely
absorbed by the genomic hypothesis within the
same trial, it might be argued that the experiment-
wise type I error rate already spent for the clinical
hypothesis can be claimed back and can be re-used
for the genomic hypothesis test. It is not immedi-
ately obvious that such a strong assumption can
be justified. It seems reasonable that the two
hypotheses, clinical and genomic, are correlated.
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
 Treatment effect, genomic classifier, prognostic-predictive clinical utility
Environmental factors are likely to play some roles
in between the clinical and the genomic hypotheses.
One has to acknowledge that the clinical utility
of predictivity is unlikely to be established from
completed studies. The predictivity is about the
future and not the past. Of note, if in truth
the genomic biomarker used as a classifier is
predictive of therapeutic effect as those shown in
Graphs (vi)–(viii), the effect size is generally much
larger than the conventionally identified treatment
effect size planned [14,17]. Thus, only a small
number of patients would be needed to prospec-
tively assess the treatment effect in the gþ
subset alone for confirmation. This is practically
doable when the prevalence of gþ is low, e.g. n ¼
73 in the gþ subset corresponding to 8% of the
randomized patients in [23].
5. SURROGATE ENDPOINT IN
PREDICTIVE BIOMARKER
FRAMEWORK
Proper validation of a candidate surrogate end-
point (S) [32] in therapeutic intervention trials that
can substitute ultimate clinical outcome (T)
requires more than a single trial, presumably, a
series of randomized clinical trials that have both S
and T measured [33,34]. Prentice criteria [35] have
been viewed to be too stringent and strict as the
correlation between both S and T and the
associated treatment effect on S and on T must
be established for a validated predictive surrogacy.
In addition, Fleming [36] argues that one can
rarely establish that surrogate endpoints are valid.
Even in that rare setting in which data on
treatment Z would allow one to view S as a valid
surrogate for T, one cannot extrapolate this
surrogacy to any new treatment Z* that could
have mechanisms of action that differ from those
of Z. Clearly, the timeframe for establishing a
valid S from a series of clinical trials needed where
T is usually the overall survival can be over-
whelming.
A major difficulty in establishing a validated S
statistically arises when S, e.g. progression free
Published in 2007 by John Wiley & Sons, Ltd.
293
survival, shows a statistically significant associa-
tion to treatment, but T, the clinical endpoint, say,
overall survival in cancer trials, cannot consis-
tently show a statistically significant benefit in a
series of clinical trials. In such cases, the effect of
treatment is shown only on S. If one were to
classify patients into two groups, those who had
longer time to progression vs those showing no
difference in (or much shorter) time to progression,
and if these two types of patients can be described
by some important baseline characteristics such as
genomic classifier or clinical classifier through post
hoc exploration, there might be a plausible link
between the genomic classifier and the therapeu-
tics, e.g. [12].
It is vital that the utility of the baseline
characteristics as a classifier so identified be
prospectively tested and studied as those described
in Section 2.1, e.g. the composite objective. With
such assessment, the classifier is used to prospec-
tively select patients for treatment that is indepen-
dent of treatment intervention, and longer survival
is likely observed in just one of the two subsets,
e.g. hazard ratio of 0.72, p50.002 in the good
prognostic group, and 1.03, p ¼ 0:90 in the poor
prognostic group. When this is the case, the
estimated hazard reduction on overall survival
(e.g. hazard ratio of 0.89, p ¼ 0:27) will be diluted
due to the differential survival benefit observed
between the gþ and g– subsets and, thus, likely to
fail the statistical significance when the two subsets
are combined for analysis.
In other words, the concept of identifying a
genomic biomarker that is used to prospectively
select sensitive patients can be a useful tool for the
problem of a candidate predictive surrogacy. It is
difficult to discern if a surrogacy is confounded by
drug intervention. A pre-specified gþ genomic
subset may at least predict one of the multiple
causal pathways influencing the drug treatment.
Turning a surrogate endpoint problem into a
biomarker classifier problem can be much more
appealing than a candidate surrogate validation
approach which can be much more difficult,
requiring a series of clinical trials, and time
consuming. If the predictive clinical utility of the
genomic classifier exists, as long as the survival
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
294 S.-J. Wang
benefit can be demonstrated in the gþ subset, even
if the survival benefit in the g– subset may be much
less or futile, the explored treatment effect in the
gþ subset can be used to generate hypothesis for
future trial planning. Unlike validation of a
surrogate endpoint, this approach does not require
a series of clinical trials to identify and confirm a
predictive gþ genomic subset treatment effect. It is
advisable to consider use of a composite hypoth-
esis that states treatment effect exists in either all
randomized patients or only the gþ patient subsets
for prospective planning and confirmation.
6. DISCUSSION
This paper discusses how pharmacogenetics and
pharmacogenomics have grown to influence how
design and analysis of clinical trials used in the last
30 years can be improved. The genomic biomarker
may be a single biomarker or a composite
biomarker derived from, e.g. transcription profil-
ing, whole-genome SNPs scan study.
To learn if a genomic biomarker used as a
classifier can offer prognostic-predictive or pre-
dictive clinical utility, there are at least three steps.
* Development of a genomic biomarker can be
achieved through a series of early prospective
clinical studies or a series of retrospective
analyses of completed clinical studies with
available banked baseline genomic samples for
meta-analytic exploration.
* Preliminary understanding of the clinical utility
may be demonstrated from randomized well-
controlled investigations. If the analytical vali-
dation of the genomic diagnostic assay has not
been established, these studies serve to provide
preliminary evidence of the genomic classifier’s
clinical utility and the diagnostic assay’s per-
formance characteristics where all biomarker-
positive and biomarker-negative patients are
included, e.g. [16,17].
* With an approved genomic diagnostic assay
that is analytically validated and a hypothesized
predictive clinical utility, one can put the
clinical utility of the genomic classifier to the
Published in 2007 by John Wiley & Sons, Ltd.
test within the same trial that consists of both
biomarker-positive and -negative patients. The
ability to assess the relevant characteristics in
the drug trial and the diagnostic trial allows
estimation of the performance characteristics,
e.g. sensitivity, specificity, positive and negative
predictive values, with better precision for drug-
diagnostic co-development or a companion
diagnostic development. Such an approach also
allows adherence to the reporting standards for
the drug trial [37] and the diagnostic trial [38]
where applicable.
A prospective randomized well-controlled investi-
gation is a much more credible standard for
definitive inference that uses proper statistical
methods for evaluation. This was acknowledged
30 years ago and remains the gold standard today.
Scientific standard of substantial evidence derived
from adequate and well-controlled investigations
will remain. This includes the concept of hypoth-
esis testing, estimation, randomization and blind-
ing that set the foundation of statistical principle
established in 1970s during the time of the
formation of PSI, materialized in ICH E-9 [39]
with further emphasis on comparative benefits and
risks that should be optimized for patients for real
world use.
In most cases, there is large uncertainty on the
predictive utility of a genomic classifier due to little
available knowledge about the drug target or all
the possible causal pathways for a therapeutic
effect. Assume the quality standard of the banked
genomic biospecimens and standardization of the
diagnostic assays are acceptable. The adaptive
feature, if well designed, to prospectively select
sensitive patients based on molecular or genomic
biomarkers, which are used as a classifier, will
likely increase the therapeutic index. This includes
the proposal to turn the surrogate endpoint
problem to a (genomic) classifier problem. These
alternative designs have great potential to bring
clinical trial research one step closer to persona-
lized medicine [25,26,40]. For the next 30 years and
more, as this research unfolds, it is envisioned that
there will be plenty of opportunities for improve-
ment to benefit/risk assessments from the conven-
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
 Treatment effect, genomic classifier, prognostic-predictive clinical utility
tional one-size-fits-all two-arm well-controlled
comparison. More research on achieving develop-
ment of drug and diagnostics to target those
patients who benefit will continue to be an ongoing
endeavor.
ACKNOWLEDGEMENTS
The author would like to thank Dr Steven Julious for
the invitation and Dr Steve Gutman and the anonymous
referees for their constructive comments during the
preparation of this work. This research work was
supported by the RSR funds #02-06, #04-06, #05-02
and #05-14 awarded by the Center for Drug Evaluation
and Research, US Food and Drug Administration.
REFERENCES
1. Vogel F. Moderne probleme der humangenetic.
Ergebnisse der Inneren Medizı¨n und Kinderheilkunde
1959; 165:835–837.
2. Kalow W. Pharmacogenetics: heredity and the
response to drugs. W.B. Saunders: Philadelphia,
PA/London, 1962.
3. The International HapMap Consortium. A haplo-
type map of the human genome. Nature 2005;
437(7063):1299–1320.
4. Simon R, Wang SJ. Use of genomic signatures in
therapeutics development in oncology and other
diseases. The Pharmacogenomics Journal 2006;
6:166–173.
5. Gutman S, Kessler LG. The US Food and Drug
Administration perspective on cancer biomarker
development. Nature Reviews: Cancer 2006; 6:565–
571.
6. van’t Veer LJ, Dai H, Vijver MJvd et al. Gene
expression profiling predicts clinical outcome of
breast cancer. Nature 2002; 415:530–536.
7. Paik S, Shak S, Tang G, Kim C, Baker J, Cronin M,
Baehner FL, Walker MG, Watson D, Park T, Hiller
W, Fisher ER, Wickerham DL, Bryant J, Wolmark
N. A multigene assay to predict recurrence of
tamoxifen-treated, node-negative breast cancer. New
England Journal of Medicine 2004; 351:2817–2826.
8. The Wellcome Trust Case Control Consortium.
Genome-wide association study of 14,000 cases of
seven common diseases and 3,000 shared controls.
Nature 2007; 447:661–683.
9. Wang SJ. Utility of high dimensional genomic
biomarkers in therapeutic and/or diagnostic devel-
opment. The IEEE Conference Proceedings for
the Fifth International Emerging Information
Published in 2007 by John Wiley & Sons, Ltd.
295
Technology Conference, Taipei, Taiwan. DOI
10.1109/EITC.2005.1544330. 2005; 13–16.
10. Maruvada P, Srivastava S. Joint National Cancer
Institute–Food and Drug Administration Work-
shop on research strategies, study designs, and
statistical approaches to biomarker validation
for cancer diagnosis and detection. Cancer Epide-
miology, Biomarkers & Prevention 2006; 15(6):
1078–1082.
11. Temple RJ. Enrichment designs: efficiency in devel-
opment of cancer treatments. Journal of Clinical
Oncology 2005; 23(22):4838–4839.
12. Baselga J. Herceptin alone or in combination with
chemotherapy in the treatment of HER2-positive
metastatic breast cancer: pivotal trials. Oncology
2001; 61(S2):14–21.
13. Jain KK. Applications of AmpliChip CYP450.
Molecular Diagnostics 2005; 9(3):119–127.
14. Wang SJ, Hung HMJ. Trials in trials: alpha
allocation strategy and sub-trial planning. The
Proceedings of American Statistical Association,
Biopharmaceutical Section [CD-ROM], American
Statistical Association: Alexandria, VA, 2005.
15. Wang SJ. ‘Regulatory update on pharmacoge-
nomics: an FDA perspective.’ Special Report in the
First Multi-track DIA Workshop on ‘DIA Congress
on the Development and Utilization of Pharmaceu-
ticals moving towards eRegulation/Risk Management
– Safety and Efficacy/Biostatistics’, Japan, 2005;
16–20.
16. Freidlin B, Simon R. Adaptive signature design: an
adaptive clinical trial design for generating and
prospectively testing a gene expression signature for
sensitive patients. Clinical Cancer Research 2005;
11(21):7872–7878.
17. Wang SJ, O’Neill RT, Hung HMJ. Approaches to
evaluation of treatment effect in randomized clinical
trials with genomic subset. Pharmaceutical Statistics
2007. DOI: 10.1002/pst.300.
18. Moye LA, Deswal A. Trials within trials: confirma-
tory subgroup analyses in controlled clinical experi-
ments. Controlled Clinical Trial 2001; 22:605–619.
19. Sargent DJ, Conley BA, Allegra C, Collette L.
Clinical trial designs for predictive marker valida-
tion in cancer treatment trials. Journal of Clinical
Oncology 2005; 23(9):2020–2027.
20. Simon R. Validation of pharmacogenomic biomar-
ker classifiers for treatment selection. Cancer Bio-
markers 2006; 2:89–96.
21. Lachenbruch PA. A note on sample size computa-
tion for testing interactions. Statistics in Medicine
1988; 7:467–469.
22. Pusztai L, Hess KR. Clinical trial design for
microarray predictive marker discovery and
assessment. Annals of Oncology 2004; 15(12):
1731–1737.
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst
296 S.-J. Wang
23. Hughes S, Hughes A, Brothers C, Spreen W,
Thorborn D. PREDICT-1 (CNA106030): the first
powered, prospective trial of pharmacogenetic
screening to reduce drug adverse events. Pharma-
ceutical Statistics 2007. DOI: 10.1002/pst.286.
24. Pepe MS. Evaluating technologies for classification
and prediction in medicine. Statistics in Medicine
2005; 24:3687–3696.
25. Sparano JA. Current clinical trial – the TAILORx
trial: individualized options for treatment. Commu-
nity Oncology 2006; 3:494–496.
26. Bogaerts J, Cardoso F, Buyse M, Braga S,
Loi S, Harrison JA, Bines J, Mook S, Decker N,
Ravdin P, Therasse P, Rutgers E, van’t Veer LJ,
Piccart M on behalf of the TRANSBIG consortium.
Gene signature evaluation as a prognostic tool:
challenges in the design of the MINDACT
trial. Nature Clinical Practice Oncology 2006;
3(10):540–551.
27. FDA Clears the First In Vitro Diagnostic Multi-
variate Index Assay. Office of In Vitro Diagnostic
Device Evaluation and Safety (OIVD) at the Food
and Drug Administration (FDA), 6 February 2007.
Available at: http://www.fda.gov/cdrh/oivd/news.
html.
28. Wang SJ, Cohen N, Katz DA, Ruano G, Shaw P,
Spear B. Retrospective validation of genomic
biomarkers – what are the questions, challenges
and strategies for developing useful relationships to
clinical outcomes – workshop summary. The Phar-
macogenomics Journal 2006; 6:82–88.
29. Kola I, Landis J. Can the pharmaceutical industry
reduce attrition rates? Nature Review: Drug Dis-
covery 2004; 3(8):711–715.
30. Simon R, Radmacher MD, Dobbin K, McShane
LM. Pitfalls in the use of DNA microarray data for
diagnostic and prognostic classification. Journal of
National Cancer Institute 2003; 95:14–18.
Published in 2007 by John Wiley & Sons, Ltd.
31. Maitournam A, Simon R. On the efficiency of
targeted clinical trials. Statistics in Medicine 2005;
24:329–339.
32. Biomarkers Definitions Working Group. Commen-
tary: biomarkers and surrogate endpoints: preferred
definitions and conceptual framework. Clinical
Pharmacology and Therapeutics 2001; 69(3):89–95.
33. Fleming TR, DeMets DL. Surrogate end points in
clinical trials: are we being misled? Annals of Internal
Medicine 1996; 125(7):605–613.
34. Buyse M, Molenberghs G, Burzykowski T, Renard
D, Geys H. The validation of surrogate endpoints in
meta-analysis of randomized experiments. Biosta-
tistics 2000; 1:49–67.
35. Prentice RL. Surrogate endpoints in clinical trials:
definition and operational criteria. Statistics in
Medicine 1989; 8:431–440.
36. Fleming TR. Evaluating therapeutic interventions:
some issues and experiences (with discussion and
rejoinder). Statistical Science 1992; 7:428–456.
37. Moher D, Schulz KF, Altman DG. The CONSORT
statement: revised recommendations for improving
the quality of reports of parallel-group randomized
trials. The Lancet 2001; 357(9263):1191–1194.
38. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA,
Glasziou PP, Irwig LM, Lijmer JG, Moher D,
Rennie D, de Vet HC. Towards complete and
accurate reporting of studies of diagnostic accuracy:
the STARD initiative. Standards for reporting of
diagnostic assay. Clinical Chemistry 2003; 49(1):1–6.
39. US Food and Drug Administration. Statistical
principles for clinical trials (ICH E-9). International
Conference on Harmonization. U.S. Food and Drug
Administration, DHHS, February 1998.
40. Wang SJ. Genomic biomarker derived therapeutic
effect in pharmacogenomicsclinical trials: a biosta-
tistics view of personalized medicine. Taiwan Clin-
ical Trials 2006; 4:57–66.
Pharmaceut. Statist. 2007; 6: 283–296
DOI: 10.1002/pst