See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/282914684

“The Use of the Analytical Target Profile in the Lifecycle of an Analytical

Procedure: with an example for an HPLC Procedure”

Article · January 2015

CITATIONS READS
3 1,472

1 author:

Ron Snee
Snee Associates, LLC
403 PUBLICATIONS   6,151 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Strategies of Formulations Development: A Step by Step Guide Using JMP View project

Data Mining and Analysis - Practical Approaches View project

All content following this page was uploaded by Ron Snee on 19 November 2015.

The user has requested enhancement of the downloaded file.

 Peer Reviewed: Analytical Procedure Journal of Validation Technology
The Use of the Analytical Target Profile in the Lifecycle of an Analytical Procedure
with an Example of for an HPLC Procedure
Jane Weitzel, Robert A. Forbes, and Ronald D. Snee
Introduction
The lifecycle management of an analytical
procedure applying Quality by Design (QbD)
concepts and using the analytical target profile
(ATP) has been introduced by the USP stimu-
li article “Lifecycle Management of Analytical
Procedures: Method Development, Procedure
Performance Qualification, and Procedure Per-
formance Verification [1].” Current regulatory
guidance around process validation is moving
away from a simple once-and-done validation,
and toward a continuous process verification
approach, also called a “lifecycle approach” [2].
The recent FDA method validation draft guid-
ance also includes some discussion of a lifecycle
approach for analytical methods [3].
A number of articles have been written in-
troducing the lifecycle management of analyti-
cal procedures [4 – 7]. These articles introduced
the concept that analytical procedures are pro-
cesses and, as such, the tools, approaches and
statistical analyses for manufacturing process-
es can be applied to analytical procedures. The
benefit of recognizing and acknowledging the
similarity between quality by design for man-
ufacturing processes and analytical quality by
design (aQbD) is that it will allow the applica-
tion of the experience, knowledge, tools, and
statistics from QbD to be applied to aQbD.
Thus, all parties, laboratories, companies, Qual-
ity Assurance, production, as well as regulatory
bodies can leverage their experience with QbD
in developing an approach to aQbD.
A key component of the aQbD paradigm
is the definition of the Analytical Target Pro-
file (ATP) [1], analogous to the Quality Target
Product Profile (QTPP) in the ICH [8]. The
ATP defines the requirements for the result of
a test method and is based on the suitability
for use of that result. This paper illustrates how
the ATP is developed and applied to assess the
performance of the method at various stages of
its lifecycle. The use of traditional Gage R&R
statistical analysis and control sample charting,
which are commonly used in manufacturing
process validation and improvement, will also
be illustrated in the development of an analyt-
ical procedure. The similarities, contrasts, and
differences between the lifecycle approach, and
the traditional approach to validation of an an-
alytical procedure as described in the USP [9
–11] and the ICH [12], will be discussed and il-
lustrated in the example. The terminology used
with both approaches will be compared and
explained.
The aQbD approach is illustrated using an
example of the potency test for a drug sub-
stance (DS) with specification limits of 98.0%
to 102.0%. The potency test is one component
in the DS control strategy utilized to ensure the
safety and efficacy of a drug product (DP) man-
ufactured from the DS. The test ensures that the
DS is of adequate potency, so that when used in
the DP, the patient will receive the appropriate
dose of the active ingredient. In this way, poten-
cy is a critical quality attribute (CQA) of the DS
that is linked to a CQA for the DP [13]. For sim-
plicity, we will assume the limits are on the as-is
basis, and no correction for water or solvents
is necessary. Obviously, there are other speci-
fication tests (e.g. impurities, residual solvents,
etc.) that are required to ensure the DS meets its
CQAs, but these will not be considered in this
illustration. It should also be noted that the DS
potency results used for this illustration, while
actual results, are provided mainly for illustra-
tive purposes.
In this example, we will focus on the preci-
sion and uncertainty performance characteris-
tics of the potency test method to demonstrate
how the gage R&R tool and experimentally de-
signed intermediate precision studies are used
for Procedure Performance Qualification and
Verification. The acceptance criterion for the
precision of the potency method will be set us-
ing the Lifecycle approach where the intended
use of the method is defined using a decision
rule. The decision rule defines the acceptable
level of probability of making an incorrect de-
cision. For this discussion, the null hypothesis
is “The potencyof the sample is within specifi-
cation,” so there are two different types of incor-
rect decisions. First, a batch that has acceptable
potency may test outside the specification limits.
Second, a batch with a potency truly outside of
the specifications limits, could test within limits
and be accepted. The first error can be consid-
ered a false positive (positive test for unaccept-
able potency), or a type I error. The second er-
rror would be a false negative (negative test for
www.ivtnetwork.com
 Jane Weitzel, Robert A. Forbes, Ronald D. Snee
unacceptable potency) or a type II error. The
consequences of each of these types of errors
are taken into consideration when developing
the decision rule. The ATP uses the probability
of each of these errors to define the maximum
allowed measurement variation (uncertainty)
for the reported result, which translates into an
the laboratory sample, continues with the treat-
ment of that sample which may involve taking
a test-portion, comminution, extraction, disso-
lution, etc. and concludes with the output, the
reportable result as shown in Figure 2.
The FDA definition for process validation is

Figure 3 QbD for analytical procedures is analogous to that terms for the major steps in the QbD approach.
for production processes. This figure shows the analogous

throughout its lifecycle of use which establish- of variability.

es scientific evidence that a method is capable
of consistently delivering quality data [4].” For
this article, the method is the same as the an-
alytical procedure and includes one full execu-
tion of the method, starting from the original
sample, to produce a reportable result as de-
fined in the US Food and Drug Administration
out-of-specification (OOS) guidance [14]. The
Figure 1 shows that the intended use of the reportable result defines the decision rule, which includes the acceptable prob- reportable result is compared to the specifica-
ability of making a wrong decision. The Analytical Target Profile uses the probability to determine the maximum allowed tion to determine compliance.
measurement variation for the reported result or target measurement uncertainty (TMU). The TMU is an acceptance
The analogy between QbD for the produc-
criterion for the performance characteristics. The process is continuous because it can be repeated if the requirements of
the ATP cannot be met. tion process and for an analytical procedure is
illustrated in Figure 3. QbD in both areas starts
acceptance criterion for the qualification of the “the collection and evaluation of data, from the with a clear definition of the intended use of the
analytical procedure as shown in Figure 1. process design stage through commercial pro- output, or product, of the process. For produc-
duction, which establishes scientific evidence tion of a drug product, this considers items such
The Analytical Procedure as a Process that a process is capable of consistently deliver- as the route of administration, dosage form,
A process is a series of actions or steps taken ing quality products [2].” Treating an analytical bioavailability, strength, and stability. For ana-
in order to achieve a particular end or output. procedure as a process, the analogous defini- lytical procedures, the output/product is the re-
The output of an analytical procedure is a re- tion is “the collection and evaluation of data portable result, and a decision rule can be used
portable result. The series of steps begins with and knowledge from the method design stage to clearly and quantifiably state the intended use
of the result [15 - 16].
For an analytical procedure, the predefined
objective is the analytical target profile (ATP).
Criteria defined in the ATP refer to the quality
data attributes of the reportable result (i.e., bias
Figure 2 shows the analytical procedure as a process that starts with the taking of the laboratory sample and concludes with [accuracy] and target measurement uncertainty
the output of a reportable result. (TMU) [precision]), which includes all sources
For the production process, the critical
quality attributes are determined. The analogy
for the analytical procedure is the performance
characteristics required of the method. These
include the accuracy/bias, linearity, limit of
detection, repeatability, intermediate precision,
etc. The critical variables are identified and
characterized and a suitable control strategy is
designed. For production the production de-
sign space is defined, while for an analytical
procedure, the performance characteristics that
make up its design space are defined.
Decision Rules
Construction of a Decision Rule
Decision rule construction requires four com-
ponents: the measurement result, its measure-
ment uncertainty, the specification limit or lim-
its, and the acceptable level of the probability of
making each type of wrong decision [15 - 16].
The analytical quality by design approach uses
risk analysis, including assessment of probabil-
ities, to determine those four components of a
decision rule. The relationship between the four
components allows one to determine if an ana-
lytical procedure is fit for use and to set accep-
tance criteria that the analytical procedure must
meet.

Journal of Validation Technology Volume XX Issue 4

 Peer Reviewed: Analytical Procedure Journal of Validation Technology
Simple Decision Rule
For the drug substance potency example, both
the decision rule and the acceptance limits are
generally defined by the expectations of reg-
ulatory authorities, so the manufacturer may
not have much flexibility to use alternative ap-
proaches. A range of 98.0 to 102.0% is generally
expected for a commercial drug substance. This
range ensures that the drug product manufac-
tured from the DS will meet its potency specifi-
cation limits, which are generally 90 to 110% in
the US and 95 – 105% in the EU. This approach
is described by a simple decision rule in which
the specification zone, the values between and
including the specification limits, is the same as
the acceptance zone and defines the values for
which the product is accepted. The values out-
Figure 4 The figure illustrates a simple decision rule for a specification with both upper and lower limits. If the measurement
result lies in the acceptance or specification zone, product is accepted, otherwise it is rejected.
side the acceptance zone comprise the rejection
zone. This simple decision rule is illustrated in
Figure 4.
This simple decision rule can be stated as:
The product will be considered compliant if the
measurement result is within the acceptance
zone.
With this decision rule, the probability for
either type of error is a function of both the pro-
cess variability and the measurement uncertain-
ty (i.e., measurement variation). A type II error
occurs when the potency test result is within
specifications, when the true batch potency is
outside of specifications. In this example, the
type II error rate will be assumed to be negli-
gible, which means that the process delivers
consistent, high quality DS (process variation is
negligible). The DS is also assumed to be of high
purity, containing negligible water (non-hygro-
scopic), solvents, and inorganic content, which
means the true DS potency is approximately
100%. The probability of a type I error (rejection
of a batch when the true batch potency is within
specifications) is determined by the measure-
ment variation in combination with the width
of the acceptance zone. This probability is also
called the type I error rate.
The consequences for a type I error will de-
termine what probability is acceptable. Since
there is little flexibility to increase the accep-
tance zone, the measurement uncertainty is the
only parameter that can be adjusted in order to
meet the acceptable error rate.
Measurement Uncertainty
Measurement uncertainty is defined in the in-
ternational vocabulary of metrology (VIM) as a
“non-negative parameter characterizing the dis-
persion of the quantity values being attributed
to a measurand (the quantity being measured),
based on the information used” [17]. For an
analytical procedure, the uncertainty includes
all the random sources of variability, such as re-
peatability, intermediate precision and the un-
certainty in the estimates of the accuracy or bias.
As such, it can be viewed as a high level, overall
requirement for the total precision.
It is important to note that there are many
definitions of different types of precision based
on the sources of variability they include. Addi-
tionally, different industries, regulatory bodies,
and references define the types of precision
differently. As a result, it is best to clearly define
the precision terms being used by identify-
ing the sources of variability that are included.
The VIM takes this approach by including the
conditions of measurement in its definition of
precision:
“2.15 Measurement Precision
closeness of agreement between indications or
measured quantity values obtained by replicate
measurements on the same or similar objects un-
der specified conditions …
Note 2 The ‘specified conditions’ can be, for exam-
ple, repeatability conditions of measurement, in-
termediate precision conditions of measurement
or reproducibility conditions of measurement”
The VIM definition of precision is very sim-
ilar to the ICH definition in the analytical pro-
Figure 5 shows the normal distribution in relation to the specifications of 98.0% to 102.0%. The width of the normal distribu-
tion is chosen so that the probability the true result is below98.0% is 2.5% and above 102.0% is 2.5%.
cedure validation guideline [12]. Reproducibil-
ity is the broadest measure of method precision
as it includes all sources of variability in the es-
timate, including multiple testing laboratories.
The ICH uses the term intermediate precision
to distinguish the precision of the method with-
in a single lab from precision over multiple labs
(reproducibility). Normally the test for compli-
ance of a given batch with specification limits
is carried out in a single laboratory. Therefore,
the intermediate precision estimate is generally
the most appropriate precision estimate for the
assessment of method capability. In this work,
the term repeatability will be used to describe
short-term variability (i.e., repeat preparations
within a single run) and intermediate precision
to describe long-term variability within a single
lab (i.e., preparations analyzed over different
runs/setups). Note that the terminology associ-
ated with the use of the Gage R&R tool does not
make the distinction between within-lab and
inter-lab precision estimates, so here the term
“reproducibility” means the same thing as “in-
termediate precision.”
The measurement uncertainty characteriz-
es the dispersion of the values being attributed
to a measurand. This dispersion is assumed to
have a normal distribution. The basis for this as-
sumption is discussed in Annex G of the GUM
[18]. Even if the distribution is not normal, the
same approach will be used.
Uncertainty can be illustrated by imposing
www.ivtnetwork.com
 Jane Weitzel, Robert A. Forbes, Ronald D. Snee
a normal distribution over the decision rule as
shown in Figure 5. In this example, the true drug
substance potency is assumed to be 100.0%,
which establishes the center of the normal dis-
tribution. The width of the normal distribution
is determined by the standard deviation which
was set to 1% to coincide with a 2.5% probabili-
ty of obtaining values each above and below the
limits. As shown in Figure 5, 5% of the potency
results for the sample will fall outside the limits
due to measurement variation (type I error rate)
for the assumed drug substance potency and
precision described above. This distribution
only captures the measurement uncertainty and
does not include process variation (variation in
the true batch to batch potency), which was as-
sumed to be negligible.
If a 5% type I error rate is acceptable, the
standard deviation of 1.0% becomes the target
measurement uncertainty for the Analytical
Figure 6 (a) shows that if the normal distribution is centered over 99.0% the probability of making a wrong decision is in-
creased to 16.3%. (b) shows the target measurement uncertainty must be reduced to 0.61% in order that the probability of
making a wrong decision remains at <5%.
Journal of Validation Technology Volume XX Issue 4
Target Profile. Target measurement uncertainty
(TMU) is defined in the VIM as the “measure-
ment uncertainty specified as an upper limit
and decided on the basis of the intended use of
the measurement results” [19]. Any analytical
procedure that yields a reportable result with a
standard uncertainty of 1.0% or less, given there
is no bias, will be fit for its intended use.
The probability associated with the decision
rule is based on the theoretical normal distri-
bution and the specified standard deviation.
It does not take into account the variability in
the estimates of the uncertainty that will be
obtained in the qualification and verification
experiments. That is dealt with in the experi-
mental designs of the development and qualifi-
cation experiments, and the desired confidence
in their resulting precision estimates. Generally,
higher confidence limits will require more ex-
perimental runs to provide better estimates of
method precision to avoid failing to meet the
TMU. Larger studies with more runs are more
costly in terms of time and resources, so one
must weigh the benefit of greater confidence in
the precision estimate, versus the additional cost
of larger studies.
Relationship Between Accuracy and Target Mea-
surement Uncertainty
The probability of not meeting the specifica-
tion depends on where the true distribution is
centered. If the true distribution is not centered
at 100.0%, as in the above example, then a dif-
ferent proportion of the measured results are
expected to be outside the specifications. When
this occurs, the target measurement uncertain-
ty must be improved in order to maintain the
same type I error rate. However, the analytical
procedure could have a bias that cannot be
removed in method development. If it is not
corrected for, then the TMU will have to be ad-
justed. Or if the drug substance does not have a
potency of 100.0%, this will impact the degree
of precision that the method will need to attain.
For example, if the DS has an actual potency
value of 99.0%, then the TMU must be reduced
to 0.6% (Figure 6).
Analytical Target Profile
Based on a risk analysis that includes consider-
ations such as the cost of OOS investigations,
the known characteristics of the product (the
drug substance is known to have few impuri-
ties and the potency is close to 100%), and the
capability of the analytical technique (HPLC) a
company can decide the probability it is willing
to accept associated with making an incorrect
decision (type I error) concerning a product
specification. In the below example, the proba-
bility selected is 5%.
The decision rule can be more detailed now:
The batch of drug substance will be considered
compliant if the test result is within the range of
98.0 to 102.0%. The probability that an accept-
able batch will fall outside this range must be
NMT 5%.
The analytical target profile can now be de-
termined. The laboratory qualifies the analytical
procedure for a concentration range wider than
the specification to allow for variability in sam-
ple concentration. The matrix and measurand
are defined clearly. There is only one known
impurity, Impurity A, and it occurs below 0.1%.
The ATP can now be written:
The analytical procedure must be able to quan-
tify the drug substance in the presence of impu-
rity A, and potential degradation products, over
a range of 80% to 120% of the nominal concen-
tration with an accuracy and uncertainty so that
the reportable result falls within ±2% of the true
value with at least a 95% probability.
Performance Characteristics for the
Analytical Procedure
The ATP can now be used to determine the
performance characteristics for the analytical
procedure. The following performance char-
acteristics need to be included in the method
qualification: accuracy, precision, specificity,
linearity, and range.
The acceptance criteria for specificity (i.e.,
resolution of the main peak from impurity A
and potential degradation products) and linear-
ity are determined based on their impact on the
accuracy and precision. As long as their impact
on the accuracy and precision is such that the
target measurement uncertainty can be met,
they are acceptable. This is a key advantage of
this approach; it defines the performance that is
“good enough.” One can confidently accept the
performance because it is directly linked to the
intended use.
The performance characteristics and the ac-
ceptance criteria are summarized as follows:
The target uncertainty is NMT 1.0%. The com-
bination of bias and uncertainty need to be such
 Peer Reviewed: Analytical Procedure Journal of Validation Technology
that the probability of not meeting the specifica-
tion is NMT 5%.
To assess the accuracy of the method when
performed in a new laboratory, a reference stan-
dard, or a previously tested sample with a de-
fined potency could be used. Results obtained
in the new laboratory would be compared to
the defined result to test for a bias. If a bias is
detected, the combination of the uncertainty
of the bias with the intermediate precision es-
timate would be compared to the allowable
measurement uncertainty defined by the ATP.
For this example, the bias (accuracy) is assumed
to be negligible so that the precision component
can be focused on. If the bias were significant,
then the analytical procedure should be im-
proved to remove that bias, or the bias should
be corrected for and the uncertainty of that cor-
rection included in the uncertainty estimate.
Now that the ATP has been established, its
use in lifecycle management of the analytical
method will be demonstrated.
Procedure Performance Qualification
During the validation of the analytical meth-
od, the precision of the method is evaluated
by studies to determine its repeatability, and its
intermediate precision. Because it includes both
long-term and short-term sources of variation,
the intermediate precision estimate is consid-
ered a representative measure of the capability
of the method for ongoing batch release test-
ing. Generally performed as one component
of a method validation protocol, a statistically
designed intermediate precision study is car-
ried out using multiple setups, incorporating
multiple analysts, multiple HPLC instruments,
and HPLC columns with different packing
batches. In order to meet the ATP, the accep-
tance criterion for the study is that the overall
variability, the combined uncertainty, must be
NMT 1.0% as a standard uncertainty or stan-
dard deviation. Traditionally, the qualification
protocol includes a criterion only for the SD
estimate obtained. However, the uncertainty in
the precision estimate obtained in the qualifi-
cation study should also be taken into account.
Generally a confidence interval (CI), or upper
confidence limit (e.g., 95% UCL), is calculated
for the standard deviation estimate to provide
an upper limit below which the actual standard
deviation is likely to fall. The width of this inter-
val is impacted by both the desired confidence
for the estimate and the number of setups in the
study. For more detail on the calculation of the
confidence interval, see the stimuli article in the
USP Pharmocopoial Forum [20].
The design and results of one such study are
shown in Table 1, where four analysts produced
duplicate potency results for one batch on each
of 8 setups using a different HPLC instrument
and column on each setup. For this study we
will choose 80% as the desired confidence for
the precision estimate. This means that if we
obtain a standard deviation of 1.0%, we will
accept a 20% chance that the actual standard
deviation may be greater. This also means that
the acceptance limit for the standard deviation
is effectively NMT 0.74%, which corresponds to
an 80% UCL of 1.0% for an 8-run study. Since
the method prescribes averaging the results for
two sample preparations on each setup, the re-
portable result for a batch is the average (AVG).
The overall standard deviation (SD) obtained
Table 1, Results of Intermediate Precision Study for DS Po-
tency Method.
in this study was 0.15%, with an 80% UCL of
0.20%, which met the acceptance criteria of
NMT 1.0% SD.
A limitation of this study was the fact that
only a single DS batch was tested, so potential
variability caused by differences in behavior of
different batches of the DS was not included. In
the case of a high-purity DS, with very consis-
tent properties from batch to batch, this source
of variation is generally small but this is not al-
ways the case.
Procedure Performance Verification
To further illustrate the utility of the ATP, its use
in the next stage of the lifecycle of the potency
method will be considered. This is the stage
in which the method is installed in the release
testing quality control laboratory (QCL). At this
stage, the method has been fully developed and
validated to demonstrate its appropriateness for
its intended use, and some history of its perfor-
mance to test manufactured DS batches at a de-
velopment scale exists. The DS is a new chemi-
cal entity that has been submitted for regulatory
approval and the manufacturing process is
being installed and validated in the long-term
manufacturing facility. The analytical method
is being transferred to the long-term release lab,
and is undergoing performance qualification at
the new site. Some data to characterize both the
capability of the method and the capability of
the manufacturing process are available.
Installation of the DS potency method in the
QCL is described as a Type 3 change in the USP
stimuli article, involving the need to operate
the method in a different environment [1]. To
support this activity, a procedure performance
qualification (PPQ) protocol would be written
to describe the analytical method validation or
comparison activities needed to demonstrate
that the method meets its performance re-
quirements as required by the ATP in the new
environment. These activities are governed by
current guidance around method transfers [2,
9]. From the perspective of a lifecycle approach,
a key element of the transfer activity is to show
that the precision of the method as performed
in the QCL is sufficient to meet the required un-
certainty as described in the ATP. The required
precision would be included in the PPQ proto-
col as an acceptance criterion for the intermedi-
ate precision study; in our example the accep-
tance criteria would be NMT 1.0%RSD.
One approach to evaluate the method preci-
sion after transfer to the QCL would be to per-
form an intermediate precision study similar to
the study performed for method qualification.
An alternative approach is to use the Gage R&R
tool which measures the repeatability and re-
producibility of the method [21]. While the
Gage R&R study may be new to many in the
pharmaceutical and biotech worlds, the method
has been used widely in the process industries
including auto, chemical and electronics. Gage
R&R studies are also widely used in Six Sigma
process improvement studies [22]. It is not un-
common to find process problems are due to
poor measurement systems with the problem
significantly reduced, if not eliminated, by im-
proving the measurement system. Note that for
the Gage R&R study (as described under Mea-
surement Uncertainty above) we use the term
reproducibility to mean the same thing as the
intermediate precision, and not inter-laborato-
ry variation as defined by the ICH. Repeatabil-
ity is the within setup precision, reproducibility
comprises precision across multiple setups. In
a Gage R&R Study 5-10 samples are evaluated
by 2-4 analysts (reproducibility) using 2-4 re-
peat tests (repeatability) sometimes involving
2-4 test instruments (reproducibility). For ex-
ample if 3 analysts measure each of 10 samples
of a product in duplicate the study will produce
3x10x2=60 test results.
A variance components analysis is then
performed on the measurements to obtain
estimates of the repeatability (within-setup
variability) and reproducibility of the method.
These statistics are then used to evaluate the fit-
ness of the method for use for product release
www.ivtnetwork.com
 Jane Weitzel, Robert A. Forbes, Ronald D. Snee
and to support process improvement efforts.
The variance component estimates are of-
ten used to determine how many within-run
replicates and/or setups are required to give the
desired method intermediate precision. If the
within-run variability (repeatability) is high,
then more within-run replicates maybe re-
quired to generate a reportable result with the
desired precision level. If setup-to-setup vari-
ability (reproducibility) is high, then multiple
setups including one or more analysts, instru-
ments, and columns maybe required to gener-
ate a reportable result with the desired preci-
sion level. The reportable result consists of the
average of multiple within run and/or setups as
prescribed in the method and is the value that is
compared to the specification and applied to the
decision rule. This kind of a replication strategy
can be employed to reduce the variability test
method that currently has a higher than desired
level of variability to continue to release product
until a better test method can be developed. In
this way production doesn’t have to be stopped
(or not initiated) because of a highly variable
test method.
To illustrate the utility of this approach for
verification of the method transfer to the new
laboratory, a Gage R&R study was performed.
Table 2. Gage R&R Study Results for DS Potency Method Transfer.
Journal of Validation Technology Volume XX Issue 4
Samples from five API batches were tested on
four setups by two analysts using each of two
instruments (each instrument used a unique
column for total to two columns). New mobile
phase was prepared for each setup and each
analyst prepared a fresh set of standards for
each run. Samples were prepared and analyzed
in duplicate on each run, producing a total of
5x2x2x2 = 40 Potency(%) test results. The data
are summarized in Table 2.
The results of the variance components
analysis for the potency measurements are
summarized in Table 3 and shown in Figure
7. In this study, multiple analysts, instruments
and columns were used to obtain more realistic
variability estimates, and could not be separate- Figure 7. Variability Chart showing the results of the Gage R&R study to verify performance of the method in a new labo-
ly evaluated for statistically significant effects. ratory.
The variance components show that 84% of
variability is due to the setup and the remaining
16% due to repeatability (within-setup).
The standard deviation of the test method
(intermediate precision for two preparations on
a single setup) is 0.71% with an upper 80% con- Table 3. Gage R&R Study Repeatability and Reproducibility Variance Components
fidence limit of1.16%. The intermediate preci- proach incorporating a confidence limit is used. ber of setups and within setup replicates.
sion estimate of 0.71% meets the acceptance The method may not be good enough for its Table 4 demonstrates that increasing the
criterion of NMT 1.0% that was defined based intended use. To address this, additional set- number of within setup measurements (re-
upon the ATP. However, the upper confidence ups could be performed to obtain a better SD peatability) from one to two has little effect on
limit exceeds 1.0%, indicating that we cannot estimate with a tighter CI. For example, if the the method intermediate precision. This is be-
say with 80% confi- results for all the batches are pooled, with the cause the within setup (repeatability) variance
dence that the standard assumption that there are no significant differ- is much smaller than the setup-to-setup vari-
deviation is not NMT ences between them, a standard deviation of ance (reproducibility). Generally, the standard
1.0%. If the verification 0.63% with an 80% UCL of 1.03% (rounds to practice for a DS potency method is to prepare
protocol utilized an 1.0%) is obtained, which meets the acceptance samples in duplicate. While the decrease in
acceptance criterion of criterion with an 80% confidence level. variation for this method is marginal, the major
NMT 1.0% for the 80% If the method still did not meet the accep- benefit of the two preparations is to control for
UCL of the SD, these tance criterion with additional runs, the test preparation errors by setting an acceptance cri-
results would not pass method variation can be reduced by employing terion on their difference. Since reproducibility
the criterion. This study a replication strategy according to Equation 1. was the larger component of variance, analyzing
provides an interest- Table 4 shows the effect of increasing the num- the samples on multiple setups would be more
ing example where the effective at reducing the method
method would pass a standard deviation, than preparing
traditional assessment, more samples for the same setup.
but does not pass when By including multiple setups (n=2)
a more rigorous ap- the method intermediate precision
could be reduced to approximately
 Peer Reviewed: Analytical Procedure Journal of Validation Technology
Table 4. Method Standard Deviation versus Replication Strategy.
0.5%, showing that this would be an effective
way to improve the intermediate precision if
necessary to meet the ATP requirements. These
results illustrate a key benefit of the R&R study
to understand the variance components. Be-
cause it is easier, some laboratories may be pre-
paring replicates at the repeatability level thus
wasting resources and giving a false source of
assurance that precision has been improved.
The estimate of the standard deviation ob-
tained in the QCL (0.63%, 80% UCL = 1.0%,
pooling the batches) was larger than the stan-
dard deviation estimate obtained during meth-
od validation (0.15%, 80% UCL = 0.20%).
However, the 80% UCL for both estimates met
the acceptance criterion of NMT 1.0% from the
ATP, demonstrating the suitability of the meth-
od for its intended use. There may be many
reasons for the apparent difference in method
performance between the two studies. For ex-
ample, the analysts performing the first valida-
tion study may have been more experienced in
performing the method.
Continued Procedure Performance
Verification
Following a successful installation of the meth-
od in the QCL, the method moves into stage 3
of its lifecycle where ongoing procedure perfor-
mance verification (PPV) is required [1]. This
is analogous to the requirement for Continued
Process Verification called out in the FDA Pro-
cess Validation Guidance, and is appropriate
since we have argued that the execution of a test
method is a process [2]. The lifecycle approach
requires one to assess the repeatability and in-
termediate precision of the test method as it
is used throughout the life of the method, not
simply as a once-and-done validation, or site
certification, exercise.
Continued performance verification is ac-
complished in part by monitoring the results of
the internal method system suitability require-
ments. System suitability requirements ensure
that the method has been set up appropriately
and that the equipment is functioning cor-
rectly. Peak shape (tailing or asymmetry) and
resolution requirements ensure the peak can
be precisely integrated, appropriate selectivity
is provided by the HPLC column, and that the
mobile phase has been prepared properly. Injec-
tion precision ensures that the sample injection
system is functioning properly, and surfaces of
internal parts have not become scratched and
worn. Precision requirements for duplicate
standard and sample preparations ensure that
the analyst has made the preparations with ad-
equate care. All these internal method require-
ments work together to ensure the measuring
system continues to be capable to meet the ATP
uncertainty requirements. System suitability re-
sults should be monitored and control charted
to maintain a state of PPV, and to provide an
early warning of drift or changes in the method
capability.
An effective way to assess the long-term sta-
bility of a test method is to periodically submit
“blind control samples” (also referred to as ref-
erence samples, or method performance sam-
ples), from a common source, for analysis along
with routine production samples in a way that
the analyst cannot determine the difference be-
tween the production samples and the control
samples. Nunnally and McConnell conclude
“…there is no better way to understand the true
variability of the analytical method” [23]. The
control sample must be homogeneous and sta-
ble in order to avoid out-of-control signals un-
related to the measurement system. The control
samples are typically tested 2-3 times (depend-
ing on the test method) at a given point in time.
The sample averages are plotted on a control
chart to evaluate the stability (intermediate pre-
cision) of the method. The standard deviations
of the sample replicate preparations are plotted
on a control chart to assess the stability of the
repeatability of the test method [21].
When using control charts typically two
types of non-random patterns are observed:
1. Test measurements outside of the con-
trol limits (typically set at the process
average ±3 standard deviations). Such
events are referred to as out-of-control
(OOC) signals.
2. Non-random patterns inside the con-
trol limits; such as trends, drifts, and
shifts. These events are referred to as
out-of-trend (OOT) signals and are de-
tected using the Western Electric rules
[21].
Both OOC and OOT signals are based on
statistical tests of significance and are included
in many statistical software packages [24].
Continued Procedure Performance Verification
by Control Charting Example
The data shown in Figures 8 - 10 are from an
evaluation of a DS potency test method over
a six-year period. Periodically blind control
samples were submitted to the lab for analysis.
Figure 8 is an I-MR (Individual Moving Range)
control chart. The top control chart is a plot of
the averages of the duplicate analyses of each
sample, and the lower control chart shows the
moving range of the averages. Control limits
were calculated from the MR chart and thereby
include both the setup-to-setup variation as well
as the within setup (repeatability) variation. The
standard deviation estimate calculated from the
MR chart was 0.48, and is an estimate for the
intermediate precision of the method for this
six-year period.
The MR chart in Figure 8 (bottom) shows
only one point above the control limit. The indi-
viduals chart in Figure 8 (top) shows two points
outside the control limits. No trends or shifts
were observed over the 48 samples. However,
after sample 23 the method intermediate preci-
sion was observed to improve. After further in-
vestigation, samples 1-23 were found to have all
been analyzed by analyst A, and the majority of
the samples afterward were tested by other ana-
lysts. Analyst technique appears to be the cause
for the observed decrease in method variability
following sample 23. Potential instrument and
column impacts on the results were also inves-
tigated but determined to have little to no effect
on the observed decrease in the method preci-
sion.
When the data are split into two sets: Stage
1 (Samples 1-23--Figure 9) and Stage 2 (Sam-
ples 24-48--Figure 10) no practical differences
between the average values of the two stages
were noted. But as mentioned above, the meth-
od intermediate precision is greater in the first
stage than in the second stage likely due to the
practices of analyst A. The stage 2 data (Figure
10) showed one sample average and one MR
outside of control limits.
Another important use of the control sam-
ple results is to verify that the method meets the
requirements of the ATP during this period of
time. The standard deviation of all the report-
able results for the control sample (sample av-
erages) generated during the six-year period
was found to be 0.62% (80% UCL = 0.83%),
which meets the ATP criterion of NMT 1.0%.
Furthermore, this estimate of the method inter-
mediate precision is based on a large number
of independent method setups consisting of
multiple analysts, instruments, and columns.
This estimate of the intermediate precision is
also consistent with the estimate from the Gage
R&R study (0.71%).
www.ivtnetwork.com
 Jane Weitzel, Robert A. Forbes, Ronald D. Snee
Figure 8. I-MR Control Charts for Sample Averages (n=2): All Samples
Figure 9. I-MR Control Charts for Sample Averages (n=2): Samples 1-23
Figure 10. I-MR Control Charts for Sample Averages (n=2): Samples 24-48
Journal of Validation Technology Volume XX Issue 4
Recommendations for Interpreting the
Control Charts
When all the results are within the control limits
and no OOT patterns are observed the meth-
od is performing as expected. However, over
the course of time OOC and/or OOT may be
observed. The course of action will depend on
the specific situation but some general guidance
can be given. The following recommendations
are similar to those regarding process monitor-
ing when using control charts.
First when out-of-control (OOC) or out-of-
trend (OOT) signals are obtained the data
collection, analysis, and control charting proce-
dures are examined to ensure that no mistakes
have occurred. If mistakes are found the data
should be corrected and the analysis repeated.
When mistakes are ruled out then an inves-
tigation should be considered to find the cause
of the OOC or OOT signals. The investigation
will likely examine if that signal can be traced to
a specific change in materials, reagents, instru-
ments, columns, reference standards, or ana-
lysts. The action taken will then depend on the
cause and the specific situation.
However, sometimes the cause of the OOC
or OOT cannot be determined. If this is the case,
the practical significance of the signal should be
considered. For example, the associated results
may be close to the control chart limits and iso-
lated with no OOC or OOT points before or af-
ter those results. The process may be perform-
ing at a very high level relative to specifications,
suggesting that measurement variation may not
be a problem.
In the case when the variation has changed
– increased or decreased – revised limits should
be considered along with the appropriate ap-
provals and change control actions taken. Such
changes should be accompanied by appropriate
training of the analyst and supervision involved.
A control plan should also be created and put
in place to ensure that the new procedures are
followed.
If unacceptable variation continues to be
observed, it may be appropriate to consider
performing a robustness evaluation of the test
method as discussed above. A test method
that is not robust can be expected to exhibit in-
creased repeatability, intermediate precision, or
both over time.
Product Quality Review – Summarize
Test Method Performance
The performance of the test method should be
reviewed periodically in accordance with the
reviews for the process and product. As part of
that review, the results of the test method con-
tinued process verification system would be
provided. The control charts of the test results
for the control samples and summary statistics
which document the test method performance
including the repeatability and intermediate
precision standard deviations and relative stan-
dard deviations, results of recent calibration
studies and other information depending on
the test method and situation should be includ-
ed in the summary. The performance summary
documentation should be stored and dissemi-
nated in accordance with the knowledge man-
agement expectations of the ICH [25].
Conclusion
In this paper, we have illustrated the application
of the lifecycle management approach for an
analytical procedure (a drug substance potency
test) and showed how a decision rule was used
to develop the analytical target profile (ATP).
The ATP was then used to define the fitness of
the method for its intended use, which in turn
enables an objective assessment of the method
performance during initial validation, transfer
to the quality control laboratory (QCL), and its
ongoing performance in the QCL.
The lifecycle approach is essentially applying
Quality by Design (QbD) principles to the exe-
cution of the method to produce a result, in an
analogous way that QbD is applied to a manu-
facturing process producing a product. We have
also demonstrated the use of the Gage R&R
 Peer Reviewed: Analytical Procedure Journal of Validation Technology
study approach to determine the repeatability
and reproducibility (intermediate precision) of
the test method within the QCL after transfer,
as a means to verify its performance, versus the
ATP criteria has also been demonstrated. The
estimate of the method precision from the QCL
results will often be based on a larger sample
size and encompass more variability than the
development studies did, leading to a more ro-
bust estimate of the long-term measurement
precision. The Gage R&R approach utilizes a
greater number of test batches than a typical
intermediate precision study, which provides a
higher degree of replication for assessment of
the measurement uncertainty.
The use of a control sample for continued
procedure performance verification has also
been explained, which is an important stage
in the lifecycle of a method that is often over-
looked. The use of an analytical method control
sample to monitor performance of the method
versus the ATP requirements during its long-
term use in the QCL has also been demonstrat-
ed. Not only does the control sample provide
assurance that the ATP is being met, but it also
detects special cause variation and/or shifts in
common cause variation. This enables continu-
al improvement of the method performance by
identifying opportunities for analyst training,
the need for equipment repairs, or detrimental
changes in consumables such as the HPLC col-
umns or reagents.
References
1. USP Stimuli Article, “Lifecycle Manage-
ment of Analytical Procedures: Method
Development, Procedure Performance
Qualification, and Procedure Performance
Verification,” Pharmacopeial Forum 39
(6), online, www.usp.org/usp-nf/notices/
stimuli-article-lifecycle-management-an-
alytical-procedures-posted-comment, ac-
cessed December 3, 2014.
2. FDA, Guidance for Industry Process Val-
idation: General Principles and Practices
(Rockville, MD, January 2011).
3. FDA, Guidance for Industry Analyti-
cal Procedures and Methods Validation
for Drugs and Biologics (Rockville, MD,
Draft--February 2014), online, www.fda.
gov/downloads/drugs/guidancecompli-
anceregulatoryinformation/guidances/
ucm386366.pdf, accessed December 3,
2014.
4. P. Nethercote, et al., “QbD for Better Meth-
od Validation and Transfer,” PharmaMan-
ufacturing.com, online, www.phar-
mamanufacturing.com/articles/2010/060.
html, April 13, 2010, accessed December
3, 2014.
5. M. Schweitzer, et al., “Implications and
Opportunities of Applying QbD Princi-
ples to Analytical Measurements,” Pharm.
Tech. 34 (2), 52–59, 2010.
6. Phil Nethercote, Joachim Ermer, “Qual-
ity by Design for Analytical Methods,
Implications for Method Validation and
Transfer,” PharmTech.com, online, www.
pharmtech.com/pharmtech/Feature+Ar-
ticles/Quality-by-Design-for-Analyti-
cal-Methods-Implicati/ArticleStandard/
Article/detail/791903, October 2012, ac-
cessed December 3, 2014.
7. Joachim Ermer, “A Lifecycle Concept for
Pharmaceutical Analysis,” European Phar-
maceutical Review, 16 (3), 16-24, 2011, on-
line, content.yudu.com/Library/A1sipk/
PATampQbDSupplement/resources/in-
dex.htm?referrerUrl=http%3A%2F%2F-
w w w. y u du. c om % 2 F ite m % 2 Fd e -
tails%2F345094%2FPAT---QbD-Supple-
ment accessed December 3, 2014.
8. ICH, Q8(R2), Pharmaceutical Develop-
ment – Part II: Pharmaceutical Devel-
opment - Annex, Step 4 version (August
2009).
9. USP 37-NF 32 General Chapter <1224>,
“Transfer of Analytical Procedures,” 1155-
1157.
10. USP 37-NF 32 General Chapter <1225>,
‘Validation of Compendial Procedures,”
1157-1162.
11. USP 37-NF 32 General Chapter <1226>,
‘Verification of Compendial Procedures,”
1162-1163.
12. ICH, Q2(R1), Validation of Analytical
Procedures: Text and Methodology, Step 4
version (November 2005).
13. ICH, Q11, Development and Manufacture
of Drug Substances (Chemical Entities
and Biotechnological/Biological Entities),
Step 4 version (May 1, 2012).
14. FDA, Guidance for Industry Investigating
Out-of-Specification (OOS) Test Results
for Pharmaceutical Production (Rockville,
MD, October 2006).
15. ASME, Guidelines for decision rules: con-
sidering measurement uncertainty in de-
termining conformance to specifications,”
B89.7.3.1-2001.
16. Eurachem/CITAC Guide, Use of uncer-
tainty information in compliance assess-
ment, First edition (2007), online, eura-
chem.org/images/stories/Guides/pdf/
Interpretation_with_expanded_uncer-
tainty_2007_v1w.pdf, accessed December
3, 2014.
17. JCGM, International vocabulary of me-
trology – Basic and general concepts
and associated terms (VIM), 3rd ed.,
JCGM 200:2012(E/F), online, www.bipm.
org/utils/common/documents/jcgm/
JCGM_200_2012.pdf, accessed Decem-
ber 3,2014.
18. ISO/IEC, Uncertainty of measurement
– Part 3: Guide to the expression of un-
certainty in measurement (GUM:1995),
98-3:2008.
19. ICH, Q2(R1), Validation of Analytical
Procedures: Text and Methodology, Step 4
version (November 2005).
20. USP Proposed General Chapter <1210>,
“Statistical Tools for Procedure Validation,
”Pharmacopeial Forum 40 (5), 1–33.
21. D. C. Montgomery, Introduction to Sta-
tistical Quality Control, John Wiley and
Sons, New York, NY, 7th Edition, 2013.
22. [22] R. D. Snee and R. W. Hoerl, Leading
Six Sigma – A Step-by-Step Guide Based
on Experience with GE and Other Six Sig-
ma Companies, FT Prentice Hall, Upper
Saddle River, NJ, 2003.
23. B. K. Nunnally and J. S. McConnell, Six
Sigma in the Pharmaceutical Industry:
Understanding, Reducing, and Con-
trolling Variation in Pharmaceuticals and
Biologics, CRC Press, Boca Raton, FL,
2007.
24. L. D. Torbeck, “OOS, OOT, OOC and
OOSC,” Pharmaceutical Technology, 35
(10), 46-47, 2011.
25. ICH, Q10, Pharmaceutical Quality Sys-
tem, Step 4 version (June 2008).
www.ivtnetwork.com
Jane Weitzel, Robert A. Forbes, Ronald D. Snee

About the Authors

Jane Weitzel has been working in analytical chem-
istry for over 35 years in highly regulated, fast
paced companies with the last 5 years at the direc-
tor/associate director level. She is currently a full
time consultant and auditor.

Robert Forbes earned his PhD from Purdue Uni-

versity working with Dr. Ben Freiser in Mass Spec-
trometry. He worked as a research scientiest with
Eli Lilly working with Clinton Laboratories, Tippe-
canoe Laboratories, and Lilly's Technology Center.

Ronald D. Snee is founder and president of Snee

Associates, a firm dedicated to the successful
implementation of process and organizational
improvement initiatives. He provides guidance
to senior executives in their pursuit of improved
business performance. He worked at the DuPont
Company for 24 years prior to initiating his con-
sulting career.

Journal of Validation Technology Volume XX Issue 4

View publication stats