Professional Documents
Culture Documents
Develop A Quality by Design Approach For Analytical Method Development
Develop A Quality by Design Approach For Analytical Method Development
Develop A Quality by Design Approach For Analytical Method Development
Jane Weitzel
mljweitzel@msn.com
8:30 – 10:00
Wednesday, Dec. 9
SESSION 1 – ANALYTICAL
PROCEDURES AND METHOD
VALIDATION
mljweitzel@msn.com 2
Jane Weitzel Biosketch
Jane Weitzel has been working in analytical chemistry for
over 35 years for mining and pharmaceutical companies with
the last 5 years at the director/associate director level. She
is currently a consultant, auditor, and trainer. Jane has
applied Quality Systems and statistical techniques, including
the estimation and use of measurement uncertainty, in a
wide variety of technical and scientific businesses. She has
obtained the American Society for Quality Certification for
both Quality Engineer and Quality Manager.
In 2014 she was pointed to the Chinese National Drug
Reference Standards Committee and attended their
inaugural meeting in Beijing
For the 2015 – 2020 cycle, Jane is a member of the USP
Statistics Expert Committee and Expert Panel on Method
Validation and Verification.
mljweitzel@msn.com 3
Topics
Develop a Quality by
Design (QbD) Approach
for Analytical Method
Development
Stage 1 in the
Lifecycle of an
Analytical
Procedure - Method
Development
Using Design of
Experiments and
ANOVA
Estimating and
Using Measurement
Uncertainty
mljweitzel@msn.com 4
Based on Approach
http://www.friesenpress.com/book
store/title/119734000004601536
mljweitzel@msn.com 5
Talks at Conference
mljweitzel@msn.com 6
Lifecycle of Analytical Procedure
Stage 1 Stage 3
Stage 2
Design Continued
Performance
Development & Performance
Qualification
Understanding Verification
mljweitzel@msn.com 7
Justification for Approach
mljweitzel@msn.com 8
FDA Guidance
http://www.fda.gov/downloads/drugs/guidancecompliancere
gulatoryinformation/guidances/ucm386366.pdf
mljweitzel@msn.com 9
Sound Science
Metrological approach
to measurements
Measurement
uncertainty
Target measurement
uncertainty
Completely
characterises the
variability
http://www.fda.gov/ScienceResearch/FieldScience/LaboratoryManual/ucm171878.htm
mljweitzel@msn.com 10
Scientifically Sound
and Appropriate
mljweitzel@msn.com 11
21 CFR 211.165 (d)
mljweitzel@msn.com 12
Topics
Develop a Quality by
Design (QbD) Approach
for Analytical Method
Development
Stage 1 in the
Lifecycle of an
Analytical Procedure
- Method
Development
Using Design of
Experiments and
ANOVA
Estimating and
Using Measurement
Uncertainty
mljweitzel@msn.com 13
USP Website
USP-NF/Notices Article
mljweitzel@msn.com 14
Lifecycle Management of an Analytical
Procedure
The Stimuli article discusses
how the modern concept of lifecycle can be
applied to analytical procedures.
In December 8 &9, 2014, the approach was
further explained and the required statistical tools
were presented at a USP workshop.
Series of articles will be published to further
explain these concepts with worked examples.
mljweitzel@msn.com 15
Analytical Target Profile (ATP)
mljweitzel@msn.com 16
Wording for ATP
Assay
The procedure must be able to quantify the
analyte in presence of (X, Y, Z) over a range
of A% to B% of the nominal concentration
with an accuracy and uncertainty so that the
reportable result falls within ±C% of the true
value with at least P% probability.
The variables in orange are specific for each
reportable result.
mljweitzel@msn.com 17
LET’S LOOK AT THESE
TERMS
mljweitzel@msn.com 18
Accuracy and uncertainty are defined
mljweitzel@msn.com 19
Accuracy
a) Error b) Error
Improving Trueness Bias
Trueness
mljweitzel@msn.com 21
Target Measurement Uncertainty
2.34
target measurement
uncertainty
target uncertainty
Measurement
uncertainty specified
as an upper limit and
decided on the basis
of the intended use of
http://www.bipm.org/en/publications/
measurement results guides/vim.html
mljweitzel@msn.com 22
TMU and ATP
mljweitzel@msn.com 23
Terminology
http://www.rsc.org/Membership/Networking/InterestGr
oups/Analytical/AMC/TechnicalBriefs.asp
mljweitzel@msn.com 24
Probability
mljweitzel@msn.com 25
Basic Concept - Probability
mljweitzel@msn.com 26
Statistical usage
probability
a real number in the scale 0 to 1 attached to a
random event
NOTE It can be related to a long-run relative
frequency of occurrence or to a degree of belief that
an event will occur.
For a high degree of belief, the probability is
near 1.
[ISO 3534-1:1993, definition 1.1]
mljweitzel@msn.com 27
Probability
mljweitzel@msn.com 28
Example - Do you release the lot?
mljweitzel@msn.com 29
Example – Release of a Lot
mljweitzel@msn.com 30
Picture
Probability
93.7
mljweitzel@msn.com 31
More Details - Probability
Nominal Concentration (Central Value) 93.7
Target Measurement Uncertainty (Standard Deviation) 1.00 Enter the largest standard deviation which results in the acceptable
Lower Limit 90.0 "Total outside limits".
Upper Limit 110.0
Calculates % below Lower Limit 0.011% % above upper limit 0.00% Total outside limits 0.0%
Probability
Lower Limit Upper Limit
Uses
EXCEL Concentration
mljweitzel@msn.com 32
Normal Distribution Curve
Y axis can
Density Function
mljweitzel@msn.com 33
Normal Distribution Curve
Y axis can
Density Function
µ
area under the curve between
two distinct points defines the
probability for that interval
mljweitzel@msn.com 34
% of population between z values
Percentage of Results Between z Factors for a Normal Distribution
99.7%
95%
68%
-4 -3 -2 -1 0 1 2 3 4
z
(Standardized Normal Variable)
For example, the probability for interval between z=1 and z=-1 is 68%
mljweitzel@msn.com 35
Probability
mljweitzel@msn.com 36
Established Concepts
mljweitzel@msn.com 37
The USER Develops Decision Rule
mljweitzel@msn.com 39
From where does information come?
Could be Arbitrary
Assign the values based on expert judgement,
past experience, regulatory requirement/request
The problem is that this assignment of values is
not based on the intended use.
mljweitzel@msn.com 40
Information can come from aQbD
mljweitzel@msn.com 41
Cyclical Process – May have to revisit
the Decision Rule
Target
Measurement
Uncertainty
Analytical
Procedure
Performance
Characteristics
mljweitzel@msn.com 42
Decision Rules – References
mljweitzel@msn.com 43
Decision Rule
mljweitzel@msn.com 44
Decision Rule – Acceptance or
Rejection
Decision rules give a
prescription for the
acceptance or rejection of
a product based on the
measurement result, its
uncertainty and the
specification limit or limits,
taking into account the
acceptable level of the
probability of making a
wrong decision.
mljweitzel@msn.com 45
Why Use Decision Rules
mljweitzel@msn.com
46
Decision Rule
To decide whether
a result indicates
compliance or
non-compliance
with a Upper
specification, it is Limit
necessary to take
into account the
measurement
uncertainty.
1 2 3 4
Result is Result above Result is Result is
above the the limit. below the below the
limit. Limit is within limit. limit.
Limit is below the expanded Limit is within Limit is above
expanded uncertainty. the expanded expanded
uncertainty. uncertainty. uncertainty.
mljweitzel@msn.com 47
Decision Rules Require 4 Components
mljweitzel@msn.com 49
Types of Decision Rules - Simple
Simple
Acceptance and Lower
Limit
Upper
Limit
Rejection
Product Rejection Zone Specification Zone Rejection Zone
mljweitzel@msn.com 50
50% Probability
If a result
is
obtained
% below Lower Limit 50.00% % above upper limit 0.00% Total outside limits 50.0% at or near
Lower Limit Upper Limit
limit the
probability
of true
value
being
90 92 94 96 98 100 102 104 106 108 110 outside
Concentration
the limit is
50%
mljweitzel@msn.com 51
Add guard band
(sometimes called internal release limit)
% below Lower Limit 2.28% % above upper limit 0.00% Total outside limits 2.3%
The guard
band is Guard Band
determined
using the
acceptable
probability
of making a
wrong
90 92 94 96 98 100 102 104 106 108 110
decision.
Concentration
mljweitzel@msn.com 52
For the Statistical Approach
mljweitzel@msn.com 53
SELECT A TECHNOLOGY OR
METHOD
mljweitzel@msn.com 54
Based on Reportable Result
Requirements
Based on TMU & probability
Select suitable technology
E.g. use ELISA screening method or ICP-MS-MS
mljweitzel@msn.com 55
Knowledge Management
mljweitzel@msn.com 56
ELISA Screening
Nominal Concentration (Central Value) 100.0
Target Measurement Uncertainty (Standard Deviation) 14.00 Enter the largest standard deviation which results in the acceptable
Lower Limit 0.0 "Total outside limits".
Upper Limit 200.0
% below Lower Limit 0.00% % above upper limit 0.00% Total outside limits 0.00%
Concentration
mljweitzel@msn.com 57
HPLC
Nominal Concentration (Central Value) 100.0
Target Measurement Uncertainty (Standard Deviation) 1.00 Enter the largest standard deviation which results in the acceptable
Lower Limit 90.0 "Total outside limits".
Upper Limit 110.0
% below Lower Limit 0.00% % above upper limit 0.00% Total outside limits 0.0%
Concentration
mljweitzel@msn.com 58
Procedure Design - Robustness
DESIGN OF EXPERIMENTS
mljweitzel@msn.com 59
DOE
DOE is a useful, efficient tool for procedure
validation
Well planned experiments
Hint: finalize and clearly define how the data will
be assessed/analyzed as part of the protocol.
Include actual formulas, charts, tables, etc.
Doing so makes for a well thought out, easily
understood protocol.
It also makes writing the report faster and easier.
You are less likely to have to repeat experiments.
Penny wise is often pound foolish.
mljweitzel@msn.com
60
Ruggedness DOE - Definitions
mljweitzel@msn.com
61
DOE Presented Here and in
Workbook
It is an “incomplete factorial design”
Also known as
AOAC Ruggedness Test
Youden
mljweitzel@msn.com 62
Robustness test
mljweitzel@msn.com
63
Run Design
Experiment (Run)
Factor 1 2 3 4 5 6 7 8
A + + + + - - - -
B + + - - + + - -
C + - + - + - + -
D + + - - - - + +
E + - + - - + - +
F + - - + + - - +
G + - - + - + + -
Result s t u v w x y z
mljweitzel@msn.com
64
Calculations for Factor A
Experiment (Run)
Factor 1 2 3 4 5 6 7 8
A + + + + - - - -
B + + - - + + - -
C + - + - + - + -
D + + - - - - + +
E + - + - - + - +
F + - - + + - - +
G + - - + - + + -
Result s t u v w x y z
mljweitzel@msn.com 66
What to do with Significant Factor?
mljweitzel@msn.com 67
Overall Standard Deviation
mljweitzel@msn.com 68
Example of Factors Selected for
HPLC
Factors
extraction heating time,
extraction temperature,
HCl concentration,
extraction volume,
column temperature,
flow rate, and
test sample weight
mljweitzel@msn.com 69
Examples in Literature
mljweitzel@msn.com 70
DOE References
mljweitzel@msn.com
71
New Proposed USP General Chapter
1210 Statistical Tools for Procedure
Validation
New proposed USP General Chapter <1210>
Statistical Tools for Method Validation.
This new proposed chapter, which was
published in PF 40(5) [Sept-Oct 2014], will be
a statistical companion chapter for USP
General Chapter <1225> Validation of
Compendial Procedures. Watch for this!
mljweitzel@msn.com 72
Analysis of Variance
ANOVA
mljweitzel@msn.com 73
Useful
mljweitzel@msn.com 74
Typical Experiment for Accuracy
Replicate Spike 1 Spike 2 Spike 3
1 99.53 102.30 100.78
Look at data
2 101.88 101.40 104.00
3 101.29 96.60 100.26
Could calculate
4 101.88 100.70 99.57 average and standard
5 94.71 102.70 100.78
6 100.12 100.60 102.09 deviation
Average 99.90 100.72 101.25 Could do t test
Std Dev 2.72 2.18 1.58
Only tell if 2 spikes
“Accuracy Experiment” were different
3 spikes done on same day
Only tells if they are
different, no probability
“It looks good. ??”
mljweitzel@msn.com 75
ANOVA One-way EXCEL Print Out
SUMMARY
Groups Count Sum Average Variance
Spike 1 6 599.41 99.90167 7.378857
Spike 2 6 604.3 100.7167 4.773667
Spike 3 6 607.48 101.2467 2.502467
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 5.5083 2 2.75415 0.563798 0.580647 3.68232
Within Groups 73.27495 15 4.884997
Total 78.78325 17
mljweitzel@msn.com 76
Calculate Precisions
Precision Symbol Formula
repeatability sr SQRT(MS Within Groups)
Between Group Standard Deviation s BG SQRT((M S Between Groups - M S Within Groups)/Count )
Intermediate Precision s IP SQRT(Sr2+SBG2)
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 5.5083 2 2.75415 0.563798 0.580647 3.68232
Within Groups 73.27495 15 4.884997
Precisions s r 2.210203
s BG #NUM! The MS's do not differ significantly. Both estimate repeatability.
s IP #NUM!
mljweitzel@msn.com 77
Repeatability
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Std Dev
Spike 1 6 599.41 99.90167 7.378857
2.716405
Spike 2 6 604.3 100.7167 4.773667
2.184872
Spike 3 6 607.48 101.2467 2.502467
1.581919
2.210203 Pooled
15 Degrees of
ANOVA Freedom
Source of Variation SS df MS F P-value F crit
Between Groups 5.5083 2 2.75415 0.563798 0.580647 3.68232
Within Groups 73.27495 15 4.884997
Total 78.78325 17
Precisions s r 2.210203
mljweitzel@msn.com 78
Between Groups &
Intermediate Precision (IP)
Precision Symbol Formula
repeatability sr SQRT(MS Within Groups)
Between Group Standard Deviation s BG SQRT((M S Between Groups - M S Within Groups)/Count )
Intermediate Precision s IP SQRT(Sr2+SBG2)
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 5.5083 2 2.75415 0.563798 0.580647 3.68232
Within Groups 73.27495 15 4.884997
Precisions s r 2.210203
s BG #NUM! The MS's do not differ significantly. Both estimate repeatability.
s IP #NUM!
mljweitzel@msn.com 79
Condition Experiment
mljweitzel@msn.com 80
ANOVA Between Group is significant
The condition is significant.
What does this mean?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Condition 1 6 596.41 99.40167 7.378857
Conditon 2 6 610.3 101.7167 4.773667
Condition 3 6 625.48 104.2467 2.502467
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 70.4683 2 35.23415 7.212728 0.006386 3.68232
Within Groups 73.27495 15 4.884997
Total 143.7433 17
Precisions s r 2.210203
s BG 2.249043
s IP 3.153282
mljweitzel@msn.com 81
Uncertainty Components
Precisions sr 2.21
s BG 2.25
s IP 3.15
mljweitzel@msn.com 82
Uncertainty Components
Precisions sr 2.21
s BG 2.25
s IP 3.15
mljweitzel@msn.com 83
Replicate Condition 1 Conditon 2 Condition 3
1 89.53 103.30 108.78
2 91.88 102.40 112.00 How should replicates be
3 91.29 97.60 108.26 placed?
4 91.88 101.70 107.57
5 84.71 103.70 108.78
6 90.12 101.60 110.09
Average 89.90 101.72 109.25
Std Dev 2.72 2.18 1.58
Anova: Single Factor
SUMMARY
The repeatability Groups Count Sum Average Variance
is not significant. Condition 1 6 539.41 89.90167 7.378857
Conditon 2 6 610.3 101.7167 4.773667
Condition 3 6 655.48 109.2467 2.502467
The between
conditions is the
critical variable ANOVA
and needs Source of Variation SS df MS F P-value F crit
Between Groups 1141.048 2 570.5242 116.7911 7.16E-10 3.68232
operational Within Groups 73.27495 15 4.884997
control.
Total 1214.323 17
Precisions sr 2.21
s BG 9.71
s IP 9.96
mljweitzel@msn.com 84
Use MU and Probability to describe
performance Nominal Concentration (Central Value) 100.0
Target Measurement Uncertainty (Standard Deviation) 9.96 Enter the largest standard deviation which results in the acceptable
Lower Limit 75.0 "Total outside limits".
Upper Limit 125.0
% below Lower Limit 0.60% % above upper limit 0.60% Total outside limits 1.2%
In this case
the
acceptable
probability
of making a
wrong
decision is 0 20 40 60 80 100 120 140 160 180 200
5%.
Concentration
mljweitzel@msn.com 85
The test item does not need to be
centered in specification
Nominal Concentration (Central Value) 92.0
Target Measurement Uncertainty (Standard Deviation) 9.96 Enter the largest standard deviation which results in the acceptable
Lower Limit 75.0 "Total outside limits".
Upper Limit 125.0
% below Lower Limit 4.39% % above upper limit 0.05% Total outside limits 4.4%
Concentration
mljweitzel@msn.com 86
Add 20% to MU to compensate for its
variability
Nominal Concentration (Central Value) 100.0
Target Measurement Uncertainty (Standard Deviation) 11.95 Enter the largest standard deviation which results in the acceptable
Lower Limit 75.0 "Total outside limits".
Upper Limit 125.0
% below Lower Limit 1.82% % above upper limit 1.82% Total outside limits 3.6%
Concentration
mljweitzel@msn.com 87
Understand & Communicate
Using DOE correctly and effectively allows
the identification of critical variables
(significant uncertainty components) during
analytical procedure design and qualification
Adequate operational controls can be
implemented
The intermediate precision is an initial
estimate of uncertainty (for those conditions
varied)
It can be compared to the target uncertainty
mljweitzel@msn.com 88
Probability
mljweitzel@msn.com 89
HOW TO ESTIMATE MU
mljweitzel@msn.com 90
How to Estimate Uncertainty
mljweitzel@msn.com 91
Process
mljweitzel@msn.com 92
2 • List the steps in the analytical process
Sampling, sub-sampling
Comminution
Weighing a test portion
Dissolving sample (not a dissolution test)
Volumetric manipulation
Calibration of measurement
Instrumental measurement
Data processing
mljweitzel@msn.com 93
Brains
3 • Identify potential sources of random variability
in each step – uncertainty components
Use experience
Brainstorm
Fishbone
Equation
SOP
√(sR2 + sr2)
mljweitzel@msn.com 94
Fish Bone – Eurachem Guide Quantifying
Uncertainty in Analytical Measurement
mljweitzel@msn.com 95
Intermediate Precision
mljweitzel@msn.com 96
4 • Design a process that permits an estimate of
each source of random variability or of a group
of sources or look for the data
mljweitzel@msn.com 97
Method Validation Studies
mljweitzel@msn.com 98
Replicate Data to Cover off
Sources of Variability
As part of method
validation studies, there
are several measures
that are repeated.
As part of routine runs,
data has been collected
that has repeated
measurements while
some parameters have
varied.
mljweitzel@msn.com 99
5 • Combine the different estimates of random
variability to get the overall uncertainty estimate
Combine data
Can organize using a table
mljweitzel@msn.com 100
Example from REAL data
mljweitzel@msn.com 101
Conclusion
mljweitzel@msn.com 103
SPREADSHEET
mljweitzel@msn.com 104