See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/334971858

VALIDATION OF ANALYTICAL PROCEDURES: GRAPHIC AND CALCULATED

CRITERIA FOR ASSESSMENT OF METHODS LINEARITY IN PRACTICE

Article · August 2019

CITATIONS READS

6 4,579

1 author:

Naum Epshtein
STADA CIS
52 PUBLICATIONS   242 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Stress Testing for methods validation View project

Attestation of reference standards View project

All content following this page was uploaded by Naum Epshtein on 05 August 2019.

The user has requested enhancement of the downloaded file.

DOI: 10.33380/2305-2066-2019-8-2-122-130
VALIDATION OF ANALYTICAL PROCEDURES:
GRAPHIC AND CALCULATED CRITERIA FOR ASSESSMENT
OF METHODS LINEARITY IN PRACTICE
Epshtein N.A. Validation of Analytical Procedures: Graphic and Calculated Criteria
for Assessment of Methods Linearity in Practice. Drug development & registration.
2019;8(2):122-130 (trans. from Russian).
Center of registration and development of medicines LLC "IRVIN 2", 115446, Russia, Moscow, Ko-
lomensky proezd, 13A
ORCID ID: orcid.org/0000-0001-8047-4078, Researcher ID: A-7114-2019
Corresponding author: Naum A. Epshtein. E-mail: naumepshtein@gmail.com
Received: 16.04.2019. Accepted: 18.04.2019
Abstract. Introduction. “Linearity” is one of the required characteristics when validating analytical
procedures. The issues of the validation of linearity procedures are still relevant.
Text. The main criteria for proving/confirming the linearity of analytical procedures and their re-
strictions are considered in detail. It is shown that these criteria cannot always give a reliable assess-
ment of the linearity of the procedure; the possible reasons for this are indicated and recommendations
are given.
Conclusion. When validating the procedures, it is necessary to prove/confirm their linearity by using
two, and, more reliably, three criteria, one of which must be the linearity of the plot of measured
response (𝑌𝑌) vs concentration of the tested substance (𝐶𝐶). On the other hand, we demonstrate here
that a formal approach should not be used for the estimation of non-linearity of the procedure when
using graphical and calculation criteria based on mathematical statistics, since they do not take into
account the possibility of practical insignificance of small deviations from the linear dependence of
𝑌𝑌 on 𝐶𝐶.
Key words: validation, linearity, procedures, acceptance criteria.

INTRODUCTION
“Linearity” is one of the required characteristics when validating analytical
methods [1-8]. According to ICH, “The linearity of an analytical procedure is its ability
(within a given range) to obtain test results which are directly proportional to the con-
centration (amount) of analyte in the sample” [1]. The goal of linearity validation of
the method, according to the USP “is to have a model, whether linear or nonlinear, that
describes closely the concentration-response relationship” (by a response, we mean the
measured signal or the result of its processing); “If linearity is not attainable, a nonlin-
ear model may be used” [2]. Therefore, the confirmation of the linear model of the
concentration-response relationship in methods validation is desirable but not neces-
sary. We emphasize that the essence of the “Linearity” test is to check the possibility
of using the linear model before confirming the accuracy and precision of the method
as the simplest and preferred practice. At the same time, the results of the “Linearity”
test may indicate the need for a non-linear model if the nonlinear model de-scribes
more closely (more accurately) the concentration-response relationship and confirms
the accuracy and precision of the method than a linear one.
 2

The linearity of methods is most often confirmed in accordance with ICH [1]:
visually by graph of dependence of response Y (the measurement signal, for example,
the peak area in HPLC) on concentration C of analyte and the acceptable value of cor-
relation coefficient R of linear regression dependence of Y on C; also one has to give
the regression equation or statistical characteristics of regression. In practice, other cri-
teria are also used to assess the linearity or non-linearity of the dependence of Y on C,
based on mathematical statistics [6-24]. There are certain reasons for this, as will be
seen from the text of this article. No wonder, therefore, that up to now remain relevant
questions which connected with criteria of linearity of analytical methods, both in prac-
tical validation of methods and in the expert appraisal of validation documentation.
The aim of the present work is to consider the main graphical and calculation
criteria for confirming the linearity of analytical methods and their limitations, and also
to give recommendations.
MATERIALS AND METHODS
The statistical calculations were made using MS Excel, including using the sta-
tistical package included in its composition. The experimental data given as examples
were obtained by HPLC on a Waters e2695 liquid chromatograph with a Waters 2998
Photodiode Array Detector.
RESULTS AND DISCUSSION
First of all, let us discuss why the linearity of the method is important, although
it is known that a nonlinear model of dependence of Y on C can be used for the cali-
bration graph [6, 11-22]. This can be explained by the following reasons. First, the
possibility of using the simplest method - absolute calibration (one concentration of a
standard solution) with negligible systematic error [6]; secondly, in the routine analysis
in case of line calibration chart it is enough to use 2-3 points while at the nonlinear
calibration schedule not less than 5 points will be required, and these are big expenses
of time especially in case of rather slow chromatographic techniques of the analysis.
The required number of experimental points and solutions used for linear-
ity validation. In the ICH validation manual, it is recommended to use at least 5 ex-
perimental points (concentrations of the substance being determined) in linearity vali-
dation of method [1]; this is consistent with the recommendation from mathematical
statistics [25]. But sometimes five points may not be enough for a reliable conclusion
about the linearity or nonlinearity of the method [6]. ICH allows for the validation of
method linearity to use solutions of required concentration by diluting the reference
standard solution. But this ignores the possibility of the influence of excipients since it
is indirectly verified by the “Accuracy” test. At the same time, according to ICH, to
confirm the linearity of the method, instead of diluting of reference standard solution
you can use the results obtained on mixtures of the analyte with placebo when testing
accuracy of the method. This usually reduces the amount of work and significantly
increases the reliability of the conclusions about the linearity or nonlinearity of the
method, since the effect of placebo is taken into account and the number of experi-
mental points increases (at least 9, distributed as evenly as possible in the analytical
 3

range of the method). In practice, we use such an approach for methods of determining
the content of main substances.
The range (analytical range) of the method for which it is necessary to vali-
date linearity depends on the type of method [1, 6]:
• For assay of a drug substance or a finished product: “not less than 80% to
120% of the nominal content” (i.e., the concentration of the tested solutions should
cover a range corresponding to at least 80,0% - 120,0% of the nominal content of the
analyte);
• For methods for determining the uniformity of dosage units: “not less than
70% to 130% of the nominal content, if a wider interval is not required for testing (for
example, for aerosols)”;
• For dissolution testing procedures: “± 20% (absolute) with respect to the up-
per/lower limits of the range specified in the specification”;
• For methods for determining impurities: “from the threshold of ignoring im-
purities (registration threshold) [26] or the limit of quantitation (LOQ) to no less than
120% of the nominal limit.” In the US Pharmacopoeia, another requirement is “from
50% to not less than 120% of the acceptance criterion” [2].
Let us consider the main criteria for assessing the linearity of the dependence of
Y on C, used in the validation of analytical methods and their limitations.
Correlation coefficient R
The Pearson’s correlation coefficient R and a graph of the measured response Y
on the concentration C of the analyte are most often used in the analytical methods
validation for confirmation of their linearity [1-6]. At the same time as the criterion of
the acceptability for the correlation coefficient R, it is possible to be guided by R≥
0,999 for methods of determination of the main substances [5, 27] and R ≥ 0,98 for
methods of determination of impurities [4]. Also other criteria of the acceptability, for
example, for methods of determination of the content of the main substances R> 0.998
for API and R> 0.990 for medicines [28], R≥ 0.995 [29] meet. R> is 0.998 for API and
R> 0,990 for medicines [28], R≥ 0,995 [29]. The lowest allowable (critical) values of
R, taking into account the possibility of rounding off of numbers, specified in the GF
XIV: " In most instances one uses the linear dependence relevant to condition R≥ 0.99,
and only in the analysis of trace quantities consider linear dependences for which R≥
0.9" [3]; in our experience such low acceptable values of R should be viewed as insuf-
ficiently reliable, accepted only at confirmation of accuracy and precision of the
method.
The above admissible values of R for determining the content of main substances
are significantly more than the critical values of the Pearson’s correlation coefficient
𝑅𝑅𝑐𝑐𝑐𝑐𝑖𝑖𝑖𝑖 used for theoretical estimates of existence of linear relationship between Y and
C. For example, for n = 5 points of experimental data for one-sided test with a proba-
bility P = 95,0% 𝑅𝑅𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 = 0,805, with P = 99,0% 𝑅𝑅𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 = 0,934, and with an increase in
the number of points, the value of 𝑅𝑅𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 only decreases (http://www.mag-
 4

niel.com/alex/spearman.html). It indicates the empirical nature of the above recom-

mendations of the values of R and can be explained by the fact that the Pearson’s cor-
relation coefficient is based on a linear model of the Y–C relationship and, moreover,
characterizes only correlation between Y and C without taking into account its practical
significance [11, 12].
Residual plot
In general, the dependence of measured response Y on C may, in fact, be non-
linear even at high value of correlation coefficient. The typical example with R =
0,99952 is given in [4]. Therefore, ICH recommends using the additional linearity
criterion - plot [1]. As a rule, it represents graphical dependence of the difference be-
tween the experimental and calculated (from the regression equation) response value
(𝑌𝑌𝑒𝑒𝑒𝑒𝑒𝑒 − 𝑌𝑌𝑐𝑐𝑐𝑐𝑐𝑐 ) from concentration C. Sometimes 𝑌𝑌𝑐𝑐𝑐𝑐𝑐𝑐 is used instead of C [9]. As an
acceptance criterion, the absence of a nonlinear trend in the location of points on the
residual plot is considered: “the residual plot must show random behavior in a constant
range, without systematic pattern or regularities” [6]. At UV-detecting it is recom-
mended to use no more, than the 10-fold range of concentration [6], otherwise, there is
the risk of making the nonlinear dependence of Y on C.
If five experimental points are not enough for an unambiguous conclusion about
the linearity or nonlinearity of the dependence of Y on C on the residual plot, it is
necessary to increase the number of experimental points, or to use the values of Y
obtained in the accuracy test (more reliable for linearity testing of methods). On the
other hand, the linearity of the method is a conditional concept, as evidenced by the
variety of criteria for assessing linearity in validation of methods. In this regard, it
should not be a formal approach to the assessment of linearity/nonlinearity of method
when using criteria based on mathematical statistics, including residual plot. Relation-
ship Y on C can be statistically significant but of no practical value. If as in Fig. 1:
- the residual plot (lower plot) indicates the nonlinearity of method from a theo-
retical point of view,
- but on the plot Y on C (upper plot) all experimental points are located on the
regression line or close to it, and obviously or confirmed by the calculations, the non-
linear model will not describe the dependence of Y on C more accurately than the lin-
ear one, then it is logical to conclude: “the method is linear” and to confirm (check)
this with the results of accuracy and precision testing of the method.
Note. The condition for obtaining correct residual plot and usual (unweighted)
linear regression is the constancy of standard deviation of residuals. The variability of
the standard deviation of residuals - heteroscedasticity, can be visually recognized by
the wedge-shaped distribution of residuals or by comparing the scatter of the results
of parallel measurements depending on concentration, as well as by methods of math-
ematical statistics [6]. If heteroscedasticity is detected, to obtain an adequate regres-
sion model it is necessary to increase the contribution (weight) of lower concentra-
tions by means of weight coefficients (weighted linear regression) [6].
 5

Fig. 1. The results of determining the dependence of the peak area (S) of the impurity on its
concentration (C) - above, and the corresponding residual plot - below, which formally indi-
cates a non-linear trend, however, this trend can be neglected from a practical point of view
(see text).
Plot of the dependence of Y/C vs. C
For assessing the linearity of the method instead of residuals plot, one can use
plot of Y/C on C called the “sensitivity plot” [6]. In accordance with the formula
Y=bC+a it can be expected that on the plot of Y/C on C in the case of linearity of the
method, the points will be located near the trend line ≈ b (characterizes the sensitivity
of method) and almost in parallel to the concentration axis. However, it should be borne
in mind that a necessary condition for estimating the linearity of the method according
to the Y/C on C versus C plot is the proximity of the free regression term a to zero,
since Y/C=b+a/C≈ b only for a ≈ 0. Therefore, before making plot of Y/C on C must
be verified that the free term of the regression equation is statistically insignificant. To
do this, you can use the student's t-test. The value of a is statistically insignificant (a≈0)
if [30]:
|𝑎𝑎|
𝑡𝑡𝑎𝑎 = < 𝑡𝑡 (P, f=n-2) (1)
𝑆𝑆𝑆𝑆𝑎𝑎
where 𝑡𝑡𝑎𝑎 is the estimated value of student's criterion, |𝑎𝑎| is the absolute value of free
member of linear dependence of Y on C; 𝑆𝑆𝑆𝑆𝑎𝑎 is the standard deviation of values 𝑎𝑎;
 6

𝑡𝑡(P, f=n-2) is the tabulated value of student's t-test at a probability of P= 95%; f is the
number of degrees of freedom; n is the number of experimental points on regression
line.
Criteria for analysis of variance (ANOVA):
F-criteria and p-value
To verify the linearity of the method, sometimes used analysis of variance
(ANOVA), together with visual confirmation of the linearity of the dependence of Y
on C. The main are two versions of ANOVA.
• LOF-test (“Lack of fit test”): a test with a statistical calculation of the re-
sults of parallel measurements. Its potential advantage is the independence of the ac-
ceptance criterion from the accepted model of the relationship between Y and C. In this
variant several repeated measurements are carried out for each concentration, for ex-
ample, 3 injections in the case of the HPLC procedure [10]. At the same time, it is
recommended to use at least 8 concentrations in the intended range of the method [6].
The LOF test based on consideration of certain residuals and their squares, on
which the variability of total measurement error depends [6, 9, 24]:
𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 = 𝑦𝑦𝑖𝑖𝑖𝑖 − 𝑦𝑦�𝑖𝑖 = 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 + 𝐿𝐿𝐿𝐿𝐿𝐿 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒, (2)
𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 = 𝑦𝑦𝑖𝑖,𝑗𝑗 − 𝑦𝑦�𝑖𝑖 ; 𝐿𝐿𝐿𝐿𝐿𝐿 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 = 𝑦𝑦�𝑖𝑖 − 𝑦𝑦�𝑖𝑖 (3)
where 𝑦𝑦𝑖𝑖,𝑗𝑗 is the result of 𝑗𝑗-th measurement at the point 𝑖𝑖 (i.e., at a concentration of 𝐶𝐶𝑖𝑖 );
𝑦𝑦� is the average value of measurement and 𝑦𝑦�𝑖𝑖 is the value of 𝑦𝑦 calculated on regression
equation at the point 𝑖𝑖. From (3) it can be seen that 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 is the deviation of meas-
ured value 𝑦𝑦𝑖𝑖𝑖𝑖 from predicted value 𝑦𝑦�𝑖𝑖 by regression equation at the point 𝑖𝑖. Residual
is the sum of Pure error (PE), which is a random error, does not depend on the regres-
sion model adopted and is the difference between the result of 𝑗𝑗-th measurement at the
point 𝑖𝑖 (𝑦𝑦𝑖𝑖𝑖𝑖 ) and the average value of the parallel measurements at point 𝑖𝑖 (𝑦𝑦�𝑖𝑖 ), and
LOF (“lack of fit error” [6]), which is the difference between the average value of the
parallel measurements at point 𝑖𝑖 (𝑦𝑦�𝑖𝑖 ) and the value of 𝑦𝑦 calculated on regression equa-
tion at the point 𝑖𝑖.
One can represent the variability of the total measurement error 𝑆𝑆𝑆𝑆𝑇𝑇 as a sum of
squares for the pure error (𝑆𝑆𝑆𝑆𝑃𝑃𝑃𝑃 ), 𝐿𝐿𝐿𝐿𝐿𝐿 (𝑆𝑆𝑆𝑆𝐿𝐿𝐿𝐿𝐿𝐿 ) and the factor associated with the re-
gression model (𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅 ) [9]:
𝑘𝑘 𝑚𝑚𝑖𝑖
2
𝑆𝑆𝑆𝑆𝑇𝑇 = � ��𝑦𝑦𝑖𝑖,𝑗𝑗 − 𝑦𝑦�� =
𝑖𝑖=1 𝑗𝑗=1
𝑘𝑘 𝑚𝑚𝑖𝑖 𝑘𝑘 𝑘𝑘

� �(𝑦𝑦𝑖𝑖,𝑗𝑗 − 𝑦𝑦�𝑖𝑖 )2 + � 𝑚𝑚𝑖𝑖 ∙ (𝑦𝑦�𝑖𝑖 − 𝑦𝑦�𝑖𝑖 )2 + � 𝑚𝑚𝑖𝑖 ∙ (𝑦𝑦�𝑖𝑖 − 𝑦𝑦�)2 (4)

𝑖𝑖=1 𝑗𝑗=1 𝑖𝑖=1 𝑖𝑖=1

𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅 𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅

𝑆𝑆𝑆𝑆𝑃𝑃𝑃𝑃 𝑆𝑆𝑆𝑆𝐿𝐿𝐿𝐿𝐿𝐿
 7

where 𝑘𝑘 is the number of concentrations, 𝑚𝑚𝑖𝑖 is the number of parallel determinations

at the 𝑖𝑖-th concentration. It can be seen from residuals 𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅 in (4) that 𝑆𝑆𝑆𝑆𝐿𝐿𝐿𝐿𝐹𝐹 depends
on the regression model adopted - since it depends on 𝑦𝑦�, and therefore 𝑆𝑆𝑆𝑆𝐿𝐿𝐿𝐿𝐿𝐿 can be
used to estimate the linearity of method. The condition for confirming the linearity of
the method is the fulfillment of the requirement for the experimental value of the F-
criterion [6, 9]:
𝑆𝑆𝑆𝑆𝐿𝐿𝐿𝐿𝐿𝐿 ⁄(𝑘𝑘−2)
𝐹𝐹𝑒𝑒𝑒𝑒𝑒𝑒 = ≤ 𝐹𝐹(P, f1,f2) = 𝐹𝐹(P, k-2, n-k) (5)
𝑆𝑆𝑆𝑆𝑃𝑃𝐸𝐸 ⁄(𝑛𝑛−𝑘𝑘)

where 𝑘𝑘 is the number of concentrations; 𝑛𝑛 is the total number of measurements

(𝑛𝑛 = 𝑚𝑚 ∙ 𝑘𝑘 when the number of parallel measurements 𝑚𝑚𝑖𝑖 = 𝑚𝑚); 𝐹𝐹 (𝑃𝑃, 𝑓𝑓1 , 𝑓𝑓2 ) is the
critical (tabular) value of the Fisher's F-test at confidence level P (usually 95%) and
degrees of freedom 𝑓𝑓1 = 𝑘𝑘- 2 and 𝑓𝑓2 =𝑛𝑛- 𝑘𝑘. Formula (5) is a Fisher criterion since the
ratio of two variances is considered: 𝑆𝑆𝑆𝑆𝑃𝑃𝐸𝐸 /(𝑛𝑛- 𝑘𝑘) - the calculated estimate of the vari-
ance 𝜎𝜎 2 ; 𝑆𝑆𝑆𝑆𝐿𝐿𝐿𝐿𝐿𝐿 /(𝑘𝑘- 2) - the calculated estimate of 𝜎𝜎 2 + 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 2 (if the model is inade-
quate).
Note that this method, like other variants of ANOVA, can sometimes lead to an
erroneous conclusion about the nonlinearity of the method [10], and therefore in the
case of 𝐹𝐹𝑒𝑒𝑒𝑒𝑒𝑒 > critical value F, look at the residual plot, as recommended in [9]. So in
[10], using the HPLC method, it was found that while the residual plot indicated a
linear dependence of 𝑌𝑌 on 𝑥𝑥, the LOF test showed that 𝐹𝐹𝑒𝑒𝑒𝑒𝑒𝑒 > 𝐹𝐹(𝑃𝑃, k-2, n-k). As it
turned out, this was due to an additional source of variation 𝑌𝑌 connected with sample
preparation. This was substantiated theoretically and introduced a different experi-
mental F-criterion (modified LOF test [10]), which confirmed the linear dependence
of 𝑌𝑌 on 𝑥𝑥.
We also observed a situation similar to [10], when, with an obvious linear de-
pendence of the peak area (S) on concentration and the absence of a non-linear trend
on the residual plot (Fig. 2), it turned out that 𝐹𝐹𝑒𝑒𝑒𝑒𝑒𝑒 = 25,2455 > F(95%, 8, 20)= 2,4471.
That is, the LOF test mistakenly did not confirm the linear dependence between S and
С. Since there was a very high convergence of the results of parallel determination of
the peak area of the API - the values of the relative standard deviation RSD were in the
range of 0,04% up to 0,22%, it was assumed that the low RSD is the reason for addi-
tional source of variations S and leads to an erroneous LOF test. To test this assump-
tion, we increased the scatter of the results of parallel determinations in Excel and com-
pared it to the experimental data: the RSD value increased to 0,54% to 1,03%. As a
result of the increase in RSD, the 𝐹𝐹𝑒𝑒𝑒𝑒𝑒𝑒 value decreased to 𝐹𝐹𝑒𝑒𝑒𝑒𝑒𝑒 = 1,5331 which is less
than F(95%, 8, 20)= 2,4471 and the LOF test confirmed the linear dependence of Y on
C as expected. At the same time, the regression dependence of Y on C and the residual
plot changed only slightly (Fig. 2). From the obtained results it follows that with high
repeatability of measurement results - RSD is less than 0.5% in our case, the unmodi-
fied LOF test can lead to an erroneous conclusion about the nonlinearity of the ana-
lytical method. On the contrary, sometimes with poor repeatability of the results of
 8

parallel determinations, one can obtain 𝐹𝐹𝑒𝑒𝑒𝑒𝑒𝑒 ≤ 𝐹𝐹 (𝑃𝑃, 𝑓𝑓1 , 𝑓𝑓2 )), and at the same time, the
method may have the necessary precision [24, p. 103].

Fig. 2. Experimental (A) and corrected (by increasing the relative standard deviation of RSD
parallel measurements) - (B), the results of determining the dependence of the peak area (S)
of the API on concentration. In case (A), due to the very high reproducibility of parallel
measurements (RSD from 0.04% to 0.22%), the LOF test did not confirm linearity, although
the points on the residual plot ( ) indicated the linearity of the technique. With an increase
in RSD of parallel measurements to 0.54% -1.03% - case (B), there was only a slight change
in the parameters of the regression dependence of Y on C and the residual plot ( ) compared
to (A), however, this turned out to be enough so that the LOF test confirms the linearity of
the method (see text for explanation).

• "F/p-test" of linear dependence: a test without statistical calculation of the

results of parallel measurements. The average values of experimental Y (𝑌𝑌𝑒𝑒𝑒𝑒𝑒𝑒 ) and cal-
culated Y from the linear regression equation (𝑌𝑌𝑐𝑐𝑐𝑐𝑐𝑐 ) are used as initial data. The theo-
retical basis of this procedure is described in detail on the Internet: https://ex-
cel2.ru/articles/proverka-znachimosti-regressii-s-pomoshchyu-dispersionnogo-ana-
liza-f-test with an Excel example https://excel2.ru/sites/de-
fault/files/node/808/edit/_f-test_for_simple_regression.xlsx
To confirm the linearity of the dependence of Y on C, it is required that the
 9

calculated value of F-criterion (𝑭𝑭𝒐𝒐 ; formula (6)) obtained using the function
=LINEST(range Y; range X; TRUE; TRUE) was greater than the critical value of the
Fisher criterion 𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 (calculated using the function =F.INV.RT(𝛼𝛼,1, k-2) and the calcu-
lated value of the criterion p (calculated using the function =F.DIST.RT(𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 ;1; k-2)
would be less than a given level of significance 𝛼𝛼 (usually 𝛼𝛼 = 0.05 or 𝛼𝛼 = 0.01, which
corresponds to a probability of 95% or 99%, respectively).
𝑆𝑆𝑆𝑆𝑆𝑆⁄1
𝐹𝐹𝑜𝑜 = , where (6)
𝑆𝑆𝑆𝑆𝑆𝑆 ⁄(𝑘𝑘−2)
𝑘𝑘 𝑘𝑘

𝑆𝑆𝑆𝑆𝑆𝑆 = � (𝑦𝑦�𝑖𝑖 − 𝑦𝑦�)2 ; 𝑆𝑆𝑆𝑆𝑆𝑆 = � (𝑦𝑦𝑖𝑖 − 𝑦𝑦�𝑖𝑖 )2 (7)

𝑖𝑖=1 𝑖𝑖=1
where k is the number of experimental data points on the plot Y on C. Note that 𝑆𝑆𝑆𝑆𝑆𝑆
corresponds to 𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅 in (4), and 𝑆𝑆𝑆𝑆𝑆𝑆 corresponds to 𝑆𝑆𝑆𝑆𝐿𝐿𝐿𝐿𝐿𝐿 with 𝑚𝑚𝑖𝑖 =1 and 𝑖𝑖=1. The
use of this variant ANOVA usually does not cause difficulties. However, it based on
the linear model, which reduces the reliability of findings. For example, for the regres-
sion of Y on C in Figure 1, in contrast to the residual plot (non-linearity), the results of
this test calculated in Excel, pointed to the lack of non-linearity:

Residual standard deviation 𝑺𝑺𝑺𝑺𝒐𝒐 and correlation index 𝑹𝑹𝒄𝒄

In Ukraine, an alternative to ICH approach has been developed for validating the
linearity of methods [8]. It based on the use of normalized coordinates and the maxi-
mum permissible uncertainty of the analytical method [7, 8]. When validating the lin-
earity of methods, residual standard deviation 𝑆𝑆𝑆𝑆0 and correlation index 𝑅𝑅𝑐𝑐 are con-
sidered as controlled characteristics. The condition for the applicability of 𝑆𝑆𝑆𝑆0 and
the approach as a whole is the statistical (or practical) insignificance of the free term
of regression dependence 𝑎𝑎 (that is, 𝑎𝑎 ≈ 0). The following algorithm are used:
In Ukraine, the alternative approach to ICH for validating the linearity of meth-
ods has been developed [8]. It based on the use of normalized coordinates and the max-
imum permissible uncertainty of the analytical method [7, 8]. When validating the
linearity of methods residual standard deviation 𝑆𝑆𝑆𝑆0 and correlation index 𝑅𝑅𝑐𝑐 are con-
sidered as controlled characteristics. The condition for the applicability of 𝑆𝑆𝑆𝑆0 and the
approach as a whole is the statistical (or practical) insignificance of the free term of
regression dependence, that is, 𝑎𝑎 ≈ 0. The following algorithm is used:
At first, they use the method of the least squares to calculate a linear regression
dependence between the normalized experimental values of the response 𝑌𝑌𝑛𝑛 and the
concentration 𝐶𝐶𝑛𝑛 :
 10

𝑌𝑌𝑛𝑛𝑛𝑛 = 𝑏𝑏𝐶𝐶𝑛𝑛𝑛𝑛 + 𝑎𝑎, (8)

where 𝑌𝑌𝑛𝑛𝑛𝑛 = 𝑌𝑌𝑖𝑖 ⁄𝑌𝑌𝑠𝑠𝑠𝑠 , 𝐶𝐶𝑛𝑛𝑛𝑛 = 𝐶𝐶𝑖𝑖 ⁄𝐶𝐶𝑠𝑠𝑠𝑠 ; the indices 𝑖𝑖 and 𝑠𝑠𝑠𝑠 refer to the 𝑖𝑖-th test solution and
the standard solution respectively; the index 𝑛𝑛 indicates normalized values. Then they
check the statistical (or practical) insignificance of the free term of the regression equa-
tion (8). After confirming that 𝑎𝑎 ≈ 0, they verify if the requirement for the residual
standard deviation 𝑆𝑆𝑆𝑆0 is satisfied by the formula:
∆𝐴𝐴𝐴𝐴
𝑆𝑆𝑆𝑆0 /𝑏𝑏 ≤ , (9)
𝑡𝑡(95%,𝑛𝑛−2)
where 𝑆𝑆𝑆𝑆𝑜𝑜 - residual standard deviation that can be determined in Excel using the
function = STEYX(range Y; range X); ∆𝐴𝐴𝐴𝐴 - maximum permissible uncertainty of ana-
lytical method , calculated using simple formulas [7, 8]; 𝑡𝑡(95%, 𝑛𝑛 − 2) - value of the
Student’s 𝑡𝑡 at 95% probability (one-sided) with n – 2 degrees of freedom; n - number
of samples (the number of experimental points on the plot).
If condition (9) is satisfied, they calculate the correlation index on the formula:
𝑆𝑆𝑆𝑆𝑜𝑜2
𝑅𝑅𝑐𝑐 =�1 − 2 , (10)
𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅
in which 𝑆𝑆𝑆𝑆𝑜𝑜 is the residual standard deviation, and 𝑅𝑅𝑆𝑆𝑆𝑆𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 is the standard deviation
of the concentrations calculated by the formula:
̅ )2
(𝐶𝐶𝑛𝑛𝑛𝑛 −𝐶𝐶𝑛𝑛
𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 =�∑ ⋅100% (11)
(𝑛𝑛−1)
where 𝐶𝐶𝑛𝑛𝑖𝑖 and 𝐶𝐶𝑛𝑛̅ is the concentration of i-th solution and the average concentration of
solutions in normalized coordinates respectively, 𝑛𝑛 is the number of experimental
points. Then they check that the requirement for correlation index is met to:
2
∆𝐴𝐴𝐴𝐴 /𝑡𝑡(95%,𝑛𝑛−2)
𝑅𝑅𝑐𝑐 ≥ �1 − � � (12)
𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅

If the requirements (9) and (12) are met the method is considered as linear. Note
that the correlation index 𝑅𝑅𝑐𝑐 does not depend on the model 𝑌𝑌 = 𝑓𝑓(𝐶𝐶) and therefore has
a significant advantage over the Pearson’s correlation coefficient 𝑅𝑅 in estimating the
linearity of the methods. This approach is considered most detail from a theoretical and
practical point of view in the book [8]. This book also contains the most complete table
with the critical values of the criteria considered above (right parts of formulas (12)
and (9) with 𝑏𝑏 ≈ 1). This facilitates the validation of methods for assay, uniformity of
dosage units and dissolution testing. Even though this approach is based on theoretical
assumptions and uses the criteria of "practical insignificance", it can be concluded,
based on an Internet search, that it is used only in validation methods in Ukraine and
partially in Russia and some other Post-Soviet states. Most researchers use ICH rec-
ommendations [1, 2].
Additional recommendations:
• If during the validation of the method its non-linearity is established, one
 11

should try to reduce the range or change the range of the concentrations under consid-
eration or use weighted linear regression if necessary.
• To verify that a particular experimental data point 𝑦𝑦𝑗𝑗 is outlier relative to
the regression model, you can use the t-test [11]. Calculate the experimental value of
test:
|𝑦𝑦𝑖𝑖 −𝑦𝑦�𝑖𝑖 |𝑚𝑚𝑚𝑚𝑚𝑚
𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒 = 2
(13)
��
1 �𝑦𝑦𝑗𝑗 −𝑦𝑦
𝑆𝑆𝑆𝑆0 ∙�1+ +
𝑛𝑛 (𝑛𝑛−1)∙𝑆𝑆𝑆𝑆2
𝑦𝑦

where |𝑦𝑦𝑖𝑖 − 𝑦𝑦�𝑖𝑖 |𝑚𝑚𝑚𝑚𝑚𝑚 is the maximum absolute value of the difference between the ex-
perimental 𝑦𝑦𝑖𝑖 and the calculated 𝑦𝑦�𝑖𝑖 (by regression) response value among all data points
(from 𝑖𝑖=1 to 𝑛𝑛); 𝑛𝑛 is the number of experimental points; 𝑦𝑦𝑗𝑗 is the experimental value
of the response at the potential outlier point 𝑗𝑗; 𝑦𝑦�= ∑ 𝑦𝑦𝑖𝑖 ⁄𝑛𝑛; 𝑆𝑆𝑆𝑆0 is the residual standard
deviation and 𝑆𝑆𝑆𝑆𝑦𝑦 is the standard deviation of the experimental response values 𝑦𝑦,
calculated by the formulas:
∑(𝑦𝑦𝑖𝑖 −𝑦𝑦�𝑖𝑖 )2 ∑(𝑦𝑦𝑖𝑖 −𝑦𝑦�)2
𝑆𝑆𝑆𝑆0 = � , 𝑆𝑆𝑆𝑆𝑦𝑦 = � (14)
𝑛𝑛−2 𝑛𝑛−1

Compare 𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒 with the tabular value of Student's t(95%, f=n- 2). If 𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒 is greater
than the tabular value of 𝑡𝑡(95%, f=n- 2), then with a 95% probability this point does
not correspond to the accepted regression model and is an outlier that requires 𝑦𝑦𝑗𝑗 to be
re-determined. An alternative to this test is the F-test given in [31].

CONCLUSION
The main criteria for assessing linearity at validation of analytical methods and
their limitations were considered. It is shown that these criteria separately cannot pro-
vide a reliable assessment of method linearity. Therefore, in practice, when validating
methods, it is necessary to prove (confirm) their linearity using two or three criteria (it
is more reliable) one of which has to be a plot of measured response (Y) on concentra-
tion (C) of the substance to be determined. On the other hand, as shown above when
using graphical and calculation criteria based on mathematical statistics one should not
formally approach the assessment of non-linearity of the method since they don’t take
into account the possibility of practical insignificance of small deviations from the lin-
ear dependence of Y on C.
REFERENCES
1. ICH Harmonised Tripartite Guideline. Validation of Analytical Procedures: Text
and Methodology Q2(R1). ICH, 2005; 13.
2. United States Pharmacopoeia. <1225> Validation of compendial procedures.
USP40–NF35. 2017; 1640-1646.
3. State Pharmacopoeia of the Russian Federation XIV edition, 2018, V.1; 276-288 (In
Russ.).
 12

4. Green J. М. А Practical Guide to Analytical Method Validation. Analytical Chemis-

try News & Features. 1996; Мау 1: 305А-309А.
5. Epshtein N.A. Validation of HPLC Techniques for Pharmaceutical Analysis, Phar-
maceutical Chemistry Journal. 2004; 38(4): 212-229.
DOI: https://doi.org/10.1023/b:phac.0000038422.27193.6c
6. Method Validation in Pharmaceutical Analysis. A Guide to Best Practice. Ed by J.
Ermer and J. H. McB. Miller. Weinheim: Wiley-VCH Verlag GmbH & Co. KGaA.
2005; 418.
7. Guide for enterprises of the pharmaceutical industry. Guidelines. Part I. Guidelines
for the validation of methods for analyzing drugs. Ed. N. V. Yurgelya, A. L. Mla-
dentseva, A. V. Burdeina i dr. M.: Publishing house «Sport and Culture – 2000». 2007;
5–92 (In Russ.).
8. Grizodub A.I. Standardized procedures for the validation of drug quality control
methods. Khar'kov: Gosudarstvennoe predpriyatie «Ukrainskiy nauchnyy farma-
kopeynyy tsentr kachestva lekarstvennykh sredstv», 2016; 396 (in Russ.).
9. Massart D.L., Vandeginste B.G.M., Buydens L.M.C., De Jong S., Lewi P.J., Smey-
ers-Verbeke J. Straight Line Regression and Calibration, in: Handbook of Chemomet-
rics and Qualimetrics: Part A, volume 20A of Data Handling in Science and Technol-
ogy. Amsterdam, Netherlands: Elsevier. 1998: 171–230.
DOI: https://doi.org/10.1016/S0922-3487(97)80038-X.
10. Vial J. and Jardy A. Taking into account both preparation and injection in HPLC
linearity studies. J. Chromatogr. Sci. 2000; 38:189-194.
11. Burke S. Regression and Calibration. LC-GC Europe Online Supplement statistics
and data analysis. 2001: 13-18.
12. Kiser M.M., Dolan J.W. Selecting the Best Curve Fit. LC-GC North America.
2004; 22 (2): 138-143.
13. Mark H., Workman J. Chemometrics in Spectroscopy. How to Test for Non-line-
arity. Spectroscopy. 2005; 20 (9): 26-35.
14. Mark H., Workman J. Linearity in calibration: Other tests for non-linearity // Spec-
troscopy. 2005; 20(4):38-39.
15. Scheilla Vitorino Carvalho de Souza, Junqueira R.G. A procedure to assess linear-
ity by ordinary least squares method. Analytica Chimica Acta. 2005; 552(1-2): 25-35.
DOI: https://doi.org/10.1016/j.aca.2005.07.043.
16. Mark H., Workman J. Chemometrics in Spectroscopy. Linearity in Calibration:
Quantifying Nonlinearity, Part II. Spectroscopy. 2006; 21 (1): 44-54
17. Bruggemann L., Quapp W., Wennrich R. Test for non-linearity concerning linear
calibrated chemical measurements. Accred Qual Assur. 2006; 11: 625-631. DOI:
https://doi.org/10.1007/s00769-006-0205-x.
18. Burrows J, Watson K. Linearity of chromatographic systems in drug analysis part
I: theory of nonlinearity and quantification of curvature. Bioanalysis. 2015; 7
(14):1731-1743. DOI: https://doi.org/10.4155/bio.15.103.
19. Burrows J, Watson K. Linearity of chromatographic systems in drug analysis part
II: a Monte Carlo justification for the use of nonlinear regressions. Bioanalysis. 2015;
7 (14):1745-1761. DOI: https://doi.org/10.4155/bio.15.104
 13

20. Burrows J, Watson K. Linearity of chromatographic systems in drug analysis part

III: examples of nonlinear drug assays. Bioanalysis. 2015; 7 (14): 1763-1774. DOI:
https://doi.org/10.4155/bio.15.105.
21. Raposo F. Evaluation of analytical calibration based on least-squares linear regres-
sion for instrumental techniques: A tutorial review. TrAC Trends in Analytical Chem-
istry. 2016; 77: 167-185. DOI: https://doi.org/10.1016/j.trac.2015.12.006.
22. Rawski R.I., Sanecki P.T., Kijowska K.M., Skital P.M. and Saletnik D.E. Regres-
sion Analysis in Analytical Chemistry. Determination and Validation of Linear and
Quadratic Regression Dependencies. S. Afr. J. Chem. 2016; 69, 166–173. DOI:
https://doi.org/10.17159/0379-4350/2016/v69a20.
23. Mymrikov A. Is your calibration a straight line? 25.12.2017. Url: https://pharm-
community.com/2017/8336/.
24. Chemometrics in Chromatography ed. by Komsta L., Vander Heyden Y., Sherma
J. Boca Raton: CRC Press, 2018; 506. DOI: https://doi.org/10.1201/9781315154404.
25. Doerffel K. Statistik in der analytischen Chemie. Leipzig: VEB Deutscher Verlag
für Grundstoffindustrie, 1966; 211. DOI: https://doi.org/10.1002/bimj.19690110612.
26. Epshtein N.A. About admissible values of disregard limits for impurities and sig-
nal-to-noise ratio when checking chromatographic system sensitivity // The Bulletin of
the Scientific Center for Expert Evaluation of Medicinal Products. 2017; 7(2): 85-91
(In Russ.).
27. Reviewer Guidance: Validation of Chromatographic Methods. Center for Drug
Evaluation and Research (CDER). Washington. 1994. URL:
https://www.fda.gov/downloads/drugs/guidances/ucm134409.pdf
28. Rosario LoBrutto and Tarun Patel. Method Validation in HPLC for Pharmaceutical
Scientists. Ed. By Kazakevich Y.V., LoBrutto R. Hoboken, New Jersey: John Wiley
& Sons, Inc. 2007; 455-502. DOI: https://doi.org/10.1002/9780470087954.ch9.
29. Gao Quanyin and Sanvordeker D.R. Analytical methods development and methods
validation for oral solid dosage forms in Generic Drug Product Development: Solid
Oral Dosage Forms, 2-nd Ed. Shargel, Leon: CRC Press/Taylor & Francis Group.
2014; 31-50.
30. Doerffel K. Statistik in der analytischen Chemie. Leipzig: Deutscher Verlag für
Grundstoffindustrie GmbH: 1990; 256.
31. Njaka N.A., Elise O.R., Herinirina N.R., Lucienne V.R., Manovantsoatsiferana H.,
R.A., Randrianarivony E. Dealing with Outlier in Linear Calibration Curves: A Case
Study of Graphite Furnace Atomic Absorption Spectrometry. World Journal of Applied
Chemistry. 2018; 3 (1): 10-16.
Note. The same article in Russian is available at Url:
https://www.researchgate.net/publication/333481029_Validacia_analit-
iceskih_metodik_graficeskie_i_rascetnye_krite-
rii_dla_ocenki_linejnosti_metodik_na_praktike?_sg=livhzJmrjkojFz3bmEEV-
ZzOUZTNbMSFQgjZyojDp2b-Hdi-jI-aho4Ueumg5lpXjfzlpiyYJGA5eQyg5ik3sgR-
gOFEf8R3z7gpczM4mD.32-xB2M4Ti9jJ126CJeikH472PnCvCKDFQIiKk-
2HcA5diPDpd-zbkEumO3_V7bnqckCKecn0UX67FDxOnNCQg

View publication stats