Statistical Method Validation and Estimation of

Measurement of Uncertainty for Test Laboratories

Allan Fraser
Allan Fraser & Associates
(Consulting Analytical Chemists and Geochemists)
Section 12

Method Development, Calibration Best Practice,

Setting out a Procedure for Method Validation
 Optimising Sample Mass
• It is important to know what the optimal mass of sample is with LECO
combustion instrumentation.
• This is because too little sample means that only a small amount of the
analyte being detected by the instrument detection system.
• In contrast, too much sample could lead to incomplete sample combustion
which results in poor precision of replicate test sample results.
• In addition, a too small sample is less representative than a larger sample
and results in imprecision in replicate measurements on a test sample.
• If powdered samples are used, ensure that the particle size is sufficiently
fine. Typically, 95% passing 75um or similar. Milling to a finer size will
improve precision.

3
Optimising Sample Mass
• Therefore, the mass that provides the best precision must be
determined.
• Another approach is to use a CRM at different masses. The
mass that gives the smallest % error in terms of measured and
certified will be the optimal mass.

4
 Optimising Sample Mass
Optimising Sulphur mass for minimum error using CRM SARM 74 mass from 0.087 g to 0.49 g

SARM74 Mass (g) % S Measured mg S Certified mg S Measured % Error Sulphur Optimal Mass at ~0.25 g
0.0872 0.0480 0.035 0.042 -0.712
1.500
0.1213 0.0443 0.049 0.054 -0.548
0.1511 0.0411 0.060 0.062 -0.156 y = 4.3508x - 1.0834
0.2039 0.0412 0.082 0.084 -0.244 1.000 R² = 0.7524

0.2050 0.0410 0.082 0.084 -0.200

0.2520 0.0415 0.101 0.105 -0.420
0.500

% Error Sulphur SARM 74

0.2503 0.0396 0.100 0.099 0.112
0.2563 0.0398 0.103 0.100 0.252
0.2911 0.0411 0.116 0.120 -0.356 0.000
0.3075 0.0384 0.123 0.120 0.300 0 0.1 0.2 0.3 0.4 0.5 0.6

0.3405 0.0395 0.136 0.130 0.620

0.3641 0.0388 0.146 0.140 0.564 -0.500
0.3777 0.0401 0.151 0.150 0.108 Smallest error =
0.4054 0.0383 0.162 0.150 1.216 -1.000 optimal mass ~0.25 g
0.4239 0.0375 0.170 0.160 0.956
0.4976 0.0389 0.199 0.190 0.904
-1.500
Mass SARM 75 (g)

-2.000

5
Linear Forced Through Zero
• It is often tempting to exclude the intercept, a, from the model because a
zero stimulus on the x-axis should lead to a zero response on the y-axis.
However, the correct procedure is to fit the full model and test for the
significance of the intercept term, i.e., when the origin (zero, zero) is
within the uncertainty of the measurement. See section on Calibration.

Sources:
http://www.itl.nist.gov/div898/handbook/mpc/section3/ mpc 361.htm Section 2.3.6.
Roland Caulcutt and Richard Boddy, 1983, "Statistics for Analytical Chemists," Chapman and Hall, New
York, ISBN 0 - 412 - 23730 - X, p 91.

6
 Calibration Best Practice (Spectrometric Methods)
• Applying calibration best practice minimises measurement uncertainty
and improves accuracy in the measurement of test samples.
• The equation for calculation of the standard uncertainty of a test
sample (Sx0), provides a view on how best practices can be applied.
• Reducing the uncertainty in measurement for analyses performed by
instrumental means will require as far as possible maximising the
terms in the denominator and minimising the terms in the numerator
of Sx0 equation.
• This will result in a small value for Sx0 which when multiplied by a t
critical value for n-2 degrees of freedom, will result in a smaller “±” or
confidence interval on the test sample result.

(Barwich, 2003; Ellison et al., 2009, Thompson & Lowthian, 2011; Colvine, 2014)
7
Minimising Sx0
1/2
Sy/x 1 1 lj 2
(𝑦0 − 𝑦)
Sx0 = ൜ + + 2ቋ
To minimise Sx0 b 𝑚 𝑛 𝑏 2 σ(𝑥𝑖 − 𝑥)lj

• Minimise Sy/x
• Minimise y0 − y
• Maximise: b, m, n and  ( xi − x )
2

• The values of the RMs should be distributed approximately

equidistantly over the range of values of the measurement System,
e.g., 0, 2, 4, 6, 8,10 and 12ppm or 1, 2, 3, 4, 5, 6, 7, 8%
• Linear model verified using SQT test

8
Minimising Sx0
Majority of sample test values should fall near centroid value. How
can this be done?
• Using historical data, calculate the mean value of the analyte of
interest.
• Design the calibration range so that the mean is near the centroid of
the calibration line.
• This ensures that many of the test sample values will have a minimum
measurement uncertainty.

9
Minimising Sx0
Majority of sample test values should fall near centroid value.
Example: 1000 test sample values have a mean value of 20 ppm.
Calibration standards are chosen as follows:
• Distributed approximately equidistantly
• Mean of calibration standards of 0, 5, 10, 15, 20, 25, 30, 35, 40 =17.5
ppm
• Test samples will then fall near the centroid value of 17.5 ppm

10
Minimum Number of Calibration
Standards
ISO 11095:1996:
• The number of RMs used to assess the calibration function should be
at least 3. For an initial assessment of the calibration function, a
number larger than 3 is recommended (at least 3 over any subinterval
where there is a doubt about the linearity of the calibration function).
• 6 to 8 RMs would allow accurate description of all possible inflection
points in the calibration.

11
 Calibration Standard
Sequence Sequence Conc.
No. (%) Intensity
When running calibration standards there are two schemes that tend to be used: 1 0.50 0.0282
2 0.75 0.0415
• Lowest to highest concentration – this can help reduce effects of 3 0.00 0.0033
any potential carry over between analysis as lower concentration 4 1.00 0.0555
does not have as much effect on higher concentrations. However, in 5 0.25 0.0172
this scheme carry over is very difficult to identify as it has the 6 0.5 0.0302
same effect on each subsequent sample. 7 0.75 0.0431
8 0.0 0.0034
• Random order – this is considered ideal as it can help identify 9 1.0 0.0562
carry over. Any carry over present will produce a higher correlation
coefficient (R2) value. 10 0.25 0.0172
11 0.5 0.0296
12 0.75 0.0435
It is good practice to randomise calibration standard 13 0.0 0.0032
concentrations during instrument calibration (as in table shown here). 14 1.0 0.0567
This method can be used to detect instrument drift during calibration 15 0.25 0.0172

see Apps NoteApplications Note: 19. Assessment of Significance of Drift During

Calibration of a Spectrometer

12 https://andyjconnelly.wordpress.com/2017/02/26/its-calibration-time/
 Assessment of Significance of Drift During
Calibration of a Spectrometer
Sequence
No. Residuals
1 -0.0015 0.0010
2 -0.0014 p=0.006

R² = 0.4481
3 -0.0002 0.0005
4 -0.0005

Residuals
5 0.0006 0.0000
0 2 4 6 8 10 12 14 16
6 0.0005
7 0.0003 -0.0005
8 -0.0001
9 0.0002 -0.0010
10 0.0006
11 -0.0001 -0.0015
12 0.0006
13 -0.0003 -0.0020
14 0.0007 Sequence number
15 0.0006
see Apps NoteApplications Note: 19. Assessment of Significance of Drift During
13 Calibration of a Spectrometer
 Internal Standard
In ICP-OES for example:
• Used to negate the effects of changes in plasma conditions or sample
introduction issues is the use of internal standards.

• An internal standard is a solution containing a known amount of an element not

found in the sample solution.

• A known and identical volume of the internal standard solution is added to

each sample. By monitoring the internal standard intensity it can be
determined if conditions have changed, e.g., did the standard solution register a
5% increase in a particular solution? - if so then all element concentrations
must be reduced by 5%.

• This is done automatically in most ICP software programs.

14
 Where the Pt concentration changes, the Y concentrations changes in the same direction, i.e., they
mirror each other (Fig 60). However, their ratio remains constant. It is this principle on which an
internal standard and its use is based.

Table 8. Replicates of a Pt bearing solution with Y as internal standard.

Replicate Pt ppm (Y) ppm Ratio

1 90.6 80.6 1.124
2 88.0 78.0 1.128
3 91.5 81.5 1.123
4 88.3 78.3 1.128
5 90.4 80.4 1.124
6 88.1 78.1 1.128
7 88.1 78.1 1.128

Figure 60. As the concentration in Pt changes it is mirrored by the

concentration of Y.
15
A Procedure for Method Validation
Method development is a requirement before validation data are
generated, to ensure optimal results and a realistic measurement of
uncertainty.

Step 1: Method Development and Review

Step 2: Validation using statistical methods

16
 1. Method Development and Review
• Establish mean of analyte of interest in sample types to be analysed.
• Design calibration range around mean analyte value so that mean is
close to centroid value of the regression line, e.g., mean =5.3 ppm,
make the centroid 5 ppm by establishing the range of, 2, 4, 6, 8, 10
ppm.
• Ensure 6 to 10 calibration standards are used.
• Check linearity using SQT test. If the regression data do not support
linearity, remove the highest standard and retest using SQT.

17
See Applications Note 41: A procedure for method validation
 2. Validation using statistical methods
• Check limit of linear response (LOLR).
• Determine LOD, LOQ and MLOD and MLOQ.
• Prepare a set of independent standards (IS); these can be either natural
materials e.g., the analyte bearing ore, or synthetically prepared standards of
known concentration. By independent we mean that they are not part of the
calibration and preferably of a different batch number and or supplier.
• The concentration of these standards needs to be known accurately. Calculate
the expected concentration of the standards taking into consideration the
certified concentration of the stock solution (and its associated uncertainty),
and all propagated uncertainties from volumetric glassware used.

18
See Applications Note 41: A procedure for method validation
 2. Validation using statistical methods
• Standards: prepare near LOQ, 1/3, centroid 2/3 and at or near LOLR.
• Measure each at least six times.
• Evaluation of potential outliers.
• Use Grubbs test for a single outlier in the assessment of potential
outliers for the results generated in the evaluation of accuracy.
• Normality in Distribution
• Kurtosis and Skewness
• Test all replicate data for significance of skewness and kurtosis
(or test the empirical rule on the data for normality).
• Mean divided by Median ~1 for normal distribution

19
See Applications Note 41: A procedure for method validation
 2. Validation using statistical methods
• Accuracy
o Use a one-sample two tailed t-test to validate the accuracy of the
observed mean compared to the expected concentration. Use the
calculated uncertainty of the expected concentration as the uμ in the t-
test formula.
o With accuracy established across the calibration (the result near the
LOQ may not demonstrate accuracy)
• Evaluate any potential bias in the results from the IS’s generated over the
working range.
• Evaluate the % RSD for each of the IS’s using Horwitz.
• Keep all records of measurements made for traceability purposes.
• With accuracy and precision validated across the working range of the
method.
20
End/