See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/226313972

Uncertainty calculations in the certification of reference materials. 2.

Homogeneity study

Article  in  Accreditation and Quality Assurance · January 2001

DOI: 10.1007/s007690000238

CITATIONS READS

144 1,026

3 authors:

Adriaan M. H. van der Veen Thomas P J Linsinger

VSL - Dutch Metrology Institute European Commission
112 PUBLICATIONS   1,934 CITATIONS    126 PUBLICATIONS   2,660 CITATIONS   

SEE PROFILE SEE PROFILE

J. Pauwels
European Commission
130 PUBLICATIONS   3,195 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Nanoparticle Metrology for Health and the Environment View project

Nanotechnology View project

All content following this page was uploaded by Thomas P J Linsinger on 27 May 2014.

The user has requested enhancement of the downloaded file.

Accred Qual Assur (2001) 6 : 26–30
Q Springer-Verlag 2001 GENERAL PAPER

Adriaan M.H. van der Veen Uncertainty calculations in the

Thomas Linsinger
Jean Pauwels certification of reference materials.
2. Homogeneity study

Received: 31 May 2000

Accepted: 29 July 2000
A.M.H. van der Veen (Y)
Nederlands Meetinstituut,
Schoemakerstraat 97, 2600 AR Delft,
The Netherlands
e-mail: avdveen6nmi.nl
Tel.: c31-15-2691 733
Fax: c31-15-261 2971
T.P. Linsinger 7 J. Pauwels
European Commission, Joint Research
Centre, Institute for Reference Materials
and Measurements, Retieseweg,
2440 Geel, Belgium
Abstract Many reference materials
undergo a batch certification,
which implies that a small number
of samples is taken from a batch,
characterised, and these results are
then assumed to be representative
of all remaining samples. An im-
portant aspect in this design is the
translation of the characterisation
data to a single sample, as usually
the laboratory will be using only
one sample of the batch. This form
of homogeneity is very important
and can be influenced to a certain
extent by well-designed sample
preparation procedures. Another
subsampling problem associated
with many reference materials is
that only a small test portion is
drawn from the sample to carry
out the measurement. Obviously,
this test portion must be represent-
ative of the sample, otherwise the
certified value is still not applica-
ble. Both kinds of homogeneity
tests are examined in the paper
and evaluated using practical ex-
amples.
Keywords Reference materials 7
Measurement uncertainty 7
Analysis of variance 7
Homogeneity study 7 Minimum
sample intake

Within-bottle homogeneity plays a role if a subsam-

Introduction
Homogeneity testing is a well-known phenomenon in
the preparation and certification of reference materials.
It also finds its application in the preparation and
checking of proficiency testing material. The design of
the homogeneity study is the same for both between-
bottle and within-bottle testing, as the same question
needs to be answered. For a “between-bottle” homo-
geneity test, where the differences among samples are
of interest, the problem can be stated as follows. What
do these differences contribute to the uncertainty of the
characterisation of material, taking into consideration
that the user of the material will only be using one sam-
ple at a time? The name “between-bottle” has been
chosen, as most reference materials (and proficiency
testing materials) are shipped in bottles; however, the
whole discussion is also valid for vials, gas cylinders,
and test pieces, to give a few examples.
pling step is required. For instance, for measuring a gas
mixture shipped in a cylinder, a subsample is taken, so
within-bottle homogeneity is an issue here. In this case,
as in most other cases of solutions and mixtures, the
heterogeneity in the bottle can effectively be removed
by shaking, rolling or some other handling that allows
the mixture to become homogeneous again. For parti-
culate materials, the problem is different. Apart from
reprocessing the material, there is no option of improv-
ing the within-bottle homogeneity. A certain minimum
amount of the sample must be taken, so that the test
portion represents the contents of the package. In
terms of measurement uncertainty, the smaller the
amount taken the higher the measurement uncertainty.
Thus, this would lead to a situation where the uncer-
tainty of the characterisation of the material is no long-
er valid for the test portion, even in the case where the
between-bottle homogeneity has been taken into con-
sideration.

Experimental set-up and the relationship with analysis
of variance (ANOVA)
A typical experimental set-up for a between-bottle ho-
mogeneity study is visualised in Fig. 1. On the left-hand
side, the case is given where subsampling is impossible,
or just not done. With test pieces, often only one test is
possible, so in this case, n, the number of replicates,
equals 1. In those cases where the sample allows multi-
ple measurements after transformation, n will generally
be greater. In those cases where n 1 1, the data can be
treated with ANOVA [1]. In this paper, a few examples
will be given.
The effect of between-bottle homogeneity is in the
variance “among groups” [1], as well as the effect of the
transformation of the sample. The variance “within
groups” [1] covers only the repeatability of the meas-
urement. In contrast, if from each sample of the batch
multiple test portions are taken, the variance “within
groups” will cover the measurement, transformation,
and subsampling. In this case, the variance “among
groups” only covers the between-bottle heterogeneity.
From the perspective of obtaining an unbiased esti-
mate, this is the ideal situation.
For obvious reasons, it is not possible to obtain an
exact estimate of the variance resulting from within-
bottle heterogeneity. From a sample, multiple test por-
tions must be drawn, which can obviously only be
transformed once (Fig. 2).
This implies that the variance resulting from within-
bottle heterogeneity will always be conservative, which
is not really a problem. For the computation of the
minimum sample intake [2], this implies in turn that the
minimum sample intake will be overestimated, which is
not a problem either. The amount of material stated
will be greater than the critical amount, the amount ac-
Fig. 1 Lay-out of a between-
bottle homogeneity study
27
tually needed to observe no effect of the sample intake
on the repeatability of a measurement. However, the
sample intake for the study to establish the minimum
sample intake differs from the sample intake for a be-
tween-bottle study: in the latter case the optimal sam-
ple intake should be chosen to obtain a good repeata-
bility, in the former, the minimum sample intake must
be used. Doing so will increase influences of within-
bottle heterogeneity and will thus reduce the influence
of method repeatability on the estimated minimum
sample intake.
Current practice versus uncertainty-based practice
In many cases, the well-known F-test [3–5] is still used
to test for significance of a heterogeneity effect. Apart
from the problems already stated in Part 1 [1], there is
another big problem: this way of treating homogeneity
data does not answer the question(s) raised in the in-
troduction. In principle, it does not matter whether the
heterogeneity observed is significant with respect to the
repeatability of the test method. The repeatability of
the test method is only of interest in the sense of the
quality of what has been demonstrated. If the F-test
does not indicate a significant result from the homo-
geneity test, but the repeatability of the test method
used is poor, then it is still possible that the heterogene-
ity among samples (between-bottle homogeneity) is sig-
nificant when compared to the uncertainty from charac-
terisation.
As pointed out by Pauwels et. al. [6] and by Van der
Veen and Alink [7], a method used for assessing sam-
pling and/or subsampling performance, including ho-
mogeneity tests, should have a good repeatability. The
repeatability of the test method, in conjunction with the
number of replicates on each sample, defines the reso-
28
Fig. 2 Lay-out of a within-bottle homogeneity study
lution of the method. The better the resolution, the
smaller the effects that can be estimated.
Looking from the perspective of the “Guide to the
expression of uncertainty in measurement” (GUM) [8],
the variation of the bottle averages (uc(bb)) is a com-
bined uncertainty consisting of the between-bottle het-
erogeneity (sbb) and the measurement variation (smeas).
The latter comprises analytical variation and within-
bottle heterogeneity, which should be pooled for the
estimation of between-bottle heterogeneity anyway.
The relationship between uc(bb), sbb and smeas can be ex-
pressed as [6]
u 2c(bb)ps 2bbcs 2meas (1)
which implies that
s 2bbpu 2c(bb)Ps 2meas (2)
Note that uc(bb), sbb and smeas were named uexp, ubetw and
umeas, respectively in [6]. smeas is the analytical variation
divided by the square root of the number of replicates
per bottle. By increasing the number of replicates per
bottle a small smeas can be obtained even for methods
with poor repeatability, thus allowing a good estima-
tion of sbb, the estimate of between-bottle variation
sought for.
Equation (1) obviously cannot be used if the varia-
tion of the measurement is large compared to the heter-
ogeneity, without looking at the repeatability standard
deviation of the measurements, smeas. In Part 1 [1], it
has been demonstrated that the variance “among
groups” is affected by the variance within groups. This
means that for small values of sbb a problem of quantif-
ication arises. There are two principle choices to deal
with this
1. Acceptance of the value for sbb, even if it is zero,
because the samples are expected to be homogene-
ous (e.g. solutions) or known from previous experi-
ence to have negligible heterogeneity when pre-
pared properly
2. Application of a more conservative estimation tech-
nique for this uncertainty source, based on sbb as ob-
served and smeas.
Both options comply with GUM, and apart from the
expectations about the heterogeneity aspect, smeas
should also fulfill the requirement, to be smaller than
the repeatability standard deviation of the measure-
ments in the characterisation. If this requirement is not
clearly fulfilled, then in any case a more conservative
estimator than the value of sbb as observed from
ANOVA is necessary. This is an effect of the existing
correlation between both standard deviations. This top-
ic will be covered in more detail in Part 4, about the
certification process, as it also has consequences for the
treatment of stability data.
For within-bottle homogeneity studies, a similar rea-
soning can be developed. The uncertainty from the ex-
periment can be expressed as [6]
u 2c(wb)ps 2wbcs 2method (3)
which leads to
s 2wbpu 2c(wb)Ps 2method (4)
with uc(wb) being the combined standard uncertainty of
the experiment, swb the within-bottle variation and
smethod the intrinsic variability of the method. These
terms were named uexp, uinh and umeas,. respectively in
[6]. The second term of this expression differs from that
of Eq. (1) due to the difference in experimental design:
Usually, smethod cannot be determined independently, as
this would require a material of the same type with per-
fect within-unit homogeneity, which renders estimation
of swb impossible. In these cases, uc(wb) must be used to
estimate the minimum sample intake. To diminish the
influence of smethod as much as possible, a sample intake
should be chosen for which swb is much larger than
smethod. Examples for this approach for (trace) elements
are solid sampling (SS) techniques like solid sampling
SS-ETAAS (Electrothermal Atomic Absorption Spec-
trometry) or solid sampling inorganic inductively cou-
pled plasma-mass spectrometry (SS-ICP-MS). In any
case, swb is not part of the uncertainty of the certified
reference material, as will be explained in Part 4. It is
only needed to establish the minimum sample intake
for which the stated uncertainty is valid.
Evaluation of a between-bottle homogeneity study
For a clay soil sample, 18 samples were taken out of a
batch for an homogeneity study on barium. The results,
expressed in mg/kg on a dry basis are given in Table 1
[Van Son M, Van der Veen AMH, Verkuil D, unpub-
 29

Table 1 Homogeneity study of barium in soil tle. The link between Eqs. (1and 5) is as follows:
MSamong is equal to n times u 2c(bb) and MSwithin is equal
Sample Data #1 Data #2 Data #3 Mean s n
to s 2method (see part 1 of this paper). As smeaspsmethod/;n,
# 0118 323 301 310 311 11 3 the equivalence of Eqs. (2 and 4) follows. As it can be
# 0201 340 334 316 330 12 3 seen, the computation of the grand mean is not needed
# 0383 320 321 309 317 7 3 for this uncertainty evaluation, although Excel internal-
# 0442 315 338 321 325 12 3
# 0557 326 338 325 330 7 3
ly will calculate this parameter.
# 0666 325 302 304 310 13 3
# 0791 324 331 317 324 7 3
# 0918 310 310 331 317 12 3 Evaluation of a between-bottle homogeneity study in an
# 1026 336 321 328 328 8 3
# 1133 310 328 312 317 10 3
alternative format
# 1249 314 314 302 310 7 3
# 1464 329 300 299 309 17 3 In the experimental set-up described above, method re-
# 1581 320 329 311 320 9 3 peatability and between-bottle heterogeneity were esti-
# 1607 322 312 311 315 6 3 mated by analysing several units n-times each. Fre-
# 1799 332 317 299 316 17 3
# 1877 313 294 293 300 11 3 quently a different approach to obtain estimates for
# 1996 324 314 335 324 10 3 smeas and sbb is used: one unit is analysed several times
# 2000 321 342 316 327 14 3 to obtain an estimate for smethod and several units are
then analysed in one replicate each to obtain an esti-
mate of uc(bb). For the estimation of the between-bottle
Table 2 Analysis of variance (ANOVA) table for barium in soil variability by this approach, it is vital that the results
from one unit and from the different units are obtained
Source of variation SS df MS F P-value Fcrit
by the same technique using the same sample intake.
Between groups 3467 17 204 1.66 0.10 1.92 As np1 in this case, smeaspsmethod and sbb can be esti-
Within groups 4412 36 123 mated according to Eq. (1).
Total 7880 53 For the certification of a mussel-tissue material [9],
ten units were used for the homogeneity study on sele-
nium. Five determinations on one unit were performed,
whereas the other nine units were analysed once. All
lished data]. The measurements were carried out on ex- analyses were done by k0–NAA without sample pre-
tracts obtained from aqua regia digestion using NEN treatment. The results of this study are given in Table 3.
6465, and the measurements were carried out using In this case, sbb amounts to 2.84%, which is the uncer-
ICP-MS. Using Excel, the following ANOVA table tainty contribution of homogeneity to the uncertainty
(one-way layout) can be computed (Table 2). The co- of one bottle.
lumn “SS” provides the sums of squares, the column Obviously, this method requires fewer measure-
“df ” the associated degrees of freedom, and the co- ments than performing the complete matrix of measur-
lumn “MS” the mean squares, which form the basis for ements for the ANOVA evaluation. This advantage is
the computation of variances as discussed in Part 1 [1]. paid for with less significant results, as measurement re-
The F-test indicates that the result of the homogeneity
is insignificant (F~Fcrit, the critical value of F for
ap5%). The P-value gives the level for which the ob- Table 3 Homogeneity study for selenium in mussel tissue
served F equals Fcrit.
5 replicates Results from
The calculation of uncertainties is now very straight- from one unit 9 different
forward. The repeatability of the test method is just the [mg/kg] units
square root of MSwithin, equal to 11 mg/kg (p3,5%). [mg/kg]
For the variance among groups, the following expres-
1.907 1.872
sion can be used 1.917 1.874
MSamongPMSwithin 1.961 1.928
s 2Aps 2bb p (5) 1.901 1.833
n 1.834 1.944
1.840
This equation can be used instead of the formulas from 1.726
Part 1, as in this case the data matrix is complete (all 1.952
groups have the same number of members, np3). The 1.861
variance is 27 mg 2/kg 2; the standard deviation between Average 1.904 1.870
Standard deviation 0.046 0.070
bottles (sbb) is 5.2 mg/kg (p1.6%), which is the contri- Variation coefficient 2.40% (smeas) 3.72% (uc(bb))
bution of heterogeneity to the uncertainty of one bot-
30

peatability is not reduced by replication. The effect is
that smeas is more likely to be larger than uc(bb), leading
to the problems already addressed about sbb values
tending to zero. This format may therefore lead to a
greater uncertainty for the certified reference material.
As in many cases a more conservative value for the un-
certainty due to between-bottle variation must be in-
serted rather than the value of sbb as obtained directly
from ANOVA. On the other hand, this example also
demonstrates how to conduct a homogeneity test in
cases where repetition of measurements is impossible.
In these cases, the results in the first column of Table 2
must be obtained from other sources (“Type B evalua-
tion” [8]), such as the quality manual of the laboratory,
validation data or some other source.
Conclusions
A general framework for the estimation of within- and
between-bottle heterogeneity has been developed. The
approach consists of correcting an estimation of hetero-
geneity, impaired by measurement variability, for
measurement variability. Reliable estimates for be-
tween- and within-bottle heterogeneity can be obtained
given that the measurement variability is small com-
pared to the heterogeneity to be detected. If the re-
quirement of low measurement variability compared to
heterogeneity is not met, more conservative ap-
proaches should be employed.
The method as such works equally well on the fully
nested ANOVA designs of experiments, and on other
formats. The underlying theory and concepts are the
same, but the implementation differs. This allows appli-
cation of the method for homogeneity tests on test
pieces in destructive testing as well, which has great
benefits.
This work also shows that carrying out homogeneity
tests cannot and must not be separated from other
parts of the certification project (e.g. stability studies,
characterisation measurements), as the accuracy of the
measurements in the homogeneity study have impor-
tant implications on the establishment of the combined
standard uncertainty of the candidate reference materi-
al.

References
1. Van der Veen AMH, Pauwels J(2000)
Accred Qual Assur 5 : 464–469
2. Pauwels J, Kurfürst U, Grobecker KH,
Quevauviller P (1993) Fresenius J
Anal Chem 345 : 478–481
3. ISO Guide 35 : 1989 (1989) Certifica-
tion of reference materials – General
and statistical principles, 2nd edn. In-
ternational Organization for Standardi-
zation (ISO), Geneva, Switzerland
4. Schiller SB (1996) Statistical aspects of
the certification of chemical batch
SRMs. NIST Special Publication
260–125. NIST, Gaithersburg, USA
5. BCR Guidelines (1994) Standards,
Measurement and Testing Programme,
Brussels, Belgium
6. Pauwels J, Lamberty A, Schimmel
H(1998) Accred Qual Assur 3 : 51–55
7. Van der Veen AMH, Alink A (1998)
Accred Qual Assur 3 : 20–26
8. BIPM, IEC, IFCC, ISO, IUPAC, IU-
PAP, OIML (1995) Guide to the ex-
pression of uncertainty in measure-
ment, 1st edn. ISO Geneva, Switzer-
land
9. Lamberty A, Muntau H (1999) The
certification of the mass fractions of
As, Cd, Cr. Cu, Hg, Mn, Pb, Se and
Zn in mussel tissue Mytilus edulis.
EUR 18840EN

View publication stats