Professional Documents
Culture Documents
Waters Proficiency Testing Program: Report No. 1119
Waters Proficiency Testing Program: Report No. 1119
1119
December 2018
Acknowledgments
PTA wishes to gratefully acknowledge the technical assistance provided for this program by
Dr M Buckley-Smith, Global Proficiency Ltd (New Zealand). Also our thanks go to Global
Proficiency Ltd (New Zealand) and to Global Proficiency Pty Ltd (Australia) for the supply
and distribution of the samples.
SD 9.17.12
CONTENTS
1. Foreword ........................................................................................................................ 1
2. Program Features and Design ........................................................................................ 1
3. Statistical Format ............................................................................................................ 2
4. PTA and Technical Adviser’s Comments ........................................................................ 4
5. Outlier Results ...............................................................................................................10
6. References ....................................................................................................................10
APPENDIX C – Documentation
SD 9.17.12
1
1. Foreword
The exercise was conducted in October 2018 by Proficiency Testing Australia (PTA).
The main aim of the program was to assess laboratories’ abilities to competently
perform the prescribed analyses.
The Program Coordinators were Dr M Li and Mrs D Mihaila and the Technical Adviser
was Dr M Buckley-Smith, Global Proficiency Ltd (New Zealand). This report was
authorised by Mrs K Cividin, PTA Quality Manager.
2.1 Each laboratory was randomly allocated a unique code number for the program to
ensure confidentiality of results. Reference to each laboratory in this report is by code
number only.
2.2 Laboratories were provided with the "Instructions to Participants" and "Results Sheet"
(see Appendix C). Laboratories were requested to perform the tests according to their
routine methods.
2.3 Participants were provided with two glass vials (labelled PTA 1 and PTA 2) for
analysis of Total Recoverable Oil and Grease.
Of these 34 laboratories, five were unable to submit results by the due date.
2.5 Results (as reported by participants) with corresponding summary statistics (i.e.
number of results, median, normalised interquartile range, uncertainty of the median,
robust coefficient of variation, minimum, maximum and range) are presented in
Appendix A (for each sample and for each of the analyses performed).
2.6 A robust statistical approach, using z-scores, was utilised to assess laboratories’
testing performance (see Section 3). Robust z-scores and ordered z-score charts
relevant to each test are presented in Appendix A.
The document entitled Guide to Proficiency Testing Australia, 2016 (reference [1])
defines the statistical terms and details the statistical procedures referred to in this
report.
SD 9.17.12
2
2.7 A tabulated listing of laboratories (by code number) identified as having outlier results
can be found on page 10.
2.8 Prior to sample distribution, a number of randomly selected samples were analysed
for homogeneity and stability. Based on the results of this testing (see Appendix B) it
was considered that the samples utilised for this program were homogeneous and
stable. As such, any results later identified as outliers could not be attributed to any
notable sample variability.
3. Statistical Format
Each determination was examined for outliers with all methods pooled. The table on
page 10 summarises the outlier results detected.
The tables in Appendix A contain the results returned by each laboratory, including
the code number for the method used and the robust z-score calculated for each
result.
Results have been entered exactly as reported by participants. That is, laboratories
which did not report results to the precision (i.e. number of significant figures)
requested on the Results Sheet have not been rounded to the requested precision
before being included in the statistical analysis.
SD 9.17.12
3
A list of summary statistics appears at the bottom of each of the results tables and
consists of:
For normally distributed data, the uncertainty of the median is approximated by:
𝜋 𝑛𝑜𝑟𝑚𝐼𝑄𝑅
√ ×
2 √𝑛
𝑛 = number of results.
Please see reference [1] for further details on these robust summary statistics.
These charts contain solid lines at +3.0 and -3.0, so that outliers are clearly
identifiable as those laboratories whose "bar" extends beyond these "cut-off" lines.
The y-axis of these charts has been limited, so very large z-scores appear to extend
beyond the chart boundary.
SD 9.17.12
4
Consensus values (median) derived from participants’ results are used in this
program. These values are not metrologically traceable to an external reference.
Sample PTA 1 was prepared from high purity mineral oil and stearic acid which has a
low viscosity in distilled water. Sample PTA 2 was prepared from a mixture of
vegetable oil, anhydrous milk fat (AMF) and stearic acid in distilled water. Samples
were prepared as ready-to-test (500 mL) samples, packaged in Teflon capped amber
glass bottles and preserved with acid to pH 2.
Solutions were stable and homogeneous, and medians obtained from this proficiency
round indicated average recoveries of 94.9% for sample PTA 1 and 86.8% for sample
PTA 2, compared to the expected levels (dope concentration), as shown in Table 1.
These are comparable to published precision data, which indicated that laboratories
should be able to yield an average recovery of 93% with the Liquid-Liquid Partition
Gravimetric method (APHA 5520 B), a recovery of 87.1% - 92.5% with the Solid
Phase Partition Gravimetric method (APHA 5520 B), and a recovery of 98.7% for
Soxhlet Extraction method (APHA 5520 D).
Table 1. Comparison of expected levels (dope concentration) and proficiency medians. The
values of the calculated uncertainty of the median are also presented.
Dope
Uncertainty
Concentration Median
Analysis Sample of the median
± 2SD (mg/L)
(mg/L)
(mg/L)
As the assigned value for each sample in this program is the median of the results
submitted by the participants, the uncertainty of the median for each sample has
been calculated and is also presented in the Table 1.
SD 9.17.12
5
Table 2 compares the Total Recoverable Oil and Grease median and robust CV from
this round to those obtained in previous PTA rounds. Both samples contained
sufficient oil and grease (>10 mg/sample) to enable testing using the gravimetric
methods. CV’s used in this study agreed well with published precision data from
APHA 5520 G Solid Phase Partition Gravimetric method which indicated that CVs
ranging between 17%-38% could be expected for various wastewater types. APHA
indicated laboratories should achieve CVs closer to 8.7% for Liquid-Liquid Partition
Gravimetric method (APHA 5520 B) or 1.86% for Soxhlet Extraction method (APHA
5520 D).
CVs for the general population of methods in this round (Table 2) were much wider
for PTA 1 than those produced by the dominant method APHA 5520 B (Liquid-Liquid,
Partition-Gravimetric Method - method code 1); which produced CVs of 17.8% and
17.4% for samples PTA 1 and PTA 2, respectively (see Table 4 in section 4.4).
Table 2. Comparison of current round variability and proficiency median of Total Recoverable
Oil and Grease testing with the results of the previous two rounds.
Median
Round Sample Robust CV (%) Participants
(mg/L)
PTA 1 76.30 29.7 28
This study
PTA 2 41.80 14.2 29
PTA 1 66.10 17.1 27
Report 1056
PTA 2 51.40 15.7 25
PTA 1 39.40 18.4 25
Report 1002
PTA 2 76.30 9.9 25
Bias / Accuracy
The Total Recoverable Oil and Grease testing was successfully performed, with
statistically satisfactory results (|z-score| ≤ 2.0) ranging between 37.2 – 100 mg/L for
sample PTA 1 and 34.6 – 52.8 mg/L for sample PTA 2.
Out of 28 results submitted for sample PTA 1 and 29 results for sample PTA 2, two
questionable results (2.0 < |z-score| < 3.0) were reported for sample PTA 1
(laboratories 353 and 439) and six questionable results were reported for sample
PTA 2 (laboratories 135, 353, 439, 472, 584 and 712).
Two outlier results (|z-score| ≥ 3.0) were obtained for sample PTA 2, requiring follow-
up action by laboratories 113 and 213. No outlier results were obtained for sample
PTA 1. Due to the tight Quality Control of the sample manufacture, it is
recommended that laboratories carry out follow-up for both questionable and outlier
results in this instance.
The data sets for samples PTA 1 and PTA 2 formed approximately normal
distributions (Figures 1 and 2) around the median. The method most frequently used
for Total Recoverable Oil and Grease testing in this round was APHA 5520 B (Liquid-
SD 9.17.12
6
9
APHA 5520 B
US EPA 1664A
8
APHA 5520 C
ASTM D 7066-04
7
US EPA 413.2
6 APHA 5520 D
Other
Frequency
5 Not Specified
0
8 16 24 32 40 48 56 64 72 80 88 96 104 112
Results (mg/L)
Figure 1. Spread of results for Total Recoverable Oil and Grease testing of sample PTA 1,
with the median of 76.30 mg/L (- - -) and doping concentration of 80.39 mg/L (- - -).
9
APHA 5520 B
US EPA 1664A
8
APHA 5520 C
ASTM D 7066-04
7
US EPA 413.2
APHA 5520 D
6
Other
Frequency
5 Not Specified
0
20 25 30 35 40 45 50 55 60 65 70 75 80 85
Results (mg/L)
Figure 2. Spread of results for Total Recoverable Oil and Grease testing of sample PTA 2,
with the median of 41.80 mg/L (- - -) and doping concentration of 48.15 mg/L (- - -).
SD 9.17.12
7
Of more concern are the low biasing results, where laboratories recovered less than
50% of the oil and grease in the samples, particularly, in the context of wastewater
discharge limits, and fitness for purpose of test methods. Laboratories receiving
questionable and outlier ratings on low biasing results are strongly recommended to
investigate their ability to adequately recover all the oil and grease adhered to the
surfaces of the glass bottle and Teflon lid liner. Laboratories using the Soxhlet
Extraction Method (APHA 5520 D) performed well on both samples. Laboratories
using solvent partition gravimetric methods APHA 5520 B (LLE) and APHA 5520 G
(SPE) may have had a problem with insufficient rinsing of the sample container with
extracting solvent. Closer inspection of the original bottles will likely show a smear of
fatty material still adhering to the side of the glass bottles.
On the other end of the scale, incomplete evaporation of the solvent can cause
erroneous high results for laboratories using the Liquid-Liquid Partition Gravimetric
method (APHA 5520 B). Cooling in a desiccator and weighing to a constant weight is
critical for obtaining acceptable results with this method. Also, if the sample is dried
with sodium sulphate, the salt that passes into the flask becomes a positive
interference in gravimetric methods. See the interferences section (d) in APHA 5520
A for additional instructions if you observe crystals in the flask after drying.
Laboratories using NIR methods (APHA 5520 C, ASTM D 7066-04 and HORIBA)
have the additional difficulty of not knowing what kind of wastewater they are
analysing, and the inherent assumptions they must make when calibrating their
equipment can cause significant error. As mentioned previously, sample PTA 1
contained high purity mineral oil and stearic acid, and sample PTA 2 contained a
mixture of vegetable oil, anhydrous milk fat (AMF) and stearic acid.
In general, the Quality Control (QC) practices identified in Table 5020:I of APHA
section 5000 should be considered an integral part of each method. The use of
Method Blanks (MB), Laboratory Fortified Blanks (LFB), and Laboratory fortified
matrix (LFM) with duplicates (LFMD), are mandatory for all Oil and Grease methods.
APHA recommends that all of these QC practices (MB, LFB, LFM & LFMD) be carried
out daily (as a minimum) or with each batch of 20 samples.
SD 9.17.12
8
The majority of participants in this round (68%) reported the measurement uncertainty
(MU) associated with their results. Table 3 below presents the number and
percentage of laboratories reporting the MU for each sample.
PTA 1 28 19 (68%)
PTA 2 29 20 (69%)
Many of the stated MUs did not accurately reflect the difference between the median
and the participant’s result for these proficiency samples. These laboratories may
have under estimated their MU, as they indicated that their MU was less than two
times the uncertainty of the median, however their results were further from the
median than the MU value.1
In order for methods to be grouped for analysis, PTA requires at least 11 sets of
results from the same method group. Please note that, for methods other than the
one presented below, there were less than 11 results submitted for each method and
reliable conclusions cannot be drawn from analysing them on this occasion.
Table 4 below presents the median and uncertainty of the median for results obtained
by method 1. Results for PTA 1 were considerably tighter than for the overall pool of
results, and method results for PTA 2 were more variable than the overall dataset.
1
MU evaluation is based on minimum / maximum uncertainty criteria (umin and umax)
described in ISO 13528:2015 [2]. It should be noted, however, that these are
informative indicators only and cannot be solely used to validate or invalidate the MUs
reported.
SD 9.17.12
9
The statistics presented in Table 4 and doping concentration from Table 1 have been
used to illustrate the variability of results in Figures 3 and 4.
Figure 3. Spread of laboratory results (●) for Total Recoverable Oil and Grease testing of
sample PTA 1, where laboratory MU show as error bars, the median is (───), the std error of
median is (- - -), dope concentration is (───), 2x NIQR is (- - - ), and 3x NIQR is (- - - ).
Figure 4. Spread of laboratory results (●) for Total Recoverable Oil and Grease testing of
sample PTA 2, where laboratory MU show as error bars, the median is (───), the std error of
median is (- - -), dope concentration is (───), 2x NIQR is (- - - ), and 3x NIQR is (- - - ).
The median for sample PTA 2 (45.8 mg/L) was much closer to the doping
concentration (48.15 mg/L) than the median produced by the general pool statistics
(41.8 mg/L). This method’s standard error of the median also encompassed the
doping concentration, whereas the general pool did not.
SD 9.17.12
10
5. Outlier Results
Laboratories reporting results that have been identified as outliers are listed in Table
5 below.
113 §
213 §
Note:
1. A “§” indicates the occurrence of a z-score outlier result (i.e. those results for which
|z-score| ≥ 3.0).
6. References
[1] Guide to Proficiency Testing Australia, 2016 (This document can be found on
the PTA website, www.pta.asn.au)
SD 9.17.12
APPENDIX A
SD 9.17.12
Total Recoverable Oil and Grease
Results
SD 9.17.12
A1
Sample PTA 1
Laboratory Code 1
Result ± MU Robust Method
2 3
mg/L z-score Code
113 54.9 ± 1.0% -0.94 4
120 54 ± 5.4 -0.98 3
135 39.4 ± 0.1 -1.63 1
140 100 ± 20 1.05 5
151 76.2 ± 15 0.00 9
165 99.8 ± 2.9 1.04 5
213 86.0 # 0.43 1
237 78.4 ± 16 0.09 9
347 94.5 ± 5.7 0.80 7
353 23.4 # -2.33 10
356 92.5 ± 9.2 0.71 4
390 79.2 ± 1.5 0.13 1
427 38.0 ± 1.9 -1.69 1
439 22.4 # -2.38 #
447 77 ± 1 0.03 1
472 37.2 # -1.73 1
508 45.0 ± 9.0 -1.38 1
510 70.2 # -0.27 9
584 89.9 # 0.60 1
603 74.3 ± 7.4 -0.09 1
634 82.6 ± 13.3 0.28 1
664 76.4 ± 11.2 0.00 1
665 69.6 ± 6.2 -0.30 10
686 72.3 ± 1.5 -0.18 1
1
Where reported, results are shown with their corresponding measurement
uncertainty (MU).
2
"§" denotes an outlier (i.e. those results for which |z-score| ≥ 3.0). Robust z-scores
are calculated as: z = (A - median) ÷ normalised IQR, where A is the participant
laboratory's result.
3
Please refer to Appendix C (page C3) for method code descriptions.
4
"na" indicates "not applicable".
5
"#" indicates that no result was returned for this sample/test.
SD 9.17.12
A2
Sample PTA 1
Laboratory Code 1
Result ± MU Robust Method
2 3
mg/L z-score Code
699 83.5 ± 20.9 0.32 1
712 85 # 0.38 1
729 73.0 # -0.15 1
736 87.4 # 0.49 1
No of Results: 28
Median: 76.30
Normalised IQR: 22.67
Uncertainty of the Median: 5.37
Robust CV: 29.7%
Minimum: 22.4
Maximum: 100
Range: 77.6
1
Where reported, results are shown with their corresponding measurement
uncertainty (MU).
2
"§" denotes an outlier (i.e. those results for which |z-score| ≥ 3.0). Robust z-scores
are calculated as: z = (A - median) ÷ normalised IQR, where A is the participant
laboratory's result.
3
Please refer to Appendix C (page C3) for method code descriptions.
4
"na" indicates "not applicable".
5
"#" indicates that no result was returned for this sample/test.
SD 9.17.12
Total Recoverable Oil and Grease - Sample PTA 1 - Robust Z-Scores
5
4
151
603
729
686
510
665
-1
113
120
135
427
472
353
439
-3
-4
-5
lab code
A3
Robust Z-Scores
5
4
3
2
1
z-score
0
-1
-2
-3
-4
-5
lab code
SD 9.17.12
A4
Sample PTA 2
Laboratory Code 1
Result ± MU Robust Method
2 3
mg/L z-score Code
113 73.4 ± 1.0% 5.33 § 4
120 40 ± 4.0 -0.30 3
135 26.0 ± 0.1 -2.66 1
140 46.5 ± 5 0.79 5
151 40.6 ± 8.1 -0.20 9
165 39.4 ± 2.0 -0.40 5
213 82.0 # 6.78 § 1
237 47.4 ± 9.9 0.94 9
347 41.8 ± 2.5 0.00 7
353 24.3 # -2.95 10
356 40.3 ± 4.0 -0.25 4
390 39.0 ± 0.8 -0.47 1
427 41.5 ± 2.1 -0.05 1
439 24.2 # -2.97 #
447 40 ± 1 -0.30 1
472 25.9 # -2.68 1
497 34.6 ± 7.2 -1.21 5
508 51.6 ± 10.3 1.65 1
510 40.5 # -0.22 9
584 57 # 2.56 1
603 44.8 ± 4.5 0.51 1
634 49.0 ± 13.3 1.21 1
664 46.8 ± 6.9 0.84 1
665 38.2 ± 3.4 -0.61 10
1
Where reported, results are shown with their corresponding measurement
uncertainty (MU).
2
"§" denotes an outlier (i.e. those results for which |z-score| ≥ 3.0). Robust z-scores
are calculated as: z = (A - median) ÷ normalised IQR, where A is the participant
laboratory's result.
3
Please refer to Appendix C (page C3) for method code descriptions.
4
"na" indicates "not applicable".
5
"#" indicates that no result was returned for this sample/test.
SD 9.17.12
A5
Sample PTA 2
Laboratory Code 1
Result ± MU Robust Method
2 3
mg/L z-score Code
686 47.0 ± 1.3 0.88 1
699 44.6 ± 11.2 0.47 1
712 58 # 2.73 1
729 42.2 # 0.07 1
736 52.8 # 1.85 1
No of Results: 29
Median: 41.80
Normalised IQR: 5.93
Uncertainty of the Median: 1.38
Robust CV: 14.2%
Minimum: 24.2
Maximum: 82.0
Range: 57.8
1
Where reported, results are shown with their corresponding measurement
uncertainty (MU).
2
"§" denotes an outlier (i.e. those results for which |z-score| ≥ 3.0). Robust z-scores
are calculated as: z = (A - median) ÷ normalised IQR, where A is the participant
laboratory's result.
3
Please refer to Appendix C (page C3) for method code descriptions.
4
"na" indicates "not applicable".
5
"#" indicates that no result was returned for this sample/test.
SD 9.17.12
Total Recoverable Oil and Grease - Sample PTA 2 - Robust Z-Scores
5
4
427
151
510
356
120
447
-1
165
390
665
135
472
353
439
-4
-5
lab code
A6
Robust Z-Scores
5
4
3
2
1
z-score
0
-1
-2
-3
-4
-5
lab code
SD 9.17.12
APPENDIX B
SD 9.17.12
B1
Samples for this program were obtained from Global Proficiency Ltd, New Zealand. As such,
all samples are subjected to rigorous quality control and homogeneity / stability testing.
Sample PTA 1 was doped with 40.19 ± 0.56 mg (95% CI) of mineral oil and stearic acid in a
500mL bottle, and sample PTA 2 was doped with 24.08 ± 0.41 mg (95% CI) of a mixture of
vegetable oil, anhydrous milk fat and stearic acid. All samples were acidified to pH < 2 using
H2SO4.
A random selection of ten samples was chosen from samples PTA 1 and PTA 2 for
homogeneity and stability testing. Seven of these were stored chilled at 4°C and the
remaining three were subjected to 35°C for three days, for an accelerated ageing stability
trial. Samples were then analysed for Oil and Grease by Hill Laboratories, New Zealand,
using method APHA 5520 D (modified) 22nd Ed. 2012.
From statistical analyses based on the results of this testing and rigorous quality control, it
was considered that all samples were sufficiently homogeneous and stable, so that any
results later identified as outliers should not be attributed to any notable sample variability.
The results of homogeneity and stability testing are presented in Table B1 below. Please
note that the mean results for these tests are not intended to be used as reference values.
Table B1. Homogeneity and stability testing of PTA 1 and PTA 2 samples.
Total Recoverable Oil and Grease
Round (g/m3)
PTA 236 Sample Sample PTA 1 Sample PTA 2
ID Test Result Quality Control Test Result Quality Control
Homogeneity H1 81.0 79.8 49.0 48.6
H2 80.0 80.2 49.0 48.6
H3 83.0 80.2 52.0 48.6
H4 77.0 80.0 49.0 48.0
H5 79.0 80.6 48.0 48.6
H6 76.0 81.6 50.0 48.6
H7 79.0 80.6 46.0 47.8
Stability S1 83.0 80.6 49.0 48.6
S2 78.0 79.8 46.0 48.2
S3 82.0 81.6 50.0 48.2
Median 79.5 80.4 49.0 48.6
RSD 3.1% 0.8% 3.7% 0.6%
SD 9.17.12
APPENDIX C
Documentation
SD 9.17.12
C1
INSTRUCTIONS TO PARTICIPANTS
**Please record (on the Results Sheet) the approximate temperature of the samples upon
receipt**
Please note the following before commencing the analysis of the samples.
1. Samples
i) Two 500 mL glass bottles, labelled PTA 1 and PTA 2, supplied by Global Proficiency Ltd.
The bottles contain oil and grease (in water) in the range of 10-50 milligrams (mg). These
have been acid preserved and should be refrigerated until ready to test.
ii) The sample must be thoroughly mixed prior to analysis. Please take extra care as the
sample may be strongly adhered to the glass neck and shoulder of the bottle.
Please Note: Where possible, proficiency testing samples should be treated as a routine
laboratory sample.
2. Sample Preparation
3. Tests Requested
If unable to perform the above please note this on your Results Sheet.
SD 9.17.12
C2
4. Safety
ii) Participants should have sufficient experience and training to take the necessary precautions
when handling the samples and reagent chemicals and during disposal.
iii) Use of safety glasses, gloves, and fume hoods, where appropriate during the
determinations, is recommended.
5. Reporting
iv) Select the appropriate method code from the Method Code Table and record it on the
Results Sheet.
v) Calculate the measurement uncertainty (MU) for each reported result. All estimates of MU
must be given as a 95% confidence interval (coverage factor k ≈ 2) and reported in mg/L.
Report MU using the same number of decimal places as for the result.
6. Testing should commence as soon as possible after receiving the samples and results reported
NO LATER THAN 9 NOVEMBER 2018 to:
Delfina Mihaila
Proficiency Testing Australia
PO Box 7507
SILVERWATER NSW 2128
AUSTRALIA
Phone: +612 9736 8397
Fax: +612 9743 6664
Email: dmihaila@pta.asn.au
7. For this program your laboratory has been allocated the code number shown on the attached
Results Sheet. All reference to your laboratory in reports associated with the program will be
through this code number, thus ensuring the confidentiality of your results.
Analyte Range
Total Recoverable Oil and Grease 20 – 100 mg/L
SD 9.17.12
C3
Hexane
US EPA 1664A Oil & Grease (HEM/SGT-HEM)
Extraction & 3
by extraction
Gravimetry
Soxhlet
APHA 5520 D. Soxhlet Extraction Method 9
Extraction
i) APHA APHA “Standard Methods for the Examination of Water and Wastewater” (18, 19, 20,
21, 22, 23 Edition) (http://www.standardmethods.org/).
ii) ASTM American Society for Testing and Materials, Annual Book of ASTM Standards, Vol.
11.01 and Vol. 11.02.
SD 9.17.12
C4
OCTOBER, 2018
RESULTS SHEET
(mg/L)
Laboratory
Code
Please note: Where possible, proficiency testing samples should be treated as a routine laboratory
sample.
SD 9.17.12
- End of Report -
SD 9.17.12