See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/291692620

A simple matrix of analytical performance to identify assays that risk patients

using External Quality Assurance Program data

Article in Clinical Biochemistry · January 2016

DOI: 10.1016/j.clinbiochem.2016.01.014

CITATIONS READS

9 1,710

3 authors, including:

Tony Badrick
The Royal College of Pathologists of Australasia
285 PUBLICATIONS 2,396 CITATIONS

SEE PROFILE

All content following this page was uploaded by Tony Badrick on 15 April 2019.

The user has requested enhancement of the downloaded file.

 Clinical Biochemistry 49 (2016) 596–600

Contents lists available at ScienceDirect

Clinical Biochemistry

journal homepage: www.elsevier.com/locate/clinbiochem

A simple matrix of analytical performance to identify assays that risk

patients using External Quality Assurance Program data
Mark Mackay a, Gabe Hegedus b, Tony Badrick a,⁎
a
RCPA QAP, St Leonards, Sydney, NSW, Australia
b
Lismore Laboratory, Pathology North, Lismore, NSW, Australia

a r t i c l e i n f o a b s t r a c t

Article history: Objectives: We propose a simple way to reliably rank assays for improvement according to patient risk, based
Received 28 July 2015 solely on EQA imprecision and biological variation data. Because the underlying technique aligns the imprecision
Received in revised form 13 January 2016 class of an assay from EQA data, peer performance can be used to assess achievable imprecision and the risk rank-
Accepted 16 January 2016 ing can not only prioritise improvement but also highlight laboratory QC operating parameters that are easy to
Available online 1 February 2016
manage and provide reliable, acceptable performance.
Design and methods: A modified Failure Modes Effects Analysis (FMEA) is applied to produce an analyte risk
Keywords:
Quality Control
rating based on three factors, each of which is graded: 1) the ease of detecting analytical errors based on the ratio
External quality assurance of allowable limits of performance to imprecision (Assay Capability) compared to absolute standards and to
Assay Capability peers, 2) the predicted frequency of errors in patient monitoring based on the ratio of within-individual biolog-
Risk ical variation to laboratory imprecision, and 3) the clinical importance of the assay as a surrogate marker for harm
FMEA arising from an error.
Results: We provide laboratory examples to illustrate these models.
Conclusion: The proposed models using only EQA data can objectively identify assays at risk of failing against
biological variation goals for monitoring patients and suggest parameters for reliable performance.
© 2016 The Canadian Society of Clinical Chemists. Published by Elsevier Inc. All rights reserved.

1. Introduction
Maintaining the quality of clinical assays continues to challenge
pathology laboratories despite improvement in data handling and
analyser capabilities. To achieve commutable and precise results
requires suppliers providing appropriate calibrators, the laboratory
using an effective QC system and careful interpretation of External
Quality Assurance results. EQA results will identify if either the laborato-
ry or the relevant analyser method group performs worse than other
laboratories in terms of bias and imprecision.
Laboratories need to identify assays which are poorly performing
and adopt QC practices that maintain results that are clinically accept-
able. Using EQA performance can give strong guidance and identify
those particular assays which warrant closer attention.
The aim of Quality Control (QC) is to ensure that laboratory test re-
sults are fit for their intended use. But laboratories manage assays
using a QC system with a single analytical goal for an analyte, even
though the clinical goal changes with the patient circumstance. For ex-
ample, the goal for patient monitoring is individual biological variation,
while for patient diagnosis it is total error based on individual and group
biological variation (BV). The adopted analytical goal for an analyte
⁎ Corresponding author.
E-mail address: tony.badrick@rcpaqap.com.au (T. Badrick).
might be regulatory (e.g. CLIA), derived by expert group (e.g. RCPA
QAP Chemical Pathology Program) or determined by the laboratory it-
self. Methods of assessing laboratory performance should focus on
both the suitability of the selected operating parameters so they ensure
compliance with the analytical goal (e.g. target SD and QC rules/algo-
rithms) and outcome measures of performance (e.g. imprecision).
Methods for calculating patient risk from errors in test results cannot
rely on analytical goals alone; they need to include clinical goals and
the harm arising from the errors.
Quality Control strategies do not usually consider patient risk, they
are concerned with detection of analytical error. The aim of QC strate-
gies was to develop QC rules (algorithms) and QC sample frequencies
that allow high error detection rates usually a 90% probability of detect-
ing a statistically significant shift in QC results (Ped), combined with a
low probability of false error detection (Pfr). These analytical goals
were usually based on stable imprecision which is not always adequate
for the relevant biological goal for patient care. The analytical error de-
tection rates were determined using power function rules [1,2] or com-
puter programs such as QC Validator [3]. Parvin [4] introduced the
variable, expected number of patient reports with an unacceptable
error condition E (Nu) which is the product of the increased probability
of a result having an unacceptable amount of error due to an error state
and the average number of results reported during an error state. There
are many different QC rules and frequencies that meet a given E(Nu).

http://dx.doi.org/10.1016/j.clinbiochem.2016.01.014
0009-9120/© 2016 The Canadian Society of Clinical Chemists. Published by Elsevier Inc. All rights reserved.
 M. Mackay et al. / Clinical Biochemistry 49 (2016) 596–600
This variable is an example of a quality goal focussing on predicting and
minimising outcomes that would affect patients, rather than merely de-
tecting analytical/statistical outliers.
In this paper, we wish to explicitly consider risk to the patient, not
just analytical error. There is a requirement under ISO 15189 [5] and
CLSI EP 23A [6] standards/guidelines that any clinical laboratory must
also have targeted processes in place that reduce patient risk. Risk is a
function of the clinical application, e.g. diagnosis or monitoring, and
whether or not an immediate intervention in treatment may follow
from an unexpected result. Risk is also dependent on the frequency of
errors for a test and how effectively the current processes identify
these errors. These components of risk will be used in the discussion
to present a simple risk based model that laboratories can use to identify
high risk assays on which they should concentrate quality monitoring
efforts.
The idea relies on three concepts, Assay Capability (Cpa) [7], a 3 × 3
matrix of achievable imprecision, and a calculation of a risk score, using
a modified Failure Mode Effects Analysis. We will base our model on
EQA results. Laboratories often undervalue EQA results by only consid-
ering short-term survey feedback. EQA results should be analysed over
longer time intervals than just a survey cycle. Long-term EQA results
are in fact quite stable and can reflect the performance of the laboratory
over the life of an instrument [8]. We will apply the idea to EQA data but
it can equally be used with QC data if peer QC imprecision data is
available.
1.1. External Quality Assurance Programs
It is important to be aware that different EQA programs have differ-
ent approaches depending on whether they are part of a regulatory
system (Proficiency Testing) or Aspirational (Quality Assurance) [9].
The differences have been summarised by the IFCC, see Table 1 [10].
There can be problems with EQA samples because of their nature.
When investigating a problem identified by an EQA sample the follow-
ing must be considered: clerical errors, methodological problems
(carryover, reagent or calibrator variation), equipment problems,
human errors with preparation of the EQA material, and problems
with the EQA material (commutability issues) [11].
Miller et al. described the key factors for interpreting PT/EQA results
as a knowledge of the commutability of the samples used and the pro-
cess used for target value assignment. A commutable PT/EQA sample
demonstrates the same numeric relationship between different mea-
surement procedures as that expected for patients' samples. Non-
commutable PT/EQA samples frequently have a matrix-related bias of
unknown magnitude that limits interpretation of results [12]. The tech-
niques used in the following analysis require an EQA program with
commutable samples and robust target setting procedures.
Table 1
Summary of differences between Proficiency Testing and External Quality Assessment.
• Laboratory performance evaluation
Proficiency testing
for regulatory purposes
• Laboratory performance and method
External Quality Assessment Schemes
evaluation
(EQAS)
• Educational
External Quality Assessment Inter-laboratory comparisons designed
Programmes (EQAP) and operated to assure one or more of:
• Participant performance analytical,
interpretive, clinical advice
• Method performance evaluation
• In vitro diagnostic device vigilance
• Education
• Training and help
597
2. Methods
For the rest of this paper we will concentrate on calculating risk for
assays based on performance in an EQA program. We have selected
the RCPA QAP Chemical Pathology program which uses paired, linearly
related samples and an estimate of imprecision is calculated from the
standard error of the linear regression line fitted to all data points [13].
2.1. Assay Capability — detection of analytical errors
The first concept to understand as we develop a model of risk is
Assay Capability. Capability is an objective measure of the ability of an
assay to meet pre-defined requirements and is defined as the analytical
goal divided by assay imprecision. This can be expressed as Cpa = AG/
CVa (equivalently AG/SD), where AG is the Analytical Goal and CVa is
the Coefficient of Variation (imprecision) of the assay in question [14].
The analytical goal can be chosen from a number of sources includ-
ing BV, State of the Art and expert opinion [15]. We will use the AG con-
cept based on biological variation which is given for a desirable target
as:
AG = [k × 0.5 x CVi] + [0.25 x √(CV2i + CV2g)] [16] where k = 1.65 for
95% confidence; CVi = within individual biological variation; CVg = be-
tween individual biological variation.
The assumption of zero systematic bias is valid in the case of many
routine clinical chemistry tests [17] and is also valid for the analysis of
most tests if the result is compared to the method or group perfor-
mance. The AG could be based on biological variation, group perfor-
mance in an EQA (standard deviation of group), or an arbitrary error
limit, such as with CLIA regulations. The Assay Capability value indicates
the number of standard deviations inside the analytical goal, so the
higher the value the better the assay. We recall that the aim of any QC
strategy is to have well defined QC rules which have at least a 90% prob-
ability of error detection (Ped) with low false rejection (Pfr) which re-
quires selecting an appropriate QC algorithm matched to laboratory
imprecision.
2.2. 3 × 3 matrix (of achievable imprecision)
The 3 × 3 matrix displays achievable imprecision for every laborato-
ry assay in a single graphic. It shows analytical laboratory performance
against peers and against performance standards, with the position of
the assay in the matrix showing responsibility for improving poorly
performing assays (see Table 2).
As is the case with many other External Quality Assurance Programs,
the RCPA QAP Chemical Pathology End-of-Cycle report [13] is useful in
establishing laboratory performance on a test-by-test basis, showing a
laboratory's imprecision, the imprecision of the top 20th percentile for
all laboratories, the imprecision of the 50th percentile for laboratories,
the median imprecision for a method group, as well as the allowable
limits of performance. However, such formats do not give insight into
a laboratory's overall performance, particularly for a laboratory having
a wide variety of instrumentation.
Using Assay Capability (Cpa) we constructed a 3 × 3 matrix to visu-
alise the performance of individual laboratory assays against better
performing laboratories in an EQA program. We chose the 20th percen-
tile as the comparator group because we considered that this was an as-
pirational performance goal achieved by a reasonably large number of
laboratories. There are a number of possible analytical goals that could
have been chosen as imprecision goals, for example the CLIA guidelines.
We selected the analytical goals of the RCPA QAP (the Allowable Limit of
Performance). The advantages of these goals are that they are based on
BV and State of the Art. It is not possible to use just BV for all analytes
because some are not endogenous, e.g. drugs. In programs where
there were insufficient members in a method group to accurately calcu-
late the 20th percentile, we propose using the average for the method
group as the comparator. This substitution is one of expediency and
598 M. Mackay et al. / Clinical Biochemistry 49 (2016) 596–600

Table 2 place. When laboratory Cpa b 4 the laboratory is forced to use reactive
The 3 × 3 matrix showing how performance is classified using Assay Capability Index of QC, i.e. patient samples must be rerun after fixing the QC problem. For
the individual laboratory assays compared to the top 20th percentile performers in the
EQA.
Cpa b 4, we score relative to the 20th percentile Cpa, i.e. based on
achievable imprecision, and do not penalise laboratories where even
Achievable assay capability
the best current equipment is inadequate. However, if laboratory
(20th percentile laboratory)
Cpa b2, we score 10 because even reactive QC fails, i.e. error detection
<4 4–6 >6 is so low that it may take more than one round of QC to detect the an-
Lab Profession world alytical error — a disastrous situation.
>6
world class class

Actual 4–6 Acceptable Acceptable Acceptable
laboratory imprecision imprecision imprecision
assay capability
Profession User group Your lab
<4
to review to review to review
does not affect the generality of our approach. This allows assays to be
benchmarked according to achievable Assay Capability.
It is possible to rate boxes in the 3 × 3 matrix and add the total assay
score to produce a peer index, but the main aim of the 3 × 3 matrix is to
specifically highlight poorly performing assays, not to generate an ag-
gregated Key Performance Indicator.
Boxes along the diagonal from bottom left to top right define the
performance of the 20th percentile laboratory. The box in top left
shows exceptionally good performance compared to 20th percentile.
The box at the bottom right shows exceptionally poor imprecision per-
formance compared to other laboratories. Analysis of at least 15 pro-
gram cycles over 9 years covering the period 1997 to 2014 shows that
the Assay Capability of the 20th percentile for each analyte has been rel-
atively stable, with minor exceptions for commutability issues and
some tests near the Cpa = 4 and Cpa = 6 boundaries. In particular
when the 20th percentile achieves Cpa ≥ 6, at least 50–80% of laborato-
ries achieve Cpa ≥ 4 (see Table 3). This means that assays in the bottom
right box of the 3 × 3 grid/matrix are the first targets for laboratory
improvement.
2.3. Risk
The final concept used was to add risk to the matrix based on FMEA.
FMEA calculates a risk which is the product of the rate of error detection,
the frequency of error occurring and the impact of this error on a patient
(harm). We describe below how each of these factors is determined and
suggest a potential scoring model for each of the three factors.
1. The error detection variable (Ped) represents the power of the QC rule
and is determined by the ratio of analytical goal to laboratory impre-
cision, i.e. Assay Capability. This follows as the lower the analytical im-
precision compared to analytical goal, i.e. the higher the Assay
Capability, the simpler the QC rule required for good error detection
and the lower the likelihood that reported results will have errors.
The scores are based on the CLSI Guideline QMS06-A3 Section 4.3.2.12
Risk Severity Matrix where the three levels of risk are described as
low, moderate and high. If laboratory Cpa N 6, then the score is 1
and if Cpa is between 4 and 6, then the score is 2. Both these situations
satisfy an error budget, i.e. proactive QC — avoiding errors in the first
2. It is impossible to calculate actual errors in patient results without at
least performing a replicate study on each patient sample, which is
impractical to say the least. We want to calculate risk on the basis
of the information available and use the fact that the ratio of clinical
goal to laboratory imprecision determines errors in patient results,
e.g. the ratio of within-individual biological variation (CVi) to impre-
cision determines the frequency of errors in patient monitoring. The
advantage of using this method is that the calculation is dependent
only on the EQA result.
An important consequence is that the ratio of clinical goal to analyt-
ical goal is what determines the frequency of patient errors at any
level of imprecision. That is Assay Capability times this ratio gives
Clinical Capability which predicts errors, i.e. AG/CVa ∗ CG/AG =
CG/CVa.
The score that we suggest is related to imprecision grade as defined
by Frazer [16] e.g. optimum = 1, desirable = 2, minimum = 5,
fail = 10. Similarly, we can score by imprecision grade for diagnosis
after substituting CVi with relevant clinical goal. Note that we are
classifying assays by risk, not calculating aggregate risk.
3. The third factor is harm from the error, i.e. impact on the patient,
which depends on the clinical impact of the test result. So the
highest risk assays will be those where changes in diagnosis or treat-
ment will follow immediately from a result. They include sodium,
potassium, calcium, troponin, and HbA1c in the case of a diagnosis.
We have used a graded score based on clinical significance. Individ-
ual laboratories will have to assign this risk factor for each test on the
basis of the risk to patients of these tests in their situation.
Our estimation of risk from pathology testing uses the product of the
factors error detectability, frequency of error and result impact of errors
on a patient (see Table 5).
3. Results
The last task is to reduce all possible risk scores into a small number
of action levels and then present this information in a simplified way to
lead to a common response to a given calculated risk. We have seen that
the risk score, being a product of individual factor scores, is the likeli-
hood of a hazardous failure of quality control to have a high impact on
a patient. High risk scores need to be easily identified. This is achieved
by overlaying the risk score on the 3 × 3 matrix of achievable analytical
imprecision.

Table 3
Achievable Standard and Assay Capability of the top best performing 20th percentile QAP laboratories as measured by Cpa.

Cpa ≤ 4 4 b Cpa b 6 Cpa ≥ 6 Year of RCPA QAP

Sodium, potassium, chloride, bicarbonate, calcium,
creatine kinase, total bilirubin, osmolality
Sodium, potassium, chloride, bicarbonate, calcium,
total bilirubin, osmolality, creatinine, magnesium,
albumin, ALP, ALT, Glucose, protein
b20% of all laboratories achieve Cpa ≥4
Urea, creatinine, magnesium, phosphate, albumin, Glucose, GGT, amylase, lipase, LD, 1997
ALP, ALT, AST, lactate, trig, HDL, protein cholesterol, iron, transferrin, lithium
Creatine kinase, urea, phosphate, AST, lactate, HDL, Trig, GGT, TrpI 2015–16
amylase, lipase, LD, cholesterol, iron, transferrin,
lithium, urate
20–50% of all laboratories achieve Cpa ≥4 50–80% of all laboratories achieve Cpa ≥4
 M. Mackay et al. / Clinical Biochemistry 49 (2016) 596–600 599

Table 4 Cpa for each test result from a recent EQA end of cycle report for the in-
The 3 × 3 matrix showing the Capability Index of the individual laboratory compared to dividual laboratory and the best in class (20th percentile) laboratories.
the top 20% of performers in the EQA for a specific laboratory in detail with a scoring sys-
tem. The numbers in each box represent the score for Ped as described above.
We see that assays that have the greatest Cpa have the lowest scores.
In Table 4 we have identified the measurands with the highest risk
20th percentile laboratory Cpa–general serum chemistry cycle 97
scores with an asterisk. We explain the derivation of those risk scores
<4 4–6 >6 in Table 5.
Table 5 shows some examples of patient monitoring risk calculated
Valp Urea GGT, Trig
>6 for a particular laboratory using the process/formula described in the
1 1 text. We see that the assay of greatest clinical risk is sodium because, al-
1
though this laboratory performs analytically better than the 20th per-
Fe, Lip, HDL,
Laboratory centile, it still performs poorly compared to clinical goals. GGT and
Chol, Lact,
assay chloride have low risk because of the low clinical importance. Glucose
Cl, ALP, K Amyl, Trf, Trp I
capability rates a high risk because of a combination of the three factors associated
2 AST, PO4, 2 with risk: detection of analytical errors, frequency of clinical errors and
(Cpa)
Urate clinical importance. For glucose, the laboratory performs significantly
4–6
2 worse than the 20th percentile and should check if the imprecision
Osml, ALT, Mg, class differs between QC and QAP. If they are the same, there is room
Gluc*, HCO3,
for analytical improvement; if QC is better, there may be a problem
LDH, CK, with the QAP material.
<4 Creat, Ca*,
Li
Alb, Prot, Tbil,
4. Discussion
Na*
2 5 10
The 3 × 3 matrix displays achievable performance with regard to im-
<2 10 10 10 precision for every laboratory assay in a single graphic. It shows analyt-
ical laboratory performance against peers, with the position of the assay
in the matrix showing responsibility for improving poorly performing
assays.
The factor scores were chosen so that if either an analytical factor of We have suggested an objective procedure for laboratories to iden-
10 or any two factors of 5 or more are given, then there is a significant tify poorly performing assays when compared to their peers and a
overall risk. In terms of the 3 × 3 matrix this means that assays in the method of determining risk based only on performance in an EQA. If
bottom right cell always flag high risk, while assays in the second row Cpa b 4, 90% error detection is not achievable, improving performance
and bottom left cell need both other factors (predicted clinical errors is difficult, and patient samples need to be rerun when the error is
AND harm) to be 5 or more, and the middle bottom cell needs either fac- fixed after a QC flag. This is potentially dangerous, costly and disrupts
tor (predicted clinical errors OR harm) to be 5 or more. In addition any laboratory operations. With assays in different Assay Capability classes,
assay with b2SD inside analytical goal scores 10 for detection of analyt- fewer mistakes are made in interpreting QC results and taking appropri-
ical errors and so is at high risk. ate action.
The following are examples from a clinical laboratory of the use of There are limitations in this approach including that External Quality
these tools. Firstly the 3 × 3 matrix is populated using the calculated Assurance samples may have matrix effects leading to questions of

Table 5
Examples of risk calculated from EQA data for a particular laboratory.

Test Score for Analytical 20th Laboratory Laboratory Score for Clinical Ratio of Laboratory Score for FMEA
harm arising goal (AG) percentile imprecision Assay predicted goal for clinical Clinical predicted patient
from errors Assay (CVa) Capability detection of patient goal to Capability frequency of monitoring
(clinical Capability (Cpa = analytical monitoring analytical (CVi/CVa clinical risk = harm
importance) (AG/CV20th) AG/CVa) errors iecg= cvi goal = AG/CVa errors x detection x
(AG/CVa) (CVi/AG) * CVi/AG) (CVi/CVa) frequency

Na 5 2.2% 2.8 0.6% 3.7 2 0.7% 0.3 1.2 10 100

Creat 10 8.0% 3.3 2.7% 3.0 2 6.0% 0.8 2.2 2 40

Cl 1 3.1% 3.4 0.6% 5.2 2 1.2% 0.4 2.0 2 4

Gluc 5 8.0% 3.6 3.2% 2.5 2 4.5% 0.6 1.4 5 50

GGT 2 12.0% 6.0 1.5% 8.0 1 13.8% 1.2 9.2 1 2

TrpI 10 20.0% 6.7 3.8% 5.3 2 9.7% 0.5 2.6 2 40

Ca 5 4.0% 2.9 1.1% 3.6 2 1.9% 0.5 1.7 5 50

600 M. Mackay et al. / Clinical Biochemistry 49 (2016) 596–600
commutability. The EQA imprecision used is based on subgroup perfor-
mance over an entire EQA cycle and as all laboratories are analysing the
same sample it does represent a reasonable sampling process. The anal-
ysis also uses a calculated capability index which is not dependent on
bias, only imprecision. As long as the EQA uses the same material during
the survey estimates of imprecision should be valid.
5. Conclusion
We have developed a proactive, predictive model, based on the abil-
ity of the quality system to detect not just laboratory analytical errors,
but those that would have an impact on patients using a measure of
the likely harm caused. The processes described allow the laboratory
to more effectively use the peer assessment which EQA provides to
prioritise assays with inferior analytical performance or increased clini-
cal risk. These tools allow laboratories to focus their improvement ef-
forts in an evidence based manner.
Both the 3 × 3 matrix and the FMEA calculation of risk are self-
contained and use novel approaches. Together they produce an inte-
grated system covering imprecision standards, assessment of laboratory
and peer imprecision performance, responsibility for improving poor
assays, and assays with high clinical risk.
References
[1] J.O. Westgard, B. Stein, Automated selection of statistical quality-control procedures
to assure meeting clinical or analytical quality requirements, Clin. Chem. 43 (2)
(1997) 400–403.
[2] J.O. Westgard, P. Hyltoft Petersen, D.A. Wiebe, Laboratory process specifications for
assuring quality in the US national cholesterol education program, Clin. Chem. 37
(5) (1991) 656–661.
View publication stats
[3] P.C. Fallest-Strobl, E. Olafsdottir, D.A. Wiebe, J.O. Westgard, Comparison of NCEP per-
formance specifications for triglycerides, HDL-, and LDL-cholesterol with operating
specifications based on NCEP clinical and analytical goals, Clin. Chem. 43 (11)
(1997) 2164–2168.
[4] C. Parvin, Assessing the impact of the frequency of quality control testing on the
quality of reported patient results, Clin. Chem. 54 (2008) 2049–2054.
[5] International Organisation for Standardisation ISO 15189 Medical Laboratories —
Particular Requirements for Quality and Competence http://www.iso.org/iso/cata-
logue_detail?csnumber=56115 (accessed 7 July 2015)
[6] Clinical and Laboratory Standards Institute http://clsi.org/blog/2011/10/01/clsi-pub-
lishes-new-guideline-laboratory-quality-control-based-on-risk-management-
ep23-a/ (accessed 7 July 2015)
[7] L. Burnett, G. Hegedus, D. Chesher, J. Burnett, G. Costaganna, Application of process
capability indices to quality control in a clinical chemistry laboratory, Clin. Chem. 42
(1996) 2035–2037.
[8] R. Rej, C.S. Norton-Wenzel, Assessing analytical accuracy through proficiency test-
ing: have the effects of matrix been overstated? Clin. Chem. 61 (2) (2015) 433–434.
[9] D. James, D. Ames, B. Lopez, R. Still, W. Simpson, P. Twomey, J. Clin. Pathol. doi:
http://dx.doi.org/10.1136/jclinpath-2013-201621
[10] D. Maziotta, D. Harel, G. Schumann, et al., Guidelines for the Requirement of Compe-
tence of EQAP Organizers in Medical Laboratories. IFCC/EMD/C-AQ, 2003.
[11] W.G. Miller, Specimen materials, target values and commutability for external qual-
ity assessment (proficiency testing) schemes, Clin. Chim. Acta 327 (2003) 25–37.
[12] W.G. Miller, G.R.D. Jones, G.L. Horowitz, C. Weykamp, Proficiency testing/external
quality assessment: current challenges and future directions, Clin. Chem. 57
(2011) 1670–1680.
[13] https://www.rcpaqap.com.au/chempath/ (accessed 12 July 2015)
[14] A. Coskum, Six sigma and calculated laboratory tests, Clin. Chem. 52 (4) (2006)
770–771.
[15] G.R.D. Jones, Analytical performance specifications for EQA schemes — need for
harmonisation, CCLM 53 (6) (2015) 919–924.
[16] C.G. Fraser, Biological Variation From Principles to Practice, AACC Press, Washington,
2001.
[17] K. Hens, M. Berth, D. Armbruster, S. Westgard, Sigma metrics used to access analyt-
ical quality of clinical chemistry assays: importance of the allowable total error
(TEa) target, CCLM 52 (7) (2014) 973–980.