EURACHEM/CITAC

AQA 2021: Assessment

of Performance and
Uncertainty in
Qualitative Tests in
the Medical
Laboratory

Paulo Pereira, Ph.D.

Contents
• What is a 'qualitative test procedure'?
• What is 'performance assessment' of qualitatives?
• What is ' uncertainty assessment of ' qualitatives?
• Case studies
What is a 'qualitative test procedure'?
 Binary, nominal, and ordinal categorizations
• Unlike quantitative analysis, qualitative analysis is reported by a qualitative result,
such as 'positive or negative' or 'pass or fail'
• The qualitative tests differ from the quantitative tests principally because the
property has no size, for example, dichotomous results
• Examples:

HCV Rapid
Blood Typing
Antibody Test
(agglutination/
(red line/+/no red
no-agglutination)
line/-)

Binary, unordered (nominal), categorizations

 Binary, nominal, and ordinal categorizations

• Examples:

Anti-SARS-CoV-2 Real-Time PCR

IgG/IgM
Immunoassay SARS-CoV-2 RNA
(positive/negative) (positive/negative)

Binary, ordered (ordinal), categorizations

What is 'performance assessment' of
qualitatives?
What is the recommended approach for performance
assessment?
• Clinical performance evaluation: Clinical sensitivity and specificity
Clinical accuracy criteria
Candidate test
Positive Negative Total
results
True-positive (TP) False-positive (FP)
Positive TP + FP
results results
False-negative True-negative
Negative FN + TN
(FN) results (TN) results
Total TP + FN FP + TN N

• Clinical sensitivity[%] = TP / (TP / FN) * 100

• Clinical specificity [%] = TN / (TN / FP) * 100
 What is the risk of misclassifying weak signals?
Weak Cutoff
Weak
negative positive
frequency, f

Risk of misclassification
Real-world scenario

True negatives True positives

S/CO
False negatives False positives
 What are the sources of uncertainty/binary
misclassification?
Blood bank case/Risk of post-transfusion infection

Epidemiological Binary results Uncertainty of

prevalence/ Seroconversion uncertainty/ proportions
Biological Blood bank "window" False results, and
variation (intra Uncertainty/ screening
and inter- period/ Interferences false-negative measurement
uncertainty
New variants/ interview
individual) viral load (FN), and (ordinal
Mutation of uncertainty uncertainty false-positive categorization
agents (FP) rate only)

Risk of Misclassification / Risk of False Results

What is ' uncertainty assessment of '
qualitatives?
 What is uncertainty of proportions?

The statistical quality of the estimated result rate that depends on the
number of samples tested and the number of false results

A wide 95 % CI for clinical sensitivity indicates that the binary

result has a significant chance to be misclassified/false-negative

A wide 95 % CI for clinical specificity indicates that the binary

result has a significant chance to be misclassified/false-positive

The interpretation of 95 % CI is, to some degree, similar to what

happens with the expanded measurement uncertainty
What is 'the number of samples' suitable to calculate the
uncertainty of proportions?
95% CILL = 89% for n = 30 (minimum no. of samples)
100%
The lower limit
for 95%
confidence for
90%

80%
+

Lower limit of confidence

clinical 70%

sensitivity and 60%

specificity 50%
based on the 40%
score 30%
confidence
20%
interval
10%

0%
0 10 20 30 40 50 60 70 80 90 100
n
Case studies
 Clinical sensitivity and clinical specificity case
• Clinical performance evaluation
Clinical accuracy criteria
Candidate test
Target sensitivity
Positive Negative Total = 100%, the target
results
of the 95% CI low
Positive 30 0 30 limit is 85%
Negative 0 30 30 Target specificity
Total 30 30 60 = 90%, the target
of the 95% CI
• Clinical sensitivity = 30 / (30 + 0) * 100 = 100% (95% CI: 89% - 100%)
low limit is 80%
• Clinical specificity = 30 / (30 + 0) * 100 = 100% (95% CI: 89% - 100%)
EURACHEM/CITAC
AQA 2021: Assessment
of Performance and
Uncertainty in
Qualitative Tests in
the Medical
Laboratory

Paulo Pereira, Ph.D.