Diagnostic Test

Kuntjoro Harimurti
kuntjoro.harimurti@ui.ac.id

Center for Clinical Epidemiology and Evidence-Based Medicine (CEEBM)

Faculty of Medicine, University of Indonesia Cipto Mangunkusumo Hospital
Test-treatment threshold

Post-test probability
Why (perfect) diagnosis?

Diagnosis: clinicians main task

Reasoned decisions about patient care, despite imperfect clinical
information and uncertainty about clinical outcome
Newman-Toker & Pronovost (JAMA, 2009):
2/3 legal claims against GPs in UK
40,000-80,000 US hospital deaths from misdiagnosis per year
Adverse events, negligence cases, serious disability more likely to be
related to misdiagnosis than drug errors
Diagnosis uses <5% of hospital costs, but influences 60% of decision
making
Ideal diagnostic tests right answers: positive results in everyone
with the disease and negative results in everyone else
Benefits of diagnostic test

Screening
to identify risk factors for disease and to detect occult disease in
asymptomatic persons
Diagnosis
to help establish or exclude the presence of disease in symptomatic
persons
Management
evaluate the severity of disease,
estimate prognosis,
monitor the course of disease (progression, stability, or resolution),
detect disease recurrence,
select drugs and adjust dosages,
select and adjust therapy
Basic principles of diagnostic test

Sensitivity, specificity
Prevalence, prior probability
Predictive values
Likelihood ratios
ROC (receiver operator characteristic) curve
Traditional 2x2 table

DISEASE
Yes No Total
3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d
Sensitivity

DISEASE
Yes No Total FALSE
NEGATIVES
3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d

Sensitivity: the proportion of patients with target disorder/disease (N=4)

who have positive test results (N=3).
Sensitivity: a/a+c = = 75%
Specificity

DISEASE
FALSE
Yes No Total POSITIVES

3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d

Specificity: the proportion of patients who dont have target

disorder/disease (N=96) with negative test results (N=89).
Specificity: b/b+d = 89/96 = 93%
Positive predictive value (PPV)

DISEASE PPV
Yes No Total
3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d

PPV: the proportion of people with positive test result (N=10)

who are truly have the disease (N=3)
PPV = a/(a+b) = 3/10 = 30%
Negative predictive value (NPV)

DISEASE NPV
Yes No Total
3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d

NPV: the proportion of people with negative test (N=90) who are
free from the disease (N=89)
NPV = d/(c+d) = 89/90 = 98%
Pre-Test Odds

DISEASE
Yes No Total
3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d

In the sample as a whole, the odds of having the disease are 4 to

96 or 4% (the PRE-TEST ODDS)
Post-Test Odds

DISEASE
Yes No Total
3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d

In the sample as a whole, the odds of having the disease are 4 to

96 or 4% (the PRE-TEST ODDS)
In those who score positive on the test, the odds of having the
disease are 3 to 7 or 43% (the POST-TEST ODDS)
Post-Test Odds

DISEASE
Yes No Total
3 7 10

a+b
Yes
a b
TEST

c d
90

c+d
No 1 89
Total 4 96 100
a+c b+d a+b+c+d

In the sample as a whole, the odds of having the disease are 4 to

96 or 4% (the PRE-TEST ODDS)
In those who score positive on the test, the odds of having the
disease are 3 to 7 or 43% (the POST-TEST ODDS)
In those who score negative on the test, the odds of having the
disease are 1 to 89 or approximately 1%
DIAGNOSTIC ODDS RATIO

DISEASE Potentially useful as an

Yes No Total overall summary measure,
but only in conjunction with
3 7 10

a+b
Yes other measures (LR,
a b
sensitivity, specificity)
TEST

c d
90

c+d
No 1 89
3
4 96 100 DOR = 7
Total 1
a+c b+d a+b+c+d 89
0.429
The Diagnostic Odds Ratio is the = = 38.2
0.011
ratio of odds of having the diagnosis
given a positive test to those of
having the diagnosis given a
negative test
BAYES THEOREM

POST-TEST ODDS =
LIKELIHOOD RATIO x PRE-TEST ODDS
LIKELIHOOD RATIO AND PRE- AND POST-TEST
PROBABILITIES

For a given test with a given

likelihood ratio, the post-test
probability will depend on the
pre-test probability (that is, the
prevalence of the condition in
the sample being assessed)
SENSITIVITY ANALYSIS OF A DIAGNOSTIC
TEST

Value 95% CI
Pre-test
35% 26% to 44%
probability
SENSITIVITY ANALYSIS OF A DIAGNOSTIC
TEST

Value 95% CI
Pre-test
35% 26% to 44%
probability
Likelihood
5.0 3.0 to 8.5
ratio

Applying the 95% confidence

intervals above to the
nomogram, the post-test
probability is likely to lie in the
range 55-85%
 The diagonal line (representing Sensitivity=0.5
and Specificity=0.5) represents performance
no better than chance

RECEIVER OPERATING CHARACTERISTIC CURVE

Overall shape is
predicted by the
reciprocal relationship
between sensitivity and
specificity
The closer the curve
gets to Sensitivity=1
and Specificity=1, the
better the overall
performance of the test
Hence the area under
the curve gives a
measure of the tests
performance

FALSE POSITIVE RATE (1-Specificity)

AREA UNDER ROC CURVES

100
Sensitivity

Sensitivity and specificity both

100% - TEST PERFECT
AREA=1.0

0 Sensitivity and specificity both

1-Specificity 50% - TEST USELESS

100
Sensitivity

The area under a ROC

AREA=0.5
curve will be between
0
0.5 and 1.0
1-Specificity
AREA UNDER ROC CURVES

100
Area = 0.7 (between
Sensitivity

0.5 and 1.0)

0
1-Specificity

Consider (hypothetically) two patients drawn

randomly from the DISEASE+ and DISEASE- groups
respectively
If the test is used to guess which patient is from the
DISEASE+ group, it will be right 70% of the time
Applying a diagnostic test in different settings

The Positive Predictive Value of a test will vary

(according to the prevalence of the condition in the
chosen setting)

Sensitivity and Specificity are usually considered

properties of the test rather than the setting, and are
therefore usually considered to remain constant

However, sensitivity and specificity are likely to be

influenced by complexity of differential diagnoses and a
multitude of other factors (cf spectrum bias)
 RECEIVER OPERATING CHARACTERISTIC
(ROC) CURVE
100 This study compared
90 the performance of a
80 dementia screening test
70 in a community sample
Sensitivity

60 (ACAT) and a memory

50 clinic sample (MC)
40
30
20 ACAT
10 Flicker L, Loguidice D,
MC
0 Carlin JB, Ames D. The
predictive value of
0 20 40 60 dementia screening
1-Specificity instruments in clinical
populations. International
Journal of Geriatric
Psychiatry 1997 ; 12 :
Critical appraisal of diagnostic test
(EBM issues)

Is this evidence about the accuracy of a diagnostic test

valid?
Does this valid evidence demonstrate an important
ability to accurately distinguish patients who do and dont
have a specific disorder?
Can I apply this valid, important diagnostic test to a
specific patient?
Is this evidence about the accuracy of a diagnostic
test valid?

Was there an independent, blind comparison with a reference

(gold) standard of diagnosis?

Was the diagnostic test evaluated in an appropriate spectrum of

patients (like those in whom we would use it in practice)?

Was the reference standard applied regardless of the diagnostic test

result?

Was the test (or cluster of tests) validated in a second, independent

group of patients?
Does this valid evidence demonstrate an important ability
to accurately distinguish patients who do and dont have
a specific disorder?

Sensitivity

Specificity

Predictive values

Likelihood ratio
Can I apply this valid, important diagnostic test to a
specific patient?

Is the diagnostic test available, affordable, accurate, and precise in our

setting?
Can we generate a clinically sensible estimate of our patients pre-test
probability?
From personal experience, prevalence statistics, practice databases, or primary
studies
Are the study patients similar to our own?
Is it unlikely that the disease possibilities or probabilities have changed since this
evidence was gathered?
Will the resulting post-test probabilities affect our management and help our
patient?
Could it move us across a test-treatment threshold?
Would our patient be a willing partner in carrying it out?
Would the consequences of the test help our patient reach his or her goals in all
this?
Diagnostic tests

y Is not about finding absolute truth, but about limiting

uncertainty
y establishes both the necessity and the logical base
for introducing probabilities, pragmatic test-treatment
thresholds...

Start thinking about

what youre going to do with the results of the
diagnostic test, and
whether doing the test will help your patients