Diagnostic Study

Sitti Wahyuni
Evidence Based Medicine
Some slides are modified from
Nia Kurniati’s
Center for Clinical Epidemiology and Evidence-Based Medicine (CEEBM)
Faculty of Medicine, University of Indonesia – Cipto Mangunkusumo Hospital
 Diagnosis

• Diagnosis in practice is the basis of medical care

• Clinicians make diagnosis following these rule:
• Patient history
• Examination
• Differential diagnosis
• Final diagnosis
• Is an imperfect process: probability rather than certainty
• Typically someone with abnormal symptoms consults a physician, who will
obtain a history of their illness and examine them for signs of diseases.
• The physician formulates a hypothesis of likely diagnoses and may or may not
order further tests to clarify the diagnosis
Sitti Wahyuni/ EBM 2
 What are tests used for?

• To increase certainty about presence/absence of disease

• To determine disease severity
• To monitor clinical course
• To asses prognosis – risk/stage within diagnosis
• To plan treatment

Sitti Wahyuni/ EBM 3

Sitti Wahyuni/ EBM 4
Before ordering a test asks your self

• What will you do if the test is positive?

• What will you do if the test is negative?
• If the answers are the same→ Do not do the test!

Myriam Hunink and Paul Glasziou

Decision Making in Health and Medicine (2001)

Sitti Wahyuni/ EBM 5

 Critical appraisal
for the accuracy (VIA) of a diagnostic study
• What you need to know when you do the test?
• Validity of a diagnostic study
• Interpret the results
• Defining the clinical question: PICO or PIRT
• Patient/Problem
• How would I describe a group of patients similar to mine?
• Index test
• Which test am I considering?
• Comparator… or …Reference Standard
• What is the best reference standard to diagnose the target condition?
• Outcome….or….Target condition
• Which condition do I want toSittirule in or rule out?
Wahyuni/ EBM 6
 Diagnostic Accuracy Studies

Series of patients

Index test

Reference standard

Compare the results of the index test with the reference standard,
blinded

Sitti Wahyuni/ EBM 7

 Appraising diagnostic studies: VIA

1. Are the results valid?→ validity

• Appropriate spectrum of patients?
• Does everyone get the reference standard?
• Is there an independent, blind, or objective comparison with the reference
standard?
2. What are the results?→ Importance
3. Will they help me look after my patients?→ Applicability/ relevance

Sitti Wahyuni/ EBM 8

 Accuracy Biases in Diagnostic Studies

• Spectrum bias: study uses only highly selected patients…….perhaps

those whom you would really suspect have the diagnosis
• Ideally, the test should be performed on a group of patients to whom it will be
applied in the real-world clinical setting (Appropriate spectrum of patients)
• Verification bias/ Reference bias: Only some patients get the reference
standard…..probably the ones whom you really suspect have the disease
• Ideally all patients get the reference standard test
• Observer bias: the test is very subjective, or done by a person who
knows something about the patient or samples
• Ideally, the reference standard is independent, blind, and objective

Sitti Wahyuni/ EBM 9

Bias

Differential Reference Bias Partial Reference Bias

Series of patients Series of patients

Index test Index test

Ref. Std. B Ref. Std.A Ref. Std. A

Blinded cross-classification Compare the results of the index

test with the reference standard,
blinded

Sitti Wahyuni/ EBM 10

Bias
Incorporation Bias Observer Bias

Series of patients Series of patients

Index test Index test

Reference standard, includes Ref. Standard

parts of Index test

Unblinded cross-classification
Blinded cross-classification

Sitti Wahyuni/ EBM 11

Effect of biases on results

Sitti Wahyuni/ EBM 12

 Appraising diagnostic studies: VIA

1. Are the results valid?→ validity

• Appropriate spectrum of patients?
• Does everyone get the reference standard?
• Is there an independent, blind, or objective comparison with the reference
standard?
2. What are the results?→ Importance
• Sensitivity, specificity
• Likelihood ratios
• Positive and Negative Predictive Values
3. Will they help me look after my patients?→ Applicability/ relevance

Sitti Wahyuni/ EBM 13

 1. Spectrum
2. Index test

3. Reference
standard

4. Blinding

Sitti Wahyuni/ EBM 14

 Calculation: 2 x 2 Table (Sensitifity, specificity)
Disease + Disease - • Sensitivity Proportion of people
Test + A B A+B WITH a disease who have a
(True positive) (False positive) positive test result → A/A+C
Test - C D C+D • Specificity Proportion of people
(False negative) (True positive
WITHOUT a disease who have a
A+C B+D negative test result → D/B+D

• Sensitivity 84/100→ A/A+C→ Sensitivity

Disease + Disease -
84%→ the test identifies 84 out of100 patient
Test + A=84 B=25 A+B= 109 WITH the disease
(True positive) (False positive)
Test - C=16 D=75 C+D=91
(False negative) (True negative) • Specificity 75/100→ Specificity 75%→ Within
100 people WITHOUT the disease, 75 has
A+C=100 B+D= 100 TOTAL 200 negative results
Sitti Wahyuni/ EBM 15
 Influenza example
Influenza Influenza
RT-PCR test (+) RT-PCR test (-)
Rapid A=27 B=3 A+B= 30
test +
Rapid C=34 D=93 C+D=127
test -
A+C=61 B+D= 96 TOTAL 157

• Sensitivity = 27/61 = 0.44 (44%)→

• There were 61 children who had influenza, the rapid test was positive in 27 of them
• Specificity = 93/96 = 0.97 (97%)
• There were 96 children who did not have influenza. the rapid test was negative in 93 of
them Sitti Wahyuni/ EBM 16
 Sensitifity, specificity, false positive rate

• Sensitivity →
• The new rapid influenza test was positive in 27 out of 61 children
with influenza (sensitivity = 44%)’
• Specificity → The new rapid influenza test was negative in 93 of the 96
children who did not have influenza (specificity = 97%)’
• False positive rate = 1 – specificity
• There were 96 children who did not have influenza, the rapid test
was falsely positive in 3 of them’
• A specificity of 97% means that the new rapid test is wrong (or falsely
positive) in 3% of children

Sitti Wahyuni/ EBM 17

Ruling In and Ruling Out

• High Sensitivity
• A good test to help in Ruling Out disease
• Means there are very few false negatives – so if the test comes back negative
it’s highly unlikely the person has the disease
• High Specificity
• A good test to help in Ruling In disease
• High specificity means there are very few false positives – so if the test comes
back positive it’s highly likely the person has the disease

Sitti Wahyuni/ EBM 18

 Calculation: 2 x 2 Table (Predictive value)
Disease + Disease - • PPV = Proportion of people with
Test + A B A+B a positive test who have the
(True positive) (False positive) disease. → A/A+B
Test - C D C+D • NPV = Proportion of people with
(False negative) (True positive)
a negative test who do not have
A+C B+D the disease → C/C+D
Influenza example
Influenza Influenza
• PPV = 27/30 = 90%
RT-PCR test (+) RT-PCR test (-) • NPV = 93/127 = 73%
Rapid A=27 B=3 A+B= 30
test +
Rapid C=34 D=93 C+D=127
test -
A+C=61 B+D= 96 TOTAL 157
Sitti Wahyuni/ EBM 19
Predictive Value: Natural Frequencies

• 100% Likely

• 50% Maybe

• 0% Unlikely
Sitti Wahyuni/ EBM 20
• Your friend went to his doctor and was told that his test for a disease was
positive. He is really worried, and comes to ask you for help!
• After doing some reading, you find that for men of his age:
• The prevalence of the disease is 30%
• The test has a sensitivity of 50% and specificity of 90%
• Given a positive test, what is the probability your friend has the disease?

30 50% 30x50/
Disease (+) Sensitivity 100=15 22 (15+7) people
100 test Positive, of
people Test(+) whom 15 have the
disease→
70 Specificity chance of disease is
70 x 10/
Disease (-) 90% or 15/22 = 68%
100= 7
False (+) rate
10% 21
Positive and Negative Predictive Value

• PPV and NPV are not intrinsic to the test, it depends on the
prevalence!

• NPV and PPV should only be used if the ratio of the number of
patients with the disease and the number of patients without
the disease is equivalent to the prevalence of the diseases in the
studied population

• Likelihood Ratio – it does not depend on the prevalence

Sitti Wahyuni/ EBM 22

Likelihood Ratios (LR)

• LR=
• Probability of clinical findings in patients withdisease
Probability of the same finding in patients without disease
• Example
• If 80% of people with a cold have a runny nose
• and 10% of people without a cold have a runny nose,
• then the LR for runny nose is: 80%/10% = 8

Sitti Wahyuni/ EBM 23

Likelihood Ratios (LR)

• Positive likelihood ratio (LR+)

• How much more likely is a positive test to be found in a person
with the disease than in a person without disease?
• LR+ = sens/(1-spec)

• Negative likelihood ratio (LR-)

• How much more likely is a negative test to be found in a person
without the disease than in a person with disease?
• LR- = (1-sens)/(spec)
Sitti Wahyuni/ EBM 24
What do likelihood ratios mean?

LR< 0,01 LR=1 LR>10

Strong NEGATIVE No diagnostic value Strong POSITIVE test
test results results

Sitti Wahyuni/ EBM 25

Example: Clinical diagnosis for appendicitis
McBurney’s point Rovsing’s sign
• If palpation of the left lower quadrant of a
person's abdomen results in more pain in
the right lower quadrant
Psoas sign
• Abdominal pain resulting from passively
extending the thigh of a patient or asking
the patient to actively flex his thigh at the
hip
Ashdown’s sign
• Pain when driving over speed bumps
Sitti Wahyuni/ EBM 26
 (LR+ = 3.4)

(LR- = 0.4)

Speed bump test (Ashdown’s sign):

LR+ = 1.4
LR- = 0.1
Sitti Wahyuni/ EBM 27
 %
Bayesian Fagan
reasoning nomogram
Post-test odds
for disease after
one test
Pre test 5% become pre-test
odds for next
?Appendicitis: test etc.
McBurney tenderness LR+ = 3.4 Post test ~20%
Speed bump test LR- = 0.1

Post-test odds =
Pre-test odds x
Likelihood ratio Post test ~0.5%

Clinical Epidemiology and Evidence-Based Medi

28
The news story…

The Journal of
Neuroscience: The
researchers detected
autism with over 90%
accuracy,

Sitti Wahyuni/ EBM 29

Sitti Wahyuni/ EBM 30
Natural Frequencies
• Autism has a prevalence of 1%., The test has sensitivity of 90% and specificity of
80%. Given a positive test, what is the probability the child has autism?

1 90% 1x
20.7 people test
Disease (+) Sensitivity 90/100=0,9
positive, of whom
100 0,9 have the
Test(+)
people disease→ chance of
disease is 0,9/20,7
99 Specificity 99x20/100=
= 4,5%
Disease (-) 80% or 19.8
Fals (+) rate
20% Sitti Wahyuni/ EBM 31
Sitti Wahyuni/ EBM 32
Sitti Wahyuni/ EBM 33
 3 steps (VIA) to
Appraising diagnostic studies:

1. Are the results valid?→ validity

• Appropriate spectrum of patients?
• Does everyone get the reference standard?
• Is there an independent, blind, or objective comparison with the reference standard?
2. What are the results?→ Importance
• Sensitivity, specificity
• Likelihood ratios
• Positive and Negative Predictive Values
3. Will they help me look after my patients?→ Applicability/ relevance
•Can I do the test in my setting?
•Do results apply to the mix of patientsI see?
•Will the result change my management?
•Costs to patient/health service?
Sitti Wahyuni/ EBM 34
Will the test apply in my setting?→ applicability/ relevance

• Reproducibility of the test and interpretation in my setting

• Do results apply to the mix of patients I see?
• Will the results change my management?
• Impact on outcomes that are important to patients?
• Where does the test fit into the diagnostic strategy?
• Costs to patient/health service?

Sitti Wahyuni/ EBM 35

Don’t believe everything they published,
Check for the Evidence!

Sitti Wahyuni/ EBM 36