Professional Documents
Culture Documents
Epidemiology - Lecture 4
Epidemiology - Lecture 4
Copyright 2008, The Johns Hopkins University and Sukon Kanchanaraksa. All rights reserved. Use of these
materials permitted only in accordance with license rights granted. Materials provided “AS IS”; no representations or
warranties provided. User assumes all responsibility for use, and all liability related thereto, and must independently
review all materials for accuracy and efficacy. May contain materials owned by others. User is responsible for
obtaining permissions for use from third parties as needed.
Evaluation of DiagnosticandScreeningTests:
Validity andReliability
SectionA
4
DiagnosticTest andScreeningTest
5
SomeCommonScreeningTests
6
Variationin BiologicValues
7
Distributionof TuberculinReactions
25
20
15
Number of
subjects
10
0
3 9 15 21 27
Diameter of induration (mm)
Source: Edwards et al, WHO Monograph 12, 1953 8
Distributionof SystolicBloodPressures:
744EmployedWhiteMales,Ages40–64
25
20
Number of
15
10
men
0
80 90 100 110 120 130 140 150 160 170 180 190 200
10
SensitivityandSpecificity
• Sensitivity
– The ability of the test to identify correctly those who have
• the disease
• Specificity
– The ability of the test to identify correctly those who do
• not have the disease
11
DeterminingtheSensitivity, Specificityof aNewTest
12
GoldStandardTest
Disease
+ –
a+c b+d
(All people
(All people with
without
disease)
disease)
13
Comparisonof DiseaseStatus:
GoldStandardTestandNewTest
Disease
+ –
a b
+ (True positives)
New test
– c d
(True negatives)
14
Sensitivity
a true positives
sensitivity = =
a+c disease+
=Pr(T+|D+)
15
Specificity
d true negatives
sensitivity = =
b +d disease −
=Pr(T− | D− )
16
ApplyingConcept of SensitivityandSpecificityto a
ScreeningTest
17
CalculatingSensitivityandSpecificity
18
EvaluatingValidity
19
ExaminingtheEffect of ChangingCut-Points
http://en.wikipedia.org/wiki/Oral_glucose_tolerance_test
http://care.diabetesjournals.org/cgi/content/full/25/suppl_1/s21 20
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
20 diabetics
High
20 non-diabetics
Blood
sugar
Low
21
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
Subjects are
High
screened using
fasting plasma
glucose with a low
(blood sugar) cut-
point
Blood
sugar Diabetics Non-
Diabetics
+ 17 14
– 3 6
20 20
Low Sens=85% Spec=30%
22
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
Subjects are
High
screened using
fasting plasma
glucose with a high
cut-point
Blood
sugar Diabetics Non-
Diabetics
+ 5 2
– 15 18
20 20
Low Sens=25% Spec=90%
23
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
In a typical
High
population, there is
no line separating
the two groups, and
the subjects are
mixed
Blood
sugar
Low
24
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
In a typical
High
population, there is
no line separating
the two groups, and
the subjects are
mixed
Blood
sugar
Low
25
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
In fact, there is no
High
color or label
Blood
sugar
Low
26
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
Ascreening test
High
using a high cut-
point will treat the
bottom box as
normal and will
identify the 7
Blood subjects above the
sugar line as having
diabetes
Low
27
Concept of SensitivityandSpecificity
Diabetics Non-diabetics
Ascreening test
High
using a high cut-
point will treat the
bottom box as
normal and will
identify the 7
Blood subjects above the
sugar line as having
diabetes;
But a low cut-point
will result in
identifying 31
subjects as having
Low diabetes
28
LessonsLearned
29
Whereto DrawtheCut-Point
39
BehindtheTest Results
Disease
+ –
a b
+ (True positives) (False positives)
Test
– c d
(False negatives) (True negatives)
31
Review
32
SectionB
Multiple Testing
Useof MultipleTests
34
Sequential Testing(Two-StageScreening)
35
Exampleof aTwo-StageScreeningProgram:
Test 1(BloodSugar)
+ 70% 350
x 500=350 1900
9500–7600=1900 2250
results
Test
– 500–350=150
150 80% x7600
9500=7600 7750
500 = 500
5% x 10,000 9500 10,000
36
Exampleof aTwo-StageScreeningProgram:
Test 2(GlucoseToleranceTest)
+ Diabetes –
• Test 1 (blood sugar)
– Sensitivity = 70% + 350 1900 2250
– Specificity = 80%
results
Test
• Test 2 (glucose – 150 7600 7750
tolerance test)
– Sensitivity = 90% 500 9500 10,000
– Specificity = 90% + Diabetes –
315 190
+ 505
results
Test
35 1710
– 1745
results
Test
• Test 2 (glucose – 150 7600 7750
tolerance test)
– Sensitivity = 90% 500 9500 10,000
– Specificity = 90% + Diabetes –
• Net sensitivity = 315 190
+ 505
315
=63%
results
Test
500 35 1710
– 1745
• New specificity =
7600+1710
=98% 350 1900 2250
9500 38
Two-StageScreening: Re-ScreenthePositivesfromthe
FirstTest
D D
AI B
• Subject is disease negative when test negative in either test
D D
AU B
39
Net Sensitivityin aTwo-StageScreeningwhenTest + in
the FirstTestAreRe-Screened
P(A | B) = P(A)
thus
thus
P(A I B) =P(A) ∗P(B)
• Net sensitivity = Sensitivity 1 x Sensitivity 2
49
Net Specificityin aTwo-StageScreeningwhenTest + in the
FirstTestAreRe-Screened
41
Other Two-StageScreening
• Screen the negatives from the first test to identify any missed
true positives from the first test
– Net sensitivity and net specificity calculation follows
similar but different logical algorithms
– What is the net effect of testing the negatives from the
first test?
� Find more true positives => net sensitivity will be
higher than sensitivity from the individual tests
� Also find more false positives => net specificity will
be lower than specificity from the individual tests
42
SimultaneousTesting
43
SimultaneousTesting: CalculateNet Sensitivity
44
SimultaneousTesting: CalculateNet Specificity
45
Exampleof aSimultaneousTesting
47
SectionC
Predictive Values
PredictiveValues
49
Another Interpretation of PPV
50
BehindtheTest Results
Disease
+ –
a b
+ (True positives) (False positives)
Test
– c d
(False negatives) (True negatives)
51
Whatthe TestShows
Disease
+ –
+ a+b
(All people with positive results)
Test
– c+ d
(All people with negative results)
52
PredictiveValue
a
• Positive predictive value =
a+b
True Positives
=
Test +
= P(D+|T+)
d
• Negative predictive value =
c+d
True Negatives
=
Test –
= P(D– |T–)
53
ApplyingConcept of PredictiveValuesto ScreeningTest
54
CalculatingPredictiveValues
Positive predictive value =
80/180 = 44%
55
CalculatingPredictiveValues
Negative
20 800 820
56
PPVPrimarilyDependsOn…
57
PPVFormula
sensitivity x prevalence
PPV =
(sensitivity x prevalence) + (1- specificity) x (1- prevalence)
58
Calculationof PPVandNPV
60
Relationshipof DiseasePrevalenceto PredictiveValue
Example: Sensitivity = 99%; Specificity = 95%
61
Prevalenceof Disease
100
90
80
Predicted Value (%)
70
60
50 For test with
40 95% sensitivity
30
20 95% specificity
10
0
0 20 40 60 80 100
Prevalence of Disease (%)
Adapted from Mausner JS, Kramer S.Epidemiology: an Introductory Text. Philadelphia, WBSaunders 1985, p221. 62
SoIf aPersonTestsPositive…
62
TheRelationshipof Specificityto PredictiveValue
Disease
+ – Prevalence = 50%
250 250 Sensitivity = 50%
+ Specificity = 50%
500
250
PPV= =50%
500
Test 250 250
– 500
63
TheRelationshipof Specificityto PredictiveValue
Disease
+ – Prevalence = 20%
100 400 Sensitivity = 50%
+ Specificity = 50%
500
100
PPV= =20%
500
Test 100 400
– 500 Change prevalence
64
TheRelationshipof Specificityto PredictiveValue
Disease
+ – Prevalence = 20%
180 400 Sensitivity = 90%
+ Specificity = 50%
580
180
PPV= =31%
580
Test 400
420 Change sensitivity
– 20
200 800 1,000
65
TheRelationshipof Specificityto PredictiveValue
Disease
+ – Prevalence = 20%
100 80 180 Sensitivity = 50%
+ 720 Specificity = 90%
100
PPV= =56%
180
Test 100
– 820 Restore sensitivity
Change specificity
200
150
100
50
0
25–29 35–39 45–49 55–59 65–69 75–79
69
PPVof First ScreeningMammography
byAgeandFamilyHistoryof Breast Cancer
79
Consequenceof Different PPVs
Reliability (Repeatability)
Reproducibility, Repeatability, Reliability
72
Intra-Subject Variation
73
Variationin BloodPressureReadings:A24-Hour Period
74
Inter-ObserverandIntra-Observer Variation
75
Agreement betweenTwoObservers
(Or TwoObservations)
Positive Negative
Positive a b
Observer 2
Negative
c d
76
PercentAgreement
a+ d
Overall Percent Agreement = x 100
a+ b + c+ d
a
Percent Positive Agreement = x 100
a+ b + c
77
Example
Radiologist A
Positive Negative
Positive 4 5
Radiologist B
Negative
2 6
4+ 6
Overall Percent Agreement = x 100 = 58.8%
4+ 5+ 2+ 6
4
Percent Positive Agreement = x 100 = 36.4%
4+ 5+ 2 79
Observer or Instrument Variation:
Overall PercentAgreement
Reading #1
Reading #2 Abnormal Suspect Doubtful Normal
Abnormal A B C D
+
Suspect E F G H
+
Doubtful I J K L
+
Normal M N O P
A+F+K+P
Percent agreement = x100
Total
80
Outcomeof Test Results
Test results
Valid but not reliable
Test results
Valid and reliable
Test results
Reliable but not valid
True value 81
Review
• Define
– Overall percent agreement
– Percent positive agreement
• Contrast overall percent agreement and percent positive
agreement
82