Epidemiology – Diagnosis

MA1610: Applied Epidemiology & Biostatistics

Part B: Epidemiology, Summer term 2017

DIAGNOSIS
(HAS A PERSON A DISEASE OF INTEREST?)

Christiane Rudolph

Christiane.rudolph@uksh.de
 Epidemiology – Diagnosis

Main Topics I

• What is a medical diagnosis, diagnostic procedure?

• Diagnostic tests
– Diagnostic quality criteria
– Gold standard

• Logic and methodology of diagnostics

2
 Epidemiology – Diagnosis

Main Topics II

• Epidemiological Tools
– Sensitivity, specificity
– Polytomous tests
– Receiver Operating Characteristics Curves (ROC-Curves)
– Predictive values
– Likelihood ratio
– Pre- and Post-test probabilities
– Odds
– Fagan nomogram

3
 Epidemiology – Diagnosis

Important Terms and Definitions

Diagnostic procedure / Diagnostics

• Process of determining health status and factors
responsible for producing it;
 applied to an individual, family, group or community

• Lots of strategies, principles and methods used during a

medical examination of a health disturbance / clinical disorder

• Diagnostics implies that the probability of a present medical

diagnosis can be increased by conducting one or more
diagnostic tests (at the same time or one after another).

4
 Epidemiology – Diagnosis

Important Terms and Definitions (cont.)

Diagnostic Test
• Clinical, laboratory, radiological or other tests,
carried out for the purpose of establishing an actual
diagnosis.

• Usually motivated by the presence of signs and / or

symptoms of disease,
which makes diagnostic testing different from screening.

(Pereira-Maxwell, F: A-Z of Medical Statistics. Oxford University Press, 1998)

5
 Epidemiology – Diagnosis

Important Terms and Definitions (cont.)

Screening
• Presumptive identification of unrecognized disease or
defect by the application of tests, examinations or other
procedures which can be applied rapidly.
• Screening tests sort out apparently well persons who
probably have the disease from those who probably do
not.
• Not intended to be diagnostic.
Persons with positive or suspicious findings must be
referred to their physicians for diagnosis and necessary
treatments. (US Commission on Chronic Illness, 1951)

6
 Epidemiology – Diagnosis

Diagnostics

• Anamnesis
• Clinical examination
• Laboratory tests
(Blood specimens,
microbiology)
• Physical function
tests
• Radiological tests
• Electrophysical tests
• Psychometric tests

7
 Epidemiology – Diagnosis

Clinical examination

• Watching (inspection)
• Hearing (auscultation)
• Touching (palpation)

• Function tests

8
 Epidemiology – Diagnosis

Diagnostics / Medical Medical need for

Diagnostic test Diagnosis action

• Anamnesis • Classification • To treat or not

• Clinical examination treat?
• Laboratory tests • Sometimes: same as
(Blood specimens, the name of an illness
microbiology) • Regularly only the
• Physical function best possible
tests description of an
• Radiological tests illness
• Electrophysical tests • Is stated for an
• Psychometric tests individual person

9
 Epidemiology – Diagnosis

What is a “Medical Diagnosis” ?

• In a very broad sense:

complete solution of a riddle

• “..and the case has now been so entirely cleared up that

I am not aware
that there is anything which has remained a secret to
us.“

10
 Epidemiology – Diagnosis

Coming back to tests: Capacity of diagnostic tests

Three quality criteria

Objectivity Reliability Validity

11
 Epidemiology – Diagnosis

Objectivity

• Property of a measurement or test, resulting in the same

result independent of the person who conducts, analyses
and interprets the test results

• Standard operation procedures

(manual of conduction and interpretation)

12
 Epidemiology – Diagnosis

Reliability

• Precision
• Overall consistency of a measure
• A test has a high reliability if it produces
similar results under consistent conditions
– Test-retest-reliability
(degree to which test scores are consistent
from one test administration to the next)
– Inter-rater reliability
(degree to which test scores are consistent
when measurements are taken by different people)

13
 Epidemiology – Diagnosis

Validity

• Accuracy (in comparison to gold standard)

• Degree to which the tool measures what it claims to
measure
– Sensitivity
(How many of the ill (=true positives) persons are tested
positive?)
– Specificity
(How many of the healthy (=true negatives) persons are tested
negative?)
• Problem if we measure „soft“ data such as pain,…
(lack of gold standard knowledge, lack of comparability)
14
 Epidemiology – Diagnosis

Validity and Reliability

Neither accurate Not accurate,

nor precise but precise

Accuracy increases
(Random and (Little random
systematic error) error, but
systematic error)

Accurate but not Accurate and

precise precise
(Random error, (Little random
but little and little
systematic error) systematic error)
Precision increases

According to: Webb and Bain, Essential Epidemiology, 2011

15
 Epidemiology – Diagnosis

Gold standard / reference

• Evaluation of accuracy depends on the knowledge whether the
disease is really present or not = gold standard.

• Problems in choosing proper comparison methods / standard:

 often relatively complex, expensive, invasive examination
procedures
 lack of objective standards for a disease
 using imperfect tests as gold standard tests

(Fletcher, Fletcher & Wagner: Klinische Epidemiologie, Ullstein Medical, 1999)

21
 Epidemiology – Diagnosis

Coming back to validity…

• Sensitivity
(How many of the true positives are tested positive?)

• Specificity
(How many of the true negatives are tested negative?)

22
 Epidemiology – Diagnosis

Helicobacter pylori / diagnostic methods

Calvet et al. Aliment Pharmacol Ther 16: 1283-1289

23
 Epidemiology – Diagnosis

UBT
• The urea breath test is based upon the ability of H. pylori to convert
urea to ammonia and carbon dioxide.
• Patients swallow urea labelled with an uncommon isotope, either
radioactive carbon-14 or non-radioactive carbon-13. In the
subsequent 10–30 minutes, the detection of isotope-labelled carbon
dioxide in exhaled breath indicates that the urea was split; this
indicates that urease (the enzyme that H. pylori uses to metabolize
urea) is present in the stomach, and hence that H. pylori bacteria
are present.

Wikipedia

24
 Epidemiology – Diagnosis

HpSA / Stool antigen test

• A stool antigen test checks to see if substances that

trigger the immune system to fight an H. pylori infection
(H. pylori-antigens) are present in your faeces.
• For 2 weeks before the test, one will be asked to avoid
certain medications such as antibiotics, antacids,
bismuth, and peptic ulcer medicines such as proton
pump inhibitors (PPIs) and H2 blockers.

25
 Epidemiology – Diagnosis

Evaluation of a test‘s validity

Gold standard / Thruth

Positive Negative

Test result Positive A B

Negative C D

Sensitivity: Propability of a positive test in a truly ill person

Sensitivity = A / (A + C)

Specificity: Propability of a negative test in a truly healthy person

Specificity = D / (B + D)

26
 Epidemiology – Diagnosis

Evaluation of a test‘s validity (UBT; Calvet et al.)

Gold standard / Thruth

Positive Negative

Test result Positive 95 3

Negative 5 97

Sensitivity: Propability of a positive test in a truely ill person

Sensi = A / (A + C) = 95%

Specificity: Propability of a negative test in a truely healthy person

Speci = D / (B + D) = 97%

27
 Epidemiology – Diagnosis

Evaluation of a test‘s validity (HpSA; Calvet et al.)

Gold standard / Thruth

Positive Negative

Test result Positive 70 20

Negative 30 80

Sensitivity: Propability of a positive test in a truely ill person

Sensi = A / (A + C) = 70%

Specificity: Propability of a negative test in a truely healthy person

Speci = D / (B + D) = 80%

28
 Epidemiology – Diagnosis

What is preferable for my patient’s situation?

A high Sensitivity Specificity
(preferably 100%)
Disease… is serious and should not be is serious. The knowledge
missed. that the disease is absent
has psychologic
or public health value.
Treatment… is possible, is not possible.
False-positive results do not lead to serious can lead to psychologic or
psychologic or economic economic trauma to the
trauma to the patient. patient.
Examples Tuberculosis, Syphilis, Multiple Sclerosis
Morbus Hodgkin,
Breast Cancer

According to: R.S. Galen, S.R. Gambino: Beyond Normality, John Wiley Verlag, 1975
29
 Epidemiology – Diagnosis

The world is not always simple…

• Sometimes tests provide more than a simple

“yes or no“ answer

 Polytomous test results

 Which value is the best cut-off value?

Calvet et al. Aliment Pharmacol Ther 16: 1283-1289

30
 Epidemiology – Diagnosis

Video by Rahul Patwari

• https://www.youtube.com/watch?v=21Igj5Pr6u4

31
 Epidemiology – Diagnosis

Test negative Test positive

healthy ill

TNF = True negative findings FNF = False negative findings

FPF = False positive findings TPF = True positive findings
32
 Epidemiology – Diagnosis

Two overlapping normal distributions

TPF: 96.7%
FPF: 58.5%

TPF: 78.2%
FPF: 20.1%

TPF: 24.2%
FPF: 0.8%

http://www.anaesthetist.com/mnm/stats/roc/Findex.htm
33
 Epidemiology – Diagnosis

ROC (Receiver-Operating-Characteristic) – Curve

Method to identify the proper cut-off value

Sensitivity

1 - Specificity
34
 Epidemiology – Diagnosis

Remember:

CAVE: ROC-Curve
Different number of groups

Calvet et al. Aliment Pharmacol Ther 16: 1283-1289

35
 Epidemiology – Diagnosis

http://www.anaesthetist.com/mnm/stats/roc/Findex.htm
36
 Epidemiology – Diagnosis

Predictive Values / Likelihood Ratios

Do test results distinguish patients with

and patients without the target disorder
among those in whom it is clinically sensible to suspect the
disorder? (effectiveness)

Impact of a (positive) test result –

Patient: Am I ill?
Physician: Do I have to treat this person?

37
 Epidemiology – Diagnosis

Predictive values

• Probability of a target condition (a disease) given by a

specific result of a test
• Positive predictive value (ppv):
Probability of being ill when the test result is positive
ppv = A / (A + B)
• Negative predicitve value (npv)
Probability of being healthy when the test result is negative
npv = D / (C + D)

38
 Epidemiology – Diagnosis

Evaluation of a test‘s validity (UBT; Calvet et al.)

Gold standard / Reality

Positive Negative

Test result Positive 95 17

Negative 5 554

Sensitivity: A / (A + C) = 95%
Specificity: D / (B + D) = 97%

prevalence = (A + C) / (A+ B +C + D) = 14.9%

positive predictive value = A / (A + B) = 85%

negative predictive value = D / (C + D) = 99%

39
 Epidemiology – Diagnosis

Evaluation of a test‘s validity (HpSA; Calvet et al.)

Gold standard / Reality

Positive Negative

Test result Positive 70 114

Negative 30 457

Sensitivity: A / (A + C) = 70%
Specificity: D / (B + D) = 80%

prevalence = (A + C) / (A+ B +C + D) = 14.9%

positive predictive value = A / (A + B) = 38%

negative predictive value = D / (C + D) = 94%

40
 Epidemiology – Diagnosis

Exercise: Prevalence and post-test probabilty (HpSA)

Hp + Hp - Total
Prevalence 14.9% Test + 70 114 184
Test - 30 457 487
100 571 671

Hp + Hp - Total
Prevalence 29.8 % Test +
Test -
671

Hp + Hp - Total
Test +
Prevalence 4.9 % Test -
671
PPV+ with 14.9 % : 38%
Sensitivity = 70%
29.8% : 60%
Specificity = 80%
4.9 % : 15%

41
 Epidemiology – Diagnosis

Exercise: Prevalence and post-test probabilty (HpSA)

Hp + Hp - Total
Prevalence 14.9% Test + 70 114 184
Test - 30 457 487
100 571 671

Hp + Hp - Total
Prevalence 29.8 % Test + 140 94 234
Test - 60 377 437
200 471 671

Hp + Hp - Total
Test + 23 128 151
Prevalence 4.9 % Test - 10 510 520
33 638 671
PPV+ with 14.9 % : 38%
Sensitivity = 70%
29.8% : 60%
Specificity = 80%
4.9 % : 15%

42
 Epidemiology – Diagnosis

Bear in mind!

• Predictive values depend on prevalence

43
 Epidemiology – Diagnosis

Predictive Values / Likelihood Ratios

Do test results distinguish patients with

and patients without the target disorder
among those in whom it is clinically sensible to suspect the
disorder? (effectiveness)

44
 Epidemiology – Diagnosis

Likelihood Ratio I
• Determine whether a test result usefully changes the probability that
a condition (such as a disease state) exists
• Assess the value of performing of a diagnostic test 
“Confirming/ excluding power” (Van den Ende et al. 2005)

• Answer the question:

How much more likely is it, that a positive test result comes from a
truly ill (true positive) person than from a truly healthy person (false
positive)

45
 Epidemiology – Diagnosis

Likelihood Ratio II
• “A likelihood ratio is simply the percentage of sick people with
a given test result divided by the percentage of well
individuals with the same result.”
– percentage of sick people with a given test result =
“True positives” = Sensitivity
– percentage of well individuals with the same result =
“False positives” = 1 - Specificity
• LR + = Sensitivity / (1 – Specificity)
LR + = [ A / (A + C) ] / [ 1 - ( D / (B + D) ) ]
LR + = [ A / (A + C) ] / [ B / (B + D) ]

46
 Epidemiology – Diagnosis

Evaluation of a test‘s validity (UBT; Calvet et al.)

Gold standard / Reality

Positive Negative

Test result Positive 95 17

Negative 5 554

positive predictive value = A / (A + B) = 85%

negative predictive value = D / (C + D) = 99%

LR+ = Sensi. / (1 - Speci.) = ( A / (A + C) ) / ( B / ( B + D)) = 95/3 = 32

LR- = (1 - Sensi.) / Speci. = ( C / (A + C)) / ( D / (B + D)) = 5/97 = 0.05

47
 Epidemiology – Diagnosis

Evaluation of a test‘s validity (HpSA; Calvet et al.)

Gold standard / Reality

Positive Negative

Test result Positive 70 114

Negative 30 457

positive predictive value = A / (A + B) = 38%

negative predictive value = D / (C + D) = 93.8%

LR+ = Sensi. / (1 - Speci.) = ( A / (A + C) ) / ( B / ( B + D)) = 70/20 = 3.5

LR- = (1 - Sensi.) / Speci. = ( C / (A + C)) / ( D / (B + D)) = 30/80 = 0.37

48
 Epidemiology – Diagnosis

Interpretation of LR

Test performance

Very good
good
moderate
bad

Mühlhauser and Höldke 1999,

arznei-telegramm

50
 Epidemiology – Diagnosis

Pre-test probability

Probability of disease among those who attend for

examination (= prevalence)

P[D] = 0.11

0 0.25 0.5
Probability of disease (P[D])

51
 Epidemiology – Diagnosis

Post-test probability

Probability of disease with positive test result

Proportion Odds Proportion

P[D] = 0.11 LR P[D] = 0.31

0 0.25 0.5
Probability of disease (P[D])

52
 Epidemiology – Diagnosis

Getting odds straight – Simply, odds are a ratio of proportions.

Suppose you and a friend are splitting a

cake after dinner. You of course, decide to take the
smaller piece (1/4) and your friend takes the rest
of the cake (3/4).

What proportion of the pie did you take?

_______ = p

What proportion of the pie is left?

_______ =1-p

What is the ratio of your portion to your friend´s?

_______ = p/(1-p) = Odds

Congratulations!
In calculating this final ratio, you converted a probability into an odds.

53
 Epidemiology – Diagnosis

Odds and proportions

p = a / N (part / total) (e.g. 1 out of 10 = 0,1)
odds = p / (1 – p) (e.g. 1 to 9 = 1/9)

odds = p / (1 – p)
p = odds / (odds + 1)

Fictive example:
Propability that the Hamburg soccer club (HSV) wins the champignons
league = 20 %
Odds are at 1 : 4

54
 Epidemiology – Diagnosis

Interpretation of diagnostic test results

Pre test propability Post test propability

(prevalence)
p

ê
Pre test Odds Likelihood-Ratio é

x LR [x LR x LR x …] = Post test Odds

It is quite simple, if you use the Fagan nomogram!

55
 Epidemiology – Diagnosis

Fagan nomogram

Prevalence (pre-test pr.) Hp = 14.9%

LR+ UBT = 31.6

LR- UBT = 0.05

STATS - STeve's Attempt to Teach Statistics

http://www.cmh.edu/stats/
56
 Epidemiology – Diagnosis

An Example (Van den Ende et al. (2005) Lancet 306:548)

57