L RELOADED: A METAMORPHOSIS

AND TRANSFIGURATION.

APPLICATION TO DATA FROM THE HEIDELBERG RETINA TOMOGRAPH.

Martin Diller, Heidelberg; Paul Artes, Halifax

Anything that can be said can be said clearly. What cannot be said, thereof one must be silent. L Wittgenstein (1889 1951). www.youtube.com/watch?v=57PWqFowq-4 God is in the details. L Mies van der Rohe (1886 1969)
Modern imaging technologies yield complex data data that are difcult to interpret without sophisticated statistical tools. Yet, users (clinicians, technicians, patients) usually do not have a background in statistics or probability theory, and thus there is a danger that results are misinterpreted. L is our (yet incomplete) attempt to nd a generalisable solution to this dilemma, applied to data from the HRT. L aims to translate statistical data a) into a simple graphical representation, and b) into clear, jargon-free, yet technically accurate, verbal statements that provide guidance to non-experts. L makes statements on the most important dimensions of a series of HRT images (quality; speed, clinical and statistical signicance of change, and the power to detect it; validity of linearity assumptions made by the analysis). Each dimension is quantied by an appropriate combination of indices (for example, image quality = mean pixel height standard deviation [MPHSD] + uniformity of illumination + + etc.), providing a hierarchy of statements that users are invited but not forced to explore. Data are scaled into an [0, 100] interval by comparison with reference quantiles from a previously published population (Reis et al, Ophthalmology 2012). We look forward to discussing the L approach with the critical audience of IMAGE.

Martin Diller and Paul Artes, IMAGE 2013

Friday, March 22, 2013

Data are complex. Analyses are complex.

Friday, March 22, 2013

Please write down your judgements! (A to F, yes or no)

Friday, March 22, 2013

Friday, March 22, 2013

A: The likelihood that this test result is abnormal is >99.5%. B: The likelihood that this patient has a normal visual eld is <0.5%. C: The likelihood that this patients eye is healthy is <0.5%. D: The likelihood that this patients eye is damaged is >99.5%.

Friday, March 22, 2013

above 95% P.L. between 95% and 99.9% P.L. beyond 99.9% P.L.

Friday, March 22, 2013

E: The likelihood that this test result is abnormal is >99.9%. F: The likelihood that this patient has a healthy optic disc is <0.1%. G: Damage in the nasal-inferior disc sector is >99.9% likely. H: The likelihood that this patient has glaucoma is >99.9%.

Friday, March 22, 2013

A - H are all wrong. (Some are dangerous.)

Friday, March 22, 2013

Why do we need ?
Studies show ~50% undiagnosed disease. Patients still go blind from glaucoma. A lot of these patients present late. Many diagnostic and therapeutic decisions are poor. All this despite a lot of technological advances.

Friday, March 22, 2013

1.0

A
0.8

30

1.0

30

A
25 0.8 25

positive rate in VFpositive group

C
response latency (s)
20

positive rate in VFpositive group

AUC=0.82
0.6

AUC=0.79
0.6

87% r=0.84

79% r=0.61

15

response latency (s)

20

15

0.4

mean.Diff=0.38 criterion=0.67

0.4

mean.Diff=0.27 criterion=1.3

10

10

A
0.2 5

B
0.2 5

A
0.0
!

0 0.0 0.2 0.4 0.6 0.8 1.0

29

40

10

11

10

0.0

0 0.0 0.2 0.4 0.6 0.8 1.0

62

18

dh ph

ns

pd dd

dh ph

ns

pd dd

positive rate in VFnegative group

response

positive rate in VFnegative group

response

Friday, March 22, 2013

Assessment of Glaucomatous Optic Disc Damage by Ophthalmology Residents at the University of So Paulo
Jayme A. Vianna1, Alexandre S. Reis1,2, Lucas P. Vicente1, Marcelo Hatanaka1, Paul H. Artes2
1Department

Jayme A Vianna, ARVO 2012, #644 jaymeavianna@gmail.com

of Ophthalmology, University of So Paulo, So Paulo, Brazil; 2Ophthalmology and Visual Sciences, Dalhousie University, Halifax, Canada

Purpose
To examine performance at diagnosing glaucomatous optic disc damage in residents at different stages of training.

Case Example
After completion of the test, each residents responses were automatically analyzed and the results presented in a graphic (Figure 2).

Table1. Performance of residents at different years of training (numbers are mean [standard deviation]) 1st Year AuROC Correlation w/ Experts Decision Criteria Response Latency (s) 0.69 (0.07) 0.66 (0.15) 1.92 (0.34) 5.6 (2.42) 2nd Year 0.74 (0.05) 0.65 (0.10) 1.85 (0.30) 7.6 (2.82) 3rd Year 0.74 (0.07) 0.65 (0.11) 1.81 (0.26) 5.4 (2.17) P* 0.61 0.86 0.52 0.04

Methods
At the end of the academic year, 40 ophthalmology residents (14, 14, 12 in the 1st, 2nd, and 3rd year or residency training) tested themselves with the Discus software.1 The software randomly displayed 100 nonstereoscopic optic disc photographs. Twenty (20%) were from patients with glaucoma and conrmed visual eld defect, and 80 (80%) from patients with suspected glaucoma or ocular hypertension with reproducibly normal visual elds. Twenty-six (26%) of images were repeated to evaluate consistency. Each image was displayed for 10 seconds, and observers had unlimited time to rate it (Figure 1).

*Kruskal-Wallis test AuROC, Area under the Receiver Operating Characteristic

Conclusion
There were considerable differences in performance, criterion, and speed, between residents in each year of training. Residents in the second and third year tended to perform better than those at the rst year of training. These differences were not statistically signicant. Discus provides a simple, rapid and objective assessment of performance that should be useful in many training programs. Our results will be useful as a reference for comparing other trainees.

Fig 2. Graphic results of Discus, containing: User ROC curve (colored) Expert reference ROC curve (gray) Area under ROC (AUC) User AUC / Expert AUC (percentage) Rank correlation of user and expert responses Mean difference of repeated images Likelihood to diagnose damage as mean of responses (criterion) Response latency for each category (inset boxplot) Graphic representation of criterion (red line) Expert reference of criterion (gray dashed line)

Fig 3. Discus results stratied by year of residency training. Each circle represents a single resident. Sizes of the circles are proportional to the response latency, colors are coded according to criterion, bold circles and horizontal dashed lines are group medians, and vertical dashed lines give the 25th and 75th percentiles. The horizontal gray line and shaded area give the mean and range from of a reference group of 10 experts (Denniss, 2011)

Title

Median AuROC was smaller in the rst year. Median response latency was larger in the second year. Decision criteria and correlation with experts were similar among the 3 years (Table 1). There was moderate correlation between the two performance measures (AuROC and correlation with experts, Spearmans = 0.61, P < 0.001), but no relationship between either performance measure and the decision criteria ( = -0.13 and -0.01, P > 0.10).

Reference
Denniss et al. Discus: investigating subjective judgment of optic disc damage. Optometry and Vision Science. 2011; 88(1):E93-101.

Results
Diagnostic performance (AuROC), response latency, decision criteria for individual participants, and summary statistics by year, are shown in Figure 3.

Try Discus yourself!

www.discusproject.blogspot.com

Fig 1. Screenshot of Discus software, showing an optic disc photograph, and the rating scale from denitely healthy (+2) to denitely damaged (-2). Images are displayed for up to 10 sec.

Sunday, April 29, 2012

Friday, March 22, 2013

Imaging really has a place in primary and secondary glaucoma care. We need to provide better guidance on use and interpretation.
Friday, March 22, 2013

Friday, March 22, 2013

Aim
jargon-free statements simple, but technically accurate understandable to non-experts useful rather than trivial

Friday, March 22, 2013

Guiding Principles
"Was sich berhaupt sagen lt, das kann man klar sagen; und wovon man nicht reden kann, darber mu man schweigen." "What can be said, can be said clearly, and what cannot be said clearly, thereof we should be silent."

Friday, March 22, 2013

Man muss die Dinge so einfach wie mglich machen. Aber nicht einfacher. Things should be made as simple as possible. but no simpler.
Friday, March 22, 2013

Categories
Change
topographical analysis rate of rim area change (mm2/y * 10-3) power (to detect a rate of -20.0 units)

Relevance of model / validity of assumptions

outliers tests of nonlinearity & autocorrelation

Data quality

Image quality (MPHSD) Overlap of images across the series

Friday, March 22, 2013

Principles
Visualisation
transform each statistic via its reference distribution. show quantiles [0-10, .., 90-100] on a simple 10-segment bar chart.

Translation into verbal statements

The data are of good quality. Typically, only 1 of 10 patients have better images. No change exceeding chance was statistically detectable over 5 years of follow-up (10 tests). If rapid deterioration had occurred, it would almost certainly have been detected. Another test should be obtained within the next 18 months.

Friday, March 22, 2013

Open Questions
Combining different indices to single metric
e.g: series quality = median MPHSD + 85th percentile MPHSD + overlap weighted mean, linear & non-linear discriminant functions

Reference data
published research studies vs large clinical databases whose data is it anyway?

Friday, March 22, 2013

We hide it effectively.
Friday, March 22, 2013