C L I N I C A L A N D E X P E R I M E N TA L

RESEARCH

Validation of an automated-ETDRS near and intermediate visual

acuity measurement

Clin Exp Optom 2019 DOI:10.1111/cxo.13018

Yi Pang OD PhD Background: The aim of this study was to determine the repeatability of an automated-ETDRS
Lauren Sparschu BS (Early Treatment Diabetic Retinopathy Study) near and intermediate visual acuity measurement
Elyse Nylin BS in subjects with normal visual acuity and subjects with reduced visual acuity. The agreement of
Department of Dean’s Education, Illinois College of automated-ETDRS with gold standard chart-based visual acuity measurement was also studied.
Optometry, Chicago, Illinois, USA Methods: Fifty-one subjects were tested (aged 23 to 91 years; 33 subjects with normal
E-mail: ypang@ico.edu visual acuity: 6/7.5 or better; 18 subjects with reduced visual acuity: 6/9 to 6/30). Near and
intermediate visual acuity of one eye from each subject was measured with an automated
tablet-computer system (M&S Technologies, Inc.) and Precision Vision paper chart in a ran-
dom sequence. Subjects were retested one week later. Repeatability was evaluated using
the 95 per cent limits of agreement (LoA) between the two visits.
Results: Average difference between automated-ETDRS near visual acuity and near visual acuity
by paper chart was 0.02  0.10 logMAR (p > 0.05). Agreement of near visual acuity between
automated-ETDRS and paper chart was good, with 95 per cent LoA of 0.19 logMAR. Furthermore,
automated-ETDRS near visual acuity showed good repeatability (95 per cent LoA of 0.20). Mean
difference between automated-ETDRS intermediate visual acuity and intermediate visual acuity by
paper chart was 0.02  0.10 logMAR (p > 0.05). Agreement of intermediate visual acuity between
automated-ETDRS and paper chart was good, with 95 per cent LoA of 0.20 logMAR. In addition,
automated-ETDRS intermediate visual acuity had good repeatability (95 per cent LoA of 0.16).
Conclusion: Automated-ETDRS near and intermediate visual acuity measurement showed
Submitted: 27 February 2019 good repeatability and agreement with the gold standard chart-based visual acuity mea-
Revised: 20 September 2019 surement. The findings of this study indicate the automated visual acuity measurement sys-
Accepted for publication: 15 October 2019 tem may have potential for use in both patient care and clinical trials.

Key words: computer, intermediate vision, near vision, tablet, visual acuity

Visual acuity is the most common and pri-
mary measurement of visual function. In
addition, visual acuity is one of most com-
mon outcome measurements in clinical
research.1–5 The Early Treatment Diabetic
Retinopathy Study (ETDRS) chart is designed
for use in clinical studies where accurate and
repeatable visual acuity measurements are
required.6,7 Although many types of visual
acuity charts are used, the ETDRS chart is
accepted worldwide as the gold standard for
accurate visual acuity measurement.
With advances in technology, automated
visual acuity tests using electronic devices have
been developed and studied.8–11 The advan-
tages of automated visual acuity testing over
paper chart testing include use of a
standardised protocol, reduced risk of patients
memorising the letters, and limiting human
error in counting letters and calculating visual
acuity. Thus, many clinical trials use automated
visual acuity testing as the gold standard to
measure treatment outcomes.3–5
The purpose of this study was to deter-
mine the repeatability of an automated-
ETDRS near and intermediate visual acuity
measurement in subjects with normal visual
acuity and subjects with reduced visual acu-
ity. In addition, agreement of automated-
ETDRS with gold standard chart-based visual
acuity measurement was also studied.
Methods
Study population and data
collection
Both the study protocol and informed con-
sent forms were approved by the Institu-
tional Review Board of the Illinois College of
Optometry. In accordance with the guide-
lines of the Declaration of Helsinki, written
informed consent was obtained from each
subject.
Test procedure
A total of 51 sequential subjects who under-
went a comprehensive eye examination at
the Illinois Eye Institute, an urban eye clinic,
were enrolled into the study. Thirty-three
subjects had normal distance visual acuity
of 6/7.5 or better, with habitual refractive
correction. The remaining 18 subjects had
reduced habitual distance visual acuity
(from 6/9 to 6/30) due to cataract, glau-
coma, degenerative myopia, retinal disease,
diabetic retinopathy, and/or uncorrected
refractive error. Patients with distance visual
acuity worse than 20/100 were excluded
from this study.

© 2019 Optometry Australia Clinical and Experimental Optometry 2019

1
Automated-ETDRS visual acuity Pang, Sparschu and Nylin
Subjects wore their habitual refractive
correction for all tests. Right eye visual acu-
ity of each subject was measured with an
automated tablet-computer system (M&S
Technologies, Inc., Niles, IL, USA) and with a
paper ETDRS chart (Precision Vision, Wood-
stock, IL, USA) in a random sequence. Left
eye visual acuity was measured in two sub-
jects whose right eye vision was worse than
the inclusion limit.
Near visual acuity at 40 cm was measured
first, followed by intermediate visual acuity
at 67 cm. Both paper and automated-ETDRS
tests were measured using high-contrast let-
ters. Visual acuity with the paper ETDRS
chart was tested at the luminance of
100–110 cd/m2. For paper ETDRS chart, sub-
jects were asked to read the smallest line
they could read. Subjects were asked to read
the line below the smallest line that they
have read. The test was terminated if no letter
in the line was correctly read. Subjects were
given at least a one-minute break between
each test. A retest was performed one week
(3 days) after the first visit. Forty-eight sub-
jects (94 per cent) completed the retest.
Automated tablet-computer
visual acuity test algorithms
ETDRS letters were displayed on a computer
screen for subjects to read. The examiner
viewed a tablet that displayed the correct
answers (Figure S1). The computer screen
was auto-calibrated to the luminance level
of 85 cd/m2 (5 per cent) after 15 minutes
warm up. A Datacolor Spyder 5 colorimeter
as part of the automated-ETDRS system was
used to confirm the luminance level of the
computer screen. Glare was controlled by
using a 90o metal display tray that the com-
puter was set into; thus, a consistent angle
for the screen display was maintained. The
resolution of the 13.3 inch computer screen
was 3,200 × 1,800, which resulted in a reso-
lution of 276 pixels per inch, a pixel size of
0.79 minutes of arc at 40 cm, and a pixel
size of 0.47 minutes of arc at 67 cm. Filtered
optotypes were used for both near and
intermediate automated-ETDRS test. Two
phases were used to determine the visual
acuity endpoint.
Phase I: initial threshold
An ETDRS chart (from 6/60 to 6/3) was dis-
played on the computer screen and subjects
were instructed to read the smallest line for
which they could read all five letters. The
examiner submitted the visual acuity level
at which subjects correctly read all letters.
Clinical and Experimental Optometry 2019
2
Phase II: threshold
ETDRS letters at the initial threshold visual
acuity level the examiner submitted were
displayed on the computer screen as well as
smaller size lines of the chart. A blue dot
was shown next to the visual acuity level
one line below the submitted letter size and
subjects were instructed to read the line
next to the blue dot. The examiner submit-
ted the correct number of letters that sub-
jects read and then continued to instruct
subjects to read letters of decreasing size
(0.1 logMAR steps). The test stopped when
subjects were unable to correctly read any
letters on a line or there were no smaller
lines available to be tested. This ending
point protocol has been utilised in previous
studies.12–14 The system automatically calcu-
lated the logMAR visual acuity using the cor-
rect letters that subjects read.
The test results were displayed on the
computer after the completion of the mea-
surement with the following parameters:
eye tested, test distance, letter score for
visual acuity, logMAR visual acuity, and
Snellen visual acuity equivalent.
Data analysis
Visual acuity was converted to logMAR for
analysis. The Kolmogorov–Smirnov test was
used to check the normality of distributions
for all parameters. Near visual acuity and
intermediate visual acuity tested with the
paper chart were not normally distributed.
Wilcoxon signed rank test was performed
to determine the difference in visual acuity
tested by paper chart and automated-
ETDRS.
Test–retest reliability of the automated-
ETDRS was evaluated using the Bland–
Altman 95 per cent limits of agreement
(LoA) method. Agreement between the
automated-ETDRS and the gold standard
paper chart was also determined by the
Bland–Altman method.
Power calculation was performed after
data of 51 subjects were collected. Based on
the standard deviation of difference
between paper and automated-ETDRS visual
acuity test, there was 93.8 per cent power to
detect difference of 0.05 logMAR (0.5
logMAR line or 2.5 letters) between the two
tests.
All data were analysed using Statistical
Package for Social Sciences (IBM SPSS ver-
sion 21.0; SPSS Inc., IBM UK, Portsmouth,
UK) and MedCalc version 12.2.1 (MedCalc
Software, Ostend, Belgium). A p-value of
< 0.05 was considered statistically significant.
Results
Table 1 lists demographic characteristics of
the subjects. The average difference between
the automated-ETDRS near visual acuity and
near visual acuity by paper chart was
0.02  0.10 logMAR (one letter difference)
without statistical significance (Figure 1A). The
mean difference between the test and retest
measurements for both automated-ETDRS
near visual acuity and near visual acuity by
paper chart A was 0.02 logMAR (one letter)
with no statistical significance. Agreement of
near visual acuity between the automated-
ETDRS and paper chart (Figure 1A) was good,
with 95 per cent LoA of 0.19 logMAR. Fur-
thermore, the automated-ETDRS near visual
acuity showed good repeatability (95 per cent
LoA of 0.20, Figure 2A) between the two ses-
sions (one week apart), slightly better than
that of near visual acuity by paper (95 per cent
LoA of 0.24, Figure 3A).
Mean difference between the automated-
ETDRS intermediate visual acuity and interme-
diate visual acuity by paper chart was
0.02  0.10 logMAR (one letter difference)
without statistical significance (Figure 1B). Aver-
age differences between the test and retest
measurements were 0.02  0.12 logMAR and
0.01  0.10 logMAR for automated-ETDRS
intermediate visual acuity and the intermedi-
ate visual acuity by paper, respectively.
Agreement of intermediate visual acuity
between the automated-ETDRS and paper
chart was good, with 95 per cent LoA of 0.22
logMAR (Figure 1B). In addition, the automated-
n (%)
Visual acuity
6/7.5 or better 33 (64.7)
6/9 to 6/30 18 (35.3)
Gender
Female 42 (82.4)
Male 9 (17.6)
Race
Non-Hispanic Black 22 (43.1)
Hispanic/Latino 8 (15.7)
Non-Hispanic White 16 (31.4)
Asian 5 (9.8)
Age (years)
Range 22.6–91.1
Mean (SD) 46.7 (17.5)
Table 1. Demographic characteristics of
the subjects (n = 51)
© 2019 Optometry Australia
 Automated-ETDRS visual acuity Pang, Sparschu and Nylin

Figure 1. Agreement between automated-Early Treatment Diabetic Retinopathy Study visual acuity (A-ETDRS-VA) test and paper chart
visual acuity (P-VA) test. The difference between the first administration of each test was plotted against the mean for the two tests.
Exact confidence intervals were calculated and plotted.32 A: Near visual acuity test (NVA). B: Intermediate visual acuity test (IVA).

ETDRS intermediate visual acuity had good
repeatability (95 per cent LoA of 0.16)
between the two sessions (Figure 2B), with a
slightly better repeatability than that of interme-
diate visual acuity by paper (95 per cent LoA of
0.20, Figure 3B).
Repeatability of automated-ETDRS near
visual acuity and intermediate visual acuity
was not significantly different between sub-
jects with normal and reduced visual
acuity using independent-samples t-test (all
p values > 0.05).
Discussion
Repeatability of visual acuity has been
extensively studied.11,15–24 Repeatability of
distance visual acuity (95 per cent LoA) has
been reported at rates varying from 0.07

Figure 2. Repeatability of automated-Early Treatment Diabetic Retinopathy Study visual acuity (A-ETDRS-VA) two measurements, one
week apart. The difference between the first and second administrations of A-ETDRS-VA was plotted against the mean of the two measure-
ments. Exact confidence intervals were calculated and plotted.32 A: Near visual acuity test (NVA). B: Intermediate visual acuity test (IVA).

© 2019 Optometry Australia Clinical and Experimental Optometry 2019

3
Automated-ETDRS visual acuity Pang, Sparschu and Nylin
Figure 3. Repeatability of paper chart visual acuity (P-VA) two measurements, one week apart. The difference between the first
and second administrations of P-VA was plotted against the mean of the two measurements. Exact confidence intervals were cal-
culated and plotted.32 A: Near visual acuity test (NVA). B: Intermediate visual acuity test (IVA).
to 0.25 logMAR.15–17,20,22,25 Many factors
could contribute to test–retest variability,
including use of different visual acuity
charts, subject age (children and older indi-
viduals showing higher variability),26,27 opti-
cal defocus (higher variability),25 and ocular
abnormalities (higher variability).20,26
Despite numerous reports on repeatabil-
ity of distance visual acuity, fewer studies of
near visual acuity11,23,24 have been con-
ducted. Aslam et al.11 tested near visual acu-
ity at two times, 15–20 minutes apart, in
78 subjects with or without eye pathologies,
using an automated computer tablet-based
visual acuity system. They reported the
repeatability was 0.17 logMAR, which is
comparable to the current study. These
authors also found that the agreement
between their automated near visual acuity
and paper near chart (near Landolt C) was
0.31 logMAR, which is worse than the pre-
sent study.
Cho and Woo23 measured near visual acu-
ity in 55 young optometry students with
normal vision and reported that the repeat-
ability of near visual acuity using a paper
chart (Waterloo Four-Contrast logMAR Visual
Acuity Chart) was 0.06 logMAR. Lam et al.
measured near visual acuity in 55 young
optometry students with visual acuity 6/6 or
better and stated that the repeatability of
near visual acuity was 0.06 logMAR for a
PolyU paper chart and 0.12 logMAR for the
Precision paper charts.24 The repeatability
Clinical and Experimental Optometry 2019
4
of near visual acuity in the present study is
lower than in both studies by Cho and Woo
and Lam et al.23,24
The two studies referred to above tested
young adults with normal visual acuity.23,24
Both factors, young adult and normal visual
acuity, have been reported to be associated
with a better visual acuity repeatability,25–27
which could contribute to the disparities
between those studies and the present
work. In addition, subjects in both studies
were optometry students, who might under-
stand the test procedure better and provide
more reliable responses.
Other factors could affect test–retest
repeatability including termination rules of
visual acuity tests. Carkeet28 suggested that
a termination rule of four mistakes or more
on a line should be used for ETDRS charts
and single-letter scoring. Shah et al.29 tested
vision in 50 normal observers and deter-
mined that test–retest variability was
smaller for a termination rule of four or five
mistakes (0.14 logMAR) compared to the
rule of three mistakes or less. The present
study used a termination rule of five mis-
takes to minimise the test–retest variability
as suggested by previous authors.28,29
With the extensive use of computers, inter-
mediate visual acuity becomes more critical
in the lives of many individuals. However, only
one study of intermediate visual acuity was
identified after a thorough literature search.
Tsilimbaris et al.30 simulated presbyopia in
34 healthy young emmetropes or myopes
without ocular abnormalities. They reported
the mean intermediate visual acuity was
0.01  0.09; however, they did not study
repeatability of intermediate visual acuity. In
the present study, both the repeatability of
intermediate visual acuity using automated-
ETDRS and agreement of automated-ETDRS
with paper charts were comparable to that
of near visual acuity.
There are some limitations of the present
study. First, the letter sizes in the
automated-ETDRS system were limited by
the screen size of the monitor; thus, the
range of testing was limited to 6/3 to 6/60.
Although a short test distance can be
utilised in clinical practice to measure visual
acuity worse than 6/60, a shorter test dis-
tance may introduce other variables includ-
ing change of refractive correction. Second,
the resolution of the computer screen in the
present study was 3,200 × 1,800, which
resulted in a resolution of 276 pixels per inch
and a pixel size of 0.092 mm, which is lower
than suggested by Carkeet and Lister31 in
their study. This may potentially impact
ETDRS letter resolution of smaller size.
In summary, the present study deter-
mined the repeatability of automated-ETDRS
near and intermediate visual acuity mea-
surement on a wide age range of individuals
with normal or reduced vision. Two-session
repeatability was measured rather than
one-session repeatability. Because visual
© 2019 Optometry Australia

acuity is measured over time on different
visits in both patient care and clinical trials,
it is critical to establish two-session visual
acuity repeatability. The present study rev-
ealed that the automated-ETDRS near and
intermediate visual acuity measurements
had good repeatability and agreement with
the gold standard paper chart, which indi-
cates this automated visual acuity measure-
ment system may have potential for use in
both patient care and clinical trials.
ACKNOWLEDGEMENTS
The authors would like to thank M&S Technol-
ogies, Inc. for providing the automated tablet-
computer visual acuity system for this study.
The authors also would like to thank Dr Kelly
Frantz in helping edit the manuscript.
REFERENCES
1. Early treatment diabetic retinopathy study design and
baseline patient characteristics. ETDRS report number
7. Ophthalmology 1991; 98: 741–756.
2. Photocoagulation for diabetic macular edema. Early
treatment diabetic retinopathy study report number
1. Early treatment diabetic retinopathy study research
group. Arch Ophthalmol 1985; 103: 1796–1806.
3. Pediatric Eye Disease Investigator Group. A random-
ized trial of atropine vs. patching for treatment of
moderate amblyopia in children. Arch Ophthalmol
2002; 120: 268–278.
4. Holmes JM, Edwards AR, Beck RW et al. A randomized
pilot study of near activities versus non-near activities
during patching therapy for amblyopia. J AAPOS 2005;
9: 129–136.
5. Holmes JM, Kraker RT, Beck RW et al. A randomized
trial of prescribed patching regimens for treatment of
severe amblyopia in children. Ophthalmology 2003;
110: 2075–2087.
6. Told R, Baratsits M, Garhofer G et al. Early treatment
diabetic retinopathy study (ETDRS) visual acuity.
Ophthalmologe 2013; 110: 960–965.
7. Koenig S, Tonagel F, Schiefer U et al. Assessing visual
acuity across five disease types: ETDRS charts are
faster with clinical outcome comparable to Landolt Cs.
Graefes Arch Clin Exp Ophthalmol 2014; 252: 1093–1099.
8. Zhang ZT, Zhang SC, Huang XG et al. A pilot trial of
the iPad tablet computer as a portable device for
© 2019 Optometry Australia
Automated-ETDRS visual acuity Pang, Sparschu and Nylin
visual acuity testing. J Telemed Telecare 2013; 19:
55–59.
9. Bokinni Y, Shah N, Maguire O et al. Performance of a
computerised visual acuity measurement device in
subjects with age-related macular degeneration: com-
parison with gold standard ETDRS chart measure-
ments. Eye (Lond) 2015; 29: 1085–1091.
10. Black JM, Jacobs RJ, Phillips G et al. An assessment of
the iPad as a testing platform for distance visual acu-
ity in adults. BMJ Open 2013; 3: e002730.
11. Aslam TM, Parry NR, Murray IJ et al. Development and
testing of an automated computer tablet-based
method for self-testing of high and low contrast near
visual acuity in ophthalmic patients. Graefes Arch Clin
Exp Ophthalmol 2016; 254: 891–899.
12. Ravikumar A, Sarver EJ, Applegate RA. Change in
visual acuity is highly correlated with change in six
image quality metrics independent of wavefront error
and/or pupil diameter. J Vis 2012; 12: 11.
13. Ravikumar A, Marsack JD, Bedell HE et al. Change in
visual acuity is well correlated with change in image-
quality metrics for both normal and keratoconic
wavefront errors. J Vis 2013; 13: 28.
14. Applegate RA, Marsack JD, Thibos LN. Metrics of reti-
nal image quality predict visual performance in eyes
with 20/17 or better visual acuity. Optom Vis Sci 2006;
83: 635–640.
15. Camparini M, Cassinari P, Ferrigno L et al. ETDRS-fast:
implementing psychophysical adaptive methods to
standardized visual acuity measurement with ETDRS
charts. Invest Ophthalmol Vis Sci 2001; 42: 1226–1231.
16. Reeves BC, Wood JM, Hill AR. Reliability of high- and
low-contrast letter charts. Ophthalmic Physiol Opt
1993; 13: 17–26.
17. Kheterpal S, Jones HS, Auld R et al. Reliability of visual
acuity in children with reduced vision. Ophthalmic Phy-
siol Opt 1996; 16: 447–449.
18. Barrio A, Antona B, Puell MC. Repeatability of mesopic
visual acuity measurements using high- and low-
contrast ETDRS letter charts. Graefes Arch Clin Exp
Ophthalmol 2015; 253: 791–795.
19. Raasch TW, Bailey IL, Bullimore MA. Repeatability of
visual acuity measurement. Optom Vis Sci 1998; 75:
342–348.
20. Blackhurst DW, Maguire MG. Reproducibility of refrac-
tion and visual acuity measurement under a standard
protocol. The macular photocoagulation study group.
Retina 1989; 9: 163–169.
21. Lovie-Kitchin JE. Validity and reliability of visual acuity
measurements. Ophthalmic Physiol Opt 1988; 8: 363–370.
22. Manny RE, Hussein M, Gwiazda J et al. Repeatability of
ETDRS visual acuity in children. Invest Ophthalmol Vis
Sci 2003; 44: 3294–3300.
23. Cho P, Woo GC. Repeatability of the Waterloo four-
contrast LogMAR visual acuity chart and near vision
test card on a group of normal young adults. Ophthal-
mic Physiol Opt 2004; 24: 427–435.
24. Lam AK, Tong C, Tse J et al. Repeatability of near
visual acuity measurement at high and low contrast.
Clin Exp Optom 2008; 91: 447–452.
25. Rosser DA, Murdoch IE, Cousens SN. The effect of
optical defocus on the test-retest variability of visual
acuity measurements. Invest Ophthalmol Vis Sci 2004;
45: 1076–1079.
26. Dougherty BE, Flom RE, Bullimore MA. An evaluation
of the Mars letter contrast sensitivity test. Optom Vis
Sci 2005; 82: 970–975.
27. Elliott DB, Yang KC, Whitaker D. Visual acuity changes
throughout adulthood in normal, healthy eyes: seeing
beyond 6/6. Optom Vis Sci 1995; 72: 186–191.
28. Carkeet A. Modeling logMAR visual acuity scores:
effects of termination rules and alternative forced-
choice options. Optom Vis Sci 2001; 78: 529–538.
29. Shah N, Dakin SC, Whitaker HL et al. Effect of scoring
and termination rules on test-retest variability of a
novel high-pass letter acuity chart. Invest Ophthalmol
Vis Sci 2014; 55: 1386–1392.
30. Tsilimbaris MK, Plainis S, Tontos C et al. Normative
distance, intermediate and near visual acuity in simu-
lated presbyopia. Invest Ophthalmol Vis Sci 2011; 52:
2833.
31. Carkeet A, Lister LJ. Computer monitor pixellation and
Sloan letter visual acuity measurement. Ophthalmic
Physiol Opt 2018; 38: 144–151.
32. Carkeet A. Exact parametric confidence intervals for
bland-Altman limits of agreement. Optom Vis Sci 2015;
92: e71–e80.
Supporting information
Additional supporting information may be
found in the online version of this article at
the publisher’s website:
Figure S1. Automated-ETDRS (Early Treat-
ment Diabetic Retinopathy Study) measure-
ment. A: Automated-ETDRS measurement at
near in a subject. B: Automated-ETDRS let-
ters viewed by subjects. C: Automated-
ETDRS letters viewed by the examiner. D: An
example of automated-ETDRS at 6/6 level
viewed by the examiner.
Clinical and Experimental Optometry 2019