GAMES FOR HEALTH JOURNAL: Research, Development, and Clinical Applications

Volume 10, Number 2, 2021

ª Mary Ann Liebert, Inc.
DOI: 10.1089/g4h.2020.0069

A Videogame-Based Approach to Measuring Information

Processing Speed in Multiple Sclerosis Patients

Marco Pitteri, PhD, Caterina Dapor, MSc, Stefano Ziccardi, PhD, Maddalena Guandalini, MSc,
Riccardo Meggiato, and Massimiliano Calabrese, MD
Downloaded by National Taiwan University N.T.U. Package from www.liebertpub.com at 08/13/22. For personal use only.

Abstract

Objective: Slowing information processing speed (IPS) is a biomarker of neuronal damage in patients with
multiple sclerosis (pwMS). A focus on IPS might be the ideal solution in the perspective of promptly detecting
cognitive changes over time. We developed a tablet-based home-made videogame to test the sensitivity of this
device in measuring subclinical IPS in pwMS.
Materials and Methods: Forty-three pwMS without cognitive impairment and 20 healthy controls (HCs) were
administered the videogame task with a tablet. Response times (RTs) and accuracy were recorded.
Results: PwMS (mean RTs = 505.5 – 73.9 ms) were significantly slower than HCs (mean RTs = 462.3 – 40.3 ms,
P = 0.014) on the videogame task. A moderate but significant correlation (r = -0.35, P = 0.03) between mean
RTs and the Symbol Digit Modalities Test was observed.
Conclusion: Our videogame showed good sensitivity in measuring IPS in apparently cognitive normal pwMS.
Computerized testing might be useful in screening initial cognitive dysfunction that should be monitored as a
marker of underlying disease progression.
IRB approval Number is 2332CESC.

Keywords: Multiple sclerosis, Information processing speed, Computerized assessment, Videogame

Introduction less sensitive in detecting initial cognitive difficulties in

M ultiple sclerosis (MS) is a chronic immune-
mediated disease of the central nervous system
(CNS), characterized by multifocal areas of inflammatory
demyelination within both the white matter (WM) and the
gray matter (GM), neurodegeneration, and axonal injury.1
In addition to physical disability, cognitive impairment
(CI) is common in people with multiple sclerosis (pwMS),
with frequencies ranging from 43% to 70%.2 CI can occur
from the early stages of the disease,3,4 playing a criti-
cal prognostic role in predicting disability progression
and accumulation of cortical atrophy in the long term.5,6
Furthermore, CI results in a considerable disruption to
personal, occupational, and social functioning of pwMS,
with a consequent detrimental effect on patients’ quality
of life.7,8 Despite the paramount importance to promptly
identify the presence of CI, initial changes in cognitive
functioning are difficult to be detected, especially in the
early stages of the disease.4
The gold standard for cognitive assessment in pwMS
consists of paper-and-pencil tests that, might be relatively
those individuals who do not manifest overt signs of cog-
nitive impairment.4,9
By contrast, computer-based behavioral assessment can
offer numerous advantages in terms of sensitivity with re-
spect to paper-and-pencil tests. Several studies provided
evidence on behalf of the adoption of computerized tasks to
improve the detection of subtle signs of cognitive dysfunc-
tions in different clinical populations.10–12 The major ad-
vantage is that computer-based tasks offer the possibility to
simultaneously manipulate and control several variables, thus
providing the opportunity to design multifaceted and flexible
conditions that better simulate complex real-life demands.
Furthermore, computer-based tasks enable to record response
times (RTs) within a millisecond precision, a parameter
potentially highly sensitive to capture small changes in pro-
cessing speed capacities.
Slowing in information processing speed (IPS) is one of
the most common behavioral signs in pwMS,13–15 whose
impairment can exert a disturbing effect on the effective
functioning of other cognitive abilities. The neuronal un-
derpinnings of IPS have been widely investigated and a

Neurology Section, Department of Neurosciences, Biomedicine and Movement Sciences, University of Verona, Verona, Italy.

115
 116
consensus has been reached over the relationship between
reduced IPS and both structural and functional brain dam-
ages related to MS.16–18
In the context of an underlying disease that determines WM
and GM damage with consequent functional reorganization,19
IPS might account for the first marker of more global cognitive
inefficiency.15 It has been argued that a decline in processing
speed is likely one of the main and direct consequences of
demyelination caused either by normal aging or by an
underlying disease of the CNS.20,21
Recently, the high potential of measurement of basic re-
sponse speed with computerized testing has been recognized
also in pwMS, with promising results.18,22,23 Moreover,
computerized testing could potentially be used as a practical
tool to monitor cognitive changes of pwMS over time.23
Downloaded by National Taiwan University N.T.U. Package from www.liebertpub.com at 08/13/22. For personal use only.
In this perspective, we developed a task for measuring IPS
running on a videogame format. Videogaming is a popular form
of leisure activity that in recent years has attracted scientific
interests, mainly with reference to the field of rehabilitation.24,25
Designing a novel videogame allowed us to implement tasks
based on strong theoretical background but still characterized
by high technological appeal. Furthermore, by designing our
own videogame, it has been possible to implement a fine control
over the stimuli, the graphical interface, the task’s difficulty, as
well as the outcome measures of interest. Our long-term aim is
to develop a videogame to be used for a remote monitoring of
pwMS cognitive performance over time, which could be easily
self-administered at home.
With this preliminary study, we aimed primarily at ex-
ploring the ability of the videogame-based task to detect IPS
in a group of pwMS classified as having no CI with respect
to paper-and-pencil neuropsychological tests, investigating
whether (1) the RTs recorded with the videogame task were a
sensitive measure to detect signs of slower IPS in pwMS with
respect to a group of healthy controls (HCs); (2) the RTs
recorded with the videogame task correlated with the gold
standard for IPS assessment in pwMS, that is, the Symbol
Digit Modalities Test (SDMT).26
Method
Participants
Seventy participants were enrolled for this study, consisting
of 49 pwMS, recruited at the MS Centre of the Verona Uni-
versity Hospital (Verona, Italy), and 21 neurologically HCs.
Inclusion criteria for pwMS were diagnosis of MS,27 no
concomitant neurological conditions (other than MS), no
psychiatric or other pathological health conditions, no sub-
stance abuse, no hearing or visual impairment, no relapses
within the previous 6 months from enrollment, and absence
of global CI. A neuropsychological assessment, composed of
the Brief Repeatable Battery of neuropsychological tests28
and the Stroop Test,29 was available for all patients. The
median time between neuropsychological assessment and
videogame testing was 6.5 months. Neuropsychological as-
sessment was carried out by experienced neuropsychologists.
To be classified as having CI, pwMS had to fail three or more
neuropsychological tests.30
Inclusion criteria for HCs consisted of absence of neuro-
logical or psychiatric conditions, no substance abuse or
medications, and normal or corrected-to-normal vision and
hearing.
PITTERI ET AL.
Owing to software technical problems during data col-
lection, six pwMS and one HC were excluded before pro-
ceeding to statistical analyses. Hence, the final study sample
was composed of 43 pwMS and 20 HCs.
Physical disability of pwMS was measured with the Ex-
panded Disability Status Scale (EDSS).31 Functioning of
upper limbs was assessed by an experienced neurologist as
part of the neurological assessment; none of the patients had
motor difficulties of the upper limbs.
At the time of videogame testing, all pwMS presented
with a relapsing remitting (RR) form; 9 pwMS were not
treated with specific disease-modifying therapy for MS,
whereas 23 were treated with dimethyl fumarate, 5 with
fingolimod, 1 with natalizumab, 2 with teriflunomide, 1 with
interferon beta-1a, 1 with cladribine, and 1 with ocrelizumab.
The study was approved by the local ethic committee and
all participants provided written informed consent before
participation in the study.
The videogame
Graphics. All the software graphics were created using
the Gimp software (www.gimp.org), a cross-platform, open-
source image editor. Stimuli were designed by an expert
designer and were composed of three frames each, which has
been proven to be the optimal compromise between fluidity
and reactivity with respect to user interaction.
Programming. The software was written in C and Java
programming languages, to have a working prototype to
adapt to different interfaces and videogame levels. After
several tests, the software architecture was replaced by a
skeleton source code, solid and almost unchangeable, which
supported an external layer of easily modifiable functions.
This structure permitted to manipulate basic functions, such
as managing inputs, writing and uploading files, viewing
characters, and managing animations.
The Gamemaker (https://www.yoyogames.com/gamemaker)
videogame engine was used, an integrated development
environment specialized in the display of high-speed two-
dimensional graphics. At the same time, the external layer of
software’s own functions was created by converting part of
the previous C and Java code into JavaScript scripts. Java-
Script permitted to translate effectively the mathematical
algorithms that allow processing of all the data received
from user interactions. After that, to improve the videogame
usability, JavaScript has been converted to GML that allows
to extend the app to multiple platforms (the videogame runs
currently on Android operative system).
Hardware. The videogame was administered through a
touch-screen tablet (Nvidia Shield) with the following technical
specifications: dimensions = 221 · 126 · 9.2 mm; weight = 390 g;
display type = capacitive touchscreen, 16M colors; display
size = 185.6 cm2 (*66.6% screen-to-body ratio); display res-
olution = 1920 · 1200 pixels, 16:10 ratio (*283 ppi density);
memory storage = 16 GB; RAM = 2 GB; processor = Nvidia
Tegra K1; operative system = Android.
Stimuli and procedure. The scenario consisted of an
apartment building with nine windows (Fig. 1). A total of 80
trials have been implemented, with each trial starting with
 A VIDEOGAME TO MEASURING PROCESSING SPEED IN MS 117
Downloaded by National Taiwan University N.T.U. Package from www.liebertpub.com at 08/13/22. For personal use only.

FIG. 1. Examples of the videogame scenario.

one out of the nine windows opening; behind the window ucation, and gender) had an impact on the dependent vari-
immediately appeared the target (i.e., a figure representing a ables (i.e., mean RTs and accuracy). Based on the results of
prisoner—Fig. 1). The window remained open for a maxi- the regression analyses, two separate analysis of covariances
mum duration of 1000 ms. Participants were asked to touch (ANCOVAs) were performed to compare videogame’s mean
on the prisoner as quickly as possible as it appeared on the RTs (calculated on correct responses only) and accuracy
screen. As the target was touched, it disappeared behind the between the HC and pwMS groups.
closing roll-up shutter. After a variable time period (200 or To offer a measure of concurrent validity of the video-
800 ms), a new window opened, and a new target appeared. game RTs as a specific measurement of IPS, correlation
The position of the target (i.e., the window in which it ap- analysis between videogame mean RTs and the SDMT total
peared) was randomized across trials. The RTs, measured score was performed.
in milliseconds, and the number of correct responses were All the statistical analyses were executed with the soft-
recorded. ware JASP (Version 0.9.0.1; JASP Team, 2019).
With the purpose of making the task more engaging for the
players, a score was assigned to each response (i.e., +3 points
for each correct response, -3 points for each error; -1 point Results
for each omission) and displayed on the upper left corner of Demographic and clinical characteristics of the study
the screen (Fig. 1). Furthermore, different audio feedbacks sample are listed in Table 1.
were given in response to correct responses and errors.
Each participant was tested individually in a quiet room.
The administration started with a practice session (warm-up) Cognitive and clinical characteristics of pwMS
to familiarize with the videogame setting. After the warm-up,
participants were administered the videogame task, which The median EDSS score was 1.5 (range: 0–3.5) and the
overall lasted *2 minutes. mean disease duration was 7.6 – 5.5 years.
Together with the videogame, pwMS were administered According to the inclusion criteria, all the patients have
the SDMT, to rule out the possibility of the presence of IPS been classified as having no CI.30 Furthermore, none of the
impairment at the time of videogame testing. The SDMT was pwMS showed evidence of IPS impairment, as assessed with
chosen for this purpose as it is a widely recognized test of the SDMT (all z-scores > -1.5), at the time of videogame
visual IPS that might serve as the gold standard for the as- testing.
sessment of IPS in pwMS.32 In the SDMT, patients are
presented with a series of nine symbols, each of which is Table 1. Demographic and Clinical
paired with a single digit (1 to 9) in a key at the top of a sheet. Characteristics of People with Multiple
A pseudo-randomized sequence of the symbols is presented Sclerosis and Healthy Controls
to the patient, who is instructed to respond, as quickly as
possible, with the digit associated with each symbol. The pwMS (n = 43) HCs (n = 20)
final score is the total number of correct responses provided Gender (M/F) 11/32 13/7
within 90 seconds. The SDMT was considered impaired with Age (years) 36.4 – 8.6 34.4 – 12.3
z scores smaller than -1.5 standard deviation (SD) compared Education (years) 14.1 – 3.1 17.5 – 2.5
with the normative data.28 EDSS 1.5 (0–3.5) —
Disease duration (years) 7.6 – 5.5 —
Statistical analyses Means – SD were provided for continuous variables. Median
(range) was provided for EDSS.
Linear regression analyses (stepwise method) were used EDSS, Expanded Disability Status Scale; HCs, healthy controls;
to assess whether demographic characteristics (i.e., age, ed- pwMS, people with multiple sclerosis; SD, standard deviation.
 118
Videogame results
The linear regression analysis run on mean RTs evidenced
age as the only significant predictor (adjusted R2 = 0.21,
P < 0.001). Similarly, the linear regression analysis run on
the total number of correct responses showed only a signif-
icant effect of age (adjusted R2 = 0.08, P = 0.03). Based on
these results, the subsequent ANCOVAs were performed
including age as covariate.
A significant difference in mean RTs was found between
pwMS and HCs (F(1,60) = 5.463; P = 0.023; g2p = 0.08), with
pwMS being significantly slower than HCs (mean – SD pwMS:
505.5 – 73.9 ms; mean – SD HCs: 462.3 – 40.3; Fig. 2). By
contrast, no significant difference between the two groups
was observed on accuracy (F(1,60) = 1.835; P = 0.181;
g2p = 0.03): both pwMS and HCs showed high accuracy at
Downloaded by National Taiwan University N.T.U. Package from www.liebertpub.com at 08/13/22. For personal use only.
the videogame task (pwMS: 97.2% of correct responses;
HCs: 98.4% of correct responses).
The correlation analysis between videogame mean RTs
and the SDMT total score showed a significant negative
correlation (r = -0.35; P = 0.03; CI [-0.60 to -0.04]): the
higher the SDMT, the faster the patients’ response on the
videogame task.
Discussion
As far as we know, this is the first study in which it has been
investigated the efficacy of a videogame task as a screening
tool for evaluating IPS in a group of pwMS as compared with
a group of HCs.
The results showed that our tablet-based videogame is
indeed an effective tool to test IPS and it is sensitive in
discriminating a group of pwMS, identified as without CI on
the paper-and-pencil neuropsychological tests, from a group
of HCs. Specifically, a significant difference between pwMS
and HCs was observed on the videogame task in reference to
mean RTs, with pwMS being on average slower than HCs.
By contrast, no significant difference in the number of cor-
rect responses between the two groups was observed, even
though pwMS were slightly less accurate than HCs.
These results are in accordance with previous studies
in pwMS,18,23 in which it has been highlighted that speed of
response (i.e., RTs), rather than accuracy, might be consid-
FIG. 2. Difference between the two groups in the mean
RTs at the videogame task. *P < 0.05. HCs, healthy controls;
pwMS, patients with multiple sclerosis; RTs, response times.
PITTERI ET AL.
ered as the ideal measurement to focus on. IPS is the key
cognitive deficit in MS:14 when allowed adequate time to
process the required information, pwMS succeed in reaching
performance accuracy with rates comparable with those of
HCs,33,34 suggesting that even difficulties observed in other
cognitive domains (e.g., working memory, executive func-
tions, and verbal memory) are actually a predominant matter
of speed. Furthermore, IPS has been identified as the first
marker of cognitive difficulties due to neuronal and axonal
damage.15
Promptly detecting initial IPS inefficiencies is of para-
mount importance considering that slowed IPS might be
suggestive of an underlying progression of the disease that
should require further clinical consideration. However, the
initial detection of slowed IPS could be challenging by using
only paper-and-pencil tests, as they offer measures mainly
related to accuracy parameters. By contrast, recording RTs
through computerized paradigms might reveal initial IPS
inefficiencies that might not be captured by paper-and-pencil
tests. Accordingly, the results of this study fit well within this
framework: videogame mean RTs, but not accuracy, have
been found sensitive to slowed IPS in a group of pwMS
identified as without IPS impairment by the gold standard of
paper-and-pencil IPS tests, that is, the SDMT.
The SDMT is a reliable brief cognitive screening measure
that has been recommended for use in both screening and
monitoring perspectives in pwMS.35 In this study, we found
a significant correlation between the videogame mean RTs
and the SDMT total score, a result that provides evidence in
support of the concurrent validity of the videogame task as a
reliable measure of IPS in pwMS. According to this result,
our videogame task might potentially reveal itself as a valid
substitute of the SDMT in screening procedures.
The potential of the videogame format is outstanding.
A videogame is a very usable and friendly tool that ideally
allows for higher compliance, especially in youngest pa-
tients, in comparison with traditional tests. Videogame for-
mat might bring the same advantages as computerized
paradigms with respect to the rigorous implementation of
sensitive tasks and the high degree of accuracy in recording
IPS within a millisecond precision; in addition, videogames
offer an inherent playfulness not comparable with the setting
of laboratory-based computer tasks. Videogames are easy-
going, amusing, dynamic, and engaging, thus offering the
opportunity to dramatically reduce performance anxiety and
to allow for potential high acceptance rate by patients.
Despite the encouraging results, this study is not free from
limitations. First, we did not control for psychological fac-
tors, such as depression and anxiety, which might have had
an impact on videogame performance. Second, the study
sample is characterized by the presence of relatively young
pwMS. Even though this kind of testing might be particularly
suitable for the youngest patients, future studies should in-
clude also older individuals to evaluate whether the results
are generalizable to a broader range of pwMS. Similarly, the
generalizability of the present results to different MS phe-
notypes beyond the RR type has to be further investigated.
An interesting aspect that might be investigated in future
studies should foresee the inclusion of measures of every-day
life functioning to evaluate whether computerized tools
(such as our videogame) might indeed capture real-life de-
mands to a greater extent than paper-and-pencil tests.
 A VIDEOGAME TO MEASURING PROCESSING SPEED IN MS
Lastly, it is worth noting that computerized testing might
come with some limits. Fundamental questions about psy-
chometric obstacles, inadequate normative data, reliability,
and validity have been raised36 and it has been argued that
cautiousness is needed when adopting computerized as-
sessment tools in neuropsychological practice.37 With specific
reference to our videogame, then, further research over its
psychometric qualities, reliability, and usability is needed be-
fore extensively releasing it to the clinical practice.
Conclusion
CI in pwMS is extremely difficult to assess and monitor
with paper-and-pencil tests considering that various forms
of compensation (i.e., neural plasticity, cognitive reserve,
practice and learning effects) might contribute in covering
Downloaded by National Taiwan University N.T.U. Package from www.liebertpub.com at 08/13/22. For personal use only.
initial cognitive inefficiency. Analyzing patients’ RTs might
boost the potential of assessing actual IPS and monitoring
individuals’ variances in IPS over time, thus allowing to
detect warning signs (red flags) of potential disease pro-
gression to be further investigated with a more extensive
and comprehensive clinical evaluation. We expect that
computerized testing will have an essential role in clinical
practice in the near future, especially in screening and
monitoring procedures.38
Acknowledgments
We thank Agnese Tamanti for helping us with the data
management and Dr. Davide Massidda for his useful sug-
gestions about the statistical analyses. We also thank all the
participants who took part in this study for their time and
effort. We extend thanks to Biogen Idec for the support
provided (grant for the best research project at XLV Italian
Congress of Neurology).
Author Disclosure Statement
No competing financial interests exist.
Funding Information
No funding was directly addressed for the completion of
the study.
References
1. Lassmann H. Multiple sclerosis pathology: Inflammation
versus neurodegeneration. In: Translational Neuroimmu-
nology in Multiple Sclerosis. R Arnon and A Miller (eds);
Academic Press; 2016: pp 3–13.
2. Grzegorski T, Losy J. Cognitive impairment in multiple
sclerosis—A review of current knowledge and recent re-
search. Rev Neurosci 2017; 28:845–860.
3. Amato MP, Portaccio E, Goretti B, et al. Cognitive im-
pairment in early stages of multiple sclerosis. Neurol Sci
2010; 31(Suppl 2): S211–S214.
4. Pitteri M, Ziccardi S, Dapor C, et al. Lost in classification:
Lower cognitive functioning in apparently cognitive normal
newly diagnosed RRMS patients. Brain Sci 2019; 9:321.
5. Calabrese M, Gajofatto A, Gobbin F, et al. Late-onset
multiple sclerosis presenting with cognitive dysfunction
and severe cortical/infratentorial atrophy. Mult Scler 2015;
21:580–589.
119
6. Pitteri M, Romualdi C, Magliozzi R, et al. Cognitive im-
pairment predicts disability progression and cortical thin-
ning in MS: An 8-year study. Mult Scler 2017; 23:848–854.
7. Campbell J, Rashid W, Cercignani M, Langdon D. Cog-
nitive impairment among patients with multiple sclerosis:
Associations with employment and quality of life. Postgrad
Med J 2017; 93:143–147.
8. Bobholz JA, Rao SM. Cognitive dysfunction in multiple
sclerosis: A review of recent developments. Curr Opin
Neurol 2003; 16:283–288.
9. Wilken JA, Kane R, Sullivan CL, et al. The utility o f
computerized neuropsychological assessment of cognitive
dysfunction in patients with relapsing remitting multiple
sclerosis. Mult Scler 2003; 9:119–127.
10. Bonato M, Deouell LY. Hemispatial neglect: Computer-based
testing allows more sensitive quantification of attentional
disorders and recovery and might lead to better evaluation of
rehabilitation. Front Hum Neurosci 2013; 7:1–4.
11. Kane RL, Roebuck-Spencer T, Short P, et al. Identifying
and monitoring cognitive deficits in clinical populations
using Automated Neuropsychological Assessment Metrics
(ANAM) tests. Arch Clin Neuropsychol 2007; 22(SUPPL.
1):115–126.
12. Wojcik CM, Beier M, Costello K, et al. Computerized
neuropsychological assessment devices in multiple sclero-
sis: A systematic review. Mult Scler J 2019; 25:1848–1869.
13. Chiaravalloti ND, DeLuca J. Cognitive impairment in
multiple sclerosis. Lancet Neurol 2008; 7:1139–1151.
14. Costa SL, Genova HM, Deluca J, Chiaravalloti ND. In-
formation processing speed in multiple sclerosis: Past,
present, and future. Mult Scler J 2017; 23:772–789.
15. Van Schependom J, D’Hooghe MB, Cleynhens K, et al.
Reduced information processing speed as primum movens
for cognitive decline in MS. Mult Scler J 2015; 21:83–91.
16. Dineen RA, Vilisaar J, Hlinka J, et al. Disconnection as a
mechanism for cognitive dysfunction in multiple sclerosis.
Brain 2009; 132:239–249.
17. Manca R, Sharrack B, Paling D, et al. Brain connectivity
and cognitive processing speed in multiple sclerosis: A
systematic review. J Neurol Sci 2018; 388:115–127.
18. De Meijer L, Merlo D, Skibina O, et al. Monitoring cog-
nitive change in multiple sclerosis using a computerized
cognitive battery. Mult Scler J Exp Transl Clin 2018; 4:1–9.
19. Calabrese M, Magliozzi R, Ciccarelli O, et al. Exploring
the origins of grey matter damage in multiple sclerosis. Nat
Rev Neurosci 2015; 16:147–158.
20. Denney DR, Lynch SG, Parmenter BA, Horne N. Cognitive
impairment in relapsing and primary progressive multiple
sclerosis: Mostly a matter of speed. J Int Neuropsychol Soc
2004; 10:948–956.
21. Miller EM. Intelligence and brain myelination: A hypoth-
esis. Pers Individ Dif 1994; 17:803–832.
22. Golan D, Wilken J, Doniger GM, et al. Validity of a multi-
domain computerized cognitive assessment battery for pa-
tients with multiple sclerosis. Mult Scler Relat Disord 2019;
30:154–162.
23. Merlo D, Darby D, Kalincik T, et al. The feasibility, reli-
ability and concurrent validity of the MSReactor comput-
erized cognitive screening tool in multiple sclerosis. Ther
Adv Neurol Disord 2019; 12:1756286419859183.
24. Kühn S, Gallinat J, Mascherek A. Effects of computer
gaming on cognition, brain structure, and function: A crit-
ical reflection on existing literature. Dialogues Clin Neu-
rosci 2019; 21:319–330.
 120 PITTERI ET AL.

25. Latham AJ, Patston LLM, Tippett LJ. The virtual brain: 30
years of video-game play and cognitive abilities. Front
Psychol 2013; 4:1–10.
26. Smith A. Symbol Digit Modalities Test: Manual. Los An-
geles: Wester Psychological Services; 1982.
27. Polman CH, Reingold SC, Banwell B, et al. Diagnostic cri-
teria for multiple sclerosis: 2010 revisions to the McDonald
criteria. Ann Neurol 2011; 69:292–302.
28. Amato MP, Portaccio E, Goretti B, et al. The Rao’ s Brief
Repeatable Battery and Stroop test: Normative values with
age, education and gender corrections in an Italian popu-
lation. Mult Scler 2006; 12:787–793.
29. Caffarra P, Vezzadini G, Dieci F, et al. Una versione ab-
breviata del test di Stroop: dati normativi nella popolazione
italiana. [A short version of the Stroop test: Normative data
in an Italian population sample.] Riv Neurol 2002; 12:111–
Downloaded by National Taiwan University N.T.U. Package from www.liebertpub.com at 08/13/22. For personal use only.
36. Gates NJ, Kochan NA. Computerized and on-line neuro-
psychological testing for late-life cognition and neuro-
cognitive disorders: Are we there yet? Curr Opin Psychiatry
2015; 28:165–172.
37. Schmand B. Why are neuropsychologists so reluctant to
embrace modern assessment techniques? Clin Neuropsychol
2019; 33:209–219.
38. Rao SM. Role of Computerized screening in healthcare
teams: Why computerized testing is not the death of neu-
ropsychology. Arch Clin Neuropsychol 2018; 33:375–378.
Address correspondence to:
Marco Pitteri, PhD
Neurology Section
Department of Neurosciences, Biomedicine

115. and Movement Sciences

30. Amato MP, Zipoli V, Goretti B, et al. Benign multiple University of Verona
sclerosis: Cognitive, psychological and social aspects in a
Neurology B, Policlinico ‘‘G. B. Rossi’’ Borgo Roma
clinical cohort. J Neurol 2006; 253:1054–1059.
Piazzale L. A. Scuro, 10
31. Kurtzke JF. Rating neurologic impairment in multiple scle-
rosis: An expanded disability status scale (EDSS). Neurology Verona 37134
1983; 33:1444–1452. Italy
32. Benedict RHB, Deluca J, Phillips G, et al. Validity of the
Symbol Digit Modalities Test as a cognition performance E-mail: marco.pitteri@univr.it
outcome measure for multiple sclerosis. Mult Scler 2017;
23:721–733. Massimiliano Calabrese, MD
33. DeLuca J, Gaudino EA, Diamond BJ, et al. Acquisition and Neurology Section
storage deficits in multiple sclerosis. J Clin Exp Neu- Department of Neurosciences, Biomedicine
ropsychol 1998; 20:376–390. and Movement Sciences
34. Demaree HA, DeLuca J, Gaudino EA, Diamond BJ. Speed University of Verona
of information processing as a key deficit in multiple Neurology B, Policlinico ‘‘G. B. Rossi’’ Borgo Roma
sclerosis: Implications for rehabilitation. J Neurol Neuro- Piazzale L. A. Scuro, 10
surg Psychiatry 1999; 67:661–663. Verona 37134
35. Benedict RHB, Amato MP, Boringa J, et al. Brief Interna- Italy
tional Cognitive Assessment for MS (BICAMS): International
standards for validation. BMC Neurol 2012; 12:55. E-mail: massimiliano.calabrese@univr.it