Aging, Neuropsychology, and Cognition

A Journal on Normal and Dysfunctional Development

ISSN: 1382-5585 (Print) 1744-4128 (Online) Journal homepage: https://www.tandfonline.com/loi/nanc20

A new tool to assess amnestic mild cognitive

impairment in Turkish older adults: virtual
supermarket (VSM)

Hatice Eraslan Boz, Hatice Limoncu, Stelios Zygouris, Magda Tsolaki,

Dimitrios Giakoumis, Konstantinos Votis, Dimitrios Tzovaras, Vesile Öztürk &
Görsev G. Yener

To cite this article: Hatice Eraslan Boz, Hatice Limoncu, Stelios Zygouris, Magda Tsolaki,
Dimitrios Giakoumis, Konstantinos Votis, Dimitrios Tzovaras, Vesile Öztürk & Görsev
G. Yener (2019): A new tool to assess amnestic mild cognitive impairment in Turkish
older adults: virtual supermarket (VSM), Aging, Neuropsychology, and Cognition, DOI:
10.1080/13825585.2019.1663146

To link to this article: https://doi.org/10.1080/13825585.2019.1663146

Published online: 04 Sep 2019.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=nanc20
AGING, NEUROPSYCHOLOGY, AND COGNITION
https://doi.org/10.1080/13825585.2019.1663146

A new tool to assess amnestic mild cognitive impairment in

Turkish older adults: virtual supermarket (VSM)
Hatice Eraslan Boz a,b, Hatice Limoncua, Stelios Zygourisc,d, Magda Tsolakic,e,
Dimitrios Giakoumisf, Konstantinos Votisf, Dimitrios Tzovarasf, Vesile Öztürka,b
and Görsev G. Yenera,b
a
Department of Neurology, School of Medicine, Dokuz Eylul University, Izmir, Turkey; bDepartment of
Neurosciences, Institute of Health Sciences, Dokuz Eylul University, Izmir, Turkey; cDepartment of
Neurology, School of Medicine, Aristotle University of Thessaloniki, Greece, Greece; dNetwork Aging
Research, Heidelberg University, Heidelberg, Germany; eGreek Association of Alzheimer’s Disease and
Related Disorders, Thessaloniki, Greece; fCentre for Research & Technology Hellas/Information
Technologies Institute (CERTH/ITI), Thessaloniki, Greece

ABSTRACT ARTICLE HISTORY

The purpose of this study was to investigate cognitive functioning by Received 10 April 2019
administering the Virtual Supermarket (VSM) test in patients with Accepted 29 August 2019
amnestic mild cognitive impairment (aMCI, N = 37) and age and KEYWORDS
education-matched healthy controls (HCs, N = 52). An extensive neu- Amnestic mild cognitive
ropsychological test battery and the VSM were administered to all impairment; computerized
participants. The aMCI group exhibited lower performance and cognitive testing; serious
required more time to complete the VSM compared to HCs. Also, games; virtual
aMCI-Multiple Domain (aMCI-MD) patients performed worse in the environments; virtual
“Correct Types”, “Correct Quantities”, “Bought Unlisted”, “Correct shopping task
Money” variables compared to HCs. Moreover, aMCI-SD patients dis-
played lower performance in “Bought Unlisted” and “Correct Money”
variables compared to HCs. The VSM variables correlated with estab-
lished neuropsychological test scores. The VSM test was found to
discriminate between aMCI and HCs with a correct classification rate
(CCR) of 81%. This is a preliminary study showing that the VSM is
a valid, brief and user-friendly test. .

Introduction
Mild Cognitive Impairment (MCI) is defined as a transitional period between dementia and
normal aging. Cognitive decline in MCI appears to be greater than in normal aging (Petersen,
2004). Persons with MCI need more time to perform complex daily tasks such as shopping and
paying bills, make more mistakes and are less efficient compared to age-matched healthy
older adults. However, they can maintain independence in daily life with little or no help
(Albert et al., 2011). Patients with aMCI have been reported to have a higher risk of developing
Alzheimer’s Disease (Bennet et al., 2005). aMCI-SD is characterized by an isolated memory
impairment that is below 1.5 standard deviations or more compared to age-corrected norms
(Larrieu et al., 2002). aMCI-MD is defined as impairments in memory and additionally in other
cognitive domains (Jicha et al., 2006; Markesbery et al., 2006; Morris, 2006; Ritchie, Artero, &

CONTACT Hatice Eraslan Boz hatice.eraslan@deu.edu.tr

© 2019 Informa UK Limited, trading as Taylor & Francis Group
2 H. ERASLAN BOZ ET AL.

Touchon, 2001). The diagnosis of MCI is reached through neurological examination, mental
status examination, a thorough clinical interview, and standardized neuropsychological test-
ing (Knopman & Petersen, 2014; Tangalos & Petersen, 2018).
Patients with MCI should be assessed regularly and monitored for the risk of
progression to dementia (Arevalo-Rodriguez et al., 2015). It is generally recom-
mended to use brief cognitive screening tools assessing global cognition, executive
function, attention, and memory (Petersen et al., 2001). For this purpose, a detailed
evaluation of memory and other cognitive functions of MCI patients, must be con-
ducted to provide an accurate clinical diagnosis (Oktem, 2003). Recently some
studies, have used computerized tests, simulated shopping tasks or virtual reality
technologies to detect MCI. A commonly used computerized test, the CogState
Battery includes the International Shopping List Test (ISL), a verbal learning task
that can be used to distinguish between MCI, AD and HCs based on the level of
verbal memory impairment (Lim et al., 2012). Furthermore a study using the brief
version of CogState, the CogState Brief Battery (CBB), indicated that cerebral Aβ is
associated with visual and verbal memory in APOE ɛ4 carriers (Ellis et al., 2012), and
demonstrated that CBB displays a high level of sensitivity and specificity for detect-
ing cognitive impairment in MCI and AD, and MCI and healthy controls (Hammers
et al., 2012; Maruff et al., 2013).
Virtual Reality (VR) technologies are increasingly being used in the evaluation of
cognitive functions. In virtual reality environments, a person is able to freely explore
the virtual world, which is a simulation of the real world, and interact with it (Zygouris
et al., 2015, 2017). Among the virtual reality tests, the Virtual Daily Living Test (VDLT)
displayed improved correct classification rates for the differentiation of healthy elderly
and MCI patients in comparison to traditional pencil and paper tests (Seo, Kim, Oh, Ryu,
& Choi, 2017). Another task, the virtual reality navigation task (VRNT), was able to
distinguish aMCI-MD from healthy controls. It displayed a strong correlation of the
VRNT with Rey Auditory Verbal Learning Test and Rey Complex Figure Test
(Mohammadi, Kargar, & Hesami, 2018). Another task, the Virtual Reality Functional
Capacity Assessment Tool (VRFCAT), assessed the functional ability related to
a shopping trip and yielded a strong relationship between the task and cognitive
performance in healthy young and older adults and in patients with subjective cognitive
decline (Atkins et al., 2018).
In this study, we aimed to evaluate cognitive decline in aMCI and its subtypes using
the virtual supermarket (VSM). A previous study showed that the VSM can differentiate
between patients with aMCI and HCs with 87% CCR in a Greek population (Zygouris
et al., 2015). Therefore, we aimed to evaluate whether the VSM is a useful screening tool
for MCI in a Turkish speaking population. In addition, we investigated whether there is
a relationship between performance in the VSM and performance in establishing neu-
ropsychological tests. Previous studies showed that the VSM is associated with neurop-
sychological test performance in the domains of memory, attention, and executive
function (Zygouris et al., 2015, 2017). To our knowledge, this is the first study using
virtual reality in the cognitive evaluation of aMCI patients in a Turkish speaking older
adult population.
 AGING, NEUROPSYCHOLOGY, AND COGNITION 3

Materials and methods

Participants
The study included a total of 37 patients with a diagnosis of aMCI (age range 53–83, mean age
70.41 years) according to Petersen criteria (Petersen et al., 2009) and 52 healthy controls (age
range 58–82, mean age 67.56 years) recruited from the dementia and general outpatient
clinics of the Dokuz Eylul University Hospital, Department of Neurology. The study was
conducted between January 2016 and November 2017. The mean education level was
11.16 years for the aMCI group and 10.54 for the healthy controls (HCs) group. There were
35 female subjects in HCs and 12 female subjects in the aMCI group. All participants under-
went complete physical and neurological examinations. Patients with aMCI were selected
according to Petersen criteria (Petersen et al., 2009). The appropriate age and education
corrected norms were used to calculate neuropsychological test scores of all participants.
Participants with normal scores according to these norms were considered, for allocation to
the HCs group and participants who scored at least 1.5 standard deviation below these norms
were considered as the MCI. All but one of our aMCI participants had an intact functioning as
the Lawton-Brody IADL scores indicated. The scale was administered to relatives who rated
the daily functioning of the participants. These scores, medical history and a clinical interview
were used in conferring a diagnosis of aMCI or HCs. Depression, other neurological and
psychiatric diseases were excluded by a clinical interview and the Geriatric Depression Scale.
The general and clinical features of the groups are summarized in Table 1. The study protocol
was approved by the Ethical Committee of Dokuz Eylul University. All participants provided
informed consent.

Neuropsychological assessment
A detailed neuropsychological test battery was administered to participants. It included:
Oktem Verbal Memory Processes Test (Oktem, 1992), Wechsler Memory Scale-Revised
(WMS-R) Visual Reproduction Subtest (Wechsler, 1987), Stroop Test (Stroop, 1935), Digit
Span, Figure Copying Test, Clock Drawing Test (Rouleau, Salmon, Butters, Kennedy, &
McGuire, 1992), the Turkish version of revised Mini-Mental State Examination Test (rMMSE-T)

Table 1. Clinical and demographical characteristics of aMCI subgroups and HCs.

HCs aMCI aMCI-SD aMCI-MD p
(N = 52) (N = 37) (N = 19) (N = 18) (MCI-HCs)
Age (Mean ± SD) 67.56 70.41 68.74 72.17 .057a
(6.044) (7.297) (7.248) (7.123)
Education Years 10.54 11.16 12.32 9.94 .568a
(Mean ± SD) (3.71) (3.149) (2.473) (3.386)
Gender (M/F) 17/35 25/12 14/5 11/7 .001b
Hand Dominance (R/L) 50/2 34/3 17/2 17/1 .389b
MMSE (Mean ± SD) 29.29 27.54 28.44 26.59 .000a
(1.016) (1.771) (0.984) (1.938)
BAI (Mean ± SD) 3.88 2.67 2.78 2.56 .542a
(4.213) (2.426) (2.798) (2.064)
GDS (Mean ± SD) 5.68 5.61 6.33 4.89 .886a
(4.186) (4.468) (4.511) (4.431)
Note. aMCI-SD/MD = Mild Cognitive Impairment Single-Domain/Multi-Domain; MMSE = Mini-Mental State Examination;
GDS = Geriatric Depression Scale; BAI = Beck Anxiety Inventory; SD = Standard Deviation, M = Male, F = Female,
a
Mann Whitney-U, bPearson Chi-Square, p < 0.05.
4 H. ERASLAN BOZ ET AL.

(Folstein, Folstein, & Mc Hugh, 1975; Keskinoglu et al., 2009), Verbal Fluency Test (categorical
and phonemic) (Martin, Wiggs, Lalonde, & Mack, 1994; Troyer, Moscovitch, Winocur, Alexander,
& Stuss, 1998; Tumac, 1997), Boston Naming Test (Kaplan, Goodglass, & Weintraub, 2001),
Wechsler Adult Intelligence Scale-III (WAIS-III) Similarities Subtest (Wechsler, 1981), Luria
Alternan Sequences Test (Luria, 1980). Also, the Beck Anxiety Inventory (BAI) (Beck, Epstein,
Brown, & Steer, 1988; Ulusoy, Sahin, & Erkmen, 1998), Geriatric Depression Scale (GDS) (Ertan,
Eker, & Sar, 1997; Yesavage et al., 1982), and the Lawton Instrumental Activities of Daily Living
(IADL) (Lawton & Brody, 1969).

Virtual supermarket
Virtual Supermarket (VSM) was developed by the Center for Research & Technology
Hellas/Information Technologies Institute (CERTH/ITI) in Association with the Greek
Association of Alzheimer’s Disease and Related Disorders (GAADRD) as a screening tool
for MCI detection (Zygouris et al., 2015). VSM was previously used in two studies in Greece
and it has been shown to be a valid and accurate tool for detecting MCI in older adults
(Zygouris et al., 2015, 2017). It can be administered through a PC or a tablet device. The
VSM software was translated and adapted to Turkish.
The VSM is based on a daily shopping activity. Before engaging in the exercise, the
participant is asked to answer questions about his or her age, gender, occupation, years
of education and possible memory complaints. A shopping list appears in the upper
right corner of the screen during the VSM exercise. The person is expected to locate the
items on this list, place them in the shopping cart, take them to the cashier desk and pay
the correct amount for the purchases. The participant is asked to move the shopping
cart and navigate inside the virtual supermarket by touching green footprints on the
screen. It is an exercise designed for examining multiple cognitive domains such as
visual and verbal memory, executive functions, attention, and spatial navigation. The
exercise, which requires simultaneous activation of different cognitive processes, has
been developed for healthy adults and MCI patients. The contents of the shopping list
are randomly generated in each trial to prevent practice effects. The VSM allows the user
to actively explore the artificial environment since active exploration enhances learning
and memory (Zygouris et al., 2015).

Administration of the VSM

Participants were asked whether they knew how to use computers or tablets before using to
the VSM, and how often they use them if they use them. All participants used the VSM
application twice at difficulty level 1. In these trial administrations, instructions were
repeated to participants and they received help by the examiner from the beginning till
the end of the testing procedure. Then they were administered the VSM at difficulty level 2
and while they were given instructions before starting the VSM administration, they did not
receive help from the examiner while completing the test. As in the previous study (Zygouris
et al., 2015) a tablet device with a 10-inch touchscreen was used during the VSM.
The VSM was scored in five variables. These are “Correct Items”, “Correct Quantities”,
“Bought Unlisted”, “Correct Money” and “Duration”. All VSM variables except “Duration”
were nominal variables, encoded on SPSS as “true” and “false”. In all nominal variables,
 AGING, NEUROPSYCHOLOGY, AND COGNITION 5

a score of 0 indicates a correct response and a score of 1 indicates a false response by the user.
Administration time for the VSM, in this study, ranges at approximately 25 minutes.

Statistical analysis
All variables were analyzed in IBM® SPSS® Statistics 22.0 program. Age was normally distrib-
uted; education, gender, the VSM and neuropsychological test variables were not normally
distributed, as assessed by the Kolmogorov-Smirnov test. Therefore, all analyzes were per-
formed in non-parametric conditions. The effect of age, gender and education on the VSM
variables was tested using Chi-square and Kruskal Wallis test. The effect of education level on
VSM variables was assessed through One-Way ANCOVA with Bonferroni adjustment.
“Correct Types”, “Correct Quantities”, “Bought Unlisted”, and “Correct Money” vari-
ables were categorical, so Chi-Square test was employed to find differences between
groups. The continuous variables such as the “Duration” VSM variable and neuropsycho-
logical test scores, were compared between groups by Kruskal-Wallis Test. Chi-Square
post-hoc analysis test with Bonferroni correction was used for pairwise comparisons for
aMCI subtypes and HCs for all categorical VSM variables, whilst Kruskal-Wallis Test and
post-hoc Dunn-Bonferroni Correction were used for “Duration” variable, and adjusted
significance level was set at p < 0.05 for all comparisons.
The neuropsychological test scores of all participants, including verbal memory, visual
memory, attention, executive functions, and language skills were transformed into
z-scores. Spearman correlation analysis was used to calculate the correlation of these
z-scores with the VSM duration variable. The point-biserial correlation analysis was
performed between all categorical VSM variables and z-scores of neuropsychological
tests. The correct classification rate, sensitivity and specificity of the VSM and neuropsy-
chological tests were analyzed with Fisher’s Linear Discriminant Function Analysis.

Results
Clinical and demographical features in aMCI and HCs
There were no significant differences between the aMCI and HCs groups in education (Mann-
Whitney U, Z = −0.571, p = 0.568) and age (Mann-Whitney U, Z = −1.905, p = 0.057). However,
there was a gender difference between the aMCI and HCs (Chi-Square = 10.551, p = 0.001).
Additionally, there were differences between the aMCI and HCs groups in MMSE scores
(Mann-Whitney U, Z = −5.182, p = .000) similarly, between the aMCI-SD and aMCI-MD groups
in MMSE scores (Mann-Whitney U, Z = −3.408, p = 0.001). Besides, there were significant
differences between aMCI-SD and aMCI-MD in education level (Mann-Whitney U, Z = −2.184,
p = 0.029). No significant differences were noted between all groups in GDS and BAI scores.
The clinical and demographical features are presented in Table 1.

Gender, education and age effect on VSM variables

There was no effect of gender and age on the “Correct Types”, “Correct Quantities”, “Bought
Unlisted” and “Correct Money” variables (Chi-square, p > 0.05). There was an effect of
education on the “Correct Types” and “Correct Quantities” variables (Kruskal Wallis Test,
6 H. ERASLAN BOZ ET AL.

p < 0.05). The effect of education on the “Correct Types” and “Correct Quantities” variables
was controlled with ANCOVA analysis. Furthermore, there was no effect of education on the
“Bought Unlisted”, “Correct Money” and “Duration” (Kruskal Wallis Test, p > 0.05). In addition,
there was no effect of age, gender, and education on “Duration” variable (Kruskal Wallis Test
and Chi-Square, p > 0.05).

Correlations between VSM variables, age, education, and neuropsychological

tests
Z-scores of neuropsychological tests including verbal memory, visual memory, executive
function, attention, language, and general cognitive status were calculated in our sample.
Attention included forward and backward digit span tests; verbal memory included the
Oktem Verbal Memory Processes Test; visual memory included the WMS-R Visual
Reproduction Test (immediate and delayed); executive function included the Stroop Test,
WAIS-III Similarities subtest, Luria Alternan Sequences Test, verbal categorical and phonemic
fluency, Clock Drawing Test; language included the Boston Naming Test; visual-spatial
construction ability included figure copying test, and general cognitive status included
the MMSE. There were positive correlations between age and “Duration” (Spearman
Correlation analysis, rho = 0.460, p = 0.000), “Correct Types” (Point-biserial correlation
analysis, rpb = 0.220, p = 0.039), Correct Quantities” (Point-biserial correlation analysis,
rpb = 0.328, p = 0.002), “Bought Unlisted” (Point-biserial correlation analysis, rpb = 0.369,
p = 0.000), and “Correct Money” (Point-biserial correlation analysis, rpb = 0.324, p = 0.002),
and negative correlations between education and “Duration” (Spearman Correlation analy-
sis, rho = −0.336, p = 0.000), “Correct Types” (Point-biserial correlation analysis, rpb = −0.270,
p = 0.011), “Correct Quantities” (Point-biserial correlation analysis, rpb = −0.372, p = 0.000),
and “Bought Unlisted” (Point-biserial correlation analysis, rpb = −0.230, p = 0.030). Many
correlations existed between the composite z-scores of neuropsychological tests and VSM
variables, and they are presented in Table 2.

Table 2. Correlations between z-scores of neuropsychological tests and VSM variables.

VSM Variables
Neuropsychological domains (composite Correct Correct Bought Correct
z-scores) Typesa Quantitiesa Unlisteda Moneya Durationb
General cognitive status -,318** -,369** -,523** -,494**
Verbal Memory -,221* -,371** -,467** -,550** -,334**
Visual Memory -,231* -,293** -,616** -,492** -,380**
Attention -,319** -,235* -,288**
Executive Function -,239* -,379** -,291** -,265*
Language -,272**
Visual-spatial construction -,355** -,209* -,377**
Note. *Correlation is significant at the 0.05 level (2-tailed); **Correlation is significant at the 0.001 level (2-tailed);
a
Point-biserial correlation; b Spearman Correlation. Correlation of VSM variables with composite z-scores of neurop-
sychological tests was indicated. The general cognitive status includes of the MMSE; verbal memory includes the
composite z-score of the Oktem Verbal Memory Processes Test; visual memory includes composite z-score of the
WMS-R Visual Reproduction Test; executive function includes the Stroop Test, the Clock Drawing Test, WAIS-III
Similarities subtest, Luria Alternan Sequences Test, verbal categorical and phonemic fluency test; attention includes
the composite z-scores of the forward and backward digit span tests; language includes the composite z-scores of the
Boston Naming Test; and visual-spatial construction ability includes the composite z score of the figure copying test.
 AGING, NEUROPSYCHOLOGY, AND COGNITION 7

VSM variables over HCs and amci

The correct classification rate (CCR), sensitivity and specificity of the VSM variables for
differentiating between aMCI and HCs were presented in Table 3. A significant difference
was found in terms of “Correct Types” (Chi-Square = 6.095, p = 0.014), “Correct Quantities”
(Chi-Square = 7.169, p = 0.007), “Bought Unlisted” (Chi-Square = 21.502, p = 0.000), “Correct
Money” (Chi-Square = 31.711, p = 0.000) and “Duration” variables (Mann-Whitney U,
Z = −3.043, p = 0.002) between aMCI and HCs.

VSM variables between aMCI subgroups and HCs

There were significant differences between aMCI-MD and HCs in terms of “Correct Types”
(One-way ANCOVA, Adjusted R2 = 0.132, F(2,85) = 7.501, p = 0.022) by controlling for effect of
education, “Correct Quantities” (One-way ANCOVA, Adjusted R2 = 0.268, F(2,85) = 13.803,
p = 0.000), “Bought Unlisted” (Chi-Square-Bonferroni Adjusted, p < 0.05), “Correct Money” (Chi-
Square-Bonferroni Adjusted, p < 0.05), and “Duration” (Kruskal Wallis, post-hoc Dunn-
Bonferroni Test, p = 0.001).
A significant difference was found in terms of “Bought Unlisted” (Chi-Square-Bonferroni
Adjusted, p < 0.05) and “Correct Money” (Chi-Square-Bonferroni Adjusted, p < 0.05)
between aMCI-SD and HCs.
In addition, there were no significant difference between aMCI-MD and aMCI-SD in
“Correct Types” (One-way ANCOVA, Adjusted R2 = 0.154, F(1,34) = 0.008, p = 0.931)
“Correct Quantities” (One-way ANCOVA, Adjusted R2 = 0.210, F(1,34) = 3.486, p = 0.071),
“Bought Unlisted” (One-way ANCOVA, Adjusted R2 = 0.148, F(1,34) = 4.053, p = 0.052),
“Correct Money” (One-way ANCOVA, Adjusted R2 = 0.031, F(1,34) = 0.982, p = 0.329, and
“Duration” (One-way ANCOVA, Adjusted R2 = 0.036, F(1,34) = 0.768, p = 0.387) control-
ling for education level (in Table 3).

Table 3. VSM variables of CCR, specificity, and sensitivity for discriminating between HCs and aMCI.
CCR Sensitivity Specificity p Pairwise
VSM Variables % % % aMCI-HCs comparisons
Correct Types 64 62 86 0.014b aMCI-SD – HCs = >0.05d
aMCI-MD – HCs = <0.05d
aMCI-SD – aMCI-MD = 0.931f
Correct Quantities 66 66 68 0.007b aMCI-SD – HCs≤0.05d
aMCI-MD – HCs = <0.05d
aMCI-SD – aMCI-MD = 0.071f
Bought Unlisted 75 73 83 0.000b aMCI-SD – HCs = <0.05d
aMCI-MD – HCs = <0.05d
aMCI-SD – aMCI-MD = 0.052f
Correct Money 79 73 100 0.000b aMCI-SD – HCs = <0.05d
aMCI-MD – HCs = <0.05d
aMCI-SD – aMCI-MD≤0.329f
Duration 67 68 67 0.002a aMCI-SD – HCs = 0.58c
aMCI-MD – HCs = 0.001c
aMCI-SD – aMCI-MD = 0.387f
Note. VSM = Virtual Supermarket; CCR = Correct classification rate; aMann-Whitney U; bPearson Chi-Square; cKruskal Wallis-
Dunn-Bonferroni; dChi-Square post-hoc analysis with Bonferroni adjusted; fOne-Way ANCOVA (between aMCI-MD and
aMCI-SD controlling for education).
8 H. ERASLAN BOZ ET AL.

Discrimination analysis of all VSM variables and the MMSE between aMCI and HCs
As shown in Table 4, the CCR, specificity, and sensitivity of the combination of all VSM
variables for differentiating between aMCI and HCs were calculated as 80%, 74%, and 85%,
respectively. The MMSE by itself had a CCR at 74%, a sensitivity of 71% and specificity of 85%
for detecting aMCI. A combination of all VSM variables and the MMSE exhibited a CCR of
82% with, sensitivity and specificity rates of 77% and 92%, respectively.

Combinations of VSM variables and the MMSE over aMCI and HCs
The classification accuracy of various combinations of VSM variables was analyzed by
using Fisher’s Linear Discriminant Functions in Original Classification and the best
combination of VSM variables for discrimination between aMCI and HCs groups, was
determined to be “Bought Unlisted”, “Correct Money” and “Duration”. The combination
of these variables displayed a CCR of 81%, a sensitivity of 79%, and a specificity of 86%.
Combining these three VSM variables with the MMSE yielded a CCR of 83%, a sensitivity
of 78%, and a specificity of 100%. These results are presented in Table 4.

Discrimination analysis of all VSM variables and the MMSE between aMCI
subtypes
The CCR, specificity, and sensitivity for differentiation between aMCI-SD and aMCI-MD by
using the MMSE and combinations of VSM variables are presented in Table 5 andTable 6.
The CCR, specificity, and sensitivity when using all VSM variables to differentiate
between MCI subtypes were calculated as 73%, 63%, and 72%, respectively. The MMSE
detected aMCI subgroups with a CCR at 81%, and sensitivity and specificity of 80% and
82%, respectively. A combination of all VSM variables and the MMSE yielded a CCR of
80%, a specificity of 79% and a sensitivity of 81% for differentiating between aMCI
subtypes.

Table 4. The CCR, specificity and sensitivity of VSM variables, neuropsychological tests, and MMSE
for discriminating between aMCI and HCs.
HCs-Predicted
aMCI and HCs HCs Yes No CCR % Sensitivity % Specificity %
All VSM variables Yes 48 4 80 74 85
No 14 23
MMSE Yes 49 3 74 71 85
No 20 17
VSM & MMSE No 15 22 82 77 92
No 15 22
Bought Unlisted & Correct Money& Duration Yes 48 4 81 79 86
No 13 24
Bought Unlisted & Correct Money& Duration & MMSE Yes 52 0 83 78 100
No 15 22
Note. VSM = Virtual Supermarket; MMSE = Mini Mental State Examination; CCR = Correct Classification Rate.
 AGING, NEUROPSYCHOLOGY, AND COGNITION 9

Table 5. The CCR, specificity, and sensitivity of VSM variables for

discriminating between aMCI-SD and aMCI-MD.
CCR Sensitivity Specificity
VSM Variables % % %
Correct Types 56.8 54.8 66.6
Correct Quantities 70.3 66.6 76.9
Bought Unlisted 70.3 72.2 68.4
Correct Money 62.2 63.1 61.6
Duration 59.5 60 58.8
Note. VSM = Virtual Supermarket; CCR = Correct classification rate.

Table 6. The CCR, specificity and sensitivity of VSM variables, neuropsychological tests, and MMSE
for discriminating between aMCI-SD and aMCI-MD.
aMCI CCR
aMCI-SD/MD aMCI aMCI-SD aMCI-MD % Sensitivity % Specificity %
All VSM variables aMCI-SD 14 5 73 63 72
aMCI-MD 5 13
MMSE aMCI-SD 16 3 81 80 82
aMCI-MD 4 14
VSM & MMSE aMCI-SD 15 4 80 79 81
aMCI-MD 5 13
Correct Quantities & Bought Unlisted aMCI-SD 12 7 68 70.5 65
aMCI-MD 5 13
Correct Quantities & Bought Unlisted & MMSE aMCI-SD 16 2 74 69.5 83
aMCI-MD 7 10
Note. VSM = Virtual Supermarket; MMSE = Mini Mental State Examination; CCR = Correct Classification Rate.

An effort to find the best combination of variables for differentiating between MCI
subtypes indicated that the best classification results were obtained by using all VSM
variables (in Table 6).

Discussion
MCI is known to be a risk factor for Alzheimer’s Disease. Therefore, early diagnosis of MCI
and suitable interventions are crucial in disease monitoring (Albert et al., 2011). The Virtual
Supermarket (VSM) has been developed to assess cognitive functioning in MCI and it can
also be used for cognitive training (Zygouris et al., 2015). In this study, we examined whether
the VSM was a potentially useful test for detecting MCI in a Turkish speaking population. We
found that the VSM is effective in distinguishing aMCI subjects from healthy older adults,
but not very effective in discriminating between MCI subtypes. In addition, performance in
the VSM is correlated with performance on neuropsychological tests that assess verbal
memory, visual memory, attention, executive function, and global cognition. The same
correlations were present in the first VSM study in a sample of Greek older adults (Zygouris
et al., 2015).

VSM variables
In the current VSM study, aMCI patients bought more wrong items and needed more
time to complete the VSM exercise compared to HCs. In all VSM measurements, aMCI-
10 H. ERASLAN BOZ ET AL.

MD showed lower performance compared to HCs. In “Bought Unlisted” and “Correct

Money”, aMCI-SD showed lower performance than HCs. These findings may be asso-
ciated with impaired attention and lack of inhibition in MCI patients (Belleville,
Chertkow, & Gauthier, 2007; Borella et al., 2017). Patients with aMCI-MD and aMCI-SD
displayed similar performance in terms of “Correct Types”, “Correct Quantities”, “Bought
Unlisted”, “Correct Money” and “Duration” controlling for education. Our findings are not
consisted with the findings of the first VSM study (Zygouris et al., 2015, 2017) since the
aMCI-SD and aMCI-MD groups differed in terms of education level.
The aMCI patient group has made more mistakes compared to HCs in the “Correct
Money” variable, which requires correct payment at the till at the end of the exercise.
Moreover, no difference was found between the groups in terms of the “Correct
Quantities” variable. These results of our study differ from the results of the first VSM
study (Zygouris et al., 2015). In our study, similarly to the first VSM study, Euro currency
was used to pay the necessary amount of money at the till. Since local Turkish currency
was not used, the use of the Euro may have created a greater challenge for patients in
our study.

Correlations between the VSM, demographical features, and neuropsychological

tests
All VSM variables were positively associated with age and negatively with education, except
for the Correct Money variable. It has been shown in several studies that education increases
cognitive performance (Wilson et al., 2009), and the speed of information processing
decreases as age increases (Birren & Fisher, 1995; Cerella, 1985; Salthouse, 2000).
The task’s “Bought Unlisted” variable showed a high level of correlation to visual
memory scores measured by WMS-R visual reproduction test (Table 2). Moreover, the
“Duration” variable was moderately correlated with the MMSE; “Bought Unlisted” corre-
lated with visual and verbal memory and “Correct Money” correlated with the MMSE,
and with verbal/visual memory. The “Duration” variable of the VSM displayed a low level
of correlation with all neuropsychological tests except the MMSE. Observing either
moderate or low correlation levels may be related to several factors, such as the small
sample size and the lack of familiarity with tablet devices in our sample. An assessment
of the reliability and validity of the VSM in a larger sample may provide clearer findings.
The current study cannot indicate whether the VSM test is unaffected by anxiety and
depression, since all participants with clinically significant depression or anxiety symp-
toms were excluded from the present study. However, in this healthy sample, VSM was
unaffected by the presence of symptoms of depression and anxiety. This finding is in
line with the findings of previous VSM studies (Zygouris et al., 2015, 2017).

Discrimination analysis of VSM variables over aMCI and HCs

In the present study, a combination of all VSM variables was able to distinguish between
aMCI and HCs with a CCR of 80%. In Zygouris et al. (2015) study, the CCR of all VSM
variables was not reported. In our study, a combination of “Bought Unlisted”, “Correct
Money” and “Duration” variables displayed the highest CCR, sensitivity, and specificity for
discriminating between aMCI and HCs. The combination of these three variables displayed
 AGING, NEUROPSYCHOLOGY, AND COGNITION 11

a CCR of 81% with 79% sensitivity and 86% specificity in our study. In a Greek population,
Zygouris et al. (2015) reported similar rates of 87.3%, 82.3%, and 95.2% respectively using
the same variable combination as it also yielded the best classification in their sample. This
small difference may be related to several factors, such as having higher numbers of
healthy controls and cultural differences or unfamiliarity with tablet devices in our study.

Performance of the MMSE in discriminating between aMCI patients and HCs

In our sample the VSM elicited higher rates of sensitivity and specificity than the MMSE;
it should be noted though that the MMSE is a poor screening tool for MCI (Hoops et al.,
2009; Luis, Keegan, & Mullan, 2009; Nasreddine et al., 2005; Smith, Gildeh, & Holmes,
2007; Trenkle, Shankle, & Azen, 2007) thus comparing it with the VSM cannot lead to
meaningful conclusions. We found that the MMSE discriminated between aMCI and HCs
with a CCR of 77%, 75% sensitivity, and 85% specificity. In a Greek population, these
rates were 66.7%, 66.7%, and 66.7% respectively (Zygouris et al., 2015). The difference in
the diagnostic accuracy of MMSE in these two studies can be attributed to the different
versions of MMSE that were used (Greek and Turkish).
The VSM performed better than the MMSE in our sample yielding a CCR of 81% with
79% sensitivity and 86% specificity in differentiating between aMCI and HCs. It should be
noted that the CCR of the VSM in our sample and in previous studies is similar to the
CCR of the MoCA (Zygouris et al., 2015, 2017), which is considered a gold standard test
for MCI screening, however, no direct comparison to the MoCA was conducted in this
study or in previous VSM studies.

Discrimination analysis of VSM variables over aMCI-MD and aMCI-SD

Combining the MMSE and the VSM variables on differentiating between aMCI-MD and
aMCI-SD allowed for higher CCR, sensitivity, and specificity compared to the previous
study on a Greek sample (Zygouris et al., 2015). This can be attributed to the higher CCR
displayed by the Turkish version of the MMSE compared to the Greek version.

Advantages of virtual reality-based neuropsychological testing

Virtual reality-based neuropsychological testing has advantages including savings of costs
and time for administering and scoring tests, suitability for unsupervised use at home or in
clinical settings, and attractiveness due to visual content and interactive format (Bauer
et al., 2012; Fredrickson et al., 2010; Snyder et al., 2011; Stricker et al., 2018; Wild, Howieson,
Webbe, Seelye, & Kaye, 2008; Zygouris et al., 2015; Zygouris & Tsolaki, 2014). The VSM takes
less time to administer than traditional neuropsychological test batteries, features auto-
matic scoring and allows for easier data extraction and interpretation. In addition, accord-
ing to our observations, participants find the task more enjoyable and are more willing to
be tested with this test than a traditional pencil and paper test. More importantly, as shown
in previous studies (Zygouris et al., 2017), this test can be self-administered thus allowing
older adults to test their cognitive functionality on their own from the comfort of their
home and only visit a specialist if initial signs of possible decline are detected by this test.
12 H. ERASLAN BOZ ET AL.

This study will provide preliminary findings for the validity and reliability of the VSM in the
Turkish population with larger samples.

Study limitations
Our study has certain limitations, which affect the generalizability of our results. Firstly, in the
present study a large number of participants had limited familiarity with new technologies
and tablet devices in particular. Secondly, the use of the Euro currency in the VSM payment
screen may have added extra complexity and cognitive burden not inherent in the task itself.
Another factor that adds complexity and limits the ecological validity of this task (in compar-
ison to real-life shopping in a grocery store) could be the lack of familiarity with tablet devices
in our sample. Therefore, future studies that account for these issues are needed in order to
assess whether VSM performance can predict actual functioning in daily living and especially
in shopping-related tasks. Thirdly, although it is known that MCI patients have subtle deficits
in complex daily functioning, preservation of the functionality of all MCI patients, according to
the Lawton-Brody IADL, which is a self-report questionnaire, indicates that personal awareness
of patients with MCI is limited (Roberts, Clare, & Woods, 2009). However, we think this is not
the case in the current study, as we gathered the information about daily functioning from the
patients’ relatives.
Future studies could examine whether the VSM is a good screening test for other patient
groups with cognitive impairment, such as cerebral small vessel disease (CSVD) patients.

Conclusions
The current study is the first to evaluate the VSM test in a Turkish sample. This study
demonstrated that the VSM may be a potentially useful cognitive assessment tool as an
adjunct to traditional pencil-and-paper neuropsychological tests. However, future VSM
studies measuring its reliability and validity in larger samples are needed.

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
This work was supported by The Scientific and Technological Research Council of Turkey-TUBITAK
(grant number 216S242).

ORCID
Hatice Eraslan Boz http://orcid.org/0000-0003-0128-4124

References
Albert, M. S., DeKosky, S. T., Dickson, D., Dubois, B., Feldman, H. H., Fox, N. C., . . . Phelps, C. H.
(2011). The diagnosis of mild cognitive impairment due to Alzheimer’s disease:
 AGING, NEUROPSYCHOLOGY, AND COGNITION 13

Recommendations from the National Institute on Aging-Alzheimer’s Association work-

groups on diagnostic guidelines for Alzheimer’s disease. Alzheimer’s & Dementia, 7(3),
270–279.
Arevalo-Rodriguez, I., Smailagic, N., Roqué I Figuls, M., Ciapponi, A., Sanchez-Perez, E.,
Giannakou, A., . . . Cullum, S. (2015). Mini-Mental State Examination (MMSE) fort he detection
of Alzheimer ‘s Disease and other dementias in people with mild cognitive impairment (MCI).
Cochrane Database of Systematic Reviews, 5(3).CD010783.
Atkins, A. S., Khan, A., Ulshen, D., Vaughan, A., Balentin, D., Dickerson, H., . . . Keefe, R. S. E. (2018).
Assessment ıf instrumental activities of daily living in older adults with subjective cognitive
decline using the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). The Journal of
Prevention of Alzheimer’s Disease, 5(4), 216–224.
Bauer, R. M., Iverson, G. L., Cernich, A. N., Binder, L. M., Ruff, R. M., & Naugle, R. I. (2012).
Computerized neuropsychological assessment devices: Joint position paper of the American
Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. The
Clinical Neuropsychologist, 26(2), 177–196.
Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety:
Psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893–897.
Belleville, S., Chertkow, H., & Gauthier, S. (2007). Working memory and control of attention in
persons with Alzheimer’s disease and mild cognitive impairment. Neuropsychology, 21(4),
458–469.
Bennett, D. A., Schneider, J. A., Bienias, J. L., Evans, D. A., & Wilson, R. S. (2005). Mild cognitive
impairment is related to Alzheimer disease pathology and cerebral infarctions. Neurology, 64(5),
834–841.
Birren, J. E., & Fisher, L. M. (1995). Aging and speed of behaviour: Possible consequences for
psychological functioning. Annual Review of Psychology, 46, 329–353.
Borella, E., Carretti, B., Mitolo, M., Zavagnin, M., Caffarra, P., Mammarella, N., . . . Piras, F. (2017).
Characterizing cognitive inhibitory deficits in mild cognitive impairment. Psychiatry Research,
251, 342–348.
Cerella, J. (1985). Information processing rates in the elderly. Psychological Bulletin, 98, 67–83.
Ellis, K. A., Lim, Y. Y., Ames, D., Darby, D., Harrington, K., Martins, R. N., & AIBL Research Group.
(2012). Aβ amyloid, cognition and APOE genotype in the preclinical stages of Alzheimer’s
disease. Alzheimer’s & Dementia, 8(4), 556–557.
Ertan, T., Eker, E., & Sar, V. (1997). Validity and reliability of geriatric depression scale in Turkish
elderly people. Archives of Neuropsychiatry, 34(2), 62–71.
Folstein, M. F., Folstein, S., & Mc Hugh, P. R. (1975). “Mini-Mental State” A practical method for
grading the cognitive state of patients for the clinician. Psychiatry Research, 12, 189–198.
Fredrickson, J., Maruff, P., Woodward, M., Moore, L., Fredrickson, A., Sach, J., & Darby, D. (2010).
Evaluation of the usability of a brief computerized cognitive screening test in older people for
epidemiological studies. Neuroepidemiology, 34(2), 65–75.
Hammers, D., Spurgeon, E., Ryan, K., Persad, C., Barbas, N., Heidebrink, J., . . . Giordani, B. (2012).
Validity of a brief computerized cognitive screening test in dementia. Journal of Geriatric
Psychiatry and Neurology, 25(2), 89–99.
Hoops, S., Nazem, S., Siderowf, A. D., Duda, J. E., Xie, S. X., Stern, M. B., & Weintraub, D. (2009).
Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease.
Neurology, 73, 1738–1745.
Jicha, G. A., Parisi, J. E., Dickson, D. W., Johnson, K., Cha, R., Ivnik, R. J., & Petersen, R. C. (2006).
Neuropathologic outcome of mild cognitive impairment following progression to clinical
dementia. Archives of Neurology, 63(5), 674–681.
Kaplan, E., Goodglass, H., & Weintraub, S. (2001). Boston naming test (2nd ed.). Austin, TX: Pro-Ed.
Keskinoglu, P., Ucku, R., Yener, G., Yaka, E., Kurt, P., & Tunca, Z. (2009). Reliability and validity of
revised Turkish version of Mini-Mental State Examination (rMMSE-T) in community-dwelling
educated and uneducated elderly. International Journal of Geriatric Psychiatry, 24(11),
1242–1250.
14 H. ERASLAN BOZ ET AL.

Knopman, D. S., & Petersen, R. C. (2014). Mild cognitive impairment and mild dementia: A clinical
perspective. Mayo Clinic Proceedings. Mayo Clinic, 89(10), 1452–1459.
Larrieu, S., Letenneur, L., Orgogozo, J. M., Fabrigoule, C., Amieva, H., Le Carret, N., . . . Dartigues, J. F.
(2002). Incidence and outcome of mild cognitive impairment in a population-based prospective
cohort. Neurology, 59(10), 1594–1599.
Lawton, M. P., & Brody, E. M. (1969). Assessment of older people: Self-maintaining and instrumental
activities of daily living. Gerontologist, 9(3), 179–186.
Lim, Y. Y., Ellis, K. A., Harrington, K., Ames, D., Martins, R. N., Masters, C. L., & The AIBL Research
Group. (2012). Use of the CogState Brief Battery in the assessment of Alzheimer’s disease-related
cognitive impairment in the Australian Imaging, Biomarker and Lifestyle (AIBL) study. Journal of
Clinical and Experimental Neuropsychology, 34, 345–358.
Luis, C. A., Keegan, A. P., & Mullan, M. (2009). Cross-validation of the montreal cognitive assessment
in community-dwelling older adults residing in the Southeastern US. International Journal of
Geriatric Psychiatry, 24, 197–201.
Luria, A. R. (1980). Higher cortical functions in man (2nd ed.). New York, NY: Consultants Bureau.
Markesbery, W. R., Schmitt, F. A., Kryscio, R. J., Davis, D. G., Smith, C. D., & Wekstein, D. R.
(2006). Neuropathologic substrate of mild cognitive impairment. Archives of Neurology, 63(1),
38–46.
Martin, A., Wiggs, C. L., Lalonde, F., & Mack, C. (1994). Word retrieval to letter and semantic cues:
A double dissociation in normal subjects using interference tasks. Neuropsychologia, 32(12),
1487–1494.
Maruff, P., Lim, Y. Y., Darby, D., Ellis, K. A., Pietrzak, R. H., Snyder, P. J., & AIBL Research Group (2013).
Clinical utility of the cogstate brief battery in identifying cognitive impairment in mild cognitive
impairment and Alzheimer’s disease. BMC Psychology, 23(1), 30.
Mohammadi, A., Kargar, M., & Hesami, H. (2018). Using virtual reality to distinguish subjects with
multiple-but not single domain amnestic mild cognitive impairment from normal elderly
subjects. Psychogeriatrics, 132–142. doi:10.1111/psyg.12301
Morris, J. C. (2006). Mild cognitive impairment is early-stage Alzheimer disease: Time to revise
diagnostic criteria. Archives of Neurology, 63(1), 15–16.
Nasreddine, Z. S., Phillips, N. A., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., . . .
Chertkow, H. (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for
mild cognitive impairment. Journal of the American Geriatrics Society, 53, 695–699.
Oktem, O. (1992). Verbal Memory Processes Test (WMPT)-A preliminary study. Archives of
Neuropsychiatry, 29(4), 196–206.
Oktem, O. (2003). General cognitive profile in patients with middle and late stage in Alzheimer
Disease. In Brain and Neuropsychology (pp. 101–111). Ankara: Çizgi Medicine Publishing.
Petersen, R. C. (2004). Mild cognitive impairment as a diagnostic entity. Journal of Internal Medicine,
256(3), 183–194.
Petersen, R. C., Roberts, R. O., Knopman, D. S., Boeve, B. F., Geda, Y. E., Ivnik, R. J., . . . Jack, J. R.
(2009). Mild cognitive impairment: Ten years later. Archives of Neurology, 66(12), 1447–1455.
Petersen, R. C., Stevens, J. C., Ganguli, M., Tangalos, E. G., Cummings, J. L., & DeKosky, S. T. (2001).
Practice parameter: Early detection of dementia: Mild cognitive impairment (an evidence-based
review). Report of the Quality Standards Subcommittee of the American Academy of Neurology.
Neurology, 56(9), 1133–1142.
Ritchie, K., Artero, S., & Touchon, J. (2001). Classification criteria for mild cognitive impairment: A
population-based validation study. Neurology, 56(1), 37–42.
Roberts, J. L., Clare, L., & Woods, R. T. (2009). Subjective memory complaints and awareness of
memory functioning in mild cognitive impairment: A systematic review. Dementia and Geriatric
Cognitive Disorders, 28, 95–109.
Rouleau, I., Salmon, D. P., Butters, N., Kennedy, C., & McGuire, K. (1992). Quantitative and qualitative
analyses of clock face drawings in Alzheimer’s and Huntington’s diseases. Brain and Cognition,
18(1), 70–87.
Salthouse, T. A. (2000). Aging and measures of processing speed. Biological Psychology, 54, 35–54.
 AGING, NEUROPSYCHOLOGY, AND COGNITION 15

Seo, K., Kim, J., Oh, D. H., Ryu, H., & Choi, H. (2017). Virtual daily living test to screen for mild
cognitive impairment using kinematic movement analysis. PloS One, 12, e0181883.
Smith, T., Gildeh, N., & Holmes, C. (2007). The montreal cognitive assessment: Validity and utility in
a memory clinic setting. Canadian Journal of Psychiatry. Revue Canadienne De Psychiatrie, 52,
329–332.
Snyder, P. J., Jackson, C. E., Petersen, R. C., Khachaturian, A. S., Kaye, J., Albert, M. S., & Weintraub, S.
(2011). Assessment of cognition in mild cognitive impairment: A comparative study. Alzheimer’s
& Dementia, 7(3), 338–355.
Stricker, N. H., Lundt, E. S., Edwards, K. K., Machulda, M. M., Kremers, W. K., Roberts, R. O., . . .
Mielke, M. M. (2018). Comparison of PC and iPad administrations of the cogstate brief battery in
the Mayo Clinic Study of Aging: Assessing cross-modality equivalence of computerized neu-
ropsychological tests. The Clinical Neuropsychologist, 10, 1–25.
Stroop, J. R. (1935). Studies of interference in serial verbal reaction. Journal of Experimental
Psychology, 18, 643–662.
Tangalos, E. G., & Petersen, R. C. (2018). Mild cognitive impairment in geriatrics. Journal of Clin
Geriatr Med, 34, 563–589.
Trenkle, D. L., Shankle, W. R., & Azen, S. P. (2007). Detecting cognitive impairment in primary care:
Performance assessment of three screening instruments. Journal of Alzheimer’s Disease : JAD, 11,
323–335.
Troyer, A. K., Moscovitch, M., Winocur, G., Alexander, M. P., & Stuss, D. (1998). Clustering and
switching on verbal fluency: The effects of focal frontal and temporal lobe lesions.
Neuropsychologia, 36(6), 499–504.
Tumac, A. (1997). The Effect Of Age And Education On Performance In Some Tests Sensitive To
Frontal Damages In Normal Subjects, Istanbul University, Social Sciences Institute, Unpublished
Psychology Master Thesis.
Ulusoy, M., Sahin, N., & Erkmen, H. (1998). The Turkish version of beck anxiety inventory:
Psychometric properties. Journal of Turkish Cognitive Psychotherapy, 12, 163.
Wechsler, D. (1981). Wechsler adult intelligence scale-revised. New York, NY: The Psychological
Corporation.
Wechsler, D. (1987). Wechsler memory scale-revised. San Antonio, TX: The Psychological
Corporation.
Wild, K., Howieson, D., Webbe, F., Seelye, A., & Kaye, J. (2008). Status of computerized cognitive
testing in aging: A systematic review. Alzheimer’s & Dementia, 4(6), 428–437.
Wilson, R. S., Hebert, L. E., Scherr, P. A., Barnes, L. L., de Leon, C. F., & Evans, D. A. (2009).
Educational attainment and cognitive decline in old age. Neurology, 72(5), 460–465.
Yesavage, J. A., Brink, T. L., Rose, T. L., Lum, O., Huang, V., Adey, M., & Leirer, V. O. (1982).
Development and validation of a geriatric depression screening scale: A preliminary report.
J Psychiatry Rese, 17(1), 37–49.
Zygouris, S., Giakoumis, D., Votis, K., Doumpoulakis, S., Ntovas, K., Segkouli, S., . . . Tsolaki, M. (2015).
Can a virtual reality cognitive training application fulfill a dual role? Using the virtual super-
market cognitive training application as a screening tool for mild cognitive impairment. Journal
of Alzheimer’s Disease : JAD, 44(4), 1333–1347.
Zygouris, S., Ntovas, K., Giakoumis, D., Votis, K., Doumpoulakis, S., Segkouli, S., . . . Tsolaki, M. (2017).
A preliminary study on the feasibility of using a virtual reality cognitive training application for
remote detection of mild cognitive impairment. Journal of Alzheimer’s Disease : JAD, 56,
619–627.
Zygouris, S., & Tsolaki, M. (2014). Computerized cognitive testing for older adults: A review.
American Journal of Alzheimer’s Disease and Other Dementias, 30(1), 13–28.