ORIGINAL STUDY

Effects of a Mixed Reality-based Cognitive Training System

Compared to a Conventional Computer-assisted Cognitive
Training System on Mild Cognitive Impairment:
A Pilot Study
Eunhee Park, MD, PhD,*† Byoung-Ju Yun, PhD,‡ Yu-Sun Min, MD, MS,*†
Downloaded from http://journals.lww.com/cogbehavneurol by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC4/OAVpDDa8K2+Ya6H515kE= on 01/29/2022

Yang-Soo Lee, MD, PhD,*§ Sung-Jun Moon, BA,* Jae-Won Huh, MD, MS,*§ Hyunsil Cha, MS,∥
Yongmin Chang, PhD,∥¶# and Tae-Du Jung, MD, PhD*§

working memory compared with the individuals with MCI who

Background: Mixed reality (MR) technology, which combines
the best features of augmented reality and virtual reality, has
recently emerged as a promising tool in cognitive rehabilitation
therapy.
Objective: To investigate the effectiveness of an MR-based cogni-
tive training system for individuals with mild cognitive impairment
(MCI).
Methods: Twenty-one individuals aged 65 years and older who
had been diagnosed with MCI were recruited for this study and
were divided into two groups. Participants in the MR group
(n = 10, aged 70.5 ± 4.2 years) received 30 minutes of training 3
times a week for 6 weeks using a newly developed MR-based
cognitive training system. Participants in the control group
(n = 11, aged 72.6 ± 5.3 years) received the same amount of
training using a conventional computer-assisted cognitive train-
ing system. Both groups took the Korean version of the Con-
sortium to Establish a Registry for Alzheimer’s Disease
(CERAD-K) both before and after intervention. To determine
the effect of the intervention on cognitive function, we compared
the difference in each group’s CERAD-K scores.
Results: There was a statistically significant interaction between
intervention (MR group vs control group) and time (before vs
after intervention) as assessed by the Constructional Recall Test.
The individuals with MCI who participated in the MR training
showed significantly improved performance in visuospatial
Received for publication August 17, 2018; accepted April 24, 2019.
From the *Department of Rehabilitation Medicine, Kyungpook National
University Chilgok Hospital; Departments of †Rehabilitation Medicine;
¶Radiology; #Molecular Medicine, School of Medicine; ‡School of
Electronics Engineering, College of IT Engineering; ∥Department of
Medical & Biological Engineering, Kyungpook National University;
and §Department of Rehabilitation Medicine, Kyungpook National
University Hospital, Daegu, Korea.
E.P. and B.-J.Y. contributed equally.
The authors declare no conflicts of interest.
Correspondence: Tae-Du Jung, MD, PhD, Department of Rehabilitation
Medicine, Kyungpook National University Chilgok Hospital,
807 Hoguk-ro, Buk-gu, Daegu, South Korea 41404 (email:
teeed0522@hanmail.net).
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.
participated in the conventional training.
Conclusion: An MR-based cognitive training system can be used
as a cognitive training tool to improve visuospatial working
memory in individuals with MCI.
Key Words: mixed reality, virtual reality, cognitive rehabilitation,
mild cognitive impairment
(Cogn Behav Neurol 2019;32:172–178)
3D = three-dimensional. CERAD-K = Korean version of the
Consortium to Establish a Registry for Alzheimer’s Disease.
MCI = mild cognitive impairment. MR = mixed reality. VR =
virtual reality.
M ild cognitive impairment (MCI) has been shown to
precede dementia in an estimated 5% to 10% of
dementia cases each year (Knopman et al, 2015; Mitchell
and Shiri-Feshki, 2009). Cognitive impairment increases
the risk of falls in individuals (Muir et al, 2012) and carries
high health costs (Callahan et al, 2015) that are propor-
tionate to the number of cognitive functions impaired and
the severity of the symptoms (Knopman et al, 2015).
Delaying the onset of dementia by improving individuals’
cognitive function and symptoms at the MCI stage could
substantially reduce the incidence of dementia, cut health
care costs, and improve the quality of life of affected individuals
(Jak, 2012).
Recently, research interest has focused on compu-
terized cognitive training, which involves structured and
standardized practice tasks. This type of training has been
used successfully with the elderly and individuals with
MCI (Clare et al, 2003; Kueider et al, 2012). A recent
meta-analysis on the effectiveness of various computerized
cognitive training tools found a moderate positive effect on
cognition in the elderly (Lampit et al, 2014) and individuals
with MCI (Hill et al, 2017). However, the drawback to most
computerized cognitive training programs is that individuals

172 | www.cogbehavneurol.com Cogn Behav Neurol  Volume 32, Number 3, September 2019

Copyright r 2019 Wolters Kluwer Health, Inc. All rights reserved.

Cogn Behav Neurol  Volume 32, Number 3, September 2019
are only able to improve on the specific cognitive task they
have trained on, without receiving any beneficial improvement
in other cognitive functions related to their day-to-day activities
(Jak, 2012).
Virtual reality (VR), first presented by Ivan Suther-
land in 1965, is intended to “make that (virtual) world
in the window look real, sound real, feel real, and respond
realistically to the viewer’s actions” (Mazuryk and Gervautz,
1992, p. 2). VR uses a combination of technologies to create
immersive, interactive, multisensory, viewer-centered, three-
dimensional (3D) computer-generated environments that
provide customized experiences that mimic the sensation of
being in a real environment (Burdea, 2003; Cruz-Neira,
1993). One of the advantages of VR is the opportunity for
experiential learning during an activity; that is, the system
controls the difficulty of the task and the number of stimuli
according to each individual user’s responses, thereby creating
a unique training path for each user (Burdea, 2003; Kang
et al, 2008; Lange et al, 2012).
VR-based cognitive training programs have been
shown to enhance cognitive function in patients with
traumatic brain injury (Man et al, 2013), stroke (Faria
et al, 2016; Kang et al, 2008), MCI (Man et al, 2012;
Optale et al, 2010), and dementia (Schreiber, 1999). VR
offers virtual worlds that can be controlled by computer
input devices, such as a mouse or joystick. However,
having to control these multiple devices may reduce par-
ticipants’ 3D virtual sensation. Furthermore, traditional
head-mounted displays cannot recreate real body move-
ments and instead may act as a barrier in a real environ-
ment (Simone et al, 2006).
Mixed reality (MR) technology blends the real world
with virtual worlds to produce visualizations where real
and digital objects co-exist in real (eg, augmented reality)
or virtual environments (eg, VR) (Wagner et al, 2009).
MR can merge and interact with both real and virtual
objects in real and virtual environments, enabling both to
be experienced in a single visual display (Milgram and
Kishino, 1994; Wagner et al, 2009). MR-based systems
provide augmented virtuality, which relays feedback
from the user’s body position in real life to sensory
stimuli that are, in turn, presented to them to create the
illusion of being immersed in a 3D virtual space with
which they can interact. Kinetic depth data have shown
that participants’ spatial perception is able to interact
with 3D virtual stimuli (Maurer et al, 2001). This pro-
prioceptive sensory stimulation exploits multimodal aspects
of goal-oriented movements and feedback on one’s actions
(Krasovsky et al, 2018), thereby enhancing visual and
auditory sensory feedback in an MR system as compared
to a VR system.
MR systems have been used as a tool for older
people and patients with stroke to practice walking and
other activities of daily living in virtual environments
(Colomer et al, 2016; Krasovsky et al, 2018). MR systems
can be designed to provide familiar virtual environments
and situations so that participants can transfer skills
learned in the clinical setting to the real world using
multimodal feedback (Krasovsky et al, 2018). Colomer
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.
Copyright r 2019 Wolters Kluwer Health, Inc. All rights reserved.
Mixed Reality-based Cognitive Training
et al (2016) reported the clinical effectiveness of MR in-
terventions for upper limb motor training in chronic
stroke patients. However, the clinical efficacy of using an
MR-based cognitive training system for individuals with
MCI has yet to be evaluated.
In this study, we used an MR-based cognitive
training system that uses an MR interface to interact with
a 3D virtual space and augment real body movement in-
puts in order to control a virtual environment. The aim of
the study was to determine the clinical effectiveness of the
MR-based cognitive training system to improve cognitive
function in individuals with MCI. We hypothesized that
there would be a significant increase in test results for
those participants who trained using the MR-based cog-
nitive training system compared with the conventional
computer-assisted cognitive training system.
METHODS
Participants
The study took place from January 2016 to January
2017. Community-dwelling older adults who had lived in
the Daegu and Kyungpook areas in South Korea were
recruited for the study using announcement posters in two
public health care centers in those areas. The inclusion
criteria for participation included aged 65 years or older;
score of less than 1 on the Korean version of the Clinical
Dementia Rating Scale (Choi et al, 2001); score greater
than 21 on the Korean version of the Mini-Mental State
Examination (Kim et al, 2001; Park and Kwon, 1990);
and diagnosis of single-domain or multiple-domain MCI,
with impairment in the range of 0.5 to 1 SD below that of
age- and education-matched z-score healthy individuals,
according to the Korean version of the Consortium to
Establish a Registry for Alzheimer’s Disease (CERAD-K;
Jak et al, 2009; Lee et al, 2004; Petersen, 2004). Exclusion
criteria included a history of stroke or other neuro-
degenerative or neuropsychiatric disorder and severe vis-
ual impairment.
We assessed 25 individuals for eligibility in our study
and excluded four because they did not meet the study
requirements. The remaining 21 individuals were enrolled
in our study and were randomly divided into two groups
according to the type of intervention: an MR-based cog-
nitive training system or a conventional computer-assisted
cognitive training system. Participants in the two groups
were matched in terms of sex, age, education level, and
CERAD-K score at baseline (Table 1). Written informed
consent was obtained from all of the participants, and
ethical approval was provided by the institutional review
board of the Kyungpook National University Chilgok
Hospital. The study is registered with the Clinical
Research Information Service as KCT 00003014.
Intervention
The participants received cognitive training using
either an MR-based cognitive training system or a con-
ventional computer-assisted cognitive training system,
www.cogbehavneurol.com | 173
Park et al Cogn Behav Neurol  Volume 32, Number 3, September 2019

1920×1080 resolution, 45-degree field of view), and a depth

TABLE 1. Demographic and Clinical Characteristics of
Participants With Mild Cognitive Impairment in a Mixed camera (Xbox One, Microsoft). A geomagnetic sensor motion
Reality-based (MR) Cognitive Training Group and a tracker in the head-mounted display provides stereoscopic
Conventional Computer-assisted Cognitive Training Group views of computer-aided design schematics. The depth camera
Group interacts with a hand tracking system to reflect a kinetic
capture of the real body. Chroma-key technology, which
MR Group Control Group enhances augmented virtuality in a 3D virtual environment,
(n = 10) (n = 11) P enables each participant to directly view an image of one’s
Demographic Data own hand. The participant’s spatial perception is based on
Female, n (%) 8 (80.0) 9 (81.8) 0.973 stereo depth cues as well as on kinetic depth cues that are
Age, M ± SD (years) 70.60 ± 4.29 (65–76) 73.36 ± 5.50 (67–81) 0.197
Education, 7.09 ± 3.36 (0–12) 7.09 ± 3.36 (0–12) 0.705 received with viewpoint variations and interactive data
M ± SD (years) visualization. Thus, the MR-based cognitive training system
Neurophysiological Assessment at Baseline* is able to provide augmented virtuality using information
CERAD (z-score)
Verbal Fluency –0.52 ± 0.57 –0.77 ± 0.63 0.359
from the participant’s real-life position in space to the sensory
Boston Naming 0.34 ± 1.13 0.41 ± 0.87 0.886 stimuli presented to the participant in order to create the
Word List Learning –0.30 ± 1.12 –0.51 ± 0.42 0.592 illusion of being immersed in a 3D virtual space, with which
Word List Recall –0.48 ± 1.13 –0.38 ± 0.45 0.803 his or her real body may interact.
Word List –0.51 ± 1.03 –0.73 ± 1.32 0.669
Recognition The software in the MR-based cognitive training
Constructional 0.01 ± 1.49 0.33 ± 1.06 0.569 system provides 15 training tasks that reflect daily activ-
Praxis ities that the study participants are likely to participate in
Constructional –0.42 ± 0.99 –0.21 ± 0.84 0.612
Recall in a home setting, such as caring for a grandchild. We
Trail Making Part A 0.48 ± 0.79 0.67 ± 1.15 0.658 developed the program scenarios and target training do-
Trail Making Part B –0.88 ± 3.04 0.08 ± 1.69 0.369 mains in conjunction with rehabilitation physicians, en-
MMSE 26.60 ± 1.35 26.73 ± 1.49 0.756
Clinical Dementia 0.70 ± 0.26 0.68 ± 0.25 0.918
gineers, and neuropsychological therapists. Together with
Rating Scale the engineers, we created a 3D virtual environment system
Beck Depression 10.20 ± 9.48 10.64 ± 0.93 0.973 using an in-home setting scenario consisting of four
Inventory rooms: a living room, a child’s room, a kitchen, and a
Modified Barthel 100 ± 0.00 99.81 ± 0.60 0.756
Index bathroom. The cognitive domains targeted were selective
attention, visual and verbal working memory, executive
*We used the Korean versions of all of the tests. function (including sequencing, planning, and problem
CERAD = Consortium to Establish a Registry for Alzheimer’s solving), and calculation. Participants were required to
Disease. MMSE = Mini-Mental State Examination. perform each task with some type of distraction.
In the living room environment, participants were
expected to perform three training tasks targeting short-
consisting of three 30-minute sessions a week for 6 weeks term memory, verbal working memory, and sequencing:
(total amount of 540 minutes). turn on a child’s favorite cartoon TV channel (short-term
memory) and make a telephone call and play a card game
MR-based Cognitive Training System (verbal working memory and sequencing). In the child’s
An MR-based cognitive training system named the room environment, participants were expected to perform
Mixed Reality System for Health was developed by an eight training tasks targeting various cognitive functions:
electronics engineering research team at Kyungpook National get a child to clean his or her room (selective attention and
University (Figure 1). The hardware consists of a four-sided categorization), pick books to read to a child(ren), (selective
booth measuring 1800 mm wide, 1200 mm deep, and 2160 attention and visual working memory), match picture
mm high. Inside the booth is a 360-degree blue screen, a blocks (localization), pack a child’s school bag (short-term
head-mounted display (Oculus Rift Development Kit 2, DK2, memory), check out shopping items (calculation), color a

FIGURE 1. Hardware (A) and software (B) of the Mixed Reality System for Health.

174 | www.cogbehavneurol.com Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.

Copyright r 2019 Wolters Kluwer Health, Inc. All rights reserved.

Cogn Behav Neurol  Volume 32, Number 3, September 2019
sketch (localization and visual working memory), and hang
a doll’s clothes and play a color-matching game (visual
working memory and sequencing). In the kitchen environment,
participants were expected to perform two training tasks
targeting procedural memory, sequencing, and planning: make
sandwiches and make a bowl of cereal. In the bathroom
environment, participants were expected to perform two
training tasks targeting problem solving and executive
function: brush a child’s teeth and train a child to use a toilet.
Each scenario was designed according to a gradation
structure: Tasks ranged from simple to more complex in
terms of the number and similarity of objects to be re-
membered as well as the duration of the distraction.
During training, the computer provided instructions to the
participants in the form of auditory cues. The difficulty of
the training tasks in each scenario was immediately ad-
justed based on each participant’s responses (correct or
incorrect)—that is, the participant’s hand positions and
movements in the real-life environment—while he or she
was performing tasks in the virtual environment. When a
participant correctly responded to a virtual task, the MR-
based cognitive training system provided positive visual,
auditory, and haptic (proprioceptive) feedback while also
increasing the difficultly of the following tasks. Con-
versely, when a participant incorrectly responded to a
virtual task, the training system provided more detailed
instruction, using both visual and auditory cueing, to help
the participant perform the task in the proper sequence.
Conventional Computer-assisted Training System
The conventional computer-assisted cognitive train-
ing was conducted using Comcog (Maxmedica), which is a
computer-assisted cognitive training system that has been
used for years in Asia (Kim et al, 2003). The system
provides 10 training activities: two visual processing tasks
that assess response time during visual stimulation; two
auditory processing tasks that assess response time during
auditory stimulation; two selective attention tasks that
track attention in distraction; three working memory tasks
that assess recognition and recall memory using visual,
auditory, and multisensory stimulation; and one emo-
tional attention task that assesses responses to pleasant or
unpleasant stimulation (Kim et al, 2003).
Neuropsychological Assessments
We assessed all of the participants for cognitive function,
mood, and independence in daily life both before the inter-
vention (baseline) and immediately after the intervention
(postintervention) using the CERAD-K (Lee et al, 2004) and
the Korean versions of the Mini-Mental State Examination,
the Clinical Dementia Rating Scale, the Beck Depression In-
ventory (Hahn et al, 1986), and the Modified Barthel Index
(Jung et al, 2007). These tests were conducted by an occupa-
tional therapist (J.-W.H.) who was blinded to the intervention
and who had completed the required cognitive certification
program. The CERAD-K consists of nine different cognitive
tests (Lee et al, 2004): the Verbal Fluency Test, which assesses
semantic memory, verbal production, and language function;
the Boston Naming Test, which assesses confrontational
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.
Copyright r 2019 Wolters Kluwer Health, Inc. All rights reserved.
Mixed Reality-based Cognitive Training
naming; the Word List Learning Test, which assesses verbal
working memory; the Word List Recall Test, which assesses
episodic memory and delayed recall in language tasks; the
Word List Recognition Test, which assesses the recognition of
specific words; the Constructional Praxis Test, which assesses
visuospatial and constructional abilities; the Constructional
Recall Test, which assesses visuospatial working memory and
recall in visuospatial tasks; Trail Making Part A Test, which
assesses attention; and Trail Making Part B Test, which as-
sesses executive function.
Statistical Analysis
All statistical analyses were performed using SPSS
23.0 statistical software. The scores from all functional
assessments were first analyzed for normal distribution
using the Shapiro-Wilk Test. The differences between each
neuropsychological assessment at baseline between the
MR group and the control group were measured using the
independent t test. A two-way repeated-measures AN-
OVA was performed to analyze the effects of interaction
between group (MR group vs control group) and time
(baseline vs postintervention) (P ≤ 0.05). The mean dif-
ferences in CERAD-K scores from baseline to post-
intervention within each group and between the two
groups were analyzed using Tukey post hoc tests consid-
ering multiple comparison (P ≤ 0.025).
RESULTS
None of the participants reported suffering from
mild headache, dizziness, or simulator sickness immedi-
ately after intervention with the MR-based cognitive
training system; thus, the intervention was well tolerated.
There was a statistically significant interaction between
group (MR group vs control group) and time (baseline vs
postintervention) on the Constructional Recall Test, which
measures visuospatial working memory and recall in visuo-
spatial tasks (F1,38 = 4.354, P = 0.044) (Table 2). There was
also a significant improvement in scores on this test in the
MR group compared with the control group (t = –2.612,
P = 0.017, effect size = 1.16/0.44 =2.63). However, there were
no significant interactions between group and time for any of
the other measures: Verbal Fluency Test (F1,38 = 2.390,
P = 0.130), Boston Naming Test (F1,38 = 0.024, P = 0.878),
Word List Learning Test (F1,38 =0.454, P = 0.505), Word
List Recall Test (F1,38 = 0.016, P =0.900), Word List
Recognition Test (F1,38 = 0.001, P = 0.982), Constructional
Praxis Test (F1,38 = 0.120, P = 0.731), Trail Making Test
Part A (F1,38 = 0.002, P = 0.996), and Trail Making Test
Part B (F1,38 = 0.899, P = 0.349).
DISCUSSION
The MR-based cognitive training system used in our
study was designed to imitate and simulate real-life daily
tasks in the home setting. It did this by combining an im-
mersive display of four realistic 3D environments with aug-
mented reality, thereby enabling the sensation of real body
movement in a virtual space. Although our study did not
show significant improvement in all cognitive domains for the
www.cogbehavneurol.com | 175
Park et al Cogn Behav Neurol  Volume 32, Number 3, September 2019

TABLE 2. Comparisons of the z-Score of the Korean Version of the Consortium to Establish a Registry for
Alzheimer’s Disease at Baseline and Postintervention Between the Mixed Reality Group and the Control Group
Mixed Reality Group Control Group
Baseline Postintervention Baseline Postintervention P
Verbal Fluency −0.522 ± 0.575 −0.647 ± 0.558 −0.772 ± 0.636 −0.312 ± 0.665 0.130
Korean-Boston Naming 0.343 ± 1.137 0.708 ± 1.298 0.413 ± 0.875 0.884 ± 0.929 0.878
Word List Learning −0.307 ± 1.121 0.412 ± 1.233 −0.514 ± 0.423 0.577 ± 0.603 0.505
Word List Recall −0.487 ± 1.137 −0.131 ± 1.265 −0.388 ± 0.459 −0.104 ± 0.598 0.900
Word List Recognition −0.511 ± 1.034 −0.017 ± 1.253 −0.738 ± 1.324 −0.260 ± 0.835 0.982
Constructional Praxis 0.013 ± 1.490 0.562 ± 0.710 0.338 ± 1.063 1.514 ± 1.885 0.731
Constructional Recall −0.422 ± 0.990 1.217 ± 0.912 −0.215 ± 0.842 0.263 ± 0.858 0.044*
Trail Making Test Part A 0.482 ± 0.798 0.890 ± 0.620 0.679 ± 1.157 1.064 ± 0.825 0.996
Trail Making Test Part B −0.888 ± 3.045 1.069 ± 0.952 0.089 ± 1.699 0.926 ± 1.355 0.349
*Significant at P ≤ 0.05.

participants trained using the MR-based system, it did show
significant improvement in visuospatial working memory.
Thus, our MR-based system can be used to improve visuo-
spatial working memory in individuals with MCI.
The ability to perform basic daily activities requires
that an individual’s cognitive (eg, problem solving, plan-
ning), motor (eg, balance, dexterity), and perceptual (eg,
sensory stimulation) functions are intact (Mlinac and
Feng, 2016). In our MR-based cognitive training system,
the training exercises simulating vital daily tasks were de-
signed to target multiple cognitive domains such as se-
lective attention, visual/verbal working memory, problem
solving, executive functions, sequencing, planning, and
calculation. This is an improvement on conventional
computer-assisted cognitive training systems, which mostly
train selective attention and simple sensory processing.
Compared with a VR system, our MR-based cognitive
training system involves multimodal sensory feedback, includ-
ing proprioception. VR games equipped with cognitive training
modules and using various input devices (eg, keyboard, mouse,
and joystick) have been shown to improve cognitive abilities in
individuals (Kim et al, 2011; Man et al, 2013; Optale et al,
2010; Pietrzak et al, 2014); however, the current MR-based
cognitive training system is more promising in that it uniquely
enables the user to actually see his or her actual body position
in the simulated environment, instead of receiving virtual data
from input devices (Brigham, 2017).
MR-based cognitive training systems can also enhance
visual or auditory sensory feedback. Using our MR-based
cognitive training system, participants’ spatial perception
based on kinetic depth data interacts with 3D virtual stimuli.
The MR interface (unlike the VR system) augments a sense
of being in and interacting with a virtual environment. In
addition, the advantages of the MR-based cognitive training
system are its portability, low cost, ease of accessibility (it
can be set up at home and should not require hospital visits),
ease of use, and good user tolerance—all of which suggest
that it can be integrated successfully into clinical practice.
Therefore, our MR-based cognitive training system could be
of clinical use in individuals with MCI during daily tasks.
It is our belief that the MR-based cognitive training
system could be used as a conduit to increase the efficiency
of cognitive training and enable the transfer of cognitive
skills learned in the clinical setting to real-world situations.
It has previously been shown that training working
memory, as carried out here using our MR-based cogni-
tive training system, can successfully transfer to other
cognitive domains (Klingberg, 2010). In addition, a virtual
learning environment that mimics the real environment
and provides enriched multisensory stimuli such as audi-
tory and visual stimuli could improve visual working
memory after intervention (Gazzaley and D’Esposito,
2007; Shams and Seitz, 2008). Wong et al (2014) pre-
viously found a significant correlation between object lo-
cation recall performance and variability in resting-state
functional magnetic resonance imaging blood oxygenation
level-dependent signals in brain regions that are important
for spatial exploration and memory after VR-based
training. However, the study did not explain the possible
neural mechanisms underlying this improvement. Further
studies using functional magnetic resonance imaging tools
are needed to explore any plastic changes in neural net-
works that may positively affect visual working memory,
and this should be studied in the context of an MR-based
cognitive training system.
Our MR-based cognitive training system displayed
stereoscopic 3D objects to participants in a realistic virtual
environment. However, because our study was limited to a
direct comparison of the effectiveness of VR- and MR-
based cognitive training, it cannot be ruled out that the
participants trained using our MR-based cognitive train-
ing system may have improved their visuospatial cognitive
function due to the effects of 3D stimuli through virtual
environments. For example, Park et al (2016), who eval-
uated the effects of 3D virtual object tasks using func-
tional magnetic resonance imaging on visuospatial
working memory, reported that stereoscopic and shaded
3D object tasks may activate salience neural networks.
Additional studies are needed to determine this and to
provide evidence of structural or functional alterations in
cognitive functions related to neural networks following
training on an MR-based cognitive training system.
The present study has several limitations. First, the
participant sample size was small, and we did not follow the

176 | www.cogbehavneurol.com Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.

Copyright r 2019 Wolters Kluwer Health, Inc. All rights reserved.

Cogn Behav Neurol  Volume 32, Number 3, September 2019
participants to see whether they were able to transfer what
they learned to the real world. Future research should in-
clude larger samples sizes, longer training sessions, and a
longer follow-up period to fully evaluate the clinical effec-
tiveness of an MR-based cognitive training system on in-
dividuals with MCI. Second, the efficacy of our MR-based
cognitive training system was evaluated as a tool for im-
proving cognitive function in individuals with MCI only,
not in individuals with other neurologic conditions such as
dementia, stroke, or traumatic brain injury. We propose
that future studies should apply an MR-based cognitive
training system to individuals with other neurologic con-
ditions so as to widen the clinical application of the
technology. Third, the results of the study are insufficient in
suggesting the mechanism through which MR may improve
cognitive function. Additional studies are needed to explore
the importance of mechanisms, such as neuroplasticity, and
changes that may be correlated to improved cognitive per-
formance after such interventions. Last, we did not assess
the usability of the MR-based cognitive training system for
any user interface issues, which is an important step in the
development process of any software. This shortcoming
should be addressed in a study that explores ways in which
to improve the MR environment.
In conclusion, we showed that an MR-based cogni-
tive training system can be used to improve visuospatial
working memory in individuals with MCI.
REFERENCES
Brigham TJ. 2017. Reality check: basics of augmented, virtual, and
mixed reality. Med Ref ServQ. 36:171–178.
Burdea G. 2003. Virtual rehabilitation—benefits and challenge. Methods
Inf Med. 42:519–523.
Callahan KE, Lovato JF, Miller ME, et al. 2015. Associations between
mild cognitive impairment and hospitalization and readmission. J Am
Geriatr Soc. 63:1880–1885.
Choi SH, Na DL, Lee BH, et al. 2001. Estimating the validity of the
Korean version of Expanded Clinical Dementia Rating (CDR) Scale.
J Korean Neurol Assoc. 19:585–591.
Clare L, Woods RT, Moniz Cook ED, et al. 2003. Cognitive
rehabilitation and cognitive training for early‐stage Alzheimer’s
disease and vascular dementia. Cochrane Database Syst Rev. 4.
doi:10.1002/14651858.CD003260
Colomer C, Llorens R, Noe E, et al. 2016. Effect of a mixed reality-based
intervention on arm, hand, and finger function on chronic stroke.
J Neuroeng Rehabil. 13:45. doi:org/10.1186/s12984-016-0153-6
Cruz-Neira C. 1993. Virtual reality overview. SIGGRAPH. 93:1.
Faria AL, Andrade A, Soares L, et al. 2016. Benefits of virtual reality
based cognitive rehabilitation through simulated activities of daily
living: a randomized controlled trial with stroke patients. J Neuroeng
Rehabil. 13:96. doi:10.1186/s12984-016-0204-z
Gazzaley A, D’Esposito M. 2007. Top-down modulation and normal
aging. Ann N Y Acad Sci. 1097:67–83.
Hahn H, Yum T, Shin Y, et al. 1986. A standardization study of Beck
Depression Inventory in Korea. J Korean Neuropsychiatr Asso. 25:
487–500.
Hill NT, Mowszowski L, Naismith SL, et al. 2017. Computerized
cognitive training in older adults with mild cognitive impairment or
dementia: a systematic review and meta-analysis. Am J Psychiatry.
174:329–340.
Jak AJ. 2012. The impact of physical and mental activity on cognitive
aging. Curr Top Behav Neurosci. 10:273–291.
Jak AJ, Bondi MW, Delano-Wood L, et al. 2009. Quantification of five
neuropsychological approaches to defining mild cognitive impair-
ment. Am J Geriatr Psychiatry. 17:368–375.
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.
Copyright r 2019 Wolters Kluwer Health, Inc. All rights reserved.
Mixed Reality-based Cognitive Training
Jung HY, Park BK, Shin HS, et al. 2007. Development of the Korean
version of Modified Barthel Index (K-MBI): multi-center study for
subjects with stroke. J Korean Acad Rehab Med. 31:283–297.
Kang YJ, Ku J, Han K, et al. 2008. Development and clinical trial of
virtual reality-based cognitive assessment in people with stroke:
preliminary study. Cyberpsychol Behav. 11:329–339.
Kim JM, Shin IS, Yoon JS, et al. 2001. Cut-off score on MMSE-K for
screening of dementia in community dwelling old people. J Korean
Geriatr Psychiatry. 5:163–168.
Kim YH, Ko MH, Seo JH, et al. 2003. Effect of computer-assisted
cognitive rehabilitation program for attention training in brain injury.
J Korean Acad Rehab Med. 27:830–839.
Kim YM, Chun MH, Yun GJ, et al. 2011. The effect of virtual reality
training on unilateral spatial neglect in stroke patients. Ann Rehabil
Med. 35:309–315.
Klingberg T. 2010. Training and plasticity of working memory. Trends
Cogn Sci. 14:317–324.
Knopman DS, Beiser A, Machulda MM, et al. 2015. Spectrum of
cognition short of dementia: Framingham Heart Study and Mayo
Clinic Study of Aging. Neurology. 85:1712–1721.
Krasovsky T, Weiss PL, Kizony R. 2018. Older adults pay an additional
cost when texting and walking: effects of age, environment, and use of
mixed reality on dual-task performance. Phys Ther. 98:549–559.
Kueider AM, Parisi JM, Gross AL, et al. 2012. Computerized cognitive
training with older adults: a systematic review. PLoS One. 7:e40588.
doi:10.1371/journal.pone.0040588
Lampit A, Hallock H, Valenzuela M. 2014. Computerized cognitive
training in cognitively healthy older adults: a systematic review and
meta-analysis of effect modifiers. PLoS Med. 11:e1001756. doi:10.1371/
journal.pmed.1001756
Lange B, Koenig S, Chang CY, et al. 2012. Designing informed game-
based rehabilitation tasks leveraging advances in virtual reality.
Disabil Rehabil. 34:1863–1870.
Lee DY, Lee KU, Lee JH, et al. 2004. A normative study of the CERAD
neuropsychological assessment battery in the Korean elderly. J Int
Neuropsychol Soc. 10:72–81.
Man DW, Chung JC, Lee GY. 2012. Evaluation of a virtual reality-based
memory training programme for Hong Kong Chinese older adults with
questionable dementia: a pilot study. Int J Geriatr Psychiatry. 27:513–520.
Man DW, Poon WS, Lam C. 2013. The effectiveness of artificial
intelligent 3-D virtual reality vocational problem-solving training in
enhancing employment opportunities for people with traumatic brain
injury. Brain Inj. 27:1016–1025.
Maurer CR, Sauer F, Hu B, et al. 2001. Augmented-reality visualization
of brain structures with stereo and kinetic depth cues: system
description and initial evaluation with head phantom. Proc SPIE.
4319:445–457.
Mazuryk T, Gervautz M. 1992. Virtual Reality. History, Applications,
Technology and Future. Institute of Computer Graphics. Technical
University of Vienna. Technical Report TR-186-2-96-06.
Milgram P, Kishino F. 1994. A taxonomy of mixed reality visual
displays. IEICE Trans Inf Syst. E77-D:1321–1329.
Mitchell AJ, Shiri-Feshki M. 2009. Rate of progression of mild cognitive
impairment to dementia—meta-analysis of 41 robust inception cohort
studies. Acta Psychiatr Scand. 119:252–265.
Mlinac ME, Feng MC. 2016. Assessment of activities of daily living, self-
care, and independence. Arch Clin Neuropsychol. 31:506–516.
Muir SW, Gopaul K, Montero Odasso MM. 2012. The role of cognitive
impairment in fall risk among older adults: a systematic review and
meta-analysis. Age Ageing. 41:299–308.
Optale G, Urgesi C, Busato V, et al. 2010. Controlling memory
impairment in elderly adults using virtual reality memory training:
a randomized controlled pilot study. Neurorehabil Neural Repair.
24:348–357.
Park JH, Kwon YC. 1990. Modification of the Mini‐Mental State
Examination for use in the elderly in a non‐western society. Part 1.
Development of Korean version of Mini‐Mental State Examination.
Int J Geriatr Psychiat. 5:381–387.
Park J-W, Kim Y-T, Yun B-J, et al. 2016. Stereoscopic 3D objects evoke
stronger saliency for nonverbal working memory: an fMRI study. Int
J Imag Syst Tech. 26:76–84.
www.cogbehavneurol.com | 177
Park et al
Petersen RC. 2004. Mild cognitive impairment as a diagnostic entity.
J Intern Med. 256:183–194.
Pietrzak E, Pullman S, McGuire A. 2014. Using virtual reality and
videogames for traumatic brain injury rehabilitation: a structured
literature review. Games Health J. 3:202–214.
Schreiber M. 1999. Potential of an interactive computer-based training in
the rehabilitation of dementia: an initial study. Neuropsychol Rehabil.
9:155–167.
Shams L, Seitz AR. 2008. Benefits of multisensory learning. Trends Cogn
Sci. 12:411–417.
178 | www.cogbehavneurol.com
Copyright r 2019 Wolters Kluwer Health, Inc. All rights reserved.
Cogn Behav Neurol  Volume 32, Number 3, September 2019
Simone LK, Schultheis MT, Rebimbas J, et al. 2006. Head-mounted
displays for clinical virtual reality applications: pitfalls in under-
standing user behavior while using technology. Cyberpsychol Behav.
9:591–602.
Wagner I, Broll W, Jacucci G, et al. 2009. On the role of presence in
mixed reality. Presence: Teleoperators and Virtual Environments.
18:249–276.
Wong CW, Olafsson V, Plank M, et al. 2014. Resting-state fMRI activity
predicts unsupervised learning and memory in an immersive virtual reality
environment. PLoS One. 9:e109622. doi:10.1371/journal.pone.0109622
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.