DOI 10.

1007/s11062-018-9741-3
Neurophysiology, Vol. 50, No. 3, June, 2018

REVIEWS

Application of Virtual Reality and Augmented Reality

in Psychiatry and Neuropsychology, in Particular
in the Case of Autistic Spectrum Disorder (ASD)
A. Fridhi,1 F. Benzarti,1 A. Frihida, and H. Amiri1
Received February 24, 2017

This paper is a combination of a review of the papers related to applications of virtual reality and
augmented reality in various aspects of psychiatry and neurophysiology, with special attention to using
the respective approaches in the case of autistic spectrum disorders (ASDs) and of a description of the
studies of the authors dealing with the corresponding problem. The major emphasis is put on the elements
that contribute to the amelioration of communicative and emotional skills in such children, mainly
related to modeled applications, computers, and an output interface (projection). This study, therefore,
focuses on the understanding of the elements simplifying the development of these new information and
communication technologies. In order to answer our queries, we used exploratory interviews and specific
reviews, such as virtual or augmented reality and virtual (preferably geo-localized) environments used
to provide children with ASD a database containing varied and multimodal information for various
purposes: (i) a display of the 3D environment and its constituent objects, (ii) understanding of the
task, (iii) an increase in the salience of certain objects, (iv) issuance of instructions associated with
interaction with the environment in order to increase the capacity of an individual to process and use
received information to allow a better performance of the activities of daily living (ADL), (v) simulation
of everyday activities, repeating them as necessary things, and graduating them in order to be able to
control a given situation, and (vi) introducing many innovative techniques that are not well-known in
the treatment of ASD.

Key words: ICT, virtual reality, augmented reality, autism, autistic spectrum disorder (ASD),
3D, virtual environment (VE), collaborative virtual environment (CVE), activities of daily living
(ADL).

VIRTUAL AND AUGMENTED REALITY
It is a “scientific and technical domain using
computing and behavioral interfaces to simulate in
a virtual environment the behavior of 3D entities
that interact in real time between them and with one
or more users” [1].
Perhaps, it is necessary to explain the terms
“virtual reality” and “augmented reality.” The
virtual environment is defined as a “computer-
generated three-dimensional simulation of a
1
LR-SITI- ENIT Laboratory. National School of Engineers of Tunis
(ENIT), Tunisia.
Correspondence should be addressed to A. Fridhi
(e-mail: adel.fridhi2013@gmail.com),
F. Benzarti (e-mail: benzartif@yahoo.fr),
A. Frihida (e-mail: ali.frihida@enit.rnu.tn),
or H. Amiri (e-mail: hamidlamiri@gmail.com).
real or imaginary environment” [2]. People who
use these environments can interact through “an
avatar”; so, users can get direct responses based
on their behavior in a virtual environment [3]. In a
collaborative virtual environment (CVE), multiple
users can communicate with each other through
their avatars. If we have a single user, we are talking
about a simple virtual environment [4].
Virtual or augmented reality has several
exploitable advantages for working with children
who suffer from autistic spectrum disorder, ASD.
Indeed, it facilitates the control of the environment,
as well as social interactions of the personality.
This can contribute to increasing the self-esteem
of patients who encounter difficulties in the real
environment in controlling social situations [5].
The CVE offers high flexibility in the sense that
social norms can be developed between users

222
0090-2977/18/5003-00222 © 2018 Springer Science+Business Media LLC
Virtual and Augmented Reality in Psychiatry and Neuropsychology
to facilitate communication. Researchers in this
field confirm that virtual objects, or “avatars,”
can facilitate social encounters, as well as the
processes of communication between people. This
communication can be simpler than face-to-face
in the reality [4]. Indeed, users can play major
roles in a virtual environment created by virtual
reality or in an environment created by augmented
reality to imitate certain specific social situations
[6]. Interaction in virtual reality does not require a
strong direct relational engagement of the autistic
individual with another person [5]; the same is true
for augmented reality. As a result, interactions are
realized slower, and ASD children have more time
to think about different ways of reacting to one
situation or another.
This system can be useful for increasing
emotions and recognition of emotional expressions
[7]. In addition, using educational potentials of
this technology, we can have the autistic person
interact with an avatar that is a street, a building,
or an animal (Fig. 1). It is also interesting to create
situations, such as to cross a boulevard or a fire
to put out, in order to anticipate events in the real
world [8]. The researchers add that the CVE allows
patients to practice real conversation in a virtual
world [2]. Indeed, if it is the researcher who chooses
the real activities and the modeled ones, he/she can
choose models of real buildings and real situations
around the real environment of the autistic. This
point takes up the consideration made previously
related to the importance of the customization of
the models.
F i g. 1. Virtual environment modeled in 3D.
223
To the question of how children with autism
interpret the virtual environment, probably the first
researchers who used virtual reality in the field of
autism were Strickland et al. [9], who employed a
complete immersion system to learn the steps to
cross a boulevard (Fig. 2). In this memory, a more
extensive study was carried out to observe that these
experiments could be transferred to a real situation.
Various analyses have also been carried out to check
whether the technology of virtual environments
with a computer, a large projection screen, and an
application of augmented reality could be applied.
In our case, Nichols [10], who was able to work
with children with ASD, attested that the virtual
environment with these systems is more accessible,
especially for this type of population. The research
subsequently disclosed was carried out with the
latter system [11]; it was demonstrated that new
technologies, such as virtual reality, are effective,
comfortable, facilitative, and offer a supportive
emotional context. The researchers continued their
assertions by adding that virtual and/or augmented
reality shares its advantages but increases the
potential of its effects. More precisely, it increases
its benefits in terms of the generalization skill,
thanks to the great ability to engage and control the
attention, control, and commitment of the emotional
participation that virtual environments can offer.
This device is practical and very useful for
working with the problem of autism because it can
provide a large amount of exploitable resources
to create a project appropriate to the needs of an
autistic child.
F i g. 2. Virtual environment modeled in 3D by adding a vehicle
on the street.
224
VIRTUAL REALITY AND AUGMENTED
REALITY (VR/AR) IN PSYCHIATRY AND
NEUROPSYCHOLOGY
The methods of virtual or augmented reality allow
ones, on the one hand, to expose children with ASD,
under the respective control, to complex, dynamic,
interactive, and three-dimensional simultaneous
stimuli. On the other hand, cognitive, behavioral,
and functional results of such application allow
one to introduce the child to actions that can be
close to those of the “real” reality and, in such a
mode, to improve evaluation of the symptoms
and the treatment. Virtual and augmented reality
technologies, therefore, offer the possibility of
developing environments for diagnostic, therapeutic,
and supportive purposes. Phobias, autism, and
rehabilitation are examples of areas where the use
of VR or AR has so far been most often considered
[12]. We will consider the use of VR or RA in two
disciplines, psychiatry and neuropsychology.
Designing and Creating an Application. In
psychiatry and neuropsychology, applications
modeled by VR/AR aim to understand the cognitive
and behavioral functioning of the child with ASD
to support him/her in his/her handicaps and to
take care of him/her in his/her problems. The
respective applications are centered on autism
and are developed for the purpose of performing a
specific activity or an exposure to specific stimuli.
The approach exposing applications realized by
virtual or augmented reality, whose objective is
to offer cognitive and sensorimotor activities to
patients, is generally in line with the expectations
of the sectors of psychiatry and neuropsychology.
The realization of an VE is simplified by the use, at
least partially, of the hierarchical model developed
recently. In neuropsychology, cognitive immersion
and operational immersion are often preferred. This
implementation phase needs a close interdisciplinary
partnership, understanding the clinical needs of
some and the specific possibilities of others.
The design of a VE or virtual application will
be developed to achieve goals. It is, therefore,
necessary to ask questions such as:
(i) The activities of the child with ASD in
the VE (Will it be a good and simple exposure
to stimuli? Or will it be active? Will it perform
actions?).
(ii) The sensory aspects to be applied (Is it a
possibility to be solicited in an VE?).
A. Fridhi et al.
(iii) The physician’s place (What will his role
be? Will it be in the same place as the patient?).
(iv) Management of the increase in the session
(Will it be entrusted to the doctor or the patient?
How will the patient’s development be estimated?).
(v) The need to save variables and performance
(What provisions will be required to measure
achievement of the goal?).
These reflections, once conducted, one can take a
look at the Virtual Behavioral Primitives (VBP) to
be developed:
(i) Navigation in the VE (What freedom of
navigation is to be proposed to the patient?)
(ii) Actions of the autistic to do the tasks.
(iii) Observation of VE (Which scale is used for
VE?).
Then the behavioral interfaces are selected or
developed with the objective of a job as natural as
possible, with a minimal moment of adaptation.
Their technical specifications (resolution, visual
field) are determined according to the children with
a targeted ASD.
Finally, the step of modeling the VE of the
various necessary elements can begin.
The design of a VE must be approved at different
stages by designers. Once performed, the virtual
reality-based application is tested on a patient
to verify its functionality and achievement of
objectives. Based on the results of these reviews,
changes may be made before moving to use the
application for the initial purpose.
VIRTUAL REALITY AND AUGMENTED
REALITY: OUR APPLICATIONS
We will describe the application based on
virtual or augmented reality-addressing therapy
and assessment. This application is conducted in
psychiatry and neuropsychology and focuses on the
evaluation of action planning for children with ASD.
The design and creation of the various VEs or the
virtual object (avatar), the choice of the interfaces,
is based on the principles described above.
Exploitation of VR and AR in Psychiatry.
Studies that use VR or AR within a
psychotherapeutic objective essentially affect ASD
for which several theories attempt to explain and
substantiate the treatment processes. These studies
based on VR or AR relied heavily on the theoretical
and methodological postulates of cognitive and
behavioral therapies [13].
Virtual and Augmented Reality in Psychiatry and Neuropsychology
Theories Regarding the Treatment of ASD. The
theory of the emotional treatment of Foa and Kozak
[14] suggests that fear memories, particularly in
autistic children, can be considered structures that
integrate information about stimuli, responses, and
meaning. Emotional integration, therefore, requires
activation of the structure of fear and habituation by
prolonged and repeated presentation of the stimuli.
On our part, the example of putting an avatar like
a dinosaur or an elephant with a very high sound
to see how the child with an ASD reacts leads us
to the activation of the structure of fear (Figs. 3
and 4). The cognitive theory of Beck’s information
processing [15] is based on the idea of cognitive
schemes. Finally, the aim of cognitive care methods
is to promote controlled (conscious, slow) cognitive
mechanisms in the process of automatic processing
of information by disturbed patterns.
F i g. 3. Activation of a dinosaur image for initiation of the
reaction of an ASD child.
F i g. 4. Activation of an elephant image for initiation of the
reaction of an ASD child.
225
Finally, the hypothesis of Bandura’s perception
of personal effectiveness [16] allows one to use
the feeling of powerlessness learned against
phobogeneous situations a central function in
psychopathology. The increase in the ability to
deal with situations will take place to the extent
that the subject considers himself/herself capable
of behaving and believes that this behavior will
succeed. This hypothesis is capable of being
considered as a cognitive interpretation of the
principles of learning that we will discover below
in the context of cognitive and behavioral care
methods.
OPERATION OF VR/AR IN
NEUROPSYCHOLOGY
Neuropsychology proceeds by dissociating
complicated behaviors into different cognitive areas,
which are:
(i) attention,
(ii) memory,
(iii) other cognitive activities, such as inductive
and deductive reasoning, and
(iv) the ability to abstract.
In the early 1980s, neuropsychology was closely
associated with the presentation of ASD compared
to theoretical concepts. Then, in the 1990s, the goal
was also to appreciate the practical repercussions
of these disorders in everyday life, a point truly
essential for the clinician.
The use of virtual reality in neuropsychological
applications is based on three fundamental
objectives:
(i) scientific studies of cognitive processes
(memory, attention, planning, and abilities),
(ii) neuropsychological assessment, and
(iii) cognitive rehabilitation.
Cognitive assessment and cognitive rehabi­
litation affect patients with ASD (learning and
developmental problems, attention deficit,
and mental retardation). Brain damage creates
degradation in the cognitive domain and in the
emotional and social domains [17]. After studying
the neuro­p sychological assessment, we will present
applications of VR in neuropsychology as a part of
the activities of daily living and cognitive rehabi­
litation. It is to be said that examinations in a
category like memory have the possibility also of
being in others like the occupations of everyday
living.
226 A. Fridhi et al.

EVALUATION IN NEUROPSYCHOLOGY is measured [19]. The recovered balances have more

Evaluation based on VR/AR. Virtual reality
(VR) techniques have the ability to address all
of the requirements formulated [18]. The above
realities (VR/AR) provide means to simulate natural
environments, in which we can present adapted
stimuli in a more ecological way, in particular
inserted with a demonstrative and familiar situation
(in a classroom or a store). Virtual reality (VR)
or augmented reality (AR) offers the possibility
to manage the control of distracting components,
complexity of the stimuli, and alteration of these
variables dynamically, in response to the actions
of autistic children. Other technical specifications
of responses (accuracy, rhythm) can be collected
to allow one to use more accurate analysis. Virtual
or augmented reality (VR / AR) allows us to
explore different cognitive sectors, e.g., memory
(Table 1).
This methodology can increase the reliability of
traditional assessment by reducing the variability
due to physician differences, the virtual testing
environment, and the effectiveness of stimuli.
Finally, it can improve its validity by allowing more
detailed and well-specialized surveys of behavior
and by advancing the ecological character of what
clinical relevance and have direct consequences on
the extension of cognitive rehabilitation systems.
Attentional Processes. Attentional difficulties
are usually encountered in subjects with ASD.
Attention is a prerequisite of all cognitive
functionalities; evaluation and rehabilitation objects
are mandatory in order to approach attentional skills
for many reasons in children with ASD:
(i) to anticipate possible concerns during
educational activities,
(ii) for decision-making on the placement
in special education, for determination of the use
and performance of some treatments, and for the
performance measurement of the treatment and
outcomes, and
(iii) to address the use of rehabilitation on
more sophisticated cognitive mechanisms, such
as memory, spatial visual capabilities, executive
functionality, and problem solving.
The level of experimental control provided
by the techniques of VR or AR correctly enables
improvement of the action of evaluation and reha­
bilitation of attention closer to those encountered in
the real environment. Therefore, the validity of the
measurement and treatment means about attention
can be increased.

T a b l e 1 Assessment of Our Applications in Neuropsychology

Intervention Goal
Evaluation of visual hemoglobin with
Evaluation
eye movement recording (follow-up)
Diagnosis of visual heminegligence
rehabilitation perspectives
Rehabilitation
Restoring the ability to cross streets in patients
(computer screen + projection table)
Spatial information transfer from VE to real
environment (RE) in disabled children (n = 10)
Object location, learning and transfer
possibilities to the real world (supermarket)
Space skills
(projection screen + computer)
rehabilitation
Study of the transfer of learning from the VE
to the RE
Description of application to children with ASD
(projection screen + computer)
Description of the creation of a virtual city
for the learning of ADL.
Evaluation activities
of daily living (ADL) Evaluation of the training of children to cross
streets in a virtual city
Rehabilitation activities Training in a virtual supermarket for students
of daily living (ADL) with disabilities
Conclusion
The integration of monitoring is well suited
to the evaluation objectives
Interest in measurements of angles of rotation
of the head. The VR or AR can better stimulate
the strategy of eye patching
VR or AR increases caution in everyday
situations
Experimentation in VE enhances knowledge
of RE
Repeating the task in the virtual supermarket
improves performance in the actual supermarket
Repeated exploration of an VE improves spatial
memory in the corresponding RE
Possible learning in VE discussion on constraints
and limitations
The VR or AR allows once to learn safe
behaviors, but the transfer in the real world
does not happen for all
Improved performance in the RE after repetitive
training in the VE
Virtual and Augmented Reality in Psychiatry and Neuropsychology
ASSESSMENT OF OUR APPLICATIONS
IN NEUROPSYCHOLOGY
The techniques of VR or AR are constantly
changing the approach to the assessment and
rehabilitation of ASD-related processes. The next
phase of this progress will allow one to provide
access via the Internet to environments based on
virtual or augmented reality. However, before this
vision is realized, several questions will have to be
addressed and resolved [20]. Applications created
by VR or AR are used to train children with learning
difficulties, such as children with ASD, on certain
safety gestures.
_______
The clinical logic that supports the use of VR or
AR in psychotherapy is now well established, even
though all the research conducted from time to time
are only feasibility studies or preparatory trials.
Controlled studies providing convincing results on
the clinical performance of the method are few in
number due to the lack of standardization of VR
systems (hardware and software) and the protocols
of the different operating groups. This feeling is at
the origin of research projects of laboratories on
signal, image, and information technology (LRSITI
ENIT). Virtual and augmented-reality technologies
are, therefore, expected to play an increasingly
important role in the creation of new diagnostic,
therapeutic, and support tools for children with ASD
who become a concern of public health.
This paper is a review, and thus the cor­
respondence of the study to the existing ethical
standards does not need special confirmation. At the
same time, some studies of the authors of the review
have been mentioned in the latter; in all these studies
involving human subjects, the existing international
ethical norms have been strictly observed.
REFERENCES
1. M. Billinghurst, A. Clark, and G. Lee, “A survey of
augmented reality,” Found. Trends in Human–Comput.
Interact., 8, Nos. 2-3, 73-272 (2015).
2. S. Cobb, L. Beardon, R. Eastgate, et al., “Applied virtual
environments to support learning of social interaction
skills in users with Asperger ’s Syndrome,” Digital
Creativ., 13, No. 1, 11-22 (2002).
227
3. T. R. Goldsmith and L. A. LeBlanc, “Use of technology
in interventions for children with autism,” JEIBI, 1, No.
2, 166-178 (2004).
4. S. Parsons, P. Mitchell, and A. Leonard, “Do adolescents
with autistic spectrum disorders adhere to social
conventions in virtual environments?” Autism, 9, No. 1,
95-117 (2005).
5. E. Klinger, R. M. Marié, and I. Viaud-Delmon,
“Applications de la RV aux troubles cognitifs et
comportementaux,” Chap. 5 du volume “Applications de
la réalité virtuelle,” in: Le Traité de la Réalité Virtuelle,
Vol. 4, P. Fuchs, G. Moreau, et al., Les Presses de
l’Ecole des Mines de Paris, Paris (2006), pp. 121-158.
6. S. Parsons, P. Mitchell, and A. Leonard, “The use and
understanding of virtual environments by adolescents
with autistic spectrum disorders,” J. Autism Dev.
Disord., 34, No. 4, 449-466 (2004).
7. S. Parsons and P. Mitchell, “The potential of virtual
reality in social skills training for people with autistic
spectrum disorders,” J. Intell.. Disabil. Res., 46, No. 5,
430-443 (2002).
8. D. Moore, Y. Cheng, P. McGrath, and N. Powell,
“Collaborative virtual environment technology
for people with autism,” Focus Autism Other Dev.
Disabilities, 20, No. 4, 231-243 (2005).
9. D. Strickland, D. McAllister, C. Coles, and S. Osborne,
“An evolution of virtual reality training designs
for children with autism and fetal alcohol spectrum
disorders,” Top. Lang. Disord., 27, No. 3, 226-241
(2007).
10. S. Nichols, Virtual Reality Induced Symptoms and
Effects (VRISE): Methodological and the Theoretical
Issues, Doct. diss. Nottingham (1999).
11. G. Herrera, F. Alcantud, R. Jordan, et al., “Development
of simboli play through the use of virtual reality tools in
children with autistic spectrum disorders,” Autism, 12,
No. 2, 143-157 (2008).
12. E. Klinger, R.-M. Marié, and P. Fuchs, “Réalité virtuelle
et sciences cognitives: Applications en psychiatrie et
neuropsychologie,” Cognito, 3, No. 2, 1–31 (2006).
13. J. Cottraux, A. Berthoz, R. Jouvent, et al., “A compa­
rative controlled study of virtual reality therapy and
cognitive behavior therapy in panic disorder with
agoaphobia: design and methodological issues,” in:
VRIC - Laval Virtual, S. Richir and B. Taravel (eds.),
Laval (2005), pp 125-130.
14. E. B. Foa and M. J. Kozak, “Emotional processing of
fear: exposure to corrective information,” Psychol. Bull.,
99, No. 1, 20-35 (1986).
15. A. T. Beck and G. Emery, Anxiety Disorders and
Phobias: A Cognitive Perspective, Basic Books, New
York (1985).
16. A. Bandura, “Self-efficacy: toward a unifying theory of
behavioral change,” Psychol. Rev., 84, No. 2,191-215
(1977).
17. M. D. Lezak, Neuropsychological Assessment, Oxford
Univ. Press, New York (1995).
228 A. Fridhi et al.

18. A. A. Rizzo, M. T. Schultheis, K. A. Kerns, and 20. A. A. Rizzo, D. Strickland, and S. Bouchard, “The
C. Mateer, “Analysis of assets for virtual reality challenge of using virtual reality in telerehabilitation,”
applications in neuropsychology,” Neuropsychol. Telemed. J. Health, 10, No. 2, 184-195 (2004).
Rehab., 14, 207-239 (2004).
19. L. Pugnetti, L. Mendozzi, A. Motta, et al., “Evaluation
and retraining of adults’ cognitive impairment: which
role for virtual reality technology?” Comput. Biol. Med.,
25, No. 2, 213-227 (1995).
Neurophysiology is a copyright of Springer, 2018. All Rights Reserved.