Received 12 September 2018; revised 24 April 2019; accepted 3 May 2019.

Date of publication 7 May 2019; date of current version 16 September 2021.

Digital Object Identifier 10.1109/TETC.2019.2915287

A Comparative Study of Virtual Reality Methods

of Interaction and Locomotion Based on Presence,
Cybersickness, and Usability
JESUS MAYOR , LAURA RAYA , AND ALBERTO SANCHEZ
stoles Madrid 28933, Spain
J. Mayor is with U-tad, Las Rozas de Madrid, Madrid 28290, Spain, and also with the Universidad Rey Juan Carlos, Mo
L. Raya is with U-tad, Las Rozas de Madrid, Madrid 28290, Spain
A. Sanchez is with the Universidad Rey Juan Carlos, Móstoles, Madrid, 28933, Spain and also with CCS, Madrid, Spain
CORRESPONDING AUTHOR: J. MAYOR (j.mayor.m90@gmail.com)

ABSTRACT In recent years, virtual reality has experienced notorious technological advances in a quite
short time. In an attempt to quickly response to this technical developments, some designs and developments
of inmersive environments have caused different symptoms such as dizziness or disorientation. This work
aims to analyze different methods of interaction and locomotion used in inmersive environments (Point of
Interest, Gamepad, Teleport, and Room-Scale) in three different aspects: presence, cybersickness, and usabil-
ity. We have designed and developed an experimental environment to carry out an empirical analysis with 48
subjects comparing the results obtained in different perceptual experiments. As a result, we provide a guide-
line for the use of these methods of interaction and locomotion in virtual reality.
INDEX TERMS Human-computer interaction, virtual reality, interaction techniques, virtual environment
modeling

I. INTRODUCTION used by VR companies. Then we analyze them by comparing

The emergence of more affordable Virtual Reality (VR) devices
for consumers (such as Oculus Go and HTC Vive) has caused
both the number of users and companies in this area to grow in
recent years. The commercial momentum of these head-
mounted displays (HMDs) has led to a heightened interest in
the use of this immersive technology in different types of appli-
cations, such as medicine [1], psychology [2], or education [3].
However, its adaptation has been excessively rapid in some
cases, e.g., offering just stereoscopy. An adapted VR experi-
ence should provide users the ability to interact with virtual
objects and move through the virtual three-dimensional space.
The capacity to interact with virtual objects in a similar way to
the real world may increase realism, improve manipulability of
virtual objects and enhance the sensation of presence [4]. Nev-
ertheless, if virtual locomotion is not correctly adapted to the
movement perceived by the vestibular system, the user could
perceive unwanted dizziness when moving in virtual environ-
ments [5]. Therefore, it is necessary to understand which factors
affect the effectiveness of virtual experiences [6].
For this purpose, first we conduct a study to identify the
locomotion and interaction methods that are most widely
the following aspects:
1) Presence. It defines a state of consciousness that indicates
the psychological sensation of being inside the virtual envi-
ronment [7]. Presence should be subjectively quantified
through the particular user experience, e.g., psychological
exposure to phobias [8], pain distractor in medical proce-
dures [9], or stress management training in the military [10].
The level of presence significantly influences how the user
perceives the VR experience. The way of interacting or
navigating in the virtual environment may affect the user’s
perception of presence [4], [11]. Matthias and Beckhaus
indicate that emotions are an essential factor to increase the
levels of presence [12]. Additionally, the simultaneous use
of the different sensory channels (vision, touch, sound)
may increase the sensation of presence [13], [14], [15].
2) Cybersickness. This effect is similar to motion sickness,
as it can generate sensations such as dizziness, nausea,
vertigo, or sweating. People who suffer or are prone to
vertigo may experience these symptoms in an increased
manner [16]. Cybersickness is the result of a discrepancy
between the visual simulation of movement and the input

2168-6750 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE
1542 permission. See ht_tps://www.ieee.org/publications/rights/index.html for more information. VOLUME 9, NO. 3, JULY-SEPT. 2021

Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
stimuli to the vestibular system [5]. This effect is quite
common in virtual experiences if they are not properly
designed, and can make people reject this technology.
Therefore, reducing cybersickness is one of the most crit-
ical milestones in the state of the art [17]. LaValle
presents a guide on how to reduce these symptoms by
correctly designing VR experiences [18]. The choice of
one or another method of locomotion could influence the
emergence of this effect.
3) Usability. The use of VR in different areas such us educa-
tion or training requires not only high presence and low
cybersickness, but also good mechanisms to perform tasks
efficiently. The way users interact and manipulate virtual
objects can be very different according to the interaction
methods used. These methods can influence the effective-
ness when carrying out a given task. A poor design may
cause frustration or discomfort [19], improper training, or
inadequate learning [20]. Thus, it is essential to evaluate
the interaction methods with usability in mind. The time it
takes to perform certain tasks (task performance) and the
dropout rate (percentage of users who cannot complete
the task) are becoming increasingly meaningful aspects to
measure usability in virtual environments [21].
These three aspects are significantly affected by the methods
of interaction and locomotion used during the virtual experi-
ence [22]. For instance, cybersickness is notoriously influenced
by the method of locomotion used [23], due to a remarkable
variation between the visual and balance feedback in each
method. An incomplete interaction may worsen usability, or a
limited locomotion may reduce the sensation of presence. In
addition, these three aspects can also influence each other. For
instance, presence can decrease as a result of the level of cyber-
sickness or usability may decrease if the user feels dizzy [24].
In this article, we carry out a simultaneous analysis of these
three virtual characteristics in order to evaluate which methods
of locomotion or interaction produce the best results in terms of
presence, cybersickness and usability in virtual worlds.
This article is organized as follows. Section II shows the
related work. Section III explains the different methods used in
our experiment. Section IV describes the experimental design.
Section V explains the results obtained for the methods ana-
lyzed. These results are discussed in Section VI, defining a
guide for selecting the most appropriate methods of interaction
and locomotion.
II. RELATED WORK
In this section we present a brief description of methods for mea-
suring VR aspects such as presence, cybersickness and usability.
Different research works indicate methods to measure the
sensation of presence in virtual worlds. Baren and Ijssel-
steijn [25] indicate two groups of methods for evaluating pres-
ence: subjective measures or objective corroborative measures.
On the one hand, presence should be measured through the
user experience by subjective methods [7], [26] because it is a
mental sensation or manifestation. Subjective measures are
usually obtained from questionnaires, qualitative methods,
VOLUME 9, NO. 3, JULY-SEPT. 2021
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
continuous evaluation, etc. being the questionnaires the most
used. Specifically, the state-of-the-art survey conducted by
Schuemie et al. [27] shows a compendium for the subjective
evaluation of presence. The authors show the different theories
that define presence and their main related questionnaires to
measure it, including the relevant and widely used Igroup Pres-
ence Questionnaire (IPQ) [28], which we have used in our
study. For example, Hernandez-Ossa et al. [29] conducted a
comparative study of a real world versus a virtual environment
performing tasks with a wheelchair. They determined a similar
level of presence between the two scenarios through the IPQ.
On the other hand, objective measures often require the
use of specialized measuring devices, such as biofeeback
sensors. For example, we can measure the skin conductance
or heart rate with sensors placed on the hands or fingers [30].
This biometric data related to emotional arousal is associated
with presence, but is also altered by other factors such as
anxiety caused by the experiment or cybersickness [12]. For
this reason, objective measures are sometimes used to rein-
force the results obtained by subjective measures [25].
Regarding cybersickness, different articles ask users about the
symptoms they felt when using a VR environment through ques-
tionnaires. One of the most relevant questionnaires regarding
cybersickness is Simulator Sickness Questionnaire (SSQ) [31].
For example, all of the following articles use SSQ to achieve their
results. Almeida et al. [32] compared the results of prior warning
or not of the possible symptoms (dizziness, feel bad, etc.). They
determine that the prior warning worsened the symptoms show-
ing a hypochondriac effect on cybersickness. Budhiraja et al.
[33] conducted a study with a first person shooter game. They
found that the use of motion blur, at moments of greater motion
dissonance, can reduce cybersickness when using a keyboard
and mouse. David et al. compared the cybersickness produced in
different scenes. They determine that a scene with more realism
(or level of detail) could produce more cybersickness [34].
Regarding usability (in this case, with reference to the task
performance and dropout rate), Bowman et al. present a com-
plete guide for designing 3D interfaces [19] where it exhaus-
tively analyzes the state of the art in order to find the best
practices and recommendations. Note that the most classic way
to measure task performance is by measuring the time it takes
to perform a task and the number of errors [25]. Yao et al.
developed other guide for the design of both 3D interfaces and
interactions with elements of the virtual environment [35].
Among other recommendations, they advise to integrate the
interfaces in the virtual environment or to take into account the
depth of vision to implement the reticle. Both guides are not
based on empirical studies. Concerning the task performance,
Christiansen et al. use this measure to determine how good was
the rehabilitation of brain-injured patients through VR [36].
The study showed the potential of this type of treatment.
Most of the above studies measure presence, cybersickness
or usability separately, without assessing the influence they
have on each other. However, these three main aspects can be
dependent on each other [24], and they could be significantly
affected according to the method of interaction or locomotion
1543
 Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
used. For instance, Weech et al. [23] reviewed articles relating
cybersickness and presence, concluding that there is a nega-
tive correlation between both variables. Focused on naviga-
tion techniques, Zielasko et al. measured different hands-free
locomotion methods inside a virtual environment through
data visualization [37]. This study evaluated the appropriate-
ness of these methods by measuring presence and cybersick-
ness. However, they did not indicate how usability might be
affected, and the methods analyzed are currently not the most
used by the VR companies. Buttussi and Chittaro measured
the presence generated by different devices using a gamepad,
[6] although natural interaction methods were not analyzed.
Fernandes and Feiner reduced the cybersickness produced by
the Gamepad method modifying the user’s field of view
(FOV) according to the amount of movement applied [38].
Langbehn et al. [11] analyzed different methods of loco-
motion: Joystick, Teleport and Redirected Walking. They
conclude that Redirected Walking provides better spatial rec-
ognition compared to the other two. They also indicate that
the Joystick is the locomotion method that produces more
cybersickness. Nonetheless, they did not analyze other meth-
ods widely used, such as Room-Scale or Point of Interest,
nor did they assess the impact of the type of interaction.
In contrast to the previous works mentioned above, our
approach evaluates four methods of interaction and locomotion
indicated as most used by VR companies: Gamepad, Points of
interest, Room-Scale and Teleport. These methods are widely
used in VR environments (see Section IV-A). We simulta-
neously analyzed the impact of the three main aspects: presence,
cybersickness and usability in these methods. The objective of
this simultaneous analysis is to define the main properties of
each method and to determine its conditions of use in virtual
environments.
III. METHOD
Our study combines subjective measures for presence and
cybersickness characteristics with objective measures to
evaluate usability in the virtual world.
We have work with two of the most relevant questionnaires
in the state of the art for presence and cybersickness. In the
case of usability, we evaluate objective measures like perfor-
mance, through the time automatically measured in completing
a task, and efficiency, through the dropout rate.
A. PRESENCE MEASUREMENT: IPQ
There are different questionnaires for measuring presence
levels. We have selected IPQ [28] because it is widely used
to measure presence [39], [40].
The IPQ was designed through the analysis of items from
other questionnaires [24], [41]. IPQ extracts 14 of the most sig-
nificant items related to presence. These evaluated items follow
a 7-point Likert scale where 0 means “completely disagree”
and 6 “completely agree”. Some questions are inverted to cor-
roborate the result where 6 means “completely disagree” and 0
“completely agree”.
1544
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
This questionnaire considers three groups of items. They
can be evaluated separately as metrics for specific aspects of
presence:
 Spatial Presence (SP). The sense of being physically
present in the virtual environment.
 Involvement (INV). Measurement of the attention to the
virtual environment and the involvement experienced.
 Realism (REAL). Measurement of the subjective experi-
ence of realism in the virtual world.
Furthermore, the questionnaire contains an additional sin-
gle-item general group, General (G), to measure a level of
overall presence. The use of specific measures (SP, INV,
REAL) is useful to validate the subjective opinion about the
general presence (G) of the user. Besides, the authors recom-
mend calculating the final scores of these groups by calculat-
ing the mean of all their items.
B. CYBERSICKNESS MEASUREMENT: SSQ
We have selected SSQ [31] among the different methods for
measuring user symptoms produced by VR exposure. This ques-
tionnaire, developed through the results obtained by multiple
flight simulators, is also widely used today. For instance, Ragan
et al. used this questionnaire to evaluate the cybersickness pro-
duced by scaling HMD rotations to increase workspace [42].
They demonstrate that the vestibular dissonance produced by a
scaled rotation causes cybersickness. Chessa et al. compared the
levels of cybersickness produced by different types of displays
(including modern VR devices) [43] in three different settings: a
railway environment, a skyscraper and a roller coaster. Balk
et al. validated a modern translation of this questionnaire into
Portuguese, demonstrating that it remains valid twenty years
after its creation [44].
The questionnaire consists of 16 items that follow a 4-point
Likert scale, where 0 means “none” and 3 “severe”, dividing it
into three item clusters. It applies specific weights to the differ-
ent answers. Consequently, the result of these clusters may
exceed the maximum value of a single question because it is
not just an average. High outcomes mean worse symptoms for
the user. The clusters defined in the SSQ are:
 Oculomotor (O). Symptoms related to eye function
problems (eyestrain, difficulty focusing, blurred vision,
headache).
 Disorientation (D). Loss of orientation or related sensa-
tions (dizziness, vertigo).
 Nausea (N). Related to stomach upset or similar reac-
tions (nausea, stomach awareness, increased salivation,
burping).
Finally, the level of cybersickness is determined by the
Total Score (TS), obtaining the results through these clusters.
C. USABILITY MEASUREMENT
As other studies [19], we refer to the task performance and
dropout rate. The time to perform a task in VR training sys-
tems can indicate the appropriateness of using one or another
interaction and locomotion technique depending on the
application [19], [36], [45].
VOLUME 9, NO. 3, JULY-SEPT. 2021
Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
We measure the task performance as the time needed to
perform a specific task using the cumulative method defined
by Das et al. [46]. The summation of the sample interval con-
stitutes a measure of task performance. The time is automati-
cally recorded and obtained by the average time taken for
each task. Additionally, we measure the dropout rate, record-
ing whether the user completes the task or not. We have mea-
sured two types of tasks in this study:
 3D Graphical User Interface: Interaction with menus
and/or 3D interfaces. This type of tasks is very common
in VR experiences, e.g., in the interaction when the user
has to select a dialog.
 Manipulation: The user must manipulate virtual objects
to complete the task. There are a wide variety of possible
manipulation tasks. Not all interaction methods allow
users to perform each one of them with the same usabil-
ity (take an object or two objects at the same time, rotate
objects, move them, etc.). Therefore, it is important to
model the interaction methods accurately. In our study,
the interactions have been designed to allow users to
manipulate the objects in different ways. We classified
the different types of interactions as: sequential and par-
allelizable; simple and complex. Sequential interactions
are those that must be performed once the previous one
is finished. Parallelizable interactions are those that can
be performed at the same time using both hands. Com-
plex interactions requires the combination of different
movements, using 6 degrees of freedom (DOF) to han-
dle the object (3 positional DOF and 3 rotational DOF).
Simple interactions only use 3 positional DOF.
IV. EXPERIMENT
This section defines the procedures and design decisions
made to carry out the developed experiment.
A. SELECTION OF THE METHODS OF INTERACTION
AND LOCOMOTION
There are multiple methods of locomotion used for virtual real-
ity. We have analyzed the most commonly used methods
under three main aspects (presence, cybersickness and usabil-
ity). To this end, we have previously conducted a study to
know the methods that companies use most frequently. We
contacted the project managers of 14 international VR compa-
nies (Future Lighthouse - USA, Virtual Voyagers - Spain, Tar-
sier Studios - Sweden, HTC Vive - USA, Estudio Future -
Spain and others). We asked them through a questionnaire
about the methods of interaction and locomotion most used in
their VR projects. All the items mentioned in the questionnaire
were widely explained, including the required explanatory vid-
eos about them. The survey was conducted by offering a cate-
gorization of the most common research methods found in the
literature (Room-scale, Points of interest, Gamepad, Teleport,
Walking in place, Specialized device to simulate real displace-
ment and Special actions with hands). If desired, they were
free to add any additional options.
VOLUME 9, NO. 3, JULY-SEPT. 2021
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
The methods with the highest percentage of use in the dif-
ferent companies were: Room-Scale with a percentage of
27.08 percent, Gamepad with 25.01 percent, Teleport with
20.83 percent, and Points of interest with 18.75 percent. The
small remaining percentage is distributed among the other
different options. We have grouped the methods obtained
according to the type of interaction with the environment:
 Directed by sight: Methods that use the sight to move, inter-
act or indicate the direction through the movement of the
head with the HMD. In Points of interest the vision deter-
mines the movement and interaction with virtual objects. In
VR experiences, Gamepad uses sight to move through the
virtual world. It is combined with buttons on an external
controller that is not visible within the virtual environment.
 Natural movement or interaction: Methods that use nat-
ural or realistic movement (with external controllers
visible within the virtual environment) to move or inter-
act. Room-Scale only uses natural movement to interact
with the virtual objects while in Teleport, the user dis-
placement is combined with external controllers.
B. DESCRIPTION OF THE IMPLEMENTED METHODS
1) POINTS OF INTEREST
The Points of interest method, also called “hotspots” or “run-
ning on rails”, is a purely directed by sight method, where the
subjects use the centre of the fixed FOV area to interact with
the different elements and navigate through the virtual
world [47]. The user can move through the virtual environ-
ment observing different visual indicators that stand out in the
scenario. In our implementation, when users stare at the indi-
cators, a reticle with the charging process is displayed to start
the default movement. Our process takes three seconds to
complete in which the user cannot change the view to another
place. This three-second stand-by confirmation is considered
not too annoying and prevents unwanted movements. We use
ray casting from the position and orientation of the HMD to
detect where the user is pointing, which does not have to
exactly correspond to where he/she is looking with his/her
eyes. Horizontal movements in the real world are not allowed
because the user remains seated, as shown in Section IV-C.
In the implementation of this method, we set the displace-
ment velocity to 1.5 m/s, which is close to the real walking
speed [38]. Yao et al. recommended the use of this human
natural walking speed [35]. This movement provides 2 DOF
in the horizontal plane in the virtual world (see Figure 1).
The interaction model with the virtual objects is similar to
the movement, showing a reticle close to the objects. We have
implemented the operation of the reticle using of an overlaying
grid to the surface pointed by the HMD according to Yao et al.
In the same way as navigating, the users must stare at the object
they wish to interact with and it is then highlighted. Once
again, after three seconds, the object is fixed to the center of
the FOV and it remains at the same distance in the direction
pointed by the HMD as long as it is selected. Then users can
move the objects using the head movement (with 3 DOF in ori-
entation). When the users want to release the object, they just
1545
 Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
FIGURE 1. The figure shows the different DOF allowed in the user’s real movement (orange/light color) and additional DOF thanks to the
virtual locomotion method used (blue/dark color). The sum of both indicates all the possibilities that the user has to move in the virtual
environment for each method.
have to keep the orientation of the HMD fixed for three sec-
onds. All these actions have visual feedback that reflects the
loading time on the displayed reticle.
The user usually moves through the virtual world and
interacts with its objects at the same time. Both actions are
performed directly by sight in this method. For this reason,
the users can keep their eyes still to look at a displacement
point without releasing the previously gripped object.
2) GAMEPAD
In this method, the user employs a gamepad controller to interact
with virtual objects and move around the scenario. The move-
ment of the used HMD (see Section IV-C) allows users to
observe the scenario in 3DOF in orientation [6], [48]. Users can
then use the joystick of the gamepad forward or backward and
point with the HMD to select the direction of movement in
2DOF (horizontal plane position on x and z axes). Vertical
movements (on y-axis) in the real world are not allowed because
the user remains seated. Our displacement velocity is also close
to 1.5 m/s, as in the previous method.
In our implementation, there is a reticle in the center of the
FOV to allow users to interact with virtual objects. Similarly
to Point of interest, the reticle indicates the objects which the
user can interact with, but instead of waiting three seconds
users can press two buttons on the remote control to interact
with the object. The first one is pressed to indicate that the
user want to grab the selected virtual object, attaching it to the
same distance in the direction pointed by the HMD. If the
user presses the same button again, the virtual object will be
released. The second button can be pressed to perform differ-
ent actions (rotate, activate, change position, etc.) depending
on the selected object. This facilitates its usability.
3) TELEPORT
In this method, users can manipulate trackable devices (which
can be seen within the virtual scene) for displacement. This type
of device detects the position of the user’s hands and, at the same
time, provide a way to select objects and to navigate through sce-
narios. When the user presses one of the buttons of the device, a
laser appears in the virtual world. The position of the laser means
1546
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
the place where the user wants to move. Users move instantly to
this position when the button is released. Simple cut-off transi-
tions or instantaneous transitions are most appropriate to avoid
breaking presence levels [39]. We have replicated the implemen-
tation made by Bozgeyikli [49] to develop this method.
In this method, the users have 3DOF in orientation and 1DOF
in position on the vertical y-axis. Users can rotate, bend down
and stand up but are not allowed to move around in the horizon-
tal plane of the real world to force them to use the trackable con-
trollers. To interact with virtual objects, the users can operate
with the same trackable controllers. They can place the controller
closer to a virtual object and press the button specified to pick it
up. They can manipulate it with the natural movement of the
hand, rotating and moving the object in 6 DOF.
4) ROOM-SCALE
In this method, the users can physically walk in the real world by
using the tracking capabilities of the HMD, observing their dis-
placement in the virtual environment. Room-Scale is the closest
to the actual user movements and provides 6 DOF of motion
(orientation and position). We develop the method in a similar
way to [50].
The user can see the limits of the tracking space thanks to vir-
tual lines. These lines allow the user to move around safely
increasing confidence within the virtual space. Similar to Tele-
port, users have to use the trackable controllers in their hands to
interact with objects. With the natural movement of the hand, the
subject can rotate and move the object with controllers in 6 DOF.
C. ENVIRONMENT
We chose HTC Vive VR HMD as the experimental device.
This device offers a high resolution (2160 x 1200, FOV-110
degrees) and performance (90-Hz) compared to other
well-known devices, such as Samsung Gear VR or CAVE
[51]. HTC Vive allows us to develop all the methods of inter-
action and locomotion studied in this article. In addition it pro-
vides lighthouses to send the signal to track the HMD. All
experiments were performed in an area of 4m  4m (limited
by HTC Vive) with the same temperature avoiding reflective
surfaces that could affect the behavior of the lighthouses (using
VOLUME 9, NO. 3, JULY-SEPT. 2021
Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
FIGURE 2. Different images of the scenario and models created
for the experiment.
black/opaque room). We configured the scenario for a correct
implementation of each method. Figure 1 shows a visual dia-
gram summarizing and explaining the environment and con-
figuration used for the experiment of each of the methods of
interaction and locomotion:
 Points of interest. This method has been performed from
a sitting position to limit the DOF of the head. We used a
rotating stool that enables the rotation of the subject to
observe the virtual environment without moving from
the same place. Although the HMD allows 6 DOF, we
limit three of them to adapt to this method.
 Gamepad. Similarly, we also use a rotating stool to
limit the DOF. We have used an Xbox controller
because of its wireless connection and ergonomic con-
sistency that make it widely used as a method of loco-
motion [48]. This gamepad allows users to perform all
the functionality explained above for this method.
 Teleport. This method allowed the user to have move-
ment only with the controllers, without having to walk
around. HTC Vive trackable controllers were used for
natural interaction [51]. Then, as the user was standing,
the center of the room was marked to check that the sub-
ject was not moving from the mark (avoiding the use of
two of their walking DOF). We wanted to differentiate
this method clearly from Room-scale by limiting the dis-
placement of the user in the real environment. We told
the subjects that they could turn or bend over to look at
the environment, but not walk around the workspace. If
at the end of the experiment the subject had moved more
than 30 cm from the mark, the test was invalidated.
 Room-Scale. For the implementation of this method we
used Vive head tracking to control the position of the
user in the room, allowing free movement (6 DOF)
through the defined workspace. We also used HTC
Vive controllers in order to track users’ hands position.
The spoken instructions and the environmental sounds of the
virtual experience were played through closed over-ear head-
phones. We have used positional audio as a narrative resource
VOLUME 9, NO. 3, JULY-SEPT. 2021
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
to guide the user through the experience. The experiment is
described as a story divided into four chapters, where each
chapter tries to associate the four possible combinations of
interactions according to our previously defined classification
(sequential-parallelizable and simple-complex). We used
Maya 2017 and Unity 5.5.2 to develop the experience, ensuring
that the computer-generated content is as realistic as possible.
A video has been provided as supplementary content, which
can be found on the Computer Society Digital Library at http://
doi.ieeecomputersociety.org/10.1109/TETC.2019.2915287, in
which the visual quality of the different chapters can be appre-
ciated. The storyboard used in each chapter is the following:
1) In the first chapter, the user can explore the virtual sce-
nario. The characters appear for the first time in a cheer-
ful and reliable atmosphere. In Figure 2 A we observe
the user’s interaction with a robot. The user has to
comb the robot antennas. The interactions defined for
this chapter are simple (requiring low maneuverability)
and designed to be performed sequentially.
2) In the second chapter the user must perform tasks indi-
cated by the virtual avatars, such as manipulating tools
around the virtual world or serving water from a teapot
to a glass, (see Figure 2 B.) In this chapter, the defined
interactions are complex (requiring 6 DOF to be com-
pleted) and designed to be performed sequentially.
3) In the third chapter the user must complete a mini puzzle
to open an inter-dimensional portal (see Figure 2 C).
When the portal is open, users discover that it is a deep
pit. In this case, the task is complex and parallelizable,
being able to use both hands in the methods that allow it.
4) The last chapter, the user must manipulate objects by mov-
ing them around the virtual environment, including dis-
placements that can cause fear. In Figure 2 D we can see
the jump into the pit that takes the user to another scenario.
In this last chapter the type of interaction is simple and
parallelizable.
The user’s movements through the virtual environment of
each chapter have been designed to be similar in distance. In
addition to the types of interaction already indicated in each
chapter, all chapters include interaction functions with 3D user
interfaces. Therefore, the same interface task can be compared
between all chapters without being affected by each of them.
D. PARTICIPANTS AND PROCEDURES
First, we carried out a pilot study with four people to evaluate
the adequacy of the duration of the experiment and the num-
ber of questions in the questionnaires. The results indicated
that the duration and number of questions had been appropri-
ately designed. The average duration of each chapter was
6.25 minutes. Between the chapters, the subjects carried out
the questionnaires with a rest time similar to the time
required to complete the VR experience.
The number of subjects who finally performed the experi-
ment was 48 participants; 24 women and 24 men, with a
median age of 28.5 (range = 18-62, mean = 30.59, standard
deviation (SD) = 11.996). The participants did not necessarily
1547
 Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
TABLE 1. Results (mean values) obtained throughout the study
for the IPQ and SSQ questionnaires including the average
duration of tasks (3D interface and manipulation).
Variable Mean/Max.
G 5.112/6
SP 4.79/6
IPQ
INV 4.46/6
REAL 3.04/6
TS 7.72/200.34
N 8.50/159.18
SSQ
O 9.71/292.32
D 20.66/92.54
Interface 15.58/300
Task Manipulation 14.28/300
Max.= Maximum value each variable can take.
have previous VR experience. All subjects carried out the
experiment under the same conditions (same device, same
computer, same procedure and in the presence of the same
supervisor). The order of the chapters was the same for all
users, but the order the locomotion methods (one different per
chapter) changed for each user in a balanced way. The balanc-
ing strategy consisted of giving each participant a possible per-
mutation of methods, completing all possible permutations
with all participants. We used the balanced latin square algo-
rithm, which allowed us to eliminate consecutive dependen-
cies between methods. This way, the participants had to
perform the experiment with all methods described above but
in a different order. Thus, the content of each chapter does not
affect the results of the selected locomotion method.
V. RESULTS
In this section, we detail the results obtained from the experi-
ments carried out. We have used SPSS v24 for statistical analysis
and chart creation. First, we evaluate whether there are statistical
significant differences between the methods of interaction and
locomotion in terms of presence and cybersickness. Then, we
present the results about usability. Subsequently, we detail the
conclusions of the subjective evaluation of the same methods.
A. GENERAL RESULTS
First, we verified whether the sample collected from the 48
participants followed a normal distribution. A Kolmogorov-
Smirnov normality test was performed resulting in a signifi-
cance level of p < 0.05, meaning that the sample do not fol-
low a normal distribution. For this reason, we use non-
parametric tests for statistical analysis.
The first experiment carried out analyzed the whole experi-
ence (the 4 chapters) in terms of presence and cybersickness.
Table 1 shows the scores for each variable relative to presence
(G, SP, INV and REAL) and cybersickness (TS, N, O, and D).
In addition it includes the average time of the tasks performed.
Second, we contrasted whether the separation into four dif-
ferent chapters may have affected affect presence or cyber-
sickness. We used the non-parametric Friedman test of
differences. The test reported the following values:
1548
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
1) In terms of presence (IPQ): SP (Chi-Square (x2 ) = 5.69,
p = 0.128); INV (x2 = 4.46, p = 0.216); REAL (x2 =
1.42, p = 0.699). Therefore, there were not any statisti-
SD cally significant differences among the chapters with
respect to presence.
1.04
0.850 2) In terms of cybersickness (SSQ): N (x2 = 3.56, p =
0.313); O (x2 = 3.77, p = 0.287); D (x2 = 3.23, p =
1.13
0.996 0.357). Similarly, no statistically significant differences
10.86 were found for cybersickness.
16.37 3) In terms of task completion times, interaction with 3D
15.73 graphical user interfaces reported x2 = 4.37 (p =
28.98 0.224), which showed no significant differences for the
14.28 chapters, as it was the same type of interaction. On the
90.90 other hand, the manipulation with objects reported x2 =
15.08 (p = 0.002) which showed statistically significant
differences, as the interaction varied by chapter.
These results indicate that the content of the different
chapters does not significantly affect the presence and cyber-
sickness. As expected, we only found statistically significant
differences for the different manipulation tasks. It will there-
fore be studied in a further analysis, separating it by chapters.
In the following sections we detail the comparative analy-
sis of the different methods of interaction and locomotion
regarding presence, cybersickness, and usability.
B. DIFFERENCES BETWEEN INTERACTION AND
LOCOMOTION METHODS
In order to determine whether there are significant differen-
ces between the different methods of locomotion we con-
ducted an inter-method analysis through hypothesis contrast
testing. We used a Friedman test with a significance level of
p < 0.05, finding significant differences. Table 2 shows the
results obtained for IPQ, SSQ and 3D interface task meas-
ures. In Table 3 we can see the results obtained by grouping
in pairs the methods that showed statistically significant dif-
ferences in the previous Friedman test. The manipulation
task is not reflected in both tables because it showed statisti-
cally significant differences among the chapters.
1) PRESENCE EVALUATION
In the case of IPQ, we detected statistically significant differ-
ences in the variables G (x2 = 22.353, p = 0.001) and SP
(x2 = 11.978, p = 0.007) (see Table 2). These differences are
reflected in the spatial perception that users have of the vir-
tual environment. We used a Wilcoxon Signed-Rank test to
study the relationship among the different locomotion
methods. Table 3 represents the results obtained for all varia-
bles that showed statistically significant differences in the
previous Friedman test (G and SP). We did not detect any
statistically significant differences for IPQ in the case of the
methods of Point of Interest and Gamepad (p > 0.05).
Nonetheless, we found statistically significant differences for
spatial presence between Gamepad and Teleport (p < 0.05),
but there are no such statistically significant differences in
general presence (G). In addition, we found statistically
VOLUME 9, NO. 3, JULY-SEPT. 2021
Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability

TABLE 2. Friedman’s test results on the different methods of

interaction and locomotion for IPQ, SSQ and the duration of the
3D interface tasks.

Variable p-value
G 0.000 **
SP 0.007 **
IPQ
INV 0.839
REAL 0.101
TS 0.000 **
N 0.122
SSQ
O 0.000 **
D 0.000 **
Task Interface 0.002 **

(* = p < 0.05, ** = p < 0.01).

significant differences between the methods of Point of Inter-

est and Teleport in REAL (p = 0.003).
Finally, following the guidelines of the IPQ test, we ana-
lyzed the final scores obtained of general presence (G) and
spatial presence by the four methods of interaction and loco-
motion. Figure 3 shows that the method that obtain the high-
est sense of presence is Room-Scale (with GP: Mean = 5.45,
SD = 0.798, and SP: Mean = 5.08, SD = 0.696) that presents
a statistically significant difference from the rest: Teleport
(with GP: Mean = 5.18, SD = 0.790, and SP Mean = 4.83,
SD = 0.756), Gamepad (with GP: Mean = 5.02, SD = 1.19,
and SP: Mean = 4.65, SD = 0.937) and Points of interest
(with GP: Mean = 4.81, SD = 1.21, and SP: Mean = 4.59,
SD = 0.921). In summary, the methods with the highest pres-
ence are those that have a closer interface to the natural one.
2) CYBERSICKNESS EVALUATION
In the case of the SSQ questionnaire, as shown in Table 2, we
detected statistically significant differences for the variables:
Oculomotor (O) (x2 = 18.495, p = 0.000), Disorientation (D)
(x2 = 31.051, p = 0.000) and Total Score (x2 = 28.401, p <
0.000). The results of Friedman test indicate that the locomo-
tion methods clearly affect the cybersickness levels.
We used a Wilcoxon signed-rank test with the aim of studying
the relationship among the different methods of locomotion.
Table 3 represents the results obtained for all variables that
showed statistically significant differences in the previous
TABLE 3. Results of the Wilcoxon signed-ranks test in pairs
between the different methods of interaction and locomotion.
Variable POI-GP POI-TL POI-RS GP-TL GP-RS
IPQ
G 0.230 0.018* 0.001** 0.401 0.029* 0.031*
SP 0.706 0.146 0.002** 0.043* 0.003** 0.042*
TS 0.261 0.000** 0.003** 0.000** 0.001** 0.327
SSQ O 0.597 0.000** 0.003** 0.000** 0.006** 0.427
D 0.323 0.000** 0.008** 0.000** 0.000** 0.272
Task Interface 0.013* 0.119 0.630 0.000** 0.182
[POI = Points of interest, GP = Gamepad, TL = Teleport, RS= Room-Scale]
(* = p < 0.05, ** = p < 0.01).
FIGURE 3. Igroup Presence Questionnaire results: Comparison
among the mean scores obtained. The score obtained for each
of the methods is shown on the vertical axis. The variables that
obtained statistically significant differences from this question-
naire are displayed on the horizontal axis. The outliers are plot-
ted by circles. The asterisks are extreme outliers (more than
three times the height of the boxes).
Friedman test (TS, O, D). We observed statistically significant
differences (p < 0.05) in all combinations of methods except
between Points of interest and Gamepad, and Teleport and
Room-Scale. These groups may be due to the fact that they have
the same type of interaction: Directed by sight or Natural move-
ment. Gamepad and Points of interest have a similar movement
speed but Gamepad do not use rotatial movements. Meanwhile,
the dissonance with the senses is similar in Teleport and Room-
Scale, although the displacement is completely different.
Finally, the weighted mean is defined as the final score of
the different groups of symptoms. The methods that produce
less cybersickness are Teleport (Mean = 4.61, SD = 8.27) and
Room-Scale (Mean = 5.35, SD = 7.7) with a statistic-
ally significant difference from the rest: Points of interest
(Mean = 9.47, SD = 12.20) and Gamepad (Mean = 11.46,
SD = 13.05). The reason for this difference between pairs of
methods may be because the methods that use vision to move
TL-RS in the virtual world produce more discrepancy between the
real (perceived by the vestibular system) and virtual move-
ment. Figure 4 shows a representation of these scores and the
differences between methods. We can see how disorientation
(D) is the symptom group that shows the most differences.
0.032* 3) USABILITY EVALUATION
In relation to the task performance, as shown in Table 2, we
observed statistically significant differences when interacting

VOLUME 9, NO. 3, JULY-SEPT. 2021 1549

Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
 Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
FIGURE 4. Simulator Sickness Questionnaire Results: Quantitative
results in the SSQ. The score obtained for each of the methods is
plotted on the vertical axis. The horizontal axis is classified
according to the symptom groups that showed significant differ-
ences. The outliers are plotted by circles. The asterisks are
extreme outliers (more than three times the height of the boxes).
with 3D interfaces (p = 0.002). We conclude that the method
of interaction affects the performance in 3D interface selec-
tions. To study the relationship between the different meth-
ods of interaction we used a Wilcoxon signed-rank test (see
Table 3). We have detected significant differences between
the Points of Interest and Gamepad methods, and between
the Teleport and Room-Scale methods.
Finally, we used the mean value of the recorded time-marks
to obtain the final score. Higher time values indicate worse per-
formance in the execution of the tasks of each method. Figure 5
shows the box plots obtained in the interaction with 3D interfa-
ces. We found that Gamepad (Mean = 11.45, SD = 9.54) and
Room-Scale (Mean = 14.6, SD = 11.73) score better when
interacting with 3D interface than Points of interest (Mean =
14.7, SD = 8.66) and Teleport (Mean = 21.55, SD = 21.65).
As it can be seen in the initial analysis, in the case of the
manipulation task, it was necessary to carry out an independent
analysis by chapter. The interaction times depend on the type of
interaction designed in each chapter. As indicated in Section III-
C, we study the interaction with virtual elements by modeling
four different types of manipulation tasks: simple-sequential,
complex-sequential, complex-parallelizable and simple-paralle-
lizable. In this case, we have used the non-parametric Kruskal
Wallis test because there are not repeated measures (in the differ-
ent chapters the users do not repeat a questionnaire for each
method). The results show statistically significant differences for
complex-sequential interactions (H(3) = 32.61, p < 0.001),
complex-parallelizable (H(3) = 26.35, p < 0.001) and simple-
parallelizable (H(3) = 16.745, p = 0.001). We made a compari-
son of methods in pairs for the different chapters, by using the
Mann-Whitney U-test. Table 4 shows the results for this test,
which has been performed at a level of p < 0.05.
Additionally, there was a 13.5 percent dropout in the 192
tests carried out. We have performed a Chi-square
1550
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
FIGURE 5. 3D Interface Task: Average efficiency results by
responding to menu options. The average time obtained for
each of the methods is measured on the vertical axis, and each
method is plotted on the horizontal axis. The outliers are plotted
by circles. The asterisks are extreme outliers (more than three
times the height of the boxes).
independence test comparing the methods of interaction with
the number of dropouts. We found a statistically significant
interaction between both variables (x2 (3) = 27.23, p < 0.001),
being the dropout ratio for each interaction and locomotion
method: Points of interest (65.4 percent), Gamepad (15.4 per-
cent), Teleport (15.4 percent) and Room-Scale (3.8 percent).
We have also analyzed the relationship between the dropout
rate and the interaction of each chapter. The Chi-square inde-
pendence test shows that there is a statistically significant inter-
action between both variables (x2 (3) = 22.24, p < 0.001).
The reported values are: complex-sequential (57.7 percent),
complex-parallelizable (30.8 percent), simple-parallelizable
(7.7 percent) and simple-sequential (3.8 percent). Figure 6
shows the average elapsed time of the tasks, where the drop-
outs can be observed. Complex-sequential interaction and
Points of Interest had the most dropouts.
These results indicate that methods with natural interaction
(Teleport and Room-Scale) allow the user to optimize the
task in the case of complex interactions. For the tasks that are
characterized as simple, the method that requires the least
time is Teleport in the case of Simple-Sequential, and Room-
Scale in the case of simple-parallelizable.
VOLUME 9, NO. 3, JULY-SEPT. 2021
Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability

TABLE 4. Results of the Mann-Whitney U-test that analyzed the dif-

ferent methods of interaction in pairs, classifying them according
to the different interactions studied in the different chapters.

Simple Complex Complex Simple

Sequential Sequential Parallelizable Parallelizable
POI&GP 0.128 0.003** 0.020* 0.002**
POI&TL 0.114 0.000** 0.049* 0.266
POI&RS 0.630 0.000** 0.000** 0.002**
GP&TL 0.114 0.189 0.799 0.006**
GP&RS 0.266 0.000** 0.000** 0.887
TL&RS 0.266 0.001** 0.000** 0.010**

[POI = Points of Interest, GP = Gamepad, TL = Teleport, RS= Room-Scale]

(* = p < 0.05, ** = p < 0.01).

4) SUBJECTIVE USER SATISFACTION

In this latest questionnaire, users indicated which method of FIGURE 6. Manipulation task: Average time of each of the meth-
locomotion they considered to be the best in general terms. The ods in each of the interactions studied is displayed on the verti-
objective was to detect the user’s preference among the different cal axis. The horizontal axis displays the interaction of the
locomotion methods analyzed and to study whether there was a chapters with the different methods.
relation with those that offer a higher presence, less cybersick-
ness or better usability. Therefore, we asked the users at the end, needed. If the experience does not require interaction, it
which method they thought was the best. The worst rated method is an alternative for developers.
according to the subjective opinion was Gamepad (2.08 percent  Gamepad. This method obtains medium values in all three
of the users selected it as the best option). Points of Interest had a aspects, standing out as the best alternative to interact with
percentage of 4.17 percent, Teleport 12.5 percent, and Room- static 3D interfaces. The results suggest that users are
Scale 81.25 percent. As expected, the most chosen methods familiar with this type of interaction. This method is appro-
were those that have obtained more presence and less cybersick- priate for users who want a familiar experience with simple
ness in the analysis with IPQ and SSQ, respectively. It was inter- interactions that does not require high levels of presence.
esting to note that, between Teleport and Room-Scale, users This method does not gets good results in cybersickness,
preferred (by a significant margin) the Room-Scale method. but there are recent studies trying to reduce it [38].
 Teleport. This method obtains high scores in cybersickness
VI. DISCUSSION and presence. However, the ability to interact with objects
The present study has analyzed the simultaneous impact of may be limited by the dependence of a hand on locomotion.
presence, cybersickness, and certain aspects of usability Teleport shows great versatility since it has no technical
according to the method of locomotion and interaction used in limitations (the workspace is unlimited; the interaction is
virtual environments. Specifically, we have designed a virtual natural and can be parallelized). This method can be used
experimental environment in which 48 subjects are exposed to in place of Room-Scale in VR experiences that require a
some of the most common methods of interaction and locomo- larger workspace. The not-so-good results reflected in the
tion: Points of Interest, Gamepad, Teleport and Room-Scale. task performance indicate that this method is not as intuitive
We found statistically significant differences between the as Room-Scale or Gamepad. Note that this difficulty does
four methods analyzed in terms of presence, cybersickness, not appear to affect the levels of presence.
and usability. In order to classify the different methods used
according to the three aspects evaluated, we summarize the TABLE 5. Classification for each of the methods, considering
data obtained in Table 5. The conclusions obtained regarding those that reported statistically significant differences between
each of the method of locomotion and interaction analyzed are them or not (equal position).
presented below: POI GP TL RS
 Points of Interest. This method obtains the lowest scores
Locomotion Cybersickness 2nd 2nd 1st 1st
in presence, cybersickness, and usability. It is notewor-
thy the high number of dropouts produced in the interac- Both Presence 3rd 2nd 1st 1st
tion task. Its low usability reduces the possibility of Simple 2nd 1st 2nd 1st
being used for applications that require high interaction Complex 3rd 2nd 2nd 1st
Interaction
with the virtual world, unless combined with another spe- Sequential 3rd 2nd 2nd 1st
Parallelizable 4th 2nd 3rd 1st
cific method of interaction. In the case of operating with
3D interfaces, this method gets better scores. The main 3D Interface 2nd 1st 3rd 2nd
advantages of this method are that it does not require the (POI = Points of Interest, GP = Gamepad, TL = Teleport and RS = Room-
use of hands, and absolute positioning devices are not Scale)

VOLUME 9, NO. 3, JULY-SEPT. 2021 1551

Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
 Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
 Room-Scale. This method achieves the best results for all
aspects. It is the best valued in presence, cybersickness, and
usability. Note that, although most of the subjects chose the
Room-Scale method in the subjective opinion question-
naire, the results through IPQ and SSQ do not reflect statis-
tically significant differences between Room-Scale and
Teleport. We found that interaction with 3D interfaces
through Room-Scale is not the fastest one. We realized that
users generally did not associate interaction with 3D interfa-
ces with physical contact interaction. Several users tried to
interact remotely with the HTC Vive controllers. Thus, it
would be advisable to interact with 3D interfaces remotely,
for example using Gamepad (it is the best method in this
type of interactions) together with Room-Scale.
Based on the results obtained during the experiments, we
can conclude that the method best valued by users in terms of
presence and cybersickness is Room-Scale. Similarly,
following the results of the IPQ and SSQ questionnaires,
Room-Scale is one of the most appropriate methods to provide
presence and reduce cybersickness. Regarding interactions
with virtual objects, Room-Scale is also ranked as the best
alternative in all types of interaction (simple, complex, sequen-
tial and parallelizable). Only in the case of interaction with 3D
interfaces, Room-Scale is ranked as the second alternative.
The main technical drawback of the Room-Scale is the
limitation of the workspace. Workspaces may be limited by
the different specifications of the HMDs or by the physical
area where the user can move. This limitation makes it diffi-
cult for Room-scale to be used in some cases.
In general, the present study has provided useful knowledge
for the future design of new, more advanced methods of interac-
tion and locomotion. Further research is needed to design new
methods of interaction that fill or improve the gaps in the meth-
ods analyzed. Improving the three aspects becomes a key ele-
ment in the development of new methods of locomotion or
interaction in virtual reality (for example, new methods should
focus on increasing presence by eliminating cybersickness).
Because the Room-scale method has the best results in terms
of presence, cybersickness, and usability, future work should
focus on eliminating its major constraint (workspace). For
example, designing new techniques that make better use of the
workspace, such as the Redirected Walking methods [52].
These new methods should, subsequently, be compared with
Room-scale method for the three aspects analyzed in this study.
ACKNOWLEDGMENTS
This work has been partly supported by the Spanish Ministry
of Economy (grant RTI2018-098694-B-I00), Indra and Fun-
dacion Universia (CicerOn Virtual Speech Coach).
REFERENCES
[1] J. L. McGrath, J. M. Taekman, P. Dev, D. R. Danforth, D. Mohan,
N. Kman, A. Crichlow, and W. F. Bond, “Using virtual reality simulation
environments to assess competence for emergency medicine learners,”
Academic Emergency Med., vol. 25, no. 2, pp. 186–195, 2018. [Online].
Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/acem.13308
1552
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
[2] M. Peskin, K. Wyka, J. Cukor, M. Olden, M. Altemus, F. S. Lee, and
J. Difede, “The relationship between posttraumatic and depressive symp-
toms during virtual reality exposure therapy with a cognitive enhancer,” J.
Anxiety Disorders, vol. 61, pp. 82–88, 2019.
[3] L. Freina and M. Ott, “A literature review on immersive virtual reality in
education: State of the art and perspectives,” in Proc. Elearn. Softw. Edu.,
2015, pp. 133–141.
[4] M. Slater, M. Usoh, and A. Steed, “Taking steps: The influence of a walk-
ing technique on presence in virtual reality,” ACM Trans. Comput.-Human
Interaction, vol. 2, no. 3, pp. 201–219, Sep. 1995. [Online]. Available:
http://doi.acm.org/10.1145/210079.210084
[5] M. E. McCauley and T. J. Sharkey, “Cybersickness: Perception of self-
motion in virtual environments,” Presence: Teleoperators Virtual Envi-
ron., vol. 1, no. 3, pp. 311–318, 1992. [Online]. Available: https://doi.org/
10.1162/pres.1992.1.3.311
[6] F. Buttussi and L. Chittaro, “Effects of different types of virtual reality dis-
play on presence and learning in a safety training scenario,” IEEE Trans.
Vis. Comput. Graph., vol. 24, no. 2, pp. 1063–1076, Feb. 2018.
[7] M. Slater and S. Wilbur, “A framework for immersive virtual environments
(FIVE): Speculations on the role of presence in virtual environments,” Pres-
ence: Teleoperators Virtual Environ., vol. 6, no. 6, pp. 603–616, 1997.
[Online]. Available: https://doi.org/10.1162/pres.1997.6.6.603
[8] H. M. Peperkorn, J. Diemer, and A. Mhlberger, “Temporal dynamics in the
relation between presence and fear in virtual reality,” Comput. Human
Behavior, vol. 48, pp. 542–547, 2015.
[9] P. Indovina, D. Barone, L. Gallo, A. Chirico, G. De Pietro, and G. Antonio,
“Virtual reality as a distraction intervention to relieve pain and distress during
medical procedures: A comprehensive literature review,” Clinical J. Pain, Feb.
2018. [Online]. Available: https://doi.org/10.1097/AJP.0000000000000599
[10] F. Pallavicini, L. Argenton, N. Toniazzi, L. Aceti, and F. Mantovani, “Vir-
tual reality applications for stress management training in the military,”
Aerosp. Med. Human Perform., vol. 87, no. 12, pp. 1021–1030, 2016.
[11] E. Langbehn, P. Lubos, and F. Steinicke, “Evaluation of locomotion tech-
niques for room-scale VR: Joystick, teleportation, and redirected walking,”
in Proc. Virtual Reality Int. Conf., Laval, 2018. [Online]. Available: http://
basilic.informatik.uni-hamburg.de/Publications/2018/LLS18a
[12] H. Matthias and S. Beckhaus, “Adaptive generation of emotional impact
using enhanced virtual environments,” Presence, vol. 21, no. 1, pp. 96–116,
Feb. 2012.
[13] J. M. Loomis, “Presence in virtual reality and everyday life: Immersion
within a world of representation,” Presence: Teleoperators Virtual Envi-
ron., vol. 25, no. 2, pp. 169–174, 2016.
[14] M. C. Whitton, “Making virtual environments compelling,” Commun.
ACM, vol. 46, no. 7, pp. 40–47, Jul. 2003. [Online]. Available: http://doi.
acm.org/10.1145/792704.792728
[15] M. S. El-Nasr, “Projecting tension in virtual environments through
lighting,” in Proc. ACM SIGCHI Int. Conf. Advances Comput. Entertain-
ment Technol., 2006. [Online]. Available: http://doi.acm.org/10.1145/
1178823.1178898
[16] M. Pavlou, R. Kanegaonkar, D. Swapp, D. Bamiou, M. Slater, and
L. Luxon, “The effect of virtual reality on visual vertigo symptoms in
patients with peripheral vestibular dysfunction: A pilot study,” J. Vestibu-
lar Res., vol. 22, no. 5, pp. 273–281, 2012.
[17] P. Budhiraja, M. R. Miller, A. K. Modi, and D. A. Forsyth, “Rotation blur-
ring: Use of artificial blurring to reduce cybersickness in virtual reality first
person shooters,” CoRR, 2017. [Online]. Available: http://arxiv.org/abs/
1710.02599
[18] S. M. LaValle, Virtual Reality. Cambridge, U.K.: Cambridge Univ. Press.,
2016, Art. no. 418.
[19] D. A. Bowman, E. Kruijff, J. J. LaViola, and I. Poupyrev, 3D User Interfa-
ces: Theory and Practice. Redwood City, CA, USA: Addison Wesley
Longman, 2004, Art. no. 512.
[20] J. Nielsen, “Usability inspection methods,” in Proc. Conf. Companion
Human Factors Comput. Syst., 1994, pp. 413–414. [Online]. Available:
http://doi.acm.org/10.1145/259963.260531
[21] C. J. Lin, H.-J. Chen, P.-Y. Cheng, and T.-L. Sun, “Effects of displays on
visually controlled task performance in three-dimensional virtual reality
environment,” Human Factors Ergonom. Manuf. Service Industries,
vol. 25, no. 5, pp. 523–533, 2015. [Online]. Available: https://
onlinelibrary.wiley.com/doi/abs/10.1002/hfm.20566
[22] Y. Ling, H. T. Nefs, W.-P. Brinkman, C. Qu, and I. Heynderickx, “The
relationship between individual characteristics and experienced presence,”
Comput. Human Behavior, vol. 29, no. 4, pp. 1519–1530, 2013.
VOLUME 9, NO. 3, JULY-SEPT. 2021
Mayor et al.: A Comparative Study of Virtual Reality Methods of Interaction and Locomotion Based on Presence, Cybersickness and Usability
[23] S. Weech, S. Kenny, and M. Barnett-Cowan, “Presence and cybersickness
in virtual reality are negatively related: A review,” Frontiers Psychology,
vol. 10, 2019, Art. no. 158, doi: 10.3389/fpsyg.2019.00158.
[24] B. G. Witmer and M. J. Singer, “Measuring presence in virtual environments:
A presence questionnaire,” Presence, vol. 7, no. 3, pp. 225–240, Jun. 1998.
[25] J. V. Baren and W. IJsselsteijn, “Measuring presence: A guide to current
measurement approaches,” Deliverable OmniPres Project, no. 5, 2004,
https://ispr.info/about-presence-2/tools-to-measure-presence/omnipres-guide/.
[26] T. B. Sheridan, “Musings on telepresence and virtual presence,” Presence:
Teleoperators Virtual Environ., vol. 1, no. 1, pp. 120–126, 1992. [Online].
Available: https://doi.org/10.1162/pres.1992.1.1.120
[27] M. J. Schuemie, P. V. D. Straaten, M. Krijn, and C. V. D. Mast, “Research on
presence in virtual reality: A survey,” Cyberpsychology Behavior: Impact Internet
Multimedia Virtual Reality Behavior Soc., vol. 4, pp. 183–201, May 2001.
[28] T. Schubert, F. Friedmann, and H. Regenbrecht, “The experience of pres-
ence: Factor analytic insights,” Presence: Teleoperators Virtual Environ.,
vol. 10, no. 3, pp. 266–281, 2001. [Online]. Available: https://doi.org/
10.1162/105474601300343603
[29] K. A. Hernandez-Ossa, B. Longo, E. Montenegro-Couto, M. A. Romero-
Laiseca, A. Frizera-Neto, and T. Bastos-Filho, “Development and pilot test
of a virtual reality system for electric powered wheelchair simulation,” in
Proc. IEEE Int. Conf. Syst. Man Cybern., Oct. 2017, pp. 2355–2360.
[30] C. Dillon, J. Freeman, and J. Davidoff, “Aroused and immersed: The psycho-
physiology of presence,” in Proc. 3rd Int. Workshop Presence, 2000, pp. 27–28.
[31] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, “Simula-
tor sickness questionnaire: An enhanced method for quantifying simulator
sickness,” Int. J. Aviation Psychology, vol. 3, no. 3, pp. 203–220, 1993.
[32] A. Almeida, F. Rebelo, P. Noriega, and E. Vilar, “Virtual reality self
induced cybersickness: An exploratory study,” in Proc. Int. Conf. Appl.
Human Factors Ergonomics, 2018, pp. 26–33.
[33] P. Budhiraja, M. R. Miller, A. K. Modi, and D. A. Forsyth, “Rotation blurring:
Use of artificial blurring to reduce cybersickness in virtual reality first person
shooters,” CoRR, 2017. [Online]. Available: http://arxiv.org/abs/1710.02599
[34] S. Davis, K. Nesbitt, and E. Nalivaiko, “Comparing the onset of cybersick-
ness using the oculus Rift and two virtual roller coasters,” in Proc. 11th
Australasian Conf. Interactive Entertainment, 2015, pp. 3–14. [Online].
Available: http://crpit.com/confpapers/CRPITV167Davis.pdf
[35] R. Yao, T. Heath, T. Davies, A. Forsyth, and P. Mitchell, N. Hoberman,
“Oculus best practices guide,” Mar. 2014. [Online]. Available: https://
static.oculus.com/documentation/pdfs/intro-vr/latest/bp.pdf
[36] C. Christiansen, B. Abreu, K. Ottenbacher, K. Huffman, B. Masel, and
R. Culpepper, “Task performance in virtual environments used for cogni-
tive rehabilitation after traumatic brain injury,” Archives Phys. Med. Reha-
bil., vol. 79, no. 8, pp. 888–892, 1998. [Online]. Available: https://doi.org/
10.1016/S0003–9993(98)90083-1
[37] D. Zielasko, S. Horn, S. Freitag, B. Weyers, and T. W. Kuhlen, “Evaluation of
hands-free HMD-based navigation techniques for immersive data analysis,” in
Proc. IEEE Symp. 3D User Interfaces, Mar. 2016, pp. 113–119.
[38] A. S. Fernandes and S. K. Feiner, “Combating VR sickness through subtle
dynamic field-of-view modification,” in Proc. IEEE Symp. 3D User Inter-
faces, Mar. 2016, pp. 201–210.
[39] L. Men, N. Bryan-Kinns, A. S. Hassard, and Z. Ma, “The impact of transi-
tions on user experience in virtual reality,” in Proc. IEEE Virtual Reality,
Mar. 2017, pp. 285–286.
[40] J. Vasconcelos-Raposo, M. Bessa, M. Melo, L. Barbosa, R. Rodrigues,
C. M. Teixeira, L. Cabral, and A. A. Sousa, “Adaptation and validation of the
igroup presence questionnaire (IPQ) in a portuguese sample,” Presence: Teleop-
erators Virtual Environ., vol. 25, no. 3, pp. 191–203, Dec. 2016.
[41] M. Slater, M. Usoh, and A. Steed, “Depth of presence in virtual environ-
ments,” Presence: Teleoperators Virtual Environ., vol. 3, no. 2, pp. 130–144,
1994. [Online]. Available: https://doi.org/10.1162/pres.1994.3.2.130
[42] E. D. Ragan, S. Scerbo, F. Bacim, and D. A. Bowman, “Amplified head
rotation in virtual reality and the effects on 3D search, training transfer,
and spatial orientation,” IEEE Trans. Vis. Comput. Graph., vol. 23, no. 8,
pp. 1880–1895, Aug. 2017.
[43] M. Chessa, G. Maiello, A. Borsari, and P. J. Bex, “The perceptual quality
of the oculus Rift for immersive virtual reality,” HumanComput. Interac-
tion, pp. 1–32, 2016. [Online]. Available: https://doi.org/10.1080/
07370024.2016.1243478
[44] S. A. Balk, M. A. Bertola, and V. Inman, “Simulator sickness question-
naire: Twenty years later,” in Proc. 7th Int. Driving Symp. Human Factors
Driver Assessment Training Vehicle Des., Mar. 2013, pp. 257–263.
VOLUME 9, NO. 3, JULY-SEPT. 2021
Authorized licensed use limited to: Oulu University. Downloaded on October 04,2022 at 12:49:24 UTC from IEEE Xplore. Restrictions apply.
[45] L. Raya, S. Bayona, L. Pastor, and M. Garcia, “A new user-adapted search
haptic algorithm to navigate along filiform structures,” IEEE Trans. Hap-
tics, vol. 7, no. 3, pp. 273–284, Jul.–Sep. 2014. [Online]. Available: doi.
ieeecomputersociety.org/10.1109/TOH.2014.2324574
[46] H. Das, H. Zak, W. S. Kim, A. K. Bejczy, and P. S. Schenker, “Operator
performance with alternative manual control modes in teleoperation,”
Presence: Teleoperators Virtual Environ., vol. 1, no. 2, pp. 201–218,
1992. [Online]. Available: https://doi.org/10.1162/pres.1992.1.2.201
[47] J. Treleaven, J. Battershill, D. Cole, C. Fadelli, S. Freestone, K. Lang, and
H. Sarig-Bahat, “Simulator sickness incidence and susceptibility during
neck motion-controlled virtual reality tasks,” Virtual Reality, vol. 19, no.
3, pp. 267–275, Nov. 2015. [Online]. Available: https://doi.org/10.1007/
s10055-015-0266-4
[48] B. Sarupuri, S. Hoermann, M. C. Whitton, and R. W. Lindeman, “Evaluat-
ing and comparing game-controller based virtual locomotion techniques,”
in Proc. Int. Conf. Artif. Reality Telexistence Eurograph. Symp. Virtual
Environ., 2017, pp. 133–139.
[49] E. Bozgeyikli, A. Raij, S. Katkoori, and R. Dubey, “Point & teleport loco-
motion technique for virtual reality,” in Proc. Annu. Symp. Comput.-
Human Interaction Play, 2016, pp. 205–216. [Online]. Available: http://
doi.acm.org/10.1145/2967934.2968105
[50] S. W. Greenwald, Z. Wang, M. Funk, and P. Maes, “Investigating social
presence and communication with embodied avatars in room-scale virtual
reality,” in Proc. Int. Conf. Immersive Learn., 2017, pp. 75–90.
[51] C. Bassano, F. Solari, and M. Chessa, “Studying natural human-computer
interaction in immersive virtual reality: A comparison between actions in
the peripersonal and in the near-action space,” in Proc. Int. Joint Conf.
Comput. Vis. Imaging Comput. Graph. Theory Appl., 2018, pp. 108–115.
[52] M. Azmandian, T. Grechkin, M. Bolas, and E. Suma, “The redirected
walking toolkit: A unified development platform for exploring large virtual
environments,” in Proc. IEEE 2nd Workshop Everyday Virtual Reality,
Mar. 2016, pp. 9–14.
JESUS MAYOR received the MS degree in com-
puter graphics and simulation from the Centro Uni-
versitario de Tecnologa y Arte Digital (U-tad;
Madrid, Spain). He is working toward the PhD
degree in computer science at the Universidad Rey
Juan Carlos (Madrid, Spain). In 2013, he worked as
a ask if this should be research fellow with the Uni-
versidad de Alcala (UAH; Madrid, Spain). Since
2016, has been a full-time lecturer and Virtual Real-
ity dynamizer with U-tad. His main fields of
research are virtual reality, user experience, com-
puter graphics, and acoustic simulation.
LAURA RAYA received the MS degree in com-
puter science, the MS degree in computer graphics,
and the PhD degree in computer science from the
Universidad Rey Juan Carlos of Madrid, in 2008,
2010, and 2014, respectively. She is a professor and
researcher with the Centro Universitario de Tecno-
loga y Arte Digital (U-tad), Spain. Since 2013, she
is the head of the master’s degree and the manager
of Virtual Reality Projects, Department of Com-
puter Science, U-tad (Madrid, Spain).
ALBERTO SANCHEZ received the MS and PhD
degrees, obtaining the Extraordinary PhD Award, in
computer science from the Universidad Politecnica de
Madrid, in 2004 and 2008, respectively, and the MS
degree in marketing from the Universidad Autonoma
de Madrid (UAM), in 2006. He is an associate profes-
sor and researcher with the Universidad Rey Juan Car-
los and CCS, Spain. He has also done long placement
abroad in some prestigious international researching
centers, such as CERN, NeSC, NRC, and the Univer-
sity of Melbourne. His primary research areas are high-
performance and large-scale computing, data analysis,
and visualization.
1553