Extended Reality (XR): The end of distance

Aradhana Deshmukh Rahul Raut

Tirth Patel
School of Computer Science & IT
(Cyber Security)
Symbiosis Skills & Professional
University Pune, India
tirthnppatel2001@gmail.com
Professor in School of Computer
Science & IT (Cyber Security)
Symbiosis Skills & Professional
University
Pune, India
aaradhana.deshmukh@sspu.ac.in
Professor in School of Computer
Science & IT (Cyber Security)
Symbiosis Skills & Professional
University
Pune, India
rahul.raut@sspu.ac.in

Abstract: As XR continues to evolve, its impact on the concept of

For centuries, physical distance has imposed limitations on
human interaction, education, and experiences. The
emergence of Extended Reality (XR), encompassing
Augmented Reality (AR), Virtual Reality (VR), and Mixed
Reality (MR), presents a paradigm shift. This paper explores
how XR technologies are blurring the lines between physical
and virtual spaces, effectively "ending distance." We
examine how XR fosters communication, collaboration,
learning, and access to experiences regardless of location.
We discuss the transformative potential of XR across
various societal sectors and explore the challenges and
ethical considerations that need to be addressed for
responsible implementation.
Keywords: Extended Reality, Virtual Reality, Augmented
Reality, Mixed Reality, Distance, Communication,
Collaboration, Education, Accessibility
I. INTRODUCTION
In an increasingly interconnected world, the concept of
distance is undergoing a profound transformation. Extended
Reality (XR), an umbrella term that includes Virtual Reality
(VR), Augmented Reality (AR), and Mixed Reality (MR), is
at the forefront of this change. By seamlessly blending digital
and physical environments, XR technologies are enabling
immersive experiences that transcend geographical
boundaries, fundamentally altering the way we interact,
collaborate, and engage with information and each other.
XR leverages cutting-edge advancements in hardware, such
as high-resolution, lightweight headsets, and sophisticated
software, including powerful graphics engines and AI-driven
interfaces. These innovations have propelled XR from niche
applications into mainstream use, offering unprecedented
opportunities across various sectors. In entertainment, XR
creates deeply interactive experiences, transforming gaming
and cinematic narratives. In education, it facilitates
immersive learning environments, allowing students to
engage with complex subjects in ways that traditional
methods cannot match. The healthcare industry is utilizing
XR for virtual surgical training, therapeutic interventions,
and enhanced patient care. Industrial applications of XR
include virtual prototyping, remote maintenance, and
workforce training, driving efficiency and innovation.
The transformative potential of XR is evident, but its adoption
is not without challenges. High development costs, technical
limitations, and concerns about privacy and data security pose
significant hurdles. Addressing these challenges requires a
concerted effort from technologists, policymakers, and
industry leaders to ensure that XR technologies are developed
responsibly and implemented effectively.
distance will only deepen, offering new ways to connect and
collaborate across the globe. This paper explores the potential
of XR to eliminate the barriers of distance, examining its
applications, benefits, and the challenges that must be
addressed to fully realize its promise. Through this
exploration, we aim to understand how XR can reshape our
interaction with the world and each other, heralding a new era
where distance is no longer a limiting factor..
A. Communication and Collaboration without Borders
Traditionally, physical distance has hindered communication
and collaboration. XR technologies offer innovative
solutions:
• VR for Immersive Communication: VR meeting
platforms create a sense of "presence" in shared
virtual spaces, fostering stronger connections and
collaboration despite geographical separation.
Imagine attending a business meeting in Tokyo or a
brainstorming session with remote team members,
all in a virtual environment that feels real.
• AR for Enhanced Communication: AR overlays
real-time information onto the real world, enhancing
remote communication and collaboration. Imagine a
doctor remotely guiding a surgery with real-time AR
annotations on the surgeon's view, or an engineer
using AR to share instructions with a colleague
working on a machine in a different location.
B. Objectives of the Research
The primary objective of this research is to develop a
comprehensive framework that supports the creation of
collaborative applications for multiple Extended Reality
(XR) devices, including Virtual Reality (VR) and
Augmented Reality (AR). This overarching goal is broken
down into several specific objectives:
1. Develop a Multi-Device XR Framework:
Create a robust and flexible framework that can support the
development of XR applications compatible with both VR
and AR devices. Ensure that the framework is adaptable to
new XR technologies as they emerge, maintaining its
relevance and utility over time.
2. Implement Collaborative Mechanisms:
Design and implement tools that facilitate effective
collaboration among users, such as user and object
highlighting, previewing, and synchronized interactions.
Ensure these collaborative mechanisms are intuitive and
enhance the user experience, making it easier for users to
work together in a shared virtual environment.
3. Address Synchronization Challenges:
Develop methods to maintain a synchronized state of the
virtual environment across multiple devices and users.
Implement an ownership mechanism to prevent conflicts
when multiple users interact with the same object
simultaneously, ensuring a smooth and consistent user
experience.
4. Evaluate User Interaction and Usability:
Conduct user testing to assess the usability and effectiveness
of the framework in facilitating collaboration. Gather
feedback to identify areas for improvement and ensure that
the interactions are seamless and intuitive for users on
different XR devices.
5. Design Multi-Device Interaction Interfaces:
Create interaction interfaces that are tailored to the unique
capabilities and constraints of different XR devices. Ensure
that interactions such as object manipulation, navigation, and
communication are optimized for both VR and AR platforms.
6. Facilitate Interoperability and Compatibility:
Ensure that the framework supports interoperability between
different XR devices, allowing users to collaborate
seamlessly regardless of the hardware they are using.
Develop guidelines and best practices for developers to
create applications that are compatible with a wide range of
XR devices.
7. Enhance User Representation and Awareness:
Implement features that enhance the representation of users
in the virtual environment, such as avatars and visual effects
that indicate user actions and presence. Develop mechanisms
to improve user awareness of important aspects of the
environment and the actions of other users.
8. Support Scalability and Flexibility:
Ensure that the framework is scalable, allowing for the
development of applications that can be used by a large
number of users simultaneously. Provide flexibility in the
framework to support various use cases and industries,
including education, healthcare, industrial training, and
remote work.
9. Explore Emerging Use Cases:
Identify and explore new use cases for collaborative XR
applications that can benefit from the multi-device
framework. Demonstrate the potential of the framework
through prototype applications in different domains,
showcasing its versatility and practical utility.
10.Provide Comprehensive Documentation and Tools:
Develop thorough documentation and tools to assist
developers in using the framework effectively.
Include examples, tutorials, and best practices to help
developers get started with creating multi-device XR
applications.
II. Overview of Extended Reality(XR):
Extended Reality (XR) is an umbrella term that encompasses
Virtual Reality (VR), Augmented Reality (AR), and Mixed
Reality (MR). These technologies blend the digital and
physical worlds to create immersive experiences that enhance
user interaction and engagement. As XR technologies
continue to evolve, they are revolutionizing various fields,
including entertainment, education, healthcare, and industrial
training.
A. Virtual Reality (VR)
Virtual Reality (VR) immerses users in a completely digital
environment, typically using head-mounted displays (HMDs)
and motion controllers. Users can explore and interact with
these virtual environments as if they were real, making VR an
ideal tool for applications such as gaming, training
simulations, virtual tours, and therapeutic interventions. The
immersive nature of VR allows users to experience scenarios
and environments that would be impossible or impractical to
replicate in the physical world.
B. Augmented Reality (AR)
Augmented Reality (AR) overlays digital information onto
the real world, enabling users to interact with virtual objects
within their physical surroundings. This is achieved through
devices such as smartphones, tablets, and AR glasses. AR is
widely used in applications such as navigation, education,
retail, and maintenance, where it enhances the user's
perception and interaction with their environment by
providing contextual information and interactive elements.
C. Mixed Reality (MR)
Mixed Reality (MR) combines elements of both VR and AR,
allowing for real-time interaction between physical and
digital objects. MR devices, such as Microsoft's HoloLens,
enable users to manipulate and interact with both physical and
virtual components in a seamless manner. This capability is
particularly useful in fields such as design, prototyping, and
collaborative work environments, where the integration of
real and virtual objects can enhance productivity and
creativity.
III. Methodology:
Interaction:
Interactions within the application serve as the foundation for
collaboration, particularly focusing on object manipulation in
the prototype, which centers on interior design. The planned
interactions include:
• Object Manipulation: Encompassing actions such
as Grabbing, Dragging, Throwing, and Scaling
objects. Synchronization of these interactions across
all users is crucial to prevent conflicts when
multiple clients attempt to interact with the same
object simultaneously.
To address conflicts, an ownership feature is implemented,
allowing users to interact with an object only once they have
ownership over it. Requests and transfers of ownership
resolve issues arising from simultaneous interactions with the
same object.
In Virtual Reality (VR), users navigate the environment and
interact with objects through teleportation, given the physical
space limitations. Initially, users move to pre-defined points
by pointing at them. However, to improve accessibility, users
are permitted to teleport to designated areas, enhancing
control over their position. For pointing, a curved line is
employed, as it proves more effective in crowded
environments where precise ground pointing is challenging.
In Augmented Reality (AR), markers determine users' current
positions. To transition to different rooms within the Virtual
Environment (VE), users synchronize their positions with
different markers, as each division in the VE is associated
with its marker.
By implementing these interaction mechanisms, users across
various XR devices can seamlessly collaborate and engage
with the application, fostering a cohesive and immersive
collaborative experience.
IV. DESGIN
A. Prototype Design:
During the development phase, insights into potential
challenges that developers may encounter when
implementing such applications were gleaned. The VR
prototype underwent development on two distinct computers
and VR systems: one utilizing the HTC Vive and the other
employing the Oculus Rift. Similarly, the AR prototype was
developed on a separate computer and underwent testing on
various mobile devices. Leveraging Unity3D for
development proved advantageous, as it provided essential
tools such as support for VR and AR devices, as well as a
robust physics and graphics engine.
Implementing the tools outlined in the previous section
facilitated the assembly of a prototype focused on interior
design. Concurrently, an AR prototype was developed
alongside the VR iteration, allowing for consistent
interactions across both platforms.
B. Interface Challenges:
Enabling support for multiple Extended Reality (XR)
technologies necessitates the creation of multimodal
interaction interfaces. Each user's interaction with the Virtual
Environment (VE) will vary depending on the device they are
using.
Virtual Reality Challenges:
In typical Virtual Reality (VR) applications, motion
controllers (MCs) are utilized for interacting with the virtual
world. However, due to the limited number of buttons on a
motion controller, it becomes challenging to accommodate
the numerous actions defined earlier. To address this, the
actions are segmented into distinct tools. These tools dictate
the actions that users can execute
with each motion controller. Users can select tools through a
menu, with only one tool active for each hand. This setup
allows users to simultaneously perform different tasks—for
instance, teleporting with one hand while grabbing an object
with the other. The tools also enable developers to map the
same input from motion controllers to various behaviors.
However, the selection of tools, which remains active at all
times, is the only input they cannot control. For the prototype,
four tools are implemented, each color-coded to enhance
differentiation. The blue tool facilitates object manipulation,
the purple one is for creating previews, and the green one is
for accepting previews. Additionally, these three tools also
manage input for teleportation and object highlighting.
Finally, the fourth tool is dedicated to highlighting the user.
Given that it encompasses only one action, it reverts back to
the previous tool after the highlighting is completed.
Augmented Reality Challenges:
In the case of our prototype, AR functionalities are leveraged
through mobile devices for visualization purposes.
Interactions in AR are primarily conducted via the device's
touch screen, distinguishing it from the VR experience where
motion controllers are typically used. AR proves
advantageous for visualizing data in the real world by
overlaying virtual elements onto the physical environment.
In our prototype, the entire Virtual Environment (VE) is
superimposed over the real-world setting.
AR users benefit from a deeper understanding of how virtual
objects interact with the real world. This understanding
enables them to place arrows within the environment,
indicating points of interest or directing actions such as where
to place objects. Furthermore, users can leave arrows in the
environment, pointing to specific locations or objects, aiding
in communication and collaboration.
These differences in interaction modalities present unique
challenges in developing XR applications. However, they
also offer opportunities for more immersive and extensive
experiences. Users can adapt their interactions with the
application by switching between devices, resulting in more
diverse and collaborative perspectives. These multimodal
interfaces foster collaboration among users, each
contributing from their unique vantage point, ultimately
enhancing the overall user experience.
C. Architecture:
To develop an application that accommodates multiple
Extended Reality (XR) devices, components reliant on
specific systems are segregated into isolated scenes. These
scenes are then loaded into the application based on the
connected device. Conversely, components such as the
environment are placed in a scene shared across all devices,
facilitating the reuse of components that do not necessitate
specific devices.
The diagram depicted in Fig 4.1 illustrates two Virtual
Reality (VR) and two Augmented Reality (AR) clients
connected to a common server. Dashed lines signify the
communication maintaining synchronized states of the
Virtual Environment (VE). Components within each box
offer high-level descriptions of different framework elements
and their interactions.
Of the four clients linked to the server, one must act as the
master client, responsible for VE synchronization in the event
of conflicts. Conflicts may arise, for instance, when multiple
clients attempt to interact with the same object
simultaneously. In such cases, the master client assigns
ownership of the object to one of the clients.
The networking tool must possess flexibility to
accommodate various server configurations. Photon Engine,
a well-established networking solution compatible with
Unity, offers such flexibility. Developers can opt to host their
own server or utilize Photon’s cloud servers. It supports
multiple clients from diverse devices to connect to the same
room and interact within a shared environment.
Through Photon Engine, object synchronization and
communication among multiple users are facilitated. In both
the VR and AR prototypes, only avatars and interactables are
synchronized across users. However, actions (denoted by
blue color) are specific to VR systems, as they hinge on input
from these devices. Additionally, the tool component for
enabling and disabling actions is visualized.(Fig 1.1)
Fig1.1: System Architecture
V. Challenges:
While Extended Reality (XR) technologies offer vast
potential and numerous applications, their development and
implementation come with significant challenges. These
challenges span technical, design, and user experience
domains, and addressing them is crucial for creating effective
and immersive XR applications.
A. Technical Challenges
Hardware Limitations:
Performance Constraints: XR applications require high
processing power and graphics capabilities to render
immersive environments in real-time. Many devices,
especially mobile ones, may struggle with these demands,
leading to lower frame rates and potential motion sickness.
Battery Life: XR devices, particularly standalone VR
headsets and AR glasses, often have limited battery life,
which can restrict prolonged use and impact user experience.
.Latency and Lag:
Real-Time Processing: Ensuring low latency is critical for
immersive experiences, particularly in VR where any delay
between user actions and system responses can cause
disorientation and discomfort.
Network Latency: For collaborative XR applications,
maintaining real-time synchronization across users in
different locations is challenging due to network latency and
bandwidth limitations.
Compatibility and Interoperability:
Diverse Platforms: Developing applications that work
seamlessly across various XR platforms (e.g., Oculus, HTC
Vive, Microsoft HoloLens) with different hardware
specifications and interaction models is complex.
Software Integration: Integrating XR applications with
existing enterprise systems, software, and data sources can be
challenging, requiring custom development and middleware
solutions.
B. Design Challenges
User Interface (UI) and User Experience (UX):
Intuitive Interactions: Designing intuitive and natural
interaction methods that leverage the unique capabilities of
VR, AR, and MR devices can be difficult. Users must be able
to navigate and interact with the virtual environment without
extensive training.
Accessibility: Ensuring XR applications are accessible to
users with disabilities involves designing adaptable interfaces
that cater to a wide range of physical and cognitive abilities.
Immersion and Presence:
Realism: Creating realistic and immersive environments
requires high-quality graphics, sound, and physics, which can
be resource-intensive and challenging to optimize.
Presence: Achieving a sense of presence, where users feel
truly immersed in the virtual environment, requires careful
attention to details such as environmental consistency, haptic
feedback, and minimizing distractions.
C. User Experience Challenges
Motion Sickness and Discomfort:
Simulator Sickness: Poorly designed VR experiences can
cause motion sickness, with symptoms such as nausea,
dizziness, and headaches. This is often due to mismatches
between visual stimuli and physical movement or low frame
rates.
Comfort: Long-term use of XR devices can cause physical
discomfort, including eye strain, neck pain, and fatigue,
particularly with heavy or poorly balanced headsets.
Learning Curve:
User Training: Users unfamiliar with XR technologies may
require significant training to use applications effectively,
which can be a barrier to adoption, particularly in enterprise
settings.
Complex Interactions: Complex or non-intuitive interaction
methods can frustrate users and reduce the overall
effectiveness of the application.
D. Development and Maintenance Challenges
Content Creation:
Resource Intensive: Creating high-quality 3D models,
animations, and environments is resource-intensive, requiring
specialized skills and significant time and effort.
Scalability: Ensuring that content can scale across different
devices and resolutions without losing quality or performance
is challenging.
Security and Privacy:
Data Protection: XR applications, especially those involving
sensitive data or personal information, must implement robust
security measures to protect user data and privacy.
User Safety: Ensuring user safety in physical spaces while
they are immersed in XR environments is critical, requiring
careful design to prevent accidents or injuries.
Continuous Improvement:
Feedback Loop: Gathering and integrating user feedback to
continuously improve the application is crucial, but can be
challenging to manage and implement effectively.
Technological Advances: Keeping up with rapid
advancements in XR technology and updating applications to
leverage new features and capabilities requires ongoing
investment and effort.
VI. Discussion:
The discussion for this seminar encompasses several key
points related to the development and implementation of
Extended Reality (XR) applications, particularly focusing on
the challenges and considerations associated with supporting
multiple XR devices. Here's an outline for the discussion:
A. Importance of Supporting Multiple XR Devices:
Establish the significance of accommodating various XR
devices, including Virtual Reality (VR), Augmented Reality
(AR), and Mixed Reality (MR), to reach a broader user base
and provide diverse user experiences.
Challenges in Developing Multi-Device XR Applications:
Identify and discuss the technical, design, and usability
challenges faced by developers when creating applications
compatible with multiple XR devices.
Address issues such as device-specific input methods,
hardware capabilities, and platform differences that impact
development efforts.
Strategies for Device-agnostic Development:
Explore strategies and best practices for developing XR
applications that are device-agnostic, allowing seamless
experiences across different XR platforms.
Discuss approaches such as scene isolation, component reuse,
and network synchronization to ensure compatibility and
consistency across devices.
Role of Networking Solutions:
Highlight the importance of networking solutions, such
Photon Engine, in enabling multiplayer and shared
experiences in XR applications.
Discuss the capabilities and limitations of networking
solutions in facilitating communication and synchronization
among users across different devices.
Considerations for User Interaction:
Examine the differences in user interaction modalities
between VR and AR devices, including input methods and
user interfaces.
Discuss design considerations for optimizing user interactions
based on the capabilities and limitations of each XR device.
Collaborative and Multi-user Experiences:
Explore the potential for collaborative and multi-user
experiences in XR applications, leveraging networking
solutions to enable real-time interaction and communication
among users.
Discuss use cases and scenarios where collaborative XR
experiences can enhance productivity, learning, and social
interaction.
Future Trends and Directions:
Speculate on future trends and developments in XR
technology, including advancements in hardware, software,
and networking solutions.
Discuss emerging use cases and applications for XR in
various industries, and the potential impact on society and
human interaction.
Ethical and Privacy Considerations:
Consider ethical and privacy implications associated with
multi-device XR applications, including data security, user
privacy, and accessibility concerns.
Discuss strategies for addressing ethical and privacy
challenges to ensure responsible development and
deployment of XR technologies.
Opportunities for Innovation and Collaboration:
Identify opportunities for innovation and collaboration in the
field of XR, including research initiatives, industry
partnerships, and community engagement.
Encourage participants to share insights, experiences, and
ideas for advancing XR technology and applications.
Conclusion and Takeaways:
Summarize key discussion points and insights gained from
the seminar.
Encourage participants to apply learnings and best practices
in their own XR development projects, and to continue
exploring the possibilities of XR technology in creating
immersive and impactful experiences.
By engaging in a comprehensive discussion covering these
topics, participants will gain a deeper understanding of the
complexities and opportunities in developing multi-device
XR applications, and be better equipped to address
challenges and drive innovation in the field.
VI I. Conclusion
Extended Reality (XR) technologies, which encompass
Virtual Reality (VR), Augmented Reality (AR), and Mixed
Reality (MR), hold immense potential to revolutionize how
we interact with digital and physical worlds. These
technologies offer transformative possibilities across various
sectors, including education, healthcare, manufacturing,
entertainment, and beyond. However, the journey to fully
harness the power of XR is fraught with challenges that need
to be meticulously addressed to ensure successful
development and implementation.
The development process of XR applications involves
multiple stages, from initial conceptualization and design to
development, testing, and deployment. Each stage presents its
own set of technical and design hurdles. These include
hardware limitations, latency issues, compatibility across
diverse platforms, and the need for intuitive and accessible
user interfaces. Additionally, maintaining a high level of
immersion and presence while avoiding user discomfort and
motion sickness is critical for user acceptance and
engagement.
The collaborative aspect of XR applications adds another
layer of complexity. Ensuring real-time synchronization,
managing network latency, and designing effective
collaboration tools are essential for creating seamless multi-
user experiences. Furthermore, the development of
multimodal interaction interfaces that cater to different
devices and user preferences is vital for maximizing the
flexibility and usability of XR applications.
VIII References
[1] V. Pereira, T. Matos, R. Rodrigues, R. Nóbrega and J.
Jacob, "Extended Reality Frameworkfor Remote
Collaborative Interactions in Virtual Environments," 2019
International Conference on Graphics and Interaction (ICGI),
2019, pp. 17-24, doi:10.1109/ICGI47575.2019.8955025.
[2] Omamah Almousa, Joana Prates, Noor Yeslam, Dougal
Mac Gregor, Junsong Zhang, VietPhan, Marc Nielsen,
Richard smith, and Karim Qayumi. Virtual Reality
Simulation Technology for Cardiopulmonary Resuscitation
Training: An Innovative Hybrid System With Haptic
Feedback. Simulation & Gaming, page 104687811882090, 1
2019.
[3] Luis Arenas, Telmo Zarraonandia, Paloma D´ıaz, and
Ignacio Aedo. A Platform for Supporting the Development of
Mixed Reality Environments for Educational Games. In
Learning and Collaboration Technologies. Lecture Notes in
Computer Science Volume 9192, pages 537 – 548. Springer,
Cham, 2015.
[4] Jeremy N. Bailenson, Andrew C. Beall, Jack Loomis, Jim
Blascovich, and Matthew Turk.Transformed Social
Interaction, Augmented Gaze, and Social Influence in
Immersive Virtual Environments. Human Communication
Research, 31(4):511 – 537, 10 2005.

Despite these challenges, the potential benefits of XR [5] John Brooke. SUS: A quick and dirty usability scale.
technologies are profound. They can eliminate the barriers of 1996.
distance, providing immersive training environments, [6] Cristiano Carvalheiro, Rui Nobrega, Hugo da Silva, and
enhancing remote collaboration, and delivering rich, Rui Rodrigues. ´ User Redirection and Direct Haptics in
interactive experiences that were previously unimaginable. Virtual Environments. In Proceedings of the 2016 ACM on
By addressing the technical, design, and user experience Multimedia Conference -MM ’16, pages 1146– 1155, New
challenges, developers can create XR applications that not York, New York, USA, 2016. ACM Press.
only meet but exceed user expectations.
[7] Phillip S. Dunston and Xiangyu Wang. Mixed Reality-
Based Visualization Interfaces for Architecture, Engineering,
Continued research and development in XR technologies are
and Construction Industry. Journal of Construction
crucial for overcoming existing limitations and exploring new
Engineering and Management, 131(12):1301 – 1309, 12
possibilities. This involves a commitment to iterative
2005.
improvement, user-centered design, and staying abreast of
technological advancements. As XR technologies evolve,
they will undoubtedly open up new frontiers in how we
experience and interact with the world, offering
unprecedented opportunities for innovation and growth.

In conclusion, while the path to effective XR implementation

is challenging, the potential rewards are significant. By
leveraging best practices in design and development,
prioritizing user experience, and fostering a collaborative and
innovative approach, we can unlock the full potential of XR
technologies and usher in a new era of digital interaction and
immersive experiences.