Introduction to ARVR

By Nilima Patil
 What is Augmented Reality
Augmented reality (AR) is an experience where designers
enhance parts of users’ physical world with computer-
generated input. Designers create inputs—ranging from
sound to video, to graphics to GPS overlays and more—in
digital content, which responds in real-time to changes in
the user’s environment, typically movement.
 Augmented reality (AR)
• Augmented reality (AR) is an interactive experience that
combines the real world and computer-generated content.

• Augmented reality (AR) augments your surroundings by

adding digital elements to a live view, often by using the
camera on a smartphone.
• Virtual reality (VR) is a completely immersive experience that
replaces a real-life environment with a simulated one.
 Examples
AR app helps you to try on a garments from the comfort of home.
App helps to see how a new piece of furniture looks in your living room

• Here are some actual applications you can engage with today.
• Ikea Place is a mobile app that allows you to envision Ikea furniture in
your own home, by overlaying a 3D representation of the piece atop a
live video stream of your room.
• YouCam Makeup lets users virtually try on real-life cosmetics via a living
selfie.
• Repair technicians can don a headset that walks them through the steps
of fixing or maintaining a broken piece of equipment, diagramming
exactly where each part goes and the order in which to do things.
• Various sports are relying on augmented reality to provide real-time
statistics and improve physical training for athletes.
• Beyond gaming and other entertainment cases, some
business examples of virtual reality include:
• Architects are using VR to design homes — and let clients
“walk through” before the foundation has ever been laid.
• Automobiles and other vehicles are increasingly being
designed in VR.
• Firefighters, soldiers and other workers in hazardous
environments are using VR to train without putting
themselves at risk.
• Target app. The Target retail app feature called
See it in Your Space enables users to take a photo
of a space in their home and digitally view an
object, like a picture on the wall or a chair, to see
how it will look there.
• Apple Measure app. The Measure app on Apple
iOS acts like a tape measure by enabling users to
select two or more points in their environment
and measure the distance between them.
• Snapchat. Snapchat filters use AR to overlay a
filter or mask over the user's Snap or picture.
• Pokemon Go. Pokemon Go is a popular mobile AR game that uses
the player's GPS to detect where Pokemon creatures appear in
the user's surrounding environment for them to catch.

• Google Glass. Google Glass is Google's first commercial attempt

at a glasses-based AR system. This small wearable computer
enables users to work hands-free. Companies such as DHL and DB
Schenker use Google Glass and third-party software to enable
frontline workers to be more efficient when it comes to global
supply chain logistics and customized shipping.

• U.S. Army. The U.S. Army uses AR in an eyepiece called Tactical

Augmented Reality (TAR). TAR mounts onto the soldier's helmet
and aids in locating another soldier's position.
 Virtual Reality
1. What is Virtual Reality?

Virtual Reality (VR) is a simulated, digital experience that can be

similar to or completely different from the real world. VR experiences

are created with computer technology and presented to the user

through a VR headset or head-mounted display (HMD), which creates

an immersive and interactive three-dimensional environment. VR

should be experienced and interacted with as if that virtual

environment were real.

• Currently, standard virtual reality systems use either virtual reality
headset or multi-projected environments to generate some realistic
images, sounds and other sensations that simulate a user's physical
presence in a virtual environment.

• A person using virtual reality equipment is able to look around the

artificial world, move around in it, and interact with virtual features or
items.

• Users may explore and interact with virtual items and characters, the
environment can react to their activities in real-time, giving them the
impression that they are physically present there.
• Virtual reality applications are applications that make use of
virtual reality (VR), an immersive sensory experience that
digitally simulates a virtual environment.

• Applications have been developed in a variety of domains,

such as entertainment ( Video games) education,
architectural and urban design, digital marketing and
activism, engineering and robotics, virtual communities, fine
arts, healthcare and clinical therapies, heritage and
archaeology, occupational safety, social science and
psychology
• Examples are: Google Cardboard, 3D- Virtual tour, 360
degree video
 Mixed Reality
• When Virtual Reality (VR) and Augmented Reality (AR) technologies are
utilized together, the result is referred to as Mixed Reality (MR). The
development of MR technology has opened the door for creative and
interactive experiences, and thus, has grown in popularity and significance.

• It enabling real-time user interaction with both virtual and actual worlds.

• MR technology is made possible by the application of cutting-edge

hardware and software, such as cameras, sensors, and specialized goggles
that detect and analyse the user’s movements and surroundings.

• In addition to interacting with the user and the actual environment, this
enables the technology to construct virtual things that seem to be present
in the real world.
 Key Features of Mixed Reality

• Physical and digital worlds are blended together in MR to produce a seamless

experience in which digital things seem to be a part of the physical
surroundings.

• With the use of MR technology, users may interact and modify virtual items
in real-time

• MR employs cutting-edge sensors and cameras to track the user’s motions

and the surroundings. As a result, the technology may produce virtual things
that can be interacted with as if they were real.

• MR uses cutting-edge graphics and high-resolution monitors to produce

lifelike virtual things that seem to be present in the real world.

• Users may fully immerse themselves in the virtual environment and

experience it as a part of themselves by using MR’s immersive experience.
 Examples of Mixed Reality

Pokemon Go: Pokemon Go is a well-known mobile game that employs augmented reality (AR)
technology to superimpose digital Pokemon figures over the surrounding environment. Real-
time Pokemon catching and global exploration are also possible for gamers.
Volvo Cars: To give its consumers a virtual showroom experience, Volvo Cars integrated MR
technology. They may see and alter several automobile models in a virtual setting using the MR
headset.
Surgery Stimulation: Medical students and surgeons may train and hone their abilities in a safe
and controlled environment, thanks to the use of MR in the creation of realistic surgical
simulators.
Education: Immersive learning experiences are created in the education sector using MR.
Students may explore and learn in a more engaging and participatory way by using it to make a
learning experience, for example, virtual field trips to historical locations or natural settings.
 Mixed Reality Applications

• Gaming: The use of MR in gaming enables the creation of immersive,

mixed-reality experiences. It makes the experience more dynamic and
engaging by enabling real-time player interaction with virtual items.

• Architecture and engineering: To visualize and simulate complicated

designs, MR is employed in these fields. Designers can view and interact
with their designs in a more realistic way through MR, which helps them
decide more wisely.

• Retail: The application of MR in the retail industry aims to give customers a

more interactive and tailored buying experience. It may be used to design
virtual apparel try-on experiences or virtual furniture showrooms, for
instance.

• Military: Real-time situational awareness and training simulations are

provided to soldiers using MR in the military. It makes training more
realistic by enabling interaction with virtual surroundings and objects.
Difference between Ar, VR & MR
 Advantages of Augmented Reality

• Augmented reality is an interactive experience

in which a real world environment is enhanced
with computer-generated visual elements,
sounds, and other stimuli.
• It can provide a user with a heightened, more
immersive experience than they would
experience otherwise that adds to the user's
enjoyment or understanding. From a
commercial perspective, augmented reality
can increase brand awareness and boost sales.
 Benefits of AR
1. Enhanced User Experience
• AR can provide interactive and immersive experiences, whether in gaming,
education, retail, or entertainment.
• For ex. In education, AR can bring textbooks to life, offering students a
dynamic and interactive learning experience.

2. Improved Product Visualization and Design

• It allows designers and engineers to create virtual prototypes and visualize
products in real-world contexts before physical production. This helps
identify design flaws, streamline the manufacturing process, and reduce
costs.
3. Enhanced Training and Skill Development
• AR can provide realistic and interactive training simulations. For
example, in healthcare, AR can assist medical professionals in
practicing complex procedures or allow students to explore human
anatomy in a three-dimensional space
4. Increased Efficiency in Maintenance and Repair
• By wearing AR-enabled devices, technicians can access step-by-
step instructions, visual cues, and virtual guides to carry out
complex repair tasks.
 Features of AR

The three main features of augmented reality are:

• Combining digital and real worlds
• Providing a medium for real-time interactions between these
two worlds
• Accurate 3D identification and overlay of virtual and real-
world objects

• Unlike Virtual Reality (VR) which creates an immersive virtual

world that detaches the user from the real world, AR allows
the user to control both worlds by combining them. AR
combines the two worlds by augmenting one another, that is,
the virtual world onto the real world.
• A camera-equipped device with AR software installed in it is
required. For instance, the AR app can use the scanners in
the camera to identify a specific object in the real world so
that it can augment it. Through computer vision and machine
learning algorithms, achieving object recognition in the real
world is possible.

• AR app creates a 3D object that can augment the detected

object in the real world through computer graphics. Once the
3D object is created, the position and orientation of the real-
world object are finalized. Lastly, machine learning
algorithms overlay the 3D object on the real-world object to
create a virtual environment for the user.
 Examples of App
• Snapchat: Snapchat has been making use of augmented reality for its selfie
filters. In 2020, they introduced an advancement in their software that could
help it detect individual landmarks worldwide and augment them.

• Google ARCore App: Google ARCore application has taken the idea of 3D
object overlay to the next level through their Depth API which has unlocked
capabilities like occlusion, realistic physics, and particle effects for its users.

• Pokémon GO: Pokémon GO is one of the apps that broke the internet when it
first came out in 2016, Pokémon GO is an AR-based app that lets users catch
Pokémon from their immediate surroundings, creating an exciting social
experience.
• Google Glass: Google Glass technology lets its users experience all
smartphone features without having to hold them in their hands. Through
voice commands and simple gestures, you can access your phone virtually.

• Google Streetview: In 2007, Google Maps introduced Streetview in a few

cities in the US, offering 360-degree pictures of the streets. Today, the
advanced version of Google Streetview allows users access to enhanced
navigation by superimposing virtual information over real-life objects

• Interior Design Apps : Myty AR is one of the top AR-powered interior design
applications, which lets you have a realistic perspective of how a furniture
item will look in your house. Such an experience increases user satisfaction
and increases customers
 Methods of AR
1. Marker-based AR
• This type of AR, also known as recognition-based AR or image
recognition, as it requires a special visual object and a camera to
scan it.
• Marker-based AR requires a marker to activate an augmentation.
Markers are distinct patterns that cameras can easily recognize
and process, and are visually independent of the environment
around them; they can be paper-based or physical objects that
exist in the real world.
• It may be anything, from a printed QR code to special signs.
• The AR device also calculates the position and orientation of a
marker to position the content, in some cases. Thus, a marker
initiates digital animations for users to view, and so images in a
magazine may turn into 3D models.
2. Markerless AR
• Marker-less AR offers the most control to the user as it allows
the user to choose where they would like to place the content.
It also allows real-life scale placement of the virtual augmented
objects. These AR experiences are highly reliant on smart
phone features such as sensors, camera, and processors.
• Marker-less AR can be split into:
• 3. Projection-based AR
• Projection-based AR relies on projectors to display 3D imagery or
digital content onto a flat two-dimensional surface, like a wall, floor,
or object. It doesn’t create fully immersive environments, mainly
holograms for events and movies.

• It is a method of delivering digital information within a stationary

context. It focuses on rendering virtual objects within or on a user’s
physical space. It is one of the simplest forms of AR where light is
projected onto a surface. The interaction occurs by touching the
projected surface physicall
• location-based or position-based augmented reality, that
utilizes a GPS, a compass, a gyroscope, and an accelerometer
to provide data based on the user's location.
• This data then determines what AR content you find or get in a
certain area. With the availability of smartphones this type of
AR typically produces maps and directions, nearby business
info.
• Applications include events and information, business ads pop-
ups, navigation support
4. Location-based AR
Location-based AR is a type of markerless AR that relies on
geographic data to deliver digital images at specific
locations.

GPS or location-based AR solutions respond to the sensors

of your mobile device. It allows for objects to be placed in
a certain location and as long as the public has the app,
they can retrieve it regardless of the time of day or
weather conditions.
• The reliability of marker-less AR on positional
information gathered from a device’s camera, GPS,
digital compass, and accelerometer makes it more
adaptable than marker-based AR as it does not need an
image or object cue to deploy.

Ex: 1 It’s a popular type of AR for gaming––Pokémon Go

2. Google app helping you navigate the city
 4. Superimposition-based AR
• Replaces the original view with an augmented, fully or
partially. Object recognition plays a key role, without it
the whole concept is simply impossible.

• This type of AR has been popularized by social platforms

such as Instagram, Facebook, and Instagram using
filters.

• The example of superimposed augmented reality in the

IKEA Catalog app, that allows users to place virtual items
of their furniture catalog in their rooms.
 CHALLENGES TO IMPLEMENTING AR

1. IMPLEMENTATION COST:
The most common concern was the perceived cost of implementing the new
technology, Of course, there are multiple ways of implementing AR, all with
different price points.
But if a company needs a custom app built from the ground up with unique
components, the price tag could run well into the six-figure range.
Ex- furniture retailer IKEA App

2. TECHNOLOGY AND SKILLS GAPS

“ talent and technological gaps” These gaps, and the fear of them, are
common with any new technology. need more guidance on training, use-
cases, implementation and maintenance.
3. RESOURCE SCARCITY AND COMPETING PRIORITIES
Executives struggled to find ways to devote already scarce financial
and human resources to the implementation of AR. It takes time to
train and/or recruit workers to operate complex systems,

4. CYBERSECURITY AND REGULATORY CONCERNS

policymakers and AR developers have not done enough to

address data privacy, copyright and liability concerns. But while
ethical challenges will emerge, businesses and governments will
need to cooperate to address them.
5. Technical Limitations
AR technology still faces some technical limitations that need
to be overcome. These include limitations in tracking
accuracy, device battery life, field of view, and visual quality.
Advancements in hardware, such as improved sensors, more
powerful processors, and longer battery life, are necessary to
enhance the overall AR experience. Overcoming these
technical challenges will enable more seamless and
immersive AR applications
6. Content Creation and Accessibility

Developing high-quality AR content requires specialized skills and

resources. Creating interactive and engaging AR experiences demands

expertise in 3D modeling, programming, and content design. The

accessibility of AR content also needs to be addressed, ensuring that it

reaches a wider audience with diverse devices and platforms.

Simplifying the content creation process and expanding accessibility

will contribute to the broader adoption of AR technology.

Unit 2: VR system
 How Does Virtual Reality Work?

• Virtual Environment Creation: VR begins with developing a digital

3D environment through computer-generated graphics or real-
world 360-degree videos. This environment is designed to look and
feel realistic, replicating various aspects of the physical world.
• Head-Mounted Display (HMD): Users wear a head-mounted display
(HMD), also known as VR goggles or a VR headset. The HMD
contains two screens, one for each eye, and provides a stereoscopic
view of the virtual environment, giving users a sense of depth and
immersion.
• Motion Tracking: VR systems use motion-tracking sensors, such as
gyroscopes and accelerometers, to detect the user’s head
movements. This allows the VR environment to respond in real-
time, adjusting the view based on the user’s head orientation and
providing a seamless experience as they look around.
• Interaction Devices: To navigate their virtual world, users may
employ hand controllers, gloves, or other
Interaction Devices: To navigate their virtual world, users may employ hand
controllers, gloves, or other input devices that allow for interaction. Such
devices allow them to perform actions within this environment, such as
grabbing objects or pressing buttons.
Rendering and Display: The virtual reality system’s software processes 3D data
of its virtual environment and renders images tailored for each eye, which are
then shown on screens inside an HMD, creating the illusion of three-
dimensional space.
Audio Simulation: To further enhance the immersion, VR systems often
incorporate spatial audio technology. This means that sound sources in the
virtual environment change based on the user’s head movements, mimicking
real-world sound behavior.
User Interaction: As users move their heads or use input devices, the VR
system continually updates the displayed images and audio to match their
actions, maintaining a seamless and responsive experience.