Received 17 October 2023; revised 21 December 2023; accepted 3 January 2024.

Date of publication 28 February 2024;

date of current version 14 March 2024. The review of this paper was arranged by Associate Editor Arokia Nathan.
Digital Object Identifier 10.1109/OJID.2024.3370888

Display and Optics Architecture for Meta’s

AR/VR Development
LINGHUI RAO1 , NAAMAH ARGAMAN 2 , JIM ZHUANG 1 , AJIT NINAN2 , CHEONHONG KIM2 ,
DAOZHI WANG2 , AND SHIZHE SHEN2
(Invited Paper)
1
Meta, Redmond, WA USA
2
Meta, Sunnyvale, CA USA

ABSTRACT We believe in the future of connection in the metaverse. The addition of immersive technologies
such as augmented reality (AR) and virtual reality (VR) is the next step in this progression. In this paper, we
will provide an overview of Meta’s progress in display and optics development for AR and VR. The overview
will cover different architectures that may lead to wide adoption of AR and VR devices in the near future as
well as solutions that will mature over time. We will also discuss different components maturity and utilizing
human perception for building systems made for human vision.

INDEX TERMS Augmented reality, display, optics, virtual reality.

I. INTRODUCTION backplane technology while they cost significantly more than

The metaverse is the next evolution in social connection and
the successor to the mobile internet, which helps people con-
nect and get closer together [1]. Meta is moving forward to
the metaverse through extensive research and development of
innovative concepts. The addition of immersive technologies
such as augmented reality (AR) and virtual reality (VR) is the
next step in this progression.
To support the most optimized visual experience for Meta
AR/VR applications, we have designed the architecture of
Meta’s Display and Optics system and developed the visual
performance evaluation matrixes. In this paper, we will intro-
duce the design concepts, challenges, and discuss the human
perception study for maximized visual performance.
II. INFINITE DISPLAY SYSTEM FOR VR ARCHITECTURE
Meta’s Infinite Display features high resolution fast-switch
display panels with an advanced lens system. Various dis-
play technologies, including LCDs [2], OLEDs (organic light
emitting diodes) [3], and micro displays [4], such as µOLEDs
(micro-OLEDs) and µLED (micro light emitting diodes) dis-
plays, have been introduced in the VR systems. The OLEDs
on glass or flexible substrate give wide color gamut and high
contrast, but its pixel density is very limited. The micro dis-
plays provide the highest pixel density thanks to silicon wafer
the other display types. Meanwhile, the LCD can achieve
fairly high pixel density at affordable price. From Oculus Rift
to Oculus Go, Quest, Rift S, Quest 2 and Quest Pro, Meta
has advanced in VR display and optical architecture design
and manufacturing to enable the immersive user experience.
In this section, we are going to focus on the review on the
most recent Quest 2 and Quest Pro products.
For both Quest 2 and Quest Pro products, the LCD tech-
nology has been selected considering the mass production
maturity, affordable price, and possibility of pixel density
increase. The innovative pancake lens works by folding the
light inside the optical system, and reduces the optical stack
thickness by 40%, compared with Quest 2. Fig. 1 below shows
the unique display and optics system designed to be harmo-
nized with the product design and to bring a hyper-realistic
visual experience for Quest Pro. Overall, we were able to
increase the system resolution for Meta Quest Pro (22 Pixels
Per Degree) by 10% compared to Meta Quest 2 (20 Pixels
Per Degree). Plus, Quest Pro achieved 25% of the full-field
visual sharpness improvement in the center view, 50% of the
peripheral region improvement, and larger color gamut than
Quest 2. The Infinite Display system also allows consumers
to adjust the lens distance from eyes with a new eye relief dial
to optimize fit, face tracking and viewing experience.

© 2024 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see
VOLUME 1, 2024 https://creativecommons.org/licenses/by-nc-nd/4.0/ 71
RAO ET AL.: DISPLAY AND OPTICS ARCHITECTURE FOR META’S AR/VR DEVELOPMENT
FIGURE 1. Infinite display system architecture for quest pro.
TABLE 1 Specifications of Meta Quest Pro and Quest 2 Displays
A. VR DISPLAY DESIGN
To lead the VR innovations for fitness, gaming and work,
we introduced high resolution, high efficiency and fast-switch
panel designs for Meta VR products. Meta Quest 2 has a
single Fast-Switch LC display panel design for both eyes
[5]. Meta Quest Pro is equipped with two individual fast-
switch liquid crystal display panels. The LCD panels are
1800 × 1920 pixels per eye. For Quest pro, the display res-
olution density is 1058 PPI, which is 37% higher compared to
Quest 2. In addition, the local dimming backlight technology
was introduced to Quest Pro to enhance the display color
gamut, contrast and improve power consumption [6].
In order to suppress visual artifacts, such as screen door
effects, mura, motion blur, ghosts and trailing effects, and to
enhance power efficiency, fast-switch (FS) LCDs have been
developed with below considerations.
1) HIGH EFFICIENCY AND HIGH RESOLUTION
FAST-SWITCHING PIXEL DESIGN
The Meta VR display team has driven the display technology
to over 1000 PPI to reduce artifacts, such as screen door
effect (SDE), and improve text readability. SDE describes a
mesh-like appearance on a screen or projected image [7]. SDE
could be observed along the gate and data lines, which is
related to the display fill factor design. For the Infinite Display
72
FIGURE 2. Typical timing diagram of the quest 2 LCDs with the split
backlight.
system, special display panel and pixel design were imple-
mented to improve the aperture ratio uniformity. With the
increased resolution, development in panel process optimiza-
tion is also implemented to ensure robust manufacturing
operation.
A fast response display is needed to improve the system
latency, which impacts the user experience on VR devices
[8]. The LC response time was engineered with LC material
tuning, cell design/process control, and also panel driving
optimization.
We customized DDIC (display driver integrated circuit)
design to minimize the horizontal line time and the optimum
value has been chosen considering panel loading and MIPI
(mobile industry processor interface) data rate.
Power consumption is important for battery life and sys-
tem thermal management. Higher PPI display intrinsically
has more challenges for power efficiency. Meanwhile, com-
pared to the Fresnel lens system, the demand of Quest Pro
display brightness was ∼4–5 times higher due to the pancake
lens system. We have taken all the system requirements into
consideration in architecting the backlight design, panel pixel
design, and overall optical stack innovation to maximize the
optical transmittance improvement.
2) BACKLIGHT DESIGN OPTIMIZATION
Low persistence illumination: Motion blur is a significantly
noticeable spatiotemporal artifact which occurs when an im-
age is presented with a finite display frame rate and eyes
move across the image. It also reduces visual acuity due to
the blurred image. This artifact becomes even more apparent,
and users will notice it all over the place in VR applications
because the display images are world-locked instead of being
head-locked. In order to minimize the motion blur artifact,
low-persistence illumination has been adopted by controlling
the duty ratio in the backlight timing. Fig. 2 below shows
the typical timing diagram of the Quest 2 FS LCD with the
split backlight. The left LED bars are illuminated after the LC
completely settles down in the left half active area. Meanwhile
the right LED bars may extend into the next frame, but the il-
lumination should be finished before the right half active area
scan gets started. The illumination timing has been optimized
VOLUME 1, 2024
FIGURE 3. Comparison of fresnel lens in rift CV1 and quest 2.
for each frame rate to get the best visual performance, such as
no ghosts and minimized stereoscopic disparity.
Local Dimming Innovation: To provide the consumers with
better contrast and visual experience, Quest Pro adopts lo-
cal dimming technology. The backlight consists of over 500
independently controllable LED zones. The local dimming
technology applied to Quest Pro is carefully designed to cope
with various challenges. For example, with the VR optical
stack, the zoning planning, and the field of view design, cer-
tain artifacts could be more noticeable compared to traditional
consumer devices. In addition, for better battery life, the Quest
Pro local dimming needs to be highly efficient in computing
resources and power consumption. We have deliberately opti-
mized the algorithm with advanced techniques in the graphics
pipeline, such as image statistics analysis, backlight level ad-
justment, and post-spatial and temporal filtering. Thus, we
have minimized the power consumption, while maintained
visual benefits and suppressed unwanted artifacts, such as
halo.
3) VR DISPLAY DESIGN CHALLENGES
There are other display module design challenges needed to
be addressed for an optimized optical system performance.
For example, special panel design was introduced to reduce
stray light in the optical system, and display optical profile
distribution output was finely tuned to match the lens design.
Beyond the optical visual design, it was also important to
have architectural consideration on display module weight,
mechanical dimension, system latency, and power consump-
tion optimization.
B. VR OPTICS DESIGN
The 2nd generation of Fresnel lens from Meta can be found
in the products such as Oculus Go, Rift S, Quest, and most
recently Quest 2, the most popular standalone VR headset
on the market. As Fig. 3 shows, the Quest 2 Fresnel lens is
a bi-convex design which provides additional design degrees
of freedom to further optimize the optical performance, such
as sharpness and pupil swim, compared with a plano-convex
optics.
The design and manufacturing of Fresnel features including
the pitch, draft surfaces and peak/valley tip radius, shown in
Fig. 4, are critical for the god-ray / glare performance of the
lens when users view the contents in a VR environment with
VOLUME 1, 2024
FIGURE 4. Cross section of fresnel lens.
FIGURE 5. AR/VR display image quality axes.
a 6DOF headset. All these aspects are carefully co-optimized
and the associated manufacturing processes are further ad-
vanced for the Meta’s 2nd generation Fresnel lens.
The pancake lens in Quest Pro represents another leapfrog
advancement, compared with Quest 2 Fresnel lens. The pan-
cake lens is a polarized catadioptric optical system in which
the refraction and reflection of the light are combined to
fold the light from display multiple times through the optical
system before it reaches the user’s eyes. As a result, a VR
optical system that is compact, lightweight, high resolution,
wide-FOV is achieved . A polarized catadioptric optical sys-
tem was first proposed by LaRussa [9], and more recently by
Lacroix [10] and Huxford [11] for head-mounted display ap-
plications. Meta has collaborated with the industry to advance
the technologies of optics, material science, high-precision
and high-volume manufacturing, which makes it possible to
design and fabricate such complex optical systems at scale and
cost effectiveness for applications in consumer electronics.
Quest Pro pancake lens features a two-element lens design.
Both elements are of the plano-convex surfaces, as shown
in Fig. 1. This optical architecture offers additional design
degrees of freedom than the Fresnel single lens and allows the
optimization for better visual performance that is important to
the users, such as the field of view, and full-field sharpness.
The field of view of the Quest Pro display optical system
is designed at 15 mm eye relief (defined as the distance from
the eye pupil to the apex of the first lens element), to accom-
modate users wearing the spectacle lens. To achieve the same
field of view, the longer the eye relief, the more challenging
73
RAO ET AL.: DISPLAY AND OPTICS ARCHITECTURE FOR META’S AR/VR DEVELOPMENT
from the design standpoint to minimize the off-axis aberra-
tion, such as color, distortion, and astigmatism, etc.
The polarization nature of the pancake lens commands the
precise control of the polarization state of the light propa-
gating from display through the optical stack, over a large
range of field angles and broad RGB spectrum of the display.
The film stacks are uniquely designed and integrated with the
optics to minimize some of the intrinsic artifacts of polarized
catadioptric optics systems, such as ghosting, and achieve
better contrast and color performance.
Quest Pro pancake lenses are made of optical thermoplas-
tic resins through injection molding process. This process is
important for cost and scale, and is equally important for mini-
mizing the optical aberrations to deliver desired a comfortable
visual experience to users. Some of the optical surfaces in
pancake lenses are reflective (polarization-state dependent),
and it is well-known that the lens optical performance is
more prone to the imperfect quality of the reflective surfaces
than the refractive surfaces in non-pancake lens systems. In
the high-volume manufacturing processes, Quest Pro pancake
lens quality is tightly controlled at each step, including surface
form error, its 1st and 2nd derivatives, and surface roughness,
etc.
Another unique requirement of the pancake lens is to
achieve the ultra-low birefringence (on the order of single
digit nanometers) in these thermoplastic lenses. Similar to the
engineering of polarization film stacks as aforementioned, the
low-birefringence lens is essential to the desired contrast and
ghosting performance.
III. AR DISPLAY AND OPTICS DEVELOPMENT
A. INTRODUCTION TO AR OPTICAL SYSTEMS
It is not accidental that VR and AR systems are often referred
to as one technology developed together and used together.
Both are designed as Head Mounted Displays (HMDs) and
as such are targeting a small form factor, to be lightweight
and as comfortable as possible while maintaining as long a
battery life as possible. But although the two do share many
similarities in many areas, they differ in the optical path,
which relates to many differences in rendering, system design,
and integration.
The use cases for VR and AR applications merge in a
pass-through VR, where world-facing cameras capture the
real world around the user and overlay digital content on top
of the pass-through video, delivered by a display positioned
in front of the user’s eyes. While the benefits of this approach
are significant from the user’s perspective, one can expect
limited adoption of such devices for activities that require
social interaction or appearance in public locations.
An AR system is designed to overlay digital content on top
of the real world while keeping the see-through qualities of
the system. This mission is quite challenging to achieve in
a battery-driven system, especially while competing with the
optical flux coming in from the world around us on a brightly
lit day. This section will discuss the options in front of an AR
74
designer who wishes to build a system that would be useful
and valuable to consumers.
When considering the adoption of both VR and AR devices,
the user experience is significant but not the only goal to
consider. The comfort, both physical and visual, of the system
is a significant part of the design goal, in addition to the size,
weight, power consumption, cost (Swap-C), and performance.
One driver for comfort is a significant weight reduction
compared to systems available today in the market. Spec-
tacles worn today for vision correction weigh between 15
and 65 g and are considered an acceptable weight. Adding
visual, audio, and AI features to the same form factor leads to
constant trade-offs in weight acceptability and a constant push
for smaller, lighter, and more efficient components across the
system.
An AR Display system consists of two main components:
The Display Engine or Light Engine, which converts bits into
photons projected at predefined angles at predefined locations
toward an optical component that merges this light with light
arriving at the eye from the world around the user. This sec-
ond component could be a mirror or a waveguide (diffractive
or refractive) but must allow light from the world to reach
the user’s eyes. Both components will be discussed in detail
below.
Due to the nature of the projection system, the image is
projected at infinity, making it uncomfortable to interact with
when objects in the real world are close to the user. Over time,
eye strain from the focus point changes could cause visual
fatigue or discomfort. For that reason, AR systems often use
push-pull lens systems that bring the projection plane closer
to the user, usually anywhere between 30 cm and 2 m. These
lenses could be diamond-turned, molded, or even 3D printed.
Meta’s recent acquisition of Luxexcel, an optical 3D company,
allows us to utilize this no-waste, fast, durable, and flexible
platform for the making of such lenses and other optical
components at an optical grade level [11]. 3D printing optical
components allows us to utilize complex geometries, increase
our integration flexibility, reduce the size and the weight of
components made with other methods.
B. AR DISPLAY ENGINES
Three main approaches are considered for the Display Engine
for Immersive Displays that are bright enough to support
both indoor and outdoor usage: Laser Beam Scanning (LBS),
Liquid Crystal on Silicon (LCoS), and micro-LEDs (uLED).
With the recent improvement in Organic LED (OLED) per-
formance, it is used in AR devices but mostly for low field of
view systems where the pupil efficiency is higher and hence
is kept out of this discussion on Immersive Display options
[13] DLP technology, although mature and very bright, is
power-hungry and poses challenges in size and wearability.
Future development in the field might prove valuable to the
world of AR, though this paper will not review it in detail.
Laser Beam Scanning (LBS) is based on three separate laser
panels projecting globally onto a set of 1D or 2D MEMS-
based mirrors that scan across the scene pixel by pixel to
VOLUME 1, 2024
generate the image. LBS offers the potential of a very small
light engine at high brightness and contrast. It enables high
resolution and reduced latency, key factors for AR immersion.
However, challenges include power consumption constraints,
complexity in packaging, and production cost considerations,
which can hinder widespread adoption. Additionally, safety
concerns related to laser radiation and eye strain need care-
ful mitigation. Historically, both Intel Vaunt that was never
launched and North Focals, launched in 2018, chose LBS as
their Display Engine of choice, but the FoV was small at 15°
[14] and was considered a limiting factor. Microsoft chose
LBS for its more immersive Hololens2, launched in 2019,
which has a FoV of 43° by 29°, demonstrating that with two
sources per color [14] and 2D MEMS scanning, a wide FoV is
possible with LBS. To date, no other major player in the AR
space launched an immersive and mass-produced commercial
AR system using LBS.
uLED Light Engines, which are just emerging in commer-
cial small FoV devices, give the Optical Display designer a
glimpse into its future potential. Unlike LBS, the uLED panels
generate the whole image spatially, eliminating the require-
ment for a high scan rate. The image is collected through a
collimating optical component to generate an image onto the
optical combiner. The resolution is set by the number of pixels
in the panel and the system field of view, and the limiting
factor is the size of the pixels themselves. The technology
provides exceptional contrast and color accuracy, significantly
enhancing the visual fidelity of AR content. Its miniature size
allows for compact device designs, increasing portability and
wearability. Yet, to this day, the processing of very small pixel
sizes at the brightness required for an AR Immersive Display
remains a challenge. Furthermore, the field of AR optics is
characterized by two distinct color generation methodologies.
The first approach involves the segmentation of red photon
generation from blue and green, necessitating the use of 2 or 3
panels. However, the challenge lies in the intricate merging of
the light emitted by these panels, resulting in a display system
that is notably large, limiting wearability. Conversely, the sec-
ond approach, RGB pixelation, while promising, remains in
its developmental stages and is not yet ready for commercial
application. This method encompasses various technologies,
such as localized epitaxial growth and the conversion of blue
or UV light to green and red via quantum dots, requiring
further refinement before practical implementation.
LCoS technology is the most mature of the three display op-
tions. LCoS light engine utilizes liquid crystal panels placed
on top of silicon backplanes, reflecting flood or global illu-
mination coming in from LEDs. The liquid crystals modulate
the light’s intensity based on the displayed image on a pixel-
by-pixel basis, and the reflected light is then directed into
the user’s eye through optics. Due to their reflective nature,
LCoS engines sometimes suffer from low contrast ratios, and
complex integration and alignment between the different com-
ponents. In addition, the liquid crystals modulate just one
polarization, thus reducing the overall efficiency of the sys-
tem. If solved for, LCoS light engines provide a good balance
VOLUME 1, 2024
between manufacturability and cost, size, performance, and
power consumption. Among the immersive systems already
shipped in volume are the Hololens 1, Magic Leap 1 and its
successor Magic Leap 2 and numerous smaller FoV systems.
C. AR COMBINERS
Historically, AR displays have shown clear trade-offs between
wearability, size, power consumption, see-through qualities
(both inside-out and outside-in), and the visual experience
of the user. Immersive Displays, even ones designed for
consumers or light-industrial applications like Microsoft’s
Hololens [15] or Magic Leap’s series [16] chose a very dif-
ferent design point compared with Google glass [17] or the
North Focals, aimed at a low field of view audience.
Our goal at Meta is to explore that gap and bring to mar-
ket the best design point products that brings our users and
the world closer together. In the following section, we will
discuss the options for AR combiners from birdbath, single to
multiple reflector combiners to pupil expansion geometric and
surface-relief combiners.
Like many other components in an AR system, birdbath
combiners for Immersive Displays also suffer from a trade-off
between form factor and efficiency. The smaller, lighter, and
more "normal looking" the glasses are, the higher the distor-
tion and lower the efficiency. Although such combiners can
be molded, making their cost at volume quite attractive, the
bulkiness and weight of a system designed for them make their
adoption quite limited.
On the other side of the combiner option space, one can find
Total-Internal-Reflection combiners (TIR). Total internal re-
flection occurs when light travels from a medium with a higher
refractive index to a medium with a lower refractive index,
and the angle of incidence exceeds a critical angle. When this
critical angle is exceeded, no refraction occurs, and all of the
light is reflected back into the higher refractive index medium.
This phenomenon is exploited in AR combiners when the
virtual images from the Display Engine are directed into the
high-refractive-index medium at specific angles, exceeding
the critical angle for TIR. As a result, the virtual images are
internally reflected at the boundary and become visible to the
user at the eyebox.
When the number of bounces is increasing, the combiner is
referred to as a waveguide.
The first in this review is a reflective waveguide using 100%
reflective pin-mirrors. In recent years, pin-mirror projection
systems were able to show a trend towards a wide field of view
at relatively high efficiency, low manufacturing cost, and com-
pelling size and weight [18]. In such systems, tiny mirrors are
embedded into the eyepiece to overlay digital content on top
of the real-world view. The dots are small enough and close
enough to the user’s eyes so that no occlusion occurs. Yet, the
expansion into a wide field of view is still challenging, and the
social acceptability of the outside-in view is still unknown.
Other Geometric Reflective Waveguides use partially re-
flective mirrors to control and direct the light from the Display
Engine [19]. The latest designs using this approach show very
75
RAO ET AL.: DISPLAY AND OPTICS ARCHITECTURE FOR META’S AR/VR DEVELOPMENT
good color and luminance uniformity at very high efficiency
and see-through transparency. The Field of View has also
increased systematically over the past several years, though
the trade-offs of FoV to thickness (and hence weight) and
efficiency still exist. Glares, ghosts, and other visual com-
fort artifacts remain a challenge using Geometric Reflective
Waveguides.
Surface Relief Gratings (SRG) and Volume Bragg Grating
(VBG) Waveguides use a different methodology to control
and manipulate the light. In VBG, diffractive gratings are
inscribed within a waveguide substrate using a photosensitive
holographic material with a refractive index slightly different
than the medium and are designed to diffract specific wave-
lengths of light while transmitting others. In contrast, SRG
waveguides employ nanostructures imprinted or etched onto
the surface of a transparent substrate [20]. Leveraging TIR and
by controlling the spacing and orientation of these gratings,
SRG waveguides can precisely steer and overlay a digital
image from the Display Engine to the user’s eyes. Both VBG
and SRG waveguides offer very thin and small form factors
compared with any other combiner technology. Since the re-
fractive index and surface materials of SRG are higher than
VBG, SRG also offers higher field of view opportunities for
the design of Immersive Displays. Yet, due to the diffractive
nature of these technologies, they show color non-uniformity
that reduces the visual comfort of the user.
D. FUTURE-LOOKING AR SYSTEMS
When looking forward to the adoption of Immersive Displays,
the road is still long ahead of us to enable price-sensitive, low
weight and size, highly reliable, and low-power Immersive
Displays that would be appealing to the mass market. In ad-
dition, some of the issues identified above remain a challenge
like the single plane of projection and mitigating Vergence
Accommodation Conflict and the requirement for a comfort-
able visual experience from both the user’s perspective and a
bystander.
IV. VISUAL PERFORMANCE
Display technology for AR/VR from a visual performance
point of view needs to be thought of differently than tra-
ditional display systems. The North Star for AR/VR is to
impedance match the display system to the human visual
system. The device needs to fit in a form factor that takes
into account human ergonomics which implies that if we are
wasteful with our system parameters such as power and light,
we have probably not achieved ideal form factor and ideal
ergonomics. On the other hand if we underdeliver on image
quality, we have compromised our imaging experience. This
equilibrium is what we target for the delivery of an optimal
visual experience. The goal is to achieve perceptual quality
for immersive experiences. We strive to deliver experiences
for humans not metrology.
The additional challenge AR/VR has above that of tradi-
tional displays is the fact that we have 2 display systems which
deliver an image to the left and right eye that must behave
76
seamlessly as one spatial display delivering an image to the
brain. The traditional vectors of visual performance apply to
these displays, but the complexity of the binocular aspects
working together compound the issue of visual performance
further. This also offers unique opportunities to optimize it as
well. To help guide us we use a 9 axes framework as described
below.
At the high level, we can think of the traditional 4 axes of
display image quality that drives AR/VR as: (1) Resolution,
(2) Dynamic range, (3) Color gamut, and (4) Frame rate. The
good thing about these four characteristics is that the industry
has had over fifty years to define and standardize them. With
small modifications, we are able to extend the existing tools
to quantify them and make them applicable to AR/VR. The
binocular head mountable image specifics are however new
and lack any standardization or quantifiable minimum viable
experience. The remaining AR/VR specific axes are: (5) Field
of view, (6) Stereo presentation, (7) Latency, (8) Degrees of
Freedom, and (9) Accommodation Vergence. These nine de-
fine not only image quality, but also visual comfort. At Meta,
we have focused perception and image quality teams who are
dedicated to this aspect of AR/VR display system designs.
A. RESOLUTION
Resolution for normal displays is quantified by horizontal
and vertical resolution. This does not adequately describe an
AR/VR system since it does not tell us anything about what
the brain will see. A better measure is pixel per degree (PPD)
since this will give us a better idea as to how it appears to the
human visual system. The human visual system is capable of
resolving 120 PPD or higher for some specific content while
60 PPD is good enough for most content. The Meta Quest
Pro has 22 PPD at the center of FOV, which provides 10%
improvement over Meta Quest 2 at 20 PPD [21].
The PPD may not be enough of a measure for resolution
since the theoretical number is simply the display total resolu-
tion divided by the field of view. This theoretical number will
be degraded by the lens system. The MTF of the lens will drop
the resolution further. This along with rendering algorithms
such as antialiasing would lower this to what we refer to as an
effective PPD. This MTF sharpness has a spatial component
to it which is inevitable for AR/VR devices to have different
degrees of sharpness from the center to the edge area. This
is due to the optical aberrations of lenses or imperfect optical
quality from the manufacturing processes. From this stand-
point, the goal of a well-designed and well-manufactured lens
is to achieve a high MTF at both the center and periphery area,
and to provide the edge-to-edge full field visual clarity to the
users. The Meta Quest Pro delivers a center sharpness of 0.98
and an edge sharpness of 0.85 (measured at 5 lp/mm), which is
25% and 50% improvement over Meta Quest 2, respectively.
To make matters worse, the persistence time has a dynamic
effect on PPD which changes the perceptual PPD. All these
must be evaluated in order to quantify what the real resolution
quality is when setting a target.
VOLUME 1, 2024
B. DYNAMIC RANGE
Most of us are familiar with contrast ratio. However, there
is also another aspect of control over the dynamic range to
render smooth gradients. The peak brightness with the simul-
taneous contrast and the sequential contrast defines the display
system contrast ratio. In AR systems the additive contrast
ratio is usually more important to characterize [22]. This takes
into account the light coming from the world mixing with
the display all contribute to the visual performance of the
changing dynamic ranger. The content must map to play-
back effectively in a way that makes content visible while
preserving the intent of the image. On the other hand, VR
Infinite display features local dimming with 500 zones. This
enables 75% higher contrast compared to Quest 2 and allows
for finer control over the rendering and results in deeper blacks
and an enhanced viewing experience. Additionally, the optical
system has polarization film stacks on the display to minimize
ghost images, especially from the mid-periphery to the edge
of the FOV. This contributes to its overall improved contrast.
C. COLOR GAMUT
Color plays a key role in contributing to the consumers’ visual
experience. The quality of a VR headset’s color is measured
by color gamut and color accuracy. Color gamut is the range of
colors within a spectrum that can be reproduced by the overall
system or a metameric equivalent. Color accuracy refers to
the display system’s ability to reproduce colors and shades as
intended. Meta Quest Pro has a wide DCI-P3 color gamut,
which enables vivid color. The calibration process to deliver
accurate color is a critical aspect of image quality. In AR,
this is exceptionally challenging due to the additive nature
of the displays. Color becomes critical to match content in
the real world for immersive experience. The resulting gamut
combined with the dynamic range defines how the content
needs to be played back.
D. FRAME RATE
Frame rate plays into the brain’s ability to fuse images into
motion and can play an important role in comfort. Display-
ing of the image due to head motion and eye motion must
be rendered seamlessly for smooth pursuit and for vestibular
ocular reflex of the eye. These are dependent on the display’s
ability to render new frames quickly. Currently, the lower
bound for comfort is at 90fps for world locked content. The
ability to lock content accurately to the world for mixed reality
is limited by this capability. Combined with the requirement
for higher frame rate, there is a need for low persistence
displays. Low persistence allows for sharper rendering and
reduced retinal blur, which contributes to key image quality
characteristics [23].
E. FIELD OF VIEW
Field of view determines how immersive the experience will
be. Different applications will require different fields of view.
Field of view can be cut with two edges. While delivering a
VOLUME 1, 2024
more immersive experience, it also requires other axes to be
tighter since visual comfort can be compromised if charac-
teristics such as latency and frame rates are not maintained
correspondingly.
F. STEREO PRESENTATION
The key to delivering a comfortable experience is delivering a
correct stereo image that is right for the user. This requires the
display to take into account how the eyes are spaced and where
your eyes are. Unlike 3D TV’s that just deliver horizontal
disparity of some kind, AR/VR will require this to be done
right. This is especially true since extended use is expected
and the distances to the virtual objects need to mix with the
real world. The system will need to do correct rendering by
leveraging all the sensors. Lens designs will need to take into
account the pupil swim that can create errors. The brightness
[24] & color matching between the displays all contribute to
correct depth rendering. To add to this, the vertical disparity
of the images will result in diplopia if exceeding the limit of
binocular fusion. More exact rendering that includes inter oc-
ular parallax would add to that naturalness of the experience.
G. LATENCY
The latency of the display not only includes how long it takes
from receiving the first information of the image but includes
the roundtrip time from the motion for the head to when the
photon is registered in the brain. The larger the perceptual
latency the more uncomfortable this experience is. This is true
for world locked content and user motion but is not as critical
for content motion.
H. DEGREES OF FREEDOM (DOF)
The ability for the display to render 3 degrees of freedom or
6 degrees of freedom rests on the need for the application and
its ability to locate the displays in space. 6 DoF requires other
axes such as frame rate, latency, stereo rendering to be tighter
to accommodate for this level of freedom. While this is not a
display characteristic the capability to do this is dependent on
the display’s capabilities.
I. ACCOMMODATION VERGENCE
The need to interact with content for extended periods of time
where content is in close proximity requires us to be able to
verge with correct stereo image but also with corresponding
focus/accommodation. This disconnect in matching accom-
modation vergence has its effect on different parts of the
population in different ways. But when it works it is natural to
everyone and for most it becomes a critical feature for com-
fort. If the display system is capable of rendering hologram’s
or lightfields there are usually trade offs with other aspects
mentioned above.
For each of these axes quantified perceptual models are
built and JND (Just noticeable difference) or JOD (Just ob-
jectionable difference) thresholds established. These are then
partitioned into what would be an acceptable minimum viable
experience and what would be diminishing returns for that
77
RAO ET AL.: DISPLAY AND OPTICS ARCHITECTURE FOR META’S AR/VR DEVELOPMENT
axis. This can be further broken down into good, better, best
recommendations for the product. Since this is done per appli-
cation the final product combines these requirements together
to decide target specifications.
V. CONCLUSION
In this paper, we have reviewed the key design architecture
and user experience metrics to maximize Meta’s AR/VR user
experience development. We are looking forward to bringing
the next leap in display and optics technologies into reality for
our next generation products.
REFERENCES
[1] Meta. [Online]. Available: https://about.meta.com/
[2] T. Matsushima et al., “Optimal fast-response LCD for high-definition
virtual reality head mounted display,” SID Symp. Dig., vol. 49,
pp. 667–670, 2018.
[3] J. Cho, Y. Kim, S. Jung, H. Shin, and T. Kim, “Screen door effect
mitigation and its quantitative evaluation in VR display,” SID Symp.
Dig., vol. 48, pp. 1154–1156, 2017.
[4] G. Haas, “Microdisplays for augmented and virtual reality,” SID Symp.
Dig., pp. 506–509, 2018.
[5] C. Kim, A. Klement, E. Park, J. Han, L. Rao, and J. Zhuang,
“High-ppi fast-switch display development for oculus quest 2 VR
headsets,” SID Symp. Dig. Tech. Papers, vol. 53, pp. 40–43, 2022,
doi: 10.1002/sdtp.15410.
[6] L. Rao et al., “Infinite display for meta quest pro,” SID Symp. Dig. Tech.
Papers, vol. 54, pp. 32–35, Aug. 2023, doi: 10.1002/sdtp.16480.
[7] J. Nguyen, C. Smith, Z. Magoz, and J. Sears, “Screen door effect
reduction using mechanical shifting for virtual reality displays,” Proc.
SPIE, vol. 11310, pp. 200–210, 2020, doi: 10.1117/12.2544479.
[8] S. T. Murdison, C. McIntosh, J. Hillis, and K. J. MacKenzie, “Psy-
chophysical evaluation of persistence- and frequency-limited displays
for virtual and augmented reality,” SID Symp. Dig. Tech. Papers, vol. 50,
pp. 1–4, 2019, doi: 10.1002/sdtp.12840.
78
[9] J. A. LaRussa and A. T. Gill, “The holographic pancake window TM,”
Proc. SPIE, vol. 162, pp. 120–129, 1978, doi: 10.1117/12.956898.
[10] M. Lacroix, “Collimation device of small size,” French Patent 2 690
534, 1993.
[11] R. B. Huxford, “Wide FOV head mounted display using hybrid optics,”
Proc. SPIE, vol. 5249, pp. 230–237, 2004.
[12] G. Groet, “Luxexcel: 3D printing: Combining prescription power and
a waveguide into a lightweight lens,” Proc. SPIE, vol. 11764, 2021,
Art. no. 117640F.
[13] E. Hsiang, Z. Yang, Q. Yang, P. Lai, C. Lin, and S. T. Wu, “AR/VR
light engines: Perspectives and challenges,” Adv. Opt. Photon., vol. 14,
pp. 783–861, 2022.
[14] KGOnTech. [Online]. Available: https://kguttag.com/
[15] B. C. Kress and W. J. Cummings, “Optical architecture of HoloLens
mixed reality headset,” Proc. SPIE, vol. 10335, pp. 124–133,
2017.
[16] K. R. Curtis, “Unveiling magic leap 2’s advanced AR platform and
revolutionary optics,” Proc. SPIE, vol. 11932, 2022, Art. no. 119320P.
[17] B. Kress and T. Starner, “A review of head-mounted displays (HMD)
technologies and applications for consumer electronics,” Proc. SPIE,
vol. 8720, pp. 62–74, 2013.
[18] LetinAR. [Online]. Available: https://letinar.com/en/pintilt
[19] A. Frommer, “11-3: Invited paper: Lumus optical technology for AR,”
SID Symp. Dig. Tech. Papers, vol. 48, pp. 134–135, 2017.
[20] B. Kress, “Optical architectures for augmented-, virtual-, and mixed-
reality headsets,” 2020.
[21] Meta. [Online]. Available: https://www.meta.com/blog/quest/vr-
display-optics-pancake-lenses-ppd/
[22] D. R. Blanc-Goldhammer and K. J. MacKenzie, “The effects of nat-
ural scene statistics on text readability in additive displays,” Proc.
Hum. Factors Ergonom. Soc. Annu. Meeting, vol. 62, pp. 1281–1285,
2018.
[23] A. Goettker, K. J. MacKenzie, and T. S. Murdison, “Differences be-
tween oculomotor and perceptual artifacts for temporally limited head
mounted displays,” J. Soc. Inf. Display, vol. 28, pp. 509–519, Jun. 2020.
[24] T. Doi, L. Wilcox, and T. S. Murdison, “Stereopsis from interocular
temporal delay: Disentangling the effects of target versus background
luminance,” in Proc. Vis. Sci. Soc. Annu. Meeting, 2023, Art. no. 5159.
VOLUME 1, 2024