The 15th International Conference on
Computer Science & Education (ICCSE 2020)
August 18-20, 2020. Online
Application of virtual reality and augmented reality
technology in Teaching1
Liping Li2
School of Information
Yunnan University of Finance and Economics
Kunmin, Yunnan,China
llplrs@126.com
Abstract—“Internet Plus Education” is just unfolding with
the upgrading of educational philosophy and the innovation
of teaching methods. The application of virtual reality and
augmented reality technology in the field of education has
broad prospects. This paper introduces the classification of
virtual technology, puts forward the composition of teaching
system and human-computer interaction system based on
AR and AR technology, describes the key technologies of
teaching system based on VR and AR, and points out the
problems of VR and AR application in teaching.
Keywords—virtual reality, augmented reality, teaching
I. INTRODUCTION
With the development of computer technology, the
subject of computer simulation science and technology
has been formed. In 1965, Dr. Sutherland, an important
founder of computer graphics, described a new display
technology in an article, put forward the idea of
simulating of the real world from the perspective of
computer display and human-computer interaction. In
1973, Krueger M. proposed the word "artificial reality"
(VR), which is an early VR term. [1] For "what is real",
the movie "The Matrix" has a classic line: If you mean
what you can feel, what you can taste and what you can
see, then "reality" is just the electronic signals your brain
compiles. Virtual Reality (VR) or Augmented Reality
(AR) formally based on this principle, through scientific
and technological means to reconstruct or strengthen the
surrounding environment.
At present, the combination of VR, AR and education
is in the ascendant, and the market demand is strong,
Domestic and foreign institutions and organizations have
laid out the VR education market. China's VR started late,
its technology development is not so sufficient, it is in the
rising stage of development, there are not many products,
and its application is limited, but, if it is well combined
with the education industry, get through the technical
barriers, and expand the scale of promotion, the
development prospect is very considerable.
The innovation of this paper lies in the combination of
virtual reality technology and teaching, it introduces the
classification of virtual technology, puts forward the
composition of teaching system and human-computer
interaction system based on AR and AR technology,
describes the key technologies of teaching system based
on VR and AR, and points out the problems of VR and
AR application in teaching.
1 Fund Project: Science Research Foundation from Education Department of Yunnan Province, China
2 Speaker
3 Correspondence Author
978-1-7281-7267-5/20/$31.00 ©2020 IEEE
Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 05:33:44 UTC from IEEE Xplore. Restrictions apply.
ThC2.1
Xiaosong Wu3
Experimental Center
Yunnan University of Finance and Economics
Kunming,Yunnan,China
wxszwd@ynufe.edu.cn
II. CLASSIFICATION OF VIRTUAL REALITY TECHNOLOGY
A. Desktop Virtual Reality
Desktop virtual reality uses computer display as a
window for users to observe the virtual world. It uses
computers and low-level workstations for simulation. The
cost of this technology is low, but the user lacks the
experience of complete immersion, and the interference
from the environment is great.
B. Immersive Virtual Reality
It uses immersive devices, such as helmet mounted
display to close the user's sense of vision and hearing. It
provides an isolated virtual space, and makes use of
simulation, sensor and other equipment, enables users to
have a sense of immersiveness. There are helmet display
system, projection display system, etc.
C. Virtual Reality Of Augmented Reality Type
It not only uses virtual reality technology to simulate
and emulate the real world, but also enhances users'
feelings to the real world. In other words, it enables users
to perceive information that they cannot or are not easy to
know in the real world, such as the 3D holographic virtual
image of teachers in the classroom, and glasses which
help people with amblyopia to buy goods, etc.
Augmented reality(AR) technology overlays the virtual
information generated by computer into the real
environment, it seamlessly integrates the expanded
content into the real environment with the help of sensing
and display devices, integrates the virtual scene and the
real environment, and presents a new sensory
environment to users.
D. Distributed Virtual Reality
Distributed virtual reality technology connects multiple
users through computer network to join a virtual space.
Multiple users can observe, operate and interact with one
virtual world at the same time, and they can work together.
The distributed virtual teaching system is shown in Figure
1. The system can connect the simulation equipment in
different places and establish a virtual teaching
environment with high interaction. Teachers and students
in different places can enter virtual classroom, virtual
laboratory and virtual practice training field through
2019J0924
543
 ThC2.1

virtual teaching system. computers, cameras and optical equipment that can create
virtual teaching environment and respond to various
operations of teachers and students in real time. The
virtual virtual virtual
classroom Laboratory practice software are virtual teaching development platform which
training field is used to realize the required functions of teaching and a
virtual teaching resource editor.
The broadcast control platform manages and controls
LAN the generated content. Authentication management
verifies the access rights of users. Teaching management
manages and controls the teaching process, teaching steps,
teaching methods and teaching tools. User management
realizes the management and control to system end user
teacher teacher teacher student student student and system maintenance personnel.
1 2 3 1 2 3
terminal entrance
Figure 1 distributed virtual teaching system PC terminal mobile integrated
(VR helmet) terminal machine
(VR glasses)

III. TEACHING SYSTEM BASED ON VR AND AR content generation

A. The Constitution Of Virtual Teaching System hardware software

The teaching system based on VR and AR is called computer panoramic optical virtual teaching virtual
virtual teaching system here, it integrates camera equipment development teaching
platform resource
high-performance computer software and hardware and editor
various advanced sensors to create a comprehensive
information environment with immersion and interaction broadcast control platform
capabilities. VR teaching system is shown in Figure 2. It content authentication teaching user
shows that there are some overlapping areas between the management management management management
real world and the virtual teaching environment, this
overlapping area forms the AR teaching environment.
That is to add virtual information to the real teaching Figure 3 composition of virtual teaching system
environment to enhance teachers and students' feelings of B. The Construction Of Human-Computer Interaction
the real teaching environment.
System In Virtual Teaching System
In an ideal situation, the virtual teaching system is
real environment
composed of multi-sensor, which is convenient for
AR teaching
human-computer interaction. The system can generate
environment high-quality images, good stereo effect, and fine natural
sense of force. It has good real-time performance, short
VR teaching delay, and users are not easy to feel tired. The
environment human-computer interaction of virtual teaching system is
shown in Figure 4.
sensing devices According to the changes of internal model and
display feedback external environment, the virtual environment generator
teachers generates a realistic virtual teaching environment through
and calculation. Stereovision generation produces a kind of
vision based on user's own viewpoint. Stereo sound
synthesis produces stereo effect based on users
themselves, which takes the user as the original point.
Figure 2 virtual teaching system Language recognition is used to identify voice commands
and conversation contents of users. Speech synthesis
Virtual teaching system consists of terminal entrance, produces the speech of human natural language. Position
content generation system and broadcast control platform, tracking and motion capture determines the position and
as shown in Figure 3. VR terminal is a terminal product direction of the user's head, eyes, hands and body.
integrating hardware, platform, computing center, flow Somatosensory system provides feedback on gravity,
entrance, industry application and other functions, which pressure, friction, temperature, etc. Kinematic system
meets the requirements of rigid demand, pain point, high provides rules and laws of motion for people and objects.
frequency, scene and connection.
The content generation system of virtual teaching
system consists of hardware and software. Hardwires are

544

Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 05:33:44 UTC from IEEE Xplore. Restrictions apply.

virtual environment generator
stereovision generation
stereo sound synthesis
language recognition
speech synthesis
position tracking
motion capture
somatosensory system
kinematic system
helmet display, VR glasses, holographic
equipment, and other visual equipment
integrated machine
stereo equipment
multisensor groups
somatosensory equipment
force sensing device
Figure 4 human-computer interaction of virtual teaching
IV.KEY TECHNOLOGIES OF TEACHING SYSTEM BASED ON
VR AND AR
A. Vision Technology
When a person is observing an object, two eyes look at
the object from different positions, the left and right eyes
see images of different sides of the object due to different
angles, these two images are simultaneously imaged in
the retina, and the brain recognizes the distance of the
object according to these two different images, so as to
produce stereoscopic sense.[2] Naked eye 3D vision uses
optical technology to separate the visual images of the left
and right eyes, resulting in the parallax of the two eyes, so
as to produce the visual illusion effect of the stereoscopic
image, it can obtain realistic stereo images with space and
depth without any auxiliary equipment (head display,
glasses).Holographic projection technology uses the
interference and diffraction principle of light to record the
intensity and depth information of light, and reproduce
the real space information of the object, it enables the
observer to see the stereoscopic image of the object from
any visual angle. In the virtual teaching system, teachers
and students can get better learning experience and effect
if they don't need to wear any visual equipment and watch
the images and objects in a natural way.
B. Sound Technology
Virtual sound technology in VR teaching system is
used to simulate sound field in real space, it uses virtual
sound recording and synthesis technology (artificial head
recording, artificial synthetic virtual sound) and virtual
sound playback technology to simulate the transmission
of sound in three-dimensional space. For example, a
micro microphone is placed in the ear canal of the
artificial head, and the pulse response of the head to
multiple sound sources at different positions is recorded,
and then real-time convolution calculation is carried out
545
Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 05:33:44 UTC from IEEE Xplore. Restrictions apply.
ThC2.1
to generate a sense of position, so that people can feel the
sound coming from all directions. [2]
C. Tracking And Motion Capture Technology
The tracking and motion capture technology in VR
teaching system needs to acquire accurately the position,
direction and posture of the target object in
three-dimensional space in real time, as well as capture
the whole body movement of the human, and then
feedback the acquired information to VR system to
realize the interaction between human and the system. For
example, users can use eye movement or gesture changes
to control the device.
D. Sensing Technology
The sensor can sense the measured information,
transform the information into electrical signal or other
forms of information according to certain rules and output
information, it can meet the requirements of information
transmission, processing, storage, display, recording and
control. The sensing devices involved in VR include
wearable devices, such as helmet display, data gloves,
data clothes, etc., which are used to identify and track the
actions and sounds of the operator's head, hands and body.
There are also various kinds of sensing devices in the
environment, such as vision, hearing, touch and force
sense devices.
E. Somatosensory Technology
The somatosensory technology in VR teaching system
is to bring the user's body feeling from the real world to
the digital world, body sensation includes vision, hearing,
kinesthetic, tactile, even taste and smell. Kinesthetic
sense is the perception of force and moment, such as the
shape, weight and hardness of objects. Tactile sensation is
the feeling of vibration, temperature and tangential force,
such as the perception of material, cold and hot.[3] In
practical training, simulation experiment and other
teaching activities, somatosensory technology enables
teachers and students to really feel the physical
characteristics of the experimental object and improve the
sense of reality and immersion.
V. THE CONTENT GENERATION OF VIRTUAL TEACHING
SYSTEM
A. 3D Content Generated By Computer
In VR system, the geometric attributes of objects are
acquired mainly by optical method and stereo parallax
method. The optical method is to project light onto the
object and measure the depth information of the object
according to the order of the returned light. According to
the principle of triangulation, the stereo parallax method
uses two cameras located at different positions to take
pictures of the object, and uses the parallax of the
corresponding points to calculate the stereo information
of the object, so as to obtain the spatial coordinates of the
sampling points. With professional 3D modeling software
(such as 3D Max), it can make exquisite 3D models and

animations, 3D scanners can also be used to quickly
create high-precision 3D models, it can also use 3D
reconstruction software (such as Agisoft Photoscan
Professional) to generate 3D models of real coordinates
by using multiple photos of objects. [3]
In terms of motion capture, there are technologies
based on Electromechanical, electromagnetic and special
optical signs. The camera and motion capture device can
be used to obtain motion data and generate motion model.
B. 3D Content Generated by Panoramic Video Shooting
VR panoramic video uses panoramic shooting
(horizontal 360o and vertical 360o) and image splicing
technology to splice the video into spherical image to
build virtual space. VR video generation includes video
acquisition, stitching, coding and other processes, and it
also goes through the later synchronization, multi video
splicing and other work. Post production software, such
as PluralEyes, can be used for simultaneous processing of
sound and picture of multiple machines. Premiere is video
editing software based on non-linear editing equipment,
which can clip, splice and synthesize video materials.
After VR video is generated, teachers and students can
use VR devices (such as VR helmets, glasses, etc.) to get
virtual space experience.
VI.CONCLUSION
Virtual reality and augmented reality bring great
benefits to teaching. But at present, most of the researches
in China are still on the interaction between human
perception system, muscle system and computer, there is
little research on "how the sensory information obtained
by human in practice is stored and processed in human
brain to understand the objective world?" For the teaching
.
546
Authorized licensed use limited to: Auckland University of Technology. Downloaded on November 03,2020 at 05:33:44 UTC from IEEE Xplore. Restrictions apply.
ThC2.1
system, if the above problems are solved and the teaching
system is examined and designed from the perspective of
educational neuroscience, the system will become an
assistant to help people learn, think and create in the
multi-dimensional information space. In addition, VR and
AR can play a better role and value when they are used in
teaching, only when the human factors are fully
considered and the human-friendly design is carried out.
REFERENCES
[1] Tan Jiefu; Zhong Zheng;Yao Yongfang. Virtual Reality Foundation
and Actual Combat [M]. Beijing: Chemical Industry Press, 2018.
[2] Wang Chengwei;Gao Wen;Wang Xingren. Theory, Implementation
and Application of Virtual Reality Technology [M]. Beijing:
Tsinghua University Press, 1996.
[3] Lu Yun;Wang Haiquan;Sun Wei. Theory, technology, development
and application of virtual reality [M]. Beijing: Tsinghua University
Press, 2015
[4] Lou Yan. Overview of Virtual Reality and Augmented Reality
Technology [M]. Beijing: Tsinghua University Press, 2016.
[5] Guido Bozzelli; Antonio Raia; Stefano Ricciardi. An integrated
VR/AR framework for user-centric interactive experience of
cultural heritage: The ArkaeVision project[J]. Digital Applications
in Archaeology and Cultural Heritage,2019.
[6] Kyle Hooks; Wesley Ferguson; Pedro Morillo; Carolina Cruz-Neira;
Evaluating the user experience of omnidirectional VR walking
simulators[J]. Entertainment Computing,2020.
[7] Steven Lo; Abdul Shakor Sandel Abaker; Fabio Quondamatteo;
Use of a virtual 3D anterolateral thigh model in medical education:
Augmentation and not replacement of traditional teaching?[J].
Journal of Plastic, Reconstructive & Aesthetic Surgery,2020.
[8] Liu Xiangqun;Guo Xuefeng;Zhong Wei. VR / AR / MR
development practice: Based on unity and U4E engine [M]. Beijing:
Mechanical Industry Press, 2017.
[9] Zhang Shanli;Shi Fen. Introduction to virtual reality [M]. Beijing:
Tsinghua University Press, 2016.
[10] He Wei. United virtual reality development Canon [M]. Beijing:
China Railway Press, 2016.
[11] Zhang Kefa. AR and VR development practice [M]. Beijing:
Mechanical Industry Press, 2016