Proceedings of the 29th International Conference
2018 Cybernetics & Informatics (K&I)
Augmented Reality as an Instrument for Teaching
Industrial Automation
Juhás Martin, Juhásová Bohuslava
Institute of Applied Informatics, Automation and Mechatronics
Faculty of Materials Science and Technology in Trnava
Slovak University of Technology in Bratislava
Trnava, Slovakia
martin_juhas@stuba.sk, bohuslava.juhasova@stuba.sk
Abstract— Augmented Reality is a progressive tool in the field
of education and learning systems. The subject of the article is the
introduction of a learning system focused on the teaching of
subjects from the field of industrial automation. However, the
modularity and versatility of its design mean that the system is
universally applicable to the teaching of any technical subject
matter. The solution is built on an open source platform, which
underlines the low cost of the system. The use of the A-Frame
framework in combination with the AR.js project allows for a
combination of learning information and real-world images on
commonly available mobile devices in augmented reality to be
offered. Additionally, the information is supplemented by real-
time process information from the process level control elements
of selected subsystems of the complex production line AFB Factory
by Festo Didactics.
Keywords—augmented reality; industrial automation; learning
system
I. AUGMENTED REALITY IN INDUSTRY
Augmented reality (AR) is a live direct or indirect view of a
physical, real-world environment whose elements are
"augmented" by computer-generated or extracted real-world
sensory input such as sound, video, graphics, haptics or GPS
data [1]. It is related to a more general concept called computer-
mediated reality, in which a view of reality is modified (possibly
even diminished rather than augmented) by a computer.
Augmented reality enhances one’s current perception of reality,
whereas in contrast, virtual reality replaces the real world with a
simulated one [2], [3].
Manufacturing is the process of transforming raw materials
and information into finished commodities with good added
value for the satisfaction of human needs. It has been a key
contribution to a nation’s economic growth for the last few
centuries and will continue to do so in the future. In the current
highly competitive and dynamic business environment, the
manufacturing industry is facing new challenges, which require
a holistic perspective on the four main classes of manufacturing
attributes, i.e., cost, time, quality and flexibility [4].
AR research in manufacturing applications is a strong and
growing area. However, it faces a higher order of accuracy,
response, and interface design. The challenge is to implement
Authorized licensed use limited to: UNIVERSIDAD DE VIGO. Downloaded on July 09,2021 at 13:11:08 UTC from IEEE Xplore. Restrictions apply.
Jan. 31 - Feb. 3, 2018, Lazy pod Makytou, Slovakia
integrated AR-assisted simulation tools that could improve
manufacturing operations, as well as product and process
development, leading to faster learning and training, hence
shorter lead-time, and consequently, reduced cost and improved
quality [5].
With new advances in computer and manufacturing
technologies, there is a growing trend of allowing users to
interact directly with the manufacturing information associated
with the manufacturing processes. AR has the ability to integrate
these modalities in real time into the real working environment,
which is useful for manufacturing activities, particularly
assembly, training, and maintenance [6].
AR applications may be created with the purpose of being
used as Technical support in manufacturing industries to
facilitate maintenance processes or assembly procedures, aiming
to reduce the operating time and training costs. Simple and
complex animations can be generated from 3D CAD models to
be displayed through an application run by a mobile device,
either a tablet PC or a smartphone [7].
Our project is focused on the use of augmented reality in
teaching the subjects in a specialized laboratory. This laboratory
is designed in accordance with Industry 4.0 concept so we want
to allow students to use their own smart devices for the education
process.
II. PORTFOLIO OF INDUSTRIAL AUTOMATION ELEMENTS
In a set of technical resources used for the teaching of
subjects in the field of industrial automation we can include the
complex hybrid manufacturing system AFB production line
from Festo Didactics (Fig. 1) that block diagram of it is shown
in Fig. 2.
Industrial training needs to be as close as possible to the
entire industrial sector covering wide range of production types,
so the main two technologies of factory automation as well as
process automation need to be taught in parallel, or even at the
same time. The entire range of different industry sectors is fairly
complex and undoubtedly sophisticated, but the range of
applications within this technology is too important, especially
in the field of so-called fully automated systems, and should
therefore be part of every trainee’s skill and knowledge
978-1-5386-4421-8/18/$31.00 ©2018 European Union
development. Festo Didactics training equipment covering the
so-called fully automated systems in industrial automation in
combination with process automation is being conducted on a
teaching factory solution called Agro, Food and Beverage
Training Factory, in short AFB Factory, providing expedient
exercises by hands-on training as well as in depth technology
training.
Fig. 1. Agro, Food and Beverage Training Factory
The AFB training factory delivers a learning environment
which consciously incorporates trends and innovations from all
areas of automation technology:
- Electric and pneumatic linear drive units;
- Semi-rotary drives and grippers;
- Valves and valve terminals;
- Sensors;
- Vision and control systems.
The central part of this learning arrangement is the intelligent
conveyor belt system in combination with different stations
arranged in so-called factory zones. Additionally, new
technologies are implemented shifting this training environment
closer to real industrial applications (RFID technology, QR
codes).
Fig. 2. Scheme of AFB Factory
Individual factory zones provides wide range of possibilities
for training and education, so these possibilities are listed in
TABLE I.
Authorized licensed use limited to: UNIVERSIDAD DE VIGO. Downloaded on July 09,2021 at 13:11:08 UTC from IEEE Xplore. Restrictions apply.
TABLE I. EDUCATION POSSIBILITIES OF AFB FACTORY
Factory
Zone
Education possibilities
Zone A Process automation with
• liquid medium filtering, mixing, temperature control;
• solid medium quality control, transport;
• recording, evaluating and controlling typical process
variables such as temperature, level, pressure, flow
rate, weight or count;
• pumping fluids;
• shutting off pipelines;
• installing, commissioning or maintaining butterfly
valves, slide valves, ball valves, electric drives.
Zone B Filling and packing with
• position and fill level of each bottle and the state of
completion of each lot recorded by various optical
sensors and a high-speed camera;
• production data fully tracked by RFID tags in the
bottle caps;
• belts with electric drives;
• various handling units;
• palletising applications;
• programmable logic controllers and the latest operator
control and monitoring tools.
Zone C Production automation with
• fast cycle times, part gripping, handling;
• detection, differentiation, separation and mounting;
• vacuum gripper technology;
• pneumatic linear and rotary drives, electrical drives;
• programming controllers, adjusting sensors;
• operating, maintaining and servicing individual lines
in a plant.
Zone D Transport and logistics with
• materials transportation, warehousing;
• chaotic or systematic warehousing, optimisation of
the material flow;
• planning and prioritisation of orders;
• signals sending over long distances to the transport
controller;
• high-performance drive units and positioning
systems;
• fast, precise movement in the automatic warehouse;
• industrial grippers using;
• communications based on AS interface or CAN;
• vision camera system, QR codes processing.
Zone E Recycling with
• robot programming;
• commissioning of the entire sequence of recycling;
• teaching a robot using the handheld device;
• safety and security applications;
• industrial high precision handling axis;
• RFID technology.
III. STRUCTURE OF THE EDUCATION SYSTEM
The concept of education system design allows improved the
teaching of subjects especially focused on industrial automation
using technological concept CPS (Cyber-Physical Systems).
Thanks to the database-oriented architecture, the database can
be supplemented by various relevant teaching materials and,
depending on the used materials, the system can be used to teach
a whole range of other technically oriented subjects. The system
can be used by students as well as pedagogues during the
learning process itself, but also during home self-study. The
system is designed to help the teacher as much as possible during
the education process, while teaching students autonomy and
analytical thinking.
 A system designed in a modular structure, as is shown in Fig.
3, will enable the students to identify individual components or
entire subsystems used in solving tasks (using a tablet,
smartphone, VR, AR, QR code), provide them with relevant
information about the component or device (manuals,
connection schemes, technical documentation, etc.) and allow
them to better understand their function by using the CPS
philosophy.
The student module has the ability to alert the teacher to a
student’s need for assistance in the given problem or task
solving. The applicants may check the status of their requests or
monitor their placement on the waiting requests list. Thanks the
digital form of the study materials as well as the objects of
interest (virtual models), the individual tasks and materials for
the student are also available outside the laboratories and beyond
the time of the defined study courses.
The teacher’s module system allows teachers to use all the
functionalities like a student, but additionally provides them
with an overview of the currently solved tasks, sample examples
with solutions, control questions for students, notification of a
request for assistance as well as of management and processing
of these requests.
From a global perspective, the system is applicable for
learning purposes not only in educational institutions, but also in
retraining, raising the knowledge level of personnel resources or
training their skills in removing failures in industrial enterprises.
Fig. 3. Modular structure of education system
IV. THE TOOLS USED FOR EDUCATION SYSTEM
IMPLEMENTATION
Implementation of the system involves the use of several
web-based technologies for process information sharing, whose
interconnection is shown in Fig. 4. The main purpose of the
solution was to use open source technologies as much as
possible, and minimizing costs while maximizing the
customizability of the proposed system.
Fig. 4. Scheme of process information sharing in web space
A. PLC WebServer
As stated in [8], a lot of technologies within Industry 4.0
have their origin in web technologies and informatics. However,
their implementation on the process level is limited by hardware
because their usage is primarily dedicated to the processes
control. Nevertheless there is an option of implementing user
web pages using the embedded webserver, which is not
necessary for primary control functions. This technology on one
hand offers added value for production management, and on the
other, the students become familiar with the web page
development. The custom user web pages offer unlimited
possibilities to visualize data according to the customer needs
from simple overviews to fully-featured web based HMI
(Human Machine Interface). The user web pages are developed
in HTML 5 using CSS and JavaScript, while the page files need
to be loaded to the PLC.
Our solution uses the aforementioned approach to the
application of embedded PLC web server with the optimal use
of system resources. To ensure a minimal PLCs load, user
webpages loaded in their memory contains station process data
in a very basic form as a process value enclosed in named html
div tag.
The embedded PLC web server in conjunction with an
auxiliary PC based web server offers a wide range of user web
pages utilization besides the production data acquisition (e.g.
linking the device documentation). The application of these
technologies appears to be much easier and less hardware
demanding, assuming the transformation of the traditional
pyramidal model of decentralized control systems in accordance
with the Industry 4.0 concept as shown in Fig. 5 and keeping the
principles of computer networks security.

Authorized licensed use limited to: UNIVERSIDAD DE VIGO. Downloaded on July 09,2021 at 13:11:08 UTC from IEEE Xplore. Restrictions apply.
 AR.js technology is marker based with possibility of hybrid
custom marker using. System implementation uses hybrid
personalized marker (markers) for each technological subsystem
with QR identification code included in it, as is shown in Fig. 6.
This module ensures just-in-time VR scene with 3D object
model re-orientation by AR marker identification and its
position and orientation evaluation.

Fig. 5. Comparison of decentralized control systems models [8]

B. Technologies of education system core

The entire architecture of the learning system is based on an
open source CMS system, WordPress, its modular structure Fig. 6. Hybrid custom AR marker
perfectly meets the requirements for such oriented systems [9].
The data base of system core realized in MySql server VR model of technological subsystem with technical data
environment is complemented by a proprietary data model of the stored in database is identified by QR code, scanned by
learning part of the system. Extensibility of the system core by JavaScript QRCode reader for HTML5 enabled browser [12].
user’s php based modules provides possibilities for creating the Again, it is an open source javascript project based on ZXing
required communication module between the core of the QRcode scanner.
learning system itself and the process level of the AFB Factory.
Process of data exchange between individual technologies is
C. Technologies of augmented reality module implementation depicted in communication diagram form in Fig. 7.
Due to the complexity of the proposed system, part of which
is implemented in the form of a standard database system, we
focused more on describing the technologies used to implement
the augmented reality module.
Mixture of real world picture with virtual scene called
augmented reality ensures open source 3D web framework for
building web based virtual reality with HTML and Entity-
Component, named A-Frame [10]. A-Frame can be developed
from a plain HTML file without having to install anything. A-
Frame is optimized from the ground up for WebVR. While A-
Frame uses the DOM, its elements don’t touch the browser
layout engine. 3D object updates are all done in memory with
little overhead under a single animation request call. A-Frame is
a powerful javascript based framework, providing a declarative,
composable, reusable entity-component structure. Developers
have unlimited access to JavaScript, DOM APIs, three.js,
WebVR, and WebGL.
A virtual scene containing schematic models of industrial
automation components grouped into functional units is created
by this tool. Object dynamics can be realized either in a simpler
way - by animating elements that compose an object model or
by more complex form - by incorporating more complex
animations directly into the model and then by their subsequent
parameter activation.
Created virtual scene is implemented into augmented reality Fig. 7. Communication diagram of augmented reality part of education system
by open source project AR.js [11]. AR.js is an efficient
Augmented Reality solution on the Web. It runs fully in client Combination of the above-mentioned technologies into one
system native web browser, which means no additional set resulted into created application that allows displaying the
application has to be installed. There is no need for a specific real-time process information from industrial automation
device. It runs on all mobile platforms: Android, iOS11 and components combined with their technical data in a web browser
Windows mobile. Depending on the used device, it can run very environment in the form of mixed reality with minimal hardware
fast, up to 60fps on relatively old hardware (about 2 years). and software requirements. An example of thus creeated

Authorized licensed use limited to: UNIVERSIDAD DE VIGO. Downloaded on July 09,2021 at 13:11:08 UTC from IEEE Xplore. Restrictions apply.
application intended for educational purposes is shown in Fig. 8
and Fig. 9.
The process information of the subsystem is displayed on the
data panel belonging to that device as well as in the graphical
form by changing the parameters of the virtual objects (position,
color, scale, rotation, opacity), directly representing the chosen
element of the industrial automation. Individual objects are
implemented as interactive elements, where the A-Frame object
of the cursor type allows the user to directly select the interest
element.
In this case, the identified subsystem is in zone A subsystem
for bulk material weighing, with its transporting through a screw
conveyor into a hopper followed by transporting by compressed
air to an input stack in zone B. In Fig. 8, the automatic weight is
selected by the user as the object of interest, to which the
information from the database is shown on the right side of the
display based on the ID of the object. On the left side of the
display are shown the real-time process information of the
selected subsystem – digital and analog inputs and outputs of the
controller – PLC.
Fig. 8. Augmented reality in education process, subsystem 01 part a
Fig. 9. Augmented reality in education process, subsystem 01 part b
V. CONCLUSION
This paper presents the possibility of using augmented
reality in the education process. The design and implementation
of the learning system is a partial objective of the project dealing
Authorized licensed use limited to: UNIVERSIDAD DE VIGO. Downloaded on July 09,2021 at 13:11:08 UTC from IEEE Xplore. Restrictions apply.
with increasing efficiency and popularity in the teaching of
technical subjects, specially focused on the area of industrial
automation. It is augmented reality which is the element that will
enable students to acquire theoretical knowledge when working
with real objects of industrial automation. The technologies used
to implement the proposed system make it possible to use
popular and modern IT tools on common mobile devices without
the need of specialized hardware or software. The individual
learning objects are represented by virtual models using the A-
Frame framework, which are transferred and mixed with the real
image of the real technological object using AR.js. The
technology of hybrid AR markers is used to carry information
about the examined object, as well as its position and orientation
with regard to the observer. The result is thus an interactive
scene based on real view, providing all necessary information
retrieved from the database part of the learning system. The
scene is also complemented by process information from the
PLC control elements of each technological subsystem of AFB
Factory line. The user - the student just-in-time sees not only the
result of change, for example, the configuration of a particular
operating element, but also in the virtual part look behind the
scenes of these changes.
ACKNOWLEDGMENT
This publication was written with financial support of the
KEGA agency in the frame of the project 040STU-4/2016
“Modernization of the Automatic Control Hardware course by
applying the concept Industry 4.0”
REFERENCES
[1] P. Schuettel, “The concise fintech compendium,” Fribourg: School of
Management Fribourg, Switzerland, 2017.
[2] J. Steuer, “Defining virtual reality: dimensions determining telepresence,”
Department of Communication, Stanford University, 15 October 1993.
[3] Introducing virtual environments national center for supercomputing
applications, University of Illinois, online, available at
http://archive.ncsa.illinois.edu/Cyberia/VETopLevels/VR.Overview.htm
l
[4] G. Chryssolouris, “Manufacturing systems – theory and practice,” 2nd ed.
Springer-Verlag, New York, 2006.
[5] A.Y.C. Nee, S.K. Ong, “Virtual and augmented reality applications in
manufacturing”, In IFAC Proceedings Volumes, Volume 46, Issue 9,
2013, Pages 15-26.
[6] A.Y.C. Nee, S.K. Ong, G. Chryssolouris, D. Mourtzis, “Augmented
reality applications in design and manufacturing,” In CIRP Annals,
Volume 61, Issue 2, 2012, Pages 657-679.
[7] D. Segovia, M. Mendoza, E. Mendoza, E. González, “Augmented reality
as a tool for production and quality monitoring,” In Procedia Computer
Science, Volume 75, 2015, Pages 291-300.
[8] M. Kopþek, “Embedded PLC webserver and possibilities of its
utilization,” In Research papers Faculty of Materials Science and
Technology Slovak University of Technology in Trnava, Vol. 24, no. 39
2016, pp. 33-42.
[9] WordPress, online, available at https://wordpress.com/
[10] A-Frame – Make WebVR, online, available at https://aframe.io/
[11] Efficient Augmented Reality for the Web, online, available at
https://github.com/jeromeetienne/AR.js
[12] QR Code scanner, online, available at https://webqr.com/about.html