Download as pdf or txt
Download as pdf or txt
You are on page 1of 132

EcoStruxure™ Augmented Operator Advisor

Configuration Training
Document Modification History

Date Version Description


30 Aug 2018 0.1 1. First issue (Draft)
12 Sep 2018 0.5 2. Refined and revised
16 Oct 2018 0.7 3. Refined and revised
01 Nov 2018 1.0 4. First release

The information contained in this document is proprietary to Schneider Electric.

This document contains proprietary information of Schneider Electric, and neither the document
nor said proprietary information shall be published, reproduced, copied, disclosed or used, in
whole or in part, for any purpose other than consideration of this document without the express
written permission of a duly authorized representative of the said company.
Contents
Chapter 1: Introduction to EcoStruxure™ Augmented Operator Advisor .................................... 9
What is EcoStruxure™ Augmented Operator Advisor ...................................................................... 10
EcoStruxure™ Augmented Operator Advisor Architecture .............................................................. 11
EcoStruxure™ Augmented Operator Advisor Examples of Use ........................................................ 12
Licensing ............................................................................................................................................ 13
Chapter 2: The Builder .............................................................................................................. 17
What is EcoStruxure™ Augmented Operator Advisor Builder .......................................................... 18
Prerequisites to Start Building a Project ........................................................................................... 19
Getting Started.................................................................................................................................. 20
EcoStruxure™ Augmented Operator Advisor Builder User Interface ............................................... 21
Chapter 3: Projects, Areas and Scenes ...................................................................................... 27
Key Concepts ..................................................................................................................................... 28
Projects ............................................................................................................................................. 29
Areas ................................................................................................................................................. 31
Scenes ............................................................................................................................................... 32
Why the Split Between Projects, Areas and Scenes?........................................................................ 33
Limitations ........................................................................................................................................ 34
Chapter 4: Points of Interest ..................................................................................................... 43
What is a Point of Interest? .............................................................................................................. 44
Types of Points of Interest ................................................................................................................ 45
Chapter 5: Building a Project ..................................................................................................... 53
Building a Project .............................................................................................................................. 54
Deploying a Project ........................................................................................................................... 55
Chapter 6: The App ................................................................................................................... 61
What is EcoStruxure™ Augmented Operator Advisor App ............................................................... 62
Using the App .................................................................................................................................... 63
App’s User Interface ......................................................................................................................... 64
Chapter 7: Subscenes ................................................................................................................ 69
What is a Subscene? ......................................................................................................................... 70
Chapter 8: Triggers .................................................................................................................... 77
What is a Trigger? ............................................................................................................................. 78
Chapter 9: The Runtime Software ............................................................................................. 83
What is EcoStruxure™ Augmented Operator Advisor Runtime........................................................ 84
EcoStruxure™ Augmented Operator Advisor Manager .................................................................... 85
Node-RED .......................................................................................................................................... 86
EcoStruxure Operator Terminal Expert / BLUE ................................................................................. 87
Chapter 10: Procedures ............................................................................................................... 93
What are Procedures? ...................................................................................................................... 94
Procedure Limitations ....................................................................................................................... 96
Chapter 11: The Manager .......................................................................................................... 105
What is the Manager?..................................................................................................................... 106
Users Management......................................................................................................................... 107
Node-RED Management ................................................................................................................. 108
Notes Management ........................................................................................................................ 109
Procedures Management ............................................................................................................... 110
Chapter 12: Image Recognition and Tracking ............................................................................ 115
Image or Tags, Which One to Use? ................................................................................................. 116
Guidelines for Taking Photographs ................................................................................................. 119
Image Limitations ............................................................................................................................ 120
Appendix A: Node-RED ................................................................................................................. 123
What is Node-RED? ......................................................................................................................... 124
Nodes and Flows ............................................................................................................................. 125
Safety Information

Important Read these instructions carefully, and look at the equipment to


Information become familiar with the device before trying to install, operate,
service, or maintain it. The following special messages may appear
throughout this documentation or on the equipment to warn of
potential hazards or to call attention to information that clarifies or
simplifies a procedure.
The addition of this symbol to a "Danger" or "Warning" safety label
indicates that an electrical hazard exists which will result in
personal injury if the instructions are not followed.

This is the safety alert symbol. It is used to alert you to potential


personal injury hazards. Obey all safety alert messages that follow
this symbol to avoid possible injury or death.

DANGER indicates a hazardous situation which, if not


avoided, will result in death or serious injury.

WARNING indicates a hazardous situation which, if not


avoided, could result in death or serious injury.

CAUTION indicates a hazardous situation which, if not


avoided, could result in minor or moderate injury.

NOTICE is used to address practices not related to


physical injury.

PLEASE NOTE
Electrical equipment should be installed, operated, serviced, and
maintained only by qualified personnel. No responsibility is
assumed by Schneider Electric for any consequences arising out of
the use of this material.
A qualified person is one who has skills and knowledge related to
the construction and operation of electrical equipment and its
installation, and has received safety training to recognize and avoid
the hazards involved.
DISCLAIMER

Schneider Electric makes no representations or warranties with respect to this manual and, to the maximum extent permitted by law,
expressly limits its liability for breach of any warranty that may be implied to the replacement of this manual with another. Furthermore,
Schneider Electric reserves the right to revise this publication at any time without incurring an obligation to notify any person of the
revision.
The information provided in this documentation contains general descriptions and/or technical characteristics of the performance of the
products contained herein. This documentation is not intended as a substitute for and is not to be used for determining suitability or
reliability of these products for specific user applications. It is the duty of any such user or integrator to perform the appropriate and
complete risk analysis, evaluation and testing of the products with respect to the relevant specific application or use thereof. Neither
Schneider Electric nor any of its affiliates or subsidiaries shall be responsible or liable for misuse of the information that is contained herein.
If you have any suggestions for improvements or amendments or have found errors in this publication, please notify us.
All pertinent state, regional, and local safety regulations must be observed when installing and using this product. For reasons of safety and
to help ensure compliance with documented system data, only the manufacturer should perform repairs to components.
When devices are used for applications with technical safety requirements, the relevant instructions must be followed.
Failure to use Schneider Electric software or approved software with our hardware products may result in injury, harm, or improper
operating results.
Failure to observe this information can result in injury or equipment damage.
© 2018 Schneider Electric. All rights reserved.
The contents of this manual are proprietary to Schneider Electric and all rights, including copyright, are reserved by Schneider Electric. No
part of this document may be reproduced in any form or by any means, electronic or mechanical, including photocopying, without express
written permission of Schneider Electric.
EcoStruxure™ Augmented Operator Advisor Configuration Training Manual

INTRODUCTION AND LEGAL NOTICE

Satisfactory completion of the course evaluation is mandatory for you to obtain a Schneider Electric certificate of completion of the training
course.
Schneider Electric will not accept any liability for action taken in reliance on this training manual.

TRADEMARKS

Schneider Electric has made every effort to supply trademark information about company names, products and services mentioned in this
manual. Trademarks shown below were derived from various sources.
EcoStruxure™, EcoStruxure™ Augmented Operator Advisor are trademarks owned by Schneider Electric or its affiliated companies. All
other trademarks are the property of their respective owners.
Excel and Windows are either registered trademarks or trademarks of Microsoft® Corporation in the United States and/or other countries.
VirtualBox is a registered trademark of Oracle Corporation in the United States and other countries.
Some product names used in this manual are used for identification purposes only and may be trademarks of their respective companies.

Validity Note

The present documentation is intended for qualified technical personnel responsible for the implementation, operation and maintenance of
the products described. It contains information necessary for the proper use of the products.

About Us

Members of Schneider Electric’s team of Instructional Designers have tertiary qualifications in Education, Educational Course Development
and are also experienced Instructors. Currently, the team is supporting a range of Schneider Electric courses in multiple languages and
multiple software environments.

Authors

Original Material: Guilherme Moreira

Contributors

Arvin Pitt
Creation Date:

15 October 2018
Course Overview

Course Objectives By the completion of this training course the student will:
 Be able to create a EcoStruxure™ Augmented Operator Advisor
project
 Understand best practices to take photographs to optimize image
recognition process

Target Audience The EcoStruxure™ Augmented Operator Advisor Configuration training


course is an integral part of the complete Educational Services curriculum.
This course is designed for:
 Users who are new to EcoStruxure™ Augmented Operator
Advisor
 Users who are going to implement a EcoStruxure™ Augmented
Operator Advisor project

Prerequisite It is expected that trainees will be familiar with:


Knowledge  The concepts of PACs and HMIs
 Ethernet networking principles
 The concepts of industrial automation
 The EcoStruxure™ architecture
 Unity Pro
 EcoStruxure Operator Terminal Expert
 Microsoft Windows

Scope of this This training manual is a supplement to the authorised training. To make
Training Manual proper use of the software, students should also refer to the documentation
that has been provided with the product such as the Help Files, User
Guides or Knowledge Base.
The graphics displaying screen captures were taken using the Windows®
10 operating system. If students are running a different version of
Windows, the screen images may differ slightly from those shown in the
training manual.
Some screen captures may have been taken from beta or earlier versions
of the software and may vary slightly from release screen captures.
Course Overview (cont.)

Course Program The training course will take two days to complete. The following
program outlines the topics that will be covered on each day:

Day Topic
 Introduction to EcoStruxure™ Augmented
Operator Advisor
 The Builder
 Projects, Areas and Scenes
1
 Points of Interest
 Building a project
 The App
 Subscenes
 Triggers
 The Runtime Software
 Procedures
2  The Manager
 Image Recognition and Tracking
 Node-RED Introduction

Course Assets For the training course, the following are also required:

Water_Plant.png Image file used as a scene.


Pump_Maintenance.jpg Image file used as an image POI.
Pump_house.jpg Image file used as a subscene.
Pump_document.pdf Document used in a Procedure step.
M340a.sta Unity Pro M340 PLC project
M340a.vxdz EcoStruxure Operator Terminal
Expert project for the HMI
AOA_QSG.pdf File for Document POI
AppendixA_NodeRED_Flo Flow to be imported in Node-RED
w.docx
AOA_2.2.0.395.zip AOA RT installation files

8
Chapter 1: Introduction to EcoStruxure™ Augmented
Operator Advisor

Overview
Introduction EcoStruxure™ Augmented Operator Advisor is a solution proposed by
Schneider Electric based on Augmented Reality technology, which allows
the fusion of a real physical world with virtual objects, on a real scene.
This chapter will take an initial overview look at the product to set the
scene for the training.
By the completion of this chapter the student will be able to:
 Describe the purpose of EcoStruxure™ Augmented Operator
Advisor
 Launch the product

Contents
What is EcoStruxure™ Augmented Operator Advisor .......................... 10
EcoStruxure™ Augmented Operator Advisor Architecture .................. 11
EcoStruxure™ Augmented Operator Advisor Examples of Use ............ 12
Licensing ................................................................................................ 13

Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor 9


What is EcoStruxure™ Augmented Operator Advisor
Brief History EcoStruxure™ Augmented Operator Advisor uses a technology called
Augmented Reality to optimize the operation and maintenance of
industrial sites and equipment.
Although AR technologies have been around for at least 15 years,
mainstream adoption is a recent phenomenon within the manufacturing
industry.
New breakthroughs in the affordability and applicability of AR
technology have accelerated the rate of adoption.
Within the realm of industrial manufacturing, augmented reality is about
two different environments converging or blending in a way that boosts
the effectiveness and efficiency of plant operators.
One environment is “real” (what you see, unassisted, in front of your own
eyes) and the other is “virtual” (not “real”, but computer generated). Both
environments can be understood in terms of a continuum, with real
environments at one end and completely virtual environments at the other.
What lies in between is augmented reality, which is, in essence, mixed
reality.
EcoStruxure™ Augmented Operator Advisor makes use of AR
technology and bring benefits to industrial operations by maximizing
operational efficiency, reducing downtime and simplifying maintenance
while increasing safety of operators and maintainers.
Increasing efficiency, reducing labour and decreasing costs are the main
benefits for users of EcoStruxure™ Augmented Operator Advisor
solution.

How Does it Below is a basic description of how EcoStruxure™ Augmented Operator


Work? Advisor works after the project has been developed, deployed and
commissioned.
 The operator points a tablet at the site or equipment to be
monitored.
 EcoStruxure™ Augmented Operator Advisor uses special image
comparison techniques to match the scene visible in the field of
view of the tablet camera with previously stored photographs of
the same scene.
 When a match is made, markers called points of interest are
overlaid on the live scene visible on the tablet.
 The operator taps the point of interest markers to display
information. A wide variety of information can be displayed,
including:
 Process variables
 Values extracted from an SQL database
 Documents or instruction sheets
 Wiring diagrams
 Web pages
 Videos
 Audio
 Procedures

10 Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor


EcoStruxure™ Augmented Operator Advisor Architecture

System EcoStruxure™ Augmented Operator Advisor is made up of the following


Components main components:
 EcoStruxure™ Augmented Operator Advisor Builder, a Web-
based application.
 EcoStruxure™ Augmented Operator Advisor Runtime, server-
based Windows software.
 EcoStruxure™ Augmented Operator Advisor App, an application
running on a tablet.
The following diagram shows the relationship between the components.

For the EcoStruxure™ Augmented Operator Advisor App to be able to


communicate with the EcoStruxure™ Augmented Operator Advisor
Runtime component, both must be connected to the same Wi-Fi network
or 3G/4G mobile network.
To view the real-time values of process variables produced by the
equipment being monitored, one of the following applications must be
used:
 Node-RED
 EcoStruxure Operator Terminal Expert (or Pro-face BLUE)
The solution also includes a Web-based management tool, EcoStruxure™
Augmented Operator Advisor Manager.

Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor 11


EcoStruxure™ Augmented Operator Advisor Examples of Use

Typical Scenarios EcoStruxure™ Augmented Operator Advisor can typically be used, in


different scenarios, such as:
 In front of a machine, having immediate access to relevant
information in the field for operations and maintenance such as
machine history, process variables, alarms, documents (products,
installations, instructions, electrical diagrams, etc.).
 In front of an electrical cabinet, allowing fast diagnosis without
lockout or electrical qualifications, being able to virtually open the
cabinet’s door and verify information on the tablet’s screen.
 An overall view of a whole industrial site.

The usage of EcoStruxure™ Augmented Operator Advisor is not limited


to these scenarios, these are only typical examples.

12 Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor


Licensing

Product Licensing Trial versions of EcoStruxure™ Augmented Operator Advisor Runtime


and EcoStruxure™ Augmented Operator Advisor Builder are available.
They are free to download and use for up to 42 days. At any time during
the trial period, the user can purchase licenses and activate the software.

EcoStruxure™ Augmented Operator Advisor is composed of 2 offers:

 Essential offer: for Point of Interest visualization only


 Performance offer: for Point of Interest and Procedures
visualization.
The following license types are available:

License Type Maximum number of...

Projects POI / Procedures


EcoStruxure™ Augmented Operator Advisor Builder
Trial Unlimited (1) Unlimited

Essential Unlimited Unlimited


Performance Unlimited Unlimited
EcoStruxure™ Augmented Operator Advisor runtime software
Trial Unlimited Unlimited
Essential Level 1 1 20 / 0
Essential Level 2 1 50 / 0
Essential Level 3 1 Unlimited / 0
Performance Level 1 1 20 / 10
Performance Level 2 1 50 / 20
Performance Level 3 1 Unlimited
Multi-area Unlimited Unlimited
(1) EcoStruxure™ Augmented Operator Advisor Builder Trial projects
containing multiple areas are incompatible with EcoStruxure
Augmented Operator Advisor runtime software Essential and
Performance licenses that support only one area.

Tablet App The EcoStruxureTM Augmented Operator Advisor App is compatible with
iPads running iOS 11.2.5 or greater and Android tablets running OS V5.0
or greater.
The EcoStruxure™ Augmented Operator Advisor App does not require a
license. It is available from the application store of the tablet.

Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor 13


Exercise – Run the Sample Project

Learning
By the completion of this exercise you will be able to:
Outcome
 Run the sample project available in EcoStruxure™ Augmented
Operator Advisor App.

1) Download and install the App on your tablet and run it.

2) Login to Sample projects from the Preferred sites page of the app. No
username or password will be needed.

3) Tap on Start after selecting the Water plant scene under Sample
folder.

4) Perform image recognition with the image gallery available for


download from the EcoStruxure™ Augmented Operator Advisor
webpage.

Note:
The image gallery for sample projects is available for download from
Schneider Electric’s website. Ask your instructor to show you how to
access it.

14 Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor


Exercise – Run the Sample Project

5) Explore the project. Notice there seems to be a problem with Tank2.

6) Tap on the POI on top of the building flashing red. This is a subscene
and the image on the tablet will change.

7) Explore the subscene.

8) To exit the subscene tap on the door POI again.

Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor 15


Summary
Summary This chapter introduced EcoStruxure™ Augmented Operator Advisor;
Where it would be used, the basic components and licensing.

Questions The following questions are used to check understanding.


 What are the 3 main components of the EcoStruxure™
Augmented Operator Advisor typical architecture?

 Why does the EcoStruxure™ Augmented Operator Advisor App


on your tablet and the Runtime device need to be connected on the
same network?

 What is the difference between the Essential and Performance


offers?

16 Chapter 1 – Introduction EcoStruxure™ Augmented Operator Advisor


Chapter 2: The Builder

Overview
Introduction The EcoStruxure™ Augmented Operator Advisor Builder is the
environment in which the user develops the project.
By the completion of this chapter the student will be able to:
 Describe the purpose of EcoStruxure™ Augmented Operator
Advisor Builder
 Understand the prerequisites to start creating a project with the
Builder.

Contents
What is EcoStruxure™ Augmented Operator Advisor Builder .............. 18
Prerequisites to Start Building a Project ............................................... 19
Getting started ...................................................................................... 20
EcoStruxure™ Augmented Operator Advisor Builder User Interface ... 21

Chapter 2 – The Builder 17


What is EcoStruxure™ Augmented Operator Advisor Builder
Access to the The EcoStruxure™ Augmented Operator Advisor Builder is a web-based
Builder UI application, hence no installation is required. The PC running
EcoStruxure™ Augmented Operator Advisor Builder must have Internet
access.
You can use any of the following web browsers:
 Google® Chrome™ (version 47 or higher)
 Microsoft® Internet Explorer™ (version 11 or higher)
 Mozilla Firefox® (version 43 or higher)
 Microsoft Edge (version 13 or higher)
To run EcoStruxure™ Augmented Operator Advisor Builder the user
must access the URL below:
 https://app.schneider-electric.com/ecostruxure-augmented-
operator-advisor
A login will be required, to create one, click on Sign Up.

EcoStruxure™ Augmented Operator Advisor Builder is the software that


is used to:
 Organize and structure areas, scenes, subscenes, and points of
interest in a project.
 Create and position the points of interest on photographs.
 Define what happens when the operator taps the point of interest.
 Build procedures to explain a set of tasks the operator must carry
out.

18 Chapter 2 – The Builder


Prerequisites to Start Building a Project
Site Visit Before using EcoStruxure™ Augmented Operator Advisor Builder,
perform a top-down analysis:
 The sites where EcoStruxure™ Augmented Operator Advisor is to
be deployed.
 The areas within each site (such as production lines, plant rooms,
storage areas, workshops, laboratories, and office areas) to be
monitored.
 The scenes within each area, for example, production lines,
machines, and electrical cabinets.
 Optionally, the subscenes, that are parts of a scene that may need
to be zoomed in on to uniquely identify all the controls on the
equipment.

Taking The first step to determine whether to use photographs or tags is to


Photographs and identify a scene.
Tagging If you are using photographs, the next step is to take high-resolution
Equipment photographs of all the scenes, and subscenes.
Store the photographs on a hard disk drive, USB key, or network folder
accessible from the device on which EcoStruxure™ Augmented Operator
Advisor Builder is used.
If you are using tags, the next step is to uniquely identify physically
similar or identical items of equipment and attach tags to them.

Note:
These steps will be covered in more details in a later chapter.

Start Building When all prerequisites are met and the images are available, start using
EcoStruxure™ Augmented Operator Advisor Builder to create a project.

Chapter 2 – The Builder 19


Getting Started
Project Workflow The following diagram can be followed as a guide to all the steps needed
to configure a project using EcoStruxure™ Augmented Operator Advisor.

20 Chapter 2 – The Builder


EcoStruxure™ Augmented Operator Advisor Builder User
Interface
User Interface The main components of the EcoStruxure™ Augmented Operator Advisor
Overview Builder user interface are as follows:

1. Toolbar. Displays icons specific to areas or scenes.


2. Project toolbar. Displays icons for project-specific actions.
3. Tabs to access the areas, scenes or procedures windows.
4. Allows you to select the project language.
5. Shows the currently logged in user name and allows access to
license information and the log out command.
6. Displays online help. Gives access to online help, terms &
conditions, downloading tags and product information.
7. Left sidebar. Project Structure in tree style.
8. Main window. Configure selected items.
9. Right sidebar. Dynamic notes.

Chapter 2 – The Builder 21


Exercise – Create an Account for the Builder

Learning
By the completion of this exercise you will be able to:
Outcome
 Create an account to login with EcoStruxure™ Augmented Operator
Advisor Builder.

1) Launch the Builder web site from the URL:


https://app.schneider-electric.com/ecostruxure-augmented-operator-
advisor

2) Click on Sign Up.

Note:
For Schneider Electric employees it is possible to login by clicking on the

icon.

3) Enter your e-mail address and click Next.

Chapter 2 – The Builder 22


Exercise – Create an Account for the Builder

4) Follow the instructions in the window to fill in the requested


information and click on Sign Up.

5) You will receive an e-mail with a link to confirm your registration.

Chapter 2 – The Builder 23


Exercise – Create an Account for the Builder

6) Open your email and click on the link provided.

7) You will now be able to login, and a pop up window will ask for a
license Activation ID. If you do not have one, click on Start Trial and
launch the builder.

8) After login, examine the User Interface of the Builder and get familiar
with the different menus and options available.

24 Chapter 2 – The Builder


Summary
Summary This chapter introduced EcoStruxure™ Augmented Operator Advisor
Builder, how to access it and the prerequisites to start using it.

Questions The following questions are used to check understanding.


 What is the purpose of the EcoStruxure™ Augmented Operator
Advisor Builder?

 Where is it accessed from?

 What are the main things to be analysed before starting to create a


project in the Builder?

Chapter 2 – The Builder 25


Chapter 3: Projects, Areas and Scenes

Overview
Introduction Projects, Areas and Scenes are essential parts of a EcoStruxure™
Augmented Operator Advisor project. These are created and configured
using the Builder.
By the completion of this chapter the student will be able to:
 Create a new project
 Configure a new area in the project
 Configure a new scene within an area

Contents
Key Concepts ......................................................................................... 28
Projects ................................................................................................. 29

Chapter 3 – Projects, Areas and Scenes 27


Key Concepts
Key Concepts EcoStruxure™ Augmented Operator Advisor introduces a few key
concepts:
 A project is a container for all the information in areas, scenes,
and subscenes.
 An area is a zone within each site or building—such as production
lines, plant rooms, storage areas, workshops, laboratories, and
office areas—that you want to monitor using EcoStruxure™
Augmented Operator Advisor.
 Scenes are things to monitor within an area, for example,
machines, electrical cabinets, or production lines.
 Subscenes are a part of a scene, typically a close view of a
particular area of the scene. A subscene can contain its own points
of interest.
 Procedures are a set of instructions that the operator must
perform.

Note:
The number of areas you can monitor depends on the type of license you
purchase.

28 Chapter 3 – Projects, Areas and Scenes


Projects
Overview A project can contain multiple areas and scenes. The EcoStruxure™
Augmented Operator Advisor Runtime will only run one single project at
a time.
In your tablet, you are also only able to perform image recognition at one
project at a time.

New Project After being signed up, logged in and having a first look at the
Creation EcoStruxure™ Augmented Operator Advisor Builder user interface you
can now start creating your first project.
In this course, we will create a project using the water plant image demo
available on: www.schneider-electric.com.
This image can be downloaded from the EcoStruxure™ Augmented
Operator Advisor web page or alternatively it can be provided by your
instructor.
On the Project Toolbar menu, you must click on Create a new project to
start.

How to Create a On the Project Toolbar menu, click on Create a new project to start.
New Project

A new window on the right sidebar will appear requesting some


information about the project to be created.

Chapter 3 – Projects, Areas and Scenes 29


Projects (cont.)
Structure of a When you open a project, a structured view of the project appears in the
Project left sidebar. Each object you create (area, scene, point of interest, and so
on) is automatically added to the structure. By default, the structure of
areas in the project is displayed as example below:

Backup / Restore You can make a backup of your project on your local device using the
icon. At a later time, you can use this backup to restore your project
into the EcoStruxure™ Augmented Operator Advisor Builder. Backups
can be encrypted using a password.
The backup / restore function is only available when you have activated an
EcoStruxure™ Augmented Operator Advisor Builder license.
When the Backup / Restore function is launched, a window is displayed as
example below:

30 Chapter 3 – Projects, Areas and Scenes


Areas
Areas Overview A project is made up of one or more areas. You can create an area, for
example, for each production line or item of equipment to be monitored.
Keep in mind the number of areas you can deploy to your Run Time
device is dependent on your type of license.

How to Create a On the Contextual Toolbar menu, you must click on the Area icon
New Area

to create a new area.

Chapter 3 – Projects, Areas and Scenes 31


Scenes
Scenes Overview A scene is a combination of:
 An image (high-resolution photograph) of, for example, an
electrical cabinet, machine, or production line.
 Points of interest, positioned on the image to indicate where
information is available to be monitored. This information can be
real-time variable data, documents or procedures.
 Subscenes. A sub-scene is an image overlaid on the main image
used to access more detailed information about a particular area of
a scene.
 Other graphics, such as arrows, lines, or text, that are overlaid on
the main image to help the operator better understand or more
easily navigate the scene.
Before creating a scene, decide whether you are going to use photographs
or tags to identify the scene. This will be discussed in more details on a
later chapter in this course.

How to Create a On the tabs menu, you can select the Scenes tab, highlight the area you are
New Scene
working with and click on the scene icon to create a new scene.

32 Chapter 3 – Projects, Areas and Scenes


Why the Split Between Projects, Areas and Scenes?
Answer is: EcoStruxure™ Augmented Operator Advisor uses this split between
Performance project, areas and scenes to optimize performance during image
recognition.
When the EcoStruxure™ Augmented Operator Advisor App on the tablet
is viewing reality, it is running an algorithm to compare the real video
coming from the tablet camera with a database of images that have been
downloaded to the EcoStruxure™ Augmented Operator Advisor Run
Time device.
When the tablet tries to recognise a scene, all registered images included
in the area are compared to the real scene.
Reducing the search to the area has better performance during image
recognition, as there is a reduced number of images to be analysed.

Chapter 3 – Projects, Areas and Scenes 33


Limitations
Maximum The following describes the maximum number of elements that can be
Numbers used in a EcoStruxure™ Augmented Operator Advisor project.
The number of elements that can be added may also be limited by the
License Requirements.

Projects and Areas


Element Maximum Number
Areas in a project 100(1)
Languages in a project 8
(1) A project must have at least one area and one scene.

Points of Interest
Element Maximum Number

Points of interest in a scene 50

Points of interest in a project 5000

Variables in a project 1000

List items in a List point of 5


interest
Characters in labels/names 50

Characters in descriptions 255

Scenes
Element Maximum Number
Scenes in a project 100(1)

Subscene levels 1
Subscenes in a scene 10
2D images in a project 100
Tags in a project 100
Characters in labels/names 50
Characters in descriptions 255
(1) 100 images in 1 scene, or 100 scenes with 1 image

34 Chapter 3 – Projects, Areas and Scenes


Exercise – Create a New Project

Learning
By the completion of this exercise you will be able to:
Outcome
 Create a new project with the Builder.

1) Click on Create a new project icon on the project toolbar.

2) Fill in the project name and project description.

3) Select a language and click the ‘+’ sign to add it to the project. More
languages can be added if required.

4) You need to indicate which language will be the reference one.

Chapter 3 – Projects, Areas and Scenes 35


Exercise – Create a New Project

5) Click Validate.

You will be warned about not being able to change the reference
language later. Agree by clicking Yes.

Note:
There is an in-built translator in case the project needs to be translated into
one of the selected languages from the reference language.

36 Chapter 3 – Projects, Areas and Scenes


Exercise – Create a New Area

Learning
By the completion of this exercise you will be able to:
Outcome
 Create a new area within your project.

1) Click on Area from the Toolbar menu.

2) You will be prompted to enter some information about the area on


the right sidebar. Click Validate after filling in.

3) Notice the left sidebar. The tree structure of the project has changed.
You also should have access to these items from the main window.

Chapter 3 – Projects, Areas and Scenes 37


Exercise – Create a New Scene

Learning
By the completion of this exercise you will be able to:
Outcome
 Create a new scene within an area of your project.

1) Click on the Scenes tab from the menu.

2) Highlight the area you are working with on the left sidebar tree
structure menu.

3) Click on the Scene icon from the Toolbar menu to start configuring a
new scene.

4) You will be prompted to fill in some information about the scene. The
Scene type will be Image Recognition.

5) Give the scene a label, in this example, we can use Water Plant.

6) Click on Upload Images and browse for the file Water_Plant.png


provided by your instructor. Alternatively, you can simply drag and
drop the image file from your windows explorer into the web
browser where the “Drop Images” dotted square box is located on
the right sidebar.

Note:
The file formats supported for the images are .PNG or .JPG and they must
have a maximum of 2MB data size.

38 Chapter 3 – Projects, Areas and Scenes


Exercise – Create a New Scene

7) Once the information is provided and the image chosen, the right
sidebar will look like this, and you can now click on Validate.

Chapter 3 – Projects, Areas and Scenes 39


Exercise – Create a New Scene

8) You will see the image placed on the main window and your new
scene populated on the tree structure menu on the left sidebar of
the page.

40 Chapter 3 – Projects, Areas and Scenes


Summary
Summary This chapter explained the main elements of a EcoStruxure™ Augmented
Operator Advisor project and how to start configuring them in the Builder.

Questions The following questions are used to check understanding.

 What are the main elements part of the project structure?

 Why is the project split between areas and scenes?

Chapter 3 – Projects, Areas and Scenes 41


Chapter 4: Points of Interest

Overview
Introduction Points of Interest are used to display information on the screen of the
tablet for the operator.
By the completion of this chapter the student will be able to:
 Understand what is a Point of Interest
 Create Points of Interest in your project

Contents
What is a Point of Interest? .................................................................. 44
Types of Points of Interest .................................................................... 45

Chapter 4 – Points of Interest 43


What is a Point of Interest?
Points of Interest A point of interest is a position in a scene for which additional diagnostic
or technical information is available. The point of interest can be placed
anywhere in the scene. The number of points of interest that can be added
to each scene is limited: refer to the Limitations section in Chapter 3.
Various types of points of interest can be defined, including technical
documentation, design drawings, information contained in a spreadsheet,
and the real-time values of process variables generated by equipment.
When the operator points the tablet camera at the scene, EcoStruxure™
Augmented Operator Advisor automatically recognizes it and using
augmented reality displays the points of interest on the scene.
You can define the following types of point of interest:
 Text
 Variables
 Documents
 External applications

44 Chapter 4 – Points of Interest


Types of Points of Interest
Variable When viewing a scene, a variable point of interest is replaced by the real-
time value of a process variable.
EcoStruxure™ Augmented Operator Advisor supports the following
variable source types:
 EcoStruxure Operator Terminal Expert (or Pro-face BLUE)
variables. The value is read from the PLC using EcoStruxure
Operator Terminal Expert (or Pro-face BLUE) software.
 Internal. The value is read from any source using Node-RED
software.
Variable points of interest are specific to an area. For example, you could
define a variable with the name “VAR101” in two different areas with
different properties.
For EcoStruxure Operator Terminal Expert variables, use the complete
name, including the device name part. If the EcoStruxure Operator
Terminal Expert variable name includes spaces, replace these with
underscores (“_”) when creating the equivalent EcoStruxure™
Augmented Operator Advisor Builder variable.

Note:
EcoStruxure™ Augmented Operator Advisor Builder variable names must
be exactly the same as the variable names defined in EcoStruxure
Operator Terminal Expert or Node-RED.

Document You associate document points of interest with specific positions in a


scene. The operator viewing the scene on the tablet can then tap the point
of interest to display the associated document. The document could be, for
example, a wiring diagram for a cable loom, a video, web page, or a user
manual.
Documents can be stored locally, which means the document file is copied
to the tablet when it is opened for the first time. Thereafter, the local copy
is opened.
If the Copy the document locally option is not selected, the document is
copied to the tablet each time the operator taps the associated document
point of interest.
To optimize performance, therefore, it is better to copy frequently-
consulted documents locally.

Note:
Storing too many documents locally can fill up the tablet storage.

Chapter 4 – Points of Interest 45


Types of Points of Interest (cont.)
Text Text point of interest is the simplest type available. Its purpose is to show
the operator some textual information on the screen to identify an area of a
site or a component of a machine, for example.

External App The App is launched when the operator taps the POI.
Any external app can be launched on the tablet screen from the point of
interest, as long as the app provides a Uniform Resource Identifier (URI)
path type.
This is used to point to an object such as a file, and defines which
application is used to open it.
The use of external apps may require the installation of third party
applications.
Below is an example of a URL used to open the e-mail app and send an e-
mail to a defined recipient.
mailto:toto@toto.fr?subject=test&body=test_mail_content

46 Chapter 4 – Points of Interest


Exercise – Create a List POI

Learning
By the completion of this exercise you will be able to:
Outcome
 Create a new point of interest within a scene.

1) Drag and drop the List POI icon from the toolbar menu on
the scene image.

2) You will see a new POI on the image, and some information being
requested on the right sidebar. Give this POI a title, such as
Pump_house, and click on Configure Information.

Note:
If you uncheck the Show checkbox, that means a trigger condition will
need to be configured for the POI to appear on the tablet screen.
The Keep opened checkbox is used to configure whether the list’s
content will be already open once the image is recognized on the tablet
screen, or if the operator will need to click the POI in order to view its
content.
The Security Level field is used to manage which users will have access to
this POI, this will be explained later in this course.

Chapter 4 – Points of Interest 47


Exercise – Create a List POI

3) You should now see the POI with its title on your image, and new
information being requested on the right sidebar. This means you are
now able to configure the first item on this List.

4) Note that you can configure this item as any POI type. In this
example, give it a label such as Water_Pumping, and leave the type
as Text, and then click on Validate.

5) Your POI should look like this:

Note:
If you wish to delete an item from the list POI, click on the icon.

If you wish to create a new item on the list POI, click on the icon.

48 Chapter 4 – Points of Interest


Exercise – Create a List POI

6) Add a second item to the List POI. Click on the icon.

7) On the right sidebar, configure it to call, for example, a location on


google maps. You can choose a location of your preference by
copying its URL from Google Maps. Below is a URL example.

https://www.google.com/maps/place/CERN/@46.2351762,6.039288
9,15z/data=!3m1!4b1!4m5!3m4!1s0x478c62fcec737b11:0x81bef3ae
7a885e31!8m2!3d46.234032!4d6.052951

8) Add a third item to the List POI, this time make it a Document type.
Use the file AOA_QSG.pdf given with the course resources.

Chapter 4 – Points of Interest 49


Exercise – Create a List POI

9) The List POI will look like this.

50 Chapter 4 – Points of Interest


Summary
Summary This chapter explained Points of Interest and their different types.
Questions The following questions are used to check understanding.

 What types of POI are available?

 For a Document POI, from which location can the document be


accessed?

Chapter 4 – Points of Interest 51


Chapter 5: Building a Project

Overview
Introduction This chapter will show how to build and deploy a project.
By the completion of this chapter the student will be able to:
 Build a project
 Deploy the built project locally in your tablet

Contents
Building a Project .................................................................................. 54
Deploying a Project ............................................................................... 55

Chapter 4 – Points of Interest 53


Building a Project
Overview When you have finished defining scenes, you can build the project. After
building the project, copy the built file to the EcoStruxure™ Augmented
Operator Advisor Runtime component database. The project is then
available for use on the EcoStruxure™ Augmented Operator Advisor
App.
The project can also be copied locally in your tablet. This way you can run
the project locally, without a server connection. This is useful for testing
purposes.

How to Build a On the project toolbar menu, you must click on the Build Project icon.
Project

Download the After the project is built, it must be downloaded to your computer. A .zip
Project file is generated containing all the project’s data.

The content of the .zip file must be deployed to your Runtime Device.
It can also be deployed locally to your tablet.

54 Chapter 4 – Points of Interest


Deploying a Project
Run Time Device After the project is built, it must be deployed to the Run Time device
Options component. This is also known as the EcoStruxure™ Augmented
Operator Advisor server.
This can be one of the below:
 Magelis HMIG5U Open Box PC
 Magelis iPC / Industrial PC
 Magelis Edge Box
 Standard Windows PC

Deploy Locally It is also possible to deploy the project locally, directly in the tablet. This
Option is mainly used for testing and demonstration purposes.
There are different methods to deploy the project to the tablet depending
on which tablet you are using.
For Android tablets, the access to the correct path can be done using
Windows Explorer, as if the tablet were a regular external hard drive.
For iOS tablets, iTunes must be used to copy the project to the correct
path in the tablet.

Chapter 4 – Points of Interest 55


Exercise –Building & Deploying a Project

Learning
By the completion of this exercise you will be able to:
Outcome
 Build a EcoStruxure™ Augmented Operator Advisor project.
 Deploy a EcoStruxure™ Augmented Operator Advisor project to a
tablet.
1) Click on the Build a project icon on the project toolbar.

2) You will be informed about what type of license is needed for the
Runtime. Click on Start Build.

3) After the process is finished, click on Download to save a .zip file that
is created containing all the project data.

4) This file will be named after the project’s name and current Builder
version.

5) Save this file in your computer.

6) Make sure you have downloaded and installed the EcoStruxure™


Augmented Operator Advisor App in your tablet.

Note:
You may need to access your tablet’s settings to give the app permission
to access the camera and data storage when you run it for the first time.

56 Chapter 4 – Points of Interest


Exercise –Building & Deploying a Project

7) Run the EcoStruxure™ Augmented Operator Advisor App for the first
time and accept the Terms & Conditions.
8) Unzip the project file that has been built in the previous exercise.

9) Copy the unzipped project folder from your computer to the tablet.
The procedure is different for iOS and Android tablets.

• On Android tablets:
..\Tablet\Android\data\com.schneiderelectric.AugmentedOpe
rator\Local

• On IOS tablets:
..\Apps\EcoStruxure Augmented Operator
Advisor\Documents\Local

Note:
It is essential that you unzip the built project file, and copy the whole
folder to the tablet.

Note:
For iOS tablets, iTunes is needed to copy the folder. There are alternative
third-party software available to manage file transferring to iOS tablets.

Chapter 4 – Points of Interest 57


Exercise –Deploying a Project with iTunes

Learning
By the completion of this exercise you will be able to:
Outcome
 Deploy the project to an iPad using iTunes

1) Unzip the built project to a folder.

2) Copy the folder into another folder named “Local”.

3) Connect the iPad to your computer using a USB cable.

4) Open iTunes and find the EcoStruxure Augmented Operator Advisor


App in the File sharing option under Settings.

58 Chapter 4 – Points of Interest


Exercise –Deploying a Project with iTunes

5) Drag and drop the Local folder from your computer to iTunes.

6) Replace the existing Local folder in the App.

Chapter 4 – Points of Interest 59


Summary
Summary This chapter explained how to build and deploy a project.

Questions The following questions are used to check understanding.


 What are the 2 options to deploy the project?

 Which devices can be used as the Runtime Device?

60 Chapter 4 – Points of Interest


Chapter 6: The App

Overview
Introduction This chapter will show how to use the app.
By the completion of this chapter the student will be able to:
 Run a project with the EcoStruxure™ Augmented Operator
Advisor App
 Get familiar with EcoStruxure™ Augmented Operator Advisor
App main features

Contents
What is EcoStruxure™ Augmented Operator Advisor App ................... 62
Using the App ........................................................................................ 63
App’s User Interface.............................................................................. 64

Chapter 6 – The App 61


What is EcoStruxure™ Augmented Operator Advisor App
Overview The EcoStruxure™ Augmented Operator Advisor App runs on a tablet. It
views reality (what the operator sees using the camera integrated in the
tablet) and compares it to photographs of scenes stored in the
EcoStruxure™ Augmented Operator Advisor project. The scene being
viewed by the camera and the stored image must match for detection to
occur. User interface controls and points of interest associated with the
scene are then displayed:

Downloading the The EcoStruxure™ Augmented Operator Advisor App must be


App downloaded from your tablet’s store. It is free of charge and does not
require a license.

App Compatibility The EcoStruxureTM Augmented Operator Advisor App is compatible with
iPads running iOS 11.2.5 or greater and Android tablets running OS V5.0
or greater.
The EcoStruxure™ Augmented Operator Advisor App does not require a
license. It is available from the application store of the tablet.

62 Chapter 6 – The App


Using the App
Preferred Sites After launching the EcoStruxure™ Augmented Operator Advisor App, the
Welcome Page Preferred sites page appears. You will be prompted to choose between
running local projects stored in the tablet, running a project from a remote
runtime device or running sample projects.

Scene Recognition When you run a project with EcoStruxure™ Augmented Operator Advisor
App, it will trigger the algorithm to compare the perceived reality through
the tablet’s camera with the image database deployed to tablet or Run
Time device.
Once image recognition is achieved, a few of the app’s features can be
used.

Chapter 6 – The App 63


App’s User Interface
Features On the left bottom corner of the user interface, these are the features
Overview available:

Access to configured Procedures.

Used to customize Points of Interest’s look and feel on


the screen. The dialog box below is opened to modify its
theme and font size.

Used to display or hide the Favorites bar on top of the


screen by default, without having to tap on a POI to
access it.
Used to adjust the pointing circle size in the middle of
the screen. The larger the circle, the more POIs will be
opened on the screen.

64 Chapter 6 – The App


App’s User Interface (cont.)
Features On the right bottom corner of the user interface, these are the features
Overview available:

Tap to freeze the display, allowing the operator to put


down or lower the tablet and still see the scene and
points of interest. Project actions continue and the
Favorites bar continues to be updated.

Tap to add a note on the scene. This can be used, for


example, to leave information for other operators.

Tap to display the content of the point of interest


contained in the adjustment circle in the middle of the
screen.

Chapter 6 – The App 65


Exercise – Run the Local Project

Learning
By the completion of this exercise you will be able to:
Outcome
 Run a local EcoStruxure™ Augmented Operator Advisor project.

1) After launching the app, select Local projects.

2) You will be prompted for login, but since we have not yet created any
users, just click on Login without filling any username and password.

3) Under your project folder, navigate down to the Scenes section and
tap Start.

4) Open the Water_Plant.png image file given by your instructor on


your computer.

5) With the App’s User Interface started on your tablet, now you can
scan the image in your computer screen.

6) Experiment the different features available and get familiar with them.

7) Test the POIs created on the exercise from Chapter 5.

66 Chapter 6 – The App


Summary
Summary This chapter presented the EcoStruxureTM Augmented Operator Advisor
App and its functionalities.

Questions The following questions are used to check understanding.


 Which tablet models supports the App?

 Does the App require a license?

Chapter 6 – The App 67


Chapter 7: Subscenes

Overview
Introduction This chapter will cover the subscene part of a project.
By the completion of this chapter the student will be able to:
 Understand what is a subscene
 Implement a subscene within a project

Contents
What is a Subscene? ............................................................................. 70

Chapter 7 – Subscenes 69
What is a Subscene?
Overview A subscene is a type of point of interest that appears in a scene. When the
operator clicks the subscene point of interest on the tablet, an image is
displayed. This image could be, for example, an electrical cabinet with the
doors opened, or a zoomed-in view of part of the scene.
Points of interest can be added to a subscene in the same way as for a
scene.
Only one Subscene level is possible to be configured. Nested subscenes is
not possible.

How to Create a
Subscene
Under the Scenes tab, drag and drop the subscene icon on the
main scene.

70 Chapter 7 – Subscenes
Exercise – Create a Subscene

Learning
By the completion of this exercise you will be able to:
Outcome
 Add a subscene to a scene.

1) Drag and drop the subscene icon to the scene where you
want the POI to appear in the image.

2) Fill in the information on the right sidebar as per the example below
and click Validate. Use the Pump_house.jpg image file provided with
the course resources.

Chapter 7 – Subscenes 71
Exercise – Create a Subscene

3) Reposition the subscene image on the scene and resize as needed.


Click on Validate on the right sidebar.

4) On the left sidebar you will notice the subscene has been populated
under the tree structure of your project.

5) You can now add Points of Interest to the subscene just as you do
with a scene.

6) Drag and drop the Variable POI to the subscene.

72 Chapter 7 – Subscenes
Exercise – Create a Subscene

7) Fill in the information on the right sidebar . Make sure the Create
variable checkbox is selected so a new variable will be created.
Validate after filling the info as below.

Note:
The variable name must match the exact Folder name and Variable Name
from EcoStruxure Operator Terminal Expert or Pro-face Blue variables.

8) Now drag and drop an image POI to the subscene.

Chapter 7 – Subscenes 73
Exercise – Create a Subscene

9) Fill in its details on the right sidebar as below. Use the image file
Pump_Maintence.jpg provided with this course resources.
Uncheck the Show checkbox.

Note:
In the next chapter, you will implement a trigger to change the visibility of
this object on the tablet screen.

74 Chapter 7 – Subscenes
Exercise – Create a Subscene

10) Resize and adjust the image position on the subscene so it looks like
the example below and validate it.

Note:
Use the Zoom In / Zoom Out function on the Project Toolbar
menu to help with positioning the image properly on the screen.

Chapter 7 – Subscenes 75
Summary
Summary This chapter explained what is a subscene and how to configure it.

Questions The following questions are used to check understanding.


 Is it possible to have a subscene without a scene?

 Is it possible to have nested subscenes?

 How can a subscene be used to make operation of an electrical


cabinet not only more effective, but also safer?

76 Chapter 7 – Subscenes
Chapter 8: Triggers

Overview
Introduction This chapter will cover the Triggers feature.
By the completion of this chapter the student will be able to:
 Create a new Trigger
 Deploy the project to a Run Time device

Contents
What is a Trigger? ................................................................................. 78

Chapter 8 – Triggers 77
What is a Trigger?
Overview A trigger contains one or more trigger conditions. A trigger condition
compares the current value of a variable to a specific value according to a
test condition and performs a specified action if the test condition is true.
For example, a point of interest could start flashing on the tablet whenever
the value of the Speed variable is less than 100.

Trigger Condition A trigger condition is a test performed on the value of a variable. If the
test result is “true”, the trigger actions are performed. If the test result is
“false”, nothing happens and the next trigger condition in the list is
evaluated.

How to Create a The Triggers menu is located under the project tree structure on the left
Trigger sidebar. After clicking on it, you can add a new trigger from the main
window.

78 Chapter 8 – Triggers
Exercise – Creating a Trigger

Learning
By the completion of this exercise you will be able to:
Outcome
 Create a trigger.

1) Click on Triggers on the left sidebar.

2) On the main window, click the “+” sign to create a new Trigger.

3) Fill in the information on the right sidebar as example below. You will
use an existing variable, so you can select it from the dropdown list
and leave the “Create Variable” checkbox unchecked. Validate.

4) On the main window, the new trigger will appear. Click on the
icon to configure its properties.

Chapter 8 – Triggers 79
Exercise – Creating a Trigger

5) Click on to add a new trigger condition and fill the right sidebar
as below:

6) Add a new Action and configure it as below:

7) That means the Image POI will blink on the screen once the variable
reaches a value greater or equal than 40.

8) Now you must configure it to stop blinking once the value is lower
than 40. Add a new Condition using the Else operator:

80 Chapter 8 – Triggers
Exercise – Creating a Trigger

9) Add two new actions as you did in previous steps, this time choosing
Change visibility to Hide and Stop Blinking action types. The end
result in your main window should look like the example below:

Chapter 8 – Triggers 81
Summary
Summary This chapter explained what is a Trigger, how to configure it and how it is
executed.

Questions The following questions are used to check understanding.


 What is the Trigger function mainly used for?

 What happens if a Trigger condition is False?

 What happens if a trigger condition is True?

82 Chapter 8 – Triggers
Chapter 9: The Runtime Software

Overview
Introduction This chapter will go through the steps required to run a project in a
Windows based HMI with the EcoStruxure™ Augmented Operator
Advisor Runtime service.
By the completion of this chapter the student will be able to:
 Understand the requirements of running a project in the Runtime
device
 Deploy the project to a Run Time device

Contents
What is EcoStruxure™ Augmented Operator Advisor Runtime ............ 84
EcoStruxure™ Augmented Operator Advisor Manager ........................ 85
Node-RED .............................................................................................. 86
EcoStruxure Operator Terminal Expert................................................. 87

Chapter 9 – The Runtime Software 83


What is EcoStruxure™ Augmented Operator Advisor Runtime
Overview EcoStruxure™ Augmented Operator Advisor Runtime Software is
installed on a Magelis HMIG5U / HMIG5U2 Open Box PC, Magelis iPC /
Industrial PC, or a standard Windows PC. It gathers together all the
information to be displayed in the points of interest and sends it to the
tablet to be displayed by the EcoStruxure™ Augmented Operator Advisor
App. It also manages the database containing the photographs of scenes
and equipment contained in the uploaded project.

Where to Find it? The EcoStruxure™ Augmented Operator Advisor Runtime Software can
be downloaded from Schneider Electric website ( http://www.schneider-
electric.com ).

Click on in the
EcoStruxure™ Augmented Operator Advisor page of Schneider Electric
website to download.

Firewall Rule It is important to configure the Windows Firewall in the HMI to allow an
Prerequisite incoming connection on port 80 (HTTP), and alternatively on port 443 (if
using HTTPS connection).

EWF Disable On Windows embedded systems, EWF (Enhanced Write Filter) protection
Prerequisite feature must be disabled before Runtime installation. It is also highly
recommended to keep EWF disabled when operating runtime, as any
database modification (notes, procedure history, user configuration) might
be lost on system reboot if EWF remains enabled.
This can be found on the Windows taskbar in the HMI / iPC being used as
Runtime device.

84 Chapter 9 – The Runtime Software


EcoStruxure™ Augmented Operator Advisor Manager
Overview The EcoStruxure™ Augmented Operator Advisor Manager is included in
the EcoStruxure™ Augmented Operator Advisor Runtime Software. The
EcoStruxure™ Augmented Operator Advisor Manager gets installed
automatically when the EcoStruxure™ Augmented Operator Advisor
Runtime software is installed.

Features Using the EcoStruxure Augmented Operator Advisor Manager software


you can:
 Manage user’s credentials (runtime, manager and operators) and
view a list of all users (added, deleted or modified) in Users tab.
 View a list of all procedures that are completed by operators in the
EcoStruxure™ Augmented Operator Advisor App in Procedures
tab.
 View a list of all notes in scenes that are created by operators in
the EcoStruxure™ Augmented Operator Advisor App in Notes
tab.
 Activate and edit Node-RED in Node-RED tab.

Chapter 9 – The Runtime Software 85


Node-RED
Overview Node-RED is also included in the EcoStruxure™ Augmented Operator
Advisor Runtime software.
Node-RED is an open source web browser based visual tool for wiring the
Internet of Things. It uses nodes which encapsulates code, and these nodes
are wired together to create flows. The idea of Node-RED is to make
programming easier, with several hundred nodes available online and
people from all around the around contributing to expand this library of
community nodes.
Node-RED is a component of EcoStruxure™ Augmented Operator
Advisor Runtime. It is used to allow transparent connectivity between the
EcoStruxure™ Augmented Operator Advisor Runtime and data sourced
from different devices at OT level and from software’s and systems at IT
level such as SQL data bases.
It allows the IT / OT convergence in accordance to Schneider Electric’s
EcoStruxure™ architecture.
Node-RED is not covered in detail on this course. It is a topic of a separate
training course.

Note:
Node-RED is pre-installed with EcoStruxure™ Augmented Operator
Advisor Runtime. If the user decides to update or change its version, or
add new nodes, Schneider does not guarantee its functionality, neither
support.

See Also:
For more information on Node RED, visit its web page on
https://nodered.org/

86 Chapter 9 – The Runtime Software


EcoStruxure Operator Terminal Expert / BLUE
Overview EcoStruxure Operator Terminal Expert / Blue is the latest software for
Magelis HMIs. The most straight forward way to display a variable to the
operator in the tablet’s screen is by sourcing it from a EcoStruxure
Operator Terminal Expert Runtime application running in the same HMI
where EcoStruxure™ Augmented Operator Advisor Runtime software is
running.

Prerequisite In order to enable the interaction between EcoStruxure Operator Terminal


Expert / Blue Runtime and EcoStruxure™ Augmented Operator Advisor
Runtime a driver must be added in the HMI Project.

Once the project is downloaded to the HMI with this driver configured,
the HMI will be ready to serve the EcoStruxure™ Augmented Operator
Advisor Runtime software with its variables data.

Chapter 9 – The Runtime Software 87


Exercise – Launch the Runtime and Check Ports

Learning
By the completion of this exercise you will be able to:
Outcome
 Check if all necessary ports are open.

1) Launch the Runtime software from the Windows Start menu in the
HMI.

2) Click on Settings on the left sidebar.

3) You will see a list of the ports in their default settings. It is possible to
change port numbers in case there are conflicts with existing
software blocking these ports.

4) Click on Check to diagnose whether these ports are available to be


used in the HMI.

5) Discuss the results with your instructor.

Note:
If port 80 is not open, you may need to create a firewall rule in Windows
firewall settings in order to open this port.

Port 8082 is only open if you have a valid license or if your trial period is
still valid.

88 Chapter 9 – The Runtime Software


Exercise – Deploy the Project to the Runtime

Learning
In Chapter 6 you have learned how to deploy and run a project locally in
Outcome
the tablet. In this exercise you will see the difference in deploying it to a
Runtime device. By the completion of this exercise you will be able to:

 Deploy your project to a Runtime device.

1) Install the Runtime component on your PC. The installation files


are given with the course resources.

If you are using a HMI as the Runtime device, check its date and
time is correct before installing the Runtime. If the date and time is
in the past, the trial period of the Runtime will not be activated.

2) Open the project M340a.vxdz in EcoStruxure Operator Terminal


Expert. This project is provided with the course resources.

3) From the toolbar in EcoStruxure Operator Terminal Expert, Simulate


the project.

4) Download the project M340a.sta to the Unity Pro simulator and run
it. This project file is provided with the course resources.

5) Build the Augmented Operator project in the Builder and Download


it to your computer.

Chapter 9 – The Runtime Software 89


Exercise – Deploy the Project to the Runtime

6) The .zip file is saved on the Downloads folder of Windows. Extract


the .zip file to a folder.

7) Open the Extracted folder and copy all its contents to the Project
folder in the Runtime device, which is your PC or the HMI (if you
have one). The project folder can be reached from the Windows Start
Menu, as shown below:

Extracted Folder Runtime Device

8) Launch the app on your tablet. Select Custom connection and enter
the IP address of the Runtime device (PC or HMI). The default
credentials are:

User name: admin


Password: admin

Remember to set up a firewall rule in your Runtime device to allow an


inbound connection to port 80.

Note:
In case you are using a PC as the Runtime Device, in order to create a
point-to-point network between the tablet and the PC hosting the Runtime,
the mobile hotspot feature will need to be used on the PC, unless both
devices (tablet and PC) are on the same wi-fi network.

90 Chapter 9 – The Runtime Software


Exercise – Deploy the Project to the Runtime

9) Start the project’s scene and perform image recognition with your
tablet. Observe its behaviour.

Chapter 9 – The Runtime Software 91


Summary
Summary This chapter explained what is a subscene and how to configure it.

Questions The following questions are used to check understanding.


 Which devices can be used as Runtime Device?

 Which port number is used on the Runtime Device to establish


communication with the tablet?

 What are the other two components installed with the Runtime
Software?

92 Chapter 9 – The Runtime Software


Chapter 10: Procedures

Overview
Introduction This chapter will go through the steps required to run a project in a
Windows based HMI with the EcoStruxure™ Augmented Operator
Advisor Runtime service.
By the completion of this chapter the student will be able to:
 Understand the requirements of running a project in the Runtime
device
 Deploy the project to a Run Time device

Contents
What are Procedures? .......................................................................... 94
Procedure Limitations ........................................................................... 96

Chapter 10 – Procedures 93
What are Procedures?
Overview Procedures are a set of instructions that the operator must perform. The
instructions are displayed on the tablet screen to guide the operator
through every step needed to perform a set of tasks related to maintenance
or operation.

Steps A procedure always includes:


 A Start step. This is the entry point for the procedure and is
always the first step executed.
 A Stop step. This is the exit point for the procedure and is always
the last step executed.
You must then add at least one of the following:
 Standard steps, such as Step 1 and Step 2 in the example below.
 Multichoice steps, with 2 possible choices, such as Is machine
running? in the example below. The operator is shown a question
and two options. The next step depends on the choice the operator
makes.
 Subprocedures. A previously created procedure that is executed
within another procedure.
The order in which these steps are executed is determined by the links
connecting the steps.

Expert Mode User It is possible to give a user the rights to skip steps during a procedure. This
is done by checking the Expert Mode when creating users in the Manager.

94 Chapter 9 – The Runtime Software


What are Procedures? (cont.)
How to Create Procedures are created under the Procedures tab in the Builder.
Procedures With a project area highlighted on the left sidebar, you can click on

icon in the Toolbar to start creating a new procedure.

Alternatively, you can click on the sign on the main window.

Chapter 10 – Procedures 95
Procedure Limitations
Maximum The following describes the maximum number of procedures that can be
Numbers of used in a EcoStruxure™ Augmented Operator Advisor project.
Procedures The number of elements that can be added may also be limited by the
License Requirements.

Element Maximum Number


Procedures in a project 100

Sub procedures in a project 10(1)


Branches in a procedure 10
Steps in a procedure 100
Characters in labels / names 50
Characters in descriptions 255
(1) Sub procedures cannot contain nested sub procedures.

96 Chapter 10 – Procedures
Exercise – Create a Procedure

Learning
By the completion of this exercise you will be able to:
Outcome
 Create Procedures

1) Click on the Procedures Tab, highlight Area1 on the left sidebar and

click on the icon.

2) On the right sidebar, give this Procedure a label and optionally, a


description. Validate.

3) The new procedure will be populated on the main window, click on

the icon to configure it.

4) Under the Scenes tab on the left sidebar, highlight Area1.

5) Click on Scene icon on the Project toolbar

6) On the right sidebar, untick the “Create new scene” checkbox.

7) Select the existing Pump_house scene and Validate.

Chapter 10 – Procedures 97
Exercise – Create a Procedure

8) Make it will be populated under the Scenes tab on the left sidebar, as
below.

9) Highlight the Check Tank procedure on the left sidebar.

10) Drag and drop a Standard step from the Toolbar to the
main window.

11) Fill in the information requested on the right sidebar as per example
below and Validate.

Use the Pump_document.pdf file given with the course resources.

98 Chapter 10 – Procedures
Exercise – Create a Procedure

12) The Standard Step will now appear as on the main


Window.

13) Move the mouse pointer over the Step 1 box to find a dot, click and
hold it to create a link with the Start step. It will look like this:

14) Create a second standard step with the below configuration and link
it to the first step.

Chapter 10 – Procedures 99
Exercise – Create a Procedure

15) Now create a Multiple Choice step. Drag and drop the
icon to the main window and configure it as below:

16) Now create Step 3 and configure it as below:

You will attach the same document attached in Step 1, but this time
you only need to select it from the drop down list.

100 Chapter 10 – Procedures


Exercise – Create a Procedure

17) Finish linking the steps together like the example below:

18) Highlight Step 1 and you will see down the bottom right of the main
window a reference to the attached scene. Click on it.

19) Drag and drop a List POI to the scene and name it Procedures.
Validate it. This List POI is now associated with Step 1 and it will be
shown on the tablet screen when this Step is active.

20) Exit the AR configuration and select the


Procedures List POI from the left sidebar menu under the Scenes tree
structure.

Chapter 10 – Procedures 101


Exercise – Create a Procedure

21) Add a Text item to the List POI and name it Pump is ON.

22) Deploy the project to the Runtime device and run it on your tablet.

23) Click on the Procedures icon on the bottom left menu of the
app.

24) From the displayed menu above, select the Check tank procedure to
start it.

25) You will need to have the Pump_house.jpg image file on your laptop
screen for the app to recognize it.

26) Once the procedure is started and the scene is recognized, notice
the List POI associated with Step 1 is on the screen as well as the Step
1 itself. You can open the attached document User Manual, and you
can proceed to the next step.

102 Chapter 10 – Procedures


Exercise – Create a Procedure

27) Proceed to Step 2. Tap on

28) Proceed until you reach the end of the procedure. A summary will be
displayed. Tap on Save.

29) Go back to the App’s welcome page and navigate to the Procedures
option under your project Area1.

30) Tap on the icon and examine the window displayed. You
can check the procedure’s history, for example.

Chapter 10 – Procedures 103


Summary
Summary This chapter explained what is a Procedure and how to configure and
execute it.

Questions The following questions are used to check understanding.


 What type of steps can be configured in a Procedure?

 If a user is configured with Expert Mode attribute, what does it


mean?

 How many steps can be configured in a Procedure?

104 Chapter 10 – Procedures


Chapter 11: The Manager

Overview
Introduction The EcoStruxure™ Augmented Operator Advisor Manager is included in
the EcoStruxure™ Augmented Operator Advisor Runtime Software. The
EcoStruxure™ Augmented Operator Advisor Manager gets installed
automatically when the EcoStruxure™ Augmented Operator Advisor
Runtime software is installed.
This chapter will cover the main features of the EcoStruxure™
Augmented Operator Advisor Manager.
By the completion of this chapter the student will be able to:
 Access the Manager and use its functions

Contents
What is the Manager? ......................................................................... 106
Users Management ............................................................................. 107
Node-RED Management ..................................................................... 108
Notes Management ............................................................................ 109
Procedures Management.................................................................... 110

Chapter 11 – The Manager 105


What is the Manager?
Overview The EcoStruxure™ Augmented Operator Advisor Manager is included in
the EcoStruxure™ Augmented Operator Advisor Runtime Software. The
EcoStruxure™ Augmented Operator Advisor Manager gets installed
automatically when the EcoStruxure™ Augmented Operator Advisor
Runtime software is installed.

How to launch it The EcoStruxure™ Augmented Operator Advisor Manager is accessed


from the Windows Start menu in the HMI with Runtime software
installed.

Open the EcoStruxure™ Augmented Operator Advisor runtime software


and click AOA Manager at the bottom of the left sidebar.
Login using default credentials.
Username: admin
Password: admin
This will launch the User Interface for the Manager.

Features Using the EcoStruxure™ Augmented Operator Advisor Manager software


you can:

 Manage user’s credentials (runtime, manager and operators) and


view a list of all users (added, deleted or modified) in Users tab.
 View a list of all procedures that are completed by operators in the
EcoStruxure™ Augmented Operator Advisor App in Procedures
tab.
 View a list of all notes in scenes that are created by operators in
the EcoStruxure™ Augmented Operator Advisor App in Notes
tab.
 Activate and edit Node-RED in Node-RED tab.

106 Chapter 11 – The Manager


Users Management
Overview There are three types of users of EcoStruxure™ Augmented Operator
Advisor:
 Builder users
 Runtime/Manager users
 Operator users
Builder users are those who will access EcoStruxure™ Augmented
Operator Advisor Builder. Users without credentials can sign-up in the
builder login screen.
Runtime/Manager users are those who will manage the EcoStruxure™
Augmented Operator Advisor runtime software device. They are
responsible for configuring users and security settings using the
EcoStruxure™ Augmented Operator Advisor Manager Runtime users
belong to Admin group.
Operator users are those who will use the EcoStruxure™ Augmented
Operator Advisor App. Based on their user rights (security level) they will
have access to different objects in the app. In the builder project, objects
are attributed with a security level numerical value. Operator users with a
security level numerical value equal to or greater than the objects security
level may view and interact with the object. Otherwise, the object is
hidden or not accessible.

Note:
 The EcoStruxure™ Augmented Operator Advisor Manager can
manage (create / modify / delete) the users (Runtime and Operator)
only. You cannot create Builder users.
 You cannot delete Admin group.

Chapter 11 – The Manager 107


Node-RED Management
Overview You can manage the Node-RED engine activation (launch or stop) and set
it up to start automatically or manually. You can also launch the Node-
RED editor to modify Node-RED parameters.

108 Chapter 11 – The Manager


Notes Management
Overview In this tab, you can see the list of all Notes created by the operators in the
App. Operators can create their own Notes in scenes or subscenes when
running the App on their tablet device. The information entered by the
operators into the Notes may be of use in order to make improvements to
your project.

Chapter 11 – The Manager 109


Procedures Management
Overview In this tab, you can see a report of all procedures followed by the
operators in the EcoStruxure™ Augmented Operator Advisor App. You
can see the name and version of the project, as well as the date and time
that the procedures were followed, the individual steps taken, and the
name of the user who followed the procedure.

110 Chapter 11 – The Manager


Exercise – Access the Manager

Learning
By the completion of this exercise you will be able to:
Outcome
 Understand the different options within the Manager.

1) Launch the Augmented Operator Runtime Software and click on AOA


Manager on the bottom of the left sidebar.

2) Login with username: admin and password: admin

3) Under the Users tab, click on Add Group.

4) Give this Group a name and a Security Level of Access such as the
example below and Validate.

5) Click on Add User.

6) Give the User a name, a password, and select the Group it will belong
to. Validate.

Expert mode is marked when you want the user to have the rights to
skip steps on Procedures.

Chapter 11 – The Manager 111


Exercise – Access the Manager

7) Now click on Node-RED tab. If not already started, click on Start to


start the Node-RED engine.

8) Click on the Procedures tab. Notice the Procedure from previous


exercise will be listed in the main window. Examine this information.

112 Chapter 11 – The Manager


Summary
Summary This chapter explained what is the Manager and how to access and use it.

Questions The following questions are used to check understanding.


 Where is the Manager accessed from?

 What are the main features to be managed in the Manager?

 When creating a new group of users, what is the Security Level


attribute used for?

Chapter 11 – The Manager 113


Chapter 12: Image Recognition and Tracking

Overview
Introduction There are best practices and guidelines recommended to be followed in
order to maximize the efficiency of image recognition and tracking when
using the EcoStruxure™ Augmented Operator Advisor App. This chapter
will cover this topic.
By the completion of this chapter the student will be able to:
 Understand the prerequisites for image recognition
 Understand the differences between image and tag recognition
 Identify when each method is preferred

Contents
Image or Tags, Which One to Use? ..................................................... 116
Guidelines for Taking Photographs ..................................................... 119
Image Limitations ................................................................................ 120

Chapter 12 – Image Recognition and Tracking 115


Image or Tags, Which One to Use?
Overview When implementing an EcoStruxure™ Augmented Operator Advisor
project, the first step is to determine whether to use photographs or tags to
identify a scene.
If you are using photographs, the next step is to take high-resolution
photographs of all the areas, scenes, and subscenes.
Store the photographs on a hard disk drive, USB key, or network folder
accessible from the device on which EcoStruxure™ Augmented Operator
Advisor Builder is used.
If you are using tags, the next step is to uniquely identify physically
similar or identical items of equipment and attach tags to them.

How to Choose? Depending on the nature of the equipment being imaged and its
surrounding environment, the following detection methods can be utilized
for the EcoStruxure™ Augmented Operator Advisor App:
 Image recognition: compares the complete image used in the
EcoStruxure™ Augmented Operator Advisor Builder scene with
the real scene.
 Tag recognition: used when image recognition is unsuccessful.
The EcoStruxure™ Augmented Operator Advisor App searches
the real scene for a known, uniquely identifiable tag. These tags
are built into the EcoStruxure™ Augmented Operator Advisor
Builder library.
There are a number of criteria that needs to be considered when deciding
if image or tag recognition should be used.
First, you should consider the environment. Is the environment stable or
unstable one?
Stability in this case refers to the equipment in the scene as viewed from a
particular viewpoint.
As we require a close match between the real scene and photo reference, a
big object added to or missing from the real scene can impact the
detection. Generally, image and reality need to match at around 90% in
order to achieve recognition.
Another consideration is the detail in the image. Does the equipment have
some identifiable and unique detail features in the scene that enable it to
be both recognizable and correctly identified?
Tag recognition should be considered in cases where there are few details,
or if the equipment in the scene is similar to other scenes or objects.

CAUTION
FALSE SCENE DETECTION
• Use tags in situations where similar equipment and/or
environments could result in false scene detection.
• Attach tags firmly to the equipment with sufficient protection
against damage or deterioration of the tag.
Failure to follow these instructions can result in injury or
equipment damage.

116 Chapter 12 – Image Recognition and Tracking


Image or Tags, Which One to Use? (cont.)
Image Recognition In the image recognition method, the real image of the equipment is
compared with the photo reference in the scene of the EcoStruxure™
Augmented Operator Advisor project.
Image recognition is suitable in the following instances:
 When the equipment being imaged and its environment are stable,
which means:
• The equipment being imaged does not change.
• There is no movement of other objects or people around
the equipment being imaged.
 When the equipment being imaged has some unique identifiable
details that can enable correct identification.
 When the equipment or scene is significantly different from other
objects or scenes in a project.
You can also take additional photos of parts of equipment to improve
detection in the following instances:
 When the equipment being imaged is moving or in a changing
environment.
 When there are people or other objects moving in the camera’s
field of view.

Tag Recognition In the tag recognition method, detection is made possible by attaching tags
to the equipment being imaged. The tag is a uniquely identifiable object
that is placed in the scene to enable detection.
Tag recognition method is suitable in the following circumstances:
 When image recognition or partial-image recognition methods do
not yield satisfactory results.
 When some of the scenes or equipment being imaged may not be
easily identified. For example, in a row of machines, tags must be
printed and attached to the machines to assist EcoStruxure™
Augmented Operator Advisor to uniquely identify the scene and
display the appropriate points of interest.
 When the equipment is in a poorly or very brightly lit area that
prevents taking high-contrast photographs.

Chapter 12 – Image Recognition and Tracking 117


Image or Tags, Which One to Use? (cont.)
Where to find the The tags can be downloaded from the Help menu in the EcoStruxureTM
tags? Augmented Operator Advisor Builder.

Note:
 The original file type of a tag is .ppm (Portable Pixmap). Before using
it in the Builder, it must be converted to either a .png or .jpg file type.

118 Chapter 12 – Image Recognition and Tracking


Guidelines for Taking Photographs
Overview The following guidelines apply to photographs:
 Take photographs using the integrated camera of the tablet. This
means that the optical aspect of the objects should be similar
when detection will take place. Detection generally works better if
the detection point is at the same point or further away than the
original image. If the detection point is closer, then parts of the
detection are removed from the field-of-view and we reduce the
match criteria.
 The equipment in the image should contain a number of
identifiable and unique objects.
 Refer to Limitations for information on the image resolution. The
higher the resolution, the better the image recognition.
 On identical items of equipment, attach tags before taking the
photograph.
 Take multiple photographs of the scene with a 30° horizontal
angle of separation to improve the match criteria.
 Fill the scene with the primary object. This includes taking close
up views of sub-assemblies or components of the equipment.
 Take photos at 1.6 m height. Take additional images if the user
will be viewing the scene from lower down or higher up.
 The more detail, contrast, and sharp changes in color are available
on the photo, the more successful detection will be.

Note:
More photos will lead to better detection and tracking, especially with
varying light conditions. A minimum of 300 lux is recommended.

Chapter 12 – Image Recognition and Tracking 119


Image Limitations
Image File Type The following describes the limitations of image files to be used in scenes
and Resolution and subscenes.

Element Maximum Number


Image Resolution 4096 x 4096 pixels max

Image File PNG or JPG, 2MB max

120 Chapter 12 – Image Recognition and Tracking


Exercise – Take Photographs and Create a New Scene

Learning
By the completion of this exercise you will be able to:
Outcome
 Take photographs and use them as scenes in your project.

1) Discuss with your instructor to use any object available in your


classroom to be used as a scene in the project.

2) Take photographs of the object.

3) Add these images to your project.

4) Place a couple POIs and test the results.

Chapter 12 – Image Recognition and Tracking 121


Summary
Summary This chapter explained what is the Manager and how to access and use it.

Questions The following questions are used to check understanding.


 What file types are allowed for the image files?

 Why is Tagging an alternative to be used for Image Recognition?

 Where are the tags catalog downloaded from?

122 Chapter 12 – Image Recognition and Tracking


Appendix A: Node-RED

Overview
Introduction This chapter will give a brief introduction on Node-RED.
By the completion of this chapter the student will be able to:
 Understand why Node-RED is used as part of the Augmented
Operator solution.

Contents
What is Node-RED? ............................................................................. 124

Appendix A: Node-RED 123


What is Node-RED?
Overview Node-Red is an open source visual editor that allows programmers of any
level to interconnect physical IO, cloud based systems, databases and most
APIs in any combination possible, wiring the Internet of Things.
It uses nodes which encapsulates code, and these nodes are wired together
to create flows. The idea of Node-RED is to make programming easier,
with several hundred nodes available online and people from all around
the around contributing to expand this library of community nodes.
Node-RED is a component of EcoStruxure™ Augmented Operator
Advisor Runtime. It is used to allow transparent connectivity between the
EcoStruxure™ Augmented Operator Advisor Runtime and data sourced
from different devices at OT level and from software’s and systems at IT
level such as SQL data bases.
It allows the IT / OT convergence in accordance to Schneider Electric’s
EcoStruxure™ architecture.
Node-RED is not covered in detail on this course. It is a topic of a separate
training course.

See Also:
For more information on Node RED, visit its web page on
https://nodered.org/

User Interface The editor window consists of four components:


• The header at the top, containing the deploy button, main menu,
and, if user authentication is enabled, the user menu.
• The palette on the left, containing the nodes available to use.
• The main workspace in the middle, where flows are created.
• The sidebar on the right with the info and debug panels.

124 Appendix A: Node-RED


Nodes and Flows
Overview Node-RED comes with a core set of useful nodes, but there are a growing
number of additional nodes available for install from both the Node-RED
project as well as the wider community.
You can search for available nodes in the Node-RED library at
http://flows.nodered.org/
Schneider Electric is also developing its own nodes for specific
applications. These are not found in the Node-RED library.
A flow is a set of nodes wired together to transmit messages representing
events flowing between nodes, triggering processing that results in
outputs.

AOA Nodes When launching Node-RED from the Runtime Device, there are two
nodes that can be used to interface data with the Runtime software. These
nodes are available in the node palette.

 Augmented Operator Variable: Creates a well formatted


AOAvariable to be PUT or GET into/from an Augmented
Operator server.
 Augmented Operator Server: Facilitates the exchange of data with
the Augmented Operator Runtime software.

Augmented When using the Augmented Operator Variables Node, the name of the
Operator Variable variable in the node must match the internal variable in the Augmented
Operator project running in the Runtime device.

Appendix A: Node-RED 125


Exercise – Display data from Node-RED in AOA

Learning
By the completion of this exercise you will be able to:
Outcome
 Create variables in the Builder to be used with Node-RED;
 Create a flow in Node-RED;
 Deploy a flow to Node-RED.

1) From AOA Manager, start the Node-RED engine.

2) After the engine is on, launch the Node-RED editor.

3) In Node-RED, drag and drop an Inject Node from the node palette in
the canvas.

4) Double click the Inject node to configure it.

5) Select JSON as the payload and configure it as image below.

126 Appendix A: Node-RED


Exercise – Display data from Node-RED in AOA

6) Drag and drop a Split node from the node palette to the canvas.

7) The Split node’s default configuration will be used. Wire it to the


Inject node.

8) Drag and drop a Switch node from the node palette to the canvas.

9) Double click the Switch node and configure it as below.

10) Wire the output of the Split node to the input of the Switch node.

11) Drag and drop a AOA Variable node from the node palette to the
canvas.

Appendix A: Node-RED 127


Exercise – Display data from Node-RED in AOA

12) Double click on the AOA Variable node and configure it as below.

13) Wire the first output of the Switch node to the input of the AOA
Variable node.

14) Repeat steps 11, 12 and 13 to create three new AOA Variable nodes
with new variable names, the flow will look like below image.

15) Drag and drop a AOA Server node from the node palette to the
canvas.

128 Appendix A: Node-RED


Exercise – Display data from Node-RED in AOA

16) Double click the AOA Server node and configure it as below.

17) Wire the outputs of all AOA Variable nodes to the input of the AOA
Server node.

18) Click the Deploy button on the top menu.

Appendix A: Node-RED 129


Exercise – Display data from Node-RED in AOA
Learning
By the completion of this exercise you will be able to:
Outcome
 Create internal variables in the Builder
 Visualize Node-RED variables on the tablet

1) In the Builder, click the Areas tab, click on Variables under the
structure of your project’s Area.

2) Create new internal variables and make their names match the
names created in AOA variable nodes in Node-RED, as example
below:

3) Create a new List POI in a scene of your choice.

4) Configure the POI to display the values of the internal variables


coming from Node-RED, such as example below using a List POI.

130 Appendix A: Node-RED


Exercise – Display data from Node-RED in AOA

5) Build and Deploy the project to the Runtime device.

6) With the App in the tablet, perform image recognition and check the
variables coming from Node-RED.

Appendix A: Node-RED 131


Summary
Summary This appendix introduced Node-RED and explained why it is a component
of the EcoStruxureTM Augmented Operator Advisor solution.

Questions The following questions are used to check understanding.


 What is the main purpose of Node-RED?

 Where is it accessed from?

 What type of variable must be created in the AOA Builder to


allow data exchange with Node-RED?

132 Appendix A: Node-RED

You might also like