Professional Documents
Culture Documents
Chapter 6 - IoT Communication Protocols v1.2 With Narration Text
Chapter 6 - IoT Communication Protocols v1.2 With Narration Text
1
Learning Objectives 2
2
Communication Protocols 3
• Devices communicate with each other (D2D), send data out to the IT
infrastructure (D2S). The IT infrastructure contain servers using these,
and data can be communicated between other servers (S2S) or back
to devices and people.
Slow (>1s)
Response
time
Fast (<10ms)
Source : Internet of Things and Data Analytics Handbook by Hwaiyu Geng
Devices can communicate with each other, or can send data to servers in the IT
infrastructure. The servers in IT infrastructure uses these data, and these data can be
communicated with other servers or back to devices and people.
D2D refers to devices at the lower level communicating with one another.
D2S refers to devices at lower level communicating to the Server at the IT infrastructure
level.
S2S refers to server communicating with another server.
Likewise the reverse direction for data transfer can happen.
Generally communication at D2D is fast with short delay that is normally below <10ms.
Response time starts to increase as D2S happens with delay that can be in term of
seconds.
Rules in which how data are sent at D2D, D2S, S2D or S2S are very important. These
rules are often known as the IoT communication protocols.
3
Some IoT Protocols for Connectivity 4
IoT communication protocols defines the way how data are formatted, communicated
and protected between connected devices.
And these are not all, they are many more IoT communication protocols which are not
covered here.
4
MQTT 5
• It collects data from many devices and transport that data to the IT
infrastructure. For large networks of small devices that need to be
monitored or controlled.
• It is TCP based.
It collects data from many devices and transport that data to the IT infrastructure. For
large networks of small devices that need to be monitored or controlled.
MQTT is efficient in terms of bandwidth, battery and resources, and uses a
publish/subscribe model.
It is TCP based.
5
How MQTT works? 6
• An authorised client can subscribe to any topic and receive the value
from the broker as they arrive from the publisher.
• The publisher (device) does not need to know about any of the
subscribers, which makes the management easier at the device level.
The IoT devices publish data to an MQTT broker. MQTT broker is a piece of software that
can run on any servers.
An authorised client can subscribe to any topic and receive the value from the broker as
they arrive from the publisher.
The publisher (device) does not need to know about any of the subscribers, which
makes the management easier at the device level.
The broker can support multiple concurrent connections from many devices.
6
How MQTT works cont’d 7
Source : https://www.processonline.com.au/content/software-it/article/connecting-the-iiot-947158535
From this figure, imagine you have devices on board several ships that measures
different speed on each ship.
Each device will to publish its own topic to MQTT Broker acting like a server in the
network. These device that publish its topic is know as MQTT publisher.
At the other end, user will subscribe to the topic of their interests. In this manner,
subscriber will be able to receive the update once the publish update data to the topic
via the MQTT broker.
Take note that each topic is a UTF-8 string, that the broker uses to filter messages for
each connected client. The topic consists of one or more topic levels. Each topic level is
separated by a forward slash, also know as, topic level separator.
Plus symobol represents single level wildcard and the Hex symbol represents multi level
wildcard.
To debug MQTT communications, a MQTT client can be installed on a local computer, for
example MQTTfx, the link is mqttfx.org.
7
Terminology of MQTT 8
• Broker – The broker is the software that receives all the messages
from the publishing clients and sends them to the subscribing clients.
• It holds the connection to persistent clients.
• Broker can become the bottleneck or result in a single point of failure, therefore it is
often clustered for scalability and reliability.
A MQTT Client is any publisher or subscriber that connects to the centralized broker over
a network is considered to be the client.
A client can be persistent or transient. Persistent clients maintain a session with the
broker, while transient clients are not tracked by the broker.
Clients often connect to the broker through libraries and SDKs. There are over a dozen
libraries available for C, C++, Go, Java, C#, PHP, Python, Node.js, and Arduino.
The broker is the software that receives all the messages from the publishing clients and
sends them to the subscribing clients. It holds the connection to persistent clients.
Broker can become the bottleneck or result in a single point of failure, therefore it is
often clustered for scalability and reliability.
8
Terminology of MQTT cont’d 9
A MQTT topic is an endpoint to that the clients connect. It acts as the central
distribution hub for publishing and subscribing messages. A topic is well-known location
created before the publisher and subscriber connect to the endpoint. Topics are simple,
hierarchical strings, encoded in UTF-8, delimited by a forward slash.
MQTT connection are based on TCP/IP on port no. 1883. For brokers who support
TLS/SSL, it typically use port 8883. For secure communication, the clients and the broker
rely on digital certificates. For example, AWS IoT is one of the secure implementations of
MQTT, which requires the clients to use the X.509 certificates.
9
MQTT cont’d 10
This figure shows a good overview of how many sensors and devices from its individual
network are sending & receiving MQTT messages to & fro its enterprise software. Each
sensor publish a topic while each smart phones may have subscribed to a topic to
receive that data, all these data are transferred via the MQTT broker that resides in the
enterprise main software.
MQTT messages are light-weight thus it is suitable for use cases that are less demanding
in data transfer operating in client-server model in order to maintain low power
consumption for devices.
10
DDS 11
11
DDS Protocol Stack 12
Source : The Data Distribution Service - The Communication Middleware Fabric for
Scalable and Extensible Systems-of-Systems by Angelo Corsaro and Douglas C. Schmidt
DDS stack is often based on UDP-IP at transport layer. But in some cases, it is specified to
use TCP-IP as well.
12
DDS Protocol Stack 13
subscribers. Scalable and Extensible Systems-of-Systems by Angelo Corsaro and Douglas C. Schmidt
DCPS (Data Centric Publish Subscribe) layer extends the publish-subscribe model to
address the specific needs of real-time, data-critical applications.
It allow application developers to control how communications works and how the
middleware handles resource limitations and error conditions.
DCPS is a mandatory low-level layer that enables efficient delivery of data injected by
publishers to subscribers.
13
DDS Protocol Stack 14
Source : The Data Distribution Service - The Communication Middleware Fabric for
Scalable and Extensible Systems-of-Systems by Angelo Corsaro and Douglas C. Schmidt
14
DDS Architecture for Real-Time Distribution 15
DDS is a publish and subscribe service. Data samples are transferred through the system
using a conceptual "Data Objects".
The "publication“, (the association of a Publisher and a DataWriter), send data samples
to one or more "subscription“, (the association of a DataReader and a Subscriber).
The application on the DDS producer side initiates the flow of data by writing a data
sample to the DataWriter. The DataWriter's publication publishes the data samples. The
publication sends the data samples to the associated subscriptions. Each associated
subscriber gives the received data sample to its DataReaders that are associated with
the sending DataWriter. The flow ends when the application on the subscribing side
retrieves the data from the DataReader.
Quality of Service (QoS) Policies control the flow of the data through the system. The
Topic, DataReader, DataWriter, Publisher, and Subscriber all have QoS polices. The QoS
policies of Publisher, DataWriter, and Topic control the data on the sending side. QoS
policies of Subscriber, DataReader, and Topic control the data on the receiving side.
The "Data Objects" are identified by Topics. Topics are compatible when they have the
same name, the same data type and the QoS policies are not in conflict. When a
DataReader's Topic is compatible with a DataWriter's Topic, then the "publication" and
"subscription" become associated and data is published between them.
Each DDS domain participants maintains a database of information about all the active
15
DataReaders and DataWriters that are in the same DDS domain
The global data space is an abstract concept with no real physical existence. It makes data
available where needed by dealing with network communication if necessary, and this is
what makes DDS is a broker-less publish-subscribe model.
There are many commercial DDS stack distributions, and the most popular one are from
RTI’s Connext DDS stack.
However, the DDS Standard is an open standard, and has open source implementations
available as well like OpenDDS stack.
15
DDS GDS 16
• A topic contains:
• a type
• data structure
• a unique name,
• a set of Quality of Service
(QoS) policies
• Durability, Reliability,
History, Transport Priority,
etc
Source : The Data Distribution Service - The Communication Middleware Fabric for Scalable
and Extensible Systems-of-Systems by Angelo Corsaro and Douglas C. Schmidt
• Topic instances are runtime entities for which DDS keeps track of
whether
• there are any live writers,
• the instance has appeared in the system for the first time, and
• the instance has been disposed (explicitly removed from the system).
Chapter 7, ver 1.2
Topic instances are runtime entities for which DDS keeps track of whether there are any
live writers, any instance has appeared in the system for the first time, and any instance
has been disposed (explicitly removed from the system).
16
DDS cont’d 17
17
OPC - History 18
OPC was originally termed as OLE (Object linking and Embedding) for Process Control),
but now refers as Open Platform Communications.
OPC is released in 1996 by OPC foundation, defines a series of standards and
specifications for the secure and reliable exchange of real time data in the industrial
automation space, and in other industries between Clients-to-Servers and Servers-to-
Servers.
It is widely adopted in thousands of factories all over the world to configure, and query
plant-floor servers such as Programmable Logic Controllers (PLCs).
OPC abstracts PLC specific protocols (such as Modbus, Profibus and etc.) into a
standardized interface allowing HMI-SCADA systems to interface with a server, who
would convert generic-OPC read and write requests into device-specific requests, and
vice-versa.
18
What is OPC? 19
• Client decides when and what data the server will fetch from the
underlying systems.
OPC is a Client-Server based communication which means that you have one or more
servers that waits for several Clients to make requests. Once the server gets a request it
answers to that and then goes back into wait state.
Client decides when and what data the server will fetch from the underlying systems.
19
Limitation of OPC 20
OPC relies on OLE and DCOM technology from Microsoft for process control and server-
client communication.
There is a need for it to be platform independent across various system integration,
therefore OPC Foundation released the OPC XML DA specification as an intermediate
step in 2002 to try to meet this demand. The OPC XML DA was basically a Web Service
Definition Language (WSDL) that requires a web server but its performance especially on
small embedded devices are poor. Therefore a better and more generic and
independent standard is needed.
20
OPC-UA 21
OPC Unified Architecture (OPC-UA), released in 2007, is an upgrade of the OPC protocol
that does not rely on OLE and DCOM.
It is Platform independence, Language independence, Protocol independence and Offers
better set of functions and services.
It unified all existing OPC specification in terms of their functionality and added
scalability, but also adds new security demands.
21
OPC-UA Client/Server Model 22
Source : https://www.novotek.com/uk/solutions/kepware-communication-platform/opc-and-opc-ua-explained/
Like OPC, OPC-UA is also using client and server communication model. Each OPC UA
Client and Server can be imagined as 2 layers. The application layer that handles the
session connection such as users are authenticated and authorised. The communication
layer that establishes a secure and reliable channel between the client and server.
Transport layer will specify the implementation methods so as to support the
communication layer.
22
OPC-UA Architecture 23
• Meta model
• Defines the base model
constructs and rules on how to
model.
This figure shows the OPC UA architecture, let’s take a look at the 2 major parts in OPC-
UA lower level. Namely, Transport and Metadata Model.
While OPC UA Meta data model part defines the basic model construct and rules how to
model information, the Transport layer serializes and de-serializes the data and
transmits it over the network.
Within the transport layer, data can be sent in one of the 3 methods as follows:.
1:. In the form of web services over HTTPs or SOAP. SOAP stands for Simple Access
Protocol, that defines the rules how 2 application will talk to each other over the
internet based on SOAP web services specification in XML formats.
2:. In the form of OPC UA BINARY ENCODING via TCP-IP method that serialize data using
an IEEE binary encoding standard. An encoding is a specific way of mapping a data type
to the actual data that appears on the wire. In Binary encoding, data is mapped to
compact binary data representations that demand fewer bytes. This lower use makes it
more efficient to transfer and process by embedded systems. Binary encoding is heavily
used by industrial automation systems.
3:. Or a hybrid way such that OPC UA binary data are sent via HTTPS.
23
OPC-UA Architecture cont’d 24
OPC UA Base Services in the middle of OPC UA architecture then defines a fixed set of
services was defined to access and use the exposed information. It abstract method
descriptions which are protocol independent and follows the request and response
paradigm.
24
OPC UA Base Services 25
• List of Services
Source : http://www.ascolab.com/en/technology-unified-architecture/base-services.html
Here is the list of services OPC UA services provided for reference only.
This unification of services and features is one of the essential innovations of OPC UA.
25
OPC-UA Architecture cont’d 26
Next, the OPC UA Information Model is an object oriented base model that fits many
use cases but can also be used by others, outside the OPC Foundation, to define their
mappings on top or on the real world. By this generic but flexible concept, any kind of
domain specific information and its related semantics can be mapped. Then at the
highest level are extended models defined that is uses the base information model
depending of application needs .
26
OPC-UA Stack 27
The transport layer serializes or deserializes the data and transmits it over the network
via either the binary, high performance optimized TCP protocol method or web service
based protocol that is also based on the reliable TCP-IP.
The data model layer defines the rules and base building blocks necessary to expose an
information model with OPC UA. It defines also the entry points into the address space
and base types used to build a type hierarchy. The OPC UA data model forms the base
for UA information models.
The UA Base Services are the interface between servers as supplier of an information
model and clients as consumers of that information model. The Services are defined in
an abstract manner. They use the transport mechanisms to exchange the data between
client and server.
There are built-in information models that extends from the base for specific
functionality like Data Acess, Alarms and Conditions, Historical Access or Programs at
higher layers. The companion information model specifies information models that
helps to integrate with different systems and lastly they could be vendor or
manufacturer specific information model too.
27
OPC-UA Security 28
• It also has user and role based access control for secure operation
cases. Security features like authorization, authentication, integrity,
availability, confidentiality, traceability and accountability were
included into the core architecture of OPC-UA.
Now that we have a better understanding of what is OPC-UA, let’s take a look at what
security features it offered.
OPC-UA is able to detect and protect against malformed messages, message alteration,
message flooding and message spooling, session hijacking, eavesdropping and rogue
clients and servers.
It also has user and role based access control for secure operation cases. Security
features like authorization, authentication, integrity, availability, confidentiality,
traceability and accountability were included into the core architecture of OPC-UA.
The new OPC UA technology integrated these security features at various levels, from
application layer down to transport layer.
In other words, security is for free as it is already implemented into the communication
stack itself.
28
Security at Application Layer 29
• The data access rights can be adjusted for each single node.
http://www.ascolab.com/en/technology-unified-architecture/technology-security.html
• Security is not optional, the features are parts of the specification and
the stack.
29
Security at Transport Layer 30
For each transportation method just mentioned, security are enforced accordingly as
shown in the flow of this figure.
In Web Services method, it used WS Secure Conversation which ensures the
compatibility to .NET and other SOAP implementations or via HTTPS.
HTTPS means using the standard HTTP protocol together with SSL-TLS. SSL-TLS secures
the route of transportation by using X.509 certificates.
The binary variant algorithms of WS Secure Conversation have been adopted and
converted to a binary equivalent, which is known as UA Secure Conversation and data is
transmitted directly using TCP-IP.
As for the hybrid way, the security aspect to send the UA encoded binary is enforced by
using HTTPs.
30
AMQP 31
Broker
The difference between MQTT and AMQP is that in AMQP broker is composed of
two components namely, exchange and queues. Both exchange and queues are
binded together. In this architecture, client applications that produce data are
known as producers ("publishers") while AMQP server is known as broker. Client
apps create messages which are given to broker. In the broker, messages are
routed and are queued. They are being read by consumers from the queues
where they are processed. These client applications that consumes data are
known as consumers ("subscribers").
31
AMQP cont’d 32
32
Which Protocol to choose? 33
• QoS control refers to the flexibility of data delivery control. DDS offers
such QoS control of “liveliness” in resource usage, discovery, and
even timing.
• In a large scale IoT systems like Smart City, it is very common to see
multiple communication protocols are being used.
QoS control refers to the flexibility of data delivery control. DDS offers such QoS control
of “liveliness” in resource usage, discovery, and even timing.
With collection-to-a-server design, MQTT well suited this case well.
OPC UA communicates the information about the system, its configuration, topology,
and data context (the “metadata”) but not designed for flexible device-to-device
interaction.
AMQP focuses on tracking all messages and ensuring each is delivered as intended,
regardless of failures or reboots.
In a large scale IoT systems like Smart City, it is very common to see multiple
communication protocols are being used.
33
Questions & Answers 34
34