MQTT Study and Revision Workbook

© 2022 by Stephen Cope. All rights reserved.

Website: www.steves-internet-guide.com

Every precaution has been taken to ensure that the information presented in this book is accurate. However,
neither the author nor the publisher shall have any liability to any person or entity with respect to any loss or
damage caused or alleged to be caused directly or indirectly by the information contained within this work.

The information is presented on an “as is” basis, there is no warranty.

Contents
Who is this Workbook For?..................................................................................................... 3
Introduction ............................................................................................................................. 3
MQTT Versions ....................................................................................................................... 4
MQTT Topics .......................................................................................................................... 4
MQTT QOS ............................................................................................................................. 5
QOS 0 – Once ......................................................................................................................... 5
QOS 1 – At Least Once ........................................................................................................... 5
QOS 2 – Only Once ................................................................................................................. 6
Questions ................................................................................................................................ 7
Answers ................................................................................................................................... 7
Important MQTT Features (All Versions)............................................................................... 7
Clean Sessions ......................................................................................................................... 8
Retained Messages .................................................................................................................. 8
Keep Alives ............................................................................................................................. 9
Last Will and Testament .......................................................................................................... 9
Authentication ....................................................................................................................... 10
MQTT v5 New Features ....................................................................................................................... 10
Connection Features .............................................................................................................. 10
Publish Features ..................................................................................................................... 12
Subscribe ............................................................................................................................... 15
General................................................................................................................................... 16
Questions .............................................................................................................................................. 17
Answers ................................................................................................................................. 18
Who is this Workbook For?
This workbook is geared towards individuals who are studying MQTT at a university or college
as part of a course.

It will also be of use for those who want to get a quick overview of MQTT and MQTT features
without going into detail.

The only pre-requisites are a basic understanding of networking and networking protocols.

Introduction
MQTT stands for MQ Telemetry Transport but previously was known as Message Queuing
Telemetry Transport.

MQTT is a binary messaging protocol and transfers messages using a publish and subscribe
model.

This model makes it possible to send messages to 0,1 or multiple clients.

A useful analogy is TV or radio.

A TV broadcaster broadcasts a TV program using a specific channel and a viewer tunes into this
channel to view the broadcast.

There is no direct connection between the broadcaster and the viewer.

In MQTT a publisher publishes messages on a topic and a subscriber must subscribe to that topic
to view the message.

MQTT requires the use of a central Broker as shown in the diagram below:
MQTT Versions
There are two different variants of MQTT and several versions.
 MQTT v3.1.0 –
 MQTT v3.1.1 – In Common Use
 MQTT v5 – Currently Limited use
 MQTT-SN – See notes later
The original MQTT was designed in 1999 and has been in use for many years.

There is very little difference between v3.10 and 3.1.1, MQTT v3.1.1 is the version in common
use. MQTTv5 is now available (2022) on all of the main brokers and many of the clients.

MQTTv5 offers all of the functionality of MQTT v3 but with many new features.

MQTT Topics
MQTT topics are a form of addressing that allows MQTT clients to share information.

MQTT Topics are structured in a hierarchy similar to folders and files in a file system using the
forward slash ( / ) as a delimiter.

Using this system you can create a user friendly and self-descriptive naming structures of your
own choosing.

Topic names are:

 Case sensitive
 Use UTF-8 strings.
  Must consist of at least one character to be valid.

Except for the $SYS topic there is no default or standard topic structure.

That is there are no topics created on a broker by default, except for the $SYS topic.

All topics are created by a subscribing or publishing client, and they are not permanent.

A topic only exists if a client has subscribed to it, or a broker has a retained or last will
messages stored for that topic.

MQTT QOS
MQTT supports three QOS levels which are designed to ensure message delivery.

 QOS 0 – Once (not guaranteed)

 QOS 1 – At Least Once (guaranteed)
 QOS 2 – Only Once (guaranteed)

The QOS levels are a way of guaranteeing message delivery and they refer to the connection
between a broker and a client.

QOS 0 – Once
This is the fastest method and requires only 1 message. It is also the most unreliable transfer
mode.

The message is not stored on the sender, and is not acknowledged.

The message will be delivered only once, or not at all.

Once the message has been sent by the client it is deleted from the outbound message queue.

Therefore with this QOS level there is no possibility of duplicate messages.

QOS 1 – At Least Once

This level guarantees that the message will be delivered at least once, but may be delivered more
than once.
Publishing with QOS of 1 requires 2 messages.

The sender sends a message and waits for an acknowledgement (PUBACK).

If it receives an acknowledgement then it notifies the client app, and deletes the message from
the outbound queue.

If it doesn’t receive an acknowledgement it will resend the message with the DUP flag set
(Duplicate Flag).

The message will continue to be resent at regular intervals, until the sender receives an
acknowledgement.

If the message is being sent to the broker then the broker will forward that message to
subscribers even though the duplicate flag is set.

Therefore subscribers can receive the same message multiple times.

QOS 2 – Only Once

This level guarantees that the message will be delivered only once.

This is the slowest method as it requires 4 messages.

1. The sender sends a message and waits for an acknowledgement (PUBREC)

2. The receiver sends a PUBREC message
3. If the sender doesn’t receive an acknowledgement (PUBREC) it will resend the message with
the DUP flag set.
4. When the sender receives an acknowledgement message (PUBREC) it then sends a message
release message (PUBREL). The message can be deleted from the queue.
5. If the receiver doesn’t receive the (PUBREL) it will resend the PUBREC message
5. When the receiver receives the PUBREL message it can now forward the message onto any
subscribers.
6. The receiver then send a publish complete (PUBCOMP) .
7. If the sender doesn’t receive the PUBCOMP message it will resend the PUBREL message.
8. When the sender receives the PUBCOMP the process is complete and it can delete the
message from the outbound queue, and also the message state.
Introductory MQTT Video
This video takes you through the basics of MQTT.

Important Points to Note:

1. Clients do not have addresses like in email systems, and messages are not sent to clients
but are sent to a topic.
2. The job of an MQTT broker is to filter messages based on topic, and then distribute
them to subscribers.
3. A client can receive these messages by subscribing to that topic on the same broker
4. There is no direct connection between a publisher and subscriber.
5. All clients can publish (broadcast) and subscribe (receive).
6. MQTT brokers do not normally store messages.

Questions
1. The latest version of MQTT is version 4?
2. How do you assign an address to a client?
3. What are topics?
4. Is the topic house/test the same as house/TEST?
5. Do you need to start the topic structure with a forward slash (/)?
6. Is house/te* a valid wild card usage?
7. Before you can publish a message on a topic you first need to create that topic on the
broker? True or False?

Answers
1. No it is version 5. There was no version 4
2. You don’t assign addresses to clients.
3. Topics are a form of addressing that clients publish and subscribe to.
4. No topics are case sensitive
5. No and it is generally not done
6. No wild cards can only be used directly after of before the / separator.
7. False.

Important MQTT Features (All Versions)

All versions of MQTT support the following features:
 QOS levels 0,1,2
 Clean sessions
 Retain messages
 Keep Alives
  Last Will and testament
 Authentication

Clean Sessions
When a client connects to a broker it can connect using either a:
 Non-persistent connection (clean session) or a;
 Persistent connection

With a non-persistent connection the broker doesn’t store any subscription information or
undelivered messages for the client.

This mode is ideal when the client only publishes messages.

It can also connect as a durable client using a persistent connection.

In this mode the broker/server will, depending on the QOS of the published messages and
subscribing client, store subscriptions and messages for the client if it is disconnected.

Retained Messages
Normally if a publisher publishes a message to a topic, and no one is subscribed to that topic the
message is simply discarded by the broker.

However the publisher can tell the broker to keep the last message on that topic by setting the
retained message flag.

This can be very useful, as for example, if you have sensor publishing its status only when
changed e.g. Door sensor. What happens if a new subscriber subscribes to this status?

Without retained messages the subscriber would have to wait for the status to change before it
received a message.

However with the retained message the subscriber would see the current state of the sensor.

What is important to understand is that only one message is retained per topic.

The next message published on that topic replaces the last retained message for that topic.
Keep Alives
MQTT uses a TCP/IP connection.

This connection is normally left open by the client so that it can send and receive data at any
time.

If no data flows over an open connection for a certain time period then the client will generate a
PINGREQ and expect to receive a PINGRESP from the broker.

This message exchange confirms that the connection is open and working.

This period is known as the keep alive period.

This is what the specification says:

The Keep Alive is a time interval measured in seconds. Expressed as a 16-bit word, it is the
maximum 529 time interval that is permitted to elapse between the point at which the Client
finishes transmitting one 530 Control Packet and the point it starts sending the next. It is the
responsibility of the Client to ensure that 531 the interval between Control Packets being sent
does not exceed the Keep Alive value. In the absence of 532 sending any other Control Packets,
the Client MUST send a PINGREQ Packet [MQTT-3.1.2-23]. 533
534
The Client can send PINGREQ at any time, irrespective of the Keep Alive value, and use the
PINGRESP 535 to determine that the network and the Server are working.
The default keep alive period for most clients is 60 secs, but it can be set to anything you want
when you establish the client connection.

Last Will and Testament

The last will and testament message is used to notify subscribers of an unexpected shut down of
the publisher.

The basic process is.

1. The publisher tells the broker to notify all subscribers to a topic, using the last will
message, in the event that the connection breaks
2. If the broker detects a connection break it sends the last will message to all subscribers
of that topic.

The last will message is sent as part of the connection payload and is optional.
Authentication
MQTT provides support for an optional username and password as part of the connection
message.

The username/password combination is transmitted in clear text and is not secure without some
form of transport encryption.

It is the only form of security provided by the protocol itself.

MQTT v5 New Features

MQTT v5 introduced many new features and changed how some existing features work.

Most clients currently default to using MQTT v3.1. To use MQTTv5 you must set it in the
connection packet.

I’ve divided the features into sections to make it easier to determine where they apply.

Connection Features
 Clean session/start
 Client Restrictions/Limitations
 Server Restrictions/Limitations
 Will Delay Interval
 User properties
 Server Redirect

Clean session/start
This has been renamed to clean start and works basically the same as before except that we now
have a session expiry timer.

However in MQTTv5 if you set the clean start to false you need to provide a session expiry
value. If you don’t then it is set to 0 which makes it basically the same as clean sessions True.

As an example if you set the expiry time to 300 seconds.

This means that if a client disconnects and reconnects within 5 minutes with clean start=
False,QOS>1 then session state data (e.g subscribed topics, queued messages) are retained.
However, if the client disconnects and reconnects after 5 minutes with clean start= False,QOS>1
then session state data (e.g subscribed topics, queued messages) are lost, and the client must re-
subscribe to topics, and messages sent while the client was disconnected are lost.

Client Limitations/Restrictions
A client can now indicate to a server various client restrictions. It can tell the server to:
 Restrict size of messages to send to the client
 Restrict the number of QOS 1 and 2 messages

Message Size Restrictions

The client tells the server the maximum message size it is prepared to receive and any messages
larger than this are dropped by the server.

Uses: Used by low powered clients that lack the memory to store the messages. See MQTT and
Mosquitto Message size restrictions

Receive Maximum
The client tells the server to limit the number of QOS 1 and 2 messages that can be sent
concurrently.

Uses: Again used by low powered clients as QOS 1 and 2 messages mean that the client needs to
send additional acknowledgement messages.

Server Limitations/Restrictions
Providing certain functionality can strain server resources and so the server may not support
them.

It is now possible for the server to indicate this is the message exchange using the properties
field.

Example: The screen shot below shows the CONNACK message properties field From the
Mosquitto Server.
There are many other limitations that the server can indicate.

By default if the server doesn’t indicate that it doesn’t support a particular feature then it is
supported.

For example it is possible for the server not to support retain messages using the Retain
Available property, but in the screen shot above it is not there, and so the server supports retain
messages.

Will Delay Interval

This is used to specify a time delay in sending the will messages.

This prevents the will message being sent for a short connection interruption.

User Properties (Connection)

User properties are defined by the user and are a collection of key value pairs.

They can be used to send connection related properties from the Client to the Server.

They are not forwarded to the destination client.

Server Reference or Server Redirect

This is a very interesting feature as it allows the server to redirect the client to another
broker/server. From the specification

The Server uses a Server Reference in either a CONNACK or DISCONNECT packet with Reason
code of 0x9C (Use another server) or Reason Code 0x9D (Server moved) as described in section
4.13.

Publish Features
 Message Expiry
  Payload Format Indicator
 Topic aliases
 User Properties
 Request Response

Message Expiry
When a message is published with a QOS of 1 or 2 and the receiving client has:
1. Connected with clean session of False
2. Subscribed with a QOS or 1 or 2

Then the broker will hold published messages for the client if it is currently disconnected.

On MQTTv3 this was indefinite, but on v5 you can set the period that the server will hold the
message before discarding.

This is done in the publish message below is a sample Python code snippet:
client_pub.publish('test', msg_out1,qos=1,message_expiry_interval=20)

With the above setting then if the client doesn’t reconnect within 20 seconds the message is
discarded.

Payload Format Indicator

Set on the Publish Message. There are only two possibilities.
 Binary – this is the same as v3.1
 utf-8 – new to v5

Uses: Useful at the receiver as is knows if it needs to decode the message payload.

User Properties
These are defined by the user and are a collection of key value pairs.

In Javascript =object

Python = dictionary

Example:
{msg_number:1,time:100}
keys=msg_number and time

values= 1 and 100

These are useful for sending information to the destination outside of the payload.

In v3.1.1 this information was usually sent as JSON encoded data as part of the payload.

So in v3.1.1 the payload would be:

{msg_number:1,time:100,payload:payload_data}

This isn’t a problem if the payload is text data but if the payload is binary data then it will need
to be converted using Base64 encoding.

User properties can be set in all packet types.

Publish user properties are sent to the destination.

Topic Aliases
Topic aliases were introduced in MQTT-SN and are mechanism for reducing the size of
published packets by reducing the size of the topic field.

A topic alias once established can be used in place of a topic string.

So if the topic houses/house1/upstairs/light1 had an alias of 1 then this could be used in place
of the topic string when publishing messages.

Before you can use topic aliases the client must indicate the topic alias maximum number to
the server.

By default if this isn’t set then it is 0 which means topic aliases aren’t allowed by the client.

Response Topic and Request /Response Pattern

In Web Applications the client (browser) makes a request and the server responds.

MQTT uses a publish and subscribe pattern where there is no direct communication between
the sending client and the destination client/server.
Using the response topic in the publish message allows you to implement the request/response
pattern that is common in Web applications.

Subscribe
 Non local publishing
 Retained Message Control
 Subscription Identifier
 Shared Subscriptions

Non Local Publishing

In MQTTv3.1.1 if you subscribe to the same topic as you publish on then you will receive all of
the messages that you publish.

By Using the non-local option the broker will not send you any messages that you have
published.

This is a subscription option.

Python Example:
client.subscribe('org/common',no_local=True)

Retained Message Control

Retained messages still work as in 3.1.1 but subscribe options have been added to control what
the client receives.

0 -Send retained messages when client subscribes. Same as in MQTT 3.1.1

1 -Send retained messages when client subscribes. if the subscription does not currently exist.
2 – Don’t send retained messages when client subscribes.

Uses: Likely to help in implementing MQTT bridges.

Subscription Identifier
This is optional and if used it is an integer associated with the subscription and used to filter
incoming messages to the correct message handler.

However this could also be done by analysing the topic itself as in MQTTv3.1.1
Python Example:
client.subscribe('test', qos=0, subscription_identifier=1)
client.subscribe('test/#', qos=0, subscription_identifier=2)

Shared Subscriptions
The idea is to provide for client load balancing.

It is accomplished use the reserved topic $SHARE

The format is:

$SHARE/Sharename/topic

The sharename cannot be a wildcard character and must be at least 1 character.

General
 Reason Codes on All ACK Messages
 Server Disconnect

Reason Codes on All ACK Messages

In MQTTv3.1.1 there were very few indications from the server as to what went wrong during
the various stages of:
 Establishing a connection
 Publishing a Message
 Subscribing to topics

All Acknowledgement packets (except PINGRESP) carry a reason code and there are 128 codes
available.

Uses and example : Makes it much easier for an application to understand what has gone wrong
and take the appropriate action.

A publish ACK with reason code 16 means that there are no matching subscribers. Therefore
in MQTTv5 it is possible to detect if a topic is in use to some degree.

Server Disconnect
In MQQTv3.1.1 only the client could send the disconnect message.
If the server encountered problems it would simply terminate the TCP/IP session.

In MQTTv5 the server can send the client a disconnect message along with a reason code.

Questions
1. MQTTv5 now contains a new Flag called Will retain? True or False

2. MQTTv5 connect message can contain user properties? True or False

3. MQTTv5 now supports authentication methods other than username and password? True or
False

4. The authentication method property of the connect message is used to request extended
authentication? True or False

5 User properties let you insert your own Key/Value Pairs ? True or False

6 The request response is used identify the sending client name? True or False

7 In MQTTv5 the broker can inform you when you send too large a message? True or False

8. In MQTTv5 you always need to use a username and password? True or False

9. User properties are carried in the properties field? True or False

10. A client can inform a broker about the maximum packet size it will accept?

11. A client can choose not to receive retained messages? True or False

12 Shared subscriptions use a reserved topic root called $subscription? True or False

13. Clients A and B subscribe to the shared subscription $share/movies/action. A message is

published to the topic action what happens next. Choose the best answer:
a) Both client A and client B receive the message
b) Only client A or client B receives the message
c) Client A receives the message
d) Client B receives the message
14. Topic Aliases are created in.
a) The publish Message
b) The subscribe message
c) The registration message

15. Once a topic alias is created by the client the client can subscribe using the topic alias? True
or False.

16. The number of topic Aliases allowed can be restricted? True or False

17. All Acknowledgement packets carry a reason code? True or False

18 The publish ACK can carry a reason code? True or False

19. Connection user properties are sent to the subscribing client? True or False

20. The AUTH control type is a new type added in MQTTv5 ? True or False

Answers
1.True 2.True 3.True 4.True 5.True
6.False 7.True 8.False 9.True 10.True
11.True 12.False 13.b 14:a 15.False
16.True 17.True 18.True 19.False 20.True