Using MQTT and Google’s Protocol Buffers to

implement efficient RPCs

Mauricio Andrada Sep 22, 2020 · 5 min read

Introduction
Over the decades there were many attempts — some more successful than others — to
implement RPCs (Remote Procedure Calls) in a programming language and network
agnostic way.

SOAP (Simple Object Access Protocol), REST (Representational State Transfer),

CORBA (Common Object Request Broker Architecture), JSON (JavaScript Object
Notation), to name a few, are some protocols and architectures created with that
purpose in mind.

In this article, I will describe our team experience using a combination of two
relatively recent technologies to accomplish an efficient implementation of RPCs,
derived from work developed by my colleague Julie Vogelman.

MQTT and Protobufs

MQTT
MQTT was authored by Andy Stanford-Clark (IBM) and Arlen Nipper (Cirrus Link,
then Eurotech) in 1999. It was standardized in 2013 by OASIS.

MQTT defines a lightweight publish/subscribe protocol designed for resource-

constrained devices and slow network links.

MQTT uses a broker that allows devices to publish data octets to topics, as well as
subscribe to topics and receive data published to them.

Topics are strings that look like a path to a folder or file:

/this/is/an/example/of/a/topic

and are not predefined; a device simply publishes to a topic and any device that
knows it can subscribe to receive data from it.

MQTT also allows devices to subscribe using wildcards. There are 2 wildcard
symbols: ‘+’ and ‘#’

So if a device publishes to topics /this/is/an/example/of/a/topic and

/this/is/an/example/of/another/topic, a subscriber can receive messages from both
topics in 2 ways:

1. Subscribe to /this/is/an/example/of/+/topic

2. Subscribe to /this/is/an/example/of/#

Another feature of MQTT brokers that made them attractive for us is that they
support both UDP and TCP, as well as one-way and two-way TLS.

In order to use MQTT developers can take advantage of open source libraries
like Paho, which already support C and Java with Lua, Python, C++ and JavaScript
on the way; and free, open source brokers like Eclipse mosquitto.
Protocol Buffers
Google initially developed protocol buffers — protobufs for short — in 2001 for
internal use; it was made public in 2008.

Protobuf specifies a language-neutral mechanism to serialize data in a simple

structure that is efficient for transmission between devices.

Its use is relatively simple: the developer defines a file with .proto with the data
structure definition, like this:

message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
required string callbacktopic = 4;

Then use language generators provided by Google to create the serialization code for
your programming language of choice. Currently Google supports C++, C#, Dart, Go,
Java and Python.

Putting it all together

Here is a simple example that illustrates how it all gets glued together. The topic will
look like the classpath for a Java class but it’s just our choice.

For this example, besides the Person message above we will define the Result
message:

message Result {

required int resultcode = 1;

Client on a device in C++

1. Subscribes to topic /com/myclient/MainClass/setPersonResult
 2. Defines the Person message in the messages.proto file

3. Defines the Result message in messages.proto file

4. Generates the C++ code to serialize/deserialize the messages

Server application in Java

1. Defines the same Person message as the client

2. Defines the same Result message as the client

3. Generates the Java code to serialize/deserialize the messages

4. Subscribes to topic /com/myserver/MainClass/+, so it can receive method

calls for all methods in this class

Flow
1. Client invokes the generated methods with the appropriate parameters; in
particular, callbacktopic set to “/com/myclient/MainClass/setPersonResult”

2. Client publishes the serialized buffer to the

topic /com/myserver/MainClass/setPerson

3. Server receives the data, deserializes and processes it by invoking

method MainClass.setPerson (in this case we took advantage of Java’s Reflection
and got the class and method straight from the topic)

4. Once processed, server invokes the generated methods to serialize the result and
publishes to /com/myclient/MainClass/setPersonResult

5. Client receives the result, invokes the methods to deserialize it and calls the
appropriate method to handle the result; in this case, because the client is in C++,
we relied on a hash table to map topics to method pointers.

This schema scaled up well to all the API calls; the broker hides the internals of the
server app, which can be protected behind VPN, firewalls or containers (we used
Kubernetes and Docker to isolate the server app by the way).

Also the broker can handle thousands of messages per second, so no problem there.
It works really well, however…

However…
The same features that make it so flexible and efficient also make it very vulnerable.
Here are some issues:

1. What if several instances of the client use the same topic for the callback?

Well, it is a problem because they will all receive the responses sent to all clients. Not
good, each client must receive the result for its own method call.

The solution is to add a unique identifier to the string so the callback topic is unique
to the client. Something like:

/com/myclient/Th8858Hhjhkjhcvb&uAA34/MainClass/setPersonResult

2. What if an attacker subscribes to MQTT for all topics, using topic ‘#’?

This is a big problem. If the MQTT interface is exposed publicly — via URL or public
IP address — then an attacker can easily eavesdrop all traffic between all clients and
the server. A HUGE no-no.

We envisioned 4 possible solutions for this problem:

1. The client and server negotiate an encryption key — for example, using Diffie-
Hellman — and use it to encrypt the content before sending it to the topic; anyone
eavesdropping will see the content but will not be able to read it

2. Whitelisting IP addresses. Only whitelisted IP addresses could connect to the

MQTT broker. Works but it is cumbersome to maintain.

3. Use VPN. Access to the MQTT interface requires VPN so only clients with the
credentials can connect. Works well if access is restricted to a client app or if
users are required to register for using the app; not so much for an open, public
API

4. Use a broker implementation that offers Access Control. Normally these are
commercial paid brokers, not free.
So as you can see, these solutions work but are not great. So, in conclusion…

Conclusion
Using MQTT and Protobufs for implementing RPCs works really well. It is efficient,
easy to implement, fast and reliable.

But security is a concern; it is not a solution for every RPC application. For public
APIs there are other approaches that are more secure and therefore preferred.

If however your use case is restricted to communication between your client

application and your server, then it is a solution that worked well for us and you may
want to check it out.

Thanks to Julie Vogelman.

Rpc Mqtt Mqtt Broker Protocol Buffers