3rd International Conference on Internet of Things & Connected Technologies (ICIoTCT), 2018

A Performance Evaluation of Linux Containers on similar ARM based

Internet of Things Development Boards
Senthil Kumaran. Sa,b , Balakannan. S.P.a,c
a
Department of Information Technology, Kalasalingam Academy of Research and Education, Krishnankoil. TamilNadu, India.
b
sk@stylesen.org
c
balakannansp@gmail.com

Abstract
Internet of Things (IoT) is evolving and there will be more than 50 billion connected things in the near future.
Security is of prime importance when so many devices are connected and it is an active research topic among
others, in the IoT world. We need innovative ways to deploy, manage and update the software installed
on these devices in order to keep the devices updated with security fixes delivered on time. Containers
can help in achieving a new mechanism to deploy and apply updates to these IoT devices, which many
System on a chip (SoC) vendors and other operating system vendors handling IoT devices are considering.
There are many ARM[1] based development boards available commercially off-the-shelf offering various
hardware specifications. In this paper we will use Linux containers on ARM based development boards such
as Dragonboard 410c, Orange Pi PC2, Pine A64 and Raspberry Pi 3 model B which are commonly used in IoT
development, in order to benchmark and measure their performance at various levels. The ARM development
boards are carefully chosen in order to have similar hardware specification and running a common Operating
System namely Debian[2] with Linux kernel 4.13 and above. The LXC environment used across all the boards
is based on Debian Stretch (9.0).
Keywords: internet of things, iot, containers, linux containers, lxc, security, benchmark, SoC, ARM,
measurement, commercial off-the-shelf

1. Introduction the software running on the device updated. Taking

Availability of commercial off-the-shelf ARM
based development boards help many researchers
and start-up companies to prototype and implement
Internet of Things (IoT) based projects and exper-
iments. Commercial off-the-shelf devices are con-
sidered as black boxes and importance is not given
for performance, scalability and reliability[3]. Also,
it is important to measure the performance of these
devices in order to use it in a better way and avoid
serious implications during deployment[4].
IoT is a growing area in computer science which
will eventually lead to proliferation of devices.
When large number of IoT devices get connected
over the internet, security is of prime importance.
The most common way to secure a device is to keep
into consideration the number of devices that will
be deployed in future, traditional software delivery
mechanisms may not scale. Many vendors involved
in developing IoT solutions are considering innova-
tive ways to deploy, manage and update the soft-
ware installed on these devices. One such innova-
tive method is to use containerization within these
devices to deliver software.
Consider a wrong software update that has been
pushed to one of these device and causing a disrup-
tion which may be unacceptable in most scenarios.
Hence there should be mechanisms to rollback the
updates that were applied and bring the device to
the previous working condition. In order to support
this rollback mechanism, containers can be of much

ELSEVIER-SSRN INFORMATION SYSTEMS & EBUSINESS NETWORK ISSN: 1556-5068

HTTPS://WWW.SSRN.COM/LINK/3RD-ICIOTCT-2018.HTML
1018

Electronic copy available at: https://ssrn.com/abstract=3178682

 3rd International Conference on Internet of Things & Connected Technologies (ICIoTCT), 2018
help. Instead of running software on the devices di-
rectly, they can run on containers and each new up-
date will be deployed to a staging container within
the device before they are moved to work as the pri-
mary container, running the service. This container
based mechanism for software updates are popular
among many SoC vendors and other operating sys-
tem vendors handling IoT devices.
Figure 1: Linux Containers (LXC)
At a high level the host operating system deployed
on these devices will be a minimal operating system
that will be capable of running containers. The con-
tainers will run on top of these host operating sys-
tem in order to deliver various services that the IoT
device is intended for. The host operating system
should be hardened and will not be exposed to the
outer world and acts as a management commodity in
order to manage containers that will get installed on
top of it. Figure 1, shows the architecture of Linux
containers (LXC)[5], which shares the same operat-
ing system kernel as the host and provides isolation
of operating systems on the host.
In this paper we will be measuring both bare metal
and within a container environment performance of
the chosen, similar ARM based development boards.
This will help us to identify the capabilities of these
boards and also the potential bottlenecks in using
containerization, if any.
ELSEVIER-SSRN INFORMATION SYSTEMS & EBUSINESS NETWORK
HTTPS://WWW.SSRN.COM/LINK/3RD-ICIOTCT-2018.HTML
Electronic copy available at: https://ssrn.com/abstract=3178682
2. Related Work
In the paper, “Benchmarking Internet of Things
devices”[3] commercial off-the-shelf IoT devices
such as Raspberry Pi, BeagleBoard-MX and Beagle-
Bone Black were bench-marked with the help of a
cross platform benchmarking tool called lmbench[6]
and a Constrained Application Protocol (CoAP)[7]
library based custom benchmarking tool. There
were two types of benchmarking done such as micro
benchmark with lmbench and macro benchmark us-
ing the CoAP based custom tool. The benchmarks
were based on memory performances, latency and
CoAP request round-trip times.
The IoT devices taken into consideration in this
paper[3] had differences in terms of hardware spec-
ification, specifically on the CPU where a Beagle-
Bone Black had a dual core CPU and Raspberry Pi
had a single core CPU. It wasn’t clear on the exact re-
visions or version of the boards that were taken into
consideration, which did had differences in memory
and CPU clock frequencies, specially the Raspberry
Pi’s.
The benchmarking tool lmbench used to carry out
benchmarks is old[6] and is not under active devel-
opment. There were differences in the Linux kernel
versions such as 3.8.x and 3.6.x used in these boards
for the benchmarks. Also the operating system envi-
ronments had Graphical User Interface in some cases
and didn’t in other cases. As discussed in the conclu-
sion of this paper[3], due to the major differences in
choice of devices with different hardware specifica-
tion, software and operating systems used for bench-
marks, devices with better and bigger hardware spec-
ifications showed good performance than the lower
equipped devices.
“Evaluating Performance of Containerized IoT
Services for Clustered Devices at the Network
Edge”[8] discusses employing lightweight virtual-
ization technologies such as containerization on IoT
devices at the network edge. A framework based on
direct interaction between two devices and another
framework based on interaction between two devices
with a supervising manager device in between were
considered for performance evaluation. It discusses
various aspects of using containerization in IoT de-
vices. Application oriented Docker[9] containers
ISSN: 1556-5068
1019
 3rd International Conference on Internet of Things & Connected Technologies (ICIoTCT), 2018

Table 1: Development Boards

Dragonboard 410c Raspberry Pi 3 Model B Orange Pi PC2 Pine A64
SoC Qualcomm Snapdragon 410E Broadcom BCM2837 Allwinner H5 Allwinner A64
CPU ARM Cortex A53 ARM Cortex A53 ARM Cortex A53 ARM Cortex A53
Cores Quad Core Quad Core Quad Core Quad Core
Clock Speed 1.2 GHz 1.2 GHz 1.3 GHz 1.2 GHz
Instruction Set ARMv8 ARMv8 ARMv8 ARMv8
RAM 1GB LPDDR3 1GB LPDDR2 1GB LPDDR3 1GB LPDDR3
eMMC 8GB - - -
Storage MicroSD MicroSD MicroSD MicroSD
Ethernet - 10/100 10/100/1000 10/100/1000
Wireless 802.11 b/g/n 802.11 b/g/n - 802.11 b/g/n (optional)
Bluetooth 4.1 4.0 - 4.0 (optional)
GPIO 40 pins 40 pins 40 pins 40 pins
Price $75 USD $35 $20 $19

were used throughout the experiments to evaluate
performance. Docker is a Linux containerization
technology that emerged from LXC in the past and
later moved to its own implementation methodology.
Yet, both Docker and LXC use the same underly-
ing mechanisms offered by Linux Kernel in order to
implement containerization, hence Docker could be
considered an equivalent and similar technology to
LXC. Thorough analysis of different parameters such
as CPU utilization, power efficiency, system load,
CoAP response times, etc. were made. The IoT de-
vices used in these experiments were same and were
running Docker based containers to measure other
system and network parameters.
In the paper, “A Performance Evaluation of
Container Technologies on Internet of Things
Devices”[10] the performance evaluation was done
on a single device namely the Raspberry Pi 2[11]
both on bare metal and within container environ-
ment. This paper also used Docker as the con-
tainerization technology and analysed benchmark re-
sults for CPU, Network I/O, Memory, Disk I/O and
Apache server. It used tools such as sysbench[18],
dd[20] and apache benchmark (ab) tool to conduct
the benchmarks. It concludes with a note on negli-
gible impact seen on employing containerization on
Raspberry Pi 2 board that was considered.
3. Proposed Model
Throughout this paper care has been taken to run
performance measurements on hardware and soft-
ware that are identical or close to identical to bring
a fair comparison rather than to compare apples with
oranges.
In the recent years, container technology has be-
come popular in GNU/Linux operating systems with
Linux Containers (LXC). We will use LXC as our
container technology.
3.1. Device Configuration
The following similar ARM based development
boards are chosen for this paper:
1) Dragonboard 410c [12]
2) Orange Pi PC2 [13]
3) Pine A64 [14]
4) Raspberry Pi 3 Model B [15]
These boards are popular among IoT developers
and are also affordable with similar hardware specifi-
cation. The detailed specification of these boards are
given in Table 1. As seen in Table 1, all these boards
have a quad core ARM Cortex A53 CPU which has a
64-bit architecture and uses ARMv8 instruction set,
though these CPUs come from different SoC ven-
dors such as Qualcomm, Broadcom and Allwinner.
The CPUs have the same clock speeds at 1.2 GHz.
Similarly the RAM size is same across the boards at
1GB. In the benchmark measurements Ethernet was
used for networking which was connected to a Gi-
gabit LAN network. For boards which did not have
Ethernet, QuantumZERO QZ-AD02 USB 3.0 Giga-
bit Ethernet Adapters[16] were used.

ELSEVIER-SSRN INFORMATION SYSTEMS & EBUSINESS NETWORK ISSN: 1556-5068

HTTPS://WWW.SSRN.COM/LINK/3RD-ICIOTCT-2018.HTML

1020
 3rd International Conference on Internet of Things & Connected Technologies (ICIoTCT), 2018

One of these boards namely the Dragonboard 410c 4.2. netperf

has inbuilt-eMMC, but the rest of them did not have Netperf is a benchmark that can be used to mea-
an eMMC. Hence to make a fair analysis, we have sure various aspects of networking performance. It
installed the Operating system on a SanDisk 16GB can be used to measure both TCP and UDP perfor-
Class 10 microSD Card. mances. [19]
3.2. Operating System and Kernel 4.3. dd
Care has been taken to run the same Operating sys- dd is a command-line utility for Unix and Unix-
tem platform - Debian aarch64 or arm64 for measur- like operating systems whose primary purpose is to
ing the performance parameters. Armbian[17] which convert and copy files. [20]
is a derivative of Debian is used for some boards such
as Orange Pi PC2 and Pine A64. All these boards run
Linux kernel version greater than or equal to 4.13.x, 5. Results and Discussion
which is the latest available at the time of this writ-
ing. This section details the benchmark results on the
four different ARM development boards that are
3.3. Performance Parameters listed in Table 1. Experiments were carried on bare
This paper focuses on measuring both bare metal metal hardware to measure the performance followed
and within LXC performance of the following pa- by measurements within the container. Each bench-
rameters: mark was run thrice and the average of the three runs
were taken as the final result value. Finally, results
a) CPU Performance are presented and discussed.
b) Memory Performance
c) Network Performance 5.1. CPU Performance Analysis
d) I/O Performance The CPU performance benchmark was done us-
ing sysbench with 4 worker threads created. This
benchmark performs prime number calculation for
each request using 64-bit integers, upto the maxi-
4. Tools Analysis mum prime number specified as 20000. The lower
the time taken for execution in seconds is better. The
benchmark was run on each of the bare metal devel-
Table 2: Benchmarking Tools Version opment boards and within LXC.
Tool Version
sysbench 0.4.12-1.2
netperf 2.6.0-2.1 Table 3: CPU Performance Benchmark Completion Time in
dd 8.26 (from coreutils) seconds
Board Bare Metal Inside LXC
Dragonboard 410c 7.50 7.49
Following sections provide the list of Benchmark- Orange Pi PC2 7.86 7.85
ing tools that were used to measure different perfor- Pine A64 11.11 11.11
mance related parameters. Table 2, provides the ver- Raspberry Pi 3 Model B 15.28 15.28
sion of each of these benchmarking tools.
Figure 2, shows that the bare metal CPU perfor-
4.1. sysbench mance matches the CPU performance measurements
Sysbench is a benchmark tool used for evaluat- taken within LXC, with negligible difference in mil-
ing CPU-bound operations and memory related per- liseconds. Dragonboard 410c completed the bench-
formance. It is modular, cross-platform and multi- mark in the lowest time around 7.5 seconds and
threaded [18]. Raspberry Pi 3 Model B took the longest time around

ELSEVIER-SSRN INFORMATION SYSTEMS & EBUSINESS NETWORK ISSN: 1556-5068

HTTPS://WWW.SSRN.COM/LINK/3RD-ICIOTCT-2018.HTML

1021
 3rd International Conference on Internet of Things & Connected Technologies (ICIoTCT), 2018
Figure 2: CPU Performance Benchmark
15.2 seconds. The detailed time taken for CPU per-
formance benchmark completion is listed in Table 3
for each development board.
5.2. Memory Performance Analysis
Memory performance analysis was done with the
help of sysbench in order to measure the memory
operations speed. All the development boards have
1GB of RAM as listed in Table 1. Each mem-
ory block size used in the performance benchmark
measurement is 1 KiB. Write operations were per-
formed by sysbench taking the entire 1GB of avail-
able memory into account. The sysbench com-
mand was started with 4 worker threads. In Fig-
ure 3, we can see that Pine A64 development board
took the longest time to execute the memory bench-
mark,whereas Dragonboard 410c took the least time
for execution. The details of total time taken for ex-
ecution is given in Table 4.
Table 4: Memory Performance Benchmark Completion Time
in seconds
Board Bare Metal Inside LXC
Dragonboard 410c 35.52 35.05
Orange Pi PC2 36.13 36.13
Pine A64 86.80 140.07
Raspberry Pi 3 Model B 63.94 64.48
Also it is worth noting that there is considerable
difference in execution time of Pine A64 bare metal
ELSEVIER-SSRN INFORMATION SYSTEMS & EBUSINESS NETWORK
HTTPS://WWW.SSRN.COM/LINK/3RD-ICIOTCT-2018.HTML
Figure 3: Memory Performance Benchmark
and within LXC run of the memory performance
benchmark, in the order of 50+ seconds.
5.3. Network Performance Analysis
The Network performance benchmark was done
using netperf tool which is capable of perform-
ing various network related benchmarking both with
TCP and UDP protocols. As listed in Table 1, not all
the development boards discussed in this paper has
an onboard Ethernet interface, which is also true for
the WiFi interface. Also, a closer look shows Rasp-
berry Pi 3 Model B has an onboard Ethernet interface
that supports 10/100 and other boards which have on-
board Ethernet interface supports 10/100/1000 band-
width. For Dragonboard 410c which does not have
an onboard Ethernet we have provided an USB Eth-
ernet dongle to get it connected to the network. In the
case of Pine A64, we could not make the onboard
Ethernet working, hence we added a USB Ethernet
dongle and used the same for running the benchmark.
QuantumZERO QZ-AD02 USB 3.0 Gigabit Ethernet
Adapters[16] were used in both the cases. The LXC
containers running inside the development boards are
connected via a virtual bridge formed with the help
of libvirtd[21].
Netperf requires a netserver service running on a
different machine to which the netperf client sends
data packets and measures the throughput of the net-
work. The netserver service was run on a laptop
ISSN: 1556-5068
1022
 3rd International Conference on Internet of Things & Connected Technologies (ICIoTCT), 2018

Figure 4: Network Throughput

Table 5: Network Throughput in MB/s

Board Bare Metal Inside LXC
TCP UDP TCP
Dragonboard 410c 113.18 98.21 105.99 98.94
Orange Pi PC2 913.98 675.04 315.82 318.74
Pine A64 123.43 131.11 122.42 131.45
Raspberry Pi 3 Model B 94.10 94.00 66.49
which was a single hop away from each of the de-
velopment boards. The laptop and all the develop-
ment boards were connected using a Gigabit switch
with CAT6[22] cable, which ensures the network can
have a maximum throughput of 1 GB/s or 1024 MB/s
(theoretically).
The measurement was taken by sending a test
packet of size 1024 bytes each to the remote host,
first using the TCP, followed by sending UDP pack-
ets of the same size. The benchmark was run on both
bare metal and within LXC setup of development
boards and the results are shown in Figure 4, where
the Y-axis represents throughput in MB/s. Orange Pi
PC2 has an onboard Gigabit Ethernet adapter which
was used directly in the analysis and that did show
peak throughput of around 900+ MB/s. All the other
development boards showed a throughput of 10/100
analogous bandwidth. There was not much differ-
ence in throughput observed between bare metal and
within LXC executions, except in the case of Orange
Pi PC2. Table 5, shows the measured value of net-
work throughput.
5.4. I/O Performance Analysis
As noted already, some of the development boards
has an onboard eMMC and some does not. In or-
der to overcome this difference the Operating Sys-
tem was installed on identical 16 GB MicroSD card,
UDP
Figure 5: I/O Performance Benchmark
90.24
Table 6: I/O Benchmark Completion Time in seconds
Board Bare Metal Inside LXC
Dragonboard 410c 9.75 9.85
Orange Pi PC2 43.86 43.94
Pine A64 10.60 10.88
Raspberry Pi 3 Model B 56.11 56.83
which serves as the storage for all the development
boards. The I/O performance analysis was done by
creating a 100 MiB file by copying 1024 KiB at a
time for 100 times using dd command. The flags
such as direct and dsync were used in the dd com-
mand to avoid any caching on the memory.
The time taken in seconds to complete creating a
100 MiB file was measured in each case as shown
in Table 6. The lower the execution time, is better,
in this case. As seen in Figure 5, Dragonboard 410c
took less than 10 seconds to completely write a 100
MiB file, whereas Raspberry Pi 3 Model B took the
longest time i.e., 56 seconds to create the same file
using dd.
6. Future Work
The development boards considered in this paper
are mostly employed as IoT Gateways / Prototypes
and Edge devices, since they have considerably big-
ger resource in terms of CPU, RAM, connectivity,
etc. which are typically not the case of constrained

ELSEVIER-SSRN INFORMATION SYSTEMS & EBUSINESS NETWORK ISSN: 1556-5068

HTTPS://WWW.SSRN.COM/LINK/3RD-ICIOTCT-2018.HTML

1023
 3rd International Conference on Internet of Things & Connected Technologies (ICIoTCT), 2018
IoT devices[23] classified as class0, class1, class2.
In such non-constrained devices it is important to
analyse the application protocols and measure the
performance of applications that will be deployed in
real world. Future work will focus on evaluating the
performance of constrained IoT devices and applica-
tions run as service in individual containers.
7. Conclusions
The principal aim of this paper was to evaluate
the performance of few of the popular commercial
off-the-shelf development boards and look for any
performance degradation in terms of employing con-
tainers on the development boards. The results that
were shown and discussed clearly brought out the
fact that, employing Linux containers does not de-
grade the performance of the development boards
in any way and provides equivalent bare metal per-
formances. Hence Linux containers can be used to
deliver and deploy software on IoT devices in order
to provide security and ensure roll back possibilities
during software updates, as discussed in the start of
this paper. This paper will also stand as a piece of
evidence to support choosing the right development
boards based on the results discussed for researchers
and start-up companies who are involved in IoT.
REFERENCES
[1] ARM architecture. https://www.arm.com/; 2018.
[2] Debian - The Universal Operating System. https://www.
debian.org/; 2018.
[3] Kruger, C. P., and Hancke, G. P. (2014). Bench-
marking Internet of things devices. Proceedings
- 2014 12th IEEE International Conference on
Industrial Informatics, INDIN 2014, 611-616.
https://doi.org/10.1109/INDIN.2014.6945583
[4] J. Villasenor and M. Tehranipoor, “Chop shop electronics”,
IEEE Spec- trum, vol. 50, no. 10, pp. 41-45, 2013.
[5] Linux Containers (LXC) https://linuxcontainers.
org/ 2018.
[6] McVoy Larry, Staelin Carl, “lmbench: Portable Tools for
Performance Analysis”, Proceedings of the USENIX An-
nual Technical Conference, 1996.
[7] Z. Shelby, K. Hartke, C. Bormann IETF RFC7252, The
Constrained Application Protocol (CoAP), 2014.
[8] Morabito Roberto, Farris Ivan, Iera Antonio, Taleb Tarik,
“Evaluating Performance of Containerized IoT Services for
ELSEVIER-SSRN INFORMATION SYSTEMS & EBUSINESS NETWORK
HTTPS://WWW.SSRN.COM/LINK/3RD-ICIOTCT-2018.HTML
Clustered Devices at the Network Edge”, IEEE INTER-
NET OF THINGS JOURNAL, VOL. 4, NO. 4, pp. 1019-
1030, 2017.
[9] Docker. https://www.docker.com/; 2018.
[10] Morabito Roberto, “A Performance Evaluation of Con-
tainer Technologies on Internet of Things Devices”, IEEE
Access, VOL. 5, pp. 8835-8850, 2017.
[11] Raspberry Pi 2 http://www.raspberrypi.org/
products/raspberry-pi-2-model-b/ 2018.
[12] Dragonboard 410c https://www.qualcomm.com/
products/dragonboard-410c 2018.
[13] Orange Pi PC2 http://www.orangepi.org/
orangepipc2/ 2018.
[14] Pine A64 https://www.pine64.org/?page_id=1194
2018.
[15] Raspberry Pi 3 Model B https://www.raspberrypi.
org/products/raspberry-pi-3-model-b/ 2018.
[16] QuantumZERO QZ-AD02 USB 3.0 Gigabit Ethernet
Adapter. https://goo.gl/ptazT8, 2018.
[17] Linux for ARM development boards. https://www.
armbian.com/; 2018.
[18] SysBench, a modular, cross-platform and multi-threaded
benchmark tool. https://github.com/akopytov/
sysbench; 2018.
[19] Netperf, is a benchmark that can be used to measure
the performance of many different types of networking.
https://hewlettpackard.github.io/netperf/;
2018.
[20] dd, UNIX command line tool to convert and copy files.
https://en.wikipedia.org/wiki/Dd_(Unix); 2018.
[21] Virtualization APIs. https://libvirt.org/; 2018.
[22] Category 6 cable, commonly referred to as Cat 6. https:
//en.wikipedia.org/wiki/Category_6_cable;
2018.
[23] C. Bormann, M. Ersue, A. Keranen RFC7228, Terminol-
ogy for Constrained-Node Networks Abstract 2014.
ISSN: 1556-5068
1024