Drones' Face off:
Authentication by Machine Learning in
Autonomous loT Systems
Mehdi Karimibiuki, Michal Aibin, Yuyu Lai, Raziq Khan, Ryan Norfield, and Aaron Hunter
Department of Computing
British Columbia Institute of Technology (BCIT)
Vancouver, Canada
Abstroct-Autonmnous Internet..,f·Things (loT) are cmnprised
of moving objects such as drenes and rovers that us. self·
control techniques to accomplish a mission while following a
path. However, losing control in such systems usually by spoofing
their sensors or hijacking with misleading cmnmands can lead
to catastrophic safety consequeoces. In this paper, we elese the
gap by autheoticating the behavior of autonmnous loT systems
during eperatlon, In particular, w. eheek the b.havior of a
moving loT object, •. g., a drone, by eYBiuating its tim....ri..
telemetry traces during the flight. W. examin. three differeot
machine-learning algorithms for this parpose, namely, K-Nearest
N.ighbour (KNN), Support Vector Machin. (SVM), aod Logistic
Regression (LR). Our results show that KNN is the best method
of the three selected techniqn.. for antheotication in dynamic
loT systems, •. g., drones. W. aemeved 93.4% in precision rate
and 100% recall rate with KNN.
Index Terms-Drones, Unmanned aircraft systems; Un1llQnned
aiTcrq/t vehit:les; Autonomous; Authe1Jtieation; Machine learning
I. INTRODUCTION
The prevalence of Internet-of-Things (loT) has em-
anated ubiquitous communication between many-to-many au-
tonomous objects. There now exists drones and rovers that
receive origin and destination instructions from a control
station to follow a path in autonomous mode. The Federal
Aviation Administration (FAA) is forecasting the number of
DAVs will grow to 7 million by 2020 [I]. Similarly, it is
estimated that there will be 10 million self-driving cars to hit
roads by 2020 [2]. The growing trends show that there is an
immediate need to secure the safety of such moving objects,
particularly, checking their behaviour during operation.
In this paper, we focus on device authentication by machine
learning algorithms. We pick an emerging loT device, a drone,
to train 3 classifiers, namely, KNN, SVM, and LR. After
training, we use the classifier to tell if the new data that are
being collected from a drone are representative of a correct
behaviour of the drone or not. We evaluate the correct behavior
of the drone during their flight operation.
A drone could easily get off track and be hijacked via
man-in-the-middle attacks [3], [4]. An attacker can potentially
infiltrate the network with computational power as low as a
Raspberry Pi [5] to fool and derail the drone by commanding
978-1-7281-3885-5/191$31.00 ©2019lEEE
Authorized licensed use limited to: SUNY AT STONY BROOK. Downloaded on July 26,2020 at 11:40:18 UTC from IEEE Xplore. Restrictions apply.
falsified information [6]. The attacker's goal is to either crash
the drone or manipulate the behavior such that the drone lands
in a hostile location [7].
A. Key Insight
In this paper, we investigate the following 3 machine-
learning classifiers to find abnormal behaviors of a drone:
• K-Nearest Neighbour (KNN) [8], which stores the train-
ing flights and classifies new traces by a majority vote of
its neighbours.
• Support Vector Machine (SVM) [9], which creates a
hyper-plane during training, then uses it to separate
correct from incorrect flight paths during new flights. For
improving accuracy the size of the hyper-plane can be
increased or decreased during the testing phase.
• Logistic Regression (LR) [10], which forms a linear
decision surface based on the training data.
If the time-series data captured from a drone during flight
operation are evaluated as false by the classifiers above, it
means they are off-track data, which consequently means the
drone is not flying in an expected path. The interpretation of
the failure is that the drone is either behaving abnormally or
if deflected from the mission planned, it could be hijacked or
malfunctioned. In this paper, we do not distinguish between
a drone that is malfunctioned and a drone that is hijacked.
Instead, we consider any authentication failure as an attack that
is trying to hijack the drone. To the best of our knowledge, we
are the first to evaluate 3 different machine learning algorithms
using flight data for authenticating drones.
B. Attack Model
We consider an autonomous loT environment where there
are drones to follow command and control (C2 [II]) in-
structions from a trosted hub [12] that manages traffic of
the friendly drones in the network. The drones upon taking
instructions from the C2 system use their feedback control
loop to manage themselves to accomplish a mission and to
arrive at destination given.
Figure I illustrates the attack model. In this scenario, we
assume an attacker can infiltrate the network by man-in-the-
middle (MitM) attack [5], record and learn communication
commands between drones and a trusted hub, then manipulate
03 29

Fig. 1: Attack model. Only one communication between a
drone and the UTM is shown. Other drones in the network
would also initially communicate with the UTM but eventually
the MitM dominate their resource availability forcing drones
to follow her commands.
and send falsified C2 commands to the drones in the network.
The attacker's goal is to derail the drones from their mission
by sending them wrong commands. The ultimate goal of the
attacker is to crash the drones or guide them toward a hostile
area. Several of such attacks have been attempted in real-
world [13], [14]. We also have demonstrated this attack by
sending 'arm' and 'disarm' commands to a hobbyist drone [6].
C. Contributions
The following are the main contributions of this paper.
• Approach: We study authentication in autonomous loT
systems by evaluating three most commonly used su-
pervised machine learning algorithms. We do this by
computing and reporting the precision and recall rates
for each algorithm (see Table I).
• Experimentation: We collect flight data and experiment
with an open-source drone simulator called ArduPi-
lot [15].
• Evaluation: We evaluate the three machine-learning al-
gorithms by recording simulation data taken from the
Ardupilot Software In The Loop (SITL) [16]. We sim-
ulated 3 flight paths and recorded 600 different instances
for every path. We analyzed and compared the results of
flight data. We find that KNN yields the highest results
with the average precision of all flights being 93.4%.
2
Authorized licensed use limited to: SUNY AT STONY BROOK. Downloaded on July 26,2020 at 11:40:18 UTC from IEEE Xplore. Restrictions apply.
II. RELATED WORK
The security of drones with respect to authorization and
authentication has been studied widely in previous work [17]-
[23]. However, in device authentication, Shoufan [24] is close
to our work in that he also used machine learning to distinguish
between authentic and malicious commands. However, our
work establishes authentication based on the behavior of the
drone itself and not the commands that are being commu-
nicated between a drone and a controller. Additionally, in
our approach, a drone could be flying in autonomous mode
without continuous communication with a control station thus
our approach can be used to tell if a drone is travelling a
correct path or a wrong direction.
In a closer work, Bartak et al. [25] used machine learning as
a mechanism to identify activities from a sequence of sensor
reading and corresponding control signals of a drone. Bartak et
al. treat drone behaviour as data that are the basis for learning.
However, Bartak et al. did not articulate if their approach could
generalize to all behaviors of a drone. Our goal is to evaluate
several machine learning techniques and suggest one that could
generally be used for moving loT systems, e.g. drones, and
their behaviour in the environment as long as the classifier is
well-trained for such.
III. METHODOLOGY
The machine-learning algorithms that we have selected
(KNN, SVM, LR) are suitable specifically for classification in
large-scale datasets. In our approach we first train the models
of each technique and then use the trained model to predict the
validity of newly generated flight paths. We use both genuine
and off-track data to measure precision and recall rates in
our experiments. Precision rate measures how much of the
correct paths are validated correctly by our machine learning
algorithms, and recall rate measures how much of incorrect
data are correctly distinguished as wrong flight paths.
A. K-Nearest Neighbour (KNN)
KNN stores all available flight traces to classify new ones by
a majority vote of its neighbours. All flights are categorized
through calculation of distance. In our research, we imple-
mented a KNN model that takes in a pre-processed dataset
that has the false data labeled using the Euclidean distance
function. When a prediction is required for an unseen data
instance, the algorithm searches through the dataset to find the
most similar one. Then, the prediction attribute of the collected
instances are summarized as prediction of the unseen instance.
The pseudocode of KNN is presented in Algorithm 1.
B. Support Vector Machine (SVM)
Support Vector Machines (SVMs) are learning classifiers
used to analyze data and recognize any patterns in supervised
mode. SVMs are typically used for classification and regres-
sion. SVM models are based on being given training data and
separating the data into categories. Any data that does not
fit into those categories will be considered false. The SVM
will also make predictions for the next points in each of the
0330
 Algoritbm 1: KNN
Data: New flight datasets, source and destination of IIight
Result: Prediction of IIight datapoint (valid - 0 I fake - I)
1 Calculate the shortest path between source and
destination of IIight
2 Set number of neighbours K = 10
3 foreach datapoint in Flight Data do
4 Calculate the Euclidean distance from that datapoint
to K neighbours.
• if Majority of neighbours are fake flights then
, I return 0
7 else
8 I return I
9 end
,. end
categories. This is useful for drone authentication because any
irregolarities typically mean that an attack has occurred [26].
In this paper, we use one-class SVM for the pattern recog-
nition on the flight data. If the drone is misguided or leads
off track by an attack, the SVM will detect an anomaly in
the pattern. A pattern will be formulated based on the training
data and any points that do not follow the pattern created from
the SVM will be considered fake. One-Class SVM will then
compare training data with every row in the dataset.
In the SVM algorithm. we are looking to find the hyperplane
that maximizes the margin between the two classes (fake
and valid data). We are using hinge loss function that helps
maximize the margin. defined as:
I coefficient. We calculate coefficients and threshold ( during
211wl12 + CL:max(O, 1- y,(wTx, + b))
, (I)
where w is the regularizer, x and y are coordinates (which
are used to check if the flight is inside the margin of the
hyperplane), and finally b is a bias parameter computed as
the average error using the sum over the (target value -
predicted value) on the training data. The pseudocode of SVM
is presented in Algorithm 2.
Algoritbm 2: SVM - Training
Data: New flight datasets, source and destination of IIight
Result: Trained model
1 Calculate the shortest path between source and
destination of IIight
2 Define an optimal hyperplane: maximize margin.
3 foreach datapoint in Flight Data do
4 if Incorrectly classified then
s I Include penalty for misclassification and decrease
the margin
, else
7 I Increase the margin
8 end
9 end
3
Authorized licensed use limited to: SUNY AT STONY BROOK. Downloaded on July 26,2020 at 11:40:18 UTC from IEEE Xplore. Restrictions apply.
We chose to use One-Class SVM as one of our classifiers
because every column in the dataset (yaw. roll, pitch. altitude.
latitude and longitude) is independent of other columns. If
one of the rows in the dataset bas fake data. but the other
rows are correct. it could potentially be considered as an
attack. One-Class SVM makes sure all columns in the path are
following a correct pattern [27]. For authentication in drones.
we implemented a One-Class SVM that uses data retrieved
from the drone flight. The SVM receives training data and
will predict whether the next given data is either false or true
based on the pattern given from the training data. The training
data is based on sets of valid and fake flights, so the SVM can
start to differentiate the differences between them.
C. Logistic Regression
Logistic Regression is a widely used machine learning
classification algorithm to predict a binary outcome using
a set of independent variables. Instead of finding a pattern.
Logistic Regression finds a relationship between features and
creates a probability of the next state. This machine learning
model is suitable for our data since our flights are labelled in
dichotomous nature. i.e, taking hinary values of either 0 (valid
IIight) or I (fake data).
In this approach. we establish set X of n = 6 indepen-
dent features (Xt.X2 .....X n). namely yaw. roll. pitch. altitude.
latitude and longitude. The probahility of attack detection is
calculated as:
p= 1+.·
.' (2)
where y = Yo + y,X, + Y2X2 + ... + YzX n and Yz is the
the training process of this model. Since logistic regression
follows a binary outcome and finds a probability. we can use
this classifier to predict the next outcome of the drones path.
The threshold ( is later used to determine if the drone does
not follow the predicted path. thus. the point is considered as
a fake one. The processing steps for testing dataset of IIights
generated on-the-fly are presented in Algorithm 3.
Algoritbm 3: Logistic Regression
Data: Coefficients Yz. threshold (. feature set X, new
flight datasets, source and destination of flight
Result: Prediction of IIight datapoint (valid - 0 I fake - 1)
1 Calculate the shortest path between source and
destination of flight
2 foreach datapoint in Flight Data do
3 Find the probability p using the Equation 2
4 ifp>(then
• I return I
• else
7 I return 0
8 end
9 end
033 1
 IV. EXPERIMENTAL SETUP
For simulating drone flights, we used the drone flight sim-
ulator program, ArduPilot [15]. ArduPilot is an open-source
autopilot software that simulates the behavior of unmanned
vehicles such as drones, rovers, etc. This program allows us
to run a drone in real geographical coordinate system (GCS)
and log traces in real-time. The flight data includes time-series
information of drones attitude such as yaw, roll, pitch, and
GCS such as altitude, latitude and longitude. We have collected
such data to train our machine learning models. Figure 2 shows
a trajectory with origin 1 and destination 2 that has been
created to fly a drone in simulator and log time-series data.
We run our simulations on a Windows 10 64-bit machine
with processor Intel(R) i5-8400 CPU @ 2.80 GHz, 6 cores
and 16 GB RAM. We have a script in Python that generates
trajectories based on given origin and destination coordinates
(latitude, longitude, and altitude). We generated 600 samples
of a same trajectory and used 80% of such data for training
and the rest for testing. We selected 3 paths on earth ranging
between 100 and 500 meters to fly a drone in simulation and
log time-series data. It took an average of about 15 minutes
to train the training functions for such paths with an average
of 120 snapshots in each log file for every path. However,
once the training is finished, validation takes 3 seconds to run
through a new path and annotate if the path is fake or real. For
creating fake paths, we have the drone fly a detour trajectory
as shown in Figure 3.
The memory overhead in our work is mainly due to
recordings of the state-space traces for the training sessions.
We have saved up to 1000 traces for each path to train the
machine learning functions. The average size of every trace
is about 54 kB and the total to save all traces is 54 MB,
which is reasonable for the UTM machine. The size of the
implementation code itself is only 11 kB. The training can
happen offline, and only the checker function needs to be run
online.
Fig. 2: lllustration of a sample drone trajectory from origin
point 1 to destination point 2.
V. RESULTS
A. Research Questions
In our study, we evaluate the approaches presented above
using two research questions:
1) RQ1. What are the precision and recall rate results in
our simulations of three different machine-leaning algo-
rithms?
4
Authorized licensed use limited to: SUNY AT STONY BROOK. Downloaded on July 26,2020 at 11:40:18 UTC from IEEE Xplore. Restrictions apply.
Fig. 3: lllustration of a fake route. A drone is flying from
origin 1 to destination 3, but taking a detour at mid-point 2.
2) RQ2. What is overhead time to detect if a path is fake or
real?
B. RQ1. Precision and Recall
In RQl, we check for what is the proportion of correct
paths are identified as correct paths (precision) as well as
what proportion of off track paths was identified correctly as
wrong paths (recall). These two measures allow us to check
the relevance of our models, by calculating the Fl score - a
harmonic average of the precision and recall. Finally, in RQ2,
we check what is the training and testing time for models to
verify their applicability to the existing systems.
First, we present answer the RQ1 using the Table I:
TABLE I: Accuracy of machine learning models.
Technique Precision Recall Fl
KNN 93.4% 100% 96.5%
SVM 95.6% 96% 95.8%
LR 87.34% 72% 78.9%
KNN is the best approach overall. It is slightly worse
than SVM in Precision metric, but it allows much higher
Recall rates. As a result, it achieves the best Fl score. It
means that we are very effective in detecting fake traffic.
SVM classifies correctly valid traffic more often, but it lacks
on the classification of fake data points. In our application
for authentication of autonomous loT systems, we should
prefer our classifier to be more sensitive; thus, high recall is
essential. The worst classifier is Linear Regression, with its
major problem in the Recall metric.
C. RQ2. Detection Time
Next, we measure the detection time to answer RQ2, in
Table II. We can observe that Linear Regression is the fastest
approach to train. On the other hand, even a trained model
takes a similar time to detect anomalies. The fastest approach
is SVM. It allows moderately fast training and a short time
to detect anomalies. If there is a specific time constraint in
our systems, we might use prefer it over KNN. KNN shows a
significantly higher computational cost due to its lazy learning
- it does not learn a discriminative function from the training
data but memorizes the training dataset instead.
033 2
 TABLE II: Average computing time of used models.
Model Training time Detection time
KNN 3 hours 3 seconds
SVM 90 seconds 2 seconds
LR 10 seconds 8 seconds
VI. CONCLUSION
Device authentication can be studied by means of machine
learning algorithms. We have used three supervised learning
methods and find that KNN yeilds the best results for a drone
fligbt dataset. Such investigations can be used in future to
shed some light over authentication of other Autonomous loT
objects such as rovers and self-driving cars. We also find that
KNN althougb it takes for it to train at longer time, it wins
over SVM and LR in precision and recall rates. Our results
were based on flying a drone simulator in three unique paths
and 600 fligbt samples at each path.
REFERENCES
[1] F. A. A. (FAA), ''Faa releases 2016 to 2036 aerospace forecast;' March
2016.
[2] forbes.com, "10 million self-driving cars will hit the road by 2020 -
here's how to profit," Mar. 2017.
[3] N. Rodday, "Hacking a professional drone," Slides at www. blackhot.
comldocslasia-16lmarerialslasia-16-Rodday-Hacking-A-Professional-
Drone. pdf, 2016.
[4] N. M. Rodday, R d. O. Schmidt, and A. Pras, "Exploring security
vulnerabilities of UDmaDDM aerial vehicles:' in NOMS 2016-2016
JEEE/lFIP Network Operations andManagement Symposium, pp. 993-
994, IEEE, 2016.
[5] jeffq, "Setting up a man-in-the-middl.e device with raspberry pi," Feb.
2014.
[6] M. Karimibiuki, "Drone back by man-in-the-middle attack," 2019.
YonThbe link: h\1pll://yootn.beISrJv04RwMUQ.
[7] J. McNeely, M. Hatfield, A. Hasan, and N. Jaban, "Detection of
uav hijacking and malfunctions via variations in flight data statistics,"
in Security Technology (ICCST), 2016 IEEE International Carnahan
Conference on, pp. 1--8, IEEE. 2016.
[8] T. M. Cover, P. E. Hart, et aL, "Nearest neighborpattern classification,"
IEEE transactions on information theory, vol. 13, no. 1, pp. 21-27,
1967.
[9] L. Wang, Support vector machines: theory and applications, vol. 177.
Springer Science & Business Media, 2005.
[10] C. M. Bishop, Patternrecognitionand machine learning. springer,2oo6.
[11] H. C. Ngoyen, R Ammirn, J. wlgerd, I. Z. Kovacs, and P. Mogensen,
"Using lte networks for uav command and control link: A rural-area
coverage analysis:' in 2017 IEEE86th Vehicular Technology Conference
(VTC-Fall), pp. 1--6, IEEE, 2017.
5
Authorized licensed use limited to: SUNY AT STONY BROOK. Downloaded on July 26,2020 at 11:40:18 UTC from IEEE Xplore. Restrictions apply.
[12] M. Karimibiuki and A. Ivanov, ''Minic1oud: A mini storage and query
service for local heterogeneous iot devices," in Proceedings of the 8th
International Conference on the Internet ofThings, p. 21, ACM, 2018.
[13] S. Shane and D. E. Sanger, "Drone crash in iran reveals secret us
surveillance effort," The New York Times. vol. 7, 2011.
[14] K. Jansen, M. Sehllfer, D. Moser, V. Lenders, C. Popper, and J. Schmitt,
"Crowd-gps-sec: Leveraging crowdsourcing to detect and localize gps
spoofing attacks," in 2018 IEEE Symposium on Security and Privacy
(SP), pp. 1018-1031, IEEE, 2018.
[15] A. D. Team. "ardupilot," URL; www. ardupilot. org, accessed, vol. 2,
p. 12,2016.
[16] A. D. Team, "Sill simnlntor (software in the loop);' 2016.
[17] A. Davauian, F. Massacci, and L. Allodi, "Diversity: A poor man's
solution to dronetakeover," in Proceedings ofthe 7th International Joint
Coriference on Pervasive and Embedded Computing andCommunication
Systems (PECCS 2017), Madrid, Spain, July 24-26, 2017., pp. 25-34,
2017.
[18] C. A. T. Bonilla, O. J. S. Parra, and J. H. D. Forero, "Common secu-
rity attacks on drones," International Journal ofApplied Engineering
Research, vol. 13, no. 7, pp. 4982-4988, 2018.
[19] D. Meodes, N. Ivnki, and H. Madeira, ''EIIects of GPS spoofing on
unmanned aerial vehicles," in 23rd IEEE Pacific Rim International
Symposium on Dependable Computing, PRDC 2018, Taipe~ Taiwan,
December 4-7, 2018, pp. 155-160,2018.
[20] S. M. Giray, ''Anatomy ofnnmanned aerial vehicle hijacking with signal
spoofing," in 2013 6th lntemational Conference on Recent Advances in
Space Thchnologies (RAST), pp. 795--800, 2013.
[21] I. Gilveny, F. Koohifar, S. Singh, M. L. Sichitiu, and D. Matolak,
"Detection, tracking, and interdiction for amateur drones," IEEE Com-
munications Magazine, vol. 56, no. 4, pp. 75--81, 2018.
[22] G. Vasconcelos, R. S. Miani., V. Guizlini, and J. R. Souza, ''Evaluation
of dos attacks on commercial wi-B.-based uevs," IJCNIS, vol. 11, no. I,
2019.
[23] M. Karimibinki, E. Aggarwal, K. Pnttabiraman, and A. Ivanov, "Dynpo-
lac: Dynamicpolicy-based access controlfor iot systems:' in 2018 IEEE
23rd Pacific Rim International Symposium on Dependable Computing
(PRDC), pp. 161-170, IEEE, 2018.
[24] A. Shoufan, "Continuous authentication of DAV flight command data
using behaviometrics," in 2017 IFIPREEE InternationalCoriference on
Very Large Scale Integration, VLSI-SoC 2017, Abu Dhab~ United Arab
Emirates, October 23-25, 2017, pp. 1--6,2017.
[25] P. Abbeel, A. Conte" M. Quigley, and A. Y. Ng, "An application of
reinforcement learning to aerobatic helicopter flight," in Advances in
Neural Information ProcessingSystems 19. Proceedings ofthe TWentieth
Annual Conference on Neural1nfonnation Processing Systems, Vancou-
ver, British Columbia, Canada, December 4-7, 2006, pp. 1-8, 2006.
[26] A. Bernardini, F. Mangiatordi, E. Pallotti, and L. Capodiferro, ''Drone
detection by acoustic signature identification," Electronic Imaging,
vol. 2017, DO. 10,pp. 60-64, 2017.
[27] S. Mukkamala andA. H. Sung, "Detecting denialof serviceattacks using
support vector machines," in The 12th IEEE Intemational Conference
on Fuuy Systems, 2003. FUZZ'03., vol. 2, pp. 1231-1236, IEEE, 2003.
0333