Autonomous UAV with Human Detection

Kshitij Karnawat, Nihar Asare, Sumit Anilkumar Singh, and Anju Kulkarni

Abstract The use of Unmanned Aerial Vehicles (UAV) is increasing tremendously

and with it the need for autonomous flight. In this paper, it is discussed how an
autonomous UAV is built with the help of a cascaded Proportional Integral Deriva-
tive controller and Simultaneous Localization and Mapping (SLAM). The simplistic
design and easily available components make a light weight quadcopter style UAV.
A Human detection neural network is also added to the UAV which can be used
for many applications such as aerial photography, search and rescue, defense. In
previous papers the UAVs processed data on board which in turn caused the UAV to
use huge amounts of memory and processing power, thereby increasing the power
consumption and cost. Whereas the UAV system proposed in this paper transmits
raw data to the main control station for processing in turn reducing the on board
processing and avoiding the use of costly controllers.

Keywords Autonomous UAV · Drones · PID · Convolutional neural network

1 Introduction

An unmanned aerial vehicle (UAV) is an aircraft without a human pilot on board. The
flight of UAVs may operate with various degrees of autonomy: either under remote
control by a human operator or autonomously.
While drones originated mostly in military applications, their use is rapidly finding
more applications including aerial photography, deliveries, agriculture, surveillance
and racing [1].

K. Karnawat (B) · S. A. Singh · A. Kulkarni

Electronics and Telecommunications Department, Dr. D. Y. Patil Institute of Technology, Pimpri,
India
N. Asare
Electrical Department, Dr. D. Y. Patil Institute of Technology, Pimpri, India

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 297
V. V. S. S. S. Chakravarthy et al. (eds.), Advances in Micro-Electronics, Embedded Systems
and IoT, Lecture Notes in Electrical Engineering 838,
https://doi.org/10.1007/978-981-16-8550-7_28
298 K. Karnawat et al.

The ability of a UAV to take off, execute a mission, and return to its base without
significant human intervention promises to reinforce UAV deployment in many
application domains.
Building an autonomous UAV is not as easy as building an autonomous car. An
autonomous car moves in a two dimensional space and has a lot of constraints on its
movement. Whereas an UAV moves in three dimensional space and has six degrees
of freedom. This increase in degrees of freedom and the number of variables that need
to be calculated cause an increase in the complexity of the -autonomous navigation
of the UAV.
The levels of autonomy can be classified into four levels in the lowest being full
manual control to a fully autonomous system.
Level 0: requires full control from the pilot as the UAV has no support systems.
Level 1: involves basic assistance features. Typically, Level 1 UAVs are equipped
with a hover control. The pilot is still in full control of the UAV but gets some
forms of help.
Level 2: corresponds to semi-autonomous UAVs. The UAV can take off and land
hover, maintain its roll, pitch and yaw autonomously.
Level 3: At this level, the UAV can dynamically plan a path and fly to a specific
location but needs to be monitored all the time by a pilot who can take over in
cases of emergency [2].
In this paper, it is discussed how to develop a fully autonomous UAV that can
autonomously take off, control its attitude, altitude, hover, land and dynamically plan
its path to a given GPS coordinate with the help of the peripherals onboard the UAV.

2 Methodology

2.1 UAV Design

2.1.1 The Microcontroller

The brain of the UAV is a Raspberry Pi controller. Traditionally most drones are
equipped with a STM32 based microcontroller instead of the Raspberry Pi. The
Raspberry Pi availability in the market and its capability to run the Robot Operating
System (ROS) are the two main reasons to choose it over other controller. The higher
processing power is also a key factor as this reduces the response time of the system
and allows us to add many different modules for further enhancement. In the case of
the STM32 based flight controllers available in the market are designed such that the
motors and a few other peripherals can be added where as in the case of Raspberry
Pi the 40 GPIO pins and the different kinds of input output ports allows the addition
of several modules.
Autonomous UAV with Human Detection 299

Fig. 1 UAV design block diagram

2.1.2 Motor and Motor Control

The UAV is a Quadcopter style design. This design allows the UAV to move swiftly
and with agility in a terrain with a lot of obstacles. Four micro coreless motors are
used for the same. Coreless motors are used for their light weight and high revolutions
per minute (RPM). These light weight motors reduce the overall weight of the UAV.
The high RPM allows the propellers of the quadcopter to achieve high amounts of
thrust in turn increasing the load carrying capacity of the UAV. These motors are
controlled via a custom built MOSFET motor driver. As the coreless motors do not
require a large amount of current to generate high RPM a simple MOSFET is enough
to supply power to it. The motor controller controls the speed of the motor just like a
traditional motor controller using pulse width modulation (Fig. 1). The custom built
motor driver is smaller, lighter and cheaper than the ones available in the market.
This helps us to reduce the overall weight of the UAV and also increase its load
carrying capacity.

2.1.3 Sensors

During the flight, the UAV may encounter several obstacles in its path. To overcome
these obstacles, and avoid collisions it is necessary to detect them as fast as possible.
For this purpose, a number of sensors are on board the UAV. To determine the position
300 K. Karnawat et al.

of the UAV in three dimensional space a GPS is used to track the location of the UAV
in latitude and longitude or the x and y coordinates. It is also the main component of
autonomy. The altitude or the z coordinate is given by a highly accurate barometer
or an altimeter. Next to determine the orientation of the UAV a nine axis Inertial
Measurement Unit (IMU) is used. It comprises of a Gyroscope, Accelerometer and a
Magnetometer. These three sensors provide the roll, pitch and yaw or the telemetric
data of the UAV thus completing all the six degrees of freedom that are required to
represent a UAV.
For avoiding obstacles, a LIDAR is being used. The point data produced by the
LIDAR is used for mapping the environment around the UAV. If any obstacles are
detected in this environment the UAV then dynamically alters its path to reach the set
point. A LIDAR is used instead of an array of infrared sensors as the LIDAR gives a
360-degree view of the surroundings whereas the infrared sensors would have some
blind spots between them.
A high resolution camera is also placed on the UAV. The main use of this camera
is to relay live feed to the control station. This feed is then used to detect human
beings.

2.2 UAV Working

The UAV achieves its autonomy with the help of a Cascaded Digital PID controller as
shown in Fig. 2. The PID Controller helps in determining if the UAV is moving in the
correct direction or not. The PID controller is a Digital controller as this reduces the
number of components that are to be used. A digital PID controller also is easier to
tune as no physical components are required giving higher accuracy than a physical
PID Controller.
The attitude controller is responsible for handling the roll, pitch and yaw of the
UAV. The IMU is used as feedback in this PID controller. The IMU provides real
time values for the roll, pitch and yaw of the UAV. These values are then used to
generate the error by comparing them to the input values. The attitude controller then

Fig. 2 Block diagram of the PID controller

Autonomous UAV with Human Detection 301

generates output in the form of PWM values. These PWM values are used to control
the speed of the motors on the UAV. Thus changing the orientation of the UAV.
The outer loop of the cascaded PID controller is the position controller. The
position controller uses the GPS and altimeter for feedback. The GPS provides the
real time latitude and longitude of the UAV which act as the x and y coordinates in
the system. The altimeter provides the altitude or the z coordinate. These values are
then used to generate the error by comparing them with the location provided by the
user from the control station. For a quadcopter style UAV the movement is produced
by changing the roll, pitch and yaw of the UAV. Therefore, the position controller
uses this error to generate the corresponding roll, pitch and yaw values to minimize
the error. These are the same roll, pitch and yaw values that are fed to the attitude
controller.
The UAV’s LIDAR is used to detect obstacles in real time. Simultaneous Local-
ization and Mapping (SLAM) is used to map the terrain and the obstacles detected
by the UAV. SLAM also helps the UAV to plan its path and dynamically alter it if
any obstruction has been met with by the UAV.

2.3 Convolutional Neural Network

The high resolution camera onboard the UAV is used for the detection of human
beings. Since the UAV does not have the processing power to run a Convolutional
Neural Network (CNN) and detect human beings only raw video is transmitted by
the UAV to the control station. The control station then passes this video feed via the
CNN to detect human beings.
The CNN converts is trained with over 6000 images of human beings and uses
techniques like Image Augmentation to improve its results. The CNN is made using
TensorFlow 2.0 an open source end to end machine learning platform.
The architecture comprises of 17 layers including four convolutional layers, four
max pooling layers, five rectifier linear unit layers, a flattening layer, two fully
connected layers and a sigmoid layer. The input layer contains an image with
150*150 pixels. The first convolutional feature map is 148*148. The second is 72*72.
Then, the third one is 34*34. The layers of max pooling have 2*2. The first max
pooling produces 74*74 feature maps. The second one gives 36*36 feature maps.
Then, the third one has 17*17 feature maps. The five Rectified Linear Unit Layers
(ReLU) are used between layers to clear the negative values. The last layers are fully
connected with 512 nodes. 1 node is connected to the sigmoid layer to give two
classes [3, 4].
302 K. Karnawat et al.

Fig. 3 CNN architecture

3 Results

The CNN trained for detecting human beings was run for a hundred epochs. The
accuracy and losses for each epoch were noted and the data was plotted as shown
in Figs. 4 and 5, respectively. On the X axis of Fig. 4 are the number of epochs and
on the Y axis the accuracy ranging from 0 to 1 is plotted. As seen in the plot the
accuracy at the start was 65% this increased after each epoch. The accuracy after
a 100 was in the range of 90–92%. Along with this the validation accuracy is also
plotted in red. The validation accuracy is not as stable as the training accuracy but
is increasing overall giving an accuracy of around 85%. Plotting the accuracy graph
helps us understand if the CNN is working properly or not.

Fig. 4 Training accuracy

achieved by the CNN
Autonomous UAV with Human Detection 303

Fig. 5 Training losses for

the CNN

In Fig. 5 the training losses have been plot similar to training accuracy the training
losses help in determining the efficiency of the CNN. The lower the loss is the better.
As seen in Fig. 5 there is a steady decrease in the loss. The losses nearing the end
are lower than 0.2. The validation losses are fluctuating and can be improved in the
future (Fig. 5).
The CNN gave an accuracy of 72% on the test dataset with an F1 score of 0.83.

References

1. Yahyanejad S, Rinner B A fast and mobile system for registration of low-altitude visual and
thermal aerial images using multiple small-scale UAVs
2. Kirkland G (2019) What are the levels of autonomy for self-driving vehicles? Robot Bus Rev
(26 July 2019)
3. AlDahoul N, Sabri AQM, Mansoor AM Real time human detection for aerial captured video
sequences via deep models
4. Krishna SL, Chaitanya GSR, Reddy ASH, Naidu AM, Poorna SS, Anuraj K Autonomous human
detection system mounted on a drone
5. Tathe SV, Narote SP Real-time human detection and trackin