Distributed Multi-Agent Deep Reinforcement

learning based Navigation and Control of UAV

Swarm for Wildfire monitoring
Adeeba Ali
Department of Computer Engineering
Aligarh Muslim University
Aligarh, India
aliadeeba98@gmail.com
Rashid Ali
Department of Computer Engineering
Aligarh Muslim University
Aligarh, India
rashidaliamu@rediffmail.com
M.F. Baig
Department of Mechanical Engineering
Aligarh Muslim University
Aligarh, India
drmfbaig@yahoo.co.uk

Abstract—Decentralized deep reinforcement learning is an
emerging and most effective approach to solve the problem of
resource allocation and coordination among the swarm of UAVs.
Nowadays, the use of autonomous aerial vehicles in wildfire
monitoring is increasing and considered as the reasonably feasible
option as the surveillance in calamity hit areas can benefit from
this kind of automation. The flocks of UAVs can generate the
maps of affected areas which could improve the process of relief
planning so that necessary aid can be reached to the burnt areas
quickly. This paper presents the Multi-agent Deep Q network
based technique for planning optimized trajectories for the UAV
swarm which can sense the wildfire in the forests and nearby
regions. In this work, the UAV agents are trained over simulated
wildfire in virtually generated forests with two reward schemes.
The simulation results verify the effectiveness of the proposed
strategy for leveraging it in real-world scenarios.
Index Terms—swarms, reinforcement learning, navigation,
wildfire, decentralized control, UAV
I. I NTRODUCTION
Since the last few years the spread of massive wildfire
all across the world has caused substantial environmental,
economical, and social effects [1]. And, the most dreadful
effects of wildfire include the death of human beings, as well
as other medical problems that arise due to high temperature,
smoke inhalation, debris, and vehicle crashes. Similarly, the
adverse effects of wildfires on environmental health include
degradation of air quality and the change in overall global
composition of atmosphere as wildfires emit Carbon Dioxide
(CO2) in large quantities, thus leading to global warming.
Furthermore, the uncontrolled wildfires are also responsible
for the extinction of some species of flora and fauna in that
particular location.
In order to reduce the amount of loss caused due to
wildfires, it is important to keep the account of the information
regarding the fuel map, movement, and spread of fire front and
climate conditions. The availability of this kind of accurate
real-time information can reduce the risk concerned with the
lives of fire fighters and increase the success in the efforts
made for controlling the spread of fire [2]. One of the main
sources to obtain real-time information regarding wildfires are
satellite images. However, these images can only be made
available if the satellite passes over that particular region
of interest. Moreover, in case of small fires, satellite images
lack high spatial resolution [3]. The other alternative source
for obtaining the valuable information are manned aerial
vehicles which are quite expensive and require a highly skilled
human pilot. The use of autonomous UAVs (Unmanned Aerial
Vehicles) in monitoring wildfires and recording the real-time
information is one of the most promising solutions [3]. The
flight controller present in the autonomous aerial vehicles
can accept the commands guiding the external maneuver in
addition to the ones communicated by remote controller. The
external commands include the GNSS coordinates and state
information provided by IMU (Inertial Measurement Unit).
External commands can be either transmitted from a base
station over a communication link, or can be sent directly
from the processor mounted on the drone and physically
connected to the flight controller. Generally, little human
resources are required by the autonomous UAVs as the system
can be launched using only a single operator, however in some
countries it is mandatory to assign at least one safety pilot to
control the drone in emergency cases.
The use of multiple UAVs instead of a single UAV increases
the robustness and efficiency of the system [2]. Furthermore,
the use of multiple smaller and cheaper aerial vehicles would
minimize the overall cost of the system, thereby increasing
the adoption of this kind of system by the organizations that
possess comparatively less financial resources. In contrast,
some additional challenges are associated with the use of
multiple UAVs, such as the need for an efficient algorithm that
can establish proper coordination among the UAVs. Although
the visible light and infrared vision based forest fire detection
and hardware required for designing a drone are in effect,
state-of-the-art algorithms designed for the control of UAV
swarms for wildfire monitoring have to be further investigated
[4-9].
In the literature, collaborative control of multiple UAV
agents can be achieved either by planning optimized and pre-
defined trajectories to the goal position or using the techniques
based on decentralized control theory or RHO (Receding Hori-
zon Optimization). In [11] the authors proposed an approach
that outperforms the traditional RHO method for wildfire
monitoring with two aircrafts. The technique presented in
[11] is based on deep reinforcement learning (Deep-RL), and
the results demonstrated in the paper shows that learning
based mechanisms for cooperative control tasks are promising
alternatives. Typically, the researchers leverage off-policy Q-
learners [12-13] which is an uncomplicated and widely used
approach in the field of multi-robot navigation and control. In
[5] the deep reinforcement learning has been previously used
for autonomous wildfire monitoring through multiple UAVs.
The objective behind this research is to investigate and
validate the techniques that can efficiently deal with the chal-
lenges associated with collaborative multi-agent reinforcement
learning leveraged for monitoring a wildfire with multiple
UAVs. Inspired by [5] we propose a DQN based solution
for cooperative control of multiple UAVs while monitoring
the wildfire. In addition to this, the effect of reward sharing
is explored in this research, which is considered as another
possible feedback scheme in swarm applications [14].
The remainder of the paper is organized as follows. The
review of related work is presented in Section II. Then the
problem is formally stated in Section III. Next, the implemen-
tation details of the formulated problem as POMDP (Partially
Observable Markov Decision Process) are discussed in Section
IV. This is followed in Section V by a detailed description
of the proposed solution for wildfire monitoring. Then the
proposed methodology is validated through simulations in
Section IV. Finally, the summary of the paper is presented
in Section VII.
II. R ELATED W ORK
UAVs are used to gather valuable data and information
through surveillance and continuous monitoring in a wide
range of applications, including smart agriculture [15], post-
disaster hazardous material [16], or wildfire monitoring [2].
In this work, the use of multiple UAVs in wildfire monitor-
ing application [2] is studied. Earlier, the infrared and visible
light cameras, as well as the combination of both of them,
have been leveraged for detecting wildfires, tracking fire front
or hot-spot searching after fire [3]. Various infrared-light-based
approaches for tracking wildfires from UAVs and satellites are
outlined in [17].
In contrast to satellite imaging, aerial wildfire detection
and monitoring provides a high quality temporal and spatial
resolution [3], [18]. In [19], the authors demonstrated the
operational characteristics of a remotely controlled UAV for
monitoring wildfire in large environments. Using multiple
UAVs for the tasks of surveillance and monitoring is more
efficient with respect to a single UAV [3]. Techniques to
control the flock of UAVs during a simulated wildfire detection
and monitoring were presented in [2], [7-8], [20-22]. In [23]
authors demonstrated an outline decentralized mechanism to
estimate the wildfire model. The presented strategy was tested
using real-time drone hardware and stochastic fire simulator
was used for generating data. Later, in [24] the authors
validated their proposed cooperative control algorithm in real-
time field experiments.
The aforementioned approaches leverage control theory
techniques to estimate the UAVs’ movements. In [11], the
authors showed that the traditional control theory techniques
are outperformed by deep reinforcement learning methods for
the task of wildfire monitoring with two aircrafts using a
random wildfire simulator.
In [6] the authors put forward a deep reinforcement learning
strategy to search for burning trees and then apply a retardant
for extinguishing the fire. Similarly, in [25] the authors inves-
tigated an equilibrium based Q-learning variant to successfully
complete the coverage task. Moreover, deep reinforcement
learning has also been utilized to gather relevant information
through learning a particular task. In [27], the authors present
the DeepIG method to allow the agents to capture valuable
and relevant information without colliding with each other.
The proposed algorithm was validated with three UAVs in a
terrain mapping experiment.
The algorithms proposed in [11], [25-27] were leveraged
for learning and establishing coordination among multi-robot
agents, thus exemplifying the challenges associated with col-
laborative multi-agent reinforcement learning problems.
III. P ROBLEM F ORMULATION
A. Environment
The simulated wildfire surveillance environment consists of
a 2-D map representing a 1 Km2 area which is discretized into
100×100 grid of burnable cells. The generated dynamic maps
of the environment are of two types:
• B(s): Fuel map in which each cell represents the amount
of flammable material present in that particular region.
• F(s): Fire map which is a boolean representation of the
environment consisting of either the value ‘0’ or ‘1’ in
each cell where ‘1’ represents the presence of fire in that
particular cell.
The simulated wildfire environment with the evolution of fire
across the maps over an episode is shown in Fig. 1. The cells
of the fuel map are updated every 2.5 sec in accordance to the
following two equations:
(
max(0, B t (s) − βp(s)), if F t (s) = 1
B t+1 (s) = (1)
B t (s), otherwise
where β is the burning rate which is set to 1, and
(
1 − s′ |d(s,s′ )<dmp (1 − P (s, s′ )Ft (s′ )), if Bt (s) > 0
Q
p(s) =
0, otherwise
(2)
where p(s) is the probability that the cell s will ignite, d(s, s′ )
is the distance between cells s and s’, dmp is the maximum
distance at which propagation can happen and P (s, s′ ) is
the probability that cell s′ ignites cell s which is defined as
follows:
P (s, s′ ) = w.d(s, s′ )−2 (3)
where w is constant proportional to the wind speed.
 (a) t = 0s. (b) t = 100s.

(c) t = 200s. (d) t = 300s.

(e) t = 400s. (f) t = 500s.

Fig. 1: An example wildfire evolution over time. Left: burning cells. Right: fuel left.

B. Agents IV. S WARMDP D ESCRIPTION

Fixed-wing aircraft are considered as an agents for this
problem. It is assumed that the aircraft will fly with a constant
velocity v of 20 m/s and each agent is allowed to decide
whether to increase or decrease its bank angle ϕ by 5 degrees
at 10 Hz frequency. The position of the aircraft is updated
according to the kinematic model of fixed-wing aerial vehicle
is represented by the following expressions:
g tan ϕ
ẋ = v cos ϕ, ẏ = v sin ϕ, ψ̇ = (4)
v
where g is acceleration due to gravity , 9.8ms−2 . The maxi-
mum bank angle that a UAV can take is set to 50 degrees and
UAV maneuvers exceeding this range will not be considered
valid. This constraint will set a realistic limit to angular
velocity.
Rather using Partially Observable Markov Decision Process
(POMDP) as a model [28], the navigation problem is framed
as a special case of Decentralized POMDP, that is swarMDP
[29]. Through this design the multi-agent settings can be
explicitly modeled by assuming that all the agents have similar
configuration.
Definition 1. The swarMDP problem is formulated in two
steps. First, a prototype A = ⟨S, A, Z, π⟩, where A represents
the instance of each UAV agent of the swarm and where:
• S represents the set of local states.
• A represents the set of local actions.
• Z represents the set of local observations.
• π : Z → A is the local policy.
A swarMDP is defined using a tuple of 7 dimensions
⟨N, A, P, R, O, γ⟩ where:
 • N is the set of indices corresponding to the agents. The D. Rewards
index values in the set should be in range from 1 to N, The reward receives by each agent i is the sum of four
where N is the total number of agents. independent components:
• A is the prototype of the agent as defined above.
• P : S × S × A → [0, 1] is the transition probability
r1 = −λ1 min ds , (5)
function where P (s′ |s, a) denotes the probability that the {s∈S|Ft (s)}
agent will move to state s’ from state s after taking action
a.
X
r2 = −λ2 1 − Ft (s), (6)
• R : S × A → R is the reward function, where R(s, a) is {s∈S|ds ≤r0 }
the reward given to an agent when it takes action a in
state s. r3 = −λ3 ϕ20 , (7)
• O : S × Z × A → [0, 1] is the observation model, where
O(z ′ |s, a) is the probability that an agent observes z’ X ρij
after playing an action a in state s. r4 = − λ4 exp(− ) (8)
ϵ
• γ ∈ [0, 1] is the discount factor for the total reward {j∈1,2,...,N ∧j̸=i}
accumulated by the agent in future.
where ds is the distance from the UAV to cell s of the map.
A. State The aforementioned equations negative rewards or penalties
for the following:
The local state of each agent depends on following factors:
• Distance from fire (r1 ): proportional to the distance to
• Fuel map B(s) which has to be updated every 2.5 sec by
the closest burning cell.
the wildfire simulator. • Safe cells nearby (r2 ): proportional to the number of
• The positional coordinates (xi , yi ).
unburnt cells in a radius r0 .
• The heading angle ψi .
• High bank angles (r3 ): proportional to the square of the
• The bank angle ϕi .
bank angle (ϕ).
• Closeness to other UAV (r4 ): sum of the contributions of
B. Actions
each one of the other UAVs, which sets to the constant
The set of local actions for each agent is given by A = ⟨ value of λ4 at the time of saturation.
decrease ϕ by 5 degrees, increase ϕ by 5 degrees ⟩. The agent The constant parameters used in above equations have been
is required to choose any of them at a period of 0.1 sec. given the following values: λ1 = 10, λ2 = 1, λ3 = 100, λ4 =
1000, r0 = 10, c = 100.
C. Observations
The reason behind tuning the parameters to these values is
Each agent in the swarm gathers two kinds of observations: that flying away from the burning region should be penalized
a feature vector representing some information regarding the heavily and, once over the fire region, the negative rewards
local state of the UAV and other UAVs; an image as the partial obtained from bank angle and getting close to other UAVs
observation of the wildfire from the perspective of the UAV. should become dominant.
The feature vector associated with each agent i is the result
of the concatenation of four other vectors related to the state V. M ETHODOLOGY
variables of UAVs: the set of all the bank angles ϕ = {ϕj |j ∈ A. Deep Q-Networks
1, 2, ..., N }; the range of distance to other UAVs ρi = {ρij |j ∈
1, 2, ..., N ∧ j ̸= i}; the relative heading angle to other UAVs, The objective of the presented approach is to find the policy
θi = {θij |j ∈ 1, 2, ..., N ∧ j ̸= i}; the relative heading angle π that can optimally map observations to actions. The degree
of other UAVs, ψi = {ψij |j ∈ 1, 2, ..., N ∧ j ̸= i}. of optimality depends on the reward function R that has been
The images are captured using the optical hemispherical defined in the previous section. From the optimal policy, pi
camera of the UAV with the view angle of 160 degrees facing the value Q(s, a) can be determined for each state-action pair.
downwards, and can take pictures up to the maximum range The value function Q(s, a) represents the expected cumulative
of 500 m. After processing the image is downsampled to the reward assigned to an agent that takes action, a in state s. The
resolution of 30 × 40 pixels representing the burning trees and value of the Q function can be determined using the following
forest. Bellman equation:
The image is captured after sampling the points of interest
X
Q(s, a) = r(s, a) + γ p(s′ |s, a) max
′
Q(s′ , a′ ) (9)
at the horizontal angle of 40 degrees from the cardinal and s′ ∈S
a ∈A
at 30 degrees of elevation from nadir to 10 degrees below
the horizon. This strategy results in a high quality image of For a given Q value function the optimal policy can be deduced
the region right below the UAV. The characteristics of this as:
image determines the degree of partial observability of the
environment. π(s) = arg max Q(s, a) (10)
a∈A
Using DQN [30], [31] the Q-value can be estimated by training
a deep neural network that can approximate the function.
Following loss function is used for training the network:
Li (θi ) = Es,a,r,s′ [e2 ] (11)
where e is the Bellman error:
e = r(s, a) + γ max
′
Q(s′ , a′ ; θ− ) − Q(s, a; θ) (12)
a
During the training of the network the agent has to go through
the trade-off between the exploration of state-action space
using random policy and the exploitation of the previous
knowledge that has been obtained through the estimation of
the optimal policy in previous states. Typically, this problem
can be tackled by leveraging exploratory policy in the first few
training iterations and then gradually increasing the fraction of
actions related to exploitation policy.
Deep Q Networks (DQN) circumvent the inherent instability
problem of deep reinforcement learning through two mecha-
nisms:
• By randomizing the training samples using experience
replay method to reduce the correlation effect of the se-
quence. This can be implemented using a replay memory
E.
• By training the target network (with parameters θ− ) at
the same time but at a lower update rate than the first
network (with parameters θ) to predict Q(s′ , a′ ), so that
the first network can learn the fixed target. The complete
algorithm of the implemented DQN based wildfire mon-
itoring task is presented in Algorithm 1.
Algorithm 1 Deep Q-Networks (DQN)
Input: Parameters: ∈, L, N
Output: π, the approximate optimal policy
1: Initialize replay memory E to capacity L
2: Initialize θ = θ − randomly
3: for each episode do
4: Initialize s0
5: for each step t in the episode do
6: Take action at π∈ (.|s) and observe rt , st+1
7: Update memory replay E with current sample et =
(st , at , st+1 , rt )
8: Sample randomly a set N of N indices from E
9: for each experienced sampled et do
10: Obtain Bt
11: Update: θ ← Adam((Bt − Q(st , at ; θ))2 )
12: Update: θ − ← θ
13: for all s ∈ S do
14: π(s) ← arg maxa′ ∈A Predict ((s, a′ ), θ − )
15: return π, θ
where Bt = rt + γ maxa′ Q(st+1 , at+1 ; θ− ) for non-
terminal st+1 and bt = rt for terminal st+1 .
B. Network Architecture
In the implemented method the deep Q networks deal with
two different types of observations separately. The feature vec-
tor ⟨N, A, P, R, O, γ⟩ goes through a series of dense hidden
layers each consisting of 100 neurons with ReLU activation
function. Next, the monocular image with partial informa-
tion of the environment is processed by a series of dense
convolutional and max pooling layers which downsampled
the image before feeding it to the dense hidden layers of
neurons. The outputs obtained from both the networks are
concatenated and processed by two additional dense layers
which finally estimate Q-values for the agents. The complete
network architecture is shown in Fig. 2.
Fig. 2: Network Architecture
C. Reward Schemes
During the training of Deep Q networks two kinds of
reward approaches are compared which differ in their level of
decentralization. Once the network is trained both the models
can be used for decentralized control.
1) Independent rewards: Each agent obtains its own reward
separately from the environment.
2) Shared reward: It is calculated as the mean of the
independent rewards and shared by all the agents.
VI. E XPERIMENTAL R ESULTS
A. Simulation Setup
Both the DQN based fire detection models trained with
two different reward configurations are tested in simulated
environments. Training of the network is performed every 100
steps after the collection of 100 new samples. In every iteration
of training a batch size of 2000 samples is used, randomly
taken from the buffer of the last 100,000 examples. The target
network is updated after every update of 10 action values, that
is, once every 1000 steps. In order to prevent the overfitting
of the network with the first sequence of samples, the learning
of the network starts after the first 2000 steps. The choice of
training the network with exploratory actions starts at fraction
of 1 and then linearly decreases during the first 70 percent of
training time and finally stays at the constant value of 0.1
 (a) t = 100s.
(c) t = 300s.
Fig. 3: An example of an episode in wildfires with two aircraft trained with DQN, team reward scheme.
until the training ends. The simulation experiments have a
duration of 10,000 steps (20 minutes). Adam [32] is used as
the stochastic optimizer of the network and the learning rate
is set to the value of 5 × 10−9 .
B. Simulation Results
The major performance metric of the proposed approach
is the accumulated reward of all simulated episodes which is
determined as the sum of the instantaneous rewards earned by
all agents over an episode. Fig. 3 represents the simulation of
(b) t = 200s.
(d) t = 400s.
an episode demonstrating the behavior learnt by the agents. As
the wildfire propagates in the region, the UAVs learn to head
towards the burning areas and start making wider circles. The
third component (r3 ) of the reward function, that is peanlty
for the collision between aircraft is effective in making the
agents to circle the burning region in the same direction and
at opposite end points of the fire while staying far away from
each other. When the region of interest (ROI) is too small,
the penalty arises due to the proximity between the UAVs
outweighs the other and one of the UAVs stays there making
small circles. Furthermore, As demonstrated in Fig. 4 with
both the reward configurations agents are allowed to learn the
monitoring task effectively by minimizing their penalties on
average. It is expected that similar performance is obtained
with both the reward schemes for this problem as three of
the four reward components (r1 , r2 , r3 ) are independent of
learning behavior other UAV agents that is, only optimization
of the relative positions of the UAV with respect to other can
affect the margin of improvement.
VII. C ONCLUSION
In this paper, a DQN based approach is proposed to provide
decentralized control to multiple UAVs for monitoring the
spread of wildfires. The deep Q networks take a processed
monocular image consisting of partial observation of the
surroundings and the local state vectors of the UAV as input
and generate the control actions for the agents. The proposed
method is tested with 2 reward configurations which differ in
their degree of federating multiple UAV agents. Simulation
results show that UAVs can effectively monitor the wildfire
in a cooperative and coordinated manner with both types of
reward schemes. Thus, the presented control design can be
applied to real time UAV aided surveillance in wildfire regions
to minimize the operational costs. The DQN are trained on
simulated fire maps from forests but the model should perform
well on all types of wildfires as long as enough data is
available for training.
Fig. 4: Comparison of reward schemes at the time of training
R EFERENCES
[1] A. M. Gill, S. L. Stephens, and G. J. Cary, “The worldwide “wildfire”
problem,” Ecological applications, vol. 23, no. 2, pp. 438–454, 2013.
[2] H. X. Pham, H. M. La, D. Feil-Seifer, and M. Deans, “A distributed
control framework for a team of unmanned aerial vehicles for dynamic
wildfire tracking,” in 2017 IEEE/RSJ International Conference on Intel-
ligent Robots and Systems (IROS). IEEE, 2017, pp. 6648–6653.
[3] C. Yuan, Y. Zhang, and Z. Liu, “A survey on technologies for automatic
forest fire monitoring, detection, and fighting using unmanned aerial
vehicles and remote sensing techniques,” Canadian journal of forest
research, vol. 45, no. 7, pp. 783–792, 2015.
[4] F. A. Hossain, Y. Zhang, and C. Yuan, “A survey on forest fire
monitoring using unmanned aerial vehicles,” in 2019 3rd International
Symposium on Autonomous Systems (ISAS), 2019, pp. 484–489.
[5] N. K. Ure, S. Omidshafiei, B. T. Lopez, A. Agha-Mohammadi, J. P. How,
and J. Vian, “Online heterogeneous multiagent learning under limited
communication with applications to forest fire management,” in 2015
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS), 2015, pp. 5181–5188.
[6] R. N. Haksar and M. Schwager, “Distributed deep reinforcement learn-
ing for fighting forest fires with a network of aerial robots,” in 2018
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS), 2018, pp. 1067–1074.
[7] Z. Lin and H. H. Liu, “Topology-based distributed optimization for
multi-uav cooperative wildfire monitoring,” Optimal Control Applica-
tions and Methods, vol. 39, no. 4, pp. 1530–1548, 2018.
[8] F. Afghah, A. Razi, J. Chakareski, and J. Ashdown, “Wildfire monitoring
in remote areas using autonomous unmanned aerial vehicles,” in IEEE
INFOCOM 2019 - IEEE Conference on Computer Communications
Workshops (INFOCOM WORKSHOPS), 2019, pp. 835–840.
[9] A. Sargolzaei, A. Abbaspour, and C. D. Crane, Control of Cooperative
Unmanned Aerial Vehicles: Review of Applications, Challenges, and Al-
gorithms. Cham: Springer International Publishing, 2020, pp. 229–255.
[10] S. Chung, A. A. Paranjape, P. Dames, S. Shen, and V. Kumar, “A survey
on aerial swarm robotics,” IEEE Transactions on Robotics, vol. 34, no.
4, pp. 837–855, 2018.
[11] K. D. Julian and M. J. Kochenderfer, “Distributed wildfire surveillance
with autonomous aircraft using deep reinforcement learning,” Journal of
Guidance, Control, and Dynamics, pp. 1–11, 2019.
[12] C. Boutilier, “Planning, learning and coordination in multiagent decision
processes,” in Proceedings of the 6th conference on Theoretical aspects
of rationality and knowledge. Morgan Kaufmann Publishers Inc., 1996,
pp. 195–210.
[13] A. Tampuu, T. Matiisen, D. Kodelja, I. Kuzovkin, K. Korjus, J. Aru,
J. Aru, and R. Vicente, “Multiagent cooperation and competition with
deep reinforcement learning,” PloS one, vol. 12, no. 4, p. e0172395,
2017.
[14] M. Hüttenrauch, A. Šošić, and G. Neumann, “Deep reinforcement
learning for swarm systems,” Journal of Machine Learning Research,
vol. 20, no. 54, pp. 1–31, 2019.
[15] L. Tang and G. Shao, “Drone remote sensing for forestry research and
practices,” Journal of Forestry Research, vol. 26, no. 4, pp. 791–797,
2015.
[16] P. G. Martin, S. Kwong, N. Smith, Y. Yamashiki, O. D. Payton, F. Russell
Pavier, J. S. Fardoulis, D. Richards, and T. B. Scott, “3d unmanned
aerial vehicle radiation mapping for assessing contaminant distribution
and mobility,” International journal of applied earth observation and
geoinformation, vol. 52, pp. 12–19, 2016.
[17] L. Hua and G. Shao, “The progress of operational forest fire monitoring
with infrared remote sensing,” Journal of forestry research, vol. 28, no.
2, pp. 215–229, 2017.
[18] R. S. Allison, J. M. Johnston, G. Craig, and S. Jennings, “Airborne op-
tical and thermal remote sensing for wildfire detection and monitoring,”
Sensors, vol. 16, no. 8, p. 1310, 2016.
[19] T. J. Zajkowski, M. B. Dickinson, J. K. Hiers, W. Holley, B. W. Williams,
A. Paxton, O. Martinez, and G. W. Walker, “Evaluation and use of
remotely piloted aircraft systems for operations and research–rxcadre
2012,” International Journal of Wildland Fire, vol. 25, no. 1, pp.
114–128, 2016.
[20] D. W. Casbeer, R. W. Beard, T. W. McLain, S.-M. Li, and R. K. Mehra,
“Forest fire monitoring with multiple small uavs,” in Proceedings of the
2005, American Control Conference, 2005. IEEE, 2005, pp. 3530–3535.
[21] P. Sujit, D. Kingston, and R. Beard, “Cooperative forest fire monitoring
using multiple uavs,” in 2007 46th IEEE Conference on Decision and
Control. IEEE, 2007, pp. 4875–4880.
[22] M. Kumar, K. Cohen, and B. HomChaudhuri, “Cooperative control
of multiple uninhabited aerial vehicles for monitoring and fighting
wildfires,” Journal of Aerospace Computing, Information, and Commu-
nication, vol. 8, no. 1, pp. 1–16, 2011.
[23] N. K. Ure, S. Omidshafiei, B. T. Lopez, A. Agha-Mohammadi, J. P. How,
and J. Vian, “Online heterogeneous multiagent learning under limited
communication with applications to forest fire management,” in 2015
IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS), 2015, pp. 5181–5188.
[24] L. Merino, F. Caballero, J. R. Martı́nez-de Dios, J. Ferruz, and A.
Ollero, “A cooperative perception system for multiple uavs: Application
to automatic detection of forest fires,” Journal of Field Robotics, vol.
23, no. 3-4, pp. 165–184, 2006.
[25] H. X. Pham, H. M. La, D. Feil-Seifer, and A. Nefian, “Cooperative and
distributed reinforcement learning of drones for field coverage,” arXiv
preprint arXiv:1803.07250, 2018.
[26] Y. Xiao, J. Hoffman, T. Xia, and C. Amato, “Multi-robot deep reinforce-
ment learning with macro-actions,” arXiv preprint arXiv:1909.08776,
2019.
[27] A. Viseras and R. Garcia, “Deepig: Multi-robot information gathering
with deep reinforcement learning,” IEEE Robotics and Automation
Letters, vol. 4, no. 3, pp. 3059–3066, 2019.
[28] K. D. Julian and M. J. Kochenderfer, “Distributed wildfire surveillance
with autonomous aircraft using deep reinforcement learning,” Journal of
Guidance, Control, and Dynamics, pp. 1–11, 2019.
[29] A. Šošić, W. R. KhudaBukhsh, A. M. Zoubir, and H. Koeppl, “Inverse
reinforcement learning in swarm systems,” in Proceedings of the 16th
Conference on Autonomous Agents and MultiAgent Systems. Inter-
national Foundation for Autonomous Agents and Multiagent Systems,
2017, pp. 1413–1421.
[30] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D.
Wierstra, and M. Riedmiller, “Playing Atari with deep reinforcement
learning,” arXiv preprint arXiv:1312.5602, 2013.
[31] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G.
Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et
al., “Human-level control through deep reinforcement learning,” Nature,
vol. 518, no. 7540, p. 529, 2015.
[32] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
arXiv preprint arXiv:1412.6980, 2014.