The 12th International Conference on Wireless Communications and Signal Processing

Research on Dynamic Probability Mechanism of

Rebroadcasting for UAV Swarm

Zhen Qin∗ , Tingjun Du† , Fei Xiong‡ , Hai Wang∗§ , Aijing Li∗ , Yajie Chen∗
∗ College of Communications Engineering, Army Engineering University of PLA, Nanjing, China
† Unit 78102 of PLA, Chengdu, China
‡ CMC Political and Law Commission, Beijing, China
§ Corresponding author: Hai Wang
Email: qzqzla912@163.com; 1874404772@qq.com; xbibid@126.com; hai.wang@gmail.com;
lishan wh@126.com; chenyajie0705@gmail.com

Abstract—Unmanned aerial vehicle (UAV) swarm needs to
broadcast routing messages to each other, accept unified control
commands, and return the discovered targets’ information to
the ground control station in time. Most of these messages are
broadcast in the UAV swarm. Therefore, in the UAV swarm
network, the efficiency of broadcasting and rebroadcasting will
determine the network efficiency of the entire network. Aiming at
the broadcasting problem of UAV network, in order to adapt to
the dynamic nature of the network and improve the broadcasting
ability and efficiency, this paper focuses on the rebroadcasting
mechanism based on dynamic probability. A prediction method
of collision probability of rebroadcasting behavior, and a multi-
agent cooperative rebroadcasting method based on virtual action
and reinforcement learning are proposed. Simulation results show
that the dynamic probability mechanism of rebroadcasting has
obvious performance advantages over fixed probability flooding
and simple flooding in dynamic scenarios after pre-learning.
Index Terms—UAV swarm, rebroadcast, dynamic probability,
reinforcement learning.
I. I NTRODUCTION
overwhelm the network. This overwhelming network results
in overly redundant rebroadcasting (some nodes receive the
same packet more than once), contention and collision, which
are referred to as the Broadcast Storm Problem (BSP) [5].
When UAVs participate in the swarm mission, there is also
the demand of network broadcasting, such as the broadcasting
of safe flight information between UAVs, the broadcasting
of group formation status information, the broadcasting of
command and control information and perception information,
etc. Therefore, the BSP is the core problem to be solved first
for achieving UAV swarm control.
To achieve efficient broadcasting and solve the BSP, many
methods have been proposed [5]– [13]. The main approach to
solve the BSP in MANET is focused on how to reduce the
amount of redundant retransmissions, which can be achieved
by preventing a subset of the network nodes from doing
the retransmission. In general, these broadcast protocols are
categorized into: (i) Probability-based methods, which are

U NMANNED aerial vehicles (UAVs) can cooperate with

each other to form a network mode of hop-by-hop
transmission, which is like traditional mobile ad hoc net-
work (MANET) [1]– [2]. All UAVs can cooperate to form
connections without the help of fixed facilities such as base
stations and access points. Each UAV communicates with the
surrounding nodes to form the routing function. If UAVs are
in the communication range, they can communicate directly.
Otherwise, data transmission between UAVs requires the as-
sistance of other UAVs [3]. Therefore, UAV swarm network
can also be considered as a new application of MANET.
Broadcasting is a basic operation and has extensive ap-
plications in MANET. Basic flooding or blind flooding is
the simplest way of broadcasting a packet to all nodes in
a network [4]. Flooding requires each node to rebroadcast
a packet to its neighbors once it receives a new packet.
The rebroadcasting behavior continues until all nodes in the
network receive the packet. The main advantage of basic
flooding is that it can always find the shortest path between
sources and destinations, since the packets have been broad-
casted through every possible path in parallel. However, the
basic flooding mechanism may trigger a large number of
packets to be forwarded in MANET which will eventually
similar to basic flooding, except that each node rebroadcasts
packets with a predetermined probability. These mechanisms
might work in dense networks when multiple nodes have
similar neighbor coverage, but they will not have a significant
effect in sparse networks. (ii) Area-based methods, in which a
node rebroadcasts a packet based on the distance between itself
and the node receiving the data packet. A rebroadcast occurs
only when the distance is longer than the predefined threshold,
so that a larger additional area can be reached. However,
area-based methods, without considering whether some nodes
actually exist within that additional area, can lead to inefficient
broadcasting. (iii) Neighbor knowledge methods, which can
be further classified as neighbor-designated methods and self-
pruning methods. In neighbor-designated methods, a node
transmits a packet and specifies one of its one-hop neighbors
which should forward the packet. In self-pruning methods,
a node receiving a packet will decide whether or not to
transmit the packet by itself. However, most existing broadcast
methods assume either the underlying network topology is
static or quasi-static during the broadcast process, such as the
neighborhood information can be updated in a timely manner.
Results of the paper [14] show that most broadcast protocols
suffer from low delivery ratio in highly mobile networks.

978-1-7281-7236-1/20/$31.00 ©2020 IEEE 1052

Authorized licensed use limited to: ULAKBIM UASL ISTANBUL TEKNIK UNIV. Downloaded on January 15,2024 at 10:32:01 UTC from IEEE Xplore. Restrictions apply.

Nodes for next hop
Nodes for current hop
A4
Nodes for previous hop
R25
R24
R22
R23 R26
R27
R21
R12 R11
R28
R13
R0
R14
R15
R16
R17
Fig. 1: Sketch map of rebroadcasting behavior
Therefore, these broadcast protocols are not suitable for UAV
swarm which has high dynamic characteristic.
In this paper, a dynamic probability-based rebroadcasting
method is formulated, in which each node forecasts the
probability of packet contention and collision according to
its neighbors’ positions dynamically, then evaluates the re-
broadcasting actions to form a dynamic probability strategy.
To model the rebroadcasting actions evaluation problem, a
collision probability prediction method, and a multi-agent
cooperative rebroadcasting method based on virtual action and
reinforcement learning (RL) method are proposed. Simulation
results validate the dynamic probability-based rebroadcasting
method is more suitable for dynamic network than conven-
tional probability-based methods. The main contributions of
this paper are summarized as follows:
• The rebroadcasting actions evaluation problem is inves-
tigated in multi-UAV scenarios, and the coordination
among UAVs is considered. And we propose a collision
probability prediction method for rebroadcasting behav-
ior.
• We propose a multi-agent cooperative rebroadcasting
method based on virtual action and RL to obtain a
dynamic probability strategy for rebroadcasting.
The rest of this paper is organized as follows. The system
model and problem formulation are presented in Section II.
Then we present a multi-agent cooperative rebroadcasting
method based on virtual action and RL in Section III. Simula-
tion results are provided and analyzed in Section IV. Finally,
we conclude the paper in Section V.
II. T HE MODEL OF REBROADCASTING ACTIONS
EVALUATION PROBLEM
A. An example
As illustrated in Fig. 1, broadcasting is in progress in the
UAV swarm. The maximum communication distance of UAV
is Cm . Node R0 is a source node or a relay node which
is broadcasting packet. NR0 = {R11 , R12 , . . . , R17 } is the
neighbors set of R0 . The nodes in NR0 will rebroadcast the
packet received from R0 . The nodes in the communication
Authorized licensed use limited to: ULAKBIM UASL ISTANBUL TEKNIK UNIV. Downloaded on January 15,2024 at 10:32:01 UTC from IEEE Xplore. Restrictions apply.
overlap areas of other nodes may lose the rebroadcasting
packet because of contention and collision. In order to improve
the rebroadcasting performance under dynamic conditions,
a dynamic probability strategy is needed. This strategy can
forecast the rebroadcasting actions of neighbors, and calculate
the probability of contention and collision. Nodes can choose
A3 to forward or drop a packet following this strategy after
A2
receiving a packet.
Let us analyse the rebroadcasting effect of node R11 . We
A1 assume that hello messages are only transmitted in one hop
range. Therefore, R11 cannot know the existence of node
R14 and R17 . R11 only consider the contention and collision
with R12 and R13 when it prepares for rebroadcasting. We
characterize NR11 ∪R12 ∪R13 = {R21 , R22 , . . . , R28 } as the set
of next hop nodes of R11 , R12 , R13 . According to the possible
contention and collision for rebroadcasting, four areas can be
divided as follow in Fig. 1:
A1 : NA1 = NR11 ∩R13 = {R27 } ∈
/ NR0 ∪R12 , (1)
A2 : NA2 = NR11 ∩R12 ∩R13 = {R26 } ∈
/ NR0 , (2)
A3 : NA3 = NR11 = {R25 } ∈
/ NR0 ∪R12 ∪R13 , (3)
A4 : NA4 = NR11 ∩R12 = {R23 , R24 } ∈
/ NR0 ∪R13 . (4)
When R11 receives a packet, it should select an action from
forwarding and dropping based on the probability of rebroad-
casting. About the rebroadcasting, we propose an evaluation
method. It has two parts: (i) The profit of the action. (ii) The
loss of the action. The action should be rewarded when the
profit is greater than the loss, vice versa.
The definition of Forward Profit (FP) is the expectation of
the number of next hop nodes which successfully receive a
new packet from this forwarding. Such as the FP of R11 is the
expectation of the number of nodes which successfully receive
a new packet from R11 ’s forwarding. These nodes should be
in set NR11 ∪R12 ∪R13 , and not in set NR0 .
The definition of Forward Loss (FL) is the expectation of the
number of next hop nodes which fail to receive a new packet
from this forwarding due to the contention and collision.
Such as the FL of R11 is the expectation of the number
of nodes which fail to receive the new packet from R11 ’s
forwarding. These nodes should be in set NR11 ∪R12 ∪R13 , and
not in set NR0 . The contention and collision are caused by the
simultaneous forwarding of R11 and other nodes in set NR0 .
The definition of Drop Profit (DP) is the expectation of
the number of next hop nodes which successfully receive a
new packet due to this dropping. Such as the DP of R11 is the
expectation of the number of nodes which successfully receive
a new packet from the forwarding of other nodes with the
dropping of R11 . These nodes should be in set NR11 ∪R12 ∪R13 ,
and not in set NR0 .
The definition of Drop Loss (DL) is the expectation of the
number of next hop nodes which fail to receive a new packet
due to this dropping. Such as the DL of R11 is the expectation
of the number of nodes which fail to receive a new packet
due to the dropping of R11 . These nodes should be in set
1053
 NR11 ∪R12 ∪R13 , and not in set NR0 . The loss is caused by the
simultaneous dropping of R11 and other nodes in set NR0 .
We characterize Pji as the probability of node i forwarding
broadcast packets from node j. The calculations of profit and
loss are as follow:
1) The FP of R11 : There is a node in set NA1 . This node
can receive the packet from R11 when R11 forwards and R13
drops. As a result, the FP of R11 in set NA1 is:
F P rR11 A1 = PR0 R11 (1 − PR0 R13 ) × 1.
There is a node in set NA2 . This node can receive the packet
from R11 when R11 forwards and R12 , R13 drop. As a result,
the FP of R11 in set NA2 is:
F P rR11 A2 = PR0 R11 (1 − PR0 R12 )(1 − PR0 R13 ) × 1.
There is a node in set NA3 . This node can receive the packet
from R11 when R11 forwards. As a result, the FP of R11 in
set NA3 is:
F P rR11 A3 = PR0 R11 × 1.
There are two nodes in NA4 . These two nodes can receive
the packet from R11 when R11 forwards and R12 drops. As
a result, the FP of R11 in set NA4 is:
F P rR11 A4 = PR0 R11 (1 − PR0 R12 ) × 2.
In summary, the FP of R11 is:
F P rR11 = F P rR11 A1 + F P rR11 A2 + F P rR11 A3 + F P rR11 A4
= PR0 R11 (1 − PR0 R13 )+
PR0 R11 (1 − PR0 R12 )(1 − PR0 R13 )+
PR0 R11 + 2PR0 R11 (1 − PR0 R12 ).
(9)
2) The FL of R11 : The node in set NA1 fails to receive
the packet from R11 when R11 and R13 forward at the same
time. As a result, the FL of R11 in set NA1 is:
F LrR11 A1 = PR0 R11 PR0 R13 × 1.
The node in set NA2 fails to receive the packet from
R11 when R11 forwards while at least one of R12 and R13
forwards. As a result, the FL of R11 in set NA2 is:
F LrR11 A2 = PR0 R11 [PR0 R12 (1 − PR0 R13 )+
(1 − PR0 R12 )PR0 R13 + PR0 R12 PR0 R13 ] × 1.
(11)
The node in set NA3 can receive the packet from R11 when
R11 forwards. As a result, the FL of R11 in set NA3 is:
F LrR11 A3 = 0.
Two nodes in set NA4 fail to receive the packet from R11
when R11 and R12 forward at the same time. As a result, the
FL of R11 in set NA4 is:
F LrR11 A4 = PR0 R11 PR0 R12 × 2.
Authorized licensed use limited to: ULAKBIM UASL ISTANBUL TEKNIK UNIV. Downloaded on January 15,2024 at 10:32:01 UTC from IEEE Xplore. Restrictions apply.
In summary, the FL of R11 is:
F LrR11 = F LrR11 A1 + F LrR11 A2 + F LrR11 A3 + F LrR11 A4
= PR0 R11 PR0 R13 + PR0 R11 [PR0 R12 (1 − PR0 R13 )+
(1 − PR0 R12 )PR0 R13 + PR0 R12 PR0 R13 ]+
2PR0 R11 PR0 R12 .
(14)
The reward of the forwarding action of R11 is:
(5)
F rR11 = F P rR11 − F LrR11 . (15)
This forwarding has more advantages than disadvantages
when F rR11 > 0, otherwise the opposite.
3) The DP of R11 : The node in set NA1 can receive the
packet when R11 drops and R13 forwards. As a result, the DP
(6) of R11 in set NA1 is:
DP rR11 A1 = (1 − PR0 R11 )PR0 R13 × 1. (16)
The node in set NA2 can receive the packet when R11 drops
while R12 or R13 forwards. As a result, the DP of R11 in set
(7) NA2 is:
DP rR11 A2 =(1 − PR0 R11 )[PR0 R12 (1 − PR0 R13 )+
(17)
(1 − PR0 R12 )PR0 R13 ] × 1.
The node in set NA3 fails to receive the packet when R11
(8) drops. As a result, the DP of R11 in set NA3 is:
DP rR11 A3 = 0. (18)
Two nodes in set NA4 can receive the packet when R11
drops and R12 forwards. As a result, the DP of R11 in set
NA4 is:
DP rR11 A4 = (1 − PR0 R11 )PR0 R12 × 2. (19)
In summary, the DP of R11 is:
DP rR11 =DP rR11 A1 + DP rR11 A2 + DP rR11 A3 + DP rR11 A4
=(1 − PR0 R11 )PR0 R13 +
(10)
(1 − PR0 R11 )[PR0 R12 (1 − PR0 R13 )+
(1 − PR0 R12 )PR0 R13 ] + 2(1 − PR0 R11 )PR0 R12 .
(20)
4) The DL of R11 : The node in set NA1 fails to receive the
packet when R11 and R13 drop at the same time. As a result,
the DL of R11 in set NA1 is:
DLrR11 A1 = (1 − PR0 R11 )(1 − PR0 R13 ) × 1. (21)
The node in set NA2 fails to receive the packet when R11
drops, while R12 and R13 forward or drop at the same time.
(12) As a result, the DL of R11 in set NA2 is:
DLrR11 A2 =(1 − PR0 R11 )[PR0 R12 PR0 R13 +
(22)
(1 − PR0 R12 )(1 − PR0 R13 )] × 1.
The node in set NA3 fails to receive the packet when R11
(13) drops. As a result, the DL of R11 in set NA3 is:
1054
 The DL of UAV i:
DLrR11 A3 = (1 − PR0 R11 ) × 1. (23) X X Y δbc
DLri = (1−Pji ) [1 − δab Pja (1 − Pjc ) ].
Two nodes in set NA4 fail to receive the packet when R11 b∈Ni n a∈Ni c c∈Ni c \a
and R12 drop. As a result, the DL of R11 in set NA4 is: (34)
The reward of the forwarding action of UAV i is:
DLrR11 A4 = (1 − PR0 R11 )(1 − PR0 R12 ) × 2. (24) X Y δab
F ri = F P ri − F Lri = Pji [2 (1 − Pja ) − 1].
In summary, the DL of R11 is: b∈Ni n a∈Ni c
(35)
DLrR11 =DLrR11 A1 + DLrR11 A2 + DLrR11 A3 + DLrR11 A4 The reward of the dropping action of UAV i is:
=(1 − PR0 R11 )(1 − PR0 R13 ) + (1 − PR0 R11 )× Dri =DP ri − DLri
X X Y δ
[PR0 R12 PR0 R13 + (1 − PR0 R12 )(1 − PR0 R13 )]+ =(1 − Pji ) {2 [δab Pja (1 − Pjc ) bc ] − 1}.
1 − PR0 R11 + 2(1 − PR0 R11 )(1 − PR0 R12 ). b∈Ni n a∈Ni c c∈Ni c \a
(25) (36)
The reward of the dropping action of R11 is: III. A MULTI - AGENT COOPERATIVE REBROADCASTING
METHOD
DrR11 = DP rR11 − DLrR11 . (26) In the UAV swarm network, due to the movement of UAVs,
the proposed rebroadcasting method needs to adjust the prob-
The dropping has more advantages than disadvantages when ability of rebroadcasting behavior according to the changes
DrR11 > 0, otherwise the opposite. of UAVs’ positions. Furthermore, the change of position is a
B. The evaluation model for the rebroadcasting actions continuous process. Therefore, the adjustment is a process of
experience learning. Based on the evaluation model for the
Define Ni = {1, 2, . . . , N } as the neighbors set of UAV
rebroadcasting actions, a multi-agent collaborative Q-learning
i. Set Ni can be divided into three subsets according to the
algorithm is proposed which is similar to [15].
positions of UAVs when UAV j ∈ Ni broadcasts first.
We make each UAV sends hello packet periodically in one
The set of previous hop node:
hop range. And their neighbors use Q-learning algorithm to
learn the information received from hello packets independent-
Ni p = {j}. (27)
ly. When a UAV receives a hello packet, it should do a virtual
The set of current hop nodes: action that the UAV assumes a rebroadcasting action has done.
Then the UAV evaluates and learns the reward of this virtual
action. Finally, the UAV updates its Q value table.
Ni c = {n|0 <k Gn − Gj k3 ≤ Cm , n ∈ Ni }. (28)
A. Q value table
The set of next hop nodes:
TABLE I: Q value table
Ni n = {n| k Gn − Gj k3 > Cm , n ∈ Ni }. (29) Fji Dji Gj tj
The communication relation function can be expressed as Sji

0, k Gp − Gq k3 > Cm As shown in Table I, UAV i creates a Q value table Qji
δpq = , (30)
1, k Gp − Gq k3 ≤ Cm for UAV j when i receives a hello packet from j for the first
where UAV p cannot communicate with UAV q when δpq = 0. time. Qji has a state Sji (it means i receives a packet from
UAV p can communicate with UAV q when δpq = 1. j), two actions (forward Fji and drop Dji ), two information
The reward function is composed of the profit and loss (GPS Gj and hello time tj ). Every time UAV i receives hello
of each UAV in set Ni n . The general forms of the reward packet from UAV j, the information should be updated. UAV
function for rebroadcasting actions are proposed as follow: i should delete Qji once it has not received hello packet from
The FP of UAV i: UAV j at time tj +∆, which means the distance between UAV
X Y i and UAV j is more than Cm . For example, the node R11 in
δ
F P ri = Pji (1 − Pja ) ab . (31) Fig. 1 should create seven Q value tables for NR11 .
b∈Ni n a∈Ni c UAV i should select a rebroadcasting action when it receives
a broadcasting packet from UAV j. The selection probability
The FL of UAV i:
X Y of forwarding is calculated according to the formula of Boltz-
δab mann:
F Lri = Pji [1 − (1 − Pja ) ]. (32)
b∈Ni a∈Ni
n c
eFji /T
The DP of UAV i: Pji = , (37)
X X Y eFji /T + eDji /T
δbc
DP ri = (1−Pji ) [δba Pja (1 − Pjc ) ]. where T is a positive parameter called the temperature [16].
b∈Ni n a∈Ni c c∈Ni ch \a
High temperatures cause the actions to be all (nearly) e-
(33) quiprobable. Low temperatures cause a greater difference in

1055
Authorized licensed use limited to: ULAKBIM UASL ISTANBUL TEKNIK UNIV. Downloaded on January 15,2024 at 10:32:01 UTC from IEEE Xplore. Restrictions apply.
 Algorithm 1 Q-learning algorithm of UAV i IV. N UMERICAL R ESULTS AND D ISCUSSIONS
1: UAV i keeps listening, and receives the hello packet from
UAV j; The proposed method is simulated by Exata. For our sim-
2: If this is the first time to receive the hello message from ulations, we assume that 100 nodes are randomly distributed
UAV j, a Q value table Qji is constructed. Fji and Dji to form a UAV swarm network in a square area of side length
are set to 1. If Qji has been constructed, we update the equal to 800m. The maximum communication distance of
location and time information of the Q value table; each UAV is 168m, and the bandwidth is 2M bits. The UAVs
3: UAV i selects forward action according the probability: move randomly with a maximum speed of 10m/s and a pause
eFji /T interval of 30 seconds. Each UAV sends a hello message every
Pji = Fji /T ; 2 seconds, and sends broadcast packets every fixed interval.
e +eDji /T
4: The reward of the actions is calculated by formula (40). The size of broadcast packet is 300kbit, TTL is 7, and the data
5: UAV i update Q value table Qji : transmission time is 1800 seconds. Experiments are carried out
in two scenarios: (i) each UAV sends broadcast packet at an

Fji = Fji + α[r(ac) + γ max{Fji , Dji } − Fji ], ac = F orward,
Dji = Dji + α[r(ac) + γ max{Fji , Dji } − Dji ], ac = Drop.
(41)
interval of 5 seconds; (ii) each UAV sends broadcast packet
at an interval of 10 seconds.
The performance of six broadcast mechanisms is evaluated,
and 10 groups of experiments are conducted for each broadcast
mechanism in two scenarios. The six broadcast mechanisms
selection probability for actions that differ in their value are as follows:
estimates.
• A: Flooding
• B: 33% probability flooding
B. Q-learning algorithm
• C: 50% probability flooding
All Q value tables are included in the hello packet when • D: 75% probability flooding
UAV j broadcasts the hello message. UAV i will do a virtual • E: Q-learning
action that assumes a rebroadcasting action has done when • F: Q-learning with delay (1000s pre-learning)
it receives the hello packet from j. UAV i should decide to In this scenario, the amount of broadcasting data is very
forward or drop the packet, and evaluate the decision, then large, and serious collision is inevitable. It is difficult for
learn the reward and update its Q value table. UAV i updates data packets to be effectively transmitted, resulting in the
its Q values according to the Bellman equation which can be poor transmission effect of the whole network. In this extreme
expressed as: condition, it is impossible to ensure the normal transmission of
each packet. Only when the network conditions permit, part of
Qt+1 (st, ac) =Qt (st, ac) + α[rt (st, ac)+ the packets can be transmitted to the outside as far as possible.
(38) Considering that the number of broadcast packets is deter-
γ max0
Qt (st0 , ac0 ) − Qt (st, ac)], mined, whether the network conditions can be improved can
ac
only be controlled by adjusting the behavior of rebroadcasting.
where Qt (st, ac) is the Q value with rebroadcasting deci- That is, when the network conditions permit, the probability
sion ac in state st at time t. rt (st, ac) is the reward func- of rebroadcasting increases correspondingly; when the network
tion when the transition is made from st with decision ac. conditions are bad, the probability of rebroadcasting decreases
max Qt (st0 , ac0 ) means the maximum possible Q values after correspondingly. Therefore, the network data transmission
ac0
the transition with next possible action ac0 . α is the learning situation can be evaluated by referring to the index of packet
rate and controls the learning speed. And γ ∈ [0, 1] is a receiving and sending ratio. The meaning of this parameter is:
discount factor to maintain stableness. Since the state space the number of packets correctly received by all UAVs divided
of st only has one state and the rebroadcasting decision has by the number of packets successfully sent by UAVs in the
two actions (forward and drop), the equation can be simplified form of broadcasting and rebroadcasting.
as below: As shown in Fig. 2 and Fig. 3, compared with other method-
s, flooding broadcast mechanism has a lower packet receiving
Qt+1 (ac) = Qt (ac) + α[rt (ac) + γ max{F, D} − Qt (ac)]. and sending ratio, which indicates that the probability-based
(39) mechanism can improve the broadcast performance, and the
performance improvement is about 40%. Comparing several
According to the evaluation model for the rebroadcasting probability-based methods in two scenarios, none of them has
actions mentioned earlier, the reward function can be defined
as below: obvious advantages in terms of packet receiving and sending
ratio. However, combined with the standard deviation of packet
(1 − Pja )δab − 1), F orward
 P Q




Pji
b∈Ni n
(2
a∈Ni c
receiving and sending ratio, it is found that the performance of
r(ac) = .


 (1 − Pji )
P
{2
P
[δab Pja
Q
(1 − Pjc )δbc ] − 1}, Drop the three methods based on fixed probability changes greatly
 b∈Ni n a∈Ni c c∈Ni c \a
(40) due to the movement of UAVs. On the contrary, the fluctuation
of the two Q-learning methods is smaller and the performance
Based on the above analysis, a Q-learning algorithm is is more stable. Especially, the method based on delayed Q-
proposed. The implementation procedure of one UAV is shown learning has better stability.
in Algorithm 1. All UAVs use Q-learning algorithm to make Through the further analysis of the experimental phenome-
decision independently. na, the following conclusions are drawn:

1056
Authorized licensed use limited to: ULAKBIM UASL ISTANBUL TEKNIK UNIV. Downloaded on January 15,2024 at 10:32:01 UTC from IEEE Xplore. Restrictions apply.
 And then it evaluates the value of rebroadcasting behavior,
while forming a dynamic broadcast probability. In order to
1 .4 S c e n a ry i model the evaluation for rebroadcasting behavior, a collision
S c e n a ry ii
P a c k e t R e c e iv in g a n d S e n d in g R a tio

1 .2
probability prediction method for rebroadcasting behavior, and
a multi-agent cooperative rebroadcasting method based on
1 .0 virtual action and RL are proposed. Simulation results show
that the dynamic probability mechanism based on Q-Learning
0 .8 has obvious performance advantages over fixed probability
0 .6 flooding and basic flooding mechanism in dynamic network
scenarios after pre-learning process. In the future, this dynamic
0 .4 probability method can combine with other broadcast mecha-
0 .2 nisms to improve, which will be conducive to the further study
of UAV network broadcast.
0 .0
A B C D E F VI. ACKNOWLEDGMENT
This work was supported in part by the National Natural
Science Foundation of China under Grants No. 61702545.
Fig. 2: Packet receiving and sending ratio
R EFERENCES
[1] P. K. Sharma, and D. I. Kim, “Random 3D Mobile UAV Networks:

0 .5
Mobility Modeling and Coverage Probability,” IEEE Transactions on
Wireless Communications, vol. 18, no. 5, pp. 2527–2538, 2019.
S c e n a ry i [2] D. Wu, D. I. Arkhipov, M. Kim, C. L. Talcott, A. C. Regan, J.
S c e n a ry ii
P a c k e t R e c e iv in g a n d S e n d in g R a tio

0 .4
A. McCann, and N. Venkatasubramanian, “ADDSEN: Adaptive Data
Processing and Dissemination for Drone Swarms in Urban Sensing,”
IEEE Transactions on Computers, vol. 66, no. 2, pp. 183–198, 2017.
S ta n d a rd D e v ia tio n o f

[3] T. Lu and J. Zhu, “Genetic Algorithm for Energy-Efficient QoS Multicast

0 .3
0 .2
0 .1
0 .0
B C
Fig. 3: Standard deviation of packet receiving and sending
ratio
(i) For the scenario with a large number of transmitted da-
ta, the performance of probability-based rebroadcasting
methods is better than that of simple flooding rebroad-
casting method due to packet conflict. That is, reasonable
reduction of transmitted data can improve the overall
performance of the network.
(ii) For dynamic network scenarios, due to the frequent
changes of network topology, the performance of fixed
probability rebroadcasting methods will be greatly affect-
ed by the topology, showing strong randomness. But the
rebroadcasting methods based on Q-Learning can adapt
to the dynamic, and have better stability.
(iii) In order to play a better performance advantage, rebroad-
casting method based on Q-Learning needs a learning
process to adapt to the dynamic nature of the network.
V. C ONCLUSION
In this paper, a rebroadcasting method based on dynamic
probability is proposed and analyzed. Based on this method,
each UAV predicts the probability of collision during rebroad-
casting according to the dynamic locations of their neighbors.
Routing,” IEEE Communications Letters, vol. 17, no. 1, pp. 31–34, 2013.
[4] C. Ho, K. Obraczka, G. Tsudik, and K. Viswanath, “Flooding for
Reliable Multicast in Multi-Hop Ad Hoc Networks,” in Proceedings of
the 3rd international workshop on Discrete algorithms and methods for
mobile computing and communications (DIALM), pp. 64-71, Augest
1999.
[5] Y. C. Tseng, S. Y. Ni, and Y. S. Chen, “The Broadcast Storm Problem
in a Mobile Ad Hoc Network,” Wireless Networks, vol. 8, no. 2–3, pp.
153–167, 2002.
D E F
[6] B. Williams and T. Camp, “Comparison of Broadcasting Techniques for
Mobile Ad Hoc Networks,” in Proceedings of International Symposium
on Mobile Ad Hoc Networking and Computing (MOBIHOC), pp. 194–
205, March 2002.
[7] J. Wu and F. Dai, “A Generic Distributed Broadcast Scheme in Ad Hoc
Wireless Networks,” IEEE Transactions on Computers, vol. 53, no. 10,
pp. 1343–1354, 2004.
[8] Y. C. Tseng, S. Y. Ni, and E. Y. Shih, “Adaptive Approaches to Relieving
Broadcast Storms in a Wireless Multihop Mobile Ad Hoc Network,”
IEEE Transactions on Computers, vol. 52, no. 5, pp. 545–557, 2003.
[9] M. Gerla and Y. Yi, “Team Communications Among Autonomous
Sensor Swarms,” ACM SIGMOD Record, vol. 33, no. 1, pp. 20–25,
2004.
[10] X. Han, L. F. Rossi, and C. Shen, “Autonomous Navigation of Wire-
less Robot Swarms with Covert Leaders,” in Proceedings of the 1st
international conference on Robot communication and coordination
(RoboComm), pp. 1–8, October 2007.
[11] N. Wisitpongphan, O. K. Tonguz, J. S. Parikh, P. Mudalige, F. Bai, and
V. Sadekar, “Broadcast Storm Mitigation Techniques in Vehicular Ad
Hoc Networks,” IEEE Wireless Communications, vol. 14, no. 6, pp.
84–94, 2007.
[12] E. Yanmaz, R. Kuschnig, and C. Bettstetter, “Achieving Air-Ground
Communications in 802.11 Networks with Three-Dimensional Aerial
Mobility,” in Proceedings of IEEE International Conference on Com-
puter Communications (INFOCOM), pp. 120–124, April 2013.
[13] R. M. Pires, A. S. R. Pinto, K. R. L. J. C. Branco, “The Broadcast Storm
Problem in FANETs and the Dynamic Neighborhood-Based Algorithm
as a Countermeasure,” IEEE Access, vol. 7, pp. 59737–59757, 2019.
[14] F. Dai and J. Wu, “Performance Analysis of Broadcast Protocols in Ad
Hoc Networks Based on Self-Pruning,” IEEE Transactions on Parallel
and Distributed Systems, vol. 15, no. 11, pp. 1027–1040, 2004.
[15] N. Vlassis, “A Concise Introduction to Multiagent Systems and Dis-
tributed Artificial Intelligence,” A Concise Introduction to Multiagent
Systems and Distributed Artificial Intelligence, Morgan & Claypool,
2007.
[16] R. S. Sutton and A. G. Barto, “Reinforcement Learning: An Introduc-
tion,” MIT Press, 1998.

1057
Authorized licensed use limited to: ULAKBIM UASL ISTANBUL TEKNIK UNIV. Downloaded on January 15,2024 at 10:32:01 UTC from IEEE Xplore. Restrictions apply.