Journal of Real-Time Image Processing (2021) 18:2319–2329

https://doi.org/10.1007/s11554-021-01124-9

ORIGINAL RESEARCH PAPER

A real‑time deep learning forest fire monitoring algorithm based

on an improved Pruned + KD model
Shengying Wang1 · Jing Zhao2 · Na Ta1 · Xiaoye Zhao1 · Mingxia Xiao1 · Haicheng Wei1 

Received: 3 February 2021 / Accepted: 4 May 2021 / Published online: 15 May 2021
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2021

Abstract
To meet the needs of embedded intelligent forest fire monitoring systems using an unmanned aerial vehicles (UAV), a deep
learning fire recognition algorithm based on model compression and lightweight requirements is proposed in this study. The
algorithm for the lightweight MobileNetV3 model was developed to reduce the complexity of the conventional YOLOv4
network structure. The redundant channels are eliminated through channel-level sparsity-induced regularization. The knowl-
edge distillation algorithm is used to improve the detection accuracy of the pruned model. The experimental results reveal
that the number of model parameters for the proposed architecture is only 2.64 million—compared with YOLOv4, this
represents a reduction of nearly 95.87%. The inference time decreased from 153.8 to 37.4 ms, a reduction of nearly 75.68%.
Our approach shows the advantages of a model with a smaller number of parameters, low memory requirements and fast
inference speed compared with existing algorithms. The method presented in this paper is specifically tailored for use as a
deep learning forest fire monitoring system on a UAV platform.

Keywords  Fire detection · YOLOv4 · MobileNetV3 · Model compression · Channel-level sparsity

1 Introduction proposed a discrimination algorithm based on K-means

As the global climate becomes warmer, the probability and
intensity of forest fires will gradually increase; therefore, it
is particularly important for people to monitor forest fires
intelligently [1]. Consequently, the use of forest fire monitor-
ing platforms based on computer vision technology mounted
on an unmanned aerial vehicle (UAV) has been expanding
rapidly.
Generally, forest fire detection methods based on com-
puter vision technology fall into two categories. One method
adopts a traditional image-processing algorithm that extracts
colour characteristics, geometric shapes, and the trajectory
of fire or smoke for fire identification. Rudz et al. studied
fire colour spaces and segmented fire images with the goal
of achieving better fire detection and determining flame
characteristics more accurately [2]. Alternatively, Wei et al.
* Haicheng Wei
wei_hc@nun.edu.cn
1
School of Electrical and Information Engineering, North
Minzu University, Yinchuan 750021, China
2
School of Information Engineering, Ningxia University,
Yinchuan 750021, China
clustering combined with a threshold method to segment
suspected fire areas. To improve the recognition, they used
the sample entropy algorithm, which can eliminate interfer-
ence from red houses, flags, and other objects with colours
similar to fire [3]. Additionally, Surit et al. calculated and
analysed the static and dynamic characteristics of smoke to
accurately and effectively detect smoke from fires [4]. In
contrast, Wirth et al. developed a modified histogram back-
projection algorithm for flame detection in the YCbCr colour
space that had a lower computational cost during detection
[5].
Beyond the traditional image-processing algorithms, the
other category of forest fire recognition methods are those
based on deep learning architecture. For example, Muham-
mad et al. explored a computationally efficient convolutional
neural network (CNN) model based on SqueezeNet archi-
tecture that used smaller convolution kernels for fire detec-
tion, location, and semantic understanding of a fire scene
[6]. In addition, Muhammad et al. modified the GoogLeNet
framework and established a fire detection CNN architec-
ture appropriate for video monitoring [7]. Zhang et al. pro-
posed the Faster R-CNN structure for wildland forest fire
smoke detection, in which the convolutional layers extracted

13
Vol.:(0123456789)

2320
various types and numbers of image features, thus avoid-
ing the complex manual feature-extraction process of the
traditional surveillance video smoke-detection methods [8].
However, the conventional detection algorithms are
unable to accurately accomplish the task of fire monitor-
ing because they are highly dependent on feature selection
and the poor robustness caused by hand-crafted thresholds.
Moreover, although the fire detection algorithms based on
deep learning algorithms have higher accuracy and strong
robustness, they are difficult to implement in practical
applications because their detection speeds are seriously
restricted by the enormous numbers of model parameters
and the resulting heavy computational burdens. Many
studies have optimized deep learning models in efforts to
improve their computational efficiency. Such optimization
can effectively reduce model complexity and reduce model
execution times [9, 10].
Therefore, to increase the robustness, decrease the com-
plexity, and develop a miniaturized forest fire monitoring
system suitable for a UAV platform, we propose a real-time
forest fire monitoring algorithm based on the lightweight
and compressed “you only look once” (YOLO) model. Ini-
tially, based on the lightweight YOLO model, redundant
incoming and outgoing connections and weights associated
with redundant channels are pruned to reduce the number
of parameters. Second, this paper designs the model loss
function and the dataset for training. The goal is to perform
model training using empty label data simplicity similar to
flame and smoke. While this approach increases the model
loss during training, it does not increase the complexity of
the model structure and can help reduce misidentified forest
fires during testing. Finally, we add the knowledge distil-
lation algorithm to allow the streamlined model to further
discriminate between positive and negative samples and
improve its sensitivity. We deployed the proposed algo-
rithm on an NVIDIA Jetson Xavier NX development kit and
achieved real-time forest fire detection from a UAV platform.
2 Lightweight and compressed YOLOv4
model
A standard YOLOv4 model needs one inference to complete
its inspection. Now in its third version, the YOLOv4 network
possesses obvious advantages, including high accuracy and
short inference times. In detail, the YOLOv4 network incor-
porates mosaic data enhancement, self-confrontation train-
ing, cross-minimum-batch normalization, and improved SPP
and PANet; these all contribute to improvements in inference
speed and recognition accuracy. To make this model even
more suitable for a UAV forest fire monitoring system, we
modify the YOLOv4 network in this paper, resulting in a
more compressed and lighter weight model.
13
Journal of Real-Time Image Processing (2021) 18:2319–2329
2.1 YOLOv4 object detection and the lightweight
model
As shown in Fig. 1, the YOLOv4 object detector is usu-
ally composed of feature extraction, feature enhancement,
detection heads, and other features. [11]. A cross-stage
partial (CSP) Darknet53 functions as its feature extrac-
tion, and CSP is added into each large partial dense block
to enhance the CNN learning capability [12]. The feature
maps, which are reduced by 8, 16, and 32 times, are used
as the output of the feature extractor and connected to
the feature enhancement module. Following this, a modi-
fied SPP [13] and PANet [14] are employed for feature
enhancement to increase the receptive field and enhance
feature fusion. Additionally, we apply three detection
heads in our proposed algorithm to enhance object detec-
tion at different scales. Although the YOLOv4 object
detection algorithm achieves an optimal trade-off between
inference speed and detection accuracy [11], there are
63.95 million model parameters, which makes this model
difficult to deploy as an embedded model on a real-time
forest fire monitoring platform.
Initially, all the convolutional layers in the YOLOv4
network used standard convolution operations. However,
some variants of convolution were produced, allow-
ing some lightweight models such as MobileNet [15],
SqueezeNet [16], and ShuffleNet [17] to be proposed in
follow-up studies, to help reduce the high memory require-
ments and need for computational resources for inference.
Taking MobileNetV3 as an example, the major changes in
the model between the MobileNetV1 and MobileNetV2
frameworks were the introduction of depth-wise separa-
ble convolutions and a squeeze and excitation (SE) block,
combined with a lightweight gating mechanism and a
reduction in channels to 1/4 of the original [18], which
leads to obvious advantages, including smaller numbers
of parameter, lower computational costs, reduced mem-
ory requirements and improved accuracy compared with
a standard CNN. These modifications make it more appro-
priate for use in a mobile terminal with limited computing
power [19, 20].
As depicted in Fig.  2, we used the YOLOv4 object
detection model as the backbone and modified its frame-
work by replacing CSPDarknet53 with a MobileNetV3
module to establish the lightweight YOLO + MobileNet
architecture. As a result, the total number of model
parameters for our proposed model fell to 23.08 million,
a decline of 63.91% compared to the original YOLOv4
network and constituting a drastic reduction in computa-
tional cost.
Journal of Real-Time Image Processing (2021) 18:2319–2329 2321

Fig. 1  Block diagram of
YOLOv4 object detection. The
small modules included are
CBM: Convolution + Batch
Normalization + Mish; CBL:
Convolution + Batch Normaliza-
tion + Leaky ReLU; and UP:
Upsampling

Fig. 2  Block diagram of
YOLO + MobileNet object
detection. The small mod-
ules included are CBh:
Convolution + Batch Nor-
malization + h-Swish; CBL:
Convolution + Batch Normaliza-
tion + Leaky—ReLU; and UP:
Upsampling

13

2322 Journal of Real-Time Image Processing (2021) 18:2319–2329

2.2 YOLO model compression

A certain amount of redundancy exists in the weights,

channels, and network layers even in the lightweight
YOLO + MobileNet model; therefore, it is necessary
to consider model compression. Under the premise of
maintaining the prediction effect of the model, model
redundancies, and insignificant items are removed. In
addition, weights in the convolution and fully connected
layers that have little effect on the recognition accuracy
are pruned to improve the application prospects of the
model [21].
Typical model compression methods include sparsity,
weight decomposition, low-rank decomposition, and
weight sharing. To avoid changing the structure of the
existing model, the approach proposed in this paper mainly
utilizes channel-level sparsity to slim the model; channel-
level sparsity is both easier to implement and more flex-
ible than are other sparsity methods. Hence, our proposed Fig. 3  Sketch of the model pruning process: a The initial network
depicts the YOLO  MobileNet model; b the compact network
+ 
method chiefly prunes the model channels as follows. A depicts the pruned model
scaling factor is inserted into each channel of the model
and adjusted by being trained together with the weights.
Finally, the channels with small scaling factors are treated
as redundant and are removed. The loss function of the
proposed model is:
∑
Loss = lossYOLO + 𝜆 g(𝛾), (1)
𝛾=Γ

where g(𝛾) = |𝛾| represents an L1 regularization for channel-

level sparsity training.
The majority of the convolutional layers in the
YOLO + MobileNet network structure are followed by
batch standardization (BN) layers [22] and an activation
function to boost the convergence speed. This approach is
easy to implement and carries no additional computational
costs because our proposed approach directly leverages
the parameters in the BN layers as the scaling factors for
sparsity training. The BN layers perform the following
operations:
x − 𝜇b
y = 𝛾√ + 𝛽,
2 (2)
𝜎b + 𝜀

where x and y denote the input and output of the BN layers,

respectively, and γ and β represent the zoom and transfer
parameters, respectively.
After the model is trained under channel-level sparsity-
induced regularization, we obtain a model in which many
scaling factors have a near-zero value; thus, we can prune
the channels with near-zero scaling factors by detecting all
their incoming and outgoing connections and correspond-
ing weights. As shown in Fig. 3, after pruning, the resulting
Fig. 4  Schematic diagram of the knowledge distillation process: the
YOLO + MobileNet model acts as the teacher network, while the
pruned model functions as the student network
narrower network has a more compact model size compared
to the initial network.
Because the pruned model eliminates neurons that
do not play significant roles during prediction, there is a

13
Journal of Real-Time Image Processing (2021) 18:2319–2329 2323
concomitant reduction in model parameter quantity and
computational cost. Inevitably, pruning even unimportant
channels will degrade the classification accuracy to some
degree. Accordingly, knowledge distillation often results
in increased classification accuracy of the smaller models
produced by compression [23]. Figure 4 shows that the
lightweight YOLO + MobileNet model is developed as a
teacher network, and the small, pruned model is consid-
ered a student network. The accuracy of the small, pruned
model is promoted by knowledge distillation. At this time,
proposed algorithm loss includes two aspects: the errors
between the predicted output from the student network and
the real labels and the difference between the predicted
outputs produced by the student and teacher networks.
It is better when the student model can distinguish simi-
lar objects and maintain an accuracy close to that of the
teacher network. Using such comparisons, we were able
to further compress the lightweight YOLO + MobileNet
model and implement it in the embedded system to realize
the airborne fire identification system.
3 Experimental methods
3.1 Experimental dataset
Dataset quality is closely linked to the final performance
of a model. Therefore, to make the model more effective at
accomplishing the forest fire detection task, it is necessary
to carefully collect, classify, and label the dataset. Because
fire image datasets in the forest environments are difficult to
Fig. 5  Some images in the data-
set. a Fire origin images with
flames and smoke; b non-fire
images that contain interference
similar to that produced by fire
and smoke
collect, the detection model used in this paper was pretrained
on the MSCOCO dataset and then the weights were trans-
ferred for additional training. At the point of transfer, the
model can already detect 80 kinds of targets but fire targets
are not among them. Fire detection can be conducted only
after fine-tuning the model on a fire image dataset. A total of
1844 images were collected for this experiment; some sam-
ples from the collected dataset are shown in Fig. 5. These
collected images included 1069 fire origin images with
flames and smoke, as displayed in Fig. 5a. The remaining
775 images do not show fire but contain interference similar
to fire and smoke, such as sunset and clouds; these images
were treated as negative samples to boost the robustness of
the proposed approach, as indicated in Fig. 5b.
3.2 Model training
Misdetection is a serious problem in fire detection. The main
reason for misdetection is that objects similar to flames and
smoke are misjudged as fire. To reduce the probability of
such occurrences, in this study, we added images similar
to flames and smoke to the dataset. When these images are
misdetected, the penalties will be increased. The theoretical
basis is that during the YOLO model training process, the
loss function should include location loss, class loss, and
confidence loss:
lossYOLO = losslocation + lossclass + lossconfidence . (3)
The calculations for location and class loss are based
on the occurrence of the target object; the corresponding
13

2324 Journal of Real-Time Image Processing (2021) 18:2319–2329

penalty is not added when there is no target object. In con- PyTorch deep learning framework, we adopted a stochastic
trast, for the confidence loss, we want a high level of con- gradient descent (SGD) optimizer and Nesterov momentum
fidence only when a target object exists. This confidence optimization. The momentum, initial learning rate and mini-
level indicates that a bounding box has a high probability of mum learning rate were set to 0.937, 5e-3, and 5e-4, respec-
bounding an object and of indicating its location accurately. tively. We applied mosaic data enhancement in our proposed
On the other hand, we want the confidence to be zero when model, and all the models were trained for 500 epochs. The
no target object exists: CIoU loss curves obtained by calculating the degree of
overlap between the predicted boxes and the real boxes, are
lossconfidence =lossconf +obj + lossconf +noobj
shown in Fig. 6. First, the CIoU losses of both the YOLOv4
G×G B
∑∑ obj [ and YOLO + MobileNet models tend to converge after
=− Iij Ci log(Ĉ i ) approximately 100 epochs. Subsequently, the loss values
i=0 j=0
] fluctuated dramatically from 100 to 300 epochs. However,
+(1 − Ci ) log(1 − Ĉ i ) the loss curve inclined and remained stable after 300 epochs,
G×G B
∑∑ especially in the YOLO + MobileNet network structure.
noobj [
− Iij Ci log(Ĉ i ) Eventually, the CIoU loss of the YOLOv4 model plateaued
i=0 j=0
] at approximately 2.2, while that of YOLO + MobileNet
+(1 − Ci ) log(1 − Ĉ i ) , (4) reached approximately 2.7, indicating that the prediction
boxes of the YOLO4 model were more in line with the real
where G*G is the grid cell of the YOLO target detector boxes than were those predicted by the YOLO + MobileNet
detection head, and B represents the bounding boxes. If the network structure.
j-th bounding box of the i-th grid contains a target object,
Iij has a value of 1; otherwise, it has a value of 0. In con-
obj
3.3 Model compression
trast, Iij has a value of 1 when the j-th bounding box of
noobj

the i-th grid does not contain a target object and has a value In this paper, we chose the lightweight YOLO + MobileNet
of 0 if it exists. The C represents the confidence level network structure as the backbone to create the compressed
(confidence = Pr(Object) × IoUpred truth
). model. We prune the unimportant channels and obtain an
During YOLO model training, the predicted bounding object detection model with a smaller number of parameters
box with the highest coincidence that is a true box boundary and a fast inference speed. Based on the channel-level spar-
is taken as a positive sample. Each bounding box contains sity algorithm proposed by Liu et al., the model first needed
and only has one positive sample. Based on the location of to be sparsely trained, then pruned, and finally, fine-tuned.
the bounding box and the contained object, the location loss, The sparsity factor λ and the percentage of model clip-
class loss, and the loss of confidence including the target ping needed to be coordinated to ensure that as the num-
object are calculated. When the overlap between the pre- ber of model parameters was reduced, the precision did not
dicted bounding box and the true bounding box is below a decrease too much. The sparsity factor λ value affects the
certain threshold, it is regarded as a negative sample, and the
location and class losses are not calculated; in addition, only
the part of the confidence loss that does not include the tar-
get object is calculated. The flame and smoke in a fire image
are considered as positive samples to improve the recall rate
of the algorithm, while other areas are considered negative
samples that cannot improve the precision rate because
objects similar to flame and smoke cause misdetections far
more often than does the background environment. In this
dataset, 1069 of the 1844 images are marked as having flame
or smoke targets. The remaining 775 images do not include
flame or smoke targets. When the model confidence level is
non-zero in these 775 images, the penalty will be increased.
We conducted experiments on an Ubuntu 18.04.3 sys-
tem equipped with an Intel(R) Core(TM) i9-9900K CPU
@ 3.60 GHz and an NVIDIA GeForce RTX 2080 Ti GPU.
We trained the YOLOv4 model in the same manner as the
lightweight YOLO + MobileNet network structure. Using the Fig. 6  The CIoU loss curves of YOLOv4 and YOLO + MobileNet

13
Journal of Real-Time Image Processing (2021) 18:2319–2329 2325

Fig. 7  The number of parameters and the change in mAP with the pruning percentage

distribution of the scaling factors γ when the model to be
clipped is sparsely trained. When λ = 0 (nonsparse train-
ing), the λ value was too large: all the scaling factors γ
approached 0, and the sparsity was excessive. Following
[19], we fixed λ = 0.001. Our proposed algorithm was first
trained under channel-level sparsity-induced regularization
for 300 epochs. After sparsity training, the inessential chan-
nels were identified based on the scaling factor in the BN
layers as described earlier. At present, model clipping can be
performed according to the distribution of the scaling factors
γ, and the model accuracy decreases as the clipping percent-
age increases. As shown in Fig. 7a, as the clipping percent-
age increased from 0% to approximately 80%, the number
of parameters decreased rapidly, and the mAP decreased
smoothly, indicating that the clipped redundant channels had
little effect on the detection accuracy. When the clipping per-
centage increased from 80 to 90%, the precision decreased
dramatically. As shown in Fig. 7b, when the percentage
of clipping reaches 91%, the mAP decreases significantly;
therefore, the clipping percentage was set to 90%.
Eventually, the architecture in this paper was fine-tuned.
In this experiment, the pruned model was trained for sev-
eral epochs, but the prediction accuracy improvements
were not obvious, which led to the introduction of knowl-
edge distillation to improve accuracy. Therefore, we took
YOLO + MobileNet as the teacher network, considered the
pruned model as the student network, and capitalized on
the knowledge distillation algorithm to boost the accuracy
of the pruned model. We selected the knowledge distilla-
tion algorithm to improve the model accuracy because the
teacher model outputs a “soft target”. The negative sample
labels with high probability showed that they were similar
to the positive sample labels. This algorithm was help-
ful for improve the generalizability of the student models.
A higher temperature parameter T (1 < T < 20) was set for
knowledge distillation when the clipping ratio was large
and greater generalizability was desired. However, the
temperature parameter T was not set to a too-large value;
otherwise, the output of the squashed teacher model was
excessive. When the clipping proportion was not large,
through fine-tuning training, the temperature parameter
was T = 1. When the 90% clipping ratio was set in this
study, the precision did not decrease significantly; there-
fore, we adopted a smaller temperature parameter of T = 3.
Finally, we named the compact network established in this
paper the Pruned + KD model.

Table 1  Comparison of Model Parameter (M) Fire (AP) Smoke + Fire Smoke (AP) All (mAP)
parameter quantity and mAP (AP)
scores among the tested object
detection models YOLOv4 63.95 0.648 0.806 0.556 0.670
YOLO + MobileNet 23.80 0.580 0.815 0.604 0.666
Pruned 2.63 0.424 0.734 0.517 0.558
Pruned + KD 2.63 0.612 0.726 0.554 0.631

13

2326 Journal of Real-Time Image Processing (2021) 18:2319–2329

4 Experimental results and analysis model. The number of Pruned + KD model parameters

was only 2.63  M, constituting a drop of approximately
First, based on the premise that the input images would 90%. Regarding the detection accuracy of the tested mod-
all have the same size, we evaluated the influence of els, the YOLOv4 model achieved mAP scores similar to
different backbone models, including the YOLOv4, the YOLO + MobileNet model for all types of objects; the
YOLO + MobileNet, Pruned and Pruned + KD models, on difference was only 0.004 because the mAP values of the
the parameter quantity and mean average precision (mAP) YOLOv4 and YOLO + MobileNet models were 0.670 and
value, as shown in Table 1. In terms of parameter quan- 0.666, respectively. In contrast, a considerable drop occurred
tity, the total number of model parameters of the original in the recognition accuracy of the Pruned model; its mAP
YOLOv4 was 63.95 million. This number was reduced value declined to 0.510. However, the Pruned + KD model
by 62.78% (to 23.80 million) for the YOLO + MobileNet achieved increased recognition accuracy, reaching a mAP
value of 0.631, which is a reduction of only 0.035 compared
with the YOLO + MobileNet model.
Furthermore, the inference speeds of three models,
Table 2  Comparison of inference speeds between different models YOLOv4, YOLO + MobileNet and the Pruned + KD model,
Development YOLOv4 (ms) YOLO + MobileNet Pruned + KD were evaluated on different embedded devices, including an
platform (ms) (ms) NVIDIA GeForce RTX 2080 Ti GPU and NVIDIA Jetson
Xavier NX, with 416 × 416-pixel input images, as shown in
RTX 2080 Ti 7.8 2.9 2.0
Table 2. On one hand, when the inference speeds were tested
Xavier NX 153.8 50.6 37.4
on the system with an NVIDIA GeForce RTX 2080 Ti GPU,

Fig. 8  The performances of different detection models: a The detection effect of YOLOv4; b the detection effect of YOLO + MobileNet; c the
detection effect of Pruned + KD

13
Journal of Real-Time Image Processing (2021) 18:2319–2329 2327
the inference speed of the lightweight YOLO + MobileNet
model was 2.69 times faster than that of YOLOv4, while the
Pruned + KD model required only 2 ms, reached a speed of
nearly 500 FPS, and its inference speed increased by 1.45
times. On the other hand, when the models were tested on
the system with an NVIDIA Jetson Xavier NX, the infer-
ence speed of the YOLO + MobileNet model was 3.04 times
faster than that of the YOLOv4 network structure. Similarly,
the inference speed of the Pruned + KD model increased by
1.35 times, and its inference time was 37.4 ms, which is
approximately 26.74 FPS. In summary, our study has dem-
onstrated that a real-time fire detection system running on
an NVIDIA Jetson Xavier NX and mounted a UAV platform
can be achieved by slightly decreasing the frame rate of the
input video.
Figure 8 clearly shows that three different models, namely,
YOLOv4, YOLO + MobileNet, and the Pruned + KD model,
detected forest fires at the same time. We evaluated the
detection performances among different models from two
aspects: object location annotations and confidence level.
On one hand, all the detection models accurately marked
the flame position in the image in the first column on the
left. Although many similar objects occurred in the middle
column of images, the YOLOv4 and YOLO + MobileNet
models could identify fires accurately. On the other hand, the
Pruned + KD model had one missing rate. The smoke + fire
area marked by the YOLOv4 model is more accurate than
that of the two compared models. Specifically, the right and
bottom borders of the label box marked by the YOLOv4
model are close to the boundary of the object area, but the
bounding boxes predicted by the two other models deviates
from the object area boundary to a small degree. Finally, the
rightmost column of Fig. 8 indicates that the three detec-
tion models can accurately locate the position of smoke.
As the second aspect, we considered the flame detection
confidence level as the criterion for assessing the detection
effect of all the models. Due to the high proportion of flame
images in our dataset, the confidence level of flame detection
was higher than that of smoke detection, mostly reaching
more than 90%, as illustrated in the leftmost image. How-
ever, as shown in the other two columns, the confidence
level of smoke detection was relatively low. In future studies,
smoke images should be aggregated into a separate dataset
to improve the accuracy of smoke area location detection to
improve the confidence level of smoke detection.
Two issues that need further focus for the forest fire detec-
tion task are missed and mistaken detections. To reduce the
rate of missed detection, this paper labelled and trained on
fire targets to ensure the detection accuracy. However, for
mistaken detection, model training using only fire image
datasets is not a good solution because it is susceptible to
Table 3  Accuracy comparison of different detection models
Model Recall (%) Precision (%) Accuracy (%)
YOLOv4 (empty) 100 99.21 99.78
YOLO + MobileNet 98.41 81.05 93.30
YOLO + MobileNet 97.62 100 99.35
(empty)
Pruned + KD 98.41 88.57 96.11
Pruned + KD (empty) 99.21 99.21 99.57
Empty represents a model that is trained with an empty label dataset,
and the bold portion represents a benchmark model and its detection
accuracy
interference from objects similar to flames and smoke. This
type of interference image was trained as an empty label
dataset in the YOLO model, and the mistaken detection rate
was reduced by increasing the confidence penalty. To evalu-
ate the effectiveness of this method, we used 200 images
containing fire objects and 300 images containing objects
similar to fire to test the models’ missed detection and false
detection rates. If an image is detected by the YOLO model
as a target frame of fire or smoke, the image is determined
as a fire image, regardless of the accuracy of the target frame
location. The accuracy of the different detection models
was calculated based on a confusion matrix. As shown
in Table 3, the YOLOv4 model trained with empty labels
has a recall of 100%. These results show that the model
can overcome the missed detection problem, that its mis-
taken detection rate is extremely low, and that the overall
detection accuracy is high. We used this YOLOv4 (empty)
model as a baseline model to evaluate the performance of
the improved model in this paper, and the results are con-
vincing. The precision of the YOLO + MobileNet model is
significantly lower than that of the baseline model, indicat-
ing that the problem of misdetection is serious. However,
the YOLO + MobileNet (empty) model achieves a precision
of 100%, and its overall accuracy is also at a high value,
indicating that after adding an empty label dataset to model
training, the problem of misdetection is effectively solved.
The precision of the Pruned + KD model is also significantly
lower than that of the baseline model and higher than that
of the YOLO + MobileNet model because knowledge distil-
lation increases model generalizability of the model. The
recall of the Pruned + KD (empty) model was 99.21%, its
precision was 99.21%, and its accuracy was 99.57%. Despite
significantly reducing the number of parameters and increas-
ing the inference speed, the missed detection and misdetec-
tion rates are effectively balanced. Adding an empty label
dataset reduces the misdetection rate, and the knowledge
distillation algorithm improves the generalizability of the
model.
13

2328
5 Conclusions
Forest fire detection from a UAV platform has the advan-
tages of a real-time remote system. Furthermore, an object
detection model suitable for deployment on an embedded
development kit is of the utmost importance for implement-
ing UAV fire detection.
In this study, we initially chose the YOLOv4 object
detection architecture as the network backbone. However,
this model is unsuitable for implementation on devices
with limited computing power due to its large number
of parameters, heavy computational burden, and sub-
stantial memory requirements. Therefore, we replaced
the backbone network of the YOLOv4 model with a
MobileNetV3 model to establish an initial lightweight
YOLO + MobileNet model and decrease the number of
model parameters and the computational burden. Then,
we further compressed the model by removing redun-
dant parts of the proposed network structure. Finally,
we improved the detection accuracy of the compressed
model using knowledge distillation and obtained the final
Pruned + KD model. During the training, images showing
fire and smoke were selected as training sets, and images
with fire-like objects were added to function as negative
samples and improve model robustness.
The total number of model parameters of our proposed
Pruned + KD architecture fell to 2.64 million—nearly
95.87% smaller than the number of parameters in the
original YOLOv4 model. When running on an NVIDIA
Jetson Xavier NX development kit, the inference time of
the Pruned + KD model was 37.4 ms, which is 4.11 times
faster than the 153.8 ms required by the YOLOv4 model.
Furthermore, for the forest fire detection task in this paper,
the mAP of the Pruned + KD model reached 0.631, which
is only 0.039 below than that of the original YOLOv4
network structure. Finally, we deployed our proposed
algorithm on an NVIDIA Jetson Xavier NX development
kit, which can provide a reference for constructing forest
fire detection and other mobile object detection models on
UAV platforms.
Acknowledgements  Data processing was supported by Ningxia Tech-
nology Innovative Team of advanced intelligent perception & control
and the Key Laboratory of Intelligent Perception Control at North
Minzu University.
Author contribution  S.W. and H.W. designed the experiments; S.W.,
J.Z., and X.Z. processed the data; S.W., M.X., and N.T. analyzed the
data and wrote the original paper. All authors have read and agreed to
the published version of the manuscript.
Funding  This research was funded by National Natural Science Foun-
dation of China (No. 61861001), Postgraduate Innovation Project of
North Minzu University (No. YCX20111).
13
Journal of Real-Time Image Processing (2021) 18:2319–2329
Data availability statement  All data generated or appeared in this study
are available upon request by contact with the corresponding author.
Declarations 
Conflict of interest  The authors declare no conflict of interest.
References
1. Donald, M., Littell, J.S.: Climate change and the eco-hydrology
of fire: Will area burned increase in a warming western USA?
Ecol. Appl. 27(1), 26–36 (2017)
2. Rudz, S., Chetehouna, K., Hafiane, A., et al.: On the evalua-
tion of segmentation methods for wildland fire. In: International
Conference on Advanced Concepts for Intelligent Vision Sys-
tems, pp. 12–23. Springer, Berlin, Heidelberg (2009)
3. Wei, H.C., Wang, S.Y., Xu, Y.J., et al.: Forest fire image rec-
ognition algorithm of sample entropy fusion and clustering. J.
Electron. Measure. Instrum. 34(01), 171–177 (2020)
4. Surit, S., Chatwiriya, W.: Forest fire smoke detection in video
based on digital image processing approach with static and
dynamic characteristic analysis. In: 2011 first ACIS/JNU Inter-
national Conference on Computers, Networks, Systems and
Industrial Engineering. IEEE, pp. 35–39 (2011)
5. Wirth, M., Zaremba, R.: Flame region detection based on his-
togram backprojection. In: 2010 Canadian Conference on Com-
puter and Robot Vision. IEEE, pp. 167–174 (2010)
6. Muhammad, K., Ahmad, J., Lv, Z., et al.: Efficient deep CNN-
based fire detection and localization in video surveillance appli-
cations. IEEE Trans. Syst. Man Cybern. Syst. 49(7), 1419–1434
(2019)
7. Muhammad, K., Ahmad, J., Mehmood, I., et al.: Convolutional
neural networks based fire detection in surveillance videos.
IEEE Access 6, 18174–18183 (2018)
8. Zhang, Q.X., Lin, G.H., Zhang, Y.M., et al.: Wildland forest fire
smoke detection based on faster R-CNN using synthetic smoke
images. Proc. Eng. 211, 441–446 (2018)
9. Kuanar, S., Rao, K.R., Bilas, M., et al.: Adaptive CU mode
selection in HEVC intra prediction: a deep learning approach.
Circuits Syst. Signal. Process. 38(11), 5081–5102 (2019)
10. Kuanar, S., Athitsos, V., Mahapatra, D., et al.: Low dose abdom-
inal CT image reconstruction: an unsupervised learning based
approach. In: 2019 IEEE International Conference on Image
Processing (ICIP). IEEE, pp 1351–1355 (2019)
11. Bochkovskiy, A., Wang ,C.Y., Liao, H.Y.M.: YOLOv4: Optimal
Speed and Accuracy of Object Detection. arXiv 2020. arXiv
preprint. arXiv:2004.10934. pp. 1–17 (2020)
12. Wang, C.Y., Mark Liao, H.Y., Wu, Y.H., et al.: CSPNet: a new
backbone that can enhance learning capability of cnn. In: Pro-
ceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition Workshops, pp. 390–391 (2020)
13. He, K., Zhang, X., Ren, S., et al.: Spatial pyramid pooling in
deep convolutional networks for visual recognition. IEEE Trans.
Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)
14. Liu, S., Qi, L., Qin, H., et al.: Path aggregation network for
instance segmentation. In: Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, pp. 8759–8768
(2018)
15. Howard, A., Sandler, M., Chu, G., et al.: Searching for mobile-
netv3. In: Proceedings of the IEEE International Conference on
Computer Vision, pp. 1314–1324 (2019)
Journal of Real-Time Image Processing (2021) 18:2319–2329 2329
16. Iandola, F.N., Han, S., Moskewicz, M.W., et al.: SqueezeNet:
AlexNet-level Accuracy with 50x Fewer Parameters and <0.5
MB Model Size. arXiv preprint. arXiv:1602.07360 (2016)
17. Zhang, X., Zhou, X., Lin, M., et al.: Shufflenet: an extremely
efficient convolutional neural network for mobile devices. In:
Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pp. 6848–6856 (2018)
18. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In:
Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pp. 7132–7141 (2018)
19. Howard, A.G., Zhu, M., Chen, B., et al.: Mobilenets: Efficient
Convolutional Neural Networks for Mobile Vision Applications.
arXiv preprint. arXiv:1704.04861 (2017)
20. Sandler, M., Howard, A., Zhu, M., et al.: Mobilenetv2: inverted
residuals and linear bottlenecks. In: Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, pp.
4510–4520 (2018)
21. Liu, Z., Li, J., Shen, Z., et al.: Learning efficient convolutional
networks through network slimming. In: Proceedings of the IEEE
International Conference on Computer Vision, pp. 2736–2744
(2017)
22. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep net-
work training by reducing internal covariate shift. In: International
Conference on Machine Learning, PMLR, pp 448–456 (2015)
23. Hinton, G., Vinyals, O., Dean, J.: Distilling the Knowledge in a
Neural Network. arXiv preprint. arXiv:1503.02531 (2015)
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Shengying Wang  received his B.Sc. degree in 2018 from Liaocheng
University, now he is postgraduate in North Minzu University. His
main research interest include image recognize.
Haicheng Wei  received his B.Sc. degree in 1997 from Xi’an jiaotong
University, received his M.Sc. degree in 2004 from Xi’an jiaotong
University, received his Ph.D. degree in 2012 from Xi’an jiaotong Uni-
versity, now he is Associate professor in Basic Experimental Teaching
and Engineering Training Center North Minzu University. His main
research interest include image recognize.
13