Measurement 175 (2021) 109130

Contents lists available at ScienceDirect

Measurement
journal homepage: www.elsevier.com/locate/measurement

Deep learning-based damage detection of mining conveyor belt

Mengchao Zhang a , Hao Shi a ,∗, Yuan Zhang a,b , Yan Yu a , Manshan Zhou a,b ,∗∗
a
College of Mechanical and Electronic Engineering, Shandong University of Science and Technology, Qingdao, SD 266590, China
b
Libo Heavy Industries Science and Technology Co., Ltd, Taian 271000, China

ARTICLE INFO ABSTRACT

Keywords: The mining conveyor belt is an important component of the coal mine belt conveyor, which plays the role
Belt conveyor of carrying materials and transmitting power. Aiming at the problem that mining conveyor belts are easily
Conveyor belt damaged under severe working conditions, based on the reclassification and definition of conveyor belt damage
Deep learning
types, a special data set for conveyor belt damage was established and a new detection method that can
Machine vision
simultaneously detect multiple faults based on improved Yolov3 algorithm was proposed. The EfficientNet
Damage detection
was adopted as the backbone feature extraction network instead of Darknet53 in the improved algorithm,
comprehensively considers the balance between network depth, width, and image resolution for network
scaling to improve the accuracy of the algorithm in limited computing resources. Experiments have proved
that the improved algorithm in this paper takes into account both detection speed and detection accuracy.
The detection speed can reach 42 FPS, and the mean average precision can reach 97.26%. Compared with the
original Yolov3 algorithm, the accuracy is increased by 10.4%, with the speed 45.9%, which provides new
ideas and methods for ensuring the safe and stable work of conveyor belts.

1. Introduction At present, the commonly used damage detection methods of the

As one of the most important transportation equipment in the field
of continuous conveyance of bulk materials, belt conveyor is devel-
oping towards the direction of long-distance, high belt speed, large
volume, and intelligence [1]. Intelligence refers to the belt conveyor
can realize its running state of self-awareness, independent decision-
making, and self-learning, by installing corresponding sensing devices
on the belt conveyor: optical cameras [2,3], infrared cameras [4,5],
pickups [6,7], vibration sensors [8–10], etc., real-time collection of its
operating state parameters is achieved, the collected state information
is analyzed and processed in real-time through the cloud servers or
other edge computing devices, then corresponding processing could be
made for finally realizing the autonomous running of belt conveyor and
unattended [11,12].
The conveyor belt, as an important part of the belt conveyor, plays a
role in carrying materials and transferring power, and its cost accounts
for 30%–50% of the total cost of the conveyor [13]. Conveyor belts
often suffer longitudinal tearing, surface scratches, and other damage
when the rollers are broken or the material is jammed [14]. If not
detected or dealt with in a timely and effective manner, the damage
will be further enlarged and lead to belt breakage and other accidents.
conveyor belt can be divided into manual detection methods and non-
destructive detection methods. Among them, artificial detection mainly
adopts the observation method of bubbling on the surface of the con-
veyor belt, length measurement method, etc., with a heavy workload.
Moreover, this method belongs to the category of static off-line detec-
tion, which is difficult to meet the actual detection requirement and
has been eliminated at present. While, there are various kinds of NDT
(Non-destructive testing) methods, such as tear sensor, eddy current
test [15], weak magnetic test [16], X-ray test [17], ECD (embedded
conductive detection) technology [18], etc., most can meet the needs
of real-time detection.
The rapid development of computer technology has greatly im-
proved the computer environment required for digital image process-
ing, making machine vision technology gradually mature, and it has
become a major subject in the computer field and has been successfully
applied in the field of belt conveyors: For example, it is used for
the classification of materials on the conveyor belt [19], the online
analysis of ore composition [20–24], and the identification of coal and
gangue [25–28].
With the further improvement of the level of intelligent develop-
ment of conveyors, a belt conveyor operating status monitoring system
based on machine vision has gradually been established, which is

∗ Corresponding author.
∗∗ Corresponding author at: College of Mechanical and Electronic Engineering, Shandong University of Science and Technology, Qingdao, SD 266590, China.
E-mail addresses: shihao@sdust.edu.cn (H. Shi), sdustzms@163.com (M. Zhou).

https://doi.org/10.1016/j.measurement.2021.109130
Received 22 November 2020; Received in revised form 4 January 2021; Accepted 31 January 2021
Available online 8 February 2021
0263-2241/© 2021 Elsevier Ltd. All rights reserved.
M. Zhang et al.
mainly divided into vision-based deviation monitoring and vision-based
tearing monitoring. Christopher C. Fromme [29] at Carnegie Mellon
University first proposed and developed a conveyor belt monitoring
system based on camera and image processing technology in 2003,
mainly for the damage of conveyor belt joints monitoring. Yan L.
et al. [30] proposed a fast image segmentation algorithm to simulta-
neously detect longitudinal tearing and deviation of the conveyor belt
from the binary belt image. Ref. [31] mainly focused on the research
of vision-based belt conveyor deviation detection, and a conveyor belt
edge detection algorithm based on improved Canny algorithm and
Hough transform was proposed. Liu W. [32] et al. proposed a tear
detection method for belt conveyors based on visible light and infrared.
This technology fusion method greatly improves the accuracy of tear
detection. Xianguo L. et al. [33] explained the application of the line
laser assisted technology in the tear detection of belt conveyors. They
judged the belt tearing by judging the slope of the line laser or whether
there are abnormal pixel points. Yu B. et al. [34] proposed a Dual Band
Infrared Detection (DBID) method for longitudinal tear detection on
conveyor belts. The method combines mid-infrared and long-infrared
vision, is not limited by visual conditions, and is more accurate and
reliable than the integrated binocular visual detection (IBVD) method.
A new early warning method for longitudinal tearing of conveyor belts
was proposed in Refs. [35,36]. An infrared camera was required to
collect the image, and then the binarization automatic threshold selec-
tion algorithm was used to analyze the infrared image to determine
whether the conveyor belt tear warning should be issued. A multi-
spectrum-based conveyor belt tear detection method was proposed
by Chengcheng Hou et al. [37]. This method combines visible light,
mid-infrared light, and far-infrared light, and has high environmental
adaptability. Hao X.L. et al. [38] proposed a damage detection method
for belt conveyors based on SVM(support vector machines), which more
considered the effectiveness of the method under both dry and wet
conditions. Starting from the cause of the tearing of the conveyor belt,
Wang, Y. et al. [39] proposed a method for identifying foreign objects
on the conveyor belt based on an improved SSD algorithm to prevent
the tearing problem from the root cause of the tearing.
In general, the current damage detection methods for belt con-
veyors based on machine vision can be divided into three types: 1.
The method based on image processing, that is, the surface image of
the conveyor belt is acquired by the camera, and the relevant image
processing algorithm is used to realize the elimination of noise and
the enhancement of tear characteristics, mostly judged by binarized
images; 2. The method for detecting damage of conveyor belts based
on line laser assistance. The line laser assisted detection method to
achieve the fracture features extracted from the image to extract the
characteristics of the laser line transformation, especially the detection
of angular point, by highlighting the characteristics of laser, make the
information accessible to extraction, effectively reduce the interference
of dust; 3. A method based on infrared cameras or thermal imagers. The
method fully considers the reasons for the damage of the conveyor belt,
and considers the local heating problem of the conveyor belt caused by
the violent friction between foreign objects and the conveyor belt. The
infrared CCD camera is used to capture and analyze the heat source
distribution on the conveyor belt surface, and then determine whether
any foreign objects are scratching the conveyor belt.
The essence of conveyor belt damage detection is target detection.
Deep learning is currently the mainstream research method in the
field of target detection. Compared with traditional target detection
methods, deep learning omits manual design features and the selection
of classifiers. By simulating a hierarchical structure similar to the
human brain, it establishes a structure from low-level semantic signals
to high-level semantics, mapping has better a fit for big data, and it
is easier to meet the requirements of image and video big data for
algorithm processing efficiency, processing accuracy, and intelligence.
It has been used in machine vision, speech recognition and natural
language processing with a good performance at present. However, as
2
Measurement 175 (2021) 109130
far as we know, the application of deep learning technology in the
detection of conveyor belt damage is rarely reported.
In order to meet the needs of the intelligent development of belt
conveyors, the article first proposed a mining conveyor belt damage
detection method based on deep learning, and proposed an improved
Yolov3 algorithm for conveyor belt damage detection, which was veri-
fied by field experiments with better detection accuracy and real-time
performance. The main contributions of this paper are summarized as
follows: (1) Better than most current conveyor belt damage detection
methods, the detection method based on deep learning realize the
detection of multiple damage types of conveyor belts, not limited to
belt tear. (2) The method proposed in this paper is a more direct
detection method, without the aid of other devices or complicated
image process. (3) The improved Yolov3 algorithm proposed in this
paper comprehensively considers the impact of model scaling on the
detection results, and achieves the best speed-accuracy trade-off on the
dataset, also could be applied to the detection of other targets in other
fields.
2. Methodology and raw data
2.1. The principle and structure of the improved algorithm
As mentioned earlier, the essence of conveyor belt damage detection
is target detection. Supervised learning-based target detection methods
at this stage can be divided into two categories: Two-stage target
detection algorithms based on candidate regions, which first generate
candidate frames through regional proposal networks(RPN), and then
use convolutional neural networks for fractionation and border correc-
tion, which are more accurate but slower and difficult to meet real-time
requirements due to more candidate frames, such as R-CNN series [40–
42]; And single-stage target detection algorithms based on regression,
represented by SSD (Single Shot MultiBox Detector) [43] and YOLO
(You only look once) [44,45]. The research in this paper is carried out
based on Yolov3.
The target detection network could be divided into two parts, one is
the backbone feature extraction network, and the other is the prediction
network. The quality of the features extracted by the feature extrac-
tion network directly affects the prediction effect of the prediction
network. This means if better features, maybe better prediction results.
Darknet53, as the backbone feature extraction network of Yolov3,
draws on the design idea of residual neural network (RNN) [46]. It
is composed of 5 residual blocks, each residual block contains multiple
residual units, and the structure of each residual unit is shown in Fig. 1.
The use of residual structure effectively solves the gradient explosion
problem caused by network deepening. A deeper network also helps to
extract deeper features and improve the accuracy of the network. At
the same time, the Yolov3 draws on the idea of the feature pyramid
network (FPN) [47], the multi-scale detection mechanism integrates
three feature layers of different scales for target detection, improves
the network’s ability to detect targets of different scales, and better
compensates for Yolov1 and Yolov2 detection of defects with poor
accuracy for small targets.
Compared with Yolov2, the backbone feature extraction network in
Yolov3 is upgraded from Darknet19 to Darknet53. The deepening of
the network depth brings a 9.8% improvement in accuracy. However,
in a limited computing unit, it is very likely to increase the amount of
calculation as the network depth increases, thereby reducing the speed
of target detection.
In general, three methods usually used for strength and expand the
backbone feature extraction network to better extract image features
and information: increasing the network width (Fig. 2(b)), increasing
the network depth (Fig. 2(c)), and increasing the resolution of the
input image (Fig. 2(d)). However, it does not mean that the network
expansion will necessarily achieve better results in terms of accuracy
or speed.
M. Zhang et al.
Fig. 1. Structure of Yolov3.
Fig. 2. Model scaling method.
The EfficientNet network [48] explores whether there is a balance
between the depth, width, or resolution of the input image to maximize
the performance of the neural network with limited resources, and for
the first time proposes and verifies the effect of the integrated scaling
of the model (Fig. 2(e)) on the effect of the neural network.
This problem is similar to the multi-objective optimization problem
in mathematics, where we try to improve the computational accu-
racy as much as possible by appropriately adjusting the depth, width,
and resolution of the input image of the network, given the limited
computational resources.
A convolutional network composed of k convolutional layers is
represented as N, and each convolutional layer can be represented as:
𝑌𝑖 = 𝐹𝑖 (𝑋𝑖 ). Where 𝑌𝑖 is the output tensor, 𝐹𝑖 is the operator, 𝑋𝑖 is the
[ ]
input tensor with tensor shape 𝐻𝑖 , 𝑊𝑖 , 𝐶𝑖 , where 𝐻𝑖 and 𝑊𝑖 are spatial
dimension and 𝐶𝑖 is the channel dimension.
The structure of the entire convolutional network can be expressed
as:
𝑁 = 𝐹𝑘 ⊙ ... ⊙ 𝐹2 ⊙ 𝐹1 (𝑋1 ) = ⊙ 𝐹𝑗 (𝑋1 )
𝑗=1,2,…,𝑘
In practice, ConvNet layers are often partitioned into multiple stages
and all layers in each stage share the same architecture, then the
3
Measurement 175 (2021) 109130
convolutional network could also be expressed as:
𝑁 = ⊙ 𝐹𝑖𝐿𝑖 (𝑋[𝐻𝑖 ,𝑊𝑖 ,𝐶𝑖 ] )
𝑖=1...𝑠
𝐿 [
where 𝐹𝑖 𝑖 denotes layer 𝐹𝑖 is repeated 𝐿𝑖 times in stage i, 𝐻𝑖 , 𝑊𝑖 ,
]
𝐶𝑖 denotes the shape of input tensor X of layer i.
Scaling of the model only changes the depth, width, and resolution
of the network, not the predefined 𝐹𝑖 in the Baseline.
Therefore, the mathematical model of objective optimization can be
expressed as:
𝑇 𝑎𝑟𝑔𝑒𝑡 ∶ max 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦(𝑁(𝑑, 𝑤, 𝑟))
𝑑,𝑤,𝑟
̂
𝑠.𝑡.𝑁(𝑑, 𝑤, 𝑟) = ⊙ 𝐹̂𝑖𝑑⋅𝐿𝑖 (𝑋<𝑟⋅𝐻̂ ,𝑟⋅𝑊̂ ,𝑟⋅𝐶̂ > )
𝑖=1...𝑠 𝑖 𝑖 𝑖
Memory(N) ≤ target_memory
Flops(N) ≤ target_f lops
where 𝑤, 𝑑, 𝑟 are coefficients for scaling network width, depth, and res-
olution; 𝐹̂𝑖 , 𝐿̂ 𝑖 , 𝐻̂ 𝑖 , 𝑊̂ 𝑖 , 𝐶̂𝑖 are predefined parameters in baseline network.
The results shown in Fig. 3 can be obtained through a large number
of repeated experiments (respectively changing the depth, width, and
image resolution of the network, and comparing the accuracy of the
M. Zhang et al.
network): A larger network has a greater width, depth or resolution,
and often more high accuracy, but the accuracy gain will quickly satu-
rate after reaching 80%, which shows the limitation of only expanding
a single dimension.
The various dimensions of model scaling are not completely in-
dependent. For large-resolution images, larger and deeper networks
should be used to facilitate the extraction of richer deep feature in-
formation, which means that each expansion dimension needs to be
balanced instead of a single dimension. Therefore, EfficientNet first
proposed a compound expansion method, that is, seeking a set of
parameters to adjust the expansion dimension of the model. We assume
that the parameter is (𝛼, 𝛽, 𝛾), which is used to balance the proportion
of depth, width and resolution respectively.
EfficientNet proposed a new compound scaling method, which use
a compound coefficient (𝜑) to uniformly scales network width, depth,
and resolution in a principled way:
𝑑𝑒𝑝𝑡ℎ ∶ 𝑑 = 𝛼 𝜑
𝑤𝑖𝑑𝑡ℎ ∶ 𝑤 = 𝛽 𝜑
𝑟𝑒𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛 ∶ 𝑟 = 𝛾 𝜑
𝑠.𝑡.𝛼 ⋅ 𝛽 2 ⋅ 𝛾 2 ≈ 2
𝛼 ≥ 1; 𝛽 ≥ 1; 𝛾 ≥ 1
where (𝛼, 𝛽, 𝛾) are constants that can be determined by a small grid
search.
Intuitively, (𝜑) is a user-specified coefficient that controls how
many more resources are available for model scaling, while (𝛼, 𝛽, 𝛾)
specify how to assign these extra resources to network width, depth,
and resolution respectively. A baseline network, EfficientNet-B0, was
designed through network structure search. As shown in Fig. 4, the net-
work structure is relatively simple, which is convenient for subsequent
testing of the effect of the compound model scaling algorithm.
Starting from the baseline EfficientNet-B0, the compound scaling
method was applied to scale it up with two steps:
∙ STEP 1: First fix 𝜑 = 1, assuming twice more resources available,
and do a small grid search of (𝛼, 𝛽, 𝛾) based on the limited com-
puting resources. In particular, the best values for EfficientNet-B0
was found: 𝛼 = 1.2; 𝛽 = 1.1; 𝛾 = 1.15, under constraint of 𝛼⋅𝛽 2 ⋅𝛾 2 ≈
2.
∙ STEP 2: Then fix (𝛼, 𝛽, 𝛾) as constants and scale up baseline
network with different (𝜑), to obtain EfficientNet-B1 to B7.
Fig. 4 shows the structure of EfficientNet-B0. The EfficientNet was
used as the backbone feature extraction network of Yolov3 instead of
Darknet53 in this paper, which can extract richer feature information
for the prediction network of Yolov3. We introduce the replacement
method with EfficientNet-B0: the specific replacement process needs
to refer to Figs. 1 and 4. In the backbone feature extraction network
of Yolov3, the feature map input to the prediction network is a com-
bination of the feature map after the input image is compressed 3
times, 4 times and 5 times through convolutional downsampling, which
achieves good results in multi-scale target detection. Similarly, we use
the EfficientNet network to down-sampling the input image, realize the
compression of the image while extracting the features, and pass the
image compressed 3, 4, and 5 times into the prediction network to
realize the replacement of the backbone feature extraction network of
Yolov3.
2.2. Definition of conveyor belt damage types
There are various types of conveyor belt, but the main structure
form is the same, which is composed of upper and lower covering layer
and middle tensile layer through vulcanization process. The reasons
of conveyor belt damage can be divided into internal and external
factors. Among them, the internal cause refers to the processing quality
4
Measurement 175 (2021) 109130
Table 1
Algorithm running environment.
OS CPU GPU Tensorflow-gpu Keras Python
Windows10 E5-2620V3*2 RTX2060S 1.13.1 2.1.5 3.6.10
of the conveyor belt itself, including the selection of raw materi-
als, vulcanization process, joint form selection and process. External
causes refer to the external working environment. The damage types
of conveyor belts can be roughly divided into four categories: surface
wear, surface damage, tear and breakdown. Surface wear is mainly
caused by large lumps of coal or coal and stone blocking the outlet
of chute. Long-term obstruction will cause the rubber covering on the
conveyor belt to thin, leaving the internal wire rope exposed. Finally,
the wire rope will be twisted and broken, causing the conveyor belt
to be torn. In addition, there are many reasons for belt tearing: belt
deviation, metal foreign bodies, large gangue, etc., which are mostly
caused by irregular operation. With the development of long-distance
belt conveyor, the number of large height difference transfer stations
gradually increases, and the belt breakdown fault caused by material
loading impact also increases gradually. The surface damage mainly
refers to the bubbling on the surface of the conveyor belt or the falling
off of the covering rubber, which is closely related to the processing
quality of the conveyor belt.
2.3. Preparation for raw data
Through on-site collection, network search and other forms, 3000
images of conveyor belt damage were collected, and the four types of
images each accounted for a quarter. A conveyor belt damage dataset
was created in the form of manual annotation for algorithm testing.
The whole training process uses the SGD (stochastic gradient descent)
to optimize the loss function. The training iterations in this article is
set 30000, the batch size is 4, the momentum used is 0.9, and the
attenuation coefficient is 0.0005. The learning rate in the initial stage
of training is set to 0.001. When the iteration reaches 20000 times,
the learning rate drops to 0.00001. The training sample was further
expanded during training by changing the saturation, hue and value of
the image, random cropping, random scaling, and other operations.
2.4. Algorithm running environment
The computer configuration parameters used for neural network
model training and testing are shown in Table 1.
3. Result and discussion
3.1. Performance comparison of multiple algorithms
In multi-object classification tasks, mAP (mean Average Precision)
[49] and FPS (Frames Per Second) are usually used to measure the
superiority of the algorithm, also adopted in this paper. mAP, the
average value of AP of each class of objects, is the average value of
AP obtained by multiple verification set individuals, which measures
the overall detection accuracy of the algorithm. FPS, which is the frame
rate that can be processed per second, is used to measure the processing
speed of the algorithm.
In order to verify the performance of the improved algorithm pro-
posed in this paper, the performance of multiple algorithms were tested
simultaneously on the conveyor belt damage dataset through transfer
learning, including Faster-RCNN, SSD, RFBnet, M2det, Yolov3, Yolov4,
etc. The usage of transfer learning greatly saves the time and computer
resources required for training. The performance results of various
algorithms can be found in Table 2.
In Table 2, the detection speed of the Faster R-CNN has reached
3.6 FPS, which is the slowest of all the test algorithms. Although
M. Zhang et al. Measurement 175 (2021) 109130

Fig. 3. Limitations of single-dimensional expansion.

Fig. 4. Structure of EfficientNet-B0.

Fig. 5. The hardware usage of different algorithms.

Table 2 the region proposal network(RPN) is used to generate ROIs, instead

The performance of different algorithms on the dataset. of the previous selective search, it greatly reduces the amount of
Model-Name FPS mAP/% Wear Damage Breakdown Tear calculation. However, compared with the Yolo algorithm or the SSD
Faster R-CNN 3.675 91.1 96 98 82 88 algorithm, which are directly based on regression prediction, the cal-
SSD 30 94.45 100 100 93 87 culation amount is still large, and hard to meet real-time needs. In
RFBnet 30.42 83.36 98 77 85 75
M2det 24.27 92.47 100 98 80 92
comparison, SSD algorithm, based on the regression mechanism, also
Yolov3 28.72 88.12 100 81 83 88 makes prediction on multi-scale feature map, which greatly improves
Improved-Yolov3* 41.91 89.12 100 81 82 94 the ability to detect both large and small objects at the same time and
Yolov4 24.39 90 100 81 96 87 achieves an accuracy of 94.45% in the dataset.
The improved algorithm is named as Improved-Yolov3 in Table 2
(*: The backbone feature extraction network of here Improved-Yolov3
is EfficientNet-B0). Its average test accuracy on the dataset reaches

5
M. Zhang et al.
Fig. 6. The test results comparison between improved-Yolov3 and the original-Yolov3: Image (a)–(d) belongs to the result of original Yolov3, (e)–(h) belongs to the improved-Yolov3.
89.12%, while the average test accuracy of original Yolov3 with Dark-
net53 as the baseline is 88.12%, which shows the improved algorithm
has a certain accuracy improvement effect, the specific improvement is
mainly manifested in the detection of tears. But it is obvious that the
improved algorithm has greatly improved in the prediction speed, and
the FPS has been increased from 28 to 42.
As the above explanation of the structure of EfficientNet, the Effi-
cientNet exists 8 structures, B0–B7. Among them, EfficientNet B1–B7
were obtained on the basis of EfficientNet-B0 by increasing the coeffi-
cient (𝜑). With the increase of (𝜑), the depth and width of network as
well as the resolution of the input image are expanding. In this paper,
we constraint 𝛼 ⋅ 𝛽 2 ⋅ 𝛾 2 ≈ 2 such that for any new (𝜑), the total FLOPS
will approximately increase by 2𝜑 .
EfficientNet B0–B7 were all tried as the backbone of Improved-
Yolov3, and the test performance was shown in Table 3. The algorithm
achieves the best speed-accuracy trade-off on the conveyor belt damage
dataset, with 97.26% AP at 26.28 FPS, 89.12% AP at 41.91 FPS.
Due to the limitations of the experimental environment and rapid
increase of the FLOPS, the problem of graphics memory overflow occurs
when trying B5. But according to the trend shown in Table 3, as the
increase of coefficient (𝜑), the model will further expand, the prediction
accuracy of the neural network will be further improved. However,
6
Measurement 175 (2021) 109130
the further increase of FLOPS will inevitably slow down the detection
speed.
3.2. Hardware usage comparison of multiple algorithms
In addition, the hardware usage of different algorithms was also dis-
cussed in the context of this paper, from the aspects of CPU, GPU(cuda),
RAM, etc. The specific comparison results are shown in Fig. 5. The
comparison takes the hardware cost into consideration, which has
guiding significance for the field staff to choose a suitable algorithm.
But there is no doubt that the high-quality hardware resources will only
increase the detection speed, not the detection accuracy.
It can be seen from Fig. 5 that the GPU and RAM usage of Improved-
Yolov3 (The backbone feature extraction network of here Improved-
Yolov3 is EfficientNet-B0) is relatively less than the original Yolov3,
and the CPU usage of the two is basically the same. Relatively, Faster
R-CNN not only takes up a lot of GPU, but also CPU, which is caused by
its huge FLOPS. Thus we can infer that with the further expansion of
the model, the improved algorithm will take up more GPU and CPU
resources, even exceeding Faster R-CNN. Besides, it is more clearly
that the proposed improved-Yolov3 algorithm based on the EfficientNet
feature extraction network has a higher timeliness while ensuring the
measurement accuracy.
M. Zhang et al. Measurement 175 (2021) 109130

Fig. 7. Experimental setup.

Table 3
The performance of different backbone feature extraction networks on the dataset.
Model-Name mAP/% FPS Wear Damage Breakdown Tear
EfficientNet-B0-Yolov3 89.12 41.91 100 81 82 94
EfficientNet-B1-Yolov3 90.57 36 100 83 85 94
EfficientNet-B2-Yolov3 91.38 33.53 100 85 85 95
EfficientNet-B3-Yolov3 93.16 30.97 100 89 89 95
EfficientNet-B4-Yolov3 97.26 26.28 100 99 93 96

3.3. Performance comparison between Yolov3 and improved Yolov3
The performance of improved-Yolov3 and the original Yolov3 on
the dataset is compared in Fig. 6. It can be seen from the figure
that both algorithms can better realize the identification and detection
of various scheduled conveyor belt damage defects. Different from
existing image processing-based methods, deep learning-based target
detection methods allow for direct target localization and classification.
The position information of the target object is shown in the rectangular
frame in the figure. The numbers on the rectangular frame are the
category and probability of the category of the target object. At the
same time, the improved algorithm is slightly higher in accuracy than
the original algorithm by comparing Fig. 6(d) and Fig. 6(h), which is
basically consistent with the results shown in Table 1, and the improved
algorithm has a higher recognition accuracy for tearing detection.
3.4. Verification of model generalization ability
Generalization is a term used to describe a model’s ability to react
to new data. That is, after being trained on a training set, a model can
digest new data and make accurate predictions. A model’s ability to
generalize is central to the success of a model.
In order to verify the generalization ability of the proposed model,
we conducted field tests in Libo Heavy Industry Technology Co., Ltd.
Due to the high cost of the conveyor belt, we chose to paste a damaged
belt on the normally running conveyor belt as the experimental test
material, as shown in Fig. 7. The camera is installed directly above
the transmission roller of the machine head. The camera frame rate
is 25 frames and the maximum resolution is 700 × 480: the width of
conveyor belt is 1.6 m, and the rated operating speed is 2.5 m/s. And
what we want to emphasize here is that the experimental platform is
only used for testing. In fact, when using machine vision technology
to detect conveyor belt damage, the camera should be placed on the
lower branch and return section of the conveyor. At the same time, it
is necessary to comprehensively consider the effects of light and surface
residues on the test results.
As shown in Fig. 7(d), the algorithm accurately identifies the tearing
of the conveyor belt with the accuracy 81% , and the real-time frame
rate is 35.43 FPS, which meets the actual needs of the project.

7
M. Zhang et al.
4. Conclusions and future work
As one of the main directions of the current belt conveyor devel-
opment, the intelligent energization of belt conveyors is the only way
for the construction of smart mines. As the main bearing and traction
component of the belt conveyor, the reliability of the operation state
of the belt seriously affects the safe and efficient operation of the con-
veyor. Compared with the traditional detection methods, the proposed
deep-learning based detection method has the following improvements
and advantages:
(1) A conveyor belt damage dataset has been created based on
a redefinition and reclassification of the damage types of conveyor
belts. Four types of damage are included: surface wear, surface dam-
age, breakdown and tear. The more detailed classification of damage
allows for more accurate monitoring of the conveyor belt’s operating
conditions and prevents unnecessary operations.
(2) A deep learning based conveyor belt damage detection method
was designed and proposed in this paper. The detection process is more
direct, without the assistance of other equipment or complicated image
processing operations. It enables the detection of four conveyor belt
damage states and can give confidence levels while classifying their
damage type.
(3) The Yolov3 algorithm is improved in this paper to compre-
hensively considers the balance between the width, depth and input
image resolution of the network during model scaling and its impact
on prediction accuracy.
(4) While comparing the accuracy and speed of algorithm operation,
the hardware usage of the algorithm during operation is also taken into
account to provide a reference for engineers to select an algorithm.
The improved algorithm brings a double improvement in detection
accuracy and speed. It has a test accuracy of up to 97% on the conveyor
belt damage dataset, and the fastest test speed is 42 FPS. At the same
time, it can be seen from the test platform that the model we proposed
has a high generalization ability.
In the next work, we will focus on the lightweight research on the
model to better adapt to the needs of edge computing. At the same time,
we will further expand the conveyor belt damage data set to improve
the robustness and generalization ability of the neural network.
CRediT authorship contribution statement
Mengchao Zhang: Methodology, Writing - original draft, Software.
Hao Shi: Data curation, Writing - review & editing. Yuan Zhang:
Supervision. Yan Yu: Investigation, Supervision. Manshan Zhou: Con-
ceptualization, Writing - review & editing.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared to
influence the work reported in this paper.
Acknowledgment
The authors wish to acknowledge Yibing Shang at University of
Exeter, United Kingdom and Zhengxu Zhang, postgraduate of Shandong
University of Science and Technology, China, for their help in collecting
the original data and labeling.
Funding
This research received no external funding.
8
Measurement 175 (2021) 109130
References
[1] M. Zhang, H. Shi, Y. Yu, M. Zhou, A computer vision based conveyor de-
viation detection system, Appl. Sci. 10 (2020) 2402, https://doi.org/10.3390/
app10072402.
[2] L.I. Zhanli, C. Jiaying, L.I. Hongan, et al., Research on intelligent monitoring
and warning method of belt conveyor, J. Graph. (2017).
[3] Y. Gao, T. Qiao, H. Zhang, et al., A contactless measuring speed system of
belt conveyor based on optical flow techniques, IEEE Access PP (99) (2019)
1, https://doi.org/10.1109/ACCESS.2019.2937317.
[4] Pytlik Andrzej, Klaudiusz, et al., Research on tightness loss of belt conveyor’s
idlers and its impact on the temperature increase of the bearing assemblies, J.
Sustain. Min. (2016) https://doi.org/10.1016/j.jsm.2016.07.001.
[5] K. Nienhaus, M. Warcholik, Christoph Büschgens, Belt positioning and skewing
prevention in lignite mining using long-wavelength infrared cameras, in: Proceed-
ings of the 12th International Symposium Continuous Surface Mining - Aachen
2014, Springer International Publishing, 2015.
[6] M. Yang, W. Zhou, T. Song, Audio-based fault diagnosis for belt conveyor rollers,
Neurocomputing (2020) 397, https://doi.org/10.1016/j.neucom.2019.09.109.
[7] S. Kato, T. Nagayama, D. Su, et al., Identification of a cross-sectional vibration
mode of a belt conveyor support structure using acoustic loading, J. Struct. Eng.
A (2018) https://doi.org/10.11532/structcivil.64A.354.
[8] Grzegorz Peruń, Attempt to evaluate the technical condition of the rollers of the
belt conveyor by vibration measurements, Vibroeng. Proc. 3 (2014) 296–299.
[9] R. Li, X. Yang, T. Liu, et al., Application of fiber optic sensors for vibration
and ignition monitoring of a belt conveyor system, in: Optical Fiber Sensors and
Communication, 2019.
[10] F. Klimenda, J. Soukup, J. Sterba, Noise and vibration analysis of conveyor
belt, Manuf. Technol. 19 (4) (2019) 604–608, https://doi.org/10.21062/ujep/
341.2019/a/1213-2489/MT/19/4/604.
[11] Y. Pang, G. Lodewijks, Simulation-based knowledge acquisition for intelligent
belt conveyor monitoring, Part. Part. Syst. Charact. 24 (4–5) (2007) 360–364,
https://doi.org/10.1002/ppsc.200601117.
[12] P.D.I.G. Lodewijks, D.I.J.A. Ottjes, D.I.Y. Pang, Intelligent belt conveyor mon-
itoring and control, Des. Eng. (2010) http://resolver.tudelft.nl/uuid:585579aa-
7406-4e57-af8e-66c7df9b83bd.
[13] S.G. Sakharwade, S. Nagpal, Analysis of transient belt stretch for horizontal and
inclined belt conveyor system, Int. J. Math. Eng. Manage. Sci. 4 (5) (2019)
1169–1179, https://doi.org/10.33889/IJMEMS.2019.4.5-092.
[14] L.P. Alviari, M.F. Anggamawarti, Y. Sanjiwani, et al., Classification of impact
damage on a rubber-textile conveyor belt: A review, Int. J. Mech. Eng. Technol.
Appl. 1 (1) (2020) 21–27.
[15] V. Redko, V. Khandetskyy, P. Novak, et al., Method and apparatus for
eddy current-based quality inspection of dry electrode structure: U.S. Patent
7355395[P]. 2008-4-8.
[16] Tiezhu Qiao, Xiaoyu Lu, Laiqing Yan, Research on the signal feature extraction
method in steel-cord conveyor belt with metal magnetic memory testing, J.
Comput. Theoret. Nanoence 11 (1) (2012) 489–492.
[17] Y. Guan, J. Zhang, Y. Shang, et al., Embedded sensor of forecast conveyer belt
breaks, in: International Conference on Fuzzy Systems and Knowledge Discovery,
IEEE Computer Society, 2008, https://doi.org/10.1109/FSKD.2008.643.
[18] Y. Pang, G. Lodewijks, A novel embedded conductive detection system for
intelligent conveyor belt monitoring, in: 2006 IEEE International Conference on
Service Operations and Logistics, and Informatics, Shanghai, 2006, pp. 803–808,
https://doi.org/10.1109/SOLI.2006.328958.
[19] H.I. Bozma, H. Yalçın, Visual processing and classification of items on a moving
conveyor: a selective perception approach, Robot. Comput.-Integr. Manuf. 18 (2)
(2002) 125–133, https://doi.org/10.1016/S0736-5845(01)00035-7.
[20] Michał Kontny, Machine vision methods for estimation of size distribution of
aggregate transported on conveyor belts, Vibroeng. Proc. (2017) https://doi.org/
10.21595/vp.2017.19151.
[21] Jayson Tessier, Carl Duchesne, Gianni Bartolacci, A machine vision approach to
on-line estimation of run-of-mine ore composition on conveyor belts, Min. Eng.
(2007) https://doi.org/10.1016/j.mineng.2007.04.009.
[22] Z. Zhang, Y. Liu, Q. Hu, et al., Multi-information online detection of coal quality
based on machine vision, Powder Technol. (2020) 374, https://doi.org/10.1016/
j.powtec.2020.07.040.
[23] C. Aldrich, G.T. Jemwa, J.C. Van Dyk, et al., Online analysis of coal on a
conveyor belt by use of machine vision and kernel methods, Int. J. Coal Prep.
Util. 30 (6) (2010) 331–348, https://doi.org/10.1080/19392699.2010.517486.
[24] M. Kistner, G.T. Jemwa, C. Aldrich, Monitoring of mineral processing systems
by using textural image analysis, Min. Eng. 52 (2013) 169–177, https://doi.org/
10.1016/j.mineng.2013.05.022.
[25] T. Andersson, M.J. Thurley, J.E. Carlson, A machine vision system for estimation
of size distributions by weight of limestone particles, Min. Eng. 25 (1) (2012)
38–46, https://doi.org/10.1016/j.mineng.2011.10.001.
[26] M.H. Zhao, S. Ma, D.X. Zhao, Image processing based on gray information
in sorting system of coal gangue, in: 2018 10th International Conference on
Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2018, https://doi.
org/10.1109/IHMSC.2018.10125.
M. Zhang et al.
[27] D. Dou, W. Wu, J. Yang, et al., Classification of coal and gangue under multiple
surface conditions via machine vision and relief-SVM, Powder Technol. 356
(2019) 1024–1028, https://doi.org/10.1016/j.powtec.2019.09.007.
[28] Y.D. Zhao, M.F. Sun, Image processing and recognition system based on DaVinci
technology for coal and gangue, Appl. Mech. Mater. 130134 (2012) 2107–2110,
https://doi.org/10.4028/www.scientific.net/AMM.130-134.2107.
[29] Christopher C. Fromme, David J. Stager, Thomas E. Pilarski, et al., Conveyor
belt inspection system and method: US, 2006.
[30] Y. Yang, C. Miao, X. Li, et al., On-line conveyor belts inspection based on
machine vision, Optik 125 (19) (2014) 5803–5807, https://doi.org/10.1016/j.
ijleo.2014.07.070.
[31] Z. Aichun, H. Gang, W. Yongxing, The research on the detection method of
belt deviation by video in coal mine, in: 2011 International Conference on
Mechatronic Science, Electric Engineering and Computer (MEC), IEEE, 2011, pp.
430–433, https://doi.org/10.1109/MEC.2011.6025493.
[32] W. Liu, T. Qiao, G. Yan, et al., Research on visible light and infrared vision
real-time detection system for conveyor belt longitudinal tear, Iet ence Meas.
Technol. 10 (6) (2016) 577–584, https://doi.org/10.1049/iet-smt.2015.0297.
[33] L. Xianguo, S. Lifang, M. Zixu, et al., Laser-based on-line machine vision
detection for longitudinal rip of conveyor belt, Optik 168 (2018) 360–369,
https://doi.org/10.1016/j.ijleo.2018.04.053.
[34] B. Yu, T. Qiao, H. Zhang, et al., Dual band infrared detection method based on
mid-infrared and long infrared vision for conveyor belts longitudinal tear, Mea-
surement 120 (2018) 140–149, https://doi.org/10.1016/j.measurement.2018.02.
029.
[35] Y. Yang, C. Hou, T. Qiao, et al., Longitudinal tear early-warning method for
conveyor belt based on infrared vision, Measurement 147 (2019) 106817, https:
//doi.org/10.1016/j.measurement.2019.07.045.
[36] R. Yang, T. Qiao, Y. Pang, et al., Infrared spectrum analysis method for detection
and early warning of longitudinal tear of mine conveyor belt, Measurement
(2020) 107856, https://doi.org/10.1016/j.measurement.2020.107856.
[37] C. Hou, T. Qiao, H. Zhang, et al., Multispectral visual detection method for
conveyor belt longitudinal tear, Measurement 143 (2019) 246–257, https://doi.
org/10.1016/j.measurement.2019.05.010.
Measurement 175 (2021) 109130
[38] X.L. Hao, H. Liang, A multi-class support vector machine real-time detection sys-
tem for surface damage of conveyor belts based on visual saliency, Measurement
146 (2019) 125–132, https://doi.org/10.1016/j.measurement.2019.06.025.
[39] Y. Wang, Y. Wang, L. Dang, Video detection of foreign objects on the surface
of belt conveyor underground coal mine based on improved SSD, J. Amb. Intell.
Hum. Comput. (2020) https://doi.org/10.1007/s12652-020-02495-w.
[40] R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for
accurate object detection and semantic segmentation, in: 2014 IEEE Conference
on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014,
pp. 580–587, https://doi.org/10.1109/CVPR.2014.81.
[41] R. Girshick, Fast R-CNN, in: 2015 IEEE International Conference on Computer
Vision (ICCV), Santiago, 2015, pp. 1440–1448, https://doi.org/10.1109/ICCV.
2015.169.
[42] S. Ren, K. He, R. Girshick, J. Sun, Faster R-CNN: Towards real-time object
detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell.
39 (6) (2017) 1137–1149, https://doi.org/10.1109/TPAMI.2016.2577031.
[43] W. Liu, D. Anguelov, D. Erhan, et al., Ssd: Single shot multibox detector, in:
European Conference on Computer Vision, Springer, Cham, 2016, pp. 21–37.
[44] J. Redmon, S. Divvala, R. Girshick, et al., You only look once: Unified, real-time
object detection, in: Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, 2016, pp. 779–788.
[45] J. Redmon, A. Farhadi, Yolov3: An incremental improvement, 2018, arXiv
preprint arXiv:1804.02767.
[46] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in:
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las
Vegas, NV, USA, 2016, pp. 770–778, https://doi.org/10.1109/CVPR.2016.90.
[47] T.Y. Lin, P. Dollár, R. Girshick, et al., Feature pyramid networks for object
detection, in: Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, 2017, pp. 2117–2125.
[48] M. Tan, Q.V. Le, Efficientnet: Rethinking model scaling for convolutional neural
networks, 2019, arXiv preprint arXiv:1905.11946.
[49] M. Everingham, L.V. Gool, C.K.I. Williams, et al., The pascal Visual Object Classes
(VOC) challenge, Int. J. Comput. Vis. 88 (2) (2010) 303–338.