2nd International Conference on Electrical Materials and Power Equipment - Guangzhou - China

Development of Power Transmission Line Defects

Diagnosis System for UAV Inspection based on
Binocular Depth Imaging Technology
Tianqi Mao1, Kai Huang2, Xianwu Zeng1, Lingran Ren2, Chuance Wang2, Shengfu Li1, Min Zhang3, Yu Chen2
1, Zunyi Power Supply Bureau, Guizhou Power Grid Corporation, Zunyi, Guizhou, P.R.China, 563000
2, School of Electrical Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, P.R.China, 710049
3, Northwest University, Xi’an, Shaanxi, P.R.China, 710127

Abstract-China's transmission lines are widely distributed. The
power department spends huge manpower and material
resources every year on transmission line patrolling. The
development of Unmanned Aerial Vehicle (UAV) technology
provides a new inspection method for overhead transmission line
patrolling. In the present days, most of the inspection images of
drones are based on monocular cameras. Due to the limitations
of two-dimensional images, this detection method is not sufficient.
Considering the issue that defects of transmission lines are not
ideal in the complex natural environment background, in this
paper a deep imaging UAV inspection transmission line defect
and lurking peril diagnosis system is proposed. This paper
introduces the related theory of binocular stereo vision, and
describes in detail related image processing techniques such as
binocular camera calibration, stereo image correction, stereo
matching, and image segmentation. By studying the imaging
model of the binocular vision system, a set of automatic
transmission line detection platform is established which greatly
improve the efficiency of the inspection of the UAV transmission
line. Using the system, real-time inspection of transmission lines,
intelligent diagnosis of defects is performed which greatly
improves the diagnosis efficiency and reduces the manually
workload.
Key Words: Power Transmission Line; Depth Imaging; SGC;
Defect Recognition; HOG; SVM
I. INTRODUCTION
UAV technology is a newly developed technology in recent
years [1]. There are not many scholars who specialize in the
use of unmanned aerial vehicle patrol technology in the world.
State Grid Corporation and China Southern Grid Corporation
have carried out the research of UAV patrol line, and have
made some achievements [2].
The research of domestic researchers mainly focuses on the
integration of hardware and software, including the
development of hardware platforms such as flight attitude
control system and the fusion of multi-sensors, and the data
acquisition of lines using UAV with high-resolution camera
and infrared thermal imager. The main method of analyzing
transmission line data is to screen out the main defects and
hidden dangers through manual analysis of video and pictures.
In foreign countries, with the help of advanced UAV flight
technology, the focus of related research has shifted to the
following aspects of image pattern recognition, compared
with domestic research level is still not high.
At present, Domestic and foreign scholars have done a lot
of research on image diagnosis of transmission line fault
inspection, but most of the research results are based on two-
dimensional image of transmission line scene, that is, by
improving or researching new image processing algorithms to
eliminate complex natural background, and then improve the
efficiency of identifying transmission line fault and defect [3].
However, the natural environment of transmission lines is
complex and changeable, and is subject to the influence of
photographic angle and color change, which has caused great
trouble to the research of image recognition and diagnosis of
specific elements of transmission lines. Therefore, using
stereo matching technology to identify and diagnose
transmission lines has high academic and industrial value.
Binocular stereovision has been developed in foreign
countries at first. So far, the research level of many foreign
countries has reached a certain level. There are also many
important achievements in the field of domestic research on
binocular vision [4].
The problem of transmission line algorithm based on
binocular vision is how to quickly and accurately get the
disparity map of transmission line image by stereo matching
and depth calculation, and segment the disparity map to
obtain the pure transmission line target. Therefore, this paper
aims to study the three-dimensional matching algorithm (SGC)
of transmission lines, and design and implement a prototype
system of transmission line detection based on binocular
vision for transmission line inspectors.
II. DESIGN OF THE SYSTEM
This paper designs a depth imaging based UAV inspection
transmission line defect diagnosis system, including: drone
platform, drone flight controller, ground vehicle mobile
workstation, power supply station patrol management center.
The UAV platform includes: a depth imaging device, an
onboard data center, and a flight control platform; the UAV
flight controller includes: a decoding module, a data
collection processing module, a monitoring module, and a
flight control module; and the ground vehicle mobile
workstation includes: image data Decoding module, image
data processing module, typical defect image intelligent
diagnosis system; power supply station patrol management
center includes: team monitoring subsystem, integrated

978-1-5386-8434-4/19/$31.00 ©2019 IEEE 478

management subsystem; wherein the depth imaging device is
used to collect the image of the area where the transmission
line is located, The grounding method is attached to the
unmanned aerial vehicle body, and the transmission line
defect diagnosis system uses relevant algorithms to
intelligently diagnose typical defects of the transmission line.
The structure diagram of the transmission line defect
diagnosis system for drone inspection based on depth imaging
is shown in Fig. 1.
Fig. 1. Schematic diagram of UAV inspection transmission line defect
diagnosis system
The schematic diagram of the application of UAV
inspection transmission line defect diagnosis system is shown
in Fig. 2.
Fig. 2. Schematic diagram of the application of UAV inspection transmission
line defect diagnosis system
The working flow chart of the transmission line defect
diagnosis system based on depth imaging is shown in Fig. 3.
For the stereo matching, this paper mainly analyzes and
improves the traditional GC stereo matching algorithm. Firstly,
the regional matching algorithm SAD (Sum of absolute
differences) is used to obtain the initial parallax, and then
linear interpolation is used to optimize the initial parallax.
When constructing the energy function, the pixel parallax in
the initial disparity map is used as the limiting term of the
energy function of the GC( GraphCuts ), and only the parallax
value within a certain range of the prior value is retained, and
the unrelated nodes and edges in the large number of trellis
diagrams are removed, and reduce the amount of calculation.
In the case of ensuring matching accuracy, the time taken for
matching is greatly reduced. In the aspect of threshold
479
segmentation, we should analyze the gray histogram of the
disparity map, and use bimodal threshold segmentation
algorithm to segment the parallax map, and then the
segmentation result is mapped to the original transmission
line image. Next the image features are extracted. The main
goal of the image feature extraction is to extract the
transmission line image features, and to reduce the
dimensionality around the direction gradient HOG (Histogram
of Oriented Gradients) feature extraction and PCA (principal
component analysis), and finally output the principal
component feature matrix. Then analyze the defect, Firstly,
we set the parameters of the support vector machine, run the
directed acyclic graph multi-classifier, match the principal
component feature matrix of the image with the defect
recognition model, and finally determine the defect category
and cause.
Fig. 3. work flow chart of the Fault Diagnosis System for UAV Inspection
Transmission lines Based on Depth Imaging
III. PRINCIPLE OF DEFECT DIAGNOSIS FOR UAV
TRANSMISSION LINE
A. Stereo matching algorithm based SGC
The goal of stereo matching is to find matching points from
different viewpoint images. The commonly used algorithms
are GC and SAD. The main issue of the GC stereo matching
method based on the complete grid graph is the huge
execution time overhead. Especially for the larger parallax
range, it requires a huge memory space overhead. To solve the
above problem, the SGC (SAD+GC) algorithm is adopted.
The region matching algorithm SAD is used to obtain the
initial parallax, and then the linear interpolation method is
used to mismatch the points, which can further improve the
quality of the initial disparity map. When constructing the
energy function, the pixel disparity in the initial disparity map
is used as a limiting term for the energy function of the GC.
Through these methods, the nodes can be reduced, thereby
reducing the amount of calculation and get the high-quality
parallax map.
The overall flow chart of the algorithm is shown in Fig.4:
Fig.4. Flow chart of improved GC stereo matching algorithm
The overall grid energy function consists of two parts: the
first part represents the gray value of each pixel point, and the
second part represents the smoothing attribute of the adjacent
pixel point pair [5]. So the smooth edge in the complete grid
map determines the value of the energy function. In the
simplified grid diagram, the priori disparity values calculated
by the local stereo matching algorithm SAD and the penalty
terms related to the smooth edges jointly determine the energy
function value.
When constructing a matching energy function, two bound
term are required for the data item and the smoothing item
[6].
In data items, the linear interpolation method of pixel
dissimilarity is adopted, that is, the dissimilarity of two pixels
cannot be measured only by the dissimilarity of pixel gray. In
order to improve the operation speed, SAD local optimization
algorithm is used to obtain the possible disparity range of the
pixels, and then to two-dimensional sub-pixel matching [7].
In order to avoid excessive smoothing in edge and multi-
texture regions, the smoothing function of Potts model is used.
B. Threshold segmentation algorithm based bimodal
threshold segmentation
Threshold segmentation method includes local threshold
method and global threshold method. The local threshold
method first divides the whole initial image into several sub-
images, and then uses the global threshold method to solve the
optimal threshold for each small image. The global threshold
method is to calculate the optimal segmentation threshold of
the whole image according to all the information of an image,
such as the gray distribution information of the whole image.
The selection of the threshold can be a single threshold or
multiple thresholds. The choice of threshold basically
determines the effect of threshold segmentation method.
Aiming at the gray value distribution characteristics
480
of disparity map of transmission line, the double peak method
is used to segment disparity threshold map, so as to obtain a
relatively pure transmission line target. Fig.5. shows the
results of stereo matching and threshold segmentation.
(a) Left, right and disparity images of normal transmission line
(b) Left, right and disparity images of strand breakage transmission line
(c) Left, right and disparity images of foreign matter in the transmission line
(d) Gray histogram of three transmission line images
(e)Divided map of three transmission lines
Fig.5. Original image of three transmission lines and pictures after depth
imaging
C. Transmission line feature extraction
The above Fig. shows the four pure transmission line
images after segmentation, so we can perform direction
gradient analysis. Stranded transmission lines and normal
transmission lines have many similar features, for example
their density distribution of gray levels is the same. The
strands of the normal transmission line are spiral around the
steel core, and the image gradient changes mainly concentrate
on the strands and the line edges, that is, the spiral rising
angle direction and the line edge direction. In the case of
strand breakage, the line condition is basically consistent with
the normal environment, and is also concentrated in the
direction of the spiral rising angle and the edge of the line, but
the broken strands are randomly distributed in other directions
in the space, and the gradient strength of spiral rising angle
direction and the line edge direction are relatively reduced.
The foreign object hangs or obscures the environment. Since
the foreign object blocks the line, there is no line strand, the
image gradient distribution has no obvious regularity, or in the
image with larger field of view, the gradient mutation will
occur. According to the above analysis, it can be found that
the gradient distribution of the broken strand defect is
different from the normal line, and the feature can be
extracted for classification.
Fig.6 is a HOG feature extraction process of a transmission
line image. The main steps are to normalize the transmission
line image that has been image preprocessed, after that
calculate the pixel gradient information, and then segment the
pixel cells to construct a gradient histogram of the pixel cells,
and combine the pixel cells into blocks and perform intra-
block normalization. Finally, the histogram vectors of all the
blocks are composed into the feature vectors of the HOG, and
the features are output.
Fig. 6.HOG feature extraction process
D. Transmission line defect classification
The purpose of the classifier in this paper is to compare the
performance improvement of each classification method with
respect to each sub-classifier. Therefore, the number of
samples misclassified by each sub-classifier is divided by the
total number of samples involved in the sub-classifier to
represent the classification error. The standard deviation
indicates the consistency of the average performance of each
sub-classifier. Fig.7 shows the classifier design of the system.
FIG.7.The classifier design diagram of the system
In this paper, there are 1150 transmission line images in the
dataset, 889 normal line images,125 strand breakage line
images and 136 foreign matter line images. Table 1 shows the
results of the HOG method before the dimension reduction.
TABLE I
RECOGNITION RESULTS BASED ON HOG METHOD
Average processing
Power transmission line
speed Recognition accuracy
type
(seconds/sheet)
Normal 0.530 81.4%
Strand breakage 0.570 76.5%
Foreign matter 0.534 77.2%
Average 0.552 80.3%
IV. CONCLUSION
Aiming at the issue of transmission line defect identification,
a UAV inspection transmission line defect diagnosis system is
designed. In this system, stereo matching algorithm based
SGC is used, SGC algorithm not only guarantees the accuracy,
but also improves the calculation speed from the aspect of
computation. Then a large number of transmission line image
pairs are collected by UAV-mounted binocular camera, and
the disparity map is obtained after processing. Based on the
481
characteristics of disparity map, the characteristics of
disparity map are analyzed, and the gray distribution bimodal
method is used to segment and compare disparity map, then
the best threshold of image segmentation is found, and the
disparity map is segmented by threshold. Finally, the complex
background in the image is removed and the pure target of
transmission line is extracted. For the image after threshold
segmentation, the transmission line image features are
extracted based on the directional gradient histogram, and the
pixel cell size and the principal component contribution
threshold of the directional gradient histogram are determined
by experimental comparison. The kernel function and optimal
parameters of support vector machine are experimentally
studied, and the optimal selection is determined. Three kinds
of multi-classifiers are compared and analyzed. Finally,
directed acyclic graph support vector machine is used to train
and extract features, and a defect recognition model is
established to realize the function of defect recognition of
transmission lines and the average recognition accuracy is as
high as 80.3%. Therefore, this system is efficient and
practical.
ACKNOWLEDGEMENT
This research was supported by China Southern Power
Grid, Science and Technology Project (GZKJXM20170280),
and Education Department of Shaanxi Province
(No.17JK0769), and National Natural Science Foundation of
China (No. 61701404) , and Specific Science Foundation of
China (No.61701404), and Specific Scientific Research
Program of Education Department of Shaan Xi
Province(17JK0769),and Science and Technology Project
Foundation of Beilin District, Shaan Xi Province(GX1703);
REFERENCES
[1] A. Cho, Y. Kang, B. Park, C. Yoo and S. Koo, "Altitude integration of
radar altimeter and GPS/INS for automatic takeoff and landing of a
UAV", 2011 11th International Conference on Control, Automation and
Systems, Gyeonggi-do, pp. 1429-1432, 2011.
[2] S. A. Korkmaz, A. Akçiçek, H. Bínol and M. F. Korkmaz, "Recognition
of the stomach cancer images with probabilistic HOG feature vector
histograms by using HOG features", 2017 IEEE 15th International
Symposium on Intelligent Systems and Informatics (SISY), pp. 339-342,
2017.
[3] S. Hamanaka, S. Kurihara, S. Fukuda, M. Oguchi and S. Yamaguchi,
"A Study on Object Lifetime in GC of Android Applications", 2016
Fourth International Symposium on Computing and Networking
(CANDAR), Hiroshima, pp. 701-703, 2016.
[4] Parveen and A. Singh, "Detection of brain tumor in MRI images, using
combination of fuzzy c-means and SVM", 2015 2nd International
Conference on Signal Processing and Integrated Networks (SPIN),
Noida, pp. 98-102,2015.
[5] Y. Yang, J. Wang and Y. Yang, "Exploiting rotation invariance with
SVM classifier for microcalcification detection," 2012 9th IEEE
International Symposium on Biomedical Imaging (ISBI), Barcelona, pp.
590-593, 2012.
[6] CHENG Gang, YANG Sui-hu, "Overview of airborne photovoltaic
systems for drones ". Applied Optics, pp. 1-42005,2015．
[7] FU Bo, JIANG Yong, WANG Hong-guang, et al, "Power line
inspection image intelligent diagnosis system", Journal of Intelligent
Systems, pp.7-9,2016.