15th

JuneIFAC Symposium
6-8, 2018. Savona,on Control in Transportation Systems
Italy
15th IFAC
June Symposium
6-8, 2018. Savona,on Control in Transportation Systems
Italy
15th
JuneIFAC Symposium
6-8, 2018. Savona,on Control in Transportation
Italy Systems
Available online at www.sciencedirect.com
JuneIFAC
15th 6-8, 2018. Savona,on
Symposium Italy
Control in Transportation Systems
June 6-8, 2018. Savona, Italy
ScienceDirect
Multi-Objective Performance Evaluation
IFAC PapersOnLine of98–105
51-9 (2018) the Detection of Catenary Support
Multi-Objective Performance Evaluation of the Detection of Catenary Support
Multi-Objective Performance Evaluation
Components Usingof the Detection of Catenary Support
DCNNs
Multi-Objective Performance Components
Evaluation Usingof DCNNs
the Detection of Catenary Support
Multi-Objective Performance Components
Evaluation Usingof theDCNNs
Detection of Catenary Support
Wenqiang Liu*, Zhigang Liu *,Components
Alfredo Núñez**,Using DCNNs
Liyou Wang*, Kai Liu*, Yang Lyu*, Hongrui Wang*
Wenqiang Liu*, Zhigang Liu *,Components
Wenqiang Liu*, Zhigang Liu *, Alfredo Núñez**,Using DCNNs
Alfredo Núñez**, Liyou Wang*, Kai Liu*, Yang Lyu*, Hongrui Wang*
Liyou Wang*, Kai Liu*, Yang Lyu*, Hongrui Wang*

Wenqiang Liu*, Zhigang Liu *, Alfredo Núñez**, Liyou Wang*, Kai Liu*, Yang Lyu*, Hongrui Wang*
Wenqiang Liu*, Zhigang * School
Liu of *, Electrical
Alfredo Nú Engineering,
ñez**, Liyou Southwest
Wang*, Jiaotong
Kai Liu*,UniversityYang Lyu*, Hongrui Wang*
* School of Electrical
China Engineering,
(e-mail: Southwest
Liuwq_2009@126.com).

Jiaotong University
* School of Electrical
China Engineering,
(e-mail: Southwest
Liuwq_2009@126.com). Jiaotong University
** SectionofofElectrical
* School Railway
China Engineering,
(e-mail: Delft University
Southwest
Liuwq_2009@126.com). Jiaotong of Technology
University
**
* Section
School of of
NetherlandsRailway
Electrical
China Engineering,
(e-mail: Delft University
A.A.NunezVicencio@tudelft.nl)
Engineering, Southwest
Liuwq_2009@126.com). Jiaotong of Technology
University
** Section of Railway
Netherlands Engineering,
(e-mail: Delft University of Technology
A.A.NunezVicencio@tudelft.nl)
** Section of China
Railway (e-mail: Liuwq_2009@126.com).
Engineering, Delft
Netherlands (e-mail: A.A.NunezVicencio@tudelft.nl) University of Technology
** Section of Railway
Netherlands Engineering,
(e-mail: Delft University of Technology
A.A.NunezVicencio@tudelft.nl)
Abstract: The goal of thisNetherlands paper is to evaluate (e-mail:from a multi-objective perspective the performance on the
A.A.NunezVicencio@tudelft.nl)
Abstract:
detection
Abstract: ofThe
The
goal ofsupport
catenary this paper
goal ofsupport
this paper
is to evaluate
components
is to evaluatewhenfrom usingaa state-of-the-art
from
multi-objectivedeep
multi-objectivedeep
perspective the performance
convolutional
perspective neural networks
the performance
on the
on the
detection
(DCNNs).ofThe
Abstract: of catenary
The detection
goal thisof
ofsupport components
components
paper is to evaluatewhen using
is thefrom state-of-the-art
firsta state-of-the-art
step towards adeep
multi-objective convolutional
complete
perspective automatized neural
the performance networks
monitoring
on the
detection
(DCNNs). catenary
The detection of components
components when
is about using
thefromfirst step towards adeep convolutional
complete automatized neural networks
monitoring
system that
Abstract:
detection
(DCNNs). willgoal
ofThe
catenary
The provide
detection thisactual
ofsupport paper
of information
is to evaluate
components
components when
is the usingdefects
first step in the catenary
a state-of-the-art
multi-objective
towards a support
perspective thedevices.
convolutional
complete performance
automatized neural A series
on the
networks
monitoring of
system
experiments
detection
(DCNNs). that
of will
in an
catenaryprovide
unified actual
test
support information
environment
components for
when about defects
detection
using of in the
components
state-of-the-art catenary
deepare support
performed
convolutional devices.
using A series
Faster-CNN,
neural networks of
system
experiments thatThewill
in
detection
an provide
unified
of
actual
test information
environment
is about
for
the first
detection
step towards
defectsof in the catenary
components
a complete
are support
performed
automatized
devices.
using Amonitoring
series of
Faster-CNN,
R-FCN,
(DCNNs).
system
experiments SSD,
The
that will
in an and YOLOv2.
detection
provide
unified of
actual
test Through
components
information
environment the
is comparison
the
forabout first step
defects
detection of different
towards
in the catenary
of components a assessment
complete
are indicators,
automatized
support devices.
performed such
monitoring
A series of
using Faster-CNN, as
R-FCN,
precision,
system
experiments SSD,
that recall,
will
in an and YOLOv2.
average
provide
unified actual
test Through
precision and the
information
environment mean comparison
forabout average
defects
detection of different
of precision,
in the
components catenary assessment
theare detection
performed indicators,
support performance
devices. such
A series
using Faster-CNN, of theas
of
R-FCN,
precision, SSD,
recall, and YOLOv2.
average Through
precision and the
mean comparison
average of different
precision, the assessment
detection indicators,
performance such
of as
the
different
experiments
R-FCN,
precision, DCNNs
SSD, in an
andmethods
unified
recall, methods YOLOv2.
average forfor
test the components
environment
Through
precision for
the
and mean of the
detection
comparison catenary
of support
components
of different devices
are is
performed
assessment analyzed,using
indicators,discussed
Faster-CNN,
such and
as
different
evaluated.
R-FCN,
precision, DCNNs
The
SSD,
recall,experiment
and YOLOv2.
average the
results components
show
Through
precision andthat
the
mean of average
among theall
comparison
average
precision,
catenaryof support
considereddifferent
precision,
the
the
detection
devices
methods, R-FCN
assessment
detection
performance
is analyzed,
is the discussed
indicators,
performance more such
of and
the
suitable
of andas
the
different
evaluated. DCNNs
The methods
experiment for the
results components
show that of
among the catenary
all consideredsupport devices
methods, is
R-FCN analyzed, discussed
is the discussed
more suitable
for the
precision,
different
evaluated. detection
recall,
DCNNs of catenary
average
methods
The experiment forsupport
precision
the
results components.
and
components
show mean of average
the precision,
catenary support the detection
devices
that among all considered methods, R-FCN is the more suitable is performance
analyzed, of the
and
for the detection
different
evaluated. DCNNs ofmethods
The experimentcatenaryfor support
results components.
the components
show that amongof theall catenary
consideredsupport devicesR-FCN
methods, is analyzed,
is the discussed
more suitable and
©
for the
2018,
Keywords:The
evaluated.
detection
IFAC of
Catenary, catenary
(International
experiment Railway support
Federation
results Systems, components.
show of Automatic
Multi-Objective
that among Control)
all Hostingmethods,
Performance
considered by Elsevier
Evaluation, Ltd. All
R-FCN Deep
is rights
the reserved.
convolutional
more suitable
for the
Keywords: detection of
Catenary, catenary support components.
Railway Systems, Multi-Objective Performance Evaluation, Deep convolutional
neural
for the networks
Keywords: detection (DCNNs)
of catenary
Catenary, Railway support
Systems, components.
Multi-Objective Performance Evaluation, Deep convolutional
neural
Keywords: networks (DCNNs)
Catenary, Railway Systems, Multi-Objective Performance Evaluation, Deep convolutional
neural networks (DCNNs) 
Keywords:
neural networks Catenary,
(DCNNs) Railway Systems, Multi-Objective  Performance Evaluation, Deep convolutional
neural networks (DCNNs) 
histogram of oriented gradients (HOG) feature to express the
1. INTRODUCTION histogram of oriented
1. INTRODUCTION  rotary
histogram double-ear,
of oriented andgradients
combined
gradients
(HOG)
(HOG) withfeature
feature
to express
the support
to express
the
vector
the
1. INTRODUCTION rotary double-ear, and combined with the support vector
With the rapid development of high-speed railway, a large machine
histogram
rotary (SVM)
of oriented
double-ear, to recognize
and gradients
combined it.(HOG)
Then,
with thetheGabor
feature wavelet
to express
support is
the
vector
With theofrapid 1. INTRODUCTION
development of high-speed railway, a large machine
utilized
histogram (SVM)
to detect
of tothe
oriented recognize
failure
gradients it.(HOG)
of Then,
the thetheGabor
component.
feature to wavelet
Authors
express is
in
the
number
With new
theofrapid railway infrastructures
1. INTRODUCTION
development of high-speedwill be constructed
railway, a largeall rotary double-ear,
machine to(SVM) and
tothe
recognizecombined it. the with
Then, support
the Gabor Authors vector
wavelet in is
number new railway infrastructures will be constructed all utilized
(Zhang
rotary
machine et detect
al.,
double-ear,
(SVM) 2017) and
tothe failure
combined
recognizecombined of the
it. Then, component.
difference
with the histograms
support
the Gabor Authors vector
wavelet in of
is
over
With
number world.
theofrapid
new The larger
development
railway ofthehigh-speed
infrastructures infrastructure,
will be railway, thea more
constructed large
all utilized to detect failure of the component.
over theofrapid
world. The (Zhang et al., 2017) combined the difference histograms of
challenges
With
over the
number new
the world. arerailway facedlarger
development
The whenofthe
infrastructures
larger
infrastructure,
deciding
thehigh-speed
will berailway,
infrastructure,
thea more
infrastructure
constructed large
the more all oriented
machine
utilized
(Zhang et gradients
(SVM)
to detect
al., to(DHOG)
2017) failureandof
recognize
the combined it.AdaBoost
Then,
the
the thealgorithm
component.
difference Gabor to detect
wavelet
Authors
histograms is
in
of
challenges are faced when deciding infrastructure oriented
the
utilized
(Zhang gradients
auxiliary
to detect
al., 2017)
etgradients (DHOG)
catenary
the wire,
failure
combined and of AdaBoost
and the judged algorithm
component.
the difference the fault to detect
of
Authors
histograms the
in
of
maintenance
number
over theof world.
challenges new tasks.
are railway
The In larger
faced thiswhen sense,
infrastructures automated
will be constructed
the infrastructure,
deciding monitoring
the more
infrastructure all oriented (DHOG) andand AdaBoost algorithm toof
detect
maintenance tasks. In larger
this sense, automated monitoring the
(Zhangauxiliary
auxiliary
oriented al., catenary
catenary
etgradients 2017) wire,
wirecombined
(DHOG) throughand judged
the circular
difference
AdaBoost the faultto
arc histograms
detection
algorithm the
and
detectof
systems
over the
challenges
maintenance capable
world.
are
tasks.to detect
The
faced
In this defects
when the in
sense, the
decidingwhole
infrastructure,
automated the more
infrastructure
monitoring the auxiliary catenary wire, and judged the fault of the
systems capable to detect defects in the whole infrastructure auxiliary
segment
oriented
the catenary
clustering.
gradients
auxiliary wire
In
(DHOG)
catenary through
(Han
wire, et
and the 2016),
al., circular
AdaBoost
and judged arc detection
segment
algorithm
the and
clustering
fault to detect
of the
will assure
challenges
maintenance
systems theare
capable safe
tasks.to operation
faced
In this
detect ofsense,
when
defects theincomplete
deciding
automated
the whole railway system.
infrastructure
monitoring
infrastructure auxiliaryclustering.
catenary wire through the circularsegment arc detection and
will assure thetasks.
safe segment In (Han et andthe 2016),
al., clustering
maintenance
systems
will assurecapable
the safeto operation
In this
detect
operation
ofsense,
defects
of
theincomplete
the automated
the
complete whole railway system.
monitoring
infrastructure
railway system.
is
theproposed
auxiliary
auxiliary
segment
is proposed
tocatenary
catenary
clustering. firstwire
to part
divide
In
firstwire
wire,
through
(Han
divide
the
et
the
independent
al., judgedsegment
circular
2016),
independent
area,
the
arc
area,
and
fault
detection utilize
of
clustering
and
the
and
utilize
Two
systems of capable
the most to important
detect defects infrastructure
in the whole components
infrastructure in the deformable
auxiliary
segment
is proposed catenary
clustering. models
In
to first models through
(Han
divide the (DPM)
et the
al., and
circular
2016),
independent latent
arcSVM
segment
area, to
detection detect
and
clustering
and utilize
will
Twoassure thethemost
of system safe operation
important ofinfrastructure
the complete railway components system. in the deformable
railway
will
Twoassure thethemost
of system areoperation
safe the catenary
important theand the track
ofinfrastructure
complete railwaysystems,
components system. as
in the is
the rod-insulator.
segment
proposed
deformable to part
clustering. first
part The
In localthe
(Han
divide
models
(DPM)
(DPM)
and latent
etnormalization
al., 2016),
independent
and latent
SVM
(LN)
segment
area,
SVM
to utilize
method detect
clustering
andto detectto
railway
shown in Fig. 1.are the
Potential catenary
failures andof the
the track
catenary systems,
or trackas the rod-insulator.
achieve
is proposed the tocontrast
firstThe local
enhancement
divide the normalization
independent of the (LN)
rail
area, method
image
and to
is
utilize
Two
railway of system
the most are important
the catenary infrastructure
and the trackcomponents
systems, in
as the
the deformable
rod-insulator. partThemodelslocal(DPM) and latent(LN)
normalization SVMmethod to detect to
shown
will
Two
railway insystem
severely
of Fig.threaten
the 1.are
most Potential
the
important
the failures
railway
catenary of the
traffic the
infrastructure
and catenary
safety.
track For
components or track
ensuring
systems, in
as achieve
proposed
the the
deformable in contrast
(Li and
part Ren,
modelsenhancement
2012),
(DPM) andand of
the the
defect
latent rail
SVM image
localization
to detectis
shown in Fig.threaten
1. Potential failures of the catenary or track achieve rod-insulator.
the contrastThe enhancement
local normalization of the (LN) method
image to
raillocalization is
will
the
shown severely
safe
railway operation
insystem 1.are
Fig.threaten ofthethe
Potential railway
high-speed
catenary
failures traffic
railway,
andof the safety.
the atrack
large
catenaryFor ensuring
number
systems,
or track of the
as proposed
based on the in (Li
projection
rod-insulator. and Ren,
profile
The local2012),
(DLBP) and isof
normalization the
used defect
to detect
(LN) defects.
method to
will severely the railway traffic safety. For ensuring achieve
proposed in contrast
(Li and Ren,enhancement
2012), and the the
defectrail image
localization is
the safe inoperation of high-speed railway, abeenlarge number of based on theprojection
monitoring
shown
will severely
the safe and
Fig. 1.detection
threaten
operation the technology
Potential failures
railway
of high-speed ofhave
traffic
railway, the catenary
safety.
abeenlargeinvestigated,
For or track
ensuring
number A
of achieve probabilistic
proposed
based in (Li
on projection andprofile
model
contrast to
Ren,
profile
(DLBP)
differentiate
enhancement
2012),
(DLBP) and isofused
is the
used theto rail
fasteners
defectdetect
to detect
defects.
tolocalization
track
image and
defects.is
monitoring
and
will
the the
severely
safe and
operation detection
corresponding
threaten of the technology
detection
railway
high-speed have
trafficequipment
railway, safety.
a largeinvestigated,
For has been
ensuring
number of A probabilistic
judge
proposed the in fault
(Li model
andof to differentiate
fasteners
Ren, 2012), based
and fasteners
the on the
defect to track and
likelihood
localization
monitoring and detection technology have been investigated, based on
A probabilisticprojection model profile (DLBP)
to differentiate is used to
fasteners detect defects.
to track and
and the operation
corresponding detection equipment has been of judge theprojection
faultmodel ofprofile
fasteners based onto2014).
the
developed
the safe
monitoring
and the and
andapplied.
detection
corresponding Intechnology
this
of high-speed paper
detection the
railway,
have focus
abeen
equipment isinvestigated,
large on catenary
number
has been probability
based
A on
probabilistic
judge the was
fault established
of to in (Feng
(DLBP)
differentiate
fasteners is etused
based al.,
fasteners
on to likelihood
detect
the In many
defects.
track and
likelihood
developed
support
monitoring
and and
thedevices andapplied.
relying on
detection
corresponding Intechnology
this
image paper
detection the
processing
have focus
equipment isinvestigated,
onhas
technology.
been catenary
been probability
aspects,
A was
these
probabilistic established
traditional
model to in (Feng
feature
differentiate etfasteners
al.,
extraction 2014).
and
to In many
machine
track and
developed and applied. In this paper the focus is on catenary judge
probabilitythe fault of
was traditional fasteners
established feature in (Feng based on the
et al., 2014). likelihood
In many
support
and thedevices
developed relying on
corresponding
and applied. image
detection
In this processing
paper theequipment
focus technology.
is onhas been judge
catenary aspects,
learning thethese
methods offor fasteners
faulttraditional object extraction
recognition
based andlikelihood
have
on 2014).
the machine
obtained
support devices relying on image processing technology. probability
aspects, was
these established in (Feng
feature et al.,
extraction and In many
machine
In (Cho andand
developed Ko, 2015), Inthethis
applied. scale-invariant
paper the focus feature
is ontransform
catenary learning
positive methods
results. for object
However, intheir recognition
performance have obtained
support
In (Chois devices
and Ko, relying
2015), ontheimage processing
scale-invariant technology.
feature transform probability
aspects,
learning these was traditional
methods established
for object feature(Feng et al., 2014).
extraction
recognition and has
have In been
many
machine
obtained
(SIFT)
support
In (Cho andemployed
devices Ko, relying to
2015), track
ontheimageand locate
processing
scale-invariant the pantograph,
technology.
feature transformand positive
difficult
aspects, toresults.
theseget However,
improved.
traditional In their
featurerecent performance
years,
extraction with
and has
the been
rapid
machine
(SIFT) isandemployed to track and locateoverhead
the feature
pantograph, and learning methods However,
positive toresults. for objecttheir recognition
performance have has obtained
been
assessing
In (Cho the
(SIFT) is employed Ko, reliability
2015), the
to track of railway
scale-invariant
and locateoverhead
the pantograph, power
transform by
and difficult
development
learning get improved.
of deep
methods forlearning,
objectIn recent
there years,
are
recognition with
methods
have the
appliedrapid
obtained in
assessing the reliability of railway power by positive
difficult toresults.
get However,
improved. In their
recent performance
years, with has
the been
rapid
measuring
In (Choisand
(SIFT)
assessing the
theKo,stagger
employed to between
2015),
reliability track and the pantograph
theofscale-invariant
locate
railway the feature
overhead and
pantograph,
power contact
transformand
by development
the industry
positive of detect
to
results. deep learning,
However,the fault there
their of are methods
equipment.
performance In applied
(Faghih-
has beenin
measuring the stagger between the pantograph and contact difficult
development to get improved.
of detect
deep learning, In recent
there years, with
are methodsInapplied the rapidin
wire.
(SIFT) The
assessing
measuring is speeded-up
employed
the to
reliability
the stagger robustrobust
track of
between features
and locate
railway (SURF)
the
overhead
the pantograph is proposed
pantograph,
power
and and
contact in
by the
Roohiindustry
difficult et to to
al.,get2016), the
alearning,
improved. deep fault of
Inconvolutional
recent equipment.
years, neural
with (Faghih-
network
the rapidis
wire. The speeded-up features (SURF) is proposed in development
the industry of
to deep
detect the fault thereof are methods
equipment. In applied
(Faghih- in
(Yang
assessing
measuring
wire. et al.,
The the 2013),
the reliability
stagger
speeded-up to extract
robust of features
between railway features
overhead
the pantograph
(SURF) and
is detect
power
and the
contact
proposed by
in Roohi
proposed et al.,
development for 2016),
the
of alearning,
analysis
deep deep of convolutional
image
there data
are forneural
the
methods network
detection
applied is
of
in
(Yang et al., the industry to detect the fault of equipment. In (Faghih-
insulator.
measuring
wire.
(Yang The
et the 2013),
Then,
speeded-up
al., 2013),the to
stagger extract
state
between
robust
to extractof the
features
the
features
(SURF) and
the pantograph
insulator
features and
detect
isisandevaluated
proposed
detect
the
contact
the
Roohi et al.,
in proposed
rail surface for 2016),
the a deep
analysis
defects. of convolutional
image data forneural
the network of
detection is
insulator. Then, the state of the insulator is evaluated the
Roohiindustry
proposed et al.,
for to
the detect
2016), aIndeep
analysis the(Gibert
of fault
image et data
of
convolutional 2017),
al., for
equipment.neural
the anetwork
In detection
(Faghih-
detection is
of
according
wire.
(Yang The
insulator. to
et al.,
Then,the
speeded-up
2013),vertical
robust
the tostate grayscale
extractfeatures
of the statistic
(SURF)
the features distribution.
is
and proposed
is detect
insulatordistribution. evaluated In
in
the rail
methodsurface
etwas defects. aIn (Gibert et data
al., 2017), anetwork
detection
according to Roohi
proposed forproposed
al., 2016),
the analysiswhich
deep is able et to
of convolutional
image improve
for the the
neural a detection
detection is
of
(Zhang
(Yang
insulator.
according al.,the
etet al.,
Then,
to 2013),
the
vertical
2016),
the tostate
vertical
grayscale
theextractof the statistic
contourlet
grayscale transform
features
insulator
statistic andis(CT)
detect along
evaluated
distribution.
In
the
In
rail surface
method
performance
proposed was for
defects.
proposed
through
the
In
which(Gibert
combining
analysis of is able
image toal.,
multiple
data
2017),
improve
for the
detectors
the
detection
detection
withinofa
detection
(Zhang et al., 2016), the contourlet transform (CT) along rail
methodsurface
was defects. In
proposed which(Gibertis able et toal., 2017),the
improve a detection
detection
with theetChan-Vese
insulator.
according
(Zhang Then,
to the (CV)
al.,the2016), state
vertical model
of the
the grayscale
contourlet is statistic
proposed
insulator
transform to detect
is(CT)
evaluated
distribution. and
along In performance
multi-task throughframework.
combining multiple detectors withinthe a
with theetChan-Vese (CV) model for is statistic
proposed to(CT)
detect and rail
method
performance waslearning
surface defects.
proposed In
which
throughframework. (Gibert
combining is ableInet (Chen
toal.,
multiple et al.,the
2017),
improve
detectors a2017),
detection
withinthe a
diagnose
according
(Zhang
with the the the
to insulator.
al.,the
Chan-Vese vertical
2016), A
the
(CV) method
grayscale
contourlet
model for failure
transform
is proposed detection
distribution.
to detect of the
along In
and multi-task
steady
method arm learning
base
waslearning
proposed was recognized
which is ableIn (Chen
and et
located
to improve al.,
with2017),
the2017),region-
detection
diagnose insulator. A method failure detection of the performance
multi-task through combining
framework. Inmultiple
(Chen detectors
et al., within thea
ear pieces
(Zhang isal.,
described
theetChan-Vese 2016),(CV) in method
the (Han et for
contourlet al., 2017), detection
transform which
to(CT) used the
along steady arm base was recognizedIn and located
with
diagnose
ear pieces the insulator.
is insulator.
described A model
in method
(Han
is proposed
et for
al., failure
2017), detection
which
detectof
used
and
the
the performance
multi-task
steady armlearning through
base was combining
recognizedmultiple
framework. (Chen
and et al.,with
detectors
located with
region-
within
2017),
region-thea
with the
diagnose Chan-Vese
the (CV)
A model is proposed
failure
ear pieces is described in (Han et al., 2017), which used the to detectof and
the multi-task
steady armlearning base was framework.
recognizedInand (Chen et al.,with
located 2017),
region-the
diagnose
ear pieces©the
Copyright insulator.
2018
is IFAC A
described in method
(Han et for al.,failure
2017), detection
which used of the 98 steady arm base was recognized and located with region-
Copyright
ear pieces©
2405-8963 © is2018
2018, IFAC
described in (Han et Federation
IFAC (International al., 2017),ofwhich Automatic the 98 Hosting by Elsevier Ltd. All rights reserved.
usedControl)
Copyright © 2018 IFAC 98
Peer review under responsibility of International Federation of Automatic Control.
Copyright © 2018 IFAC
10.1016/j.ifacol.2018.07.017 98
Copyright © 2018 IFAC 98
 2018 IFAC CTS
June 6-8, 2018. Savona, Italy Wenqiang Liu et al. / IFAC PapersOnLine 51-9 (2018) 98–105 99

based convolutional neural networks (RCNN) first, and then quickly and accurately using the traditional image detection
on the basis of the installation relationship, the steady arm technology all those types of defects. In this paper, catenary
was detected with Hough transform, and the slope of the support device images are used to train DCNNs, and evaluate
angle of steady arm was calculated. Although the progress in the detection performance of different DCNNs.
recent research efforts in the literature (Jamshidi et al., 2017;
Liu et al., 2017; Chen et al., 2018; Psuj, 2018), there is still a
gap towards the application of DCNN techniques in the
railway industry.
In the literature, different state-of-the-art deep convolutional
neural networks (DCNNs) structures are available. However,
for the specific task of detection in railway systems
environments, not all the neural network structures will
perform in the same way. Therefore, this paper proposes a
simple assessment methodology based on multi-objective
performance evaluation. The most advanced and the most
representative structures are used for the detection of Fig. 2. XLN4C inspection vehicle
components in the catenary support, and the intention is to
provide a systematic approach to support on their evaluation.
The rest of the paper is organized as follows. First, the
inspection system is shown in Section 2. Four of the state-of-
the-art DCNNs are introduced in Section 3. Next, the
12
experiment results are analyzed and discussed in Section 4.
Last, some conclusions are summarized in Section 5. 11

Suspension Device

Support Device
Catenary Fig. 3. Catenary support device

3. DCNN STRUCTURES OVERVIEW

Currently, DCNNs-based deep learning method has been the
mainstream for solving object detection problems. There are
Railway Tracks a lot of the state-of-the-art DCNNs structures developed,
proposed and used in the various fields. Among all, region-
based and regression-based DCNNs structures are two main
research directions, for which the representative algorithms
Fig. 1. Catenary and track devices are Faster RCNN, R-FCN, SSD, and YOLOv2. The core idea
and network structure of these algorithms are described
2. INSPECTION SYSTEM below.

XLN4C is a comprehensive inspection vehicle developed by

the China Railway Inc., shown in Fig. 2. It is equipped with
six detection and monitoring systems including
comprehensive pantograph and catenary monitor system
(CPCM-1C), catenary-checking video monitor system
(CCVM-2C), catenary-checking on-line monitor system
(CCLM-3C), high-precision catenary-checking monitor
system (CCHM-4C), catenary and pantograph video monitor
system (CPVM-5C), and ground monitor system for catenary
and power supply equipment (CCGM-6C), and called as the
6C system. In particular, the 4C system is mainly used to
detect and diagnose the condition of catenary by analyzing
the collected 2D images as the one shown in Fig. 3. A large
amount of catenary 2D images are acquired from this system
and state of the art image processing technology used for the
analysis.
Catenary support device has a complex structure with many
components. In addition, its defect type and defect level are
various, as shown in Appendix A. It is difficult to detect
3.1 Faster RCNN
Faster RCNN (Ren et al., 2015) is a series of RCNN
(Girshick et al., 2014). It is an integration of region proposal
networks (RPNs) and Fast R-CNN (Girshick, 2015), which
uses RPNs to extract the detection areas and then uses
conventional neural networks (CNNs) to extract image
features. Lastly, SoftMax function is used to achieve the
classification and bounding boxes regression to locate the
position of objects. The structure diagram is shown in Fig. 4.
3.2 Region-based fully convolutional networks (R-FCN)
R-FCN (Dai et al., 2016) is seen as an improved version of
the Faster RCNN. It moved several full connection layers
behind the region of interest (RoI) pooling layer to the front,
which greatly improved the detection speed. Meanwhile, the

99
2018 IFAC CTS
June 6-8, 2018. Savona, Italy
100 Wenqiang Liu et al. / IFAC PapersOnLine 51-9 (2018) 98–105

feature extraction network was replaced with ResNet. The Bboxes generation
with dimension priors and lacation prediction
structure diagram is shown in Fig. 5.
Objectscls
Feature
3.3 Single shot multibox detector (SSD) extraction
Objectsreg

SSD (Liu et al., 2016) can be simply seen as a combination YOLOv2

of YOLO(v1) (Redmon et al., 2016) and anchor boxes idea,
which is based on bounding boxes regression. By using a
small convolution kernel on different feature maps to predict
a series of box offsets of bounding boxes (Bboxes), the goal
of the object detection can be achieved. The structure
diagram is shown in Fig. 6.
3.4 YOLOv2
YOLOv2 (Redmon and Farhadi, 2016) is an improved
version of YOLO(v1), which is also based on the bounding
boxes regression. YOLOv2 draws on the anchor ideas in the
Faster R-CNN, which samples on the convolution feature
map with sliding window. Then, each centre predicts nine
different sizes and proportions of the proposed box. Since
there is no need for reshaping the convolution layer, the
spatial information is kept, solving the shortcomings of
YOLO (v1). The structure diagram is shown in Fig. 7.
RPN
Fig. 7. Structure diagram of YOLOv2
4. EXPERIMENT AND RESULTS
In order to evaluate the detection performance of the four
different DCNNs methods presented in the previous section,
different indexes are chosen to assess their performance. The
experiment environment is as follows. Deep learning open
source framework Caffe (Jia et al., 2014), Ubuntu 14.04,
32GB RAM, CPU clocked at 3.2 GHz, and GTX 1080 GPU
with 8GB memory.
4.1 Dataset and Parameter settings
The catenary dataset is made by the tool called “labelImg”
provided from the website 1 . The total image amount of
dataset is 5022, among which the training dataset is 2417, the
validation dataset is 1036 and the test dataset is 1569. The
experiment parameter settings are as follows. Momentum and
weight decay are 0.9 and 0.0005, and learning rate is 0.001,
iterations are 15, 000.

Objectscls 4.2 Evaluation Indexes

Feature
extraction
Objectsreg Some indexes are chosen including the precision and recall,
Faster RCNN
RoI pooling layer
average precision (AP), mean average precision (mAP) and
frames per second (FPS). Some curves and charts are drawn
including precision and recall curve (P-R curve) and loss
Fig. 4. Structure diagram of Faster RCNN curve.
TP
precision   100% (1)
TP  FP
RPN
TP
recall   100% (2)
TP  FN
1
Objectscls AP  0 p ( r ) d r (3)
Feature
Q
 q 1 AP( q )
extraction
Objectsreg
mAP  (4)
RFCN
RoI pooling layer
Q
where TP is true positive, FP is false positive, TN is true
Fig. 5. Structure diagram of R-FCN negative, Q is the number of the component class.
Bboxes generation
with default fixed boxes
4.3 Experiment Results
Objectscls
1) Examples of component detection effects
Feature
extraction
Objectsreg
In order to show the detection effect of the four different
SSD structures under the simple environment and complex
Extra feature extraction
environment, two sets of examples are shown in Fig. 8. One
Fig. 6. Structure diagram of SSD
1 https://github.com/tzutalin/labelImg

100
 2018 IFAC CTS
June 6-8, 2018. Savona, Italy Wenqiang Liu et al. / IFAC PapersOnLine 51-9 (2018) 98–105 101

only includes one set of catenary support device, and the other contains two sets of catenary support device.

(a)

(b)

(c)

(d)
Figure. 8. Two examples of component detection effects for the four different DCNNs architecture. (a) row is the detection
with Faster RCNN, (b) row is the detection with R-FCN, (c) row is the detection with SSD, and (d) row is the detection with
YOLOv2.

101
2018 IFAC CTS
June 6-8, 2018. Savona, Italy
102 Wenqiang Liu et al. / IFAC PapersOnLine 51-9 (2018) 98–105

(a) (b) (c) (d)

(e) (f) (g) (h)

(i) (j) (k) (l)

Figure. 9. P-R Curve for the detection of all the parts of the catenary support device (a) Insulator, (b) Rotary double-ear, (c)
Binaural sleeve, (d) Brace sleeve, (e) Steady arm base, (f) Bracing wire hook, (g) Double sleeve connector, (h) Messenger wire
base, (i) Windproof wire ring, (j) Insulator base, (k) Isoelectric line, (l) Brace sleeve screw, among, (a)~(j) are the large objects,
(k) and (l) are the small objects.

Table 1. Detection average precision on catenary dataset for the four different DCNNs architecture
Rotary Bracing Double Messen Brace
Catenary Binaural Brace Steady Windproof Insulator Isoelectric
Insulator double- wire sleeve ger wire sleeve mAP FPS
Dataset sleeve sleeve arm base wire ring base line
ear hook connector base screw
Faster
0.783 0.842 0.785 0.796 0.767 0.563 0.843 0.804 0.508 0.65 0.181 0 0.627 1.46
RCNN
R-FCN 0.757 0.88 0.861 0.88 0.835 0.732 0.846 0.787 0.682 0.846 0.334 0 0.703 2.02

SSD 0.877 0.856 0.818 0.834 0.876 0.459 0.812 0.782 0.223 0.683 0.716 0.003 0.662 2.30

YOLOv2 0.886 0.261 0.381 0.225 0.73 0.587 0.703 0.615 0.586 0.743 0.511 0.027 0.521 3.70

102
 2018 IFAC CTS
June 6-8, 2018. Savona, Italy Wenqiang Liu et al. / IFAC PapersOnLine 51-9 (2018) 98–105 103
Figure. 10. Detection average precision on catenary dataset for the four different DCNNs architecture
2) P-R curve
For analyzing the relationship between the false detection
rate and missed detection rate of each component for four
different DCNNs method, the curve of the precision and
recall (P-R curve) of every part is drawn up, as shown in Fig.
9.
3) AP and mAP
For comparing the detection accuracy and detection
efficiency of different models, the mathematical statistics for
the test dataset are carried out through AP and mAP as well
as FPS, as show in Table 1 and Fig. 10.
4) Training loss
To measure the robustness of the training model, the training
loss curve is plotted as the number of iterations increases, as
shown in Fig. 11.
Fig. 11. Training loss for the four different DCNNs
architecture
103
4.4 Results analysis
(1) From Fig. 8, the detection results based on regression
method, SSD and YOLOv2 have the result of missed
detection even if it is under the simple background
environment (as shown in Fig. 8(c)-left and Fig. 8(d)-
left). Among the detection results with Faster RCNN
and R-FCN, the latter performs better.
(2) It can be found from Table 2 and Fig. 10 that the
detection results with R-FCN are the best among these
DCNNs, whether it is a single object AP or the whole
mAP. For the smaller objects isoelectric line and brace
sleeve screw, the regression-based SSD and YOLOv2
achieve some effects though they are not ideal. In
addition, for the bigger objects, the SSD also performs
relatively good and its speed is also very fast.
(3) Fig. 9 shows that the P-R curve of the YOLOv2 is the
worst. It is difficult to achieve a good balance between
the precision and recall. In other words, if the model
wants to improve its false detection rate, then it has to
sacrifice its missed detection rate. In contrast, the P-R
curve of R-FCN which have a better performance.
(4) As seen from Fig. 11, the robustness of the training
model of the region-based DCNNs, Faster RCNN and
R-FCN, are the best. Their convergence speed is also
faster. To the opposite, the loss curve of SSD fluctuated
greatly and YOLOv2 has a low convergence speed.
5. CONCLUSION
Combined with the most advanced DCNNs learning method
technology, the development of advanced detection
methodologies is crucial for the railway infrastructure
maintenance. This paper compared the state-of-the-art and
the most representative DCNNs learning algorithms with
different multi-objective evaluation indexes.
2018 IFAC CTS
June 6-8, 2018. Savona, Italy
104 Wenqiang Liu et al. / IFAC PapersOnLine 51-9 (2018) 98–105
Through the above results analysis, for catenary image
detection, the region-based R-FCN is more suitable for the
detection of catenary support components. However, its
shortcoming is a poor performance in detection of the smaller
objects. A possible further research direction could be to
incorporate the capabilities of SSD and YOLOv2 into an
integrated scheme that make use the structure R-FCN achieve
the desired results. SSD and YOLOv2 could be used as a
RPNs, and the proposed regions would be sent into the R-
FCN to improve the detection effects of the DCNN.
This paper does not study the characteristics of each network
individually. It gives a general comparative result, which
expects that the follow-up research work is able to focus on a
core framework for optimization and improvement, and
ultimately establishing a detection network that is more
suitable for the maintenance of the railway infrastructure.
REFERENCES
Cho, C.J. and Ko, H. (2015). Video-based dynamic stagger
measurement of railway overhead power lines using
rotation-invariant feature matching. IEEE Transactions
on Intelligent Transportation Systems, 16(3), 1294-1304.
Yang, H.M., Liu, Z.G., Han, Y., and Han, Z.W. (2013).
Defective condition detection of insulators in electrified
railway based on feature matching of speeded-up robust
features. Power System Technology, 37(8), 2297-2302.
Zhang, G.N., Liu, Z.G., and Han, Y. (2016). Automatic
recognition for catenary insulators of high-speed railway
based on contourlet transform and Chan–Vese model.
Optik - International Journal for Light and Electron
Optics, 127(1), 215-221.
Han, Ye., Liu, Z.G., Geng, X., and Zhong, J.P. (2017).
Fracture detection of ear pieces in catenary support
devices of high-speed railway based on HOG features
and two-dimensional Gabor transform. Journal of the
China Railway Society, 39(2), 52-57.
Zhang, G.N., Liu, Z.G., Han, Y., and Han, Z.W. (2017). Loss
fault detection for auxiliary catenary wire of high-speed
railway catenary wire holder. Journal of the China
Railway Society, 39(5), 40-46.
Han, Y., Liu, Z.G., Lee, D.J., Zhang, G.N., and Deng, M.
(2016). High-speed railway rod-insulator detection using
segment clustering and deformable part models. In
Image Processing (ICIP), 2016 IEEE International
Conference on, 3852-3856, IEEE.
Li, Q.Y. and Ren, S.W. (2012). A real-time visual inspection
system for discrete surface defects of rail heads. IEEE
Transactions on Instrumentation and Measurement,
61(8), 2189-2199.
Feng, H., Jiang, Z.G., Xie, F.Y., Yang, P., Shi, J., and Chen,
L. (2014). Automatic fastener classification and defect
detection in vision-based railway inspection systems.
IEEE Transactions on Instrumentation and Measurement,
63(4), 877-888.
Faghih-Roohi, S., Hajizadeh, S., Núñez, A., Babuska, R., and
De Schutter, B. (2016, July). Deep convolutional neural
networks for detection of rail surface defects. In Neural
Networks (IJCNN), 2016 International Joint Conference
on, 2584-2589, IEEE.
104
Gibert, X., Patel, V., and Chellappa, R. (2017). Deep
multitask learning for railway track inspection. IEEE
Transactions on Intelligent Transportation Systems,
18(1), 153-164.
Chen, D.J., Zhang, W.S., & Yang, Y. (2017). Detection and
recognition of high-speed railway catenary locator based
on Deep Learning. Journal of University of Science and
Technology of China, 47(4), 320-327.
Jamshidi, A., Faghih-Roohi, S., Hajizadeh, S., Núñez, A.,
Babuska, R., Dollevoet, R., Li, Z., and De Schutter, B.
(2017). A big data analysis approach for rail failure risk
assessment. Risk Analysis, 37(8), 1495-1507.
Liu, Z.G., Wang, L.Y., Yang, C.J. and Han, Z.W. (2017). A
high-precision loose strands diagnosis approach for
isoelectric line in high-speed railway. IEEE Transactions
on Industrial Informatics, DOI
10.1109/TII.2017.2774242.
Chen, J.W., Liu, Z.G., Wang, H.R., Nunez, A. and Han, Z.W.
(2018). Automatic defect detection of fasteners on the
catenary support device using deep convolutional neural
network. IEEE Transactions on Instrumentation and
Measurement, 67(2), 257-269.
Psuj, G. (2018). Multi-sensor data integration using deep
learning for characterization of defects in steel elements.
Sensors, 18(2), 292.
Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014).
Rich feature hierarchies for accurate object detection and
semantic segmentation. In Proceedings of the IEEE
conference on computer vision and pattern recognition,
580-587.
Girshick, R. (2015). Fast R-CNN. In Proceedings of the
IEEE international conference on computer vision,
1440-1448.
Ren, S.Q., He, K.M., Girshick, R., and Sun, J. (2015). Faster
R-CNN: Towards real-time object detection with region
proposal networks. In Advances in neural information
processing systems, 91-99.
Dai, J.F., Li, Y., He, K.M., and Sun, J. (2016). R-FCN:
Object detection via region-based fully convolutional
networks. In Advances in neural information processing
systems, 379-387.
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu,
C.Y., and Berg, A.C. (2016, October). SSD: Single shot
multibox detector. In European conference on computer
vision, 21-37, Springer, Cham.
Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016).
You only look once: Unified, real-time object detection.
In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, 779-788.
Redmon, J. and Farhadi, A. (2016). YOLO9000: better, faster,
stronger. arXiv preprint arXiv:1612.08242.
Jia, Y.Q., Shelhamer, E., Donahue, J., Karayev, S., Long, J.,
Girshick, R., Guadarrama, S., and Darrell, T. (2014,
November). Caffe: Convolutional architecture for fast
feature embedding. In Proceedings of the 22nd ACM
international conference on Multimedia, 675-678, ACM.
 2018 IFAC CTS
June 6-8, 2018. Savona, Italy Wenqiang Liu et al. / IFAC PapersOnLine 51-9 (2018) 98–105 105

Appendix A

Table 2. Defect type and level of the components of catenary

support device

Part name Defect type Defect level

Cracking B
○
1 Insulator
Flashover B
Cracking A
○
2 Rotary double-ear Cotter pin
A
losing
○
3 Binaural sleeve Cracking A
○
4 Brace sleeve Cracking A
Cracking A
○
5 Steady arm base
Nut looseness A
Cracking A
○
6 Bracing wire hook
Nut looseness A
Double sleeve Cracking A
○
7
connector Nut looseness A
Cracking A
Opposite
C
○
8 Messenger wire base direction
Balance line
C
losing
Cracking A
○
9 Windproof wire ring
Nut looseness B
Cotter pin
○
10 Insulator base
losing
B

○
11 Isoelectric line Line looseness C
○
12 Brace sleeve screw Nut looseness B
Note: Isoelectric line is the joint area between steady arm base and
registration arm, and brace sleeve screw is subarea of the brace sleeve.

105