Professional Documents
Culture Documents
Automatic Inspection of Railway Carbon Strips Based On Multi-Modal Visual Information
Automatic Inspection of Railway Carbon Strips Based On Multi-Modal Visual Information
Abstract— Electric rail vehicles are driven by the current [3] the wear of the contact strip is predicted by monitoring
collected from the electrical lines of the railway catenary the running distance of the pantograph as well as the DC
system. For this purpose, trains are equipped with a current component of the locomotive current. In [4] and [5] the
collector (the pantograph) that comes in contact with the wires
by means of a pair of carbon strips. The sliding movement authors apply image processing methods and analyse the
along the wires subjects the carbon strips to wear and damage interaction points between the pantograph and the catenary
so that frequent inspection is essential to ensure the train and to detect defects and anomalies for fault diagnosis. In [6] a
catenary safety. In this paper we describe an automatic visual triangulation laser is employed to scan the strips and obtain
inspection system, made of a 3D laser scanner and a 2D high a depth profile to compare to a reference pattern, in order
resolution camera, which is able to automatically determine
the health state of the railway contact strips thanks to a to compute the maximum wear. In [7] an RGB camera is
sophisticated data processing tool. The system collects color used and the authors exploit the color histograms and other
and geometrics information on the carbon strips and performs statistical parameters extracted from the image and give them
several automated assessments both on the 3D and 2D data. For to a statistical classifier in order to classify the type of
each test, the system provides an index and decides between damage. In [8] the authors make use of the ensemble of
three different levels of wear (low, medium, high) to support
the human operator in inspection and maintenance decisions. a lateral and a frontal 2D camera. They carry out image
Experimental results reveal the effectiveness of the system, segmentation and morphological operations to individuate
comparing the human judgment with the automated analysis. worn areas of the contact strips. [9] is an attempt to improve
the work in [6] in terms of normalization of the measures
I. INTRODUCTION
with respect to a global frame. At last, in [10] the authors
Electric rail vehicles are actuated thanks to a system of aim to detect defects through the Wavelet Transform of the
overhead contact lines from which the current is collected depth profiles.
by means of a specific collector positioned on the roof, Each of the previously mentioned works seems to face a
the pantograph. The pantograph comes in contact with the single aspect of the wear. The work in [6] focuses on the
electrical lines through a pair of carbon contact strips which strip wear in terms of thickness, in [7] they attempt to
continuously slide along the wires during the train travel. localize material defects by recurring to statistical classifiers
To maintain a stable contact, a pneumatic system provides a (which always need a consistent training data set), while
contact force in the range of [80 − 200]N which increases in [10] the authors focus only on crackpoints detection.
with the train speed [1]. Hence, since the strips are subject Furthermore, the use of mono-modal data limits the number
to high friction and mechanical shocks, they wear out and/or of possible defects that can be detected, since, for example,
get damaged over time [2]. In fact, the contact strips tend a 2D image is not as suitable as a 3D scan to establish the
to shrink, splinter and lose material. Damaged contact strips thickness of the strip or, on the contrary, a 2D image is
can shatter the electrical transmission and increase the risk of more suitable for detecting extended regions of wear from
catenary rupture. Therefore, frequent inspection is essential the cromatic contrasts. Differently than the previous works,
for the correct operation of the system. Railway companies our work wants to deal with all the aspects that concern
periodically carry out manual inspection trough human op- the damage and wear of the carbon strips, exploiting multi-
erators. Nevertheless, manual inspection operations can be modal data (2D and 3D), from which we are able to extract
expensive in terms of time and costs. Indeed, the train has a lot of information and provide details for each type of
to be directed in opportune maintenance sheds and travels damage/defect. The measurement system consists of a highly
are interrupted for the whole inspection time. Furthermore, accurate 3D triangulation laser scanner and a high resolution
human inspection is based on subjective considerations and 2D camera. The data provided are input to a sophisticated
manual measurements can be subject to human error and processing tool which provides several information about the
result inaccurate or neglect important details that can be strip wear condition.
pointed out by deeper computational analysis. The aim of our automatic visual monitoring system is to
There are some previous works that face the problem of provide the results of each applied analysis and accordingly
carbon-strip inspection and monitoring. Some of them make decide a level of wear of the contact strips, supporting the
use of visual sensors, while others employ other tools. In human operators in the train monitoring and maintenance
*PERCeptual RObotics Laboratory, Scuola Superiore Sant’Anna process. In fact, in this way useless manual inspections can
** Trenitalia S.p.A. be avoided and, on the other hand, dangerously damaged
978-1-5090-6000-9/17/$31.00 ©2017 IEEE 178
pieces can be replaced once the system has detected a high
level of wear. The human operators are therefore continu-
ously supported by an automated monitoring and inspection
tool. The present work belongs to a researh project that is
being carried out between Scuola Superiore Sant?Anna and
Trenitalia S.p.A. cooperation and plans to develop a full set
of wayside inspection tools to monitor the health status of
trains.
The remaining of the paper is structured as follows: in
section II we illustrate the measurement setup, in section
III we explain the different processing and analysis applied
on the scanned points, in section IV we explain the different
processing and analysis applied on the 2D images, in section
V we show the experimental results and in section VI we
discuss the conclusions and future work.
0.5
0.1 0.4
possible defects. y 0
0
0.02
0.01
0.1
0.7
0.8
0.5
0.1
0.4
strips. The first one is taken from a new, unworn carbon strip, 0.05
0.2
x
0.3
0.8
the second one is the scan of a slightly damaged, but thinned 0 0.1
y
0 0.7
0.6
strip, whereas the third one belongs to a deeply worn and 0.02
0.01
0 0.5
z
0.4
0.02
179
A. Thickness
The obtained point cloud allows to establish the local where STD indicates the standard deviation of the thickness
thickness (T (p) = z(x, y)) of every point on the carbon applied in a local area of the carbon strip.
strip. The first information we want to obtain is the average B. Thickness Derivative
thickness, since, for a perfect functionning of the electrical
The smoothness indicator is important but not sufficient to
transmission, it must be over a specific threshold value.
detect relevant issues in carbon strip profiles. An important
For this purpose we take the depth z profiles of the strip
aspect that concerns the damage of carbon strips is the
and compute the mean of them. The resulting mean profile
presence of crackpoints and surface discontinuities, caused
represents the average thickness of the strip:
by material loss and abrupt impacts. One way to individuate
such points is by differentiation and thresholding. If we
T avg = mean T (p) (2) denote with cd = (xd , zd ) the crackpoints, td the threshold
p∈strip
value and δ an arbitrary small value, we compute the set
Figure (5) shows the average profile obtained from the
scan of the slightly worn, but thinned test strip shown in
n o
> td and d2 z2 ≤ δ
dz
C d = (x, z) s.t dx dx
(7)
figure (4) and the reference profile obtained from the scan of
the new, unworn strip. The figure points out the gap between If the test profile derivative exceeds the threshold ampli-
the reference and the test profile. tude, we are in the presence of a crackpoint. Figure (6) shows
an example test profile obtained from the scan of the deeply
0.016
Average Thickness
Reference profile
worn and cracked strip shown in figure (4) and the individ-
Test profile
0.014
uation of crackpoints through the differentiation method. In
0.012
the end we compute the number of total crackpoints N cd
0.01
resulting from each profile of the strip:
X
z[m]
0.008
N cd = cd (8)
0.006
i
0.004
0.002
10
Crackpoint
Fig. 5. Comparison of test mean profile (in blue) thickness with reference
profile (in red) thickness. 6
z [m], dz/dx
Threshold
-2
T max = max T (p) (4) Fig. 6. Individuation of crackpoints through differentiation of the depth
p∈strip
profile. The figure shows the test z profile (in blue), its derivative (in green),
These values are commonly thresholded to detect the the threshold amplitude lines (magenta dashed lines) and the black circles
are the individuated crackpoints.
overcome of limits.
Another relevant index is the symmetry index, which aims C. Frequency Spectrum
at detecting smooth irregularities among the left/right area Another strong mathematical tool that can help us to
of the carbon strip and can be a symptom af an irregular investigate the spatial trend of the carbon strip is the spatial
pressure or misalignement of the pantograph. If we set a Discrete Fourier Transform (sDFT). The sDFT allows us to
coordinate x = 0 in the center of the carbon strip, we have: compute the frequency content of the single depth profiles.
Z Intuitively, a rippled profile presents high frequency com-
T sym = xT (x, y)dxdy (5) ponents, whereas in a homogeneous profile low frequencies
x,y prevail. In addition to derivative methods the sDFT allows
to compute cracks unregarding to their position in space,
Lastly we consider the smoothness index as an indicator of
and benefits from the integral formulation to reduce possible
the smoothness of the wearing as a local/punctual variation
errors due to noise components of the meaure (to which
among the lateral geometry:
the derivative function is instead
very sensitive). Once the
amplitude spectrum P (f ) of the test profile has been
T smt (r) = max( STD (T (x, y))) (6)
x xi∈[x−r,x+r] obtained, we compute
180
Autocorrelation Functions
1
Z Test profile autocorrelation coefficients
Reference profile autocorrelation coefficients
af = P (f ) df (9) 0.8
f >tf
0.6
Autocorrelation Coefficients
0.4
np -0.2
X 0 5 10 15 20 25 30 35 40
Lag
Af = af (10)
i=1
Fig. 8. Normalized autocorrelation coefficients: coefficients of the refer-
Figure (7) shows the amplitude spectrum of the previous ence profile (in red) and coefficients of the test profile (in blue). The test
example profile compared to the reference pattern and the profile coefficients suggest the presence of discontinuities in the profile.
coordinate of the threshold frequency.
12
Test profile spectrum
Crosscorrelation Function
1
Threshold frequency
2
0.8
0 10 20 30 40 50 60 70 80 90 100
-1
f [m ]
0.6
Crosscorrelation Coefficients
0.4
2.5
z [m]
0.006
0.004
Crackpoint
0.002 2
0
0 100 200 300 400 500 600 700
sample point
4 Threshold
2 1
-2
-4
0.5
-6
-8
-10
0 100 200 300 400 500 600 700 0
sample point
0 50 100 150 200 250
Fig. 10. Individuation of crackpoints through Wavelet Decomposition. Fig. 12. Image histogram. In red: New, unworn strip image histogram. In
Top: z profile and individuated crackpoints circled in black. Bottom: 8-th blue: Worn strip image histogram.
level WT and threshold amplitude (magenta dashed line)
B. Cromatic Derivative
IV. PROCESSING OF THE 2D IMAGES
We assume that material defects such as cracks and chip-
Figure (11) shows three example 2D images. As we can pings are defined by cromatic discontinuities. These points
see, the first one shows a new, unworn carbon strip, the can be individuated as for the depth profile by differentiation
second one is the image of a slightly damaged, but thinned of the single image rows and by thresholding of the resulting
strip, whereas the third one shows a deeply damaged and function. If i is the generic row coordinate and j the generic
thinned sample. The data processing and analysis tests that column coordinate of the 2D image, for each row we find
we carry out are described in the following sections. the set
n 2 o
P c = pi,j s.t dI
d I
dj > tI and dj 2 ≤ δ (15)
100
I, dI/dj
Fig. 11. Example images of carbon strips. Top image: Image of a new
strip. Middle image: Image of a slightly damaged, but thinned strip. Bottom 50
Crackpoint
0
200 300 400 500 600 700 800 900 1000
A. Image Histogram j
183
As we can see from table (I), most of the analysis results [3] S. Ostlund, A. Gustafsson, L. Buhrkall, and M. Skoglund, “Condition
agree among them and with the human judgments. The wear monitoring of pantograph contact strip,” 2008.
[4] I. Aydin, M. Karaköse, and E. Akin, “A new contactless fault diagno-
level decisions given by the correlation analysis seem to be sis approach for pantograph-catenary system,” in MECHATRONIKA,
more conservative with respect to the depth profile derivative 2012 15th International Symposium. IEEE, 2012, pp. 1–6.
or Wavelet analysis, as sometimes the first one decide for [5] E. Karakose, M. T. Gencoglu, M. Karakose, I. Aydin, and E. Akin,
“A new experimental approach using image processing based tracking
medium wear, whereas the second ones decide for high for an efficient fault diagnosis in pantograph-catenary systems,” IEEE
wear. In each case the results suggest that the automated Transactions on Industrial Informatics, 2016.
inspection is efficient and can serve as a good support in the [6] L. Jarzebowicz and S. Judek, “3d machine vision system for inspection
of contact strips in railway vehicle current collectors,” in Applied
maintenance process. Electronics (AE), 2014 International Conference on. IEEE, 2014,
pp. 139–144.
[7] M. Sacchi, S. Cagnoni, D. Spagnolettix, L. Ascariz, G. Zunino, and
A. Piazzi, “Pavisys: A computer vision system for the inspection
of locomotive pantographs,” Proc. Pantograph Catenary Interaction
Framework for Intelligent Control, 2011.
[8] L. G. Hamey, T. Watkins, and S. W. T. Yen, “Pancam: in-service
inspection of locomotive pantographs,” in Digital Image Computing
Techniques and Applications, 9th Biennial Conference of the Aus-
tralian Pattern Recognition Society on. IEEE, 2007, pp. 493–499.
[9] S. Judek and L. Jarzebowicz, “Algorithm for automatic wear estima-
tion of railway contact strips based on 3d scanning results,” in Elec-
trical and Power Engineering (EPE), 2014 International Conference
and Exposition on. IEEE, 2014, pp. 724–729.
[10] ——, “Wavelet transform-based approach to defect identification in
railway carbon contact strips,” Elektronika ir Elektrotechnika, vol. 21,
no. 6, pp. 29–33, 2015.
[11] E. Di Stefano, E. Ruffaldi, and C. A. Avizzano, “Automatic 2d-3d
vision based assessment of the attitude of a train pantograph,” in Smart
Cities Conference (ISC2), 2016 IEEE International. IEEE, 2016, pp.
1–5.
[12] P. F. Dunn, Measurement and data analysis for engineering and
science. CRC press, 2014.
Fig. 16. Experimental setup: a pantograph bow mounted on an actuated [13] Y. Meyer, Wavelets and operators. Cambridge university press, 1995,
slider and surmounted by the capturing system. vol. 1.
[14] D. Shapira, S. Avidan, and Y. Hel-Or, “Multiple histogram matching,”
in Image Processing (ICIP), 2013 20th IEEE International Conference
on. IEEE, 2013, pp. 2269–2273.
VI. CONCLUSIONS [15] J. Canny, “A computational approach to edge detection,” IEEE Trans-
actions on pattern analysis and machine intelligence, no. 6, pp. 679–
The paper presents an automatic visual inspection system 698, 1986.
that is able to determine the state of health of railway contact [16] E. R. Dougherty, “An introduction to morphological image process-
strips autonomously. The automated system is conceived to ing,” Tutorial texts in optical engineering, 1992.
[17] H. Samet and M. Tamminen, “Efficient component labeling of images
support human operators in maintenance decisions and in the of arbitrary dimension represented by linear bintrees,” IEEE Transac-
monitoring of the strips wear conditions. Several assessments tions on Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp.
are carried out both on 2D and 3D data, obtained respectively 579–586, 1988.
[18] R. M. Haralick and L. G. Shapiro, “Image segmentation techniques,”
from a 3D tringulation laser scanner and a 2D high resolution Computer vision, graphics, and image processing, vol. 29, no. 1, pp.
camera. The system provides a wear level decision for each 100–132, 1985.
applied analysis, choosing between low, medium and high
wear, on the basis of a comparison of the analysis result with
three different reference values. Tests were performed on a
set of samples characterized by different types of defects and
the wear level decisions produced by the various analysis
were compared to each other and to the human judgment.
The results obtained show the effectiveness of the tool.
The module described in this paper will be integrated in a
more complex capturing system installed in a portal dedi-
cated to the visual inspection of trains and their condition-
based maintenance.
R EFERENCES
[1] Commission-Decision, “Technical specification for interoperability
relating to the energy subsystem of the trans-european high-speed rail
system,” Official Journal of the European Communities, vol. L245,
pp. 280–369, 2002.
[2] G. Poetsch, J. EVANS, R. Meisinger, W. Kortüm, W. Baldauf, A. Veitl,
and J. Wallaschek, “Pantograph/catenary dynamics and control,” Ve-
hicle System Dynamics, vol. 28, no. 2-3, pp. 159–195, 1997.
184