See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/252022440

Real-time image processing for edge inspection and defect detection in

stainless steel production lines

Article · May 2011

DOI: 10.1109/IST.2011.5962196

CITATIONS READS

6 1,233

5 authors, including:

Carlos G. Spinola Juan Miguel Cañero-Nieto

University of Malaga University of Malaga
29 PUBLICATIONS   52 CITATIONS    14 PUBLICATIONS   23 CITATIONS   

SEE PROFILE SEE PROFILE

J. M. Bonelo M. J. Martín-Vázquez
Universidad de Cádiz University of Malaga
27 PUBLICATIONS   53 CITATIONS    23 PUBLICATIONS   181 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Automation quality control in stainless steel production lines View project

All content following this page was uploaded by Carlos G. Spinola on 28 May 2014.

The user has requested enhancement of the downloaded file.

 Real-Time Image Processing for Edge Inspection and
Defect Detection in Stainless Steel Production Lines
Carlos G. Spinola1 (IEEE Member), Juan Canero1,2, Gonzalo Moreno-Aranda1,2, Jose M.Bonelo3 (IEEE Member),
Manuel. Martin-Vazquez1,
*1 Department of Electronics, Malaga University,
29071 Malaga, SPAIN, Phone: +34 952131388, Fax: +34 952132781, e-mail: cspinola@uma.es
*2 T.C.C., S.A.,
C/Robinson Crusoe, 5 Loc-2. 29006 Malaga, SPAIN, Phone: +34 952335999, e-mail: ofic@tcc.e.telefonica.net
*3 ACERINOX, S.A. Fábrica del Campo de Gibraltar.
11370 Los Barrios. (Cádiz), SPAIN, Phone: +34 956 629 305, Fax: +34 956 629 306, e-mail: josemaria.bonelo@acerinox.com

Abstract— An image acquisition and processing system to
measure the width and inspect the quality of the stainless steel
strip in a production line is presented in this paper. It is based on
the real-time processing of the images acquired by a twin linear
camera system. Image processing algorithms to detect defects
and anomalies in the edges have been implemented. The core of
the image processing is submitted to the GPU of the graphic card
to reduce the total processing time. A system like this has been
proved and installed in a stainless steel production line for
quality control purposes.
Image processing; edge line search; edge defect detection;
GPU, CUDA, graphic card processing; quality control.
I. INTRODUCTION
Stainless steel is a material of high added value and
increasing importance in our world. It is initially produced in
the melting shop, and in the subsequent hot rolling mill it is
usually shaped as strip coils several hundred meter long and
different width and thickness (Fig. 1). The strip width and
thickness must be monitored as the dimensions are important
features to fulfill the customer orders [1]. The strip surface and
edge must also be accurately inspected to detect any kind of
anomalies in the edges for quality control purposes and to
avoid troubles in subsequent processing lines, especially in
cold rolling mill, where damages could be produced in the very
smooth rolls and even breakage of the strip could happen.
This project was conceived to simultaneously fulfill both
goals, namely continuous width strip measuring and edge
quality inspection for all coils processed in a production line.
Previous works have presented real time inspection system
based on image processing in industrial environment using area
cameras [2][3] and linear cameras [4]-[8]. So it was decided to
design a width measuring industrial system based on image
processing that could also afford edge inspection and defect
detection such as cracks, scratches, strike marks, etc. As we
explain later, due to the precision required, the range of values
to measure and the peculiarity of the available places to
position the system, a twin linear camera system was used.
The description of the system, the basic model of the
measuring task and the inspection algorithms implemented are
presented in this paper. The need to accomplish all these
objectives in real time and in an industrial environment is very
demanding, so the initial algorithms developed to be processed
in a standard CPU were modified and adapted to be processed
in the GPU (Graphic Processor Unit) of the graphic card. A
dramatic reduction in the processing time was achieved using
its ability to parallel processing, and making room for
additional and more detailed inspection in the future. This can
be considered an important feature of the presented system.
This device has been installed, tested and it is in operation
in the Acerinox factory in Spain, one of the major producers of
stainless steel in Europe.

II. DESCRIPTION OF THE SYSTEM

The proposed system consists of a couple of cameras which
simultaneously inspect the two coil edges all over its length
(Fig. 2).
To achieve the required accuracy the cameras must be
placed where the coil sheet is very stable, without any
vibrations or displacements. It is not easy to find such a place
in a production line, unless in a roll applying tension, where the
strip is firmly adjusted to its surface. A drawback of this
location is the increased distortion introduced in the images. To
avoid this effect along the vertical direction of the images,
Fig. 1. Stainless steel coils.

Fig. 2. Width measurement system.
linear cameras has been used. This has the advantage of an
easier illumination system and the ability to take long pictures
without vertical distortion. To trigger the camera the pulses
from an encoder fixed in the roll shaft are used.
The acquired images are taken with two 2048 pixel CCD
scan line cameras covering each one 400 mm so that the ratio
pixels/mm is 5 and the accuracy of a millimeter could be
reached. The vertical size has been selected as 500 mm in
length in order to have two width values in each meter of the
coil. The distance between each camera and the strip surface is
approximately 750 mm and the focal length of the lenses used
for each camera is 50 mm. The size of each camera sensor unit
is 28.67 mm and the camera lens F-number is set to 16, so the
depth of field obtained is approximately 94.4 mm. Detailed
information about the geometry of system can be found in [8].
One purpose of this system is measuring the width of the
coils along all its length, typically several hundred meters long.
But it is also necessary to allocate each measurement to a
position along the length of the coil.
The continuous process of coils in a production line,
involves welding the end of a coil with the beginning of the
following one. Therefore the detection of the welding line is
necessary to initialize the length count automatically.
Other purpose of the system is detecting edge anomalies to
alert control quality staff and to store them in the data
warehouse for supervision.
The image processing was afforded considering the real
time restriction and trying to use the minimum processing time
in a standard CPU.
III. WELDING LINE DETECTION
Welding lines appear in the images as a horizontal pattern
(Fig. 3) all along the width. Its detection is performed using a
simplified and specialized version of the classic approach of
line detection as described:
a) Selection of the inner area of the image (Fig. 3) as the
ROI (region of interest). This is the easiest area in which
Fig. 3. Welding line between coils and ROI.
to find the welding line and therefore the computation
cost is reduced.
b) Horizontal edge detection. A Sobel filter [9] is applied
using a simplified mask considering only the horizontal
gradient. The resulting gradients are binarized in two
different matrix depending on their positive and negative
values, so that the increase and decrease of luminance are
kept. The luminance threshold level, Ith, is calculated for
each image using its luminance average Iavg and its
standard deviation σ and it is calculated as in (1).
I th = I avg + 3σ (1)
c) A simplified Hough Transform [10] is applied looking for
horizontal lines in the positive and negative edge arrays.
As only lines with 0º angles are looked for, the 2D
parameter space is reduced to a 1D vector making the
algorithm more efficient. Two vectors are obtained each
one with the number of votes of horizontal lines with
negative and positive gradient.
d) Analysis of vectors. A welding line is detected when there
are positive and negative lines at a distance less or equal
the typical welding line mark. The threshold for the
minimum number of votes of a line is selected according
to the width of the selected ROI.
IV. COIL EDGE DETECTION AND WIDTH MEASUREMENT
The width of the coils is constant enough to be sufficient to
give one width value for each meter along the coil. That makes
easier the real time representation and the data warehousing for
quality control traceability. Therefore an image processing
algorithm was designed to calculate an overall width for each
couple of images (Fig. 4).
(a) (b)
Fig. 4. Images of the coil edge: (a) left and (b) right coil edge image.
 First of all the vertical line that best fits the overall position
of the coil edge in the image is looked for. The detection
approach makes use of the different luminance of the roll and
steel surface. The algorithm can be summarized in the
following steps:
1. Each image array I(x,y) is converted in one vector V(y)
each element being the average luminance of each column
in the image. To speed up this operation the decimation of
the rows is done.
2. The S(y) vector is the gradient of V(y) calculated as the
slope of V(y).
3. The S(y) vector is low-pass filtered in Sf(y).
4. The coil edge is placed in the pixel with maximum value
in Sf(y).
The edge line positions dreal1 and dreal2 regarding the optical
center of the image are determined in each twin acquired
images. The distance k between the two cameras is known
from the system calibration; therefore a representative value of
the coil width dwd in this pair of images can be calculated as in
(2).
d wd = d real1 + d real 2 + k
An ultrasonic sensor installed between the two cameras
informs about the sheet thickness, which is necessary to
calculate the width.
The detailed geometrical model that considers the different
thickness of the coils, possible displacement from the central
axis and the correction implemented to assure the required
accuracy is presented in [7] and [8].
V. EDGE DEFECT DETECTION
A common feature of most edge defects is that they affect
the straightness of the coil edge as being caused by a hit or
strike (Fig. 5.a) and this fact is used to detect such defects in
the image.
The defect detection algorithm is complementary with the
edge detection and it is processed once the position of the coil
edge in the image has been determined. The approach consists
on analyzing in more detail a narrow region around the found
edge. The steps of the algorithm are described below:
a) Selecting the ROI centered around the coil edge (Fig. 5.a).
b) This ROI is binarized separating roll from steel pixels, in
order to improve the robustness of the edge detection
algorithm when working with small regions.
c) Afterwards, the ROI is divided in n-windows along the
edge high enough to achieve the required accuracy but
maintaining a low computational cost in order to allow
real time processing. In this application windows of
50x600 pixels (10 x 120 mm) have been selected.
d) Each binarized window i is processed in similar way
described earlier to detect the edge position in it. After
completing this step a vector E(n) is obtained where
(a)
(b)
Fig. 5. (a) ROI and window i over acquired image (image left-
rotated 90º); (b) Edge vector E(n).
element E(i) contains the relative position of the local
(2) edge in the i-th window with regard to the global edge
line in the image (Fig. 5.b).
e) Finally, E(n) is processed to detect abrupt variations in the
local edge position. See positions 25 and 42 in Fig. 5.b.
VI. GPU IMPLEMENTATION
So far it has been considered that the image processing runs
in a standard CPU, which could be placed in the same
production line. But the number of images acquired and
processed depends on the line speed and the image processing
use almost all the available time. A way to solve this problem
is to use the computational power of the GPU of present
graphic cards, with its parallel processing capability, using its
SIMD (Single Instruction stream Multiple Data) architecture
[11][12][13].
There are many high level libraries to program the GPU.
But most of them are oriented to very specific graphic
programming applications and harder to use being not
applicable to a general purpose application like ours.
CUDA (Compute Unified Device Architecture) is a general
parallel programming framework developed by NVIDIA
[14][15] to be used in general purpose C language applications
[16] to take advantage of the parallel processing capability of
the graphic card. CUDA introduce the concept of threads or
elemental processors that can be grouped in 1, 2 or 3
dimensional grid that runs the same operator over a data
matrix. These simple abstractions allow some independence of
the details of the graphic card model and shorten the
developing time.
VII. GPU IMAGE PROCESSING
When adapting the described algorithms to the GPU, it is
possible to avoid some of the restrictions introduced earlier to
reduce the execution time, as decimation and ROI selection.
Therefore coil edge detection, width measurement, and defect
detection, described as sequential tasks in IV and V paragraph
can be combined.
The image processing tasks explained in paragraphs III, IV and
V can be submitted to GPU as described in the following steps.
1. Memory allocation and transfer. GPU Memory allocation
and transfer of the image matrix to it.
2. Welding Line Detection. This step is essentially identical
to the one described in III. The GPU implementation of
classical algorithm as Sobel filter is developed in a similar
way as that described in references [17][18].
3. Averaging the luminance of image columns. The image is
logically divided in N windows of the same height. A
thread is devoted to each column and the 1xw grid of
threads, defined as shown in Fig. 6 (step 3), processes
each row in parallel. Each time the average of a window
is completed the result is stored. Finally, a matrix M(n,w)
is obtained being each row the average luminance of the
columns of the i-window.
4. Filtering and gradient calculation of each row of the
M(n.w) matrix. Each thread is assigned to an element
whose result depends on the contiguous ones in the same
row. The 1xw grid of threads, defined as shown in Fig. 6
(step 4) processes each row in parallel. The result is
stored in S(n,w) matrix.
5. Getting the position of local coil edge. A thread is
devoted to process each row. The nx1 grid of threads
defined in this step processes each column in parallel.
After completing it, a vector E(n) is obtained whose
element E(i) stores the position of the local edge in the i-
th window Fig. 6 (step 5).
Vector E(n) is the final result of the GPU processing and it
is transfer to the CPU where the rest of the processing is done
because there are only a few elements and no much parallelism
is involved. The mean value of E(n) can be considered as the
coil edge location in the image dreal1 or dreal2, depending on the
side of the image. These values are used to calculate the coil
width, as in (2). The analysis of vector E(n) to detect edge
defects is the same explained in V.
Fig. 6. Grids of threads.
VIII. PERFORMANCE IMPROVEMENTS
It has been compared the processing time of the presented
algorithms applied to 2048x2048 images when implemented in
CPU and GPU. It has also been considered the transfer time of
the image to the GPU which is a time consuming operation.
The hardware use as the test-bench was:
• CPU: Intel Pentium Dual Core E5200 2.52GHz, 1GB
RAM.
• GPU: NVIDIA GeForce GTX 285, 1 GB Global
Memory. 240 CUDA cores. Compute capability 1.3.
TABLE I shows the processing times obtained. The
speeding up got in weld line detection is 10x and in edge
inspection 14x, being the overall 9x.
TABLE I. COMPARISON OF TASK PROCESSING TIMES.
Time (ms)
Task
CPU GPU Speed-Up
Load Image - 3.75 -
Weld Line Detection 78 7.81 10x
Edge Detection 63 4.40 14x
TOTAL 141 15.96 9x
There is also additional CPU miscellaneous processing for
each image apart from the E(n) vector analysis, such as data
base accessing, results storing, image compression, etc. Taking
advantage that there are two processors in the system, it is
possible to achieve an extra processing time reduction if the
tasks are done in parallel rather than sequentially. The GPU
processing time Tg and the CPU miscellaneous processing time
Tp can be overlapped as shown in Fig. 7. In each period, the
processing of the i-image is done in the GPU while the CPU
performs the miscellaneous tasks corresponding to the previous
image i-1. It is supposed that there is always a new available
image at the beginning of each period.
Fig. 7. Time improvement using overlapped tasks..
IX. SYSTEM IMPLEMENTATION
The schematic layout of the placement of cameras and
lights over the roll is presented in Fig. 8. The line covered by
each camera is 400 mm and so it is possible to measure and
inspect the edge of coils between 900 and 1600 mm wide.
The cameras are triggered by the pulse train generated by
an incremental encoder fixed in the roll shaft. The speed of the
line can vary between 20 and 60 m/min depending on the
 TABLE II. COMPARISON OF MANUAL AND AUTOMATIC MEASUREMENTS.
STATISTICAL RESULTS OF ITS DIFFERENCE.

Width* Thickness* n Mean* Standard Deviation*

[2,3] 995 -0.61 1.73
900-1000
[3,4] 318 0.38 2.02
[2,3] 2637 -0.94 1.70
[3,4] 1510 -1.06 2.33
1000-1200 [4,5] 801 -0.86 2.03
[5,6] 312 -0.71 1.91
[6,7] 146 -0.84 1.71
[2,3] 2912 -0.41 1.79
[3,4] 1866 -0.61 1.89
1200-1400
[4,5] 1024 -0.43 1.78
[5,6] 349 -0.49 2.01
[2,3] 1376 0.17 1.57
[3,4] 1461 0.12 1.94
Fig. 8. System Architecture. [4,5] 1315 0.09 1.92
[5,6] 947 -0.15 1.83
operation conditions and coil width. The acquired images are 1400-1600
sent via a gigabit Ethernet link to an industrial embedded [6,7] 272 -0.18 1.50
computer where the edge detection and inspection processing is [7,8] 47 0.17 1.83
performed. This computer also receives the ultrasonic sensor [8,9] 250 -0.08 1.81
signal via RS-422 to correct the apparent position of the edge [9,10] 18 -0.61 1.72
according to the geometrical model [8].
*Units are in millimeters
The processing computer is connected to the factory LAN
so that the processing results are presented in real time to the An important benefit of this system is that it allows
quality control operators and edge images can be reviewed and inspection of the coil edges as well. Fig. 10 shows some
even stored for back office supervision. defects automatically detected by the system.
At this moment only general defect detection has been
X. RESULTS AND CONCLUSIONS afforded, but when an enough population of images of different
The system presented in this paper has shown to be able to defects had been collected, an automatic classification of them
measure the width of coils in a production line and to inspect could be carried out.
continuously the coil edges in real time.
A graph of the width of a coil 589 meter in length is shown
in Fig. 9, where the nominal width and tolerance limits are also
represented. It is noticeable the head and tail where the width is
out of the tolerance limits and can be considered unacceptable.
Continuous width measurements of each coil and images of
detected defects are stored for quality control audit purposes.
Automatically acquired measurements are routinely
compared with measurement manually taken by quality control
operators. Results are presented in TABLE II sorted by range
of width and thickness. The n value is the number of samples
taken for each width and thickness range. Statistics values of
mean difference and standard deviation of the difference
between manual and automatic measurements are shown. As it
can be seen quite good results have been achieved.

Fig. 10. Some detected coil edge defects.

Fig. 9. Measured width of a complete stainless steel coil.
 ACKOWLEDGEMENT
We would like to thank Acerinox management for
supporting this investigation and the Information Systems and
Quality Control departments for its help and essential
suggestions offered in this research.
REFERENCES
[1] B. Jha, P. Jha and C. D. Singh. “Process Technology for the Continuous
hot band annealing of 17% Cr ferritic stainless steel,” J. of Material Eng.
and performance, pp. 11, 2002.
[2] C. Fernández, C. Platero, P. Campoy and R. Aracil, “Vision system for
on-line surface inspection in aluminium casting process”, in Int. Conf.
on Ind. Electron., Control and Instrumentation, Maui, U.S.A., Nov. 15-
19, 1993, vol. 3, pp. 1854-1859.
[3] H. Zhang, G. Wu, X. Sun, J. Xu, and K. Xu, “A New Fast Border Search
Algorithm and Its Expansion,” in IEEE Int. Conf. on Automation and
Logistics, Jinan, China, Aug. 18-21, 2007, pp. 1886-1890.
[4] Y. Fernández, R. C. Gonzalez, P. F. Díaz and J. M. Enguita, “On-line
wavy strip and off-center analysis of high speed production lines by a
linear camera,” in IEEE Conf. on Emerging Technologies and Factory
Automation, Lisbon, Portugal, Sep. 16-19, 2003, vol. 2, pp. 552-559.
[5] Z. Hocenski, T. Keser and A. Baumgartner, “A Simple and Efficient
Method for Ceramic Tile Surface Defects Detection,” in IEEE Int. Symp.
on Ind. Electron., Vigo, Spain, Jun. 4-7, 2007, pp. 1606-1611.
[6] Y. Li, T. Y. Young and C. Huang, “Noncontact Measurement Using
Line-Scan Cameras: Analysis of Positioning Error,” in IEEE Trans. on
Indust. Electron., vol. 36, no. 4, pp. 545-551 Nov., 1989.
[7] C. G. Spínola, J. M. Cañero-Nieto, G. Moreno-Anda, M. J. Martín-
Vázquez, J. M. Bonelo and J. Vizoso, “Sistema de Visión para la
Medida del Ancho de Bobinas de Acero,” en XVII Seminario Anual de
Automática, Electrónica Industrial e Instrumentación (SAAEI’10).
Bilbao, España, 7-9 de Julio, 2010.
View publication stats
[8] C. G. Spínola, J. M. Cañero-Nieto, G. Moreno-Aranda, J. M. Bonelo and
M. J. Matrín-Vázquez, “Continuous Real-Time Optical Measuring of
Strip Width and Edge Inspection in Stainless Steel Production Lines,” in
IEEE Int. Instrumentation and Measurement Technology Conf..
Binjiang, Hangzhou, China, May. 2011.
[9] I. Sobel, “Camera Models and Machine Perception”. PhD Thesis,
Standford University, Standford, CA, U.S.A., 1970.
[10] R. O. Duda and P. E. Hart, “Use of the Hough Transformation to Detect
Lines and Curves in Pictures,” Commun. of the ACM, vol. 15, no. 1, pp.
11-15, Jan. 1972.
[11] I. K. Park, N. Singhal, M. H. Lee, S. Cho and C. Kim, “Design and
Performance Evaluation of Image Processing Algorithms on GPUs,” in
IEEE Trans. On Parallel and Distributed Syst., vol. 22, no. 1, pp. 91-
104, Jun. 2010.
[12] Z. Yang, Y. Zhu and Y. Pu, “Parallel Image Processing Based on
CUDA,” in Int. Conf. on Comput. Sci. and Software Eng., Wuhan, Hube,
pp. 198-201, Dec. 2008..
[13] J. Kim, M. Hwangbo and T. Kanade, “Parallel Algorithms to a Parallel
Hardware: Designing Vision Algorithms for a GPU,” in IEEE 12th Int.
Conf. on Comput. Vision Workshops (ICCV Workshops), Kyoto, Japan,
pp. 862-869, Sep. 2009..
[14] NVIDIA CUDA Programming Guide 3.0, NVIDIA Corporation, Feb.
2010.
[15] S. I. Park, S. P. Ponce, H. Jing, C. Yong and F. Quek, “Low-cost, High-
speed Computer Vision using NVIDIAS’s CUDA Architecture,” in
IEEE 37th Appl. Imaginery Pattern Recognition Workshop (AIPR’08).
Washington DC, U.S.A., pp. 1-7, Oct. 15-17, 2008.
[16] NVIDIA CUDA C Programming Best Practices Guide. CUDA Toolkit
2.3, NVIDIA Corporation, Jul. 2009.
[17] N. Zhang, Y. Chen and J. Wang, “Image Parallel Processing Based on
GPU,” in 2nd Int. Conf. on Advanced Comput. Control (ICACC’10).
Shenyang, China, pp.:367-370, Mar. 27-29, 2010..
[18] J. Fung and S. Mann, “OpenVIDIA: Parallel GPU Computer Vision,” in
Proc. of the 13th Annu, ACM Int. Conf. on Multimedia, Singapore, pp.
849-852, 2005.