Download as pdf or txt
Download as pdf or txt
You are on page 1of 9

Welding International

ISSN: 0950-7116 (Print) 1754-2138 (Online) Journal homepage: https://www.tandfonline.com/loi/twld20

Real-time vision-based control of weld pool size

William Lucas , Dan Bertaso , Geoff Melton , Jeremy Smith & Chris Balfour

To cite this article: William Lucas , Dan Bertaso , Geoff Melton , Jeremy Smith & Chris Balfour
(2012) Real-time vision-based control of weld pool size, Welding International, 26:4, 243-250,
DOI: 10.1080/09507116.2011.581336

To link to this article: https://doi.org/10.1080/09507116.2011.581336

Published online: 05 Oct 2011.

Submit your article to this journal

Article views: 183

View related articles

Citing articles: 3 View citing articles

Full Terms & Conditions of access and use can be found at


https://www.tandfonline.com/action/journalInformation?journalCode=twld20
Welding International
Vol. 26, No. 4, April 2012, 243–250
Selected from Przeglad Spawalnictwa 2009 (1) 11 – 16

Real-time vision-based control of weld pool size


William Lucasa, Dan Bertasoa, Geoff Meltona, Jeremy Smithb and Chris Balfourb
a
TWI, Abington, Cambridge, UK; bLiverpool University, Liverpool, UK

A computer vision technique based on the real-time measurement and control of the upper surface or ‘topface’ weld pool
size is discussed. The primary objective is to demonstrate the feasibility of using vision-based image processing to provide
measurements and subsequent control of a weld’s geometrical properties during the weld formation or molten phase. A novel
reference feature – a correlation-based image analysis algorithm – has been developed, which may be configured to operate
with a number of different welding processes. Upper surface weld pools with regulation via a simple classical feedback
control algorithm have then been demonstrated to show how this approach may be used to improve weld uniformity and
repeatability.

Introduction However, a disadvantage of this solution is that the camera


For arc welding automation, a number of methods were arrangements, arc light stability, as well as weld and native
prepared from utilizing an artificial intelligence method, material’s properties, such as the surface conditions, will
via automated selection of welding parameters, and finally all affect the created images’ quality. In order to deal with
to use control sensors and welding process monitoring, these issues, a series of tests should be performed to ensure
as well as advanced control strategies designed to process that the camera’s arrangement to the welded component is
information from sensors1 – 7. optimal from the point of view of coherence of the
In any sensor-based regulation system, obtaining provided images.
stable measurements is often difficult due to the welding This paper describes how images from the weld face
arc producing both heat and intense light. may be processed in order to perform real-time geometric
Therefore, a universal approach was discussed here measurements. Also, parameters selection and a closed
relying on a computer possessing a high-quality image of loop process system regulation type were presented, which
the upper weld, the aim of which would be to provide uses vision sensor measurements of upper weld pool
real-time, credible measurements of upper welding pool surface and processes them by applying a simple closed
width which may be used in the process control system. loop regulation technique. Additionally, controller par-
The welding control system was designed to regulate ameters of closed loop welding process were presented in
the upper pool’s surface width to obtain more homogenous which weld pool surface measurements are done by using
welds in their changeable arrangement conditions. a traditional control technique.
Possible sources of disturbances in an uncontrolled
welding process include heterogeneous heat absorption
caused by the welding clamp arrangement, differences in Digitization and image processing
base material features, and systematic declination in the Figure 2 shows the diagram of a simplified image
welding process, as well as unstable arc ignition registration and processing. A combined video signal from
parameters. the CCD camera is registered and processed into a digital
A key factor in configuration of a successful vision form by either TV interface or a WinTV card installed in
sensors system is generation of stable, high-quality any PC, usually with the Microsoft Windows 2000
welding process images, which may be analysed by the
operators. Figure 1 shows a simplified diagram of an upper
pool surface sensor arrangement.
In front of the welding clamp and wire feeding
Wire
components, a CCD camera with an IR bandpass filter is feeder
fitted. The filter’s operational duration is similar to the Welding clamp Camera
camera’s wavelength, so most of the arc’s light is blocked.
Also, the camera is arranged at an angle to the welded Video
component’s plane; therefore, the amount of arc light to signal
which the camera’s sensors are exposed is further limited. Weld
Arc
Such a sensor configuration, i.e. limiting the range of the
arc’s light transmission in order to illuminate the observed Filter
weld area, may produce high weld pool quality in the Welded sheet
‘turned-on arc’ state. This eliminates the necessity of using
an additional light source to illuminate the weld pool. Figure 1. Topface sensor arrangement for arc welding.
ISSN 0950-7116 print/ISSN 1754-2138 online
q 2012 Taylor & Francis
http://dx.doi.org/10.1080/09507116.2011.581336
http://www.tandfonline.com
244 W. Lucas et al.

Control computer

Video
Camera signal
PCI card

C++ Software
Digital video signal

Frames
analysis
algorithm
Frames Pool width
processing measured
from the
Processed Required arc side
frames Welder
dimensions
display controller
generation

Figure 2. Image capture and processing.

operating system. This card effectively generates a stream software-generated cross, which is centred in the
of digital video images from the camera. registered weld pool image, as presented in Figure 3(a).
The analysis algorithm was made in Microsoft Visual Subsequent images’ processing starts from a left
Cþ þ to provide real-time, frame-by-frame digital video horizontal calibration position. This allows to exclude the
flow processing by means of the Video for Windows major registered image part from processing, thanks to
(VFW) software8,9. The addition of diagnostics infor- which the algorithm efficiency is significantly improved.
mation to the processed video frames before displaying The second stage of the processing algorithm’s configur-
them on the computer’s screen allows monitoring of the ation phase involves sampling of a pair of characteristic
image analysis algorithm’s operation. Usually, this edges’ features with a single-dimensional matrix of
information will contain cursors indicating that the these features’ image intensity values, as presented in
measured weld pool edges’ arrangements before the Figure 3(c), (d). The cross on the left side in Figure 3(b)
processed frame appear. Thus, a visual presentation of indicates the centre of the weld pool’s left edge’s features,
the processing algorithm’s operations may be created. and the two vertical belts on each side of the cross show the
The final stage of the camera processing cycle is the limits of these features.
generation of the weld pool’s width measurements. The Similarly, the cross and vertical belts on the right side
generated measurements may then be sent to a process of Figure 3(b) show right edge features’ placement.
Figure 3(c), (d) shows a reference feature graph for the
control system to allow weld pool width control
Inconel 718 alloy, which refers to the weld images in
implementation within a closed loop. The image analysis
Figure 3.
system may process an analogue image into a digital one
After the delivery of the initial configuration
(digitizing it) and process the registered images at the
information, software can be used to process the live
camera’s maximum frame rate, which is usually 25 Hz for
video images through the focus position in the processed
standard interlaced video. image, finding the best fit with previously stored features
If the image algorithm analysis operates between of the reference images, using the formula
subsequent video frames, then measurements may be
performed with the output results updated at a speed equal
X
Fs
to the camera’s frame rate. The image analysis technique F me ½i ¼ jRf ½j 2 I m ½i þ Jj for 0 # i , I w ; ð1Þ
used is based at an eight-bit grey-scale intensity correlated J¼0
with gathered characteristics from experimental and
featured images10. This technique was combined with a where Fme is a matched characteristic or correlation error
series of simple calibration procedures that were used in matrix value, Fs is a reference feature’s size in pixels, Rf is
order to limit the amount of image data needed to be a single-dimensional matrix storage of a feature’s intensity
processed in each frame. Figure 3 presents this method values, and Im is the pixel intensity of a real-time image,
using upper face images of Inconel 718 alloy. The imaging both in position and while moving from left to right in the
camera is arranged between the weld clamp and the horizontal.
welded component in such a way that the processing Upon completion of matching error calculations, the
software can analyse just a specific area of the weld pool. matching error matrix of Fme searches for a position where
Algorithm calibration is then made feasible using a the smallest correlation error occurs, which corresponds to
Welding International 245

(a) (b)

(c) (d)
250 250
Light intensity

Light intensity
200 200
150 150
100 100
50 50
0 0
0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100
Pixel number Pixel number

Figure 3. Image feature correlation: (a) calibration; (b) feature extraction; (c) left extracted feature graph; and (d) right extracted feature
graph.

the best property match as presented in this equation: correlation position. Figure 4 shows the processed image
and corresponding surface error correlation graph.
Crosses on the left and right side of Figure 4(a) show
Bm ! min{F me ½i for 0 # i , lw }; ð2Þ detected locations of the weld pool’s edges. A line made
across the bottom third of Figure 4(a) shows a normalized
where Bm corresponds to the best property match during single-dimensional presentation of the calculated corre-
movement from left to right in the horizontal. Therefore, lation error. The two lowest dimples in this line present the
the image vertical position, which is also represented by the smallest error correlation positions and, as a result, the best
i value and which reflects Bm the lowest correlation error, match with the reference feature’s edges. Figure 4(b)
refers to the best processed feature match. The correlation shows a 3D error correlation presented on a surface graph.
algorithm’s effectiveness is improved by searching only The best correlation points with edge properties
for the location of a left property for the left calibration clearly appear at app. 100 and 500 image width pixels.
position. A similar calculation method for the best probable As presented in Figure 5, when welding the Inconel, high-
match of a right-side reference feature is undertaken for intensity variations were caused in the weld image when
locations, which are on the right of an image-processing using arc pulsing. Excessive intensity or high-impulse

(a) (b)

10000
Matching difference

8000

6000

4000

2000

0
60
40 400
Line number 20 200
0 0 Pixel number

Figure 4. Processed weld images: (a) pulsed DC TIG on Inconel 718 and (b) correlation surface plot.
246 W. Lucas et al.

Figure 5. Inconel 718 image quality variation due to arc pulsing: (a) low-pulse image and (b) high-pulse image.

image saturation, according to Figure 5(b), may cause which shows a set of reference features, a processed
difficulties with credible data processing. image, and AC TIG on aluminium correlation plot.
In order to solve this problem, the processing When welding with aluminium, a transition in
algorithm was improved for the program to be able to intensity level around the edge location on the image,
differentiate between high arc (pulse current) and low present more or less half way along reference features, is
pulse (base current) being processed. This was achieved by clearly visible (Figure 6(a,b)).
calculating an average image-processing intensity, from Reference features for Inconel 718 nickel alloy show a
tendency to higher intensity gradients without a clear
left to right, at a previously defined calibration position.
transition at weld pools’ edges (Figure 3(b,c)).
Then, in order to differentiate low pulses from high
Figure 6 shows that images with various shapes of
pulses, a variable threshold was used. To take even more
edge reference features may be captured by the image-
stable measurements, depending on image quality, a high- processing algorithm and properly processed. When the
pulse from low-pulse processing exclusion was applied. image-processing algorithm is calibrated and correctly
In the example presented in Figure 5, in order to ‘learns’ a specific processing image type, configuration
improve the results, only low pulses were processed. The parameters such as edge size features, reference feature
property correlation technique may be applied in various intensity, processed calibration location, and pulsating arc
welding methods through the configuration of a user- exclusion setup may be saved as data files for later re-use,
defined series of parameters. This is illustrated in Figure 6, so a well-prepared algorithm may be loaded as soon as the

(a) 100 (b) 100


Light intensity

Light intensity

80 80
60 60
40 40
20 20
0 0
0 20 40 60 80 100 0 20 40 60 80 100
Pixel number Pixel number
(c) (d)

8000
Correlation error

6000

4000

2000

40
400
20 200
Line number 0 0 Pixel number

Figure 6. AC TIG on aluminium: (a) left pool edge reference feature; (b) right pool edge reference feature; (c) processed image; and (d)
correlation surface plot.
Welding International 247

Set pool Difference Set welding current


width Process Welder
+ Welding clamp
– contoller Set wire speed controller Camera

Image
Wire
processing Weld
feeder
Measured algorithm
pool width Welded material

Weld image

Figure 7. Closed loop process control schematic.

set-up is ready for processing. The calibration and learning with the analogous welding current signal and wire
procedure described earlier is necessary only when image feeding speed of the welding device13.
processing of non-standard classifications is required. ð
V trimI ¼ K PI ebw þ K II ebw þ K DI ðebw 2 ebw21 Þ; ð3Þ

Welding process control where Vtriml is the welding current adjustment, ebw is the
In order to present a closed loop control process, a PID weld pool width being the difference between the set
Digital Controller was used 11. Figure 7 shows a simplified and measured pool width, KPI is a proportional current
diagram. increase, KII refers to integral current increase in the last x
Control of fundamental activities involves comparison samples, where x is an integer ranging from 1 to 100. KDI is
of the width of a weld pool’s upper face from the image a differential current increase, and ebw – 1 is the previous
processing algorithm with a set value. If there is a pool width error:
difference between the set and measured pool width, ð
process control is applied, which can adapt both welding V trimWFS ¼ K PWFS ebw þ K IWFS ebw þ K DWFS ðebw 2 ebw21 Þ;
current and wire feeding speed. This active adjustment of
ð4Þ
current and wire feeding speed allows balancing of
differentiations in the weld pool’s size. Process control where VtrimWFS is the wire feeding speed adjustment, KPWFS
requires use of two independent PID controllers as shown is the proportional wire feeding speed increase, KIWFS
in Equations (3) and (4). refers to the integral wire feeding speed increase in the last x
The PID equation is introduced to the software by a samples, where x is an integer ranging from 1 to 100, and
separate program that works simultaneously with the KDWFS is the differential wire feeding speed increase.
image processing software on the control computer. The A real-time process control system requires an Agilent
CAN serial communication protocol is used to adjust VEE/14 graphical user interface. A typical Control
welding current and wire feeding speed12. This requires computer screen is presented in Figure 8.
use of a pair of built-in microprocessor systems acting as The VEE software has a display panel containing
process-operated CAN nodes. The fundamental operation dialogue boxes, which are presented on the left side
of the nodal points is to compare (deduct) the set value (Figure 8), allowing the following operations: equation

Figure 8. VEE and visual Cþþ image processing software.


248 W. Lucas et al.

parameter selection for the PID System Controller, general performed using the parameters presented in Table 2 in
data configuration, and user log-in process controls. order to illustrate the problems, which may be encountered
A series of graphic presentations including a pie chart when using a pulsing arc.
of a measured pool width are used to illustrate information The graphs in Figure 11(a) compare manually
on both current welding and feed wire values. Image measured upper weld pool width after welding, with
processing software, presented on the right side in Figure 8, automated measurements taken from the image processing
is controlled with the VEE user interface and operates as a software.
simultaneous control computer process. By means of the In order to demonstrate the measurement technique
TCP/IP protocol, measurements from the image proces- used, the first stage involved post-weld comparisons of
sing software are transmitted to the VEE system using the manually measured parameters with upper weld pool
client – server model15. surface measurements performed by means of image
processing. Measurement graphs made for uncontrolled
Experiment’s course and results hardening of the AC TIG aluminium by means of
parameters shown in Table 1 are presented in Figure 10.
The first stage of our experiment required a series of tests,
The graph in Figure 11(b) shows the effect of pulsed
which were conducted to determine the optimal camera
arc welding on automated measurements’ stability, where
arrangement for viewing the pool’s liquid top face. As a
the measured pool size shows an increasing tendency to
result, two separate camera configurations were applied:
One for aluminium and the other for Inconel alloy. As synchronization with the arc’s pulse frequency. This
presented in Figure 9(a), the camera as arranged on the occurs most of all in relation to high deviations of camera
weld pool’s edge and below the wire feeding level created image intensity during the application of the pulsating arc.
more stable images with TIG welding of AC aluminium. In order to avoid this problem, an automated measurement
For welding of the Inconel alloy, the best results were with high arc pulse exclusion was used. This required an
obtained with the camera mounted behind the weld clamp, image-processing software configuration that gave pool
as presented in Figure 9(b). measurements only during the low pulse of the base
According to the schedule of our experiment, a series current intensity. As a result, a more stable measurement
of tests were then carried out to assess whether pool width was obtained, which is presented in Figure 11(c), which
measurements based on image processing data would be complies with the manual pool width readings as shown in
stable/constant. Measurements performed after welding, Figure 11(a).
presented in Figure 10(a), are compliant with the The final stage of the experiment required use of a
automated measurements in Figure 10(b), and each closed loop control process. A pair of 2-mm Inconel sheets
graph shows a nominal pool width value of 8 mm. An with a 1.4-mm transition lip somewhere in the middle of
uncontrolled hardfacing on the Inconel alloy was then the sample was selected for this purpose.

(a) (b)

Wire
Wire
feeder
feeder

Torch
Camera Camera
Torch

Figure 9. Practical weld imaging arrangements: (a) AC TIG welding with aluminium and (b) Inconel welding.

(a) 16 (b) 16
14 14
Pool width (mm)

Pool width (mm)

12 12
10 10
8 8
6 6
4 4
2 2
0 0
0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100
Weld length (mm) Weld length (mm)

Figure 10. AC TIG on aluminium weld measurements: (a) manually measured upper pool width and (b) automated upper pool width
measurement.
Welding International 249

(a) (b)
10 10

Pool width (mm)

Pool width (mm)


8 8
6 6
4 4
2 2
0 0
0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90
Weld length (mm) Weld length (mm)

(c) 10

Pool width (mm)


8
6
4
2
0
0 10 20 30 40 50 60 70 80 90
Weld length (mm)

Figure 11. Inconel 718 measurements: (a) manually measured upper pool width; (b) automated upper pool width measurement; and (c)
automated measurement with high-pulse exclusion.

The aim of this stage was to obtain a comparison measurements were taken upon the completion of each
between controlled and uncontrolled samples of welded weld made for both the heat source and joint penetration
materials showing a sudden change in thickness, to sides and are presented in Figure 12.
demonstrate closed loop process control activity. For controlled welds, the weld pool upper surface
A PI welding current controller was used to implement width regulation also decreased the degree of weld
regulation of weld pool upper surface width with penetration. This is clearly illustrated by the increased
controlled welds. A 6-mm weld pool width was set and homogeneity of the measurements in the graph for the
experimental tests were used to show that the proportional post-welding weld pool surface width from the top and
current increase KPI ¼ 1 and the integral increase bottom sides, presented in Figure 12(b), as compared with
KII ¼ 0.01 created acceptable results. Weld pool surface the appropriate measurements for uncontrolled welding
presented in Figure 12(a).
Table 1. Aluminium welding parameters.

Parameter Value
Conclusions
Method 100 Hz frequency TIG AC This image processing technique, based on the correlation
Material Aluminium 2024, 1.6 T4
Electrode Ø 3.2-mm zirconium – tungsten electrode
of spatial domain features, was developed, thanks to a
Shielding gas Pure argon, 15 l/min desire for unfailingly high-quality image processing of
Welding speed 3.3 mm/s upper weld pools. Use of the calibration procedure and a
Welding current 80 A pair of edge reference features from the sample images
Wire feeding Ø 1.2-mm 5356 wire, speed 15 mm/s allowed use of this innovative and efficient algorithm
analysis, which can perform real-time upper weld pool
surface width measurements. The image processing
Table 2. Inconel welding parameters. software is completely configurable, and so it is adjustable
and can learn how to deal with the images from various
Parameter Value welding processes. This was proved by the measurement
Method Pulsed TIG DC at 2 Hz frequency results for the Inconel alloy and the aluminium, which
Material 2.0-mm thick Inconel 718 confirmed acceptable (with regard to the results)
Electrode Ø 3.2-mm zirconium – tungsten electrode measurement techniques for TIG welding of various
Shielding gas Pure argon, 14 l/min
materials.
Forming gas Pure argon, 18 l/min
Welding speed 1.5 mm/s Further improvements to the processing algorithm
100 A impulse current (high); involved introduction of an average image intensity
Welding current 30 A base current (low) threshold. This allowed distinguishing of high-pulsing
Impulse times 250 ms high, 250 ms low welding arcs (impulse current) from low (base current)
Ø 0.8-mm Inconel 718, speed
arcs. The processing algorithm may, therefore, be
Wire feeding 10 mm/s
configured in such a way so as to exclude measurements
Notes: Translated into English from a Polish translation of an original English taken during high or low arc phases. This gives more stable
paper. Original Polish translation by Ilona Pawlik and Adam Jastrzebski at the
Instytutu Laczenia Metali (Institute for Joining of Metals) in Kraków, Poland. measurements by effective processing in only specific
image intensity ranges, and as a result the processing
250 W. Lucas et al.

(a) (b)
10 10

Pool length (mm)

Pool length (mm)


8 8
6 6
4 4
Pool width on heat source side Pool width on heat source side
2 2
Pool width on the joint penetration side Pool width on the joint penetration side
0 0
0 20 40 60 80 100 120 140 0 20 40 60 80 100 120 140
Weld length (mm) Weld length (mm)

Figure 12. Post-weld pool width measurements: (a) uncontrolled weld and (b) controlled weld.

technique shows a higher tolerance to the large deviations References


in image intensity which often occur when using pulsating 1. Kim IS, Jeong YJ, Yarlagadda PKDV. Prediction of welding
arcs. Closed loop process control was implemented by parameters for pipeline welding using an intelligent system.
means of two PID controller equations, which allow upper Int J Adv Man Tech. 2003;22(9 –10).
surface weld pool regulation to a set value by adjusting 2. Balfour C, Lucas J, Maqbool S, Smith JS, Maqbool S, Meilroy
both current and wire feeding speed. The process control L. A neural network model for MIG welding parameter
prediction. Proceedings of 10th International Conference on
involved comparison of controlled and uncontrolled welds
Computer Technology in Welding. June; 2000.
on Inconel alloy, performed with the same welding 3. Gaqo J, Wu C. Extracting weld penetration information in
parameters. tungsten-inert gas welding. Proc Inst Mech Eng B J Eng
A change in metal sheet thickness was used as a final Man. 2002;216(2).
control test: in control of a weld with the regulation of the 4. Zhao J, Houghton MA, Balfour C, Smith JS, Lucas J. Real
upper surface weld pool, welding current regulation was time image processing for penetration control in automated
conducted by means of a PI controller. welding. Proceedings of 8th International Conference on
This solution allows successful upper surface weld Computer Technology in Welding. June; 1998.
5. Luo H, Devanathan R, Wang J, Cheng X, Sun Z. Vision
pool regulation and effects weld penetration depth for test
based neurofuzzy logic control of weld pool geometry. Sci
samples with a leap thickness change half way of welding. Technol Weld Joining. 2002;7(5).
As a result of this test, it was clearly stated in which way 6. Zhang YM, Kovacevic R. Neurofuzzy model-based
welding control system with computer image analysis predictive control of weld fusion zone geometry. IEEE
may be used to prepare stable profile welds when heat Trans Fuzzy Syst. 1998;6(3).
abstraction conditions change during process. 7. Zhao DB, Chen SB, Wu L, Dai M, Chen Q. Intelligent
The discussed experiments were conducted with a PC control for shape of the weld pool in pulsed GTAW with
fitted with a standard, inexpensive WinTV Card with filler metal. 2001;80(11).
8. Zaration B. Microsoft visual Cþ þ6.0 programmer’s guide.
control software created in modern visual programming
Microsoft Press; 1998.
languages. Therefore, the experimental welding control 9. Weiskamp J, Johnson N, Gauldt F, Florence M. How to
system discussed in this paper is considered a system digitize video. Wiley; 1994.
prototype with a potentially commercial application. In the 10. Sonka M, Hlavac V, Boyle R. Image processing, analysis
future, the feasibility of moving from the current old- and machine vision. 2nd ed. PWS Publishing; 1999.
fashioned Windows-based video technique to implemen- 11. Phillips CL, Nagle HT. Digital control system analysis and
tation of the more modern ActiveX image registration design. 3rd ed. Prentice-hall; 1995.
software should be considered. This should also allow for 12. CAN Specification Version 2.0, Robert Bosh GmbH (1991).
a broader combination of interfaces between imaging 13. Balfonr C, Smith JS. Controller area network (CAN) applied
to welding systems. Proceedings of 12th International
equipment and image processing software.
Conference on Computer Technology in Welding. August;
Further improvement in image analysis and algorithm 2002.
method should allow testing for potential accommodation 14. VEE Proc User’s Guide, Agilent Technologies Inc; 2000.
of image quality deviations and detailed edge features, in 15. Gregory K. Using visual Cþþ.Net. Que/Sams Publishing;
order to prepare a more tolerant measurement technique. 2002.

You might also like