A report on:

Autonomous Seam Acquisition for Robotic

Welding Based on Passive Vision

Presented under the

Presented by:
guidance of:
Kartik Kumar (M.Tech)
Dr. Ruby Mishra
Machine design
2040001

School of Mechanical Engineering

KIIT deemed to be university
Bhubaneswar, Odisha, India
27th March, 2021
 Abstract

A method of autonomous seam acquisition for arc welding robot is proposed. The
seam information is real-timely obtained by one CCD camera in front of the
anterior seam, while the host computer is incharge of image processing and edge
features extracting. Thus the deviation between the projection of the tungsten
electrode and the central line of welding seam can be obtained. The method can
also be used to finish seam acquisition in the whole welding process autonomously
even if the robot is never taught. The results validate feasibility of the method and
imply the improvement of autonomy and intelligence for robot welding.
 Introduction

Nowadays, a great number of industrial robots have been used in the fields of
automatic arc welding. However, most industrial robots are still working in the
teaching and playback mode which can’t meet the requirements of intelligent welding
process and welding quality [1]. Hence, various sensors are applied in the welding
process. Sensing technology is used to monitor the status of the process and acquire
characteristic information. It is one of the most important steps for robotic welding
automation. Many research papers, specific to different interferences and features of
welding conditions, proposed various sensing methods, for example, ultrasonic
method [2], infrared thermometry [3] and positive vision such as laser sensing [4].
With the development of the computer vision technology and image processing
methods, numerous welding robots and some automatic welding machines are
equipped with corresponding vision sensors to achieve different welding tasks in
severe conditions.
 Continued...

Kim [5] developed one welding robot system with both laser sensors and passive
vision, which could be used for welding environment identification. Besides, K.Y.
Bae [6] studied the online seam tracking of steel tube based on machine vision. But
the weld gap was 2-4mm, which decreased the difficulty of image processing. J.S.
Smith [7] studied GTAW real-time top-face vision and used PID controller to assure
the welding formation and quality. Due to its great reliability, small size and low
cost, passive sensing technology with CCD sensors has been taken as one of the
most common methods for robot welding sensing. This paper focuses on the
autonomous seam acquisition for arc welding robot, presents a method of weld
seam extraction with the CCD sensor. To some extent, it would be of importance to
improve the level of autonomy and intelligence for robotic welding.
Fig.1(a) Real Experiment System
Fig.1(b) Components and Communication of System
 Experiment System
Fig. 1(b) shows the main components and their relationships of the experiment system. Fig. 1(a)
shows the robot welding system and scene photograph. RH6 robot is used in the experiment
system which can accomplish motions of six joints in the process of trajectory planning. The
interval that host computer could send data to the controller for robot motion is 16ms. The
communication between the robot controller and the host computer is achieved by CAN bus.
Welding torch is fixed at the end of the flange joint, while the CCD camera is fixed at the front of
the torch. The images obtained by the camera are converted into digital signals through the
CG400 image acquisition card.Since the host computer extracts the feature information of these
images, such as seam track, corner and projection of tungsten, it can translate image information
to robot-known data so as to control the robot to follow the seam track. During the movement,
the coordinate values of the robot position are fetched by the CAN bus for the host computer’s
record. Thus, the host computer could calculate the deviation between the projection of tungsten
and seam central line, and revise the path of the robot in the form of incremental coordinate
values.Welding for butt plates is implemented to verify the effectiveness and adaptability. Before
welding, the torch of the robot is guided to the initial welding position, while the welding path is
not taught. The CCD camera is incharge of capturing the real-time images for seam information
extraction so that the host computer can calculate deviation to help the robot planning the
welding path and complete the welding task.
 Image Process and Seam Extraction

The image of welding seam and molten pool taken by the CCD camera is shown in
Fig. 2(a). The images are captured on the moment of base pulse current. In order to
reduce the redundant information and accelerate image processing speed, a small
window in front of the molten pool is selected. In the small window, seam can be
obtained quickly. Meanwhile, to eliminate the random noise interference in the
image acquisition and transmission, the regular techniques of smoothing and
sharpening are adopted in the window as pre-processing. Median filter with 3x3
template is adopted. Image sharpening is also used for enhancing the feature
information. The pre-processing window image is shown in Fig. 2(b).
Fig. 2. (a) Origin Image Fig.2.(b) Extracted
Window
In order to improve the precision of the image processing and reduce the reflection interference
of arc light, a new algorithm is presented to extract the feature of the seam edges. The basic
procedures are expressed as follows:
1) Sobel-operator edge detection, aiming at extracting the edge information.
2) Threshold segmentation, converting the image into binary pattern easy to subsequent
processing.
3) Image thinning, by which the main skeleton can be distinguished.
4) Adaptive area filtering, eliminating clutter points and edges in order to remove pseudo-edge.
5) Welding region locking, locking the most interested object for seam tracking.
6) Image restoration and Canny-operator extraction, here, processing the seam edges in the
region locked as Canny operator detection contains more details which is necessary for seam
tracking.
7) Area filtering again and fitting the seam edges, thus the central line of the weld seam can be
obtained easily finally.
The procedures and results are shown in Fig. 3, while Fig. 4 shows the final result.
The whole algorithm costs less than 300ms to accomplish all steps. What’s more,
different welding experiments prove the feasibility of the method as the average
offset is less than two pixels, about 0.017mm.
After seam edges extraction, two sets of data are stored to fit the curve. Linear
fitting by least squares method is taken to obtain the linear equation of the central
line of the weld seam, as shown in the Equation (1).
• f (x)= ax +b
• a=( k1+k2)/2; b =(b1+b2)/2 ...(1)
where , k1,k2, b1,b2 are the coefficients of the fitting curves for the upper and
lower weld seam edges.
Fig. 3. Image Procedures and Results
Fig. 4. Result of the Image Processing
 Seam Deviation Definition
As shown in the Fig. 5, there are two coordinate systems, one is the image system,
i.e. u-v coordinate system and the other is the base system, i.e. x-y coordinate
system. Here, the image coordinate is defined as image captured by CCD camera. Its
origin is the left lower point. The base coordinate system means the real coordinate
system used for robot motion. In the image coordinate system, the distance in the v-
axis direction, which is between the tungsten projection and the central line of the
seam, is defined as weld seam position deviation d . Also, when the tungsten
projection is above the seam in the image, d is defined as positive. Otherwise, it is
negative. Defined that angle deviation of weld seam α is the angle between the
tangent of the central line of the seam and the positive u-axis. Then, according to
geometric relationship and ratio between actual distance and image solution, the
weld seam position deviation d can be obtained as follows:
Where pu and pv are the u-axis and v-axis coordinates of the tungsten
projection. so is the actual size of a pixel in the v-axis direction.
 Experiments and Results Analysis

The robot is guided to the initial welding position as well as the weld seam is not
taught. Passive visual sensing is used for monitoring the whole welding process.
Here the length of the weld seam is set in advance. Thus the motion cycle can be
calculated and set based on the velocity and frequency of the welding. When the
welding cycle equals the set motion cycle, both the robot motion and welding will
stop automatically. The welding experimental conditions are carried out in Table 1.
Fig. 7(a) displays the initial tracking error curve while Fig. 7(b) shows the
error curve based on Kalman filtering. It is obvious that through Kalman filtering,
the tracking trace appears much more smoothed. The error with and
without the Kalman filtering is in the range of ± 0.4mm and ± 0.5mm separately.
From the comparison, data process like Kalman filtering is important and necessary.
Fig. 8 exhibits an example of welded work-piece. However, some abrupt
motions occur due to the deviation mutation and instruction delay which
influence the
welding quality most. If one more advanced algorithm is adopted to quicken
the
image processing, it would improve the motion smoothness of the robot and
raise
the precision of seam tracking much better.
 Conclusions

A method of autonomous seam acquisition with passive vision for arc welding
robot is proposed. By means of obtaining the weld seam information of anterior
seam in the welding direction and generating corresponding seam coordinates, the
robot can track the weld seam and finish the whole weld task with a PD controller.
The experimental results after Kalman filtering show that the method can achieve
the alloy butt weld task and the precision meets the production demands, which
improved the level of autonomy and intelligence for robot welding.
 References
[1] Chen, S.: On the Key Technologies of Intelligentized Welding Robot. Robotic
Welding, Intelligence and Automation, 105–115 (2007)
[2] Fenn, R.: Ultrasonic Monitoring And Control During Arc Welding. Welding Journal
(Miami, Fla) 64(9), 18–22 (1985)
[3] Nagarajan, S., et al.: Control of the welding process using infrared sensors. IEEE
Transactions on Robotics and Automation 8(1), 86–93 (1992)
[4] Peiquan, X., et al.: An active vision sensing method for welded seams location using
"circle-depth relation" algorithm. International Journal of Advanced Manufacturing
Technology 32(9-10), 918–926 (2007)
[5] Kim, M.Y., et al.: Visual sensing and recognition of welding environment for
intelligent shipyard welding robots (2000)
[6] Bae, K.Y., Lee, T.H., Ahn, K.C.: An optical sensing system for seam tracking and weld
pool control in gas metal arc welding of steel pipe. Journal of Materials Processing
Technology 120(1-3), 458–465 (2002)
[7] Smith, J.S., Balfour, C.: Real-time top-face vision based control of weld pool size.
Industrial Robot 32(4), 334–340 (2005)
Thank you