The International Journal of Advanced Manufacturing Technology

https://doi.org/10.1007/s00170-018-1636-3

ORIGINAL ARTICLE

Automatic gap tracking during high power laser welding based

on particle filtering method and BP neural network
Yan-xi Zhang 1 & De-yong You 1 & Xiang-dong Gao 1 & Suck-Joo Na 2

Received: 28 September 2017 / Accepted: 15 January 2018

# Springer-Verlag London Ltd., part of Springer Nature 2018

Abstract
A numerical simulation model was established to investigate characteristics of keyhole and molten pool during the laser butt
welding. The sharp point in front of the keyhole and molten pool revealed the position of the gap, and its deviation in transversal
direction between the centroid of the keyhole demonstrated the real-time deviation between the laser beam and real gap. A visual
system was designed to capture real-time infrared images of keyhole and molten pool, and the real-time deviation data between
the laser beam and gap was extracted from these images. The state and measurement equations of real gap position prediction
were developed based on the welding system. The particle filtering (PF) method was employed to improve the accuracy of the
prediction of the gap position. Considering the non-linear and unknown distribution of system error and measurement error, a
Back Propagation (BP) neural network was developed to compensate for these errors. The effectiveness of the established PF
method combined with BP network was validated by experimental results, and higher prediction accuracy of gap position
tracking was achieved.

Keywords Gap tracking . BP neural network . Particle filtering . Numerical simulation

1 Introduction workpiece. Therefore, real-time gap tracking is especially cru-

High-power fiber laser can not only be delivered by fiber as its
short wavelength of 1.06 μm, but also produce high-intensity
heat sources which induce deep penetration and excellent me-
chanical properties of weld joints. In laser butt welding, the
deviation between the laser beam and weld gap would easily
result in bad weld bead and largely reduce the weld strength,
even with precise joint alignment and fixtures of the
* Xiang-dong Gao
gaoxd666@126.com
Yan-xi Zhang
yanxizhang@126.com
Suck-Joo Na
sjoona@kaist.ac.kr
1
Guangdong Provincial Welding Engineering Technology Research
Center, Guangdong University of Technology, No.100 West
Waihuan Road, Higher Education Mega Center, Panyu District,
Guangzhou 510006, China
2
Department of Mechanical Engineering, KAIST, 291 Daehak-ro,
Yuseong-gu, Daejeon 305-701, Republic of Korea
cial during high-power laser butt welding.
The keyhole and molten pool were the most important cou-
pling physical phenomena during laser welding, and their char-
acteristics provide rich information of the welding process.
Many numerical simulation methods have been carried out to
further analyze the transient characteristics of the keyhole and
molten pool due to its high resolution of the welding process.
Kaplan et al. applied an energy balance equation coupling with
a simple multiple-reflection model to predict the keyhole shape.
Due to the assumptions that multiple reflections were consid-
ered in a conical shape with an average angle of the keyhole
wall, only rough predictions were achieved [1]. A volume-of-
fluid (VOF) method was applied to numerically investigate the
mechanism of keyhole instability [2, 3]. Ye et al. developed a
3D model of full-penetration laser welding to study the mech-
anism and temperature distribution inside the workpiece [4].
Du et al. carried out a simulation to study the effect of molten
pool flow during a full-penetration laser beam welding of tita-
nium alloy [5]. Cho et al. also used the VOF method in con-
junction with a ray tracing algorithm to calculate the keyhole
shape and realize the real-time multiple reflections [6]. The
same algorithm was used to estimate the results of the drilling
process vis-à-vis the polarization of the laser beam [7] as well

as the results of laser-arc hybrid welding [8]. Zhang et al. ap-
plied a 3D numerical model to investigate the transient keyhole
and molten pool dynamics during a full penetration of laser
welding of two thick workpieces [9]. These publications mainly
focused on the behavior of the keyhole and molten pool during
laser bead-on-plate welding, but very few researches were de-
voted to characteristics of keyhole and molten pool during laser
butt welding.
Infrared sensing technology has been applied in welding pro-
cess monitoring. Amin et al. applied a low-cost autonomous
infrared radiation (IR) imager combined with a novelty detection
algorithm to monitor the state of the welding process [10], but
the sampling speed of their IR imager is less than 8 frames/s.
With this low frame frequency, the detailed dynamics of keyhole
and molten pool cannot be analyzed as the high-power laser
welding process is transient and drastic. An IR sensor system
has been employed to control the arc welding process coupling
with quantitative thermal image analysis techniques [11]. The
captured thermal images were used to measure the variations of
torch offset during arc welding. However, the heat transfer
mechanisms between the laser welding and arc welding are
totally different, so the method used in arc welding is difficult
to apply in laser welding. Huang et al. applied IR temperature
measurement during hybrid laser–tungsten inert gas welding
process to measure and calibrate the temperature field of the
hybrid welding process [12], but no further investigation of
how to determine the direction and position of gap from the heat
distribution of a molten pool was achieved.
The PF method has been widely applied in robot localiza-
tion and tracking [13], seam-tracking in arc welding [14],
battery health monitoring [15], prediction of crack growth
[16], spiraling ballistic missile state estimation [17], and wire-
less sensor networks [18]. However, the accuracy of tradition-
al PF algorithm depended on the statistical characteristics of
the process noise and measurement noise, which were difficult
to accurately obtain in real-time laser welding process.
Fortunately, neural networks have been reported to be
employed to compensate the errors due to the absence of sta-
tistical characteristics of the above noise in the Kalman filter-
ing method [19, 20].
In this work, a 3D numerical simulation of high-power
laser butt welding with a weld gap width of 0.1 mm was
established to investigate the characteristics of thermal distri-
bution and the dynamics of keyhole and molten pool. The
simulation showed the thermal distribution was influenced
by the gap; therefore, a near-infrared camera equipped with
a combination of specific filters was used to capture the ther-
mal images of the keyhole and molten pool. The gap position
was identified by the image processing algorithm. A PF model
combined with a BP neural network was employed to elimi-
nate the measured noises of gap position and help acquire the
precise weld gap position. Experimental results showed that
the proposed PF method compensated by BP neural network
Int J Adv Manuf Technol
could effectively improve the robustness and accuracy of gap
tracking during high-power laser butt welding.
2 Experimental setup and numerical
simulation
2.1 Experimental setup
The experiment system consisted of an IPG fiber laser
welding equipment (maximum power 10 kW), the shielding
gas (argon), a Panasonic six-axis robot and a welding platform
equipped with a high-speed camera, servo motors, and fixing
device. The schematic drawing of the experiment system is
shown in Fig. 1. The workpiece was type 304 stainless steel
plates of 10 mm in thickness, 150 mm in length, and 100 mm
in width. The laser welding speed was 2.5 m/min, with a laser
beam diameter of 200 μm. The sketch of the laser beam track
during laser butt welding experiment and numerical simula-
tion was shown in Fig. 2. The width of the gap was 0.1 mm.
2.2 Numerical simulation setup of laser butt welding
A numerical model based on computation fluid dynamic
(CFD) was developed and implemented for analysis of the
heat transfer and dynamics of the keyhole and molten pool
during laser butt welding. In the simulation, four governing
equations including mass conservation, momentum conserva-
tion (Navier-stokes Equation), energy conservation, and VOF
were calculated and their solutions were solved for analysis of
the thermal distribution and the characteristics of keyhole dy-
namic during laser butt welding. The molten pool metal is
assumed to be an incompressible laminar flow with
Newtonian viscosity during the simulation.
With these assumptions, the four governing equations can
be expressed in Cartesian coordinates as follows:
(1) Mass conservation equation:
∇ν ¼ 0 ð1Þ
Where, v is the velocity vector.
(2) Navier-stokes equation:
∂ν 1
þ ν∇ ν ¼ − ∇ P þ υ∇ 2 ν−Kν þ G ð2Þ
∂t ρ


Where, K ¼ C F 2s =ðð1−F s Þ 3 þ BÞÞ and υ = υ0(1.0 − (Fs/Fs,
cr)), ρ is the fluid density, P is the pressure, υ is the dynamic
viscosity, K is the drag coefficient for a porous media model in
the mushy zone, G is the body acceleration due to body force,
Int J Adv Manuf Technol

Fig. 1 The sketch of the welding High power laser

system setup (fiber)

Six-axe robot

Near IR camera
Reflection
Filters
Computer
Radiation Nozzle Shielding Gas (Ar)
Weld Gap

Workpiece

C is a constant representing the mushy zone morphology, Fs is (4) VOF equation:

the solid fraction, B is the positive zero used to avoid the
∂F
division by zero, υ0 is the dynamic viscosity of the liquid þ ∇ ∙ðνF Þ ¼ 0 ð5Þ
phase, and Fs,cr is a user-defined constant denoting the critical ∂t
solid fraction value at which all fluid in the mushy zone
freezes. It is assumed that the drag coefficient is 0 and the The laser beam profile is assumed to be characterized by
dynamic viscosity is constant in the molten pool (over liquid the type of Gaussian axisymmetric distribution. The detail
temperature), but in the mushy zone they are dependent on the description of the laser beam model and ray tracing algorithm
solid fraction shown in Eq. (2). can be found in the papers [6–8]. The Fresnel reflection model
when the laser ray interacts with the workpiece was simplified
(3) Energy conservation equation: as Eq. (6) by Hagen-Rubens relation under the assumption
∂h 1 that the workpiece material is a good conductor and the laser
þ ν∇ h ¼ ∇ ðk∇ T Þ ð3Þ beam wavelength is sufficiently long. ε is determined by the
∂t ρ
material properties and the laser type. In this investigation, the
fiber laser and 304 stainless steel workpiece were used, and
Where, h is computed by Eq. (4). the ε value of 0.25 was applied to result in good agreement
8 with experiments.
> ρs C s T ðT ≤ T s Þ
< T −T s
h ¼ hðT s Þ þ hsl ðT s ≤ T ≤ T l Þ ð4Þ !
>
: T l −T s 1 1 þ ð1−εcosφÞ2 ε2 −2εcosφ þ 2cos2 φ
hðT l Þ þ ρl C l ðT −T l Þ ðT l ≤ T Þ Rcs ¼ þ 2 ð6Þ
2 1 þ ð1 þ εcosφÞ ε þ 2εcosφ þ 2cos2 φ

Start point End point When the keyhole is formed, the recoil pressure is the most
y important driving force of liquid flow in the molten pool dur-
Welding direction ing laser welding. The model in Eq. (7) is used to calculate the
induced recoil pressure on the free surface [8].
1mm
1mm !
T −T b
Gap size:0.1mm PR ≅0:54Psat ðT Þ ¼ 0:54P0 exp Lv ð7Þ
RT T b

0 Where, Psat is the saturated pressure, P0 is the atmospheric

x pressure, Lv is the latent heat of vaporization, and Tb is the
Fig. 2 The sketch of the laser beam track boiling point.
 Int J Adv Manuf Technol

The Marangoni flow related to the surface tension is gen- deviation of 1 mm at the ending point. The top views of
erated because of the shear stress on the surface of the weld simulation results were shown in Fig. 3. The heat energy im-
pool induced by temperature gradient and surface activation. posed on the workpiece induced by the laser beam accumu-
The surface tension can be expressed by Eq. (8). lated at the gap, as the thermal conductivity of gap, which is
filled with air, is quite smaller than the workpiece. The accu-
γðT Þ ¼ γ 0m −AðT −T m Þ ð8Þ mulated heat melted the workpiece, and a sharp area coupled
with a sharp point in front of the molten pool and keyhole
Where, γ 0m is the surface tension at the melting point of the
appeared in Fig. 3. Therefore, this sharp area indicated the
material, A is the surface tension gradient of the metal (this
exact position of the gap.
value is negative when the material is a pure metal), and Tm is
Two section views of the welded joints both in simulation
the melting point.
and experiment were shown in Fig. 4. Figure 4a shows the
Buoyancy force can be expressed as in Eq. (9) by using the
section view when the laser beam focused on the left side of
Boussinesq approximation as only the effect of temperature
the gap, and Fig. 4b shows the section view when the laser
difference is considered in this work.
beam focused on the gap. The weld joint is good when the
F b ¼ ρgβðT −T 0 Þ ð9Þ laser beam focused on the gap; however, the two workpieces
were not reasonably joined together when the laser beam de-
Where, g is the gravity, β is the thermal expansion rate, and T0 viated from the gap in Fig. 4b, and the strength of the joint
is the reference temperature. would highly decrease consequently.
The energy on the top of the free surface is balanced be-
tween the laser heat flux, the heat dissipation by convection
and radiation, and the heat loss due to metal evaporation. The 3 Characteristics extracted for seam tracking
energy balance on the bottom of the free surface is similar with from experiments
the top surface without the laser heat flux when the keyhole
was blocked. The gap surface boundary condition was also set The radiation of the molten pool and plume in visible frequen-
the same value as the top surface boundary condition as the cy was highly stronger leading to the overexposed images;
laser ray was imposed on the gap surface when the laser beam therefore, it is difficult to acquire the images of keyhole and
focused on the gap. molten pool with visible light camera. The heat radiation spec-
The thermal parameters of the SUS 304L material was trum of the molten pool was mainly covered the near-infrared
tabulated in Table 1. band, while the plasma radiation focused at the ultraviolet
band. In order to eliminate interference of plume and obtain
2.3 Simulation results and analysis clear near-infrared image of a molten pool, a combination of
narrow-band filters at a specific frequency was placed ahead
The simulation setup is shown in Fig. 2. In the first 50 ms, the of the camera lens. With this narrow-band filter combination,
laser beam gradually moved from the left side of the gap with the high-speed camera was sensitive to a spectral band of 960–
a deviation of 1 mm at the starting point, to the gap center, and 990 nm, which corresponded to the thermal radiation of the
then gradually moved from the gap to its right side with a molten pool. Experiments show that the application of the
filter combination leads to sharper molten pool images. The
Table 1 The thermal properties of the 304 stainless steel [21] response curves of the narrow-band filters are shown in Fig. 5,
Properties Values
and the red area represents the effective light which can pass
through the combination of the two narrow-band filters. All
Solid density 7900 kg/m3 molten pool images were captured at 1000 frames/s by a NAC
Liquid density 6900 kg/m3 (company producing camera system) high-speed CMOS cam-
Viscosity 5.9 × 10−3 kg/m·s era. The calibration of images is 11.4 pixel/0.1 mm. Three
Surface tension 1.87 N/m sequences of captured images of molten pool are shown in
Surface tension gradient −0.43 × 10−3 N/m·K Fig. 6. The captured IR images of molten pool and keyhole
Specific heat of solid 760 J/kg·K demonstrated similar results with the simulation, and the de-
Specific heat of liquid 720 J/kg·K viation between the laser beam and gap can be extracted.
Latent heat of fusion 2.47 × 105 J/kg As shown in Figs. 2 and 4, the deviation in transversal
Latent heat of vaporization 6.52 × 105 J/kg direction resulted in bad weld joint, but the deviation in x
Liquids temperature 1697 K direction was not so harmful due to the small heat effects of
Solidus temperature 1727 K laser beam. Only the deviation in the transversal direction was
Boiling temperature 3023 K considered in this work. Combined with the simulation re-
sults, the data of the 245th row (marked in blue) of the ROI
Int J Adv Manuf Technol

Fig. 3 Keyhole dynamics during (0,0)

Welding direction
laser welding. a The laser beam is x
focusing on the right side of the
gap; b the laser beam is focusing
on the gap; c the laser beam is Gap
(a)
focusing on the left side of the gap
y
Molten
(b) pool

Keyhole
(c)

images shown in Fig. 6 was extracted and plotted in Fig. 7a.
The y coordinate of peak point in gray value curve in Fig. 7a
can be identified as the position of the gap in the y direction, as
the thermal accumulation gathered at the gap and made it
brighter in the IR images. The peak point denoted as P1. If
the peak gray values are located in a range of y coordinates,
the center of this range should be considered as the position of
the gap in the y direction. The second way to acquire the
information of gap position marked P2 in Fig. 7b was to mea-
sure the sharp point in front of the keyhole area in captured
images. The two measured gap positions, P1 and P2, were
combined and averaged to determine the gap position, and
the combined value was denoted as Pmeasured. As the laser
welding track was set to be an oblique line shown in Fig. 2,
and the IR camera was fixed together with the laser scanning
head, therefore, the laser beam can be located with fixed co-
ordinates of the IR images. The deviation between the laser
beam and measured gap position Pmeasured can be calculated
according to Eq. (10).


Di ¼ C calibration  Pmeasured;i −Lcenter ð10Þ
Where, Di is the measured deviation in the ith image, and
P measu re d, i is the measured gap position in the ith
image, Lcenter is the coordinate of laser beam which has a fixed
value according to the relation between the laser scanning
head and IR camera, and Ccalibration is the calibration of the
image, here Ccalibration =0.88 mm/100 pixels.
4 Automatic gap tracking with PF method
combined by BP network
4.1 PF method
As the laser welding track has been set in the experiment in an
oblique line shown in Fig. 2, and the true deviation was
expressed in Eq. (11) after calibrating the workpiece coordi-
nate system.
0:556
T¼ y−0:278; 0 ≤ y ≤ n; n ¼ 1000 ð11Þ
n
Where, T is the true welding track in the laser welding exper-
iment; n is the total frame number captured in the experiment,
and in this research n = 1000; y is the image sequence number.
The measured deviation by IR images and the true devia-
tion between laser beam and real gap are plotted in Fig. 8. It
shows that the measured deviation had the same trend with the
true deviation, but there are still some large fluctuations with

Fig. 4 Section views of the

workpiece and simulation results.
a The laser beam focused on the (a) (b)
right side of the gap; b the laser
beam focused on the gap
 Int J Adv Manuf Technol

100
velocity in the y direction, and t is the sampling time. The
process model was established by choosing the state parame-
80
ters x1, x2, and x3, which were defined by Eq. (13).
8
Spectral response (%)

60 < x1 ðk Þ ¼ ydeviation ðk Þ
x2 ð k Þ ¼ v ð k Þ ð13Þ
:
x3 ðk Þ ¼ ydeviation ðk−1Þ
40

Where, ydeviation(k) denotes the gap position at t = k, v(k) de-

20 notes the velocity at t = k, and ydeviation(k − 1) denotes the gap
position at t = k − 1.
0 The state equation can be written in Eq. (14).
400 600 800 1000 1200 1400
2 3 2 32 3
Fig. 5 The spectral response curve of the combined filtering x 1 ðk Þ 1 t 0 x1 ðk−1Þ
X ðk Þ ¼ 4 x2 ðk Þ 5 ¼ 4 0 1 0 54 x2 ðk−1Þ 5
x 3 ðk Þ 1 0 0 x3 ðk−1Þ
the measured deviation curve. The fluctuations of measured 2 3
0:5t 2
deviation curve were partly caused by the IR image processing þ 4 t 5W ðk Þ
algorithms which were sensitive to the gray range of captured 0
images of the molten pool and keyhole. Moreover, the key-
hole shape was drastically affected by the recoil pressure dur- ¼ A∙X ðk−1Þ þ B∙W ðk Þ ð14Þ
ing laser welding; the coordinates of the sharp point in the
front part of the keyhole may be affected consequently. The Where, X(k) and X(k − 1) is the state vector at t = k and t = k−1,
respectively. W(k) is the process noise at t = k, A ¼ ½ 1 t 0
third reason can be attributed to the internal errors of the 
welding system; for example, the system errors of the welding 010 100, B ¼ 0:5t 2 t 0.
robot and servo system of welding platform. The measurement position of the gap is given by Eq. (15).
In order to guarantee accurate tracking of the gap during 2 3
x1 ðk Þ
laser butt welding, the gap position parameters were extracted
Y ðk Þ ¼ ½1 0 04 x2 ðk Þ 5 þ M ðk Þ ¼ H∙X ðk Þ þ M ðk Þ ð15Þ
as an eigenvector and a dynamic mathematic model was
x3 ð k Þ
established. Considering that the deviation between the laser
beam and the gap mainly occurred along the y axis, the state Where, H ¼ ½ 1 0 0  is the measurement matrix, and M(k)
equation can be expressed by Eq. (12). is the measurement noises of the welding system at t = k; Y(k)
1 is the measured gap position at t = k.
ydeviation ðk þ 1Þ ¼ ydeviation ðk Þ þ t∙vðk Þ þ t 2 ∙W ðk Þ ð12Þ Kalman filtering is popular in object tracking domain, and
2
the minimum conditional variance criterion was employed to
Where, ydeviation is the deviation between the laser beam and obtain the best estimation of the state vector Y(k) under the
the gap in the y direction, W(k) is the system noise, v(k) is the assumptions that the noises W(k) and M(k) are random

Fig. 6 Keyhole images during

y Molten pool Keyhole
laser welding. a The laser beam is (0,0)
focusing on the right side of the
Welding direction

gap; b the laser beam is focusing

on the gap; c the laser beam is (a)
focusing on the left side of the gap

(b)
x

(c)

t=n t=n+1 t=n+2 t=n+3 t=n+4 t=n+5 t=n+6

Int J Adv Manuf Technol

Fig. 7 The measured position of y

250
(a) P1
the gap. a Measured gap position (0,0)
by thermal distribution of molten (b) Keyhole center
200
pool. b Measured gap position by

Gray value (a.u.)

Deviation
keyhole shape 150

100

50 x
P2
0
0 50 100 150 200 250

y coordinate (pixel)

Deviation between the laser beam and gap/mm

Fig. 8 The curves of the
measured and true deviation 0.4
between the laser beam and real
gap in the experiment
0.2

The real deviation

-0.2 The measured deviation by IR images

-0.4
0 100 200 300 400 500 600 700 800 900 1000
Sampling point/frame

distribution of Gaussian white noise sequences with 0 mean, econometrics, robotics, and navigation. Compared with the
and are statistically uncorrelated with each other. However, in Kalman filter and extended Kalman filter, the principal advan-
an actual welding process, the system process noise W(k) and tage of the PF methods is that these algorithms do not rely on
the measurement noise M(k) are always the color noise rather any local linearization technique or any crude distribution ap-
than white noise, and their statistical characteristics were dif- proximation of the noise.
ficult to determine and often resulted in the divergence of PF algorithms firstly produce many sampling particle pairs,
Kalman filter algorithm. {xik , ωik |i = 1,2…N}, from an empirical importance density
Since PF algorithms were introduced in 1993, they have function, and N is the number of the pairs of particles. The
become very popular in solving the optimization-oriented es- new particles at t = k + 1 were calculated according to the state
timation problems for non-linear and non-Gaussian state equation Eq. (14), and by comparing with the measurement
space models numerically in an online manner, and are now value at t = k + 1, the weights of all particles were updated, and
routinely used in fields as diverse as computer vision, the posteriori probability density function was acquired. In

Fig. 9 The acquired deviation by 0.4

Deviation between laser beam and gap/mm

PF method and IR image 0.3

measurement
0.2

0.1

-0.1
Predicted deviation by Particle filter method
-0.2 The real deviation
The measured deviation by IR images
-0.3

-0.4
0 100 200 300 400 500 600 700 800 900 1000
Smapling point/frame
 Int J Adv Manuf Technol

Fig. 10 The deviation error by PF 0.1

Deviation between laser beam and gap/mm

method and IR images

0.05

-0.05
Predicted deviation error by Particle filtering method
Measured deviation error by IR images
-0.1
0 100 200 300 400 500 600 700 800 900 1000
Sampling point/frame

order to reduce the degeneracy of the particles, residual resam- calculated and shown in Fig. 10. The statistical data of mini-
pling algorithm was adopted to delete the particles which had mum error, maximum error, mean error, and the variation of
low weights and increase the number of the particles which the predicted deviation were tabulated and compared in
had large weights. Table 2. Figure 10 and Table 2 show that the PF prediction
The algorithm can be summarized as follows: curve was more accurate than the measured deviation by IR
images, but the predicted errors were still large. The reasons
1. Initialize the particle pairs collection at t = k = 0. All par- were discussed as follows. The variation of the gap position is
ticles were generated uniformly and their weights were non-linear and its exact distribution remained unknown, and it
1/N, where N is the total number of particles. The particle is linearly written in Eq. (14) for simplicity in this work.
pairs collection can be expressed as {xik , wik |i = 1,2…N}. Therefore, the linear approximation errors will be inevitably
2. The following processes were looped until t = M, where M incorporated. The nature knowledge of the welding process
is the total number of samples. noise is difficult to be determined. The desired or more phys-
ical prediction of the gap position should be expressed in Eq.
(a) Update the particles {~xik } with the state equation (16).
expressed in Eq. (14) and acquire the new particles
xk ¼ ^xk þ Δxmodel þ Δxnoise ð16Þ
{~xikþ1 }.
 i 
(b) Update the weights of all particles w ~ kþ1 according Where, Δxmodel denotes the system modeling error by Eqs.
to the distance between the particle {~xikþ1 } and the (14) and (15), and Δxnoise denotes the unknown noise error.
measured value ximeasured , and all weights were
normalized.
(c) Resample the particle pairs collection
4.2 BP network
~ ikþ1 ji ¼ 1; 2…N } with residual resam-
{~xikþ1 ; w
In this work, a method based on BP network was established
pling algorithm. The particle pairs collections were
to compensate for Δxmodel and Δxnoise. BP network is one of
resampled as {xikþ1 ; N1 ji ¼ 1; 2…N }. the most widely used and acknowledged artificial neural net-
(d) Calculate the estimation of the ^xkþ1 state vector at works nowadays [22–24]. Its powerful ability of non-linear
t = k + 1 by ^xkþ1 ¼ ∑Ni¼1~xikþ1 w
~ ikþ1 ; i ¼ 1; 2…N. prediction is utilized to obtain the predictions of the combina-
tion of Δxmodel and Δxnoise, which is denoted as xcompensated.
The prediction curve of the deviation by the PF model was BP network typically includes one input layer, one or more
shown in Fig. 9, and the predicted deviation error was also hidden layer(s), and one output layer [25]. It is trained by a

Table 2 The statistical data of the

measured deviation and predicted Samples Measured deviation error Deviation error predicted by PF
deviation by PF method
Min. Max. Mean Variation Min. Max. Mean Variation

1–500 − 0.0723 0.0896 0.0197 6.316e-04 − 0.0537 0.0346 0.0163 4.201e-04

501–1000 − 0.0718 0.0980 0.0281 8.112e-04 − 0.0252 0.0685 0.0198 4.075e-04
Int J Adv Manuf Technol

Input layer Hidden layer Output layer The calculation process of BP algorithm can be summa-
H rized as follows.

Emeasured 1. Calculate the output of hidden layer neurons. The output

H of neuron j, denoted as Hj, can be calculated by Eqs. (17)
I
xcompensate and (18).
H O
Epredicted
 
I H

H j ¼ f ∑ wij xi þ b j ¼ f net j where f ðxÞ
i

H 1
¼ ; i ¼ 1; 2; …; n; j ¼ 1; 2; …; l: ð17Þ
1 þ e−x
Fig. 11 The structure of BP network in this work n
net j ¼ ∑ wij xi þ b j ; ð j ¼ 1; 2; …l Þ ð18Þ
i¼1
supervised learning algorithm with a lot of given samples.
During the BP network learning process, the errors were sub- Where, wij denotes the connection weights between neuron i
sequently backward propagated through the network to adjust in the input layer and the neuron j in the hidden layer; xi
the weights and thresholds of the connections between any denotes the input variable i; f is the transitive function of
two layers, to minimize the total sum of the mean squared hidden layer neurons; netj is the activation value of neuron j.
error (MSE) of the output layer [26]. Specifically, the process
modified the weights and thresholds of the network in an 2. Calculate the result of the output layer. With H j
iterative way to make the trained network fit the training sam- representing the outputs of hidden layer neurons, the re-
ples well. sults of output layer neurons Ok can be calculated with
The optimal structure of the BP neural network is deter- Eq. (19), where ak denotes the threshold of neuron k in the
mined by its input data and output data. Considering the state output layer.
equation and measurement equation in Eqs. (14) and (15),
respectively, the measurement error Emeasured and the predict-
ed error by PF Epredicted which had relationship with the sys-
l
tem modeling error Δxmodel and unknown noise error Δxnoise Ok ¼ ∑ H j wjk −ak ; k ¼ 1; 2; …; m: ð19Þ
were selected as the input data of BP network. The resulting j¼1
data in output layer is xcompensated, the compensation of ^xk . The
robustness of the neural network was highly affected by the
setup of the hidden layer. To achieve better prediction results,
the Hecht–Nelson law was applied to determine the number of 3. Calculate the error ek by Eq. (20), where Yk denotes the
neurons in the hidden layer, given the number of neurons in desired output.
the input layer was p, the number of neurons in the hidden
layer should be set as 2p + 1. In this work, the setup of the BP
network structure can be determined and shown in Fig. 11. ek ¼ Y k −Ok ; k ¼ 1; 2; …; m: ð20Þ

Fig. 12 The predicted deviation 0.4

Deviation between laser beam and gap/mm

of different methods
0.3

0.2

0.1
Predicted deviation by Particle filtering method
Predicted deviation by BP+particle filtering method
0 Real deviation
Measured deviation by IR images

-0.1
500 550 600 650 700 750 800 850 900 950 1000
sampling points/frame
 Int J Adv Manuf Technol

Fig. 13 Acquired deviation errors

by different methods 0.1 Real deviation error
Predicted deviation error by particle filter method
Measured deviation error
Predicted deviation error with particle filter method compensated by BP network

Deviation error/mm
0.05

-0.05

500 550 600 650 700 750 800 850 900 950 1000
Sampling point/frame

4. With the error ek, the weight wij which connects the ith
neuron in the input layer and the jth in the hidden layer
can be updated by Eq. (21), and the weight wjk which
connects the jth neurons in the hidden layer and the kth
in the output layer can be updated by Eq. (22).

 m ed 500 samples was used to validate the effectiveness of the
wij ¼ wij þ ηH j 1−H j xðiÞ ∑ wij ek ; i ¼ 1; 2; …; n; j
k¼1
¼ 1; 2; …; l:
wjk ¼ wjk þ ηH j ek ; j ¼ 1; 2; …l; k ¼ 1; 2; …; m:
5. Thresholds of hidden layer neurons and output layer neu-
rons can be updated by Eqs. (23) and (24), respectively.
5 Results and discussion
The measured deviation data by the IR images and the real
deviation data were divided into two collections. The first
collection which included 500 samples was used to train the
BP neural network, and the other collection which also includ-
proposed PF algorithm compensated by the BP neural
network.
ð21Þ In order to compare the effectiveness of the proposed
ð22Þ methods in this work, the absolute error and variation of
the predicted deviation between the laser beam and gap
by different methods are defined by Eqs. (25) and (26)
respectively.
Errpredicted ¼ jpredict i j−jreal i j; i ¼ 0; 1; 2…:M ð25Þ

 m qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
a j ¼ a j þ ηH j 1−H j ∑ ωjk ek ; j ¼ 1; 2; …; l ð23Þ 2
k¼1 MSE predicted ¼ ∑M i¼1 ððpredict i −real i Þ=M Þ =M ; i

bk ¼ bk þ ek ; k ¼ 1; 2; …; m ð24Þ ¼ 0; 1; 2…M ð26Þ

Figure 12 shows the deviation curve predicted by differ-

6. The iteration continues until the MSE between the calcu- ent methods. The deviation predicted by the PF method
lated and desired outputs is less than the termination compensated by BP neural network is closer to the real
criteria, or the number of iteration reaches to the fixed deviation compared with other methods. The predicted de-
maximum value. viation errors are plotted in Fig. 13. The PF method

Table 3 The statistical data of the

deviation errors by different Methods Minimum error Maximum error Mean error Variation
methods (mm) (mm) (mm)

Measured deviation error − 0.0718 0.0980 0.0281 8.112e-04

Predicted deviation error by PF − 0.0252 0.0685 0.0198 4.075e-04
Predicted deviation error by PF with BP − 0.0444 0.0298 0.0133 2.729e-04
algorithm
Int J Adv Manuf Technol
compensated by BP neural network gave the most reason-
able prediction results compared with other mentioned
methods in this work. The statistical errors of the deviations
measured by IR images, predicted by only the PF method
and predicted by the PF method compensated by BP neural
network, were tabulated in Table 3, respectively. The PF
method compensated by the BP neural network shows ob-
vious advantages in maximum error, minimum error, mean
error, and variation of the errors compared with the other
two methods. The PF method compensated by the BP neu-
ral network can predict a more accurate gap position during
laser butt welding, which is crucial in laser welding
automation.
6 Conclusion
A numerical simulation model was developed to investigate
the characteristics of the molten pool and keyhole. A high-
speed near-infrared camera equipped with a combination of
different band filters was employed to capture the keyhole and
molten pool images, and the deviations between the laser
beam and gap were measured in real time. The conclusions
of this research can be reached as follows.
1. The simulation results provide the background analysis to
use a near-infrared camera to acquire the visual informa-
tion of the keyhole and molten pool during high-power
laser butt welding.
2. A PF method was developed to estimate the gap position,
and it provides better accuracy than the measured gap
position by IR images. Furthermore, the PF method based
on the BP network was proposed to compensate for the
uncertain noises and the linearization error to acquire
more accuracy in gap position tracking during high-
power fiber laser butt welding.
3. The experimental results show that the mean error and the
error variation were highly reduced by applying the BP
network compensate to the PF method compared with
other methods mentioned in this work. The welding ex-
perimental results have confirmed its effectiveness in im-
proving the accuracy of the detection of the gap position
during high-power laser butt welding.
Funding information This work was partly supported by the National
Natural Science Foundation of China (Grant Number 51675104), the
Science and Technology Planning Project of Guangzhou, China (Grant
Numbers 201510010089, 201707010197), the Science and Technology
Planning Public Project of Guangdong Province, China (Grant Number
2016A010102015), the Guangdong Provincial Natural Science Foundation
of China (Grant Number 2017A030310494, 2016A030310347), and Youth
Science Foundation of Guangdong University of Technology (Grant
Number 16ZK0010).
References
1. Kaplan A (1994) A model of deep penetration laser welding based
on calculation of the keyhole profile. J Phys D Appl Phys 27(9):
1805–1814. https://doi.org/10.1088/0022-3727/27/9/002
2. Hirt CW, Nichols BD (1981) Volume of fluid method for the dy-
namics of free boundaries. J Comput Phys 39(1):201–225. https://
doi.org/10.1016/0021-9991(81)90145-5
3. Lee JY, Ko SH, Farson DF, Yoo CD (2002) Mechanism of keyhole
formation and stability in stationary laser welding. J Phys D Appl
Phys 35(13):1570–1576. https://doi.org/10.1088/0022-3727/35/13/
320
4. Ye XH, Chen X (2002) Three-dimensional modelling of heat trans-
fer and fluid flow in laser full-penetration welding. J Phys D Appl
Phys 35(10):1049–1056. https://doi.org/10.1088/0022-3727/35/10/
313
5. Du HB, Hu LJ, Liu JH, Hu XY (2004) A study on the metal flow in
full penetration laser beam welding for titanium alloy. Comput
Mater Sci 29(4):419–427. https://doi.org/10.1016/j.commatsci.
2003.11.002
6. Cho JH, Na SJ (2006) Implementation of real-time multiple reflec-
tion and Fresnel absorption of laser beam in keyhole. J Phys D Appl
Phys 39(24):5372–5378. https://doi.org/10.1088/0022-3727/39/24/
039
7. Cho JH, Na SJ (2007) Theoretical analysis of keyhole dynamics in
laser drilling considering the polarization of laser. J Phys D Appl
Phys 40(24):7638–7647. https://doi.org/10.1088/0022-3727/40/24/
007
8. Cho JH, Na SJ (2009) Three-dimensional analysis of molten pool in
GMA-laser hybrid welding. Weld J 88:35–43s
9. Zhang LJ, Zhang JX, Gumenyuk A, Rethmeier M, Na SJ (2014)
Numerical simulation of full penetration laser welding of thick steel
plate with high power high brightness laser. J Mater Process
Technol 214(8):1710–1720. https://doi.org/10.1016/j.jmatprotec.
2014.03.016
10. Amin AH, Robert P (2003) An autonomous low-cost infrared sys-
tem for the on-line monitoring of manufacturing processes using
novelty detection. Int J Adv Manuf Technol 22:249–258
11. Nagarajan S, Banerjee P, Chen WH, Chin BA (1992) Control of the
welding process using infrared sensors. IEEE Trans Robot Autom
8(1):86–93. https://doi.org/10.1109/70.127242
12. Huang RS, Liu LM, Song G (2007) Infrared temperature measure-
ment and interference analysis of magnesium alloys in hybrid laser–
TIG welding process. Mater Sci Eng A 447(1-2):239–243. https://
doi.org/10.1016/j.msea.2006.10.069
13. Tomás de JMS, Francisco PG (2016) Toward simple strategy for
optimal tracking and localization of robots with adaptive particle
filtering. IEEE-ASME Trans Mechatron 21:1–1
14. Chang D, Son D, Lee J, Lee D, Kim TW, Lee KY, Kim J (2012) A
new seam-tracking algorithm through characteristic-point detection
for a portable welding robot. Robot Comput Integr Manuf 28(1):1–
13. https://doi.org/10.1016/j.rcim.2011.06.001
15. Saha B, Goebel K, Christophersen J (2009) Comparison of prog-
nostic algorithms for estimating remaining useful life of batteries.
Control Trans Inst Meas 3(4):293–308
16. Zio E, Peloni G (2011) Particle filtering prognostic estimation of the
remaining useful life of nonlinear components. Reliab Eng Syst Saf
96(3):403–409. https://doi.org/10.1016/j.ress.2010.08.009
17. Kim J, Vaddi SS, Menon PK, Ohlmeyer EJ (2012) Comparison
between nonlinear filtering techniques for spiraling ballistic missile
state estimation. IEEE Trans Aerosp Electron Syst 48:313–328
18. Vázquez MA, Míguez J (2017) A robust scheme for distributed
particle filtering in wireless sensors networks. Signal Process 131:
190–201. https://doi.org/10.1016/j.sigpro.2016.08.003

19. Xiong SS, Zhou ZY (2003) Neural filtering of colored noise based
on Kalman filter structure. IEEE Trans Instrum Meas 52(3):742–
747. https://doi.org/10.1109/TIM.2003.814669
20. Gao XD, You DY, Katayama S (2012) Seam tracking monitoring
based on adaptive Kalman filter embedded Elman neural network
during high-power fiber laser welding. IEEE Trans Ind Electron
59(11):4315–4325. https://doi.org/10.1109/TIE.2012.2193854
21. Tan W, Bailey NS, Shin YC (2013) Investigation of keyhole plume
and molten pool based on a three-dimensional dynamic model with
sharp interface formulation. J Phys D Appl Phys 46(5):055501.
https://doi.org/10.1088/0022-3727/46/5/055501
22. Zhang YX, Gao XD, Katayama S (2015) Weld appearance predic-
tion with BP neural network improved by genetic algorithm during
disk laser welding. J Manuf Syst 34:53–59. https://doi.org/10.1016/
j.jmsy.2014.10.005
23. Suganthi XH, Natarajan U, Sathiyamurthy S, Chidambaram K
(2013) Prediction of quality responses in micro-EDM process using
Int J Adv Manuf Technol
an adaptive neuro-fuzzy inference system (ANFIS) model. Int J
Adv Manuf Technol 68(1-4):339–347. https://doi.org/10.1007/
s00170-013-4731-5
24. Zhang LQ, Wang RJ (2013) An intelligent system for low-pressure
die-cast process parameters optimization. Int J Adv Manuf Technol
65(1-4):517–524. https://doi.org/10.1007/s00170-012-4190-4
25. Li W, Gao K, Wu J, Hu T, Wang J (2014) SVM-based information
fusion for weld deviation extraction and weld groove state identifi-
cation in rotating arc narrow gap MAG welding. Int J Adv Manuf
Technol 74(9-12):1355–1364. https://doi.org/10.1007/s00170-014-
6079-x
26. Sun A, Jin X, Chang Y (2016) Research on the process optimiza-
tion model of micro-clearance electrolysis-assisted laser machining
based on BP neural network and ant colony. Int J Adv Manuf
Technol 88:3485–3498