Knowledge-Based Systems 261 (2023) 110212

Contents lists available at ScienceDirect

Knowledge-Based Systems
journal homepage: www.elsevier.com/locate/knosys

Cross-attention-based multi-sensing signals fusion for penetration

state monitoring during laser welding of aluminum alloy
Longchao Cao a,b , Jingchang Li c , Libin Zhang a,b , Shuyang Luo c , Menglei Li c ,
∗
Xufeng Huang c ,
a
School of Mechanical Engineering & Automation, Wuhan Textile University, Wuhan 430074, PR China
b
Hubei Key Laboratory of Digital Textile Equipment, Wuhan Textile University, Wuhan 430200, PR China
c
School of Aerospace Engineering, Huazhong University of Science & Technology, Wuhan 430074, PR China

article info a b s t r a c t

Article history: A precision multi-sensor monitoring strategy is required to meet the challenges posed by increasingly
Received 3 October 2022 complex products and manufacturing processes during laser welding. In this work, an acoustic sensor
Received in revised form 29 November 2022 and a photoelectric sensor were adopted to collect the signals during the laser welding of aluminum
Accepted 15 December 2022
alloy. The dataset was divided into three categories according to the morphologies of the top and
Available online 19 December 2022
back sides. The cross-attention fusion neural network (CAFNet) was proposed to interactively capture
Keywords: photoelectric and acoustic information for effective quality classification without prior time–frequency
Penetration state monitoring analysis and feature learning. Its effectiveness and superiority were compared with the five types
Laser welding of deep learning (DL) based methods. It demonstrates that the proposed CAFNet method achieved a
Cross-attention mean testing accuracy of 99.73% and a standard deviation of 0.37%, which outperforms other compared
Multi-sensing signals fusion
models. At the same time, the proposed CAFNet achieved the highest average testing accuracy of 94.34%
Deep learning
when utilizing limited and imbalanced data, which suggested that the proposed method has stronger
robustness than other methods. This approach is a new paradigm in the monitoring of laser welding
and can be exploited to provide feedback in a closed-loop quality control system.
© 2022 Elsevier B.V. All rights reserved.

1. Introduction to identify and control the penetration state of weld seams in

Owing to the advantages of low heat input, small heat-affected
zone, and high welding efficiency, laser welding has extended
applications in the welding of aluminum alloy [1]. However, it
suffers from several weld-induced imperfections such as poor
weld appearance and cracks due to the special physical properties
of aluminum alloy of high reflectivity, large thermal expansion
coefficient as well as low liquid viscosity, [2,3]. Thereinto, the
high reflectivity of the aluminum alloy decreases the absorption
of laser energy resulting in incomplete penetration of the weld
bead. In addition, the surface temperature and roughness which
have a significant influence on the absorptivity are dynamically
changed during laser welding [4,5]. Therefore, the penetration
state may fluctuate along the same seam, which is intolerable in
engineering practice. An intelligent method is urgently needed
∗ Correspondence to: School of Aerospace Engineering, Huazhong University
of Science & Technology, 1037 Luoyu Road, Wuhan 430074, PR China.
E-mail addresses: longchao2017@gmail.com, clc@wtu.edu.cn (L. Cao),
jingchangli@hust.edu.cn (J. Li), zhanglibin0701@126.com (L. Zhang),
luoshuyang@hust.edu.cn (S. Luo), m202071363@hust.edu.cn (M. Li),
huangxufeng@hust.edu.cn (X. Huang).
industrial fields.
With the continuous development of intelligent manufactur-
ing, deep learning (DL)-based monitoring is emerging, which pro-
vides a promising manner to in-situ detect the welding states [6,
7]. To obtain suitable information of welding states and develop
reliable DL-based monitoring methods, several signals (includ-
ing the optical [8–13], acoustic [14–16], and thermal [17–19]
signals) dynamically generated during laser-material interaction
are monitored by various sensors, which contain important in-
formation related to weld morphology and penetration status.
High-speed cameras, industrial cameras, and photodiode sen-
sors are the most frequently used sensors for optical signals. It
found that plasma plume formation and weld pool oscillation
were significant signal sources during the laser welding process.
Therefore, any instability on those occasions would consequently
affect the weld states, thence it is important to monitor the
dynamic behaviors of the plasma plume and weld pool [20]. To
monitor the behavior of vapor plumes during high-speed laser
welding (higher than 10 m/min), Xue et al. [21] applied a high-
speed camera to obtain images of vapor plumes and analyze
the vapor plume characteristics considering the welding quality.
In this context, the random forest with the Bagging method
was adopted to classify the three ejection regimes of the vapor

https://doi.org/10.1016/j.knosys.2022.110212
0950-7051/© 2022 Elsevier B.V. All rights reserved.
L. Cao, J. Li, L. Zhang et al.
plumes. Cai et al. [22] used a high-speed camera to monitor
the laser welding process, an image adaptive fusion method was
proposed to reduce the interference of the metal vapor plume. A
two-dimensional convolutional neural network (2D-CNN) model
was constructed to recognize the penetration state and the ac-
curacy was improved to 99.86%. While Liu et al. [23] proposed a
hybrid-driven method for vision-based recognition of penetration
status in laser welding, which uses a label semantic attention
mechanism to guide the 2D-CNN for accurate recognition of four
penetration statuses including lack of penetration (LP), partial
penetration (PP), full penetration (FP), and excessive penetration
(EP). Although the high-speed camera is always used to monitor
the welding process intuitively, the photodiode sensor with a
higher sampling frequency is less expensive. At the same time,
the amount of the collected one-dimension data is smaller, which
is convenient for storage and real-time processing. It may be
more suitable for real-time monitoring and feedback control in
industrial practice. Besides, the microphone was attracted to ac-
quire the acoustic signal for welding quality assessment. Giovanni
et al. [24] built a photodiode-based monitoring system to capture
the signals during remote laser welding of a copper-to-steel thin
foil of 300 µm. The part-to-part gap variations and EP conditions
were diagnosed according to energy intensity and scatter level.
Similarly, Lee et al. [25] developed a DL identification framework
to predict three penetration states (i.e., LP, PP, and FP) in laser
overlap welding, which used cost-effective photodiode signals
and a one-dimensional convolutional neural network (1D-CNN).
Luo et al. [26] applied the microphone array to localize and
track the acoustic signals during the welding processing, while
the welding defects were recognized by source localization. Fur-
thermore, some researchers focus on monitoring the penetration
status using thermal signals. Yusof et al. [16] found it feasible
to monitor the penetration status and explored the correlation
between the laser energy and penetration depth. The thermal
signal was always captured by the pyrometer or infrared camera.
Xiao et al. [27] utilized a coaxial infrared pyrometer to capture
the temperature of the molten pool. They concluded that there
was a linear relationship between the thermal signal and the
penetration depth.
The above studies monitor the laser welding process through
a single sensor or signal, the effectiveness is acceptable, how-
ever, a single sensor is unable to obtain comprehensive infor-
mation during laser welding, therefore, the detecting accuracy
maybe not be enough accordingly. Recent studies have shown
that a welding monitoring system utilizing multiple sensors with
data fusion schemes can yield improved identification results
[20,28–33]. Zhang et al. [28–30] designed a multiple-sensor sys-
tem (including a high-speed camera, a photodiode sensor, a spec-
trometer, etc.) to get detailed information of the laser weld-
ing process. After the analyses of the multiple sensors, different
classification models (e.g., stacked sparse autoencoder and deep
belief network) were employed to discriminate defects from the
extracted features and achieve satisfactory classification perfor-
mance. Shevchik et al. [31] adopted laser back-reflected (LBR)
and acoustic emissions (AE) sensors to monitor the laser welding
process of aluminum alloy, while the X-ray radiographic imaging
method was applied to observe the process zone and provided
ground truth of welding quality. By fusing the LBR and AE fea-
tures, a temporal 2D-CNN model was built to classify the five
statuses (i.e., conduction welding, stable keyhole, unstable key-
hole, blowout, and pores) with an average accuracy of 94.6%. Ma
et al. [32] applied multi-sensing signals including a high-speed
camera signal and a coherent light signal to recognize and detect
the porosity defects in the laser welding of aluminum alloy. Key-
hole 3D morphological characteristics were built and the keyhole
depth signal was captured. A 2D-CNN was developed to detect
2
Knowledge-Based Systems 261 (2023) 110212
porosity with an accuracy of 95.67%. Li et al. [33] introduced a
multi-feature back propagation neural network (BPNN) to predict
the weld penetration state (i.e., LP, PP, and FP) with an accuracy
of 97%, which using the extracted features of plasma plume and
spectrum as input.
However, these methods rely strongly on hand-crafted prepro-
cessing or time–frequency analysis, which demanding of costly
specialized knowledge. Additionally, the feature extraction of dif-
ferent sensors is performed individually rather than utilizing
information from both sensors jointly, which will lead to the
loss of some critical high-level features from both sensors and
further induce performance deterioration of diagnostic models.
In recent years, the self-attention mechanism of the Transformer
model [34] has been proven to be an effective fusion scheme in
different domains, such as point cloud learning [35], autonomous
vehicles [36], and prognostics and health management [37]. Nev-
ertheless, no related works have adopted the self-attention mech-
anism to fuse the time-domain signals of the photodiode and
microphone sensors for quality monitoring in laser welding. As a
simple, responsible high sampling frequency, low cost, and easy
automation monitoring and feedback control system, microphone
and photodiode sensors are attractive to monitor the optical
and acoustic signals generated from plasma plumes and molten
pools. Therefore, this study aims to develop a photoacoustic-
based diagnosis approach to provide accurate identification of
the welding penetration states. This approach is based on one-
dimensional CNN (1D-CNN) and a cross-attention mechanism,
which can interactively learn the photoacoustic information in
the time domain for the improvement of classification accuracy.
The main contributions of this article are summarized below:
(1) A multi-sensing system consisting of a microphone and a
photodiode sensor is established and welding experiments are
conducted to construct a sizable dataset;
(2) An innovative cross-attention fusion neural network
(CAFNet) is proposed, which could interactively extract the pho-
toacoustic features;
(3) The CAFNet performs well under imbalanced data or lim-
ited training ratio and could work directly on raw temporal
signals without any time–frequency analysis or pre-denoising
methods.
The structure of this paper is as follows: Section 2 introduces
the experimental setup and the signals monitoring platform. The
acquired data during laser welding were preliminary analyzed. In
Section 3, the acquisition and processing of datasets and the pro-
posed CAFNet are introduced. In Section 4, the performance of the
CNN model for penetration state monitoring of laser welding was
analyzed and compared with other network models, followed by
the conclusion in Section 5.
2. Laser welding equipment and monitoring system
2.1. Laser welding equipment and materials
Fig. 1 demonstrates schematic diagrams of the laser welding
and monitoring system. A fiber laser IPG YLS-30000 with a laser
wavelength of 1070 nm was employed in the experiments. The
maximum output laser power is 30 kW. The spot diameter of the
focused laser beam is 0.5 mm. The laser beam was transmitted
to the welding head through an optical fiber. The welding head
was mounted on a 6-axis KUKA robot. Argon was utilized as the
shielding gas with a flow rate of 1.5 m3 /h. The 2A12 aluminum
alloy workpieces with a size of 100 mm ×50 mm ×3.8 mm were
utilized.
The chemical composition of the material is shown in
Table 1. Before welding, the surface of the workpieces was pol-
ished by a polisher and cleaned with acetone to remove the oxide
film that results in metallurgical porosity interference during the
experiments [38].
L. Cao, J. Li, L. Zhang et al.
Fig. 1. Schematic diagrams of laser welding and monitoring system: (a) the laser welding experimental platform, (b) the photodiode and microphone sensor-based
signals acquisition system.
Table 1
The chemical concentration of 2A12 aluminum alloy (weight in %).
Element Cu Mg Mn
Content (%) 3.92 1.08 0.62
2.2. Generated mechanism of light and acoustic signals
Understanding the generated mechanism of light and acoustic
signals during laser welding can widen the option for moni-
toring the laser welding process on a real-time basis. During
laser welding, the acoustic signal source includes structure-borne
acoustic and air-borne acoustic emission signals. The structure-
borne acoustic signal is caused by cracks, porosity, phase transfor-
mation, and the back-reflected laser. While the air-borne acoustic
signal is induced by plasma plume formation, weld pool, and key-
hole oscillation, as well as the gas jet pulsation in the weld zone.
The structure-borne acoustic signal is collected by a piezoelectric
sensor which is contacted directly on the surface of a workpiece.
Meanwhile, a microphone sensor is used to acquire the air-borne
acoustic signal, which does not need to be contacted with the
surface of the workpiece. Therefore, it is convenient to utilize
the microphone sensor to monitor the high-temperature laser
welding process. As the laser beam interacts with the materials,
visible and near-infrared (NIR) radiates from the laser interaction
zone such as the metal vapor (plasma or plume), the molten, and
the base metal due to high temperature.
2.3. Photodiode and microphone based signals acquisition system
Fig. 1(b) demonstrates the built photodiode and microphone
sensor-based signals acquisition system. This system consists of a
photodiode sensor, a microphone sensor, a DC power supply, one
data acquisition card (DAQ), a personal computer, and a software
system. A C10439-11 photodiode made by Hamamatsu is applied.
It is an InGaAs-based photodiode with a circle photoactive area of
3 mm in diameter. The available measurement wavelength range
is 0.5 µm to 1.7 µm and the sensitivity of the peak wavelength is
1550 nm. Since processing parameters are varied greatly during
different experiments, a neutral density filter with 40% light in-
tensity reduction is placed above the photoactive area to prevent
probe optical signal intensity from exceeding the capacity of the
photodiode. The acoustic signal is acquired by a CCP-free field
Knowledge-Based Systems 261 (2023) 110212
Fe Si Zn Ti Al
≤ 0.5 ≤ 0.5 ≤ 0.3 ≤ 0.15 Bal.
standard microphone manufactured by GRAS. The model is 46AE
1/2’’ with a response frequency of 3.15∼51.2 kHz. The sensitivity
of the microphone is 50 mV/Pa with a sensor sensitivity range
from 3.15 Hz to 20 kHz (±2 dB). A data acquisition card NI 9221
with a maximum sampling rate of 800 kHz is adopted to acquire
the photodiode signals while the data acquisition card NI 9218
was applied to capture the acoustic signals. The data acquisition
cards were connected to the personal computer. The acquisition
parameters and schemes can be set by the NI-DAQmax driver and
LabVIEW software. The sensors were fixed on a magnetic holder
installed on the worktable. The distance between sensors and
weldment is about 300 mm at an angle of approximately 45f̊rom
the workpiece surface.
In this work, a synchronous sampling method was developed
to avoid the unsynchronized capturing signals due to the different
traveling speeds of light and sound. Once the laser beam is turned
on, a trigger pulse is produced to start the data acquisition.
Therefore, the start point of the signals of laser-material inter-
action can be captured by microphone and photodiode with a
time difference due to the difference in the traveling speeds. As
shown in Fig. 2, there is a short period without valid data at the
beginning time of the acoustic signal. When inputting data into
machine learning models, the short period without valid data is
removed. Then the acoustic signal and photodiode signals can
be synchronous. In multi-sensor fusion, each sensor has its own
sampling frequency based on its physics and capability. However,
in terms of the sensors with different sampling frequencies, it
is feasible to resample the signal from low sampling frequency
to high sampling frequency [39,40]. In this paper, the acoustic
signals are resampled to the sampling frequency of photodiode
signals using a polyphase antialiasing filter. In this context, the
acoustic signals have the same sampling points as the photodiode
signals.
2.4. Processing parameters of the experiment
To acquire enough acoustic and light signals, fifteen combi-
nations of processing parameters were intentionally selected to
3
L. Cao, J. Li, L. Zhang et al. Knowledge-Based Systems 261 (2023) 110212

Fig. 2. The diagram of a short period with valid data.

Table 2
The laser welding processing parameters.
No. Laser welding Energy density Linear energy
power (W) speed (mm/s) (J/mm3 ) (J/mm)
1 2800 30 118.90 93.33
2 3500 70 63.69 50.00
3 3500 40 111.46 87.50
4 4000 60 84.93 66.67
5 4000 50 101.91 80.00
6 3200 50 81.53 64.00
7 3200 35 116.47 91.43
8 3500 50 89.17 70.00
9 3500 45 99.08 77.78
10 4000 70 72.79 57.14
11 3200 45 90.59 71.11
12 3200 40 101.91 80.00
13 3500 60 74.31 58.33
14 3500 60 74.31 58.33
15 4000 80 63.69 50.00

produce three penetration states after numerous trials and errors.
The length of every weld seam is 100 mm and the defocusing is
zero. The processing parameters are shown in Table 2. The laser
welding experiments were performed at room temperature.
2.5. Preliminary analysis of the welding bead and collected data
In some situations, like welding a closed cavity or pipe, it is
not convenient to distinguish the penetration state according to
the bottom view. Exploring valuable information through the top
morphology or the captured light and acoustic signals to judge
the penetration state is effective and promising. The three typical
penetration states which are incomplete penetration (IP), normal
full penetration (NFP), and excessive penetration (EP) are judged
by the morphologies of the top and back side of the weld seam.
Morphologies of both sides were taken for each seam. As shown
in Fig. 3, the excessive penetration is caused by high linear heat
input which consequently caused a significant level of top surface
concavity and root sagging with the combined actions of recoil
pressure, surface tension, and hydrodynamic pressure. While the
reinforcement and incomplete penetration occurred because of a
lack of heat input, which may lead to serious deformation and
deteriorative mechanical performances [41].
After laser welding, the surface morphologies of each weld
bead were measured by stereoscope microscope and laser scan-
ning confocal microscope. Fig. 4 demonstrates the bead mor-
phologies of the three penetration states measured by the stere-
oscope microscope and laser scanning confocal microscope. The
4
penetration states can be distinguished by the top and bottom
views. The top surface height measured by laser scanning confo-
cal microscope can be used further confirm the penetration states.
Especially, when the underfill occurred, the obvious groove on the
top surface can be observed. And then excessive penetration is
formed. The depth of the groove is about 1450 µm, as shown
in Fig. 4(a). When the incomplete penetration is formed, the
reinforcement height is about 700 µm, as shown in Fig. 4(b).
When the penetration is normal, the reinforcement height is
about 500 µm, as shown in Fig. 4(c). Therefore, the penetration
status can be judged by the groove or reinforcement size.
After observing the top and bottom surfaces, the cross-section
of each weld bead was cut from each workpiece by an electric
cutting machine (STDX600). As illustrated in Fig. 5(a), to reduce
the uncertainty and random errors, three cross-sections were
sampled from each weld bead with a certain interval. A total of
45 metallographic samples were prepared. The penetration and
reinforcement height were measured on an optical microscope
together with measurement software CSM1 after the samples
were polished with suitable abrasive paper and diamond paste
and etched. As illustrated in Fig. 5(b)–(d), the penetration states
can be distinguished from the cross-section. When the pene-
tration depth is smaller than the thickness of the workpiece,
it is incomplete penetration, as shown in Fig. 5(b). When the
workpiece was penetrated with obvious underfill on the top
surface and serious sagging on the bottom surface, it is excessive
penetration, as shown in Fig. 5(d), due to the surface tension
and recoil pressure. When the penetration depth is slightly larger
L. Cao, J. Li, L. Zhang et al. Knowledge-Based Systems 261 (2023) 110212

Fig. 3. The details of the welding bead and collected data for three typical penetration states. (a) the top and bottom views; (b) photodiode signals; (c) microphone
signals.

Fig. 4. The bead appearances measured by stereoscope microscope and laser scanning confocal microscope: (a) EP, (b) IP, and (c) NFP.

5
L. Cao, J. Li, L. Zhang et al.
Fig. 5. The diagrammatic of the penetration conditions. (a) the diagram of the weld bead and the sampling positions, the red arrows present the observation direction,
(b) IP, (c) NFP, and (d) EP.
than the thickness of the workpiece with no underfill on the top
surface, it is normal full penetration, as shown in Fig. 5(c).
The captured acoustic and light data are labeled by the pene-
tration states. The raw photodiode and microphone signals of the
three penetration states are different. However, it is difficult to
distinguish the states just by a large amount of data intuitively.
Data processing and a deep learning model are needed to classify
the penetration states.
3. Proposed method
In this section, the preprocessing of the photodiode and acous-
tic signals is firstly introduced. Then, the overall architecture
and build details of the proposed cross-attention fusion neural
network (CAFNet) are described.
3.1. Data preprocessing
The state-of-the-art (SOTA) researches [32,42] adopted time–
frequency analysis methods, including wavelet packet transform
(WPT) and continuous wavelet transform (CWT), to convert raw
signals into images for the classification of welding defects.
Nonetheless, converting raw signals from the time domain to the
frequency domain may result in the loss of partial information. In
addition, since time–frequency analysis methods need redundant
parameters (i.e., scale and translation parameters, the number
of the wavelet coefficients, etc.) and computing processes, these
methods are time-consuming and demand practical knowledge.
In this section, the sliding window with L2 normalization is used
for one-dimensional (1-D) data preprocessing, as shown in Fig. 6.
To speed up the training process and eliminate the effects of
dimensional difference, a robust L2 normalization technology is
used for data preprocessing, which obtains normalized by the
energy (measured by Euclidean norm) in the raw signal. The
6
Knowledge-Based Systems 261 (2023) 110212
distribution of the raw signal will not be changed after L2 nor-
malization, which means the information is lossless and suitable
for further model training and testing. As shown in Fig. 6, the
distribution of the normalized signal is the same as that of the
raw signal, and only the amplitude (e.g., upper point a and lower
b) is reduced by a factor of ∥x∥2 . The L2 normalization can be
formulated as,
x x
xnormlization = = √ (1)
∥ x∥ 2 xT x
where ∥·∥2 is the Euclidean norm (i.e., 2-norm), x is the raw
signal, xnormlization is the normalized signal.
To prepare a suitable dataset for the deep learning-based
(DL-based) models, this paper uses a sliding window for data
preprocessing, that is, slicing the data samples with overlap.
However, it is necessary to find a suitable length of the sliding
window (i.e., the length of the sample) and shift size to better
classify the different penetration states. For instance, a too-large
length of the sliding window will increase the size of the input
layer in DL-based models, which will increase the computational
cost of the neural network; while a too-small length of the sliding
window will not cover enough features of the vibration signal,
which will lead to the generation of confused samples and further
deteriorates the classification performance. In this research, each
segment has 1024 points with a shifting size of 256, which is
easy to implement normalization and training algorithms for the
DL-based models. Accordingly, the welding dataset consists of
three penetration states is prepared, which include excessive
penetration (EP), normal full penetration (NFP), and incomplete
penetration (IP). The EP, NFP, and IP consist of 922, 796, and
608 samples, respectively, and the entire dataset contains 2326
segments in the time domain.
3.2. Cross-attention fusion neural network (CAFNet)
The proposed CAFNet aimed at utilizing the self-attention
mechanism of the transformer model [34] to interactively cap-
ture photoelectric and acoustic information for effective quality
L. Cao, J. Li, L. Zhang et al. Knowledge-Based Systems 261 (2023) 110212

Fig. 6. Data preprocessing using a sliding window with L2 normalization.

classification without prior preprocessing and feature learning. Then, a non-linear transformation block is used to calculate
Fig. 7 shows the overall architecture of the proposed CAFNet, the output attentive features Aout and P out , which consists of two
which consist of two-branch one-dimensional convolutional neu- linear layers, one layer normalization (i.e., LayerNorm or LN), and
ral networks (1D-CNNs) and one cross-attention (CA) block. The one non-linear activation function. This process can be defined
1D-CNN takes the 1-D signal as input and extracts photoelectric as:
or acoustic features through four convolutional (Conv) blocks. {
Aout = f (CA1 ) + CA1
Each Conv block consists of one 3 × 1 Conv layer with L filters, (4)
one batch normalization (BN) layer, one leaky rectified linear unit P out = f (CA2 ) + CA2
(LReLU), and one max pooling layer with 2 × 1 kernel size. Herein,
where f (·) represents the function of non-linear transformation,
the BN layer is adopted to alleviate the internal covariate shift and
Aout and P out are of the same shape as the corresponding input
accelerate model convergence, while the LReLU with a negative
features. Specifically, we used the Gaussian error linear units
slope of 0.001 is used to activate the normalized feature maps
(GELU) [43] as the activation function rather than LReLU, fol-
from the BN layer.
lowing the SOTA transformer-based models (e.g., GPT-3 [44] and
Different from the basic feature-level fusion models, a CA
BERT [45]). The classifiers using a GELU can be more robust to
block is used for the fusion of intermediate representations be-
noised inputs, which is suitable for the classification of signals
tween two sensors instead of fusing features at the end of two
without any denoising preprocess. The GELU function can be
branches directly. Actually, multi-sensor fusion is performed with
defined by the following formula:
a CA block in each branch after the first two Conv Blocks. In this
way, the feature representations of different sensors are learned 1 √
GELU(x) = x · [1 + erf(x/ 2)] (5)
jointly, which can not only obtain the commonality of photodiode 2
and microphone data but also retain the difference between where x is the input, and erf (·)
photoelectric and acoustic features. For given acoustic features Af ∫ xis the2 Gauss error function, which
can be given by erf (x) = √2π 0 e−t dt.
and photodiode features Pf , the CA block firstly uses embedding
Finally, the attentive features Aout and P out are added to the
blocks with linear projections to obtain Aq , Ak , Av and Pq , Pk , Pv .
output of the second Conv Block in the corresponding branch.
Here, q, k, v represent query, key, and value, respectively. These
The sum features of the two branches will pass through the
tensors can be formulated as,
two Conv Block to generate the final features, then, the final
Aq = Af ⊙ Waq , Ak = Af ⊙ Wak , Av = Af ⊙ Wav
{
features of the two branches are concatenated and fed into the
p
(2) final layer (i.e., fully connected layer with softmax function) for
Pq = Pf ⊙ Wpq , Pk = Pf ⊙ Wk , Pv = Pf ⊙ Wpv
classification.
where W represents the learnable linear weights, ⊙ represents
the operation of element-wise multiplication (i.e., dot-product). 4. Results and discussion
After obtaining Aq , Ak , Av and Pq , Pk , Pv , the attention matrices
CA1 and CA2 are calculated by the cross self-attention function, To ensure the fairness and consistency of the experiments, all
and this process is
( given as)follows: the competing models in our experiments were performed under
⎧ Pq ⊗ ATk the same running environment, which included: Intel Core i7-
⎪
⎪ CA1 = softmax √ ⊙ Av 9700K@3.60 GHz CPU, an Nvidia RTX 2080 GPU, Ubuntu 22.04,
⎪ DAk
Python 3.9.12, CUDA 11.3.1 and PyTorch 1.11.0.
⎨
(3)
( )
Aq ⊗ Pk T
CA = softmax ⊙ P
⎪
⎩ 2 v 4.1. Compared methods
⎪
⎪ √
DPk

where DAk represents the dimension of Ak , DPk represents the
dimension of Pk , ⊗ represents the operation of matrix multipli-
cation, the softmax function is applied to obtain the normalized
weights in the interval [0,1]. Take CA1 for example, the soft-
max function is adopted to calculate the relative importance
of acoustic features by aggregating the scores corresponding to
photoelectric features for each attribute of acoustic features. As a
result, we obtain an attention vector that can be used to highlight
more relevant attributes of acoustic features.
7
To demonstrate the effectiveness and superiority of the pro-
posed CAFNet, different DL-based methods are adopted to an-
alyze the same dataset for comparisons. The main settings and
parameters of compared methods are described as follows:
(1) 1D-CNN with acoustic signal (1D-CNN-A):
First, the one-branch 1D-CNN is implemented, which follows
a similar architecture as one of the branches in CAFNet, as shown
in Fig. 8. Specifically, the pre-processed data from the acoustic
signals are used to train and test its model.
L. Cao, J. Li, L. Zhang et al.
Fig. 7. Network architecture of the proposed CAFNet.
(2) 1D-CNN with photoelectric signal (1D-CNN-P):
To compare the effect of different sensors, the 1D-CNN-P is
implemented as shown in Fig. 9, which takes the pre-processed
data from the photodiode signals as input, and has the same
architecture as 1D-CNN-A.
(3) ResCNN with acoustic signal (ResCNN-A):
Considering the signal-to-image conversion method and im-
proved residual block, the CNN with residual learning (ResCNN) is
implemented, which achieves SOTA results in mechanical fault di-
agnosis [46–48]. Particularly, the acoustic signals are converted to
gray images with 32×32×1 pixels and used to train and test the
model. More details about the ResCNN and the signal-to-image
conversion method can be seen in Ref. [39].
(4) ResCNN with photoelectric signal (ResCNN-P):
Similarly, the ResCNN-P is implemented to analyze the ef-
fect of the different sensors, which has an identical skeleton to
ResCNN-A.
(5) 1D-CNN with acoustic and photoelectric signal (1D-CNN-
AP):
To show the effect of the cross-attention scheme, the 1D-
CNN-AP is implemented where the CA block is removed from the
proposed CAFNet, as shown in Fig. 10. The 1D-CNN-AP focus on
fusing features at the end of two branches directly.
(6) ResCNN with acoustic and photoelectric signal (ResCNN-
AP):
8
Knowledge-Based Systems 261 (2023) 110212
Actually, the original ResCNN in Ref. [39] is based on multisen-
sory fusion and CNN, which uses principal component analysis to
convert multisensory signals into red-green-blue images. How-
ever, this research only considers two sensors (i.e., microphone
and photodiode sensors). To further examine the fusion abil-
ity of the proposed CAFNet, the ResCNN-AP is implemented on
fusing features at the end of two branches (i.e., ResCNN-A and
ResCNN-P) directly, which same as 1D-CNN-AP.
Due to the difference in model complexity, different opti-
mizers with different initial learning rates are used to optimize
the cross-entropy loss function of compared methods. According
to the grid search tuning, the stochastic gradient descent (SGD)
optimizer with the initial learning rate of 3 × 10−2 is adopted to
optimize the loss function of 1D-CNN-A, 1D-CNN-P, 1D-CNN-AP,
and the proposed CAFNet, while the adaptive moment estimation
(Adam) optimizer with the initial learning rate of 3 × 10−3 is
adopted to optimize the loss function of ResCNN-A, ResCNN-P and
ResCNN-AP. In addition, the cosine annealing scheduler is utilized
to decay the learning rate for each batch, as follows,
( ( ))
(ηinitial − ηmin ) 1 + cos TTmax
cur
π
ηt = ηmin + (6)
2
where ηt is the current learning rate, ηmin is the minimum learn-
ing rate, ηinitial is the initial learning rate, Tcur is the number of
L. Cao, J. Li, L. Zhang et al.
Fig. 8. Network architecture of 1D-CNN-A.
Fig. 9. Network architecture of 1D-CNN-P.
Fig. 10. Network architecture of 1D-CNN-AP.
current epochs, and Tmax is the number of epochs in each restart.
In this paper, the training phase stopped after 200 epochs, while
Tmax and ηmin is fixed as 10 and 3 × 10−5 , respectively.
4.2. Classification performance
Unless otherwise stated, all experiments were performed 10
times (i.e., with different random seeds) to avoid contingency
during the test, and the average values were regarded as the
final results of classification for analysis. Also, the dataset was
randomly split into subsets of training, validation, and testing.
The ratios of each subset were defined as 80%, 10%, and 10%,
which were commonly used. The comparison bar and classifica-
tion performance of different methods are shown in Fig. 11 and
Table 3, respectively. Some interesting observations are listed as
follows:
(1) Among the single-sensor-based methods (i.e., 1D-CNN-A,
1D-CNN-P, ResCNN-A, ResCNN-P), the model using acoustic
signal produces better testing accuracy than that using the
photoelectric signal under the same network architecture,
which confirms that using information from sole sensor
might result in low classification accuracy. On the other
hand, the ResCNN-based method produces a more satis-
fying classification performance with an average testing
accuracy of more than 90%. However, the signal-to-image
conversion method and residual learning in ResCNN will
increase the space and time complexity of the model. As
can be seen from Table 3, the space complexity (measured
by the number of model parameters) of the ResCNN-based
method is about 2.83 times more than that of the 1D-CNN-
based method, while the time complexity (measured by the
9
Knowledge-Based Systems 261 (2023) 110212
test time per sample) is about 1.84 times longer than that
of the 1D-CNN-based method.
(2) Compared with the multi-sensor-based method except for
1D-CNN-AP, the single-sensor-based methods (i.e.,
1D-CNN-A, 1D-CNN-P, ResCNN-A, ResCNN-P) obtain infe-
rior average testing accuracy, because they only utilize sole
information from microphone or photodiode sensor and
fail to obtain the common information from different sen-
sors for better classification. Interestingly, ResCNN-A yields
improvements of 4.59% in terms of mean testing accuracy
compared with the 1D-CNN-AP, which demonstrates that
the complex and stronger feature extractor can still obtain
superior classification performance even using information
from one of the sensors. However, it’s the space and time
complexity are larger.
(3) As can be seen from Fig. 11, compared with directly per-
forming feature-level fusion at the end of the model (i.e.,
1D-CNN-AP and ResCNN-AP), introducing the cross-
attention mechanism into the model improves the clas-
sification performance. In the three multi-sensor-based
methods, the best performance is achieved by the pro-
posed CAFNet, with 99.73% ± 0.37% accuracy, while the
performance of other methods only has an accuracy of
93.39% ± 1.96% and 99.13% ± 0.67%, respectively. Al-
though the ResCNN-AP achieves satisfactory mean value
and standard deviation in terms of testing accuracy, the
space complexity of the ResCNN-AP is about 1.89 times
more than that of the proposed CAFNet while the time
complexity is about 1.40 times longer than that of the
proposed CAFNet. The results showed that the proposed
CAFNet can make a tradeoff between high accuracy and
L. Cao, J. Li, L. Zhang et al.
Fig. 11. The comparisons result of different methods.
Table 3
Classification performance of different methods.
Method Testing accuracy
(Mean ± Std %)
1D-CNN-A 85.36 ± 2.06
1D-CNN-P 81.97 ± 3.92
Single-sensor-based
ResCNN-A 97.98 ± 1.51
ResCNN-P 91.97 ± 1.75
1D-CNN-AP 93.39 ± 1.96
Multi-sensor-based ResCNN-AP 99.13 ± 0.67
CAFNet 99.73 ± 0.37
relatively low computational cost. Accordingly, the pro-
posed method using cross-attention fusion can effectively
and efficiently handle information from multiple sensors.
Fig. 12 shows the multi-class confusion matrix obtained from
compared methods on one of the ten trials. The actual labels
are given on the y-axis and the predicted labels are shown on
the x-axis. The diagonal terms of the confusion matrix denote
the correctly classified quality types in terms of category per-
centages, while the off-diagonal terms denote false classification.
Notably, the phenomenon of data imbalance exists in our dataset
(i.e., the sample size of the three types is different, and the
sample size of IP is 44 and 42 less than EP and NFP, respectively),
which is common in real industry and significantly deteriorates
the model performances. As shown in Fig. 12(c), the 1D-CNN-
P achieves the worst multi-class results where the NFP has the
highest misclassification rate (24.49% of IP and 16.13% of EP are
misclassified as NFP). The corresponding confusion matrix of the
proposed CAFNet is given in Fig. 12(a), which demonstrates that
the EP has the lowest testing accuracy of 98.92% (the 1.08% of
EP is misclassified as NFP), while the NFP and IP have a test
accuracy of 100%. For the fine-grained classification under the
data imbalance, the proposed CAFNet is significantly better than
the compared methods.
To further show the feature learning of the proposed CAFNet,
a stochastic neighbor embedding (t-SNE) [49] is adopted to vi-
sualize the features of different quality states in 2-D spaces.
The goal of the t-SNE algorithm is to perform dimensionality
reduction on high-dimensional data and obtain the similarities
between data points in low-dimensional space. Fig. 13 shows the
t-SNE representations (where the dimension 1 and 2 of features
are shown on the x-axis and y-axis, respectively) on the testing
subset obtained at different structures of the proposed CAFNet,
namely, Input, Before Cross-attention Block, After Cross-attention
Block, and Before Fully connected with Softmax. As shown in
Fig. 13, the input data points from different states are highly
10
Knowledge-Based Systems 261 (2023) 110212
Testing time per Model
sample (ms) params (M)
0.85 0.06
0.87 0.06
1.55 0.17
1.60 0.17
1.45 0.12
3.14 0.34
2.25 0.18
overlapped together, and unable to be distinguished. After two
Conv Blocks (Before Cross-attention Block), the observations of
some data points are able to be distinguished but not clearly
separated. Although some classes are not clearly separated after
Cross-attention Block, the clustering of data points is much better
and data points from IP are almost split. It can be concluded
that cross-attention can strengthen the feature representations of
different sensors. Before the fully connected layer with Softmax,
the data points are completely split and divided into different
groups. Hence, the features extracted by the developed CAFNet
will be utilized to conveniently classify the labels of the different
quality states.
4.3. Limited training samples
In practice, the limited training samples hinder the develop-
ment of the DL-based quality monitoring method. Therefore, six
experiments using different training ratios were performed to
further examine the effectiveness and robustness of the proposed
CAFNet method. Table 4 presents the testing accuracy with a lim-
ited training ratio of different methods. The bold values in Table 4
represent not only the proposed method but also the best results.
From the classification results shown in Table 4, the proposed
CAFNet can accurately classify welding penetration states with a
limited training ratio, with an average testing accuracy of 94.34%,
outperforming all other methods. Particularly, the average test-
ing accuracies of 1D-CNN-A, 1D-CNN-P using raw single-signal
are 73.98% and 73.21%, respectively, inferior to that of the pro-
posed CAFNet. And the performance of 1D-CNN-AP based on
feature-level fusion is still inferior to the proposed CAFNet, with
an average testing accuracy of only 86.14%. Surprisingly, the
ResCNN-A provides satisfactory classification performance with
an average testing accuracy of 89.84%, which demonstrates that
ResCNN using only one microphone sensor is still compatible
with other multi-sensor-based methods (i.e., ResCNN-AP and 1D-
CNN-AP). Moreover, the proposed CAFNet achieve satisfactory
L. Cao, J. Li, L. Zhang et al. Knowledge-Based Systems 261 (2023) 110212

Fig. 12. Multi-class confusion matrix of different methods on one trial (random Seed: 56351).

Fig. 13. t-SNE-based features visualization of the proposed CAFNet.

Table 4
Testing accuracy with limited training ratio of different methods.
Method Testing accuracy (Mean ± Std %) Average
10% 20% 30% 40% 50% (Mean %)

1D-CNN-A 67.45 ± 1.04 71.55 ± 1.68 74.28 ± 0.72 77.54 ± 0.93 79.09 ± 1.37 73.98
1D-CNN-P 66.96 ± 1.74 70.92 ± 1.10 74.10 ± 1.06 76.31 ± 1.17 77.76 ± 1.77 73.21
ResCNN-A 79.67 ± 5.97 88.39 ± 1.90 91.50 ± 1.31 94.58 ± 1.24 95.05 ± 1.17 89.84
ResCNN-P 46.27 ± 5.70 58.05 ± 7.23 68.45 ± 8.09 80.80 ± 2.90 86.71 ± 1.55 68.06
1D-CNN-AP 78.77 ± 1.12 84.28 ± 0.64 87.74 ± 1.48 89.62 ± 1.41 90.27 ± 1.70 86.14
ResCNN-AP 76.62 ± 3.19 83.52 ± 2.48 88.88 ± 1.86 93.73 ± 1.15 94.47 ± 1.40 87.44
CAFNet 86.70 ± 0.46 91.67 ± 0.57 96.89 ± 0.41 97.94 ± 0.36 98.49 ± 0.39 94.34

The bold values represent not only the performance of the proposed method but also the best results.

performance with 86.70% ± 0.46% even under a 10% training The visualization of comparative results in different training
ratio, outperforming all other methods. ratios is depicted in Fig. 14. As shown in Fig. 14, the overall
11
L. Cao, J. Li, L. Zhang et al.
Fig. 14. Visualization of comparative results in different training ratios.
performance improves in all tested methods with an increase
in the training ratio, which means the DL-based methods are
sensitive to the number of training samples and will provide
unsatisfactory classification performance with a limited training
ratio. In a word, the proposed CAFNet achieves good performance
in six experiments and outperforms all other methods. There-
fore, the effectiveness and robustness of the proposed CAFNet
method with different limited training ratios are validated in this
section.
5. Conclusion and the future work
This article presents a novel penetration state monitoring
method based on multi-sensing signals and CAFNet during laser
welding. A dataset with three penetration states measured by
photodiode and microphone sensors is created and used to val-
idate the effectiveness of the proposed method. The conclusions
can be summarized as follows:
(1) Laser welding equipment and a signals monitoring system
including photodiode and microphone sensors were built, and
a series of experiments were conducted to collect enough data.
Three typical penetration states (i.e., IP, NFP, and EP) are judged
by qualitative and quantitative criteria, including the surface mor-
phologies of weld bead measured by stereoscope microscope and
laser scanning confocal microscope, the cross-section of weld
bead measured by an optical microscope with measurement soft-
ware CSM1;
(2) Compared with the SOTA researches of welding moni-
toring, the proposed method does not require need redundant
parameters and computing processes for data preprocessing. A
robust 1-D data preprocessing approach was developed to elimi-
nate the effects of dimensional differences and prepare a suitable
dataset, which is based on the sliding window and L2 normaliza-
tion;
(3) An improved CA block was designed and combined with
1D-CNN to construct the CAFNet, which can significantly increase
the accuracy of the CAFNet at the expense of a small amount of
computational cost. With the better interactive ability of feature
extraction and fusion, the proposed CAFNet would improve the
final identification performance on welding quality monitoring;
(4) The proposed CAFNet was validated to classify the welding
penetration states on a laboratory dataset under data imbalance.
For the full dataset, the proposed CAFNet achieves a mean value
of 99.73% and a standard deviation of 0.37% in terms of testing
accuracy, which outperforms other compared models. The results
12
Knowledge-Based Systems 261 (2023) 110212
showed that the proposed CAFNet takes only 2.25 ms to classify
one sample with 1024 data points, which further demonstrates
the proposed method can make a tradeoff between high accuracy
and relatively low computational cost. Additionally, when utiliz-
ing a dataset with limited training samples, the proposed CAFNet
achieved the highest average testing accuracy of 94.34%, which
shows that the proposed method has stronger robustness than
other methods.
Although the experimental results demonstrate and validate
the effectiveness of CAFNet when applied to multi-sensor, unbal-
anced, and limited labeled samples for welding quality monitor-
ing, there are still some limitations to be explored in future work:
(1) For complex monitoring systems, data come from various
sensors and are mostly non-structured, multi-modal and hetero-
geneous which make the model much more complex. Further
research is required in the future to leverage heterogeneous infor-
mation in CAFNet while not reducing the training efficiency; (2)
In engineering scenarios, the welding robot usually works with
optimal processing parameters and incomplete or excessive pen-
etration states are rare. Thus, it is hard to obtain sufficient data
of welding quality from engineering scenarios directly to support
the training of CAFNet. Further study on the application of few-
shot and zero-shot learning in the data-scarce industry practice
will be carried out; (3) This paper only considers three typical
penetration states, in the future, more comparative experiments
with state-of-the-art welding monitoring of other defects will be
investigated.
CRediT authorship contribution statement
Longchao Cao: Data Curation, Funding acquisition, Conceptu-
alization, Writing – original draft. Jingchang Li: Writing – review
& editing, Validation. Libin Zhang: Investigation, Funding acqui-
sition. Shuyang Luo: Visualization. Menglei Li: Formal analysis.
Xufeng Huang: Supervision, Methodology, Software, Writing –
review & editing.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared
to influence the work reported in this paper.
Data availability
Data will be made available on request.
L. Cao, J. Li, L. Zhang et al.
Acknowledgments
This work was supported by the National Natural Science
Foundation of China (NSFC) under Grant No. 52105446, Knowl-
edge Innovation Program of Wuhan-Shuguang Project, PR China
under Grant No. 2022010801020252, and NSFC under Grant No.
62204178. The authors would like to thank Prof. Ping Jiang and
Shaoning Geng, PhD. for their providing the laser welding equip-
ments.
References
[1] N. Kashaev, V. Ventzke, G. Cam, Prospects of laser beam welding and fric-
tion stir welding processes for aluminum airframe structural applications,
J. Manuf. Process. 36 (2018) 571–600.
[2] R.S. Xiao, X.Y. Zhang, Problems and issues in laser beam welding of
aluminum-lithium alloys, J. Manuf. Process. 16 (2014) 166–175.
[3] G. Cam, G. Ipekoglu, Recent developments in joining of aluminum alloys,
Int. J. Adv. Manuf. Technol. 91 (2017) 1851–1866.
[4] F. Dausinger, P. Berger, H. Hügel, Laser welding of aluminum alloys:
Problems, approaches for improvement and applications, in: International
Congress on Applications of Lasers & Electro-Optics, Arizona, USA, 2002,
287255.
[5] Z. Malekshahi Beiranvand, F. Malek Ghaini, H. Naffakh Moosavy, M. Sheikhi,
M.J. Torkamany, M. Moradi, The relation between magnesium evaporation
and laser absorption and weld penetration in pulsed laser welding of
aluminum alloys: Experimental and numerical investigations, Opt. Laser
Technol. 128 (2020).
[6] W. Cai, J. Wang, P. Jiang, L. Cao, G. Mi, Q. Zhou, Application of sensing
techniques and artificial intelligence-based methods to laser welding real-
time monitoring: A critical review of recent literature, J. Manuf. Syst. 57
(2020) 1–18.
[7] M.F.M. Yusof, M. Ishak, M.F. Ghazali, Acoustic methods in real-time welding
process monitoring: Application and future potential advancement, J.
Manuf. Sci. Eng. 15 (2021) 8490–8507.
[8] X. Gao, D. You, S. Katayama, The high frequency characteristics of laser
reflection and visible light during solid state disk laser welding, Laser Phys.
Lett. 12 (2015).
[9] P. De Bono, C. Allen, G. D’Angelo, A. Cisi, Investigation of optical sensor
approaches for real-time monitoring during fibre laser welding, J. Laser
Appl. 29 (2017).
[10] H.-H. Chu, Z.-Y. Wang, A vision-based system for post-welding quality
measurement and defect detection, Int. J. Adv. Manuf. Technol. 86 (2016)
3007–3014.
[11] Z. Zhang, B. Li, W. Zhang, R. Lu, S. Wada, Y. Zhang, Real-time penetration
state monitoring using convolutional neural network for laser welding of
tailor rolled blanks, J. Manuf. Syst. 54 (2020) 348–360.
[12] L. Yang, J. Fan, B. Huo, E. Li, Y. Liu, A nondestructive automatic defect
detection method with pixelwise segmentation, Knowl. Based. Syst. 242
(2022).
[13] X. Dong, C.J. Taylor, T.F. Cootes, Automatic aerospace weld inspection using
unsupervised local deep feature learning, Knowl. Based. Syst. 221 (2021).
[14] W. Huang, R. Kovacevic, A neural network and multiple regression method
for the characterization of the depth of weld penetration in laser welding
based on acoustic signatures, J. Intell. Manuf. 22 (2009) 131–143.
[15] S. Lee, S. Ahn, C. Park, Analysis of acoustic emission signals during laser
spot welding of SS304 stainless steel, J. Mater. Eng. Perform. 23 (2013)
700–707.
[16] M.F.M. Yusof, M. Ishak, M.F. Ghazali, Feasibility of using acoustic method
in monitoring the penetration status during the pulse mode laser welding
process, IOP Conf. Ser.: Mater. Sci. Eng. 238 (2017).
[17] H. Köhler, C. Thomy, F. Vollertsen, Contact-less temperature measurement
and control with applications to laser cladding, Weld. World 60 (2015)
1–9.
[18] N. Chandrasekhar, M. Vasudevan, A.K. Bhaduri, T. Jayakumar, Intelligent
modeling for estimating weld bead width and depth of penetration from
infra-red thermal images of the weld pool, J. Intell. Manuf. 26 (2013)
59–71.
[19] Z. Chen, X. Gao, Detection of weld pool width using infrared imaging
during high-power fiber laser welding of type 304 austenitic stainless steel,
Int. J. Adv. Manuf. Technol. 74 (2014) 1247–1254.
[20] L. Kong, X. Peng, Y. Chen, P. Wang, M. Xu, Multi-sensor measurement and
data fusion technology for manufacturing process monitoring: A literature
review, Int. J. Extreme Manuf. 2 (2020).
13
Knowledge-Based Systems 261 (2023) 110212
[21] B. Xue, B. Chang, D. Du, Monitoring of high-speed laser welding process
based on vapor plume, Opt. Laser Technol. 147 (2022).
[22] W. Cai, P. Jiang, L. Shu, S. Geng, Q. Zhou, Real-time laser keyhole welding
penetration state monitoring based on adaptive fusion images using
convolutional neural networks, J. Intell. Manuf. (2021).
[23] T. Liu, J. Bao, H. Zheng, J. Wang, C. Yang, J. Gu, Learning semantic-specific
visual representation for laser welding penetration status recognition, Sci.
China Technol. Sci. 65 (2022) 347–360.
[24] G. Chianese, P. Franciosa, J. Nolte, D. Ceglarek, S. Patalano, Characterization
of photodiodes for detection of variations in part-to-part gap and weld
penetration depth during remote laser welding of copper-to-steel battery
tab connectors, J. Manuf. Sci. Eng. 144 (2022).
[25] K. Lee, S. Kang, M. Kang, S. Yi, C. Kim, Estimation of Al/Cu laser weld
penetration in photodiode signals using deep neural network classification,
J. Laser Appl. 33 (2021) 042009.
[26] Z. Luo, W. Liu, Z. Wang, S. Ao, Monitoring of laser welding using source
localization and tracking processing by microphone array, Int. J. Adv.
Manuf. Technol. 86 (2015) 21–28.
[27] X. Xiao, X. Liu, M. Cheng, L. Song, Towards monitoring laser welding
process via a coaxial pyrometer, J. Mater Process. Technol. 277 (2020).
[28] Y. Zhang, D. You, X. Gao, C. Wang, Y. Li, P.P. Gao, Real-time monitoring of
high-power disk laser welding statuses based on deep learning framework,
J. Intell. Manuf. 31 (2020) 799–814.
[29] Y. Zhang, D. You, X. Gao, N. Zhang, P.P. Gao, Welding defects detection
based on deep learning with multiple optical sensors during disk laser
welding of thick plates, J. Manuf. Syst. 51 (2019) 87–94.
[30] Y. Zhang, D. You, X. Gao, S. Katayama, Online monitoring of welding
status based on a DBN model during laser welding, Engineering 5 (2019)
671–678.
[31] S. Shevchik, T. Le-Quang, B. Meylan, F.V. Farahani, M.P. Olbinado, A. Rack,
G. Masinelli, C. Leinenbach, K. Wasmer, Supervised deep learning for real-
time quality monitoring of laser welding with X-ray radiographic guidance,
Sci. Rep. 10 (2020) 3389.
[32] D. Ma, P. Jiang, L. Shu, S. Geng, Multi-sensing signals diagnosis and CNN-
based detection of porosity defect during Al alloys laser welding, J. Manuf.
Syst. 62 (2022) 334–346.
[33] J. Li, Y. Zhang, W. Liu, B. Li, X. Yin, C. Chen, Prediction of penetration based
on plasma plume and spectrum characteristics in laser welding, J. Manuf.
Process. 75 (2022) 593–604.
[34] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L.
Kaiser, I. Polosukhin, Attention is all you need, in: Proceedings of the 31st
International Conference on Neural Information Processing Systems, Long
Beach, CA, USA, 2017, pp. 6000–6010.
[35] G. Wang, Q. Zhai, H. Liu, Cross self-attention network for 3D point cloud,
Knowl.-Based Syst. 247 (2022) 108769.
[36] A. Prakash, K. Chitta, A. Geiger, Multi-modal fusion transformer for end-
to-end autonomous driving, in: Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, Virtual, 2021, pp. 7077–7087.
[37] D. Xiao, C. Qin, J. Ge, P. Xia, Y. Huang, C. Liu, Self-attention-based adaptive
remaining useful life prediction for IGBT with Monte Carlo dropout,
Knowl.-Based Syst. 239 (2022) 107902.
[38] L. Cao, Q. Zhou, H. Liu, J. Li, S. Wang, Mechanism investigation of the
influence of the magnetic field on the molten pool behavior during laser
welding of aluminum alloy, Int. J. Heat Mass Transfer 162 (2020).
[39] T. Xie, X. Huang, S.-K. Choi, Intelligent mechanical fault diagnosis using
multi-sensor fusion and convolution neural network, IEEE T. Ind. Inform.
18 (2021) 3213–3223.
[40] T. Hu, T. Tang, R. Lin, M. Chen, S. Han, J. Wu, A simple data augmentation
algorithm and a self-adaptive convolutional architecture for few-shot fault
diagnosis under different working conditions, Measurement 156 (2020)
107539.
[41] P. Norman, J. Karlsson, A. Kaplan, Monitoring undercut, blowouts and
root sagging during laser beam welding, in: WLT Conference on Lasers
in Manufacturing, Munich, Germany, 2009.
[42] R. Miao, Z. Shan, Q. Zhou, Y. Wu, L. Ge, J. Zhang, H. Hu, Real-time
defect identification of narrow overlap welds and application based on
convolutional neural networks, J. Manuf. Syst. 62 (2022) 800–810.
[43] D. Hendrycks, K. Gimpel, Gaussian error linear units (GELUs), 2016, arXiv
preprint arXiv:1606.08415.
[44] T. Brown, B. Mann, N. Ryder, M. Subbiah, J.D. Kaplan, P. Dhariwal, A.
Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss,
G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter,
C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner,
S. McCandlish, A. Radford, I. Sutskever, D. Amodei, Language models are
few-shot learners, in: Advances in Neural Information Processing Systems,
Virtual, 2020, pp. 1877–1901.
L. Cao, J. Li, L. Zhang et al.
[45] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep
bidirectional transformers for language understanding, in: Proceedings
of the 2019 Annual Conference of the North American Chapter of the
Association for Computational Linguistics, Minneapolis, USA, 2019, pp.
4171–4186.
[46] B. Zhao, X. Zhang, H. Li, Z. Yang, Intelligent fault diagnosis of rolling bear-
ings based on normalized CNN considering data imbalance and variable
working conditions, Knowl.-Based Syst. (2020) 105971.
14
Knowledge-Based Systems 261 (2023) 110212
[47] S. Luo, X. Huang, Y. Wang, R. Luo, Q. Zhou, Transfer learning based on
improved stacked autoencoder for bearing fault diagnosis, Knowl.-Based
Syst. 256 (2022) 109846.
[48] X. Li, H. Jiang, R. Wang, M. Niu, Rolling bearing fault diagnosis using
optimal ensemble deep transfer network, Knowl.-Based Syst. 213 (2021)
106695.
[49] L. Van der Maaten, G. Hinton, Visualizing data using t-SNE, J. Mach. Learn.
Res. 9 (2008).