Liu 2020

This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2985715, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 1
Data-Driven Deep Learning for Signal Classification

in Industrial Cognitive Radio Networks
Mingqian Liu, Member, IEEE, Guiyue Liao, Nan Zhao, Senior Member, IEEE, Hao Song,
and Fengkui Gong, Member, IEEE
Abstract—With the proliferation of mobile access services frequency bands [4]. Thus, industrial cognitive radio networks
and wireless devices, spectrum resources have been increasingly (ICRNs), aimed at achieving effective spectrum sharing and
becoming scarce. Industrial wireless sensor networks (IWSNs) high spectrum efficiency, have been widely studied [5]-[7].
may have to share frequency bands with other systems and
suffer from considerable interference. To address that, industrial Wireless signal classification is a fundamental and critical
cognitive radio networks (ICRNs) were developed for effective technologies in ICRNs, which is used to identify modulation
spectrum sharing, where signal classification is a fundamental information of wireless signals and have many applications in
and important technology, especially for industrial wireless de- industries, such as signal demodulation, suspicious transmis-
vices, which need to identify suspicious transmissions. In this sion monitor, anomaly detection, and interference localization
paper, a novel framework of signal intelligent classification is
proposed based on deep learning networks in ICRNs. In the pro- [8], [9]. On shared spectrum, some wireless devices may
posed framework, wireless signals will be pre-processed first by possess a higher priority, which should be deemed as priority
Choi-Williams distribution (CWD) time-frequency analysis and users and protected from harmful interference, while devices
represented by two-dimensional time-frequency images. Then, with a lower priority should operate as secondary users [10].
features of wireless signals are extracted through stack hybrid In ICRNs, a secondary user is required to perform accurate
autoencoders (SHAE). To accommodate general cases, we design
multiple signal classification methods, including unsupervised, wireless signal classification, as it need to identify signals from
semi-supervised and supervised methods, which are processed by primary users for primary user protections, and meanwhile
Softmax function, semi-supervised linear discriminant function, identify signals of itself for desired signal demodulation as
and Fisher discriminant function, respectively. Finally, simulation well as signals from other second users for suspicious trans-
studies are conducted and the corresponding simulation results mission monitor and interference localization. However, in
show that the proposed framework is able to learn hierarchical
features accurately and achieve excellent signal classification industries, it is very challenging to realize accurate wireless
performance. Moreover, it can effectively overcome the negative signal classification due to complicated wireless environments
impact caused by feature parameters uncertainty. in industries, since a secondary user may have to carry out
Index Terms—Choi-Williams distribution, deep learning net- classification for mixed signals from primary user and other
works, industrial cognitive radios networks, signal classification, unexpected secondary users. Therefore, in this paper, we focus
stack hybrid autoencoders on studying high-performance wireless signal classification
methods in ICRNs.
Existing signal classification methods can be classified into
I. I NTRODUCTION
two main categories, statistical pattern recognition and deci-
NDUSTRIAL spectrum resources are becoming increasing- sion theory [11]. The identification method of decision theory
I ly scarce with the improvement of industrial technology. To
cope with the issue, wireless communication systems in indus-
is based on hypothesis testing theory, which can be regarded as
a problem of multiple hypothesis testing. Due to the fact that
tries have to greatly extend their spectrum to enhance system priori information is very hard to obtain in signal classification,
capacity and meet the requirement of massive industrial device pattern recognition methods are commonly adopted in practice
connections [1]-[3]. Unfortunately, wireless communication [12]-[13]. In general, the statistical pattern recognition method
systems in industries are unlikely to obtain adequate licensed consists of three components, including signal preprocessing,
feature extraction and classification. Unfortunately, as an es-
This work was supported in part by the National Natural Science Foundation sential process of pattern recognition methods, the derivation
of China under Grant 61501348, Grant 61801363 and Grant 61871065, in part
by the Shaanxi Provincial Key Research and Development Program under of the likelihood function is complicated and computationally
Grant 2019GY-043, in part by the China Postdoctoral Science Foundation intensive. As a result, pattern recognition methods are difficult
under Grant 2017M611912, in part by the 111 Project under Grant B08038, to extract features and high-dimensional features may easily
and in part by the China Scholarship Council under Grant 201806965031.
(Corresponding author: Nan Zhao.) incur dimensional disasters in a classifier. When processing
M. Liu, G. Liao and F. Gong are with the State Key Laboratory of Integrated with high ambiguity signals, a large number of unpredictable
Service Networks, Xidian University, Shaanxi, Xi’an 710071, China (e-mail: results will be produced.
mqliu@mail.xidian.edu.cn; litchi00000@163.com; fkgong@xidian.edu.cn).
N. Zhao is with the School of Information and Communication Engineer- Recently, machine learning and deep learning technologies,
ing, Dalian University of Technology, Dalian 116024, P. R. China. (email: as powerful tools for data analysis and features extraction, have
zhaonan@dlut.edu.cn). been applied in the field of wireless signal classification [14].
H. Song is with the Bradley Department of Electrical and Comput-
er Engineering, Virginia Tech, Blacksburg, VA, 24060, USA (e-mail: Nowadays, in the context of Industry 4.0, signal classification
haosong@vt.edu). in industrial equipments are required to be intelligent [15].
1551-3203 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on June 14,2020 at 03:16:03 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2985715, IEEE
Chen et al. used the same residual neural network model to in unsupervised methods, the time-frequency character-
identify wireless signals. However, with this method wire- istics of the Choi-Williams distribution (CWD) is used
less signal data cannot be fully exploited to perform signal to process wireless signals, which not only has the
classification, which restricts classification performance [16]. least cross-term interference, high resolution and high
The authors in [17] proposed a wireless signal classification classification accuracy for signals at different times or
method using high order spectrum and machine learning. frequencies, but also reduces the effect of signal noise.
Because of high dimensional characteristics of signal data, • Through the stack hybrid autoencoders (SHAE), wireless
the signal data volume were considerably large, causing high signals’ features are automatically extracted to overcome
complexity. In [18], the authors adopted dropout instead of the disadvantages of traditional manual feature extraction-
pooling operation and designed a new type of convolution- s. In this way, overfitting can be effectively supressed,
al neural network (CNN) with a constellation diagram to the amount of data can be reduced and implicit features
identify wireless signals. Unfortunately, the performance of can be extracted. The proposed deep learning network
signal classification still does not reach a preferable level. has good generalization ability, which is able to extract
Rajendran et al. proposed a classification model with the short- more abstract and sparse features, thereby improving the
time memory network (LSTM), which did not require expert robustness of the network and the accuracy classification
features such as high-order cyclic moments. This method can of signals.
only adapt to the data with standardized bandwidth parameters, • Integrate the idea of fish linear discrimination into the
while actual signals may have various bandwidth [19]. The SHAE and add the distance measurement function be-
authors in [20] studied the adaptive problem of convolutional tween classes and the distance measurement function
neural network to complex time-domain signals. Although within the class to the loss function of each layer of the
the considered neural network has been proven to possess SHAE. As a result, the functions learned from the SHAE
a strong nonlinear fitting ability and solid robustness, it is have smaller differences in the same categories, while
mainly aimed at the classification of large samples, while the differences in different categories would be more
small samples of signals may encounter severe over-learned significant, which is more conducive to classification.
and over-fitted issues in signal classification. The authors in • By combining SHAE and semi-supervised linear discrim-
[21] extracted the characteristic values of the wireless signals inant, only a small amount of sample set with labels are
and converted them into two-dimensional images. Then, signal needed for nonlinear feature extraction and classification.
classification is carried out using CNN. However, only limited With the aid of insufficient data samples of wireless signal
types of signals can be identified by this methos. Yar et with label, the semi-supervised classification method can
al. utilized the short-time Fourier transform to process the enhance the generalization performance and improve the
signals and used CNN to identify the signals [22]. In [23], classification accuracy leveraging a large amount of un-
spectral characteristics and high-order cumulants of signals labeled wireless signal data that could be acquired easily.
were used as characteristics input into an adaptive classifier. The remainder of this paper is organized as follows. In
The shortcoming of this method is obvious that its stability is Section II, the system that we consider is introduced. In
poor when signal to noise ratio (SNR) is in a low region. Section III, the intelligence representation of wireless signals
Huang et al. proposed a new compressed CNN, in which is shown. The intelligent classification methods of wireless
images of multiple constellations are used as the input data signals based on deep learning networks are proposed in
and classification decisions are made by CNN with small Section IV. Simulation studies are conducted to verify the
convolutional kernel. Furthermore, a new compression loss classification performance in Section V. Finally, Section VI
is designed to strengthen the compactness within classes and concludes the whole paper.
the separability between classes. Unfortunately, this method
has complicated training processes with large training time
II. S YSTEM M ODEL
and its performance cannot adapt to low SNR environments
[24]. In [25], phase components and orthogonal components of A ICRN, consisting of multiple primary user devices and
received baseband signals are employed as extracted features. secondary user devices, is considered in this paper, as shown
Additionally, CNN and recursive neural networks (RNN) are in Fig. 1 [7]. In such an network, a wireless device has to co-
utilized to develop classifiers, classifying six types of signals in exist with other devices on the same frequency bands. To fulfill
additive Gaussian white noise (AWGN) channel and Rayleigh multiple tasks, such as suspicious transmissions detection,
channel. However, the pooling layer of CNNs results in primary users protection, and desired signal demodulation,
the loss of signal features, make it have poor classification secondary user devices need to carry out signal classification.
performance at low SNR. Let s(k) denotes the received wireless signals of secondary
user devices, which can be expressed as [14]
By exploiting a large amount of unlabeled data and high-
dimensional characteristics of wireless signals, in this paper a s(k) = x(k) + σw · w(k), k = 0, 1, ..., N − 1, (1)
novel signal classification framework based on deep learning
networks is proposed in ICRNs. The main innovative contri- where w(k), σw , N and x(k) are the gaussian white noise, the
butions of the proposed framework is summarized as follows: noise level, the length of the signal, and the transmitted wire-
less signal, respectively. The transmitted wireless signals may
• To eliminate the redundancy between related information be comprised of constant wave (CW) signal, linear frequency
(a)BPSK (b)CW
Primary User Link
(c)FSK (d)FSK-BPSK
Primary User Primary User

Sensing Link
(e)LFM (f)LFM-BPSK
Secondary User (g)NLFM (h)QPSK
Wireless Intelligent Intelligent

Signals Representation Classification
Fig. 1. System model for signal classification in industrial cognitive radio Fig. 2. Two-dimensional images of wireless signals by CWD.
networks.
III. I NTELLIGENT R EPRESENTATION OF W IRELESS

modulation (LFM) signal, frequency shift keying (FSK) signal, S IGNALS
binary phase shift keying (BPSK) signal, quadrature phase
shift keying (QPSK) signal, non-linear frequency modulation CWD is an improved algorithm for cross interference in
(NLFM) signal, LFM-BPSK composite modulation signal and wigner-ville distribution (WVD), which belongs to Cohen
FSK-BPSK composite modulation signal. class time-frequency distribution. Cohen proposed a general
CW signal is given by representation of time-frequency distribution as
∫∫∫
1 τ τ
t
s(t) = Arect( )ej2π(fc t+ϕ) , (2) Cs = s(u + )s∗ (u − )g(θ, τ )e−j(θt+uθ) dudτ dθ,
T 2π 2 2
(11)
LFM signal can be expressed as and different kernel functions correspond to different type-
t k 2 s of time-frequency distributions. If take g(θ, τ )=1, the
s(t) = Arect( )ej2π(fc t+ 2 t +ϕ) , (3)
T time-frequency distribution expressed is WVD. If take
)2
FSK signal can be expressed as g(θ, τ )=exp[ (πξτ
2σ ], the corresponding time-frequency distri-
t bution is CWD, which has a better effect of suppressing cross
s(t) = Arect( )ej(2πfk t+ϕ) , (4) interference. CWD time-frequency distribution of signal s(t)
T
is expressed as follows
BPSK signal is expressed as ∫∫∫
s(t) = Aej(2πfc t+φ(t)) , (5) C(t, w) = ej2πξ(s−t) f (ξ, τ ) · s(s + τ /2)
∞ (12)
QPSK signal can be expressed as · s∗ (s − τ /2)e−jwτ dξdsdτ,
2
s(t) = Aej(2πfc t+φ(t)) , (6) where f (ξ, τ ) = exp[ (πξτ )
2σ ] denotes the distribution of kernel
NLFM signal is given by function, σ is the attenuation coefficient and σ=1, s(t) is
observation signal, s* (t) stands for the conjugate s(t), t and w
t
s(t) = Arect( )e[jφ(t)] , (7) represent the time and frequency, respectively. The function of
T the kernel function is equivalent to the low-pass filter in two-
LFM-BPSK composite modulation signal can be expressed as dimensional space. σ determines the bandwidth of the filter.
t k 2 By controlling the size of σ, the cross term interference can be
s(t) = Arect( )ej2π(fc t+ 2 t +φ(t)) , (8) effectively suppressed. The larger σ, the higher the resolution
T
and FSK-BPSK composite modulation signal is expressed as of signal item, but the more serious the cross item. The smaller
σ is, the greater the inhibition of the cross term will be, but
t
s(t) = Arect( )ej(2πfk t+φ(t)) , (9) it will cause a large trailing effect and the resolution of the
T signal term is poor. Therefore, the value of σ should be a
where A and fc represent the amplitude and the carrier fre- compromise between signal term resolution and cross term
quency, respectively. ϕ denotes the initial phase, k is the slope suppression, depending on the characteristics of the signal. If
of frequency modulation, T stands for the pulse width and fk the amplitude and frequency of the signal change rapidly, the
represents the frequency coding modulation function. Besides, larger σ and the smaller σ should be taken.
φ(t) denotes the phase modulation function, and the different Two-dimensional time-frequency original images obtained
signals have the different phase modulation functions. rect( Tt ) through CWD time-frequency analysis are shown in Fig. 2.
represents the rectangular signal, which can be expressed as The function of CWD is to pay more attention to the shape
{
t 1 0 ≤ t ≤ T, of the signal rather than the amount of energy. Therefore,
rect( )= (10) the time-frequency images of the wireless signals are con-
T 0 else.
verted into a black-and-white binary images by an iterative
relatively ideal W and b, so that the output value is similar

(a)BPSK (b)CW (c)FSK (d)FSK-BPSK
to x, then the hidden layer can be considered as the ideal
encoding input x.
2) Denoising Autoencoder: Denoising autoencoder (DAE)
basic principle is: first of all, add a certain degree of noise
into the training data to destroy the original data to a certain
(e)LFM (f)LFM-BPSK (g)NLFM (h)QPSK
extent. The processed data is then input into the encoder for
training, and the output is as much as possible the same as
the data without adding noise. If the data added with noise is
processed by encoding and decoding, the original input can be
reconstructed. Like AE, it mainly includes encoding stage and
decoding stage. x represents the input data, and x′ denotes
the input after adding noise. The error function expression of
Fig. 3. Intelligent representation of wireless signals by CWD.
denoising autoencoder is as follows
L = ||x − g(f (x′ ))||2 . (15)
method to simplify the burden of the classifier. Then, the area
where the signals exist are searched by the image cropping Thus, the cost function is
algorithm, and the area without the signal distribution are ∑ λ
J(W, b) = (L(x, y)) + ||W ||2F ). (16)
cropped. Because the data dimension of the original image 2
input classifier is too large, the cropped time-frequency image 3) Sparse Autoencoder: Sparse autoencoder (SAE) is an
is adjusted to a 28 * 28 image by a bicubic interpolation AE formed by adding the restriction condition of sparse
algorithm. The processed wireless signals classification feature representation in the coding process of traditional AE, which
diagram are shown in Fig. 3. As can be seen from Fig. 3, requires the output of most neurons to be 0 and a small part
the processed images still retain relatively complete feature to be 1. According to the auto-coding principle, the input data
information, and the features of different types of signals are can be reconstructed according to the feature representation
significantly different, so the features can be used as the input of the hidden layer, which means that if the dimension of
to the deep networks. the hidden layer is less than the dimension of the input layer,
the network can learn compression characteristics. However,
IV. I NTELLIGENT C LASSIFICATION OF W IRELESS S IGNALS when the dimension of the hidden layer is larger than the
BASED ON D EEP L EARNING N ETWORKS dimension of the input layer, the SAE can still learn some
A. Unsupervised Classification Based on SHAE potential features. The basic principle of the SAE as following:
training data set x after encoding process for encoding the
1) Autoencoder: Autoencoder (AE) is a typical unsuper- characteristics of the said h, then through sparse has restricted
vised learning method, which uses unlabeled training data. part of neurons were inhibited, and the other are active, the
Assuming that the input data is image data, the AE hopes that reconstruction in the decoding process and the input data to
the image mapped by the network will be consistent with the get y, finally through loss function using mean square error
original image, so as to achieve the reconstruction of image as the objective function reconstruction error calculation.
data. The coding and decoding process of the autoencoder can Let yj denotes activation degree of hidden layer neurons
be described as follows Encoding: h = f (x) = σe (W1 x + b1 ), j, aj (x) = σe (W x + b1 ) is the output probability of the
Decoding: y = g(h) = σd (W2 h + b2 ), where x is input, W1 hidden layer neurons j under the condition of input x, where
and b1 stand for coding weights and bias, W2 and b2 represent W represents the link weight between the neuron and the
decoding weights and bias, σe is nonlinear transformation, neuron in the previous layer. In order to describe the feature of
σd can be the same as the encoding process of nonlinear activation, we introduce the mean activation ρ̂j , and activation
transformation and affine transformation. In this paper, mean can be expressed as
square error (MSE) function is used as the error function of
1 ∑
N
data reconstruction, and the function expression is
ρ̂j = aj (xi ), (17)
N i=1
L = ||x − g(f (x))|| . 2
(13)
Therefore, the cost function can be expressed as follows where xi represents the i input sample, and N represents the
dimension of the input x. By modifying the objective function,
∑ λ
J(W, b) = (L(x, y)) + ||W ||2F ), (14) adding the penalty factor, and setting the sparsely coefficient
2 parameter ρ, the ρ̂j approximates ρ, we can set the coefficient
where x represents the input sample, W denotes the weight, limit on hidden layer neurons. Therefore, the SAE increases
and λ is the regularization coefficient. the sparse penalty term on the cost function, which can be
The training goal of AE is to minimize the error between expressed as
the output layer and the input layer, and try to learn the
∑
L
ρ 1−ρ
identity function g(f (x)) = x. If the model is trained by error KL( ρ∥ ρ̂i ) = ( ρ log + (1 − ρ) log ), (18)
antimissile and batch gradient descent algorithms to obtain i=1
ρ̂i 1 − ρ̂i
Algorithm 1 The unsupervised classification process of wire-

less signals.
h(1) 1: CWD time-frequency analysis is carried out on the re-
X(1) ceived signals, and obtain two-dimensional time-frequency
h(2) r(1) images;
Z(1) Y(1)
X(2) 2: Use unlabeled training sample {x1 , x2 , · · · , xn }to training
(3) (2)
h r
Z(2) Y(2)
DAE, and the features extracted from the hidden layer of
X(3)
the DAE are taken as input into the SAE. Similarly, the

h(4) r(3)
Z(n) Y(n) hidden layer of the SAE is taken as input of the AE, and
X(4)

the encoding layer parameters of each AE is reserved as

h(5) r(n)
X(n)
W;

h(n)
3: The two-dimensional time-frequency images of the test
sample input into the SHAE for feature extraction;
Input
Denoising Sparse
Autoencoder Output 4: Realize wireless signals classification by softmax classifi-
Autoencoder Autoencoder
er.
Fig. 4. Stack hybrid autoencoders structure. B. Semi-supervised Classification Based on SHAE and sLDA
Semi-supervised linear discriminant analysis (sLDA) is a

dimensionality reduction method that combines the semi-
where ρ̂i represents the average activation probability of the i supervised learning method with linear discriminant analysis.
hidden layer node. Therefore, the overall objective function is sLDA adds regular term into the classical linear discriminant
analysis method. In order to maintain the geometric structure
of data in the process of dimensionality reduction, the data
∑
N
Jsparse (W, b) = J(W, b) + β KL( ρ∥ ρ̂i ). (19) also maintains the original geometric structure in the low-
j=1 dimensional space.
Suppose the data set is X =
{x1 , x2 , · · · , xl , xl+1 , xl+2 , · · · xl+u }(l + u = n), where
In the training phase of the neural network, the parameters of
xi is d dimension vector, l represents the number of marked
each layer of the SAE are trained by minimizing the cost
data sets, u represents the number of unmarked data sets,
function, and then the reconstruction layer of the training
and n stands for the number of total data sets. The projected
network is removed from the AE in the signal classification
result of the sample point is Y = {y1 , y2 , · · · , yn } ,where
process, and the hidden layer of the AE is obtained as the
yi = W T xi , W = {w1 , w2 , · · · , wd } is the projection matrix.
output layer.
The inter-class divergence matrix Sb of the data set and the
4) Stack Hybrid Autoencoders: In the wireless signal clas- intra-class dispersion matrix Sw of the data set are
sification, to solve the problem of missing feature or dimension
of explosion, SHAE is adopted in this paper. SHAE contains k ∑
∑ T
a DAE, a SAE and an AE. The network structure of SHAE Sw = (xi − µi )(xi − µi ) , (20)
is shown in Fig. 4. The first layer of the structure of the i=1 xi ∈ci
SHAE as the input layer, the second layer of the structure is
DAE, in order to make the study to the characteristics of more
∑
k
T
comprehensive, random add some noise to the original signal, Sb = ni (µi − µ)(µi − µ) , (21)
and then use the AE for training. If reconstruct the original i=1
signal, show hidden layer of DAE learn the characteristics
of low dimension into the original data. The third layer of 1
∑
m
where ni is weight, µ= m xi denotes the mean of all the
the structure is a SAE. In order to reduce the dimension of i=1
features and extract abstract features, some sparse restrictions samples, µi represents the mean of the sample of i class, ci
are added to the AE to make the hidden layer dimension of the is the data set of the i class, and k stands for the number of
encoder less than the dimension of the input layer, and form categories of the data set.
a SAE. Then the SAE is used for training. If the input layer When constructing a homogeneous neighbor graph, we treat
can be reconstructed, the encoder can reduce the dimension the data points as the vertices of the graph. In the sample
of the feature and extract some more abstract features.The data set, the labeled point category label is known. We set the
fourth layer of the structure is an AE which further processes condition a that belongs to the same class and is a neighbor,
the feature and outputs it as the final feature. Signal classi- and the value at this time is set to 1. When at least one of the
fication based on SHAE can be accomplished unsupervised. two points is unmarked and is a neighbor, condition b is set,
The unsupervised classification process of wireless signal is and it is assigned by thermonuclear method. According to the
summarized in Algorithm 1. above principle, we construct a homogeneous neighbor matrix
S ′ = (Sij ′ ), where Algorithm 2 The semi-supervised classification process of

 wireless signals.
 1 homogeneous neighbors ,
1: CWD time-frequency analysis is carried out on the re-
Sij ′ = exp(−||xi − xj ||2 /β) at least one unlabeled ,
 ceived signals, and obtain two-dimensional time-frequency
0 other,
images;
(22)
2: Use unlabeled training sample {x1 , x2 , · · · , xn } to train-
β represents the scaling factor that adjusts the kernel function.
ing DAE, and the features extracted from the hidden layer
According to the principle of linear discriminant analysis
of the DAE are taken as input into the SAE. Similarly, the
method, in order to ensure more compact sample points of the
hidden layer of the SAE is taken as input of the AE, and
same kind, the following optimization problem is constructed
the encoding layer parameters of each AE is reserved as
∑ 2 W;
min ∥yi − yj ∥ Sij ′ , (23)
3: The two-dimensional time-frequency image of the test
ij
sample input into the SHAE for feature extraction;
and can be got 4: Initializes the parameters of semi-supervised linear dis-
∑ 2 criminant methodčňand the adjustment coefficient α in
∥yi − yj ∥ Sij ′ =2WT XL′ X T W. (24)
the objective function is fixed between 0 and 1. A small
ij
number of labeled samples are used for dimensionality
When constructing the heterogeneous nearest neighbor graph, reduction by semi-supervised linear discriminant method
the marked points have category information. We assign the and classification features are further extracted;
points that are not in the same category and are nearest 5: Realize wireless signals classification by Softmax classi-
neighbors by thermonuclear method. When at least one of the fier.
two points is unmarked and non-neighbor, its value is 1 and the
other cases are 0. According to the principle of heterogeneous Algorithm 3 The supervised classification process of wireless
neighbor matrix constructed S ′ = (Sij ′′ ), where, signals.

 1 at least one unlabeled 1: CWD time-frequency analysis is carried out on the re-
Sij ′′ = exp(−||xi − xj ||2 /β) homogeneous neighbors ceived signals, and obtain two-dimensional time-frequency
 images;
0 other
(25) 2: Use unlabeled training sample {x1 , x2 , · · · , xn } to train-
β is the scaling factor that adjusts the kernel function. ing DAE based on Fisher discrimination, and the features
Similarly, points that are not in the same category are more extracted from the hidden layer of the DAE based on
sparse, and the optimization function is constructed as follows Fisher discrimination are taken as input into the SAE
∑ 2 based on Fisher discrimination. Similarly, the hidden layer
max ∥yi − yj ∥ Sij ′′ , (26) of the SAE based on Fisher discrimination is taken as
ij input of the AE, and the encoding layer parameters of
and can be got each AE is reserved as W ;
∑ 2 3: The two-dimensional time-frequency image of the test
∥yi − yj ∥ Sij ′′ =2WT XL′′ X T W. (27) sample input into the SHAE for feature extraction;
ij 4: Realize wireless signals classification by softmax classifi-
And then we can obtain er.
∑ 2 ∑ 2
FL = ∥yi − yj ∥ Sij ′ − ∥yi − yj ∥ Sij ′′
ij ij (28) the AE have little difference between the same class and
′ ′′
= W XL W − W XL W.
T T
big difference between the different class, which is more
conducive to classification. Taking SAE as an example, the
Therefore, the objective function of semi-supervised linear
original objective function is
discriminant analysis can be obtained as follows
T ∑
N
W Sb W
J(W ) = . (29) Jsparse (W, b) = J(W, b) + β KL( ρ∥ p̂i ). (30)
|W T Sw W + α(W T XL′ W − W T XL′′ W )| j=1
In summary, the semi-supervised classification process of The improved objective function is

wireless signals is summarized in Algorithm 2.
J = Jsparse (W, b) + αJW − βJB , (31)
C. Supervised Classification Based on SHAEs and Fisher where

1∑∑ i
C cN
Discrimination JW = a c − mc 2 , (32)
2 c=1 i =1
Linear discrimination is integrated into the AE, and the c
inter-class distance measurement function and intra-class dis-

1∑ ∑
C C
2
tance measurement function are added into the loss function JB = ∥mc − mk ∥ , (33)
of each layer of AE so that the characteristics learned from 2 c=1
k=c+1
∑
Nc
from the SHAE can realize the feature of the large distance
aic
ic =1 between classes and small distance within classes, which lays
mc = , (34)
Nc a good foundation for subsequent classification.
where C is the total number of categories, Nc denotes the first Fig. 6-Fig. 9 show the classification performance of differ-
class c sample, aic is the first i input data to the first class ent deep learning networks with different SNRs. As can be
c in the encoder output value of the hidden layer, and mc seen from Fig. 6 to Fig. 9, when the SNR is greater than 0
represents in the first class c sample average output of hidden dB, the average classification rate of the signal is higher than
layer. 90%. (a) in Fig. 6-Fig. 9 show the classification performance
In summary, the supervised classification process of wireless of different deep learning networks method with different
signals is summarized in Algorithm 3. SNRs, (b) in Fig. 6-Fig. 9 show the classification performance
of different deep learning networks and sLDA method with
different SNRs and (c) in Fig. 6-Fig. 9 show the classification
V. N UMERICAL R ESULTS AND D ISCUSSION performance of different deep learning networks and Fisher
In order to verify the effectiveness of the proposed method, discrimination method with different SNRs. It can be seen
simulation studies are conducted, in which eight types of from Fig. 6 to Fig. 9 that the SHAE method has a better clas-
wireless signals are considered, including CW, LFM, FSK, sification performance and a higher average classification rate
BPSK, QPSK, LFM-BPSK, FSK-BPSK, and NLFM. Simula- at a low SNR than AE, DAE and SAE, and the classification
tion parameters of eight types of wireless signals used in the performance of SHAE and Fisher discrimination method is
simulation are described as follows [14]: the two frequency better than SHAE method and SHAE and sLDA method. From
points of FSK signals are 20MHz and 40MHz, the two fre- the above, we can be seen that the classification performance
quency points of FSK-BPSK signals are 25MHz and 35MHz, of supervised learning method is better than semi-supervised
the carrier frequency of the rest signals are 30MHz. The pulse learning method and unsupervised method.
width is set to be 10µs with a 120MHz sampling frequency. In the same simulation environment and the same signal
The bandwidth of LFM signals is 2MHz. FSK signals coding parameters setting such as symbol rate, carrier frequency, sam-
rule and the phase encoding rule of BPSK signals are [100110] pling frequency and sampling points. The proposed method
and [11100010010], respectively. In addition, The phase en- compared with the deep sparse capsule network (DSCN)
coding rule of BPSK signals is [01230312211200112012]. The method and the capsule network (CN) method in [14]. As
bandwidth of LFM-BPSK signals is 5MHz and the phase can be seen from Fig. 10, when the SNR is lower than 5dB,
encoding rule is [11100010010]. The phase encoding rule the average classification rates of the proposed method are
of FSK-BPSK signals is [11100010010]. The NLFM signals significantly higher than that of DSCN and CN. The charac-
are a sinusoidal frequency modulation signal. The number of teristic computational complexity of the proposed method is
samples used in the training and testing of each type of signals O(N log N ), while the computational complexity of [14] is
are 50000 and 5000, respectively. O(N log N ) and O(KN log N ), respectively, where K is the
Tab. I shows the classification performance of SHAE under number of data segmentation of higher order spectrum. When
the settings of the neuron number of different hidden layers. the training samples of each signal with each SNR are 5000,
Tab. II shows the classification accuracy of SHAE and fisher the off-line training time of SHAE and Fisher discrimination
discrimination with the settings of different hidden layer method is 898.272 seconds, the off-line training time of SHAE
neurons. From Tab. I, 100 neurons in the first layer are selected method is 834.295 s, and the off-line training time of SHAE
as the number of hidden layers. Then, when the hiddensize of and sLDA method is 835.713 s. The offline training time of CN
the first layer is fixed to 100, the number of hidden layer method in [14] is 24025.85 seconds, and the offline training
neurons of the second layer of SAE is determined. Taking the time of DSCN method in [14] is 6811.5 seconds. From the
output feature of DAE as the input of the second layer SAE, above computational complexity analysis, we can be seen that
the classification accuracy of SAE tends to be stable when the the computational complexity of proposed methods are lower
number of hidden layer neurons is about 300. Similarly, the than that of CN method and DSCN method in [14].
first two AE parameters are fixed as 100 and 300, and the
classification accuracy of the last AE tends to be stable when
VI. C ONCLUSION
the number of hidden layer neurons is 200. As can be seen
from Tab. II, the classification accuracy of the SAE and Fisher In this paper, a novel signal classification framework is pro-
discrimination tends to be stable when the number of neurons posed using stack hybrid autoencoders in industrial cognitive
is 100. Similarly, when the parameters of the first two AE are radio networks. To extract features that can fully reflect char-
fixed at 200 and 100, the number of hidden layer neurons of acteristics of wireless signals, wireless signals are processed
the last fisher auto-encoder is 100, the classification accuracy by Choi-Williams distribution time-frequency analysis first.
of SHAE and Fisher discrimination tends to be stable. And then, stack hybrid autoencoders are utilized to generate
After the parameters of the AE are fixed, the classification input data. Next, multiple methods, including stack hybrid
features distribution of the proposed methods are shown in autoencoders, stack hybrid autoencoders and semi-supervised
Fig. 5. As shown in Fig. 5, SHAE and Fisher discrimination linear discriminant analysis, and stack hybrid autoencoders and
has the best separation effect among classes, while the SHAE Fisher discrimination are employed to realize wireless signal
and sLDA has a better separation effect. The feature extracted unsupervised classification, semi-supervised classification and
TABLE I
C LASSIFICATION PERFORMANCE OF SHAE WITH DIFFERENT NUMBER OF HIDDEN LAYER NEURONS .
Hiddensize
16 64 100 200 300 400
N etworks
DAE 81.45% 83.25% 85.29% 85.31% 85.69% 85.47%
SAE 83.87% 85.91% 86.45% 87.70% 88.75% 88.84%
AE 91.74% 94.57% 96.46% 97.21% 97.24% 97.77%
TABLE II
C LASSIFICATION PERFORMANCE OF SHAE AND F ISHER DISCRIMINATION WITH DIFFERENT NUMBER OF HIDDEN LAYER NEURONS .
Hiddensize
16 32 64 100 200 300
N etworks
DAE 77.25% 86.75% 90.62% 90.75% 90.87% 91.12%
SAE 84.36% 90.62% 91.81% 92.68 92.41% 92.50%
AE 90.50% 95.78% 96.25% 97.75% 97.87% 97.73%
(a) SHAE (b) SHAE+sLDA (c) SHAE+Fisher
Fig. 5. Classification feature distribution of the proposed methods.
100 R EFERENCES
99
[1] X. Wang, L. Wan, M. Huang, C. Shen and K. Zhang, “Polarization
98 channel estimation for circular and non-circular signals in massive
Classification Correct Rates(%)
97 MIMO systems,” IEEE J. Sel. Topics Signal Process., vol. 13, no. 5,
96
pp. 1001-1016, Sept. 2019.
[2] L. Wan, L. Sun, X. Kong, Y. Yuan, K. Sun and F. Xia, “Task-driven
95
resource assignment in mobile edge computing exploiting evolutionary
94 computation,” IEEE Wireless Commun., vol. 26, no. 6, pp. 94-101, Dec.
93
2019.
DSCN [3] X. Wang, L. Wan, M. Huang, C. Shen, Z. Han and T. Zhu, “Low-
92 CN
SHAE+sLDA complexity channel estimation for circular and noncircular signals
91 SHAE+Fisher
SHAE
in virtual MIMO vehicle communication systems,” IEEE Trans. Veh.
90 Technol., Feb. 2020, DOI: 10.1109/TVT.2020.2970967.
-5 -4 -3 -2 -1 0 1 2 3 4 5
[4] H. Song, X. Fang, and C. Wang, “Cost-reliability tradeoff in licensed and
SNR(dB)
unlicensed spectra interoperable networks with guaranteed user data rate
requirements,” IEEE J. Sel. Areas Commun., vol. 35, no. 1, pp. 200-214,
Fig 10. Classification performance comparison of the different methods. Jan. 2017.
[5] P. K. Sahoo, S. Mohapatra and J. Sheu, “Dynamic spectrum allocation
algorithms for Industrial cognitive radio networks,” IEEE Trans Ind.
Informat., vol. 14, no. 7, pp. 3031-3043, July 2018.
[6] N. Li, M. Xiao and L. K. Rasmussen, "Optimized cooperative multiple
access in industrial cognitive networks,” IEEE Trans Ind. Informat., vol.
supervised classification, respectively. To testify the effective- 14, no. 6, pp. 2666-2676, June 2018.
ness of the proposed framework, extensive simulation studies [7] M. Liu, L. Liu, H. Song, Y. Hu, Y. Yi, and F. Gong, “Signal estimation
are conducted, which indicates the proposed framework have in underlay cognitive networks for industrial internet of Things,” IEEE
Trans Ind. Informat., Nov. 2019. DOI: 10.1109/TII.2019.2952413.
better classification performance than existing state-of-the-art [8] Y. Liang, K. Chen, G. Y. Li and P. Mahonen, “Cognitive radio network-
methods, such as methods based on capsule network and deep ing and communications: an overview,” IEEE Trans. Veh. Technol., vol.
sparse capsule network. Future work is to improve the method 60, no. 7, pp. 3386-3407, Sept. 2011.
[9] T. O’Shea, T. Roy and T. Clancy, “Over-the-air deep learning based
to make it has better classification performance over fading radio signal classification,” IEEE J. Sel. Topics Signal Process., vol. 12,
channels. no. 1, pp. 168-179, Feb. 2018.
100 100 100
95 95 95
Classification Correct Rates(X100%)

90 90
90
85 85
85
80 80
80
75 75
75
BPSK 70 BPSK 70 BPSK
QPSK QPSK QPSK
70
CW 65 CW 65 CW
NLFM NLFM NLFM
65 LFM LFM LFM
60 60
LFM-BPSK LFM-BPSK LFM-BPSK
60 FSK-BPSK 55 FSK-BPSK 55 FSK-BPSK
FSK FSK FSK
55 50 50
-5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5
SNR(dB) SNR(dB) SNR(dB)
(a) AE (b) AE+sLDA (c) AE+Fisher
Fig. 6. Classification performance of different AE versus different SNRs.
100 100 100
95 95
95

90 90
90
85 85
85 80 80
80 75 75
BPSK BPSK BPSK
QPSK QPSK QPSK
70 70
CW CW CW
75
NLFM NLFM NLFM
65 LFM 65 LFM
LFM
70 LFM-BPSK LFM-BPSK LFM-BPSK
FSK-BPSK 60 FSK-BPSK 60 FSK-BPSK
FSK FSK FSK
65 55 55
-5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5
(a) DAE (b) DAE+sLDA (c) DAE+Fisher
Fig. 7. Classification performance of different DAE versus different SNRs.
100 100 100
95 95 95
90
90 90
85
85 85
80
80 80
BPSK BPSK BPSK
75
QPSK QPSK QPSK
CW 75 CW 75 CW
70 NLFM NLFM NLFM
LFM LFM LFM
LFM-BPSK 70 LFM-BPSK 70 LFM-BPSK
65 FSK-BPSK FSK-BPSK FSK-BPSK
FSK FSK FSK
60 65 65
-5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5
(a) SAE (b) SAE+sLDA (c) SAE+Fisher
Fig. 8. Classification performance of different SAE versus different SNRs.
100 100 100
95
98
95
90
85 96
90
80
94
75
BPSK 85 BPSK BPSK
QPSK QPSK QPSK
70 92
CW CW CW
NLFM NLFM NLFM
65 LFM LFM LFM
80
LFM-BPSK LFM-BPSK 90 LFM-BPSK
60 FSK-BPSK FSK-BPSK FSK-BPSK
FSK FSK FSK
55 75 88
-5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5 -5 -4 -3 -2 -1 0 1 2 3 4 5
(a) SHAE (b) SHAE+sLDA (c) SHAE+Fisher
Fig. 9. Classification performance of different SHAE versus different SNRs.
[10] H. Song, X. Fang, L. Yan, and Y. Fang, “Control/user plane decoupled

architecture utilizing unlicensed bands in LTE systems,” IEEE Wireless
Commun., vol. 24, no. 5, pp. 132-142, Oct 2017.
[11] O. Dobre, A. Abdi, Y. Bar-Ness and W. Su, “Survey of automatic mod-
ulation classification techniques: classical approaches and new trends,”
IET Commun., vol. 1, no. 2, pp. 137-156, Apr. 2007.
[12] P. Ghasemzadeh, S. Banerjee, M. Hempel and H. Sharif, “Accuracy
analysis of feature-based automatic modulation classification with blind
modulation detection,” in Proc. 2019 International Conference on Com-
puting, Networking and Communications, Honolulu, USA, Apr. 2019,
pp. 1000-1004.
[13] Y. Hu, M. Liu, C. Cao, B. Li, “Modulation classification in alpha stable
noise,” in Proc. 2016 IEEE 13th International Conference on Signal
Processing, Chendu, China, Nov. 2016, pp. 1-4.
[14] M. Liu, G. Liao, Z. Yang, H. Song and F. Gong, “Electromagnetic signal
classification based on deep sparse capsule networks,” IEEE Access, vol.
7, pp. 83974-83983, June 2019.
[15] Y. Jiang, S. Yin and O. Kaynak, “Data-driven monitoring and safety
control of industrial cyber-physical systems: basics and beyond," IEEE
Access, vol. 6, pp. 47374-47384, Aug. 2018.
[16] S. Chen, S. Zheng, L. Yang and X. Yang, “Deep learning for large-
scale real-world ACARS and ADS-B radio signal classification,” IEEE
Access, vol. 7, pp. 89256-89264, June 2019.
[17] Y. Zhang, G. Wu, J. Wang and Q. Tang, “Wireless signal classification
based on high-order cumulants and machine learning,” in Proc. 2017
International Conference on Computer Technology, Electronics and
Communication, Dalian, China, Dec. 2017, pp. 559-564.
[18] Y. Wang, M. Liu, J. Yang and G. Gui, “Data-driven deep learning for
automatic modulation recognition in cognitive radios,” IEEE Trans. Veh.
Technol., vol. 68, no. 4, pp. 4074-4077, Apr. 2019.
[19] S. Rajendran, W. Meert, D. Giustiniano, V. Lenders and S. Pollin, “Deep
learning models for wireless signal classification with distributed low-
cost spectrum sensors,” IEEE Trans. Cognitive Commun. Netw., vol. 4,
no. 3, pp. 433-445, Sept. 2018.
[20] T. O’Shea, J. Corgan and T. Clancy, “Convolutional radio modulation
recognition networks,” in Proc. Int. Conf. Eng. Appl. Neural Netw.,
Springer, Cham, Aug. 2016, pp. 213-226.
[21] J. Lee, K. Kim and Y. Shin, “Feature image-based automatic modulation
classification method using CNN algorithm,” in Proc. 2019 International
Conference on Artificial Intelligence in Information and Communication,
Okinawa, Japan, Feb. 2019, pp. 1-4.
[22] E. Yar, M. Kocamis, A. Orduyilmaz, M. Serin and M. Efe, “A complete
framework of radar pulse detection and modulation classification for
cognitive EW,” in Proc. 2019 27th European Signal Signal Process.
Conference, Coruna, Spain, Nov. 2019, pp. 1-5.
[23] A. Wahla, L. Chen, Y. Wang, R. Chen and F. Wu, “Automatic wireless
signal classification in multimedia internet of things: an adaptive boost-
ing enabled approach,” IEEE Access, vol. 7, pp. 160334-160344, Nov.
2019.
[24] S. Huang, L. Chai and Z. Li, et al., “Automatic modulation classification
using compressive convolutional neural network,” IEEE Access, vol. 7,
pp. 79636-79643, June 2019.
[25] Z. He, Y. Peng and Y. Zhao, et al., “Deep learning-based automatic
modulation recognition algorithm in non-cooperative communication
systems,” in Proc. 2019 11th International Conference on Wireless
Commun. and Signal Process., Xi’an, China, Oct. 2019, pp. 1-6.

Liu 2020

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Liu 2020

Uploaded by

Copyright:

Available Formats

This article has been accepted for publication in a future issue of this journal, but has not been

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 1

Data-Driven Deep Learning for Signal Classification

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 3

Primary User Link

Primary User Primary User

Secondary User (g)NLFM (h)QPSK

Wireless Intelligent Intelligent

III. I NTELLIGENT R EPRESENTATION OF W IRELESS

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 4

relatively ideal W and b, so that the output value is similar

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 5

Algorithm 1 The unsupervised classification process of wire-

Semi-supervised linear discriminant analysis (sLDA) is a

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 6

S ′ = (Sij ′ ), where Algorithm 2 The semi-supervised classification process of

In summary, the semi-supervised classification process of The improved objective function is

C. Supervised Classification Based on SHAEs and Fisher where

inter-class distance measurement function and intra-class dis-

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 7

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 8

(a) SHAE (b) SHAE+sLDA (c) SHAE+Fisher

Fig. 5. Classification feature distribution of the proposed methods.

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 9

100 100 100

Classification Correct Rates(X100%)

Classification Correct Rates(X100%)

Classification Correct Rates(X100%)

(a) AE (b) AE+sLDA (c) AE+Fisher

Fig. 6. Classification performance of different AE versus different SNRs.

100 100 100

Classification Correct Rates(X100%)

(a) DAE (b) DAE+sLDA (c) DAE+Fisher

Fig. 7. Classification performance of different DAE versus different SNRs.

100 100 100

Classification Correct Rates(X100%)

Classification Correct Rates(X100%)

(a) SAE (b) SAE+sLDA (c) SAE+Fisher

Fig. 8. Classification performance of different SAE versus different SNRs.

100 100 100

Classification Correct Rates(X100%)

Classification Correct Rates(X100%)

(a) SHAE (b) SHAE+sLDA (c) SHAE+Fisher

Fig. 9. Classification performance of different SHAE versus different SNRs.

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 10

[10] H. Song, X. Fang, L. Yan, and Y. Fang, “Control/user plane decoupled

You might also like