Int. J. Modelling, Identification and Control, Vol. 34, No.

2, 2020 75

A review on data-driven approaches for industrial

process modelling

Wei Guo and Tianhong Pan*

School of Electrical Engineering and Automation,
Anhui University,
Hefei, Anhui 230601, China
and
School of Electrical Information and Engineering,
Jiangsu University,
Zhenjiang, Jiangsu 212013, China
Email: guow1989@163.com
Email: thpan@ahu.edu.cn
*Corresponding author

Zhengming Li
School of Electrical Information and Engineering,
Jiangsu University,
Zhenjiang, Jiangsu 212013, China
Email: lzming@ujs.edu.cn

Guoquan Li
Jiangsu Hengshun Vinegar Industry Co., Ltd.,
Zhenjiang, Jiangsu 212043, China
Email: 13775556788@139.com

Abstract: Data-driven techniques in industrial processes have been continually attended during
the past decades. However, there are many challenging issues in this field when the collected
data presents different characteristics. In order to sketch the principle of different modelling
methods under various working conditions, data-driven modelling methods from perspectives
of data structures and model structures are reviewed in this paper. Firstly, the data collection
and preprocessing procedure are inspected. Then, popular methods from linear (including the
multivariate linear regression (MLR), to latent variable projection (LVP), etc.) to nonlinear
methods (including artificial intelligence, Gaussian process regression (GPR), local model, etc.)
are discussed. Finally, the model calibration strategies (including offset-based method, recursive
method, moving window method) are also reviewed. The major purpose is to support the industrial
process modelling for technical users by providing a set of data-driven methods.

Keywords: data-drive modelling; industrial process; machine learning; data analytics; model
structure.

Reference to this paper should be made as follows: Guo, W., Pan, T., Li, Z. and Li, G. (2020)
‘A review on data-driven approaches for industrial process modelling’, Int. J. Modelling,
Identification and Control, Vol. 34, No. 2, pp.75–89.

Biographical notes: Wei Guo received his BSc in Electronic Information Engineering from
Jingjiang College of Jiangsu University, Zhenjiang, China; and the MS in School of Electrical
Information and Engineering of Jiangsu University, Zhenjiang, China, in 2012, and 2016,
respectively. He is currently pursuing the PhD in Control Science and Engineering in Jiangsu
University. His research interests include process modelling, clustering algorithm and artificial
intelligence.

Tianhong Pan received his BSc in Mechatronics Technology and Applications from Anhui
Agriculture University, Hefei, China; the MS in Power Electronics and Electric Power Drives from
the Gansu University of Technology, Lanzhou, China; and the PhD in Control Theory and Control
Engineering from Shanghai Jiao Tong University, Shanghai, China, in 1997, 2000, and 2007,

c 2020 Inderscience Enterprises Ltd.

Copyright
76 W. Guo et al.

respectively. He has been a Professor with the School of Electrical Engineering and Automation,
Anhui University, Hefei, Anhui, China. His research interests include multiple model approach and
its application, machine learning, virtual metrology, predictive control and run-to-run control theory
and practice, etc.

Zhengming Li received his BSc in Automation in 1982 from Jiangsu University, Zhenjiang,
China and the MS in Control Theory and Engineering in 1988 from Xi’an Jiao Tong University,
Xian, China. He has been a Professor in the School of Electrical and Information Engineering,
Jiangsu University. His research interests include R2R control theory and practice, complex process
modeling and control, system identification.

Guoquan Li has been a Director in the Jiangsu Hengshun Vinegar Industry Co., Ltd.

1 Introduction (RF) (Ahmad et al., 2014), and support vector machine

With the development of computer science and the related
measurement techniques, it makes a significant contribution
to the collection and storage of operating data in industrial
processes. At the same time, such convenience provides
adequate supports for database, data mining, and data-driven
technologies. The data-driven method is more flexible to
facilitate the construction of industrial process systems as
against the traditional knowledge-based method. Therefore,
the data-driven method has been extensively used for
industrial processes including process modelling, controller
designing, and system optimising (Ahmad et al., 2014; Liu
et al., 2019a; Ge, 2017; Gopakumar et al., 2018; Smith and
Jin, 2014; Tamboli and Chile, 2018).
By mining the information of the collected data, various
machine learning algorithms can be employed for industrial
process monitoring (Fezai et al., 2018; Jiang et al., 2016;
Peng et al., 2015; Qin, 2012; Wang et al., 2019a), soft
sensing (Kadlec et al., 2009; Kaneko and Funatsu, 2014;
Yao and Ge, 2017; Yuan et al., 2018), and stage division
(Frigieri et al., 2016; Quiñones-Grueiro et al., 2019; Zhao
et al., 2014; Zhu et al., 2011). There are three different
algorithms in machine learning techniques, which are
summarised as supervised learning, unsupervised learning,
and semi-supervised learning. Among them, unsupervised
and supervised learning are the most widely adopted, which
may have been used for 80-90% of applications (Ge et al.,
2017). In supervised learning structures, data that consist
of inputs along with their corresponding labelled targets
are used for model construction. Such targets may include
process outputs, key variable indices, product quality, and
fault pattern, etc. The soft sensor is a typical application
by using supervised learning, which belongs to a regression
model (Kadlec et al., 2009; Shao and Tian, 2015). If the
labelled target is composed of discrete values, supervised
learning can be applied for fault diagnosis (Ge et al.,
2017). Several supervised learning methods are widely
used in industrial processes, whose algorithms include
multivariate linear regression (MLR) (Huang and Zhao,
2018), canonical correlation analysis (CCA) (Zhu et al.,
2017), principal component regression (PCR) (Yuan et al.,
2014a), partial least squares (PLS) (Ahmed et al., 2009),
artificial intelligence (AI) (Smith and Jin, 2014), Gaussian
process regression (GPR) (Zhou et al., 2015), random forest
(SVM) (Pooyan et al., 2015). Different from the supervised
learning framework, unsupervised learning methods use
against data without labels. Correspondingly, unsupervised
learning methods have been employed to reveal the hidden
information from process data, which includes control limits
of normal work conditions, distribution of clusters, outlier
criteria, and latent structures. For example, the fault detection
model can detect an abnormal event during the industrial
process by using the Q-statistic and the Hotelling’s T 2 index
(Liu et al., 2014). And the clustering algorithm simplifies
the complex data distribution (Khediri et al., 2012). There
are several well known unsupervised learning algorithms in
the process industry, which includes principal component
analysis (PCA) (Du et al., 2017), factor analysis (FA) (Ge
and Song, 2012), independent component analysis (ICA) (Liu
and Zhang, 2015), quality trend analysis (QTA) (Zhao et al.,
2017), clustering algorithms (Zhu et al., 2011), autoencoders
(Shang et al., 2014). Recently, much attention has been
gained in the industrial process using semi-supervised
learning. For a similar purpose as supervised learning,
however, a small volume of expensive labelled measurements
and a large number of cheap unlabelled measurements are
used for training sometimes. Thus, the high cost of data
labelling cannot guarantee a full sample training in the
modelling process (Ge and Song, 2012). Therefore, semi-
supervised learning makes a connection between supervised
learning and unsupervised learning. Moreover, those two
learning frameworks can also help each other for process
modelling. For example, clustering algorithms can simplify
a complex soft sensing model into several simple models,
and predictive key variables can also be added into the
calculation of control limits. The relationship of unsupervised
learning, supervised learning, and semi-supervised learning
is presented in Figure 1.
In order to cooperate with the high requirement
of industrial production, many modelling structures are
presented, such as lazy learning (Liu et al., 2012), multi-
model (Stoica et al., 2004), fuzzy model (Qi et al.,
2014), ensemble learning (Kaneko and Funatsu, 2014),
and transfer learning (Liu et al., 2019b). A wide variety
of methods can be constructed by using different model
structures and algorithms. Therefore, this paper provides a
systematic review related to data-driven process modelling
methods, from the viewpoints of basic algorithms and
 A review on data-driven approaches for industrial process modelling
model structures. Under each specific topic, the principle
of different methods or structures is discussed, with several
corresponding industrial applications.
Figure 1 The relationship of three modelling categories
(see online version for colours)
Unlabeled Labeled
Data Data
Semi-supervised
Learning
Unlabeled Unlabeled
Data Data
Unsupervised Supervised
Learning Learning
(PCA, ICA, FA, k-means, etc.) (PCR, PLS, SVM, GPR, etc.)
The rest of this paper is organised as follows. The main
methodology of process modelling is provided in Section 2.
Several approaches to data-driven modelling are carried out in
Section 3, including mathematical methods (Section 3.1) and
model selection (Section 3.2). In Section 4, some perspectives
on the topic of process modelling for further applications are
presented. At last, a conclusion is obtained in Section 5.
2 Methodology
The general procedure of process modelling includes data
collection and its pre-processing, structure and model
selection, model training, validation, and the corresponding
maintenance. An overview of process modelling is presented
in Figure 2, in which, the main methodology procedure is
linked with black arrows. It can be seen that the technological
process of modelling is divided into two parts:
• modelling preparation (i.e., data collection and its
pre-processing, and model selection)
• process modelling and analysis (i.e., model training,
model analysis, and model maintenance).
Note that a large amount of dramatic works related to each
procedure have been presented in the last decades, only some
functional descriptions are discussed in this section, and the
details of basic approaches are presented in Section 3.
2.1 Data collection
As the first step for developing a data-driven model, the
main task of data collection is to collect modelling data
77
from a historical database. For the purpose of constructing
an effective training model, the operating region of processes
should be noticed. Thus, the impractical samples recorded
by any invalid sensors will not be selected. Moreover,
the category of selected dataset is not same for different
modelling purposes. For example, in soft sensing or fault
detecting, the dataset from a stable operating condition should
be selected. But in fault diagnosis models, abnormal data
collection should be carried out as well. Another important
issue is to carry out the variable selection. Typically, the
variables measured in the industrial process may have co-
linear and redundant problems. There are two ways to
handle the collinearity problem. One is to use the statistic
analysis method, such as PCA (Jiang et al., 2016), PLS
(Rinnan et al., 2014), and CCA (Ogura, 2010). Another is
to use the computational learning research for the feature
Labeled selection, such as support vector regression (SVR) (Ben
Data
Ishak, 2016), genetic algorithms (GA) (Lu et al., 2016), and
RF (Hapfelmeier and Ulm, 2013).
2.2 Data pre-processing
Data collection in industrial processes always comes with a
series of well-known problems, including outliers, missing
data, and so on. Therefore, data pre-processing is urgently
required for data quality improvement. The outlier can be
caused by measurement noises, communication errors, or
sensor degradations. The 3σ-rule is the most well-known
univariate approach for outlier detection. And for multivariate
models, it often works by using the interaction among
variables, such as the distance, angle, and density (Souza
et al., 2016). On the other hand, there may be some missing
values that are failed to be recorded. It is mainly caused
by the outlier deletion or the incomplete transportation in
data collection step. The observation containing missing
values can be directly discarded if there is a small
number of missing objects. Otherwise, the missing value
is needed to be imputed by using maximum likelihood
(ML) method, expectation-maximisation (EM) method, and
regression method, etc (Souza et al., 2016; Zhu et al., 2018).
In addition, sampling rates, measurement delays, dynamical
characteristics and data normalisation are also needed to be
taken into consideration (Kadlec et al., 2009; Qin, 2012).
2.3 Model selection
Before the training step, there is a critical step that has to be
addressed, i.e., the model selection. As mentioned in previous
section, large amounts of models can be constructed by taking
a combination of different model structures and mathematical
methods. Subsequently, two thorny issues arise: what kind
of model should be employed, and how much complexity
the model structure is required? Obviously, the most complex
model may not be the best one. Taking the regression based
modelling as an example, a nonlinear model is unnecessary
while the relationship between process variables and key
variables is linear. Model selection is not only highly related
to the modelling purpose but also associated with data
relationships and characteristics (Ge, 2017). Consequently,
78 W. Guo et al.

Figure 2 Overview of methodology for industrial processes modelling (see online version for colours)

Variable Outlier
Data Collection
Selection Detection
and
Historical
Sample Missing Pre-processing
Database Selection Data

Structure
Structures Selection Semi-supervised
Learning
Training
Supervised Unsupervised
Model
Learning Learning
Mathematical Selection
Methods

Query Data Constructed Model Feedback Data

Collection Model Maintenance Collection

Key Variable Quality Pattern Inner-phase

Model Prediction Assessment Analysis Analysis
Analysis Fault Fault Correlation
Fault Isolation
Detection Diagnosis Analysis

there is no unified guidance for selecting a structure or a until the termination of feedbacks (Qin, 1998). It should
mathematical method. However, several useful rules such be noticed that the feedback sample needs to satisfy the
as Bayesian information criterion and Akaike information requirement of the criteria in data collection and data pre-
criterion could promote the model selection (Bishop, 2006). processing.

2.4 Model training and model validation

Once the modelling structure has been determined, the data-
driven model is trained by gathering all of contributions
of previous steps. In the training step, different algorithms
depend on their corresponding inner parameter structures.
Such algorithms actually can be performed as an optimisation
to get optimal parameters, such as the optimised regression
parameter, control limitation, cluster centre, density
distribution, decision policy, and stage division. It should
be noticed that the obtained model is needed to be validated
before the online application. Two steps are mainly contained
in model validation, namely, the self-validation and the
cross-validation (Khatibisepehr et al., 2013). Self-validation
can exclude the inadequate model obtained from the last
step. Cross-validation can guarantee the capability of
generalisation by analysing the identification performance
with the dataset that has not been used during the training
phase. Some other validation methods are also presented in
Bishop (2006).
2.5 Model maintenance and calibration
In order to deal with the slow change caused by aging
equipment, climate-varying, or adapt new work conditions
in industrial processes, a good data-driven model should
contain the mechanism of online self-maintenance. There are
two schemes being mainly used for model calibrations: one
needs to be adjusted only if the significant variation from
the identified model is detected in the online application (Ma
et al., 2009), and the other one keeps calibrating operations
3 Algorithms review
A systematic review of machine learning algorithms for
industrial process modelling is presented in this section.
As has been mentioned before, applications on data-
driven modelling are mainly partitioned into three types,
i.e., unsupervised, supervised, and semi-supervised learning
frameworks. Meanwhile, each type may not be the only used
scheme that addresses the modelling process independently.
Therefore, mathematical methods including linear models,
nonlinear models, and adaptive models are firstly reviewed.
And then, a number of optional model structures above these
mathematical methods are supplemented. Detailed reviews
are provided as follows.
3.1 Mathematical methods
3.1.1 Linear models
(1) Multivariate linear regression (MLR)
The MLR is a generalised model by considering the
linear relationship between one dependent variable and a
number of independent variables (Abyaneh, 2014). The
ordinary least squares (OLS) regression is the most widely
known approach, which promotes the development of other
algorithms, indirectly. Since the OLS has the numerical
problem when independent variables are strongly co-linear.
Several feature selection methods are presented, such as the
stepwise regression (Rothman et al., 2013), genetic algorithm
 A review on data-driven approaches for industrial process modelling
(Lu et al., 2016), etc. On the other hand, the ridge regression
and latent variable can also deal with the collinear problem
(Piepho, 2009). Most of applications using MLR focus on
the key variable prediction. For example, Wang et al. (2019b)
applied the MLR to influent chemical oxygen demand and
total phosphorus prediction in wastewater treatment plants.
Liu et al. (2018a) created a multivariate phase space by
referencing the phase state and predicted the very-short-term
wind power. Besides the predictive purpose, the MLR can
also be applied for process monitoring (Eyvazian et al., 2011).
(2) Latent variable projection (LVP) model
The latent variable projection (LVP) model is the most
popular method to deal with the numerical problems with the
large dimensional data or the high co-linear data by projecting
multidimensional process variables from the original space
onto a subspace, which mainly includes PCA, PCR, PLS, and
PLS regression (PLSR). In PCA, there are three mathematical
calculation methods to compute the independent principal
component (PC) and loading matrix, i.e., the nonlinear
iterative partial least squares (NIPALS), the eigenvalue
decomposition and the singular value decomposition (SVD)
(Geladi and Kowalski, 1986; Qin, 2012). The PCA is an
autoencoder in the unsupervised learning model. Meanwhile,
the obtained PCs can also be addressed to the regression
model by combining the labelled response key variable, i.e.,
PCR. While PCA obtains the PC that neglecting the reference
to the variance of key variables. Consequently, those PCs
may not be related to the variation of the key variable.
Contrary to PCA, PLS obtains constrained latent variables
(LVs) by simultaneously decomposes key variables and
process variables. Such LVs not only contain the maximum
correlation between key variables (Y ) and process variables
(X) but also retain the maximum variation both in Y and X
(Guo et al., 2016). Similarly, such linear relationship among
LVs can also be used for regression, namely PLSR. The
relationship between PCA, PCR, PLS, and PLSR is shown
in Figure 3, where, P and Q are the loads, T and U are the
corresponding scores of the LV.
Large amounts of applications of LVP models have been
made for process monitoring (Fezai et al., 2018; Jiang et al.,
2016; Zhang et al., 2019), soft sensing (Chiplunkar and
Huang, 2019; Kaneko and Funatsu, 2016; Poerio and Brown,
2018), outlier detection (Peng et al., 2012), and missing data
(Bao et al., 2015) purposes, which becomes a basic tool for
data-driven modelling. Because of the space limitation of this
paper, only a part of related works is reviewed here.
(3) Independent component analysis (ICA)
Motivated by the poor capability of PCA in dealing with
a non-Gaussian dataset, an emerging technique, namely,
independent component analysis (ICA) is used for modelling.
ICA aims to find those statistically independent components
that are non-Gaussian through a separating matrix (shown
in Figure 4), which is different form PCA. For this reason,
the non-Gaussianity is the fundamental constraint, which
means without this restriction, the estimation is impossible at
79
all. Different objective functions can be derived through the
inherent concept of the maximum non-Gaussianity, and such
optimised objective functions enable the estimation of the
ICA model (Hyvärinen and Oja, 2000). Originally, ICA was
proposed to deal with the problem of blind source separation
(Back and Weigend, 1997). Then, ICA was introduced into
data-driven modelling methods (Ding et al., 2017; Lee et al.,
2007; Liu and Zhang, 2015; Liu et al., 2019a; Zhang and Qin,
2007).
Figure 3 The relationship among PCA, PCR, PLS, and PLSR
(see online version for colours)
PCR
X T U Y
PLS
PT QT
PCA PLSR
Figure 4 The principle of ICA (see online version for colours)
X2 X2
Separating
Non-independent X1 Independent X1
(4) Probabilistic model
The probabilistic model has the similar structure as
LVP, which contains factor analysis (FA), probabilistic
PCA (PPCA), probabilistic PLS (PPLS), etc. Since the
process variable is often contaminated with uncertainties
or noises as the consequence of transmission disturbances
and measurement variations. It is more reasonable to carry
out the probabilistic framework in industrial processes by
considering the process variable as a random one (Yuan
et al., 2017). Taking the FA as an example, the original
variable space is treated as linear combination of factors
(shown in Figure 5), where, the variables, factors and noises
are formulated as the form of probability density function
(PDF). Therefore, different from the deterministic PCA, the
EM algorithm is always be used for maximum likelihood
estimation by repeating E-step and M-step (Ge, 2016). In
addition, the desired probabilistic model is decided by the
assumption of noise variance (Kim and Lee, 2003). The most
application on probabilistic model is to deal with the noised
data (Zhao et al., 2015; Raveendran et al., 2018; Zheng and
Song, 2018).
80 W. Guo et al.
Figure 5 The principle of FA (two factors) (see online version
for colours)
X boundary of a dataset can be estimated for detecting the novel
Noise
F2
α1
X̂
Unit
α2 circle
F1
(5) Canonical correlation analysis (CCA)
As the supervised learning method for process monitoring,
CCA projects original space dimensions of data onto the
most correlated space between X and Y (shown in Figure 6)
(Zhu et al., 2017). Thus, both Tx2 and Ty2 are required for
judgement. Among them, the first index is used for examining
whether a fault occurs in X by referring the correlated
information from Y . Similarly, the second index examines
the fault in Y by referring the correlated information from
X (Jiang et al., 2019). However, in order to obtain the key
variable, the predictive model has to get a high accuracy
extremely, or the key variable is assumed to be online
available or measurable without a large time delay (Liu et al.,
2018b). Due to lack of attention to variance analysis, CCA
is not as popular as other data-driven methods. However, it
is still an optional modelling method for industrial processes
modelling (Ogura, 2010; Pilario and Cao, 2018).
3.1.2 Nonlinear models
(1) Support vector model
The SVM and the support vector regression (SVR) are two
well known nonlinear supervised learning methods. Among
which, the SVM is associated with binary classification
problems, and the SVR is most used for regression. Generally,
without the consideration of constrained differences in
optimisation objective function, SVM and SVR can be unified
as SVM (Kadlec et al., 2009). The essence of SVM is to find a
best-fitting function, and a search of the decision hyperplane
is developed for splitting a given dataset into two sub-datasets
(Shown in Figure 7). The fundamental advantage is that
the SVM utilises the precept of structural risk minimisation
(SRM) to balance the complexity of the approximating
function against the data quality. Meanwhile, a suitable
optimisation strategy can overcome the shortage in dealing
with large dataset modelling (Guo et al., 2016). Kaneko and
Funatsu (2014) developed an online soft sensor for plants
with various states, Pooyan et al. (2015) separated each
faulty class from the others and detected the unknown space
between those faults, Ben Ishak (2016) made a comparative
study of variable selection by using SVM and RF, etc.
Besides, by compressing the binary-class into one-
class, the SVM is specified as the support vector data
description (SVDD). It is considered as a special form of
SVM for dealing with unsupervised problems (Quiñones-
Grueiro et al., 2019). With the volume minimum principle
of a hypersphere in feature spaces, the spherically shaped
data or outliers. The most application related to SVDD is the
model development for process monitoring (Ge et al., 2017).
Figure 6 The principle of CCA (see online version for colours)
Maximum
X correlation Y
X* Y*
PT QT
Figure 7 The principle of SVM (see online version for colours)
Y
Hyperplane
Support vectors
Margin
X
(2) Artificial intelligence (AI) techniques
As the development of modern computers, artificial
intelligence (AI) techniques have been extensively used
in several different fields, including process modelling,
computer vision, natural language processing, and
bioinformatics, etc (Gopakumar et al., 2018). The basic unit
of AI networks is named as neurons, which is inspired by
biological neural systems. All neurons and the associated
input-output are connected by numerical weights directly.
By tuning the value of each weight, make them adapt to
inputs and be capable of learning like human brains. The
AI technique is a family of methods contained several
architectures of the network, such as feature extraction
network (FEN), multi-layers perceptron (MLP), and recurrent
neural network (RNN).
FEN is a type of unsupervised learning method for
producing a nonlinear feature space from the original space
of data. The application related to FEN also includes several
methods: Liukkonen et al. (2013) developed a self-organising
map (SOM) driven based approach for monitoring industrial
wastewater treatment. Shang et al. (2014) pre-trained a series
of process data weights using the deep belief network (DBN),
which is regarded as the initial weight of the subsequent
supervised learning for product quality prediction in the
crude-oil distillation unit. Wu and Zhao (2018) presented a
 A review on data-driven approaches for industrial process modelling
fault diagnosis algorithm for the Tennessee Eastman (TE)
process, which uses the convolutional neural network (CNN)
for feature extraction.
The MLP is an extension of traditional artificial neural
network (ANN) with several hidden layers. Liukkonen et al.
(2012) built a dynamic MLP model used for the nitrogen
oxide content estimation of the flue gas. Souza et al. (2013)
utilised MLP for variable selection, which is assumed that
the mean square errors of two identified results with different
inputs are similar.
RNN is a time-correlating neural work model, which is
reflected in the memory networks. Such mnemonic ability
in RNN is incorporated with the state of neurons, which is
stored and used for the prediction of the next state sequence
(Smith and Jin, 2014). Sun and Ge (2019) developed a soft
sensor model using Gaussian-Bernoulli restricted Boltzmann
machine and RNN on carbon dioxide absorption column.
Duchanoy et al. (2017) used RNN successfully predict the
area of car tyres contacted with the ground.
In addition, the relationship of above techniques is shown
in Figure 8.
Figure 8 The relationship between FEN, MLP, and RNN
(see online version for colours)
Features
MLP
Y
FEN
RNN
Previous neurons Current neurons
X
(3) Nonlinear latent variable projection (NLVP) model
In order to extend the linear LVP model to the nonlinear
applications, there are three main methods, the input
extension model, the polynomial projection model, and
the kernel LVP model (shown in Figure 9). The input
extension is the simplest model of the nonlinear latent
variable projection (NLVP). It extends the variable space
with associated nonlinear combinations (such as square
values, cross-products, logarithms) of the original input.
Then the linear LVP analysis on the extended data space is
performed (Baffi et al., 1999). Wold et al. (1989) presented
the polynomial projection algorithm, in which, inner relations
between LVs were modified as a nonlinear (polynomial)
relationship. Due to the lack of prior knowledge related to
underlying nonlinear relationships, these two methods are not
getting as much attention as the kernel LVP.
The kernel LVP (KLVP) retains the linear framework of
the traditional LVP. It firstly maps the original inputs onto a
feature space, namely, the reproducing kernel Hilbert space
(RKHS). The RKHS can be treated as a generalised space
form in SVM and GPR (Rosipal and Trejo, 2001). Then the
LV can be obtained on such functional space. The kernel
based method is widely used in industrial processes, which
81
includes model division (Khediri et al., 2012; Zhai et al.,
2018), soft sensor (Yuan et al., 2014a; Zhang et al., 2017),
and process monitoring (Fan et al., 2014; Fezai et al., 2016,
2018), etc.
Figure 9 The principle of three nonlinear LVP models (see online
version for colours)
X
Xe T X Tp Xk T
PT PT PT
Input extension Polynomial Kernel LVP
model projection model model
(4) Gaussian process regression (GPR)
The GPR is inspired by the concept of Gaussian process
(GP), which is used for representing a distribution over
functions from the viewpoint of the normal distribution.
Statistically, a sample in original spaces is associated with a
random variable that obeys a Gaussian distribution. The GP
can be seen as a generalised infinite-dimensions functional
space of multivariate normal distributions (Rasmussen and
Williams, 2006). The inference of continuous variables with
a GP prior in Bayesian framework is known as GPR.
By referencing the distribution over functional space and
Bayesian inference, the posterior distribution of requested
estimation is provided (shown in Figure 10). Meanwhile,
such distribution estimation is a crucial difference between
the GPR and other methods. In recent years, GPR has
been mostly used for applications on the predictive purpose,
such as soft sensing modelling, output or quality related
monitoring; Zhou et al. (2015) developed an adaptive quality
monitoring model with the use of predictive errors from the
GPR in the fed-batch process; Liu et al. (2017) made a
multi-step prediction using GPR for removing pollutants in
wastewater treatment plants; DuyTrinh et al. (2019) presented
a fuzzy neural network combined GPR model for predicting
the grinding wheel wear and surface roughness of alloy.
Besides, Wang and Mao (2019) also extended the GPR for
outlier detection.
3.1.3 Adaptive models
(1) Offset-based calibration
The output offset comes from suspecting data reliability
or grade-drifting caused by aging sensors. The offset-based
calibration technique uses the deviation between the output
82 W. Guo et al.
feedback and the online prediction to fix the upcoming
predictive output (Ahmed et al., 2009). Therefore, this
method is mainly used for applications under supervised
learning frameworks. For example, Ni et al. (2011) used
the output basis robustly update the GPR model by adding
an offset smoother; Ahmed et al. (2015) used the offset-
based model into the least squares support vector machine
(LSSVM) to predict the nitrogen oxide discharged from the
coal-fired power plant.
Figure 10 Illustration of GPR for the posterior distributions
(see online version for colours)
5
True function Confidence
interval
4 Training data
3
2 models are firstly constructed from the training set. And
Y
1 which also includes a membership function. Then, a final
0 from all sub-models (shown in Figure 11). For data-driven
−1
−1.5 −1 −0.5 0 0.5 1 1.5
X set have similarities among them and differences from the
(2) Recursive method
The recursive method uses the previously obtained model
and the current feedback (including sample-wise and block-
wise operations) to update the model, which avoids a
recalculation and the modelling cost. There are two widely
used strategies for the model maintenance, i.e., the direct
calculation approach and the data replacement approach.
In direct calculation approaches, the desired parameter is
obtained directly by adding the old parameters with an
incremental term. The typical method of such approach
includes the recursive least squares (RLS) (LeBreux et al.,
2012), the recursive Bayesian parameter estimation (Jing
et al., 2017), and the Kalman filter (Ma et al., 2009). In
data replacement approaches, continuously accumulate data
can be replaced by the equivalent matrix stored by the
last iteration step. In this way, it unnecessary to store the
original data, such as recursive PCA (RPCA) (Hu et al.,
2014), recursive PLS (RPLS) (Poerio and Brown, 2018) and
recursive GPR (Zhou et al., 2015). In addition, the forgetting
factor can also be used as the definition of adaptive strength
for recursive models (Kadlec et al., 2011).
(3) Moving window (MW) approach
The moving window (MW) strategy is considered as a
case, in which the old model is calibrated using a set of
measurements. Those measurements contain the maximum
relevance under the current working condition. Commonly,
a window with a fixed-length containing the most recent
measurements is employed, which ensures the maximum
relevance to the current condition (Kadlec et al., 2011). In the
MW, the newest samples are received and the same number
of oldest measurements is excluded as the window slides
along the record sequence. Then the identified model needs
to be recalculated as well (Fezai et al., 2016; Jiang and Yan,
2013). Sometimes, it is unnecessary to build a new model
immediately when the window has been slid. Therefore, the
hypothesis-testing scheme could make a decision whether the
model needs to be modified (Shao and Tian, 2015; Yao and
Ge, 2017).
3.2 Different model structures
3.2.1 Switch-based model
In switch-based models, several specified amounts of sub-
each sub-model corresponds with a steady mode condition,
model is obtained by choosing the maximum membership
models, the sub-model is always unknown for process
modelling, and it is essential to separate the database into
several exclusive sub-sets by using clustering algorithms.
Subsequently, those measurements belonging to the same
measurements belonging to the other sets (Quiñones-Grueiro
et al., 2019).
Figure 11 The framework of switch-based model (see online
version for colours)
Sub-model
1
Sub-model
2
Final
model
Training set Sub-model
3
Sub-model
M
The k-means method generates several measurement clusters
with some criteria such as Euclidean distance, Mahalanobis
distance, or angle, which is widely used for switch-based
models, for example: Wang et al. (2012) utilised k-means
algorithm for the model division and a two-step analysis of
ICA-PCA is introduced for furnace temperature monitoring
in continuous annealing line; Khediri et al. (2012) separated
the nonlinear process model by using kernel k-means
clustering; Du et al. (2017) defined a new similarity index
between different models using k-means clustering. Besides
the k-means clustering technique, the piecewise affine (PWA)
model computes the affine map that by using identified
parameters of each sub-model and corresponding regressor
 A review on data-driven approaches for industrial process modelling
space (Bemporad et al., 2003; Breschhi et al., 2016). Both k-
means and PWA require the specified cluster number at the
initialisation step. Alternatively, the cluster number also can
be determined during the clustering step by using the affinity
propagation (AP) clustering algorithm (Liang et al., 2019;
Zhang et al., 2018), or the subtractive clustering algorithm
(Pan et al., 2010; Norhayati and Rashid, 2018).
3.2.2 Mixture model
Different from the switch-based division strategy, the mixture
model converts those weighted sub-outputs into the final
output with a linear superposition (shown in Figure 12).
The main issue of the mixture model is how to assign
the weight to each sub-model. Several mixture algorithms
have been presented including different assignment rules.
For example, the fuzzy clustering method is an extension of
k-means, in which the sample has a series of membership
values to match each sub-model (Quiñones-Grueiro et al.,
2019). Moreover, by taking different distance spaces into
consideration, Gustafson-Kessel (GK) clustering (Kim et al.,
2004; Teslić et al., 2011) and the fuzzy c-means (FCM)
clustering (Askari et al., 2017; Qi et al., 2014) are specified.
Figure 12 The framework of mixture model (see online version
for colours)
ω1
Sub-model
1 Besides the bagging and boosting methods, the ensemble
Sub-model ω2
2
Final
∑ to the input data (Ge and Song, 2014; Jadhav et al., 2014).
Training set Sub-model ω3 model
3
ωM
Sub-model
M (Poerio and Brown, 2018; Shao and Tian, 2015).
Additionally, the Gaussian mixture model (GMM) refers to
a probabilistic model, in which, the distribution of a dataset
is constructed as the composite model by using a finite
number of local Gaussian distributions. Subsequently, the
membership value is replaced by the posterior probability
of observations to each sub-model through the Bayesian
framework. Considering a sufficient amount of Gaussian
components and their linear combinations, the GMM could
approximate to almost any continuous density (Yuan et al.,
2014b). Thus, the GMM shows strong ability in complicated
industrial processes (Frigieri et al., 2016; Peng et al., 2015).
Besides, the GMM is also helpful in the non-Gaussian data
processing (Ge et al., 2017).
3.2.3 Ensemble learning model
Ensemble learning is a powerful method, which integrates
several sub-models to generate the final model. This method
seems to be equivalent to the mixture model as discussed
before. In practice, the ensemble learning is a type of random
re-sampling algorithm rather than model division techniques
(Cao et al., 2010). One sample could be extracted more than
83
one time or zero, which leads to the diversity of samples in
each sub-model (shown in Figure 13).
Figure 13 The framework of ensemble learning model
(see online version for colours)
Nonduplicate
data extraction
Base ω1
model 1
Base ω2
model 2
Training set
Base ω3
model 3
Re-sampling
Base ωM ∑ Final
model M
model
The ensemble learning is especially applicable for sensitive
models (Ge and Song, 2014). The bootstrap aggregation
(bagging) and the AdaBoost family of algorithms (boosting)
are two popular techniques. Bagging creates several
individuals to construct an ensemble model, in which, every
classifier is trained with a random redistribution of the
dataset (Breiman, 1996). Boosting combines the rough and
moderately inaccurate rules of thumb in order to produce
an accurate rule for prediction, which provides a general
and provably effective predictive model (Cao et al., 2010).
method on the random subspace carries out the re-sampling
though the direction of variables (or feature space) related
Nowadays, ensemble learning has become a more generalised
model than the defined structure and the data collection of
each individual model is becoming more and more flexible
3.2.4 Local learning model
Local learning also termed as lazy learning or just-in-time
learning (JITL) is an instance-based learning algorithm,
which is mostly used for supervised learning (Cheng and
Chiu, 2004). Different from the traditional modelling method,
local learning builds a disposable dynamic model upon
the query (shown in Figure 14). Such disposable model is
consisted of selected relevant samples by using k-nearest
neighbours (kNN) criterion. Then, the predictive output of the
query is obtained depending on whether the local model is
used for regression or classification (Ge et al., 2017).
Local learning provides an attractive alternative method
for nonlinear process modelling by extracting the local linear
model from the training set. For example, Cheng and Chiu
(2005) utilised the residual between local model predictions
and process outputs to monitor the reaction process of
continuous stirred tank reactors (CSTR); Liu et al. (2012)
proposed a cumulative similarity factor in the local model,
which includes a weighted similarity of query and a size
of the relevant set; Hu et al. (2013) used the local kernel
space for the batch process monitoring; Su et al. (2016)
proposed the local state-space model for representing the
84 W. Guo et al.
high nonlinearity in batch processes; Yuan et al. (2017)
constructed a local weighted log-likelihood function to
approximate the nonlinear relationship of data; Zheng et al.
(2018) developed a semi-supervised soft sensing model by
extracting the information from the local model including
both labelled and unlabelled data.
Figure 14 The framework of instance-based local model
(see online version for colours)
Training set
Instance-based model
Local
Query Relevant model Output
data
3.2.5 Transfer learning model
As a sub-field of machine learning techniques, transfer
learning is used to deal with the target domain without or little
amount of labelled data, which is disabled for training any
model. A transferred model can be constructed by permitting
the difference in distributions, domains, and tasks in training
and testing phase (Pan and Yang, 2010). Of cause, it should
be on the assumption that enough samples are contained in
at least one related source domain, which provides a primary
basis for modelling and associated information extraction.
Additionally, the proper identification of the source domain is
essential as the sample obtained from dramatically different
source domains may adversely affect the training process,
which is termed as negative transfer (Salaken et al., 2017).
Transfer learning is mostly used in image recognition and
text classification, etc. (Pan and Yang, 2010). Until recent
years, it has been introduced to the process modelling (Facco
et al., 2012, 2014; Kang, 2018; Liu et al., 2019b; Shao et al.,
2019; Tomba et al., 2014). Actually, the transfer learning
is a kind of domain-based adaptation method with a model
trained on the source domain and calibrated on the target
domain (Kang, 2018; Salaken et al., 2017).
4 Perspectives for future research
Over the past decades, data-driven approaches used for
industrial processes have continually caught widespread
attention from both academics and engineers, which takes
the benefit from the distributed control systems (DCS) and
the power of modern computers. Meanwhile, more and more
new developed data-driven algorithms have been introduced
to cooperate new industrial characteristics. However, some
fundamental issues related to data-driven approaches should
be considered for process modelling.
4.1 Practical applications
Challenges of the development from theoretical researches to
practical applications are still urgently needed to be addressed
(Quiñones-Grueiro et al., 2019). This is mainly because
the practical application is more like a stochastic discrete
event system on aspects of multi-rate sampling, missing
data, dynamic character, and maintenance frequency. Such
uncertain elements should be put in place for the development
of process monitoring, soft sensing, fault diagnosis, etc. From
the practical viewpoint, the proposal of process modelling
should handle this issue currently rather than waiting for
offline analysis.
4.2 Data mining and analysing
With the increasing requirement of the product quality,
modern plant-wide processes are becoming more and more
complex. Most of modelling techniques are limited to product
quality prediction, normal working condition monitoring,
fault classification, and model division. However, tracing
back to the root cause of process changes is still difficult.
Therefore, a cause-to-effect relationship among the dataset
should be considered for further analysis, which reveals
the root cause of process changes and directs the process
operation. Although several methods have been used, e.g.,
Bayesian network (Gonzalez et al., 2015) and relative
importance analysis (Yan et al., 2017), investigations for the
application in plant-wide processes are still needed to be
executed (Ge, 2017).
4.3 Diversified data
As the coming era of big data, massive volumes of data are
collected from industrial processes. Besides the depth of data
mining, the diversification of collected data is also needed
to be paid attention, such as spectral data, image data, audio
data, video data, and so on. Basically, the diversified data
still contains information related to the nature of industrial
processes. Meanwhile, such data is much easier and faster
to collect. However, how to combine the traditional data and
the diversified data, or how to utilise the diversified data
independently for process modelling is a great challenge for
future works.
4.4 Economic model
Due to the factor of sustainability, recently, not only
for researches of energy efficiency, but also for matters
related to environmental protection have been taken into
consideration for process modelling (Hanes and Bakshi,
2015). As mentioned in early section, modelling methods
actually can be performed as optimisation problems. By
introducing new constraints inspired by the economic model
predictive control (EMPC) (Ellis et al., 2014), both energy
efficiency and environmental sustainability of the process
should be monitored as well. To this end, how to effectively
incorporate the process knowledge into industrial processes
needs more consideration in future works.
 A review on data-driven approaches for industrial process modelling
5 Conclusion
In this paper, several most widely used machine learning
algorithms for industrial process modelling are reviewed,
with the methodology, the mathematical method, and the
structure selection. Discussions related to extensions and
challenges are also carried out for future researches. Actually,
data-driven modelling is a field relying on information
fusion. From the perspective of algorithms, none of them
can be treated as a universal method for addressing any
industrial situation. From the perspective of applications,
similarly, one modelling task may need several algorithms to
be implemented. Therefore, the data-driven method has an
important potential for practical applications and academic
researches, which plays an important role in industrial
processes.
Acknowledgement
The project is funded in part by the National Natural Science
Foundation under Grant 61873113, Key R&D Program of
Jiangsu Province, China under Grant BE2018370, and the
Postgraduate Research and Practice Innovation Program of
Jiangsu Province (KYCX17 1785).
References
Abyaneh, H.Z. (2014) ‘Evaluation of multivariate linear regression
and artificial neural networks in prediction of water quality
parameters’, Journal of Environmental Health Science and
Engineering, Vol. 12, p.40.
Ahmad, I., Kano, M., Hasebe, S., Kitada, H. and Murata, N. (2014)
‘Gray-box modeling for prediction and control of molten steel
temperature in Tundish’, Journal of Process Control, Vol. 24,
No. 4, pp.375–382.
Ahmed, F., Cho, H.J., Kim, J.K., Seong, N.U. and Yeo, Y.K. (2015)
‘A recursive pls-based soft sensor for prediction of the melt
index during grade change operations in hdpe plant’, Korean
Journal of Chemical Engineering, Vol. 32, No. 6, pp.1029–
1036.
Ahmed, F., Nazir, S. and Yeo, Y.K. (2009) ‘A recursive pls-based
soft sensor for prediction of the melt index during grade
change operations in HDPE plant’, Korean Journal of Chemical
Engineering, Vol. 26, No. 1, pp.14–20.
Askari, S., Montazerin, N., Fazel Zarandi, M.H. and Hakimi,
E. (2017) ‘Generalized entropy based possibilistic fuzzy c-
means for clustering noisy data and its convergence proof’,
Neurocomputing, Vol. 219, pp.186–202.
Back, A.D. and Weigend, A.S. (1997) ‘A first application of
independent component analysis to extracting structure from
stock returns’, International Journal of Neural Systems, Vol. 8,
No. 4, pp.473–484.
Baffi, G., Martin, E.B. and Morris, A.J. (1999) ‘Non-linear
projection to latent structures revisited: the quadratic pls
algorithm’, Computers and Chemical Engineering, Vol. 23,
pp.395–411.
85
Bao, L., Yuan, X.F. and Ge, Z.Q. (2015) ‘Co-training partial least
squares model for semi-supervised soft sensor development’,
Chemometrics and Intelligent Laboratory Systems, Vol. 147,
pp.75–85.
Bemporad, A., Garulli, A., Paoletti, S. and A, V. (2003) ‘A greedy
approach to identification of piecewise affine models’, Hybrid
Systems: Computation and Control, Vol. 2623, pp.97–112.
Ben Ishak, A. (2016) ‘Variable selection using support vector
regression and random forests: a comparative study’, Intelligent
Data Analysis, Vol. 20, pp.83–104.
Bishop, C.M. (2006) Pattern Recognition and Machine Learning,
Springer-Verlag, New York.
Breiman, L. (1996) ‘Bagging predictors’, Machine Learning,
Vol. 24, No. 2, pp.123–140.
Breschhi, V., Piga, D. and Bemporad, A. (2016) ‘Piecewise
affine regression via recursive multiple least squares and
multicategory discrimination’, Automatica, Vol. 70, pp.155–
162.
Cao, D.S., Xu, Q.S., Liang, Y.Z., Zhang, L.X. and Li, H.D. (2010)
‘The boosting: A new idea of building models’, Chemometrics
and Intelligent Laboratory Systems, Vol. 100, pp.1–11.
Cheng, C. and Chiu, M.S. (2004) ‘A new data-based methodology
for nonlinear process modeling’, Chemical Engineering
Science, Vol. 59, pp.2701–2810.
Cheng, C. and Chiu, M.S. (2005) ‘Nonlinear process monitoring
using JITL-PCA’, Chemometrics and Intelligent Laboratory
Systems, Vol. 76, pp.1–13.
Chiplunkar, R. and Huang, B. (2019) ‘Output relevant slow feature
extraction using partial least squares’, Chemometrics and
Intelligent Laboratory Systems, Vol. 191, pp.148–157.
Ding, Z.Y., Zhang, J. and Liu, Y. (2017) ‘Ensemble non-gaussian
local regression for industrial silicon content prediction’, ISIJ
International, Vol. 57, No. 11, pp.2022–2027.
Du, W.Y., Fan, Y.P. and Zhang, Y.W. (2017) ‘Multimode process
monitoring based on data-driven method’, Journal of the
Franklin Institute-Engineering and Applied Mathematics,
Vol. 345, No. 6, pp.2613–2627.
Duchanoy, C.A., Moreno-Armendariz, M.A., Urbina, L., Cruz-
Villar, C.A., Calvo, H. and Rubio, J.D. (2017) ‘A novel
recurrent neural network soft sensor via a differential evolution
training algorithm for the tire contact patch’, Neurocomputing,
Vol. 235, pp.71–82.
DuyTrinh, N., Yin, S.H., Tan, N.N., Son, P.X. and Duc, L.A. (2019) ‘
‘A new method for online monitoring when grinding Ti-6Al-4V
alloy’, Materials and Manufacturing Processes, Vol. 34, No. 1,
pp.39–53.
Ellis, M., Durand, H. and Christofides, P.D. (2014) ‘A tutorial review
of economic model predictive control methods’, Journal of
Process Control, Vol. 24, No. 8, pp.1156–1178.
Eyvazian, M., Noorossana, R., Saghaei, A. and Amiri, A. (2011) ‘
‘Phase II monitoring of multivariate multiple linear regression
profiles’, Quality and Reliability Engineering International,
Vol. 27, No. 3, pp.281–296.
Facco, P., Largoni, M., Tomba, E., Bezzo, F. and Barolo, M. (2014)
‘Transfer of process monitoring models between plants: Batch
systems’, Chemical Engineering Research and Design, Vol. 92,
No. 2, pp.273–284.
Facco, P., Tomba, E., Bezzo, F. and Garcı́a-Muñoz, S. (2012)
‘Transfer of process monitoring models between different
plants using latent variable techniques’, Industrial and
Engineering Chemistry Research, Vol. 51, No. 21, pp.7327–
7339.
86 W. Guo et al.
Fan, J.C., Qin, S.J. and Wang, Y.Q. (2014) ‘Online monitoring of
nonlinear multivariate industrial processes using filtering kica-
pca’, Control Engineering Practice, Vol. 22, pp.205–216.
Fezai, R., Mansouri, M., Taouali, O., Harkat, M.F. and Bouguila,
N. (2018) ‘Online reduced kernel principal component analysis
for process monitoring’, Journal of Process Control, Vol. 61,
pp.1–11.
Fezai, R., Taouali, O., Harkat, M.F. and Bouguila, N. (2016) ‘A
new fault detection method for nonlinear process monitoring’,
International Journal of Advanced Manufacturing Technology,
Vol. 87, No. 9-12, pp.3425–3436.
Frigieri, E.P., Campos, P.H.S., Paiva, A.P. and Balestrassi, P.P.
(2016) ‘A mel-frequency cepstral coefficient-based approach
for surface roughness diagnosis in hard turning using acoustic
signals and gaussian mixture models’, Applied Acoustics,
Vol. 113, pp.230–237.
Ge, Z.Q. (2016) ‘Supervised latent factor analysis for process
data regression modeling and soft sensor application’, IEEE
Transactions on Control Systems Technology, Vol. 24, No. 3,
pp.1004–1011.
Ge, Z.Q. (2017) ‘Review on data-driven modeling and monitoring
for plant-wide industrial processes’, Chemometrics and
Intelligent Laboratory Systems, Vol. 171, pp.16–25.
Ge, Z.Q. and Song, Z.H. (2012) ‘Multivariate statistical process
monitoring using modified factor analysis and its application’,
Journal of Chemical Engineering of Japan, Vol. 45, No. 10,
pp.829–839.
Ge, Z.Q. and Song, Z.H. (2014) ‘Ensemble independent component
regression models and soft sensing application’, Chemometrics
and Intelligent Laboratory Systems, Vol. 130, pp.115–122.
Ge, Z.Q., Song, Z.H., Ding, S.X. and Huang, B. (2017) ‘Data mining
and analytics in the process industry: The role of machine
learning’, IEEE Access, Vol. 5, pp.20590–20616.
Geladi, P. and Kowalski, B.R. (1986) ‘ ‘Partial least-squares
regression: a tutorial’, Analytica Chimica Acta, Vol. 185,
pp.1–17.
Gonzalez, R., Huang, B. and Lau, E. (2015) ‘Process monitoring
using kernel density estimation and bayesian networking with
an industrial case study’, ISA Transactions, Vol. 58, pp.330–
347.
Gopakumar, V., Tiwari, S. and Rahman, I. (2018) ‘A deep learning
based data driven soft sensor for bioprocesses’, Biochemical
Engineering Journal, Vol. 136, pp.28–39.
Guo, W., Pan, T.H. and Li, Z.M. (2016) ‘Development of a soft
sensor for processes with multiple operating regimes using
adaptive multi-state partial least squares regression’, Journal of
the Taiwan Institute of Chemical Engineers, Vol. 67, pp.20–28.
Hanes, R.J. and Bakshi, B.R. (2015) ‘ ‘Sustainable process design
by the process to planet framework’, AICHE Journal, Vol. 61,
No. 10, pp.3320–3331.
Hapfelmeier, A. and Ulm, K. (2013) ‘ ‘A new variable selection
approach using random forests’, Computational Statistics and
Data Analysis, Vol. 60, pp.50–69.
Hu, Y., Ma, H.H. and Shi, H.B. (2013) ‘Enhanced batch process
monitoring using just-in-time-learning based kernel partial
least squares’, Chemometrics and Intelligent Laboratory
Systems, Vol. 123, pp.15–27.
Hu, Z.K., snd W.H.Gui, Z.W.C. and Jiang, B. (2014) ‘Adaptive pca
based fault diagnosis scheme in imperial smelting process’, ISA
Transactions, Vol. 53, No. 5, pp.1446–1455.
Huang, J.Y. and Zhao, J. (2018) ‘Identification of multi-model lpv
model with two scheduling variables using transition test’,
International Journal of Modelling, Identification and Control,
Vol. 29, No. 1, pp.31–43.
Hyvärinen, A. and Oja, E. (2000) ‘Independent component analysis:
algorithms and applications’, Neural networks, Vol. 13,
Nos. 4–5, pp.411–430.
Jadhav, S., Nalbalwar, S. and Ghatol, A. (2014) ‘Feature elimination
based random subspace ensembles learning for ecg arrhythmia
diagnosis’, Soft Computing, Vol. 18, No. 3, pp.579–587.
Jiang, Q.C. and Yan, X.F. (2013) ‘Weighted kernel principal
component analysis based on probability density estimation
and moving window and its application in nonlinear chemical
process monitoring’, Chemometrics and Intelligent Laboratory
Systems, Vol. 127, pp.212–131.
Jiang, Q.C., Yan, X.F. and Huang, B. (2016) ‘Performance-driven
distributed PCA process monitoring based on fault-relevant
variable selection and bayesian inference’, IEEE Transactions
on Industrial Electronics, Vol. 63, No. 1, pp.377–386.
Jiang, Q.C., Yan, X.F. and Huang, B. (2019) ‘Review and
perspectives of data-driven distributed monitoring for industrial
plant-wide processes’, Industrial and Engineering Chemistry
Research, Vol. 58, pp.12899–12912.
Jing, S.X., Pan, T.H. and Li, Z.M. (2017) ‘Variable knot-based
spline approximation recursive bayesian algorithm for the
identification of wiener systems with process noise’, Nonlinear
Dynamics, Vol. 90, No. 4, pp.2293–2303.
Kadlec, P., Gabrys, B. and Strandt, S. (2009) ‘Data-driven soft
sensors in the process industry’, Computers and Chemical
Engineering, Vol. 33, pp.795–814.
Kadlec, P., Grbic, R. and Gabrys, B. (2011) ‘Data-driven soft sensors
in the process industry’, Computers and Chemical Engineering,
Vol. 35, No. 1, pp.1–24.
Kaneko, H. and Funatsu, K. (2014) ‘ ‘Adaptive soft sensor based
on online support vector regression and bayesian ensemble
learning for various states in chemical plants’, Chemometrics
and Intelligent Laboratory Systems, Vol. 137, pp.57–66.
Kaneko, H. and Funatsu, K. (2016) ‘ ‘Ensemble locally weighted
partial least squares as a just-in-time modeling method’,
AICHE Journal, Vol. 62, No. 3, pp.717–725.
Kang, S. (2018) ‘On effectiveness of transfer learning approach
for neural network-based virtual metrology modeling’, IEEE
Transactions on Semiconductor Manufacturing, Vol. 31, No. 1,
pp.149–155.
Khatibisepehr, S., Huang, B. and Khare, S. (2013) ‘Design of
inferential sensors in the process industry: A review of bayesian
methods’, Journal of Process Control, Vol. 23, pp.1575–1596.
Khediri, I.B., Weihs, C. and Liman, M. (2012) ‘Kernel k-means
clustering based local support vector domain description
fault detection of multimodal processes’, Expert Systems with
Applications, Vol. 39, No. 2, pp.2166–2171.
Kim, D. and Lee, I.B. (2003) ‘Process monitoring based on
probabilistic PCA’, Chemometrics and Intelligent Laboratory
Systems, Vol. 67, pp.109–123.
Kim, Y.I., Kim, D.W., Lee, D. and Lee, K.H. (2004) ‘A cluster
validation index for GK cluster analysis based on relative
degree of sharing’, Information Sciences, Vol. 168, Nos. 1–4,
pp.225–242.
LeBreux, M., Desilets, M. and Lacroix, M. (2012) ‘Control of
the ledge thickness in high-temperature metallurgical reactors
using a virtual sensor’, Inverse Problems in Science and
Engineering, Vol. 20, No. 8, pp.1215–1238.
 A review on data-driven approaches for industrial process modelling
Lee, J.M., Qin, S.J. and Lee, I.B. (2007) ‘Fault detection of non-
linear processes using kernel independent component analysis’,
Canadian Journal of Chemical Engineering, Vol. 85, No. 4,
pp.526–536.
Liang, H.B., J, L.Z., Khan, M.J. and Han, J.X. (2019) ‘An sand
plug of fracturing intelligent early warning model embedded
in remote monitoring system’, IEEE Access, Vol. 7, pp.47944–
47954.
Liu, Y. and Zhang, G. (2015) ‘Scale-sifting multiscale nonlinear
process quality monitoring and fault detection’, Canadian
Journal of Chemical Engineering, Vol. 93, No. 8, pp.1416–
1425.
Liu, Y., Gao, Z.L., Li, P. and Wang, H.Q. (2012) ‘Just-in-time kernel
learning with adaptive parameter selection for soft sensor
modeling of batch processes’, Industrial and Engineering
Chemistry Research, Vol. 51, No. 11, pp.4313–4327.
Liu, Q., Qin, S.J. and Chai, T.Y. (2014) ‘Multiblock concurrent
pls for decentralized monitoring of continuous annealing
processes’, IEEE Transactions on Industrial Electronics,
Vol. 61, No. 11, pp.6429–6437.
Liu, Y.Q., Pan, Y.P., Huang, D.P. and Wang, Q.L. (2017)
‘Fault prognosis of filamentous sludge bulking using an
enhanced multi-output Gaussian processes regression’, Control
Engineering Practice, Vol. 62, pp.46–54.
Liu, R.S., Peng, M.F. and Xiao, X.H. (2018a) ‘Ultra-short-term
wind power prediction based on multivariate phase space
reconstruction and multivariate linear regression’, Energies,
Vol. 11, No. 10, pp.2763.
Liu, Y.Q., Liu, B., Zhao, X.J. and Xie, M. (2018b) ‘A mixture
of variational canonical correlation analysis for nonlinear and
quality-relevant process monitoring’, IEEE Transactions on
Industrial Electronics, Vol. 65, No. 8, pp.6478–6486.
Liu, S.T., Gao, X.W., Qi, W.H. and Zhang, S.M. (2019a) ‘Soft sensor
modelling of propylene conversion based on a takagi-sugeno
fuzzy neural network optimized with independent component
analysis and mutual information’, Transactions of the Institute
of Measurement and Control, Vol. 41, No. 3, pp.737–748.
Liu, Y., Yang, C., Liu, K.X., Chen, B.C. and Yao, Y.
(2019b) ‘Domain adaptation transfer learning soft sensor
for productquality prediction’, Chemometrics and Intelligent
Laboratory Systems, Vol. 192, p.103813.
Liukkonen, M., Hälikkä, E., Hiltunen, T. and Hiltunen, Y. (2012)
‘Dynamic soft sensors for NOx emissions in a circulating
fluidized bed boiler’, Environmental Modelling and Software,
Vol. 97, pp.1483–490.
Liukkonen, M., Laakso, I. and Hiltunen, Y. (2013) ‘Advanced
monitoring platform for industrial wastewater treatment:
Multivariable approach using the self-organizing map’,
Environmental Modelling and Software, Vol. 48, pp.193–2016.
Lu, L., Yan, J.H. and de Silva, C.W. (2016) ‘Feature selection
for ecg signal processing using improved genetic algorithm
and empirical mode decomposition’, Measurement, Vol. 94,
pp.372–381.
Ma, M.D., Ko, J.W., Wang, S.J., Wu, M.F., Jang, S.S., Shieh, S.S.
and Wong, D.S.H. (2009) ‘Development of adaptive soft sensor
based on statistical identification of key variables’, Control
Engineering Practice, Vol. 17, No. 9, pp.1026–1034.
Ni, W.D., Tan, S.K. and Ng, W.J. (2011) ‘Recursive gpr for nonlinear
dynamic process modeling’, Chemical Engineering Journal,
Vol. 173, pp.636–643.
87
Norhayati, I. and Rashid, M. (2018) ‘Adaptive neuro-fuzzy
prediction of carbon monoxide emission from a clinical
waste incineration plant’, Neural Computing and Applications,
Vol. 30, No. 10, pp.3049–3061.
Ogura, T. (2010) ‘ ‘A variable selection method in principal
canonical correlation analysis’, Computational Statistics and
Data Analysis, Vol. 54, No. 4, pp.1117–11237.
Pan, S.J. and Yang, Q.A. (2010) ‘A survey on transfer learning’,
IEEE Transactions on Knowledge and Data Engineering,
Vol. 22, No. 10, pp.1345–1359.
Pan, T.H., Wong, D.S.H. and Jang, S.S. (2010) ‘Development of a
novel soft sensor using a local model network with an adaptive
subtractive clustering approach’, Industrial and Engineering
Chemistry Research, Vol. 49, No. 10, pp.4738–4747.
Peng, J.T., Peng, S.L. and Hu, Y. (2012) ‘Partial least squares
and random sample consensus in outlier detection’, Analytica
Chimica Acta, Vol. 719, pp.24–29.
Peng, K.X., Zhang, K., You, B. and Dong, J. (2015) ‘Quality-
related prediction and monitoring of multi-mode processes
using multiple pls with application to an industrial hot strip
mill’, Neurocomputing, Vol. 168, pp.1094–1103.
Piepho, H.P. (2009) ‘Ridge regression and extensions for
genomewide selection in maize’, Crop Science, Vol. 49, No. 4,
pp.1165–1176.
Pilario, K.E.S. and Cao, Y. (2018) ‘Canonical variate dissimilarity
analysis for process incipient fault detection’, IEEE
Transactions on Industrial Informatics, Vol. 14, No. 12,
pp.5308–5315.
Poerio, D.V. and Brown, S.D. (2018) ‘A frequency-localized
recursive partial least squares ensemble for soft sensing’,
Journal of Chemometrics, Vol. 32, No. 5, pp.e2999.
Pooyan, N., Shahbazian, M., Salahshoor, K. and Hadian, M. (2015)
‘Simultaneous fault diagnosis using multi class support vector
machine in a dew point process’, Journal of Natural Gas
Science And Engineering, Vol. 23, pp.373–379.
Qi, C.K., Li., H.X., Li, S.Y., Zhao, X.C. and Gao, F. (2014) ‘A fuzzy-
based spatio-temporal multi-modeling for nonlinear distributed
parameter processes’, Applied Soft Computing, Vol. 25,
pp.309–321.
Qin, S.J. (1998) ‘ ‘Recursive pls algorithms for adaptive data
modeling’, Control Engineering Practice, Vol. 22, pp.503–514.
Qin, S.J. (2012) ‘Survey on data-driven industrial process
monitoring and diagnosis’, Annual Reviews in Control, Vol. 36,
pp.220–234.
Quiñones-Grueiro, M., Prieto-Moreno, A., Verde, C. and Llanes-
Santiago, O. (2019) ‘Data-driven monitoring of multimode
continuous processes: a review’, Chemometrics and Intelligent
Laboratory Systems, Vol. 189, pp.56–71.
Rasmussen, C.E. and Williams, C.K. (2006) Gaussian Processes for
Machine Learning, MIT Press, Cambridge, Mass.
Raveendran, R., Kodamana, H. and Huang, B. (2018) ‘Process
monitoring using a generalized probabilistic linear latent
variable model’, Automatica, Vol. 96, pp.73–83.
Rinnan, A., Andersson, M., Ridder, C. and Engelsen, S.B. (2014)
‘Recursive weighted partial least squares (RPLS): an efficient
variable selection method using pls’, Journal of Chemometrics,
Vol. 28, No. 5, pp.439–447.
Rosipal, R. and Trejo, L.J. (2001) ‘Kernel partial least squares
regression in reproducing kernel Hilbert space’, Journal of
Machine Learning Research, Vol. 2, No. 2, pp.97–123.
88 W. Guo et al.
Rothman, M.J., Rothman, S.I. and Beals, J. (2013) ‘Development
and validation of a continuous measure of patient condition
using the electronic medical record’, Journal of Biomedical
Informatics, Vol. 46, No. 5, pp.837–848.
Salaken, S.M., Khosravi, A., Thanh, T. and Nahavandi, S. (2017) ‘
‘Extreme learning machine based transfer learning algorithms:
a survey’, Neurocomputing, Vol. 267, pp.516–524.
Shang, C., Yang, F., Huang, D.X. and Lyu, W.X. (2014) ‘Data-driven
soft sensor development based on deep learning technique’,
Journal of Process Control, Vol. 24, pp.223–233.
Shao, S.Y., McAleer, S., Yan, R.Q. and Baldi, P. (2019) ‘Highly
accurate machine fault diagnosis using deep transfer learning’,
IEEE Transactions on Industrial Informatics, Vol. 15, No. 4,
pp.2446–2455.
Shao, W.M. and Tian, X.M. (2015) ‘Adaptive soft sensor for quality
prediction of chemical processes based on selective ensemble
of local partial least squares models’, Chemical Engineering
Research and Design, Vol. 95, pp.113–132.
Smith, C. and Jin, Y.C. (2014) ‘Evolutionary multi-objective
generation of recurrent neural network ensembles for time
series prediction’, Neurocomputing, Vol. 143, pp.302–311.
Souza, F. A.A., Araújo, R. and Mendes, J. (2016) ‘Review of soft
sensor methods for regression applications’, Chemometrics and
Intelligent Laboratory Systems, Vol. 152, pp.69–79.
Souza, F. A.A., Araújo, R., Matias, T. and Mendes, J. (2013) ‘
‘A multilayer-perceptron based method for variable selection
in soft sensor design’, Journal of Process Control, Vol. 23,
pp.1371–1378.
Stoica, P., Selen, Y. and Jian, L. (2004) ‘Multi-model approach to
model selection’, Digital Signal Processing, Vol. 14, No. 5,
pp.399–412.
Su, Q.L., Hermanto, M.W., Braatz, R.D. and
Chiu, M.S. (2016) ‘Just-in-time-learning based
extended prediction self-adaptive control for batch
processes’, Journal of Process Control, Vol. 43,
pp.1–9.
Sun, Q.Q. and Ge, Z.Q. (2019) ‘Probabilistic sequential network
for deep learning of complex process data and soft sensor
application’, IEEE Transactions on Industrial Informatics,
Vol. 15, No. 5, pp.2700–2709.
Tamboli, D. and Chile, R. (2018) ‘ ‘Multi-model approach for 2-
dof control of nonlinear cstr process’, International Journal
of Modelling, Identification and Control, Vol. 30, No. 2,
pp.143–161.
Teslić, L., Hartmann, B., Nelles, O. and Škrjanc, I. (2011)
‘Nonlinear system identification by gustafson- kessel fuzzy
clustering and supervised local model network learning for the
drug absorption spectra process’, IEEE Transactions on Neural
Networks, Vol. 22, No. 12, pp.1941–1951.
Tomba, E., Meneghetti, N., Facco, P. and Zelenkova, T. (2014)
‘Transfer of a nanoparticle product between different mixers
using latent variable model inversion’, AICHE Journal,
Vol. 2602, No. 1, pp.123–135.
Wang, B. and Mao, Z.Z. (2019) ‘Outlier detection based on gaussian
process with application to industrial processes’, Applied Soft
Computing, Vol. 76, pp.505–516.
Wang, F.L., Tan, S., Peng, J. and Chang, Y.Q. (2012) ‘Process
monitoring based on mode identification for multi-mode
process with transitions’, Chemometrics and Intelligent
Laboratory Systems, Vol. 110, No. 1, pp.144–155.
Wang, R., Wang, X.Y., Sun, H. and Huang, Y.T. (2019a) ‘Analysis
of estimator and energy consumption with multiple faults
over the distributed integrated WSN’, International Journal
of Modelling, Identification and Control, Vol. 32, No. 2,
pp.154–168.
Wang, X.D., Kvaal, K. and Ratnaweera, H. (2019b) ‘Explicit
and interpretable nonlinear soft sensor models for influent
surveillance at a full-scale wastewater treatment plant’, Journal
of Process Control, Vol. 77, pp.1–6.
Wold, S., Kettaneh-Wold, N. and Skagerberg, B. (1989) ‘Non-linear
PLS modelling’, Chemometrics and Intelligent Laboratory
Systems, Vol. 7, pp.53–65.
Wu, H. and Zhao, J.S. (2018) ‘Deep convolutional neural network
model based chemical process fault diagnosis’, Computers and
Chemical Engineering, Vol. 115, pp.185–197.
Yan, Z.B., Kuang, T.H. and Yao, Y. (2017) ‘Multivariate fault
isolation of batch processes via variable selection in partial
least squares discriminant analysis’, ISA Transactions, Vol. 70,
pp.389–399.
Yao, L. and Ge, Z.Q. (2017) ‘Online updating soft sensor
modeling and industrial application based on selectively
integrated moving window approach’, IEEE Transactions
on Instrumentation and Measurement, Vol. 66, No. 8,
pp.1985–1993.
Yuan, X.F., Ge, Z.Q. and Song, Z.H. (2014a) ‘Locally weighted
kernel principal component regression model for soft sensing of
nonlinear time-variant processes’, Industrial and Engineering
Chemistry Research, Vol. 53, No. 35, pp.13736–13749.
Yuan, X.F., Ge, Z.Q. and Song, Z.H. (2014b) ‘Soft sensor model
development in multiphase/multimode processes based on
gaussian mixture regression’, Chemometrics and Intelligent
Laboratory Systems, Vol. 138, pp.97–109.
Yuan, X.F., Ge, Z.Q., Song, Z.H., Wang, Y.L., Yang, C.H. and
Zhang, H.W. (2017) ‘Soft sensor modeling of nonlinear
industrial processes based on weighted probabilistic projection
regression’, IEEE Transactions on Instrumentation and
Measurement, Vol. 66, No. 4, pp.837–845.
Yuan, X.F., Huang, B. and Wang, Y.L. (2018) ‘Deep learning-
based feature representation and its application for soft sensor
modeling with variable-wise weighted sae’, IEEE Transactions
on Industrial Informatics, Vol. 14, No. 7, pp.3236–3243.
Zhai, L.R., Zhang, Y.W., Guan, S.P., Fu, Y.J. and Feng, L. (2018)
‘Nonlinear process monitoring using kernel nonnegative matrix
factorization’, Canadian Journal of Chemical Engineering,
Vol. 96, No. 2, pp.554–563.
Zhang, S., Li, L.J., Yao, L.J., Yang, S.P. and Zou, T. (2018) ‘Data-
driven process decomposition and robust online distributed
modelling for large-scale processes’, International Journal of
Systems Science, Vol. 49, No. 3, pp.449–463.
Zhang, S.M., Zhao, C.H. and Gao, F.R. (2019) ‘Incipient fault
detection for multiphase batch processes with limited batches’,
IEEE Transactions on Control Systems Technology, Vol. 27,
No. 1, pp.103–117.
Zhang, X.M., Kano, M. and Li, Y. (2017) ‘Locally weighted
kernel partial least squares regression based on sparse
nonlinear features for virtual sensing of nonlinear time-varying
processes’, Computers and Chemical Engineering, Vol. 104,
pp.164–171.
Zhang, Y. and Qin, S. (2007) ‘Fault detection of nonlinear processes
using multiway kernel independent component analysis’,
Industrial and Engineering Chemistry Research, Vol. 46,
No. 23, pp.7780–7787.
 A review on data-driven approaches for industrial process modelling
Zhao, L.P., Zhao, C.H. and Gao, F.R. (2014) ‘Regression
modeling and quality prediction for multiphase batch processes
with inner-phase analysis’, Chemometrics and Intelligent
Laboratory Systems, Vol. 135, pp.1–16.
Zhao, L., Peng, T., Xie, Y.F., Yang, C.H. and Gui, W.H. (2017)
‘Recognition of flooding and sinking conditions in flotation
process using soft measurement of froth surface level and qta’,
Chemometrics and Intelligent Laboratory Systems, Vol. 169,
pp.45–52.
Zhao, Z.G., Li, Q.H., Huang, B., Liu, F. and Ge, Z.Q. (2015)
‘Process monitoring based on factor analysis: Probabilistic
analysis of monitoring statistics in presence of both complete
and incomplete measurements’, Chemometrics and Intelligent
Laboratory Systems, Vol. 142, pp.18–27.
Zheng, J.H. and Song, Z.H. (2018) ‘Semisupervised learning for
probabilistic partial least squares regression model and soft
sensor application’, Journal of Process Control, Vol. 64,
pp.123–131.
Zheng, W.J., Liu, Y., Gao, Z.L. and Yang, J.G. (2018) ‘Just-
in-time semi-supervised soft sensor for quality prediction
in industrial rubber mixers’, Chemometrics and Intelligent
Laboratory Systems, Vol. 180, pp.36–41.
89
Zhou, L., Chen, J.H. and Song, Z.H. (2015) ‘Recursive gaussian
process regression model for adaptive quality monitoring
in batch processes’, Mathematical Problems in Engineering,
p.761280.
Zhu, J.L., Ge, Z.Q., Song, Z.H. and Gao, F.R. (2018) ‘Review and
big data perspectives on robust data mining approaches for
industrial process modeling with outliers and missing data’,
Annual Reviews in Control, Vol. 46, pp.107–133.
Zhu, Q.Q., Liu, Q. and Qin, S.J. (2017) ‘Concurrent quality
and process monitoring with canonical correlation analysis’,
Journal of Process Control, Vol. 60, pp.95–103.
Zhu, Z.B., Huan, Z.H. and Palazoglu, A. (2011) ‘Transition
process modeling and monitoring based on dynamic ensemble
clustering and multiclass support vector data description’,
Industrial and Engineering Chemistry Research, Vol. 50,
No. 24, pp.13969–13983.