ISA Transactions xxx (xxxx) xxx

Contents lists available at ScienceDirect

ISA Transactions
journal homepage: www.elsevier.com/locate/isatrans

Research article

A novel support vector machine ensemble model for estimation of free

lime content in cement clinkers
∗
Xiaoyan Liu a,b , , Jiao Jin a , Weining Wu a , Fabian Herz c
a
College of Electrical and Information Engineering, Hunan University, 410082 Changsha, China
b
Hunan Key Laboratory of Intelligent Robot Technology in Electronic Manufacturing, 410082 Changsha, China
c
Department of Applied Biosciences and Process Engineering, Anhalt University of Applied Sciences, 06366 Köthen, Germany

article info a b s t r a c t

Article history: Free lime (f-CaO) content is a crucial quality parameter for cement clinkers in rotary cement kiln. Due
Received 25 July 2018 to lack of hardware sensors, f-CaO content in cement clinker is mostly obtained by offline laboratory
Received in revised form 1 September 2019 measurement, making timely control rather difficult and even impossible. In this work, a soft sensor
Accepted 1 September 2019
approach named as support vector machine ensemble (ESVM) model is proposed to estimate f-CaO
Available online xxxx
content. The process data employed to train and test the model were collected from a cement plant in
Keywords: China, covering a time span of about 30 days. The raw data were preprocessed by filters and time-series
Free lime content matching. The processed data were then clustered by fuzzy c-means clustering algorithm to capture
Soft sensor process features at different operating conditions. For each individual cluster, a base SVM regressor was
Support vector machine trained to estimate f-CaO content. Finally, an ensemble model consisting of four base SVM regressors
Fuzzy c-means clustering
was established to estimate f-CaO content at multifarious process conditions. The effectiveness of
Ensemble model
the proposed ESVM model was investigated by comparing it with manual measurements and other
models available in literature. The results demonstrate that the proposed ESVM model achieves
improvements in model accuracy as well as generalization capability. The proposed ESVM model has
a broad application space in cement production process for automatic monitoring of f-CaO content.
© 2019 ISA. Published by Elsevier Ltd. All rights reserved.

1. Introduction and material, temperatures and pressures) in the plant database.

In cement industries, raw materials are calcined to be cement
clinker at a high temperature environment in rotary kilns. The
free lime (f-CaO) content in cement clinker greatly decides the
cement quality. The f-CaO content is affected by many factors, for
example, the combustion process, the material transport process
as well as the cooling process of the clinker [1–3]. In practice,
the f-CaO content is usually set at 0.5%–1.5% as required quality.
However, due to lack of hardware sensors and harsh environment
(high temperature, strong erosion, heavy dust etc.) within the
rotary kiln, f-CaO content is mostly obtained offline by manual
analysis in laboratory with a long sampling interval (typically
one hour), resulting in difficulties in timely control of cement
clinker quality. Therefore, it is necessary to develop methods for
automatic monitoring of f-CaO content.
In recent years, data-driven soft sensors have been introduced
for automatic monitoring f-CaO content. These soft sensors are
characterized by deriving an input–output model based on large
amount of historical process data (for example feed rates of fuel
∗ Corresponding author at: College of Electrical and Information Engineering,
Hunan University, 410082 Changsha, China.
E-mail address: xiaoyan.liu@hnu.edu.cn (X. Liu).
The soft sensor models available in literature differ mainly in the
choice of model inputs and modeling algorithms (see Table 1).
The first research on soft sensing of free lime was conducted by
Lin B et al. [4]. In their work, Principal Component Regression
(PCR) and Partial Least Squares Regression (PLSR) algorithms
were used as f-CaO content estimation model. Thirteen process
variables were chosen as model inputs. 2000 testing data were
used to test the models. Their results indicated that PLSR model
is more suitable for estimation of f-CaO content than PCR model
due to its ability to capture more relevant information. However,
all the values of f-CaO content to be predicted were in the
range 0.5%–1.5% (corresponding to product quality in normal pro-
duction conditions). As the authors themselves pointed out, the
question of integration of information from irregular sampling
offline quality measurements into weighted vectors still requires
study.
Pani AK et al. [5] developed and compared three different
neural networks (Back Propagation neural network (BP), Radial
Basis Function neural network (RBF) and Generalized Regression
neural network (GR)) for f-CaO content estimation. Nine pro-
cess variables including compositions of the raw material were
used as model inputs. It was found that RBF model has better
performance than BP and GR. Later, Pani AK et al. [6] applied

https://doi.org/10.1016/j.isatra.2019.09.003
0019-0578/© 2019 ISA. Published by Elsevier Ltd. All rights reserved.

Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
2 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx

Table 1
Soft sensor models for estimation of f-CaO content.
Authors Model Model inputs Number of Value (%) distribution
testing samples in testing samples
<0.5 0.5–1.5 >1.5
Lin B et al. [4] PCR, PLSR 13 inputs, including: kiln current, fuel flow rates to 2100 0% 100% 0%
calciner and kiln, kiln feed and several temperature
measurements within the kiln system
Pani AK et al. [5] BP, RBF, GR 9 inputs, including: 110 1.8% 91.8% 6.4%
Pani AK et al. [6] T–S inference technique Raw meal quality (SiO2 , Al2 O3 , Fe2 O3 , CaO),
kiln operating variables (kiln feed rate, kiln RPM, raw
meal inlet temperature, coal feed rate, kiln current)
Li W et al. [7] RVFL ensemble model 14 inputs, including: 80 10% 65% 25%
flame image features (color of ROI, global configuration
of ROI, local configuration of ROI),
kiln operating variables (coal feeding, opening degree of
induced draft fan, kiln current, raw material pump
current, tail temperature, head temperature, head
pressure),
raw material quality (lime saturation factor, silicic acid
rate, alumina modulua, granularity)

Takagi–Sugeno (T–S) fuzzy inference technique to improve esti-
mation performance. The model accuracy was verified using 110
testing samples and achieved good performance. However, 91.8%
of the values of f-CaO content to be predicted are ranged between
0.5%–1.5%, corresponding to product quality in normal production
conditions. The adaptability and prediction accuracy of this model
for other cases (f-CaO>1.5%, abnormal product quality) remains
to be further investigated.
Li W et al. [7] employed a random vector functional-Link
(RVFL) nets ensemble model for automatic estimation of f-CaO
content. The model was developed by generating 20 RVFL net-
works trained by the same training samples, and the final pre-
dicted result was computed by the mean f-CaO content values
predicted by the 20 RVFL networks. Fourteen variables including
features of flame images were used as inputs of the ensemble
model. The model was verified by 80 testing data with a relatively
wide range of f-CaO values (25% of the values are bigger than
1.5%). However, as pointed out by the authors themselves [7], the
random weights assigned by this RVFL ensemble model are not
optimal. Moreover, this method requires that the flame images
have good quality. In the cases of heavy smoke and dusts in
rotary kiln [8], however, the flame images captured by CCD cam-
era become very blurry or even invisible. This brings difficulties
in image feature extraction and thus affects greatly the model
performance.
The above literature analysis shows that data-driven soft sen-
sor models are effective for automatically estimating f-CaO con-
tent, and it is still necessary to develop novel algorithms to
improve model accuracy and model generalization capability by
using representative datasets that cover a wider range of cement
clinker quality.
Support Vector Machine (SVM) has been recognized as a tra-
ditional and reliable approach to evaluate nonlinear relationship
of the inputs–outputs. Compared with other algorithms, SVM
has advantages of simple structure and strong generalization
ability, and thus SVM is particularly suitable for modeling data
with insufficient quantity and multi-collinearity [9–11]. There-
fore, SVM could be a good choice for estimating the f-CaO content
in clinker. However, clinker production is a complicated and 24
hour-continuous process involving many coupled sub-processes,
such as combustion, chemical reactions, mass and heat trans-
fer [12–14]. The processes have the characters of non-linearity,
strong coupling, and especially big time lag [15,16]. It would
be difficult to measure the f-CaO content under various process
conditions by use of a single SVM model.
Motivated by the above-mentioned problems, this work pro-
posed a novel support vector machine ensemble (ESVM) model
to estimate f-CaO content in cement clinker, aiming at improving
its estimation accuracy and generalization capability. The process
data used to train and test the model were collected from a local
cement production line, covering a relatively long time span (30
days) and wide range of process conditions ( 0.3<f-CaO<3.4).
The raw data were first preprocessed based on time-lag feature
of the cement production process. The processed data were then
clustered by fuzzy c-means clustering (FCM) for capturing process
features at different conditions. For each individual cluster, a
base SVM regressor was trained using sub-dataset in the cluster.
Finally, an ensemble model consisting of several base SVM regres-
sors was established to monitor the f-CaO content under various
process conditions.
In Section 2, the cement clinker calcination process is briefly
described along with data collection and preprocessing proce-
dure; in Section 3, the ESVM model is proposed and described in
detail; in Section 4, the performance of the proposed model for
f-CaO content estimation is evaluated and compared with other
models, followed by conclusions in Section 5.
2. Cement clinker calcination process and data preprocessing
2.1. Cement clinker calcination process
The typical cement clinker calcination process includes four
stages (preheating, calcination, sintering and cooling) and in-
volves four main apparatuses (cyclone preheater, precalciner,
rotary kiln and grate cooler, in Fig. 1). Specification of the cement
clinker production line in a local cement company are listed in
Table 2. The raw material is fed into the multistage cyclone pre-
heater (mostly a five-stage cyclone preheater). During its vertical
downward movement in the cyclones, the material is pre-heated
via countercurrent hot flue gas. After that, it is discharged into a
precalciner (the temperature is about 800 ◦ C). Due to the complex
dynamics of gas–solid two phase flow in high temperature, it is
difficult to establish mathematical models to calculate the time
that the material needed to pass the cyclonepreheater and the
precalciner. Li and Jian [17] measured the residence time of the
material in a five-stage cyclone preheater in cold tests (27 ◦ C),
by use of an on-line RTD (retention time distribution) measuring
instrument using electromagnetic method. It was found that the
residence time of the material in five cyclones is approximately
8 s. Chen et al. [18] performed numerical simulations to predict
the gas–solid flow in the precalciner in cold tests. The calculated
result showed that the material stayed in the precalciner for
about 10 s. In reality, the residence time is much bigger owing

Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx
Table 2
Parameter list of the cement clinker production process in a local cement
company.
Parameters
Length of the kiln (m)
Inner diameter of the kiln (m)
Inclination of the kiln to horizontal (◦ )
Rotation speed of the kiln (rpm)
Effective area of the grate plate (m2 )
Thickness of bed material in the cooler (m)
Repose angle of the material (◦ )
Feeding rate of the raw material (t/h)
Density of clinker in the cooler (kg m−3 )
Flow rate of the clinker into the cooler (kg min−1 )
to the turbulent flow of the flue gas in high temperature field.
Based on results of experiments and simulations in [17,18] as well
as the technical reports from the local cement company, the total
time that the material needs to pass the cyclone preheater and
the precalciner is estimated to be about 30 s.
Coming out from the precalciner, the material then enters the
rotary kiln. The temperature of the burning zone of the rotary kiln
reaches about 1300 ◦ C. Under such high temperature, the liquid
phase of tetracalcium aluminoferriate (C4 AF), tricalcium alumi-
nate (C3 A) and dicalcium silicate (C2 S) in the material absorbs
f-CaO to form the major component of cement clinker which
namely tricalcium silicate (C3 S). The mean residence time tk of
the material in rotary kiln can be estimated using the classical
empirical equation developed by Sullivan et al. [19] as below:
L θ
tk = 1.77 × × (1)
D nc × β
where L represents kiln length; D denotes the inner diameter of
the kiln; θ is the repose angle of the material; nc is the rotation
speed of the kiln; β is the inclination of the kiln to horizontal.
Based on the parameters listed in Table 2, tk is calculated to be
21 min. The hot clinker is then discharged from rotary kiln into
a grate cooler and is cooled by the air coming from cooling fans.
The residence time tc of the clinker in the cooler is calculated as
m AH · ρ
tc = = (2)
W W
where m is the total mass of the material in the grate cooler; W
is the flow rate of the clinker into the cooler; A is the effective
area of the grate plate; H is the thickness of bed material in the
cooler; ρ is the density of the clinker in the cooler. Accordingly,
tc is calculated to be 29 min.
The cement clinker quality is confirmed offline in laboratory
by detecting the f-CaO content in the cement clinker samples
collected periodically from the discharge end of the grate cooler.
The sampling interval is one hour.
2.2. Data collection and preprocessing
2.2.1. Data collection
As described above, cement clinker production process is
rather complex. There are material flow, gas flow and fuel flow,
involving physical and chemical reactions, mass transfer and heat
transfer. Variations of the process conditions greatly affect the
f-CaO content. In practice, about 70 sensors are installed for
automatic monitoring of process variables such as temperatures,
pressures, flow rates etc. Due to the strong coupling of the pro-
cesses, some sampled variables are correlated with one another.
For example, the temperature in the precalciner is affected by
the temperature in the five-stage cyclone preheater, and the
electrical current increased greatly with increased rotation speed.
Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
3
Table 3
Inputs and output of the proposed soft sensor model.
Model inputs Model output
Values Feed rate of the raw material (M)
60 Temperature in precalciner (T )
3.6 Secondary air pressure at kiln head (Ph ) f-CaO content
2.29 Air pressure in the second chamber of grate cooler (Pc )
3.3–3.9 Electrical current of the drive motor of the rotary kiln (I)
63.6
0.6
35
about 180 If too many variables are chosen as inputs of the model for f-CaO
about 1500
content estimation, the time for model training and computing
about 1973.7
will be dramatically increased and cannot meet the real-time
requirement in practical applications.
For the above reasons and considering the fact that f-CaO is
mostly generated in rotary kiln, we consider mostly the signals
sampled from rotary kiln as well as its neighborhood equipments
(precalciner, two chambers of the grate cooler). According to the
process mechanism and experience of the kiln operators, five
process variables are selected in the present work as inputs of
f-CaO content estimation model (see Table 3 and Fig. 1). Among
them, the feed rate of the raw material (M) is a variable from
material flow process; the temperature in precalciner (T ) is a
variable from heat transfer process; the secondary air pressure
at kiln head (Ph ) and the air pressure in the second chamber
of grate cooler (Pc ) are variables from the air flow process; the
electrical current of the drive motor of the rotary kiln (I) is a vital
parameter reflecting the material load and thermal conditions in
the kiln [20,21]. Although some other variables may be also used
as model inputs, we will show later with experiments that these
five input variables are enough for soft sensor modeling of f-CaO
content.
In order to train the model, we collected data from a cement
production line (Fig. 1 and Table 2) of a cement plant in Jiangxi
Province, China. The process variables in Table 3 were measured
by sensors installed on site. The values were sampled and saved in
computer by the automatic monitoring system that was already
installed for the production line. The sampling interval was one
second. The f-CaO content was measured manually by offline
laboratory analysis with an interval of one hour. The acquired
data cover a long time span of 30 days (from April 25th to May
24th), reflecting various process conditions.
2.2.2. Data preprocessing
The data preprocessing procedures are illustrated in Fig. 2. The
process variables were sampled with an interval of 1 s for 30
days, leading to a huge amount of data points (86 400 × 30) and
redundancy in data. In the present work, a combination of mean
filter and 3σ criterion (‘‘mean filter-3σ method’’) is employed to
process the data, that is, the raw data are processed firstly by
mean filter with a period of 5 min to reduce redundant informa-
tion and then processed by 3σ criterion to delete contaminated
data caused by sensor failure. In principle, one can also adopt
the processing sequence ‘‘3σ -mean filter’’, which could however
lead to more computing load since outliers have to be eliminated
from 86 400 × 30 data points when using 3σ criterion. In or-
der to investigate the influence of data preprocessing sequence
‘‘3σ -mean filter’’ and ‘‘mean filter-3σ ’’, we compare the final
processed datasets (288 data points) for precalciner temperature
T on May 5th. As shown in Fig. 3, there is no big difference
between datasets obtained by use of ‘‘mean filter-3σ ’’ method
and the ‘‘3σ -mean filter’’ method. Among the 288 data points,
only 13 data points scatter near the centerline, and the relatively
differences are less than 0.6%.
As already described in Section 2.1, the cement clinker pro-
ducing process involves preheating, decomposing, calcination and
4 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx

Fig. 1. Schematic diagram of the cement clinker calcination process.

Fig. 2. Flow chart of data preprocessing.

Table 4
Example of some input–output pairs.
Sampling Sample M (t/h ) T (◦ C) I (A) Ph (Pa) Pc (kPa) f-CaO
date number content (%)
4–25 1 182.5 843.0 381.0 −22.7 6.8 1.6
2 182.8 844.1 340.6 −6.6 6.9 1.1
3 182.1 843.4 356.8 −0.1 6.5 1.2
. . . . . . .
. . . . . . .
. . . . . . .
23 182.7 843.2 279.3 14.7 6.4 0.7
24 182.8 841.0 328.1 −17.1 6.0 1.2
4–26 25 183.0 840.3 343.9 −29.9 6.1 1.1
26 182.4 838.3 294.3 −56.1 6.1 1.6
. . . . . . .
. . . . . . .
. . . . . . .
232 180.0 843.3 368.5 −13.7 6.5 0.3
. . . . . . .
. . . . . . .
. . . . . . .
415 182.0 851.8 343.3 −21.1 6.3 3.4
. . . . . . . .
. . . . . . . .
. . . . . . . .
5–23 660 179.8 851.9 229.3 −9.2 6.4 1.2
Minimum 167.5 825.5 134.5 −73.0 4.4 0.3
Maximum 185.0 865.5 461.3 36.3 7.7 3.4

cooling. There exists much time lag between the corresponding
process variables. The changes of f-CaO content in cement clinker
are caused by the cumulative impacts of operating variables
over the past time period [22]. It is thus important to consider
the time-series matching between each input and output (f-CaO
content) of the model. A useful method was proposed in our
previous work [16] to solve such a problem by analyzing the
residence time of the material in the four main apparatus (Fig. 4).
In the local cement company, the f-CaO content was manually
measured by collecting clinker samples at about the 50th minute
of every hour (labeled by XX: 50 in the figure). Using XX: 50
as reference time, the matching time of the 5 process variables
(Fig. 4) can be obtained by determining the sampling locations of
process variables and calculating the residence time of the ma-
terial in corresponding apparatuses. As described in Section 2.1,
the material stays in the preheater and precalciner for about 30 s,

Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx
Fig. 3. Comparison of the final processed datasets (288 data points) for precal-
ciner temperature T on May 5th (the data were processed by ‘‘3σ -mean filter’’
method and ‘‘mean filter-3σ ’’ method, respectively).
and the total residence time of the material in rotary kiln and
cooler is about 50 min, therefore the matching time of variable
M in the preheater and the variable T in the precalciner could
be roughly estimated as XX: 00 with respect to the reference
sampling time (XX: 50) of f-CaO content. In consideration of the
sampling position of Pc in the grate cooler and the residence time
(29 min) of the material in the cooler, the matching time of Pc is
approximately XX: 25. The variables Ph and I reflect the working
conditions in the rotary kiln. According to the infrared image of
the kiln shell, the high-temperature calcination of the material
has nearly accomplished at the distance of 51 m to the kiln tail.
The time for the material to pass the rest distance (9 m) to the
discharge end of the kiln is calculated to be 3 min based on Eq. (1),
and the time to pass through the grate cooler is about 29 min, that
is, 32 min in total. Considering the sampling positions of the two
variables Ph and I, the matching time of them can be estimated
as XX: 18 with respect to the reference sampling time (XX: 50)
of f-CaO content.
After data preprocessing, a dataset containing 660 sets of
inputs–output pairs are obtained, arranged in time sequence or-
der. The first 560 sets of data will be used for model building
(training data) and the remaining 100 sets for model validation
(testing data). This means that the proposed model needs to
predict the clinker quality over 4 successive days (from May 19th,
0:00 to 23nd, 9:00) of continuous operation of the rotary kiln.
Some of the preprocessed input–output pairs are shown in
Table 4 for example. The collected data cover a long time span
and have relatively wide ranges, being representative for various
process conditions. For example, the input variable I ranges from
134.5 A to 461.5 A; the values of f-CaO content vary between 0.3%
(over-burning state) and 3.4% (under-burning state), covering a
relatively wider range than those used in literature. To ensure
equal relevance for all variables, the inputs–output pairs are
normalized before they are used for soft sensor model training.
3. Framework of the ESVM model
In this section, a support vector machine ensemble model
(ESVM) is proposed for f-CaO content prediction. The basic idea
is to increase the adaptability of the SVM regressor to the com-
plexity and variety of cement clinker calcination process, thus im-
proving model generalization. Fig. 5 illustrates the general archi-
tecture of the proposed ESVM model. In order to capture the pro-
cess features at different operating conditions, the training data
Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
5
Fig. 4. Residence time of the material in apparatuses and time-series matching
scheme for inputs–output of the model.
Table 5
The sum of Euclidean distances between of the testing samples and its
corresponding cluster centers.
Number of clusters n=3 n=4 n=5 n=6
Sum of Euclidean distances d 3.96 3.17 3.43 3.55
is grouped into n clusters C1 , C2 , . . . , Cn using fuzzy c-means al-
gorithm (FCM). Each cluster indicates similar process conditions.
The corresponding data in each cluster (denoted as sub-dataset
S1 , S2 , . . . , Sn , respectively) are then used to train a base SVM
regressor (denoted as SVM1 , SVM2 , . . ., SVMn , respectively) for
f-CaO content estimation under the corresponding process condi-
tions. The established ESVM model {SVM1 , SVM2 , . . ., SVMn } can
be then employed to estimate f-CaO content under new process
conditions, by performing following procedures: (i) determine the
cluster that the new input data {M , T , I , Ph , Pc } belong to; (ii) use
the corresponding trained base SVM regressor to estimate f-CaO
content. The procedure will be represented in following sections.
3.1. Data clustering based on FCM
Fuzzy c-means algorithm (FCM) is an objective function-based
clustering tool. FCM has been successfully used due to its ef-
ficient and outstanding performance [23,24] and is thus ap-
plied in the present work. Based on FCM algorithm, the train-
ing data {M , T , I , Ph , Pc } can be grouped into several clusters
(C1 , C2 , . . . , Cn ). The inputs–output pairs {M , T , I , Ph , Pc , f -CaO}
of each cluster are taken as a sub-dataset (S1 , S2 , . . . , Sn , re-
spectively). Each sub-dataset will be used to train a base SVM
regressor.
In the usage of FCM, the number of clusters n should be ap-
propriately selected. Too many clusters will lead to complicated
result that is difficult to interpret and analyze, while too few
clusters will make information to be lost and may misguide the
final decision [25]. For compromise, n is preset in the range 3–6.
For determining the most suitable number of clusters, a sum of
calculated Euclidean distances (denoted as d) between the testing
samples and their corresponding cluster centers are employed for
evaluation. As shown in Table 5, d is minimum for the case n = 4,
indicating that there is a higher similarity between the testing
samples and their corresponding cluster centers. Therefore, it is
reasonable for us to group the training data into four clusters with
four cluster centers {v1 , v2 , v3 , v4 }.
6 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx

Fig. 5. Architecture of the proposed ESVM model for estimation of f-CaO content.

3.2. Training of base SVM regressors capability in modeling, and its expression is [9]:
xi − xj 2
(   )
(  )
K xi , xj = exp − xi − xj 2
( ) 
Support vector machine (SVM) is a competent algorithm for = exp −γ (6)
dealing with regression issue and has the advantages of sim-
σ2
ple structure and strong generalization ability [26–29]. The brief where γ is the kernel parameter.
notions of SVM for the case of regression are introduced. Con- In the training of the base SVM regressors for f-CaO content
sidering a training dataset {(xi , yi )}Ni=1 where input xi ∈ X ⊆ Rd prediction, the inputs and output are X = {M , T , I , Ph , Pc } and
and output yi ∈ Y ⊆ R, SVM performs regression estimation by Y = {f − CaO}, respectively. The performance of SVM is directly
establishing a linear model, given by: bound up with the regularization parameter c, the kernel param-
eter γ and the parameter ε of the loss function. In determining
f (x) = w T • Φ (x) + b (3) the learning parameters of SVM, one can adopt trial-and-error
method [31–33] or adopted optimization methods such as grid
where w defines the weight vector; b represents the bias term search [34,35] and genetic algorithm (GA) [36,37]. In the present
which is a constant; Φ (x) is the representation of a nonlinear work, we combine trial-and-error method with GA to find out
transformation function to map the input space into a higher the learning parameters of SVM. Firstly, trail-and-error procedure
dimensional feature space. This is equivalent to solving the con- is performed to determine the rough ranges of the parameters,
straint optimization problem as follows [26,30]: i.e. c ∈ [0, 10], γ ∈ [0.25, 5] and ε ∈ [0, 0.25]. Then, these
learning parameters are optimized by GA in the preset ranges. We
n
1 ∑ use GA instead of grid search algorithm, because GA consumes
wT w + c ξi + ξi∗
( )
min less computing time and are suitable to solve complex parameter
w,b,ξi ,ξi∗ 2
i=1 optimization problems using merely three genetic operations of
⎧ selection, crossover and mutation [37].
⎨y[ i − w Φ (xi )] + b ≤ ε + ξi
[ T ]
⎪ The optimization algorithm can be realized by libsvm-mat
toolbox (https://www.csie.ntu.edu.tw/~cjlin/), which was devel-
subject to wT Φ (xi ) + b − yi ≤ ε + ξi (4)
oped by Chih-Jen Lin. The maximum number of generations and
ξi , ξi∗ ≥ 0, i = 1, 2, . . . , n
⎪
⎩
the population size of GA are set to 250 and 20, respectively. The
probability of crossover operation is set as 0.9 and the probability
where ξi and ξi∗ denote slack variables which are represented
of mutation operation is set as 0.01. The individual fitness values
as the upper and lower boundary hyperplane, respectively; c is are calculated by the fitness function defined as [37]:
penalty parameter which controls the trade-off between mini-
l ⏐ ⏐
mization of the training error and the maximization of the mar- 1 ∑ ⏐ yi − ŷi ⏐
f (i) = (7)
gin; ε is an insensitive loss function [30] and determines the
⏐ ⏐
ng ⏐ yi ⏐
i=1
maximum error for the approximation. The solution of this con-
strained optimization function is obtained by converting it to be where f (i) represents the fitness function value of sample i, ng
a Lagrange dual problem, and then the regression function can be denotes the sample size of samples of the validation dataset (in
the training subsets, one of them is taken as validation dataset
simplified by solving the dual problem as [10]:
in turn, others are taken as training dataset). yi and ŷi represent
n the actual values and prediction values of the validation dataset,
∑
f (x) = (αi − αi∗ )K xi , xj + b
( )
(5) respectively. The selection operation is performed by calculating
i=1 the individual fitness values. Chromosomes with higher fitness
values get more chances to survive and are more likely to recur
where αi and( )αi denote Lagrange multipliers; and K xi , xj =
∗
( )
in the new population. The selection process is repeated until 20
Φ (xi ) • Φ xj is the kernel function which is satisfied the Mer-
T
(the population size) members have been selected. The optimized
cer’s condition. In this article, the Gaussian radial basis function learning parameters of the four base SVM regressors are listed in
is employed as kernel function by the reason that it has excellent Table 6.

Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx 7

Table 6 Table 7
Learning parameters of the base SVM regressors obtained by genetic algorithm. Values of cluster centers.
c γ ε Cluster centers M T I Ph Pc
SVM1 5.0 1.70 0.118 v1 0.16 0.35 0.65 0.51 0.61
SVM2 3.5 4.02 0.062 v2 0.79 0.38 0.62 0.47 0.57
SVM3 2.0 1.05 0.013 v3 0.74 0.57 0.34 0.61 0.57
SVM4 1.8 0.40 0.142 v4 0.70 0.66 0.47 0.44 0.39

Table 8
Comparison of testing datasets used in literature and our work.
Researchers Number of testing Percentage of data in a certain
samples range of f-CaO content
<0.5 0.5–1.5 >1.5
Lin B et al. [4] 2200 0% 100% 0%
Pani AK et al. [5,6] 110 1.8% 91.8% 6.4%
Li W et al. [7] 80 10.0% 65.0% 25%
Our work 100 3.0% 63.0% 34%

used for model testing. The data were collected over 4 successive
days (from May 19th, 0:00 to 23nd, 9:00) during continuous
Fig. 6. (a) Distribution of the training data; (b) Distribution of the training data
operation of the rotary kiln and were not used for model training.
after FCM clustering.
Compared to testing data used in literature (see Table 8), our test-
ing samples reflect a variety of clinker quality (0.3%<f-CaO<3.3%)
with 34% data in the range of f-CaO>1.5 (unsatisfactory clinker
3.3. Testing the ESVM model quality). Such variety in testing samples is very suitable for eval-
uation of model performance, that is to say, how well the model
For new testing samples, the prediction of f-CaO content is performs when it is supplied with data collected in quite different
implemented with following steps: process conditions.
(1) Determine the cluster that the new sample belongs to
by calculating the Euclidean distances between the sample and 4.2.2. Predicted f-CaO content using ESVM model and comparison
the four obtained cluster centers {v1 , v2 , v3 , v4 }. The cluster with with other models
minimal Euclidean distance value is then selected for the sample. To evaluate the performance of the proposed ESVM model, the
Samples within the same cluster are considered to have similar predicted results by ESVM model are compared with manually
process conditions. measured values as well as with the predicted values by the
(2) If the new sample belongs to a certain cluster Ci , the trained available models that were used for f-CaO prediction in literature,
base SVM regressor SVMi is then selected to predict the f-CaO including:
content. (1) Single SVM model. Compared to the proposed ESVM archi-
tecture, it has only one SVM regressor. For comparison, the single
4. Results and discussions SVM model was trained using the same training dataset as ESVM
model.
4.1. Results of FCM clustering of the training dataset (2) PLSR model. PLSR was employed for estimation of f-CaO
content in literature [4]. For comparison, the PLSR model was
Distributions of the original training data and FCM clustering trained using the same training dataset as ESVM model.
results are visualized (Fig. 6(a) and (b)) using Radviz visualization (3) RBF neural network model. RBF was used for f-CaO content
technique [38]. Fig. 6 shows that the original distribution of prediction in literature [5]. For comparison, RBF was trained using
training data is imbalanced. After FCM clustering, four clusters the same training dataset as ESVM model.
(C1 , C2 , C3 , C4 ) located in different areas can be obtained. The (4) T–S fuzzy neural network model (T–S-FNN) [39]. It com-
distribution of each cluster is separated from other clusters. Each bines the advantages of neural network and T–S fuzzy inference
cluster has its own distribution characteristics, which is con- technique. Similar technique was used in literature [6] to predict
ducive to the modeling of base SVM learners and enhancing the f-CaO content. In the present work, T–S-FNN model was trained
diversity of learners. using the same training dataset as ESVM model.
The cluster centers after normalization are given in Table 7. Three commonly used performance indices are adopted in the
The values of each cluster center are different from those of present work to evaluate model performance, i.e. mean squared
other cluster centers. In particular, the cluster center for M varies error (MSE), Theil’s inequality coefficient (TIC) [6] and correlation
greatly, from 0.16 in the first cluster to 0.79 in the second cluster. coefficient (R):
Such variations can be also observed in the cluster centers of ∑N ( )2
other 5 variables (0.35–0.66 for T , 0.34–0.65 for I, 0.44–0.61 for Ph i=1 ỹi − yi
MSE = (8)
and 0.39–0.61 for Pc ). This indicates that each cluster has its own
√ N (
characteristic, which greatly benefits the modeling of the ESVM ∑N )2
yi − ỹi
model. TIC = √
i=1
√∑ (9)
∑N 2 N
i=1 yi + i=1 ỹi 2
4.2. Results of f-CaO content prediction and model performance ( )
∑N
i=1 (yi − y) ỹi − ỹ
4.2.1. Testing samples R= √ (10)
To validate the generalization capability of ESVM model pro- ∑N ∑N ( )2
i=1 (yi − y)
2
posed in this work, 100 new samples (or measured values) are i=1 ỹi − ỹ

Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
8 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx
Fig. 7. Comparison of measured f-CaO content with predicted results by ESVM
model, PLSR model, RBF model and T–S-FNN model.
Fig. 8. Comparison of measured f-CaO content with predicted results by ESVM
model and single SVM model.
where yi and ỹi are measured and model predicted f-CaO content,
respectively; y and ỹ are the average of yi and ỹi , respectively; N
denotes the total number of testing samples. MSE is a frequently
used index that represents the differences between the measured
and predicted f-CaO contents. The less is the value of MSE, the
smaller the mean fitting error. The value of TIC varies from 0 to
1. The closer TIC value is to 0, the more similar the prediction
series is to the measurement series. R describes the degree of
correlation between prediction and measurement, ranging from
−1 to 1. The closer is the value of R to 1, the stronger relationship
of the predictions and the measured values.
The prediction capability of ESVM model proposed in present
work is compared with single SVM model, PLSR model, RBF model
and T–S-FNN model in Figs. 7 and 8. MSE, TIC and R produced by
different models are given in Table 9. It is found that PLSR model,
RBF model and T–S-FNN model produce similar results (Fig. 7,
2nd, 3th and 4th column of Table 9), with MSE = 0.28, 0.28 vs.
0.29 and TIC = 0.17, 0.18 vs. 0.17. In comparison, the proposed
ESVM model has the smallest MSE (MSE = 0.21), followed by
the single SVM model with MSE = 0.25. Since smallest MSE
indicates that the mean squared error between model prediction
and measurement is smallest, it can be concluded that ESVM
model proposed in our work outperforms the other 4 models.
Considering the facts that the ESVM model is tested with data
sampled in 4 successive days in clinker production and these
data were not used for model training, the prediction accuracy
of ESVM model is relatively great.
Except for model accuracy, the generalization ability is also
very important for the estimation model, which can be evaluated
by use of correlation coefficient R [40]. It is obvious from Table 9
that the ESVM model proposed in this work has a better gen-
eralization capabilities than the other four models. The highest
R value (R = 0.69) of the ESVM model is due to its ensemble
Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
Table 9
Performance of different models using the same testing samples.
Modeling methods MSE TIC R
PLSR model 0.28 0.17 0.55
RBF model 0.28 0.18 0.56
T–S-FNN model 0.29 0.17 0.60
Single SVM model 0.25 0.17 0.63
ESVM model in the present work 0.21 0.15 0.69
architecture of several base SVM regressors that were trained by
clustered process data.
The above analysis demonstrates that the proposed ESVM
model has advantages both in generalization capability and in
model accuracy. However, it can be also observed that relatively
big deviations between prediction and measurement may occur
for testing samples with extremely off grade f-CaO content (>3%
or <0.5%, Fig. 8). This situation mainly caused by the fact that
the local cement company seldom produces extremely off grade
cement clinkers. Consequently there are less training data avail-
able in such extreme ranges. The model prediction accuracy for
such extreme cases can be further improved by collecting more
training data in the future.
5. Conclusions
The f-CaO content in cement clinkers largely reflects the qual-
ity of cement produced in the cement plants. Due to lack of hard-
ware sensors and harsh environment (high temperature, strong
erosion, heavy dust etc.), data-driven soft sensor modeling has
become an attractive approach for estimation of f-CaO content.
In the present article, a novel support vector machine ensemble
(ESVM) model was proposed to estimate f-CaO content, with
following features and advantages:
(1) ESVM model consists of several base SVM regressors. Each
base SVM regressor was trained using clustered dataset based on
fuzzy c-means clustering algorithm. Therefore, process features at
different operating conditions can be captured by the model, and
the prediction accuracy of this model is then improved.
(2) As compared to soft sensor models available in literature,
ESVM model is relatively simple and it needs only five variables
as model inputs, including feed rate of the raw material, tem-
perature in precalciner, secondary air pressure at kiln head, air
pressure in the second chamber of grate cooler, and electrical
current of the drive motor of the rotary kiln. These five variables
are common process variables that can be directly measured
online. Results demonstrated that they are enough to build f-CaO
content estimation model.
(3) The effect of time lag between process variables and f-CaO
content was considered for building the model by processing the
data with time-series matching technique.
(4) As compared to soft sensor models available in literature,
the proposed ESVM model was tested by new samples that re-
flect a wider range of clinker quality (f-CaO content was ranged
between 0.3% and 3.3%). The testing samples were collected over
4 successive days during continuous operation of the rotary kiln
and were not used for model training.
(5) Compared to the single SVM model, PLSR model, RBF
model and T–S-FNN model, the proposed ESVM model has the
lowest mean square error and the highest correlation coefficient
when tested on the validation dataset, indicating that it has
improvements both in prediction accuracy and in generalization
capability.
The proposed ESVM model has an excellent application
prospect in cement clinker production line for automatic mon-
itoring of f-CaO content, enabling plant operators to maintain the
quality of cement clinkers.
 X. Liu, J. Jin, W. Wu et al. / ISA Transactions xxx (xxxx) xxx
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared
to influence the work reported in this paper.
Acknowledgments
This work was supported by National Natural Science Founda-
tion of China [No. 61973108], Hunan Key Laboratory of Intelligent
Robot Technology in Electronic Manufacturing [IRT.2018001].
References
[1] Mikulčić H, Vujanović M, Fidaros DK, Priesching P, Minić I, Tatschl R,
et al. The application of CFD modelling to support the reduction of CO2
emissions in cement industry. Energy 2012;45:464–73.
[2] Chen H, Zhang X, Hong P, Hu H, Yin X. Recognition of the temperature
condition of a rotary kiln using dynamic features of a series of blurry flame
images. IEEE Trans Ind Inform 2016;12(1):148–57.
[3] Vogelbacher M, Waibel P, Matthes J, Keller HB. Image-based characteriza-
tion of alternative fuel combustion with multifuel burners. IEEE Trans Ind
Inform 2018;14(2):588–97.
[4] Lin B, Recke B, Knudsen JK, Jørgensen SB. A systematic approach for soft
sensor development. Comput Chem Eng 2007;31(5):419–25.
[5] Pani AK, Vadlamudi VK, Mohanta HK. Development and comparison of
neural network based soft sensors for online estimation of cement clinker
quality. ISA Trans 2013;52(1):19–29.
[6] Pani AK, Mohanta HK. Online monitoring of cement clinker quality us-
ing multivariate statistics and Takagi-Sugeno fuzzy-inference technique.
Control Eng Pract 2016;57:1–17.
[7] Li W, Wang D, Chai T. Multisource data ensemble modeling for clinker
free lime content estimate in rotary kiln sintering processes. IEEE Trans
Syst Man Cybernet: Syst 2015;45(2):303–14.
[8] Li W, Wang D, Chai T. Burning state recognition of rotary kiln using ELMs
with heterogeneous features. Neurocomputing 2013;102:144–53.
[9] Lv Y, Liu J, Yang T, Zeng D. A novel least squares support vector machine
ensemble model for NOx emission prediction of a coal-fired boiler. Energy
2013;55:319–29.
[10] Fan G-F, Peng L-L, Hong W-C, Sun F. Electric load forecasting by the
SVR model with differential empirical mode decomposition and auto
regression. Neurocomputing 2016;173:958–70.
[11] Khormali A, Addeh J. A novel approach for recognition of control chart
patterns: Type-2 fuzzy clustering optimized support vector machine. ISA
Trans 2016;63:256–64.
[12] Bakdi A, Kouadri A, Bensmail A. Fault detection and diagnosis in a cement
rotary kiln using PCA with EWMA-based adaptive threshold monitoring
scheme. Control Eng Pract 2017;66:64–75.
[13] Meyer V, Pisch A, Penttilä K, Koukkari P. Computation of steady state
thermochemistry in rotary kilns: Application to the cement clinker
manufacturing process. Chem Eng Res Des 2016;115:335–47.
[14] Lima RN, de Almeida GM, Braga AP, Cardoso M. Trend modelling with arti-
ficial neural networks. Case study: Operating zones identification for higher
SO3 incorporation in cement clinker. Eng Appl Artif Intell 2016;54:17–25.
[15] Sharifi A, Aliyari Shoorehdeli M, Teshnehlab M. Identification of cement
rotary kiln using hierarchical wavelet fuzzy inference system. J Franklin
Inst 2012;349(1):162–83.
[16] Wu W, Liu X, Xu X, Jin J, Zhang M. Time series analysis method
for the soft measurement of cement clinker quality. Control Theory
Appl 2018;35(7):1029–36. http://dx.doi.org/10.7641/cta.2017.70501, [in
Chinese].
Please cite this article as: X. Liu, J. Jin, W. Wu et al., A novel support vector machine ensemble model for estimation of free lime content in cement clinkers. ISA Transactions
(2019), https://doi.org/10.1016/j.isatra.2019.09.003.
9
[17] Li C, Jian M. The applications of the instrument for measuring powder
retention time online in studies of preheaters. J Wuhan Univ. Technol
1991;13(3):54–9, [in Chinese].
[18] Chen Z, Lu H, Peng J, Dou H, Shi L, Huang J. Numerical simulation of the
gas solid two-phase flow in a reinforced suspension precalciner. J Chin
Ceram Soc 2006;34(1):54–8, [in Chinese].
[19] Sullivan JD, Maier CG, Ralson OC. Passage of solid particles through rotary
cylindrical kilns. US BurMines Tech Pap 1927.
[20] Bogiatzidis IX, Safacas AN, Mitronikas ED. Detection of backlash phe-
nomena appearing in a single cement kiln drive using the current
and the electromagnetic torque signature. IEEE Trans Ind Electron
2013;60(8):3441–53.
[21] Xu X. Cement clinker f-CaO soft sensor model based on time series analysis
and support vector machine (Master thesis), Hunan University; 2017, [in
Chinese].
[22] Li W, Wang D, Zhou X, Chai T. An improved multi-source based soft sensor
for measuring cement free lime content. Inform Sci 2015;323:94–105.
[23] Izakian H, Pedrycz W. Agreement-based fuzzy C-means for clustering data
with blocks of features. Neurocomputing 2014;127:266–80.
[24] Jie L, Liu W, Sun Z, Teng S. Hybrid fuzzy clustering methods based on
improved self-adaptive cellular genetic algorithm and optimal-selection-
based fuzzy c-means. Neurocomputing 2017;249:140–56.
[25] Xu R, Ii DCW. Ieee, survey of clustering algorithms. IEEE Trans Neural Netw
2005;16(3):645–78.
[26] Choudhury S, Ghosh S, Bhattacharya A, Fernandes KJ, Tiwari MK. A real
time clustering and SVM based price-volatility prediction for optimal
trading strategy. Neurocomputing 2014;131:419–26.
[27] Bai X, Wang W. Saliency-SVM: An automatic approach for image
segmentation. Neurocomputing 2014;136:243–55.
[28] Pan X, Pang X, Wang H, Xu Y. A safe screening based framework for
support vector regression. Neurocomputing 2018;287:163–72.
[29] Wang T, Qi J, Xu H, Wang Y, Liu L, Gao D. Fault diagnosis method
based on FFT-RPCA-SVM for cascaded-multilevel inverter. ISA Trans
2016;60:156–63.
[30] Vapnik V. the nature of statistical learning theory. New York:
Springer-Verlag; 1995.
[31] Borhani TNG, Bagheri M, Manan ZA. Molecular modeling of the
ideal gas enthalpy of formation of hydrocarbons. Fluid Phase Equilib
2013;360(9):423–34.
[32] Gholami AR, M. Shahbazian. Soft sensor design based on fuzzy c-means
and rfn_svr for a stripper column. J Nat Gas Sci Eng 2015;25:23–9.
[33] Yu L, Wang S, Lai KK. Credit risk assessment with a multi-
stage neural network ensemble learning approach. Expert Syst Appl
2008;34(2):1434–44.
[34] Zhang Z, Wen G, Chen S. Audible sound-based intelligent evaluation for
aluminum alloy in robotic pulsed gtaw: mechanism, feature selection and
defect detection. IEEE Trans Ind Inform 2018;14(7):2973–83.
[35] Gao X, Hou J. An improved svm integrated gs-pca fault diagnosis approach
of tennessee eastman process. Neurocomputing 2016;174:906–11.
[36] Kumar A, Kumar R. Time-frequency analysis and support vector machine
in automatic detection of defect from vibration signal of centrifugal pump.
Measurement 2017;108:119–33.
[37] Gu J, Zhu M, Jiang L. Housing price forecasting based on genetic algorithm
and support vector machine. Expert Syst Appl 2011;38(4):3383–6.
[38] Rubio-Sánchez M, Raya L, Díaz F, Sanchez A. A comparative study
between RadViz and Star Coordinates. IEEE Trans Vis Comput Graph
2016;22(1):619–28.
[39] Han M, Sun Y, Fan Y. An improved fuzzy neural network based on T–S
model. Expert Syst Appl 2008;34(4):2905–20.
[40] Ticknor JL, Hsu-Kim H, Deshusses MA. A robust framework to pre-
dict mercury speciation in combustion flue gases. J Hazard Mater
2014;264(2):380–5.