Professional Documents
Culture Documents
Predicting Permeability of Tight Carbonates Using A Hybrid Machine Learning Approach of Modified Equilibrium Optimizer and Extreme Learning Machine
Predicting Permeability of Tight Carbonates Using A Hybrid Machine Learning Approach of Modified Equilibrium Optimizer and Extreme Learning Machine
Predicting Permeability of Tight Carbonates Using A Hybrid Machine Learning Approach of Modified Equilibrium Optimizer and Extreme Learning Machine
https://doi.org/10.1007/s11440-021-01257-y (0123456789().,-volV)
(0123456789().,-volV)
RESEARCH PAPER
Received: 9 February 2021 / Accepted: 29 May 2021 / Published online: 5 July 2021
Ó The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2021
Abstract
It is a problematic task to perform petro-physical property prediction of carbonate reservoir rocks in most cases, specif-
ically for permeability prediction since a carbonate rock most commonly contains grains of heterogeneous size distribu-
tions. Consequently, the permeability calculation of tight rocks in laboratories is costly and very time-consuming.
Therefore, this study aims to tackle this issue by developing novel hybrid models based on combination of the modified
version of the equilibrium optimizer (EO), i.e., MEO, and two conventional machine learning algorithms, namely extreme
learning machine (ELM) and artificial neural network (ANN). The MEO employs a mutation mechanism in order to avoid
trapping in local optima of EO by increasing the search capabilities. In this study, ELM-MEO and ANN-MEO, novel
metaheuristic ELM-based and ANN-based algorithms, were constructed to predict the permeability of tight carbonates. In
addition, ANN, ELM, RF, RVM and MARS combined with particle swarm optimization and genetic programming
algorithm have a better insight into the performances for preferably predicting the permeability carbonates. The results
illustrate that the proposed ELM-MEO model with R2 = 0.9323, RMSE = 0.0612 and MAE = 0.0442 in training stage and
R2 = 0.8743, RMSE = 0.0806 and MAE = 0.0660 in testing stage, outperformed other ELM-based and ANN-based
metaheuristic models in predicting the permeability of tight carbonates at all levels.
1 Introduction
123
1240 Acta Geotechnica (2022) 17:1239–1255
represented as the cementation exponent m. The grain size characterization is a multi-variable problem that is difficult
can vary in the depth and sideward directions at large to approach. Xue et al. [16] found that the use of GA and
scales. They also most commonly undergo considerable backpropagation neural networks (BPNNs) in combination
changes in a small distance [5]. could make fracture zone predictions via shallow and deep
Permeability (k) is an essential variable to characterize electrical logs in the form of inputs. The determination of
and manage reservoirs. Although permeability can be facies and petro-physical properties is an evident advantage
measured on samples of cores in laboratory settings, it of ML techniques. BPNNs and convolutional neural net-
cannot be directly obtained by measuring the downhole. As works have been employed for the improvement of total
it is costly to measure core samples and such samples cover organic carbon estimates and both volatile and remaining
merely a small portion of reservoir intervals, it is required hydrocarbon calculations within a shale oil reservoir [17].
to adopt other techniques. As a result, many empirical They have higher performance than conventional tech-
models were developed to obtain permeability through niques in such estimates. Lim and Kim [18] introduced
numerous proxy measurements that can commonly be neural network and fuzzy logic methods for reservoir
performed down the hole. Even so, these problems are permeability and porosity prediction. They demonstrated
extremely complex because of the highly nonlinear stress– that the methods had further development potential. Tang
strain relationships, as well as elastoplastic performance [19] and Tang et al. [20] adopted probabilistic neural net-
under loading and unloading conditions. Therefore, tradi- works for the purpose of carbonate reservoir facies clas-
tional empirical equations are insufficient to adequately sification and succeeded to some extent. Zhou et al. [21]
describe the mapping relation between rock properties and integrated diagenesis and a deep autoencoder random for-
permeability (k). The primary aim of this research is to est algorithm in order to explore how various diagenesis
develop an easier, cheaper, and more effective way to states were related to the electrical variables (e.g., m and n)
assess the permeability of extremely tight, heterogeneous of tight gas sandstone reservoirs. Zhu et al. [22, 23]
reservoir rocks that would help characterize these reser- employed a complicated combination of ML and NMR
voirs as efficient and reliable technical tools. inputs to reasonably predict tight gas sandstone reservoir
Researchers have employed machine learning (ML) for permeability. Their results were found to be comparable to
prediction purposes. The present study does not aim to those of Rashid et al. [3], who adopted conventional pre-
review machine learning algorithms (MLAs) and the use of diction methods of permeability.
such algorithms in geophysical problems. MLAs have a Many studies attempted to improve conventional ML
rich literature, and proper reviews of their applications are performance, e.g., extreme learning machines (ELMs) and
available [6–10]. This work incorporates several recent ML ANN. Despite their higher outcomes than statistical
applications for geo/petro-physical purposes. Although ML approaches, traditional MLAs are more prone to local
can be employed for many possible purposes, advance- minima trapping rather than capturing the exact global
ments have recently included every petro-physical aspect, minima. This yields undesired outcomes. Hence, to tackle
such as logging via facies determination, rock characteri- this problem, studies are employing optimization algo-
zation, and key variable determination for the permeability rithms for the purpose of updating classical ML parame-
and volumetrics calculations of reservoirs. Studies ters, reporting significant findings. This work employs the
employed integrated deep learning models (DLMs) [11] Equilibrium Optimizer [24] as a novel metaheuristic opti-
and integrated hybrid neural networks (IHNNs) [12] to mization algorithm that was recently introduced. The
substantially enhance total organic carbon (TOC) estimates combination of ELM and EO (ELM-EO) and a combina-
and improve shale gas reservoir characterization. Onalo tion of ANN and EO (ANN-EO) are introduced in this
et al. [13, 14] adopted a nonlinear autoregressive neural study. As a new metaheuristic optimization algorithm, EO
network with exogenous input (NARX) for the estimation is rapid and powerful and has a high population-based
of shear and compressional sonic travel times within well performance. According to experimental EO results, it has
logs. They predicted actual sonic well logs at adequately higher performance than other well-known approaches
high efficiency. They obtain sonic properties, such as [24], yielding significantly comparable results. Thus, the
Poisson’s ratio and sonic porosity. Also, ML was exploited present study adopts EO to predict permeability in tight
to find optimal reservoir parameters. For example, Zhu carbonates.
et al. [15] recently studied the saturation of water within Like other metaheuristic algorithms, however, EO
organic shale reservoirs. They calculated shale petro- encounters a number of limitations. Therefore, the present
physical parameters via the genetic algorithm (GA) opti- work aims to enhance the standard EO. To this end, a
mization approach. This method requires no electrical combination of mutation-based searching and EO algo-
measurements in the form of inputs. Thus, it perfectly suits rithm is employed for the performance enhancement of
organic shale reservoirs. Reservoir fracture standard EO. Then, a modified EO (MEO) is utilized to
123
Acta Geotechnica (2022) 17:1239–1255 1241
build the ELM-based modified EO (ELM-MEO) and ANN- 2.2 Artificial neural network (ANN)
based modified EO (ANN-MEO) to estimate tight car-
bonate permeability. Furthermore, the potential of ELM- Structurally inspired by the human brain, ANNs comprises
MEO, ELM-EO, ANN-MEO, and ANN-EO for general- of simple and small processing units or the so-called arti-
ization is compared to the hybridization of particle swarm ficial neurons or nodes to be used in computational meth-
optimization (PSO) with ELM and ANN, i.e., ELM-PSO ods. This structure has turned ANNs into effective
and ANN-PSO. In addition, to have a better insight of mathematical tools for different objectives [27, 28],
performances of the hybrid models, four typical ML including function estimation and pattern recognition. An
models including random forest (RF), genetic program- ANN comprises of three main layers where neurons lie.
ming (GP), multivariate adaptive regression spline Neurons in the input layer are equal input variables in
(MARS), and relevance vector machine (RVM) approaches number and those in the output layer equal output vari-
are also employed on the dataset. ables. At least one hidden layer lies between these two
The rest of this paper is organized as follows. Section 2 layers. Signal is transmitted through the input layer, hidden
covers the methodology of this study including the detailed layers function as the computational engine of the network
description of the applied models and the hybridization and the output layer makes predictions based on the input
process. The data processing and analysis as well as per- variables. As the main parameters in an ANN, bias and
formance indices are presented in Sect. 3, while Sect. 4 weights, respectively, determine the degree of freedom of
illustrates and compares the obtained results of the applied the network and relationships between interconnected
models in every possible way. Finally, a summary and neurons of a layer [29–31]. Except for the input nodes,
conclusion are provided in Sect. 5. every node utilizes a nonlinear activation transfer function
to determine its output while providing a set of inputs. The
output determined is then employed as an input for the next
2 Methodology node, and this process continues until a reliable solution to
the original problem is found.
2.1 Extreme learning machine (ELM)
2.3 Metaheuristic optimization algorithms
An ELM refers to a neural network that serves as an
innovative data-driven instrument. It makes use of a 2.3.1 Equilibrium optimizer (EO)
modern single-layer feed-forward network (SLFN) to
produce a closed-form solution to the output weights once EO algorithm which is inspired from dynamic mass bal-
the weights and biases of the hidden layer have been fixed. ance was first introduced by Faramarzi et al. [24].
To use an ELM for data modelling, a straightforward three- According to EO approach, an equation of a mass balance
stage process is utilized. The stages involve (1) randomly employs various source and sink mechanisms of a non-
(and not iteratively) building the biases and weights of the reactive component in a control volume in order to define
hidden layer, (2) proceeding the inputs via the parameters its concentration. The equation of mass balance offers the
of the hidden layer to generate the output matrix of the fundamental mechanics for the preservation of the entering,
hidden layer, and (3) making output weight estimates leaving and producing mass. Basically, mathematical
through the inversion of the output matrix of the hidden model of the EO algorithm is comprised of three main steps
layer, in which the generalized Moore–Penrose inverse as follows:
matrix is applied, before calculating the response variable 1. Initialization
(that is, the solving of some linear equations). To this end, EO employs a set of particles in this step, in which each
it is required to randomize the hidden neurons after iden- particle reflects the vector of concentration that includes
tifying their nodes. Typically, trial-and-error is applied to a the solution for the optimization problem. The preliminary
cross-validation dataset to identify the hidden neurons. For vector concentrations are developed at random in the
real-time executions, ELMs provide a prominent advantage search space as below:
over other techniques with low rates of convergence, lower
generalizability, local minima problems, iterative tuning, vi ¼ cmin þ ðcmax cmin Þ r
~ i ¼ 0; 1; 2; . . .; n ð1Þ
and data over-fitting [25, 26]. As a result, the quick and where the particle i concentration vector is shown by ~ vi ,
enhanced ELM performance is beneficial in real-time while lower and upper bound of dimensions are determined
applications. by cmin and cmax , respectively. Parameter n indicates the
number of particles and r is a randomly assigned number in
the range of [0, 1].
123
1242 Acta Geotechnica (2022) 17:1239–1255
123
Acta Geotechnica (2022) 17:1239–1255 1243
candidate solution population that was derived after the EO best position of an individual particle is viewed to be the
procedure update undergoes sorting based on the fitness of personal best location. The position updating of a particle
the solutions. Then, the population is divided into three is performed based on the personal best position and the
equal parts. The first part of candidate solution having global best position direction. At the same time, the
better fitness is preserved for next search process. The velocity updating of a particle is grounded on the differ-
remaining portion of the new population involves the ence between personal and global best locations. The
candidates that are produced via mutation applied to the particles obtain convergence in the vicinity of the optimal
top N3 candidates and those that are produced below applied solution by an exploitation–exploration mixture.
to the top N3 population candidates. The theoretical details of other employed soft computing
0 techniques, i.e., RF, RVM, MARS, and GP can be referred
C i ¼ C i :ð1 þ GaussianðhÞÞ ð11Þ from the literature; [35–38] for RF, [39–41] for RVM,
!
!itþ1 !it !it ! !
[42–46] for MARS, and [47–49] for GP.
C i ¼ C eqð1Þ C m r 3 ½ C min þ r 4 C max C min
2.4 Hybridization process
ð12Þ
~it 1X N
~ it This work adopted EO as a newly developed metaheuristic
Cm ¼ C ð13Þ algorithm for ANN and ELM learning parameter opti-
N i¼1 i
mization to perform tight carbonate permeability predic-
where r 3 and r 4 are random variables ranging from 0 to 1. tion. The learning parameters of an ANN include input
A movement length of (which refers to the difference weights, hidden neuron biases, output weights, and output
between Cmax and Cmin ) with random scaling (r 4 ) is added bias, whereas those of an ELM involve input weights,
to Cmin . GaussianðhÞ represents the Gaussian distributed hidden biases, and output weights. Once the ANN and
random vector. It is assumed that the Gaussian mutation- ELM have been initialized, EO is employed for the opti-
produced candidate performs solution space exploration in mization of learning parameters—that is, the biases and
the vicinity of the top one-third of the population candi- weights. In this respect, the EO algorithm is set up prior to
dates in terms of fitness. Hence, such a mutation procedure ML learning parameter optimization. This includes the
helps explore search areas in the vicinity of the elite optimization of the population size, maximum iteration
solution space regions. Mutation also helps to deal with count, lower and upper bounds, and the hidden layer neu-
sub-optimal search traps. Likewise, exploration in the ron count of the ELM and ANN. Then, EO optimizes the
vicinity of the elite candidates that have been derived by ANN and ELM biases and weights. The same process as
Eq. (12) is helpful in the formation of the one-third of the that of ANN-EO and ELM-EO models is performed in
population. Such exploration is also helpful in convergence ANN-MEO and ELM-MEO. The optimized learning
rate enhancement and enables searching in the elite areas. parameter values are found by the smallest root mean
As a result, the proposed approach utilizes these concepts square error (RMSE). Even though the parameters, such as
for exploring a large area within the search space while the hidden neuron count, population size, maximum itera-
enhancing candidate solution diversity in the search. Fur- tion count and lower and upper bounds, remain unchanged,
ther details can be referred from the studies of Gupta et al. the optimized learning parameter values change in EO and
[32]. Other optimization such as Bayesian type can be MEO. Furthermore, it is worth noting that these parameters
referred to Zhang et al. [33]. have a significant impact on optimization. Thus, it is nec-
essary to appropriately tune such parameters in the
2.3.3 Particle swarm optimization (PSO) hybridization procedure. Fig. 1 depicts the hybridization
procedure of ELM-based and ANN-based models. As can
Kennedy and Eberhart [34] proposed PSO as a swarm- be seen, the ELM-based and ANN-based models devel-
grounded metaheuristic approach. Bird and fish schooling opment steps are represented.
and flocking patterns mainly inspired PSO, and the primary
objective is to identify solutions of global optimality within
a multidimensional space. At the beginning of PSO, ran- 3 Data processing and performance metrics
dom particle positions and velocities are initialized. Then,
each of the particles undergoes location updating to find the The present work proposes ELM-PSO, ELM-EO, and
best position within the multidimensional space on the ELM-MEO as ELM-based hybrid metaheuristic approa-
ground of the velocity, personal best position, and global ches and ANN-PSO, ANN-EO, and ANN-MEO as ANN-
best position. Nonetheless, the best position of all the base hybrid metaheuristic approaches. The min–max
particles is treated to be the global best position, while the method is applied to normalize the entire dataset. In soft
123
1244 Acta Geotechnica (2022) 17:1239–1255
Fig. 1. Flow chart showing the methodological steps of the hybridization process
computing, data normalization as a pre-processing stage of used as input variables, while permeability k (mD) is used
solving problems is viewed as the most essential stage. In as a dependent variable to accomplish this goal. Because of
general, data are normalized for the dimensional effect the physical nature of the tight carbonates, the helium
nullification of variables. Therefore, during pre-processing, porosity method was chosen to calculate Ø instead of the
the data undergo normalization in an upper limit (1) and water saturation porosity method. Additionally, for each
lower limit (0)-defined range by the ‘min-max’ method. case, the fluid permeability method and mercury injection
Next, the main dataset which is collected from experiments capillary pressure measurements were used to assess the
[50, 51] is classified into a training subset and a testing related variable. Table 1 lists the summary of the dataset
subset. To this end, 75% (98 observations) of the data are
randomly used as the training data, whereas the remaining
25% (32 observations) of the data is employed as the Table 1 Statistical summary of the input and output variables
testing data. Then, the training data is utilized for model Particulars Ø (-) F (-) Dpt (m) k (mD)
training, whereas the testing data are utilized for model
Mean 0.1790 62.2692 2.8844e-08 0.005244
validation. According to [50], the Portland Formation,
Standard deviation 0.0370 29.5644 4.0637e-08 0.020163
located at 50°330 1000 N 02°260 2500 W in southern England,
Minimum 0.1070 17.0000 3.9200e-10 0.000002
provided the core plugs. The aim of this research is to
25% 0.1512 40.0000 6.5800e-09 0.000032
estimate the permeability of tight carbonates using a small
number of independent parameters that can be obtained 50% 0.1730 55.0000 1.3650e-08 0.000213
inexpensively and conveniently from standard core plug 75% 0.2150 74.7500 3.2500e-08 0.001227
measurements. Three properties, porosity Ø, formation Maximum 0.2650 200.0000 2.2800e-07 0.185000
resistivity factor F, and pore throat diameter Dpt (m), are
123
Acta Geotechnica (2022) 17:1239–1255 1245
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
used in this study, while Fig. 2 presents the correlation 1 Xn
between the input variables and permeability of tight RMSE ¼ i¼1 i
ðy ybi Þ2 ð30Þ
n
carbonates.
Pn
yi by i
To perform model performance evaluation, the perfor- i¼1
yi
yi
mance indexes of Adj:R2 , R2 ,PI, MAE, RMSE and WMAPE ¼ Pn ð31Þ
i¼1 yi
WMAPE are measured and compared in all possible
aspects. These indexes are mathematically shown as: where n stands for the observation count, p is the number of
input parameters, yi and yˆi are the real and modelled value
ðn 1Þ
Adj:R2 ¼ 1 ð1 R2 Þ ð26Þ i of permeability (i.e., the output), and ymean represents the
ðn p 1Þ
mean input. To obtain a desirable prediction model, it is
Pn 2 Pn
2 i¼1 ðyi ymean Þ bi Þ2
i¼1 ðyi y
required that these indexes should be set to the desirable
R ¼ Pn 2
ð27Þ value, as shown in Table 2.
i¼1 ðyi ymean Þ
Fig. 2. a Correlation matrix and b relationships between input variables and the output variable
123
1246 Acta Geotechnica (2022) 17:1239–1255
Table 2 Performance indices and their ideal values. Table 3 Parameter configuration of the hybrid models
Trend indices Ideal value Error indices Ideal value Parameters ELM- ELM- ELM- ANN- ANN- ANN-
EO MEO PSO EO MEO PSO
Adj:R2 1 MAE 0
2 1 RMSE 0 No. of hidden 10 10 10 10 10 10
R
neuron (N)
PI 2 WMAPE 0
No. of hidden 1 1 1 1 1 1
layer (Nh)
Population size 30 30 30 30 30 30
(P)
4 Results and discussion Max. no. of 500 500 500 500 500 500
iterations (itr)
Inertia weights – – 0.9, – – 0.9,
This study carried out a detailed evaluation of parameters
(wmax ; wmin ) 0.4 0.4
to identify the best prediction model. All the data were
Acceleration – – 1,2 – – 1, 2
classified into training data (with 98 observations) and coefficient
testing data (with 32 observations). The former was (c1 ; c2 Þ
employed to build ANN- and ELM-based metaheuristic Random – – 0–1 – – 0–1
models, while the latter was utilized for model validation. parameters
(r 1 ; r 2 )
The above-mentioned indexes were assessed for predictive
performance estimation. The proposed models were com- Upper and lower ±1 ±1 ±1 ±1 ±1 ±1
bound (ub; lbÞ
pared in terms of prediction performance below. Moreover,
No. of learning 40 40 40 51 51 51
the ANN- and ELM-grounded models are comparatively parameters
assessed in terms of metaheuristic parametric configura- Generation 0.50 0.50 – 0.50 0.50 –
tions, introducing the best prediction model. probability
(GP)
4.1 Implementation of models Exploration 2 2 – 2 2 –
constant (a1 )
Once the experimental data have been arranged, the EO,
MEO, and PSO approaches are utilized for the bias and
weight optimization of the ANN- and ELM-based algo-
rithms. As metaheuristic algorithms are of a random nature,
it is essential to find the parametric configuration (that is,
the parameters that can be adjusted) in all optimization
models. Hence, it is required to tune the adjustable param-
eters during process execution. The optimal hidden layer
neuron count is found to be 10 by using trial-and-error in
the ANN. Here, deterministic parameters undergo robust
analyses, as shown in Table 3. MEO is observed to have
effective performance (within the test stage based on the R2
and RMSE results) within the ELM and ANN. Figures 3
and 4 show iterative performance of ELM-based and ANN-
based models, respectively.
ELM-PSO, ANN-EO, ANN-MEO, ELM-EO, and ELM- Fig. 3. Convergence curves of ELM-based hybrid models
MEO are developed as optimized metaheuristic ANN- and
ELM-based techniques for tight carbonate permeability error is applied to determine the hidden neuron count as
estimation. Once the data have been classified, the training they have a random nature. Furthermore, the linear
data are utilized to train the Ann and ELM by using EO, (trainlm) and tangent sigmoid (tansig) functions are
MEO, and PSO prior to retrieving the optimized biases and applied as the training and transfer functions, respectively.
weights. It is required to incorporate a hidden layer con- ELM-EO, ELM-MEO, ELM-PSO, ANN-EO, ANN-MEO,
taining 10 hidden neurons for the ANN models. Trial-and- and ANN-MEO are developed based on Table 3.
123
Acta Geotechnica (2022) 17:1239–1255 1247
123
1248 Acta Geotechnica (2022) 17:1239–1255
Fig. 5. a–j Regression plots between measured and predicted values at the training stage
the other hand, regression error characteristic (REC) curve MEO is seemingly in a shorter distance from the reference
[54–56] represents the error tolerance on the x-axis and point, while, Figs. 9 and 10 illustrate the REC curves of the
regression accuracy on the y-axes, respectively. The error proposed models in the training and testing stages,
on the x-axis is typically discussed based on the absolute respectively. It is also seen that the AUC quantities of the
deviation (AD) or squared error (SE). However, the entire metaheuristic prediction models are seemingly high;
obtained curve is a measure of the cumulative distribution ELM-MEO exhibited the largest AUC quantities of 0.9636
function of the error. Typically, a lower distance of the and 0.9446 in the training and testing phases, respectively,
curve from the top-left corner represents higher accuracy. turned out to be the best predictive model.
The area under the curve (AUC) is calculated to obtain the
overall accuracy of a predictive model.
The Taylor graphs of the training and testing data are
plotted in Figs. 7 and 8, respectively. The position of ELM-
123
Acta Geotechnica (2022) 17:1239–1255 1249
Fig. 6. a–j Regression plots between measured and predicted values at the testing stage
4.5 Uncertainty analysis (UA) absolute error between the observed data (yi ) and predicted
quantities (b
y i ) as:
The proposed models and typical MLAs are subjected to ei ¼ jyi ybi j ð34Þ
quantitative evaluation (i.e., they are analyzed for uncer-
tainty) in terms of tight carbonate permeability prediction. Additionally, one can obtain the mean of error (MOE)
To this end, the results of the training and test data were and standard deviation (SD) for a given prediction as:
combined. The datasets involved 130 real-time observation PN
ei
data points. Therefore, it may be beneficial to subject MOE ¼ i¼1 ð35Þ
N
predictive results to logical comparison to perform the
reliability evaluation of the models. Furthermore, one can
adopt the UA index to examine other comparative evalu-
ation parameters. In this respect, it is possible to obtain the
123
1250 Acta Geotechnica (2022) 17:1239–1255
Fig. 7. Taylor diagram of the developed models: a ELM-based models; b ANN-based models; c other models for the training dataset
Fig. 8. Taylor diagram of the developed models: a ELM-based models; b ANN-based models; c other models for the testing dataset
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
PN 2 and is capable of prediction at higher accuracy. According
i¼1 ðei MOEÞ to the UA results, ELM-MEO yielded the smallest WCB
SD ¼ ð36Þ
N1 and ME values of 0.0334 and 0.0167, respectively. Thus,
in which ei denotes the absolute error, N stands for the ELM-MEO is found to have the highest predictive accu-
observation count. After calculation of the MOE and SD of racy. Furthermore, smaller SE, SD, and MOE quantities
the training and test data, this study calculated the margin suggest higher reliability for ELM-MEO than the other
of error (ME) at a confidence interval of 95% to obtain the proposed models. As a result, the proposed ELM-based
width of confidence bound (WCB). Then, the standard approaches, specifically ELM-MEO, are observed to have
error (SE), lower bound (LB) and upper bound (UB) were the lowest uncertainty and highest confidence. Fig. 11
computed as: illustrates the UA results in horizontal bar chart, demon-
strating the MOE, SD, SE, ME, LB, UB, and WCB values
SD for the purpose of a better comparison.
SE ¼ pffiffiffiffiffiffiffiffiffiffiffiffi ; UB ¼ MOE þ ME; LB
N1
¼ MOE ME; WCB ¼ UB LB ð37Þ 4.6 Discussion on results
WCB stands for a range of errors where nearly 95% of
the data rest. Table 6 provides a detailed description of UA. Previous subsections provided a detailed result description
As can be seen, the number of observations (N), SD, MOE, of the proposed ELM and ANN-based metaheuristic
ME, SE, UB, WCB, and LB are provided. According to models. This subsection will present a summarized dis-
Table 6, it is possible to perform the performance evalua- cussion based on the results obtained in this study. At the
tion of the models. It should be noted that a smaller WCB beginning of developing ELM and ANN-based meta-
represents higher certainty. In other words, a smaller WCB heuristic approaches, training data were employed to
implies that the model undergoes a smaller level of error examine the goodness of fitness (GOF). Also, test data
123
Acta Geotechnica (2022) 17:1239–1255 1251
123
1252 Acta Geotechnica (2022) 17:1239–1255
ELM-EO, ELM-MEO, ELM-PSO, ANN-EO, ANN-MEO diagrams, convergence curves, REC curves, and accuracy
and ANN-PSO). Among common ML approaches, how- matrices to indicate the strengths of the proposed models.
ever, R2 was obtained to be below 0.90 for RF The Taylor diagrams mathematically described the models
(R2 = 0.8242). Nonetheless, ELM-MEO exhibited the based on RMSE,R2 and variance ratios. Similarly, an
highest predictive performance, with R2 = 0.9323, accuracy matric reveals the accuracy degree of a given
RMSE = 0.0612 and MAE = 0.0442, upon the training model for a given performance index. The results of ELM-
data. Concerning the testing data, ELM-MEO outper- MEO model were compared to the results of [50] which
formed the other algorithms in terms of prediction. How- used the same dataset and influential variables as this study
ever, the remaining four models did not follow the same with feedforward multilayer perceptron network (FFMLP)
descending order on the testing data as that of the training and genetic algorithm (GA) as ML models. R2 values for
data. In other words, the prediction performance ranks FFMLP and GA in [50] are 0.908 and 0.888 in training
were found to be in the order of ELM-MEO [ ELM- stage and 0.886 and 0.858 in testing stage, respectively. It
EO [ GP [ RVM [ ELM-PSO [ ANN-EO [ RF [ exhibits the superiority of the proposed ELM-MEO with R2
ANN-MEO [ ANN-PSO [ MARS. In addition, graphical
result demonstrations were provided in the forms of Taylor
123
Acta Geotechnica (2022) 17:1239–1255 1253
values of 0.9323 and 8843 in training and testing stages, Acknowledgements This work was financially supported by the
respectively. High-end Foreign Expert Introduction program (No. G20200022005)
and Science and Technology Research Program of Chongqing
Municipal Education Commission (No. KJCXZD2020002).
123
1254 Acta Geotechnica (2022) 17:1239–1255
12. Zhu L, Zhang C, Zhang C, Wei Y, Zhou X, Cheng Y et al (2018) 30. Kardani MN, Baghban A, Hamzehie ME, Baghban M (2019)
Prediction of total organic carbon content in shale reservoir based Phase behavior modeling of asphaltene precipitation utilizing
on a new integrated hybrid neural network and conventional well RBF-ANN approach. Petrol Sci Technol 37(16):1861–1867
logging curves. J Geophys Eng 15(3):1050–1061 31. Kardani N, Zhou A, Nazem M, Lin X (2021) Modelling of
13. Onalo D, Adedigba S, Khan F, James LA, Butt S (2018) Data municipal solid waste gasification using an optimised ensemble
driven model for sonic well log prediction. J Petrol Sci Eng soft computing model. Fuel 289:119903
170:1022–1037 32. Gupta S, Deep K, Mirjalili S (2020) An efficient equilibrium
14. Onalo D, Oloruntobi O, Adedigba S, Khan F, James L, Butt S optimizer with mutation strategy for numerical optimization.
(2019) Dynamic data driven sonic well log model for formation Appl Soft Comput 96:106542
evaluation. J Petrol Sci Eng 175:1049–1062 33. Zhang W, Wu C, Zhong H, Li Y, Wang L (2021) Prediction of
15. Zhu L, Zhang C, Zhang C, Zhang Z, Zhou X, Zhu B (2019) An undrained shear strength using extreme gradient boosting and
improved theoretical nonelectric water saturation method for random forest based on Bayesian optimization. Geosci Front
organic shale reservoirs. IEEE Access 7:51441–51456 12:469–477
16. Xue Y, Cheng L, Mou J, Zhao W (2014) A new fracture pre- 34. Kennedy J, Eberhart R (1995) Particle swarm optimization. In:
diction method by combining genetic algorithm with neural Proceedings of ICNN’95-international conference on neural
network in low-permeability reservoirs. J Petrol Sci Eng networks, vol. 4. IEEE, pp 1942–1948.
121:159–166 35. Ho TK (1995) Random decision forests. In: Proceedings of 3rd
17. Wang H, Wu W, Chen T, Dong X, Wang G (2019) An improved international conference on document analysis and recognition,
neural network for TOC, S1 and S2 estimation based on con- vol 1. IEEE, pp 278–282
ventional well logs. J Petrol Sci Eng 176:664–678 36. Kardani N, Zhou A, Nazem M, Shen S-L (2021) Improved pre-
18. Lim J-S, Kim J (2004) Reservoir porosity and permeability diction of slope stability using a hybrid stacking ensemble
estimation from well logs using fuzzy logic and neural networks. method based on finite element analysis and field data. J Rock
In: SPE Asia Pacific oil and gas conference and exhibition. Mech Geotech Eng 13(1):188–201
Society of Petroleum Engineers. 37. Zhang W, Wu C, Li Y, Wang L, Samui P (2019) Assessment of
19. Tang H (2008) Improved carbonate reservoir facies classification pile drivability using random forest regression and multivariate
using artificial neural network method. In: Canadian international adaptive regression splines. Georisk Assess Manag Risk Eng Syst
petroleum conference. OnePetro Geohazards 15:1–14
20. Tang H, Meddaugh WS, Toomey N (2011) Using an artificial- 38. Zhang W, Zhang R, Wu C, Goh AT, Wang L (2020) Assessment
neural-network method to predict carbonate well log facies suc- of basal heave stability for braced excavations in anisotropic clay
cessfully. SPE Reserv Eval Eng 14(1):35–44 using extreme gradient boosting and random forest regression.
21. Zhou X, Zhang C, Zhang Z, Zhang R, Zhu L, Zhang C (2019) A Undergr Space. https://doi.org/10.1016/j.undsp.2020.03.001
saturation evaluation method in tight gas sandstones based on 39. Kardani N, Bardhan A, Kim D, Samui P, Zhou A (2020) Mod-
diagenetic facies. Mar Petrol Geol 107:310–325 elling the energy performance of residential buildings using
22. Zhu L-q, Zhang C, Wei Y, Zhang C-m (2017) Permeability advanced computational frameworks based on RVM, GMDH,
prediction of the tight sandstone reservoirs using hybrid intelli- ANFIS-BBO and ANFIS-IPSO. J Build Eng 35:102105
gent algorithm and nuclear magnetic resonance logging data. 40. Tipping ME (2000) The relevance vector machine. In: Solla S,
Arab J Sci Eng 42(4):1643–1654 Leen T, Müller K (eds) Advances in neural information pro-
23. Zhu L, Zhang C, Wei Y, Zhou X, Huang Y, Zhang C (2017) cessing systems, vol 12. MIT Press, pp 652–658
Inversion of the permeability of a tight gas reservoir with the 41. Viswanathan R, Jagan J, Samui P, Porchelvan P (2015) Spatial
combination of a deep Boltzmann kernel extreme learning variability of rock depth using simple kriging, ordinary kriging,
machine and nuclear magnetic resonance logging transverse RVM and MPMR. Geotech Geol Eng 33(1):69–78
relaxation time spectrum data. Interpretation 5(3):T341–T350 42. Wang L, Chongzhi W, Xin G, Liu H, Mei G, Zhang W (2020)
24. Faramarzi A, Heidarinejad M, Stephens B, Mirjalili S (2020) Probabilistic stability analysis of earth dam slope under transient
Equilibrium optimizer: a novel optimization algorithm. Knowl seepage using multivariate adaptive regression splines. Bull Eng
Based Syst 191:105190 Geol Environ. https://doi.org/10.1007/s10064-020-01730-0
25. Kardani N, Bardhan A, Samui P, Nazem M, Zhou A, Armaghani 43. Wengang Z, Runhong Z, Wei W, Fan Z, Goh ATC (2019) A
DJ (2021) A novel technique based on the improved firefly Multivariate adaptive regression splines model for determining
algorithm coupled with extreme learning machine (ELM-IFF) for horizontal wall deflection envelope for braced excavations in
predicting the thermal conductivity of soil. Eng Comput 1:1–20 clays. Tunnel Undergr Space Technol 84:461–471
26. Wang L, Chongzhi Wu, Tang L, Zhang W, Lacasse S, Liu H, Gao 44. Zhang W, Goh ATC (2013) Multivariate adaptive regression
L (2020) Efficient reliability analysis of earth dam slope stability splines for analysis of geotechnical engineering systems. Comput
using extreme gradient boosting method. Acta Geotech Geotech 48:82–95
15(11):3135–3150 45. Zhang W, Goh ATC (2016) Multivariate adaptive regression
27. Kardani N, Zhou A, Nazem M, Shen S-L (2020) Estimation of splines and neural network models for prediction of pile driv-
bearing capacity of piles in cohesionless soil using optimised ability. Geosci Front 7(1):45–52
machine learning approaches. Geotech Geol Eng 46. Zhang W, Zhang Y, Goh AT (2017) Multivariate adaptive
38(2):2271–2291 regression splines for inverse analysis of soil and wall properties
28. Zhang R, Wu C, Goh AT, Böhlke T, Zhang W (2020) Estimation in braced excavation. Tunnel Undergr Space Technol 64:24–33
of diaphragm wall deflections for deep braced excavation in 47. Deo RC, Samui P (2017) Forecasting evaporative loss by least-
anisotropic clays using ensemble learning. Geosci Front square support-vector regression and evaluation with genetic
12(1):365–373 programming, Gaussian process, and minimax probability
29. Chen L, Zhang W, Gao X, Wang L, Li Z, Böhlke T, Umberto machine regression: case study of Brisbane City. J Hydrol Eng
(2020) Design charts for reliability assessment of rock bedding 22(6):05017003
slopes stability against bi-planar sliding: SRLEM and BPNN 48. Holland JH (1992) Genetic algorithms. Sci Am 267(1):66–73
approaches. Georisk. https://doi.org/10.1080/17499518.2020.
1815215
123
Acta Geotechnica (2022) 17:1239–1255 1255
49. Samui P, Dalkiliç Y, Jagan J (2015) Site characterization using 53. Taylor KE (2001) Summarizing multiple aspects of model per-
GP. MARS and GPR handbook of genetic programming appli- formance in a single diagram. J Geophys Res Atmosp
cations. Springer, Berlin, pp 345–357 106(D7):7183–7192
50. Al Khalifah H, Glover P, Lorinczi P (2020) Permeability pre- 54. Bi J, Bennett KP (2003) Regression error characteristic curves.
diction and diagenesis in tight carbonates using machine learning In: Proceedings of the 20th international conference on machine
techniques. Mar Petrol Geol 112:104096 learning (ICML-03), pp 43–50
51. Barton C, Woods M, Bristow C, Newall A, Westhead R, Evans 55. Kumar M, Bardhan A, Samui P, Hu JW, Kaloop RM (2021)
DJ et al (2011) Geology of south Dorset and south-east Devon Reliability analysis of pile foundation using soft computing
and its World Heritage Coast: special memoir for 1: 50,000 techniques: a comparative study. Processes 9(3):486
geological sheets 328 Dorchester, 341/342 West Fleet and 56. Zhang W, Li H, Li Y, Liu H, Chen Y, Ding X (2021) Application
Weymouth and 342/343 Swanage, and parts of sheets 326/340 of deep learning algorithms in geotechnical engineering: a short
Sidmouth, 327 Bridport, 329 Bournemouth and 339 Newton critical review. Artif Intell Rev. https://doi.org/10.1007/s10462-
Abbot. British Geological Survey 021-09967-1
52. Ray R, Kumar D, Samui P, Bahadur Roy L, Goh ATC, Zhang W
(2021) Application of soft computing techniques for shallow Publisher’s Note Springer Nature remains neutral with regard to
foundation reliability in geotechnical engineering. Geosci Front jurisdictional claims in published maps and institutional affiliations.
12:375–383
123