Professional Documents
Culture Documents
Das2020 Article NewApproachesInMetaheuristicTo
Das2020 Article NewApproachesInMetaheuristicTo
Das2020 Article NewApproachesInMetaheuristicTo
https://doi.org/10.1007/s13369-019-04026-y
Received: 14 January 2019 / Accepted: 2 July 2019 / Published online: 10 July 2019
© King Fahd University of Petroleum & Minerals 2019
Abstract
Artificial Neural Networks have been applied as a useful technique for classification. Based on the above, we propose a
method called Velocity-Enhanced Whale Optimization Algorithm which is hybridized with Artificial Neural Network for
classifying and diagnosing the different cancer data like breast cancer, cervical cancer and lungs cancer. The proposed
algorithm is compared with four other benchmark algorithms available in the texts, i.e., C4.5, Learning Vector Quantiza-
tion, Linear Discriminate Analysis and Factorized Distribution Algorithm. Robustness of proposed approach is validated by
calculating the Correct Classification Rate, Averaged Square Classification Error which demonstrates that its performance
is better than other techniques.
Keywords Velocity-Enhanced Whale Optimization Algorithm (VEWOA) · Artificial Neural Network · Linear Discriminate
Analysis (LDA) · Learning Vector Quantization (LVQ) · Factorized Distribution Algorithm (FDA)
13
Vol.:(0123456789)
2460 Arabian Journal for Science and Engineering (2020) 45:2459–2471
13
Arabian Journal for Science and Engineering (2020) 45:2459–2471 2461
to the performance of different models has been given. In N = set of cases; p = number of partitions; |Ni | = number of
[27], pap smears which are performed 2 years prior to like- cases in the partition i; |N| = number of cases in N.
lihood of cervical cancer are discussed. False-negative pap Entropy formula is:
cytology causes fatal errors causing patients to be diagnosed
in advanced stages. They discussed the importance of such p
∑
tests. A detailed survey on positron emission tomography/ Entropy (N) = −ni log2 ni (2)
magnetic resonance imaging for detection of cervical cancer i=1
tissues is proposed in [28]. In [29], a technique to classify N = set of cases; p = number of partitions; ni = proportion of
the lung cancer while highlighting the importance of each Ni to N.
gene group from minority classes is presented. The proposed
method can identify more significant genes from the pool 3.2 Learning Vector Quantization (LVQ)
of all genes adaptively. The anatomic extent of a cancer is
provided through stage classification. The eighth edition of LVQ is a special case of ANN which is a prototype-based
lung cancer stage classification is considered to be world- supervised classification algorithm. It is similar to KNN and
wide standard as of January 2017 [30]. Lung cancers are is developed by Kohonen [35]. When we are having labeled
classified as malignant and benign and calculated the degree input data then we can use Learning Vector Quantization.
of their malignancy with Fourier Transform Infrared (FTIR) It is supervised version of vector quantization method.
spectroscopy [31]. The FTIR is also combined with principal Classifier decision regions are improved when the learn-
component analysis–linear discriminant analysis. In [32], ing method slightly reposition the Voronoi vectors by using
researchers provided a systematic literature review and ana- class information. Two stages are involved in this technique,
lyzed the lung cancer data from International Association i.e., self-organizing map (SOM) followed by LVQ. This
for the study of lung cancer database. They also developed method is used in pattern classification problems. The first
proposals for revision involving multispecialty international step involves feature selection, i.e., a small set of feature is
review. They discovered four patterns of the disease. In [33], selected using unsupervised learning method that contains
the correct and consistent regional lymph node classification concentrated input data. In the second step, classification is
for early detection of lung cancer is provided. The effect done in which individual classes are assigned with feature
of International Association for the Study of Lung Cancer domains. LVQ algorithm starts from trained SOM having
(IASLC) lymph node map in lung cancer classification is {p} input and Voronoi weight {Vj}. Each Voronoi cell {Vj}
also discussed in detail. gets the best classification labels of input. A Voronoi cell
boundary does not match with classification boundaries.
This problem is solved with the help of LVQ by shifting of
3 Classification Algorithms boundaries:
13
2462 Arabian Journal for Science and Engineering (2020) 45:2459–2471
In this manner, better classification is carried out than SOM Step-1 P ≤ 0
algorithm. (1 − n) × N ≫ 0 points are generated randomly as
per n × N points using location approximation
3.3 Linear Discriminate Analysis (LDA) method
Step-2 Promising points are selected
LDA can be used both as a dimensionality reduction as well Step-3 K t (𝜋xi b|𝜋yi b) , the conditional probabilities are
as a linear classifier. It is quite similar to principal compo- computed
nent analysis (PCA) classifier [36]. Step-4 A new population according to the given equation
In LDA, each of the class probability density function
is assumed to be model as normal density function. All the ( )
l
K(b, P + 1) = 𝜋i=1 K t 𝜋xi b|𝜋yi b (10)
classes having normal density have same covariance. Let
there are ‘P’ classes. Let Np be a m × Tp matrix of Tp sam- is generated.
ples, as ‘m’ dimensional columns of data from class ‘p’. The Step-5 When termination criteria are met then FINISH
prior probabilities 𝜋p , the common covariance matrix ‘L’ and Step-6 The best points of previous generation are added to
means ‘ 𝜇p ’ for each class are defined as follows: the generation points (elitist)
Step-7 Then p ≤ p + 1
T Go to Step-3.
𝜋p = ∑p b (6)
i=1
Tb
FDA needs finite samples of points. Optimum conver-
gence of FDA depends on the size of samples. FDA can run
Np 1Tb by using any selection method, mostly truncation selection
𝜇p = (7) method is used.
Tb
∑p � T
��
T
�T
4 Proposed Algorithm
N − 𝜇 1 N − 𝜇 1
i=1 p p T b
p p T b (8)
L=
T −P Whale Optimization Algorithm (WOA) is simple when
where 1t is a t × 1 matrix of 1’s. global solutions are explored and positions of search agents
After that a new data point n is classified as are updated. A new optimization algorithm, i.e., VEWOA
is proposed to improve the solution accuracy, reliability
1 of search and convergence speed of WOA. Before going
argp max nt 𝛴 −1 𝜇p − 𝜇pT 𝛴 −1 𝜇p (9)
2 through proposed algorithm, some of the features of WOA
It is used in pattern classification and machine learning are discussed for easy understanding of VEWOA method.
application as a preprocessing step. Goal of LDA is projec-
tion of a feature space/dataset into a lower subspace without 4.1 Overview of WOA
effecting the class discriminator information [37]. It is also
similar to analysis of variance (ANOVA) and regression WOA is a stochastic optimization algorithm which has taken
analysis. inspiration from the special bubble-net hunting behavior of
whales of humpback types [39]. Here, the population of
3.4 Factorized Distribution Algorithm (FDA) swarm is used to get the global optimum (global maximum
or minimum) for any kind of optimization problem, the
FDA is a type of evolutionary algorithm which uses distribu- search process starts by making a random solutions set and
tion in order to combine mutation and recombination [38]. upgrading set till the end condition is met.
Distribution is done with the set of selected points. Then, In order to hunt their prey, whales first create a helix-
these selected points are used for generation of new points structured path and then they follow the bubbles to locate the
for the next generation. A distribution with n binary vari- position of their prey. In WOA, this behavior is simulated by
ables has 2n parameters. FDA works for exact factorization alternating between spiral models and encircling mechanism
as well as for approximate factorization. Factorization can to update the position with 0.5 probabilities. The mechanism
also be used at the time of initialization of the local approxi- can be defined as follows:
mation method is given by:
13
Arabian Journal for Science and Engineering (2020) 45:2459–2471 2463
B=2×B×r−B (13)
C = 2×r (14)
where P* = best position; Pt+1 = whale position; t = current
iteration; B = decreased linearly from 2 to 0 over the course
of iterations and r = uniformly distributed random number
Fig. 1 Shrinking encircling mechanism
within the range of [0, 1]. D corresponds to the distance
between tth whale and the best-positioned whale.
D� = ||P∗t − Pt || (16)
where b = constant defining shape of the logarithmic spiral;
D′ = corresponds to the updated distance; l = uniformly dis-
tributed random number within the range of [− 1, 1].
4.1.3 Global Exploration
13
2464 Arabian Journal for Science and Engineering (2020) 45:2459–2471
4.3 VEWOA‑Trained ANN
Fig. 3 Velocity updation After each search, agent has associated weight and bias; it is
then proceeded to find the optimal solution.
The VEWOA-trained NN can be summarized as follows:
13
Arabian Journal for Science and Engineering (2020) 45:2459–2471 2465
(i) Initialize: in this step, each agent is assigned the ran- with at first, the data are preprocessed. After the preprocess-
dom set of weights and bias. ing step is over, the missing values are all replaced with the
(ii) Evaluate fitness: in this step, the optimal set of instance mean, and all the inputs are normalized through
weights and biases is generated according to the min–max normalization. The formulae for min–max nor-
minimized mean-squared error (MSE), which can malization can be summarized as follows:
also be treated as fitness function. Xorig − Xmin
(iii) Update position: the position of the whales is updated Xnorm = (21)
accordingly. Xmax − Xmin
Fig. 4 Workflow model of
Initialize
VEWOA populations of
Training Dataset search agents
Calculate fitness
Assign search of each search
agents to agent
network
NO
Termination
Update POS* if
condition
their is a best
met?
solution
YES
13
2466 Arabian Journal for Science and Engineering (2020) 45:2459–2471
5.3 Result Analysis
Fig. 5 Workflow model of overall network
In Tables 1, 2 and 3, validation results of WBC data, cervical
ck = number of observations in class k cancer and lung cancer are shown. Different classification
nk = number of appropriately classified observa- algorithms are compared on the basis of no. of parameters
tions in class k. for training, time taken for calculating mean and standard
(iii) Computational cost: it is the time taken to evaluate deviation, CCR of mean and standard deviation, ASCE of
the classifier. mean and standard deviation. Algorithms such as C4.5,
FDA, LDA and LVQ consume much more time as com-
We have conducted the experiment in 30 runs each con- pared to other methods. The proposed method VEWOA-NN
taining 100 iterations. The computational cost is obtained provides optimum value of CCR, minimum value of ASCE
by simulating each algorithm in the same machine for the and less time-consuming being the best one of all.
same no. of iterations. The output can be one of the two, Tables 4, 5 and 6 depict the confusion matrices for the
i.e., either 0 (benign tumor) or 1(malignant tumor). The above-mentioned data. In WBC data (Table 4), the benign
results are summarized by computing the mean and respec- data as predicted by the classifier became 448 while the
tive standard deviation. In addition, the graphs are shown to malignant ones came to 6, thereby resulting in a classifica-
analyze results more clearly. Graphs help in better under- tion accuracy of 97.8 in case of benign and 97.5 for malig-
standing of the results and presenting a comparison. We nant ones. In cervical cancer (Table 5), the benign data
13
Arabian Journal for Science and Engineering (2020) 45:2459–2471 2467
Table 4 The confusion matrix obtained from VEWOA-trained NN in Table 5 The confusion matrix obtained from VEWOA-trained NN in
WBC cervical cancer
found out to be 790 as opposed to 5 in malignant causing the Table 6 The confusion matrix obtained from VEWOA-trained NN in
classification accuracy to rise up to 98.3 and 90.9 for benign lung cancer
and malignant tumors, respectively. In lung cancer (Table 6), CLASS Type-1 Type-2 Type-3 % Classification
the type-I and type-II classification accuracy found out to
be 88.89 and 84.6 while type-III classification accuracy is Type-1 8 1 0 88.89
80. The classification accuracy of VEWOA-NN is compared Type-2 1 11 1 84.6
with other classification techniques in Tables 7, 8 and 9. Type-3 1 1 8 80
13
2468 Arabian Journal for Science and Engineering (2020) 45:2459–2471
13
Arabian Journal for Science and Engineering (2020) 45:2459–2471 2469
0.6
CCR (Mean)
0.4 CCR (SD)
0.2
0
1 2 3 4 5 6
0.8
Fig. 14 Convergence of MSE curve in lung cancer
0.6
CCR (Mean)
0.4 CCR (SD)
6 Conclusion
0.2
13
2470 Arabian Journal for Science and Engineering (2020) 45:2459–2471
References 20. Donate, J.P.; et al.: Time series forecasting by evolving artificial
neural networks with genetic algorithms, differential evolution
and estimation of distribution algorithm. Neural Comput. Appl.
1. Valigopalini, S.; et al.: Trends in cervical cancer incidence and
22(1), 11–20 (2013). https://doi.org/10.1007/s00521-011-0741-0
mortality in Oklahoma and the United States. Cancer Epidemiol.
21. Lee, S.; Geem, Z.W.: A new meta-heuristic algorithm for continu-
56, 140–145 (2018)
ous engineering optimization: harmony search theory and prac-
2. Zheng, M.: Classification and pathology of lung cancer. Surg.
tice. Comput. Methods Appl. Mech. Eng. 194, 3902–3933 (2005).
Oncol. Clin. N. Am. 25(3), 447–468 (2016)
https://doi.org/10.1016/j.cma.2004.09.007
3. Senapati, M.R.; Dash, P.K.: Local linear wavelet neural net-
22. Mao, C.: Harmony search-based test data generation for branch
work based breast tumor classification using firefly algorithm.
coverage in software structural testing. Neural Comput. Appl.
Neural Comput. Appl. 22(7–8), 1591–1598 (2013). https://doi.
25(1), 199–216 (2014). https://doi.org/10.1007/s00521-013-1474-z
org/10.1007/s00521-012-0927-0
23. El-Sebakhy, E.A., Faisal, K.A., Helmy, T., Azzedin, F., Al-
4. Sakim, H.A.M.; Salleh, N.M.; Othman, N.H.: Neural network inputs
Suhaim, A.: A evaluation of breast cancer tumor classification
selection for breast cancer cells classification. Stud. Comput. Intell.
with unconstrained functional networks classifier. In: IEEE Inter-
199, 1–11 (2009). https://doi.org/10.1007/978-3-642-00909-9
national Conference on Computer Systems and Applications, pp.
5. Ster, B., Dobnikar, A.: Neural network in medical diagnosis:
281–287 (2006). https://doi.org/10.1109/aiccsa.2006.205102
comparison with other methods. In: Engineering Applications of
24. Adem, K.; et al.: Classification and diagnosis of cervical cancer
Neural Networks, pp. 427–430 (1996)
with softmax classification with autoencoder. Expert Syst. Appl.
6. Moghaddam, S.S.; Keshteli, M.H.; Mahmoodjanloo, M.: New
115, 557–564 (2018)
approaches in metaheuristics to solve the fixed charge transpor-
25. William, W.; et al.: A review of image analysis and machine learning
tation problem in a fuzzy environment. Neural Comput. Appl.
techniques for automated cervical cancer screening from pap-smear
(2017). https://doi.org/10.1007/s00521-017-3027-3
images. Comput. Methods Programs Bio Med. 164, 15–22 (2018)
7. Delen, D.: Analysis of cancer data: a data mining approach.
26. Devi, M.A.; et al.: Classification of cervical cancer using artificial
Expert Syst. 26, 100–112 (2009). https : //doi.org/10.111
neural networks. Procedia Comput. Sci. 89, 465–472 (2016)
1/j.1468-0394.2008.00480.x
27. Philip, Lauren; et al.: Pap tests in the diagnosis of cervical cancer:
8. Sisodia, D.; Sisodia, D.S.: Prediction of diabetes using classifica-
help or hinder? Gynecol. Oncol. 150(1), 61–66 (2018)
tion algorithms. Procedia Comput. Sci. 132, 1578–1585 (2018).
28. Shih, I.L.; et al.: PET/MR hybrid imaging of cervical and endo-
https://doi.org/10.1016/j.procs.2018.05.122
metrial cancer. J. Cancer Res. Pract. 5(3), 91–98 (2018)
9. Patra, A.; Das, S.; Mishra, S.N.; Senapati, M.R.: An adaptive local
29. Li, J.; et al.: Adaptive multinomial regression with overlapping
linear optimized radial basis functional neural network model for
groups for multi class classification of lung cancer. Comput. Biol.
financial time series prediction. Neural Comput. Appl. 28(1),
Med. 100, 1–9 (2018)
101–110 (2015). https://doi.org/10.1007/s00521-015-2039-0
30. Frank, C.; et al.: The eighth edition lung cancer stage classifica-
10. Dash, S.; Senapati, M.R.; Jena, U.R.: K-NN based automated rea-
tion. Chest 151(1), 193–203 (2017)
soning using bilateral filter based texture descriptor for comput-
31. Kazonowska, E.; et al.: The classification of lung cancers and
ing texture classification. Egypt. Inform. J. (2018). https://doi.
their degree of malignancy by FTIR, PCA-LDA analysis, and a
org/10.1016/j.eij.2018.01.003
physics-based computational model. Talanta 186, 337–345 (2018)
11. Dash, S.; Jena, U.R.; Senapati, M.R.: Homomorphic normaliza-
32. Frank, C.; et al.: The IASLC lung cancer staging project: summary
tion-based descriptors for texture classification. Arab. J. Sci. Eng.
of proposals for revisions of the classification of lung cancers with
(2018). https://doi.org/10.1007/s13369-017-2961-9
multiple pulmonary sites of involvement in the forthcoming eighth
12. Nayak, T.; Senapati, M.R.; Prasad, S.: A survey on web text
edition of TMM classification. J. Thoracic Oncol. 11(5), 639–650
information retrieval in text research. J. Appl. Sci. Eng. Tech-
(2016)
nol. 10(10), 1164–1174 (2015). https://doi.org/10.19026/rjase
33. Ahmed, H.; et al.: Cross-disciplinary analysis of lymph node clas-
t.10.1884
sification in lung cancer on CT scanning. Chest 151(4), 776–785
13. Senapati, M.R.; Das, S.P.; Champati, P.K.; Routray, P.K.: Local
(2017)
linear radial basis function neural networks for classification of
34. Sudrajat, A.R.; Irianingsih, I.; Krisnawan, D.: Analysis of data mining
breast cancer data. Prog. Sci. Eng. Res. J. 2(6), 033–042 (2014)
classification by comparison of C4.5 and ID algorithm. In: IOP Con-
14. Das, S.; Patra, A.; Mishra, S.; Senapati, M.R.: A self-adaptive
ference Series: Materials Science and Engineering, vol. 166 (2017)
fuzzy-based optimised functional link artificial neural network
35. Kohonen, T.: Learning vector quantization. In: Self-Organiz-
model for financial time series prediction. Int. J. Bus. Forecast.
ing Maps. Springer Series in Information Sciences, vol. 30,
Mark. Intell. 2(1), 19–36 (2015). https://doi.org/10.1504/IJBFM
pp. 175–189. Springer, Berlin, Heidelberg (1995). https://doi.
I.2015.075358
org/10.1007/978-3-642-97610-0_6
15. Sahoo, S.; Sahoo, S.S.; Senapati, M.R.; Dash, P.K.: Local linear
36. Izenman, A.J.: Linear discriminant analysis. In: Modern Mul-
wavelet neural network and RLS for usable speech classification.
tivariate Statistical Techniques. Springer Texts in Statistics,
IJCS 8, 412–418 (2011)
pp. 237–280. Springer, New York, NY (2013). https: //doi.
16. Abonyi, J.; Szeifert, F.: Supervised fuzzy clustering for the identi-
org/10.1007/978-0-387-78189-1_8
fication of fuzzy classifiers. Pattern Recognit. Lett. 24(14), 2195–
37. Garrett, D.; Peterson, D.A.; Anderson, C.W.; Thaut, M.H.: Com-
2207 (2003). https://doi.org/10.1016/S0167-8655(03)00047-3
parison of linear, nonlinear, and feature selection methods for
17. Majhi, S.K.: An efficient feed foreword network model with sine
EEG signal classification. IEEE Trans. Neural Syst. Rehabil. Eng.
cosine algorithm for breast cancer classification. Int. J. Syst. Dyn.
11(2), 141–144 (2003)
Appl. (IJSDA) 7(2), 1–14 (2018). https://doi.org/10.4018/IJSDA
38. Zhang, Q.: On stability of fixed points of limit models of univari-
.201804010
ate marginal distribution algorithm and factorization distribution
18. Mahfoud, S.; Mani, G.: Financial forecasting using genetic algo-
algorithm. IEEE Trans. Evolut. Comput. 8(1), 80–93 (2004)
rithms. Appl. Artif. Intell. 10, 543–565 (1996)
39. Abdel-Basset, M.A.; et al.: A hybrid whale optimization algorithm
19. Neshat, M.: FAIPSO: fuzzy adaptive informed particle swarm
based on local search strategy for the permutation flow shop sched-
optimization. Neural Comput. Appl. 23, 95–116 (2013). https://
uling problem. Future Gener. Comput. Syst. 85, 129–145 (2018)
doi.org/10.1007/s00521-012-1256-z
13
Arabian Journal for Science and Engineering (2020) 45:2459–2471 2471
40. Majdi, M.; et al.: Hybrid whale optimization algorithm with 44. Fernandes, K., Cardoso, J.S., Fernandes, J.: Transfer learning
simulated annealing for feature selection. Neuro-computing 260, with partial observability applied to cervical cancer screening. In:
302–312 (2017) Iberian Conference on Pattern Recognition and Image Analysis.
41. Kaveh, A.; Ghazaan, M.I.: Enhanced whale optimization algo- Springer (2017)
rithm for sizing optimization of skeletal structures. Mech. 45. Hong, Z.Q.; Yang, J.Y.: Optimal discriminant plane for a small
Based Des. Struct. Mach. 45(3), 345–362 (2016). https://doi. number of samples and design method of classifier on the
org/10.1080/15397734.2016.1213639 plane. Pattern Recognit. 24(4), 317–324 (1994). https://doi.
42. Mirjalili, S.; Lewis, A.: The whale optimization algorithm. Adv. org/10.1016/0031-3203(91)90074-F
Eng. Softw. 51, 67 (2016). https://doi.org/10.1016/j.advengsoft 46. Dua, D., Graff, C.: UCI machine learning repository. University of
.2016.01.008 California, School of Information and Computer Science, Irvine,
43. Wolberg, W.H.; Mangasarian, O.L.: Multisurface method of pat- CA. http://archive.ics.uci.edu/ml (2019)
tern separation for medical diagnosis applied to breast cytology.
Proc. Natl. Acad. Sci. USA 87, 9193–9196 (1990)
13