A Systematic Literature Review on Support Vector

Machines Applied to Classification

Miguel Angel Cano Lengua Erik Alex Papa Quiroz
Facultad de Ingeniería de Sistemas e Informática Universidad Nacional Mayor de San Marcos
Universidad Nacional Mayor de San Marcos Universidad Privada del Norte
Lima, Perú Lima, Perú
mcanol@unmsm.edu.pe erikpapa@gmail.com

Abstract—This paper aims to identify the current state of the art • Review planning: In this phase the research questions are
of the latest research related to support vector machines through asked and the review protocol is defined.
a literature review system according to the methodology proposed • Carrying out the review: In this phase, the plan is executed
by Kitchenham and Charter, in order to answer the following and the main studies are selected, according to the inclusion and
research questions: Q1: In which research areas are they used? exclusion criteria.
Q2: What are the main applications related with classification?
Q3: What optimization methods or algorithms are used in SVMs? • Review report: In this phase, the statistics, analysis of the
Keywords— Vector Support Machines, optimization algorithms, selected documents, and discussions are presented.
classification, hyperplane, linear and nonlinear separation.
II.1 Review planning
I. INTRODUCTION We search papers that answer the following questions:
The classification-regression method of Vector Support Q1: In which research areas SVMs are used?
Machines (SVMs) was developed in the 90's, within the field of Q2: What are the latest related applications?
Computational Science [1]. Although it was originally Q3: What optimization methods or algorithms are used in SVMs?
developed as a binary classification method, its applications
have extended to multiple classification and regression The search was carried out in the following databases: Web of
problems. SVMs have turned out to be one of the best classifiers Science, Science Direct, IEEE Xplore Digital Library and ACM
for a wide range of situations, which is why it is considered one Digital Library, Scopus and Springer Link. The review covers
of the benchmarks in the field of statistical learning and the period from 1992 to 2020.
machine learning. It was used the following stream search: TITLE-ABS-KEY
("areas where SVM are used") OR TITLE-ABS-
In this paper, using the methodology of Kitchenham and
KEY("Applications related to classification") OR TITLE- ABS-
Charter [2], we present a review of the literature on SVMs. The
KEY("Optimization algorithms used in SVMs"), which have
main contributions of this paper are the followings: We will been applied on the title, abstract and keywords. Then, the
know the applications of SVMs in diverse areas of research; the selection and exclusion criteria showed in Tables 1 and 2 were
recent applications of SVMs in classification and the applied.
optimization algorithms or optimization methods used for this
class of machines. II.2 Carrying out the review:
This research is organized by the following sections: In Section
2, we present the proposed methodology. In Section 3, we Table 1: Inclusion Criteria
analyze the information. In Section 4, we analyze the results Criterion Justification
and in Section 5, we present the conclusions of the work.
Finally, we give the sources of references. Research focus papers with the text support vector
machines as a keyword
Quantitative Articles must be published in
II.RESEARCH METHODOLOGY information indexed journals
Importance factor SVM linked to classification and
The systematic review of the literature that we will consider
applied to Artificial Intelligence
is the procedure proposed by Kitchenham and Charter [2]. This
methodology identifies, evaluates, and interprets all available Language English
research relevant to particular questions, subject area, or
phenomenon of interest. It will be used devote its structure is
Table 2: Exclusion Criteria
accordance with the objectives of our research. This
methodology has been recently used by several researches, see Criterion Justification
[25,26]. However, there are other methodologies, for example
[3] and [4], which do not fit to the field of engineering. The Type of publication Books, dissertations, book chapters
proposed methodology involves the following phases: between 1990-2000 (Not updated)

978-1-7281-8367-1/20/$31.00 ©2020 IEEE

Authorized licensed use limited to: University of Canberra. Downloaded on May 20,2021 at 09:21:25 UTC from IEEE Xplore. Restrictions apply.
 Analysis unit papers that do not consider SVM as
their interest focus
Importance factor Investigations that do not clearly
present a development methodology.
Length of this paper Number of pages allowed (4 pages)

Fig. 1 shows the sources from which the research was

obtained, using the selection criteria, inclusion and exclusion,
the filtering of the papers according to the established criteria.
Fig. 1 Systematic literature review process

III. STUDY ANALYSIS

From 30 selected investigations, we will answer the research
questions (Q1,Q2 and Q3) according the following frame:

Fig. 3: General framework proposed for the revision of the

literature.

II.3 Review report

108 306 research articles were found related to the classification III.1 Research areas
of support vector machines, which are applied to the different
research areas, see fig 1. The following articles were found in It is related to the first research question (Q1). Table 4, shows
the databases of the following sources: the four sub-questions of Q1.
Tabla 3: Selected papers
Source Potentially Selected studies Table 4. Questions related with
eligible studies " Research areas"
Science Direct 3959 3 Contributions Research questions
IEEE Xplore 8327 4 Algorithms related to Q1.1. What contributions are
ACM 42 921 2 Artificial Intelligence, [5]. found in engineering?
Springer Link 46 914 5 Techniques for classifying Q1.2. What contributions are
Scopus 11385 9 plants based on their shape, found in Biomedical
Web Of Science 6 185 7 [6] Engineering?
Total 108 306 30 Computer aided diagnosis of Q1.3. What contributions are
Alzheimer's based on SVM, found in Neuroscience
On the other hand, using the flow search in SCOPUS [27] we [7,8] engineering?
obtained the trends of the publications presented in Fig. 2. We Hybrid model to improve the Q1.4.What contributions are
may observe that the first published paper indexed in SCOPUS performance of the energy found in Computer
on Support Vector Machines applied to classification was [28]. management system, [9]. Information Systems?

Fig. 2 Publication trends of support vector machines applied to Other works in another reserach areas are the follownig: [33].
classification in SCOPUS [34] and [35].

III.2 Applications related to classification.

Authorized licensed use limited to: University of Canberra. Downloaded on May 20,2021 at 09:21:25 UTC from IEEE Xplore. Restrictions apply.
Support vector machines have applications in compound An inexact proximal Q3.4.What optimization
classification [11], mental workload selection [10] and other algorithm with generalized methods or algorithms are
applications such as magnetic resonance imaging [12] and distance to solve used in the area of Computer
verification of digital signatures [13]. It is also applied in deep Symmetric convex Information Systems?
learning (DL) algorithms in hyperspectral image classification optimization problems with
(HSI). Imaging (HSI) is popular in diverse applications such as separable structures, [24].
geosciences, biomedical imaging, molecular biology,
agriculture, astronomy, food safety and quality assessment,
surveillance, and physics-related research, see [36]. Other IV. ANALYSIS OF RESULTS
applications are: Classification of schizophrenia based on
artificial intelligence [37], the aesthetics of photographic IV.1 Research areas (Q1)
images [38]. Next, we develop the sub-questions according to
the framework of Fig. 1 The percentages presented in this section are obtained in
references to 30 studies analyzed without redundancy.
Table 5. Research questions related to
"Applications related to classification" Fig. 4 Research areas
Applications Research questions
Simultaneous SVM Q2.1. What applications are
classification and feature related to classification in
selection engineering?
The electronegativity of Q2.2. What applications are
atoms and the standard related to classification in
enthalpy of compound Biomedical Engineering?
formation
Applications of functional Q2.3. What applications are
magnetic resonance imaging related to classification in
using data-based analysis Neuroscience?
A signature verification Q2.4. What applications are
system based on global and related to classification in
local characteristics Computer Information
Systems?

III.3 Optimization methods or algorithms used in SVMs.

The figure shows the percentage of distribution according to the
The main optimization techniques are the following: Interior research areas, we mainly see that 17% is related to Computer
point algorithms, [14]; Coordinate descent [15], Newton's Science and 10% Engineering Electrical Electronic and
method for the Primal problem [16,17]; the subgradient method Computer Science Information Systems.
[18]; Secant plane algorithms [19], [20].
IV.2 Applications related to classification (Q2)
Table 6. Research questions related to "Optimization In this section we identify that classification-related to SVMs
algorithms used in SVMs" have applications in: expert systems, classification of
Optimization algorithms Investigation questions compounds, magnetic resonance and digital signatures.
The non-parallel epsilon Q3.1.What optimization
support vector regression (ε- methods or algorithms are IV.3 Optimization algorithms used in SVMs (Q3)
NPSVR) which generalizes used in the engineering area? The optimization algorithms used in SVM depend on the area
the support vector regression of application. For example, in [21] it was proposed a new twin
ε-twin the ε-TSVR, [21] SVM method. Note that, second order cone programming for
Maximum Focus Range for Q3.2. What optimization support vector machines (SOCP-SVM) presents good
Multiclass Support Vector methods or algorithms are predictive performance, see [22]. Robust non-parallel
Machines Based on Second- used in the area of hyperplane (RNH-SVM) builds nonlinear models with superior
Order Cone Programming, Biomedical Engineering? performance and attractive geometric properties, see [23].
[22] Another important contribution in the optimization field is the
Hyperplane Non-parallel Q3.3. What optimization convex proximal multiplier method for symmetric cone
Robust SVM (RNH-SVM). methods or algorithms are optimization, this decomposition algorithm can solve cone
Linear and its Dual SVM used in the area of symmetric optimization with separable structures [24]. Other
formulation (RNH-SVM), Neuroscience engineering? interesting optimization methods have been introduced in
[23]. [29-32].

Authorized licensed use limited to: University of Canberra. Downloaded on May 20,2021 at 09:21:25 UTC from IEEE Xplore. Restrictions apply.
 CONCLUSIONS
This research allows us to know that SVM has a considerable
number of applications in the different areas. It is worth
mentioning that there are also applications in regression, such
as forecasting models, which may be the subject of others
research.
REFERENCES
[1] C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn.,
1995.
[2] B.A.Kitchenham,.; and Charters, S. “Guidelines for performing
systematic literature reviews in software engineering”version 2.3.
Retrieved January 9, 2014.
[3] R.N. Kostoff, Literature-related discovery (LRD): introduction and
background, Technol. Forecast. Soc. Change 75 (2) 186–202, 2008
[4] Porter, A.L., and Newman, N.C.,” Mining external R&D” Search
Technology, Inc., 6025 The Corners Pkwy, Norcross, GA 30092, USA.
Technovation 31, 171–176, 2011.
[5] H.Azarmdel, Jahanbakhshi, A., Mohtasebi, S.S., Muñoz, A.R.
“Evaluation of image processing technique as an expert system in
mulberry fruit grading based on ripeness level using artificial neural
networks (ANNs) and support vector machine (SVM)” Postharvest
Biology and Technology, 166, art. no. 111201, .2020.
[6] Singh, K., Gupta, I., Gupta, S. “Retrieval and classification of leaf shape
by Support Vector Machine using binary Decision Tree, probabilistic
neural network and generic Fourier Moment technique: A comparative
study” Proc. of the IADIS Int. Conf. - Computer Graphics, Visualization,
Computer Vision and Image Processing, CGVCVIP 2010, Visual
Commun., VC 2010, Web3DW 2010, Part of the MCCSIS 2010, pp.
412-417., 2010
[7] Gotsopoulos, A., . H, Saarimäki et al. “Reproducibility of importance
extraction methods in neural network based fMRI classification”
NeuroImage, 181, pp. 44-54. 2018.
[8] B.F Marghalani., M. Arif, “Automatic Classification of Brain Tumor and
Alzheimer's Disease in MRI” Procedia Computer Science, 163, pp. 78-
84. 2019.
[9] N.Guo, et al. “Using improved support vector regression to predict the
transmitted energy consumption data by distributed wireless sensor
network” Eurasip Journal on Wireless Communications and
Networking, art. no. 120, 2020.
[10] S. Maldonado,J. López, , A. Jimenez-Molina., H. Lira,. “Simultaneous
feature selection and heterogeneity control for SVM classification: An
application to mental workload assessment” Expert Systems with
Applications, 143, art. no. 112988, 2020.
[11] Z. Wan,.Quantitative structure-property relationship of standard
enthalpies of nitrogen oxides based on a MSR and LS-SVR algorithm
predictions. Journal of Molecular Structure, 1221, art. no. 128867, 2020
[12] M. Allegra, S. Seyed-Allaei et al. fMRI single trial discovery of
spatio-temporal brain activity patterns (2017) Human Brain Mapping,
38 (3), pp. 1421-1437, 2020.
[13] T. Longjam, D.R..Kisku, “A supervised manipuri offline signature
verification system with global and local features” 7th International
Symposium on Embedded Computing and System Design, ISED 2017,
2018-January, pp. 1-6, 2018.
[14] Baygorrea, N., Papa Quiroz, E.A., Maculan, N. Inexact Proximal Point
Methods for Quasiconvex Minimization on Hadamard Manifolds (2016)
Journal of the Operations Research Society of China, 4 (4), pp. 397-424
[15] C. Hsieh, K. Chang, C. Lin, S. Keerthi, S. Sundararajan,” A dual
coordinate descent method for large-scale linear SVM”, in: Proceedings
of the 25th International Conference on Machine Learning, , pp. 408–
415, 2008.
[16] S. Keerthi, D. DeCoste, “A modified finite Newton method for fast
solution of large scale linear SVMs”, Journal of Machine Learning
Research 6 , 341–361, 2005.
[17] O.Mangasarian,” A finite Newton method for classification, Optimization
Methods and Software “17 , 913–929. 2002.
[18] J. Kivinen, A. Smola, R. Williamson, Online learning with kernels, IEEE
Authorized licensed use limited to: University of Canberra. Downloaded on May 20,2021 at 09:21:25 UTC from IEEE Xplore. Restrictions apply.
Transactions on Signal Processing 52 (2004) 2165–2176.
[19] J. Kelley, “The cutting-plane method for solving convex programs,
Journal of the Society for Industrial Applied Mathematics 8” 703–712,
1960.
[20] C. Teo, Q. Le, A. Smola, S. Vishwanathan, A scalable modular convex
solver for regularized risk minimization, in: Proceedings of the 13th
ACM Conference on Knowledge Discovery and Data Mining, 2007, pp.
727–736.
[21] M. Carrasco,J. López, ., Maldonado, S. Epsilon-nonparallel support
vector regression Applied Intelligence, 49 (12), pp. 4223-4236.2 019
[22] J .López,s. Maldonado,M.A. Carrasco, “A novel multi-class SVM model
using second-order cone constraints” Applied Intelligence, 44 (2), pp.
457-469,2016
[23] M.Carrasco,J.López, J.,S. Maldonado,. “A second-order cone
programming formulation for nonparallel hyperplane support vector
machine” Expert Systems with Applications, 54, pp. 95-104. 2016.
[24] E. Papa, J.lopez, M. Cano. “A Proximal Multiplier Method for
Convex Separable Symmetric Cone Optimization” International
Conference on Multimedia Systems and Signal, ICMSSP 2020, May 28–
30, 2020 pp. 92–97. May 2020.
[25] Vargas, P.S.C., Mauricio, D. A review of literature about models and
factors of productivity in the software factory (2018) International
Journal of Information Technologies and Systems Approach, 11 (1), pp.
48-71
[26] Wong, L.R., Mauricio, D., Rodriguez, G.D. A systematic literature review
about software requirements elicitation (2017) Journal of Engineering
Science and Technology, 12 (2), pp. 296-317.
[27] SCOPUS (2020). List of articles published on Support vector machines
applied to classification. Retrieved August 13, 2020, from
http://www.scopus.com/
[28] Bonneville, M., Meunier, J., Bengio, Y., Soucy, J.-P. Support vector
machines for improving the classification of brain PET images (1998)
Proceedings of SPIE - The International Society for Optical Engineering,
3338, pp. 264-273.
[29] Wu, S.-J., Pham, V.-H. Lost-min voting strategies for speeding up multi-
SVMs (2020) Proceedings of 2019 the 9th International Workshop on
Computer Science and Engineering, WCSE 2019, pp. 65-71.
[30] Ergen, T., Kozat, S.S. A novel distributed anomaly detection algorithm
based on support vector machines (2020) Digital Signal Processing: A
Review Journal, 99, art. no. 102657.
[31] Ergen, T., Kozat, S.S. A novel distributed anomaly detection algorithm
based on support vector machines (2020) Digital Signal Processing: A
Review Journal, 99, art. no. 102657.
[32] Baygin, N., Baygin, M., Karakose, M. A SVM-PSO Classifier for Robot
Motion in Environment with Obstacles (2019) 2019 International
Conference on Artificial Intelligence and Data Processing Symposium,
IDAP 2019, art. no. 8875921.
[33] Liao, X., Huang, X., Huang, W. ML-LUM: A system for land use mapping
by machine learning algorithms (2019) Journal of Computer Languages,
54, art. no. 100908.
[34] Xu, S., Zhao, Q., Yin, K., Zhang, F., Liu, D., Yang, G. Combining random
forest and support vector machines for object-based rural-land-cover
classification using high spatial resolution imagery (2019) Journal of
Applied Remote Sensing, 13 (1), art. no. 014521.
[35] Shi, H., Yu, Y., Wang, Y. Early Warning Method for Sea Typhoons using
Remote-Sensing Imagery Based on Improved Support Vector Machines
(SVMs) (2018) Journal of Coastal Research, 82, pp. 180-185.
[36] Kalidindi, K.R., Gottumukkala, P.S.V., Davuluri, R. Derivative-based
band clustering and multi-agent PSO optimization for optimal band
selection of hyper-spectral images (2020) Journal of Supercomputing, 76
(8), pp. 5873-5898.
[37] Tikka, S.K., Singh, B.K., Nizamie, S.H., Garg, S., Mandal, S., Thakur,
K., Singh, L.K. Artificial intelligence-based classification of
schizophrenia: A high density electroencephalographic and support
vector machine study (2020) Indian Journal of Psychiatry, 62 (3), pp.
273-282.
[38] Almobarak, A.S., Almohammadi, H.R., Aboalnaser, S.A., Syed, L.
Classification of aesthetic photographic images using SVM and KNN
classifiers (2019) Proceedings. International Conference on
Developments in eSystems Engineering, DeSE, October-2019, art. no.
9073087, pp. 170-175.