SUPPORT VECTOR

MACHINES
(SVM)

MAARIA JAHANGIR
STAT 390
 WHAT IS SVM?

Possible
• Uses: Regression and Classification hyperplanes

tasks
• Aim: find the hyperplane in an N-
dimensional space that best classifies
data
• N=# of features
• Best hyperplane maximizes distance
between data classes
 HYPERPLANES & SUPPORT VECTORS
• Hyperplanes: Decision boundaries
used to classify data
• N=2 -> Hyperplane is a line
• N=3 -> Hyperplane becomes 2-
D plane

• Support Vectors: used to maximize margin

• Influence position & orientation of hyperplane
 LOSS FUNCTION

• Hinge-Loss: Error due to misclassification

• Regularization Term (also seen as C)
• with λ as regularization coefficient
 KERNEL FUNCTIONS

• Set of algorithms used in pattern analysis

• RBF Kernel good 1st choice if N is small
• Has similar performance to linear, sigmoid kernels for certain parameters
• Less hyperparameters than polynomial kernel
 NONLINEAR DATA

• Solution: Map data to higher dimension – computationally intense

• Kernel Trick: Uses dot product to obtain nonlinear classifier
• No data transformations
• Also used with other classifiers, logistic regression
Pros Cons
Kernal
Memory
Parameters
Efficient
Selection

High Non-
Dimensionality Probabilistic

Versatile
DATA PREPROCESSING

• Convert categorical attributes into numeric data

• Scaling – important when attributes have different numeric ranges

• Larger values cause issues with linear, polynomial kernels
MULTI-CLASS SVMS

• OVA Classification: One-versus-all

• One-versus-one classifiers: choose class selected by most classifiers

• Directed Acyclic Graph Support Vector Machines (DAGSVM)

 CODING

• Package: e1071
• y2 <- factor(y)
• svm( y2 ~ .,
data=<dataframe>,
kernel=“linear, radial”,
“polynomial”, “sigmoid”,
cost=<#>, scale=< T or F>
• plot( <svm model>,
<dataframe>)
CODING - NONLINEAR

• library(kernlab)
• ksvm( x, y, data=<dataframe>, scaled=<T or F>, type=<see list below>, kernel=<listed below>
• Types: Other types available for regression and novelty detection
• “C-svc” – C classification
• “nu-svc”,”C-bsvc” – nu classification
• Many more types – Hyperbolic Tangent, Laplacian, Bessel, Spline, String, or ANOVA RBF
kernel
• Can also specify “gamma”
REFERENCES

• HTTPS://TOWARDSDATASCIENCE.COM/SUPPORT-VECTOR-MACHINE-INTRODU
CTION-TO-MACHINE-LEARNING-ALGORITHMS-934A444FCA47
• HTTPS://MONKEYLEARN.COM/BLOG/INTRODUCTION-TO-SUPPORT-VECTOR-M
ACHINES-SVM/
• HTTPS://WWW.DATACAMP.COM/COMMUNITY/TUTORIALS/SUPPORT-VECTOR-
MACHINES-R
• HTTP://UC-R.GITHUB.IO/SVM
• HTTPS://NLP.STANFORD.EDU/IR-BOOK/HTML/HTMLEDITION/MULTICLASS-SV
MS-1.HTML
• HTTPS://WWW.CSIE.NTU.EDU.TW/~CJLIN/PAPERS/GUIDE/GUIDE.PDF
• HTTPS://DATA-FLAIR.TRAINING/BLOGS/SVM-KERNEL-FUNCTIONS/