Support Vector

Machines
George-Eduard Zaharia

Slow down and speed-up once in a while

 Introduction - SVMs

◎ Used in a large array of domains (bioinformatics,

security)
◎ Supervised machine learning algorithm
◎ Used for classification or regression problems
◎ Separates data based on defined labels/outputs
◎ Introduced in 1963 and further developed in 1992
Classifying Data

◎ Find a separator between the

two groups

◎ Classify any new added point

Classifying Data
Maximum Margin
Model

◎ The points: (x1, y1), ... , (xn, yn)

◎ The hyperplane: f(X) = <w · X> + b

◎ The margin: 2/||w||

◎ Our purpose is to maximize the

margin
Soft-Margin SVM

◎ The hard margin SVM can work only when data is

completely linearly separable, without any errors

◎ Soft margin SVM allows error in the training set

◎ It produces a more general model when applied to

new datasets
◎
Non-Linear Separation
 Non-Linear Separation

Wherever we draw a line, the two groups of

points cannot be linearly separable
Non-Linear Separation

Mapping function: φ : X -> F

Non-Linear Separation
Kernel Trick

◎ Operate in a feature space without computing the

coordinates of the data in that space

◎ They compute the inner products between the images

of all pairs of data in the feature space

◎ Computationally cheaper
Kernel Example - Gaussian RBF Kernel
Kernel Example - Gaussian RBF Kernel
Other Kernel Functions

◎ Polynomial kernel: 𝐾(𝑋, 𝑌) = (𝑎 ∗ < 𝑋 · 𝑌 > + 𝑏) 𝑝

◎ Sigmoid kernel: 𝐾(𝑋, 𝑌) = 𝑡𝑎𝑛ℎ (𝑎 ∗ < 𝑋 · 𝑌 > + 𝑏)

◎ Linear kernel: 𝐾(𝑋, 𝑌) = < 𝑋 · 𝑌 > + 𝑏

Conclusion

Support vector machines prove to be strong classifiers

for various task:
● text categorization
● biology and other sciences
● image classification
THANK YOU!