SVM

Presented by:

M. Saqib Iqbal

Gull Muhammad

Presented to:

Mr. Imran Ali khan

Artificial intelligence

NATIONAL COLLEGE OF BUSSINESS

ADMINISTRATION & ECONOMICS MULTAN
 “What are Support Vector Machines and how to build a
classifier based on that”
Support Vector Machine or SVM is one of the most popular Supervised
Learning algorithms, which is used for Classification as well as Regression
problems. However, primarily, it is used for Classification problems in Machine
Learning.
The goal of the SVM algorithm is to create the best line or decision boundary
that can segregate n-dimensional space into classes so that we can easily put the
new data point in the correct category in the future. This best decision boundary
is called a hyper-plane.
SVM chooses the extreme points/vectors that help in creating the hyper-plane.
These extreme cases are called as support vectors, and hence algorithm is
termed as Support Vector Machine. Consider the below diagram in which there
are two different categories that are classified using a decision boundary or
hyper-plane:
Example: SVM can be understood with the example that we have used in the
KNN classifier. Suppose we see a strange cat that also has some features of
dogs, so if we want a model that can accurately identify whether it is a cat or
dog, so such a model can be created by using the SVM algorithm. We will first
train our model with lots of images of cats and dogs so that it can learn about
different features of cats and dogs, and then we test it with this strange creature.
So as support vector creates a decision boundary between these two data (cat
and dog) and choose extreme cases (support vectors), it will see the extreme
case of cat and dog. On the basis of the support vectors, it will classify it as a
cat. Consider the below diagram:

SVM algorithm can be used for Face detection, image classification, text

categorization, etc.

Types of SVM

SVM can be of two types:

o Linear SVM: Linear SVM is used for linearly separable data, which

means if a dataset can be classified into two classes by using a single
straight line, then such data is termed as linearly separable data, and
classifier is used called as Linear SVM classifier.
o Non-linear SVM: Non-Linear SVM is used for non-linearly separated
data, which means if a dataset cannot be classified by using a straight
line, then such data is termed as non-linear data and classifier used is
called as Non-linear SVM classifier.
Hyperplane and Support Vectors in the SVM algorithm:

Hyperplane: There can be multiple lines/decision boundaries to segregate the

classes in n-dimensional space, but we need to find out the best decision
boundary that helps to classify the data points. This best boundary is known as
the hyperplane of SVM.

The dimensions of the hyperplane depend on the features present in the dataset,
which means if there are 2 features (as shown in image), then hyperplane will
be a straight line. And if there are 3 features, then hyperplane will be a 2-
dimension plane.

We always create a hyperplane that has a maximum margin, which means the
maximum distance between the data points.

Support Vectors:

The data points or vectors that are the closest to the hyperplane and which affect
the position of the hyperplane are termed as Support Vector. Since these vectors
support the hyperplane, hence called a Support vector.

How SVM works?

As we have a clear idea of the terminologies related to SVM, let’s now see how
the algorithm works. For example, we have a classification problem where we
have to separate the red data points from the blue ones.
Since it is a two-dimensional problem, our decision boundary will be a line, for
the 3-dimensional problem we have to use a plane, and similarly, the
complexity of the solution will increase with the rising number of features.

As shown in the above image, we have multiple lines separating the data points
successfully. But our objective is to look for the best solution.

There are few rules that can help us to identify the best line.

Maximum classification, i.e the selected line must be able to successfully

segregate all the data points into the respective classes. In our example, we can
clearly see lines E and D are miss classifying a red data point. Hence, for this
problem lines A, B, C is better than E and D. So we will drop them.
The second rule is Best Separation, which means, we must choose a line such
that it is perfectly able to separate the points.

If we talk about our example, if we get a new red data point closer to line A as
shown in the image below, line A will miss classifying that point. Similarly, if
we got a new blue instance closer to line B, then line A and C will classify the
data successfully, whereas line B will miss classifying this new point.

The point to be noticed here, In both the cases line C is successfully classifying
all the data points why? To understand this let’s take all the lines one by one.
How to build a classifier based on SVM
An example about SVM classification of cancer UCI datasets using machine
learning tools i.e. scikit-learn compatible with Python.

Here an example about SVM classification of cancer UCI datasets using

machine learning tools i.e. scikit-learn compatible with Python.

Pre-requisites: Numpy, Pandas, matplot-lib, scikit-learn

Let’s have a quick example of support vector classification. First we need to

create a dataset:

# importing scikit learn with make_blobs

from sklearn.datasets.samples_generator import make_blobs

# creating datasets X containing n_samples

# Y containing two classes

X, Y = make_blobs(n_samples=500, centers=2,random_state=0,
cluster_std=0.40)

import matplotlib.pyplot as plt

# plotting scatters

plt.scatter(X[:, 0], X[:, 1], c=Y, s=50, cmap='spring');

plt.show()

Output
What Support vector machines do, is to not only draw a line between two classes
here, but consider a region about the line of some given width. Here’s an example of
what it can look like:

# creating line space between -1 to 3.5

xfit = np.linspace(-1, 3.5)

# plotting scatter

plt.scatter(X[:, 0], X[:, 1], c=Y, s=50, cmap='spring')

# plot a line between the different sets of data

for m, b, d in [(1, 0.65, 0.33), (0.5, 1.6, 0.55), (-0.2, 2.9, 0.2)]:

yfit = m * xfit + b

plt.plot(xfit, yfit, '-k')

plt.fill_between(xfit, yfit - d, yfit + d, edgecolor='none',

color='#AAAAAA', alpha=0.4)

plt.xlim(-1, 3.5);

plt.show()
Importing datasets

This is the intuition of support vector machines, which optimize a linear

discriminant model representing the perpendicular distance between the
datasets. Now let’s train the classifier using our training data. Before training,
we need to import cancer datasets as csv file where we will train two features
out of all features.

# importing required libraries

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

# reading csv file and extracting class column to y.

x = pd.read_csv("C:\...\cancer.csv")

a = np.array(x)

y = a[:,30] # classes having 0 and 1

# extracting two features

x = np.column_stack((x.malignant,x.benign))

# 569 samples and 2 features

x.shape

print (x),(y)
 Fitting a Support Vector Machine
Now we’ll fit a Support Vector Machine Classifier to these points. While the
mathematical details of the likelihood model are interesting, we’ll let read
about those elsewhere. Instead, we’ll just treat the scikit-learn algorithm as a
black box which accomplishes the above task.
# import support vector classifier

# "Support Vector Classifier"

from sklearn.svm import SVC

clf = SVC(kernel='linear')

# fitting x samples and y classes

clf.fit(x, y)

After being fitted, the model can then be used to predict new values:

clf.predict([[120, 990]])

clf.predict([[85, 550]])

Let’s have a look on the graph how does this show.

This is obtained by analyzing the data taken and pre-processing methods to
make optimal hyper-planes using matplotlib function.