But there can be several decision boundaries that can divide the data points without any

errors. For example, in Figure 3, all decision boundaries classify the datasets correctly. But
how do we pick the best decision boundary?

SVM Figure 3: Other Possible Decision Boundaries

Well, here’s the tip: the best decision boundary is the one which has maximum distance from
the nearest points of these two classes, as shown in Figure 4.

SVM Figure 4: Maximum Distance from the Nearest Points

Also remember that the nearest points from the optimal decision boundary that maximize the
distance are called support vectors.
 SVM Figure 5: Margin and Maximum Margin Classifier

The region that the closest points define around the decision boundary is known as the
margin.
That is why the decision boundary of a support vector machine model is known as the
maximum margin classifier or the maximum margin hyperplane.

In other words, here’s how a support vector machine algorithm model works:

 First, it finds lines or boundaries that correctly classify the training dataset.
 Then, from those lines or boundaries, it picks the one that has the maximum distance
from the closest data points.

Alright, in the above support vector machine example, the dataset was linearly separable.
Now, the question, how do we classify non-linearly separable datasets as shown in Figure 6?

SVM Figure 6: Non-linearly Separable Dataset

Clearly, straight lines can’t be used to classify the above dataset. That is where Kernel SVM
comes into the picture.

SVM Figure 7: After Using Kernel Support Vector Classifier

What does Kernel SVM do? How does it find the classifier? Well, the Kernel SVM projects
the non-linearly separable datasets of lower dimensions to linearly separable data of higher
dimensions. Kernel SVM performs the same in such a way that datasets belonging to
different classes are allocated to different dimensions. Interesting, isn’t it?
Well, before exploring how to implement SVM in Python programming language, let us take
a look at the pros and cons of support vector machine algorithm.

Applications of SVM
Some of the areas where Support Vector Machines are used are as follows –

 Face Detection
SVMs are capable of classifying images of persons in an environment by creating a square
box that separates the face from the rest.

 Text and hypertext categorization

SVMs can be used for document classification in the sense that it performs the text and
hypertext categorization. Based on the score generated, it performs a comparison with the
threshold value.

 Bioinformatics
In the field of bioinformatics, SVMs are used for protein and genomic classification. They
can classify the genetic structure of the patients based on their biological problems.

 Handwriting recognition
Another area where support vector machines are used for visual recognition is handwriting
recognition.
Advantages and Disadvantages of Support
Vector Machine
Advantages of SVM
 Guaranteed Optimality: Owing to the nature of Convex Optimization, the solution will always
be global minimum not a local minimum.
 Abundance of Implementations: We can access it conveniently, be it from Python or Matlab.
 SVM can be used for linearly separable as well as non-linearly separable data. Linearly
separable data is the hard margin whereas non-linearly separable data poses a soft margin.
 SVMs provide compliance to the semi-supervised learning models. It can be used in areas
where the data is labeled as well as unlabeled. It only requires a condition to the
minimization problem which is known as the Transductive SVM.
 Feature Mapping used to be quite a load on the computational complexity of the overall
training performance of the model. However, with the help of Kernel Trick, SVM can carry
out the feature mapping using simple dot product.

Disadvantages of SVM
 SVM is incapable of handling text structures. This leads to loss of sequential information and
thereby, leading to worse performance.
 Vanilla SVM cannot return the probabilistic confidence value that is similar to logistic
regression. This does not provide much explanation as confidence of prediction is important
in several applications.
 Choice of the kernel is perhaps the biggest limitation of the support vector machine.
Considering so many kernels present, it becomes difficult to choose the right one for the
data.