Department of Computer Science &

Engineering

Regulation 21

Semester: III

Course Code: CS3491

Course Name: Artificial Intelligence and Machine Learning

K.Sumithra Devi
Assistant Professor
CSE
1
KCG DEPARTMENT OF CSE 1
 UNIT III SUPERVISED LEARNING – MAXIMUM MARGIN CLASSIFIER

CO 3 Build supervised learning models

K3

KCG DEPARTMENT OF CSE 2

 Maximum Margin Classifier

• Review of linear classifiers

• Linear separability
• Perceptron

• Support Vector Machine (SVM) classifier

• Wide margin
• Cost function
• Slack variables
• Loss functions revisited
• Optimization
3
KCG DEPARTMENT OF CSE 3
 Binary Classification
Given training data ( x i , yi ) for i = 1 . . . N ,
with x i ∈ Rd and yi ∈ {−1, 1}, learn a
classifier f (x) such that
f i ≥ 0 yi = +1
(x ) < 0 yi = −1
i.e. y i f ( x i ) > 0 for a correct classification.

4
KCG DEPARTMENT OF CSE 4
 Linear separability

linearly
separable

not
linearly
separable

5
KCG DEPARTMENT OF CSE 5
 Linear classifiers
f (x) =
A linear classifier 0
X2
has the form
f (x) = w*x +
b
f (x) < f (x) > 0
0
X1

• in 2D the discriminant is a line

• W is the normal to the line, and b the bias
• W is known as the weight vector

6
KCG DEPARTMENT OF CSE 6
 What is the best w?

• maximum margin solution: most stable under perturbations of the inputs

7
KCG DEPARTMENT OF CSE 7
 8
KCG DEPARTMENT OF CSE 8
 9
KCG DEPARTMENT OF CSE 9
 10
KCG DEPARTMENT OF CSE 10
 11
KCG DEPARTMENT OF CSE 11