m1

Hyperplane is defined by  w j j ( x)  b  0
j 1
m1
or w 
j 0
j j ( x)  0; where 0 ( x)  1

writing :  ( x)  [0 ( x), 1 ( x),..., m1 ( x)]

w  ( x)  0
T
we get :
N
from optimality conditions : w   ai d i  ( x i )
i 1
N

 ai d i  ( x i ) ( x)  0
T
Thus :
i 1
N
and so boundary is :  ai d i K ( x, xi )  0
i 1
N
and Output  w  ( x)   ai d i K ( x, xi )
T

i 1
m1
where : K ( x, xi )    j ( x) j ( x i )
j 0
 MultiClass SVMs
One-versus-all
Train n binary classifiers, one for each class against all other classes.
Predicted class is the class of the most confident classifier
One-versus-one
Train n(n-1)/2 classifiers, each discriminating between a pair of classes
Several strategies for selecting the final classification based on the output
of the binary SVMs
 Total C classes

One-versus-all (OVA)

Total C binary SVMs

One-versus-one

All-vs-all (AVA)

Total C(C-1)/2 binary SVMs

OVA AVA
We studied 2 methods for multi class classification: one versus all (one
versus rest) and one versus one. The third method is DAGSVM
One against all
One against all
The three places a point in the space can end up (w.r.t our
classifiers):
1. On the positive side of a single classifier
2. On the positive side of more than one classifier
3. On the negative side of all classifiers

The points that lie solely on the positive side of

the classifier only should clearly belong to the
class.
They clearly satisfy the condition

Therefore to get the associated label y for these

points, we can write
 Points on the positive side of more than one classifier

Which class should we assign each point to?

 Points on the negative side of all classifiers

In this case we cannot argue that one classifier is more 'confident' in the class identity of such points.
However, we can ask which classifier is the least 'unsure' about the class identity of such points?
If we repeat this logic for every point in the
region ...
 Directed Acyclic Graph SVM (DAGSVM)

A Directed Acyclic Graph (DAG) is a graph whose

edges have an orientation and no cycles.
Decision Directed Acyclic Graph (DDAG) contains
N(N-1)/2 nodes, each with an associated one
against one classifier.
Number of leaf nodes are N
 Directed Acyclic Graph SVM (DAGSVM)

A Directed Acyclic Graph (DAG) is a graph whose

edges have an orientation and no cycles.
Decision Directed Acyclic Graph (DDAG) contains
N(N-1)/2 nodes, each with an associated one
against one classifier.
Number of leaf nodes are N
A 4-class classification problem
To evaluate a particular DDAG , starting at the root node,
the binary function at a node is evaluated.
The node is then exited via the left edge, if the binary
function is zero; or the right edge, if the binary function
is one.
The value of the decision function is the value associated
with the final leaf node.
The list is initialized with a list of all classes.
A test point is evaluated against the decision node that
corresponds to the first and last elements of the list.
If the node prefers one of the two classes, the other class
is eliminated from the list, and the DDAG proceeds to
test the first and last elements of the new list.
The DDAG terminates when only one class remains in the
list.