 Make decision regarding class membership of

patterns

 Decision Making attempts to determine the actions

the system should take in the current situation
◦ Should a function be automated ?
◦ What should be done next ?
Prof. Rajeev Srivastava
CSE, IIT (BHU)
 For decision making we need to establish some rules known as
decision function or discriminating function
 Patterns in some space or domain are naturally separated in
groups or classes by class boundaries which may be-

 Linear function
 Non-linear function
 Pattern classes can be separated from each other by using
linear boundary only
 Boundary separating individual classes
Pattern can be separated by a straight line segment

In Euclidean space having coordinates x1 and x2. here w1, w2 and

w3 are decision parameter values. A two class classifier needs to
determine the values of these decision parameter values
 The general form of the equation can be written as
 Let us consider a situation where each of the 3classes is separate
from others by a single decision boundary. How can we
determine class membership of a pattern to any of these 3
classes?
 Solution: Here the 3 classes C1, C2 and C3 are separated by three decision
boundaries

Since a decision function will attain value>0 on one side and a value
< 0 on the other side, any pattern x can be classified as
 Let us consider the pattern vector x=(5,3). For this pattern we get,

The pattern vector x=(5,3) belongs to class c2. the decision boundaries
and the decision regions for the given classes are shown as
 Can we determine class membership of a pattern x = (9,3) to
any of the three classes?
 Solution: simply using three decision boundaries we can not solve
the classification problem completely. this is because any of the
decision functions may have positive (>0) or negative(<0) values for
more than one region. In our example, the decision space is divided
into 7 different regions separated by the boundaries
 For the pattern vector x =(9,3), we get
 Boundary separating class pairs
If we have n classes then a total n(n-1)/2 number of decision functions.
The decision boundary are of the form
 If the pattern x belongs to class Ci, then we have
 Consider position of pattern classes C1, C2and C3 as shown in
below. Using the concept of decision boundaries class pairs, find
out the decision regions.
 Solution: no pattern class is separable from the others by a single
decision boundary. However, each boundary is capable of separating
any two of the classes C1, C2 and C3.
Each of the decision boundaries now separate exactly two classes
without regard to the position of remaining classes specifically dij(x)
separates classes ci and cj only. if dij(x)> 0 then x ℇ cj and if dij(x)<0
then x ℇ cj.
 Boundary separating classes and class pairs
 There exist n decision functions d1(x), d2(x),…..,dn(x) and a pattern
x is assigned to the class Ci if
Illustrate example using the above boundary concept.
 Solution: the initial boundary definitions, we started with, were

If a pattern x falls in class C1,

then we must have
 Two classes having non-convex shapes may be linearly non-
separable
 A number of classes may have distribution of patterns that prohibit
them from linearly separable
 Non-linear decision boundaries can be represented mathematically
in a similar manner as their linear counter parts
 The concept decision function may assume as

It may also be written as

 Pattern classification is to used distance functions
 Pattern classes are separable by proximity concept
 Many cases classes do not exhibit this property fully
 When pattern vector of each class are very close to each other(for
member of the same class) minimum distance based classifier may
be very useful technique for classification.
 If there exist K classes, and their representative patterns may be
z1,z2,z3…..zk
 The Euclidean distance between an arbitrary vector x and the ith
prototype zi is given by
 A minimum distance classifier computes the distance from a pattern
x to a prototype of a class and assigns the pattern to the closest class