Professional Documents
Culture Documents
Pattern Classification: All Materials in These Slides Were Taken From
Pattern Classification: All Materials in These Slides Were Taken From
Pattern Classification: All Materials in These Slides Were Taken From
Classification
All materials in these slides were taken from
Pattern Classification (2nd ed) by R. O.
Duda, P. E. Hart and D. G. Stork, John Wiley
& Sons, 2000
with the permission of the authors and the
publisher
Chapter 2 (part 3)
Bayesian Decision Theory
(Sections 2-6,2-9)
Discriminant Functions for the Normal Density
Bayes Decision Theory Discrete Features
Pattern Classification, Chapter 2 (Part 3)
2
Discriminant Functions for the
Normal Density
We saw that the minimum error-rate
classification can be achieved by the
discriminant function
g
i
(x) = ln P(x | e
i
) + ln P(e
i
)
Case of multivariate normal
) ( P ln ln
2
1
2 ln
2
d
) x ( ) x (
2
1
) x ( g
i i
1
i
i
t
i i
e E t + =
+ = =
+ =
Pattern Classification, Chapter 2 (Part 3)
4
A classifier that uses linear discriminant functions
is called a linear machine
The decision surfaces for a linear machine are
pieces of hyperplanes defined by:
g
i
(x) = g
j
(x)
Pattern Classification, Chapter 2 (Part 3)
5
Pattern Classification, Chapter 2 (Part 3)
6
The hyperplane separating R
i
and R
j
always orthogonal to the line linking the means!
) (
) ( P
) ( P
ln ) (
2
1
x
j i
j
i
2
j i
2
j i 0
e
e
o
+ =
) (
2
1
x then ) ( P ) ( P if
j i 0 j i
e e + = =
Pattern Classification, Chapter 2 (Part 3)
7
Pattern Classification, Chapter 2 (Part 3)
8
Pattern Classification, Chapter 2 (Part 3)
9
Case E
i
= E (covariance of all classes are
identical but arbitrary!)
Hyperplane separating R
i
and R
j
(the hyperplane separating R
i
and R
j
is generally
not orthogonal to the line between the means!)
| |
) .(
) ( ) (
) ( P / ) ( P ln
) (
2
1
x
j i
j i
1 t
j i
j i
j i 0
E
e e
+ =
Pattern Classification, Chapter 2 (Part 3)
10
Pattern Classification, Chapter 2 (Part 3)
11
Pattern Classification, Chapter 2 (Part 3)
12
Case E
i
= arbitrary
The covariance matrices are different for each category
(Hyperquadrics which are: hyperplanes, pairs of
hyperplanes, hyperspheres, hyperellipsoids,
hyperparaboloids, hyperhyperboloids)
) ( P ln ln
2
1
2
1
w
w
2
1
W
: where
w x w x W x ) x ( g
i i i
1
i
t
i 0 i
i
1
i i
1
i i
0 i
t
i i
t
i
e E E
E
E
+ =
=
=
= + =
=
=
=
+ =
=
=
e e
e
e