IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO.

3, AUGUST 1996 385

Correspondence
The Possibilistic C-Means Algorithm:
Insights and Recommendations
Raghu Krishnapuram and James M. Keller

Abstruct- Recently, the possibilistic C-means algorithm (PCM) was

proposed to address the drawbacks associated with the constrained
memberships used in algorithms such as the fuzzy C-means (FCM). In
this issue, Barni et al. report a difficulty they faced while applying the
PCM, and note that it exhibits an undesirable tendency to converge to
coincidenlal clusters. The purpose of this correspondence is not just to
address the issues raised by Barni et al., but to go further and analytically
examine the underlying principles of the PCM and the possibilistic
approach, in general. We analyze the data sets used by Barni et al. and
interpret the results reported by them in the light of our findings.

I. BACKGROUND
The main motivation behind the possibilistic approach to clustering
in [I] w,as to address the problems associated with the constraint
on the memberships used in the fuzzy clustering algorithms such Fig. 1. A plot of the PCM membership function for various values of the
as the fuzzy C-means (FCM). As pointed out in [l], the constraint fuzzifier parameter m.
causes the FCM to generate memberships that can be interpreted
as degrees of sharing but not as degrees of typicality. Thus, the
will discuss the issue of coincidental clusters raised by Barni et al.
memberships in a given cluster of two points that are equidistant
in Section IV. In Section V, we will explain and interpret the results
from the prototype of the cluster can be significantly different and
reported by Barni et al. In Section VI, we briefly discuss the role of
memberships of two points in a given cluster can be equal even
clustering in pattern recognition. Finally, in Section VII, we present
though the two points are arbitrarily far away from each other. This
the summary and conclusion.
gives rise to poor performance in the presence of noise and outliers.
In [l], a modification to the FCM objective function was proposed,
and a ne.w clustering algorithm, namely the possibilistic C-means 11. A FUNDAMENTAL DIFFERENCE BETWEENTHE FCM
(PCM) algorithm, was derived. The second term in the PCM objective AND THE PCM: PARTITIONING VERSUS MODE-SEEKING
function contains a parameter 17 whose value is to be estimated from The FCM is primarily a partitioning algorithm. It will find a fuzzy
the data. It is important to remember that the PCM is just a particular C-partition of a given data set, regardless of how many “clusters” are
implementation of the general idea of the possibilistic approach. The actually present in the data set. In other words, each component of the
possibi1is;ticapproach simply means that the membership value of a partition may or may not correspond to a “cluster.” In contrast, the
point in a cluster (or class) represents the typicality of the point in the PCM is a mode-seeking algorithm, i.e., each component generated by
class, or the possibility of the point belonging to the class. Typicality the PCM corresponds to a dense region in the data set. In the PCM,
is one of the most commonly used interpretations of memberships in the prototypes are automatically attracted to dense regions in feature
applications of fuzzy set theory. It was shown in [l] that by relaxing space as iterations proceed. This can be shown as follows.
the constraint on memberships, one can generate memberships that In the PCM algorithm, each cluster is independent of the other
represent typicality. It was also shown that since noise points or clusters. Hence, the objective function corresponding to cluster i can
outliers are less typical, typicality-based memberships automatically be formulated as
reduce the effect of noise points and outliers, and improve the results N
considerably.
In this correspondence, we revisit the PCM algorithm to provide
more insights and recommendations. This work is driven partially
In (l), p z represents the prototype associated with cluster i , U , (which
by the critique of Barni et al. [12] and partially by the results of
contains all the memberships associated with cluster i) represents the
our own investigations. In Section 11, we reformulate the objective
ith row of the membership matrix U , and 7. is the “bandwidth”
function of the PCM and show that the objective function consists of
or “resolution” or “scale” parameter. The membership matrix U
C-independent subfunctions, and the local minima of the subfunctions
generated by the PCM is, strictly speaking, not a “partition matrix,”
correspond to dense regions in feature space. In Section 111, we will
since its columns no longer satisfy the constraint
explain the significance of the parameter 17 as well as the fuzzifier
m in the PCM. Using the results derived in Sections I1 and 111, we
Manuscript received February 8, 1996.
The authors are with the Department of Computer Engineering and Com-
puter Science, University of Missouri, Columbia, MO 6521 1 USA. The parameter 7. needs to be prespecified, and can be estimated from
Publisher Item Identifier S 1063-6706(96)05623-S. the distance statistics of the data set X . The membership update
1063-6706/96$05.00 0 1996 IEEE
386 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 1996

2e+4 I
2e+4

Oe+O Oe+O
0 40 80 120 160 200 0 40 80 120 160 200
Location of center Location of center
(a) (b)

3e+5 1
2e+5

2e+5
7 c
2e+3

2e+3
.I

Y
.-
0
I
U 3
c
5 2e+5 z le+3
.*>
I
>
.-
c
U
.j$le+5 e,
3
0 0 5e+2
5e+4

Oe+O Oe+O
0 40 80 120 160 200 0 40 80 120 160 200
Location of center Location of center
(c) (4

2e+4 I

5e+3

Oe+O Oe+O
0 40 80 120 160 200 0 40 80 120 160 200
Location of center Location of center
(e) (0
Fig. 2. Plots o f the PCM subobjective functions for the case of a one-dimensional data set consisting of two Gaussian clusters. The data points are shown
as circles on the horizontal axis. Plots are shown for the following choices of parameters. (a) fuzzifier m = 1.5. (b) Fuzzifier m = 2. (c) Scale parameters
'r/ overestimated by an order o f magnitude. (d) Scale parameters 71 underestimated by an order o f magnitude. (e) Fuzzifier m = 1.5 with the clusters moved
closer to each other. (t] Fuzzifer 7n = 2.0 with the clusters moved closer to each other.
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 1996 387

rmo P O I

X
X

Fig. 3. The classification results of the FCM on a data set with two clusters in (a) noiseless and (b) noisy conditions. Poinls belonging to different
classes are shown using different symbols.

equation in the PCM is 111. THE IMPORTANCE AND MEANINGOF THE SCALE
PARAMETER 7). AND THE FUZZIFIER
PARAMETER m

It can be shown that the parameter qr is related to the resolution

parameter in the potential function [2] and the deterministic annealing
Solving for d2(x,,/3,)in terms of u L Jfrom (2), we obtain [3] approaches. It is also related to the idea of "scale" in robust
statistics [4]. More(over, the PCM is equivalent to a collection of
(3) C independent roblist fV1-estimators [5]-[7]. A detailed discussion
of these connections, as well as others, will be the subject of
We can now eliminate d Z ( z , , at) from the objective function in (1) another paper [8]. In any case, as explained in [l], the value of 7%
using (3). This gives us determines the distance at which the membership becomes 0.5. Thus,
1
V vz determines the "zone of influence" of a point. A point xJ will
J,(P<, U,;X ) = v <E(1 - have little influence on the estimates of the prototype parameters of
,=1 a cluster if d2(xJ,,!7c)is large when compared with q L . Therefore, a
ball-park value for qz is the variance (average intracluster distance)
For a giten value of q L , minimizing J , (Pz,U,; X ) is equivalent to
maximizing of cluster i [l].
The "fuzzifier" mi, determines the rate of decay of the membership

J l ( P Z > U Z ; x ) =qZ x[l

N

J=l
- (1 - u t J ) m - ' ] = 17%
1
V

where U : , = [l - (1 - u ~ , ) ~ can
- ~be ] interpreted as a modified
memberslhip. It is to be noted that U : , is obtained from U,, via a
monotonic mapping since
d l
__ u t J = ( m - 1)(1- >O for
du,,
Hence, U : , varies the same way as u C Ji.e., , u L J= 0 + U:, =
0; U,, = 1 + U : , = 1; both arc monotonically decreasing functions
of d 2 ( z J ,P L ) .Furthermore, for the special case of m = 2, (4) reduces
to
N Fig. 1, for this value of m the membership function decays too slowly
From (4) and (3,we see that for a given value of q t , each
of the C-subobjective functions is maximized by choosing the
prototype location such that the sum of the (modified) memberships
is maximized. This is achieved if the prototype is located in a dense
region since the (modified) membership is a monotonically decreasing
function of the distance to the prototype. If there are indeed C-dense
regions in feature space (corresponding to C-distinct clusters), then,
with proper initialization, each prototype will converge to a dense
region. In such a situation, even if all qz are equal (and, hence, all
subobjective functions become identical) each of them will still have
C-distinct minima corresponding to the C-dense regions.
,=1
~L:J (4)
value. Fig. 1 shows the membership function generated by the PCM
as a function of 'm. When m = 1, the memberships are crisp, i.e., all
points with d2(x,,[?t) greater than q A will have zero memberships.
When m + 00, the membership function does not decay to zero
at all. Also, in the FCM, m = 1 corresponds to the crisp case and
m + m corresponds to the maximally fuzzy case. However, the
interpretation of m is different in the FCM and the PCM. In the FCM,
m > 1. increasing values of m represent increased sharing of points among
all clusters, whereas in the PCM, increasing values of m represent
increased possibility of all points in the data set completely belonging
to a given cluster. Thus, the value of m that gives us satisfactory
performance is different in the two algorithms. A value of m N 2 is
known to give good results with the FCM. However, as can be seen in
in the case of the PCM. For example, if the clusters are Gaussian,
then about 95% of the points will be within d 2 ( z J PL)= 4a2 = 4qZ.
Therefore, the membership function should almost completely decay
to zero at d 2 ( x J ,P z ) / q z= 4. This is certainly not the case for 'm =
2. A more appropriate choice seems to be m 1.5.
Fig. 2 illustrates the ideas discussed in Sections I1 and I11 in the
one-dimensional case. Fig. 2(a) shows the two subobjective functions
J,(aL, U z ;X ) for a data set consisting of two Gaussian clusters with
centers at 65 and 135. Each cluster has 50 poin;ts,and the variances of
the two clusters arc 144 and 100, respectively. The objective functions
are plotted with respect to the center parameter. The variances have
been used as the estimates of q z ,and a value of 1.5 was used for
m. The data points are shown as small circles on the x-axis. Two
388
. .
(e)
Fig. 4. A comparison of the classification results of the PCM and the FCM or( a data set consisting of three Gaussian clusters (courtesy of Barni et
al.) (a), (c), and (e) show the points belonging to the three classes according to the PCM, and (b), (d), and (f) show the points belonging to the
three classes according to the FCM.
minima can be seen in each subobjective function. The upper curve
corresponds to the first cluster, and the lower curve to the second. If
the first center is initialized in the left half, and the second one in the
right half (as will be the case if the FCM is used for initialization)
then the two individual centers do converge to the two distinct local
minima. However, if both centers are initialized on the right half,
then they will both find the same local minimum, thus completely
missing cluster 1. Fig. 2(b) shows the plots of the objective functions
for the same data set when m = 2. It can be seen that the valleys
become shallow, and the local minima slightly move toward each
other. Fig. 2(c) shows the two subobjective functions when the q.
are overestimated by an order of magnitude (i.e., ql = 1440 and
q 2 = 1000). It can be seen that the two valleys tend to merge into
one shallow valley. This illustrates the fact that at this “resolution”
or “scale” the algorithm perceives only one cluster. Fig. 2(d) shows
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 1996
. .
(0
the two subobjective functions when the q1 are underestimated by
an order of magnitude (Le, 711 = 14 and q 2 = 10). This results
in rather jagged subobjective functions with multiple local minima.
For example, there is a small local minimum corresponding to the
three left-most points, which form a legitimate cluster at this scale.
Fig. 2(e) and (0shows the subobjective functions when m = 1.5
and when m = 2, respectively, for a data set obtained by moving
the two clusters closer to each other so that the centers are now at
84 and 119 respectively. It can be seen that with m = 2 it is no
longer possible to find the two distinct local minima, and no matter
what the initialization, both centers will converge to approximately
the same point.
The above examples illustrate that for the PCM to give meaningful
solution, i) we need good estimates of the scale parameters q c ,ii) we
need to initialize each prototype so that we obtain C-distinct local
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 1996
2r
0-
-2 - (7)
0
0
0
0 d 2 ( z J P, z ) , this formulation may be more appropriate when clusters
-4 - 0
0 a monotonically decreasing function in [0,1], similar to (1 - u , , ) ~ .
8
-6 -1
-2 -1 0 1
Fig. 5. Ii scatter plot of the seven-dimensionaldata set from the multichannel
satellite image (courtesy of Bami et al.) mapped to two-dimensions using
Dzwinel’s algorithm. Only two distinct clusters can be seen.
minima ((if in fact there are C-distinct clusters), and iii) the value
of m needs to be chosen so that the decay rate in the membership
function is meaningful. We now briefly address these issues.
The scale parameter q7 of a cluster corresponds to the “zone of
influence” or “size” of the cluster. Therefore, when the data set is
relatively noise-free, the FCM algorithm can be used to initialize
the partition and obtain a good estimate of the 7.. For example, we
could use
N initialization for the prototypes or memberships is provided. In other
11x7 > o!
or
c1
where d:3 = d2(x,,PL),and Q is an appropriate threshold. The
FCM will also provide a good initialization for the prototypes, so
that they will converge to C-distinct local minima, provided there
indeed are C-distinct dense regions. However, when the data is noisy,
the initial partition produced by the FCM will be poor, resulting in
bad initializations for the prototypes and improper estimates for the
scale parameters q 8 . Fig. 3 shows an example of such a situation.
Fig. 3(a) shows the “classification” generated by the FCM on a data
set with two clusters. The two classes are indicated by square and
cross symbols, respectively. Fig. 3(b) shows the classification when
“noise” points are added. The FCM lumps the two good clusters into
one class and all the noise points into the other. In such a situation,
both the initial partition and the scale estimates will be unacceptable.
Fortunatlely, there are methods in robust statistics [4] which can be
used to rescue us from this situation. A detailed discussion of these
issues may be found in [8].
As seen in Fig. I , a good choice for the value of the fuzzifier m
for the I’CM seems to be around 1.5. As an aside, we would like
to remaIk that a similar comment applies to the Gustafson-Kessel
algorithrn [9] (see, for example, [lo]. On the other hand, as pointed
out in Section I, the PCM is a particular implementation of the
possibilistic approach. In fact, we could eliminate m altogether by
choosing alternative formulations of the PCM. For instance, if we
389
The prototype update equations will remain unchanged. Since the
exponential function decays much more rapidly for large values of
are expected to be close lo one another. Note that u z Jlog ut3- uL,is
If we use the Mahalanobis distance for d2(x,. /A), then we can even
eliminate 7%(i.e., set 7 , = 1). A robust estimate of the covariance
matrix will take itc; place.
2 IV. COINCIDENT
CLUSTERS:
A BLESSING
IN DISGUISE
As mentioned in Section 11, the FCM is primarily a partitioning
algorithm, whereas the PCM is primarily a mode-seeking algorithm.
If we were to smooth the feature histogram by a kernel function
that is shaped like the PCM membership function, then the modes
corresponding to the peaks in the smoothed histogram would be the
locations of convergence of the PCM. A distinguishing characteristic
of the PCM is that the membership ut3 of a point x, in cluster i is not
relative and depends only on the distance of . ~ from j p i . Therefore,
the PCM objective function can be viewed as a collection of C-
independent functions. This means that even if the true value of C
i s unknown, the outcome of the algorithm will be useful as long as
the desired scale qz is specified or estimated accurately and a good
words, the algorithm can potentially find C good clusters from a data
set that may have more than C clusters. (As shown in Section 11, by
definition, “good” clusters in the PCM correspond to dense regions.)
On the other hand, if the data set has less than C clusters, then the
algorithm can still potentially find C good clusters, out of which
some of them may be identical. It follows that the specification of C
is somewhat irrelevant in the PCM. In particular, if C is chosen to
be one, then the PCM will find one good cluster. Thus, it can be seen
that the PCM approach has the potential for solving one of the major
problems with the FCM, namely, the need to know the number of
clusters (see, also, [ll]).
The behavior of the FCM is quite different. It will always find
the specified number of “clusters” by arbitrarily splitting or merging
clusters in the data set. In other words, the FCM makes no guarantees
about the kind of “clusters” that it will find, and the partition
generated by the FCM may or may not correspond to C intuitively
correct clusters. To illustrate this point, let us consider the case when
there is only one cluster, and let us assume that we apply the FCM
with C = 2. The FCM will find two clusters by artificially splitting
the single cluster in the middle. Although the FCM “fails to recognize
the structure underlying the data set” (to borrow a phrase that Barni
et al. attribute to the PCM), it does not mean that the FCM result is
“bad.” It simply means that the assumptions under which the FCM
produces good clusters has been violated because the FCM assumes
that the number of clusters is correctly specified. For the situation
described above, if we apply the PCM, it will find two coincident
clusters. In this case, the PCM does recognize the structure of the
data better. Similarly, if for a particular initialization the PCM gives
390 IBEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 1996

Fig. 6. The segmentation result produced by the FCM when all seven channels are used.

Fig. 7. The segmentation result produced by the FCM when only channels five and six are used

coincidental clusters, it does not mean that the result is bad. By
merging coincident clusters after overspecifying C or by sequentially
removing clusters after repeatedly applying the PCM with C = 1,
one can, in fact, determine the number of clusters, thus addressing
the problem of cluster validity [ I l l .
v. EXPERIMENTAL RESULTSREPORTEDBY
BARNIet al.: AN ALTERNATIVE
INTERPRETATION
In [12] Barni et al., reported some difficulties they encountered
while running the PCM. They also graciously agreed to supply us
with the data sets so we were able to run the same experiments
and reproduce their results. Here, we analyze and reinterpret the
results and explain why they are consistent with the derivations and
explanations given in Sections 11-V.
We first discuss the example with three Gaussian clusters (Fig. 4
in [ 121). This data set is relatively clean and has no noise. Barni
et al. reported that even when initialized with the FCM, the PCM
finds three coincidental clusters, i.e., it just finds the larger cluster
in the center and ignores the smaller clusters on either side, thus,
“failing to recognize the structure underlying the data set.” A closer
examination reveals that this behavior is very easily explained by
an incorrect choice of the fuzzifier parameter. Since the clusters are
not well separated but actually touch one another, the membership
functions need to be chosen in such a way that they decay rapidly
enough outside the zone of influence. As explained in Section 111, a
value of two for m is simply not the best choice. Fig. 4 shows the
results of the PCM on the same data set when m = 1.5. The results
of the FCM are also shown for comparison. It can be seen that the
two results are very similar. We can also completely do away with
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 1996 391

250 -1
200 - 1 e

V I I ‘ I ’ I ’ I - I

0 50 100 150 200 250 0 50 100 150 200 250

channel 5 channel 5
(a) (b)

250

200 4
\D
150
C

8
100

50

0 50 100 150 200 250

channel 5
(C)

Fig. 8. (a) A scatter plot of the feature points from channels five and six. (b) Clusters found by the FCM. (c) Clusters found by the PCM.

the fuzzifier parameter by formulating the PCM differently, as in (6)
and (7).
We now discuss the multichannel satellite image data set. It so
happens that this data set is ideal for explaining and illustrating the
differemes between the FCM and the PCM. The image corresponding
to one of the seven channels is shown in Fig. 1 in [12]. Barni et al.
applied tlhe FCM with C = 4. Even though it may be true that there
are four main classes (water, wooded areas, agricultural land, and
urban areas) in this image, when viewed in proper resolution, one can
easily find many regions that do not fit into any of the four categories.
For example, what about beaches? Do we distinguish between roads,
bridges, and buildings, or lump them all into the category of urban
areas? In the latter case, do the features allow us to do that? There
is much variation in the intensity values within the so-called “urban
areas” no matter which channel is viewed. Similarly, some of the
water is so highly reflective that it appears white. Similar variations
occur in the wooded areas. In other words, unless more sophisticated
features such as texture are taken into account, it is quite impossible to
use a clustering technique to “separate” these class by intensity values
alone, and one can hardly expect the seven-dimensional feature space
to contain four distinct “clusters.”
To verify our suspicion, we used Dzwinel’s, algrorithm [13] based
on the molecular dynamics approach to find the global minimum of
Sammon’s criterion for mapping higher dimensional data to lower
dimeusions. The resulting two-dimensional rendition of the seven-
dimensional data set is shown in Fig. 5. Sime the number of data
points is very large (512 x 699), to prevent clutter, a subsample
of the data set is shown. Only two distinct clusters are seen. The
water region appears as the smaller and denser cluster, because
in this region, there is relatively less variation in the intensity
values in all channels. The highly reflective areas that appear white
in the image show up as outliers in this mapping. The larger
cluster includes samples from all the remaining regions, and it is
hard if not impossible to distinguish the three classes within this
cluster.
Another interesting fact about this data set is that the channels are
highly correlated. To illustrate this fact, we show the segmentation
result of the FCM when all seven channels are used (Fig. 6) and
392
Fig. 9. The segmentation result produced by the FCM when only channels five and six are used after three coincident clusters have been merged
when only channels five and six are used (Fig. 7). It can be seen that
the results are similar. Thus, the remaining channels do not contribute
much to the final result. Moreover, from Fig. 6 it can be seen that the
FCM misclassification rate is unacceptably high. This is mainly due
to the choice of the features. They are not sufficiently homogeneous
within each class and distinct between classes.
Since the channels are highly correlated, and since a two-
dimensional feature space can be easily plotted and viewed, here
we use the two-dimensional data (of channels five and six) to explain
the results reported by Barni et al. Fig. 8(a) shows the scatter plot
of the two-dimensional feature vectors. (Again, to prevent clutter a
subsample of the data set is shown.) The similarity between this plot
and the one in Fig. 5 can be readily seen. The bottom left comer of the
plot contains the small elongated cluster corresponding to the water
region which gradually merges into the large cluster represented by
the remaining regions. The 4-partition generated by the FCM on the
data set is shown in Fig. 8(b) after assigning each point to the cluster
with the highest membership. The points in different “clusters” are
shown using different symbols. It can be seen that the FCM arbitrarily
divides the large cluster into three components which may or may
not correspond to the ideal partition from a classification point of
view. In this case, it apparently does not, judging from the high rate
of misclassifications. Moreover, the components hardly represent
“clusters.” Again, this result does not mean that the FCM is not
useful. It simply means that it has been misapplied. Fig. 8(c) shows
the result of the PCM when it is applied with C = 4.As reported by
Barni et al., the PCM identifies the smaller cluster but produces three
coincidental clusters rather than splitting the bigger cluster into three.
(In Fig. 8(c), the three coincidental clusters have been merged.) The
resulting segmentation is shown in Fig. 9.
Whether the result in Fig. 8(b) or the one in Fig. 8(c) is better
depends on the application. When the data points do not fall into
the desired number of clusters, the arbitrary partition produced by
the FCM may happen to be reasonably close to the ideal partition
from a classification point of view. However, this is by no means
guaranteed. On the other hand, from a clustering point of view, one
can argue that the PCM produces a better result. In any case, one
can do much better by generating more meaningful features for this
application.
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 1996
VI. THE(MIS)USEOF CLUSTERING IN PATTERN RJ3COGNITION
Clustering is a powerful technique with a wide variety of ap-
plications not only in pattern recognition, but also in areas such
as probability density estimation (mixture density decomposition),
membership function estimation, image coding, model identification
in control systems, and neural network training. Nevertheless, it is
to be remembered that clustering is an unsupervised technique, and
it does not use label information. Thus, in the training phase of a
pattern-recognition system, when the labels for the pattern vectors
are available, one is better off using a supervised technique than a
clustering algorithm. Unfortunately, too often one sees comparisons
made between a supervised technique such as the Bayes method
or a neural-network method and a clustering technique such as
the FCM. There is, however, an interesting and useful way to
use clustering methods in a supervised mode. This is done by
applying a clustering algorithm to data points belonging to each class
separately. This enables us to estimate the parameters of the class
prototype(s). Note that depending on the features, the data points
corresponding to a particular class may or may not fall into one
cluster. Therefore, a correct number of clusters needs to be chosen to
extract a representation of the data points corresponding to a class in
terms of prototype parameters. Once the prototype parameters have
been estimated for each class, test data can be classified using a
suitable rule. This procedure is roughly equivalent to estimating the
probability density functions of the classes and will yield similar
results (see, for example, [14]).
The use of clustering for classification (rather than for parameter
estimation) may be justified if it is known that the classes are well
separated in feature space and points belonging to different classes
do form clusters. It has been argued that fuzzy clustering techniques
are more meaningful when clusters touch or overlap. Hence, it might
seem that they can be used for classification even when the classes
are not well separated. However, the more meaningful result of fuzzy
clustering is only from a clustering (or data representation) point
of view, and not from a classification point of view. For example,
the center point in the famous “butterfly” data set in [15] gets a
membership of 0.5 in both clusters when fuzzy clustering is used.
This does make sense because this point is exactly between the two
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 4, NO. 3, AUGUST 7996
clusters. However, if we in fact knew that the center point had a
label that was the same as that of the cluster on the left, then a
membership of 0.5 does not make sense from a classification point
of view. The partition generated by a fuzzy clustering algorithm has
more to do with the distance measure chosen than the class labels
associated with the data points. Thus, although in specific cases the
partition may roughly correspond with the correct classification, this
is simply fortuitous unless we, in fact, know the distributions of the
clusters in feature space and choose the distance measures and the
clustering algorithm accordingly.
VII. CONCLUSION
Since the objective function of the PCM is a modification of that
of the FCM, the PCM may appear to be a close cousin of the FCM.
However, as shown in this correspondence, there are fundamental
differences between the two algorithms. The FCM is primarily a
partitioning algorithm, whereas the PCM is primarily a mode-seeking
algorithm. The power of the PCM does not lie in creating partitions,
but rather in tinding meaningful clusters as defined by dense regions.
Its strengths are that it overcomes the need to specify the number of
clusters and it is highly robust in the presence of noise and outliers.
Its weakness is that it requires a good initialization and a reliable
scale estimate to function effectively. When the data is not severely
contaminated, the FCM can provide a reasonable initialization and a
scale estiimate. Thus, with the proper choice of the scale and fuzzifier
parameters, the PCM can be used to improve the results of the FCM.
The situation is quite different when the data set is highly noisy.
The least squares (LS) algorithm is a general regression algorithm
in statistics that minimizes the sum of the squares of the residues,
and it has been used extensively in engineering applications. It
has been shown [6], [7] that the FCM is a generalization of the
least squares technique that uses harmonic means of distances to
prototypes as residues. It is well known that the LS analysis is
severely compromised by a single outlier in the data set. Thus, the
FCM can completely break down in the presence of a single outlier. In
contrast, It has been shown [6]-[SI that the PCM is a robust parameter
estimation technique that is related to the M-estimator [16] which
has been widely used in robust statistics with good results. Robust
techniques can tolerate up to 50% noise [4]. As shown in Fig. 3,
the FCM is quite inadequate as an initialization and scale estimation
tool when noise is present. Fortunately, there are many techniques in
robust statistics that can help us in this regard.
REFERENCES
R. Krishnapuram and J. Keller, “A possihilistic approach to clustering,”
IEEI? Trans. Fuzzy Syst., vol. 1, pp. 98-110, May 1993.
J. T. Tau and R. C. Gonzales, Pattern Recognition Principles. Reading,
MA: Addison-Wesley, 1974.
G. ELeni and X. Liu, “A least biased fuzzy clustering method,” IEEE
Trans. Pattern Anal. Machine Intell., vol. 16, pp. 954-960, Sept. 1994.
P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier
Detection. New York Wiley, 1987.
R. DavB and R. Krishnapuram, “Robust algorithms for clustering,” in
Proc. Inr. Fuzzy Syst. Assoc. Congress, SZo Paula, Brazil, July 1995,
vol. I, pp. 561-564.
J. Kim, R. Krishnapuram, and R. Dav: “On robustifying the C-means
algorithms,” in Proc. North Amer. Fuzzy Informat. Processing Soc. Coni,
College Park, MD, Sept. 1995, pp. 630-635.
0. Nasraoui and R. Krishnapuram, “Crisp interpretations of fuzzy and
possibilistic clustering algorithms,” in Proc. Eur. Congress F u u y Intell.
Technol., Aachen, Germany, Sept. 1995, pp. 1312-1318.
1063-6706/96$05.00 0 1996 IEEE
393
[SI R. DavC and R. Krishnapuram, “Robust clustering methods: a unified
view,” IEEE Truns. Fuzzy Syst., to be published.
[9] D. E. Gustafson, and W. C. Kessel, “Fuzzy clustering with a fuzzy
covariance matrix,” in Proc. IEEE CDC, San Diego, CA, 1979, pp.
761-766.
[lo] R. Krishnapuram and C.-P. Freg, “Fitting an unknown number of lines
and planes to image data through compatible cluster merging,” Pattern
Recogn., vol. 25, Apr. 1992, pp. 385-400.
[ 1I] R. Krishnapuram, “Generation of membership functions via possibilistic
clustering,” in 3rd IEEE Con$ Fuzzy Syst., Orllando, FL, July 1994, pp.
902-908.
[ 121 M. Barni, V. Cappellini, and A. Mecocci, “Comments on ‘A Possibilistic
Approach to Clustering,”’ IEEE Trans. Fuzzy Syst., vol. 4, pp. 393-396,
Aug. 1996
[ 131 W. Dzwinel, “In search of the global minimum in problems of feature
extraction and selection,” in Proc. Eur. Congress Intell. Tech. So@
Computing, Aachen, Germany, Sept. 1995, pp. 1326-1 330.
1141 S. Medasani, J. Kim, and R. Krishnapuram, “Ektimation of membership
functions for pattern recognition and computer vision,” in Fuzzy Logic
and its Applications 10 Engineering, Information Sciences, and Intelli-
gent Systems, K . C. Min and Z. Bien, Eds. Norwell MA: Kluwer,
1995, pp. 45-54.
[ 151 J . C. Bezdek, Puttern Recognition with Fuzzy Objective Function Algo-
rithms. New York Plenum, 1981.
[16] P. J. Huber, Robust Statisrics. New York: Wiley, 1981.
Comments on “A Possibilistic Approach to Clustering”
M. Barni, V. Cappellini, and A. Mecocci
Abstract-In this comment, we report a difficulty with the application
of the possibilisticapproach to fuzzy clustering (PCM) proposed by Keller
and Krishnapuram. In applying this algorithm we found that it has the
undesirable tendency to produce coincident clusters. Results illustrating
this tendency are reported and a possible explanation for the PCM
behavior is suggested.
I. INTRODUCTION
In their paper,’ Krishnapuram and Keller presented a new approach
to fuzzy clustering [possibilistic e-means (PCM)]. By relaxing the
constraint that the memberships of a data point across classes sum to
one, each cluster is disentangled from the others and the membership
values are interpreted as the compatibilities of the point to the
class prototypes. Besides, the possibilistic aplproach leads to higher
noise immunity with respect to classical algorithms derived from
Bezdek’s fuzzy e-means (FCM) [l].Indeed the novel approach is very
interesting since by recasting fuzzy clustering into the framework of
possibility theory, membership functions are directly related to the
typicality of data points with respect to the given classes. In this
way, classification tasks are made easier and the impact of spurious
points on the tinal partition is reduced.
The purpose here is to describe our experience in applying the
PCM algorithm, whose performance, we found, is severely compro-
Manuscript received March 30, 1995; revised August 1, 1995.
M. Bani and V. Cappellini are with the Department of Electronic Engi-
neering, University of Florence Via S. Marta, 3 50139 Firenze, Italy.
A. Mecocci is with the Department of Electronic Engineering, University
of Pavia Via Abbiategrasso, 209-27100 Pavia, Italy
Publisher Item Idenltifier !S 1063-6706(96)05624-X.
‘R. Krishnapuram and J. M. Keller, IEEE Trans. Fuzzy Syst., vol. 1, pp.
98-1 10, May 1993.