Professional Documents
Culture Documents
Clustering - Fuzzy C-Means
Clustering - Fuzzy C-Means
Introduction|Kmeans|FuzzyCmeans|Hierarchical|MixtureofGaussians|Links
FuzzyCMeansClustering
TheAlgorithm
Fuzzycmeans(FCM)isamethodofclusteringwhichallowsonepieceofdatatobelongtotwoormoreclusters.Thismethod(developedbyDunn
in1973andimprovedbyBezdekin1981)isfrequentlyusedinpatternrecognition.Itisbasedonminimizationofthefollowingobjectivefunction:
wheremisanyrealnumbergreaterthan1,uijisthedegreeofmembershipofxiintheclusterj,xiistheithofddimensionalmeasureddata,cjisthe
ddimensioncenterofthecluster,and||*||isanynormexpressingthesimilaritybetweenanymeasureddataandthecenter.
Fuzzypartitioningiscarriedoutthroughaniterativeoptimizationoftheobjectivefunctionshownabove,withtheupdateofmembershipuijandthe
clustercenterscjby:
Thisiterationwillstopwhen ,where is a termination criterion between 0 and 1, whereas k are the iteration steps. This
procedureconvergestoalocalminimumorasaddlepointofJm.
Thealgorithmiscomposedofthefollowingsteps:
1.InitializeU=[uij]matrix,U(0)
2.Atkstep:calculatethecentersvectorsC(k)=[cj]withU(k)
3.UpdateU(k),U(k+1)
4.If||U(k+1)U(k)||< thenSTOPotherwisereturntostep2.
Remarks
Asalreadytold,dataareboundtoeachclusterbymeansofaMembershipFunction,whichrepresentsthefuzzybehaviourofthisalgorithm.Todo
that,wesimplyhavetobuildanappropriatematrixnamedUwhosefactorsarenumbersbetween0and1,andrepresentthedegreeofmembership
betweendataandcentersofclusters.
Forabetterunderstanding,wemayconsiderthissimplemonodimensionalexample.Givenacertaindataset,supposetorepresentitasdistributedon
anaxis.Thefigurebelowshowsthis:
Lookingatthepicture,wemayidentifytwoclustersinproximityofthetwodataconcentrations.WewillrefertothemusingAandB.Inthefirst
approachshowninthistutorialthekmeansalgorithmweassociatedeachdatumtoaspecificcentroidtherefore,thismembershipfunctionlooked
likethis:
IntheFCMapproach,instead,thesamegivendatumdoesnotbelongexclusivelytoawelldefinedcluster,butitcanbeplacedinamiddleway.In
thiscase,themembershipfunctionfollowsasmootherlinetoindicatethateverydatummaybelongtoseveralclusterswithdifferentvaluesofthe
membershipcoefficient.
Inthefigureabove,thedatumshownasaredmarkedspotbelongsmoretotheBclusterratherthantheAcluster.Thevalue0.2ofmindicatesthe
degreeofmembershiptoAforsuchdatum.Now,insteadofusingagraphicalrepresentation,weintroduceamatrixUwhosefactorsaretheones
takenfromthemembershipfunctions:
(a)(b)
Thenumberofrowsandcolumnsdependsonhowmanydataandclustersweareconsidering.MoreexactlywehaveC=2columns(C=2clusters)
andNrows,whereCisthetotalnumberofclustersandNisthetotalnumberofdata.Thegenericelementissoindicated:uij.
Intheexamplesabovewehaveconsideredthekmeans(a)andFCM(b)cases.Wecannoticethatinthefirstcase(a)thecoefficientsarealways
unitary.Itissotoindicatethefactthateachdatumcanbelongonlytoonecluster.Otherpropertiesareshownbelow:
AnExample
Here,weconsiderthesimplecaseofamonodimensionalapplicationoftheFCM.Twentydataandthreeclustersareusedtoinitializethealgorithm
andtocomputetheUmatrix.Figuresbelow(takenfromourinteractivedemo)showthemembershipvalueforeachdatumandforeachcluster.The
colorofthedataisthatofthenearestclusteraccordingtothemembershipfunction.
Inthesimulationshowninthefigureabovewehaveusedafuzzynesscoefficientm=2andwehavealsoimposedtoterminatethealgorithmwhen
.Thepictureshowstheinitialconditionwherethefuzzydistributiondependsontheparticularpositionoftheclusters.No
stepisperformedyetsothatclustersarenotidentifiedverywell.Nowwecanrunthealgorithmuntilthestopconditionisverified.Thefigurebelow
showsthefinalconditionreachedatthe8thstepwithm=2and =0.3:
Isitpossibletodobetter?Certainly,wecoulduseanhigheraccuracybutwewouldhavealsotopayforabiggercomputationaleffort.Inthenext
figurewecanseeabetterresulthavingusedthesameinitialconditionsand =0.01,butweneeded37steps!
Itisalsoimportanttonoticethatdifferentinitializationscausedifferentevolutionsofthealgorithm.Infactitcouldconvergetothesameresultbut
probablywithadifferentnumberofiterationsteps.
Bibliography
J. C. Dunn (1973): "A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact WellSeparated Clusters", Journal of
Cybernetics3:3257
J.C.Bezdek(1981):"PatternRecognitionwithFuzzyObjectiveFunctionAlgoritms",PlenumPress,NewYork
TariqRashid:Clustering
http://www.cs.bris.ac.uk/home/tr1690/documentation/fuzzy_clustering_initial_report/node11.html
HansJoachimMuchaandHizirSofyan:NonhierarchicalClustering
http://www.quantlet.com/mdstat/scripts/xag/html/xaghtmlframe149.html
FuzzyCmeansinteractivedemo
Previouspage|Nextpage