1 s2.0 S0020025521007271 Main

Information Sciences 578 (2021) 297–322
Contents lists available at ScienceDirect
Information Sciences
journal homepage: www.elsevier.com/locate/ins
GT2-CFC: General type-2 collaborative fuzzy clustering method

Fariba Salehi a, Mohammad Reza Keyvanpour b,⇑, Arash Sharifi a
a
Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
b
Department of Computer Engineering, Alzahra University, Tehran, Iran
a r t i c l e i n f o a b s t r a c t
Article history: This paper presents a new algorithm called general type-2 collaborative fuzzy clustering
Received 27 August 2020 (GT2-CFC), which focuses on uncertainty associated with the structures discovered for
Received in revised form 4 July 2021 the same data stored in multiple data sites in different feature spaces while maintaining
Accepted 7 July 2021
data-sharing restrictions. The uncertainty of membership degrees is directly controlled
Available online 13 July 2021
by using the available data rather than by reinforcing the lower-order fuzzy model through
general type-2 fuzzy sets (GT2FS) including primary and secondary memberships. Primary
Keywords:
membership degrees show the obtained membership without the knowledge gained from
General type-2 fuzzy sets
General type-2 membership degrees
other data sites, and secondary membership degrees indicate the belonging degree to the
Collaborative fuzzy clustering data in the presence of external knowledge. In addition to the ability of GT2FS in uncer-
Horizontal distribution tainty modeling, due to the lack of any type reduction and defuzzification during the pro-
Validity index posed clustering process, there is no data loss, and time complexity becomes is reduced
Relative entropy leading to more powerful uncertainty modeling. Finally, a new validity index is proposed
to evaluate the quality of collaboration using the relative entropy-based global consistency.
The superiority and efficiency of the proposed method are illustrated with several
experiments.
Ó 2021 Elsevier Inc. All rights reserved.
1. Introduction
Clustering mainly aims to partition data into clusters with a maximum similarity in a cluster (homogeneous), as well as a
maximum dissimilarity between clusters (heterogeneous). One data could be assigned to multiple clusters by applying fuzzy
logic to clustering. In a fuzzy clustering based on the objective function, the aims of clustering such as an error or a quan-
titative criterion could be provided by an objective function. The main trends in developing these clustering methods are as
follows [18]:
Structure-based such as noise detection or outliers [1,46], cluster identification with various shapes and densities [12,14],
and fuzzifier parameter correction [16].
Knowledge-based such as the hints related to the data structure being searched, which guides the clustering process, like
an auxiliary supervision mechanism, leading to semi-supervised [38], constraint [15], proximity [18], directional [28],
consensus [29], and collaborative [8,23,47,49] fuzzy clustering methods.
⇑ Corresponding author.
E-mail address: keyvanpour@alzahra.ac.ir (M.R. Keyvanpour).
https://doi.org/10.1016/j.ins.2021.07.037
0020-0255/Ó 2021 Elsevier Inc. All rights reserved.
F. Salehi, Mohammad Reza Keyvanpour and A. Sharifi Information Sciences 578 (2021) 297–322
Therefore, collaborative fuzzy clustering is a knowledge-based clustering which aims to improve the local clusterings of
the data stored in multiple data sites by observing data sharing restrictions and using the knowledge gained from the struc-
ture of other data sites. In addition, it is considered as a distributed fuzzy clustering due to the exploration of data structure
in the distributed data. Data organization in a distributed manner and the existence of many restrictions such as privacy and
technicality results in creating the interaction between data sites at a certain level. Information granular level (IGL) meets
these restrictions since there is no direct access to the data objects and knowledge of other data sites is recognized.
Distributed fuzzy clustering could be performed based on two different purposes: (a) improve local clustering through
collaboration mechanism [5,27,40,47,48], and (b) achieve a single clustering obtained by the consensus mechanism [50].
ð1Þ ð1Þ
As shown in Fig. 1, the result of consensus-based clustering leads to a single and global clustering from C ½1 and C ½2 to C,
ð1Þ ð1Þ ðt Þ ðt Þ
and collaboration-based clustering improves clustering C ½1 and C ½2 to C ½1 and C ½2 without any global consensus or fusion
to two data sites either during clustering or at the end of the communication for the two data sites X ½1 and X ½2 after T com-
munication steps. However, the collaboration framework is preferred over the consensus framework in many cases since
dataset bias may not be desirable on a global level due to data access restrictions. Therefore, the present study focuses on
the collaborative fuzzy clustering with horizontal data distribution, i.e., the distribution of the same data in different feature
spaces.
Based on the two main types of data distribution, collaborative fuzzy clustering is divided into a horizontal scheme where
the data sites with the same objects are available in different feature spaces [27,34,40], and a vertical scheme where the data
sites with different objects are available in the same feature spaces [19,23]. In this study, the horizontal layout is considered.
The horizontal distribution of data is common in various application fields such as social networks, marketing, information
retrieval, bioinformatics, chemistry, medicine, and image processing. Social networks usually have different types of infor-
mation from the same people who are the members of different social networks. Regarding marketing, information is avail-
able for the same customers of the department stores and banks [39]. In image processing, an image can be described with
various types of visual features [8]. For example, information retrieval is considered on web pages with different aspects such
as text, links, and structures.
The collaborative fuzzy clustering method was first proposed for the horizontal mode by Pedrycz in [27]. The collabora-
tion between the findings obtained was realized in terms of the partition matrix, and the disagreement between the data
sites was weighted through a non-negative coefficient, which indicates priori information about the desired collaboration
between the two data sites. In addition, the confidentiality of the data was maintained, and the storage and bandwidth costs
reduced considerably. Pedrycz et al. [31] proposed another model for the same IGL, in which collaboration was realized in
the form of the induced prototypes. In [34] was modeled horizontal collaborative fuzzy clustering with induced partition
matrices by preprocessing prototypes before collaboration. Yu et al. presented the realization of horizontal collaboration
by emphasizing common features, as well as all features through partition matrices and prototypes, respectively [44]. A col-
laborative fuzzy clustering approach was suggested by focusing on the discovery of distributed or ubiquitous knowledge [9].
In addition, the collaborative fuzzy clustering for the vertical mode was realized through aggregating the benefits of fuzzy
and rough sets [21]. Further, collaborative fuzzy clustering was proposed using the multiple kernel technique for the vertical
mode [7]. An extension of vertical collaborative fuzzy clustering was proposed through interval type-2 fuzzy sets for control-
ling the uncertainties of fuzzifier parameter [6,23]. In [47], reconciliation between the structures of data sites was achieved
based on relative entropy, followed by its development to interval type-2 fuzzy sets [48]. The Partial collaboration of struc-
ðt Þ ðt Þ
Fig. 1. Distributed fuzzy clustering for two data sites X ½1 and X ½2 based on consensus C and collaboration C ½1 ; C ½2 after T communication steps with the
ð1 Þ ð1 Þ
results of local clustering C ½1 and C ½2 .
298
tured data was proposed by Fusheng et al. [43] since there are cases where the complete sharing of structured data is not
ideal or practical such as more technical and security constraints.
A growing body of research has addressed the interaction coefficient, also called the collaboration coefficient, which is
one of the most important factors affecting the performance of collaboration clustering [5,27]. In [27], the quantity of the
collaboration coefficient was obtained empirically or based on experts’ knowledge and information. However, in a collabo-
rative fuzzy clustering process, only the pieces of information which can be involved in determining this coefficient are con-
sidered as knowledge due to security restrictions and privacy or potential confidentiality [28]. However, using this kind of
knowledge is somehow challenging for direct identification. Fushang et al. automatically determined collaboration strength
by considering the partition matrices [42]. In this method, collaboration strength was determined based on the established
principles, which resulted in proposing two approaches for calculating coefficients. In horizontal collaborative fuzzy cluster-
ing, Falkon et al. [10] determined the intensity of collaboration in a two-step method. Rough sets and particle swarm opti-
mization (PSO) meta-heuristic were used in the initialization and tuning steps of collaboration strength, respectively.
Depaire et al. [9] used the particle swarm optimization algorithm. Falkon et al. [11] used a multi-objective particle swarm
optimization (MOPSO). In [5], adaptive or constant coefficients were investigated for each pair of data sites, and the calcu-
lation methods were presented in an adaptive mode.
Clustering algorithms are usually exposed to different sources of uncertainty, which should be managed well. These
uncertainties increase with distributed data and algorithms. In the literature, modeling uncertainty related to the fuzzifier
parameter was emphasized by using the interval type-2 fuzzy sets. Therefore, less attention has been paid to the uncertainty
of membership degrees, as well as the ability of general type-2 fuzzy sets to model different types of uncertainty. In addition,
in most type-2 fuzzy clustering methods, type-2 membership degrees are defuzzified to type-1 membership degrees in each
iteration leading to data loss and computational complexity, which necessitates further study in controlling the uncertainty
using fuzzy mathematics [13]. Thus, the general type-2 collaborative fuzzy clustering method is proposed in this study, by
focusing on the uncertainty associated with the membership degrees in the presence and absence of the knowledge obtained
from other data sites. In addition, it directly specifies the membership degrees with the available data using the general type-
2 fuzzy sets. Further, the proposed algorithm lacks type reduction and defuzzification steps. In the proposed objective func-
tion is provided by regularizing the collaboration strength between the data sites, adjusting the fuzziness rate of clusters
along with deriving a closed-form solution. The proposed validity index and existing indices are used to evaluate the collab-
oration quality and effect of the proposed method with the type-1 and interval type-2 collaborative fuzzy clustering meth-
ods. The contributions of the present study are summarized as follows:
Modeling the uncertainties related to knowledge obtained from peer data sites and defining membership degrees of data
objects by considering external knowledge and without it by using general type-2 membership degrees
Designing a new algorithm for the general type-2 collaborative fuzzy clustering method with some advantages such as
handling the uncertainty of membership degrees with no type reduction and defuzzification, the possibility of adjusta-
bility of the fuzziness in the extracted clusters, determination of collaboration strength regularly, and the objective func-
tion with a closed-form solution
Developing a validity index for evaluating collaboration quality based on the relative entropy divergence measure
The remainder of this study is organized as follows: Section 2 reviews collaborative fuzzy clustering methods. Section 3
describes the proposed method for the collaborative fuzzy clustering algorithm. Section 4 elaborates the proposed validity
index. Experimental results are addressed in Section 5. Finally, conclusions are described in Section 6.
2. Prerequisites
In this section, the main existing type-1 collaborative fuzzy clustering methods found in the literature are surveyed. Then,
the type-2 collaborative fuzzy clustering methods are briefly reviewed, although some are related to the vertical distribution
of the data.
2.1. Type-1 collaborative fuzzy clustering
The data objects were assigned to multiple clusters with varying degrees of membership by adding the type-1 fuzzy set
theory to clustering methods. In addition, collaborative fuzzy clustering made it possible to improve local clustering by rein-
forcing the knowledge gained from other data sites. The collaborative fuzzy clustering method was first proposed by Pedrycz
in [27] for the same data stored in different feature spaces. In fact, data sites are consistent with horizontally distributed
data. Local clusterings are improved in conformity with data sharing restrictions through the collaboration mechanism.
The collaboration between the findings obtained in terms of the partition matrix is realized, and the disagreements between
the data sites are weighted through a non-negative coefficient, which indicates priori information about the desired collab-
oration between the two data sites. The confidentiality of the data is maintained, and the storage and bandwidth costs
decrease considerably. The first formulation for collaborative stage iterative algorithm is implemented based on minimizing
the following objective function:
299
X
N X
C
2
X
S X
N X
C
2 2
J ½ i ¼ u2cn½i dcn½i þ b½i;j ucn½i ucn½ j dcn½i ð1Þ
n¼1 c¼1 j¼1;j–i n¼1 c¼1

where X is dataset with S data sites, X ½i shows ith data site with N data objects and C clusters. dcn½i ¼ xn½i v c ½i indicates the
2
distance between nth object xn½i ; cth cluster center prototype v c½i ; ucn½i denotes the membership degree of nth data to the cth
cluster center, and b½i;j is the effective degree ith data site with jth data site. In this version, the knowledge communicated
between the data sites is information granular level in the form of partition matrices, which is meaningful because of the
horizontal distribution of the data and reconciliation between the data sites is achieved by partition matrices. Making the
clustering based on the ith data site being aware of other partition matrices is the role of the second term in the above equa-
tion. The differences between the partition matrices tend to be lower when the structures of data sites are similar.
In [47], the reconciliation is made between the discovered structures with relative entropy, the objective function of
which is defined as follows:
XN X C X S X N X C
2 ucn½i
J ½ i ¼ cn½i dcn½i þ c½i
um ucn½i ln ð2Þ
n¼1 c¼1 j¼1;j–i n¼1 c¼1
ucn½ j
where m indicates degree of fuzziness, c½i refers to the nonnegative coefficient of interaction of ith data site with other data
sites, and the rest of the notations are the same as those in the previous equation in which relative entropy concept was used
as the communication method. Data sites communicate their findings in the form of membership degrees, and each data site
proceeds with its optimization by focusing on local data.
Among the problems related to the above-mentioned methods, the fuzzifier value directly affects the position and quality
of the cluster partition. However, the fuzziness rate of the clusters is fixed with 2 in Eq. (1); otherwise, the membership
degrees should be numerically determined for any value of the fuzzifier other than 2, which leads to additional computation.
In Eq. (2), updating the membership degrees and collaboration strength do not have exact solutions although adjusting the
fuzzifier is possible. On the other hand, with type-1 fuzzy sets, the uncertainty of data belonging to clusters was modeled
with precise membership degrees between ½0; 1. The fuzzifier value should normally reflect the fuzzy nature of the input
data. However, in most applications, pinpointing the appropriate value of the fuzzifier becomes either difficult or impossible.
In addition, its determination by using an exact numerical value seems counterintuitive since the fuzzifier value cannot be
accurately determined. Nevertheless, these precise membership degrees are unable to manage uncertainties by increasing
uncertainty. In the following subsection, type-2 collaborative fuzzy clustering methods are reviewed, although some meth-
ods are related to the vertical mode.
2.2. Type-2 collaborative fuzzy clustering
Type-2 fuzzy sets are considered as an improvement on the fuzzy sets. Wu et al. emphasized the use of type-2 fuzzy sets
[41] since type-2 fuzzy systems suggest some novel partitions of the input domain which cannot be gained by type-1 fuzzy
systems, the use of which is appropriate for membership function uncertainties and model words, while using type-1 fuzzy
sets is not correct. Type-2 fuzzy systems proved a better performance in many applications [22,25].
Different types of uncertainty for systems are related to the uncertainty of the input data, interpretation of the calculated
result, as well as appropriate values for different parameters [20]. Disagreements about the extracted expert knowledge,
uncertainty measurements, and noise data used to adjust parameters lead to uncertainties in fuzzy set parameters. Various
parameters in fuzzy membership design in clustering and pattern recognition algorithms are faced with some uncertainties
in selecting fuzzifier degree, prototype, and distance criterion.
The uncertainty in interval type-2 fuzzy clustering fuzzifier parameter has been investigated by researchers [6,16,23,35].
The fuzzifier parameter controls the fuzzy value of the final partitions in fuzzy clustering. Therefore, to control and handle
the uncertainty of the fuzzifier parameter m, the degree of belonging to a dataset using two fuzzifier values m1 and m2 is
extended to the interval type-2 fuzzy set, which forms the footprint of uncertainty related to the fuzzy parameter m. Fuzzi-
fiers m1 and m2 represent different fuzzy degrees. In fact, in these methods, the interval of the primary membership degree of
a data object is displayed by its lowest and highest primary membership degrees. These values are determined using the
lower and upper membership functions through two different values for the fuzzifier. As mentioned in the previous section,
an exact and definite number is used to determine the representation of the uncertainty of the nth data given to the cth clus-
ter in the type-1 fuzzy clustering methods. However, the uncertainties are modeled in interval type-2 fuzzy clustering meth-
ods, and the primary membership degree nth data to cth cluster can be specified with the help of interval ½unc ; u nc where unc
and u nc are equal to lower and upper membership functions, respectively. All secondary membership degrees of primary
membership degrees are equal to one.
Ngo et al. [23] introduced an interval type-2 fuzzy collaborative clustering for the vertical mode using two different fuzzi-
fier values. The use of weighting exponents representing different fuzzy membership degrees allows various objective func-
tions to be minimized in clustering as follows:
300
X X m 2
N ½i C
X
S X X
N ½i C
2
J m 1 ½ i ¼ ucn1½i dcn½i þ b½ijj um
cn½i dcn½ j
1
ð3Þ
n¼1 c¼1 j¼1;j–i n¼1 c¼1
X X m 2
N ½i C
X
S X X
N ½i C
2
J m 2 ½ i ¼ ucn2½i dcn½i þ b½ijj um
cn½i dcn½ j
2
ð4Þ
n¼1 c¼1 j¼1;j–i n¼1 c¼1
According to the vertical distribution of data, i.e., the storage of different data in the same feature space, the number of
data per data site is not necessarily the same, and N ½i represents the number of data objects of F-dimensional in ith data sites.
Two fuzzifiers m1 and m2 are related to upper and lower bounds of the interval membership degree.
P 2
dcn½i ¼ Ff¼1 xnf ½i v cf ½i shows distance of nth data object xn½i with cth center prototype v c½i in ith data site. ucn½i denotes
2
membership degree of nth object with cth cluster center. When the data objects are assigned to prototypes in the data site,
the distance of objects is minimized with the centers of the corresponding clusters of the other data sites through
P 2
dcn½ j ¼ Ff¼1 xnf ½i v cf ½ j . The number of data sites and cluster centers are indicated by S and C, respectively. Collaboration
2
and communication are accomplished at the prototype level with collaboration strength b½ijj between ith and jth data site.
Dang et al. used the same method to model uncertainty in collaborative fuzzy clustering for the vertical mode with the
following objective function [6].
X
C X
N ½i
X
S X
C X
N ½i

cn½i v c½i v c½ j
2 1 ~ 2
J m1 ½i ¼ um 1
d
cn½i cn½i þ b um ð5Þ
c¼1 n¼1 j¼1;j–i c¼1 n¼1
X
C X
N ½i
X
S X
C X
N ½i

cn½i v c½i v c½ j
2 2 ~ 2
J m2 ½i ¼ um
cn½i dcn½i þ b
2
um ð6Þ
c¼1 n¼1 j¼1;j–i c¼1 n¼1
PS PS
b v ~ C
is taken as an average of b½ijj , and v with v
~ c½ijj ¼ v c0 ½ j j min v c ½i v c0 ½ j 2 is cth
b
j¼1;j–i ½ijj j¼1;j–i ½ijj c½ijj
where b ¼ S1
~ c ½ j ¼ P S
b c ¼1 0
j¼1;j–i ½ijj
induced cluster center prototype. The rest of the notations are the same as the previous equation. In this method, the rec-
onciliation was made based on the distance between the prototypes of one data site and the induced prototypes of other data
sites.
In [48], the uncertainty of membership degrees is directly modeled using interval arithmetic theorems with interval type-
2 fuzzy sets. Although this method does not require any type-reduction, updating the collaboration coefficient and member-
ship degrees fails to have an exact solution.
As a result, finding the interval type-2 fuzzy membership functions using the appropriate values for the fuzzifier param-
eter is considered as the main objective of these methods [6,23,35]. Updating cluster center positions should consider the
interval membership degree leading to interval cluster coordinates. The positions of the interval cluster centers are calcu-
lated by using the definition of the interval centroid center of interval type-2 fuzzy sets. The fuzzifier value varies from
m1 to m2 . Determining the right value for the fuzzy parameter is not easy, which depends on the problem and cannot be
easily generalized. Therefore, several different combinations of fuzzifier parameters m1 and m2 should be considered. The
individual values of the left and right cluster boundaries in each dimension are ordered by sorting the data objects in a par-
ticular dimension and are calculated by using as an iterative procedure such as Karnik–Mendel (KM) [17]. Then, the exact
position of the centroid can be calculated by defuzzificating the interval centroid. Hence, the cluster center prototypes
require type reduction and defuzzification in each iteration of the algorithm, leading to more computational complexity
due to its iterative procedure and loss of some information. This reduction in efficiency increases in general type-2 fuzzy sets
due to an increase in discretization steps in type reduction algorithms. A large body of research has been conducted to create
a more efficient type reduction in using type-2 fuzzy systems [4,24,33]. On the other hand, general type-2 fuzzy sets have
more freedom than interval type-2 fuzzy sets. They are more powerful tools for handling uncertainties. Therefore, they can
outperform the interval type-2 fuzzy sets [25,36,37].
As a result, the introduction of a collaborative fuzzy clustering method based on general type-2 fuzzy sets is necessary for
real-world applications. The designing methods of type-2 fuzzy membership degrees are divided into two groups [13] based
on the results of type-1 fuzzy clustering methods, i.e., the reinforcement of type-1 fuzzy models and using available data
directly [3,13,45,48].
In the proposed method, the latter is considered for designing membership degrees. Therefore, this study proposes a gen-
eral type-2 fuzzy collaborative clustering method in which the uncertainty related to membership functions is considered as
the main concern. In other words, general type-2 fuzzy membership functions are directly modeled using the available data.
In addition, there is no type reduction or defuzzification, leading to better management of the uncertainty and computational
complexity.
301
3. Proposed general type-2 collaborative fuzzy clustering method
In this section, the proposed general type-2 collaborative fuzzy clustering (GT2-CFC) method is described. This model is
developed based on the uncertainty associated with the knowledge of induced peer data sites. In this section, the concept of
knowledge uncertainty is explained. In the next subsections, the proposed method and theoretical analyses are explained.
3.1. Knowledge uncertainty
Clustering algorithms, especially in the distributed datasets, are commonly subject to various sources of uncertainty that
should be appropriately managed. Several forms of uncertainty have been identified such as uncertainty related to the input
data, interpretation of results [20], and selection of appropriate values of the parameters in a fuzzy membership design
[23,35]. However, the uncertainty addressed in this study is related to the obtained external knowledge from peer data sites.
In popular collaborative fuzzy clustering models, data sites exchange their findings with an active interaction. Then, each
acts according to the differences between the results received from the environment. Data sites adjust their local findings by
reforming information granules. These granules are based on locally available data, but they provide the particulars of the
information granules supplied by other data sites at the same time. The received information granules contain noise cause
incorrect bias of the algorithm, which is visually addressed in Section 5.4.
Therefore, we will face uncertainty in reconciling the local structure to the findings of peer data sites. In such cases, the
lack of a suitable model, which considers the knowledge uncertainty in the structure of the clustering algorithm, is a severe
problem of structure discovery for the distributed data. Most interval type-2 fuzzy clustering methods [16,35] and interval
type-2 collaborative fuzzy clustering methods [6,23] have focused on representing and managing uncertainty, which are pre-
sented in the fuzzy memberships of data objects related to fuzzifier changes.
The lack of a proper clustering analysis method, which can consider existing uncertainties in the gained knowledge from
peer data sites, led us to develop a general type-2 collaborative fuzzy clustering to model these uncertainties. In this study,
the proposed method modifies the structure of the collaborative fuzzy clustering methods and defines a new collaborative
fuzzy clustering. Therefore, local structures are improved with the obtained knowledge by controlling the caused uncertainty
with a general type-2 fuzzy set. Considering the uncertainty, strengthening the level of collaboration, and not limiting the
amount of fuzzifier parameter, we expect that the proposed general type-2 collaborative fuzzy clustering models uncertainty
better than type-1 and type-2 collaborative fuzzy clustering methods.
3.2. Proposed collaborative fuzzy clustering method
A novel collaborative fuzzy clustering is proposed for the same data stored in different feature spaces called general type-
2 collaborative fuzzy clustering (GT2-CFC). Given that improving local clustering by adhering to data-sharing restrictions is
the main purpose of collaborative fuzzy clustering, the improvement is achieved through an auxiliary mechanism of external
knowledge derived from the discovered data structures in other data sites. The proposed model focuses on uncertainty about
the degrees of membership in the presence or absence of external knowledge. In the proposed model, membership degrees
are directly modeled with the help of the available data using general type-2 fuzzy sets, which are powerful tools in handling
uncertainties. Thus, minimizing the problem is modeled by considering the primary and secondary membership degrees for
the collaboration stage as follows:
ð7Þ
The constraints for Eq. (7) are as follows:
ð8Þ
302

where X ¼ X ½i ji ¼ 1; 2; . . . ; S is dataset including S data sites, ith data site
n o
X ½i ¼ xn½i jxn½i ¼ xn1 ½i ; . . . ; xnf ½i 2 Rf ½i ; n ¼ 1; . . . ; N consists of N data objects of f ½i -dimensional clustered into C clusters,
½i

and xn½i denotes nth data object in ith data site. U e ½i ¼ u ~ cn½i indicates general type-2 fuzzy membership degree matrix
CN
of data site ith, which is produced by the proposed general type-2 collaborative fuzzy clustering algorithm. For ith data site,
_ n_ o
U ½i ¼ u cn½i is the primary membership degree matrix which indicates the obtained membership degrees without
CN
external knowledge gained from other data sites, and is the secondary membership degree matrix
related to the membership degrees, which are achieved by the auxiliary mechanism resulting from the knowledge related to
n o
discovered data structures from other data sites. V ½i ¼ v c½i jv c½i ¼ v c1 ; . . . ; v cf 2 Rf ½i ; c ¼ 1; . . . ; C indicates the C cluster
½i
center prototypes, v c½i is the center of f ½i -dimensional of the cth cluster in ith data site, and m > 1 shows a weighting expo-
nent called fuzzifier. dcn½i denotes the distance between nth object and cth center prototype in ith data site, and
n o
ðBÞSS ¼ b½i½ j shows the collaboration strength matrix in which each entry b½i½ j is the effect level ith data site with jth data
site. d is the non-negative coefficient of entropy of collaboration strength.
The first term of Eq. (7) minimizes the weighted distance of data objects and cluster center prototypes without the effect
of external knowledge, while the second term reflects the effect of structures formed on all remaining data sites. The distance
between the reference data site and other data sites in the presence of external knowledge must be minimized. This require-
ment is obtained by this term of the objective function. The third term is the negative entropy of the collaboration strength
which is regularized according to the available data, and models important collaboration intensities to provide a better con-
tribution to the collaboration between data sites. The terms could be balanced for finding the optimal solution with a proper
choice of d.
_
~ cn½i with primary membership degree u cn½i and secondary membership degree
Theorem 1. General type-2 membership degree u
which optimizes the objective function Eq. (7) and satisfies the constraints Eq. (8) obtained with the following explicit
solutions:
C
X 2
_ dcn½i m1
u cn½i ¼ ð9Þ
c0 ¼1
dc0 n½i
ð10Þ
Cluster center prototype and collaboration strength are updated by Eq. (11) and Eq. (12), respectively as follows:
ð11Þ
ð12Þ
_
e V; b optimization is equivalent to J
Proof. J GT2CFC ½i U; ~ cn½i entries of
GT2CFC ½i U , , V; bÞ minimization because all u
P _
e ½i 8c; n
U are independent. By adding constraint terms Eq. (8) C
¼ 1; 8 n; i,
i¼1 u cn and
PS
j¼1;j–i b½i½ j ¼ 1; 8i in the objective function Eq. (7) by using Lagrangian multipliers technique k0n½i ; k00n½i ; and k½i is led
to unconstrained optimization problem as follows:
303
ð13Þ
The four first order necessary conditions (FONC) are as follows:
ð14Þ
@J0GT2CFC ½i
With the necessary condition _ ¼ 0, we have:
@ u cn½i
_ 0 2
0 ¼ mu m1
cn½i dcn½i kn½i ð15Þ
PC _
So, Lagrangian multiplier k0n½i with constraint term i¼1 u cn ¼ 1; 8 n; i is calculated as:
!m1
1
m1
1 X
C
1
k0n½i ¼ 2
ð16Þ
c0 ¼1 mdc0 n½i
By replacing Eq. (16) in Eq. (15), we obtain:

C
X 2
_ dcn½i m1
u cn½i ¼ ð17Þ
c0 ¼1
dc0 n½i
Based on the second FONC, the following equation is obtained:
ð18Þ
ð19Þ
k00n½i is calculated with constraint term . Thus, we have:
PS
PC b u0
j¼1;j–i ½i½ j c n½ j
1 c0 ¼1 P S
b
j¼1;j–i ½i½ j
k00n½i ¼ PC ð20Þ
c0 ¼1 PS 1
2 b d2
j¼1;j–i ½i½ j c0 n½i
The following equation is obtained by substituting Eq. (20) in Eq. (19):
304
ð21Þ
The third FONC shown in Eq. (14) and considering Euclidean distance for the distance between data objects and center pro-
totypes lead to the following expression:
ð22Þ
ð23Þ
To find the last FONC related to optimality, the gradient J 0GT2CFC ½i with respect to b½i½ j becomes equal to:
ð24Þ
ð25Þ
PS
By computing Lagrangian multiplier k½i with the constraint term j¼1;j–i b½i½ j ¼ 1; 8i; b½i½ j which optimizes Eq. (13) is updated
by the following equation:
ð26Þ
_
_
If the second derivatives of J GT2CFC ½i U , , V; bÞ with respect to u cn½i , , v c½i , and b½i½ j are positive, then JGT2CFC½i is a min-
imizing objective function. The second derivatives are as follows:
@ 2 J 0GT2CFC ½i _ 2
_ ¼ mðm 1Þu m2
cn½i dcn½i > 0 ð27Þ
@ 2 u cn½i
ð28Þ
ð29Þ
@ 2 J 0GT2CFC ½i d1

2
¼ >0 ð30Þ
@ b½i½ j b½i½ j
Examination of the second derivatives Eqs. (27)–(30) shows that J GT2CFC ½i is a minimizing objective function. The proposed
algorithm is presented below:
305
Algorithm 1. GT2-CFC: General Type-2 Collaborative Fuzzy Clustering

Input: Dataset X ¼ X ½i ji ¼ 1; 2; . . . ; S ,
n o
Data site X ½i ¼ xn½i jxn½i 2 Rf ½i ; n ¼ 1; . . . ; N
(.N data object stored in S data sites in different feature spaces.)
Output: Clusters of X ½i ; 8i
Local Stage:
Run FCM clusteing for X ½i ; 8i
Collaborative Stage:
Broadcast U ½i ; 8i to all other data sites.
If C ½i is not the same among all other data sites then
Calculate optimum value of validity indices VI½i ; 8i
Communicate U ½i ; 8i corresponding optimum value of VI½i .
End
_
Initialize , u cn½i with ucn½i ; 8i.
Fix entropy coefficient of collaboration strength d, fuzzifier m.
Repeat
Repeat for each data site X ½i ; 8i
Calculate b½i½ j ; 8j, using Eq. (12).
_
Compute u cn½i ; 8n; c, using Eq. (9).
Calculate , 8n; c, using Eq. (10).
Update v c½i ; 8c, using Eq. (11).

e ðoldÞ e ðnewÞ
Until U ½i U ½i < e
Broadcast U e ½i ; 8i to all other data sites
P
s e ðoldÞ e ðnewÞ
Until i¼1 U ½i U ½i < e Total
In this section, the proposed method for collaborative fuzzy clustering is described which consists of two stages. In the
local stage, each data site can perform local clustering individually, independently, and even in parallel. In the collaborative
stage, homogenizing them with the algorithm [5] is completed first if the number of clusters in data sites is not the same.
Finally, in the main step of this stage, local clusterings are improved with the relevant explicit solutions along with managing
the uncertainties of membership degrees with the tool of the powerful general type-2 fuzzy sets directly with the existing
data. The membership degrees, cluster center prototypes, and collaboration intensities are updated until converging data
site. Collaboration between data sites is accomplished until satisfying the termination condition.
3.3. Theoretical analyses of GT2-CFC
This section describes analyzing the GT2-CFC method, including the profound concept of Theorem 1 and the benefits of
the proposed work. The proposed method was organized in Section 3.2. The problem in Eq. (7) and its constraints in Eq. (8) is
a constrained optimization problem which aims to derive cluster center prototypes and the collaboration strength, along
with the primary and secondary membership degrees of data objects for a given data site by utilizing a collaboration mech-
anism. The Lagrangian method is used for the optimal solutions of the problem as in Eqs. (9)–(12). It can be clearly seen that
the cluster centers (Eq. (11)), collaboration levels (Eq. (12)), primary (Eq. (9)), and secondary membership degrees (Eq. (10))
are affected by the mechanism of collaboration and modeling knowledge uncertainties of which occurs using the obtained
knowledge in peer data sites. Obviously, the strength of the collaboration between each pair of data sites improves the
uncertainty modeling as well as the clustering process. Thus, this factor must be carefully considered to obtain the appro-
priate ensemble effect from collaborative activities leading to an overall description of the distributed data. The collaboration
strength is established by forming the augmented objective function which incorporates data as well as quantifying the dif-
ferences between findings. The optimal level of collaboration depends on the number of parameters in the collaborative clus-
tering method, especially the number of clusters and number of data sites participating in the collaboration. This factor can
also depend on the data objects. The third augmented objective function in Eq. (7) and third constraint in Eq. (8) affect the
optimal and automatic determination of this factor. In other words, the cluster center prototypes in Eq. (11) are updated by
the primary and secondary membership degrees, in which the components of both types of membership are calculated based
on the data, previous memberships, and cluster prototypes. Further, the collaboration strength and knowledge gained from
other data sites are involved to determine the secondary membership components. As a result, regulating the next results
306
based on the previous results is an appropriate approach. The meaning of Theorem 1 is not only the reflection of the contri-
butions stated in Section 1 in the computational process, which can be easily presented as an algorithm in Section 3.2.
The advantages of the proposed method are related to the following cases. First, the proposed algorithm can be used in
various problems which require fast processing. No type reduction and defuzzification is applied in the proposed algorithm.
Since there is no need for the defuzzification of membership functions during each iteration of this algorithm, they will not
lose any information. Lack of type reduction improves computational time since this process, usually accompanied by sorting
patterns is considered an iterative procedure. As a result, a reduction occurs in the total time. In addition, the clustering qual-
ity of the outputs is better than the outputs of the relevant collaborative fuzzy clustering algorithms due to the analysis in
Section 3.1. Second, it is possible to adjust the value of the fuzzifier in the proposed objective function. The fuzzifier value
influences the location and quality of partitions in fuzzy clustering algorithms. The fuzzifier controls the fuzziness of the
extracted clusters. The boundaries of the clusters become softer by a larger of the fuzzifier. In many fuzzy clustering algo-
rithms based on knowledge [43,27,28,26,32], the value of fuzzifier degree is limited to two or one, which can be related to
the lack of explicit solutions for each other value leading to extra computational activities for determining numerical solu-
tions or not having a solution. As a result, fuzzy clustering is reduced to hard clustering. Third, selecting the appropriate
objective function is one of the most important steps of objective function-based clustering algorithms. In collaborative
fuzzy clustering algorithms, the augmented objective function in directing of optimization is critical and accomplished based
on available data and information granules. In the new objective function, the above objectives are considered as their linear
combination simultaneously. In this combination, the effect of each objective with its assigned weight is determined. In
addition, the minimization of the new objective function has explicit solutions for its constituent variables (closed-form
solution). Finally, the type of uncertainty modeling in the GT2-CFC method can be considered as a tutorial for developing
knowledge-based fuzzy clustering methods such as semi-supervised fuzzy clustering algorithms for modeling the uncer-
tainty associated with with the received internal knowledge including supervision mechanism.
4. Proposed validity index
Quantitative evaluation methods related to the results of a clustering algorithm are known as cluster validity. These
methods are indices for evaluating the resulting partition. Therefore, the results are validated after the clustering step lead-
ing to the definition of a good clustering scheme for a given dataset. Also, validation indices in the procedure of clustering
algorithms play a key role in terminating clustering algorithms. The algorithm terminates when the difference between val-
ues obtained in two consecutive iterations is less than a predetermined value. As a result, finding appropriate validation cri-
teria is one of the most important topics in cluster analysis. In this section, the proposed validity index is based on relative
entropy for evaluating collaboration quality. Thus, the indices of collaboration quality and then relative entropy divergence
are first introduced in this section.
4.1. Collaboration quality validity indices
The validation of clusters for evaluating clustering results to discover the partitioning which best fits the underlying data
is considered as one of the most important cluster analysis issues.
Data objects are stored in several data sites, and collaborative fuzzy clustering methods should be aware of dependencies
in all data sites to discover the global structures while maintaining data sharing restrictions due to confidentiality, privacy,
and technically. The collaborative fuzzy clustering methods necessitate two aspects for evaluating the validity of clusters due
to data distribution status and global structure discovery. The first aspect is based on evaluating the collaboration effect in
order to find the best clustering scheme which a clustering algorithm can find under certain parameters and assumptions.
The latter focuses on assessing the collaboration quality quantitatively, upon which the results of a clustering algorithm are
evaluated by reconciling global specifications imposed on data sites. The effect of collaboration between data sites is exam-
ined locally. Relevant criteria show how the change occurs through the collaboration mechanism, which is optimized, and
more complete clustering is established after the collaboration.
A noteworthy point in evaluating the effect of collaboration is its dependence on the collaboration quality. In other words,
the local and global evaluation schemes are accomplished with the effect and quality of collaboration, respectively.
4.2. Relative entropy divergence measure
Let X is a space of all probability distributions. A divergence or a contrast function on X is a function Dð:jj:Þ : X X ! R
satisfying.
1. DðPjjQ Þ P 0 8 P; Q 2 X
2. DðPjjQ Þ ¼ 0 () P ¼ Q
where P and Q are two discrete probability distributions on the same probability space X.
307
In mathematical statistics, the relative entropy divergence measure [38,45] indicates the degree of difference between
one and another probability distribution. Divergence P from Q with the Kullback–Leibler divergence measure is also called
relative entropy [46,47], and is represented by DðP kQ Þ as follows:
X
P ð xÞ
DðPjjQ Þ ¼ PðxÞ ln ð31Þ
x2X
Q ð xÞ
In the above definition, the following conventions exist:
0 def 0 def P def

0 log ¼ 0; 0 log ¼ 0; 0 log ¼0 ð32Þ
0 Q 0
4.3. Relative entropy (RE) validity index
A new validity index is proposed for the quality of collaboration using the global consistency of structures in different
data sites based on relative entropy divergence measure. Degree of compatibility between the corresponding discovered data
structures in the form of partition matrices is realized using relative entropy divergence. The use of this statistical divergence
measure is more appropriate and consistent compared to the indices [27,30,31] which use distance metrics such as Ham-
ming or Euclidean distance.
Each cluster is a fuzzy set of the reference set (data site) and the fuzzy membership degree is interpreted as probability.
Since each fuzzy cluster is a probability distribution and the divergence of cluster cth in data sites ith and jth is defined as
follows:

X N
ucn½i
D U c½i jjU c½ j ¼ ucn½i ln ð33Þ
n¼1
ucn½ j

where U c½i 1N ¼ ucn½i represents membership degrees of cth cluster in ith data site and ucn½i denotes the belonging degree
of the nth object to the cth cluster in the ith data site. N is number of data objects.
In type-1 fuzzy clustering, the membership degree of nth data object to cth cluster for ith data site is determined by ucn½i .
_
In the general type-2 fuzzy clustering, both the primary and secondary membership degrees u cn½i and indicate the
membership degree nth data object to the cth cluster in ith data site. Therefore, both should be considered for the degree
of belonging to the cluster. In the proposed method, the degree of belonging is limited as follows:
ð34Þ
The degree of divergence obtained for the two clusters according to Eq. (33) shows their degree of divergence. The consis-
tency between the two data sites is equal to the sum of the consistency between their clusters. Thus, we have:
X
C

RE½ijjj ¼ D U c½i jjU c½ j ð35Þ
c¼1
where RE½ijjj is the consistency of the ith data site with the jth data site according to the discovered data structures. C shows
the number of clusters. The consistency of the ith data site is equal to its sum of consistency with other data sites, which is
represented by RE½i and is calculated as follows:
X
S
RE½i ¼ RE½ijjj ð36Þ
j¼1;j–i
where S is the number of data sites of the dataset. Therefore, the global consistency is defined as the sum of consistency in
the discovered data structures for all data sites as follows:
X
S
RE ¼ RE½i
i¼1
ð37Þ
X
S X
S X
C X
N
u
¼ ucn½i log u cn½i
cn½ j
i¼1 j¼1;j–i c¼1 n¼1
The lower values of RE index indicate the better quality of collaboration since lower RE½ijjj leads to less divergence i.e.,
more consistency of the structure of data site ith with data site jth.
308
5. Experimental results
This section is divided into three subsections. The first is concerned with evaluating the collaboration quality and effect of
each algorithm using the existing indices and proposed index. The second subsection is related to determining the number of
collaboration iterations of each algorithm. Finally, robustness analysis of GT2-CFC is conducted by enforcing noise in the col-
laboration mechanism. The performance of the proposed algorithm is compared with the five well-known methods namely
collaborative fuzzy clustering (CFC) [27], Collaborative intelligent agents clustering (CIAC) [31], Partially collaborative fuzzy
c-means (PCFC) [43], Horizontal relative entropy collaborative fuzzy c-means (HRECFC) [47], and Interval type-2 horizontal
relative entropy collaborative clustering (IT2HRECFC) [48].
In addition, nine well-known real-world and benchmark datasets are selected for the experiments. Table 1 indicates the
characteristics of these datasets including the number of data objects, attributes, and clusters. These selected datasets are
Banknote Authentication, Breast Tissue, Glass Identification, Iris, User Knowledge Modeling, Leaf, Vertebral Column, Wine,
and Wine Quality (Red), which are available in UC Irvine (UCI) machine learning repository datasets [2].
5.1. Parameter settings
The following settings were created to evaluate the proposed algorithm and compare its performance with other algo-
rithms. The attributes of datasets were normalized to prevent the attributes with large numerical values from dominating
the distance-based objective functions. Standard score (Eq. (38)) and min–max feature scaling (Eq. (39)) methods were
selected for normalization, which are defined as follows:
vA vA
v 0A ¼ i
v v þ vA ð38Þ
i v A v A A A
vA lA
v 0A ¼ i
ð39Þ
i rA
where v Ai is ith the original value of the feature, A;
v 0A indicates ith the normalized value of the feature A. The minimum and
i

maximum values of the feature A are denoted with v A ¼ min v A ; v A ¼ max v A , respectively. v A and v
i i
A are the minimum
i i
and maximum ranges of normalized values, respectively. The normalized range of min–max feature scaling was considered
½0; 1 in all of the experiments. lA shows the mean of values of the feature A and rA is their standard deviation. Iris and Leaf
datasets were normalized with standard score method and the rest of the datasets were normalized with min–max feature
scaling method.
Table 2 reports the optimal values of the parameters in each algorithm for each horizontal distribution for each dataset.
The second column of this table is related to the distributions used for each dataset in which square brackets inside the
braces represent the data sites, and the numbers inside the square brackets show the used features in that data site. For
example, f½1; 3; 4; ½2g is a horizontal distribution with two data sites, the first of which consists of attribute 2 and the second
of which includes attributes 1; 3, and 4. Three different horizontal distributions of each dataset were provided without con-
sidering the common features to demonstrate the superiority of the proposed algorithm regardless of the horizontal distri-
bution type.
The grid search strategy combined with collaboration quality indices (Eqs. (40), (41)) was used to find the optimal param-
eters for each algorithm. Parameter ranges for grid search for the fuzzifier parameter were varied from 1:1 to 5 with an inter-
val of 0:1 and d with the values of 0:1; 0:01; 0:001, and 0:0001. Lower and upper membership functions for the IT2HRECFC
algorithm were determined empirically.
Algorithms were performed based on optimal parameters as shown in Table 2. All algorithms are executed in each data
site under the initial random membership degrees, along with termination condition 103 and the maximum iteration 100 to
the convergence of each data site. In addition, the termination condition 101 and the maximum iteration 100 are considered
for the reconciliation between the structures of data sites.
Table 1
Characteristics of the datasets.
Dataset #Instances #Attribute #Clusters

Banknote 1372 4 2
Breast Tissue 106 9 6
Glass 214 9 3
Iris 150 4 3
Knowledge 258 5 4
Leaf 340 15 30
Vertebral Column 617 6 4
Wine 178 13 3
Wine Quality (Red) 1599 11 6
309
Table 2
Optimal values of parameters for each algorithm on each dataset.
Dataset Distribution Algorithms

CFC CIAC PCFC HRECFC IT2HRECFC GT2-CFC
Banknote f½1; 3; 4; ½2g – m¼5 – m ¼ 2:1 m1 ¼ 1:5; m2 ¼ 4; m ¼ 3:1 m ¼ 2; d ¼ 102
f½1; 2; ½3; 4g – m ¼ 1:1 – m ¼ 4:8 m1 ¼ 1:5; m2 ¼ 5; m ¼ 2:3 m ¼ 2; d ¼ 103
f½2; 3; ½1; 4g – m ¼ 1:7 – m ¼ 3:6 m1 ¼ 2; m2 ¼ 5; m ¼ 2:2 m ¼ 2; d ¼ 102
Breast Tissue f½1; 2; 3; 4; 5; ½6; 7; 8; 9g – m ¼ 3:3 – m ¼ 2:4 m1 ¼ 1:5; m2 ¼ 4; m ¼ 5 m ¼ 2; d ¼ 102
f½1; 3; 5; 7; ½2; 4; 6; 8g – m¼5 – m ¼ 2:3 m1 ¼ 2; m2 ¼ 5; m ¼ 2:5 m ¼ 2; d ¼ 102
f½2; 3; 5; 7; ½1; 4; 6; 8; 9g – m ¼ 2:1 – m ¼ 1:8 m1 ¼ 1:5; m2 ¼ 4; m ¼ 1:3 m ¼ 2; d ¼ 102
Glass f½1; 2; 3; 4; 5; 6; ½7; 8; 9g – m ¼ 1:9 – m ¼ 1:7 m1 ¼ 2; m2 ¼ 5; m ¼ 2:4 m ¼ 2; d ¼ 102
f½5; 6; 3; 2; ½1; 4; 7; 8; 9g – m ¼ 2:9 – m ¼ 1:8 m1 ¼ 1:5; m2 ¼ 4; m ¼ 2:2 m ¼ 2; d ¼ 102
f½1; 4; 5; 6; ½2; 3; 7; 8; 9g – m ¼ 1:1 – m ¼ 1:9 m1 ¼ 1:5; m2 ¼ 4; m ¼ 3:2 m ¼ 2; d ¼ 102
Iris f½1; 4; ½2; 3g – m ¼ 1:2 – m ¼ 1:5 m1 ¼ 2; m2 ¼ 5; m ¼ 1:4 m ¼ 2; d ¼ 102
f½1; 2; ½3; 4g – m¼5 – m ¼ 1:6 m1 ¼ 1:5; m2 ¼ 4; m ¼ 1:4 m ¼ 2; d ¼ 103
f½2; 4; ½1; 3g – m¼5 – m ¼ 1:6 m1 ¼ 1:5; m2 ¼ 4; m ¼ 1:7 m ¼ 2; d ¼ 102
Knowledge f½1; 2; 3; ½4; 5g – m ¼ 4:8 – m ¼ 1:4 m1 ¼ 1:5; m2 ¼ 5; m ¼ 5 m ¼ 2; d ¼ 102
f½1; 4; 2; ½3; 5g – m ¼ 1:1 – m¼2 m1 ¼ 1:5; m2 ¼ 4; m ¼ 4:3 m ¼ 2; d ¼ 102
f½2; 3; 4; ½1; 5g – m ¼ 1:7 – m ¼ 1:5 m1 ¼ 1:5; m2 ¼ 4; m ¼ 5 m ¼ 2; d ¼ 102
Leaf f½1; 2; 3; 4; 5; 6; 7; ½8; 9; 10; 11; 12; 13; 14; 15g – m¼2 – m ¼ 2:1 m1 ¼ 2; m2 ¼ 5; m ¼ 2 m ¼ 2:2; d ¼ 103
f½1; 14; 2; 4; ½9; 11; 13; 15; 10; ½6; 3; 5; 7; 8; 12g – m ¼ 2:1 – m¼2 m1 ¼ 1:5; m2 ¼ 4; m ¼ 2 m ¼ 3:7; d ¼ 103
f½1; 2; 3; 4; ½5; 6; 7; 8; 9; 10; ½11; 12; 13; 14; 15g – m ¼ 1:1 – m ¼ 2:1 m1 ¼ 1:5; m2 ¼ 4; m ¼ 1:1 m ¼ 4; d ¼ 103
Vertebral f½1; 2; 3; ½4; 5; 6g – m ¼ 3:2 – m ¼ 1:7 m1 ¼ 2; m2 ¼ 5; m ¼ 1:1 m ¼ 2; d ¼ 102
Column
f½2; 3; 5; ½1; 4; 6g – m¼5 – m ¼ 1:1 m1 ¼ 1:5; m2 ¼ 4; m ¼ 1:1 m ¼ 2; d ¼ 102
f½3; 5; ½4; 6; ½1; 2g – m ¼ 2:5 – m ¼ 2:1 m1 ¼ 1:5; m2 ¼ 4; m ¼ 2 m ¼ 1:7; d ¼ 102
Wine f½1; 2; 3; 4; 5; 6; 7; ½8; 9; 10; 11; 12; 13g m¼2 m ¼ 2:7 m1 ¼ 2; m2 ¼ 5; m ¼ 2:1 m ¼ 2; d ¼ 103
f½2; 4; 6; 8; 10; 12; ½1; 3; 5; 7; 9; 11; 13g – m¼2 – m¼2 m1 ¼ 2; m2 ¼ 5; m ¼ 2 m ¼ 2; d ¼ 102
½1; 2; 3; 4; ½5; 6; 7; 8; 9; ½10; 11; 12; 13 – m¼2 – m¼3 m1 ¼ 1:5; m2 ¼ 4; m ¼ 2:3 m ¼ 2; d ¼ 102
Wine Quality f½1; 2; 3; 4; 5; ½6; 7; 8; 9; 10; 11g – m ¼ 2:1 – m ¼ 2:5 m1 ¼ 1:5; m2 ¼ 4; m ¼ 2:7 m ¼ 2; d ¼ 101
(Red)
f½1; 3; 5; 7; 9; 11; ½2; 4; 6; 8; 10g – m¼2 – m ¼ 3:1 m1 ¼ 1:5; m2 ¼ 4; m ¼ 2:1 m ¼ 2; d ¼ 103
f½1; 2; 8; 9; 11; ½3; 4; 5; 6; 7; 10g – m ¼ 2:3 – m¼2 m1 ¼ 2; m2 ¼ 5; m ¼ 2:1 m ¼ 2; d ¼ 102
5.2. Comparison of collaboration quality and effect
In this section, In this section, two main directions are concluded aimed at the evaluation of the collaboration-building
clustering environment. So, First, the quality of collaboration (global evaluation scheme) is investigated, and we examine
the collaboration effect (local evaluation scheme).
The quality of collaboration means the degree of consistency between the revealed structures of data sites using the rel-
evant indices and proposed index. The results of collaborative fuzzy clustering algorithms and the proposed algorithm are
evaluated by the four existing collaboration quality indices which are W prox (Eq. (40)) [47,30], W (Eq. (41)) [31,40], d (Eq.
(42)) [27], D (Eq. (43)) [27] and the proposed index RE (Eq. (37)). These indices are defined as follows:
N
XS
2 X S X N X

W prox ¼ 2
prox½i xn1 ; xn2 prox½ j xn1 ; xn2 ð40Þ
i¼1 N j¼1;j–in1 ¼1n2 >n1
X
S X
S X
C X
N 2
w¼ u2cn½ j xn½i v~ c½ijj ð41Þ
i¼1 j¼1;j–i c¼1 n¼1
1 X S X S X C X N
ucn½i ucn½ j
d¼ ð42Þ
SCN i¼1 j¼iþ1 c¼1 n¼1
1 X S X C X N
ucn½i ucn½source
D¼ ð43Þ
SCN i¼1 c¼1 n¼1 i
310
where S is number of data sites, C denotes number of clusters, N is number of data objects, and xn½i shows nth data in ith data
site. ucn½sourcei and ucn½i are the membership degree of nth data to cth cluster center prototype before and after collaboration,
P P
respectively. prox½i xn1 ; xn2 ¼ Cc¼1 min ucn1 ½i ; ucn2 ½i and prox½ j xn1 ; xn2 ¼ Cc¼1 min ucn1 ½ j ; ucn2 ½ j are the sum of the mini-
mum membership degrees of n1 th and n2 th data to the clusters in the ith data site and jth data site, respectively.
P P
v~ c½ijj ¼ Nn¼1 u2cn½ j xn½i = Nn¼1 u2cn½ j ; is induced from cluster center prototype which is calculated with data objects in ith
data site and membership degree of data objects in jth data site.
All of the mentioned indices and the proposed index show the level of the consistency at the global level for the discov-
ered structures. Thus, the lower value of these indices leads to better quality of collaboration. However, the D index shows
the level of the changes in the structures obtained after receiving the knowledge of other data sites. Clearly, this index is
better when its value is larger.
To evaluate the effect of collaboration, the amount of correctness and fitness of discovered data structures are assessed in
the data sites for this purpose. Validity indices considered for evaluation in this aspect are the partition coefficient (PC) and
classification entropy (CE) [23]. PC and CE for ith data site are defined as follows:
1X C X N
PC ½i ¼ u2 ð44Þ
N c¼1 n¼1 cn½i
1X C X N
CE½i ¼ ucn½i log ucn½i ð45Þ
N c¼1 n¼1
where ucn½i is the membership degree of nth data to cth cluter center in ith data site. The PC i and CEi are values of validity
indices related to the collaboration effect and the final PC and CE are their average in data sites. The better algorithm has a
larger value of PC and a lower value of CE.
As shown in Table 3, the collaboration quality of GT2-CFC based on W prox index is better than those of five collaborative
fuzzy clustering algorithms, i.e., CFC, CIAC, PCFC, HRECFC, and IT2HRECFC. Based on the W results of all algorithms in Table 4,
the W index value of GT2-CFC is smaller than other algorithms for all of the datasets except for Wine dataset in each of the
three distributions. Furthermore, Table 5 indicates the dominant superiority of the proposed algorithm based on RE collab-
oration quality index over other algorithms. Based on the results in Tables 6 and 7, the quality of collaboration in GT2-CFC
based on d and D is highly better than those other algorithms. Nevertheless, there are some cases in which GT2-CFC results in
lower clustering quality. For example, d index value of GT2-CFC has lower collaboration quality for Vertebral Column dataset
in the third distribution. Similar observations of both d and D indices are found for the Wine and Leaf datasets in the second
and third distribution. However, these cases are not considerable, and GT2-CFC often has better collaboration quality than
other relevant algorithms. The quality of algorithms varies depending on the dataset and type of distribution. Uncertainty
control improves the quality of collaboration fuzzy clustering compared to other algorithms. As observed in the results sum-
marized in Tables 3–7, the proposed algorithm provides an acceptable collaboration quality based on related indices in all
datasets. The values of PC and CE related to the collaboration effect indices are also reported in Tables 8 and 9, respectively.
The results confirm the efficiency of GT2-CFC. Bold fonts in all tables show the superiority of the algorithm rather than its
series of rival techniques in terms of index
Non-parametric statistical tests are employed to investigate the significant differences between the proposed and com-
parable algorithms. Given that the algorithms being compared are more than two algorithms, the selected statistical tests are
Friedman and Kruskal tests [38]. Friedman is a one-way analysis of variance by ranks and Kruskal–Wallis is a two-way anal-
ysis of variance by ranks. Table 10 shows the mean ranks obtained through the Friedman and Kruskal–Wallis tests. These
mean ranks are calculated based on collaboration quality indices as shown in Tables 3–7. As expected, the GT2-CFC algo-
rithm is first in mean rank, i.e., it has the lowest mean rank based on W prox ; W; RE; d, and CE indices and the highest mean
rank based on D and PC index. In the next mean ranks, algorithms are different depending on the evaluation index and even
type of test. For example, in the term of W prox index, HRECFC, and IT2HRECFC are the second mean rank in Friedman and
Kruskal–Wallis test, respectively. Similar observations are found for the d index.
Statistical hypothesis tests examine whether a null hypothesis (there is no significant difference between the algorithms)
is rejected or not. The null hypothesis is accepted as untrue (rejected) or true (retained) with a p-value of zero and one,
respectively.The transition point between rejecting and retaining the null hypothesis is made with a p-value between 0
and 1 and significance level. The significance level is considered as 0:05. Table 11 shows the results of examining the signif-
icant difference between the algorithms with two tests through five quality indices and two effect indices, separately. As
shown, there is the test statistic (chi-square) value with 5 degrees of freedom, and there is the asymptotic significance of
the test (p-value). This table reports a significant difference for the proposed algorithm with the algorithms compared
through Friedman test on the results obtained by algorithms based on collaboration quality and effect indices for datasets.
In other words, Friedman test rejects the null hypothesis with a 95% confidence level based on each of the validity indices on
the results obtained by algorithms. Further, Table 11 presents rejecting the null hypothesis with Kruskal–Wallis based on
W prox ; d; PC, and CE indices. In all of the cases where the null hypothesis is rejected, the p-value is very close to zero. There-
fore, due to the lower p-value than the significance level, the confidence level is stronger for rejecting the null hypothesis.
311
Table 3
Results of W prox index obtained by algorithms on three different horizontal distributions of real-world datasets.
Distribution Dataset Algorithms

D#1 Banknote 8:99E 02 1:06E 01 6:25E 02 8:64E 03 1:32E 03 4:20E 05
Breast Tissue 1:44E 02 1:68E 02 3:29E 02 2:59E 02 3:21E 03 6:66E 06
Glass 5:91E 02 1:65E 04 3:17E 02 1:58E 02 3:08E 03 1:16E 06
Iris 7:89E 03 1:57E 02 2:53E 03 3:99E 03 9:06E 02 9:55E 07
Knowledge 1:97E 02 9:76E 02 1:60E 01 4:64E 02 1:30E 03 3:54E 06
Leaf 1:20E 01 2:16E 01 1:07E 01 8:25E 02 5:62E 02 6:54E 03
Vertebral Column 6:48E 03 4:25E 02 6:06E 03 2:77E 03 5:56E 03 3:20E 06
Wine 1:44E 02 3:64E 02 2:19E 02 1:15E 02 1:67E 02 3:05E 06
Wine Quality (Red) 2:13E 01 1:19E 02 7:30E 02 8:64E 02 7:87E 02 1:78E 04
Glass 9:32E 02 9:27E 02 2:13E 02 3:74E 03 5:98E 03 7:19E 06
Iris 2:29E 02 8:76E 02 4:93E 02 1:05E 02 3:13E 02 5:47E 07
Leaf 1:80E 02 8:08E 02 5:16E 02 5:38E 02 1:64E 01 2:08E 04
Wine 5:15E 03 1:68E 02 7:51E 03 9:02E 03 1:97E 02 7:04E 06
Glass 4:91E 02 1:63E 05 3:19E 02 3:50E 02 8:66E 03 6:06E 06
Iris 6:41E 03 4:26E 02 2:85E 02 9:87E 03 4:30E 02 5:01E 06
Leaf 1:77E 01 2:88E 01 1:78E 01 1:21E 01 1:40E 01 2:74E 04
Wine 6:00E 02 1:94E 01 7:53E 02 9:95E 03 1:67E 02 6:29E 03
Table 4
Results of W index obtained by algorithms on three different horizontal distributions of real-world datasets.

D#1 Banknote 2:92E þ 02 2:67E þ 02 3:22E þ 02 3:15E þ 02 7:97E þ 02 1:64E þ 02
Breast Tissue 3:85E þ 01 3:14E þ 01 3:73E þ 01 6:71E þ 01 2:00E þ 02 1:04E þ 01
Glass 9:10E þ 01 1:19E þ 02 9:50E þ 01 4:66E þ 01 2:11E þ 02 3:60E þ 01
Iris 1:27E þ 02 1:49E þ 02 1:27E þ 02 1:29E þ 02 2:29E þ 02 1:22E þ 02
Knowledge 2:15E þ 02 2:68E þ 02 2:88E þ 02 1:00E þ 02 5:84E þ 02 7:27E þ 01
Leaf 2:92E þ 03 2:34E þ 03 3:14E þ 03 1:55E þ 03 2:92E þ 03 1:45E þ 03
Vertebral Column 3:69E þ 02 3:11E þ 02 3:80E þ 02 2:34E þ 02 5:29E þ 02 1:05E þ 02
Wine 7:71E þ 02 8:08E þ 02 7:75E þ 02 7:73E þ 02 8:79E þ 02 8:41E þ 02
Wine Quality (Red) 2:80E þ 03 4:45E þ 03 3:75E þ 03 5:79E þ 03 4:17E þ 03 1:26E þ 03
312
Table 5
Results of RE index obtained by algorithms on three different horizontal distributions of real-world datasets.

Glass 3:31E 01 3:39E 01 3:27E 01 3:02E 01 3:81E 01 3:00E 01
Iris 1:90E 01 1:99E 01 1:77E 01 1:19E 01 2:84E 01 1:14E 01
Leaf 5:62E 02 6:18E 02 5:73E 02 5:56E 02 5:53E 02 4:65E 02
Wine 2:57E 01 2:58E 01 2:60E 01 4:04E 01 2:65E 01 1:31E 01
Glass 3:39E 01 3:25E 01 3:24E 01 3:08E 01 3:76E 01 3:04E 01
Iris 1:76E 01 1:50E 01 1:50E 01 1:13E 01 1:63E 01 8:33E 02
Leaf 5:79E 02 6:20E 02 5:66E 02 5:34E 02 5:52E 02 5:47E 02
Wine 2:60E 01 2:60E 01 2:69E 01 4:05E 01 1:68E 01 1:36E 01
Glass 3:32E 01 4:44E 01 3:38E 01 3:15E 01 3:97E 01 3:03E 01
Iris 1:75E 01 1:68E 01 1:71E 01 1:26E 01 1:63E 01 1:10E 01
Leaf 5:98E 02 6:43E 02 5:97E 02 5:53E 02 5:54E 02 4:75E 02
Wine 2:37E 01 2:49E 01 2:54E 01 5:46E 02 5:37E 02 1:16E 01
Table 6
Results of d index obtained by algorithms on three different horizontal distributions of real-world datasets.

D#1 Banknote 2:07E þ 02 7:82E þ 02 1:60E þ 02 3:12E þ 00 8:08E 01 7:35E 04
Breast Tissue 2:94E þ 02 3:11E þ 02 1:44E þ 01 2:04E þ 02 8:80E 02 8:05E 06
Glass 2:83E þ 01 2:35E þ 02 3:45E þ 01 7:29E þ 00 3:57E þ 00 2:23E 06
Iris 2:18E þ 01 7:23E þ 01 1:62E þ 01 1:30E þ 01 3:94E þ 02 9:14E 06
Knowledge 3:65E þ 02 2:52E þ 02 2:04E þ 02 1:72E þ 01 6:17E 02 2:06E 05
Vertebral Column 9:91E þ 01 4:65E þ 02 1:05E þ 02 3:31E þ 01 1:33E þ 01 1:57E 05
Wine 6:15E þ 00 2:53E þ 01 7:38E þ 00 7:70E 01 2:86E þ 01 1:39E 06
D#2 Banknote 1:21E þ 02 1:02E þ 03 1:97E þ 02 5:54E þ 00 1:99E þ 00 9:52E 06
Breast Tissue 7:57E þ 01 4:15E þ 02 2:65E þ 02 2:81E þ 01 5:93E þ 01 1:08E 05
Glass 2:56E þ 02 4:16E þ 02 3:08E þ 01 7:04E þ 00 5:72E 01 1:50E 06
Knowledge 1:21E þ 02 3:10E 04 6:15E þ 01 1:86E þ 02 9:38E 02 2:34E 05
Vertebral Column 5:70E þ 02 6:74E þ 02 8:82E þ 01 2:22E 01 8:08E 02 2:56E 05
Wine 7:66E þ 00 3:59E þ 01 5:97E þ 00 1:07E þ 00 2:89E þ 01 3:68E 06
D#3 Banknote 6:40E þ 02 1:37E þ 03 1:31E þ 02 5:75E þ 00 8:68E 01 2:58E 05
Breast Tissue 2:50E þ 01 2:83E þ 02 2:84E þ 01 5:13E þ 01 3:14E þ 01 1:94E 05
Glass 2:76E þ 01 1:24E 03 2:77E þ 01 6:57E þ 00 1:39E þ 00 1:94E 06
Knowledge 1:63E þ 02 1:76E þ 02 6:65E þ 01 1:90E þ 01 1:39E 01 1:00E 03
313
Table 7
Results of D index obtained by algorithms on three different horizontal distributions of real-world datasets.

Table 8
Results of PC index obtained by algorithms on three different horizontal distributions of real-world datasets.

Glass 5:78E 01 5:66E 01 6:01E 01 7:14E 01 5:31E 01 8:09E 01
Iris 7:03E 01 6:86E 01 7:03E 01 8:88E 01 7:16E 01 8:93E 01
Leaf 1:61E 01 6:13E 02 1:22E 01 2:10E 01 2:26E 01 4:43E 01
Wine 4:85E 01 4:92E 01 4:81E 01 3:44E 01 5:29E 01 7:23E 01
Glass 5:76E 01 6:63E 01 5:97E 01 6:62E 01 4:12E 01 7:80E 01
Iris 6:93E 01 7:64E 01 7:16E 01 8:76E 01 7:96E 01 9:18E 01
Leaf 1:10E 01 5:57E 02 1:34E 01 2:67E 01 2:35E 01 3:02E 01
Wine 4:81E 01 4:94E 01 4:69E 01 3:45E 01 6:57E 01 7:13E 01
Glass 5:66E 01 3:33E 01 5:45E 01 6:11E 01 3:85E 01 7:76E 01
Iris 7:00E 01 7:38E 01 7:08E 01 8:63E 01 8:62E 01 8:98E 01
Leaf 9:62E 02 3:44E 02 8:98E 02 2:48E 01 2:70E 01 3:96E 01
Wine 5:17E 01 5:38E 01 4:89E 01 9:12E 01 9:19E 01 9:74E 01
314
Table 9
Results of CE index obtained by algorithms on three different horizontal distributions of real-world datasets.

Breast Tissue 1:00E þ 00 9:55E 01 1:01E þ 00 1:26E þ 00 1:79E þ 00 4:43E 01
Glass 7:12E 01 7:39E 01 6:83E 01 5:12E 01 7:66E 01 3:24E 01
Iris 5:29E 01 5:53E 01 5:31E 01 2:17E 01 4:50E 01 1:96E 01
Knowledge 1:02E þ 00 1:11E þ 00 1:13E þ 00 7:51E 01 1:39E þ 00 5:91E 01
Vertebral Column 1:03E þ 00 9:61E 01 1:03E þ 00 8:84E 01 1:13E þ 00 6:13E 01
Wine 8:80E 01 8:69E 01 8:87E 01 1:09E þ 00 7:89E 01 5:08E 01
Breast Tissue 1:02E þ 00 8:37E 01 9:81E 01 1:22E þ 00 1:39E þ 00 4:71E 01
Glass 7:26E 01 5:90E 01 7:01E 01 5:92E 01 9:84E 01 3:68E 01
Iris 5:48E 01 4:29E 01 5:08E 01 2:45E 01 3:26E 01 1:46E 01
Knowledge 1:16E þ 00 1:39E þ 00 1:14E þ 00 1:18E þ 00 1:39E þ 00 6:71E 01
Vertebral Column 9:46E 01 8:78E 01 9:92E 01 6:87E 01 1:07E þ 00 5:54E 01
Wine 8:85E 01 8:64E 01 9:02E 01 1:08E þ 00 6:12E 01 5:22E 01
Breast Tissue 1:07E þ 00 9:74E 01 1:09E þ 00 8:49E 01 5:32E 01 4:69E 01
Glass 7:45E 01 1:10E þ 00 7:78E 01 6:77E 01 1:01E þ 00 3:81E 01
Iris 5:35E 01 4:74E 01 5:21E 01 2:66E 01 2:30E 01 1:87E 01
Knowledge 1:02E þ 00 1:19E þ 00 1:18E þ 00 9:48E 01 1:39E þ 00 6:60E 01
Wine 8:32E 01 8:00E 01 8:73E 01 1:74E 01 1:54E 01 4:52E 02
Table 10
Mean rank of the algorithms based on W prox ; W; RE; d; D ,PC, and CE indices based on the results of Friedman and Kruskal–Wallis tests.
Test Algorithm Validity indices

Quality Effect
W prox W RE d D PC CE
Frideman CFC 3:44 3:44 3:78 3:96 2:52 3:07 3:85

CIAC 3:74 3:74 4:33 4:89 1:89 3:11 3:93
PCFC 3:93 3:93 4:00 4:15 3:22 2:52 4:56
HRECFC 3:26 3:26 3:37 3:07 4:00 3:37 3:70
IT2HRECFC 5:22 5:22 4:41 3:52 3:67 2:93 3:96
GT2-CFC 1:41 1:41 1:11 1:41 5:70 6:00 1:00
Kruskal–Wallis CFC 107:22 82:85 83:89 98:59 73:89 72:70 89:44
CIAC 99:33 85:33 86:89 112:37 70:44 73:22 88:85
PCFC 105:22 86:48 85:30 94:67 81:85 68:96 92:15
HRECFC 81:00 77:15 82:04 71:81 85:15 81:78 82:19
IT2HRECFC 78:74 97:63 89:00 73:19 81:52 71:26 88:44
GT2-CFC 17:48 59:56 61:89 38:37 96:15 121:07 47:93
Post hoc tests are a necessary part of the analysis of variance (ANOVA). When ANOVA is used to test the significant dif-
ferences of at least three groups mean, statistically significant results indicate that not all of the group means are equal. In
fact, at least two groups have significant differences. However, ANOVA results cannot specify which specific groups are sig-
nificantly different from each other. Therefore, post hoc tests are used to explore the differences between multiple groups
means while controlling the experiment-wise error rate. The post hoc test is employed according to results presented in
Table 11, which indicates the existence of a significant difference based on particular validity indices. As shown in Table 10,
the best rank in both tests is associated with GT2-CFC algorithm. Thus, it is selected as the control algorithm. In addition,
confidence interval and significance level are 95% and 0:05, respectively. Based on the results of the post hoc test related
to Friedman test in Table 12 the control algorithm (GT2-CFC) performs significantly better regarding all of the collaboration
quality and effect indices than the remaining algorithms with a significant level of 0:05. In addition, the results of the post
hoc test related to Kruskal–Wallis test in Table 13 confirm that the control algorithm is better than the other algorithms in
the W prox ; d; PC, and CE indices.
315
Table 11
Results of Friedman and Kruskal–Wallis tests based on validity indices.
Test Hypothesis Type of collaboration evaluation Index Test statistic P-value

Frideman Quality W prox 58:979 1:98E 11 Rejected H0
W 58:979 1:98E 11 Rejected H0
RE 58:386 2:62E 11 Rejected H0
d 54:958 1:33E 10 Rejected H0
D 67:656 3:15E 13 Rejected H0
Effect PC 60:884 7:98E 12 Rejected H0
CE 61:138 7:07E 12 Rejected H0
Kruskal–Wallis Quality W prox 69:309 1:43E 13 Rejected H0
W 9:84 7:99E 02 Retain H0
RE 6:016 3:05E 01 Retain H0
d 42:228 5:30E 08 Rejected H0
D 5:008 4:15E 01 Retain H0
Effect PC 24:222 1:97E 04 Rejected H0
CE 17:257 4:04E 03 Rejected H0
Note: The degree of freedom for both tests is 5.
Table 12
Results of post hoc test based on Friedman test (GT2-CFC is the control algorithm).
Type of collaboration evaluation Validity index Algorithm i a=i Test Statistic Std. Test Statistic P-value Hypothesis
Quality W prox HRECFC 1 0:05 1:852 3:637 2:76E 04 Reject

CFC 2 0:025 2:037 4:001 6:32E 05 Reject
CIAC 3 0:0167 2:333 4:583 4:59E 06 Reject
PCFC 4 0:0125 2:519 4:946 7:56E 07 Reject
IT2HRECFC 5 0:01 3:815 7:492 6:77E 14 Reject
W HRECFC 1 0:05 1:852 3:637 2:76E 04 Reject
CFC 2 0:025 2:037 4:001 6:32E 05 Reject
CIAC 3 0:0167 2:333 4:583 4:59E 06 Reject
PCFC 4 0:0125 2:519 4:946 7:56E 07 Reject
RE HRECFC 1 0:05 2:259 4:437 9:12E 06 Reject
CFC 2 0:025 2:667 5:237 1:63E 07 Reject
PCFC 3 0:0167 2:889 5:674 1:40E 08 Reject
CIAC 4 0:0125 3:222 6:328 2:48E 10 Reject
d HRECFC 1 0:05 1:667 3:273 1:06E 03 Reject
CFC 3 0:0167 2:556 5:019 5:19E 07 Reject
PCFC 4 0:0125 2:741 5:383 7:34E 08 Reject
CIAC 5 0:01 3:481 6:837 8:06E 12 Reject
D CIAC 1 0:05 3:815 7:492 6:77E 14 Reject
CFC 2 0:025 3:185 6:256 3:96E 10 Reject
PCFC 3 0:0167 2:481 4:874 1:10E 06 Reject
HRECFC 5 0:01 1:704 3:346 8:20E 04 Reject
Effect PC PCFC 1 0:05 3:481 6:837 8:06E 12 Reject
CFC 3 0:0167 2:926 5:746 9:12E 09 Reject
CIAC 4 0:0125 2:889 5:674 1:40E 08 Reject
HRECFC 5 0:01 2:630 5:164 2:41E 07 Reject
CE HRECFC 1 0:05 2:704 5:310 1:10E 07 Reject
CFC 2 0:025 2:852 5:601 2:13E 08 Reject
CIAC 3 0:0167 2:926 5:746 9:12E 09 Reject
PCFC 5 0:01 3:556 6:983 2:89E 12 Reject
Note: Std Error is 0.509.
5.3. Comparison of computational complexity
In this section, the computational time of algorithms is assessed through Average Iteration Number (AIN). The results
reported in Table 14 clearly show that the proposed algorithm has fewer iterations than other algorithms. The number of
iterations indicates the number of reconciliations between the structures obtained in the data sites in the collaborative stage.
Thus, the value is directly related to the complexities. In other words, the fewer iterations of the collaboration lead to less
316
Table 13
Results of post hoc test based on Kruskal–Wallis test (GT2-CFC is the control algorithm).
Type of collaboration evaluation Validity index Algorithm i a=i Test Statistic Std. Test P-value Hypothesis
Quality W prox IT2HRECFC 1 0:05 61:259 4:798 0:000 Reject

HRECFC 2 0:025 63:519 4:975 0:000 Reject
CIAC 3 0:0167 81:852 6:411 0:000 Reject
PCFC 4 0:0125 87:741 6:872 0:000 Reject
CFC 5 0:01 89:741 7:029 0:000 Reject
W – – – – – – –
RE – – – – – – –
d HRECFC 1 0:05 33:444 2:620 0:009 Reject
IT2HRECFC 2 0:025 34:815 2:727 0:006 Reject
PCFC 3 0:0167 56:296 4:409 0:000 Reject
CFC 4 0:0125 60:222 4:717 0:000 Reject
CIAC 5 0:01 74:000 5:796 0:000 Reject
D – – – – – – –
Effect PC PCFC 1 0:05 52:111 4:082 0:000 Reject
IT2HRECFC 2 0:025 49:815 3:902 0:000 Reject
CFC 3 0:0167 48:370 3:789 0:000 Reject
CIAC 4 0:0125 47:852 3:748 0:000 Reject
HRECFC 5 0:01 39:296 3:078 0:002 Reject
CE HRECFC 1 0:05 34:259 2:683 0:007 Reject
IT2HRECFC 2 0:025 40:519 3:174 0:002 Reject
CIAC 3 0:0167 40:926 3:206 0:001 Reject
CFC 4 0:0125 41:519 3:252 0:001 Reject
PCFC 5 0:01 44:222 3:464 0:001 Reject
Note: Std Error is 12.767.
Table 14
Iteration number obtained by algorithms on three different horizontal distributions of real-world datasets.

D#1 Banknote 5 5 8 30 100 4
Breast Tissue 8 13 7 47 100 5
Glass 9 4 9 100 100 3
Iris 3 3 6 55 65 3
Knowledge 3 7 7 100 100 3
Leaf 100 14 100 100 100 3
Vertebral Column 19 100 31 100 100 3
Wine 4 3 5 100 100 3
Wine Quality (Red) 8 5 100 100 100 4
D#2 Banknote 7 4 8 51 100 3
Breast Tissue 4 4 20 100 100 3
Glass 6 3 8 100 100 3
Iris 4 3 5 51 80 3
Knowledge 9 5 8 100 100 3
Leaf 100 24 100 100 100 4
Wine 4 3 5 100 100 3
Wine Quality (Red) 22 13 27 100 100 5
D#3 Banknote 4 3 20 45 100 3
Breast Tissue 12 3 12 100 100 4
Glass 6 10 8 100 100 3
Iris 4 3 6 67 87 3
Knowledge 3 4 10 100 100 3
Leaf 100 9 100 100 100 5
Wine 6 3 7 100 100 3
Wine Quality (Red) 10 10 89 100 100 5
time and communication complexity, and vice versa. The direct relationship is especially significant in communication com-
plexity. CFC, CIAC, PCFC, HRECFC algorithms, and the proposed algorithm communicate asymptotically in OðST R CN Þ, and
IT2HRECFC algorithm has communication complexity equal to Oð2ST R CNÞ. The proposed algorithm is run in

O ST r T ‘ C 2 NSF þ CNS2 F time, where S = number of data sites, N = number of data objects, F = number of the dimension
317
of the data, C = number of clusters, T r = number of iterations of reconciliation between structures, T ‘½i = number of conver-
gence iterations of the ith data site, and F ½i = number of dimensions of the ith data. Since the number of iterations to reach
convergence in data sites are not the same. The data objects are described in different feature spaces in horizontal distribu-
S
tion. Thus, the number of features is not necessarily equal in all of the data sites. In fact, we have T ‘ ¼ max T ‘½i , and
i¼1
F¼ maxSi¼1 F ½i for calculating computational complexity with the asymptotic notation of big O.
5.4. Evaluation of controlling uncertainty
Clustering algorithms are commonly subject to various sources of uncertainty. This section aims to investigate controlling
the uncertainty related to external knowledge and to make comparisons with CFC, CIAC, PCFC, HRECFC, and IT2HRECFC algo-
rithms. In this investigation, knowledge containing noise is applied on the synthetic dataset. As a result, a synthetic dataset is
generated to compare the behavior of the clustering methods dealing with the noises involved in knowledge. A four-
dimensional dataset with 300 data objects is considered. The horizontal distribution of the dataset is considered
f½1; 2; ½3; 4g. Both data sites are generated based on the multivariate normal distribution. The parameters applied for gen-
erating these two data sites are as follows:

0:5 0
X ½1 : r1½1 ¼ ; l1½1 ¼ ½3; 1:5r2½1 ¼ r1½1 ; l2½1 ¼ ½3; 4:5r3½1 ¼ r1½1 ; l3½1 ¼ ½6; 4:5
0 0:5

4 1 2 0 3 0
X ½2 : r1½2 ¼ ; l1½2 ¼ ½2; 2r2½2 ¼ ; l2½2 ¼ ½1; 2r3½2 ¼ ; l3½2 ¼ ½4; 7
1 1 0 2 0 1
where X is the synthetic dataset of 4-dimensional, X ½i is ith data site. 100 data objects belonging to cth cluster center pro-
totype in ith data site are generated based on the multivariate normal distribution with mean value lc½i and variance rc½i .
The standard score method is used for normaliz features in the range ½0; 1. These two data sites are demonstrated in
Fig. 2. The initial value of local membership memberships for all algorithms was generated by the FCM algorithm with a
fuzzifier value of 2. The collaboration quality indices (Eq. (40) and Eq. (41)) were used to find the optimal parameters for
each algorithm through the grid search strategy. Parameter ranges in grid search for the fuzzifier parameter were changed
from 1:1 to 5 with an interval of 0:1 and value of 3; 2, and 2 were gained for CFC, HRECFC, and GT2-CFC respectively. The
optimal value of the d parameter for the GT2-CFC algorithm was 0:01. This value was found between the values tested
0:1; 0:01; 0:001, and 0:0001. Lower and upper membership functions for the IT2HRECFC algorithm were determined empir-
ically with m1 ¼ 1:5 and m2 ¼ 4.
Noise is added to the external knowledge at each stage of collaboration in collaborative fuzzy clustering algorithms. We
randomly selected 30% of the data objects whose knowledge should become noisy. These data objects are shown in Fig. 3
with the black star mark. Information granular level is in the form of partition matrices for the proposed and comparison
algorithms in the collaboration mechanism. As shown in the previous section, the number of convergence iterations in some
algorithms such as HRECFC and IT2HRECFC is high. Thus, only the data objects knowledge specified in Fig. 3 are involved
with noise at each stage of the collaboration, which is accomplished for a better comparison of algorithms.
Fig. 4 presents the collaborative fuzzy clustering results obtained by the algorithms in the first data site. The results of the
second data site obtained through clustering methods are shown in Fig. 5. For the sake of clarity, we use a black star to mark
Fig. 2. Four-dimensional dataset horizontally distributed to two-dimensional data sites (a) X ½1 , and (b) X ½2 .
318
Fig. 3. Dataset with selected data objects for collaboration noise in (a) X ½1 , and (b) X ½2 .
Fig. 4. The results of the clustering on the first data site (X ½1 ) of the synthetic dataset obtained by (a) CFC, (b) CIAC, (c) PCFC, (d) HRECFC, (e) IT2HRECFC, (f)
GT2-CFC methods.
319
the data objects which could not find their true cluster due to their knowledge contained noise. The simulation results show
the out-performance of GT2-CFC in terms of robustness by enforcing collaboration noise, compared to the state-of-the-art
collaborative fuzzy clustering methods. As shown in Figs. 4 and 5, the HRECFC and IT2HRECFC algorithms are extremely
weak against this noise, and even many of the data objects are distorted due to noise. PCFC is a partial collaborative fuzzy
clustering algorithm; thus, all knowledge does not participate in the collaboration. In this experiment, 90% of the knowledge
participate leading to less noise. So the PCFC algorithm performs better than other algorithms after the proposed algorithm.
The reason for noise robustness is related to the general type-2 membership degrees in which the primary membership
degrees are obtained without communicated knowledge. In other words, membership degrees are not directly influenced
by the knowledge gained.
However, we evaluate the performance of GT2-CFC method by adding noise in the collaboration mechanism and the
obtained W prox ; W; RE; d; D ,PC; CE, and AIN values are compared with the results obtained from CFC, CIAC, PCFC, HRECFC,
and IT2HRECFC methods. The results are reported in Table 15. Similar to previous results for real-world datasets, the pro-
posed method is the most effective one in all cases.
6. Conclusions and future work
Clustering algorithms, especially distributed data mining algorithms, are often exposed to different sources of uncer-
tainty, which must be well managed. In this paper, the general type-2 collaborative fuzzy clustering algorithm was suggested
to consider the uncertainties related to membership degrees with and without external knowledge. Modeling uncertainties
related to membership degrees using general type-2 fuzzy sets have more power and capability compared to type-1 fuzzy
sets and interval type-2 fuzzy sets. In the literature, type-1 or interval type-2 fuzzy clustering methods were investigated due
to the computational complexity of general type-2 fuzzy sets. However, in the proposed method, no information was lost
Fig. 5. The results of the clustering on the second data site (X ½2 ) of the synthetic dataset obtained by (a) CFC, (b) CIAC, (c) PCFC, (d) HRECFC, (e) IT2HRECFC,
(f) GT2-CFC methods.
320
Table 15
Results obtained by algorithms on synthetic dataset.
Algorithm Validity indices

Quality Effect AIN
W prox W RE d D PC CE
CFC 1:72E 02 6:93E þ 01 1:95E 01 2:08E þ 02 2:37E þ 02 7:24E 01 5:11E 01 3

CIAC 2:08E 02 9:23E þ 01 2:01E 01 2:11E þ 02 1:37E þ 02 6:19E 01 6:77E 01 4
PCFC 2:46E 02 7:66E þ 01 1:84E 01 2:24E þ 02 1:44E þ 02 6:74E 01 5:86E 01 5
HRECFC 8:03E 01 8:77E þ 01 2:21E 01 2:31E þ 02 1:27E þ 02 6:44E 01 6:42E 01 88
IT2HRECFC 1:54E 01 2:72E þ 02 3:54E 01 2:53E þ 02 6:27E þ 01 4:36E 01 9:42E 01 100
GT2-CFC 5:27E 03 2:62E þ 01 8:17E 02 2:11E þ 02 1:82E þ 02 8:52E 01 2:77E 01 3
since no type reduction and defuzzification were observed during the clustering process and related updates, leading to bet-
ter uncertainty modeling and less time complexity. Further, controlling the uncertainty using fuzzy mathematics require fur-
ther studies. In the proposed objective function, in addition to having a closed-form solution, the fuzzifier parameter was not
limited to 2, and the strength of collaboration was regularly determined. Several well-known real-world datasets were pro-
vided with different dimensions, the number of the data, and number of different clusters along with varying types of hor-
izontal distributions. The suggested validity index and other indices were used to evaluate the performance of the proposed
algorithm. The experimental results indicated the superiority and efficiency of the proposed method compared to other type-
1 and interval type-2 collaborative fuzzy clustering algorithms.
The proposed method can be extended for segmenting multimodal remote sensing image for future work. Collaborative
fuzzy clustering and modeling uncertainties can result in discovering the common underlying structures of these images,
and quality segmentation. We will consider a deep learning feature extraction to improve the time complexity.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.
References
[1] S. Askari, N. Montazerin, M.F. Zarandi, E. Hakimi, Generalized entropy based possibilistic fuzzy c-means for clustering noisy data and its convergence
proof, Neurocomputing 219 (2017) 186–202.
[2] K. Bache, M. Lichman, UCI machine learning repository: Data sets, School of Information and Computer Science, University of California, 2013.
[3] O. Castillo, P. Melin, W. Pedrycz, Design of interval type-2 fuzzy models through optimal granularity allocation, Applied Soft Computing 11 (8) (2011)
5590–5601.
[4] C. Chen, D. Wu, J.M. Garibaldi, R.I. John, J. Twycross, J.M. Mendel, A comprehensive study of the efficiency of type-reduction algorithms, IEEE
Transactions on Fuzzy Systems (2020).
[5] L.F. Coletta, L. Vendramin, E.R. Hruschka, R.J. Campello, W. Pedrycz, Collaborative fuzzy clustering algorithms: Some refinements and design guidelines,
IEEE Transactions on Fuzzy Systems 20 (3) (2011) 444–462.
[6] T.H. Dang, L.T. Ngo, W. Pedrycz, Interval type-2 fuzzy c-means approach to collaborative clustering, in: 2015 IEEE International Conference on Fuzzy
Systems (FUZZ-IEEE), IEEE, 2015, pp. 1–7.
[7] T.H. Dang, L.T. Ngo, W. Pedrycz, Multiple kernel based collaborative fuzzy clustering algorithm, in: Asian Conference on Intelligent Information and
Database Systems, Springer, 2016, pp 585–594. .
[8] T.H. Dang, D.S. Mai, L.T. Ngo, Multiple kernel collaborative fuzzy clustering algorithm with weighted super-pixels for satellite image land-cover
classification, Engineering Applications of Artificial Intelligence 85 (2019) 85–98.
[9] B. Depaire, R. Falcón, K. Vanhoof, G. Wets, Pso driven collaborative clustering: A clustering algorithm for ubiquitous environments, Intelligent Data
Analysis 15 (1) (2011) 49–68.
[10] R. Falcon, G. Jeon, R. Bello, J. Jeong, Learning collaboration links in a collaborative fuzzy clustering environment, in: Mexican International Conference
on Artificial Intelligence, Springer, 2007, pp. 483–495.
[11] R. Falcón, B. Depaire, K. Vanhoof, A. Abraham, Towards a suitable reconciliation of the findings in collaborative fuzzy clustering, in: 2008 Eighth
International Conference on Intelligent Systems Design and Applications, IEEE, vol 3, 2008, pp. 652–657. .
[12] F.V. Farahani, A. Ahmadi, M.H.F. Zarandi, Hybrid intelligent approach for diagnosis of the lung nodule from ct images using spatial kernelized fuzzy c-
means and ensemble learning, Mathematics and Computers in Simulation 149 (2018) 48–68.
[13] S.M.M. Golsefid, M.F. Zarandi, I. Turksen, Multi-central general type-2 fuzzy clustering approach for pattern recognitions, Information Sciences 328
(2016) 172–188.
[14] D. Graves, W. Pedrycz, Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study, Fuzzy Sets and Systems 161 (4) (2010)
522–543.
[15] G. Hu, Z. Du, Adaptive kernel-based fuzzy c-means clustering with spatial constraints for image segmentation, International Journal of Pattern
Recognition and Artificial Intelligence 33 (01) (2019) 1954003.
[16] C. Hwang, F.C.H. Rhee, Uncertain fuzzy clustering: Interval type-2 fuzzy approach to c-means, IEEE Transactions on Fuzzy Systems 15 (1) (2007) 107–
120.
[17] N.N. Karnik, J.M. Mendel, Centroid of a type-2 fuzzy set, Information Sciences 132 (1–4) (2001) 195–220.
[18] V. Loia, W. Pedrycz, S. Senatore, P-FCM: a proximity-based fuzzy clustering for user-centered web applications, International Journal of Approximate
Reasoning 34 (2–3) (2003) 121–144.
[19] H. Mashayekhi, Collaborative fuzzy clustering of distributed concept-drifting dynamic data using a gossip-based approach, Applied Intelligence 48 (12)
(2018) 4905–4922.
[20] P. Melin, E. Ontiveros-Robles, C.I. Gonzalez, J.R. Castro, O. Castillo, An approach for parameterized shadowed type-2 fuzzy membership functions
applied in control applications, Soft Computing 23 (11) (2019) 3887–3901.
321
[21] S. Mitra, H. Banka, W. Pedrycz, Rough–fuzzy collaborative clustering, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 36 (4)
(2006) 795–805.
[22] J.E. Moreno, M.A. Sanchez, O. Mendoza, A. Rodriguez-Diaz, O. Castillo, P. Melin, J.R. Castro, Design of an interval type-2 fuzzy model with justifiable
uncertainty, Information Sciences 513 (2020) 206–221.
[23] L.T. Ngo, T.H. Dang, W. Pedrycz, Towards interval-valued fuzzy set-based collaborative fuzzy clustering algorithms, Pattern Recognition 81 (2018) 404–
416.
[24] E. Ontiveros, P. Melin, O. Castillo, High order a-planes integration: a new approach to computational cost reduction of general type-2 fuzzy systems,
Engineering Applications of Artificial Intelligence 74 (2018) 186–197.
[25] E. Ontiveros, P. Melin, O. Castillo, Comparative study of interval type-2 and general type-2 fuzzy systems in medical diagnosis, Information Sciences
(2020).
[26] W. Pedrycz, Algorithms of fuzzy clustering with partial supervision, Pattern Recognition Letters 3 (1) (1985) 13–20.
[27] W. Pedrycz, Collaborative fuzzy clustering, Pattern Recognition Letters 23 (14) (2002) 1675–1686.
[28] W. Pedrycz, Knowledge-based Clustering: From Data to Information Granules, John Wiley & Sons, 2005.
[29] W. Pedrycz, K. Hirota, A consensus-driven fuzzy clustering, Pattern Recognition Letters 29 (9) (2008) 1333–1343.
[30] W. Pedrycz, P. Rai, Collaborative clustering with the use of fuzzy c-means and its quantification, Fuzzy Sets and Systems 159 (18) (2008) 2399–2427.
[31] W. Pedrycz, P. Rai, A multifaceted perspective at data analysis: A study in collaborative intelligent agents, IEEE Transactions on Systems, Man, and
Cybernetics, Part B (Cybernetics) 39 (4) (2009) 834–844.
[32] W. Pedrycz, J. Waletzky, Fuzzy clustering with partial supervision, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 27 (5)
(1997) 787–795.
[33] A. Pourabdollah, J.M. Mendel, R.I. John, Alpha-cut representation used for defuzzification in rule-based systems, Fuzzy Sets and Systems (2020).
[34] M. Prasad, L. Siana, D.L. Li, C.T. Lin, Y. Liu, A. Saxena, A preprocessed induced partition matrix based collaborative fuzzy clustering for data analysis, in:
2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), IEEE, 2014, pp. 1553–1558.
[35] E. Rubio, O. Castillo, F. Valdez, P. Melin, C.I. Gonzalez, G. Martinez, An extension of the fuzzy possibilistic clustering algorithm using type-2 fuzzy logic
techniques, Advances in Fuzzy Systems (2017).
[36] G. Ruiz-García, H. Hagras, H. Pomares, I.R. Ruiz, Toward a fuzzy logic system based on general forms of interval type-2 fuzzy sets, IEEE Transactions on
Fuzzy Systems 27 (12) (2019) 2381–2395.
[37] A. Sakalli, T. Kumbasar, J.M. Mendel, Towards systematic design of general type-2 fuzzy logic controllers: Analysis, interpretation and tuning, IEEE
Transactions on Fuzzy Systems (2020).
[38] F. Salehi, M.R. Keyvanpour, A. Sharifi, SMKFC-ER: Semi-supervised multiple kernel fuzzy clustering based on entropy and relative entropy, Information
Sciences, 2020.
[39] M. Savargiv, B. Masoumi, M.R. Keyvanpour, A new ensemble learning method based on learning automata, Journal of Ambient Intelligence and
Humanized Computing (2020) 1–16.
[40] Y. Shen, W. Pedrycz, Collaborative fuzzy clustering algorithm: some refinements, International Journal of Approximate Reasoning 86 (2017) 41–61.
[41] D. Wu, J.M. Mendel, Recommendations on designing practical interval type-2 fuzzy systems, Engineering Applications of Artificial Intelligence 85
(2019) 182–193.
[42] F. Yu, J. Tang, F. Wu, Q. Sun, Auto-weighted horizontal collaboration fuzzy clustering, in: Fuzzy Information and Engineering, Springer, 2007, pp 592–
600. .
[43] F. Yu, J. Yu, J. Tang, The model of generalized partially horizontal collaborative fuzzy c-means, in: 2009 Chinese Control and Decision Conference, IEEE,
2009, pp. 6095–6099.
[44] S. Yu, F. Yu, Incorporating prototypes into horizontal collaborative fuzzy c-means, in: 2010 Chinese Control and Decision Conference, IEEE, 2010, pp.
3612–3616.
[45] M. Zarinbal, M.F. Zarandi, I. Turksen, Interval type-2 relative entropy fuzzy c-means clustering, Information Sciences 272 (2014) 49–72.
[46] M. Zarinbal, M.F. Zarandi, I. Turksen, Relative entropy fuzzy c-means clustering, Information Sciences 260 (2014) 74–97.
[47] M. Zarinbal, M.F. Zarandi, I. Turksen, Relative entropy collaborative fuzzy clustering method, Pattern Recognition 48 (3) (2015) 933–940.
[48] M. Zarinbal, M.F. Zarandi, I. Turksen, M. Izadi, A type-2 fuzzy image processing expert system for diagnosing brain tumors, Journal of Medical Systems
39 (10) (2015) 110.
[49] S. Zeng, X. Wang, H. Cui, C. Zheng, D. Feng, A unified collaborative multikernel fuzzy clustering for multiview data, IEEE Transactions on Fuzzy Systems
26 (3) (2017) 1671–1687.
[50] J. Zhou, C.P. Chen, L. Chen, H.X. Li, A collaborative fuzzy clustering algorithm in distributed network environments, IEEE Transactions on Fuzzy Systems
22 (6) (2013) 1443–1456.
322

1 s2.0 S0020025521007271 Main

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1 s2.0 S0020025521007271 Main

Uploaded by

Copyright:

Available Formats

Information Sciences 578 (2021) 297–322

Contents lists available at ScienceDirect

GT2-CFC: General type-2 collaborative fuzzy clustering method

2.1. Type-1 collaborative fuzzy clustering

2.2. Type-2 collaborative fuzzy clustering

3. Proposed general type-2 collaborative fuzzy clustering method

3.1. Knowledge uncertainty

3.2. Proposed collaborative fuzzy clustering method

The constraints for Eq. (7) are as follows:

The four first order necessary conditions (FONC) are as follows:

By replacing Eq. (16) in Eq. (15), we obtain:

Based on the second FONC, the following equation is obtained:

k00n½i is calculated with constraint term . Thus, we have:

The following equation is obtained by substituting Eq. (20) in Eq. (19):

@ 2 J 0GT2CFC ½i d1

Algorithm 1. GT2-CFC: General Type-2 Collaborative Fuzzy Clustering

3.3. Theoretical analyses of GT2-CFC

4. Proposed validity index

4.1. Collaboration quality validity indices

4.2. Relative entropy divergence measure

In the above definition, the following conventions exist:

0 def 0 def P def

4.3. Relative entropy (RE) validity index

5.1. Parameter settings

Dataset #Instances #Attribute #Clusters

Dataset Distribution Algorithms

5.2. Comparison of collaboration quality and effect

Distribution Dataset Algorithms

Distribution Dataset Algorithms

Distribution Dataset Algorithms

Distribution Dataset Algorithms

Distribution Dataset Algorithms

Distribution Dataset Algorithms

Distribution Dataset Algorithms

Test Algorithm Validity indices

Frideman CFC 3:44 3:44 3:78 3:96 2:52 3:07 3:85

Test Hypothesis Type of collaboration evaluation Index Test statistic P-value

Note: The degree of freedom for both tests is 5.

Quality W prox HRECFC 1 0:05 1:852 3:637 2:76E 04 Reject

Note: Std Error is 0.509.

5.3. Comparison of computational complexity

Quality W prox IT2HRECFC 1 0:05 61:259 4:798 0:000 Reject

Note: Std Error is 12.767.

Distribution Dataset Algorithms

5.4. Evaluation of controlling uncertainty

6. Conclusions and future work

Algorithm Validity indices

CFC 1:72E 02 6:93E þ 01 1:95E 01 2:08E þ 02 2:37E þ 02 7:24E 01 5:11E 01 3

Declaration of Competing Interest

You might also like