Professional Documents
Culture Documents
1 s2.0 S0020025520306423 Main
1 s2.0 S0020025520306423 Main
Information Sciences
journal homepage: www.elsevier.com/locate/ins
a r t i c l e i n f o a b s t r a c t
Article history: The edge minimum sum-of-squares clustering problem (E-MSSC) aims at finding p proto-
Received 4 April 2019 types such that the sum of squares of distances from a set of vertices to their closest pro-
Received in revised form 7 April 2020 totype is minimized, where a prototype is either a vertex or an interior point of an edge.
Accepted 27 June 2020
This paper proposes a highly effective memetic algorithm for E-MSSC that combines a ded-
Available online 7 July 2020
icated crossover operator for solution generation with a reformulation local search for
solution improvement. Reformulation local search is a recent algorithmic framework for
Keywords:
continuous location problems that is based on an alternation between original (continu-
Heuristics
Clustering problems
ous) and discrete formulations of a given problem. Furthermore, the proposed algorithm
Memetic algorithm uses a simple strategy for population updating, and a mutation operator to prevent from
Reformulation local search premature convergence. The computational results obtained on three sets of 132 bench-
mark instances show that the proposed algorithm competes very favorably with the exist-
ing state-of-the-art algorithms in terms of both solution quality and computational
efficiency. We also analyze several essential components of the proposed algorithm to
understand their contribution to the algorithm’s performance.
Ó 2020 Published by Elsevier Inc.
1. Introduction
Cluster analysis, which consists in grouping a collection of patterns into clusters based on similarity, is a powerful tool in
many fields such as pattern recognition [5,28], machine learning [3] and image processing [38]. The simplest form of clus-
tering is partition based clustering that aims at partitioning a given data set into disjoint clusters (subsets), such that some
specific clustering criteria are optimized. Among many criteria encountered in cluster analysis, the most well-known is the
Minimum Sum-of-Squares Clustering (MSSC) [14]. Assuming that the entities are given as points in the N-dimensional space
RN , MSSC consists in finding p points, called prototypes, in such a way that the sum of squares of the Euclidean distances
between the entities and their nearest prototypes is minimized. While MSSC is shown to be an NP-hard problem [1], exact
algorithms can be used to solve it on small to medium-sized problem instances [2]. Due to the complexity of the problem,
heuristics and metaheuristics are necessary [6,15,16,23,24,40] for large problem instances.
Recently, Carrizosa et al. [13,14] introduced two Minimum Sum-of-Squares Clustering (MSSC) problems arising in net-
works, namely the Vertex-MSSC (V-MSSC) and the Edge-MSSC (E-MSSC). Given a connected weighted undirected network
G ¼ ðV; EÞ, where V and E are the set of vertices and edges respectively, V-MSSC seeks to determine p prototypes from V such
that the weighted sum of the squared distances from the vertices to their closest prototype is minimized. According to [14],
⇑ Corresponding author.
E-mail addresses: qingzhouqz03@gmail.com (Q. Zhou), una@satalia.com (U. Benlic), qinghuawu1005@gmail.com (Q. Wu).
https://doi.org/10.1016/j.ins.2020.06.056
0020-0255/Ó 2020 Published by Elsevier Inc.
272 Q. Zhou et al. / Information Sciences 541 (2020) 271–296
V-MSSC is very similar to the classic p-median problem and both exact and heuristic algorithms [4,22] can be successfully used
to solve V-MSSC. E-MSSC is similar to V-MSSC, with the distinction that prototypes are sought along the edges [26]. Meanwhile,
the authors also proposed several efficient variable neighborhood search algorithms for the two problems [14]. While these
heuristics apply the same K-means method for local improvement, they make use of different strategies to generate initial solu-
tions. Extensive experiments show that the VNS-based heuristic (denoted as VNS-1 throughout this paper), which employs a
sophisticated strategy for generating its starting solution, considerably outperforms the VNS variant that starts with a random
initial solution. In [34], Nikolaev et al. proposed two additional VNS-based heuristics for E-MSSC, denoted as VNS-2 and VNS-3
in our work, which are based on two different variants of Variable Neighborhood Descent (VND). The three VNS-based heuris-
tics (VNS-1, VNS-2, VNS-3) are considered as the current state-of-the-art approaches for E-MSSC.
In this work, we are interested in the E-MSSC problem. We propose a highly effective population-based memetic algo-
rithm (MA) to enrich the set of solution methods for solving this computationally challenging problem. The main contribu-
tions of this work can be summarized as follows:
From the algorithmic perspective, the proposed algorithm integrates several original ingredients, including a dedicated
crossover operator and a reformulation local search procedure for local improvement. Motivated by the observation that
high-quality local optima share common building blocks, the crossover preserves the components (prototypes) that are
present in both parents. The reformulation local search is an iterative algorithm that alternates between a K-means algo-
rithm and tabu search. While K-means performs the search in the continuous space and can easily become trapped in a
local optimum, tabu search is applied to further refine the local optimum by exploring a finite set of points from the con-
tinuous space. Through an iterative application of these two complementary search procedures, the algorithm is able to
explore various zones of the search space without being easily confined to a local optimum.
From the computational perspective, the proposed algorithm MA achieves highly competitive performances in terms of
both solution quality and computational efficiency on three sets of 132 benchmark instances, including 40 OR-Lib
instances, 80 TSP-Lib instances and 12 network graphs from the University of Florida Sparse Matrix Collection.
The rest of the paper is organized as follows. Section 2 gives a mathematical formulation of E-MSSC, followed by a
description of some essential properties of E-MSSC in Section 3. Section 4 describes in detail the main elements of the pro-
posed memetic algorithm. Computational results and comparisons with several state-of-the-art algorithms are given in Sec-
tion 5. Before concluding, Section 6 studies some key components of the proposed algorithm.
E-MSSC can be formally described as follows. Let G ¼ ðV; EÞ be a connected undirected network, where
V ¼ fv 1 ; v 2 ; . . . ; v n g is the set of n vertices and E is the edge set with a positive length ljk for each edge ðj; kÞ. We consider
the network G to be a spatial network: for any edge e ¼ ðj; kÞ 2 E and any x 2 ½0; 1, the pair ðe; xÞ is identified by a point
on edge e at a distance x ljk from j, and at a distance ð1 xÞ ljk from k. Let PðGÞ denote the set of all the pairs ðe; xÞ,
i.e., the set of points of the network G. Then the E-MSSC problem is to determine p prototypes from the set of points
PðGÞ on edges, such that the weighted sum of squared distances from the vertices to their closest prototype is minimized
[14]:
P
v 2V wv d ðv ; fx1 ; . . . ; xp gÞ
2
min
ð1Þ
s:t: x1 ; . . . ; xp 2 PðGÞ
where positive weights wv are given for each v 2 V. The distance between two points u and v is calculated as the shortest
path dðu; v Þ connecting u and v. The distance dðv ; XÞ between v and a non-empty finite set X of points in the network is
defined as the distance to the closest point in X, i.e., dðv ; fx1 ; . . . ; xp gÞ ¼ min dðv ; xi Þ.
16i6p
The V-MSSC problem is analogous to E-MSSC, where prototypes can only be selected from the vertex set
V ¼ fv 1 ; v 2 ; . . . ; v n g. A solution to E-MSSC can be denoted as fx1 ; . . . ; xp g, where x1 ,. . .,xp are the p prototypes. Given the
set of p prototypes, the p clusters can be easily determined by assigning each vertex v 2 V to its closest prototype.
Fig. 1 provides an example of E-MSSC. Consider a graph G ¼ ðV; EÞ with V ¼ fv 1 ; v 2 ; v 3 ; v 4 ; v 5 g and
E ¼ fðv 1 ; v 2 Þ; ðv 2 ; v 3 Þ; ðv 3 ; v 4 Þ; ðv 4 ; v 5 Þg. The lengths of the edges ðv 1 ; v 2 Þ; ðv 2 ; v 3 Þ; ðv 3 ; v 4 Þ; ðv 4 ; v 5 Þ are respectively equal to
5, 4, 1 and 3. In this example, we assume that the number of clusters p ¼ 2 and that the weights of all the vertices are equal
to 1. The top left side of the figure illustrates a feasible solution s ¼ fx1 ; x2 g with clusters C 1 ¼ fv 1 ; v 2 g and C 2 ¼ fv 3 ; v 4 ; v 5 g,
induced by points x1 ¼ fðv 1 ; v 2 Þ; 0:5g and x2 ¼ fðv 4 ; v 5 Þ; 0:2g. The data for calculating the objective value f ðsÞ is displayed on
the right side of the figure.
E-MSSC can be formulated as a Mixed-Integer Program [34] with the following decision variables:
Variables
– xj : Binary variable, where xj ¼ 1 if and only if vertex j is selected to be a prototype, and xj ¼ 0 otherwise.
– zij : Binary variable, where zij ¼ 1 if and only if vertex i is assigned to the vertex prototype j, and zij ¼ 0 otherwise.
Q. Zhou et al. / Information Sciences 541 (2020) 271–296 273
– yjk : Binary variable, where yjk ¼ 1 if and only if one of the prototypes lies on edge ðj; kÞ, and yjk ¼ 0 otherwise.
– cjk : Continuous variable which defines the location of the prototype on edge ðj; kÞ with a distance ljk cjk from vertex j and
with a distance ljk ð1 cjk Þ from vertex k.
– mijk1 : Binary variable, where mijk1 ¼ 1 if and only if a cluster with a prototype on edge ðj; kÞ contains vertex i such that the
shortest path (denoted as nijk1 ) between i and this prototype passes through vertex j.
- mijk2 : Binary variable, where mijk2 ¼ 1 if and only if vertex i is assigned to a prototype on edge ðj; kÞ such that the shortest
path (denoted by nijk2 ) between i and this prototype passes through vertex k.
Given the above decision variables, E-MSSC can be formulated as the following Mixed-Integer Program [34]:
Xn Xn 2
Xn X X2
minimizef ðsÞ ¼ i¼1 j¼1
½dði; jÞ zij þ i¼1 ðj;kÞ2E
n2 m
s¼1 ijks ijks
ð2Þ
Xn X X2
subject to j¼1 zij þ ðj;kÞ2E
m ¼ 1;
s¼1 ijks
i 2 f1; . . . ; ng ð3Þ
where the objective of E-MSSC is expressed as (2), which is equivalent to the function defined in Eq. (1). Constraint (3)
guarantees that each vertex is located in exactly one cluster, while constraints (4) and (5) ensure that each vertex is only
274 Q. Zhou et al. / Information Sciences 541 (2020) 271–296
allocated to the selected prototype. Constraint (6) guarantees that the number of clusters is exactly equal to p. Finally, con-
straints (7) and (8) define the shortest path between vertex i and the prototype on edge ðj; kÞ that passes through vertex j and
vertex k respectively. The binary values for variables zij ; xj ; yjk and mijks are imposed in constraints (9–12), while constraint
(13) enforces a continuous range ½0; 1 for variable cjk .
3. E-MSSC properties
The E-MSSC problem has two important properties that motivate the design choices of the algorithm described in the
next section. The proofs of these properties were presented in [14], and can be formally described as follows.
Property 1. Let e ¼ ðj; kÞ 2 E, at most one optimal prototype x lies on the interior of e. In that case, both endpoints j and k have x
as its nearest prototype.
Property 2. If an optimal prototype x is located at a vertex v 2 V, then there cannot be an optimal prototype on the interior of
any of the edges adjacent to v.
Memetic algorithm has shown to be an effective framework for tackling combinatorial optimization problems
[25,27,30,31] as it provides a good trade-off between search intensification and diversification by means of a combined
use of a local optimization procedure and a crossover operator. Motivated by an observation that high-quality local
optima share a large number of common prototypes, which can be viewed as building blocks of E-MSSC (refer to Sec-
tion 6.4), we design a dedicated crossover to preserve the components (prototypes) that are present in both parents.
The main scheme of the proposed MA algorithm is given in Algorithm 1. Starting with an initial population (line 3), each
iteration of MA (lines 5 to 14) first proceeds with a random selection of two solutions (i.e., parents, see line 6) that are
used as input to a crossover operator that generates an offspring solution (line 7). Additionally, a mutation operator is
further applied to the offspring solution to prevent from premature convergence (line 8). Following crossover and muta-
tion, the offspring is improved with a reformulation local search method (line 9). The best found solution S is updated
each time an improved solution has been found by the reformulation local search (lines 10 to 12). Finally, a population
updating strategy is applied to update the population (line 13). Before presenting a more detailed description of the main
MA components, we first discuss the search space explored by the MA algorithm. As E-MSSC consists in selecting p pro-
totypes from PðGÞ, the explored search space X contains all the possible subsets of PðGÞ of cardinality p, i.e.,
X ¼ fS PðGÞ : jSj ¼ pg. Given a selection of p prototypes, the p clusters can easily be obtained by assigning each vertex
v 2 V to its closest prototype.
Q. Zhou et al. / Information Sciences 541 (2020) 271–296 275
Each solution from the initial population is obtained with a random construction algorithm (see Algorithm 2) from
[14,34], which ensures satisfaction of Properties 1 and 2. Given the set of previously selected prototypes, a point is feasible
provided that Properties 1 and 2 are still satisfied. We say that an edge is feasible if it contains feasible points.
Algorithm 2 is an iterative procedure, where each iteration consists of the following steps. First, a feasible edge e is chosen
at random from the set of feasible edges, followed by the selection of a feasible point y on e from the set of feasible points.
The selected point acts as a prototype that extends the partial solution X 0 . Once prototype y has been selected, we next dis-
card from future consideration all the points and edges which are in conflict with the selected prototype according to Prop-
erties 1 and 2. More precisely, if y is an interior point of e, we discard from future consideration all the points corresponding
to e that are in conflict with Property 1. If y is an endpoint of an edge, we eliminate from future consideration y and all the
interior points of each edge adjacent to y so as to satisfy Property 2. Finally, we remove from the set of feasible edges all the
edges that contain no feasible point. This process is repeated until p points are obtained.
The solution created by Algorithm 2 is then improved with the local optimization procedure described in Section 4.5. It is
then inserted into the population if and only if it is not identical to any existing solution in the population. This procedure
leads to an initial population of relatively high quality. It stops as soon as the number of solutions in the population reaches
jpj (jpj is the population size).
A crossover operator is the key diversification mechanism of a population-based algorithm. It is well known that an effec-
tive crossover operator should not only generate diversified solutions but also transfer meaningful components from parents
to offspring [25,39]. In case of E-MSSC, an analysis conducted on a sample of locally optimal solutions disclosed that high
quality solutions share a large number of common prototypes, which might constitute building blocks of an optimal solution
(refer to Section 6.4). Given two parent solutions P 1 ¼ fx11 ; x12 ; . . . ; x1p g and P 2 ¼ fx21 ; x22 ; . . . ; x2p g, the proposed crossover for E-
MSSC thus aims to preserve common prototypes shared by both P1 and P2 . More precisely, the first step of the crossover is to
simply copy over to offspring S0 the subset of prototypes shared in both parents, i.e., S0 ¼ P 1 \ P2 . Given the partial offspring
solution S0 , a randomized procedure is then applied to extend S0 to a complete solution containing exactly p prototypes. Each
iteration of the procedure consists in randomly selecting and extending S0 with a point in PðGÞ that satisfies both Properties 1
and 2.
Given a graph consisting of 7 vertices (v 1 ; v 2 ; . . . ; v 7 ) and 6 edges (e1 ; e2 ; . . . ; e6 ), Fig. 2 illustrates the crossover procedure
on two parents P1 ¼ fv 1 ; v 3 ; v 0ðe4 ;0:2Þ g and P 2 ¼ fv 1 ; v 6 ; v 0ðe4 ;0:2Þ g, where fv 0ðe4 ;0:2Þ g represents an interior point on e4 at a distance
0:2 le4 from the first-endpoint of e4 , i.e., vertex v 4 . In this example, we assume that the number of clusters p ¼ 3. In the
276 Q. Zhou et al. / Information Sciences 541 (2020) 271–296
first step, the common prototypes shared by parents P 1 and P2 (i.e., fv 1 g and fv 0ðe4 ;0:2Þ g) are transferred to the partial offspring
S0 . In the second step, a feasible point (i.e., fv 0ðe5 ;0:6Þ g) is selected at random to obtain a complete solution S0 .
Once an offspring So is obtained with the crossover procedure, a random mutation operator is applied to So consisting of l
random SwapMove moves, where l is a parameter called the mutation strength. A SwapMove move is performed by ran-
domly selecting a point from PðGÞ and then exchanging this point with a randomly chosen prototype in So , such that Prop-
erties 1 and 2 remain satisfied.
When a new offspring solution is generated with the crossover operator, it is first improved by the reformulation local
search procedure (see Section 4.5) and and then inserted into the population if: (i) it is better than the worst individual
in the population in terms of solution cost; and (ii) there is no individual in the population identical to the offspring solution.
If both criteria are satisfied, offspring replaces the worst individual of the population, and is discarded otherwise. One poten-
tial risk of this classic updating strategy is premature convergence as So is added in the population regardless of its similarity
(see Section 6.4) to other individuals of the population. The proposed algorithm uses a random mutation operator as
described in Section 4.3 to avoid premature convergence by providing a diversified offspring solution. Therefore, the pro-
posed algorithm is not sensitive to the population updating strategy.
Local search is another key component to our memetic algorithm that plays an essential role in exploring the search
space. The fast K-means method has been frequently used as a local search method in many clustering applications. As a
K-means algorithm can easily become trapped in a local optimum just as a descent-based local search [23], it is frequently
embedded into a variable neighborhood search framework [14,34]. In this work, we propose an effective reformulation local
search (RLS) for E-MSSC, which has been successfully applied to a wide variety of continuous optimization problems [10,11].
The main idea behind RLS is to exploit the relation between the continuous model and its discrete counterpart. More pre-
cisely, the K-means method is first carried out in the continuous space until a local optimum is reached, followed by a search
in the discretized space of the continuous model in an attempt to find an improved solution. The alternation between the
two problem formulations continues until no further improvement can be achieved. Furthermore, new potential prototypes
produced in the continuous space are added to the discrete space in order to improve the quality of the discrete approxima-
tion. For search exploration in the discrete space, RLS employs a tabu search approach that has shown to be highly effective
Q. Zhou et al. / Information Sciences 541 (2020) 271–296 277
for difficult discrete optimization problems [21]. Let FS denote a finite set of points and let X be a subset of p points from FS.
The discrete formulation explored by our tabu search is then given by:
minXFS f ðXÞ ð14Þ
Before the beginning of the MA search, the finite set FS is initialized with V ¼ fv 1 ; . . . ; v n g as recommended in [10]. During
the reformulation local search, new prototypes found in the continuous space are being added to FS each time a K-means
method attains a local optimum. If the size of FS reaches a threshold value w (w is a parameter), FS is re-initialized to
V ¼ fv 1 ; . . . ; v n g so as to maintain the discrete model at a manageable size.
The general procedure of RLS is summarized in Algorithm 3. Each iteration alternates between the K-means and the tabu
search approach (lines 5–12), where K-means is applied first to identify a local optimum sc in the continuous space and to
expand FS with the newly found prototypes in sc (line 7). This is followed by the improvement of sc with the tabu search
procedure to obtain a new starting point for the next round of the K-means local search. The search stops as soon as no
improvement is achieved with the tabu search, and returns the best solution found during this process.
Additional experimental results demonstrate that the combined use of the discrete and the continuous problem formu-
lations constitutes an effective hybridization that plays an important role for the RLS performance (Section 6.1). Detailed
descriptions of K-means and the tabu search are given in the following subsections.
Note that by applying these two compound move operators, the number of prototypes in the current solution always
remains equal to p.
The general procedure of our tabu search is given in Algorithm 4. At each iteration, the choice between the two compound
move operators is made in a random manner. Each basic non-tabu move (ADD and DROP) is selected and performed such that
the corresponding move gain value is minimized (see Section 4.5.3). Each time a point is added to the current solution with
the ADD operator, it is forbidden to remove it with the DROP operator for the next tt iterations to prevent from short-term
cycling, where tt is a parameter called the tabu tenure. TS terminates after maxIter iterations (maxIter is a parameter), and
returns the best solution found during the search.
278 Q. Zhou et al. / Information Sciences 541 (2020) 271–296
Given a solution S, each time an ADD move is performed by adding a point pi R S from the finite set FS into the current
solution S, the set of vertices (denoted as Aðpi Þ) which are closer to pi than to their nearest prototype will be reassigned to pi .
The resulting move gain value is computed as:
X X
Df ¼ j2Aðpi Þ
ðdðpi ; jÞ dðpi ; jÞÞ j2Aðpi Þ
ðDS½j½0 DS½j½0Þ ð16Þ
The matrix DS is initialized before each call of the TS procedure in Oðn p 2Þ time, and is updated by recalculating DS in
Oðn p 2Þ after each basic move. For the DROP and the ADD move, the best move can be respectively identified in OðnÞ and
Oðn jFSjÞ with the help of the matrix DS. Therefore, the time complexity of each tabu search iteration is bounded by
Oðn p 2 þ n þ n jFSjÞ.
5. Computational experiments
To evaluate the effectiveness of the proposed algorithm, we carry out extensive experiments on well known benchmarks
and make comparisons with three state-of-the-art E-MSSC algorithms from the literature: VNS-1 [14], VNS-2 [34] and VNS-3
[34]. To the best of our knowledge, these are the current best performing approaches for the E-MSSC problem. In addition, we
also make comparisons between MA and two effective heuristics (GRASP [37] and HYBRID [37]), which were originally
developed for the classic p-median problem.
Q. Zhou et al. / Information Sciences 541 (2020) 271–296 279
OR-Lib: This set was introduced by Beasley [8] and includes 40 p-median instances. The size of these instances ranges
from 100 to 900 vertices and from 200 to 16200 edges, while the number of clusters is in the ranges from 5 to 200.
TSP-Lib: The set was originally designed for the traveling salesman problem [36]. Many instances in TSP-Lib have been
used before for the classic p-median problem [20,22]. Following Fast and Hicks [20], we consider 20 instances from this
set with sizes ranging from 400 to 3038 vertices and from 4922 to 282004 edges. For each instance four different values of
p (number of clusters) are used, i.e., p 2 f10; 50; 100; 200g, thus giving a total of 80 TSP-Lib instances.
University of Florida Sparse Matrix (UFSM) Collection [17]: This set is a widely used set of sparse matrix benchmark
collected from a variety of real applications. According to our knowledge, we are the first to use it for evaluation of an
E-MSSC approach. Three network graphs (named USAir97, Harvard500 and Email respectively) are considered with the
number of vertices ranging from 332 to 1133. For each graph, four values of p (p 2 f10; 50; 100; 200g) were used, giving
a total of 12 graphs.
Table 1 provides details for the three data sets. Column ‘Instance’ indicates the instance name. Columns ‘n’ and ‘m’ respec-
tively refer to the number of vertices and the number of edges of the graph. All the input files are available in the Mendeley’s
public repository1 to facilitate future research.
The MA search is controlled by means of 5 parameters: the population size jpj, the search depth of tabu search maxIter, the
mutation strength l, the tabu tenure tt and the size w of the finite set of points considered during tabu search. To tune these
parameters, we use the ‘irace’ software package [29] that implements the iterated racing method [7,9] for finding the best
configuration from a finite set of possible parameter configurations. For the tuning process, we select the 20 largest OR-Lib
instances (i.e., instances pmed21-40), and set the tuning budget to 1000 executions each limited to 60 s of run-time. The
tested and the final parameter values (suggested by irace) are shown in Table 2. For all the experiments conducted in this
paper, we use the same setting of parameters determined by irace. The proposed algorithm was programmed in C++ lan-
guage2, running on a computer with a 2.10 GHz processor (Xeon-E5 2695) and 2 GB RAM, under Windows operating system.
This section provides an extensive computational comparison of MA with several state-of-the-art algorithms. According
to the previous studies [14,34], three heuristics based on the Variable Neighborhood Search (VNS) framework (denoted as
VNS-1, VNS-2 and VNS-3) are considered as the state-of-the-art algorithms for the E-MSSC problem. In addition, two efficient
heuristics GRASP and HYBRID [37] are also considered for the comparative study. Both GRASP and HYBRID are state-of-the-
art algorithms developed for the classic p-median problem, which is similar to E-MSSC. The main difference between the p-
median problem and E-MSSC is that the p-median problem seeks for the p prototypes from the set of vertices only, while E-
MSSC considers not only the vertices but also the points along the edges. As the main references for comparisons, we thus
use these five algorithms, whose source codes were kindly provided by the authors. All the algorithms in our experimental
studies were programmed in CnC++ and compiled using g++ compiler without any option flag. For the five reference algo-
rithms and MA, we followed the same experimental protocol as described in [34], i.e., each algorithm is executed 10 times
per instance with the time limit t max set to 60 s.
Tables 3–5 summarize the statistical results for all the compared algorithms on the OR-Lib benchmark, the TSP-Lib bench-
mark and the UFSM benchmark respectively. Row ‘#Best/Avg’ shows the number of cases for which an algorithm produces
the best result among all the compared algorithms in terms of the best/average objective value. Row ‘Average g best /g av g (%)’
indicates the average percentage gap of the best/average objective value to the best result obtained within this experiment,
while the average running time in seconds required by each algorithm to reach its best objective value across all the tested
instances is given in row ‘Average time(s)’. To verify whether there is a significant statistical difference between MA and the
reference algorithms in terms of the best and the average objective value, the p-values from the non-parametric Friedman
test [19] are provided in rows ‘p-valuebest’ and ‘p-valueavg’. The best performance is indicated in bold. Detailed results for OR-
Lib, TSP-Lib and UFSM benchmarks are respectively presented in Tables 20–22 of the Appendix.
In terms of the best objective value, we observe from Table 3 that VNS-1, VNS-2, VNS-3, GRASP, HYBRID and MA respec-
tively produce the best result in 14, 21, 25, 7, 7 and 39 cases out of the 40 OR-Lib instances. When considering the average
objective value, MA yields the best result on 39 instances, while VNS-1, VNS-2, VNS-3, GRASP and HYBRID respectively give
the best result on 3, 19, 20, 7 and 7 instances. On this bechmark, MA further reports the smallest average percentage gap of
1
https://data.mendeley.com/datasets/5sk2ps2d23/1.
2
the source code of our algorithm is going to be made available for future research.
280 Q. Zhou et al. / Information Sciences 541 (2020) 271–296
Table 1
List of benchmarks.
Table 2
Setting of parameters.
Table 3
Statistical results for MA and five state-of-the-art algorithms on the OR-Lib instances with a time limit of 60 s.
Table 4
Statistical results for MA and five state-of-the-art algorithms on the TSP-Lib instances with a time limit of 60 s.
the best/average objective value to the best solution (0.00%/0.13%). The non-parametric Friedman tests (rows p-valuebest and
p-valueavg) reveal that there exists a statistically significant difference between MA and all the reference algorithms in both
the best and the average performance (p-value 60.05). Table 6 presents the results of the Wilcoxon signed ranks test
Q. Zhou et al. / Information Sciences 541 (2020) 271–296 281
Table 5
Statistical results for MA and five state-of-the-art algorithms on the University of Florida Sparse Matrix instances with a time limit of 60 s.
Table 6
Results of the Wilcoxon signed ranks test for MA and the reference algorithms on the OR-Lib instances with a time limit of 60 s.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
Table 7
Results of the Wilcoxon signed ranks test for MA and the reference algorithms for the TSP-Lib instances with a time limit of 60 s.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
Table 8
Results of the Wilcoxon signed ranks test for MA and the reference algorithms on the University of Florida Sparse Matrix instances with a time limit of 60 s.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
Table 9
Statistical results for MA and five state-of-the-art algorithms on the OR-Lib instances with a time limit of 600 s.
Table 10
Statistical results for MA and five state-of-the-art algorithms on the TSP-Lib instances with a time limit of 600 s.
Table 11
Statistical results for MA and five state-of-the-art algorithms on the University of Florida Sparse Matrix instances with a time limit of 600 s.
Table 12
Results of the Wilcoxon signed ranks test for MA and the reference algorithms on the OR-Lib instances with a time limit of 600 s.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
Table 13
Results of the Wilcoxon signed ranks test for MA and the reference algorithms on the TSP-Lib instances with a time limit of 600 s.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
Table 14
Results of the Wilcoxon signed ranks test for MA and the reference algorithms on the University of Florida Sparse Matrix instances with a time limit of 600 s.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
6. Analysis
In this section, we study some key elements of the proposed MA algorithm to understand their contribution to the MA’s
performance, including the benefit of using reformulation local search, the role of the memetic framework and the influence
of the mutation operator.
The main objective of this section is to investigate the effectiveness of the combined use of the K-means method and the
tabu search (TS) within the reformulation local search (RLS) framework. For this purpose, we perform experiments with RLS
and its two underlying components (i.e., the K-means method (Section 4.5.1) and and the tabu search method (Section 4.5.2)
on a selection of 20 representative large OR-Lib instances. For each instance and approach, we perform 10 independent runs
with the time limit set to 60 s. Due to a fast convergence of K-means and for a relatively fair comparison, all the three meth-
ods are performed in a multi-start way, until the end of the run-time limit is reached, with a randomly generated solution for
each restart. Specifically, the TS method is restarted every 100 iterations, K-means is restarted as soon as convergence is
reached, while RLS restarts once its two underlying components arrive to the same solution.
To compare the performances of the three algorithms, we show in Fig. 3 the boxplot graphs which represent, for each
algorithm, the distribution of the obtained results for the 20 selected instances. These results are expressed in the form of
percentage gaps, computed as: y ¼ ðf AVG f BK Þ=f BK 100, where f BK is the best-known solution reported in the literature
Table 15
Comparisons between multi-start versions of RLS, K-means and TS. The best performance is indicated in bold. The average computing time to reach the final
objective value is in seconds.
pmed21 5 202777.00 202777.00 0.0 237038.00 254410.13 28.5 202777.00 202777.00 0.0
pmed22 10 183329.00 183329.00 0.6 238380.36 245311.65 28.1 183329.00 183329.00 1.5
pmed23 50 62794.58 62847.91 35.1 88601.75 91061.68 33.9 63855.04 64202.22 19.5
pmed24 400 27068.65 27153.62 23.8 40575.68 42034.85 25.1 31786.41 32410.12 35.6
pmed25 167 11995.44 12128.71 42.1 18372.05 19069.63 30.1 19777.38 20702.09 38.1
pmed26 5 194461.00 194461.00 0.2 225027.00 242234.73 28.9 194461.00 194461.00 0.2
pmed27 10 144800.55 144800.70 23.9 183159.85 195214.82 34.3 144802.00 144802.00 0.8
pmed28 60 49970.32 50054.23 21.5 71397.27 74072.05 23.5 51389.91 51539.12 18.3
pmed29 120 24689.11 24842.67 24.2 36988.73 37803.62 24.4 30585.08 30994.26 38.6
pmed30 200 11303.19 11380.08 23.7 16886.28 17845.97 32.0 21165.92 21918.85 33.2
pmed31 5 169806.00 169806.00 0.1 205572.00 223944.33 34.5 169806.00 169806.00 0.1
pmed32 10 154078.00 154078.00 1.1 202279.50 209706.33 15.5 154078.00 154078.00 1.0
pmed33 70 45351.54 45456.13 40.8 68836.89 70202.72 22.6 47652.12 47999.98 31.3
pmed34 140 20436.60 20651.06 23.1 31496.93 32080.91 33.1 27741.06 28566.51 34.6
pmed35 5 157540.00 157540.00 0.3 193846.00 206456.34 26.1 157540.00 157540.00 0.3
pmed36 10 150918.00 150918.00 1.3 200516.95 210999.53 40.1 150918.00 150918.00 0.5
pmed37 80 46789.40 46952.93 24.5 71429.60 73384.82 26.3 50455.34 50870.03 17.1
pmed38 5 157956.74 157956.75 40.7 200485.00 214890.93 39.7 157972.00 157972.00 0.6
pmed39 10 123424.94 123429.19 15.5 158556.74 171186.28 23.9 123425.00 123435.00 11.5
pmed40 90 41675.55 41811.86 30.2 65339.54 66248.96 29.6 46253.78 46760.92 25.7
#Best/Avg 20 20 0 0 7 7
p-value 7.74e-6 7.74e-6 3.11e-4 1.00e-3
Average 99058.28 99118.74 18.63 127739.31 134908.01 29.00 101488.50 101753.61 15.43
Table 16
Wilcoxon signed ranks test results between RLS and its two underlying components, with a level of significance a ¼ 0:05.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
Table 17
Comparisons of MA with a multi-start version of RLS. The best performance is marked in bold. The average computing time required to find the final objective
value is in seconds.
Instance p MA RLS
f best f av g t av g f best f av g t av g
Table 18
Wilcoxon signed ranks test results between MA and RLS with a level of significance a ¼ 0:05.
Comparison Rþ
best
R
best p-value Rþ
avg
R
avg p-value
and f AVG is the average objective value over 10 independent runs. The performance of an algorithm improves as its corre-
sponding percentage gaps approach zero. Fig. 3 indicates that RLS is greatly superior to both TS and K-means. Furthermore,
the percentage gaps obtained with RLS are very close to 0, implying that RLS is highly competitive compared to the best per-
forming algorithms in the literature.
Detailed comparison results are given in Tables 15 and 16. Column ‘p’ in Table 15 refers to the number of clusters while
columns ‘f best ’ and ‘f av g ’ present the best and average objective value over 10 independent runs for each compared algorithm.
In column ‘tav g ’, we report the average computing time in seconds required to find the final objective value. Row ‘Average’
shows the averaged value across all the runs and tested instances. We observe that RLS, K-means and TS respectively pro-
duce the best results on 20, 0, and 7 instances in terms of both the best and the average objective value. The p-values from
the non-parametric Friedman test reveal a significant difference between RLS and its two underlying approaches both in
terms of the best and the average performance, while the Wilcoxon signed ranks test in Table 16 indicates that RLS outper-
forms the two compared algorithms with a 0.05 significance level.
As shown in Section 6.1, our reformulation local search (RLS) method (see Section 4.5) appears to be highly effective for
the E-MSSC problem. We next evaluate if the integration of RLS within a memetic framework can further improve on the
Fig. 4. Percentage gap between the best-known and the average results obtained by the two versions of MA with and without the mutation operator.
Table 19
Analysis of structural similarity between high-quality solutions for 10 large E-MSSC instances.
search capacity. For this purpose, we use the same experimental protocol as described in Section 6.1 and report the results of
this study in Tables 17 and 18 for the 20 selected instances. In terms of the best objective value, MA and RLS respectively
yield the best result on 20 and 10 instances. MA reaches a better average objective value in 13 cases, while the average
results of RLS are either equal to or worse than that of MA in all the 20 cases. Moreover, the non-parametric Friedman test
reveals a significant difference between MA and RLS in terms of both the best and the average performance with p-values of
0.007 and 0.001 respectively. The Wilcoxon signed ranks test in Table 18 further discloses the advantage of MA with a 0.05
significance level. This experiment indicates that the memetic element of our algorithm has an important contribution to the
overall algorithm’s performance.
As described in Section 4.3, a random mutation operator is applied to prevent from premature convergence. To evaluate
the importance of the mutation operator, Fig. 4 provides comparative results of the two MA versions with and without the
mutation operator on the selection of 20 large OR-Lib instances (pmed21-40). For each instance and algorithm, the y-axis
shows the percentage gap of the average results to the best-known solutions. These results clearly reveal the importance
of the mutation operator.
To explain the reason behind the efficiency of the proposed crossover, this section provides an analysis of structural sim-
ilarity between local optima of different qualities, in terms of the percentage of commonly shared prototypes. The analysis is
a commonly used and effective tool for justifying the use of the recombination operator [32]. Given two local optima sa and
a b
sb , their similarity is defined as the proportion of the commonly shared prototypes: simðsa ; sb Þ ¼ js js\s j
aj .
This analysis is performed on a random selection of 10 large OR-Lib instances. For each instance, we produce 1000 local
optima of different qualities using MA and its underlying reformulation local search with different time limits. We then take
the top 10% (100) local optima with the smallest objective values to represent the set of ‘high-quality solutions’, and the bot-
tom 10% (100) with the largest objective values to form the set of ‘low-quality solutions’.
Table 19 summarizes the average degree of similarity between the 100 high-quality solutions (column ‘shq ’), the average
degree of similarity between all the 1000 sampled local optima (column ‘sall ’) and the average degree of similarity between
the 100 low-quality solutions (column ‘slo ’). We observe that the average degree of similarity between high-quality solutions
is very large, ranging from 0.66 to 0.96, implying that a significant number of these prototypes might constitute building
blocks of a globally optimal solution.
7. Conclusions
This work proposes the first population-based memetic algorithm (MA) for the Edge Minimum Sum-of-Squares Cluster-
ing (E-MSSC) problem, which integrates a dedicated crossover operator for offspring generation with a reformulation local
search method for local optimization. Computational results on three sets of benchmarks show that the proposed algorithm
is very competitive with the state-of-the-art algorithms from the literature.
An analysis of the key components of the proposed MA reveals the following points. First, the reformulation local search,
that combines the K-means algorithm with tabu search, shows to be a very effective local optimization procedure within our
MA framework. Second, the population based memetic framework is very powerful for tackling E-MSSC. Finally, the muta-
tion operator helps maintain a healthy diversity of the population.
As memetic algorithm shows to be an effective approach for E-MSSC, it is worth considering other population-based evo-
lutionary computing algorithms for the given problem, such as the distributed Evolutionary Algorithms (dEAs) [18,33]. Our
proposed algorithm could also be easily adapted to similar and more complex versions of the network clustering problems
and tested on real-life instances.
Qing Zhou: Writing - original draft, Software, Visualization. Una Benlic: Writing - review & editing, Supervision. Qinghua
Wu: Conceptualization, Methodology, Project administration, Funding acquisition.
The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.
Table 20
Comparisons of MA with five state-of-the-art algorithms on the OR-Lib instances with a time limit of 60 s.
pmed01 5 434190.94 0.00 0.00 0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.04 0.04 17.91 0.04 0.04 23.21 0.00 0.00 0.09
pmed02 10 252639.75 0.00 0.00 0.90 0.00 0.00 0.10 0.00 0.00 0.00 1.38 1.38 11.27 1.38 1.38 14.56 0.00 0.00 0.99
pmed03 10 259643.17 0.00 0.00 1.20 0.00 0.00 0.10 0.00 0.00 0.00 8.26 8.26 11.99 8.26 8.26 15.29 0.00 0.00 2.21
pmed04 20 145461.30 0.00 0.09 20.90 0.00 0.00 21.70 0.00 0.00 15.40 5.70 5.70 9.22 5.70 5.70 11.89 0.00 0.00 5.11
pmed05 33 39211.95 0.00 0.08 13.70 0.30 0.47 17.80 0.00 0.08 36.50 13.02 13.02 8.90 13.02 13.02 11.43 0.00 0.00 15.66
pmed06 5 383058.00 0.00 0.05 20.10 0.00 0.00 0.00 0.00 0.00 0.00 0.98 0.98 19.01 0.98 0.98 24.60 0.00 0.00 0.04
pmed07 10 212248.00 0.00 0.17 28.00 0.00 0.00 4.20 0.00 0.00 5.10 0.83 0.83 15.22 0.83 0.83 20.06 0.00 0.00 0.25
pmed08 20 149175.44 0.00 0.28 39.90 0.00 0.08 20.50 0.00 0.03 21.10 4.48 4.48 12.70 4.48 4.48 16.54 0.00 0.00 3.85
pmed09 40 60454.76 0.00 1.18 40.20 0.00 0.00 26.20 0.00 0.00 11.70 8.74 8.74 9.93 8.74 8.74 12.43 0.00 0.00 16.83
287
288
Table 21
Comparisons of MA with five state-of-the-art algorithms on the TSP-Lib instances with a time limit of 60 s (part I).
rd400 10 3.649436e + 07 1.86 4.02 39.30 0.00 0.00 21.70 0.00 0.00 15.70 0.03 0.03 47.97 0.03 0.03 48.14 0.00 0.00 0.75
rd400 50 1.339953e + 07 2.89 4.96 48.90 0.08 0.17 42.70 0.00 0.24 45.00 1.29 1.29 31.56 1.29 1.29 31.82 0.05 0.13 28.56
rd400 100 6.191909e + 06 3.95 5.92 45.20 0.76 1.11 50.60 0.00 0.68 51.60 5.49 5.52 29.36 5.49 5.49 29.75 0.29 1.09 51.29
rd400 200 1.673771e + 06 3.51 3.70 46.70 2.23 3.57 48.60 0.10 2.72 53.60 23.46 23.75 28.19 23.46 23.47 26.43 0.00 2.32 53.16
pr439 10 1.987408e + 09 0.69 4.05 34.20 0.00 0.00 21.80 0.00 0.00 18.70 1.97 1.97 40.73 1.97 1.97 40.49 0.00 0.00 7.75
pr439 50 6.814545e + 08 4.21 6.01 50.90 0.21 0.69 44.20 0.00 0.19 46.40 5.82 5.83 27.83 5.82 5.82 28.16 0.59 0.77 37.73
pr439 100 2.724956e + 08 5.51 7.10 51.60 1.74 3.16 58.50 1.31 4.36 58.70 9.46 9.46 26.44 9.46 9.46 26.18 0.00 1.09 47.87
pr439 200 6.502722e + 07 3.08 5.49 53.30 3.27 9.44 56.00 3.09 10.96 56.20 13.32 13.51 23.24 13.32 13.33 22.71 0.00 2.34 48.34
pcb442 10 3.936676e + 08 1.09 2.86 35.80 0.00 0.00 15.90 0.00 0.00 15.00 0.07 0.07 46.13 0.07 0.07 46.60 0.00 0.00 3.88
pcb442 50 1.306841e + 08 3.55 6.06 48.50 0.01 0.40 41.90 0.13 0.38 41.00 0.82 1.15 27.27 0.82 0.85 28.54 0.00 0.19 48.46
pcb442 100 6.347818e + 07 3.25 5.45 53.60 0.45 1.25 52.40 0.00 0.92 46.50 3.66 3.77 25.03 3.67 3.67 24.98 0.10 0.94 51.03
pcb442 200 1.830759e + 07 0.51 4.56 52.10 0.00 2.34 50.30 1.40 2.17 53.70 18.46 18.98 21.12 18.46 18.46 21.17 1.71 3.77 50.19
pcb1173 10 2.898943e + 08 3.03 10.67 49.20 0.00 0.00 37.60 0.00 0.02 44.50 0.29 0.29 50.22 0.29 0.29 50.55 0.00 0.01 28.04
pcb1173 50 1.237713e + 08 12.68 15.29 35.00 9.73 12.50 51.90 8.35 10.83 52.60 0.04 0.04 30.00 0.04 0.04 32.09 0.00 0.26 43.91
Table 21 (continued)
pcb1173 100 7.949999e + 07 10.16 11.83 49.50 10.85 13.13 51.70 11.67 13.92 53.40 0.02 0.06 25.14 0.00 0.00 26.45 0.44 0.79 29.48
pcb1173 200 4.491868e + 07 6.01 8.38 47.50 16.39 17.02 51.00 13.50 14.74 55.50 0.07 0.23 23.86 0.00 0.01 25.18 0.32 1.42 45.95
rl1304 10 1.014519e + 10 4.72 15.64 50.90 0.00 0.00 40.80 0.00 0.00 53.70 0.11 0.11 56.32 0.11 0.11 56.78 0.00 0.00 24.39
rl1304 50 4.724233e + 09 12.95 15.26 41.70 2.37 3.48 55.50 2.03 2.27 52.80 0.42 0.42 31.88 0.42 0.42 34.00 0.00 0.29 32.82
rl1304 100 3.070192e + 09 11.75 13.98 39.40 3.54 7.89 56.50 8.13 9.08 52.20 0.15 0.16 26.28 0.15 0.15 27.74 0.00 0.34 55.16
rl1304 200 1.768823e + 09 10.50 12.71 28.30 13.91 15.08 56.30 13.52 15.58 54.40 0.82 0.96 24.43 0.82 0.83 24.73 0.00 1.00 50.72
nrw1379 10 2.104002e + 08 4.91 11.06 53.30 0.00 0.00 53.80 0.00 0.00 59.30 0.00 0.00 53.40 0.00 0.00 53.13 0.00 0.00 18.80
nrw1379 50 9.565609e + 07 12.38 16.71 41.40 36.95 37.46 58.60 36.88 37.17 51.30 0.34 0.39 30.53 0.34 0.36 32.72 0.00 0.37 24.54
nrw1379 100 6.196648e + 07 12.56 15.06 36.40 43.69 45.35 58.00 45.50 46.25 51.90 0.00 0.00 27.38 0.00 0.00 28.52 0.12 1.18 45.36
nrw1379 200 3.618693e + 07 8.03 9.32 34.80 33.01 35.71 57.70 31.12 33.97 53.10 0.00 0.20 25.03 0.10 0.11 26.19 0.20 1.02 34.30
fl1400 10 3.427747e + 07 43.93 128.50 59.90 0.00 89.24 52.30 0.00 54.92 54.20 4.13 4.13 39.29 4.13 4.13 39.35 0.00 0.01 27.41
fl1400 50 1.073865e + 07 9.21 19.87 55.60 1.10 2.47 51.60 2.63 5.83 58.30 14.62 14.62 29.97 14.62 14.62 31.32 0.00 0.43 55.04
fl1400 100 5.243616e + 06 11.96 17.12 55.00 10.39 23.07 50.20 18.33 24.94 53.60 18.14 18.14 30.30 18.14 18.14 32.00 0.00 1.40 43.05
289
290
Table 22
Comparisons of MA with five state-of-the-art algorithms on the University of Florida Sparse Matrix instances with a time limit of 60 s.
Harvard500 10 643.00 0.00 2.32 33.70 0.00 0.00 0.10 0.00 0.00 0.30 0.00 0.00 34.75 0.00 0.00 34.01 0.00 0.00 0.07
Harvard500 50 434.00 0.46 1.21 48.70 0.23 0.43 35.20 0.00 0.08 28.80 3.69 3.69 15.88 3.69 3.69 16.25 0.00 0.14 17.53
Harvard500 100 357.50 0.14 0.39 32.90 0.42 1.17 37.70 0.00 0.14 28.90 11.89 11.89 9.34 11.89 11.89 9.92 0.14 0.39 37.02
Harvard500 200 300.00 0.00 0.00 51.26 0.00 0.00 42.39 0.00 0.00 57.64 0.00 0.00 5.50 0.00 0.00 4.91 0.00 0.00 48.32
USAir97 10 472.94 3.88 5.89 43.60 0.65 0.70 9.40 0.63 0.63 1.60 0.86 0.86 36.35 0.86 0.86 34.65 0.00 0.24 7.49
USAir97 50 264.00 0.95 1.74 42.10 1.14 1.53 20.20 0.19 0.38 19.80 6.82 6.82 17.75 6.82 6.82 17.93 0.00 0.25 28.27
USAir97 100 189.00 0.00 0.11 14.70 0.53 1.08 31.30 0.00 0.00 11.00 22.75 22.75 9.63 22.75 22.75 9.03 0.26 0.69 39.22
USAir97 200 132.00 0.00 0.00 47.64 0.00 0.00 45.24 0.00 0.00 54.18 0.00 0.00 6.62 0.00 0.00 6.87 0.00 0.00 35.16
Email 10 4145.09 4.07 7.21 39.80 0.12 0.24 12.80 0.00 0.21 25.50 0.26 0.27 34.90 0.26 0.26 35.13 0.09 0.12 8.24
Email 50 2109.50 8.38 13.39 46.20 0.43 6.65 54.20 0.58 6.12 57.00 0.26 0.30 22.36 0.26 0.26 22.16 0.00 0.34 43.38
Email 100 1456.00 11.23 13.19 50.40 0.71 3.67 58.20 1.61 4.52 57.40 0.00 0.10 19.79 0.00 0.00 21.21 0.19 0.59 34.18
Email 200 935.50 15.93 17.32 52.00 11.20 13.50 59.80 7.27 12.31 58.10 2.94 5.28 16.02 2.94 3.00 14.39 0.00 0.70 48.96
pmed01 5 434190.94 0.00 0.00 0.10 0.00 0.00 0.00 0.00 0.00 0.00 0.04 0.04 26.24 0.04 0.04 34.10 0.00 0.00 0.27
pmed02 10 252639.75 0.00 0.00 0.30 0.00 0.00 0.00 0.00 0.00 0.00 1.38 1.38 16.65 1.38 1.38 21.47 0.00 0.00 7.42
pmed03 10 259643.17 0.00 0.00 0.60 0.00 0.00 0.10 0.00 0.00 0.00 8.26 8.26 17.29 8.26 8.26 22.39 0.00 0.00 1.48
pmed04 20 145461.30 0.00 0.02 119.00 0.00 0.00 74.70 0.00 0.00 189.00 5.70 5.70 13.38 5.70 5.70 17.38 0.00 0.00 28.42
pmed05 33 39211.95 0.00 0.00 42.60 0.00 0.00 156.70 0.00 0.00 207.00 13.02 13.02 12.92 13.02 13.02 16.69 0.00 0.00 41.68
pmed06 5 383058.00 0.00 0.00 10.60 0.00 0.00 0.00 0.00 0.00 0.00 0.98 0.98 27.99 0.98 0.98 36.53 0.00 0.00 0.01
pmed07 10 212248.00 0.00 0.00 27.00 0.00 0.00 1.60 0.00 0.00 4.90 0.83 0.83 22.47 0.83 0.83 29.56 0.00 0.00 0.33
pmed08 20 149175.44 0.00 0.00 59.20 0.00 0.00 145.90 0.00 0.00 46.10 4.48 4.48 18.81 4.48 4.48 24.68 0.00 0.00 20.88
pmed09 40 60454.76 0.00 0.09 203.30 0.00 0.00 105.20 0.00 0.00 48.20 8.74 8.74 14.55 8.74 8.74 18.83 0.00 0.00 18.54
291
292
Table 24
Comparisons of MA with five state-of-the-art algorithms on the TSP-Lib instances with a time limit of 600 s (part I).
rd400 10 3.649436e + 07 0.04 0.95 387.40 0.00 0.00 148.40 0.00 0.00 93.10 0.03 0.03 52.98 0.03 0.03 60.10 0.00 0.00 0.85
rd400 50 1.339953e + 07 0.62 1.48 516.20 0.00 0.01 333.30 0.00 0.01 446.00 1.29 1.29 34.90 1.29 1.29 39.54 0.03 0.05 205.98
rd400 100 6.176798e + 06 0.80 1.84 550.30 0.40 0.73 301.90 0.13 0.32 178.50 5.74 5.74 53.38 5.74 5.74 36.43 0.00 0.40 418.38
rd400 200 1.614626e + 06 2.11 3.08 468.50 0.08 1.53 389.30 0.00 0.96 301.68 27.99 28.17 49.77 27.99 27.99 43.53 0.00 0.80 506.80
pr439 10 1.987408e + 09 0.06 0.47 481.20 0.00 0.00 6.30 0.00 0.00 44.90 1.97 1.97 44.43 1.97 1.97 50.60 0.00 0.00 177.25
pr439 50 6.798665e + 08 2.02 3.21 534.10 0.38 0.51 368.80 0.00 0.13 301.30 6.07 6.07 81.00 6.07 6.07 34.38 0.47 0.70 389.71
pr439 100 2.719918e + 08 1.59 2.65 531.70 0.06 0.23 368.70 0.00 0.16 397.20 9.66 9.66 25.30 9.66 9.66 32.15 0.05 0.41 375.66
pr439 200 6.317237e + 07 2.07 2.84 539.20 0.41 2.08 388.50 0.00 1.35 330.50 16.65 16.71 24.62 16.65 16.65 127.21 0.59 1.78 469.50
pcb442 10 3.936676e + 08 0.00 0.47 414.40 0.00 0.00 39.70 0.00 0.00 60.60 0.07 0.07 51.24 0.07 0.07 58.38 0.00 0.00 0.22
pcb442 50 1.306005e + 08 2.31 3.06 538.00 0.06 0.20 281.80 0.14 0.19 286.50 0.88 1.02 98.88 0.88 0.88 134.36 0.00 0.08 319.60
pcb442 100 6.287998e + 07 1.02 2.32 548.70 0.52 0.92 270.60 0.00 0.38 239.20 4.61 4.70 127.31 4.63 4.63 131.11 0.63 1.09 492.00
pcb442 200 1.773801e + 07 1.05 2.74 541.00 1.92 3.07 263.80 0.00 1.47 307.60 22.26 22.62 83.05 22.26 22.26 26.11 1.79 2.90 510.91
pcb1173 10 2.898943e + 08 0.94 3.52 414.20 0.00 0.00 97.20 0.00 0.00 79.50 0.29 0.29 55.66 0.29 0.29 63.38 0.00 0.00 78.87
pcb1173 50 1.234250e + 08 7.47 9.20 457.00 0.00 0.01 319.20 0.00 0.00 406.80 0.32 0.32 35.05 0.32 0.32 39.81 0.04 0.13 383.38
Table 24 (continued)
pcb1173 100 7.888991e + 07 6.11 7.95 543.50 0.29 0.94 596.10 0.07 1.63 505.70 0.72 0.77 188.93 0.72 0.72 282.88 0.00 0.28 461.09
pcb1173 200 4.435643e + 07 5.12 6.73 519.50 3.03 4.13 505.70 3.20 4.31 506.80 1.30 1.35 87.61 1.25 1.28 182.49 0.00 0.54 545.05
rl1304 10 1.014519e + 10 0.26 5.09 273.50 0.00 0.00 160.00 0.00 0.00 123.70 0.11 0.11 62.40 0.11 0.11 70.95 0.00 0.00 144.48
rl1304 50 4.713066e + 09 9.24 10.98 515.50 0.00 0.04 313.10 0.00 0.07 322.20 0.66 0.66 38.12 0.66 0.66 42.70 0.00 0.18 508.52
rl1304 100 3.052560e + 09 8.39 11.13 516.90 0.00 0.21 479.00 0.12 0.23 528.80 0.73 0.73 85.32 0.73 0.73 34.72 0.09 0.22 460.47
rl1304 200 1.745278e + 09 7.39 9.02 501.90 0.78 2.00 506.30 2.12 2.68 505.40 2.01 2.14 158.57 2.07 2.12 293.56 0.00 0.58 463.33
nrw1379 10 2.104002e + 08 2.50 4.11 440.30 0.00 0.00 272.10 0.00 0.00 241.90 0.00 0.00 58.96 0.00 0.00 67.02 0.00 0.00 22.24
nrw1379 50 9.551156e + 07 7.53 10.91 396.60 0.02 0.94 548.00 0.04 1.09 504.80 0.49 0.51 95.45 0.49 0.49 94.45 0.00 0.11 370.68
nrw1379 100 6.161060e + 07 7.85 10.40 494.40 23.51 26.35 580.70 24.33 25.86 502.70 0.58 0.58 31.50 0.58 0.58 35.64 0.00 0.36 485.23
nrw1379 200 3.569954e + 07 6.41 7.66 514.80 20.56 23.78 504.70 20.74 23.38 505.20 1.31 1.43 108.25 1.39 1.40 183.15 0.00 0.50 512.08
fl1400 10 3.427747e + 07 0.05 2.15 405.20 0.00 0.00 129.70 0.00 0.00 126.50 4.13 4.13 43.09 4.13 4.13 49.37 0.00 0.00 112.98
fl1400 50 1.071524e + 07 10.74 12.96 356.50 0.00 0.05 292.20 0.00 0.17 433.60 14.87 14.87 34.63 14.87 14.87 38.89 0.00 0.07 272.14
fl1400 100 5.211615e + 06 8.55 12.36 364.80 0.62 1.34 560.20 0.00 1.81 570.90 18.63 18.82 194.13 18.62 18.68 138.76 0.26 0.98 518.16
293
294
Table 25
Harvard500 10 643.00 0.00 0.28 216.60 0.00 0.00 0.20 0.00 0.00 0.10 0.00 0.00 352.04 0.00 0.00 326.02 0.00 0.00 0.15
Harvard500 50 433.50 0.12 0.29 391.50 0.00 0.23 197.90 0.00 0.06 157.80 3.81 3.81 160.95 3.81 3.81 158.69 0.00 0.06 246.41
Harvard500 100 356.50 0.14 0.42 232.00 0.42 0.81 361.00 0.00 0.27 93.70 12.20 12.20 99.23 12.20 12.20 96.82 0.14 0.27 334.17
Harvard500 200 300.00 0.00 0.00 67.18 0.00 0.00 59.65 0.00 0.00 84.63 0.00 0.00 68.86 0.00 0.00 53.93 0.00 0.00 49.17
USAir97 10 472.94 0.71 0.98 273.90 0.00 0.42 164.50 0.00 0.44 93.10 0.86 0.86 383.91 0.86 0.86 324.83 0.00 0.00 118.10
USAir97 50 264.00 0.19 0.63 257.00 0.38 0.83 428.70 0.00 0.21 151.10 6.82 6.82 176.70 6.82 6.82 178.00 0.00 0.04 154.87
USAir97 100 188.50 0.00 0.45 268.70 0.53 0.66 358.70 0.27 0.27 8.00 23.08 23.08 98.74 23.08 23.08 96.89 0.00 0.34 307.90
USAir97 200 132.00 0.00 0.00 64.11 0.00 0.00 58.49 0.00 0.00 74.14 0.00 0.00 61.92 0.00 0.00 67.64 0.00 0.00 67.18
Email 10 4145.09 1.74 3.36 397.70 0.12 0.12 214.70 0.00 0.19 234.20 0.26 0.26 354.70 0.26 0.26 346.90 0.05 0.11 33.09
Email 50 2107.29 5.45 7.55 537.30 0.29 0.69 424.90 0.09 0.30 372.60 0.37 0.39 222.73 0.37 0.37 222.89 0.07 0.28 374.69
Email 100 1443.14 8.45 9.97 551.90 0.93 1.32 507.40 0.00 0.22 466.10 0.89 1.02 198.68 0.89 0.89 196.65 0.22 0.44 397.53
Email 200 920.10 13.98 15.06 523.80 3.03 3.89 549.10 1.21 2.43 580.40 4.66 6.33 152.48 4.66 4.66 153.64 0.00 0.27 473.97
Acknowledgments
We are grateful to the reviewers for their valuable comments which helped us to improve the paper. We thank Dr. A.
Nikolaev for providing us with the source codes of their VNS-1, VNS-2 and VNS-3 algorithms and Dr. I. Vasil’eV for sending
us the source codes of GRASP and HYBRID heuristics. This work is partially supported by the National Natural Science Foun-
dation of China (Grant Nos. 71850410545, 71821001, 71771099, 71810107003, 71620107002).
Appendix A
The purpose of this appendix is to present computational results and comparisons with the proposed algorithm on OR-
Lib, TSP-Lib and University of Florida Sparse Matrix Collection benchmarks (Tables 20–25). Columns ‘Instance’ and ‘p’ show
the instance name and the number of clusters. Column ‘cost’ indicates the best objective values obtained across all the com-
pared algorithms. Columns ‘g best ’ and ‘g av g ’ respectively refer to the percentage gap of the best objective value and the aver-
age objective value across 10 independent runs to the best solution value from column ‘cost’. The percentage gaps of the best
and the average objective value are calculated as ðf f Þ=f 100, where f is the best result obtained across all the com-
pared algorithms and f is the best or the average objective value achieved by the given algorithm. Column ‘tav g ’ shows
the average run-time in seconds to reach the final objective value. The best result among all the compared algorithms is
marked in bold.
References
[1] D. Aloise, A. Deshpande, P. Hansen, P. Popat, NP-hardness of Euclidean sum-of-squares clustering, Mach. Learn. 75 (2) (2009) 245–248.
[2] D. Aloise, P. Hansen, L. Liberti, An improved column generation algorithm for minimum sum-of-squares clustering, Math. Program. 131 (1–2) (2012)
195–220.
[3] A.Y. Al-Omary, M.S. Jamil, A new approach of clustering based machine-learning algorithm, Knowl.-Based Syst. 19 (4) (2006) 248–258.
[4] P. Avella, A. Sassano, I. Vasil’ev, Computational study of large-scale p-Median problems, Math. Program. 109 (1) (2007) 89–114.
[5] A.M. Bagirov, Modified global k-means algorithm for minimum sum-of-squares clustering problems, Pattern Recogn. 41 (10) (2008) 3192–3199.
[6] A.M. Bagirov, S. Taheri, J. Ugon, Nonsmooth DC programming approach to the minimum sum-of-squares clustering problems, Pattern Recogn. 53
(2016) 12–24.
[7] P. Balaprakash, M. Birattari, T. Stützle, Improvement Strategies for the F-Race Algorithm: Sampling Design and Iterative Refinement, in: T. Bartz-
Beielstein et al. (Eds.), Hybrid Metaheuristics, Lect. Notes Comput. Sci., Vol 4771, Springer, Berlin, 2007, pp. 108–122.
[8] J.E. Beasley, A note on solving large p-median problems, Eur. J. Oper. Res. 21 (2) (1985) 270–273.
[9] M. Birattari, T. Stützle, L. Paquete, K. Varrentrapp, A racing algorithm for configuring metaheuristics, in: Proceedings of the 4th Annual Conference on
Genetic and Evolutionary Computation, Morgan Kaufmann Publishers Inc., 2002, pp. 11–18.
[10] J. Brimberg, Z. Drezner, N. Mladenović, S. Salhi, A new local search for continuous location problems, Eur. J. Oper. Res. 232 (2) (2014) 256–265.
[11] J. Brimberg, Z. Drezner, N. Mladenović, S. Salhi, Using injection points in reformulation local search for solving continuous location problems, Yugoslav
J.Oper. Res. 27 (3) (2016) 291–300.
[12] J. Carrasco, S. García, M.M. Rueda, S. Das, F. Herrera, Recent trends in the use of statistical tests for comparing swarm and evolutionary computing
algorithms: Practical guidelines and a critical review, Swarm Evolut. Comput. (2020), https://doi.org/10.1016/j.swevo.2020.100665, in press.
[13] E.J. Carrizosa Priego, N. Mladenović, R. Todosijević, Sum-of-squares clustering on networks, Yugoslav J. Operat. Res. 21 (2) (2011) 157–161.
[14] E. Carrizosa, N. Mladenović, R. Todosijević, Variable neighborhood search for minimum sum-of-squares clustering on networks, Eur. J. Oper. Res. 230
(2) (2013) 356–363.
[15] L.R. Costa, D. Aloise, N. Mladenović, Less is more: basic variable neighborhood search heuristic for balanced minimum sum-of-squares clustering, Inf.
Sci. 415–416 (2017) 247–253.
[16] T.H. Cuong, J.C. Yao, N.D. Yen, Qualitative Properties of the Minimum Sum-of-Squares Clustering Problem, 2018. arXiv preprint arXiv:1810.02057..
[17] T.A. Davis, Y. Hu, The university of Florida sparse matrix collection, ACM Trans. Math. Software (TOMS) 38 (1) (2011) 1–25.
[18] J. Del Ser, E. Osaba, D. Molina, X.S. Yang, S. Salcedo-Sanz, D. Camacho, S. Das, P.N. Suganthan, C.A.C. Coello, F. Herrera, Bio-inspired computation: Where
we stand and what’s next, Swarm Evolut. Comput. 48 (2019) 220–250.
[19] J. Derrac, S. García, D. Molina, F. Herrera, A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary
and swarm intelligence algorithms, Swarm Evolut. Comput. 1 (1) (2011) 3–18.
[20] C.C. Fast, I.V. Hicks, A Branch Decomposition Algorithm for the p-Median Problem, INFORMS J. Comput. 29 (3) (2017) 474–488.
[21] F. Glover, M. Laguna, Tabu Search, in: D.Z. Du, P.M. Pardalos (Eds.), Handbook of Combinatorial Optimization, Springer, Boston, 1998, pp. 2093–2229.
[22] P. Hansen, N. Mladenović, Variable neighborhood search for the p-median, Location Sci. 5 (4) (1997) 207–226.
[23] P. Hansen, N. Mladenović, J-MEANS: a new local search heuristic for minimum sum of squares clustering, Pattern Recogn. 34 (2) (2001) 405–413.
[24] P. Hansen, B.K. Cheung, E. Ngai, N. Mladenović, Analysis of global K-means, an incremental heuristic for minimum sum-of-squares clustering, J. Classif.
22 (2) (2005) 287–310.
[25] J.K. Hao, Memetic Algorithms in Discrete Optimization, in: F. Neri, C. Cotta, P. Moscato (Eds.), Handbook of Memetic Algorithms, Studies in
Computational Intelligence, Vol 379, Springer, Berlin, 2012, pp. 73–94.
[26] J.N. Hooker, R.S. Garfinkel, C.K. Chen, Finite dominating sets for network location problems, Oper. Res. 39 (1) (1991) 100–118.
[27] X. Lai, J.K. Hao, A tabu search based memetic algorithm for the max-mean dispersion problem, Comput. Oper. Res. 72 (2016) 118–127.
[28] A. Likas, N. Vlassis, J.J. Verbeek, The global k-means clustering algorithm, Pattern Recogn. 36 (2) (2003) 451–461.
[29] M. López-Ibáñez, J. Dubois-Lacoste, L.P. Cáceres, M. Birattari, T. Stützle, The irace package: Iterated racing for automatic algorithm configuration, Oper.
Res. Perspect. 3 (2016) 43–58.
[30] Y. Lu, U. Benlic, Q. Wu, A memetic algorithm for the orienteering problem with mandatory visits and exclusionary constraints, Eur. J. Oper. Res. 268 (1)
(2018) 54–69.
[31] L. Lu, J.K. Hao, Q. Wu, Hybrid evolutionary search for the traveling repairman problem with profits, Information Sciences 502 (2019) 91–108.
[32] P. Moscato, An introduction to population approaches for optimization and hierarchical objective functions: a discussion on the role of tabu search,
Ann. Oper. Res. 41 (2) (1993) 85–121.
[33] M.C. Naldi, R.J.G.B. Campello, Comparison of distributed evolutionary k-means clustering algorithms, Neurocomputing 163 (2015) 78–93.
[34] A. Nikolaev, N. Mladenović, R. Todosijević, J-means and I-means for minimum sum-of-squares clustering on networks, Optim. Lett. 11 (2) (2017) 359–
376.
[35] B. Ordin, A.M. Bagirov, A heuristic algorithm for solving the minimum sum-of-squares clustering problems, J. Global Optim. 61 (2) (2015) 341–361.
296 Q. Zhou et al. / Information Sciences 541 (2020) 271–296
[36] G. Reinelt, TSPLIB – a traveling salesman problem library, ORSA J. Comput. 3 (4) (1991) 376–384.
[37] M.G.C. Resende, R.F. Werneck, A hybrid heuristic for the p-median problem, J. Heuristics 10 (1) (2004) 59–88.
[38] J.M. Squyres, A. Lumsdaine, R.L. Stevenson, Cluster-based parallel image processing toolkit, in: Image and Video Processing III, International Society for
Optics and Photonics, vol. 2421, 1995, pp. 228–240..
[39] Q. Wu, J.K. Hao, A hybrid metaheuristic method for the Maximum Diversity Problem, Eur. J. Oper. Res. 231 (2) (2013) 452–464.
[40] Q. Wu, J.K. Hao, A clique-based exact method for optimal winner determination in combinatorial auctions, Inf. Sci. 334–335 (2016) 103–121.