See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/285758470

Multi-objective end optimization based on an improved cross-entropy method.

A case study of a micro-scale manufacturing process

Article in Information Sciences · December 2015

DOI: 10.1016/j.ins.2015.11.040

CITATIONS READS

12 277

3 authors:

Gerardo Beruvides Ramon Quiza

Hitachi Europe GmbH University of Matanzas "Camilo Cienfuegos"
38 PUBLICATIONS 98 CITATIONS 35 PUBLICATIONS 520 CITATIONS

SEE PROFILE SEE PROFILE

Rodolfo Elias Haber Guerra

Spanish National Research Council
152 PUBLICATIONS 1,226 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Flexible FE/BE Sensor Pilot Line for the Internet of Everything (IoSENSE) View project

Heurísticas inteligentes para optimización multiobjetivo con aplicación a los procesos industriales [Intelligent heuristics for multiobjective optimization with industrial
processes applications] View project

All content following this page was uploaded by Gerardo Beruvides on 14 February 2018.

The user has requested enhancement of the downloaded file.

 Multi-objective optimization based on an improved
cross-entropy method. A case study of a micro-scale
manufacturing process
Gerardo Beruvidesa,∗, Ramón Quizab , Rodolfo E. Habera
a
Center for Automation and Robotic (UPM-CSIC),
Ctra. de Campo Real km. 0,200, La Poveda, 28500 Madrid, Spain
b
Department of Mechanical Engineering, University of Matanzas,
Autopista a Varadero, km 3 1/2, 44740 Matanzas, Cuba

Abstract
The strong points of Estimation-of-Distribution algorithms (EDAs) and specif-
ically cross-entropy methods are widely acknowledged. One of the main ad-
vantages of EDAs is that the fusion of prior information into the optimization
procedure is straightforward, thereby reducing convergence time when such
information is available. This study presents the modified Multi-Objective
Cross-Entropy (MOCE+) method, based on a new procedure for addressing
constraints: i) the use of variable cutoff values for selecting the elitist popu-
lation; and, ii) filtering of the elitist population after each epoch. We study
the proposed method in different test suites and compare its performance
with some other well-known optimization methods. The comparative study
demonstrates the good figures of merit of the MOCE+ method in complex
test suites. Finally, the proposed method is applied to the multi-objective
optimization of a micro-drilling process. Two conflicting targets are consid-
ered: total drilling time and vibrations on the plane that is perpendicular
to the drilling axis. The Pareto front, obtained through the optimization
process, is analyzed through quality metrics and the available options in the
decision-making process. Overall, the quality metrics of the MOCE+ method
were better than the metrics of the other optimization methods considered in

∗
Corresponding author
Email addresses: gerardo.beruvides@car.upm-csic.es (Gerardo Beruvides ),
ramon.quiza@umcc.cu (Ramón Quiza), rodolfo.haber@car.upm-csic.es (Rodolfo E.
Haber)

Preprint submitted to Information Sciences January 7, 2016

this work. The reported optimization of the micro-drilling process with the
proposed method could potentially have a direct impact on improvements in
industrial efficiency.
Keywords: Multiobjetive optimization, Cross-entropy method, Genetic
algorithm, micro-drilling

Glossary of terms

Problem definition variables

f : Rn → Rm Optimization objectives function
g : Rn → Rp Constraints function
lj ∈ R, j = 1 . . . n Lower boundary of the j-th variable
m ∈ N+ Number of optimization objectives
n ∈ N+ Number of decision variables
p ∈ N+ Number of constraints
uj ∈ R, j = 1 . . . n Upper boundary of the j-th variable
γj ∈ R, j = 1 . . . p Weight of the j-th constraint

Algorithm parameters
N ∈ N+ Epoch numbers
Smax ∈ N+ Maximum evaluation number
+
Z∈N Population size
∗ +
Zmax ∈ N Maximum elite population size
α ∈ [0, 1] Smoothing factor
+
εmax ∈ R Convergence limit
κ ∈ R+ Elite threshold decreasing factor
+
λ0 ∈ R Initial elite threshold
ϕ ∈ [0, 1] Frequency inversion likelihood

Algorithm variables
r Number of inner histogram intervals
S Evaluations count
t Epoch number
xij , i = 1 . . . Z, j = 1 . . . n Value of the j-th variable for the i-th solution
x∗ij , i = 1 . . . Z ∗ , j = 1 . . . n Value of the j-th variable for the i-th elite solution
yij , i = 1 . . . Z, j = 1 . . . m Value of the j-th objective for the i-th solution

2
yij∗ , i = 1 . . . Z ∗ , j = 1 . . . m Value of the j-th objective for the i-th elite solution
Z∗ Elite population size
δi , i = 1 . . . Z ∗ Distance between the i-th and (i + 1)-th elite elements
εj , j = 1 . . . n Change in the standard deviation for the j-th variable
λt , t = 1 . . . N Elite threshold at the t-th epoch
µj , j = 1 . . . n Mean value of the j-th variable
σj , j = 1 . . . n Standard deviation of the j-th variable
φk , k = 1 . . . r + 2 Frequency of the k-th histogram interval

Random numbers
U(a, b) Uniform random number in the interval [a, b]
N (µ, σ, a, b) Gaussian random number with mean µ and deviation σ,
truncated to the interval [a, b]

1. Introduction
The current literature on Estimation-of-distribution algorithms contains
an abundant range of deterministic and stochastic methods for solving multi-
objective optimization problems [18, 34]. Fig. 1 shows the basic steps to
formulate a multi-objective optimization for a general process. The opti-
mization of physical processes involves the use of models, represented by
functions that will never fulfill the conditions of continuity, differentiability
and unimodality, which are usually required for conventional analytical and
numerical techniques [44]. The alternative is the use of heuristics for opti-
mization, based on the use of soft computing tools [5]. These techniques are
especially useful in multi-objective optimization, where several different and
often interconnected objectives are considered [33].
Many optimization methods can in fact be applied, ranging from genetic
algorithms to particle swarm optimization [27, 4, 37, 10, 54, 55, 59]. Evolu-
tionary algorithms (EAs) have demonstrated their suitability as a method for
multi-objective optimization [25]. EAs store a set of simultaneous trade-off
solutions with the potential to exploit the synergies of a parallel search across
all possible solutions. However, EAs are usually experimentally assessed
through various test problems, because an analytical assessment of their be-
havior is very complex. Thus, their performance on random problems cannot

3
 Figure 1: Classic procedure for optimization of physical processes

be guaranteed prior to application [60, 23]. Estimation-of-Distribution Al-

gorithms (EDAs) have emerged in the middle ground between Monte-Carlo
simulation and EAs [19, 26]. A probabilistic model based on elite individuals
is built and subsequently sampled to produce a new population of better indi-
viduals. One of the main advantages of EDAs is that the fusion of prior infor-
mation into the optimization procedure is straightforward, thereby reducing
convergence time when such information is available. From the standpoint of
computational costs, it involves fewer heuristics than the other gradient-free
optimization methods [6].
Among the broad range of optimization possibilities for computational
architectures, optimal tuning of the parameters was adopted, rather than
the optimization of the structure or topology. It is computationally simpler

4
and sometimes brings better results than non-linear systems [35]. One of the
main applications of these techniques is the optimal setting of the parameters
(scaling factors or gains) for non-trivial and sometimes intractable tasks [22].
The optimal setting of fuzzy controller strategies based on stochastic
gradient-based optimization has been reported in different works [38, 39, 1].
However, many of these optimization techniques have yet to be applied to real
industrial processes, due to the high complexity of optimization algorithms,
the need to define appropriate cost functions and performance indexes, and
inappropriate performance, as well as an absence of commonly used empir-
ical formulas in industrial contexts. For all the above reasons, we selected
the so-called Cross-Entropy method (CE) [47, 48] as the multi-objective op-
timization algorithm. Its application to a micro-manufacturing process is
reported in this paper. The most attractive feature of cross entropy is that,
for a certain family of instrumental densities, the updating rules can be an-
alytically calculated, making them extremely efficient and fast. Moreover,
the theoretical background of CE enables theoretical studies of the method,
which can provide sound guidelines on the potential applications of this al-
gorithm in artificial cognitive architectures.
The concept of cross-entropy has been thoroughly covered in the litera-
ture from different perspectives. For instance, very promising results have
recently been described in the literature in relation to cross entropy for un-
certain variables and the minimum cross-entropy principle for uncertain op-
timization and machine-learning problems [7]. The concept has also served
as a foundation for developing a cross-entropy clustering method [52].
The optimal parameter settings are still an open research issue, mainly
due to their varied influence on improving process behaviors. Evolutionary-
based optimization algorithms have already shown improvements in perfor-
mance with very promising results for iterative feedback tuning methods in
discrete-time single-input single-output systems currently reported in the lit-
erature [40]. On-going research in this field includes new methods derived
from those that are already well-established such as the five-stage adaptive
PSO algorithm proposed by Precup et al. [39]. Moreover, Fu et al. [17]
applied the cross-entropy method to optimize the scaling factors and the
membership functions of fuzzy controllers, so as to guarantee fail-safe nav-
igation of unmanned autonomous vehicles. Finally, Giagkiosis et al. [19]
and Bekker et al. [3] recently published two seminal papers on cross-entropy
optimization.
This paper sets out two central contributions. The first contribution con-

5
cerns three key modifications of the cross-entropy method: a new procedure
for addressing constraints, the use of a variable cutoff value for selecting the
elitist population, and the filtering of the elitist population after each epoch.
The second contribution is the real application to a micro-manufacturing
process represented by the micro-drilling process of a titanium alloy.
The paper consists of six sections. Following this introduction, the sec-
ond section explains the basic cross-entropy concepts and the modified multi-
objective cross-entropy method. The third section presents the formalization
of the optimization problem for manufacturing processes. Subsequently, some
solutions are presented that use both methods and the experimental study
in which the empirical data was obtained from the micro-drilling process
is described. The results are then compared and analyzed in the fifth sec-
tion. Finally, the conclusions are presented and future lines of research are
discussed.

2. Algorithm description
The Cross-Entropy (CE) method is inspired by an adaptive variance min-
imization algorithm for estimating the probabilities of rare events on stochas-
tic networks [49]. The main rationale of CE is the construction of a random
sequence of solutions which converges probabilistically to an optimal or a
near-optimal solution in two iterative stages [9]. In the first iteration, a sam-
ple of random data is generated according to a specified random mechanism.
A better sample is produced in the next iteration and the parameters of the
random mechanisms (i.e. parameters of the probabilistic density functions)
are updated with the corresponding data [29, 58]. Let X be a random vari-
able on a space X , Px is its probability density function (PDF), and let the
score f be a real function in X . The CE method aims to find the minimum
of f over X , and the corresponding states x∗ that satisfy this minimum:

y ∗ = f (x∗ ) = min f (x); (1)

x∈X

The CE method provides a methodology for creating a sequence of x0 , x1 , ..., xN

and levels y0 , y1 , ..., yN such that y converges to y ∗ and x converges to x∗ .
We are concerned with estimating the probability Ω(y) of an event Ev =
{x ∈ X |f (x) ≥ y}, y ∈ R+ . Defining a collection of functions Iv for x ∈
X , y ∈ R+ [22].

6
 The proposed algorithm, called MOCE+, is inspired by the multi-objective
cross-entropy approach, proposed by Bekker and Aldrich [3]. All the func-
tions and flowcharts of the MOCE+ algorithm described below are available
from the following repository [20].
The algorithm is focused on solving a multi-objective optimization prob-
lem, given by:
min y = f (x) : x ∈ Rn , y ∈ Rm ; (2)
where:
li ≤ xi ≤ ui , i = 1 . . . n ; (3)
constrained by:
gi (x) ≤ 0, i = 1 . . . p . (4)
It starts by creating an empty elite population, {x∗ij } = {∅}, which will
store the best solutions throughout the execution time of the algorithm.
Then, a loop (outer loop) of N iterations is performed, where the values of
the means and standard deviations, µi and σi : i = 1 . . . n, for each decision
variable are computed from the corresponding intervals, by:

µi = li + U(0, 1)(ui − li ), i = 1 . . . n ; (5a)

σi = 10(ui − li ), i = 1 . . . n . (5b)

Note that while the means are randomly chosen from the interval, the stan-
dard deviation is deterministically computed as 10 times the length of the
interval. Then, a nested loop (inner loop) takes place until it reaches some
stopping conditions. It starts by creating the working population, {xij , i =
1 . . . Z, j = 1 . . . n}, which can be done in two different ways. If the elite
population is empty or if it is the first iteration in the loop, the working pop-
ulation is generated from normal random distributions, with means µi and
deviations σi , truncated in the intervals [li , ui ], for each decision variable.
Otherwise, the working population is created from the elite population.
The creation of the working population from the elite solutions is carried
out in a different way, as suggested by Bekker and Aldrich [3]. In the origi-
nal approach, a frequency histogram, with r = t + 2 intervals (where t is the
epoch) is constructed for each variable, from the solutions contained in the
elite population. Then, with some likelihood, 0 ≤ ϕ ≤ 1, the frequencies of
the histogram are inverted, in order to avoid convergence to a narrow sector
of the Pareto front. Finally, the mean and standard deviations are computed
for each histogram interval and a subset of new solutions are created from a

7
truncated normal random distribution; each subset contains a number of so-
lutions that are proportional to the frequency of the corresponding histogram
interval. As the number of elements in the subset is an integer, round errors
may occur. In such cases, some elements are added to the last interval until
the prescribed working population size, Z, is reached.
Two improvements were introduced in this section of the algorithm. Firstly,
the number of histogram intervals, r, was computed as the minimum of t + 2
and a fifth of the elite population size, Z ∗ , avoiding an excessive number of
intervals for small elite populations. A further improvement was integrated
in which the added solutions compensated the rounded errors. These solu-
tions were incorporated into randomly selected subsets, instead of into the
subset. As modifications, they were intended to avoid inappropriate trends
in the creation of those solutions.
When the new working population was created, each solution was evalu-
ated, obtaining the corresponding set of objective vectors, {yij = fj (xi1 , . . . , xin ), i =
1 . . . Z, j = 1 . . . m}. After this step, a further improvement was added, in
order to deal with constraints. This variation relies on the penalty approach
that modifies the values of the objective functions through the expression:
p
X
yij = yij + (γk max(0, gk (xi1 , . . . , xin ))), i = 1 . . . N, j = 1 . . . m ; (6)
k=1

where, γk , k = 1 . . . p, are the prescribed weights for each constraint. Then,

each solution in the working population was ranked by its Pareto dominance:
in other words, by considering the number of other solutions which dominate
it. The elite population, {x∗ij }, was then increased, by adding all the solutions
with a lower Pareto dominance than some threshold value, λt .
In the next step, the means and the standard deviation of each variable
were updated from the values of the elite population by using a smoothing
factor, α:
(t−1)
µti = αµi + (1 − α)mean(x∗i ), i = 1 . . . n ; (7a)
(t−1)
σit = ασi + (1 − α)stddev(x∗i ), i = 1 . . . n . (7b)

Three stopping conditions were then evaluated. Firstly, the change in the
standard deviation was computed for each variable:

εj = σj − σjold . (8)

8
 A condition that is fulfilled when all the changes are lower than or equal
to a prescribed value, εmax :
n
max(εj ) ≤ εmax . (9)
j=1

The second condition is fulfilled, if the number of evaluations, S, which

is incremented by Z in each iteration of the inner loop, is greater than or
equal to a prescribed value, Smax :
S ≥ Smax . (10)
If any of the previously described conditions are fulfilled, the inner loop
∗
stops and the elite population is ranked and filtered to store only Zmax solu-
tions. At this point, another modification was introduced, by decreasing the
threshold value, λt , by a factor, κ:
λt = (1 − κ)λt−1 . (11)
This change meant that we used broader threshold levels for the first
iterations (where preservation of diversity is important) and finer threshold
levels for the final iterations (yielding an elite population closer to the actual
Pareto front).

3. Comparison with other approaches

A comparative study was performed on the basis of other approaches
reported in the literature, to analyze the main drawbacks and advantages of
the proposed algorithm. A comparative study with all available gradient-free
methods is beyond the scope of this paper. So, only some of the most well-
established algorithms of similar scope are included in this study. Firstly, the
comparison with the first version of MOCE [3] was performed, in some test
problems that are widely used as benchmarks in multi-objective optimization
(see Table 1) .
MOP and ZDT problems were addressed by using the parameters shown
in Table 2. These values were obtained by trial and error, and by considering
those reported in various studies. The trade off between the Pareto front
quality was also taken into account for setting these parameters.
The quality of the Pareto front was evaluated by using four metrics [3].
While generational distance, GD, and convergence, CV , measure front con-
vergence (i.e., how close the obtained front is to the true Pareto front), and

9
 Table 1: Optimization problems solved by MOCE and MOCE+
Problem Reference Geometry Modality
MOP1 [50] Convex Unimodal/Unimodal
MOP2 [16] Concave Unimodal/Unimodal
MOP4 [30] Disconnected Multimodal/Unimodal
ZDT1 [60] Convex Multimodal/Unimodal
ZDT2 [60] Concave Multimodal/Unimodal
ZDT3 [60] Disconnected Unimodal/Multimodal

Table 2: Parameters values for solving problems MOP’s and ZDT’s

Parameter Value
Convergence limit, εmax 0.01
Smoothing factor, α 0.9
Epochs number, N 100
Population size, Z 50
Frequency inversion likelihood, φ 0.3
Initial elite threshold, λ0 2
Decreasing threshold factor, κ 0.05
∗
Maximum elite population size, Zmax 160
Maximum evaluation number, Smax 500

spacing, SP , measures diversity (i.e., how uniformly distributed the solutions

are); the maximum Pareto front error, M E, combines both criteria.
A total of 50 replications were performed for each problem. The mean
values and confidence intervals (at a 95% confidence level) of each metric are
shown in Table 3. The generational distances, GD, were worse than those
generated by MOCE for MOPs, but the values were still low. There was no
significant difference for ZDT2, but for ZDT1 and ZDT3 the values offered
by MOCE+ were remarkably better. Convergence, CV , was lower for all
the problems, except for MOP1 where no significant difference was found.
Spacing, SP , was also low, except in MOP4. For MOP1, ZDT1 and ZDT2
the values from MOCE+ was slightly better than those from MOCE, while
they were slightly worse for both MOP2 and ZDT3. Finally, the maximum
Pareto front error, M E, was significantly lower for MOP2, MOP4 and ZDT3;
it was notably higher in MOP1; while there were no significant differences in
ZDT1 and ZDT2.

10
 Summarizing, it can be concluded that the proposed algorithm, MOCE+,
performed in a similar way to MOCE for the MOP and the ZDT test suites
considered in this paper. It should be noted that its behavior was better for
ZDT problems than for MOP problems.

Table 3: Comparison between MOCE and MOCE+ for solving MOP and ZDT problems
Metrics [×10−3 ]
Problem Approach
GD CV SP ME
MOCE 0.0 6.8 20.23 3.8
MOP1
MOCE+ 0.6 ± 0.0 6.8 ± 0.0 8.7 ± 0.4 10.4 ± 2.8
MOCE 0.0 11.2 0.6 12.8
MOP2
MOCE+ 1.0 ± 0.0 10.2 ± 0.0 2.2 ± 0.4 1.4 ± 0.2
MOCE 0.2 95.7 15.7 290.3
MOP4
MOCE+ 0.8 ± 0.1 5.5 ± 0.4 66.9 ± 3.6 6.2 ± 0.7
MOCE 1.2 71.6 3.9 23.5
ZDT1
MOCE+ 0.8 ± 0.0 8.0 ± 0.2 2.4 ± 0.0 23.1 ± 0.9
MOCE 1.2 254.5 2.7 23.5
ZDT2
MOCE+ 1.2 ± 0.1 12.6 ± 1.3 2.5 ± 0.1 25.2 ± 3.3
MOCE 3.9 28.9 2.4 130.0
ZDT3
MOCE+ 0.6 ± 0.0 5.8 ± 0.1 4.5 ± 0.2 17.4 ± 0.8

A comparison with other multi-objective algorithms, including the archive-

based hybrid scatter search (AbYSS) [36], the strength Pareto evolution-
ary algorithm (SPEA2) [61], the non-dominated sorting genetic algorithm II
(NSGAII) [14], the non-dominated sorting genetic algorithm II + learning
paradigm based on jumping genes (NSGAII+JGBL) [32], the elite-guided
multi-objective artificial bee colony (EMOABC) [24], a self-adaptive multi-
objective harmony search (SAMOHS) [12], a real-coded NSGA-II with sim-
ulated binary jumping gene operators (RNSGA-II-SBJG) [45] and the pre-
viously mentioned multi-objective cross entropy (MOCE) method [3], are all
shown in Fig. 2. It may be noted that the generational distance of the solu-
tions provided by MOCE+ are better than those provided by MOCE (except
for MOP1 and MOP2), but they are worse than those provided by the other
evolutionary approaches. Nevertheless, the generational distance values are
very low in all the problems; therefore, the solutions are acceptable for many
applications.
Quite different results were obtained when attempting to solve more com-

11
 Figure 2: Comparison of different approaches for MOP’s and ZDT’s problems
*Only for the ZDT’s test suite fuctions

plex problems, especially those from the WFG test suite [23] (see Table 4).
These problems were addressed by using the parameters shown in Table 5.
The increases in the epoch numbers, N , and the population size, Z, were no-
table, while the convergence limit, ε, and the maximum evaluation number,
ηmax , all decreased.
Quality metrics for the WFG problems are shown in Table 6.
A comparison of the results, for the WFG test suite, with multi-objective
cross-entropy (MACE) [19], multi-objective cross-entropy optimization us-
ing generalized decomposition (MACE-gD) [19], multi-objective evolution-
ary algorithms based on decomposition (MOEA/D) [42], real-coded NSGA-II
with simulated binary jumping gene operators (RNSGA-II-SBJG) [45] and
regularity model-based estimation of distribution algorithms (RM-MEDA)
[41], are shown in Fig. 3. This figure shows a much better performance of
MOCE+ in relation to the WFG test suite and in comparison with the other
approaches.
Even though the performance of MOCE+ for simple problems was no bet-
ter than other previously published approaches, it is evident that MOCE+

12
 Table 4: Optimization problems from the WFG test suite [23]
Problem Geometry Modality
WFG2 Convex, disconnected Unimodal/Nultimodal
WFG3 Linear, degenerated Unimodal/Unimodal
WFG4 Concave Multimodal
WFG5 Concave Deceptive
WFG6 Concave Unimodal
WFG7 Concave Unimodal
WFG8 Concave Unimodal
WFG9 Concave Multimodal, deceptive

Table 5: Parameters values for solving problems WFG’s

Parameter Value
Convergence limit, εmax 0.005
Smoothing factor, α 0.1
Epochs number, N 2500
Population size, Z 160
Frequency inversion likelihood, φ 0.1
Initial elite threshold, λ0 4
Decreasing threshold factor, κ 0.5
∗
Maximum elite population size, Zmax 160
Maximum evaluation number, Smax 130

performed notably better for complex problems. The main rationale to ex-
plain this behavior is the complexity of MOCE+, which is a powerful tool
for solving complex problems, but which requires more computational re-
sources to be effective at solving simple problems. The main disadvantage
of MOCE+ is linked to problem-dependence parametrization. The optimal
setting of most gradient-free multi-objective algorithms is a challenging and
unsolved problem. However, the fast convergence of MOCE+ and the pa-
rameter settings reported in this paper serve as guidelines to help overcome
this limitation.

13
Table 6: Optimization outcomes for the WFG test suite with MOCE+[23]
Metrics [×10−3 ]
Problem
GD CV SP ME
WFG2 2.5 ± 0.3 6.8 ± 0.7 117.9 ± 10.2 28.7 ± 4.4
WFG3 1.5 ± 0.2 6.8 ± 0.3 119.2 ± 13.6 20.3 ± 1.8
WFG4 1.5 ± 0.4 10.3 ± 1.7 125.2 ± 7.9 18.7 ± 6.0
WFG5 3.3 ± 0.3 8.7 ± 1.2 96.3 ± 10.0 45.9 ± 4.1
WFG6 6.2 ± 0.8 39.2 ± 8.7 721.0 ± 32.0 52.7 ± 4.9
WFG7 2.4 ± 0.2 7.4 ± 0.7 59.2 ± 11.7 33.7 ± 3.6
WFG8 8.3 ± 1.1 36.7 ± 39.7 333.0 ± 68.3 96.9 ± 10.9
WFG9 1.8 ± 0.4 8.7 ± 1.7 80.9 ± 22.0 22.6 ± 3.8

Figure 3: Comparison of different approaches for WFG’s problems

14
4. Application to manufacturing processes: Optimization of micro-
drilling process
4.1. Current techniques and procedures for the optimization of machining
processes
Selection of the optimal cutting conditions is a very important task in the
design of machining processes [11]. The optimization of cutting conditions
involves two different challenges: on the one hand, obtaining accurate robust
models linking the objectives and constraints to the decision variables; on the
other hand, developing effective and efficient strategies that can negotiate the
optimization problems that have been formulated [43]. Both tasks are of high
physical and mathematical complexity.
Firstly, chip formation occurs over geometrically complex domains that
change quickly over time [15]. Moreover, metal-cutting processes involve
complex mechanical and thermal phenomena (such as, thermo-viscoplasticity,
friction and fracture). Most of these issues have yet to be fully understood
and there are, therefore, no reliable phenomenological models for these pro-
cesses. Consequently, in spite of some partially successful approaches, such
as Oxley’s predictive theory [31], there is at present no reliable analytical
model of machining processes that can be used for industrial applications.
Another commonly used approach is the finite-element method, which
provides approximate solutions for problems where analytical solutions can-
not be obtained [13]. Nevertheless, the accuracy of the finite-element-based
model is not good enough for use in real machining processes. Moreover,
the solutions of these models are computationally expensive, a key issue in
optimization, where multiple evaluations of the model are required. Con-
sequently, empirical models, either based on statistical regressions or based
on fuzzy modeling [46] and artificial intelligence tools [51] are currently the
most reliable choice in cutting-process modeling [53].
The optimization of the cutting processes involves the use of the above-
mentioned models, represented by functions that do not fulfill the conditions
of continuity, differentiability and unimodality, usually required for conven-
tional analytical and numerical techniques. One alternative is the use of
optimization heuristics, based on the use of soft-computing tools. These
techniques are especially useful in multi-objective optimization, where sev-
eral different, but often connected objectives are considered.
Simulated annealing [57], the ant-colony algorithm [2] and particle swarm
optimization [56] may all be mentioned among the most widely used heuris-

15
tics for the optimization of machining processes. Nevertheless, genetic algo-
rithms are still the most popular technique for a posteriori multi-objective
optimization, due not only to their robustness and efficacy, but also to their
ability to obtain the so-called Pareto front in a single processing run [8].
Several approaches have been proposed for this target; the most widely re-
ported are perhaps the multi-objective genetic algorithm (MOGA), the fast
non-dominated sorting genetic algorithm (NSGA-II) and the niched Pareto
genetic algorithm (NPGA) [28], which have been widely applied to the opti-
mization of cutting processes [35].

4.2. Cross-entropy-based multi-objective optimization of a micro-drilling pro-

cess
The cross-entropy method requires a precise or an approximate mathe-
matical model of the physical process yielded from input-output data. In
the previous sections, the improved cross-entropy method for multi-objective
optimization was assessed using well-known benchmarks with mathemati-
cal models. Micro-scale processes are non-linear, time-variant processes that
are difficult to represent with precise mathematical equations. Therefore,
two models of force and acceleration were obtained using a well-established
method inspired in Artificial Intelligence. For the sake of clarity, a feed-
forward neural network was considered for modeling purposes. A multilayer
perceptron was specifically selected for modeling the micro-drilling process.
The neural network was composed of four input layers (each input corre-
sponding to one independent variables), 5 neurons in the hidden layers and
a single output neuron, which offer the predicted value of the force. Hidden
neurons used a sigmoid transfer function while the output neuron used a lin-
ear transfer function. The training process was performed by means of the
error backpropagation algorithm, with an adaptive learning rate and momen-
tum. The following training parameters were selected: initial learning rate,
0.001; ratio to increase learning rate, 1.05; ratio to decrease learning rate,
0.70; momentum constant, 0.9; and minimum performance gradient, 10−10 .
The stop condition was established after 500000 epochs.
The first model was obtained for representing the relationship between
the thrust force, Fz , and the cutting parameters: cutting speed (vc ), feed-
rate per tooth, (fz ), peck drilling step (s) and hole depth (hd ). The available
experimental data was divided into a training set, composed of 768 data
samples (corresponding to the 80% of the total dataset), which was used for

16
fitting the models, and a validation set composed of the remaining 192 data
samples, which was used for testing their generalization capabilities.
The second model was generated to represent the vibration on the plane
perpendicular to the drilling axis (xy-plane), r̈P , which also depends on the
cutting parameters (vc , fz , s and hd ). This parameter was selected because
it is closely related to the quality of the drilled holes.
Therefore, the main goal of the optimization process was to select the
most convenient cutting parameters (i.e., speed, vc , feed rate, fz , and step,
s) for carrying out the drilling process of a 0.8mm-depth hole. All the im-
plemented functions for the micro-drilling case study are available at the
following repository [21]. All these variables were considered in their respec-
tive experimental intervals:
9.4 m/min ≤ vc ≤ 27.6 m/min ; (12a)
10 mm/min ≤ fz ≤ 300mm/min ; (12b)
0.02 mm ≤ s ≤ 0.06 mm . (12c)
The experiment was performed on a titanium-aluminum-vanadium alloy
(Ti6Al4V), with 0.2 mm-diameter drills.
A Kern-Evo high-precision machining center (see Fig. 4) was used in the
experimental setup, equipped with a Kistler Minidyn 9256 piezoelectric force
dynamometer for capturing the three components of the force signal (Fx , Fy ,
Fz ).

Figure 4: Experimental setup

Before the modeling step, both signals were filtered using a finite impulse
response filter. With regard to thrust force, the mean value of each peak

17
was computed to characterize the signal at each peak, while the root mean
square (RMS) was used in the case of vibrations for the same purpose. Two
objectives were simultaneously considered. The first was drilling time, τ ,
which can be computed with the equation:
"  #
2  
s hd hd hd
τ= + −2 + ; (13)
f0 s s fz

where, f0 is the fast-feed rate used for the backward motion. The second
objective was the amplitude of the vibrations on the plane perpendicular
to the drilling axis, which was modeled, in the previous section, by using a
neural network:
r̈P = φNN (vc , fz , s) . (14)
Indeed, both objectives must be minimized. With this combination, a
high-productivity drilling process that guarantees high-hole quality (closely
related with the perpendicular vibrations) may be expected.
Furthermore, certain constraints must be fulfilled. The thrust force, Fz ,
should be lower than the allowable thrust force, Fzal , which is pre-established
to avoid buckling-based breakage of the tool. This constraint can be ex-
pressed by:
Fz
−1≤0; (15)
Fzal
while the thrust force is computed with the previously obtained neural network-
based model:
Fz = φNN (vc , fz , s) ; (16)
and the allowable force can be determined with Euler’s equation:

π 2 Ey Imin
Fzal = ηf ; (17)
µb L
where, Ey = 650 MPa is the Young’s modulus of the tool material, Imin = 9.52×
10−6 mm4 is the minimum area moment of inertia of the drill cross-section,
µb = 2 is the coefficient, which takes into account the boundary conditions,
L = 2.5 mm is the length of the drill flute, and ηf = 0.5 is a security factor.
The optimization process was carried out with the parameters shown in
Table 7. These parameters were selected based on the previously obtained
results for standard test problems. It can be noted that the convergence limit

18
 Table 7: Parameters values for solving the drilling optimization problem
Parameter Value
Convergence limit, εmax 0
Smoothing factor, α 0.5
Epochs number, N 100
Population size, Z 5000
Frequency inversion likelihood, φ 0.1
Initial elite threshold, λ0 4
Decreasing threshold factor, κ 0.1
∗
Maximum elite population size, Zmax 500
Maximum evaluation number, Smax 105

was reduced to zero, keeping only the maximum evaluation number as the
stopping criterion.
Altogether, 25 runs were performed to increase the reliability of the op-
timization results. Fig. 5 depicts the Pareto fronts that were obtained.
Three zones can be easily noted in the graph. Zone I includes those so-
lutions with lower vibration amplitudes and longer drilling times. Hence,
it shows cutting parameters that will obtain high quality holes, but over
excessive operating times. Zone I comprises parameters in the intervals:
10.6 m/min ≤ vc ≤ 24.8 m/min, 280 mm/min ≤ fz ≤ 300 mm/min and
0.025 mm ≤ s ≤ 0.060 mm.
Zone II contains the best combination of solutions in both objectives, so
they are the most convenient parameters for most of the operations, giving
reasonably good hole quality within low drilling times. The cutting pa-
rameters in this zone are the intervals: 9.4 m/min ≤ vc ≤ 23.2 m/min,
245 mm/min ≤ fz ≤ 299 mm/min and 0.031 mm ≤ s ≤ 0.060 mm.
Finally, Zone III involves solutions with shorter execution times, but
with higher vibration amplitudes. Consequently, these are only solutions
for holes where quality is not an important requirement. Solutions in this
zone have cutting parameters within the following intervals: 10.0 m/min ≤
vc ≤ 25.5 m/min, 276 mm/min ≤ fz ≤ 300 mm/min and 0.025 mm ≤ s ≤
0.060 mm.
Regardless of their proper characteristics, the cutting parameters of the
three zones share some features. The Pareto front solutions, in all the zones,
cover almost all of the cutting speed intervals, vc , and steps, s. On the

19
 Figure 5: Pareto front of the drilling process

contrary, the values of the feed rates, fz , are higher than 245 mm/min in all
of the zones.
Table 8 displays 95% confidence intervals for the execution time and some
characteristics and quality metrics of the Pareto fronts that were obtained.

Table 8: Outcomes of the drilling process optimization

Parameter Confidence interval
Number of solutions 280 ± 57
Execution time [s] 763.4 ± 30.4
Hyperarea 1.8733 ± 0.0026
Spacing 0.0117 ± 0.0078
fz1 ≡τ 1.5292 ± 0.0001
Minimun value
fz2 ≡ r̈P 0.4444 ± 0.0002
f ≡τ 3.0900 ± 0.0761
Maximun value z1
fz2 ≡ r̈P 0.7637 ± 0.0146

As may be noted, there is a good coincidence between most of the pa-

rameters, specially in the quality metrics (hyperarea and spacing), where the
variability is lower than 1%. The minimum values of the objectives in the
Pareto fronts are also of low variability, but the maximum values are a little

20
more widespread.

5. Conclusions
This paper has presented a modified cross-entropy method to address
multi-objective optimization problems (MOCE+). The modifications that
have been introduced consist of a new procedure for addressing constraints:
the use of variable cutoff values for selecting the elitist population, and the
filtering of the elitist population after each epoch. A comparative study with
other techniques dealing with well-known testing problems are also reported
to assess the behavior of the proposed method. The comparison with a
previously published study showed no significant improvements for simple
testing problems such as those included in the MOP and ZDT test suites.
However, for more complex testing problems, represented in the paper by the
WFGs test suite, the proposed approach outperformed the other optimization
methods that were analyzed on the basis of the quality metrics of the Pareto
front.
Indeed, the parametrization of MOCE+ depended on the problems that
were addressed; however good convergence and the performance indices cor-
roborated some advantages of the proposed method. This paper presents
the application of MOCE+ for multi-objective optimization of the micro-
drilling process. The resulting Pareto front demonstrated the suitability of
the proposed approach for solving practical problems, including those involv-
ing constraints and using neural network-based models.
Further studies will be conducted to determine the influence of these
algorithmic parameters on unconstrained and constrained problems. This
future study will provide more specific guidelines for selecting the proper
parameters. New engineering applications will be also explored to test the
potential application and suitability of the proposed method.

Acknowledgment
This work was supported by the Spanish Ministry of Economy and Com-
petitiveness through DPI2012-35504 CONMICRO and by FEDER. The au-
thors also wish to thank the ACCUS and the DEMANES projects co-funded
by Artemis JU and the Ministry of Industry, Energy and Tourism under Ref:
ART-010000-2013-2 and ART-010000-2012-2, respectively.

21
References
[1] S. Barchinezhad and M. Eftekhari. A new fuzzy and correlation based
feature selection method for multiclass problems. International Journal
of Artificial Intelligence, 12(2):24–41, 2014.
[2] N. Baskar, P. Asokan, G. Prabhaharan, and R. Saravanan. Optimization
of machining parameters for milling operations using non-conventional
methods. International Journal of Advanced Manufacturing Technology,
25:1078–1088, 2005.
[3] J. Bekker and C. Aldrich. The cross-entropy method in multi-objective
optimisation: An assessment. European Journal of Operational Re-
search, 211:112–121, 2011.
[4] S. Bououden, M. Chadli, and H.R. Karimi. An ant colony optimization-
based fuzzy predictive control approach for nonlinear processes. Infor-
mation Sciences, 299(0):143–158, 2015.
[5] M. Chandrasekaran, M. Muralidhar, C. M. Krishna, and U. S. Dixit.
Application of soft computing techniques in machining performance pre-
diction and optimization: A literature review. International Journal of
Advanced Manufacturing Technology, 46:445–464, 2010.
[6] T. Chen, K. Tang, G. Chen, and X. Yao. Analysis of computational time
of simple estimation of distribution algorithms. IEEE Transactions on
Evolutionary Computation, 14:1–22, 2010.
[7] X. Chen, S. Kar, and D. A. Ralescu. Cross-entropy measure of uncertain
variables. Information Sciences, 201:53–60, 2012.
[8] C. C. Coello, G. B. Lamont, and D. A. Van Veldhuizen. Evolutionary
algorithms for solving multi-objective problems. Springer Science and
Business Media, 2007.
[9] A. Costa, O. D. Jones, and D. Kroese. Convergence properties of
the cross-entropy method for discrete optimization. Oper. Res. Lett.,
35(5):573–580, 2007.
[10] D. Sidorov D. Gao, O. Khamisov. Editorial for special issue on methods
of optimisation and their applications. International Journal of Artificial
Intelligence, 13(1):120–122, 2015.

22
[11] D. M. D’Addona and R. Teti. Genetic algorithm-based optimization of
cutting parameters in turning processes. In 46th CIRP Conference on
Manufacturing Systems (CIRP CMS 2013), pages 323–328, 2013.

[12] X. Dai, X. Yuan, and Z. Zhang. A self-adaptive multi-objective har-

mony search algorithm based on harmony memory variance. Applied
Soft Computing, 35:541–557, 2015.

[13] C. R. Dandekar and Y. C. Shin. Modeling of machining of compos-

ite materials: A review. International Journal of Machine Tools and
Manufacture, 57:102–121, 2012.

[14] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist

multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evo-
lutionary Computation, 6:182–197, 2002.

[15] P. M. Dixit and U. S. Dixit. Modeling of metal forming and machin-

ing processes: by finite element and soft computing methods. Springer
Science and Business Media, 2008.

[16] C. M. Fonseca and P. J. Fleming. Multiobjective genetic algorithms

made easy: selection sharing and mating restriction. In First Interna-
tional Conference on Genetic Algorithms in Engineering Systems: In-
novations and Applications, pages 45–52, 1995.

[17] C. Fu, M. A. Olivares-Mendez, R. Suarez-Fernandez, and P. Campoy.

Monocular visual-inertial slam-based collision avoidance strategy for
fail-safe uav using fuzzy logic controllers: Comparison of two cross-
entropy optimization approaches. Journal of Intelligent and Robotic
Systems: Theory and Applications, 73:513–533, 2014.

[18] I. Giagkiozis and P.J. Fleming. Methods for multi-objective optimiza-

tion: An analysis. Information Sciences, 293(0):338–350, 2015.

[19] I. Giagkiozis, R. C. Purshouse, and P. J. Fleming. Generalized decom-

position and cross entropy methods for many-objective optimization.
Information Sciences, 282:363–387, 2014.

[20] GAMHE Group. Multi-objective Optimization Cross Entropy (MOCE+)

function repository. URL: http://gamhe.eu/downloads/?route=.%
2FCEMOO%2FMOCE%2B.

23
[21] GAMHE Group. Multi-objective Optimization Cross Entropy (MOCE+)
samples repository. URL: http://gamhe.eu/downloads/?route=.%
2FCEMOO%2Fsamples.

[22] R. E. Haber, R. M. Del Toro, and A. Gajate. Optimal fuzzy control

system using the cross-entropy method. a case study of a drilling process.
Information Sciences, 180:2777–2792, 2010.

[23] S. Huband, P. Hingston, L. Barone, and L. While. A review of mul-

tiobjective test problems and a scalable test problem toolkit. IEEE
Transactions on Evolutionary Computation, 10:477–506, 2006.

[24] Y. Huo, Y. Zhuang, J. Gu, and S. Ni. Elite-guided multi-objective

artificial bee colony algorithm. Applied Soft Computing, 32:199–210,
2015.

[25] H. Jiang, J. Chen, and T. Liu. Multi-objective design of an fbg sen-

sor network using an improved strength pareto evolutionary algorithm.
Sensors and Actuators A: Physical, 220:230–236, 2014.

[26] H. Karshenas, R. Santana, C. Bielza, and P. Larranaga. Multiobjective

estimation of distribution algorithm based on joint modeling of objec-
tives and variables. Evolutionary Computation, IEEE Transactions on,
18(4):519–542, 2014.

[27] M. S. Kiran, H. Hakli, M. Gunduz, and H. Uguz. Artificial bee colony

algorithm with variable search strategy for continuous optimization. In-
formation Sciences, 300(0):140–157, 2015.

[28] A. Konak, D. W. Coit, and A. E. Smith. Multi-objective optimization

using genetic algorithms: A tutorial. Reliability Engineering and System
Safety, 91:992–1007, 2006.

[29] D. P. Kroese, R. Y. Rubinstein, and Th. Taimre. Application of the

cross-entropy method to clustering and vector quantization. J. of Global
Optimization, 37(1):137–157, 2007.

[30] F. Kursawe. A variant of evolution strategies for vector optimization.

1991.

24
[31] D. I. Lalwani, N. K. Mehta, and P. K. Jain. Experimental investigations
of cutting parameters influence on cutting forces and surface roughness
in finish hard turning of mdn250 steel. Journal of Materials Processing
Technology, 206:167–179, 2008.

[32] K. Li, S. Kwong, R. Wang, K.-S. Tang, and K.-F. Man. Learning
paradigm based on jumping genes: A general framework for enhancing
exploration in evolutionary multiobjective optimization. Information
Sciences, 226:1–22, 2013.

[33] A. Lopez-Jaimes and C. A. Coello Coello. Including preferences into a

multiobjective evolutionary algorithm to deal with many-objective en-
gineering optimization problems. Information Sciences, 277:1–20, 2014.

[34] S. Mirjalili and A. Lewis. Novel frameworks for creating robust multi-
objective benchmark problems. Information Sciences, 300:158–192,
2015.

[35] I. Mukherjee and P. K. Ray. A review of optimization techniques in

metal cutting processes. Computers and Industrial Engineering, 50:15–
34, 2006.

[36] A. J. Nebro, F. Luna, E. Alba, B. Dorronsoro, J. J. Durillo, and A. Be-

ham. Abyss: Adapting scatter search to multiobjective optimization.
IEEE Transactions on Evolutionary Computation, 12:439–457, 2008.

[37] C. Pereira, L. Gonalves, and M. Ferreira. Exudate segmentation in

fundus images using an ant colony optimization approach. Information
Sciences, 296(0):14–24, 2015.

[38] R. E. Precup, R. C. David, E. M. Petriu, S. Preitl, and M. B. Rdac.

Fuzzy logic-based adaptive gravitational search algorithm for optimal
tuning of fuzzy-controlled servo systems. IET Control Theory and Ap-
plications, 7:99–107, 2013.

[39] R. E. Precup, R. C. David, E. M. Petriu, M. B. Radac, and S. Preitl.

Adaptive gsa-based optimal tuning of pi controlled servo systems with
reduced process parametric sensitivity, robust stability and controller
robustness. IEEE Transactions on Cybernetics, 44:1997–2009, 2014.

25
[40] R. E. Precup, M. B. Radac, M. L. Tomescu, E. M. Petriu, and S. Preitl.
Stable and convergent iterative feedback tuning of fuzzy controllers for
discrete-time siso systems. Expert Systems with Applications, 40:188–
199, 2013.

[41] Z. Qingfu, Z. Aimin, and J. Yaochu. Rm-meda: A regularity model-

based multiobjective estimation of distribution algorithm. IEEE Trans-
actions on Evolutionary Computation, 12:41–63, 2008.

[42] Z. Qingfu and L. Hui. Moea/d: A multiobjective evolutionary algorithm

based on decomposition. IEEE Transactions on Evolutionary Compu-
tation, 11:712–731, 2007.

[43] R. Quiza, M. Rivas, and E. Alfonso. Genetic algorithm-based multi-

objective optimization of cutting parameters in turning processes. En-
gineering Applications of Artificial Intelligence, 19:127–133, 2006.

[44] S. Bharathi Raja and N. Baskar. Optimization techniques for machin-

ing operations: a retrospective research based on various mathematical
models. International Journal of Advanced Manufacturing Technology,
48:1075–1090, 2010.

[45] M. Ramteke, N. Ghune, and V. Trivedi. Simulated binary jumping

gene: A step towards enhancing the performance of real-coded genetic
algorithm. Information Sciences, 325:429–454, 2015.

[46] Q. Ren, M. Balazinski, L. Baron, K. Jemielniak, R. Botez, and

S. Achiche. Type-2 fuzzy tool condition monitoring system based on
acoustic emission in micromilling. Information Sciences, 255:121–134,
2014.

[47] R. Rubinstein. A stochastic minimum cross-entropy method for combi-

natorial optimization and rare-event estimation. Methodology and Com-
puting in Applied Probability, 7:5–50, 2005.

[48] R. Rubinstein. Semi-iterative minimum cross-entropy algorithms for

rare-events, counting, combinatorial and integer programming. Method-
ology and Computing in Applied Probability, 10:121–178, 2008.

[49] R. Y. Rubinstein and D. P. Kroese. The Cross Entropy Method: A Uni-

fied Approach To Combinatorial Optimization, Monte-carlo Simulation

26
 (Information Science and Statistics). Springer-Verlag New York, Inc.,
2004.

[50] J.D. Schaffer. Multiple objective optimization with vector evaluated

genetic algorithms. In Proceedings of the 1st International Conference
on Genetic Algorithms, pages 93–100, 1985.

[51] B. Sieben, T. Wagner, and D. Biermann. Empirical modeling of hard

turning of aisi 6150 steel using design and analysis of computer experi-
ments. Production Engineering, 4:115–125, 2010.

[52] J. Tabor and P. Spurek. Cross-entropy clustering. Pattern Recognition,

47:3046–3059, 2014.

[53] S. Velchev, I. Kolev, K. Ivanov, and S. Gechevski. Empirical models

for specific energy consumption and optimization of cutting parameters
for minimizing energy consumption during turning. Journal of Cleaner
Production, 80:139–149, 2014.

[54] H. Wang, Zh. Wu, Sh. Rahnamayan, H. Sun, Y. Liu, and J. Pan. Multi-
strategy ensemble artificial bee colony algorithm. Information Sciences,
279(0):587–603, 2014.

[55] Y. Wang and Y. Yang. Particle swarm optimization with preference

order ranking for multi-objective optimization. Information Sciences,
179:1944–1959, 2009.

[56] N. Yusup, A. M. Zain, and S. Z. M. Hashim. Overview of pso for opti-

mizing process parameters of machining. In 2012 International Work-
shop on Information and Electronics Engineering (IWIEE 2012), pages
914–923, 2012.

[57] A. M. Zain, H. Haron, and S. Sharif. Optimization of process parameters

in the abrasive waterjet machining using integrated sa-ga. Applied Soft
Computing Journal, 11:5350–5359, 2011.

[58] J. Zerovnik. The cross entropy method: A unified approach to com-

binatorial optimization, monte-carlo simulation and machine learning.
Journal of the Operational Research Society, 57:1503–1503, 2006.

27
 [59] J. Zhang, J. Zhuang, H. Du, and S. Wang. Self-organizing genetic algo-
rithm based tuning of pid controllers. Information Sciences, 179:1007–
1018, 2009.

[60] E. Zitzler, K. Deb, and L. Thiele. Comparison of multiobjective evo-

lutionary algorithms: Empirical results. Evolutionary Computation,
8:173195, 2000.

[61] E. Zitzler, M. Laumanns, and L. Thiele. Spea2: Improving the strength

pareto evolutionary algorithm. 2001.

28

View publication stats