Professional Documents
Culture Documents
Tec 2008
Tec 2008
Abstract— We present a novel method for handling uncertainty Evolutionary Algorithms (EAs) are intrinsically robust to
in evolutionary optimization. The method entails quantification uncertainties present in the evaluation of the objective function
and treatment of uncertainty and relies on the rank based due to the implementation of a population [5], [11]. In order to
selection operator of evolutionary algorithms. The proposed
uncertainty handling is implemented in the context of the Co- improve their robustness to uncertainty two common methods
variance Matrix Adaptation Evolution Strategy (CMA-ES) and are available. First, the implementation of larger population
verified on test functions. The present method is independent of size most often increases the robustness to uncertainty [6],
the uncertainty distribution, prevents premature convergence of [32]. Second, multiple objective function evaluations can be
the evolution strategy and is well suited for online optimization as conducted for each population member and the objective
it requires only a small number of additional function evaluations.
The algorithm is applied in an experimental set-up to the function is usually represented by the mean value. Both ap-
online optimization of feedback controllers of thermoacoustic proaches however increase the number of function evaluations
instabilities of gas turbine combustors. In order to mitigate these per generation typically by a factor between three and 100.
instabilities, gain-delay or model-based H∞ controllers sense the Hence the large number of required function evaluations makes
pressure and command secondary fuel injectors. The parameters the methods prohibitively expensive for applications requiring
of these controllers are usually specified via a trial and error
procedure. We demonstrate that their online optimization with an online optimization.
the proposed methodology enhances, in an automated fashion, the In this paper we propose an alternative approach to enhance
online performance of the controllers, even under highly unsteady
operating conditions, and it also compensates for uncertainties the capabilities of EAs for online optimization under uncer-
in the model-building and design process. tainties. We develop a novel uncertainty handling algorithm
and, motivated by the combustion problem, we demonstrate
its effectiveness in the online optimization of a Gain-Delay
I. I NTRODUCTION and an H∞ controller of an experimental combustor test-rig
using the CMA evolution strategy. The uncertainty handling
Environmental considerations impose stringent emission
method distinguishes uncertainty measurement and uncertainty
regulations for modern gas turbines. These requirements dic-
treatment. The uncertainty is measured by rank changes among
tate the development of lean premixed combustion systems
members of a population. This quantification of uncertainty is
operating with excess air to lower the combustion temperature
well suited for any ranking-based search algorithm. It requires
and decrease the NOx emission levels [37]. In turn, the
only a few additional function evaluations per generation, and
operation of the combustor in the lean regime makes it prone
does not rely on an underlying uncertainty distribution. The
to thermoacoustic instabilities that may cause mechanical dam-
uncertainty measurement is combined with two treatments for
age, energy losses by heat transfer to walls and increased noise
high uncertainty levels to prevent the failure of the algorithm.
and pollutant emissions. Thermoacoustic instabilities arise due
The uncertainty treatments aim to ensure that the signal-to-
to a feedback loop between pressure fluctuations, flow velocity
noise ratio remains large enough to maintain the effectiveness
and heat release. Active control is a prevalent method to reduce
of the evolutionary optimization algorithm.
thermoacoustic instabilities [38], [21]. In active control of gas
turbine combustors a feedback controller receives input from The paper is organized as follows: In Section II the test rig,
pressure sensors and commands a secondary fuel injection. built at ETH Zurich, is presented. We cast the optimization
The adjustment of the controller parameters into a feasible of the controller parameters as an optimization problem under
working regime can be formulated as an optimization prob- uncertainties and we discuss previous work. We address the
lem distinguished by two important factors: The stochastic problem of thermoacoustic instabilities for gas combustors and
nature of the combustion process introduces uncertainty in the introduce their handling by active control strategies. Section III
computation of the objective function value while the unsteady addresses evolutionary optimization under uncertainties. In
operating conditions require the online tuning of the controller Section IV the uncertainty handling method is introduced and
parameters. combined with the CMA evolution strategy. Section V presents
the verification of the algorithm on test functions. Section VI
A.S.P. Niederberger and L. Guzzella are with the Measurement and Control reports experiments on the test rig with the different controller
Laboratory
N. Hansen and P. Koumoutsakos are with the Institute of Computational structures for two operating conditions. The paper concludes
Science with a Summary and Outlook in Section VII.
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 2
downstream
A. Experimental set-up of the ETHZ combustor test rig duct
D. Adaptive Controllers premature convergence have been neglected and the method
Unsteady combustion introduces pressure waves in the employs a pre-specified maximum number of iterations.
combustor. Their reflection from the boundaries modifies in
turn the combustion process resulting in a potentially unstable E. Evolutionary Algorithms for Control
feedback cycle. Rayleigh [50] first noted that if the heat release An in-depth overview of evolutionary algorithms applied to
is in phase with the pressure fluctuations, the instability grows, controller optimization is given in [25]. One can distinguish
while combustion is stable if they are out of phase. In a study between online and offline optimization. Online applications
of a swirl-stabilized premix burner [47], it was found that large are rare and due to safety and time-constraints only very few
coherent structures are associated with this instability. The online applications have been conducted in a real system [1],
interaction between flow instabilities and acoustic resonant [43].
modes excites unstable modes, leading to periodic combustion In order to evolve the controller either the controller param-
in large-scale structures. Fuel feedline dynamics are another eters are directly optimized [19], [1], or the design parameters
mechanism causing equivalence ratio fluctuations [3]. of control algorithms such as Linear Quadratic Gaussian
(LQG) or H∞ are manipulated [20], and the controller is
noise calculated automatically. In order to improve the feasibility
of the online application of evolutionary algorithms, tuning of
- pn
an existing controller can be performed [39], [40]. Our method
u pc pm
controller plant is based on this latter approach.
-
only if, for two candidate solutions x1 and x2 , the variation the population [2], [58] or on the empirical variances of the
due to Nf (x1 ) and Nf (x2 ) exceeds the difference |f (x1 ) − measurements [18]. The number of reevaluations is limited by
f (x2 )| such that their ordering reverses. If the uncertainties an upper bound to avoid divergence and to maintain adaptivity
tend to exceed the difference |f (x1 ) − f (x2 )| we cannot in online applications. Despite these efforts, methods that re-
anymore conclude from two single measurements, L(x1 ) and duce the uncertainty based on reevaluations typically increase
L(x2 ), whether f (x1 ) > f (x2 ) or f (x1 ) < f (x2 ) holds with the number of function evaluations per generation by a factor
a small enough error probability (in a ranking-based search between three and a 100.
algorithm this is the only decision based on the L-values). In A slightly different approach to reduce the uncertainty uses
other words, referring to |f (x1 ) − f (x2 )| as signal and to the the already evaluated solutions. Instead of resampling L and
variations from Nf as noise, the uncertainty is problematic taking statistics of the samples a surrogate function (or meta-
if and only if the signal-to-noise ratio is too small. This model) is built from the already evaluated solutions [54],
observation readily implies that there can only be two ways to [14], [17]. In case of ”benign” uncertainty distributions the
cope with uncertainties for a ranking-based search algorithm. surrogate smooths the noisy landscape. For the CMA evolution
1) Increasing the signal, or strategy a local quadratic surrogate model could speed-up the
2) reducing the uncertainty. convergence on a noisy sphere function by a factor of two
in small dimensions [34]. In general, the surrogate approach
Efficient optimization techniques for problems with uncer-
will be less effective with increasing search space dimension
tainties must address successfully at least one of these two
or when a large population spread is already realized by the
issues. A broad literature overview of uncertainties addressed
underlying search algorithm.
in evolutionary optimization is given in [33].
A third approach addresses uncertainties in the objective
The most common technique to approach uncertainties in function by using a large population, also referred to as
the objective function value is resampling, that is the repeated implicit averaging [33]. The effect of a large population size
evaluation of L(x) for a given solution x [2], [15], [18], [58]. in an evolution strategy (ES) is twofold. First, the population
A statistics L(x)
b of the repeated samples of L(x) replaces spread can be larger. For example, on the sphere function the
the single measurement. Usually the mean value is taken as optimal step-size of the (µ/µI , λ)-ES is proportional to the
statistics L
b and the question whether it is an appropriate statis-
parent number µ, given intermediate multi-recombination and
tics
for uncertainty reduction is ignored. If the second moment µ ∝ λ 6 n [51, p.148]. Second, recombination smooths
E Nf2 exists, the variance of the mean statistics equals to the the effect of erroneous selection in search space. Conse-
variance of L divided by the number of independent samples quently increasing only λ is inferior to resampling [9], [26],
and hence the mean leads to a reduction of uncertainty. Taking but increasing µ and λ is preferable to resampling [5]. A
the median as statistics reduces the uncertainty
under much prerequisite for this advantage is that step-sizes are adapted
milder assumptions than the mean. If E Nfα exists for some properly, because the population spread is decisive. Otherwise
2
α > 0 then E L b exists for any sufficiently large sample size
increasing the population size can even be counterproductive
(note that E Nf0 = 1). If in addition, Nf has a continuous [26].
positive density in some neighborhood of the true median, Modifications of the selection scheme have been proposed
then L b is asymptotically normally distributed with variance
to compensate for uncertainties. In a (1 + 1)-ES a non-zero
inversely proportional to the sample size [36]. threshold for accepting the offspring is advantageous [42]. In
The main drawback of repeated evaluations is the increased the (µ/µI , λ)-ES the optimal ratio between µ and λ is 0.5
cost of the algorithm (in our case the evaluation of L is by [10] corresponding to a maximal step-size for a given λ. The
far the most time consuming part of the optimization). Given stochastic tournament selection can be modified to make up
the sphere function f (x) = kxk2 and Nf normally distributed for the stochastics introduced by the uncertain selection [16],
with standard deviation σε an evolution strategy can reach while in evolution strategies the selection scheme is already
√
a final distance to the optimum of R∞ ∝ σε [9] [10]. deterministic.
Consequently, to reduce the final distance to the optimum R∞ Overall, the handling of uncertainties in the objective func-
by a factor of α < 1 the number of necessary L-samples grows tion has been mainly addressed by uncertainty reduction rather
with α−2 . than signal improvement. In this paper we will use both
Usually the number of necessary samples varies in time approaches. First, a resampling approach is taken and adopted
and cannot be predicted in advance. In particular in the early to the specific application to reduce the uncertainty. Second,
stages of optimization the signal-to-noise ratio is expected to and more importantly, the signal is improved explicitly by in-
be large for two reasons. First, the distance between population creasing the population spread. Both approaches are controlled
members is large producing more likely a large difference in by a uncertainty measurement and hence implemented in an
objective function values. Second, the difference between L- adaptive way.
values of “bad” solutions is usually larger than for “good”
solutions. In order to reduce the associated cost, adaptive
IV. A N U NCERTAINTY-R ESISTANT E VOLUTIONARY
reevaluation methods have been proposed [2], [15] [18]. The
A LGORITHM
number of reevaluations is determined by the outcome of a
statistical test, for example the t-test [18], [58]. The choice of In this section we describe an evolutionary algorithm that
the solutions to be reevaluated can depend on their ranking in serves to minimize an objective function as defined in Equation
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 5
(2). The algorithm consists of two parts: A ranking-based objective function L(x) = rand that is independent of x,
evolutionary algorithm, the CMA-ES, and the uncertainty the first moment of the object parameters x is unbiased.
handling method. We first describe the CMA-ES and then The expectation of newly generated solutions is equal to
introduce the proposed novel uncertainty handling technique. the weighted mean of the previously selected solutions. The
second moment is described by covariance matrix and step-
A. The CMA Evolution Strategy size. The covariance matrix in the CMA-ES is unbiased,
We employ the evolution strategy (ES) with Covariance because under random selection the updated covariance matrix
Matrix Adaptation (CMA) [30], [31], [29], [28]. This choice is equal to the previous covariance matrix in expectation.
is motivated by several reasons: Analogously, the step-size σ is unbiased on the log scale. For
the second moment, the population variance, a bias towards
• CMA-ES is a non-elitist continuous domain evolutionary
increase or decrease will entail the danger of divergence or
algorithm. Non-elitism avoids systematic fitness overval-
premature convergence, respectively, whenever the selection
uation on noisy objective functions [5], because even
pressure is low. Next we describe the algorithm in detail.
solutions with (erroneously) exceptionally good fitness
Given an initial mean value m ∈ Rn , the initial covariance
values survive only one generation.
matrix C = I and the initial step-size σ ∈ R+ , the λ candidate
• The selection in CMA-ES is solely based on the ranking
solutions xk of one generation step obey
of solutions. This provides additional robustness in a
noisy environment. Ranking-based selection is in par- xk = m + σy k , k = 1, . . . , λ, (3)
ticular invariant to strictly monotonic (order-preserving)
transformations of the value L. where y k ∼ N (0, C) denotes a realization of a normally
• The CMA-ES provides an effective adaptation of the
distributed random vector with zero mean and covariance
search distribution to the landscape of the objective matrix C. Equation (3) implements mutation in the EA by
function. adding a random vector. The solutions xk are evaluated on
• The CMA-ES can be reliably used with small population
L and ranked such that xi:λ becomes the i-th best solution
sizes allowing for a fast adaptation in an online applica- vector and y i:λ the corresponding random vector realization.
tion. In the remainder we describe the updates of m, σ, and C
for the next generation step. For µ < λ let
The CMA-ES adapts the covariance matrix of a normal search
µ µ
distribution to the given objective function topography. On X X
convex-quadratic objective functions, nearly optimal covari- hyi = wi y i:λ , w1 ≥ · · · ≥ wµ > 0, wi = 1 (4)
ance matrices are thus achieved. The adaptation procedure i=1 i=1
operates efficiently and independently of the given population be the weighted mean of the µ best ranked y k vectors. The
size, which is small by default. Particularly on non-separable, recombination weights sum to one. The so-called variance
badly scaled problems often a speed-up by several orders of effective selection mass
magnitude can be achieved in terms of number of function 1
evaluations to reach a given function value and in terms of µeff = Pµ ≥1 (5)
i=1 wi2
CPU-time. The CMA-ES was evaluated on a variety of test
functions [31], [29], [28], [7] and was successfully applied to will be used in the following. Given µeff , the particular setting
a variety of real-world problems.1 of the recombination weights is, in our experience, secondary.
The CMA-ES follows two fundamental design principles From the definitions follows 1 ≤ µeff ≤ µ and µeff = µ for
employed in the development of the algorithm. The first design equal recombination weights. The role of µeff is analogous to
principle is invariance. We distinguish between invariance the role of the parent number µ with equal recombination
to transformations R → R of the function value L, as weights and usually µeff ≈ λ/4 is appropriate. Weighted
considered in the beginning of this section, and invariance recombination is discussed in more detail in [4].
to transformations S → S of the solution vector x in (2). The mean of the new distribution becomes
The CMA-ES reveals invariance to rigid (angle-preserving) µ
X
transformations of the solution vector, like translation, rotation m ← m + σhyi = wi xi:λ . (6)
and reflection, given that the initial solution is transformed i=1
respectively. The CMA-ES reveals invariance to overall scaling Equation (6) determines the center of the next population. The
of the search space, given that the initial scale σ is chosen equation implements selection by using µ < λ. Using different
accordingly. Finally, The CMA-ES reveals even invariance recombination weights must also be interpreted as selection
to any full rank linear transformation, given that the initial mechanism. The equation implements recombination by taking
covariance matrix is chosen respectively. Invariance properties a (weighted) mean of parental solutions.
induce equivalence classes of objective functions and therefore For step-size control the “conjugate” evolution path pσ ∈
allow for generalization of empirical results. Rn is introduced. The evolution path cumulates an exponen-
Second, the variation of object and strategy parameters tially fading pathway of the population mean in the generation
is unbiased [12], [31]. Given random selection, that is an sequence. Assuming that the optimal step-size leads to conju-
1 See http://www.inf.ethz.ch/personal/hansenn/cec2005.html and gate steps, the length of the conjugate evolution path can be
http://www.inf.ethz.ch/personal/hansenn/cmaapplications.pdf used as adaptation criterion for σ. If the evolution path is long,
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 6
σ must be increased, whereas if the evolution path is short, σ population sizes λ. Given cc ∝ 1/n the rank-one update can
must be decreased. Initialized with pσ = 0 the update of pσ reduce the number of function evaluations needed to adapt
(so-called cumulation) and σ reads : to a straight ridge topography roughly from O(n2 ) to O(n)
p 1 [29]. The rank-µ update exploits the information prevalent
pσ ← (1 − cσ ) pσ + cσ (2 − cσ )µeff C − 2 hyi (7)
in a large population. Given a sufficiently large population,
cσ kpσ k say λ ≈ n + 3, it reduces the number of generations needed
σ ← σ × exp −1 (8)
dσ χbn to adapt a complex but globally constant topography roughly
where 1/cσ > 1 determines the backward time horizon of from O(n2 ) to O(n) [29].
the evolution path pσ , damping dσ ≈ 1 controls the change The default parameter values for all parameters, namely
magnitude of σ, and χ bn is the expected length of a random offspring population size λ, parent number µ, recombination
variable distributed according to N (0, I). The evolution path weights wi , cumulation parameter cσ , step-size damping dσ ,
1 def
is appropriately normalized. We have C − 2 = BD −1 B T , cumulation parameter cc , mixing number µcov , and learning
2 T
where C = BD B is an eigendecomposition of the sym- rate ccov are [28] :
metric, positive definite covariance matrix C.2 The transfor- Selection and recombination:
1
mation C − 2 rescales hyi into an isotropic reference system. λ = 4 + b3 ln nc, µ = bλ/2c,
Given y i:λ distributed according to N (0, C), as under random
√ 1 ln(µ + 1) − ln i
selection, we can derive that µeff C − 2 hyi is distributed wi = Pµ for i = 1, . . . , µ,
according to N (0, I). Therefore, if pσ ∼ N (0, I) holds µ ln(µ + 1) − j=1 ln j
before applying (7), the same holds after applying (7). The Step-size control:
transformations make the expected length of pσ independent
of its orientation and allow the comparison of the length µeff + 2
cσ = ,
of pσ with its expected length χ bn in (8). Step-size σ is n + µeff + 3
increased if and only if kpσ k > χ bn , and decreased if and r !
µeff − 1
only if√kpσ k < χ bn . In practice we use the approximation dσ = 1 + 2 × max 0, − 1 + cσ
√ n+1
bn = 2 Γ( n+1
χ 2 )/Γ( n
2 ) ≈ n 1 − 1 1
4n + 21n2 .
Similar to (7) an evolution path pc is constructed to update Covariance Matrix Adaptation:
the covariance matrix. The covariance matrix admits a rank- 4
one and a rank-µ update. cc = , µcov = µeff
p n+4
pc ← (1 − cc ) pc + hσ cc (2 − cc )µeff hyi (9)
ccov T 1 2
C ← (1 − ccov )C + pc p ccov = √
µcov | {z c} µcov (n + 2)2
rank-one update
1
2µeff − 1
µ + 1− min 1,
(n + 2)2 + µeff
X
1 µcov
+ ccov 1 − wi y i:λ y T
i:λ (10)
µcov i=1 A detailed discussion of the strategy parameters can be found
| {z } in [31]. The identification procedure for ccov with rank-µ
rank-µ update update is described in [29]. Parameters for step-size adap-
where ccov ≤ 1 is a learning rate, µcov ≥ 1 determines the tation, cσ and dσ , were accommodated for use with a large
portion between
rank-one and rank-µ
p updates, and hσ = 0 population size in [28]. With increasing µeff the backward time
1 horizon and the change rate are reduced, such that the impact
if kpσ k > 1.5 + n−0.5 χ bn 1 − (1 − cσ )2(g+1) , and 1
otherwise, where g is the generation counter. Consequently, of step-size control diminishes in particular for µeff n.
the update of pc is stalled whenever pσ is considerably longer All experiments in this paper are conducted with the default
than expected. This mechanism is decisive after a change in parameter settings.
the environment which demands a significant increase of the Finally, we note that the structure of CMA-ES bears sim-
step-size whereas fast changes of the distribution shape are ilarities with other stochastic optimization procedures, see
postponed until after the step-size is increased to a reasonable e.g. [57], as well as with recursive estimation procedures,
value. in particular of the “Gauss-Newton” type [41, pp. 366, 371,
For the covariance matrix update the cumulation in (9) 375]. The analysis of these similarities and differences of these
serves to capture dependencies between consecutive steps. algorithms are far beyond the scope of this paper.
Dependency information would be lost for cc = 1, as a
change in sign of pc or y i:λ does not matter in (10). The B. Box Constraint Handling
rank-one update is particularly efficient with small offspring In the present algorithm parameter constraints are accounted
2 Columns of B are an orthonormal basis of eigenvectors, B B =T by introducing a penalty term in the cost function. This penalty
T
BB = I. Diagonal elements of the diagonal matrix D are square roots term quantifies the distance of the parameters from the feasible
of the corresponding positive eigenvalues. The matrix D can be inverted parameter space. The feasible space is a hypercube defined
by inverting its diagonal elements. From these definitions it follows that
y k ∼ σBDN (0, I) which allows the generation of the random vector by lower and upper boundary values for each parameter. We
realizations on the computer. implement a box boundary handling algorithm such that each
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 7
Pn
evaluated solution is guaranteed to lie within the feasible ξi = exp 0.9 log(Cii ) − n1 j=1 log(Cjj ) scales
space. This algorithm affects solely the evaluation of the the distance coordinate wise with respect to the covari-
solutions and entails the following steps. ance matrix of the distribution, disregarding its overall
• The cost of a solution x is obtained by evaluating the size. The number 0.9 serves as regularizer to an isotropic
function L at xfeas , where xfeas is the feasible solution shape (choosing zero would make all ξi isotropically
closest to x (with minimal kxfeas −xk). Hence, a feasible equal to one). √
solution is evaluated itself and an infeasible solution is Given that xfeas
i − xi = σ Cii is as large as typically
evaluated on the boundary of the feasible space. The new sampled by the given search distribution (i.e. a one-σ
feasible solution xfeas is used for the evaluation on L sample) then the i-th summand in Equation (12) equals
and for computing a penalty term, and it is discarded γi × σ 2 Cii /ξi ≈ γi × σ 2 . With Equation (11) we have
afterwards. γi × σ 2 ≈ 2 δfit which is a desired contribution. In par-
• A penalty term is added to the function value L penalizing ticular the contribution of each component (summand)
infeasible solutions. The penalty depends on the distance becomes identical and therefore the perturbation from
to the feasible space and is weighted and scaled in each the penalization on the covariance matrix adaptation
coordinate. The weights are set depending on observed procedure is minimized.
function value differences in L and are increased if The additive penalization in Equation (12) is a quadratic
necessary, depending on the distance of the distribution function with its minimum located on the boundary [27, p.76].
mean m to the feasible space. The scaling is based on Equation (12) has two important properties. First, it guarantees
the covariance matrix diagonal elements. that the minimum of the resulting function L cannot be outside
The complete boundary procedure, applied after the candidate the feasible domain. Second, the construction results in a
solutions of one generation are generated, reads as follows. comparatively unproblematic function topography, because the
0) Initialization: the boundary weights γi are initialized partial derivative ∂L(x)/∂xi approaches zero if the distance
once in the first generation step as γi = 0, for i = xfeas
i − xi approaches zero from the infeasible domain. For
1, . . . , n. ∂L(x)/∂xi 6→ 0 a sharp ridge along the boundary can result
1) Set weights: if the distribution mean m is out-of-bounds which is quite undesirable.
and either the weights were not set yet or the second
generation step is conducted, set for all i = 1, . . . , n, C. A Method for Handling Uncertainty
2 δfit We introduce a novel uncertainty-handling (UH) method,
γi = 1
Pn (11) suitable for evolutionary optimization algorithms that employ
σ2 × n j=1 Cjj
rank based selection operators. The rank based selection
where δfit is the median from the last 20 + 3n/λ operation allows for a robust quantification and handling of
generations of the interquartile range of the unpenalized uncertainties in the cost function as shown in the following
objective function values and Cjj is the j-th diagonal sections. We emphasize that the development of the proposed
element of covariance matrix C. The setting in Equation uncertainty-handling method is independent of the other op-
(11) is explained in conjunction with Equation (12) erators employed in the evolutionary optimization algorithm.
below. In the present work the UH is discussed, without loss of
2) Increase weights: for each component i, if the distribu- generality, in its implementation within the CMA-ES and the
tion mean mi is out-of-bounds and the distance
√of mi to overall algorithm is referred as UH-CMA-ES. The proposed
√ uncertainty handling preserves all invariance properties of the
the bound is larger than 3 × σ Cii × max 1, µeffn (the
typical distance to the optimum on√ the sphere function is CMA-ES mentioned above. The method however biases the
coordinate wise proportional to σ n/µeff ), the weight population variance when too large an uncertainty level is
γi is increased according to detected.
The uncertainty handling consists of two separate compo-
µeff
γi ← γi × 1.1max(1, 10n ) . nents.
• Quantification of the uncertainty effect on the ranking of
This adjustment prevents the mean value of the distribu- the members of the population
tion from moving too far away from the feasible domain • Treatment of the uncertainty, if necessary, to prevent the
(where far is naturally defined in terms of the given search algorithm from premature convergence
search distribution).
a) Uncertainty Quantification: We propose a reevalua-
3) Compute the penalized function value for each candidate
tion technique that provides a quantification of the uncertainty
solution x as
for any ranking-based search algorithm. The uncertainty in the
n
def 1 X (xfeas − xi )2 objective function can affect a ranking-based search algorithm
L(x) = L(xfeas ) + γi i (12)
n i=1 ξi only if changes of the ordering of solutions occur. Hence, the
uncertainty quantification is based on rank changes induced by
where xfeas is the feasible solution closest to x with out- reevaluations of solutions. A small perturbation can be applied,
of-bounds components set to the respective boundary before the reevaluation is done, to cover “frozen noise”, i.e.
values. Only xfeas is actually evaluated on L. Finally when the objective function is not a random variable itself
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 8
where rank(L·i ) is the rank of the respective function too low a signal-to-noise ratio, because in these
value in the set L = {Lold new regions the movement of the population is amplified.
k , Lk |k = 1, . . . , λ}.
5) Compute the uncertainty level, s. The rank change value, • Premature convergence is prevented.
∆i , is compared with a given limit ∆lim θ . The limit The following uncertainty treatment algorithm is applied
is based on the distribution of the rank changes on a after each generation step employing uncertainty measurement
random function L and the parameter θ. Formally we s.
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 9
functions the increase of the step-size might occasionally help that only σ is changed to treat the noisy environment, this
to locate the domain of a better local optimum, but we believe is a remarkable result. The mean vector fluctuates around the
that this effect is of minor relevance. Overall, we do not expect optimum zero (upper right), while the size of the fluctuations
that the uncertainty-handling would impair the performance differs for different variables (coordinates), according to their
of the algorithm on multi-modal functions. Hence we do not sensitivities ai .
include experiments in multi-modal functions and we present Figure 5 shows a single run of UH-CMA-ES, switching
experiments on several unimodal functions with uncertainty. from LC C
sphere to Lelli after 3000 function evaluations, and back
These functions can also be interpreted as rugged, highly again after 6000 function evaluations. Here, tmin = 1, tmax =
multi-modal (non-noisy) functions, because any single solution 10 and teval is shown in the upper left. The initial σ is chosen
is (virtually) never evaluated twice. The “re-evaluations” are far too small. Consequently σ increases from 10−2 to 2 in the
−1
conducted with a slightly mutated solution (compare point beginning. During convergence on LC sphere σ drops to 2×10 ,
3 in the uncertainty measurement algorithm). Therefore no while teval increases to the upper bound tmax = 10. When the
difference between “stochastic” and “frozen noise” can be objective function is switched teval drops to the lower bound,
observed. because the uncertainty becomes negligible when compared
The test functions obey to the differences in the function values f . In addition σ
n increases fast, because the optimum has been displaced. At
X
L(x) = f (x) + Nf (x) = ai (xi − b)2 + N/tβeval , (14) about 4000 function evaluations teval starts to increase again,
i=1 because the uncertainty term becomes significant. As expected,
the covariance matrix adapts to the new topography and the
where ai , b ∈ R are chosen function-dependent and the
x-variables move to the new optimum, arranged according
uncertainty term is independent of x but scaled with tβeval , and
to their relevance. Switching back to LC sphere reveals a sim-
β > 0. While this test function is additively decomposable and
ilar picture. The step-size increases, teval decreases, and the
hence unrealistic simple, all our simulation results also hold
isotropic topography has to be re-learned. Because there is no
for non-decomposable (rotated) versions. Three functions are
distinct coordinate system the re-learning takes as much time
derived, where β = 0.5.
as learning the elliptic topography starting from a spherical
LCspherethe isotropic sphere function, where ai = 1, for distribution. This procedure can be naturally accelerated by
i = 1, . . . , n, b = 0, and N is standard Cauchy reseting the covariance matrix. Further experiments, conducted
distributed. i−1 with larger dimensions, on non-quadratic objective functions,
Lelli the ellipsoid function, where ai = 106× n−1 , b = 0, and with x-dependent uncertainty terms, give similar results
and N is standard normally distributed. The condi- (not shown).
tion number is 106 . The principal axis lengths are
equidistant on the log scale.
VI. E XPERIMENTAL R ESULTS
LCelli the ellipsoid function Lelli , where b = 5, and N is
standard Cauchy distributed. A. Implementation of the Algorithm on the Test Rig
Figure 3 shows five independent runs on Lelli in 10D. Pre- The UH-CMA-ES delivers a set of controller parameters
mature convergence is observed without uncertainty handling
√ to be evaluated together with a requested function evaluation
(CMA-ES, left). The smallest standard deviation σ λmin , time. The controller parameters undergo an affine transfor-
where λmin is the smallest eigenvalue of C, exhibits a drift mation from the interval [0, 1] onto respective intervals spec-
with all values clearly below 10−4 . With uncertainty handling ified below, and the initial values are set to the middle of
(UH-CMA-ES, right) we choose tmin = tmax = 1 here for the interval. The controller is assembled and written to the
simplicity, implying constant teval = 1. Therefore only σ is real-time board. In order to avoid any risks stemming from
changed by √the uncertainty treatment. The smallest standard inappropriate parameter settings delivered by the algorithm,
deviation σ λmin reaches a stationary value and does not the gain of the new controller is ramped up over the course
drop below 10−4 . If teval is chosen much larger such that of two seconds, such that the human operator can intervene
no rank changes occur (non-noisy case), about 20% fewer in case of a developing harmful situation. After the data
function evaluations are required to reach a function value acquisition has been completed, the controller gain is ramped
of 1 (not shown). Note that even though the smallest standard down, and an intermediate controller keeps the combustor in
deviation is larger than in cases without uncertainty handling, a stable regime. Meanwhile, pressure data is logged, a new
the function values are clearly better and give first evidence controller is developed and transferred to the real-time board.
that the uncertainty handling works effectively. The total cycle time thus consists of ramping the controller
Figure 4 shows a single run of UH-CMA-ES on Lelli in gain up and down (about 2 s each), pressure data acquisition
8D. The course of σ (upper left) reveals that the uncertainty (determined by the algorithm, 1-10 s), data logging (1 s) and
treatment enters after about 2000 function evaluations. After UH-CMA-ES computation time (negligible). The maximum
about 5000 function evaluations the adaptation of the covari- time that pressure can be logged is currently limited to 10 s,
ance matrix is completed. The eigenvalues of the covariance due to real-time board memory constraints. The controller is
matrix correspond to the inverse coefficients a−1 i of Lelli sampled at 10 kHz, the frequency content of the pressure signal
and indicate an almost perfect adaptation to the function warrants no aliasing effects during sampling. For the following
topography, despite the noisy environment. In view of the fact experiments, the preheat temperature and the mass flow are
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 11
2 2
10 10
function value
function value
1 1
10 10
0 0
10 10
−1 −1
10 10
0 10000 20000 0 10000 20000
function evaluations function evaluations
0 0
10 10
smallest standard deviation
−4 −4
10 10
−6 −6
10 10
−8 −8
10 10
0 10000 20000 0 10000 20000
function evaluations function evaluations
Fig. 3. Five runs on Lelli , where n = 10, and tmin = tmax = 1. Left: CMA-ES without uncertainty handling; right: UH-CMA-ES (with uncertainty
handling); above: function value of the population mean, f (m); below: standard deviation in the smallest principal axis. The uncertainty handling keeps the
smallest standard deviation above 10−4 and prevents premature convergence.
kept constant at 700 K and 36 g/s, respectively. Two values Lps; λ=2.1
for the air/fuel ratio λ are investigated, namely λ = 2.1 and
λ = 1.875. 150
4
10 15
5
0
10
0
−2
10 −5
0 5000 10000 15000 0 5000 10000 15000
function evaluations function evaluations
standard deviations in coordinates
0
10
−2
10
−2
10
−4
10
−4 −6
10 10
0 5000 10000 15000 0 5000 10000 15000
function evaluations function evaluations
Fig. 4. A single run of the UH-CMA-ES (with uncertainty handling) on Lelli , where n = 8, and tmin = tmax = 1. Upper left: function value of the
distribution mean, f (m) (thick line with dots), step-size σ (mostly increasing line), ratio between largest
√ and smallest principal axis length of C (flattening
line). Upper right: components of mean vector m. Lower left: coordinate-wise standard deviations σ × Cii , where Cii is the i-th diagonal element of C.
Lower right: square root of eigenvalues of C
Leq [dB] Leq and teval teval [s] controller parameters (2D)
CMA-ES algorithm are −1 × 10−7 and 1.5 ms, respectively. 10
During the first 4800 seconds Leq rises as the combustor 147.2 Gain
147 Delay
heats up, and the optimal value of the gain increases from 0.4
146.8 5
about −2.5 × 10−4 at 1000 s to −1.8 × 10−4 at 4800 s. The
146.6 0.2
rise of Leq is related to the persistent change of the system
146.4 Leq teval
conditions during heat-up and seems to have no adverse effect 146.2 0 0
on the optimization. During the first 1000 s the evaluation time 0 2000 4000 6000 8000 0 2000 4000 6000 8000
time [s] time [s]
increases and reaches 10 s, the maximum allowed. That means
uncertainty is becoming an issue. The standard deviations std in coordinates principal axes lengths of C
0
decrease during the first 4000 seconds and rise again as Gain 10 axis 1
the operating condition is changed. At 4800 s, the operating Delay axis 2
−1
condition is changed from λ = 2.1 to λ = 1.875, and the 10
evaluation time is manually set to 1 s.
Four cost function landscapes for different time intervals are 0 2000 4000 6000 8000 0 2000 4000 6000 8000
shown in Fig. 8. They are obtained by Delauney triangulation time [s] time [s]
4
10 6
2
0
10
0
−2
10 −2
0 2000 4000 6000 8000 0 2000 4000 6000 8000
function evaluations function evaluations
1
10
standard deviations in coordinates
0
10
−2
−1
10
10
−4
−2 10
10
−3 −6
10 10
0 2000 4000 6000 8000 0 2000 4000 6000 8000
function evaluations function evaluations
Fig. 5. A single run of the UH-CMA-ES switching between LC C
sphere and Lelli at 3000 and 6000 function evaluations, where n = 5, β = 0.5 (see Equation
0.5
(14)), initial σ = 10−2 , initial teval = 1. The graphs remain identical for any β > 0 given αt = 1.5 β . Upper left: function value √
of the distribution mean
f (x) (line with dots), and teval . Upper right: components of mean vector m. Lower left: coordinate-wise standard deviations σ × Cii , where Cii is the
i-th diagonal element of C. Lower right: square root of eigenvalues of C
are now narrowed down to the smaller black polygon. If this C. Experiment: H∞ Controller, Two Parameters Optimized,
result is compared to the last plot showing function evaluations λ = 1.875.
325-390 (3800-4800 s), the optimal values for the gain and
An H∞ controller has been designed for the operating con-
the delay are confirmed, but the cost function evaluated Leq
dition with λ = 1.875, where the goal was to simultaneously
rises. This is in accord with the observation that the combustor
decrease the three peaks at 250 Hz and around 330 Hz. In
exhibits slowly rising sound pressure levels for λ = 2.1.
order to keep the number of parameters small and to speed
up convergence, only the gain and the frequency shift are
At run 395 (4800 s), the lambda value is changed to optimized.
λ = 1.875. This operating conditions exhibits less thermal In the top plot of Fig. 10 the intervals for frequency shift,
drift than the previous one. According to Fig. 9 the changing [0.95, 1.05], and gains [0.4, 1.1], are mapped onto [0, 1]. The
operating conditions can clearly be discerned in the cost comparatively good values for Leq in the beginning are related
function Leq . The algorithm finds a new minimum, where the to the short evaluation time. The shorter the evaluation time
gain can be more negative for this case. is, the larger is its variation due to the uncertainty. Therefore
better values occur more often. The bottom plot shows the cost
The evaluation time increases immediately again, indicating function landscape. It is obtained by DACE, a Matlab toolbox
no big improvement of the signal-to-noise ratio, even though for working with kriging approximations, which has been
the controller is less close to its optimal regime. This suggests kindly provided by Hans Bruun Nielsen from the Technical
that σ should be increased together with teval . The course of University of Denmark3 . A second-order polynomial has been
σ supports this conjecture. It increases by a factor of three used as regression model and a Gaussian correlation. For this
and shows the adaptive capability of the algorithm. It takes experiment with an H∞ controller, the cost function is flatter
8 generations until the increase of σ appears, and we do not than with the Gain-Delay controller, because the controller is
know whether this reflects a sensible adaptation or whether σ model-based, and thus already performs well. However, the
should have increased beforehand. Finally, the UH-CMA-ES optimization shows that in order to decrease Leq , the gain has
successfully adjusts the controller parameters to new improved
values, as shown in Fig. 7. 3 http://www2.imm.dtu.dk/∼hbn/dace/
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 14
Leq; λ=2.1; f.evals 1−150 Leq [dB] Leq; λ=2.1; f.evals 395−900 Leq [dB]
1.4 1.4
149.5 147.8
1.2
1.2
1 149 147.6
Delay [ms]
Delay [ms]
0.8 148.5
147.4
0.8
0.6 148
0.6 147.2
0.4
147.5
0.2 0.4 147
−3 −2.5 −2 −1.5 −1 −0.5 0.2 146.8
Gain x 10
−4
−4 −3 −2 −1
Leq; λ=2.1; f.evals 150−250 Leq [dB] Gain −4
x 10
1.4 Fig. 9. UH-CMA-ES optimization of a Gain-Delay controller for λ = 1.875.
149.5 Function evaluations 395 − 740 (4900-9800 s), see Fig. 8 for explanations
1.2
1 149
Delay [ms]
0.8 148.5 to be reduced from the design value of 1 to about 0.7. This
is explained by the fact that the H∞ -design process has been
0.6 148 laid out primarily to decrease the three peaks in the spectrum,
0.4 without special concern given to the reduction of Leq .
147.5
0.2
−3 −2.5 −2 −1.5 −1 D. Experiment: H∞ Controller, Three Parameters Optimized,
Gain x 10
−4
λ = 1.875.
Leq; λ=2.1; f.evals 250−325 Leq [dB]
For the following experiment, three parameters are opti-
1.4
mized by the UH-CMA-ES, namely gain, frequency shift and
149.5 time delay of the H∞ controller.
1.2 The evolution of the parameters is shown in Fig. 11
1 149 (frequency shift interval [0.95, 1.05], gains [0.4, 1.1], delays
Delay [ms]
[1, 10]). Since the cost function takes three arguments, only
0.8 148.5
four cost function landscapes with fixed delays of 0.1 ms to
0.6 148 0.4 ms are shown in Fig. 12. The topmost plot corresponds to
0.4 the bottom plot of Fig. 10, where only frequency shift and gain
147.5 are adjusted, but the delay is kept at 0.1 ms for all experiments.
0.2
Gain and frequency shift have similar values but exhibit a
−3 −2.5 −2 −1.5 −1 larger variation. The minimum Leq is lower for a delay of
Gain x 10
−4
0.2 ms, and even lower for a delay of 0.3 ms, while it increases
Leq; λ=2.1; f.evals 325−390 Leq [dB] again for 0.4 ms (bottom plot).
The Bode plots of the designed and optimized H∞ con-
1.4 trollers are shown in Fig. 13. The superior performance of
149.5
1.2 the H∞ controller goes hand in hand with a more complex
1 149 shape. The optimized controller has nearly the same phase as
Delay [ms]
the designed one, but the gain is lower. Since the delay is
0.8 148.5 adjusted additionally, it is possible to move the controller in
0.6 148 the frequency domain keeping the same phase.
0.4 As a result for the operating condition characterized by λ =
147.5 1.875, the spectra achieved with the optimized Gain-Delay and
0.2
H∞ controllers are compared to the uncontrolled plant in Fig.
−3 −2.5 −2 −1.5 −1
Gain −4 14. The Leq of the uncontrolled plant is 148.72 dB, the Gain-
x 10 Delay controller reduces it to 146.67 dB, and finally the H∞
Fig. 8. UH-CMA-ES optimization of a Gain-Delay controller for λ = 2.1, controller reaches 146.16 dB, which is about 15% lower again.
heating up. Function evaluations, from top plot down: 1-150; 150-250; 250- Moreover, the H∞ controller is able to simultaneously push
325; 325-390. Pentagrams show the best parameter set for each generation,
the larger they are, the later they have been acquired for each plot. The black down all three peaks and to attain the flattest spectrum. This is
polygon is the convex hull of all controller parameter values tried in the given achieved thanks to the model-based approach, conferring the
time range. most design freedom to the engineer.
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 15
Leq [dB] Leq and teval teval [s] controller parameters (2D)
10 Leq; delay 0.1 ms; λ=1.875 Leq [dB]
1.2 Freq. Shift
146.2 1 Gain 1.05
0.8
146 5
0.6 146.7
145.8 0.4
Leq teval
H∞ frequency shift
0.2
0 1000 2000 3000 4000
0
0 1000 2000 3000 4000
146.6
time [s] time [s]
146.3
−1
10 −0.8
10 0.95
0 1000 2000 3000 4000 0 1000 2000 3000 4000 0.4 0.6 0.8 1
time [s] time [s] Gain
Leq; λ=1.875 Leq [dB] Leq; delay 0.2 ms; λ=1.875 Leq [dB]
1.05
1.04
146.7 146.7
H∞ frequency shift
H∞ frequency shift
1.02 146.6 146.6
146.5 1 146.5
1
146.4 146.4
0.98 146.3
146.3
0.95
0.96 0.4 0.6 0.8 1
0.4 0.6 0.8 1 Gain
Gain
Leq; delay 0.3 ms; λ=1.875 Leq [dB]
Fig. 10. Parameter evolution for the UH-CMA-ES optimization of a H∞
controller for λ = 1.875 (top) and the resulting cost function landscape 1.05
(bottom). Pentagrams show the best parameter set for each generation.
146.7
H∞ frequency shift
Leq [dB] Leq and teval teval [s] controller parameters (3D)
146.6
10
1
146.4 1 146.5
0.5
5
146.2 0 146.4
Freq. Shift
Gain
Leq teval −0.5
146 Delay 146.3
0
0 2000 4000 6000 0 2000 4000 6000
time [s] time [s] 0.95
0.4 0.6 0.8 1
std in coordinates principal axes lengths of C Gain
0
10 axis 1 Leq; delay 0.4 ms; λ=1.875 Leq [dB]
axis 2
axis 3
1.05
Delay
10
−0.9
146.6
0 2000 4000 6000 0 2000 4000 6000
time [s] time [s]
1 146.5
Fig. 11. Parameter evolution for the UH-CMA-ES optimization of a H∞
controller for λ = 1.875. 146.4
146.3
H∞ and CMA−ES
Lps; λ=1.875
−60 uncontrolled
130
C(jω)dB
Gain−Delay
−70 H∞
125
−80
Lps [dB]
120
0 100 200 300 400 500
115
π
110
∠C(jω)
0 105
H∞ designed 0 100 200 300 400 500
Frequency [Hz]
H∞ CMA−ES
Fig. 14. Comparison of the pressure spectra when the plant is uncontrolled,
−π Gain-Delay and H∞ controlled. Both controllers are UH-CMA-ES optimized,
0 100 200 300 400 500 λ = 1.875.
Frequency [Hz]
Fig. 13. Bode plots for the designed and the UH-CMA-ES-optimized
H∞ controllers for λ = 1.875. Amplitude (top) and phase (bottom) of the Leq [dB] Leq and teval teval [s] controller parameters (2D)
controller. 10 0.8
147.8 Freq. Shift
147.6 0.6 Gain
gain interval [0.4, 1.1]), and the cost function landscape at 147.4 5 0.4
147.2 0.2
the bottom. A lower gain decreases Leq compared with the
147 Leq teval
designed controller with gain one, but the frequency shift 0
0
does not have a decisive effect. The final standard deviations 0 1000 2000 3000 4000 0 1000 2000 3000 4000
time [s] time [s]
for gain and frequency shift differ by a factor of about
three (lower left of the top plot of Fig. 15) reflecting the std in coordinates principal axes lengths of C
0
10
different sensitivities of the parameters. The Bode plots of the axis 1
designed and the optimized controllers are shown in Fig. 16. axis 2
The controller phase is quite flat and therefore tolerant against
frequency shifts. The combustor pressure spectrum exhibits Freq. Shift
−1
10
only one very distinct peak, and it suffices to provide the right Gain
amount of gain and phase at this frequency. 0 1000 2000 3000 4000 0 1000 2000 3000 4000
time [s] time [s]
To compare the UH-CMA-ES optimized Gain-Delay and
H∞ controllers with the uncontrolled plant, their spectra are Leq; λ=2.1 Leq [dB]
plotted in Fig. 17, the values of Leq are 159.87 dB, 147.48 1.05
dB and 147.35 dB, respectively. They are shown for the plant 148.8
which has been running for several hours and is thus heated up. 148.6
H∞ frequency shift
H∞ and CMA−ES term without additional function evaluations and with only
minor computational effort.
The algorithm has been validated on test functions and it
−70 has been applied to the optimization of feedback controllers
C(jω)dB
100 R EFERENCES
0 100 200 300 400 500
Frequency [Hz] [1] M. Ahmad, L. Zhang, and J. Readle. On-line genetic algorithm
tuning of a pi controller for a heating system. In Proceedings of
GALESIA—genetic algorithms in engineering systems: innovations and
Fig. 17. Comparison of the uncontrolled with the Gain-Delay and H∞ applications, pages 510–515, 1997.
controlled plant for λ = 2.1. Both controllers are optimized by UH-CMA- [2] A. Aizawa and B. Wah. Scheduling of genetic algorithms in noisy
ES. environment. Evolutionary Computation, pages 97–122, 1994.
[3] A. M. Annaswamy and A. F. Ghoniem. Active control of combustion
instability: Theory and practice. IEEE Control Systems Magazine,
22(6):37–54, 2002.
The novel uncertainty-handling algorithm needs few addi- [4] D. Arnold. Weighted multirecombination evolution strategies.
tional cost function evaluations per generation and is therefore Theoretical Computer Science, 1(361):18–37, 2006.
well suited for online applications. The algorithm distinguishes [5] D. V. Arnold. Noisy Optimization with Evolution Strategies, volume 8.
Kluwer, Boston, 2002.
between uncertainty measurement and uncertainty treatment. [6] D. V. Arnold and H.-G. Beyer. Local performance of the (µ/µi , λ)-
The uncertainty measurement is based on rank changes in- es in a noisy environment. In W. Martin and W. Spears, editors,
duced by reevaluations of solutions. Two adaptive uncertainty Foundations on Genetic Algorithms FOGA, pages 127–142. Morgan
Kaufmann, 2000.
treatments are proposed: increasing the time for evaluating the [7] A. Auger and N. Hansen. Performance evaluation of an advanced local
controller up to a prescribed bound, and increasing the pop- search evolutionary algorithm. In Proceedings of the IEEE Congress on
ulation diversity. Both treatments improve the signal-to-noise Evolutionary Computation, 2005.
[8] A. Banaszuk, K. B. Ariyur, M. Krstić, and C. A. Jacobson. An
ratio. The former reduces the uncertainty variance comparable adaptive algorithm for control of combustion instability. Automatica,
to resampling of solutions, while the latter improves the signal 40(11):1965–1972, 2004.
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 18
[9] H.-G. Beyer. Toward a Theory of Evolution Strategies: Some Asymp- [32] G. Harik, E. Cantu-Paz, D. E. Goldberg, and B. L. Miller. The gambler’s
totical Results from the (1,+ λ)-Theory. Evolutionary Computation, ruin problem, genetic algorithms, and the sizing of the populations.
1(2):165–188, 1993. Evolutionary Computation, 7:231–253, 1999.
[10] H.-G. Beyer. Evolutionary Algorithms in Noisy Environments: Theoret- [33] Y. Jin and J. Branke. Evolutionary optimization in uncer-
ical Issues and Guidelines for Practice. Computer Methods in Applied tain environments—a survey. IEEE Transactions on Evolutionary
Mechanics and Engineering, 186(2–4):239–267, 2000. Computation, 9(3):303–318, 2005.
[11] H.-G. Beyer and D. Arnold. Qualms regarding the optimality of cumula- [34] S. Kern, N. Hansen, and P. Koumoutsakos. Local meta-models for
tive path length control in CSA/CMA-evolution strategies. Evolutionary optimization using evolution stratgies. In T. P. Runarsson et al., editors,
Computation, 11(1):19–28, 2003. Parallel Problem Solving from Nature—PPSN IX, Proceedings, pages
[12] H.-G. Beyer and K. Deb. On self-adaptive features in real-parameter evo- 939–948. Springer, 2006.
lutionary algorithms. IEEE Transactions on Evolutionary Computation, [35] M. Krstić, A. Krupadanam, and C. Jacobson. Self-tuning control of
5(3):250–270, 2001. a nonlinear model of combustion instabilities. IEEE Transactions on
[13] R. Blonbou, A. Laverdant, S. Zaleski, and P. Kuentzmann. Active Control Systems Technology, 7(4):424–436, 1999.
adaptive combustion control using neural networks. Combustion Science [36] T. L. Lai, H. Robbins, and K. F. Yu. Adaptive choice of mean or median
and Technology, 156:25–47, 2000. in estimating the center of a symmetric distribution. Proc. Natl. Adac.
[14] J. Branke. Creating robust solutions by means of evolutionary algo- Sci., 80(18):5803–5806, 1983.
rithms. In A. Eiben, T. Bäck, M. Schoenauer, and H.-P. Schwefel, [37] A. H. Lefebvre. Gas turbine combustion. Taylor and Francis, Philadel-
editors, Parallel Problem Solving from Nature—PPSN V, Proceedings, phia, 2nd edition, 1999.
pages 119–128. Springer, Berlin, 1998.
[38] T. Lieuwen and V. Yang. Combustion Instabilities in Gas Turbine
[15] J. Branke and C. Schmidt. Selection in the presence of noise. In
Engines: Operational Experience, Fundamental Mechanisms, And
E. Cantu-Paz, editor, Genetic and Evolutionary Computation Conference
Modeling. Progress in Astronautics and Aeronautics, Vol. 210. AIAA,
(GECCO’03), pages 766–777. LNCS 2273, Springer, JUL 2003.
2005.
[16] J. Branke and C. Schmidt. Selection in the presence of noise. In
G. Goos, J. Hartmanis, and J. van Leeuwen, editors, Genetic and [39] D. A. Linkens and H. O. Nyongesa. Genetic Algorithms For
Evolutionary Computation Conference, GECCO, Proceedings, pages Fuzzy Control.1. Offline System-Development And Application. Iee
766–777. Springer, 2003. Proceedings-Control Theory And Applications, 142(3):161–176, 1995.
[17] J. Branke, C. Schmidt, and H. Schmeck. Efficient fitness estimation [40] D. A. Linkens and H. O. Nyongesa. Genetic Algorithms For
in noisy environments. In Genetic and Evolutionary Computation Fuzzy Control.2. Online System-Development And Application. Iee
Conference (GECCO). Morgan Kaufmann, 2001. Proceedings-Control Theory And Applications, 142(3):177–185, 1995.
[18] E. Cantu-Paz. Adaptive sampling for noisy problems. In K. e. a. [41] L. Ljung. System Identification, Theory for the User. Prentice Hall,
Deb, editor, Proceedings of the Genetic and Evolutionary Computation 2nd edition, 1999.
Conference (GECCO-2004), pages 947–958, Berlin Heidelberg, 2004. [42] S. Markon, D. V. Arnold, T. Baeck, T. Beielstein, and H.-G. Beyer.
Springer. Thresholding - a selection operator for noisy ES. In Proceedings of the
[19] A. Chipperfield and P. Fleming. Multiobjective gas turbine engine 2001 Congress on Evolutionary Computation CEC2001, pages 465–472,
controller design using genetic algorithms. IEEE Transactions on COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul,
Industrial Electronics, 43(5):583–587, 1996. Korea, 27-30 2001. IEEE Press.
[20] N. V. Dakev, J. F. Whidborne, A. J. Chipperfield, and P. J. Flem- [43] H. Moriyama and K. Shimizu. On-line optimisation of culture tem-
ing. Evolutionary H-infinity design of an electromagnetic suspension perature for ethanol fermentation using a genetic algorithm. Journal of
control system for a maglev vehicle. Proceedings of the Institution Chemical Technology and Biotechnology, 66(3):217–222, 1996.
of Mechanical Engineers Part I- Journal of Systems and Control [44] A. S. P. Niederberger, B. B. H. Schuermans, and L. Guzzella. Modeling
Engineering, 211(5):345–355, 1997. and active control of thermoacoustic instabilities. In 16th IFAC World
[21] A. P. Dowling and A. S. Morgans. Feedback control of combustion Congress, Prague, 2005.
oscillations. Annual Review Of Fluid Mechanics, 37:151–182, 2005. [45] N. S. Nise. Control systems engineering. Bejamin/Cummings Pub. Co.,
[22] S. Evesque. Adaptive Control of Combustion Oscillations. PhD thesis, Redwood City, Calif., 2nd edition, 1995. Norman S. Nise. ill.; 24 cm.
University of Cambridge, 2000. [46] C. O. Paschereit and E. Gutmark. Proportional control of combustion
[23] S. Evesque, A. M. Annaswamy, S. Niculescu, and A. P. Dowl- instabilities in a simulated gas-turbine combustor. Journal of Propulsion
ing. Adaptive control of a class of time-delay systems. Journal of and Power, 18(6):1298–1304, 2002.
Dynamic Systems Measurement and Control-Transactions of the ASME, [47] C. O. Paschereit, E. Gutmark, and W. Weisenstein. Coherent structures
125(2):186–193, 2003. in swirling flows and their role in acoustic combustion control. Physics
[24] S. Evesque, A. P. Dowling, and A. M. Annaswamy. Self-tuning of Fluids, 11(9):2667–2678, 1999.
regulators for combustion oscillations. Proceedings of the Royal Society [48] C. O. Paschereit, B. B. H. Schuermans, and D. Büche. Combustion
of London Series A- Mathematical Physical and Engineering Sciences, process optimization using evolutionary algorithm. In 2003 ASME
459(2035):1709–1749, 2003. Turbo Expo, volume 2, pages 281–291, Atlanta, USA, 2003. ASME,
[25] P. J. Fleming and R. C. Purshouse. Evolutionary algorithms in International Gas Turbine Institute.
control systems engineering: a survey. Control Engineering Practice,
[49] C. O. Paschereit, B. B. H. Schuermans, and D. U. Campos Delgado.
10(11):1223–1241, 2002.
Active combustion control using an evolution algorithm. In 39th
[26] U. Hammel and T. Bäck. Evolution strategies on noisy functions: How
Aerospace Sciences Meeting and Exhibit, Reno, USA, 2001.
to improve convergence properties. In Y. Davidor, H.-P. Schwefel,
and R. Männer, editors, Proceedings of theInternational Conference [50] L. Rayleigh. The explanation of certain acoustical phenomena. Nature,
on Evolutionary Computation, theThirdConference on Parallel Problem 18:319:321, 1878.
Solving from Nature(PPSN III), volume 866, pages 159–168, Jerusalem, [51] I. Rechenberg. Evolutionsstrategie &94. Frommann-Holzboog, 1994.
9–14 1994. Springer. [52] A. J. Riley, S. Park, A. P. Dowling, S. Evesque, and A. M. Annaswamy.
[27] N. Hansen. Verallgemeinerte individuelle Schrittweitenregelung in der Adaptive closed-loop control on an atmospheric gaseous lean-premixed
Evolutionsstrategie. Mensch und Buch Verlag, Berlin, 1998. combustor. In 2003 ASME Turbo Expo, volume 2, pages 347–358,
[28] N. Hansen and S. Kern. Evaluating the CMA evolution strategy on mul- Atlanta, USA, 2003. ASME, International Gas Turbine Institute.
timodal test functions. In X. Yao et al., editors, Parallel Problem Solving [53] A. J. Riley, S. Park, A. P. Dowling, S. Evesque, and A. M. An-
from Nature - PPSN VIII, LNCS 3242, pages 282–291. Springer, 2004. naswamy. Advanced closed-loop control on an atmospheric gaseous
[29] N. Hansen, S. D. Müller, and P. Koumoutsakos. Reducing the time lean-premixed combustor. Journal Of Engineering For Gas Turbines
complexity of the derandomized evolution strategy with covariance And Power-Transactions Of The ASME, 126(4):708–716, 2004.
matrix adaptation (CMA-ES). Evolutionary Computation, 11(1):1–18, [54] Y. Sano and H. Kita. Optimization of noisy fitness functions by means
2003. of genetic algorithms using history of search. In M. Schoenauer, K. Deb,
[30] N. Hansen and A. Ostermeier. Adapting arbitrary normal mutation G. Rudolph, X. Yao, E. Lutton, J. J. M. Guervós, and H.-P. Schwefel,
distributions in evolution strategies: The covariance matrix adaptation. In editors, PPSN, volume 1917 of Lecture Notes in Computer Science,
Proceedings of the 1996 IEEE Conference on Evolutionary Computation pages 571–580. Springer, 2000.
(ICEC ’96), pages 312–317, 1996. [55] B. B. H. Schuermans. Modeling and Control of Thermoacoustic
[31] N. Hansen and A. Ostermeier. Completely derandomized self-adaptation Instabilities. PhD thesis, Ecole Polytechnique Fédérale de Lausanne,
in evolution strategies. Evolutionary Computation, 9(2):159–195, 2001. Switzerland, 2003.
PAPER DRAFT, FINAL VERSION ACCEPTED FOR PUBLICATION TO IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 19
A PPENDIX
A. Matlab Implementation of the Uncertainty Measurement
%%% compute rank change limits using both ranks(1,...) and ranks(2,...)
for i = 1:lamreev
sumlim(i) = ...
prctile(abs((1:2*lam-1) - (ranks(1,i) - (ranks(1,i)>ranks(2,i)))), ...
theta*50) ...
+ prctile(abs((1:2*lam-1) - (ranks(2,i) - (ranks(2,i)>ranks(1,i)))), ...
theta*50);
end