This article was downloaded by: [134.245.254.

52] On: 13 July 2017, At: 03:44

Publisher: Institute for Operations Research and the Management Sciences (INFORMS)
INFORMS is located in Maryland, USA

INFORMS Journal on Computing

Publication details, including instructions for authors and subscription information:
http://pubsonline.informs.org

Scatter Search and Local NLP Solvers: A Multistart

Framework for Global Optimization
Zsolt Ugray, Leon Lasdon, John Plummer, Fred Glover, James Kelly, Rafael Mart,

To cite this article:

Zsolt Ugray, Leon Lasdon, John Plummer, Fred Glover, James Kelly, Rafael Mart, (2007) Scatter Search and Local NLP Solvers:
A Multistart Framework for Global Optimization. INFORMS Journal on Computing 19(3):328-340. https://doi.org/10.1287/
ijoc.1060.0175

Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions

This article may be used only for the purposes of research, teaching, and/or private study. Commercial use
or systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisher
approval, unless otherwise noted. For more information, contact permissions@informs.org.

The Publisher does not warrant or guarantee the articles accuracy, completeness, merchantability, fitness
for a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, or
inclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, or
support of claims made of that product, publication, or service.

Copyright 2007, INFORMS

Please scroll down for articleit is on subsequent pages

INFORMS is the largest professional society in the world for professionals in the fields of operations research, management
science, and analytics.
For more information on INFORMS, its publications, membership, or meetings visit http://www.informs.org
 INFORMS Journal on Computing informs
Vol. 19, No. 3, Summer 2007, pp. 328340
issn 1091-9856
eissn 1526-5528
07
1903
0328 doi 10.1287/ijoc.1060.0175
2007 INFORMS

Scatter Search and Local NLP Solvers: A Multistart

Framework for Global Optimization
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.

Zsolt Ugray
Management Information Systems Department, Utah State University, Logan, Utah 84322,
zsolt.ugray@usu.edu

Leon Lasdon
Department of Information, Risk, and Operations Management, McCombs School of Business,
The University of Texas at Austin, Austin, Texas 78712, lasdon@mail.utexas.edu

John Plummer
Department of Computer Information Systems and Quantitative Methods, Texas State University,
San Marcos, Texas 78666, jp05@txstate.edu

Fred Glover
University of Colorado, Boulder, Colorado 80309, fred.glover@colorado.edu

James Kelly
OptTek Systems, Inc., Boulder, Colorado 80302, kelly@opttek.com

Rafael Mart
Departamento de Estadstica e Investigacin Operativa, University of Valencia,
46100 Burjassot, Valencia, Spain, rafael.marti@uv.es

T he algorithm described here, called OptQuest/NLP or OQNLP, is a heuristic designed to nd global optima
for pure and mixed integer nonlinear problems with many constraints and variables, where all problem
functions are differentiable with respect to the continuous variables. It uses OptQuest, a commercial implemen-
tation of scatter search developed by OptTek Systems, Inc., to provide starting points for any gradient-based
local solver for nonlinear programming (NLP) problems. This solver seeks a local solution from a subset of these
points, holding discrete variables xed. The procedure is motivated by our desire to combine the superior accu-
racy and feasibility-seeking behavior of gradient-based local NLP solvers with the global optimization abilities
of OptQuest. Computational results include 155 smooth NLP and mixed integer nonlinear program (MINLP)
problems due to Floudas et al. (1999), most with both linear and nonlinear constraints, coded in the GAMS
modeling language. Some are quite large for global optimization, with over 100 variables and 100 constraints.
Global solutions to almost all problems are found in a small number of local solver calls, often one or two.
Key words: global optimization; multistart heuristic; mixed integer nonlinear programming; scatter search;
gradient methods
History: Accepted by Michel Gendreau, Area Editor for Heuristic Search and Learning; received August 2002;
revised January 2005, September 2005; accepted January 2006. Published online in Articles in Advance
July 20, 2007.

1. Introduction uses the generalized reduced gradient NLP solver

This paper describes OQNLP, a multistart heuristic LSGRG2 (Smith and Lasdon 1992).
algorithm designed to nd global optima of smooth The most general problem this algorithm can solve
constrained nonlinear programs (NLPs) and mixed has the form
integer nonlinear programs (MINLPs). It uses the minimize f x y (1)
OptQuest Callable Library (OCL) implementation of subject to the nonlinear constraints
scatter search (Laguna and Marti 2002) to generate
trial points, which are candidate starting points for gl Gx y gu (2)
a local NLP solver. These are ltered to provide a and to the linear constraints
smaller subset from which the solver attempts to nd
l A1 x + A2 y u (3)
a local optimum. Our GAMS implementation can use
any GAMS NLP solver, and the stand-alone version x S y Y (4)
328
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS
where x is an n-dimensional vector of continuous
decision variables, y is a p-dimensional vector of dis-
crete decision variables, and the vectors gl, gu, l, and
u contain upper and lower bounds for the nonlinear
and linear constraints, respectively. The matrices A1
and A2 are m2 by n and m2 by p, respectively, and con-
tain the coefcients of any linear constraints. The set
S is dened by simple bounds on x, and we assume
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.
that it is closed and bounded, i.e., that each compo-
nent of x has a nite upper and lower bound. This
is required by the OptQuest scatter-search procedure.
The set Y is assumed to be nite, and is often the set
of all p-dimensional binary or integer vectors y that
satisfy nite bounds. The objective function f and the
m1 -dimensional vector of constraint functions G are
assumed to have continuous rst partial derivatives at
all points in S Y . This is necessary so that a gradient-
based local NLP solver can be applied to the relaxed
NLP sub-problems formed from (1)(4) by allowing
the y variables to be continuous.
2. Multistart Algorithms for Global
Optimization
In this section, which reviews past work on mul-
tistart algorithms, we focus on unconstrained prob-
lems where there are no discrete variables, since to
the best of our knowledge multistart algorithms have
been investigated theoretically only in this context.
These problems have the form of (1)(4) with no
y variables and no constraints except the bounds
x S in (4). All global minima of f are assumed to
occur in the interior of S. By multistart we mean any
algorithm that attempts to nd a global solution by
starting a local NLP solver, denoted by L, from mul-
tiple starting points in S. The most basic multistart
method generates uniformly distributed points in S,
and starts L from each of these. This converges to
a global solution with probability one as the num-
ber of points approaches innityin fact, the best
of the starting points converges as well. However,
this procedure is very inefcient because the same
local solution is located many times. A convergent
procedure that largely overcomes this difculty is
called multilevel single linkage (MLSL) (Rinnooy Kan
and Timmer 1987a, b). MLSL uses a simple rule to
exclude some potential starting points. A uniformly
distributed sample of Ns points in S is generated,
and the objective f is evaluated at each point. The
points are sorted according to their f values, and
the qNs best points are retained, where q is an algo-
rithm parameter between 0 and 1. L is started from
each point of this reduced sample, except if there is
another sample point within a certain critical distance
that has a lower f value. L is also not started from
sample points that are too near the boundary of S, or
329
too close to a previously discovered local minimum.
Then, Ns additional uniformly distributed points are
generated, and the procedure is applied to the union
of these points and those retained from previous itera-
tions. The critical distance referred to above decreases
each time a new set of sample points is added. The
authors show that, if the sampling continues inde-
nitely, each local minimum of f will be located, but
the total number of local searches is nite with proba-
bility one. They also develop Bayesian stopping rules,
which incorporate assumptions about the costs and
potential benets of further function evaluations, to
determine when to stop the procedure.
When the critical distance decreases, a point from
which L was previously not started may become a
starting point in the next cycle. Hence all sample
points generated must be saved. This also makes the
choice of the sample size Ns important, since too
small a sample leads to many revised decisions,
while too large a sample will cause L to be started
many times. Random-linkage (RL) multistart algo-
rithms introduced by Locatelli and Schoen (1999)
retain the good convergence properties of MLSL, and
do not require that past starting decisions be revised.
Uniformly distributed points are generated one at a
time, and L is started from each point with a proba-
bility given by a nondecreasing function d, where
d is the distance from the current sample point to
the closest of the previous sample points with a bet-
ter function value. Assumptions on this function that
give RL methods the same theoretical properties as
MLSL are derived in the above reference.
Recently, Fylstra et al. have implemented a ver-
sion of MLSL that can solve constrained problems
(http://www.solver.com/xlspremsolv3.htm). Limited
to problems with no discrete variables y, it uses the
L1 exact penalty function, dened as


m
P1 x w = f x + wi violgi x (5)
i=1
where the wi are nonnegative penalty weights, m =
m1 + m2 , and the vector g has been extended
to include the linear constraints (4). The function
violgi x is the absolute amount by which the ith
constraint is violated at the point x. It is well known
(Nash and Sofer 1996) that if x is a local optimum of
(1)(4), u is a corresponding optimal multiplier vec-
tor, the second order sufciency conditions are satis-
ed at x  u  and
wi > absui  (6)
then x is a local unconstrained minimum of P1 . If
(1)(4) has several local minima, and each wi is larger
than the maximum of all absolute multipliers for con-
straint i over all these optima, then P1 has a local min-
imum at each of these local constrained minima. Even
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
330
though P1 is not a differentiable function of x, MLSL
can be applied to it, and when a randomly generated
trial point satises the MLSL criterion to be a starting
point, any local solver for the smooth NLP problem
can be started from that point. The local solver need
not make any reference to the exact penalty func-
tion P1 , whose only role is to provide function val-
ues to MLSL. We will use P1 in the same way in our
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.
OQNLP algorithm. We are not aware of any theoret-
ical investigations of this extended MLSL procedure,
so it must currently be regarded as a heuristic.
3. The OQNLP Algorithm
3.1. The Global PhaseScatter Search
Scatter search (ScS) is a population-based meta-
heuristic algorithm devised to intelligently perform a
search on the problem domain (Glover 1998). It oper-
ates on a set of solutions called the reference set or
population. Elements of the population are maintained
and updated from iteration to iteration. ScS differs
from other population-based evolutionary heuristics
like genetic algorithms (GAs) mainly in its emphasis
on generating new elements of the population mostly
by deterministic combinations of previous members
of the population as opposed to the more extensive
use of randomization. ScS was founded on strategies
that were proposed as augmentations to GAs more
than a decade after their debut in ScS. It embodies
principles and strategies that are still not emulated by
other evolutionary methods and prove to be advan-
tageous for solving a variety of complex optimization
problems. For the most recent and complete descrip-
tion of ScS, see Laguna and Marti (2003). Also see
the Online Supplement to this paper on the journals
website.
3.2. The Local PhaseGradient-Based NLP
Solvers
Many papers and texts discuss gradient-based NLP
solvers, e.g., Nash and Sofer (1996), Nocedal and
Wright (1999), and Edgar et al. (2001). These solve
problems of the form (1)(4), but with no discrete y
variables. They require a starting point as input, and
use values and gradients of the problem functions
to generate a sequence of points that, under fairly
general smoothness and regularity conditions, con-
verge to a local optimum. The main classes of algo-
rithms in widespread use are successive quadratic
programming (SQP) and generalized reduced gradi-
ent (GRG); see Edgar et al. (2001, chapter 8). The algo-
rithm implemented in the widely used MINOS solver
(Murtagh and Saunders 1982) is similar to SQP. If
there are nonlinear constraints, SQP and MINOS gen-
erate a sequence of points that usually violate the non-
linear constraints, with the violations decreasing to
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS
within a specied feasibility tolerance as the sequence
converges to a local optimum. GRG algorithms have
a simplex-like phase 1phase 2 structure. Phase 1
begins with the given starting point and, if it is not
feasible, attempts to nd a feasible point by minimiz-
ing the sum of constraint violations. If this effort ter-
minates with some constraints violated, the problem
is assumed to be infeasible. However, this local opti-
mum of the phase 1 objective may not be global, so a
feasible point may exist. If a feasible point is found,
phase 2 uses it as its starting point, and proceeds to
minimize the true objective. Both phases consist of a
sequence of line searches, each of which produces a
feasible point with an objective value not worse (and
usually better) than its predecessor.
Several good commercially available implementa-
tions of GRG and SQP solvers exist; see Nash (1998)
for a review. As with any numerical-analysis software,
a local NLP solver can fail to nd a local solution from
a specied starting point. The problem may be too
badly conditioned, badly scaled, or too large for the
solver, causing it to terminate at a point (feasible or
infeasible) that is not locally optimal. While the reli-
ability of the best current NLP solvers is quite high,
these difculties occurred in our computational test-
ing, and we discuss this in more detail later.
Let L be a local NLP solver capable of solving
(1)(4), and assume that L converges to a local opti-
mum for any starting point x0 S. Let Lx0  be the
locally optimal solution found by L starting from x0 ,
and let xi , i = 1 2     nloc be all the local optima
of the problem. The basin of attraction of the ith local
optimum relative to L, denoted by Bxi , is the set
of all starting points in S from which the sequence
of points generated by L converges to xi : Bxi  =
!x0
x0 S Lx0  = xi ".
One measure of difculty of a global optimization
problem with unique global solution x1 is the volume
of Bx1  divided by the volume of the rectangle, S,
the relative volume of Bx1 . The problem is trivial if
this relative volume is 1, as it is for convex programs,
and problem difculty increases as this relative vol-
ume approaches zero.
3.3. Comparing Heuristic Search Methods and
Gradient-Based NLP Solvers
For smooth problems, the relative advantages of a
heuristic search method like ScS over a gradient-
based NLP solver are its ability to locate an approx-
imation to a good local solution (often the global
optimum), and the fact that it can handle discrete
variables. Gradient-based NLP solvers converge to
the nearest local solution, and have no facilities
for discrete variables, unless they are embedded in a
rounding heuristic or branch-and-bound method. Rel-
ative disadvantages of heuristic search methods are
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS
their limited accuracy, and their weak abilities to deal
with equality constraints (more generally, narrow fea-
sible regions). They nd it difcult to satisfy many
nonlinear constraints to high accuracy, but this is a
strength of gradient-based NLP solvers. Search meth-
ods also require an excessive number of iterations to
nd approximations to local or global optima accurate
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.
to more than two or three signicant gures, while
gradient-based solvers usually achieve four- to eight-
digit accuracy rapidly.
The motivation for combining search and gradient-
based solvers in a multistart procedure is to achieve
the advantages of both while avoiding the disadvan-
tages of either. Surprisingly, we have been unable to
locate any published efforts in this direction, besides
the Frontline extended MLSL method discussed in
Section 2.
3.4. The OQNLP Algorithm
A pseudo-code description of the simplest OQNLP
algorithm follows:
INITIALIZATION
Read_Problem_Parameters (n, p, m1 , m2 , bounds,
starting point);
Setup_OQNLP and OptQuest_parameters and Op-
tions (problem size, stage 1 and 2 iteration limits,
population size, accuracy, names and types of vari-
ables and constraints, bounds on variables and con-
straints);
Initialize_OptQuest_Population;
Stage 1 iterations = Stage 2 iterations = 0;
STAGE 1: INITIAL OPTQUEST ITERATIONS
AND FIRST L CALL
WHILE (Stage 1 iterations < Stage 1 iteration limit)
DO {
Get (trial solution from OptQuest);
Evaluate (objective and nonlinear constraint
values at trial solution,);
Put (trial solution, objective and constraint
values to OptQuest database);
Stage 1 iterations = Stage 1 iterations + 1;
} ENDDO
Get_Best_Point_from_OptQuest_database (starting
point);
Call_L (starting point, local solution);
Threshold = default value;
IF (local solution feasible) THEN {
Insert local solution in linked list;
Penalty weights = max(positive lower limit,
absolute multiplier values from L call);
threshold = P1 value of local solution; }
Penalty weights = max(positive lower limit,
absolute multiplier values from L call)
331
STAGE 2: MAIN ITERATIVE LOOP
WHILE (Stage 2 iterations < Stage 2 iteration limit)
DO {
Get (trial solution from OptQuest);
Evaluate (objective and nonlinear constraint
values at trial solution,);
Put (trial solution, objective and constraint
values to OptQuest database);
Calculate_Penalty_Function (trial solution,
Penalty weights, P1 );
IF (distance and merit lter criteria are
satised) THEN {
Replace threshold with current P1 value;
Call_L (trial solution, local solution);
IF (local solution feasible) THEN {
Insert local solution in linked list;
Penalty weights = max(positive lower
limit, absolute multiplier values
from L call); }
}
ELSE IF (P1 > threshold for waitcycle
consecutive iterations) {increase threshold}
Stage 2 iterations = Stage 2 iterations + 1;
} ENDDO
After initialization, there are two main stages. In
the initial OptQuest iterations stage, the objective
and constraint values at all trial points generated by
the initial OptQuest population (including the pop-
ulation points themselves) are evaluated, and these
values are returned to OptQuest, which computes its
penalty function, POQ , at each point. The point with
the best POQ value is selected, and L is started from
this point. If there are any discrete variables y, they
are xed at their current values during the L solution
process. Figure D in the Online Supplement shows a
graph of these trial points for a two variable uncon-
strained problem. In general, they are scattered within
the rectangle dened by the bounds on the variables,
so choosing the best corresponds to performing a
coarse search over this rectangle. If the best point falls
inside the basin of attraction of the global optimum
relative to L (as it often does), then if the subsequent
L call is successful, it will nd a global optimum.
This call also determines optimal Lagrange multiplier
values u for the constraints. These are used to deter-
mine initial values for the penalty weights wi satisfy-
ing (6), which are used in the exact penalty function,
P1 , dened in (5). All local optima found are stored in
a linked list, along with the associated Lagrange mul-
tipliers and objective values. Whenever a new local
optimum is found, the penalty weights are updated
so that (6) is satised over all known local optima.
The main iterative loop of stage 2 obtains trial
points from OptQuest, and starts L from the subset
of these points determined by two lters. The dis-
tance lter helps insure that these starting points are
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
332
diverse, in the sense that they are not too close to any
previously found local solution. Its goal is to prevent
L from starting more than once within the basin of
attraction of any local optimum, so it plays the same
role as the rule in the MLSL algorithm of Section 2,
which does not start at a point if it is within a crit-
ical distance of a better point. When the nal point
found by L is feasible, it is stored in a linked list,
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.
ordered by its objective value, as is the Euclidean dis-
tance between it and the starting point that led to
it. If a local solution is located more than once, the
maximum of these distances, maxdist, is updated and
stored. For each trial point t, if the distance between t
and any local solution already found is less than
distfactor maxdist, L is not started from the point, and
we obtain the next trial solution from OptQuest.
This distance lter implicitly assumes that the
attraction basins are spherical, with radii at least
maxdist. The default value of distfactor is 0.75, and it
can be set to any positive value. As distfactor ap-
proaches zero, the ltering effect vanishes, as would
be appropriate if there were many closely spaced local
solutions. As it increases, the ltering effect increases
until eventually L is never started in stage 2. Its de-
fault value is chosen empirically to achieve a reason-
able compromise between these extremes.
The merit lter helps insure that the L starting
points have high quality, by not starting from can-
didate points whose exact penalty function value P1
in (5) is greater than a threshold. This threshold is set
initially to the P1 value of the best candidate point
found in the rst stage of the algorithm. If trial points
are rejected by this test for more than waitcycle con-
secutive iterations, the threshold is increased by the
updating rule:
threshold threshold + threshfactor
10 + absthreshold
where the default value of threshfactor is 0.2 and that
for waitcycle is 20. The threshfactor value is selected
to provide a signicant but not-too-large increase in
threshold, while the waitcycle value insures that these
increases happen often enough but not too often.
Using the default values of 200 stage 1 and 800 stage 2
candidate points, threshold can be increased in stage 2
at most 40 times. The additive 1.0 term is included
so that threshold increases by at least threshfactor when
its current value is near zero. When a trial point is
accepted by the merit lter, threshold is decreased by
setting it to the P1 value of that point.
The combined effect of these two lters is that L
is started at only a few percent of the OptQuest trial
points, yet global optimal solutions are found for
a very high percentage of the test problems. Some
insight is gained by examining Figure 1, which shows
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS
1.0
0.2155 0.8
1.0316
0.6 1.6071
0.4
0.2
0 0
2.0 1.5 1.0 0.5 0 0.5 1.0 1.5 2.0
0.2
0.4
1.6071 0.6
1.0316
0.8 0.2155
1.0
Figure 1 Local Optima and Ten L Starting Points for Six-Hump Camel-
back Function
the stationary point at the origin and the six local
minima of the two-variable six-hump camelback func-
tion (Dixon and Szeg 1975) as dark squares, labeled
with their objective value. The ten points from which
OQNLP starts the local solver are shown as nine
white diamonds, plus the origin. The local minima
occur in pairs with equal objective value, located sym-
metrically about the origin. There were 144 trial points
generated in the initial OptQuest iterations stage,
and these, plus the ten points in the initial population,
are shown in Figure D in the Online Supplement. The
best of these 154 points is the population point 0 0,
so this becomes the rst starting point for the local
solver. This happens to be a stationary point of F , so it
satises the optimality test (that the norm of the gra-
dient of the objective be less than the optimality toler-
ance), and the local solver terminates there. The next
local solver start is at iteration 201, and this locates the
global optimum at 00898 07127, which is located
twice. The other global optimum at 00898 07127
is found rst at iteration 268, and is located six times.
The limit on total OQNLP iterations in this run was
1,000. L was started at only nine of the 846 OptQuest
trial points generated in the main iterative loop of
stage 2. All but two of the starting points are in the
basin of attraction of one of the two global optima.
This is mainly due to the merit lter. In particular, the
threshold values are always less than 1.6071, so no
starts are ever made in the basin of attraction of the
two local optima with this objective value. The merit
lter alone rejected 498 points, the distance lter alone
57, and both rejected 281.
Figure 2 illustrates the dynamics of the merit-lter-
ing process for iterations 155 to 407 of this problem,
displaying the objective values for the trial points as
white diamonds, and the threshold values as dark
lines. All objective values greater than 2.0 are set
to 2.0.
The initial threshold value is zero, and it is raised
twice to a level of 0.44 at iteration 201, where the
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS 333

2.0

1.5

1.0
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.

0.5

0.0
150 200 250 300 350 400

0.5

1.0

1.5

Figure 2 Objective and Threshold Values for Six-Hump Camelback Function for Iterations 155 to 407

trial-point objective value of 029 falls below it. L
is then started and locates the global optimum at
00898 07127, and the threshold is reset to 029.
This cycle then repeats. Nine of the ten L starts are
made in the 252 iterations shown in the graph. In
this span, there are 12 points where the merit lter
allows a start and the threshold is decreased, but L is
not started at three of these because the distance lter
rejects them.
Figure 3 shows the same information for iterations
408 to 1,000. There is only one L start in this span. This
is not due to a lack of high-quality trial points: there
are more good points than previously, many with val-
ues near or equal to 10310 (the global minimum is
10316), and the merit threshold is usually 10310
as well. Every time this threshold is raised, the merit
lter accepts one of the next trial points, but 51 of the
52 accepted points are too near one of the two global
optima, and they are rejected by the distance lter.
This simple example illustrates a number of impor-
tant points:
1. Setting the bounds on the continuous or discrete
variables to be too large in magnitude is likely to
slow the OQNLP algorithm (or any search algorithm)
and may lead to a poorer nal solution. In the above
example, if the variable bounds had been (2 2)
rather than (10 10), the trial points generated by the
initial population would have had much lower objec-
tive values. OptQuest can overcome this when the
initial population is updated.
2. L found a highly accurate approximation to the
global solution of this unconstrained problem at its

2.0

1.5

1.0

0.5

0.0
407 507 607 707 807 907

0.5

1.0

1.5

Figure 3 Objective and Threshold Values for Six-Hump Camelback Function for Iterations 408 to 1,000

Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.
, so if the values yt have not
Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
334
second call. OptQuest alone would have taken many
more iterations to achieve this accuracy.
3. The best trial point generated by the initial pop-
ulation may not have as good an objective value as
those generated from the second or succeeding ones,
especially if the variable bounds are too large. Using
the best rst-generation point as the initial L start-
ing point may not lead to as good a local solution
as if some second-generation points had been con-
sidered. For this reason our base-case computational
results use a rst stage of 200 OptQuest trial points,
which in this example would include all 144 rst-
generation points and 56 from the second generation.
3.5. Filtering Logic for Problems with Discrete
Variables
The ltering logic described above must be extended
when there are discrete variables (the y variables in
the problem statement (1)(4)). There are two distinct
modes: (1) Optquest is aware of both the x and y vari-
ables and all problem constraints, and (2) Optquest
is aware only of the y variables and the constraints
involving y only. Tests thus far do not show conclu-
sively that one of these is preferred, so both modes
are described and comparisons thus far are presented
later.
In mode (1), when a trial point xt  yt  provided by
OptQuest passes the two ltering tests and is passed
to the local solver L, xt acts as a starting point and
is changed by L, but the yt values are xed and are
not changed. Each new set of yt values denes a dif-
ferent NLP for L to solve, say NLP(yt ), with its own
set of local minima in x space, so both lters must
be made specic to NLP(yt ). For the distance lter, it
is irrelevant if xt is close to any local minima (in x
space) previously found that correspond to problems
NLP(y) with y different from yt . Hence the distance
lter is based on the distance from xt to local minima
of NLP(yt ) only. Similarly, the tests and threshold val-
ues in the merit lter must be specic to the problem
NLP(yt ) currently being solved. However, the weights
w in the exact penalty function P1 x y w used in the
merit lter are based on the maximum absolute mul-
tipliers over all local optima for all vectors yt , because
these weights are large enough to ensure that this
function is exact for all problems NLP(y).
Therefore, in stage 2 of the algorithm, the exact pen-
alty function, P1 xt  yt  w, is calculated at each trial
point xt  yt , and L is started at xt  yt  if P1 is smaller
than the current threshold for NLP(yt ). This thresh-
old is initialized to +
routines provided by GAMS Development Corpora-
occurred in a previous stage 2 trial point, L will be
called at this point. This leads to many more local
solver calls in problems with discrete variables, as we
show later in the computational-results sections.
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS
In mode (2), Optquest presents candidate y vectors
only to L, which are xed while L nds correspond-
ing (locally) optimal x values. The starting values for
x can be chosen to minimize computational effort. We
are experimenting with an option that obtains all pos-
sible trial points for the current population, sorts them
in terms of their distance from each other, and calls
L in that sorted order, starting each call of L from
the previous optimum. It is expected that ys that are
close to one another will have xs with that prop-
erty, so the previous optimum will be a good starting
point.
In mode (2), there is no stage 1, and L must be
called at each y vector that has not been produced
previously. As a result, the local solver call where
the best value is found typically comes later than
with mode (1). On the other hand, OptQuests effort
is reduced since it processes a much smaller prob-
lem, and the information returned to it by the local
solver (the optimal objective value over the continu-
ous variables) is of much higher quality than in the
base case (the penalized objective value at OptQuests
trial point).
An important option involves the return of infor-
mation from the local solver to OptQuest, which
is absent in the above procedure, i.e., local solu-
tions found by the local solver are not returned to
OptQuest. Such solutions are generally of very high
quality, and might aid the search process if they were
incorporated into the OptQuest population, because
at least a subset would likely be retained there. How-
ever, this should be done so as to preserve the diver-
sity of the population.
4. Computational Results
The algorithm described in the previous section has
been implemented as a callable C-language function.
In this form, the user supplies a C function that evalu-
ates the objective and constraint functions, an optional
routine that evaluates their rst partial derivatives
(nite-difference approximations are used otherwise),
and a calling program that supplies problem size,
bounds, an initial point, and invokes the algorithm.
Algorithm parameters and options all are set ini-
tially to default values, and any changes are speci-
ed in an options le. The local NLP solver is the
LSGRG2 implementation of Smith and Lasdon (1992).
We have also developed an interface between our C
implementation and the GAMS algebraic modeling
language (http://www.gams.com/), using C library
tion. The user function routine is replaced by one that
calls the GAMS interpreter, and a special derivative
routine accesses and evaluates expressions developed
by GAMS for rst derivatives of all nonlinear problem
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS 335

Table 1 Floudas et al. (1999) Test Problem Set Characteristics

Max. Max. discrete Max. linear Max. nonlinear

Series Problems variables variables constraints constraints Problem type

EX2_1_x 14 24 0 10 0 Concave QP (min)

EX3_1_x 4 8 0 4 6 Quadratic obj and constraints
EX4_1_x 9 2 0 0 2 Obj or constraints polynomial
EX5_2_x 2 32 0 8 11 Bilinear-pooling
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.

EX5_3_x 2 62 0 19 34 Distillation column sequencing

EX5_4_x 3 27 0 13 6 Heat exchanger network
EX6_1_x 4 12 0 3 6 Gibbs free energy min
EX6_2_x 10 9 0 3 0 Gibbs free energy min
EX7_2_x 4 8 0 3 12 Generalized geometric prog
EX7_3_x 6 17 0 10 11 Robust stability analysis
EX8_1_x 8 6 0 0 5 Small unconstrained, constrained
EX8_2_x 5 55 0 6 75 Batch plant design-uncertainty
EX8_3_x 14 141 0 43 65 Reactor network synthesis
EX8_4_x 8 62 0 0 40 Constrained least squares
EX8_5_x 6 6 0 2 2 Min tangent plane distance
EX8_6_1 N 3N 6 0 0 0 Lennard-Jones energy min
EX8_6_2 N 3N 6 0 0 0 Morse energy min
EX9_1_x 10 29 6 27 5 Bilevel LP
EX9_2_x 9 16 3 11 6 Bilevel QP
EX12_2_x 6 11 8 9 4 MINLP
EX14_1_x 9 10 0 4 17 Innity norm solution of equations
EX14_2_x 9 7 0 1 10 Innity norm solution of equations
Total: 142 + 2N

functions. GAMS identies all linear terms in each
function, and supplies their coefcients separately,
thus identifying all linear constraints. This enables us
to invoke the OptQuest option that maps each trial
point into a point that satises the linear constraints.
The derivative information supplied by GAMS signif-
icantly enhances the performance of the local solver
since only nonconstant derivatives are re-evaluated,
and these are always available to full machine pre-
cision. As mentioned earlier, this GAMS version can
call any GAMS NLP solver.
For our computational experiments we used the
large set of global optimization test problems coded
in GAMS from Floudas et al. (1999). Table 1 shows
the characteristics of 142 individual and two groups
of problems.
Most problems arise from chemical engineering,
but some are from other sources. Most are small,
but a few have over 100 variables and a comparable
numbers of constraints, and 13 have both continu-
ous and discrete variables. Almost all of the problems
without discrete variables have local solutions dis-
tinct from the global solution, and the majority of
problems have constraints. Sometimes all constraints
are linear, as with the concave quadratic programs
of series EX2_1_x, but many problems have nonlin-
ear constraints, and these are often the source of the
nonconvexities. The best known objective value and
(in most cases) the corresponding variable values are
provided in Floudas et al. (1999). The symbol N in the
rows for the series EX8_6_1 and EX8_6_2 is the num-
ber of particles in a cluster whose equilibrium cong-
uration is sought via potential-energy minimization.
(For more details on these problems see Section 4.2.)
4.1. Continuous VariablesThe Base Case
This section describes results obtained when OQNLP
is applied to 128 of the problems in the Floudas
et al. (1999) test set with no discrete variables. A few
problems for which no GAMS NLP solver can nd a
feasible solution in 800 solver calls are omitted. Com-
putations were performed on a DELL OptiPlex PC
with a 1.2 GHz Pentium IV processor and 261 Mbytes
of RAM, running under Windows 2000.
The options and main algorithm parameters used
are shown in Table 2 (see Section 3.4 for denitions).
The lter parameter values (waitcycle, threshfactor, dist-
factor) correspond to fairly tight lters, and these must
be loosened to solve some problems. The OptQuest
use linear constraints option, which projects trial
points onto the linear constraints, is not used be-
cause it is very time-consuming for larger problems.
SNOPT, an SQP implementation, was used for the
largest problems because many calls to the GRG
solvers CONOPT (Drud 1994) and LSGRG2 termi-
nate infeasible on these problems. The 8_3_x problems
include many pooling constraints, which have bilin-
ear terms. In almost all these terminations, the GRG
solvers nd a local minimum of the phase 1 objec-
tive. SNOPT has no phase 1, and never terminates
infeasible.
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
336 INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS

Table 2 Base Case OQNLP and OptQuestGRG Parameters and Computational effort needed to achieve these re-
Options Used sults is quite low, and increases slowly with problem
Total iterations = 1000 size, except for the geometric mean solution time for
Stage 1 iterations = 200
the largest problems. The best OQNLP value is also
Waitcycle = 20
found very early: in the rst solver call in 80 of the
Threshfactor = 02
118 solved problems, and the second call in 19 more.
Distfactor = 075
This shows that, for these test problems, stage 1 of
OQNLP is very effective in nding a point in the
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.

Use linear constraints = no

OptQuest search type = boundary
Boundary search parameter = 05
NLP solver = LSGRG2, except SNOPT for 110 to 141 range
Table 3 shows outcomes and average-effort statis-
tics for 128 of the Floudas et al. (1999) collection of test
problems with continuous variables only, sorted into
six groups by number of variables. Geometric rather
than arithmetic means are used to reduce the effects
of outliers: function calls, iterations, and times for
the larger problem sets typically include a few prob-
lems with values much larger than all others. Com-
putational effort is measured by OptQuest iterations,
solver calls, function calls (each function call evalu-
ates the objective and all constraint functions), and
computation time. The three to best columns show
the effort required to nd the best OQNLP objective
value. Function calls are not available for the largest
problems because the SNOPT interface does not yet
make them available.
Since all problems have known solutions, we de-
ne failures as problems with a solution gap of
more than 1%. This gap is the percentage difference
between the best feasible OQNLP objective value,
fOQNLP, and the best known feasible objective value
fbest, dened for minimization problems as gap =
100foqnlp fbest/1 +
fbest
, and the negative of the
above for maximization, so positive gaps indicate that
the best known solution was not reached. Nine of the
128 problems failed to achieve gaps smaller than 1%,
with gaps ranging from 2.2% to 80%. All these are
solved with more iterations or by loosening the lters.
Percentage gaps for almost all 119 solved problems
are less than 104 , and the largest gap among solved
problems is 0.37%.
basin of attraction of the global optimum. The ratio of
the to best effort to total effort is also small. For iter-
ations, since there are always 200 stage 1 iterations,
we subtract 200 before computing the ratio, giving
513/800 = 006. The solver call ratio is 0.17 and the
time ratio 0.23. This implies that, for these problems,
a criterion that stops OQNLP when the fractional
change in the best feasible objective value found thus
far is below a small tolerance for some (reasonably
large) number of successive iterations would rarely
terminate the algorithm before the best solution was
found. The ratio of total solver calls to locals found, a
measure of lter efciency, varies from 3 to 5, and is
nearly 1 for the largest problems.
Table 4 shows results obtained in solving the nine
failed problems with looser lters and an OptQuest
boundary search parameter of 1. Seven of these nine,
the 2_1_x series, are quadratic programs (QPs) with
concave objectives (to be minimized), so each has an
optimal extreme-point solution, and every extreme
point is a local solution. The base case and new
parameter values are in Table 5.
The looser merit lter increases its threshold every
ten iterations, replacing the old value by old value +
10 1 +
old value
. The looser distance lter rejects a
trial solution if its distance from any previously found
local solution is less than 01 maxdist, where maxdist
is the largest distance traveled to reach that solution.
A search parameter of 1 causes more OptQuest trial
points to have values on the boundary of the rectangle
dened by the variable bounds, which helps solve the
seven concave QPs.
Eight of the nine unsolved problems are solved
with these new parameters, and the other, EX14_1_8,
achieves a gap of 1.15%. It is solved by using 1,000
stage 1 iterations and 5,000 total, with all other param-
eters as in the base case.

Table 3 Results and Effort Statistics for 128 Continuous-Variable Floudas et al. (1999) Problems

Variable Iterations Solver Solver Locals Function calls Function Time to Total First Second
range Problems Variables Constraints to best calls to calls found to best calls best time Failed L call L call

1 to 4 32 25 17 206.3 1.1 75 19 2639 21580 0.1 05 1 27 3
4 to 7 31 55 57 214.4 1.3 58 19 3815 47667 0.2 06 0 22 6
8 to 12 21 94 81 238.2 1.5 132 30 5756 196980 0.1 08 3 10 4
13 to 20 18 159 117 303.5 2.4 74 26 9682 52119 0.3 07 4 8 1
22 to 78 13 379 276 259.6 2.1 141 31 15624 230779 0.6 25 1 5 3
110 to 141 13 1164 801 305.0 2.7 237 227 NA NA 6.6 641 0 7 2
Total/avg. 128 251.3 1.8 105 35 5379 71909 0.4 17 9 80 19
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS 337

Table 4 Solving Nine Failed Problems with Looser Filters and Boundary Parameter = 1

Iterations Solver calls Total solver Base-case Locals Function calls Total Time Total
Problem name Variables Constraints to best to best calls solver calls found to best function calls to best time Gap

EX14_1_8 3 4 338 21 122 17 2 38141 1218680 049 225 115

EX9_2_5 8 7 344 2 6 45 3 690 2832 021 06 0
EX2_1_6 10 5 203 3 107 7 17 389 138749 02 138 0
EX2_1_9 10 1 201 1 111 66 39 252 342345 016 145 0
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.

EX2_1_7_1 20 10 279 8 27 2 19 5051 60043 039 125 0

EX2_1_7_3 20 10 269 4 65 22 36 1321 286213 03 192 0
EX2_1_7_4 20 10 253 6 29 4 9 3676 83989 039 141 0
EX2_1_7_5 20 10 254 5 29 3 15 2620 71004 033 134 0
EX2_1_8 24 10 226 3 120 8 25 981 1071730 029 287 0
Means(geom) 2586 41 487 105 128 17607 1378768 03 15

Table 5 Base Case and Loosened Filter Parameter Values
Parameter Base-case value
Waitcycle 20
Threshold_factor 02
Distance_factor 075
Boundary search parameter 05
Comparing the total solver calls and base-case
solver calls columns shows that the new parameters
represent a substantial loosening of both lters. The
looser lters result in many more solver calls in all but
problem 9_2_5, and the geometric mean solver calls
is 48.7 with the loose lters vs. 10.5 with the tighter
ones. The behavior of 9_2_5 is surprising (six solver
calls with loose lters versus 45 with tighter ones), but
the run with looser lters nds the global minimum
at iteration 344, and after that its merit thresholds and
set of local solutions differ from those of the base-
case run.
Table 6 shows the geometric performance means
and totals obtained from solving the 14 concave QP
problems with base-case parameters, with and with-
out the OptQuest use linear constraints option,
which maps each trial point into a nearest point fea-
sible for the linear constraints. Since these are linearly
constrained problems, invoking this option guaran-
tees that all trial points are feasible.
Clearly, this option helps: there are roughly twice
as many solver calls on average when using it, and
only two failures, vs. seven in the base case. The gaps
for the two unsolved problems (2_1_7_5 and 2_1_9)
are between 1% and 3.5% in both cases. However,
this option increases run times here by about a factor
of 30, so it is currently off by default.
4.2. The Lennard-Jones and Morse
Energy-Minimization Problems
Looser value
The Floudas et al. (1999) set of test problems includes
10 two GAMS models that choose the locations of a clus-
10 ter of N particles to minimize the potential energy of
01
the cluster, using two different potential energy func-
10
tions, called Lennard-Jones and Morse. The decision
variables are the x y z coordinates of each parti-
cle. Particle 1 is located at the origin, and three posi-
tion components of particles 2 and 3 are xed, so
each family of problems has 3N 6 variables. These
problems have many local minima, and their number
increases rapidly with problem size, so they constitute
a good test for global optimization algorithms.
Results of applying OQNLP to 14 of these prob-
lems, using 200 stage 1 and 1,000 total iterations,
are shown in Tables 7 and 8. Each problem set was
solved with LSGRG2 and CONOPT. These results
use CONOPT for the Lennard-Jones problems and
LSGRG2 for the Morse, because they provide slightly
better results, illustrating the value of being able to
call several solvers. Because of the many distinct
local minima, the number of local minima found is
equal to the number of solver calls for the three
largest Lennard-Jones problems and for all the Morse
problems.
The Lennard-Jones problems are the more difcult
of the two. The three largest problems have gaps of
roughly 1% to 2%, using the looser lter parameters
in Table 5. The default lter parameters led to positive
gaps for the last three problems totaling 7.8%, while
this sum in Table 7 is 3.8%. The objective approaches
innity as the distance between any two particles
approaches zero, so its unconstrained minimization
for N = 20 leads to about 50,000 domain violations

Table 6 Solving Concave QP Problems With and Without Use Linear Constraints

Iterations Solver calls Total Locals Fcn calls Total fcn Time Total
Case to best to best solver calls found to best calls to best time Failed

No use 284.8 2.3 66 3.7 643.8 3,875.1 0.3 06 7

Use 247.1 2.1 121 3.1 437.7 3,827.6 6.9 190 2
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
338 INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS

Table 7 Solving Six Lennard-Jones Problems Using CONOPT and Loose Filters

Solver calls Total Locals Time

Problem name Variables Constraints to best solver calls found to best Total Gap, %

EX8_6_1_5 9 10 1 152 39 109 2183 0.00

EX8_6_1_10 24 45 21 130 114 1856 6834 0.00
EX8_6_1_15 39 105 6 104 100 1363 16509 0.00
EX8_6_1_20 54 190 67 118 118 25762 39621 1.12
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.

EX8_6_1_25 69 300 42 94 94 32582 73068 1.84

EX8_6_1_30 84 435 16 59 59 13435 43456 0.88

Table 8 Solving Eight Morse Problems Using LSGRG2 and Default 4.3. Problems with Discrete Variables
Parameters There are 11 MINLP problems in the Floudas et al.
Solver Total (1999) test set, with the total number of variables
Problem calls solver Locals Time to Total ranging from three to 29 and the number of binary
name Variables to best calls found best time Gap variables ranging from one to eight. Two of these,
EX12_2_3 and EX12_2_4, had been reformulated so
EX8_6_2_5 9 1 5 5 023 061 0.0000
EX8_6_2_10 24 1 15 15 057 444 0.0000 that all binaries appeared linearly, and we restored
EX8_6_2_15 39 1 6 6 141 643 0.0000 them to their original state where the binaries appear
EX8_6_2_20 54 2 43 43 420 5120 0.0000 nonlinearly. OQNLP allows such representations,
EX8_6_2_25 69 4 20 20 1344 5838 0.0000 while the other GAMS MINLP solvers do not. The
EX8_6_2_30 84 17 43 43 6856 16019 0.0000 nal test set contains 13 problems. These are far
EX8_6_2_40 114 7 33 33 6629 27391 0.0000
EX8_6_2_50 144 20 25 25 33720 40396 0.1251
too small to yield meaningful inferences about the
power of OQNLP on problems of practical size, but
allow preliminary testing of the two MINLP modes
(either divide by zero or integer power overow), and described in Section 3.5. The geometric means of some
this number grows rapidly with N . Hence we added measures of computational outcomes and effort for
constraints lower bounding this distance by 0.1 for all both modes are shown in Table 9, using the LSGRG2
distinct pairs of points, and the number of these con- NLP solver.
straints is shown in the table. None are active at the The rst table row is for mode (2), where OptQuest
best solution found. manipulates only the discrete variables. Each NLP
Table 8 shows the Morse potential results using the problem was cold started from the same initial
LSGRG2 solver and the default OQNLP parameters point in these runs, so the number of function calls
shown in Table 2. The objective here has no singu- could be reduced substantially by warm starts. All
larities, so there are no difculties with domain vio- runs are terminated by OptQuest after the small num-
lations, and the only constraints are variable bounds. ber of possible binary-variable combinations have
All problems are solved to very small gaps except the been completely enumerated. The optimal solution is
largest (144 variables), which has a gap of 0.125%. found on average about midway through the solution
The number of solver calls is much smaller than for process, but we expect that this will occur earlier
the Lennard-Jones problems, because the lters are as the number of discrete variables increases. The
much tighter. Each call leads to a different local opti- OptQuest logic requires that at least one population
mum. The largest problem is solved to a gap less than of binary solutions be evaluated before any learning
104 % with 5,000 total and 1,000 stage 1 iterations can occur, and the average number of solver calls to
and the same lter parameters. This run terminated nd the best solution here is about equal to the pop-
because the 3,000-second time limit was exceeded, ulation size of ten.
took 4,083 iterations, and found 210 distinct local The last three rows of Table 9 show results for
optima in 210 solver calls, compared to only 25 in the mode (1), where OptQuest manipulates both binary
base case. and continuous variables. In rows 2 and 3, we do

Table 9 Solution Statistics for 13 Problems with Discrete Variables

Discrete Options and Iterations L calls Total L Fcn calls Total Time Total
var mode parameters to best to best calls to best fcn calls to best time Failures

Discrete only Default 102 102 201 20658 57941 0.8 15 0
All Default 2617 40 321 10657 250222 0.3 08 7
All 1000 5000 15519 12 891 101968 2240257 0.8 37 1
All Default, use 2724 36 1158 11780 889830 9.9 261 0
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS
Table 10 Average Solver Calls vs. Number of Binary Variables
Binaries 1 3 4 5 6 8
Problems 1 3 2 1 4 2
Discretes only 2 7 125 27 525 81
All 52 357 175 109 1955 5515
not require trial points to satisfy linear constraints,
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.
while in row 4 we do. Without using linear con-
straints, the default number of stage 1 and total iter-
ations of 200 1000, are not enough to nd the best
solution for about seven of the 13 problems. This is
because many OptQuest trial points have the same
values for the binary variables but different values
for the continuous variables, so complete enumera-
tion takes far longer than in mode (2). However, 1,000
stage 1 and 5,000 total iterations solve all but one
problem (its gap is 9.4%), and the ratio (solver calls
to best)/(total solver calls) of about 1/9 is favorably
small. Row 4 shows that, if trial points are required
to satisfy linear constraints, all problems are solved in
1,000 total iterations. This is because these problems
have mostly linear constraints (geometric mean of lin-
ear constraints is 9.1 and of total constraints is 9.9),
so the projected trial points tend to contain an opti-
mal set of binary variable values earlier, after only 3.6
solver calls on average. However, solving the MILPs,
which map trial points into nearest points that sat-
isfy the linear constraints, increases total solution time
by a factor of about 30 (compare the times in rows
2 and 4).
Table 10 shows that total solver calls increase
quickly with the number of binary variables for
the two discrete-variable modes, especially the all
mode. When more than one problem has the same
number of binaries, averages over those problems are
given. The values for the Discretes only mode are
the number of feasible binary vectors, averaged over
the number of problems shown.
5. Summary and Future Research
The results of Section 4 show that OQNLP is a
promising approach for smooth nonconvex NLPs
with continuous variables. It solves all 142 of the
test problems with no discrete variables with very
reasonable solution effort. While there is no guar-
antee of optimality and no gap is available, it
can be combined with other algorithms that pro-
vide this information, e.g., LGO (Pinter 2004) or
BARON (Tawarmalani and Sahinidis 2002). Informa-
tion on these solvers is also available at http://www.
gamsworld.org/(link to global world and then to global
solvers). The lower bounds provided by these proce-
dures can be used to estimate the gap for the OQNLP
solution, and the solution produced by any algorithm
can be used as a warm start for any other.
339
Future research includes enhancing the lter logic.
As described above, the lters needed to be loosened
to solve nine of the Floudas et al. (1999) test problems,
and this loosening could be done automatically. The
merit lter parameter threshfactor (new threshold =
threshfactor 1 + old threshold could be calculated
dynamically. Each time a penalty value is above
the threshold, calculate the value of threshfactor that
would cause the new threshold to equal the penalty
value. If this happens for waitcycle consecutive itera-
tions, set threshfactor to the smallest of these values, so
the new threshold would have just accepted the low-
est of the penalty values. Similar logic can be devel-
oped for the distance lter, reducing a basin radius
maxdist if that basins distance lter rejects trial points
for waitcycle consecutive iterations.
Also, the current distance-lter logic allows over-
lap of the spherical approximations to the attraction
basins. The true basins can have no points in com-
mon, so we can impose this condition on the spheres.
If the spherical model basins for any 2 local solu-
tions xi and xj have radii ri and rj, these must satisfy
ri + rj di j
where di j is the Euclidean distance between xi and
xj. If this inequality is violated, the radii ri and rj can
be reduced by the same scale factor so that it holds as
equality. We plan to test these options soon.
Another important aspect is the communication
of locally optimal solutions back to OptQuest, to
improve its search process. These solutions usually
have substantially lower penalty values than do typi-
cal OptQuest trial points, so they are likely ultimately
to be included in OptQuests population. However,
their penalty values often become the merit-lter
thresholds, causing most other trial points to be
rejected. Also, the local optima and their nearby chil-
dren will be rejected by the distance lter. We have
seen these effects in preliminary tests.
NLP algorithms can fail by failing to nd a feasible
point in cases where the problem instance is feasi-
ble. With GRG algorithms, this usually happens when
phase 1 terminates at a local optimum of the phase 1
objective. OQNLP can be applied to such problems,
if they are reformulated by dropping the true objec-
tive, adding deviation variables into all constraints,
and minimizing the sum of these deviation variables.
This approach could greatly improve the ability of
existing NLP solvers to diagnose infeasibility. More
generally, OQNLP can improve NLP solver reliability
by starting the solver from as many points as desired,
while insuring that these points balance diversity and
quality.
The performance of OQNLP in solving MINLPs is
less clear, because the 13 MINLP test problems used
 Ugray et al.: Scatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
340
here are so small. More extensive testing is needed,
which should clarify the relative merits of the two
MINLP modes discussed in Section 4.3. If OptQuest
manipulates only the discrete variables, then all trial
points generated by the current population may be
generated at once, and the solver calls at these points
may be done in any order. The points can be sorted by
increasing distance from their nearest neighbor, and
Downloaded from informs.org by [134.245.254.52] on 13 July 2017, at 03:44 . For personal use only, all rights reserved.
each NLP call can be started from the previous opti-
mum. The NLPs can also be solved in parallel.
Finally, comparative studies of OQNLP and other
global and MINLP solvers are needed. This testing
is facilitated by the existing GAMS interfaces for
BARON, LGO, DICOPT, and SBB. The MINLP
World and Global World websites developed
by GAMS Development Corporation (http://www.
gamsworld.org/) provide solver information and test
problems with known solutions.
Acknowledgments
This research was partially supported by the Ministerio
de Educacin y Ciencia of Spain under Reference Code
TIN2006-02696.
References
Dixon, L., G. P. Szego. 1975. Towards global optimization. Proc.
Workshop at the University of Cagliari, Italy, North Holland,
Amsterdam, The Netherlands.
Drud, A. 1994. CONOPTA large-scale GRG-code. ORSA J. Com-
put. 6 207218.
Edgar, T. F., D. M. Himmelblau, L. S. Lasdon. 2001. Optimization of
Chemical Processes. McGraw-Hill, New York.
INFORMS Journal on Computing 19(3), pp. 328340, 2007 INFORMS
Floudas, C. A., P. M. Pardalos, C. S. Adjiman, W. R. Esposito,
Z. Gumus, S. T. Harding, J. L. Klepeis, C. A. Meyer, C. A.
Schweiger. 1999. Handbook of Test Problems for Local and Global
Optimization. Kluwer Academic Publishers, Boston, MA.
Glover, F. 1998. A template for scatter search and path relinking.
J.-K. Hao, E. Lutton, E. Ronald, M. Schoenauer, D. Snyers, eds.
Articial Evolution, Lecture Notes in Computer Science, Vol. 1363.
Springer Verlag, New York, 1354.
Laguna, M., R. Marti. 2002. The OptQuest callable library. S. Voss,
D. Woodruff, eds. Optimization Software Class Libraries. Kluwer
Academic Publishers, Boston, MA, 193218.
Laguna, M., R. Marti. 2003. Scatter Search: Methodology and Imple-
mentations in C. Kluwer Academic Publishers, Boston, MA.
Locatelli, M., F. Schoen. 1999. Random linkage: A family of accep-
tance/rejection algorithms for global optimization. Math. Pro-
gramming 85 379396.
Murtagh, B. A., M. A. Saunders. 1982. A projected Lagrangian algo-
rithm and its implementation for sparse nonlinear constraints.
Math. Program. Study 16 84117.
Nash, S. G. 1998. Nonlinear programming. OR/MS Today 25 3645.
Nash, S. G., A. Sofer. 1996. Linear and Nonlinear Programming.
McGraw-Hill, New York.
Nocedal, J., S. J. Wright. 1999. Numerical Optimization. Springer
Series in Operations Research, New York.
Pinter, J. D. 2004. Computational Global Optimization in Nonlinear Sys-
tems. An Interactive Tutorial. Lionheart Publishing, Atlanta, GA.
Rinnooy Kan, A. H. G., G. T. Timmer. 1987a. Stochastic global opti-
mization methods; part I: Clustering methods. Math. Program-
ming 37 2756.
Rinnooy Kan, A. H. G., G. T. Timmer. 1987b. Stochastic global opti-
mization methods; part II: Multi level methods. Math. Program-
ming 37 5778.
Smith, S., L. Lasdon. 1992. Solving large sparse nonlinear programs
using GRG. ORSA J. Comput. 4 315.
Tawarmalani, M., N. Sahinidis. 2002. Convexication and Global Opti-
mization in Continuous and Mixed-Integer Nonlinear Programming.
Kluwer Academic Publishers, Boston, MA.