lEE TRANSACTIONS ON CIRCWIS AND SYSTEMS--I: FUNDAMENTAL m R Y AND APPLICATIONS, VOL. 43, NO.
2, FEBRUARY 19% 99
A Systematic Search Method for Obtaining
Multiple Local Optimal Solutions
of Nonlinear Programming Problems
Hsiao-Dong Chiang, Senior Member, IEEE, and Chia-Chi Chu, Student Member, IEEE
Abstract- We propose, in this paper, a systematic method to
find several local optimal solutions for general nonlinear opti-
mization problems. We have developed some analyticalresults for
quasi-gradient systems and reflected gradient systems, applying
these results to derive topological and geometric properties of the
critical points of the underlying objective function. A mechanism
has also been devised to escape from a local optimal solution
and proceed into another local optimal solution via decomposition
points. By properly switching between quasi-gradientsystems and
reflected gradient systems, our proposed method can attain a set
of local optimal solutions. The proposed method is applied to two
test examples with promising results.
I. INTRODUCTION
G LOBAL optimization is concerned with the characteriza-
tion and computation of the global minima or maxima of
nonlinear optimization problems. Such problems are important
to a wide range of applications such as engineering design
and control, allocation and location problems, and optimal
power flow in electric networks [12], [25]. Standard nonlinear
optimization techniques usually obtain local optimal solutions.
Such local optimal solutions, however, will only be the global
optimal solution when certain conditions - such as whether
the objective function is quasi-convex or whetether the feasible
region is convex - are satisfied. In general, several local
optimal solutions may exist and their corresponding function
values may be very different. The need for obtaining global
optimal solutions are obvious for practical applications.
The problem of locating global optimal solutions is in gen-
eral almost intractable. Becuase global optimization problems
belong to the class of NP-hard problems, its computational
time is expected to grow exponentially with the size of the
problem. In the context of information-based optimization,
this problem is, even for the 1-D case, at least as hard as
the “partition problem” which is known to be NP-complete
[7], [26]. Despite such difficulties, however, some encourag-
ing development in this area still stimulates further research
activities.
The existing methods for solving global optimization
problems can be classified into two categories: deterministic
Manuscript received August 26, 1993; revised July 25, 1994 and June 21,
1995. This work was supported in part by the NSF under Grants ECS-8957878
and ECS-8913074. This paper was recommended by Associate Editor A.
Premoli.
The authors are with the School of Electrical Engineering Cornel1 Univer-
sity, Ithaca, NY 14853 USA.
Publisher Item Identifier S 1057-7122(95)00736-2.
1057-7122/96$05.00 0 1996 IEEE
methods and stochastic methods [lo], [20], [25]. Deterministic
methods include the branch and bound, cutting planes,
decomposition-based, enumerative techniques, homotopy,
interval analysis, and interior point methods. Stochastic meth-
ods include random search, simulated annealing, simulated
evolution, genetic algorithms, and simulated diffusion. In
general, stochastic methods can find global optimal solutions
in an asymptotical way, while deterministic methods are not
guaranteed, except in special cases, to find global optimal
solutions for general problems.
In this paper, we will present a systematic method, which
is deterministic in nature, to find several local optimal solu-
tions for general nonlinear optimization problems. Given an
objective function, this method builds an associate vector field
which is gradient-like. This method then generates a set of
trajectories covering a set of several local optimal solutions.
We will also study the topological and geometrical properties
of the associated gradient-like vector field. These topological
and geometrical properties lead to the development of the
systematic method, that is presented in this paper.
Our proposed method was stimulated by recent progress in
the generalized descent method [25]. Fiodora first proposed a
conceptual algorithm for finding several local minima through
the dynamical system approach. Even though this conceptual
algorithm works well in lower dimensional cases, Fiodora
points out that it may lead unsatisfactory results for higher
dimensional cases [13]. Branian tried to modify the steepest
descent method by using information provided by the Jacobian
of the objective function to find the local minima. Reportedly,
such modifications endowed the method with a much wider
convergent region than other methods [l]. However, Treccani
observed that Branian’s method is not globally convergent
in general, and that not every local minimum can be found
[24]. On the other hand, among all existing methods, there
is no theoretical evidence that the existing algorithms can
systematically generate a trajectory out of the stability region
of a local minimum and find another local minimum.
Our proposed method consists of two distinct levels. In
the lower level, an associated quasi-gradient system is first
built. A local optimal solution can be obtained by intergating
the associated quasi-gradient system. Our construction of the
quasi-gradient system is so general that most of the previous
methods can be unified in such a quasi-gradient framework.
Once a local minimum is found, the trajectory of a reflected-
qradient system is used to find another local minimum. Since
100 lwEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-I F'Ul 'JDAMlNTAL THEORY AND APEICATIONS, VOL. 43, NO. 2, FEBRUARY 1996

this kind of the system is continuous in nature, it can be x = -gradRc(x) (5)

synthesized in programmable analog nonlinear circuits, such
as the neural network circuits, for example 121-[4], [14]. It can System (5) is also called a generalized gradient system in [5]
be also implemented in a conventional digital computer by or quasi-gradient system [9], [16]. In the next section, some
the discretization of such a continuous process. In the upper relations between ( 3 ) and (4) are discussed. In particular, it
level, an associated graph, which determines the connection will be shown that the local minima of (1) can be obtained via
structure among local optima and decomposition points, is the stable equilibrium points of (5).
constructed. Clearly, it is a discrete algorithm. If we can find
all the adjacent decomposition points of a given local optimum, In. QUASI-GRADIENT
SYSTEMS
then the associated graph is a cmnected tree and the global
optimum can also be obtained by comparing the function Mathematical Preliminary
values of each local optimum. Although current computer
Consider a nonlinear dynamical system described by
technology cannot adopt both the analog and the discrete
algorithm simultaneously, it is still possible, as new technology k ( t )= f ( 4 t ) ) (6)
advances, to implement the proposed algorithm in such a
hybrid computer. where the state vector x ( t ) of this dynamical systems belongs
to the Euclidean space Rn,and the function f : R" -+
11. PROBLEM
FORMULATION Rn satisfies the sufficient condition for the existence and
uniqueness of the solutions. The solution curve of (6) starting
Consider the following unconstrained global optimization from 2 at t = 0 is called a trajectory, denoted by @(x, .) :
problem R 4 P.
min{c(x) : 11: E Rn} A state vector 2 is called an equilibrium point of ( 6 ) if
(1)
f(2)= 0. A state vector x is called a regular point if it is
where ~ ( xis) assumed to be a bounded below smooth function not an equilibrium point. The set of equilibrium points will
so that its global minimal solution exists and the number of be denoted by E. We say that an equilibrium point of (6) is
local minimal solutions are finite. Moreover, there exist e and hyperbolic if the Jacobian of f ( . )at 9, denoted J f ( 2 ) ,has
6 such that no eigenvalues with a zero real part. A hyperbolic equilibrium
point is a (asymptotically) stable equilibrium point if all the
IIVc(x)I/ > E unless x E B s ( 2 ) , 2E E (2) eigenvalues of its corresponding Jacobian have negative real
parts; otherwise it is an unstable equilibrium point. If the
where E = {x : x E R2", VC(J) = O}. The central idea of our Jacobian of the equilibrium point D has exactly k eigenvalues
proposed method to find multiple local optimal solutions is as with positive real parts, then it is called a type-k equilibrium
follows: if it is possible to find a nonlinear dynamical system point. If k = 0, then the equilibrium point is called a stable
such that all the critical points of c are identical to all the equilibrium point, or a sink (or attractor). If k = n, then the
equilibrium points of the corresponding vector field, then it is equilibrium point is called a source (or repeller). A dynamical
possible to find all the local optimal solutions of c by finding system is called completely stable if every trajectory of the
stable equilibrium points of the corresponding vector field. system approaches one of its equilibrium points.
More precisely, every stable equilibrium point corresponds to a Let 2 be a hyperbolic equilibrium point. Its stable and
(local) optimal solution. One way to implement the above idea unstable manifolds W s ( 2 )W , u ( 2 )are defined as follows:
is to build an associate gradient vector field. Before explaining
this approach, we willreview the concept of gradient vector W s ( 2 ):= {x E R" : lim @ ( z , t =) 2}
ti"
fields. The following is a general definition of gradient vector
fields which admit inner products depending on the point in
sli" [17], [23]. W a ( 2 ):= {x E Rn : liin @(x,t) = 2 }
ti-"
DeJinition: Let U c R" be an open subset and S be the
space of symmetric n x n real matrices. A C"- variable metric I f f is a type-k equilibrium point, then the dimension of W"( 2 )
or C"-Riemannian metric on U is a C"-mapping R : U + S , and W u ( 2 )is n - k and k , respectively. The stable manifolds
where for all x E U , R ( x ) is a positive definite matrix and and unstable manifolds of equilibrium points are said to satisfy
there exist constants k and T such that k 5 llR(x)II 5 T . the transversality condition if either they do not intersect at all,
If c E Cm(U,X) and R is a CO"-Riemannian metric on U , or at every intersection point z,the tangent space of W'(&)
then the C"-gradient vector field gradRc(x) is defined as and Wu(2i.,) spans 32" at x for 2%and 2, in E.
A useful tool for the analysis of nonlinear systems is the
gTadRc(x) = R(x)-1Vc(x) (3) Liapunov function theory. If there exists a smooth function
and the negative gradient vector field is V ( . ) : En + R for (6) such that

-gradRc(x) = -R(x)-1Vc(z). (4) V ( @ ( Z , t ) )< 0 (7)

With this definition, we will build the following gradient for all x not in E , and t E R+.Any smooth function satisfying
vector field associated with the global optimization problem: (7) will be termed a Liapunov function for the nonlinear
CHIANG AND CHU: MULTIPLE LOCAL OPTIMAL SOLUTIONS 101

dynamical system (6). The (lower) level set of the Liapunov

function V ( x ) is defined as
SI,(x)= v-l((-m,k]) = {x E F : V(z) 5 I C }
Generally speaking, the set SI, can be very complicated with
several different components. Recall that the space X is called
path-connected if, and only if, for all xo,x1 E X , there exist
a continuous map 4 : [0,1] + X such that $(O) = xo and
stability region practical stability region
4(1) = 21. A path component of X is a nonempty, path-
connected, equivalent class of X . #Sh denotes the total number
- stability boundary - practical stability boundary
of path components of S I , ( Z ) .With the above definition, (a) (b)
the level set Sk(x) can be decomposed into disjoint path- Fig. 1. The difference between the stability region and the practical stability
connected components, i.e., region: (a) the stability region A(z,) and the corresponding stability boundary
3A(zs); (b) the practical stability region A p ( z , ) and the corresponding
r practical stability boundary aA, (zs).
Sk(2) = U
3=1
SI,,3(4

where S ~ Care, ~disjoint path-connected components. The total

number of path components of SI, is r .
For a hyperbolic stable equilibrium point, it can be shown
that there exists a number S > 0 such that 11zo - 211 < 6
implying that @(Q, t ) -+ D as t + CQ. If S is arbitrarily large,
then D is called a global stable equilibrium point. There are
many physical systems containing stable equilibrium points
but not global stable equilibrium points. A useful concept for
this kind of system is that of stability region (or region of
attraction). The stability region of a stable equilibrium point
x, is defined as
A(%,) := {x E Rn : lim @(x,t) = x,}.
t-oo

From a topological point of view, the stability region A(%,)

is an open, invariant, and connected set. The boundary of
stability region A(%,) is called the stability boundary (or
separatrix) of x, and will be denoted by aA(x,) 161, [271.
The stability boundary is topologically, a ( n - 1)-dimensional
closed, invariant set. A new concept in stability regions is the
quasi-stability region (or practical stability region) [I 11, [27],
which will play an important role in the development of our
method. The practical stability region of a stable equilibrium
point IC,, denoted by A,(x,), is the open set int A(x,), where
A denotes the closure of A and int A denotes the interior
of A. It has been shown that the practical stability boundary
aA,(xS)is equal to dA(x,). The practical stability boundary
is a subset of its stability boundary. It eliminates the complex
portion of the stability boundary which has no "contact" with
the complement of the closure of the stability region. The
difference between the stability boundary and the practical
stability boundary is roughly described in Fig. 2. If the portion
of the stability boundary lying inside int A(z,) is eliminated,
then the stability boundary is equal to the quasi-stability
boundary. To illustrate the idea of the practical stability, we
cite the following concrete example [ l l , p. 181:
2=
( ( d m -3)(x2+y2+ (y - 2 ) d m - 2y + 1.5) +.)~1
jl=
(( d m - -3) (x2$y2+ (y - 2) d m - 2y+ 1.5)+y) -2'.
(8)
Fig. 2. The corresponding phase portrait of (7). (a) The stability boundary of
the stable equilibrium point (0,O) consists of Ws(O,-3) and Ws(O,1.5).
(b) The practical stability boundary of the stable equilibrium point (0,O)
consists of W "(0, -3) only. In other words, the practical stability boundary
is the union of the stable manifold Ws(O,- 3 ) and the u.e.p. (0,3).
From the phase portrait shown in Fig. 2, it is easy to see that
the stability boundary of stable equilibrium point (0,O) consists
of Ws(0,-3) and Ws(0,1.5). Since the stable manifold
Ws(0,1.5) is inside the stability region of stable equilib-
rium point (O,O), the practical stability boundary consists
of ws(O, -3) only. In other words, the practical stability
boundary is the union of the stable manifold W'(0, -3) and
the u.e.p. (0,3).
It can be shown that an equilibrium point xd is on the practi-
cal stability boundary aA,(z,) of the stable equilibrium point
x, if Xd E dA(z,) and W'(zd) n(A(x,))" # 0 [ll, p. 211.
Furthermore, if there exists another type-k equilibrium point
2 , and IC > 1, then there exists a type-one equilibrium point
z1E aA,(zs) such that z E W s ( z : ,[)l l , p. 291. Motivated by
these results, we shall call a type-one equilibrium point lying
on a practical stability boundary dA,(zs) a decomposition
point. The decomposition point Xd distinguishes itself from
the other type-one equilibrium point in the following way:
as the Liapunov function value increases from c ( z d ) - E to
, number of the path components of S,(,,)(X)
c ( z ~ ) + Ethe will
be decreased by one, i.e., #Sc(,d)+B(~) = #Sc(,d)-E(z)- 1. On
I02 J
EE TRANSACTIONS ON CIRCUITS AND SYSTEMS-I FUNDAMENTAL TBEORY AND APPLICATIONS, VOL. 43, NO. 2, FEBRUARY 1996
Fig. 3. For a completely stable system, every trajectory converges to one of
its equilibrium points. In other words, every trajectory converges to one of its
stable equilibrium points or to one of its unstable equilibrium point.
the other hand, if x1 is a type-one equilibrium point other than
a decomposition point, then the number of path components
remains the same as the Liapunov function value increases
+
from c ( x d ) - E to c ( m ) e [ll, p. 421.
Characterizations
We will derive in this subsection several geometrical and
topological properties of generalized gradient system ( 5 ) which
will be useful in the development of our systematic method.
Proposition I (Completely stable): The negative quasi-
gradient system (5) is completely stable and ).(. is an
associated Liapunov function.
Prooj See Appendix.
Proposition 1 implies that every trajectory of ( 5 ) is bounded
and converges to one of the equilibrium points. This system
does not admit any closed, quasi-periodic, or chaotic trajec-
tories. Moreover, the state-space of ( 5 ) is composed of the
closure of the stability regions of each stable equilibrium point
of the system (see Fig. 3 ) . Proposition 1 also sheds some light
on the structure of the stability boundary as described in the
following corollary [6].
Corollary 2: (Characterization of stability boundary):
Suppose that all the equilibrium points of quasi-gradient
system ( 5 ) are hyperbolic. Let ot,z = 1 , 2 , . . . be the
equilibrium points on the stability boundary dA(x,) of a
stable equilibrium point, say 2 , . Then
Next, Corollary 2 can be extended to a characterization of
the practical stability boundary in terms of the stable manifold
of the decomposition point.
Proposition 3 (Characterizationof practical stability bound-
ary): Suppose all the equilibrium points of quasi-gradient
system (5) are hyperbolic. Let ot,i = 1 , 2 , . . . be the de-
composition points on the quasi-stability boundan/ dA, (2,)
of the stable equilibrium point x,. Then
Prooj See Appendix.
Proposition 3 asserts that the practical stability boundary is
contained in the union of the closure of the stable manifolds
of all the decomposition points on the practical stability
boundary. Thus, if the decomposition points can be identified,
then an explicit characterization of the practical stability
boundary can be established via (10). In Section V, we will
use the information regarding the decomposition points on
the practical stability boundary to develop a systematic search
method for obtaining multiple local optimal solutions.
AND LOCAL MINIMA
Iv. EQUILIBRIA
In this section we will examine some analytical and com-
putational issues between the critical points of c in (1) and
the equilibrium points in (5). Without loss of generality, it is
assumed that that the critical points of c are nondegenerate, i.e.,
the Hessian matrix at every equilibrium point is nonsingular.
We first present a result showing the relationship between
equilibrium points and extreme values.
Proposition 4 (Equilibrium points and extreme values): An
equilibrium point of a negative quasi-gradient system
-gradRc(x) is hyperbolic if, and only if, the corresponding
critical point of c is nondegenerate. Moreover, if 2 is a
hyperbolic equilibrium point of (5), then
1) 3 is a stable equilibrium point of (5) if, and only if, c ( x )
has an isolated minimum at 2 .
2) rC is a source of (5) if, and only if, C(Z) has an isolated
maximum at z.
Proof See Appendix.
We next present that all the quasi-gradient systems have the
same equilibrium points with the same type. This convenient
property allows us to design a computationally efficient algo-
rithm to find stable equilibrium points of (5), and thus, to find
local optimal solutions of (1).
Proposition 5 (Type-persistence of equilibrium points):
Consdider the quasi-gradient system (5). If Rl(x)# Rz(x)
are both positive definite matrices for all x E U , then we have
the following results.
1) the locations of the equilibrium points of gradR,c(x)
is the same as those of the equilibrium points of
gradRz e(.).
2) a type-k equilibrium point of gradR, e(.) is also a type-
k equilibrium point of g r a d ~ , c ( x ) .
Prooj See Appendix.
The above lemma states that the static behaviors of any
quasi-gradient system (2) are the same. Their dynamic behav-
iors, such as stability regions, can be quite different. However,
it can be shown that two quasi-gradient systems characterized
by two positive definite matrices Rl(x)and Rz(x)have
different stability regions, but their stability boundaries contain
the same equilibrium points, provided that R1(x)and Rz(x)
are "close" to each other r51.
CHIANG AND CHU MULTIPLE LOCAL OPTIMAL SOLUTIONS
(b)
Fig. 4. (a) The unstable manifold Wu(p) of the decomposition point z d
converges to two stable equilibrium points and zz. (b) A mechanism to
escape from a local minima and move on toward another local minimum can
be accomplished by the following procedure: staring from the local minima,
one can move along the unstable manifold of a decomposition point in reverse
time as indicated in the dashed line.
One important issue in global optimization (instead of local
optimization) is how to escape from a local optimal solution
and move on toward another local optimal solution. This prob-
lem has been studied by several researchers (see, for examples,
[lo], [22], [25]). In terms of our proposed formulation, we
can see this problem as a question of how to escape from the
stability region of a stable equilibrium point and enter stability
regions of other stable equilibrium points so as to search for
other stable equilibrium points (correspondingly, local optimal
solutions). The following results will offer some insight into
this issue.
Proposition 6 (Decompositionpoints and the stable equilib-
rium points): Let X Q be a stable equilibrium point of quasi-
gradient system (5) and Xd be a decomposition point on the
practical stability boundary dA(z:).If every equilibrium point
of quasi-gradient system (5) lying on aA(z;) is hyperbolic
and its stable and unstable manifolds satisfy the transversal
condition, then there exists another stable equilibrium point
~f to which the 1-D unstable manifold of X d converges.
Prooj See Appendix.
Proposition 6 suggests that if one starts from a stable
equilibrium point, and moves along the unstable manifold of
a decomposition point in reverse time, then one can escape
from a local optimal solution and move on toward another
local optimal solution by moving along the other "direction"
of the unstable manifold in positive time. (See Fig. 4)
Locating the 1-D unstable manifold of the decomposition
point can be accomplished by the following procedure [6]
[21, p. 1501:
103
Find the Jacobian at the decomposition point X d .
Find the normalized unstable eigenvector E" of the
Jacobian.
Find the intersection of the open ball B E ( X d ) and the
+
straight line {X : x = X d XE", X E R}, say X d €E" +
and X d - €E".
Numerically integrate the vector field backward from
+
the intersection point z d €EUand Z d - €EU.If both
trajectories remain inside the open ball B, ( X d ) , then
go to the next step; otherwise replace E = 196, where
0 < 19 < 1, and repeat Step 3.
Numerically integrate the vector field forward from the
+
intersection points Xd €Eu and x d - €E".
METHOD
V. A SYSTEMATIC
As shown in the previous sections, a conceptual method to
systematically find several local minima includes the following
steps:
Step 1: Approach a local minimum.
Step 2: Escape from the local minimum and pass through
a decomposition point.
Step 3: Approach another local minimum by moving along
the opposite direction of the unstable manifold of
the decomposition point.
The first step can be implemented by the descent method, or
by numerically integrating the negative quasi-gradient system
(5). Various R ( x )can be chosen. For example, if R ( x )= I ,
it is called the steepest descent method; if R ( x ) = v2c(z),
it is called the Newton method; if R ( x )= u d j ( v 2 c ( ~ ) ) - 'it,
+
is Branin's method [1]; if R ( z ) = (1 11 v2c(z)ll)I,it is
Treccani's method [24]. Different R(x)'sin the quasi-gradient
system will have different numerical convergent properties.
For a given objective function, it is not yet clear how to choose
an optimal R ( x ) which provides the best convergent result.
The second step is the most challenging task. A heuristic
way is to move along the direction of the eigenvector of
the Jacobian matrix evaluated at the stable equilibrium point
associated with the largest eigenvalue. Then, continue the
movement until the matrix becomes semi-definite or until an
unbounded increase of the objective function is confirmed.
However, this method of ascent does not consistently produce
satisfactory results. A theory-based approach for escaping from
a local minimum will be proposed in the next section. The
third step involves direct numerical integration of the negative
quasi-gradient system.
From a theoretical viewpoint, the above method builds a
graph G describing the connections between the stable equi-
librium points and decomposition points with the following
elements (see Fig. 5):
the vertices of G are stable equilibrium points x:, . . ' , x;
and decomposition points xi,. . , xi of the negative
e
quasi-gradient vector field -gradRc(z), where r and s
are the total number of stable equilibrium points and
decomposition points, respectively.
the edge of G can only,connect vertices of different types;
xi is connected with X: if, and only if, W"(x;)nA(x%,) #
0 and W U ( x ; )n (A(x",)" # 0.
104 IEEE TRANSACIlONS ON CIRcuTfS AND SYSTEMS-1: FUNDAMENTAL. THEDRY AND APPLICATIONS, VOL. 43, NO. 2, FEBRUARY 1%
Fig. 5. A graph showing the connections between stable equilibrium points
and decomposition points.
According to Proposition 6, each vertex x; is connected
by two edges, which connects two different stable equilibrium
points. Moreover, s = r - 1. The total number of edges is
equal to 2(7- - 1) and the total number of vertices is equal to
21- - 1. The graph G is a connected tree without cycling if the
critical points of the objective function have distinct function
values [18, p. 4331.
We are now in a position to propose an algorithm to
systematically find several local minima. Let variables V,,,
and Vd record the location of local minima and decomposition
points, respectively; and let E list the edges of graph G
with the element of the form ( x ~ , ~the ~ )set
; V denote
vertices of graph G. Each vertex in V will be examined to
determine whether it is adjacent to a new local minimum or
to a decomposition point. At each iteration of the algorithm,
a point II: in V will be chosen: If z is a local minimum,
determine all new adjacent decomposition points; otherwise
z is a decomposition point which will be used to find another
local minimum. The element of V will be added if some new
vertices appear. When the set V is empty, this algorithm will
be terminated.
An Algorithm to Find Multiple Local Minima
Step 1: [Initialize the data]
Step 1.1: set V,,, = Vd = V = 0;
Step 1.2: set an initial guess zo;
Step 1.3: set a sufficient small number E;
Step 2: [Find the first local minimum]
Step 2.1: numerically integrate the negative
quasi-gradient vector field -gradRc(z) with
initial condition zo until the trajectory approaches
one of a local minimum zQof e(.) (cfi Proposition
3);
Step 2.2: set V, = {zt} and V = {z:};
Step 3: [Build the corresponding graph G]
w ~ i ~V e# 0 do
begin
choose z E V, set V = V \ {z};
if z E V,
find all the adjacent decomposition points
xi,.. . , xpd;
set V = VU({z,!j,...,xpd}\Vd);
set Vd = Vd U{zi,. . . , zpd};
set E = E U{(z, xi),. . . , (2, zpd)};
else
compute the unstable eigenvector E" of
J-gradRc(z) ;
numerically integrate the negative quasi-
gradient vector field -gradRc(z) with initial
points f €23" until it approach a stable
equilibrium point y, where y V.
set V = V U ( ( y > \ Vmzn);
set Vmzn = Vmzn U{Y} ;
set E = E U{(z, Y)};
endif;
end;
end;
At the end of the algorithm, two partitions of vertices V,
and Vd and an edge-list E will be obtained. Since the graph
G is connected, any two local minima is connected by one
path which is composed of a sequence of edges in E . Such a
graph G is a by-product of the proposed algorithm. It make
the algorithm noniterative among existing local minimum, i.e.,
it prevents the deadlock of the algorithm. The global optimum
can thus be obtained by comparing the values of all the local
minima. It is clear that the key step in the proposed solution
algorithm is to find all the adjacent decomposition points.
VI. IDENTIFYING POINTS
DECOMPOSITION
As we have seen in the previous section, the search for
various local minima of an objective function can be ac-
complished with the aid of decomposition points. Hence, it
is essential to determine such points. A straightforward way
to formulate this problem is via a nonlinear optimization
problem: through which we may find the local minima over the
practical stability boundary. However, since we cannot derive
an explicit expression to characterize the practical stability
boundary, this method seems very difficult. An alternative way
is to build a new vector field such that,in the new vector field,
there exists a trajectory which starts from a stable equilibrium
point and converges to a decomposition point. To this end, we
will employ a method called reJEectedgradient [ 181, [ 191. This
method can be used to change the quasi-gradient vector field
in a systematic way such that the location of any equilibrium
point is invariant while its type will be altered in a more
explicit way.
Let x be an equilibrium point of (5) and Al(z) 5
X,(x) . . . 5 Xn(z) be the eigenvalues of the Hessian
matrix v2c( z) , and wuz(z) be the normalized eigenvector
corresponding to AJ(z),i.e., IivJ(z)lj= 1, for J = l , . . . , n.
Let ~ , ( z )= w,(z)wT(z), j = I , . . . , % . Since v2c(z) i s
symmetric, P3(x)is the orthogonal projection matrix on the
eigenspace associated with A, (z).By the spectral factorization
theorem [15, p. 1751, the Hessian matrix v2c(z) can be
decomposed into the direct sum of orthogonal projection
matrices P J ( z ) ,i.e.,
CHIANG AND CHU: MULTIPLE LOCAL OPTIMAL SOLUTIONS 105

Now, we define a family of matrices Mk(x), k = 1,. . ,n,
K n
and the k-th order reflected gradient vector field Fh(x),
boundary a A ( x s )based on the reflected gradient method. The
variable D record all the adjacent decomposition points of ji..
Step l:[Initialize the data]
Step 1.1:choosep points P j , j = l , . . . p , from a
neighborhood of a local minimum, say 2 , of f ;
Step 1.2: set a sufficient small number E ;

Step 2: [Find decomposition points]

Note that if k = 0, then Fo(z)= - v c(x); and if k = n,
then Fn (x)= vc(x). By the orthogonalproperty of the matrix for j = 1 to p do
Pj (z) , the Jacobian of the k-th order reflected gradient vector Step 2.1: numerically integrate the first order
field Fk(x) evaluated at x = % can be expressed as reflected gradient vector field F1 (x) with the
k n
initial conditions P3 until I(Fl(x)J(reaches a
local minimum, say PF, or I(Fl(x)l( becomes
JF,, (3)= A, (z)P3(5)- A, (%)P3(5)
J=1 ,=k+l
unbounded.
Step 2.2: solve the algebraic equation e(.) = 0
The above formula can be interpreted as the spectral fac- using the initial condition PF. The resulting point
torization of the Jacobian of the kth-order reflected gradient is x3.
vector field F~(z) at z = 5. Therefore, the eigenvalues of Step 2.3: check the type of the equilibrium points
JF,, (z) are with respect to the negative gradient vector field
-v 41
..
X I ( % ) , . . . >Xk(x),-Xk+l(x), . . . > - A n ( x ) if x3 is a type-one equilibrium point,
It can be shown that with respect to two vector fields - v c ( x ) compute the unstable eigenvector E" of
and Fk(x),the type of equilibrium points can be changed J-vc(z) ( 2 3 ) ;
according to a simple formula. numericallyintegrate negative gradient sys-
Proposition 7 (Alternation rule for the types of the equilib- tem (4) with initial points x3 tE" and x3 - +
rium points) [18], [19]: If 2 is a type-i equilibrium point of EE" until it approaches equilibriumpoints y1
the negative gradient vector field - v).(C and W8(D)is the and 9 2 , respectively (cf.: Proposition 6)
stable manifold of the equilibrium point i of the k-th order if y1 # y2, D = D U { n 3 } ;endif;
reflected gradient vector field F k (x),then
endif ;
1) W"( 2 ) is a smooth manifold of dimension n - I k - il .
2) with respect to the kth-order reflected gradient vector end;
field Fk (x), It is clear that the ability of the proposed approach to find all
a) E is a type-zero equilibrium point if, and only if, local minima depends significantly on the convergence of the
k = i. employed nonlinear solver as well as on the ability to find all
b) D is a type-n equilibrium point if, and only if, decompostion points. Moreover, the number of decomposition
Ik - il = n. points on the practical stability boundary of a local minimum
Applying the above proposition to k = 1, we have the plays a central role in the performance of the algorithm. The
first order reflected gradient vector field F l ( z ) , which will issue of how many points should be chosen in Step 1.1 is
make any trajectory with its initial condition lying inside the not easy to address. This issue is realted to the structure of
neighborhood of a local minimum of e(.) converge to a type- the objective function. For example, if the variables of the
one equilibrium point on its stability boundary. Proposition 6 objective function have certain periodic properties, then the
can then be used to check whether the type-one equilibrium upper bound and the lower bound of the decomposition points
point is a decomposition point in the following way: along the can be estimated in terms of the dimension of the variables
two directions of the unstable eigenvectors of the Jacobian of [22]. As a result, the number of starting points p in the above
the type-one equilibrium point, numerically integrate the quasi- algorithm can be determined a priori if the number of the
gradient system. If both trajectories converge to two different decomposition points can be estimated.
equilibrium points, then this type-one equilibrium point is
a decomposition point. Otherwise, this type-one equilibrium
VII. NUMERICALSTUDIES
point lies inside the practical stability region. We will next
present this algorithm in detail. For the purpose of illustrating the systematic method pro-
posed in Section VI, the following two examples are studied.
An Algorithm to Compute Decomposition Points Example I: Minimize the function
2
Assume 2 , is a local minimum of e(.). This algorithm c(x,y) = - ( x 2 f y 2 - 1 ) 2 - ( ( 2 2 2 - 1 ) 2 + ( 2 + 1 ) 2 - - ) 2 .
will identify decomposition points on the practical stability 3
It is clear that the associated negative-gradient system is of TABLE I
the form LOCALMINIMAAND DECOMPOSITION
POINTSOF EXAMPLE
1

332 76
x = -x
3
+ 68xy2 - --IC
3
+
128x7 Iteration 1: Step 2 Iteration #2: Step 3 Iteration #3: Step 3
lnitlal z = $,0.2) initial z = z: initial z =
+
- 192x5 128x3y4 - 64xy4 - 128x3y2 (1 1)
332 76
y = 68x2y + -y3
3
- -y
3
+
128x4y3 - 64x4y
+
- 128x2y3 128y7 - 192y5

the associated reflected gradient system can be written as

the following form: (see (12) at the bottom of the page) v = {si} v = {Zi,z:,z;, z:} v = {z?, 22,si,Z;}
where PI ( x ) and P2 (x) are normalized orthogonal matrices Iteration #4: Step 3 Iteration # 5 . Step 3 I t q a t i o n #6: Step 3
initial z = z: lnltlal2 = 2; initial x = z :
associated with the eigenvalues X I ( x ) and X2(2), respectively,
and Xl(x) 5 Xz(x) are the eigenvalues of the Hessian
matrix g c ( x ) . First, we choose an initial guess x0 = (0,0.2)
and set E = 0.01. By integrating the negative-gradient sys-
tem, the first local minimum x: = (0,O) is obtained. Then
switching to the reflected gradient algorithm with 30 initial
points PJ = x,' + €(cos %,sin %), j = 0;..,29, we
v = (22, zi,22) v = {z;, z;, z;} v = {z:, zfi}
numerically integrate the first-order reflected gradient system
Fl(ic). Four decomposition points x i = (0.5213,0.5213),
Iteration e?
initial z = zd
Step 3 Iteration #8: Step 3
lnltlal z = 2
:
Iteration #9: Step 3
initial z = 22

x i = (-0.5213,0.5213), xi (-0.5213, -0.5213), and

x: = (0.5213,-0,5213) are obtained. At Step 3 of each
iteration, if initial point x E V is a local minimum, then 30
+
initial points P~ = x €(cos %, sin s),
j = 0 , . . ,29 are
used to locate decomposition points on the practical stability
boundary aA,(x); otherwise, we use x f €E", where E" is (v3=' {z:}
the unstable eigenvector of J-vc(z), as the initial conditions
to compute other local minima. The detailed results after each Iteration #lo: Step 3
initial z = z
:
iteration is summerized in Table I.
After the algorithm is executed, we find that this objective
function has 5 local minima and 4 decomposition points. Their
values are listed in Table 11. By comparing the values of
local minima, we find that the global minimum is -2.7778
at (2, y) = x i = (0,O). It can be verified that a corresponding
graph GI has 2(5 - 1) = 8 edges.
Example 2: [8]: Minimize the Hump Camel-Back function
described by
c ( x , y) = ax2 + bx4 + c26 - xy + dy2 + ey4 TABLE I1
AND DECOMPOSITION
LOCALMINIMA POINTSOF EXAMPLE
1
where a = 4, b = - 2 . 1 , ~= 1 / 3 , d = -4,e = 4. iocal minima 1 c(.) (1 decomposition points
The associated negative-gradient system is of the form: 4 = (0,O) I -2.7778 11 :2 = (0.5213,0.5213)
~f = (0.7071,0.7071) -0.4444 25 = (-0.5213,0.5213)
k = -2ax - 4bx3 - 6cx5 +y (13) ~f = (-0.7071,0.7071)
2: = (-0.7071, -0.7071)
-0.4444
-0.4444
22 =
2
:
(-0.5213, -0.5213)
= (0.5213, -0.5213)
y = x - 2dy - 4ey3 (14) , 25 = (0.7071, -0.7071) -0.4444

and the reflected-gradient system is of the form:

Then switching to the reflected gradient algorithm with 20
initial points p3 = x: +€(COS sin g, g),
j = 0, . . . ,19, we
numerically integrate the first-order reflected gradient system
First, we choose an initial guess 5 0 = (0.5,0.5) and e = 0.01. Fl(x). Three decomposition points ici = (1.10921,0.76823),
By performing step 1 of the multiple local minima algorithm, x: = (0,0), x i = (1.29607, -0.60508) are obtained. At
the first local minimum obtained is x: = (0.08984,0.71266). Step 3 of each iteration, if initial point x E V is a local
CHIANG AND CHU: MULTIPLE LOCAL OPTIMAL SOLUTIONS 107

TABLE I11 V, we can verify that the total number of the edges in the
THE ITERATION
PROCESS 2. 0:
IN EXAMPLE graph G is 14 = 2(8 - 1).Since the corresponding graph G2
LOCALMINIMUM: 0 : DECOMPOSITION
POINT.
contains three loops, it can be concluded, from the discussions
in Section V, that some critical points of the objective function
Iteration f.1, Step 2 Iteration # Z Step 3 Iteration #3: Step 3 must have the same function value. The function values of the
initial z = 0 5 0 5 ) initial L = z! initial z = z:
local minima in Table IV confirm this result.

mFm
v = {z!)
Iteration # 4 Step 3
initial z = 2:

2: =:
v = {z:,zf,z:)
Iteration # 5 Step 3
initial L = zi

=:
2,
2
:
v = {z:,.;,.:}
Iteration #6: Step 3
initial z = z:

z:
4
2, *:
4
VIII. CONCLUDING
REMARKS
In this paper, we have proposed a systematic method to
find several local optimal solutions for general nonlinear
optimization problems. Instead of using the concept of the
stability region which is commonly used in existing literature,
we employed the concept of quasi-stability regions to develop
a new mechanism to escape from a local minimum via
v = {z:,z$,.:) v = (z:,.:,.:) v = {& 4 6) the unstable manifold of a decomposition point [l], [2],
Iteration #I: Step 3
initial z = zi
Iteration #S: Step 3
initial t = z:
Iteration #9: Step 3
initial z = z:
[lo], [13], [20], [22], [24], [25]. We have also developed
o-g-&-g 4 2:
some analytical results for quasi-gradient systems and applied
them, in combination with a theory of reflected gradient
systems to derive the topological properties of the critical
=t 4 points of the underlying objective function. Moreover, we
v = {z:,z:,s:,z:} v = {z:,z:,z:) have shown that the unstable manifold of any decomposition
Iteration # 1 0 Step 3 Iteration #11: Step 3 Iteration #12: Step 3
initial z = z: initial z = zf initial z = s: point converges to two stable equilibrium points (i.e., two
local minima solutions). This result helped us to reduce
computational burden. By properly switching between quasi-
gradient systems and reflected gradient systems, our proposed
method can obtain a set of local minima and also construct the
Iteration #13: Step 3 corresponding graph describing the connection configuration
initial t = z:
among the local optimal solutions and the decomposition
points. The global optimal solution can be obtained if the
associated graph contains all the decomposition points. The
proposed method is comprised of two levels: the lower level is
V=B continuous while the upper level is discrete in nature. Further
improvements are desirable in locating all decomposition
points.

TABLE IV APPENDIX
local minima e(.) decomposition points
z: = (0.08984,0.71266) -1.0316 z1= (1.10921,0.76827)
zf= (1.70361,0.79608) -0.2236 z! = (0,O)
Proof of Proposition I
ZS = (-0.08984, -0.71266) -1.0316 Z? = (1.29607,-0.60508) This proof consists of two parts. First, we will show
= (-i.60710,0.56s65). 2.0985 = (i.63807,-0.2~867)
2: = (1.60710,-0.56865) 2.0985 ~ f =; (1.29607,-0.60508) that every bounded trajectory will converge to one of the
2: = (-1.70361, -0.79608) -0.2236 zs (-1.10921,-0.76827) equilibrium points. Second, we will show that every trajectory
+d - (-1.29607,0.60508)
is bounded.
Let @(x,t) denote the bounded trajectory starting at
minimum, then 20 initial points P~ = 2 + €(COS sin 2, g), x. Consider the time derivative & c ( @ ( s , t ) )along the
j = 0,. f . ,19 are used to locate the decomposition points on trajectory. It is clear that
the practical stability boundary dA, (2); otherwise, we just use d
2 f €EU,where E" is the unstable eigenvector of J-vc(z), -c(@(x,t)) =
dt
as the initial conditions to compute other local minima. Table - (VC(@(Z, t)))TR(@(x,
t))-lVc(qz, t ) )5 0
111 describes the value of variables at each iteration of the
algorithm. Moreover, & c ( @ ( x , t ) )= 0 if, and only if, 2 E E .
From the result of Table 111, we find that this objective Therefore, c ( x ) is a Liapunov function of the negative
function has 6 local minima and 7 decomposition points, i.e., quasi-gradient system (5) and the w-limit point of any
T = 7 and s = T - 1 = 6. The complete results are shown bounded trajectory consists of equilibrium points only,
in Table IV. i.e., any bounded trajectory will approach one of the
By comparing the values of local minima, we can conclude equilibrium point [17, p. 2031.
that the global minimum is -1.0316, which occurs at x: Since e(.) is bounded below, we want to show that every
and 2:. Recalling the graph structure described in Section trajectory @(z,t ) is also bounded. Suppose @(x,t ) is not
10s IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS--I: FUND- THEORY AM3 APPLICmONS, VOL. 43, NO. 2, FEBRUARY 1W6

bounded, we will show that this leads to a contradiction. Proof of Proposition 5

Two possible cases will be considered: Let T,(z,) denote the Jacobian matrix of g r u d R % c ( z eval-
)
a) After t > T*, @ ( x , t ) U z E ~ B ~ ( 2Since
). uated at the equilibrium point E,, i = 1 , 2 . H ( z e ) denotes
IIVc(x)II > 6 for x $ U Z E E B ~ ( ~ ) , [02c(xe)]-’.Since g c ( x ) vanishes at z = x,, J,(xe) =
c ( @ ( xt, ) )- e(@(? T * ) ) R,(ze)-’ v2e(%,). Moreover, H ( x e ) is a nonsingular sym-

= /I: .(a(., t ) ) d t 5 -k
L*
IIVc(x)112dt 5 -kc2(t - T*)
metric matrix.

where k is a positive constant. As t + 03, the right-

hand side of the above inequality approaches -CO, By Sylvester’s inertia theorem [15, p. 4451, I n ( J l ( z , ) ) =
which contradicts the fact that c ( @ ( xt, ) )is bounded I n ( H ( x e ) )= fn(J2(ze)),where In(.) denotes the integer
below. triple which consists of the number of eigenvalues of the given
b) There is an infinite sequence 2, E E and a strictly matrix with the negative, zero and positive real parts, respec-
increasing sequence p , such that @ ( z , p 3 ) E B6(&). tively. Therefore, a type-k equilibrium point of gradEl c(x) is
Let us define two strictly increasing sequences t, also a type-k equilibrium point of gradRi,c(z).This completes
and s,: t , is the first time that @(x,t ) enters the ball the proof. 0
Bs(2,) and sz is the first time that @(x,t ) leaves the
ball B&(&), where tz+l > s, > t,. Fix an integer
m > 0; then for t 2 tm+l we have Proof of Proposition 6
1) Since xd is a decomposition point, by Proposition 2.2.3
of [111, there exists a stable equilibrium point xt such at
Wu(xd)nA(z:)# 0 and W U ( x d ) \ { ~ d } n ( i l (#x 0. ~)c
Moreover, quasi-gradient system (5) is completely sta-
ble, and !Rn can be decomposed as the union of A(x“,),
where xi is the stable equilibrium point of the quasi-
gradient system (5) and (A(x;))‘ = U,.+,;A(z:). In
other words, there exists at least one xl other than x;
,=I such that V P ( z d ) \ { z d } n A ( x i ) # 0.By Theorem 3.3
of [6],we can conclude that xd E aA(xZ,).
where k is a positive constant. If m 403, then the
2) On the other hand, since the decomposition point must
right-hand side of the above inequality approaches
be type-one, the unstable manifold W U ( x d )is a 1-D
-03, which contradicts to the fact that e(@(., t ) )is
manifold. Remove xd from the unstable manifold to
bounded.
form two half-manifolds, W”+(xd)and W U - ( x d )Each .
Therefore, any trajectory @(z,t ) is bounded. half-manifold is, itself an invariant manifold consisting
0 of only a single trajectory only. Also quasi-gradient
system (5) is completely stable, and the stable and
Proof of Proposition 3 unstable manifolds satisfy the transversality condition.
Since the definition of the Liapunov function e(.) belongs Thus, by applying Lemma 3-5 of [6], it follows that these
to the class of energy functions [ 6 ] ,we can apply Theorem two trajectories must converge to two stable equilibrium
2.2.5 in [13] to obtain the above results. 0 points.
From the above argument, it can be confirmed that every
Proof of Proposition 4 decomposition point x$ can be connected to exactly two
Since different stable equilibrium points via its unstable manifold.0
1) e(.) is a Liapunov function for negative gradient system
REFERENCES
(51,
2) every trajectory within the stability region A(Z) con- [I] F. H. Branin, “Widely convergent methods for finding multiple solutions
verges to 2 , and of simultaneous nonlinear equations,” IBM J. Res. Develop., pp. 504-
3) the function value e(.) decreases along every nontrivial 522, 1972.
[2] L. 0. Chua, “Global optimization: a naive approach,” ZEEE Trans.
trajectory. Circuits Syst., vol. 37, pp. 966-969, 1990.
It is clear that 2 is a sink of (5) if, and only if, there exists 131 L. 0. Chua and G. N. Lin, “Nonlinear optimization with constraints:
a cookbook approach,” Inf. J. Circuit Theory Applicat., vol. 11, pp,
a neighborhood N such that e(.) reaches the local minimum 141-159, 1983.
at x = 2 . On the other hand, x is a source of - g r a d E c ( x ) [4] -, “Nonlinear optimization with constraints,” IEEE Trans. Circuits
Syst., vol. CAS-31, pp. 182.188, 1984.
if, and only if, is a sink of g r a d R c ( z ) and -e(.) is a [ 5 ] H. D. Chiang, F. F. Wu, and P. P. Varaiya, “Foundations of the potential
Liapunov function of gradRc(x). By the same argument of energy boundary surface method for power system transient stability
the above Liapunov analysis, we can conclude that z is a analysis,” ZEEE Trans. Circuits Syst., vol. 35, pp. 712-728, 1988.
[61 H. D. Chiang, M. W. Hirsch, and F. F. Wu, “Stability region of nonlinear
source of - g r a d e c ( z ) if, and only if, e(.) has an isolated autonomous dynamical systems,” IEEE Trans. Automatic Contr., vol.
maximum at 2. 0 33, pp. 16-27, 1988.
CHIANG AND CHU: MULTIPLE LOCAL OPTIMAL SOLUTIONS
[7] T. F. Coleman, “Large-scale numerical optimization: Introduction and
overview,” Tech. Rep., TR91-1236, Dept. Computer Sciences, Comell
Univ., NY, 1991.
[8] C. R. Corles, “The use of regions of attraction to identify global
minima,” in Toward Global Optimization, I. C. W. Dixon and G. P.
Szego, Eds. Amsterdam: North-Holland, 1975, pp. 55-93.
[9] C. L. De Marco, “Approximating power system dynamics and en-
ergy functions by quasi-gradient models,” in Int. Symp. Circuits Syst.
(ISCAS’89), 1989, pp. 1966-1969.
[lo] I. C. W. Dixon and G. P. Szego, Toward Global Optimization, vol. 1
and 2. Amsterdam: North-Holland, 1975.
[ l l ] L. Fekih-Ahmed, “New theoretical results on the stability regions and
bifurcations of nonlinear dynamical systems and their applications to
electric power systems analysis,” Ph.D. dissertation, Comell Univ., NY,
1991.
1121 P. Fletcher, Practical Methods of Optimization. New York Wiley,
1987.
1131 I. Fodorova, “Search for the global optimum of multiextrementalprob-
lems,” in Optimal Decision Theory, vol. 4, (Inst. Math. Cybem., Lithua-
nian SSR, Acad. of Sci.,) 1978, pp. 93-100.
1141 M. P. Kennedy and L. 0. Chua, “Nonlinear networks for nonlinear
programming,” IEEE Trans. Circuits Syst., vol. 35, pp. 554-562, 1988.
[15] P. Lancaster and M. Tismenetsky, The Theory of Matrices with Appli-
cations. Orlando, FL: Academic, 1985.
1161 R. Lum and L. 0. Chua, “The identificationof general pseudoreciprocal
vector fields,” IEEE Trans. Circuits Syst., vol. 39, pp. 102-123, 1992.
[17] M. W. Hirsch and S. Smale, Differential Equations, Dynamical Systems
and Linear Algebra, Academic: New York, 1974.
[18] H. Th. Jongen, P. Jonker, and F. Twilt, Nonlinear Optimization in Rn,
vol. 2. Frankfurtam Main: Verlag Peter Lang, 1986.
[19] H. Th. Jongen, and J. Sprekels, “The index-k-stabilizing differential
equation,” OR Spectrum, vol. 2, pp. 223-225, 1981.
[20] P. M. Pardalos and J. B. Rosen, Constrained Global Optimization:
Algorithms and Applications (Lecture Notes in Computer Science) vol.
268. New York Springer-Verlag, 1987.
[21] P. S. Parker and L. 0. Chua, Practical Numerical Algorithmsfor Chaotic
Systems. New York Springer-Verlag, 1989.
1221 H. Shashikala, N. K. Sancheti, and S. S. Keerthi, “A new approach to
global optimization using ideas from nonlinear stability theory,” in Int.
Symp. Circuits Syst. (ISCAS’92), 1992, pp. 2817-1820.
[23] S. Smale, “On gradient dynamical systems,” Annals. ofMath., vol. 74,
no. 1, pp. 199-206, 1961.
109
1241 G. Treccani, “On the convergence of Brain’s method,” in Toward Global
Optimization. Amsterdam: North-Holland, 1975, pp. 107-116.
[25] A. Tom and A. Zilinskas, Global Optimization, Lecture Notes in
Computer Science, vol. 350. New York Springer-Verlag, 1987.
[26] S. Vavasis, Nonlinear Optimization: Complexity Issues. New York
Springer-Verlag, 1990.
1271 J. Zaborszky, G. Huang, B. Zheng, and T-C. Leung, “On the phase
portrait of a class of large nonlinear dynamic systems such as the power
system,” IEEE Trans. Automat. Contr., vol. 33, pp. 4-15, 1988. ~
Hsiao-Dong Chiang (M’87-SM’91) received the Ph.D. degree in electrical
engineering and computer sciences from the University of California at
Berkeley in 1986.
He is currently an associate professor of electrical engineering at Comell
University, Ithaca, NY. His research interests include power systems, nonlin-
ear systems, optimization theory and neural networks.
Dr. Chiang was a recipient of the Engineering Research Initiation Award
(1988) and of the Presidential Young Investigator Award (1989) both from
the National Science Foundation. He was an Associate Editor of the IEEE
TRANSACTIONS ON CIRCUITS AND SYSTEMS (1990-1991). In 1990 he was
selected by a Comell Merrill Presidential Scholar as the faculty member who
had the most positive influence on that student’s education at Comell. He is
currently the Associate Editor for Express Letters of the IEEE TRANSACTIONS
ON CRCUITS AND SYSTEMS I: FUNDAMENTAL THEORY AND APPLICATIONS.
Chia-Chi Chu (S’91) was bom in Tiupei, Ta-
wan, Republic of China, on September 4, 1965. He
received the B.S. and M.S. degrees both in electri-
cal engineering from National Taiwan University,
Taipei, Tawan, in 1987 and 1989, respectively.
After two-year militrary services, he entered Cor-
ne11 University. Currently he is working toward the
Ph.D. degree in electrical engineering with a minor
in applied mathematics. His research interests focus
on nonlinear dynamical system theory, nonlinear op-
timization techniques, and nonlinear control theory.
Mr. Chu is a student member of SIAM.