INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL, VOL.

2, 291-311 (1992)

FEEDBACK CONTROL OF NONLINEAR SYSTEMS*

ALBERT0 ISIDORI
Dipartimento di Informatica e Sistemistica, Universita di Roma ‘La Sapienza’, 00184 Rome, Italy, and
Department of Systems Sciences and Mathematics, Washington University, St. Louis, MO 63130, U.S.A.

SUMMARY
This paper deals with the design of (memoryless) state-feedback laws for systems modelled by nonlinear
differential equations which are affine in the inputs. The purpose of the design is to obtain a (locally)
internally stable closed-loop system in which the effect of exogenous inputs on a prescribed error (or,
more in general, on a penalty variable) is attenuated. Two standard setups are considered: in the first
one, the ratio between the energy associated with the penalty variable and that associated with the
exogenous input is required to be bounded by a constant 0 < y; this setup includes (to some extent) the
standard H , controlproblem of linear system theory. In the second one, the penalty variable is required
to converge to 0 as t - m ; this setup generalizes the so-called servomechanism problem of linear system
theory.

1. INTRODUCTION
In the last decade, the development of specific methodologies for the design of feedback laws
in order to control systems described by nonlinear mathematical models has experienced a
major development. At the beginning of the decade, primarily because of the application of
mathematical concepts derived from the field of differential geometry, it became clear how to
design feedback laws which transform a nonlinear system into an equivalent linear system
(feedback linearization), and to design feedback laws which render certain outputs
independent of certain inputs (disturbance decoupling and non-interacting control). In
particular, feedback linearization techniques were successfully applied to the very difficult
problem of controlling an aircraft with multi-axis nonlinear dynamics. Moreover, the
geometric concepts were instrumental to a full understanding of the nonlinear equivalent of
the notion of zero of a transfer function. Around the middle of the decade, as a result of the
introduction of mathematical concepts derived from the field of differential algebra, further
thrust was added, to the research in nonlinear control, by the appearance of new methods for
the analysis of nonlinear input-output differential relations. Basic system theoretic concepts
such as left invertibility, right invertibility, rank of a system and their fundamental role in the
solution of the problem of achieving non-interaction via dynamic feedback were fully
understood. Towards the end of the decade a renewed interest took place in the longstanding

* Paper presented, as a plenary address, at the 1st European Control Conference, Grenoble, France, 2-5 July 1991.
This paper was recommended for publication by editor M. J. Grirnble

1049-8923/92/O40291-21$15 S O Received November 1991

0 1992 by John Wiley & Sons, Ltd. Revised May 1992
292 ALBERT0 ISIDORI

problem of asymptotic stabilization, leading to the development of systematic methods for the
design of (locally or globally) stabilizing as well as adaptive& stabilizing feedback laws for
selected classes and/or interconnected structures of systems. In the meantime, methods of the
solution of two outstanding problems of major engineering interest, the asymptotic tracking
of prescribed reference signals as well a the attenuation (below a specified threshold) of
exogenous disturbances, gradually became available.
Reasons of space, on the one hand, and the desire to provide a reasonably detailed and self-
contained deescription, on the other hand, have restricted this survey to the coverage of just
one of the specific research topics indicated above. Objective relevance to engineering practice,
and the author's personal interest in the subject, have led to the selection of the last one,
namely the design of control laws which internally stabilize a system and impose the fulfilment
of prescribed specifications (usually requiring the attenuation, or the asymptotic annihilation,
of a suitable penalty variable) on the response to exogenous inputs (which may include
references as well as disturbances).

2. NONLINEAR DESIGN WITH PERFORMANCE CRITERIA ON THE

INPUT-OUTPUT RESPONSE
Consider a nonlinear system modelled by equations of the form
i = F ( x, w,u )
z = H ( x , w,u )
y =K(x,w)
The first equation of this system describes a plant with state x , defined on a neighbourhood
X of the origin in IR" with control input uE IR" and subject to a set of exogenous input
variables w E IT?' which includes disturbances (to be rejected) and/or references (to be tracked).
The second equation defines a penalty variable z E IR", which may include a tracking error, for
instance the difference between the actual plant output and its desired reference behaviour,
expressed as a function of some of the exogenous variables w, as well as a cost of the input
u needed to achieve the prescribed control goal. The third equation defines a set of measured
variables y E IRp, which also are functions of the state plant x and the exogenous input w.
The mappings F: X x IR" x IR' + IR", H: X x IR" x IR' --i Rs,K X x IR' + IRp are smooth
(i.e. C") mappings. We assume also that F(0, 0,O)= 0, H (0 ,0,O)= 0 and K(0,O)= 0.
The control action to (1) is to be provided by a compensator which processes the measured
variable y and generates the appropriate control input u. This compensator could be either
static, i.e. modelled by an equation of the form
24 = a ( Y )

or dynamic, i.e. modelled by equations of the form

i= H(t9 Y)
= 8(E, Y )
in which E is defined on a neighbourhood ," of the origin in IR' and 7: ," x Rp IR", +

8: ," x IRp --i IR"' are C k functions (for some k 2 l), satisfying ~ ( 0 ~=00)and O(0,O) = 0.
In what follows, for the sake of simplicity, we will focus our attention on the design of static
feedback laws, and we will assume that the set y of measured variables consists of both the
plant state x and the exogenous input w. This is the so called full-information confrguration,
 CONTROL OF NONLINEAR SYSTEMS 293

and we will in this case express the feedback law more explicitly as
u =a ( x , w) (2)
The purpose of the control is twofold: to achieve closed-loop stability and to reduce (to some
extent) the influence of the exogenous input w on the penalty variable z. A compensator which
stabilizes the equilibrium x = 0 of the closed-loop system (in a sense that will be made precise
in the sequel) is said to be an admissible compensator. The requirement of disturbance
reduction, on the other hand, may be dealt with in different ways, depending on the specific
class of exogenous signals to be considered and/or the performance criteria chosen to evaluate
the penalty variable. Two different setups which are rather classical in control theory are the
following ones:
(1) Disturbance attenuation. Given a time T > 0, a set Wof piecewise continuous functions
w: [0, TI -, R', and a real number 0 < y, it is required to find, if possible, an admissible
compensator such that for each wE W , the response z: [0, TI R s of the closed loop
-+

system (1)-(2) starting from the initial state x(0) = 0 is well-defined and satisfies

s T

0
z'(s)z(s) d s < y2 so
T
w T ( s ) w ( s )d s

This setup includes the standard H, control problem of linear system theory.
(2) Output regulation. Given an autonomous differential system
w =s(w) (3)
where s is a smooth vector field defined in a neighbourhood W of the origin in Rr, it
is required to find, if possible, an admissible compensator such that for each initial
condition (x(O),w(0)) in a neighbourhood of the point (x, w )= (0,O) the response
z: [0, TI R s of the closed loop system (1)-(2)-(3) satisfies
-+

lim z ( t )= 0
t-'m

This setup generalizes the so-called servomechanism problem of linear system theory.
In Sections 3 and 4 we will illustrate how these two control problems can be dealt with.

3. DISTURBANCE ATTENUATION
The results described in this section are (simple) elaborations on a number of relevant research
papers dealing with the state-space approach to the so-called standard Hacontrol problem of
linear system theory (in particular References 1-6), its connection with the theory of
differential games (discussed in detail in Reference 7), a game-theoretic approach to the
nonlinear equivalent of the Hacontrol problem (discussed in Reference 8) and the synthesis
of control laws yielding disturbance attenuation (in the sense of an L2-induced norm, discussed
in Reference 9 for a class of affine nonlinear systems). The author is indebted to T. Basar,
J. W. Helton, P. P. Kargonekar, A. A. Stoorvogel and A. Van der Schaft for having provided
him with a number of most important preprints, on which the material presented in
subsections 3.1-3.3 and 3.5 is based. The results described in subsection 3.4 do not seem to
have been published before. In particular, the author wishes to thank T. Basar for having
drawn his attention to the fact the so-called Isaacs equation is perhaps the most natural and
easy point of departure to understand a number of key developments in Ha control.
294 ALBERT0 ISIDORI

3.1. A n associated digerential game.

One of the objectives of the control problem described above is to render

1 zT(s)z(s)
0
T
d s < y2 10
T
wT(s)w(s) d s

Thus, as suggested for example by References 8 and 7, it is natural to associate with this
problem a two players, zero sum, diferential game
i = F(x, w,u )
of fixed duration [0, T I , with value functional

J(u, w ) = 10
T
L ( x ( s ) , N S ) , u(s))d s

where L ( x , w,u ) is given by

L ( x , w, u ) = H T ( x ,w , u)H(x,w,u ) - y2w'w

The minimizing player controls u and the maximizing player controls w (the ultimate goal of
the minimizing player being, in the present setting of the problem of disturbance attenuation,
to render J(u, w ) non-positive for all w).
In view of the fact that the strategy of the minimizing player is to be implemented via
feedback, it is natural to set the game in question under the so-called feedback information
conJguration, i.e. allowing the values of u and w at time t to depend only on the value of x
at this time t (and possibly, on the value of r itself).
The theory of these kind of games is analysed e.g. in Reference 10, from which we quote
the following result, whose consideration is useful in the present setting.

Theorem
For a two-person zero-sum differential game of prescribed fixed duration [0, TI a pair of
strategies u*(t, x ) , w*(t, x ) provides, under feedback information pattern, a saddle point
solution (i.e. a solution such that J(u*, w ) < J(u*, w*)< J(u, w*)) if there exists a C'
function E [0, TI x IR" IR satisfying the partial differential equation
-+

- Vt = min max [ VxF(x,w,u ) + L ( x , w,2411

u w

= max min[VxF(x, w,u ) + L ( x , w , u)l (4)

w u

= V./xF(x,w*(t,x), u*(t, x ) ) + L ( x , w*(t, x ) , U*(t,x ) )

with V(T,x ) = 0 .
Here Vt and Vx denote first order partial derivatives of V , namely

vt =
av
at Vx =
av
ax
The partial differential equation (4) was first obtained by Isaacs and is often called by his
name. The requirement of interchangeability of the min and max operations in (4) is known
as fsaacs condition.
 CONTROL OF NONLINEAR SYSTEMS 295

3.2. The Isaacs equation f o r afine nonlinear systems

We consider henceforth the case of a plant described by equations of the form
i =f(x) + gl(x)W + gz(x)u (5)
z = h(x) + ki(x)w + kz(X)U (6)
Moreover, we suppose the matrix kz(x) to have rank m (i.e. equal to the number of its
columns) for each x. Without loss of generality (after possibly an x-dependent co-ordinates
transformation in the space of the values of the control input) this is equivalent to assuming
that
kT(x)kz(x) = z
for each x.
In the notation of the previous subsection, we now have
F ( x , W , u ) = A x ) + g1 (x)w + gz(x)u
H(x, W , U ) = h(x) + ~ I ( X )+W~ Z ( X ) U
and therefore the quantity
H(x, V,, W , U ) = V,F(x, W , U ) + L(x, W , U )
is a polynomial of degree two in (w,u). In fact,

where

and

with

( W * , U*) = 0, -
aH ( W * , U*) = 0
au aw
then clearly H(x, V,, W , u ) can be expressed as
296 ALBERT0 ISIDORI

or, also, as
H(X, v,, W , U ) = H ( X , vX,w*, u*) + [ W - W*I T ( ~ w w -Q : ~ Q ~ ~ ) [-WW*I
+ [U - U * + Q u w ( w - w*)]T [ -~ U * +Q ~ ( w- w * ] (8)
The solution ( w * , u * ) of (7) exists and is unique if and only if the matrix R ( x ) is invertible,
and this occurs if and only if so is the matrix (recall Q u u ( X ) = Z)
Q ~ ~ ( x- )Q S ( X ) Q ~ ~ ( X=) ~ : ( x ) [ I - k z ( x ) k ~ ( x ) i k l ( x -
) y21 (9)
If this is the case, then w* and u* have the expressions

Note that, since Vx is a function of x and t, so are w* and u*.

It is now clear that Zsaacs condition is satisfied if the matrix Qww(x) is negative dejinite for
each x. In this case, in fact, the matrix (9) is non-singular, the expression (8) for H ( x , V,, w , u )
holds, the min and max operations in (4) interchange, and indeed ( w * , u*) is a saddle point
for H ( x , V,, w, u ) namely
H ( x , V,, w, u * ) < H ( x , Vx,u*,w * ) ,< H ( x , Vx,w*,W )
Note also that, by construction, the fulfilment of Isaacs equation requires
Vt + H ( x , Vx, w * ( x , Vx), U * ( S V.1) = 0 (1 1)

3.3. The time invariant Isaacs equation for a@ne nonlinear systems
We return now to the analysis of the disturbance attenuation problem. To this end we may,
as we shall see in a moment, drop the assumption that the matrix Q w w ( x ) is negative definite,
and replace it by the weaker assumption that the matrix
Q w w ( x )- Q L W ( x ) Q U w W
is negative definite. Under this assumption the expression (8) is still valid but the pair ( w * , u * )
given by (10) is not necessarily anymore a saddle point for H ( x , V,, w, u ) . In other words, we
do not anymore assume that H ( x , V,, w, u ) satisfies Isaacs condition. We continue to suppose,
however, that the pair (w",a*) is a zero of Vt + H ( x , V,, w, u),i.e. that the equality (1 1) holds.
In this situation, it is immediate to check that the control law
u = C Y ( X , W ) = u * ( x , Vx) - Q U w ( x ) ( w - w * ( x , Vx))
renders
vt+ H ( X , vx,w,(Y(x,w ) ) = [W - ~7 T ( ~ w w (-~ Q
) Z ( X ) Q ~ ~ (-XW)*)I [,<
Wo (12)
This property is the basis of our subsequent developments.
Since we are interested in finding a time invariant control law, we seek a time-independent
function K R" -,IR which fulfils the previous conditions. The basic assumption that
Q w w ( x ) - Q : w ( ~ ) Q u w ( ~ ) is negative definite is not affected by this choice, the term V, is
vanishing, V, is independent of t, and the first order partial differential equation ( 1 1 ) which
characterizes V assumes the form
2 *T
0 = V x c f ( x ) + S l ( X ) W * ( X , V,) + $fZ(X)U*(X, V x ) l - y w ( x , V X ) W * ( X , V*)
+ [ h ( x ) + k l ( X ) W * ( X , Vx) + k Z ( X ) U * ( X , V x ) ] T [ h ( X ) + k l ( X ) W * ( X , Vx) + k Z ( X ) U * ( X , VxN
(13)
 CONTROL OF NONLINEAR SYSTEMS 297

with (w*(x, V,),.u*(x, V,)) still given by (10). The equation (13) is a Hamilton-Jacobi-type
equation and can be called the ffamilton-Jacobi-~soac equation associated with the
disturbance attenuation problem.
Suppose now that there exists a positive semide3nite solution V(with V(0)= 0 and, thus,
V,(O) = 0) of the Hamilton-Jacobi-Isaacs equation and, also, suppose that for some fixed
exogenous input function w: [O,T] IR", the integral curve x ( t ) of
-+

i = A x ) + g l ( x ) w + gz(x)Cr(x, w )
satisfying x(0) = 0 is defined for all CE [0, TI.
Integration of H(x, V,, w , a(x, w ) ) along this curve yields, in view of (13),

V ( x ( t ) )- V(0)+ 1' (zT(s)z(s)

0
- y2w'(s)w(s)) ds <0

i.e.

for all t c [0, T I .

We summarize this fact in the following statement.

Proposition 1
Consider the nonlinear system (5)-(6). Suppose
kT(x)kz(x)= I
k T ( x ) [ I - kZ(X)k2T(X)]kl(X)- y Z I < 0
at each x. Suppose there exists an open neighbourhood U of x = 0 in R" and a positive
semidefinite 12'function V: U+R which satisfies the Hamilton-Jacobi-Isaacs equation (13).
Consider the feedback law
u = U * ( X , V,) - k Z ( X ) k l ( X ) ( W - W * ( X , V,)) (14)
with u * ( x , V,) and w * ( x , V,) given by (10). Each exogenous input function w : [0, T ] -+ R'
such that the integral curve of (5)-(14) initialized at x(0) = 0 is in U for each t E [0, TI yields
a response z: [0, T J 3 R" satisfying

for all t~ [O, T I .

Some remarks are in order at this point.

Remark (i). If Q W W ( x<) 0, i.e. if the Isaacs (saddle point) condition is satisfied, the same
result expressed by the previous Proposition, namely the attenuation of the exogenous input,
can be obtained by means of the w-independent law
u = a ( x ) = U * ( X , V,)
In fact, as we have seen, this assumption implies
) )H(x, V,, w, u * ( x , K)) < H(x, V . , w * ( x , V,), u * ( x , VX))= 0.
Hfx, V,, w , ( ~ ( x =
298 ALBERT0 ISIDORI

Thus integration along the solution x ( t ) of

i = f ( x ) + g1 ( x ) w + gt(x)cY(x) x(0) = 0
yields again (15) (compare with Reference 6 , pp. 49-50, for a corresponding result in the case
of a linear system).
The two hypotheses Qww(x)< 0 and Q w ( x ) - QEW(x)QUw(x) < 0 indeed coincide if
Quw(x)= kT(x)ki ( X I = 0
This is the case when, for instance, the penalty variable z has a form
z= [“‘I
z2
- [hdx) + q1(x)w]
-
92 ( x ) u
in which z1 is a quantity which represents a tracking error (see also Reference 1 for similar
assumptions in the case of linear systems).

Remark (ii). It is well known that even in the case Q w ( x ) < 0, i.e. when H(x, V,, w ,u ) has
a unique saddle point, the strategy (u*(x, Vx),w*(x, V . ) ) is not necessarily a feedback saddle
point solution, on the injinite horizon, of the differential game considered in subsection 3.1
(see, for example, Reference 11). It is, however, possible, as done in Reference 11 for the case
of a linear system, to show that if V ( x )is a positive semidefinite solution of the time-invariant
Isaacs equation 13 satisfying V(0)= 0 and Y(t,x ) is the solution of the time-dependent Isaacs
equation 11 satisfying V ( T ,x ) = 0 and defined for all t E [0, T I , then
V(X)2 Wl, x ) 2 V(t2, x )
for all 0 < t l < tz < T.
Remark (iii). The solution of the Hamilton-Jacobi-Isaacs equation is discussed, for
example, in Reference 12, pp. 148-155. Setting
v, = p T
the solution of this equation is reduced to the integration of the Hamiltonian system

dp-
dt 4a w ,
- pT,w * ( x , pT),U ( X , pT))
ax )
If V is Cz, differentiation with respect to x of the identity (1 l), that is
H ( x , V., w*(x, K), u*(x, Vx))= 0
yields

because
 CONTROL OF NONLINEAR SYSTEMS 299

Thus, the graph of the mapping T: x - +V,' is an invariant manifold of the Hamiltonian
system (16). Moreover, the restriction of (16) to this invariant manifold is exactly given by
i = F(x, W*(X, Vx),U*(X, V,))

Remark (iv). In the case of a linear system

i=Ax+B1w+Bzu
y = c x + DllW + D12u
the basic assumptions of Proposition 1 become
DW12 =
DT1 [ I - D12DT21Dll - y 2 z< 0
A quadratic function X'XX is a solution of the Hamilton-Jacobi-Isaacs equation (13) if and
only if X is a solution of the algebraic Riccati equation
A T X +X A + CTC- F T ( X ) R F ( X =) 0 (17)
where

and the latter is exactly the Riccati equation considered in the standard H , control of linear
systems (see, for example, References 2 and 6).

3.4. The issue of asymptotic stability

So far, we have shown that the feedback law (14) guarantees disturbance attenuation-on
finite time intervals-for each disturbance in a suitable class. Of course, it is of interest also
to examine the stability properties of the closed-loop system induced by this particular
feedback law. To this end, it is possible to prove the following result.

Proposition 2
Consider the nonlinear system (5)-(6). Suppose
kZ(x)kz(x)= Z
k T ( x ) [ Z - k2(X)kZT(X)]kl(X)
- y2ZC 0

at each x . Suppose there exists an open neighbourhood U of x = 0 in R" and a positive deJnite
C' function V U - t R which satisfies the Hamilton-Jacobi-Isaacs equation (13) and such that
the system
x = f ( x ) + gl (X)W*(X,V,) + gz(x)u*(x,V*) (19)
has a locally asymptotically stable equilibrium at x = 0. Then, the feedback law (14), with
w = 0, locally asymptotically stabilizes the equilibrium x = 0 of the corresponding closed-loop
system. If U = R", if Vis proper and if the equilibrium x = 0 of (19) is globally asymptotically
stable, so is the equilibrium x = 0 of the closed-loop system (5)-(14).
300 ALBERT0 ISIDORI

Proof. Set w = 0 in the closed-loop system (5)-(14), to obtain

i = f ( x ) + ~ ~ ( x ) [ u * V( x. ), + Quw(x)w*(x, K)l (20)
Recall that, by assumption, the matrix Q w ( x ) - Q z W ( x ) Q U wis( ~negative
) definite and that,
therefore, there exists a unique non-singular matrix T ( x ) such that
Qww(x)- Qzw(x)Quw(X) = - T T ( x ) T ( x )
Consider the function y: U-IR defined by
Y ( X ) = T ( x ) w * ( x ,V1.
Setting w = 0 and u = u*(x, V.) + QUw(x)w*(x, V x ) in the identity (8), recalling (11) and
integrating along the trajectories of the system (20), we obtain

= - sf
0
Iv(x(s))l T ~ ( ~ (d ss <) 0)

From this, we deduce that V ( x ( t ) )is non-increasing along the trajectories of (20). Since V is
positive definite, for any sufficiently small a > 0, the set V - ' ( [0, a]) is compact, and therefore
the equilibrium x = 0 of the closed loop is Lyapunov stable. Choose a sufficiently small initial
-
condition x o , let xo( ) denote the corresponding trajectory and let yo denote its w-limit set
(which is non-empty, compact and invariant). Since l i m p mV ( x ( t ) )= a. 2 0, by continuity of
V , V ( x )= a0 at each point x of yo. Let [ be a point of yo and E ( . ) the corresponding
trajectory. Since E ( . ) c yo, then V(E(t))= a0 for all t 2 0 and

0 = UEW) - UE(0))< - if0

[Y(E(S))l TY(E(s))d s

implies y ( E ( t ) ) = 0 for all t 2 0. We will show now that this implies limt+&(t) = 0.
In fact, observe that the closed-loop system (20) can be expressed as
i = f ( x ) + gz(x)u*(x, K) + g2(x)Quw(x)w*(x,Vx)
= f ( x ) + gz(x)u*(x, K) + gi(x)w*(x, Vx) + [ - g i ( x ) + g ~ ( ~ ) Q u w ( xT)-I' ( x ) Y ( x )
Any trajectory x ( . ) of this system yielding y ( x ( t ) )= 0 for all t 2 0 is also a trajectory of the
system (19). Thus, since the latter has by hypothesis an asymptotically stable equilibrium at
x = 0, for each initial condition t in a neighbourhood of x = 0, we have limf+&(t) = 0.
Returning to the previous discussion, we can deduce that a0 = 0. Thus, limf+mV ( x o ( t ) )= 0,
i.e. limt+mxO(t)= 0. This proves local asymptotic stability of the equilibrium x = 0. The
arguments for global asymmptotic stability are similar.
Merging the results of Propositions 1 and 2, we can summarize the results of this and of
the previous subsection in the following terms.

Theorem 1
Consider the nonlinear system (5)-(6). Suppose
 CONTROL OF NONLINEAR SYSTEMS 301

at each x . Suppose there exists an open neighbourhood U of x = 0 in IR" and a positive dejnite
C' function E U+IR which satisfies the Hamilton-Jacobi-Isaacs equation (13) and such that
the system
i =f ( x ) + g l ( x ) w * ( x , V,) + g2(x)u*(x, V,)
has a locally asymptotically stable equilibrium at x = 0. Then, the feedback law (14) locally
asymptotically stabilizes the equilibrium x = 0 of the corresponding closed-loop system (with
w = 0). Moreover, each exogenous input function w: [0,T ] R r such that the integral curve
-+

of (5)-(14) initialized at x(0) = 0 is in U for each t f [0, TI yields a response z: [0, TI IR"
-+

satisfying

for all f E [0, TJ

It is of course interesting to examine the form assumed by this result in the case of linear
systems. In the notation of Remark (iv), the result expressed by Theorem 1 says that if there
is a positive definite matrix X which:
(i) satisfies the Riccati equation (17),
(ii) is such that the matrix
A + B I F I+ B2F2
(with FI and F2 given by (18)) is a stable matrix,
then the feedback law
U = F I-X )
FZX-DT~DII(W
asymptotically stabilizes the corresponding closed-loop system (for w = 0) and renders its
&-induced norm less than or equal to y. Thus, the condition expressed by Theorem 1 is a
nonlinear equivalent of a well known suficient condition (which is known to be also necessary,
under appropriate hypotheses) for the existence of a linear feedback law of the form
u = Klw + KZXwhich asymptotically stabilizes a system and renders its H , gain less than or
equal to y (see, for example, References 3 and 2).
The two conditions (i) and (ii) above have been recently given a very expressive
characterization in Reference 1. Recall that a Hamiltonian matrix

"=[" - Q "1
-AT

is said to belong to dom ( R i c ) if it has no eigenvalue on the imaginary axis and its generalized
(n-dimensional) stable eigenspace can be expressed as

for some P (see Reference 1). If this is the case, the (uniquely determined) matrix P is written
as
P = Ric(H)
Alternatively, HE dom(Ric) if and only if there exists a symmetric matrix X which solves the
302 ALBERT0 ISIDORI

algebraic Riccati equation

A ~ +
X XA + XRX + Q = o
and (A + RX) has all eigenvalues with negative real part (in which case X is necessarily equal
to Ric(H)).
Proceeding as in Reference 1 , it is not difficult to check that the existence of a matrix X
satisfying the conditions (i) and (ii) stated above is equivalent to the fact that the Hamiltonian
matrix

belong to dom(Ric) (in which case, X is necessarily equal to Ric(H,)). Thus the existence of
a positive definite matrix X satisfying the conditions (i) and (ii) is equivalent to the following
pair of conditions:
H, E dom( Ric) (22)
Ric(Hm) > 0 (23)
Returning now to the case of a nonlinear system, one may observe that, in view of the
properties of the Hamiltonian system introduced in Remark (iii), a function V: U+R which
satisfies the Hamilton-Jacobi-Isaacs equation (13) is such that the system (19) has a locally
asymptotically stable equilibrium at x = 0 (i.e. satisfies the conditions required by Theorem 1)
if and only if the restriction of the Hamiltonian system (16) to the invariant manifold
A= ( ( x ,p ) : p = VXT(x))
has an asymptotically stable equilibrium at x = O . This formulation appears to be an
appropriate nonlinear analogue of the condition H , E dom(Ric). Note also that the manifold
&is not necessarily the stable manifold of the Hamiltonian system (1 6), because the restriction
of (16) to A, namely the system (19), does not need to have an exponentially stable
equilibrium at x = 0.

3.5. A suficient condition for disturbance attenuation

The purpose of this subsection is to show that the existence of a function V satisfying the
conditions expressed by Theorem 1 can actually be decided on the basis of the knowledge of
the linear approximation of the nodinear system (5)-(6) around the equilibrium
( x , w , u ) = ( O , O , 0).
Let the linear approximation of (5)-(6) at the equilibrium ( x , w , u ) = (0, 0,O) be denoted, as
before, by
i =A x + B*w + B2U
y = CX + Diiw + D 1 2 ~
and consider the associated Hamiltonian matrix Hm defined by (21).
Adapting to the present setting the arguments used in Reference 9 for the particular case in
which hT(x)k2(x)= 0 and k l ( x ) = 0, one can easily arrive at the following result.

Proposition 3
Suppose HmE dom(Ric) and suppose X , = Ric(H,) > 0. Then there exists an open
neighbourhood U of x = 0 in IR" and a positive definite C 2 function K U-IR which satisfies
 CONTROL OF NONLINEAR SYSTEMS 303

the Hamilton-Jacobi-Isaacs equation (13) and such that the system

i =f ( x ) + g l ( x ) w * ( x ,V.) + & ( X ) U * ( X , V.)
has a locally asymptotically stable equilibrium at x = 0.

Sketch of the proof. By construction, the linear approximation, at ( x , p ) = (0,0), of the

Hamiltonian system (16) considered in the Remark (iii) is exactly

Thus, since H, E dom( Ric), this Hamiltonian system has an n-dimensional stable submanifold
which can be expressed, locally around ( x , p ) = (0,0), as the graph of a mapping a: x -,p .
Moreover, by Proposition 3 of Reference 9, it can be shown that there exists a solution V ( x )
of the Hamilton-Jacobi-Isaacs equation (10) such that
a(x)= VX'(x)
In particular, the Hessian matrix of V ( x ) at x = 0 coincides with 2Ric(Hm) and therefore as
a consequence of the assumption X m > 0, V ( x ) is positive definite in a neighbourhood of
x = 0.
Merging the results of Theorem 1 and Proposition 3, one arrives immediately at the
following conclusion.

Corollary
Assume
kZT(X)kZ(X)= z
k ? ( x ) [ I - k2(X)kP(X)lkl(X)- y 2 1 < 0
H m 6 dom( Ric)

Ric(Hm) > 0

Then the feedback law u = a ( x , w ) given by (14), with V positive definite solution of the
Hamilton-Jacobi-Isaacs equation (13), which is defined for all ( x , w ) in a neighbourhood
U x W of ( x , w ) = (0,0 ) , locally asymptotically stabilizes the corresponding closed-loop system
at the equilibrium ( x , w ) = (0,O) and renders

for all t E [0,TI and all exogenous input functions w: [0, TI -, IR' such that the integral curve
x ( t ) of (5)-( 14) satisfying x(0) = 0 remains in U for each s E [ O , TI .

4. OUTPUT REGULATION
The results described in this section summarize a number of relevant findings which appeared
in a series of papers, 1 3 - 1 5 where the problem of output regulation for a nonlinear system was
addressed and locally solved, together with some interesting related results on the
approximation of the corresponding nonlinear control laws (discussed in Reference 16).
304 ALBERT0 ISIDORI

4.1. The steady state response of a nonlinear system

In the case of a linear system, the problem of output regulation amounts to the design on
an (internally) asymptotically stable closed loop system whose steady state response exactly
coincides with a prescribed reference input. In order to understand how the design of a
controller inducing a similar behaviour in a nonlinear plant is possible, it is convenient to
examine first to what extent the notion of steady state response can be extended to a nonlinear
system.
Loosely speaking, the steady state response of a system to some specific input is a function
of time to which the actual response 'converges' as time increases, provided that such a
convergence takes place. In more rigorous terms, this concept can be expressed as follows.
Consider a system
i =f ( x , u ) (24)
with f(0,O) = 0, and let x(t, xo, u( .)) denote the value of the state achieved at time t, starting
from the initial state xo at time t = 0 , under the effect of the input u ( . ) . Let u s s ( . )be a
prescribed input function and suppose there exists an initial state xsswith the property that
Iimx(t, xo, u s s ( . ) -
) x(t, xss,u s s ( . )=
) 0
1-w

for all x o in a neighbourhood of xSS.If this is the case, then the response
x s s ( t ) = x(t, xss, usd.))

is called the steady state response of the system to the specific input ass( ). -
) generated by some dynamical system of the form (3), i.e.
Suppose now the input u s s ( . is
uss = d w)
w = s(w)
and, moreover, suppose the system (3) satisfies the following assumption, that we will call
assumption of neutral stability: the equilibrium w = 0 is stable and, for some neighbourhood
W of w = 0, the set of states which are Poisson stable is dense in W. Then, it is easy to prove
that, if the equilibrium x = 0 of i =f ( x , 0) is locally exponentially stable, for each initial
condition w(0) in a suitable neighbourhood of w = 0, the input us&) = q ( w ( t ) ) induces a
steady state response. More precisely, we have:

Proposition 4 (Reference 15)

Suppose (3) satisfies the assumption of neutral stability. Suppose the equilibrium x = 0 of
i = f ( x , 0 ) is locally exponentially stable. Then, there exists a C k ( k 2 2) mapping x = ~ ( w ) ,
locally defined in a neighbourhood of w = 0, which satisfies

Any reference input uo( .) = q(w( .)), produced by (3) initialized at some initial condition
w(0) = w o in a neighbourhood of w = 0, induces a steady state response, which is given by
) r(w0(t))
x s s ( t= (26)
Equation (25) expresses the fact that the graph of the mapping w -+ ~ ( wis)an invariant
 CONTROL OF NONLINEAR SYSTEMS 305

manifold (more precisely, a centre manifold) for the composite system

i = f ( x , 4(wN
w = s(w)
From (26) it is seen that the initial condition from which the steady state response to u o ( . )
takes place is exactly equal to ?r(wo),namely to the projection on x-space of the (unique) point
of the graph of 7r whose projection on the w-space is wo.

4.2. Output regulation

Consider again a system of the form (1) and suppose the penalty variable z does not include
the control input u explicitly, i.e. suppose
2 = H ( x ,w) (1b' 1
Moreover suppose, as announced in Section 2, that the exogenous inputs are generated by a
system of the form (3) (which is called the exosystem). The problem of (local) output
regulation consists in finding, if possible, a feedback law u = a ( x , w ) such that:
(i) the closed loop system
x = F ( x , 0, a ( x ,0))
has a locally exponentially stable equilibrium at x = 0,
(ii) for each w o in a neighbourhood of w = 0, the steady state response x,,(t) of
x = F(x, w,a(x, w))
to the exogenous input w o ( t )satisfying the initial condition wo(0)= w oexists and is such
that W(x,,(t) ,wo(t)) = 0.
Alternatively, the second requirement may be expressed by asking that for each pair (xo,w o )
in a neighbourhood of (x, w ) = (0,0), the response of
i = F(x, w,a(x, w))
w = s(w)
satisfying (x(O), w(0))= (xo, wo),be such that
lim z(t) = lim H(x(t), w(t)) = 0
t-oo I-m

We will describe now a necessary and sufficient condition for the solvability of the problem
under consideration.

Theorem 2 (Reference 14)

Suppose the exosystem (3) satsifies the assumption of neutral stability. Then the problem of
output regulation is solvable if and only if:
(i) the linear approximation of F(x, 0, u ) at (x, u ) = (0,O)is stabilizable;
(ii) there exists a pair of C k ( k 2 2) mappings, x = ~ ( wand ) u = c ( w ) (satisfying a(0) = 0
and c(0)= 0) such that
ax
- s ( w ) = F ( a ( w ) ,w,c ( w ) ) (27a)
aw
0 = H(?r(w),w ) (27b)
306 ALBERT0 ISIDORI

(Constructive) proof of the suflciency. By assumption (i), there exists a matrix K such that
all the eigenvalues of the Jacobian matrix

have negative real part. Choose the control law

u = (Y(x,W ) = C ( W ) + K ( x - T(w)) (28)
where C ( W ) and n(w) are mappings which satisfy (27). Then, the closed-loop system (la)-(28),
i.e. the system
i = F(x, W , C ( W ) + K ( x - T(w)))
satisfies the assumptions of Proposition 4 and in particular, because of (27a), the mapping
x = n(w) satisfies a condition which corresponds to (25). As a consequence, for each
sufficiently small wo,in the closed loop system (la)-(28), the steady state response to the input
w0(.) produced by the exosystem (3) initialized at ~ ' ( 0=)w o exists and is given by
xss(t)= a(wO(t)) t20
Thus, because of (27b), H(x,,(t),w ( t ) )= 0, and the result follows. 0

Remark (v). In the case of a linear system

i== A X + B ~ w BZU
+
w = sw
z=cx+DllW
the output regulation problem has been addressed and solved by several authors (see, for
example, References 17-19). In particular, Reference 19 has shown that, if the pair ( A ,B z ) is
stabilizable, a compensator yielding closed-loop stability and output regulation can be found
if the following pair of matrix equations
ns = A ~+I~~r+ B ]
0 = cn + Dll
is solvable, in the unknowns n and .'I The solvability of these equations becomes also a
necessary condition for the solvability of the linear regulator problem is the exosystem is
antistable, i.e. the eigenvalues of the matrix S are all in the close right-half complex plane. Of
course, in the case of a linear system, equations (27) reduce to (30).

Remark (vi). Note that the two equations (30) can be merged into the single matrix equation

which, according to a result of Reference 20, is solvable for any B1 and Dll if and only if the
polynomial matrix

has rank equal to the number of its rows for each X which is an eigenvalue of S . The matrix
R(X) is actually the well-known Rosenbrock's system matrix of the plant. The property that
this matrix has rank equal to the number of its rows for some X corresponds to the property
 CONTROL OF NONLINEAR SYSTEMS 307

that the plant is right invertible, and the values at which its rank may drop are exactly the
transmission zeros of the system. Thus, the result of Reference 20 says that the equations (30)
are solvable (i.e. the linear output regulation problem is solvable) for any B1 and DIIif and
only if the plant is right-invertible and none of its transmission zeros coincides with an
eigenvalue of the exosystem.
Similar conditions for the solvability of the regulator equations (27) have been established
in Reference 13 for a nonlinear system, under appropriate regularity assumptions. These
conditions, that cannot be summarized here in detail for reasons of space, reduce the problem
of the existence of a pair ( ~ ( w )c (,w ) ) solution of the regulator equations (27) to a property
of the so-called zero dynamics of the composite system
i = F(x, w , u )
w =s(w)
z = h(x,w)
We recall that the zero dynamics of a system is an object that extends, to the case of
nonlinear systems, the notion of numerator polynomial of the transfer function of a (single-
input single-output) linear system. For a multivariable and invertible nonlinear system having
the same number of inputs and outputs, under appropriate regularity assumptions, the zero
dynamics can be given (see, for example, Reference 21, pp. 290-293) the form of an
autonomous differential (sub)system
f = f *(r)
evolving on a submanifold Z* of the system state-space. The bridge with the notion of
numerator polynomial of a (single-input single-output) linear system lies in the fact that, if the
system (of which the zero dynamics are considered) is linear, then f*({)also is linear in (,
and has a characteristic polynomial which coincides with the numerator polynomial of the
system transfer function. In particular, f *({) is a linear operator whose eigenvalues coincide
with the zeros of the system.
In Reference 13, it is shown that the existence of a pair (~(w),c ( w ) )solution of the regulator
equations (27) is a property of the zero dynamics (Z:, f:) of the composed system (31),
namely the existence of two complementary invariant submanifolds, 2, and Zf characterized
by the following properties:
(i) the restriction of 2: to Z , is a diffeomorphic copy of the exosystem
w = s(w)
(ii) the restriction of Z: to Zf is a diffeomorphic copy of the zero dynamics of
i = F(x,0 , u )
2 = H ( x ,0 )

From this rather theoretical characterization it is possible to derive a very simple and
expressive sufficient condition for the solvability of the regulator equations (27). More
precisely, it is possible to show l 3 that, under the same regularity assumptions required for the
general result to hold, the regulator equations (27) are solvable for a pair ( ~ ( w )c,( w ) ) if the
linear approximation of the plant, written in the form (30), satisfies

rank[
A-joZ B2
01=n+s

for each w E R, i.e. is right invertible and has no transmission zeros on the imaginary axis.
308 ALBERT0 ISIDORI

As in Section 3, we see that the existence of a solution to a specific design problem for a
nonliner system is guaranteed by the existence of a solution of the corresponding problem for
the linear approximation. It must be stressed, however, the actual solutions of the problem
(for the original nonlinear systems and for its linear approximation) need not be coincident and
are in fact different.

4.3. Approximate solutions of regulator equations

In general, it may not be possible to find a solution of the regulator equations (27) in closd
form. However, to the purpose of designing a compensator, approximate solutions may be
convenient as well. Suppose m ( w ) and ck(w) are polynomials of degree k 2 2 such that

H(Tk(W), w ) = R2(w)
where Rl(w) and &(w) are residual terms which vanish at w = 0 together with their partial
derivatives of order less than or equal to k.
Starting from these polynomials, define a new compensator
u = cYk(X, w ) = C k ( W ) + K ( x - 7rk(W)) (33)
By construction, the system
i = F(x, w,w ( x , w))
w = s(w)
satisfies again the assumptions of Proposition 4. As a consequence, for each sufficiently small
wo, in the closed-loop system (la)-(33), i.e. in the system (34a) there exists a steady state
response to the input wo(.)produced by the exosystem (3) initialized at ~'(0) = wo. This
steady state response has the following form
xss(t)= p ( w O ( t ) ) t20
where x = p(w) is a mapping satisfying the equation (which corresponds to (25))

i.e. whose graph is a centre manifold for (34).

From centre manifold theory, it is known that the mappings x = p(w) and m ( w ) agree up
to order k. Thus, since H ( x , w ) is continuous, (32), implies
HOl(w), w ) = R3 (w)
where R3(w) is a residual term which vanishes at w = 0 together with its partial derivatives of
order less than or equal to k. We can therefore conclude that, for each initial condition xo in
a neighbourhood of x = 0, the steady state response x o ( . )of the system (la)-(1b)-(33), to the
input wo( .) produced by the exosystem (3) initialized at ~ ' ( 0 =) wo,exists and satisfies
zss(t) = H(xss(t),w ( t ) )= H ( p ( w ( f ) ) w
, ( t ) )= R3(w(t))
i.e. is of order k in w ( t ) .
The problem of solving the equations (32) for any arbitrary finite order k has been addressed
 CONTROL OF NONLINEAR SYSTEMS 309

by Huang and Rugh, l6 who have shown that this is possible if the linear approximation of the
controlled plant satisfies

for each w E IR. Note that this is exactly the condition, already encountered in the previous
subsection, that the linear approximation of the controlled plant is right-invertible and has no
zero on the imaginary axis.
A complete set of necessary and sufficient conditions for the solvability of the regulator
equations (32) up to a fixed order k has been established, more recently, by Krener.” This
important work describes a number of useful algorithms for the actual computation of
polynomial approximations of nonlinear feedback laws, including those arising in feedback
linearization and optimal control.

4.4. Robust regulation

A problem of major relevance in the theory of linear control systems is the design of robust
regulators. In the sense of Davison l7 and Francis, l9 robust means that if the parameters which
characterize the model of the plant are perturbed around their nominal values, as long as the
perturbations are such that the closed-loop system remains asymptotically stable, the penalty
variable z ( t ) (which usually characterizes the error in the tracking/rejection of the prescribed
exogenous reference/disturbance input w ( t ) ) still converges to 0 as t+m. There is a simple
structural reason for which a dynamic controller of a single-input single-output linear system
is a robust regulator. As a matter of fact, the asymptotic decay of the error requires the plant
to be driven by a controller which incorporates a copy of the exosystern and, once a copy of
the exosystem is there, no matter what the actual plant parameters are (provided that the
closed-loop system remains stable) the asymptotic decay of the error takes place. The analysis
is conceptually the same (but technically more elaborate) in the multi-input multi-output
systems, where it is known”’19 that the controller should incorporate as many copies of the
exosystem as the number of components of the error.
Unfortunately, in the control of a nonlinear plant, the situation is rather more complicated
With the only exception of the particular case in which the exosystem produces constant inputs
(studied in Reference 18), it is possible to show that the robustness properties of a linear
regulator no longer hold if nonlinear perturbations (in the mathematical model of the plant)
are allowed. In other words, the presence of a copy of the exosystem in the compensator may
no longer be sufficient to guarantee asymptotic decay of the penalty variable in presence of
some unknown nonlinear term in the mathematical model of the plant.

Example. Consider the system

x= - x + p ( x ) + u
z=x-w1

where x , u, z E IR, w E R 2 and p ( x ) is some unknown perturbation term. Let the exosystem be
310 ALBERT0 ISIDORI

The classical theory of linear servomechanisms says that the compensator

El = WE2
Ez= --UEl+Z

u = K&
is a robust controller (if 0 < K < W ) against any linear perturbation, i.e. a perturbation of the
form p ( x ) = (us, with arbitrary a. However, this controller is not robust against nonlinear
perturbations (if, for instance, p ( x ) = ax2, then the error decays to zero only for a = 0).

5 . CONCLUSIONS
The design of feedback laws for systems characterized by complicated nonlinear dynamical
behaviour is a challenging research task which has attracted an increasing interest in recent
years. Among the many problems which must be confronted, a most important one is the
design of feedback laws achieving tracking of prescribed exogenous commands and rejection
of exogenous disturbances. The approaches to the problem of trackinglrejection of exogenous
inputs in a control system depend upon the model chosen to describe the family of exogenous
inputs which are expected to affect the system. There are currently two approaches to the
problem of tracking/attenuation of exogenous inputs in a nonlinear system; one is the
nonlinear extension of the classical servomechanism problem of linear system theory, in which
the task of the regulator is to achieve asymptotic decay of a tracking error (the difference
between the desired behaviour and the actual behaviour of the controlled variables). The other
one is the nonlinear equivalent of the so-called H,-optimal control problem of linear system
theory, in which the task of the regulator is to minimize the maximal amplitude of the
frequency response of the system.
The purpose of the paper was to highlight some recent progresses in the design of feedback
laws in these two research areas. The new design methodologies here summarized, together
with the appropriate computational methods which are also available, can be regarded as a
substantial addition to the control systems designer’s toolbox.

REFERENCES
1. Doyle, J. C., K. Glover, P. P. Kargonekar and B. A. Francis, ‘State-space solutions to standard HZ and H ,
control problems’, IEEE Trans. Aut. Contr., AC-34, 831-846 (1989).
2. Glover, K., and J. C. Doyle, ‘A state space approach to H , optimal control’, in Three Decades of Mathematical
System Theory, H. Nijmejier and H. Schumacher (Eds), LNCIS 135, Springer Verlag, Berlin, 1989, pp. 179-218.
3. Kargonekar, P. P., ‘State-space H , control theory and the LQG problem’, Mathematical Systems Theory, A. C .
Antoulas (Ed.), Springer Verlag, 1991, pp. 159-176.
4. Tadmor, G., ‘Worst case analysis in time domain’, Math. Control Sign. Systems, 3, 301-324 (1990).
5 . Scherer, C., ‘The Riccati inequality and state-space H , optimal control’, PhD Thesis, University of Wuerzburg,
1990.
6. Stoorvogel, A. A,, ‘The H, control problem: a state space approach’, PhD Thesis, Technical University of
Eindhoven, 1990.
7. Basar, T., and P. Bernhard, H,-optimal control and related Minimax problems, Birkhauser, Boston, 1991.
8. Ball, J. A,, and J. W. Helton, ‘H, control for nonlinear plants: connections with differential games’, 28th IEEE
CDC, pp. 956-962, 1989.
9. Van der Schaft, A. J., ‘On a state-space approach to nonlinear H , control’, Syst. and Contr. Lett., 16, 1-8
(1991).
10. Basar, T., and G. J. Olsder, Dynamic Noncooperative Game Theory, Academic Press, London, 1982.
11. Mageirou, E. F., ‘Values and strategies for infinite duration linear quadratic games’, IEEE Trans. Aut. Contr.,
AC-21, 547-550 (1976).
12. Friedman, A., Differential Games, Wiley-Interscience, New York, 1971.
 CONTROL OF NONLINEAR SYSTEMS 31 1

13. Isidori. A,, and C. I. Byrnes, ‘Output regulation of nonlinear systems’, IEEE Trans. Aut. Contr., AC-35,
131-140 (1990).
14. Byrnes, C. I., and A. Isidori, ‘Output regulation of general nonlinear systems’, C.R. Acud. Sci. Paris, 309,
527-530 (1989).
15. Byrnes, C. I., and A. Isidori, ‘Steady-state response, separation principle and the output regulation of nonlinear
systems, 28th IEEE CDC, pp. 2247-2251, 1989.
16. Huang, J., and W. J . Rugh, ‘An approximation method for nonlinear servomechanism problem’, Tech. Rep.
JHU/ECE, 90/08, Johns Hopkins University 1990.
17. Davison, E. J., ‘The robust control of a servomechanism problem for a lilnear time-invariant multivariable
system’, IEEE Trans. Aut. Contr., AC-21, 25-34 (1976).
18. Francis, B. A., and M. W. Wonham, ‘The internal model principle of control theory’, Automaticu, 12, 457-465
(1976).
19. Francis, B. A., ‘The linear multivariable regulator problem’, SIAM J. Contr. Optimiz., 15, 486-505 (1977).
20. Hautus, M., ‘Linear matrix equations with applications to the regulator problem’, in Outils and Modeles
Mathematiques pour I’Automatique..., I. D. Landau (Ed.) CNRS, Paris, 1983, Vol. 3, pp. 399-412.
21. Isodori, A., Nonlineur ControI Systems (2nd edn), Springer-Verlag. Berlin, 1989.
22. Krener, A. J., ‘The construction of optimal linear and nonlinear regulators’, Systems, Models and Feedback, A.
Isidori and T. J. Tarn (Eds), Birkhauser, Boston, 1992, pp. 301-322.