AIAA Guidance, Navigation, and Control Conference AIAA 2009-5627

10 - 13 August 2009, Chicago, Illinois

Nonlinear Stochastic Control Part I: A Moment-

based Approach
Yunjun Xu1
Department of Mechanical, Materials, and Aerospace Engineering
University of Central Florida, Orlando, FL, 32816
Prakash Vedula2
School of Aerospace and Mechanical Engineering, University of Oklahoma, Norman, OK, 73019

This paper describes a new stochastic control methodology for nonlinear affine systems
subject to parametric and functional uncertainties, with random excitations. The primary
objective of this method is to control the statistical nature of the state of a nonlinear system
to designed (attainable) statistical properties (e.g. moments). This methodology involves a
constrained optimization problem for obtaining the undetermined control parameters,
where the norm of the error between the desired and actual stationary moments of
state/output responses is minimized subject to constraints on moments corresponding to a
stationary distribution. To overcome the difficulties in solving the associated Fokker-Planck
equation, generally experienced in nonlinear stochastic control and filtering problems, an
approximation using the direct quadrature method of moments is proposed. In this
innovative approach, the state probability density function is expressed in terms of a finite
collection of Dirac delta functions, with the associated weights and locations determined by
moment equations. The advantages of the proposed method are: (1) robustness with respect
to parametric and functional uncertainties; (2) ability to control any specified stationary
moments of the states/output probability density function; and (3) the state process can be
Non-Gaussian. A numerical simulation is used to demonstrate the capability of the proposed
nonlinear stochastic control method.

I. Introduction

M ost applications of dynamical systems have intrinsic uncertainties such as environmental disturbance, random
noise, and approximation in mathematical modeling. Control of a stochastic dynamical system is a more
challenging problem as compared to its deterministic counterpart. Monte Carlo simulation approaches, which are
commonly used in analysis of statistical dynamic systems, can be used to find proper control parameters, if a desired
statistic of the closed loop system performance is required. However, these become intractable when the system is
large which may result in a prohibitive number of control design iterations (through trial and error) and CPU time.
To reduce the computational cost and provide a systematic design methodology, extensive research efforts were
spent for both linear and nonlinear stochastic systems, as discussed below.
On one hand, the stochastic control for linear systems has been well studied such that desired statistical behavior
of state variables (e.g. mean and variance) [1-6] is approximately ensured. Just to list a few, an LQG type regulator
is formulated by Davis and Vinter in [1], and an observer based covariance control method has been proposed by
Iwasaki [2] for linear systems. Sinha and Miller [3] designed an optimal sliding mode regulator for a linear
stochastic system where the linear dynamics is assumed to perfectly known. Bratus et al. [5] proposed an optimal
control (in the nonlinear form) for linear systems excited by a zero-mean Gaussian white noise forces through the
Hamilton-Jacobi-Bellman equation. Grigoriadis and Skelton [4] designed the minimum energy covariance control
for linear systems with a zero mean white noise process. The shortcomings of the above mentioned linear stochastic
control methods are obvious as (1) one might want to attain a state with a desired mean and covariance instead of a

1
Assistant Professor, The Department of Mechanical, Materials, and Aerospace Engineering, University of
Central Florida, Email: yunjunxu@mail.ucf.edu, AIAA Senior Member
2
Assistant Professor, School of Aerospace and Mechanical Engineering, University of Oklahoma, Email:
pvedula@ou.edu.

1
American Institute of Aeronautics and Astronautics

Copyright © 2009 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.
scalar weight performance index (e.g. quadratic form index) [6] and (2) the approximation as linear system is only
valid in a small region around the trim point.
On the other hand, the nonlinear robust control of stochastic systems is always a challenging task and has
attracted extensive interests. The methods investigated to date can be broadly put into three categories.
The first method used in handling nonlinear stochastic control problems is to linearize the system so that linear
stochastic control methods [7] could be applied. However, this technique suffers from the same drawbacks as LQG
type control and only works in a small region around the linearization points [8].
The second approach to solve nonlinear stochastic control problems is via the solution of the associated Fokker-
Planck Equation once the structure of the closed loop system is known. In order to evaluate the expectation or
covariance, one must have knowledge of the response PDF [9]. However for nonlinear systems, the exact PDF is
typically unknown and just like the case in the FPE and Bayes’ based nonlinear filtering, solving the associated
Fokker-Planck Equation (FPE) [10-14] make this problem cumbersome or unsolvable with few special exceptions
[15] due to the curse of dimensionality. To mitigate the computational cost, methods such as path integral method
[16], cell-mapping method [17] and adaptive grids methods [18-20] have been tried. However, the computational
cost is still high even for low dimension nonlinear control problems.
Many interests have been attracted to the somewhat promising third category, consisting of approximation
methods [21-25], through which the pre-specified response mean and covariance in steady state can be achieved.
The Gaussian Closure method investigated by Sun and Hsu [21] is one of them, in which the first and second order
moments are controlled through the proposed sliding mode control. In this approach, the moments of order higher
than two are approximated in terms of the first and second order moments through the Gaussian Closure method.
However the expectation of nonlinear terms, such as E[ x13 sign( s )] , have to be integrated over a domain big enough
such that majority of the probability can be captured. In the approach proposed by Sobczy and Wojtkiewicz in [8,
22], a maximum entropy method has been applied thus a system of nonlinear equation can be formed for user-
specified response moments. The availability of the response PDF makes the evaluation of higher order response
moments fairly effective, even though the PDF is approximate. In the paper by Chang [23], describing functions
were applied to study the covariance control for a nonlinear system. Kim and Rock [24] used dual properties in
designing a suboptimal stochastic control where the feedback control structure is pre-defined and the mean value of
the performance is controlled. However, the weighting matrix for the error covariance needs to be diagonal and no
equality or inequality constraints can be included. Forbes [25] used Gram-Charlier expansions as the PDF basis
functions and obtained an approximately parameterized response PDF to track the target PDF in steady state,
however, without a guaranteed stability.
In brief, most of the methods mentioned above have one or several of the following limitations: (a) the difficulty
in solving the FPE, (b) some expectations of nonlinear terms cannot be obtained effectively, (c) the form of the
response PDF has to be know a priori (d) it is not easy to handle inequality and equality constraints and (e) stability
is not guaranteed.
In the present work, these limitations for nonlinear stochastic control problems will be addressed based on the
novel quadrature based moment approach (referred as the direct quadrature method of moments - DQMOM), along
with an asymptotically stable nonlinear tracking control. This approach involves representing the state PDF in terms
of a finite summation of Dirac delta functions, whose weights and locations (abscissas) are determined based on
moments constraints. Using a small number of scalars, the method is able to efficiently and accurately model
stochastic processes described by the multidimensional FPE through a set of ordinary differential equations (ODEs).
Together with the DQMOM approach [26, 42], a nonlinear controller is designed here based on the concepts of
sliding manifold and input-output feedback linearization with guaranteed asymptotic tracking stability. Different
from the commonly used sliding mode control (SMC) [27-36], the high speed switching (discontinuous) function
shown in the SMC has been removed to satisfy the continuity requirement in the partial derivatives of the FPE, and
in addition the inherent chattering problem experienced in the commonly used SMC can be eliminated.
The main contributions of the paper can be summarized as follow. First, the controller proposed is robust to
bounded parametric as well functional uncertainties. The existence of a finite moment index implies that the control
system is stable up to the highest order of moments included in the design. Second, selected order moments of the
variables can be controlled accurately in steady state. Third, the state process need not be a Gaussian.
The rest of this paper is organized as follows. First, the system model of an affine nonlinear stochastic system is
described along with the control objectives. Second, a nonlinear robust control method is proposed based on the
concepts of input-output feedback linearization and sliding manifold. Following this, the governing equations of the
weights and abscissas, which are used to represent the state PDF, are derived from the FPE using the proposed
quadrature based moment approach (DQMOM). Next, the undetermined control parameters, weights and abscissas

2
American Institute of Aeronautics and Astronautics
are optimized offline through a constrained nonlinear optimization. Finally, a numerical example is illustrated
followed by the conclusion.

II. The System Model and Control Objectives

Let us consider the following affine nonlinear stochastic differential equation (SDE) with additive noise
m Nw
xi( ni ) = f i ( x1 ,..., xn , t ) +  bij (x1 ,..., xn )u j +  gij ( x1 ,..., xn ) w j (t ), i = 1,..., n (1)
j =1 j =1

and an output model to be

yi = hi ( x1 ,..., xn ), i = 1,..., p (2)
T
where xi =  xi ,..., xi( ni −1)  ∈ ℜni and xi( ni −1)  d ni −1 xi / dt ni −1 are states with up to ni − 1 derivatives.
u = [u1 ,..., um ]T ∈ ℜm is the control input and B = bij ( x1 ,..., xn )  ∈ ℜn× m is the input matrix with linearly
independent vector fields. f = [ f1 ,..., f n ]T ∈ ℜn is the nonlinear state function. The relative degree for the output
y = h = [ y1 ,..., y p ]T ∈ ℜ p is r = [r1 ,..., rp ]T ∈ ℜ p . To avoid numerical errors in the pseudo inverse associated with
the proposed controller, p ≤ m is required. The case when p > m , singular perturbation or multi-time scale
decomposition methods can be used [37]. Furthermore, in this model, w ( t ) ∈ ℜ N w is assumed to be a Weiner
process [10] with a zero-mean and a covariance matrix of Q ( t ) , and G =  gij ( x1 ,..., xn )  ∈ ℜn× Nw is the associated
diffusion matrix.
The control objectives are (1) to stabilize the system while being capable of tracking the desired trajectory
yi , d , i = 1,..., p (the subscript d denotes the desired signal), and (2) to achieve desired stationary moments of the
states/output probability density function (PDF). For example, the desired stationary PDF distribution can be
characterized by (but not limited to) the stationary values of mean, variance, and high order moments.
The nonlinear robust controller will be designed based on the following nominal system
m
xi( ni ) = fˆi ( x1 ,..., xn , t ) +  bˆij (x1 ,..., x n )u j , i = 1,..., n (3)
j =1

with a nominal output to be

yi = hˆi ( x1 ,..., xn ), i = 1,..., p (4)
where ∧ represents the nominal information, fˆ = [ fˆ1 ,..., fˆn ] ∈ ℜ and Bˆ = bˆij ( x1 ,..., xn )  ∈ ℜ
T n n× m
. The parametric
 
uncertainties of the input matrix are bounded by Δij ≤ Dij , i, j = 1,..., p as
+
( I + Δ) =  LB Lrf−1h( x )   LBˆ Lrfˆ−1hˆ ( x )  , Δ ∈ ℜ p× p (5)
 
where I is an identity matrix with the proper dimension. B and Β̂ are assumed to satisfy the matching condition,
i.e. the maximum eigenvalue of the matrix D satisfy λmax ( D) < 1 . “ L ” and “ + ” are used to denote the Lie
derivative and the pseudo inverse, respectively. The error between the nominal and actual state functions is bounded
by F = [ F1 ,..., Fp ]T ∈ ℜ p as
Fi = − Lrif hi + Lrfiˆ hˆi , i = 1,..., p (6)
Note that the bounds of the noise are not included which will be handled by the proposed nonlinear stochastic
control later.

III. Nonlinear Robust Control

In this section, a nonlinear robust controller will be proposed. Unlike a commonly used SMC approach [27-30],
there is no discontinuous function involved. The later fact is preferred by the FPE based approach because of
continuity requirements in the partial derivatives. Let us define the sliding manifold s = [ s1 ,...,s p ]T ∈ ℜ p as
ri − 2
si = λ−1,i  ei dt +  λk ,i ei( k ) + ei( ri −1) , i = 1,..., p (7)
k =0

3
American Institute of Aeronautics and Astronautics
where λk ,i > 0, k = −1,..., ri − 2, i = 1,... p can be any positive number and the error signal is defined to be
ei = yi , d − yi , i = 1,..., p .
For a general nonlinear system (1) with bounded parametric and functional uncertainties (5) and (6), the
proposed MIMO feedback control scheme
+ dr y r −2 
u =  LBˆ Lrfˆ−1hˆ ( x )   r d − Lrfˆ hˆ ( x ) + λ−1 ⋅ e +  λk ⋅ e ( k +1) + k ⋅ s  (8)
 
 dt k =0 
guarantees that the closed-loop system is globally asymptotically stable for tracking desired signal yi , d . Note that
the element-wise multiplication a ⋅ b = [a1b1 ,..., a p bp ]T is applied.
The time varying feedback gain k = [k1 ,..., k p ]T ∈ ℜ p can be uniquely solved from
d r yd r −2
F+D
dt r
− Lr ˆ
ˆ
f
h( x ) + λ− 1 ⋅ e + 
k =0
λk ⋅ e ( k +1) + η ⋅ s = ( I − D)k ⋅ s (9)

for any positive numbers λ and η . In case of si → 0 , the magnitude of ki si ( ς i = ki si ) instead of ki will be
calculated using (9) because the proposed controller (8) only uses ki si . The sign of ki si is determined by si since
ki > 0 . The stability proof of Eq. (8) and Eq. (9) can be found in [40, 41].
Note that the state tracking control y = x is a special of the proposed robust nonlinear control.

IV. Nonlinear Stochastic Control based upon DQMOM

Now, the state equation of the closed loop system SDE (Eqs. 1) can be rewritten as
m Nw
xi( ni ) = fi ( x1 ,..., xn , t ) +  bij (x1 ,..., xn )u j ( x1 ,..., xn , λ ,η ) +  gij ( x1 ,..., xn ) w j (t ), i = 1,..., n (10)
j =1 j =1

which is then converted into the form of the Itô SDE as

  Nw

dxi = fi ( x1 ,..., xn , λ, η,t )dt +  gij ( x1 ,..., xn )d β j (t ), i = 1,..., N s (11)
j =1
n

where w j (t ) ~ d β j (t ) / dt and N s =  ni is the number of states xi in the first order system. Also in this model,
i
   
d β j (t )  (0, dt ) , G = [ gij ] ∈  N s × Nw , and x (t ) = [ xi ] ∈  N s .
If the process described by the SDE is a Markovian diffusion process, the probability density function
characterizing this process is governed by the FPE [10-12] as
  
∂p N s ∂  pf 
1 Ns Ns ∂
2 

p GQG T 
ij 
( )
= −    +     
i
(12)
∂t i =1 ∂xi 2 i =1 j =1 ∂xi ∂x j

where p = p ( x (t ) ) is the state PDF. The first term on the right hand side (RHS) of the FPE is the drift term
 
whereas the second one is the diffusion term and D = 1 / 2GQG T is defined.
However, the central issue associated with solving the FPE is the high computational cost, which explains the
reason of why this type of nonlinear stochastic control and nonlinear filtering has not been widely used. Here a new
quadrature based moment approach is proposed for solving the FPE. This method involves the representation of the
state PDF in terms of a finite summation of Dirac delta functions as
N Ns
  
p ( x (t )) = 
α
wα (t )∏ δ [ x
=1 j =1
j − xj
α
] (13)

where N is the number of nodes, wα = wα (t ) denotes the corresponding weight for node α , α = 1,..., N , and
 
x j = x j (t ), j = 1,..., N s represents the property vector of node α (called “abscissas”). The weights and
α α
abscissas are determined based on the constraints from the evolution of moments. Using a small number of scalars

4
American Institute of Aeronautics and Astronautics
(in the Dirac delta function), this method is able to efficiently and accurately model stochastic processes described
by the multivariable FPE through a set of ordinary differential equations (ODEs) as demonstrated below.

The dynamics of the abscissas x j and weights wα are governed by the following differential algebraic
α
equations
N  Ns  Ns  N Ns Ns
  
  ∏ 
k j −1
∏
kk kk
 1 − kj  xk  aα + k j xj xk b jα = S k1 ,...k N (14)

α =1 
 α  α α s
j =1  k =1  α =1 j =1 k =1, k ≠ j

with the definitions of

dwα / dt  aα , α = 1,..., N (15)
and
d ς jα / dt  b jα , j = 1,..., N s ; α = 1,..., N (16)

where the weighted abscissas ς jα  wα x j is introduced. The k1 , k2 ,..., k N s moment constraint is derived as
α
S k1 ,...k N  S k11 ,...k N + S k21 ,...k N , in which
s s s
Ns N
       

k1 ki −1 ki −1 ki +1 k Ns
Sk11 ,...k N = k wα (t ) i x1 α
⋅⋅⋅ xi −1 α
xi α
xi +1 α
⋅⋅⋅ xN s fi ( x1 α
,... xN s ) (17)
s α α
α
i =1 =1
when i ≠ j ,
Ns Ns N  Ns
    
 ∏
kk
Sk21 ,...k N = wα ki k j  xk  / xi ⋅ xj [ D( x )]ij   (18)
s  α  α α x1 ,... xN s
i =1 j =1 α =1  k =1  α α

whereas when i = j ,
 Ns  k   2N

s
wα ki (ki − 1) 
 
xk αk  / xi α ⋅ [ D( x )]ij 
Sk21 ,...k N =  ∏
x1 α ,... xN s
(19)
α =1  k =1  α

The derivation of Eq. (17) through Eq. (19) is shown in Appendix A. Equation (14) can be rewritten in the
matrix form as
Aμ = s (20)
T
where μ =  aα b jα  , α = 1,..., N ; j = 1,..., N s . Once the abscissas and weights are calculated, any selected
moment of the state PDF can be found from
N Ns

 ∏
k1k2 ...k N s kα
M  wα xj (21)
α
α =1 j =1
where k1 , k2 ,..., k N s are nonnegative integers.
For any selected nonnegative integer k1 , k2 ,..., k N s , the corresponding stationary moments of the PDF is
governed by
S k1 ,...k N = 0 (22)
s

Proof: In steady state, the abscissas and weights of the moments will not change in time; therefore, based on Eq. 14,
the corresponding RHS is zero.

V. Offline Nonlinear Constrained Optimization

One of the control objectives is to find the control parameters λ and η for getting desired stationary moments
by minimizing the p-norm (e.g. 2-norm) of the error between the desired and actual stationary k1 , k2 ,..., k N s
moments as
k1k2 ...k N s k k2 ...k N s
J= M − M d1 (23)
p

5
American Institute of Aeronautics and Astronautics
 
in which the PDF is characterized by the weights wα and abscissas x j . This problem can be formulated as an
α
NLP where the objective function is described as Eq. (23). Equation (22) for selected choice of k1 , k2 ,..., k N s and
N


α
wα = 1
=1
(property of the PDF) are the equality constraints, whereas η > 0 (stability requirement), λ > 0

(stability requirement), and wα > 0 (property of the PDF) are inequality constraints. The parameters to be optimized

are control parameters λ and η , and the weights wα and abscissas x j . Note that due to the flexibility of the
α
NLP approach used here, the performance index can be extended to a more general form including the quadratic
type index.
k k2 ...k N s
The method is summarized and illustrated in Fig. 1. First, the desired stationary moments M d1 are
specified. After that an NLP optimal control algorithm will be used to obtain desired optimal control parameters λ
and η through the DQMOM approach. Finally, the obtained controller parameters will be implemented in the
nonlinear robust control.

Fig. 1 Outline of the stochastic robust control algorithm

VI. A Simple Numerical Example

To demonstrate the effectiveness of the proposed algorithm, a numerical example modified from [33] is shown
below.
x1 = x2 ; x2 = − ax22 cos 3 x1 + u + w (24)
where the mean and variance values of the noise applied here are E[ w] = 0 and E[ w2 ] = σ 2 respectively. The
uncertainty parameter a is bounded by 1 ≤ a ≤ 2 and the nominal value is aˆ = 1.5 . The output is y = x = [ x1 , x2 ]T .
According to Eq. (8), the nonlinear robust controller is designed as
ˆ 22 cos 3 x1 + x2, d − ks − λ ( x2 − x2, d )
u = ax (25)
where the uncertainty bound is: F = 0.5 x 2 cos 3 x . The sliding surface is defined as s = ( x2 − x2, d ) + λ ( x1 − x1, d )
where subscript d denotes the desired values. When s > 0 , ks = F + η s ; when s < 0 , ks = − F + η s ; whereas
ks = 0 when s = 0 . The Itô stochastic differential equation of the system is
 dx1   x2   0 
 dx  =   dt +   , dw  (0, dt ) (26)
σ dw
2
ˆ
 2   −ax2 cos 3 x1 + u 
where the control u (Eq. 26) is a function of states and control parameters λ and η . The corresponding FPE
governing the state PDF is derived as

2 ∂  pf  2 ∂  p ( D( x ) ) 
2
∂p  i
2
+  
ij 
= − (27)
∂t i =1 ∂ xi i =1 j =1 ∂x i ∂x j

6
American Institute of Aeronautics and Astronautics
  
ˆ 22 cos 3x1 + u . In this example, the state PDF is
where D11 = D12 = D21 = 0 , D22 = Qdt , f1 = x2 , and f 2 = − ax
modeled as
N 2
p ( x1 , x2 ) = 
α
wα (t )∏ δ [ x
=1 j =1
j − xj
α
] (28)

and the k1 , k2 moments of the state PDF is derived as

N 2

 ∏
kj
M k1k2  x1ki x2k2 = wα xj (29)
α
α =1 j =1
The objectives of the controller are to (1) track the desired output y1 = 1 and y2 = 0 and (2) achieve a desired
tracking performance statistic distribution under different noise level as shown in Table 1. In this simulation, the
performance index in the NLP optimization is modeled as J = M d − M , where M = [ M 10 , M 01 , M 20 , M 02 ]T . The
MATLAB® function fminsearch is used for the NLP optimization. The moments to be constrained in Eq. (22) are
selected to be k1 = [1, 0, 2,1, 0,3, 2,1, 0, 4,3, 2] and k1 = [0,1, 0,1, 2, 0,1, 2,3, 0,1, 2] for the case when N = 4 . The actual
statistics is achieved through two thousand Monte Carlo runs, and after that the control parameter is found in the
optimization. The results are compared with the case, where arbitrarily selected parameters λ = 1 and η = 1 is used
(without the offline stochastic optimization). To simplify the description, here the proposed stochastic robust
controller is denoted as “method 1”, and the nonlinear robust controller method using arbitrarily selected control
parameters is represented as “method 2”. In Table 1, the subscript d , a , and t denote the desired, actual value
found from “method 1”, and actual value found through “method 2”. The residuals of the performance index (Eq.
23) in optimization for all the simulation cases and the achieved control parameters η a and λa are listed as well.
The small value of 10−6 indicates the converged solution is obtained in the optimization.
Since similar conclusions can be drawn from all the four cases shown in Table 1, only the figures found from
Case 1 and Case 4 are illustrated.
The mean values of the position ( x1 ) and velocity states ( x2 ) are successfully controlled to the desired values in
both methods as shown in Figs. 2 and 3. Also as shown in Table 1, the desired stationary mean value needs to be
unity for the position and zero for the velocity. The values achieved by method 1 are one for the position and zero
for the velocity, whereas those values from method 2 are 0.99 and zero. Therefore, in terms of mean value control,
there are no significant difference between method 2 and the proposed method 1. The same conclusion can be made
for other cases. For example, in Figs. 6 and 7 (case 4 in Table 1), the mean values of the position and velocity found
through both methods are one and zero respectively, which satisfy the requirement.
The significance and advantage of method 1 can be easily seen from Figs. 4, 5, 8 and 9. Method 1 has
successfully controlled the moments M a20 and M a02 to the desired value 1.01 and 0.105 in case 1. However in
method 2, the actual stationary moments M a20 and M a02 are 1.4 and 0.45, which is quite different from the desired
statistic. In case 4, the desired moments are M d20 = 1.2 and M d02 = 0.5 . Through method 1, the accomplished
stationary moments are 1.19 and 0.5 whereas in method 2, the corresponding values are 1.86 and 0.88.
1.4
1.04
1.2 Method 1
Mean Value of the Position

Mean Value of the Position

Method 1
1.02
1

0.8 1
Method 2
0.6 0.98 Method 2
0.4 0.96
0.2
0.94
0
0 10 20 30 40 10 15 20 25 30 35
Time (s) Time (s)
Fig. 2(a) Mean value of the position (case 1) Fig. 2(b) Stationary mean value of the position (case 1)

7
American Institute of Aeronautics and Astronautics
 0.8
0.06
Method 1
Mean Value of the Velocity

Mean Value of the Velocity

0.6
0.04

0.4 0.02

0
0.2 Method 2
-0.02
0 -0.04
Method 2
Method 1 -0.06
-0.2
0 10 20 30 40 10 20 30 40
Time (s) Time (s)
Fig. 3(a) Mean value of the velocity (case 1) Fig. 3(b) Stationary mean value of the velocity (case 1)
1.5 0.7
Method 2
0.6
Method 2
0.5
1
Moment 20

Moment 02
0.4
Method 1
0.3
0.5 Method 1
0.2

0.1

0 0
0 10 20 30 40 0 10 20 30 40
Time (s) Time (s)
Fig. 4 M a20 and M t20 of the output PDF (case 1) Fig. 5 M a02 and M t02 of the output PDF (case 1)

Table 1. Stochastic Control Scenario and Results

Cases, Noise M 10 M 01 M 20 M 02
10
M d M a10 M10
t M 01
d M a01 M a01 M 20
d M a20 M t20 M02
d M a02 M t02
1, Q = 1 1.0 1.0 0.99 0 0 0 1.01 1.01 1.4 0.1 0.105 0.45
2, Q = 1 1.0 1.0 0.99 0 0 0 1.02 1.05 1.4 0.2 0.21 0.44
3, Q = 2 1.0 1.0 1.0 0 0 0 1.02 1.02 1.77 0.2 0.22 0.84
4, Q = 2 1.0 1.0 1.0 0 0 0 1.2 1.19 1.86 0.5 0.5 0.88
Cases, Noise 1, Q = 1 2, Q = 1 3, Q = 2 4, Q = 2
Residual of J 10−6
10 −6
10 −6
10−6
λa = 1 ηa =8.817704 ηa =3.721468 ηa =8.237894 ηa =2.696682

1.4
1.06 Method 1
1.2 Method 1
Mean Value of the Position

Mean Value of the Position

1.04
1
1.02
0.8
Method 2 1
0.6
0.98 Method 2
0.4 0.96
0.2 0.94
0
0 10 20 30 40 10 20 30 40
Time (s) Time (s)

8
American Institute of Aeronautics and Astronautics
 Fig. 6(a) Mean value of the position (case 4) Fig. 6(b) Stationary mean value of the position (case 4)
0.6
0.06 Method 1
0.5
Mean Value of the Velocity

Mean Value of the Velocity

0.04
0.4 Method 1
0.02
0.3 Method 2
0
0.2
-0.02
0.1 -0.04
0 -0.06 Method 2
-0.1
0 10 20 30 40 10 20 30 40
Time (s) Time (s)
Fig. 7(a) Mean value of the velocity (case 4) Fig. 7(b) Stationary mean value of the velocity (case 4)
2 1

0.8
1.5
Method 2
Method 2
Moment 20

Moment 02
0.6
1
0.4
Method 1 Method 1
0.5
0.2

0 0
0 10 20 30 40 0 10 20 30 40
Time (s) Time (s)
Fig. 8 M a20 and M t20 of the output PDF (case 4) Fig. 9 M a02 and M t02 of the output PDF (case 4)

VII. Conclusion
This paper describes an innovative robust stochastic control methodology for nonlinear affine systems subject to
parametric and functional uncertainties with random excitations. The method is of interest because the process is
not necessary to be Gaussian and any specified set of (attainable) stationary moments can be precisely controlled.
Two particular attributes of the methodology are: (1) the deterministic part of the controller is robust with respect to
parametric and functional uncertainties without discontinuous functions involved; and (2) the state PDF is expressed
in terms of a finite collection of Dirac delta functions, and the associated weights and locations are governed by
moment constraints. In this approach, the associated Fokker-Planck equation need not be solved. The advantages of
the method are (1) ability to control the distribution of any specified stationary moments of the states/output
probability density function (PDF), and (2) robustness with respect to parametric and functional uncertainties. Our
numerical simulations have successfully demonstrated the capability of the proposed nonlinear stochastic control
method.

Appendix A – Moments Equation in DQMOM

To show the derivation, the FPE of the state PDF (Eq. 12) is repeated here as
  
∂p N s ∂  pf 
1 Ns Ns ∂
2 

p GQG T 
ij 
( )
= −    +     
i
(A-1)
∂t i =1 ∂xi 2 i =1 j =1 ∂xi ∂x j
The DQMOM method was initially developed by Marchisio and Fox [39] to address the population balance
problems in multiphase flows [39]. Based on this method, we developed a novel extension of this method with
applications in nonlinear estimation and controls. In our approach, the state PDF is written as a summation of a
multi-dimensional Dirac delta function

9
American Institute of Aeronautics and Astronautics
 N Ns
  
p ( x (t )) = 
α =1
wα (t ) ∏δ [x
j =1
j − xj
α
(t )] (A-2)

Substituting Eq. (A-2) into Eq. (A-1), the left hand side (LHS) of Eq. (A-1) becomes

∂p
N
 ∂wα 
Ns
 
N Ns Ns
  ∂δ jα ∂ xj
= 

∂t α =1  ∂t 
 ∏
j =1
δ [x j − x j
α
]− 
α
wα  ∏
=1 j =1 k =1, k ≠ j
δ [ xk − xk α
] 
∂ xj ∂t
α

α (A-3)

N Ns
 ∂wα 
N Ns Ns ∂ xj
= 
α
∏ δ α 
=1 j =1
j
∂t
−
 α =1
 ∏ j =1 k =1, k ≠ j
wα δ kα δ 'jα
∂t
α

  
where δ jα  δ [x j − x j ] and δ 'jα
 ∂δ jα / ∂ xj .
α α

If the weighted abscissas ς jα  wα x j is introduced, after some manipulations, Eq. (A-3) can be written as
α

∂p 
N Ns
 ∂wα 
Ns Ns
 ∂wα 
N Ns Ns
∂ς jα
= ∏

∂t α =1 
δ jα 
 ∂t
+

∏ xj
α
δ kα δ 'jα −
∂t  α =1  ∏ δ kα δ 'jα
∂t
(A-4)
 j =1 j =1 k =1, k ≠ j  j =1 k =1, k ≠ j

Notice that wα , ς jα , and δ jα are functions of time, thus the partial derivatives of the functions can be written as
total derivatives.
∂p N Ns
 dwα 
Ns Ns
 dwα  N
Ns Ns
d ς jα
= ∏

∂t α =1 
δ jα 
 dt
+

∏ xj
α
δ kα δ 'jα −
dt  α =1  ∏ δ kα δ 'jα
dt
(A-5)
 j =1 j =1 k =1, k ≠ j  j =1 k =1, k ≠ j

With the definitions

dwα / dt  aα , α = 1,..., N (A-6)
and
d ς jα / dt  b jα , j = 1,..., N s ; α = 1,..., N (A-7)
Eq. (A-5) (LHS of Eq. (A-1)) can be further simplified as
N  Ns Ns Ns  N  Ns Ns 
∂p 
= 
∂t α =1  j =1 ∏
δ jα +
α∏
x j δ kα δ 'jα aα −


α =1 
δ kα δ 'jα  b jα

∏ (A-8)
 j =1 k =1, k ≠ j   j =1 k =1,k ≠ j 
The RHS of Eq. (A-1) is now given by the expression
Ns  Ns Ns 2  
 ∂pfi ∂ [1 / 2 p (GQG T )ij ]
Sx ( x ) = −
i =1
 +
∂xi i =1 j =1
  
∂xi ∂x j  (A-9)

The FPE can be written in terms of the multi-variable Dirac delta function as
N  Ns  N Ns Ns N  Ns Ns 

∏ 

α =1  j =1
δ jα aα +
  ∏ x j δ kα δ 'jα aα −
α

 
∏
δ kα δ 'jα b jα = S x ( x ) (A-10)
 α =1 j =1 k =1, k ≠ j α =1  k =1, k ≠ j
j =1 
There are total N (1 + N s ) parameters which need to be found in order to construct the conditional PDF

p ( x ( t ) ) : aα and b jα , j = 1,..., N s , α = 1,..., N . The DQMOM method applies an independent set of user defined
moment constraints to construct N (1 + N s ) algebraic ordinary differential equations (ODEs).
Given the following three Dirac delta function properties
+∞      k
x k δ ( x − x α )dx = x α  −∞
(A-11)
+∞     

k −1
xkδ ' (x − x α
)dx = −k x α
(A-12)
−∞
and
+∞     

k −2
xkδ " (x − x α
)dx = k (k − 1) x α
(A-13)
−∞
The k1 , k2 ,..., k N s moments of Eq. (A-10) can be written as followed

10
American Institute of Aeronautics and Astronautics
 k   k  
N Ns Ns N Ns Ns Ns
+∞ +∞   +∞ +∞   
−∞
...  −∞
x1k1 ⋅⋅⋅ xNN s 
s  ∏ δ jα aα 
 ∏ dxl +  −∞
...  −∞
x1k1 ⋅⋅⋅ xNN s 
s   ∏ xj
α
δ kα δ 'jα aα 
 ∏ dx l
 α =1 j =1  l =1  α =1 j =1 k =1, k ≠ j  l =1
(A-14)
+∞ +∞  N  Ns Ns   Ns
+∞ +∞
Ns
 k   k  
− 
−∞
...  −∞
x1k1 ⋅⋅⋅ xNNs
s

∏
 α =1  j =1 k =1, k ≠ j
δ kα δ 'jα  b jα


 ∏ dx =  l
−∞ 
...
−∞
x1k1 ⋅⋅⋅ xNNs [ Sx ( x ) ]
s ∏ dx l
    l =1 l =1

After rearranging and simplifying Eq. (A-14), the N (1 + N s ) unknown parameters can be found in
N  Ns  Ns  N Ns Ns
  
  ∏ 
k j −1
∏
kk kk
 1 − kj  xk  aα + k j xj xk b jα = S k1 ,...k N (A-15)

α =1 
 α  α α s
j =1  k =1  α =1 j =1 k =1, k ≠ j

where Sk1 ,...k N  Sk11 ,...k N + Sk21 ,...k N . The first term in the LHS of Eq. (A-15) can be simplified as
s s s

k     
N Ns N Ns
 +∞ +∞  
∏ ∏
kj

−∞ −∞
x1k1 ⋅⋅⋅ xNNs 
s 
 α =1
 ...  j =1
δ jα aα  dx1...dxN s =




α =1  j =1
xj
α
 aα


(A-16)

whereas the second term in the LHS of Eq. (A-15) is

     
Ns N Ns Ns N Ns Ns
+∞ +∞   
  ∏  ∏  ∏
kk
... xmkm  xj δ kα δ 'jα aα dx1...dxN s = −  kj xk aα (A-17)
−∞ −∞  α =1 α  
α =1 
α 
m =1  j =1 k =1, k ≠ j  j =1 k =1 
In the same way, the third term of the LHS in Eq. (A-15) is
N Ns +∞  k  Ns   
N Ns

Ns

  
k j −1
∏ ∏
kk
− b jα x1k1 ⋅⋅⋅ xNNs δ 'jα  δ  dx ...dx = kj xj xk b jα (A-18)
−∞ s  k =1, k ≠ j kα  1 Ns α α
α =1 j =1   α =1 j =1 k =1, k ≠ j

The k1 ,..., k N s moments of the RHS of Eq. (A-15) are then derived to be

 k1  k N s  ∂pfi  
+∞ +∞ n ∞
 k    
Sk1 ,...k N 
s  −∞
... −∞
x1k1 ⋅⋅⋅ xNNs Sx ( x )dx1...dxN s = −
s  i =1
−∞
x1 ⋅⋅⋅ xN   dx1...dxN s
s 
 ∂xi 

 T (A-19)
∞  k  Ns Ns
1 ∂ [ p (GQG )ij  
2

+  −∞
x1k1 ⋅⋅⋅ xNNs
s
 
 i =1 j =1
2
 
∂xi ∂x j
dx1...dxN = Sk1 ,...k + Sk2 ,...k
 s 1 Ns 1 Ns

where
Ns  Ns
 k1  k Ns  ∂pfi  
∞ N
    
 
k1 ki −1 ki −1 ki +1
Sk11 ,...k N =− x1 ⋅⋅⋅ xN   dx1...dxN s = k wα (t ) x1 ⋅⋅⋅ xi −1 xi xi +1 ⋅⋅⋅
s   i α α α α
s
i =1
−∞ ∂x
 i  α
i =1 =1 (A-20)
 k Ns   
xN s fi ( x1 α ,... xN s )
α α
When i ≠ j ,
 T
k  s 1 ∂ [ p(GQG )ij  
N Ns 2
 ∞ 
Sk21 ,...k N
s
=
−∞ 
x1k1 ⋅⋅⋅ xNNs 
s
 i =1
 j =1
2
 
∂xi ∂x j
dx1...dxN
 s

(A-21)
Ns Ns N  Ns
   
 ∏
kk
= wα ki k j  xk  / xi ⋅ xj [ D( x )]ij  
 α  α α x1 ,... xN s
i =1 j =1 α =1  k =1  α α

whereas when i = j ,
2 
∞   k ∂ f [ D( x )]ii  
Sk21 ,...k N =
s  −∞
x1k1 ⋅⋅⋅ xNNs
s 
∂xi2
dx1...dxN s
(A-22)
N  Ns
   
 ∏
kk 2
= wα ki (ki − 1)  xk  / xi [ D( x )]ij  
 α  α x1 ,... xN s
α =1  k =1  α α

11
American Institute of Aeronautics and Astronautics
Thus, the N (1 + N s ) ODEs can be constructed using a set of independent moment constraints k1 ,..., k N s as shown in
Eq. (20).

References
1
Davis, M. H. A., and Vinter, R. B., Stochastic Modeling and Control, Chapman and Hall, London, 1985.
2
Iwasaki, T., and Skelton, R. E., “On the observer-based structure of covariance controllers,” System Control Letter, 22(1),
1994, 17-25.
3
Sinha, A., and Miller, D. W., “Optimal sliding-mode control of a flexible spacecraft under stochastic disturbances,” Journal
of Guidance, Control, and Dynamics, 18(3), 1995, 486-492.
4
Grigoriadis, K. M. and Skelton, R. E., “Minimum-energy covariance controllers,” Automatica, 33, 1997, 569-578.
5
Bratus, A., Dimentberg, M., and Iourtchenko, D., “Optimal bounded response control for a second-order system under a
white-noise excitation,” Journal of Vibration and Control, 6(5), 2000, 741–755.
6
Hotz, A., and Skelton, R. E., “A covariance control theory,” International Journal of Control, 46(1), 1987, 13-32.
7
Young, G. E., and Chang, R. J., “Optimal control of stochastic parametrically and externally excited nonlinear control
systems,” ASME Journal of Dynamical System, 110, 1988, 114-119.
8
Wojtkiewicz, S. F., and Bergman, L. A., “A moment specification algorithm for control of nonlinear system driven by
Gaussian white noise,” Nonlinear Dynamics, 24(1), 2001, 17-30.
9
Elbeyli, O., and Sun, J. Q., “Covariance control of nonlinear dynamic systems via exact stationary probability density
function,” Journal of Vibration and Acoustic, 126(1), 2004, 71-76.
10
Jazwinski, A., Stochastic Process and Filtering Theory, New York, NY, Academic Press, 2007 (re-publication of the
version of 1970) pp. 72-74.
11
Fokker, A., Annalen der Physik, 43(810), 1940.
12
Planck, M., Sitzungsber. Preuss. Akad. Wissens., 1917, 324.
13
Zhou, Y. and Chirikjian, G. S., “Probabilistic models of dead-reckoning error in nonholonomic mobile robots,” Proceedings
of IEEE Int.Conference on Robotics and Automation, 2, 2003, 1594-1599.
14
Spencer Jr., B. F., and Bergman, L. A., “On the numerical solution of the Fokker equations for nonlinear stochastic
systems,” Nonlinear Dynamics, 4, 1993, 357–372.
15
Daum, F., “Nonlinear filters: beyond the Kalman filter,” IEEE Transaction on Aerospace and Electronic Systems Magazine,
20(8), Part 2, 2005, 57-69.
16
Naess, A., and Johnson, J. M., “Response statistics of nonlinear dynamics systems by path integration,” IUTAM Symposium
on Nonlinear Stochastic Mechanics, 1992, 401–414.
17
Sun, J. Q., and Hsu, C. S., “The generalized cell mapping method in nonlinear random vibration based upon short-time
Gaussian approximation,” ASME Journal of Applied Mechanic, 57, 1990, 1018–1025.
18
Challa, S., and Bar-Shalom, Y., “Nonlinear filter design using Fokker-Planck-Kolmogorov probability density evolutions,”
IEEE Transaction on Aerospace Electronics System, 36(1), 2000, 309-315.
19
Musick, S., Greenswald, J., Kreucher, C., and Kastella, K., “Comparison of particle method and finite difference nonlinear
filters for low SNR target tracking,” The 2001 Defense Applications of Signal Processing Workshop, 2001.
20
Yoon, J., and Xu, Y., “Relative position estimation using Fokker-Planck and Bayes’ equations,” 2007 AIAA Guidance,
Control, and Dynamics Conference, August 20-23, Hilton Head, South Carolina, 2007.
21
Sun, J. Q., and Hsu, C. S., “Cumulant-neglect closure method for asymmetric nonlinear systems driven by Gaussian white
noise,” Journal of Sound and Vibration, 135(2), 1989, 338-345.
22
Sobczy, K., and Trebicki, J., “Maximum entropy principle in stochastic dynamics,” Probabilistic Engineering Mechanics,
3(5), 1990, 102-110.
23
Chang, K. Y., Wang, W. J., and Chang, W. J., “Constrained control for stochastic multivariable systems with hysteresis
nonlinearity,” International Journal of Systems Science, 28(7), 1997, 731-736.
24
Kim, J., and Rock, S., “Stochastic feedback controller design considering the dual effect,” AIAA Guidance, Navigation, and
Control Conference and Exhibit, Keystone, Colorado, Aug. 21-24 2006.
25
Forbes, M. G., Forbes, J. F., and Guay, M., “Regulatory control design for stochastic processes: shaping the probability
density function,” American Control Conference, Denver, Colorado, June 4-6, 2003, 3998-4003.
26
Attar, P., and Vedula P., “Direct quadrature method of moments solution of the Fokker–Planck equation,” Journal of Sound
and Vibration, 317, 2008, 265-272.
27
Utkin, V. I., “Variable structure systems with sliding modes,” IEEE Transaction on Automatic Control, 22(2), 1977, 212-
222
28
Bartolini, G., Punta, E., and Zolezzi, T., “Simplex methods for nonlinear uncertain sliding mode control,” IEEE Transaction
on Automatic Control, 49(6), 2004, 922-933.
29
Hung, J. Y., Gao, W., and Hung, J. C., “Variable structure control: a survey,” IEEE Transaction on Industrial Electronics,
40(1), 1993, 2-22.
30
Bartolini, G., Ferrara, A., Usai, E., and Utkin, V. I., “On multi-input chattering-free second order sliding mode control,”
IEEE Transaction on Automatic Control, 45(9), 2000, 1711-1717
31
Perruquetti, W., and Barbot, J. P., Sliding Mode Control in Engineering, New York: Marcel Dekker, Inc., 2002, pp. 12-14.

12
American Institute of Aeronautics and Astronautics
 32
Lee, J. G., Park, C. G., and Park, H. W., “Sliding-mode controller design for spacecraft attitude tracking maneuvers,” IEEE
Transaction on Aerospace and Electronic System, 29(4), 1993, 1328-1333.
33
Slotine, J. E., and Li, W., Applied Nonlinear Control, New Jersey: Prentice Hall, 1990, pp. 267-307.
34
Buffington, J. M., and Shtessel, Y. B., “Saturation protection for feedback linearizable systems using sliding mode theory,”
American Control Conference, Philadelphia, PA, 1998, 1028-1032.
35
Phuah, J., Lu, J., and Yahagi, T., “Chattering free sliding mode control in magnetic levitation system,” IEEJ Transactions
on Electronics, Information and Systems, 125(4), 2005, 600-606.
36
Yao, B., and Tomizuka, M., “Smooth robust adaptive sliding mode control of manipulators with guaranteed transient
performance,” American Control Conference, Baltimore, MD, 1994, 1176-1180.
37
Kokotovic, P., Khalil, H. K., and O’Reilly, J., “Singular perturbation methods in control, analysis and design,” Society for
Industrial and Applied Mathematics, Philadelphia, 1999, 1-45.
38
LaSalle, J. P., “Some extensions of Lyapunov’s second method,” IRE Transaction on Circuit Theory, CT-7, 1960, 520-527.
39
Marchisio, D. L., and Fox, R. O., “Solution of population balance equations using the direct quadrature method of
moments,” Journal Aerosol Science, 36, 2005, 43-73.
40
Xu, Y. “Chattering free robust control for nonlinear systems,” IEEE Transactions on Control Systems Technology, 16(6),
2008, pp. 1352-1359.
41
Hopkins, R., and Xu, Y., “Position tracking control for a simulated miniature helicopter,” 2008 AIAA Guidance, Navigation,
and Control Conference and Exhibit, Honolulu, Hawaii, August 18-21, 2008.
42
Yunjun Xu and Prakash Vedula, “A Moment-based Approach for Nonlinear Stochastic Tracking Control,” Submitted for
publication in Nonlinear Dynamics.

13
American Institute of Aeronautics and Astronautics