Professional Documents
Culture Documents
On The Time-Varying Riccati Difference Equation of
On The Time-Varying Riccati Difference Equation of
CONTROL AND OPTIMIZATION (C) 1992 Society for Industrial and Applied Mathematics
Vol. 30, No. 6, pp. 1251-1269, November 1992 001
OPTIMAL FILTERING*
GIUSEPPE DE NICOLAOt
Abstract. This paper studies the time-varying Riccati difference equation (RDE) for the filtering problem.
In particular, existence, stabilizability, and attractiveness properties of the real symmetric solutions that
remain bounded on (-o, +oo) (infinite-time solutions) are investigated. Under the assumption of uniform
detectability, conditions for the existence of the maximal and stabilizing solutions are given. Analogous
results are worked out for the minimal and antistabilizing solutions by making reference to the uniform
antidetectability notion. Moreover, it is zhown that, under uniform observability, the set of all symmetric
infinite-time solutions constitute an infinite number of lattices with common minimal and maximal elements.
Key words. Riccati difference equation, linear time-varying systems, Kalman filtering, optimal control,
stabilizability and detectability
1. Introduction. Since the early 1960s, it has become clear that the matrix Riccati
equation is the keystone of a number of filtering and control problems. Thirty years
later, due to the effort of a multitude of authors, the theory of the time-invariant Riccati
equation, although yet in progress, appears rich and consistent. It would be beyond
the scope of the present Introduction to mention all the relevant contributions, and
the following summary does not advance any claim of completeness. To make life
easy, we will only refer to the Riccati equation for the filtering problem, taking for
granted that the results extend by duality to the optimal control Riccati equation.
In the pioneering works of Kalman 1] and Bucy [2], the algebraic Riccati equation
(ARE) was investigated under the assumptions of controllability and observability to
derive the uniqueness of the symmetric positive semidefinite (SPS) solution, the stability
of the closed loop system, and the asymptotic convergence properties. The relaxation
of these hypotheses to the weaker ones of stabilizability and detectability is due to the
works of Wonham [3] and Kucera [4], in continuous time, and Caines and Mayne I-5]
in discrete time. Starting from the early 1970s, there was a growing interest in the study
of the nonstabilizable case. We can mention the contributions of J. C. Willems [6],
Martensson [7], Kucera [8], Molinari [9], and, more recently, Callier and J. L. Willems
[10], Chan, Goodwin, and Sin [11], and De Souza, Gevers, and Goodwin [12]. In the
nonstabilizable case, the ARE admits more than one SPS solution, and the problem
of the classification of the solutions arises. It has been shown that under detectability
assumptions, a maximal solution exists, and conditions for this solution to be stabilizing
have been provided. The convergence of the SPS time-varying solutions of the time-
invariant differential or difference Riccati equation toward the SPS solutions of the
ARE was also studied. The classification of all the real symmetric solutions (positive,
negative, or even nondefinite) of the ARE calls for the works of J. C. Willems [6],
Coppel [13], Callier and J. L. Willems [10], and Shayman [14]. A major result was
the characterization of all the real symmetric solutions as a distributive lattice having
Received by "the editors May 9, 1990; accepted for publication (in revised form) April 12, 1991.
Research for this paper was supported by the Centro di Teoria dei Sistemi of the Italian National Research
Council (CNR) and by the Ministry of University and Scientific and Technological Research (MURST)m
Project "Model Identification, Systems Control, Signal Processing."
"
Centro Teoria dei Sistemi--Consiglio Nazionale delle Ricerche, c/o Dipartimento di Elettronica,
Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, Italy.
1251
1252 GIUSEPPE DE NICOLAO
the maximal and minimal solution as extremal elements. When we turn to the time-
varying Riccati equation, much of the richness of the time-invariant case is lost. By
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
suitably defining the notions of uniform reachability and observability, Kalman [1]
and Bucy [2] (Deyst and Price [15] for the discrete-time case) showed that their main
results also hold true in the time-varying case. Indeed, they proved that, under these
assumptions, there is a unique SPS "moving equilibrium" and that the closed-loop
system is asymptotically stable for any solution, independently of the (SPS) initial
condition. An attempt to extend the RDE analysis with stabilizability and detectability
to the time-varying discrete-time case was made by Hager and Horowitz [16] and
Anderson and Moore [17]. The major contributions in [17] were (a) the exploration
of equivalent definitions of uniform stabilizability and detectability; (b) the demonstra-
tion of a time-varying version of the Lyapunov lemma under stabilizability assumptions;
and (c) the exponential stability of the closed-loop system under stabilizability and
detectability. Only recently, in the context of continuous-time infinite-dimensional
systems, Da Prato and Ichikawa [18] have established some existence and convergence
results for the SPS solutions of the time-varying Riccati equation. In particular, a
necessary and sufficient condition for the existence of a bounded SPS solution has
been proved together with some results relative to the stabilizing and maximal solutions.
However, to the author’s knowledge, the study of the negative semidefinite solutions,
together with the notions of minimal and antistabilizing solution and the classification
of all the symmetric solutions, has remained an unexplored region, at least for the
most general time-varying case. Only for the class of periodically time-varying systems,
the theory of the periodic Riccati equation appears almost as complete as its stationary
counterpart; see, e.g., [19]-[22].
The purpose of this paper is to fill some of the gaps between the theory of the
stationary RDE and the theory of the time-varying RDE. The analysis will be carried
out only in discrete time, but a derivation of the analogous continuous time results
should, in principle, be possible. The attention will be focused on the solutions
remaining bounded on (-oo, +oo), which will be termed "infinite-time" solutions.
Conversely, the solutions that are obtained starting with a given initial condition will
be called "finite-time solutions." The symmetric infinite-time solutions will be shown
to enjoy many of-the properties that in the stationary case characterize the constant
solutions of the RDE. In particular, uniform detectability and stabilizability guarantee
the existence of a unique SPS infinite-time solution, which is attractive for all the SPS
finite-time solutions. When the stabilizability assumption is removed, the uniqueness
falls. However, it can be proved that a maximal infinite-time solution exists and is
attractive for a certain set of finite-time solutions. By means of a device that allows us
to reduce the study of the symmetric negative semidefinite (SNS) solutions of the RDE
to the study of the SPS solutions of a suitably modified RDE, the SNS solutions are
explored, also. In this context, the notions of antistabilizability, antidetectability, SNS,
antistabilizing, and minimal solution take the place of stabilizability, detectability,
SPS, stabilizing, and maximal solution. Finally, a classification of all the symmetric
solutions is provided. It is shown that, under certain assumptions, the set of all
symmetric infinite-time solutions can be grouped in an infinite number of isomorphic
distributive lattice that share the same maximal and minimal elements (the maximal
and the minimal solution, respectively).
The layout of the paper is as follows. In 2 some preliminary definitions are given
and the notion of "infinite-time" solution is introduced. Sections 3 and 4 are devoted
to the SPS and SNS solutions, respectively. In 5 the results of the previous sections
are put together to give a comprehensive picture of all the symmetric solutions and
TIME-VARYING RICCATI EQUATION 1253
the lattice structure is proved. Finally, the Appendix includes a number of auxiliary
lemmas that are employed throughout the paper.
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
2. Preliminaries.
2.1. Structural properties. Consider the following discrete-time, time-varying,
linear system:
there exists a bounded matrix function K(.) such that A(.)+B(.)K(.) [A(.)+
K (.) C (.) is exponentially antistable.
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
2.2. Lyapunov and Riccati equations. It is well known that the state covariance
matrix Var Ix(t)] obeys the following Lyapunov difference equation (LDE):
solution of the LDE. If the uniqueness of the solution is to be preserved, the correct
statement would be as follows:
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
Suppose that (A(.), B (.)) is uniformly stabilizable and A(. and B(.
are bounded. Then, if there exists a reversed finite-time SPS solution X(. of
the LDE (2) on (-oo, to], A(. is exponentially stable on (-oo, to]. Conversely,
if A(. is exponentially stable on (-oo, to], there exists a unique reversed
finite-time SPS solution X(. of the LDE on (-oo, to].
Then, by taking the limit for to--> oo, Lemma 1 is obtained.
Finally, note that, if A(. is exponentially stable, the infinite-time solution of the
LDE proves to be a moving equilibrium for the set of all SPS finite-time solutions.
3. Maximal and nonnegative solutions. The present section will be devoted to the
analysis of the nonnegative definite solutions of the RDE under the hypotheses of
stabilizability and detectability, as well as detectability alone. We begin with two basic
definitions.
DEFINITION 8 (maximal solution). A symmetric infinite-time solution X/(.) of
the RDE is said to be maximal if, for any symmetric infinite-time solution X(.) of
the RDE, X/( t) X( t) >- O, for all t.
DEFINITION 9 (stabilizing solution). A stabilizing solution is an infinite-time
symmetric solution X(. of the RDE such that the corresponding closed-loop matrix
F(. is exponentially stable on (-oo, /oo). A finite-time stabilizing solution is a sym-
metric finite-time solution X(. of the RDE on [to, /oo) such that the corresponding
closed-loop matrix F(. is exponentially stable on [to, /oo).
Below, a sufficient condition for the existence of the maximal solution will be
proved. Such a condition can be seen as an extension of a time-invariant result [7],
[8]. However, differently from [7] and [8], herein the demonstration calls for a
quasi-linearization (Newton) technique. The application of the Newton algorithm to
the ARE traces back to [26], [27] for the continuous-time case, and [5], [28] for the
discrete-time one. In all these papers, controllability and observability assumptions
(sometimes relaxed to stabilizability and detectability) were made, so as to ensure the
existence of a unique SPS solution of the ARE. In [6] it was pointed out that, under
the sole hypothesis of observability, the Newton algorithm converges to the maximal
(minimal) solution of the continuous-time ARE whenever it is started with a stabilizing
(antistabilizing) gain. As for the time-varying case, we refer to [29], where, under the
hypotheses of uniform reachability and uniform observability that guarantee the
existence of a unique SPS moving equilibrium, the convergence of the quasi-lineariz-
ation technique to such an equilibrium is proved. Finally, in a recent paper on the
periodically time-varying difference Riccati equation [22], the sole-detectability has
been shown to be a sufficient condition for the convergence of the Newton method to
the periodic maximal solution. We generalize this result to the time-varying case by
means of the following result.
THEOREM 1. Let (A(.), C(’)) be uniformly detectable and consider the sequence
of LDEs
(4) Xi+(t+l):Ai(t)Xi+l(t)Ai(t)’+Q(t)+Ki(t)R(t)Ki(t)’, i>-O,
where
The stabilizing property follows directly from Theorem 2. Any SPS solution
Proof.
being stabilizing, Proposition 1 implies that the SPS solution is unique and maximal.
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
We now prove that the solution X/( is a moving equilibrium. Indeed, let X(. be
any SPS finite-time solution of the RDE. It can be seen that
X+(t + 1) X(t + 1) F+( t)[X+(t) X( t)]F( t)’,
where F/(.) and F(.) are the closed-loop matrices relative to X/(.) and X(.),
-
-A(t)X(t)C(t)’[C(t)X(t)C(t)’+ R(t)]-lc(t)Xi(t)A(t) ’,
with Xi( to) Xi, o, 1, 2. Then, Xl,o>_- X2, 0 and Ql(t) >= Qe( t), >-_ to, imply that X( t) >-
Xe(t), to
Proof. The proof is completely analogous to the proof given in [30] for the
time-invariant version of the same lemma, and is therefore omitted. Note that the proof
in [30] relies on a result originally proved by Nishimura [31] in a time-varying context.
THEOREM 4. Assume that (A( ), C( )) is uniformly detectable and let X+( denote
the maximal solution. Then, if X(. is a finite-time solution of the RDE with initial
condition X(to) Xo>= X+(to), limt_ooX(t)-X+(t)=0.
Proof. The proof is inspired by [12], where an analogous result is proved in the
time-invariant case. Consider the family of RDEs
Xk( 1) A( t)Xk( t)A( t)’ + Qk( t)
d-
-A( t)Xk( t)C( t)’[ C( t)Xk( t)C( t)’ + R( t)] -1C(t)Xk(t)A(t)’,
(5) Xg(to)=Xo,
1
Qk( t) Q( t)
Then, in view of Lemma 2, we have
+ I, k 1, 2,.’..
X+(t)<-X(t)<-X,+,(t)<-Xk(t),
t>=to, k:l,2,’".
By the definition of Q(t), (A(.), Qk(" )) is uniformly stabilizable. Then, by Theorem
3, limt. Xk(t) X(t), where X{(. is the maximal solution of (5). It is not difficult
to see that
X+(t)<X ++,(t)<-X(t)
+
Vt, k= 1,2,....
TIME-VARYING RICCATI EQUATION 1259
Now, we take the limit for k-oo. X-(t) is monotonically nonincreasing (in the sense
that X-+(t)<-_X-(t)) and bounded below by X+(t). Hence, there exists some .(.)
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
such that limk_ooX-(t)--(t)=0, for all t. Recalling (5), such a limit 3(.) proves
to be an infinite-time solution of the RDE (3). Moreover, ;( t) >= X/( t), for all t.
Therefore, X/( being maximal, .,’(. )= X/( ). It follows easily that lim,_oo X(t)-
x+() =0.
An analogous convergence result can be found in [18, Prop. 3.2], where the
attractiveness from above of the stabilizing solution is established. Apart from the fact
that [18] refers to a different context (infinite-dimensional, continuous-time, Riccati
equations), Theorem 4 is therefore more general, in that the stabilizing solution, if
any, is maximal, but the maximal" solution is not necessarily stabilizing.
4. Minimal and nonpositive solutions. In this section, the nonpositive solutions of
the RDE will be considered. The aim is to derive a set of results parallel to those
established in the previous section. The key notions will be antidetectability and
antistabilizability, as well as the notions of minimal and antistabilizing solution that
are given below.
DEFINITION 10 (minimal solution). A symmetric solution X-(. of the RDE (3)
is said to be minimal if, for any symmetric solution X(. of the RDE, X-(t)-X(t) <-0,
for all t.
DEFINITION 11 (antistabilizing solution). A symmetric solution X(. of the RDE
is said to be antistabilizing if the corresponding closed-loop matrix function F(. is
exponentially antistable.
The analysis will be carried out by showing that there is a one-to-one correspon-
dence between the nonpositive solutions of the RDE (3) and the nonnegative solutions
of a suitably redefined RDE. In the continuous-time case, such a correspondence
would be easily established by simply reversing the time axis. Given a Riccati differential
equation characterized by (A(t), Q(t), C(t), R(t)), consider the "reversed" Riccati
differential equation corresponding to (-A(-t), Q(-t), C(-t), R(-t))" if X(t) is an
SNS solution of the original Riccati equation, -X(-t) is an SPS solution of the
"reversed" equation, and vice versa. In discrete time, as shown below, the correspon-
dence is not so easily worked out, and some technicalities are required. From now on,
it is implicitly assumed that A(.)- exists and is bounded. Some discussion on the
merits of such assumption can be found at the end of this section. First, let us define
the "reversed system."
DEFINITION 12 (reversed system). Assume that A(t) is nonsingular for each
and let
(6a) .(t)= A(-t) -,
(6b) ((t) A(-t) -1Q(-t)A(-t) -1’,
(6c) C(t)=C(-t+l),
(6d) R(t)=R(-t+l).
Then the reversed system associated with system (1) is
(7a) ( + 1) A( t)(t) + 7(t),
(7b) 97(t) C( t):[.( t) + (t),
where (.) and if(.) are independent zero-mean white noises having covariance
matrices 0(" and/(. ), respectively.
1260 GIUSEPPE DE NICOLAO
Observe that, if (r)=X(--r+ 1), (t)=--(t)v(--t), and (t)= w(-t+ 1), t_ > z,
then 2(t)=x(-t+ 1) and (t)=y(-t+ 1), for each t- > z. In other words, the reversed
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
system is nothing but a representation of system (1) when the time axis is reversed.
Note also that by reversing system (7), we turn back to the original system (1).
Associated with the reversed system (7) is the following reversed Riccati difference
equation:
(8)
(t / 1) (t)(t)(t)’/ ((t)
-
-,( t)’( t)( t)’[ ( t)( t)( t)’+/ (t)] ( t)’( t)A( t)’.
To clarify the relationships between the solutions of (8) and (3), we first provide a
result on the structural properties of (7). The proof is straightforward and is therefore
left to the reader.
PROPOSITION 2.
(a) Matrix ,(. is exponentially stable (antistable) if and only if A(. is exponen-
tially antistable (stable).
(b) The pair (,(.), 0(’)) [(,(’), ((’))] is uniformly reachable (observable) if
and only if the pair (A(. ), Q(.)) [(A(.), C(.))] is uniformly reachable
(observable).
(c) The pair (A(.), Q(. )) is uniformly stabilizable (antistabilizable) if and only if
the pair (A(.), Q(. )) is uniformly antistabilizable (stabilizable).
(d) The pair (,( ), ( )) is uniformly detectable (antidetectable) if and only if the
pair (A(.), C (.)) is uniformly antidetectable (detectable).
We are now in a position to prove the following theorem that establishes a
one-to-one correspondence between the SPS (SNS) solutions of (8) and the SNS (SPS)
solutions of (3).
THEOREM 5. Assume that A(t) is nonsingularfor each t. Let Y(-t + 1) -X(t), and
(9) ’(t + 1)- ,(t) Y(t)(t)’+ O(t).
Then
(a) (.) SPS (SNS) solution of the RDE (8) if and only fiX(.) is an SNS
is an
(S PS) solution of the RDE (3);
(b) Denoting by K(.) the Kalman gain associated with an SNS (SPS) solution
X(.) of the RDE (3) and letting I(t)=(t)f(t)(t)’[(t)ff(t)(t)’+
(t) ]-1, the closed-loop matrix A(. K (.) C (.) is antistable (stable) if and
only if the closed-loop matrix (. I7((. (.) is stable (antistable);
(c) Letting XI(" ), X2(" be two SNS (SPS) solutions of the RDE (3) and 1(’),
2(" ), the correspondin SPS (SNS) solutions of the RDE (8), XI(" )-> X2(.
if and only if X1 (") <- X2(. ).
Proof of (a). Assume that X(. is an SNS solution of the RDE (3). Then
V(t) -(t) -1Y(t + 1)(t)’-I + (t)-i ((t)(t)
-(t) -1Y(t + 1)(t / 1)’[/(t / 1)- (t / 1) Y(t / 1) ((t / 1)’] -1
Conversely, suppose that X(. is an SPS solution of the RDE (8). From (8) and
(9), follows that (11) holds. Now, in view of Lemma A3, (10) holds, also, so that
it
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
by substituting (10) into (9), X(. is easily shown to satisfy the RDE (3).
As for the SPS solutions of the RDE (3), recall that the reversed system associated
with (7) coincides with system (1). Therefore, there also exists a one-to-one correspon-
dence between the SPS solutions of (3) and the SNS solutions of (8).
Proof of (b). Let us consider an SNS solution X(.) of (3). First, note that the
invertibility of A(t) and Lemma A4 imply that (t)-(t)(?(t) is nonsingular, for
all t. By resorting to the matrix inversion lemma,
.(t) --/ (t) (t) [[1 + ( t)( t)’J (t) ( t)]( t)-l]
-1
-.
Define F(-t+l)=[I+2(t)(?(t)’(t)-(?(t)]. Note that, since 2(.), (.), and
/(.)- are bounded, F(.)- is bounded. As seen above, expression (10) holds true.
Then, the matrix inversion lemma implies that
2( t)(( t)’/ (t)-’ ((t)
=[ Y(t) + Y(t)(t)’[(t)-(t)Y(t)(t)’]-a(t)Y(t)](t)’(t)-’(t)
Y( t)( t)’[/ (t) (t) Y( t)( t)’] (t).-1
Therefore,
F(t) I-X(t)C(t)’[R(t)+ C(t)X(t)C(t)’]-lC(t) and
fi(t)-I(t)((t)=A(-t)-lF(-t+ 1) -1.
Denote by (k, s) and (k, s) the transition matrices relative to A(. )- K(. )C(.
and (.)-/(.)((.), respectively. Then, (k,s)=A(-k+l)-lq(-s,-k+2)-lF
(-s + 1)-1 and the thesis is proved
As for the SPS solutions of the RDE (3), the same observation as at the end of
the proof of (a) applies.
Proof of (c). The proof is straightforward in view of (9). U
When X(. is SNS, ’(. defined in (9) is SPS and satisfies the RDE (8). Hence,
(. can be seen as the covariance of the state prediction error relative to system (7).
Now, (9) is the classical time-update equation of the Kalman filter, so that Y(.)
coincides with the covariance of the. state filtering error. Therefore, given any SNS
solution X(. of (3), its reversed opposite Y(. can be interpreted as the covariance
of a state filtering error relative to the reversed system (7). On the other hand, the SPS
solutions are, as usual, interpreted as the covariance of a state prediction error relative
to system (1). This is a main difference with respect to the continuous-time case, where
both the SPS solutions and the reversed opposite of the SNS solutions of the Riccati
differential equation can be seen as the covariance of a state filtering error relative to
a suitable (standard or reversed) system.
Now, in view of Theorem 5 and Proposition 2, the following results on the SNS
solutions of (3) are direct consequences of the parallel results on the SPS solutions
that were proved in the previous section.
THEOREM 6. Let (A(.), C(.)) be uniformly antidetectable. Then the RDE (3)
admits a minimal solution X-(. ), which is, in fact, negative semidefinite.
PROPOSITION 3. The antistabilizing solution of the RDE (3) (if any) coincides with
the minimal solution and is unique.
THEOREM 7. If (A(. ), C(. )) is uniformly antidetectable and (A(. ), Q(. )) is uni-
formly antistabilizable, then the RDE (3) admits a unique SNS solution, which is, in
fact, antistabilizing. Moreover, such a solution is a moving equilibrium for all the SNS
reversed finite-time solutions of the RDE.
1262 GIUSEPPE DE NICOLAO
THEOREM 8. Assume that (A(.), C(. )) is uniformly antidetectable and let X-(.
denote the minimal solution. Then, if X(. is a reversed finite-time solution of the RDE
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
Proof. The first point straightforwardly follows from [32, Lemma 3.1], and the
second one is verified by inspection. As for detectability and observability, recall that
they are feedback invariant properties; see, e.g., [25] and [17].
We wonder whether, when passing from (3) to (12), the properties concerning
the boundedness of A(.) translate into analogous properties relative to F(.). The
answer can be found in the following result.
LEMMA 4. Assume that A(.) and A(.)-I are bounded. Consider the closed-loop
matrix F(. corresponding to any infinite-time solution X(. ). Then, F(. and F(. )-1
are bounded, too.
Proof. By means of the matrix inversion lemma, we can see that
F(t) A(t) K(t)C(t) A( t)[I + X(t)C(t)’g(t) -1 C( t)].
Then, the thesis easily follows from the boundedness of X(. ), C(. ), and R(. )-1.
By means of Lemma 3, in the case where a minimal solution X-(.) exists, we
can take X(. X-(. ), so that any infinite-time solution of the RDE (3) corresponds
to a positive semidefinite solution of (12). Then, when X-(.) is antistabilizing, the
RDE (12) is just a particular case of the RDE (3) with A(. exponentially antistable
and Q(t)= 0. A result concerning the maximal solution of such a particular RDE is
now established.
LEMMA 5. Assume that A(.) is exponentially antistable, Q(t)=0, for all t, and
(A(. ), C(. )) is uniformly detectable. Then the maximal solution X+( of the RDE (3)
is positive definite, and X +(. )-1 is bounded.
Proof. By Theorem 4, a finite-time solution X (.) of the RDE with initial condition
X (to) -> X / (to) converges to the maximal solution. In particular, assume that X (to) > 0.
We will prove that the maximal solution is positive definite by showing that X(t) -1
remains bounded for each t. Consider the RDE (3) with Q(t)=0, for all t. By means
of the matrix inversion lemma, we obtain
X(t + 1)= A(t)[X(t)-l + C(t)’g(t)-lC(t)]-lA(t) ’,
(13)
X + 1 )-1 A( t)’-lX (t)-lA(t) -1 + A(t) ’-1 C( t)’g( t) -1 C( t)A( t) -1.
Recall that the exponential antistability of A(-) entails the exponential stability of
A(. )--1o Since A(. )-1, C(. ), and R(. )-1 are bounded, X(. )-1 is bounded, also.
Interestingly enough, this last result enables us to clarify some relationships
between the maximal and the minimal solution. In particular, we focus on the so-called
gap between these solutions and on their stabilizing and convergence properties.
COROLLARY 1. Assume that (A(.), C(.)) is uniformly observable and that the
minimal solution is antistabilizing. Then the gap X+(t)-X-(t) between the maximal
and the minimal solution of the RDE is positive definite for each t.
Proof With reference to Lemma 3, let X(. )= X-(. ). By the assumptions, F(.
is exponentially antistable. Then, the gap X+( )-X-(. turns out to be the maximal
solution of the RDE (12), and the result follows from Lemma 5.
LEMMA 6. The maximal solution of the RDE (3) is stabilizing if and only if the
minimal solution is antistabilizing.
Proof Suppose that the minimal solution is antistabilizing. In view of Lemma 3,
there is no loss of generality in assuming that Q(t) 0, for all t, and A(. is exponentially
antistable. Then, by definition of infinite-time solution and by Lemma 5, the maximal
solution X+(.), as well as its inverse, is bounded. Under the given assumptions, the
1264 GIUSEPPE DE NICOLAO
where A l(t) and Xl( t) are square matrices of the same constant dimensions,
Xl(to) > 0, and * denotes a term we do not consider specifically;
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
(b) Partition matrix C(t)T(t)’ as C(t)T(t)’--[Cl(t)*], where Cl(t) has the same
number of columns as A l(t). Then XI t), >= to, satisfies the following reduced-
order RDE:
_ _
where X-(. is the maximal solution of (15). Moreover, X(. has the same
supporting subspace as X(. );
(f) Let Xa(" and Xb(" be two (finite-time or infinite-time) SPS solutions of the
RDE and denote by (. and b(" the corresponding supporting subspaces.
a
Then, if s gTb S [, (s) b (S)] at an arbitrary time point s, :Ta Wb
_
[W(t) Wb(t)], for any t;
(g) Let X(.) and Xb(’) be two infinite-time SPS solutions of the RDE. Then,
t) b t) implies X (.) Xb (");
(h) Given a time point s and a subspace gT of R n, there exists one and only one
infinite-time SPS solution X(. such that [X(s)] ;
(i) Let X(. and Xb(" be two infinite-time SPS solutions of the RDE, and denote
by (.) and (.) the corresponding supporting subspaces. Then, if (s)
b(S) at an arbitrary time point s, Xa(t)>= Xb(t), for all t.
Proof of (a). First, note that T(t) defines a change of basis for system (1). It can
be seen that X(.) is a solution of the RDE (3) if and only if ’(t) T( t)X( t) T( t)’
satisfies the RDE
(t + 1) 3( t))"(t)( t)’
(17)
where A(t)= T(
, , -
(t) (t)(( t)’[ (t) (t)(( t)’+ R( t)] (( t)’( t)3( t)’,
+ l )A( t) T( t)’ and (t)=C(t)T(t)’.
Since X(to) is SPS, there exists a unitary T(to) such that
T(to)X(to)T(to),=[Xl(to) 0]
0 0
with X(to) positive definite. Now, denoting by r the rank of X(to), let the first r
columns of T(to+ 1) -1 be an orthonormal basis of [A(to)X(to))] and choose the
other columns so as to make T(to+ 1) unitary. Then, A(to) has the structure given in
(14). By substituting (to) and ’(to) into (17), X(to+ 1) also takes on the partitioned
structure (14). Obviously, the procedure can be iterated for any => to, proving the thesis.
1266 GIUSEPPE DE NICOLAO
Proof of (b). The proof follows by simple substitution of ,(t), (t), and )(t)
into (17).
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
_
positive definite for each t, so that [Xoo(t) [X (t) ], for all t.
Proof of (f). Assume that gTa(s)=gb(s). As seen in the proof of point (a),
[X(s + 1)] depends only on IX(s)] and not on the particular value of X(s).
Therefore, Ta (t)= b(t)-t--> S. By making reference to the reversed RDE, we can see
that also [X(s 1)] depends only on IX(s)], and the first part of the proof is
completed.
As for (s) b (S), consider the following nonsingular transformation Ta (")
that performs the decomposition (14) on Xa(. ):
t>=s,
_
with Xbl(t) of the same dimensions as X,l(t). Therefore,
Analogously, by means of the reversed RDE, it can be shown that o(s) Tb(S) implies
that ( t) b( t), < s.
_
(t) b (t), _--> S.
Proofof (g). In view of point (f), a(. fb(" )-For both solutions, decomposition
.
_
(14) can be performed by means of the same transformation T(.). Then, point (e)
_
implies that both X(. and Xb(" coincide with Xoo(" defined in (16).
Proof of (h). Consider any finite-time solution X(. such that [X(s)]
X(.)=Xoo(.) be the infinite-time solution to which, in view of point (e), X(.)
converges. Then, by point (g), X(.) is the only infinite-time solution such that
[X(s)]=.
Let
Proofof (i). First, note that, by point (f), f(s) gTb(S) implies that fa (t) b(t),
for all t. By the assumptions, for any r, it is possible to choose two initial conditions
TIME-VARYING RICCATI EQUATION 1267
X,(z) and Xt(z) such that [X(z)] a(z), [Xt(z)] b(’) and X(z)>-Xt(z).
Denote by X(., -) and X(., -) the finite-time solutions such that X (-, -)= X (z),
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
X (-, -) X (’). Lemma 2 implies that X, (t, z) >_- X (t, z), _-> -. Since Xa (t)
lim__ X,(t, -) and Xb(t)=lim__X(t, z), the thesis follows.
THEOREM 9. Let (A( ), C (.)) be uniformly observable and assume that the minimal
solution of the RDE (3) is antistabilizing. Then, the infinite-time solutions of the RDE
constitute an infinite number of isomorphic distributive lattices with common minimal and
maximal elements.
Proof Consider the set of all subspaces in R". In view of Lemma 8(h), to each
of these subspaces corresponds one and only one infinite-time solution. We now show
that this infinite number of solutions can be organized in an infinity of isomorphic
lattices. Take n independent one-dimensional subspaces Wi, i= 1,..., n, and call
5 {i} the set of these subspaces. The set 3- formed by all subspaces that can be
obtained by means of the operations of intersection and sum between the elements of
,
turns out to be a distributive lattice with respect to such operations. Note that,
independently of the choice of O the origin and R belong to 3- and constitute the
maximal and minimal element of 3-. By suitably varying 5, we can see that all the
subspaces in R" can be organized in an infinity of isomorphic lattices. Thanks to the
one-to-one correspondence between subspaces and solutions of the RDE, this reflects
in an analogous structure for the infinite-time solutions. The partial ordering by
inclusion of the subspaces is translated into the partial ordering _-> for symmetric
matrices (Lemma 8(i)), while the maximal and minimal elements of the lattices are
given by the maximal and minimal solution.
Remark 4. Obviously, the above theorem also holds for the time-invariant RDE.
Then, the problem of determining the number and the structure of the solutions of
the ARE is equivalent to the problem of determining which infinite-time solutions are
time-invariant, i.e., which supporting subspaces do not depend on time. Therefore, it
is not surprising that in the literature these solutions have been associated with
subspaces that, according to our terminology, would be denoted as "A-invariant
supporting subspaces." Analogously, in the study of the periodic solutions of the
periodically time-varying Riccati equation, the key task consists in the classification
of the supporting subspaces that are periodic with the same period as the coefficients
of the equation.
Appendix. Herein, we report the statements of some technical lemmas that are
needed throughout the paper. The proofs, which rely mostly on simple matrix manipula-
tions, can be found in [33].
LEMMA A1. Let X(. be an SNS solution of the RDE (3). Then, C(t)X(t)C(t)’+
R(t) > O, for all t.
LEMMA A2. Let R > 0 and H >- 0 be such that R CII C’] -1 > 0 and let P
II + HC’[R CHC’]-CH. Then, H P- PC’[R + CPC’]-CP.
LEMMA A3. Let H P PC’JR + CPC’] -1CP, with P >= 0 and R > O. Then, P
II + H C’[ R CH C’]-I CII.
LEMMA A4. Let F I PC’[ R + CPC’] -1 C, with R > 0 and P >- O. Then, det F 0.
LEMMA A5. Assume that Q(.) is bounded and A(.) is exponentially stable. Let
X(. be an infinite-time SPS solution of the LDE
X( + 1) A( t)X( t)A( t)’ + Q( t).
Then, if there exists a constant k > O, such that X(t) >= kI, for all t, thepair (A(. ), Q( ))
is uniformly reachable.
1268 GIUSEPPE DE NICOLAO
REFERENCES
1] R. E. KALMAN, New methods in Wiener filtering theory, in Proc. 1st Sympos. on Engrg. Applications
of Random Function Theory and Probability, J. Bogdanoff and F. Kozin, eds., John Wiley, New
York, 1963, pp. 270-388.
[2] R. S. Bucv, Global theory of the Riccati equation, J. Comput. Systems Sci., (1967), pp. 349-361.
[3] W. M. WONHAM, On a matrix Riccati equation for stochastic control, SIAM J. Control, 6 (1968),
pp. 681-698.
[4] V. KUCERA, A contribution to matrix quadratic equations, IEEE Trans. Automat. Control, AC-17 (1972),
pp. 344-347.
[5] P. E. CAINES AND D. Q. MAYNE, On the discrete-time matrix Riccati equation of optimal control,
Internat. J. Control, 12 (1970), pp. 785-794; 14 (1971), pp. 205-207.
[6] J. C. WILLEMS, Least-squares stationary optimal control and algebraic Riccati equation, IEEE Trans.
Automat. Control, AC-16 (1971), pp. 621-634.
[7] K. MARTENSSON, On the matrix Riccati equation, Inform. Sci., 3 (1971), pp. 17-49.
[8] V. KUCERA, On nonnegative definite solutions to matrix quadratic equations, Automatica, 8 (1972),
pp. 1413-1423.
[9] I. P. MOLINARI, The stabilizing solution of the algebraic Riccati equation, SIAM J. Control, 11 (1973),
pp. 262-271.
10] F. M. CALLIER AND J. L. WILLEMS, Criterion for the convergence of the solution of the Riccati differential
equation, IEEE Trans. Automat. Control, AC-26 (1981), pp. 1232-1242.
11] S. W. CHAN, G. C. GOODWIN, AND K. S. SIN, Convergence properties of the Riccati difference equation
in optimal filtering of nonstabilizable systems, IEEE Trans. Automat. Control, AC-29 (1984),
pp. 110-118.
[12] C. E. DE SOUZA, M. R. GEVERS, AND G. C. GOODWIN, Riccati equations in optimal filtering of
nonstabilizable systems having singular state transition matrices, IEEE Trans. Automat. Control,
AC-31 (1986), pp. 831-838.
[13] W. A. COPPEL, Matrix quadratic equations, Bull. Australian Math. Soc., 10 (1974), pp. 377-401.
[14] M. A. SHAYMAN, Geometry of the algebraic Riccati equation, Parts and II, SIAM J. Control Optim.,
21 (1983), pp. 375-394, 395-409.
15] J. J. DEYST, JR. AND C. F. PRICE, Conditions for asymptotic stability of the discrete minimum-variance
linear estimator, IEEE Trans. Automat. Control, AC-18 (1968), pp. 702-705; AC-18 (1973), pp. 562-
563.
[16] W. W. HAGER AND L. H. HOROWlTZ, Convergence and stability properties of the discrete Riccati
operator equation and the associated optimal control and filtering problems, SIAM J. Control Optim.,
14 (1976), pp. 295-312.
[17] B. D. O. ANDERSON AND J. B. MOORE, Detectability and stabilizability of time-varying discrete-time
linear systems, SIAM J. Control Optim., 19 (1981), pp. 20-32.
[18] G. DA PRATO AND A. ICHIKAWA, Quadratic control for linear time-varying systems, SIAM J. Control
Optim., 28 (1990), pp. 359-381.
[19] H. KANO AND T. NISHIMURA, Periodic solutions of matrix Riccati equations with detectability and
stabilizability, Internat. J. Control, 29 (1979), pp. 471-487.
[20] M. m. SHAYMAN, On the phase portrait of the matrix Riccati equation arising from the periodic control
problem, SIAM J. Control Optim., 23 (1985), pp. 717-751.
[21] C. E. DE SOUZA, Riccati differential equation in optimal filtering of periodic non-stabilizable systems,
Internat. J. Control, 46 (1987), pp. 1235-1250.
[22] S. BITTANTI, P. COLANERI, AND G. DE NICOLAO, The difference periodic Riccati equation for the
periodic prediction problem, IEEE Trans. Automat. Control, AC-33 (1988), pp. 706-712.
[23] J. C. ENGWERDA, Stabilizability and detectability of discrete-time time-varying systems, IEEE Trans.
Automat. Control, AC-35 (1990), pp. 425-429.
[24] B. D. O. ANDERSON AND J. B. MOORE, Comments on "stabilizability and detectability of discrete-time
time-varying systems," IEEE Trans. Automat. Control, to appear.
[25] , New results in linear system stability, SIAM J. Control, 23 (1969), pp. 398-414.
[26] D. L. KLEINMAN, On an iterative technique for Riccati equation computations, IEEE Trans. Automat.
Control, AC-13 (1968), pp. 114-115.
TIME-VARYING RICCATI EQUATION 1269
[27] N. R. SANDELL, JR., On Newton’s method for Riccati equation solution, IEEE Trans. Automat. Control,
AC-19 (1972), pp. 254-255.
Downloaded 01/03/13 to 132.206.27.25. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
[28] G. A. HEWER, An iterative technique for the computation of the steady state gains for the discrete optimal
regulator, IEEE Trans. Automat. Control, AC-16 (1971), pp. 382-384.
[29] R. S. Bucv AND P. D. JOSEPH, Filtering for Stochastic Processes with Applications to Guidance, John
Wiley, New York, 1968.
[30] R. R. BITMEAD, M. R. GEVERS, I. R. PETERSEN, AND R. J. KAYE, Monotonicity and stabilizability
properties of solutions of the Riccati difference equation: Propositions, lemmas, theorems, fallacious
conjectures and counterexamples, Systems Control Lett., 5 (1985), pp. 309-315.
[31] T. NISHIMURA, On the a priori information in sequential estimation problems, IEEE Trans. Automat.
Control, AC-11 (1966), pp. 197-204; AC-12 (1967), p. 123.
[32] C. E. DE SOUZA, On stabilizing properties of solutions of the Riccati difference equation, IEEE Trans.
Automat. Control, AC-34 (1989), pp. 1313-1316.
[33] G. DE NICOLAO, On the time-varying Riccati difference equation of optimal filtering, Internat. Rep.
90-02, Centro Teoria dei Sistemi--CNR, Dipartimento di Elettronica, Politecnico di Milano,
Italy, 1990.