104 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, FEBRUARY 1975

plot of the number of modal filters per scan for each model. The
algorithm must consider many measurement combinations of measure-
ments to get started.
CONCLUSION
A branching algorithm is an efficient, powerful, and flexible procedure
for discriminating and tracking objects in a noisy environment. The
computer program developed by the authors is being used to augment
the visual discrimination and tracking of several moving objects against
fixed and random background clutter.
REFERENCES
I71
181
I91
H. E. Rauch, Combining the functions of signal detection and trajectory estima-
tion, in Proc. 2nd Symp. Nonlinear Esrimarion Theor). and i u Applicalions. San
Diego. Calif.. Sept 1972.
R. S. Bucy and K. 0. Senne, Realization of optimum discrete-time nonhnear
estimations. in Proc. S ww. Nonlinear Esrimarion 7heory and its Applications. San
Diego. Calif.. 1970.
A. G. Jaffer and Y. Bar-Shalom, On optimal tracking in multiple-target environ-
men&. In Proc. 3rd Symp. Nonlineur Esrrmarion Theory and ils Applicarions. San
Diego. Calif., Sept. 1972.
identification: Dlscrete-data case, Inr . 1. Conrr., vol. 17. 1973.
D. G. Lainiotis and S. K. Park, On joint detection. estimation. and system
J. S. Thorp, Optimal tracking of maneuvering targets, IEEE Trans. Aerosp.
Elerrron Sy s r , voi. AES-9, pp. 512-519. July 1973.
Gauss-Markov process, Div. Eng. Appl. Phys., Harvard Uni v.. Cambridge. Mass.,
P. M. Newbold and Y. C. Ho, Detection of changes in characteristics of a
Tech. Rea. 531. 1967.
. . ~~
of imprecisely determined origin in surveillance
systems.
In Proc. IEEE
Conl.
RI-A. S&e;and JI J. Stein, An optimal tracking filter for
processing
sensor data
Precision and Conrrol , Univ. Fl ori da, Gainesville. Dec. 1971.
K. P. Prabhu, On the detection of a sudden change in system parameten. IEEE
Truns. Inform Theory (Corresp.). vol. IT-16. pp. 497-500. July 1970.
Systems Theory Lab., Stanford Electronics Lab.. Stanford, Calif.. Tech. Rep. 6302-
D. T. Magill, Optimal adaptive estimation of sampled stochastic processes,
3. 1963.
A. W. F. Edwards. Likelihood. Cambridge. Mass.: Cambridge Univ Press, 1972.
T. Kailath. The diversence and Bhattacharyya distance meaSures in signal selec-
eon.'. IEEE Trans. Commun Technot.. vol. COM-15. pp. 5260. Feb. 1967. (Exten-
sive bibliography.)
Identification of Autoregressive
Moving-Average
Parameters of Time Series
D. GRAUPE, MEMBER, IEEE, D. J . K UUSE, MEMBER. IEEE,
AND J . B. MOORE
Abmuct-A procedure for sequentially estimating the parameters and
orders of mixed autoregressive moving-average signal models from time-
series data is presented. Identification is performed by f i i identifying a
purely antoregressive signal model. The parameters and orders of the
mixed autoregressive moving-average process are then given from the
solntion of simple algebraic equations involving the purely autoregressive
model parameters.
I. INTRODLCTION
Many control system design algorithms and filtering algorithms in the
literature assume knowledge of the parameters of the signal process
model. In practice rarely is there Q priori knowledge of these parameters.
and so there exists the need to identify a signal model first. For
stationary stochastic time-series an autoregressive moving-average
(ARMA) model is frequently used since it is the minimum parameter
linear model of such time series.
Important contributions [1H6] to the problem of identifying the
parameters of an ARMA model have been made in the last few years.
The text of Box and J enkins [I] is probably the most complete book to
date on the identification of stationary and nonstationary ARMA mod-
els. In particular, Durbin [3] has treated the problem of identifying the
autoregressive (AR) parameters of an ARMA model given the moving
W. Sorenson, Chanman of the IEEE S-CS Estimation and Identification Committee.
Manuscnpt received June 23. 1972: revised August 16. 1973. Paper recommended by H.
Fort Collins. Colo.
D. Graupe IS with the Department of Electrical Engineering. Colorado State Univenlty.
University. FOR Colllns, Colo. He IS now wt h the Department of Electrical Engineerin&
D. J. Krause was w~t h the Department of Electrical Engmeenng. Colorado State
North Dakota State University. Fargo. h.D.
Netvcastie. ?IS.%.. Australia and the Department of Electncal Engineenng. Colorado
J. B. Moore is with the Department of Electncal Engineenng. University of Newcastle.
State Univen~ty, Fort Collins. Colo.
average (MA) parameters (and vice versa). His work is based on the
important studies of purely AR models by Mann and Wald [7]. How-
ever, a limitation of his algorithms is that he requires an assumption of
the order for the AR or ARMA process. Lee [4] and Gersch [ 5] also
achieved results for the problem of estimating the AR parameters of a
mixed ARMA model of given order assuming a knowledge of the MA
parameters.
Mehra [6] has presented a method for identifying the state variable
model for a Gaussian process which can be executed in a recursive
manner. His method is computationally convenient for estimating AR
parameters of an ARMA model, but is rather complex when dealing
with the MA parameters.
This paper is an extension of earlier work by the authors [8], [9]. It is
concerned with the estimation of parameters and orders of stationary
mixed ARMA time series without Q priori assumptions on parameters or
on orders. The resulting models may be further transformed to yield the
parameters of a linear state space model [IO]. In this respect, the
algorithms of this paper may be viewed as an alternative approach to
that taken by Mehra [6]. having their main advantage in the estimation
of the MA parameters of the ARMA model.
11. MIXED ALTOREGRESSIVE MOVING-AVERAGE MODEL
A well-known property of stationary time sequences is that they may
be represented by a linear filter model driven by white noise [IO]. Let us
consider the ARMA signal model
where {wa] is the input white noise sequence with zero mean and
variance o:.; (yk) is the output sequence; el, . ,en are the AR
parameters: and 8, . 8,; . . .Om are the MA parameters. Without loss of
generality [8] we take Bo= 1. Equation (2.1) is commonly termed the
mixed ARMA model and may be written in operator notation as
+( B) ya=B( B) wa; Bh k = c k - , (2.2)
where B is a delay operator.
The system is assumed to be stable and invertible. That is. all the roots
of both + ( B) (for stability) and B( B) (for invertibility) lie outside the unit
circle to guarantee that both +- ( B) B( B) and B- ( B) +( B) form con-
vergent series [2]. We comment that in cases of noninvertible processes.
an invertible equivalent model, having the same first and second order
statistics of the original process output. will always be identified and is in
fact the only model that can be identified. (The roots of the MA
polynomial of this equivalent ARMA model will be the reciprocal of
those roots of the original process that are inside the unit circle, whereas
all other roots will be as in the original process.)
111. ESTIMATION OF MIXED ARMA PROCESSES
A. Pure A R Processes
The problem of consistent least squares estimation of the AR parame-
ters in a purely AR process [(2.1) with m =01has already been solved by
Mann and Wald [7]. For Gaussian sequences. their algorithms are,
moreover. asymptotically efficient. A recursive version of the algorithm
is described by Lee [4] (see also [ 1 I]) using sequential regression and will
not be repeated here. Consistency of the AR model identification can
also be proven using stochastic approximation theory [I21 for stable
signal model cases.
B. Mixed A RMA Processes
When a process involves m >1 MA terms, weproceed as follows.
Cross multiplying both sides of (2.2) by V 1 ( B ) as in [13]. [14], we
obtain
B - ( B ) 9 ( B ) y , = y ( B ) , ~ a = ~ ~ a . (3.1)
Here y ( B) = - l ( B) @( B) is an in-finite power series that is convergent for
SHORT PAPERS
IB I 4 1 as noted in Section 11. Consequently, ( y k) may be represented by
y k = x y ~ ~ - ~ + w ~ + e , = , x q l y k - i +Gi , f=estimateofy (3.2)
S m
i = I I = 1
where E[ej!] can be made arbitrarily small by choosing a sufficiently
high finite order s. Actually, an upper bound & on E[ej!] that vanishes
with increasing s can be evaluated by expanding E[Zf=,(y,- i i ) y k - ; +
Zj "s+l yj yk- j ] 2, as follows [ 15] :
where Q is the largest time constant of the envelope of E [ Y ~ ~ - ~ ] , given
that i > 1. As shown in. [16], such an envelope always exists for wide-
sense stationary yk.
Since (3.1) and (3.2) represent a purely AR process whose residuals
converge to discrete white noise wk, we may employ sequential regres-
sion as in Section 111-A to consistently identify yi (thisidentification
being asymptotically efficient for Gaussian yk) . The order s may be
chosen as some large integer and checked by computing the autocorrela-
tion of 4. As long as the sequence { Gi } is correlated for a given order s,
a larger value for s must be chosen. We note that the sequential
regression estimation of yi is extremely fast, even for large s, (in ex-
amples worked, a value of s =20 is more than adequate). Furthermore,
with s sufficiently large, slight changes in s are usually of little con-
sequence, as is indicated by Tables I1 and 111.
C. Derivation of ARMA Parameters and Orders from AR Model
Once the parameters of the pure AR model have been consistently
identified, an ARMA model of the same process can be derived. The
ARMA parameters and order can be obtained directly from the
parameters of the purely AR model, noting the relation
Yk 1
-=
wk 1 + y , ~ + y 2 ~ 2 + . . .
- - 1+Ql B +Q2B 2+~~~ +Q,,,Bm
I++,B+C$@~+... ++nBn
(3. 4)
where y, are the parameters of the purely AR model and Qi , Qj are the
AR and the MA parameters of the mixed ARMA model of the same
process. Cross multiplication and equating the coefficients of like powers
of B yields
+l =Ql +Y I
+~=~~+Q ~Y I +Y z
where 8, =0 for j =m +1, m +2, . . . , n. The preceding relationshps give
US the useful matrix equations
. Y,-*+Cl
. Yn- m+2
...
Y" Qm
=-
Y"+ I
Yn +2
Yn+m
(3.5)
+
1
X
forj =1,2,-. . ,where again it is understood that q =O f orj =m+ I , m+ 2
and+,=Oforj=n+l,n+2,-~-.Cl~rly,withm,n,andyl,~~,~~~.~n+m
known, the parameters Qi and +; can be determined from (3.5) and (3.6).
Note that the square matrix involving the yi in(3.5) must be nonsingular
for a unique solution of Qi. Such wiU be the case if the m and n specified
are minimal orders.
Actually, the determination of the minimal orders m and n from just
knowledge of the AR parameters is straightforward. Denoting the square
matrix involving the yi in(3.5) for n =3 and m =@i by ,TE,=, the rank of
XE,= is tested for [ ~ , ~ ] = [ O , O ] , [1,1], [],a], [2,2], [2,11, [2,0], etc., until for
some m and n,
IAE,el =O, for i i >n, m>m.
-
(3. 7)
In practice, only estimates of the true yi and of IAr,=l are available.
Hence the condition (3.7) is replaced by a test
lX,,,E12 <E, for some E >n,m >rn (3.8)
and some small E >O, or better, the values l & E1 2 can be examined for a
range of @ and ?fi and the region for which lXE,E\z becomes small to
obtain good estimates of m and n. Since the ARMA parameter estimates
above are based on estimates of pure AR model parameters which have
been shown in Section I1 to beconsistent (if the order of the purely AR
model is correct, or otherwise to be within upper bounds as in [l5D, and
noting [17,theorem 2.3.31, the ARMA parameter estimates will also be
consistent (or bounded for unknown AR order). There is likelihood of
difficulties with a signal model if m and n are estimated on the low side
of the true values since then there is not the possibility of omitting
dynamics which are essential in a signal description. On the other hand,
if the values for m and n are overestimated, all that happens is that
negligibly small extra coefficients are introduced into the model which
correspond to the addition of small-magnitude high-frequency terms
which for signal models in other than control applications do not usually
spell disaster. Some examples wi l lbe considered to give some feel for
what can happen in working with estimates.
Example I : Consider a pure ARMA process model
1-0.5B - Q( B) .
1 +1.5B+0.625B2 +( B) '
mo= I ; n0=2.
The correct purely AR model for the preceding process is y ( B) = 1 +
2 + 1 . 6 2 5 B 2 + 9 B 3 + v B 4 + - . . . Identification of y ( B) has yielded
f , =2. 018; $z= 1.639; $,=0.801; i4=0. 402; T5=0.198.
Consequently, for assuming correct orders: the following ARMA
parameters where obtruned- (via r,, r2, y3): GI = 1.519 (correct: 1.5);
GZ =0.649 (correct: 0.625); 0, =- 0.489 (correct: - 0.5).
~ Via f4 $stead of i3, the parameter estimates became GI = 1.518;
+2 =0.650; Q1 =- 0.490, which i s very close to the estimates via PI to T3.
Underfitting of orders, i.e.,* assuming m =1; n =1 wil! yield for the
present example (via TI; id Ql = -0.8 (correct: -0.5); +,=1.218 (cor-
rect: 1.5), wpereas using 9, to im+n+l=$3 instead of i2 will yield
8, =-.049; +, =1.528, which differs considerably from when only PI
106 EEE WS ACTI ONS ON AUTOblAnC CONTROL, FEBRUARY 1975
TABLE I
01 c2 '3 tl - ? " s p q Length
Sa"pl
True Val ue
I denti f i ed
1.8 -1.3
1.841 -1.385 0.455 1.012 0.196
0.4 1.1 0.28
(Gaussi an)
True Val ue 2.0 -1.7 0.5 1.5 0.685
I denti fi ed 2.016 -1.66 0.532 1.l.66 0.631
(Gaussi an)
True Val ue -1.5 -0.625 --
I denti f i ed
0.$1 2.152s
-1.486 -0.612 _ _ 0.391 0.1503
(Gaussi an)
True Val ue 1.8 -1.3 0.4 1.1 0.28
i denti f i ed
( Non-Gaussian)
1.78 -1.31 0.416 1.07 0.318
1.0 --
0.9Li 20 3 3 500
3 2 _ _
1.0 -- 3 2 _ _
1.036 15 3 2 _ _
G.09 --
G.161 10 3 2 500
2 2 LOO0
1.0 --
0.930 111 3 2 i 01l
3 2 _-
TABLE I1
S
$1 $2 $3 31 6 2
r-
5 2.038 -1.762 0.550 1.121 0.513 1.386
10 2.018 -1.638 0.537 1.428 0.609 1.070
15 2.016 -1.661 0.532 1.466 0.631 1.036
20 2.016 -1.677 0.529 1.470 0.623 1.031
True
Val ues 2.0 -1.7 0.5 1.5 0.685 1.0
N = 1000 i n all cases
and T2 are employed. Note that 4, is now assumed to be 0 instead of
0.625.
Overfitting of orders, i.e., assuming rn =1 ; n =3 (mo=I ; no= 21, yields
very close estimates for d2, and 8, as when the correct orders are
assumed either when qm+, isemployed or when ?,,,+"+I is employed
"stead of-ym+n. The overfitting above yields (via y4) also a? estimate for
+3 where Q~ =- 0.0192 (correct: 0). A similar estimate for +3 is obtained
via is.
Example 2: Consider a process given by:
1-0.5B
1 +1.5B+0.66B2+0.08B3
namely, rno=l; n,=3. Here, Tl =1. 51; Tz=1.40; T3=0.78; T4=0.39;
fs=0.196; 9,=0.0981. . . . The preceding Ti yield, for assuming rn =2:
n =2, an estimate of 6, =- 0.0775 via f4 and T2 =- 0.060 via i s (the true
8, being 0 in both cases). However, a correct assumption of m and of n
yields ARMA parameters that are close to the true ones via either T4 and
ys.
IV. CQ~WLTATIONAL RESULTS
Table I gives several examples of computational results where the
present procedure was employed (using Fortran on a CDC 6400 com-
puter).
Results illustrating the effect of changing the order of the AR model
on the ARMA parameter values are given in Table 11.
Table I11 illustrates the effect of various orders m and n on the
variance of the one-step prediction error yk -; k. j k being obtained from
the ARMA model.
V. CONCLUSIONS
A procedure has been presented for identifying the parameters and
orders of linear mixed ARMA models of Gaussian and non-Gaussian
time series. This procedure differs from that of [6], and from procedures
based on [6], in that no computation of covariances and of spectral
factorization is required and in the simplicity of deriving the MA
parameters. Also the proofs of consistency do not require a Gaussian
TABLE I11
Xumber of Number of var[yk - ykl
AR Parameters ?lA Parameters
1
2
3
1
2*
3
1
2
3
- - ~ _____
unstabl e
1.423
1.042
1.052
1.031
1.034
1.039
1.031
1.036
s = 20, N = 500 sampl es *True order
assumption. Concrete criteria for determining the AR and ARMA
orders are given.
Extensions of the method to input-utput noise models, and to some
nonstationary processes are possible, as is a direct transformation [9] to a
state-space formulation, for cases of noise-free and of noisy measure-
ments.
We note that the present approach can use a stochastic approximation
subroutine with a scalar correction coefficient ( p of [IS]) instead of
employing a sequential least squares regression subroutine as in [9], [l l .
ch. 61. However, the convergencs rate wi l l inevitably be much slower
The analysis above indicates that ARMA signal model parameter and
order estimation can be performed by a sequential pure AR identifica-
tion followed by a solution of a set of algebraic equations (3.5) and (3.6),
the latter requiring only the storing of rn +n +1 AR parameters. The
computation effort is therefore virtually that of the very fast sequential
regression identification of the pure AR model, which requires storage of
only s measurements where s is the AR model order. Hence. complete
identification can be executed with microprocessing hardware at great
speed.
[ 181.
REFEREWES
G. E. P. Box and G. M. J enkins. T z m Ser m Ana!vsis, Forecarring and Conrrol.
San Francisco: Holden-Daq. 1970.
G. E. P. Box. G. M. J enkins. and D. V'. Bacon. "Models for forecastme seasonal
and- non-seasonal bme series.- in Adcanced Senunar on Specrra/ Anal*.& of T i m
Serier. B. Hams. Ed. Neu, York: Wiley. 1967. p. 271.
J. Durbln. "The F~tri ng of Time Series Models." Rer INI. Inr de Srur.. Vol. 28. pp.
27L747 19hn -. - . , . . . . .
R. C. K. Lee. Op r i mu l Esrrmarron Idemt$rorion and Conrrol. Research hlonogaph
No. 28. Cambridge. Massachusetts WIT Press. 1964.
W. Gersch. "Estimation of the autoregressive parameters of a mixed autoregressive
moving-average Ume sene&^ IEEE Trom. Auromar. Conrr. (Shon Papers). vol.
AC-15. pp. 58S-588. Oct. 1970.
R. K. Mehra. "On-line ldenufication of hear dynamic systems mth applxations to
Kalman filtering," IEEE Trans. Auromar. Conrr.. vol. AC-16. pp.l2-21. Feb. 1971.
difference equations:' Economcrrica. vol. I I . pp. 173-220. 1913.
H. B. Mann and A. Wald. "On the statistical treatment of linear stocharuc
predlcror and nonlinear systems parameters." in Proc. 14rh Mldwesr S~. mp. Ctrcuir
D. Graupe and D. J. Krause. "Sequential regasi on ldentlficatlon procedure for
Theon.. 1971. pp. 8.6.1-8.6.10.
predictor parameterr" in Proc. 2nd Ss mp . Nonlrnear Esrimarron Theon.. San Diego.
D. J. Krause and D. Graupe. "Identifxatlon of autore_eresslre-moving-average
Cahf., 1971.
SHORT PAPERS 107
R. E. Kalman, A new approach to hearfiltering and prediction problems, Trunr.
ASME, J. Bmic Eng.. ser. D, vol. 82, pp. 34-45.
D. Graupe, Idenrifcution of Systems. New York: Van Nostrand, 1972. I
G. N. Sar i d i s and G. Stein, Stochastic approximation algorithms for discrete-time
system identification, IEEE Trunr. Automar. Conrr., vol. AC-13, pp. 515-523, Oct.
1968.
[I 31 J. -Durbin. Efficient estimation of parameters of moving average models,
1141 H. Wold, A St& in the Aml ysi s of Smrionury Time Series. Uppsala, Almquist and
Biometriku, vol. 49, pp. 3W316, 1959.
. .
[I 51 D. Graupe,: Estimation of upper bounds of errors in identifying ARMA models,
1161 J. L. Doob, Sr ochr i c Procerres. New York: Wiley, 1953, p. 524.
(171 E. Lukacs. Srocharric Conaergence. Lexington, Mass.: Heath, 1968.
1181 G. N. Sari&, Comparison of 5 popular identification al gori t hmsA survey, in
Wiksells. 1938.
in Proc. 4rh Sy w. Nodi nar Esrimarion Theory, San Diego, Calif., 1973.
~~
Proc. Decision and Control ConJ. New Orleans. La., 1972.
Approximate Non-Gaussian Filtering
with Linear State and Observation Relations
C. J. MASRELIEZ
ABstracr-Two approaches to the non-Gaussian filtering problem are
presented. The proposed filters retain the computationally attractive re-
cursive sfructure of the Kalman fiiter and they approximate well the exact
minimum variance filter in cases where either 1) the state noise is
Gaussian or its var i ance small in comparison to the observation noise
variance, or 2) the observation noise is Gaussian and the system is one step
observable. In both cases, the state estimate is formed as a l i nearpredic-
tion corrected by a nonlinear function of past and present observations.
Some simulation results are presented.
I. INTRODUCTION
Since R. Kalman and R. Bucy presented their famous papers on
filtering for discrete linear systems in 196Ck1961 [8],[9], a great number
of researchers have worked on the prediction, filtering, and smoothing
problem, extending Kalman-Bucys result, which holds for a linear state
and observation model and white noise sequences, to cases where the
state and observation relations are nonlinear or the noises correlated.
Comparatively little has been done, however, to extend the discrete
Kalman filter, which is optimal for Gaussian disturbances, to situations
where the distributions for the dsturbances are non-Gaussian. This may
partially be explained by the fact that the Kalman filter constitutes the
best linear filter, and by the argument that the disturbances often will
have approximately Gaussian distributions due to central limit theorem
type effects.
However, in practice we will often have to deal with densities which
either from the physics of the particular situation are highly non-
Gaussian or densities which may have Gaussian shape in the middle but
possess small but potent deviations from normality in the tails. It iswell
known [6],[12] that the latter type of deviations from normality may
seriously degrade the performance of a linear estimator. Thus there
appears to be considerable motivation for considering non-Gaussian
filters.
A few contributions in this direction have been made. We mention, for
example, the works by Bucy et al. [2],[3], who attack the nonlinear
non-Gaussian filter problem from the Bayesian viewpoint investigating
efficient and simplified methods for approximating and computing the
conditional probability density for the state. This approach assumes
perfect knowledge of the pertinent densities as well as a good feel for
how to represent these by any of a number of proposed methods.
Unfortunately, the method becomes unattractive if the state ha! higher
dimensionality than say three or four due to the large amount of digital
processing required. Another disadvantage is that-the qualitative features
of these filters tend to be obscured by the computational complexity.
We also mention the work of Alspach and Sorenson 111 who also
employ the Bayesian approach approximating the pertinent densities by
Manuscript received August 22, 1973; revised April 22, 1974. Paper recommended by
H. W. Sorenson, Chairman of the IEEE SCS Estimation and Identification Committee.
This research was supported by the NSF under Grant GK37670.
a Gaussian sum, minimizing, for example, the L2 distance. The resulting
filter could be characterized as a symphony of Kalman filters whereb,. a
set of mean values and corresponding variances are computed. The
conditional state density (as well as other densities of interest) is then
reconstructed as a Gaussian sum using the aforementioned mean values
and variances. One obvious disadvantage of this method is the problem
of findmg an appropriate Gaussian sum representation, a task of consid-
erable delicacy. A second difficulty is that some method of reducing the
number of terms in the sums approximating the densities has to be found
since this number grows exponentially with the number of steps pro-
cessed.
The one fact which stands out in reviewing the results so far in the
field of non-Gaussian filtering is that a nonGaussian minimum variance
filter usually is considerably more difficult to implement than the Kal-
man filter. As a result, a filter designer of today often will have to use the
Kalman filter even in situations where he knows that the noise distribu-
tions are significantly nowGaussian and that potentially large gains in
terms of decreased variability of the state estimate could be obtained by
a more efficient estimator. What is needed then is an estimation method
for non-Gaussim situations which is computationally attractive as well
as easy to understand and implement.
The two filter representations presented here are contributions in this
general direction. The first filter applies to situations in which the
densities for the predicted state errors are approximately Gaussian at
each stage, but the observation disturbances are non-Gaussian. The
second filter applies to the converse situation, i.e., the case of Gaussian
observation disturbances and non-Gaussian plant noises.
11. PRELIMINARIES
The following familiar problem is to be considered. We are to estimate
the n X 1 dimensional state vector x, of a dynamic system from rX I
dimensional observations (zI,z2;. . ,zk), where x, and zk satisfy the
linear state and observation relations
x k +l =Gk x k i wk
zk =Hkxk i v, ( 1 )
with +,=nXn the state transition matrix, H, =r Xn the observation
matrix, and wk and ck disturbance vectors of dimensions n X 1 and r X 1,
respectively.
As our optimal estimator we take the minimum variance estimator
which is given by the conditional expectation
P, =E ( X k l P )
where we have used the notation Z k ( z 3 , z 2 , z 3 ; ~ ~ ,+).
The assumption is made that wk and u, are zero mean independent
sequences which also are motually independent but could be non-
Gaussian. We also assume that the initial state x, is independent of the
future disturbances wk and v,. We will show that Kalman type filter
representations for 2, exist for certain non-Gaussian situations of practi-
cal interest. These filters may be characterized as Kalman filters employ-
ing nonlinear gain functions operating on the innovation vector zk -
Hk + k - l Pk - ] .
111. THE FIRST FILTER
In the following we let p( zkl Zk- ) denote the observation prediction
density (which we assume exists twice differentiable), i.e., the density of
zk conditioned on prior observations z1,z2,-. . , z k- ,. Similarly,
p( xkl Zk- ) is the predicted state density conditioned on past observa-
tions. Let prime denote the transpose. Then we have the following
theorem.
Theorem I : Assume thatp(x,lZK-) is a Gaussian density wi thmean
a( k) and covariance matrix Mk and that E {w( k) w ( j ) ) =Q(k)S,. Then
the conditional expectation Pk and its conditional covariance
ton, Seattle, Wash. 98195.
The author is azith the Department of Electrical Engineering University of Washing-
Pk A E {(2, - X,)( 2, - X,)/ Z k }