PRENTICE HALL SIGNAL PROCESSING SERIES Alan V. Oppenheim, Series Editor ANDREWS AND HUNT Digital Image Restoration Baicuam The Fast Fourier Transform BRIGHAM The Fast Fourier Transform and Its Applications BURDIC Underwater Acoustic System Analysis, 2/E CASTLEMAN Digital Image Processing CowAN AND Grant Adaptive Filters CROCHIERE AND RABINER Multirate Digital Signal Processing DUDGEON AND MERSEREAU Multidimensional Digital Signal Processing HamMine Digital Filters, 3/E HAYKIN, BD. Advances in Spectrum Analysis and Array Processing, Vols. 1 & I HAYKIN, ED. Array Signal Processing JAYANT AND NOLL Digital Coding of Waveforms JONSON AND DUDGEON Array Signal Processing: Concepts and Techniques Kay Fundamentals of Statistical Signal Processing: Estimation Theory Kay Modern Spectral Estimation : Kino Acoustic Waves: Devices, Imaging, and Analog Signal Processing LEA, ED. Trends in Speech Recognition Lim’ Two-Dimensional Signal and Image Processing Lim, eb. Speech Enhancement Lim AND OPPENHEM, EDS. Advanced Topics in Signal Processing MaRpLe Digital Spectral Analysis with Applications MCCLELLAN AND RADER Number Theory in Digital Signal Processing MENDEL Lessons in Digital Estimation Theory OPPENHEIM, ED. Applications of Digital Signal Processing OpPeNteim AND NawAB, EDS. Symbolic and Knowledge-Based Signal Processing Oppenuens, WILLSKY, WITH YOUNG Signals and Systems OPPENHEIM AND SCHAFER Digital Signal Processing OPPENHEIM AND SCHAPER Discrete-Time Signal Processing QUACKENBUSH ET AL, Objective Measures of Speech Quality RABINER AND GOLD Theory and Applications of Digital Signal Processing RABINER AND ScuaFER Digital Processing of Speech Signals ROBINSON AND TREITEL Geophysical Signal Analysis STEARNS AND DaviD Signal Processing Algorithms STEARNS AND HusH Digital Signal Analysis, 2/E TRIBOLET Seismic Applications of Homomorphic Signal Processing VAIDYANATHAN Multirate Systems and Filter Banks WIDROW AND STEARNS Adaptive Signal Processing Fundamentals of Statistical Signal Processing: Estimation Theory 4493 Steven M. Kay University of Rhode Island con AE, PTR Upper Saddle River, NJ 07458Contents Preface xi 1 Introduction 1 1,1 Estimation in Signal Processing 1 1.2. The Mathematical Estimation Problem 7 1.3 Assessing Estimator Performance 9 14 Some Notes to the Reader : 2 2. Minimum Variance Unbiased Estimation 18 2. Introduction 15 22. Summary 15 213 Unbiased Estimators 16 24 Minimum Variance Criterion : 19 25 Existence of the Minimum Variance Unbiased Estimator 20 26 Finding the Minimum Variance Unbiased Estimator 21 2.7 Extension to a Vector Parameter 2 3 Cramer-Rao Lower Bound ar 3.1 Introduction 7 32 Summary 7 3.3 Estimator Accuracy Considerations ‘i 8 34 Cramer-Rao Lower Bound 30 3.5. General CRLB for Signals in White Gaussian Noise : 35 36 Transformation of Parameters a7 37 Extension to a Vector Parameter 39 3.8. Vector Parameter CRLB for Transformations 45 3.9 CRLB for the General Gaussian Case 47 3.10 Asymptotic CRLB for WSS Gaussian Random Processes .- 50 43.11 Signal Processing Examples 33 3A. Derivation of Scalar Parameter CRLB 87 SB Derivation of Vector Parameter CRLB 0 3C Derivation of General Gaussian CRIB... 3 3D Derivation of Asymptotic CRLB. , 7= contenrs 4 Linear Models 4.1 Introduction 4 42 Summary a 83 4.3. Definition and Properties 83 4.4 Linear Model Examples : 36 4.5. Extension to the Linear Model Fi 94 5 General Minimum Variance Unbiased Estimation 101 5.1 Introduction 10l 5.2 Summary 101 5.3 Sufficient Statistics 102 5.4 Finding Sufficient Statisties : 104 5.5. Using Sulficiency to Find the MVU Estimator 107 5.6 Extension to a Vector Parameter 16 5A. Proof of Neyman-Fisher Factorization ‘Theorem (Scalar Parameter) |_| 197 5B Proof of Rao-Blackwell-Lehmann-Scheffe Theorem (Scalar Parameter) | 130 6 Best Linear Unbiased Estimators 133, 6.1 Introduction . 133 62 Summary : 133 6.3 Definition of the BLUE 134 6.4 Finding the BLUE : 136 6.5 Extension to a Vector Parameter 139 6.6 Signal Processing Example ai 6A Derivation of Scalar BLUE 151 6B Derivation of Vector BLUE 153 7 Maximum Likelihood Estimation 157 7.1 Introduction 72 Summary 7.3 An Example 74 Finding the MLE 7.5 Properties of the MLE 7.6 MLE for Transformed Parameters 1.7 Numerical Determination of the MLE 7.8 Extension to a Vector Parameter 7.9 Asymptotic MLE 7.10 Signal Processing Examples 7A Monte Carlo Methods : 7B Asymptotic PDF of MLE for a Scalar Parameter TC Derivation of Conditional Log-Likelihood for EM Algorithm Example 8 Least Squares 219 8.1 Introduction a 82 Summary ae CONTENTS 83 The Least Squares Approach 84 Linear Least Squares 85 Geometrical Interpretations 86 Order-Recursive Least Squares 87. Sequential Least Squares 88. Constrained Least Squares 89 Nonlinear Least Squares 8.10 Signal Processing Examples 3A. Derivation of Order-Recursive Least Squares 8B Derivation of Recursive Projection Matrix 8C Derivation of Sequential Least Squares 9 Method of Moments 9.1 Introduction 9.3. Method of Moments 9.4 Extension to a Vector Parameter 9.5 Statistical Evaluation of Estimators 96. Signal Processing Example 10 The Bayesian Philosophy 101 Introdvetion 102 Summary ; 10.3 Prior Krowiedge and Extimation {04 Choosing a Prior PDF 10.5 Properties of the Gausian PDF 10.8 Bayesian Linear Model, 107 Nuisance Parameters pee 10.8 Bayesian Eatimation for Deterministic Parameters TOA Derivation of Conditional Gaussian PDF 11 General Bayesian Estimators 11.1 Introduction i 112 Summary 13 Risk Functions Boat 1L4 Minimum Mean Square Error Estimators 11.5 Maximum A Posteriori Estimators 11.6 Performance Description 117 Signal Processing Example TIA Conversion of Continuous-Time System to Discrete-Time System 12 Linear Bayesian Estimators 121 Introduction 122 Summary... 123 Linear MMSE Estimation 20 223 226 232 242 251 254 260 282 285 286 289 289 280 289 292 204 209 309 309 310 316 321 325 32812.4 Geometrical Interpretations 12.5 The Vector LMMSE Estimator 12.6 Sequential LMMSE Estimation 127 Signal Processing Examples - Wiener Filtering 12A. Derivation of Sequential LMMSE Estimator 43 Kalman Filters 18.1 Introduction . 182 Summary . 13.3 Dynamical Signal Models 184 Scalar Kalman Filter 13.5 Kalman Versus Wiener Filters 136 Vector Kalman Filter . 13.7 Extended Kalman Filter 188 Signal Processing Examples ASA Vector Kalman Filter Derivation 13B Extended Kalman Filter Derivation 14 Summary of Estimators 14. Introduction 142 Estimation Approaches 143 Linear Model 144 Choosing an Estimator 15 Extensions for Complex Data and Parameters 15.1 Introduction . . 15.2 Summary : 15.3 Complex Data and Parameters 15.4 Complex Random Variables and PDFs 15.5 Complex WSS Random Processes 15.6 Derivatives, Gradients, and Optimization 15.7 Classical Estimation with Complex Data 15.8 Bayesian Estimation 15,9 Asymptotic Complex Gaussian PDF 15.10Signal Processing Examples 15A. Derivation of Properties of Complex Covariance Matrices 15B_ Derivation of Properties of Complex Gaussian PDF 15C Derivation of CRLB and MLE Formulas A1 Review of Important Concepts ‘ALLL Linear and Matrix Algebra 5 ‘AL2 Probability, Random Processes, and Time Series Models, A2 Glossary of Symbols and Abbreviations INDEX CONTENTS 384 389 302 4400 415 419 a9 486 489 493 493 493 494 500 513 517 524 332 535 539 355 558 563 567 367 am 583 589 Preface Parameter estimation is a subject that is standard fare in the many books available or statistics. These books range from the highly theoretical expositions written by Statisticians to the more practical trestments contributed by the many users of applied Statistics. "This text is an attempt to strike a balance between these two extremes. ‘Toe particular audience we have in mind is the community involved in the design ‘and implementation of signal processing algorithms. As such, the primary focus is on obtaining optimal estimation algorithms that may be implemented on a digital opiter “The data fel a7 therebre aatumed To be samples of a continaots-time Saveform or a sequence of data points, The choice of topics reflects what we believe te be the important approaches to obtaining an optimal estimator and analyzing its performance. AS a consequence, some of the deeper theoretical issues have been omitted with references given instead. Tt is the author's opinion that the best way to assimilate the material on parameter estimation is by exposure to and working with good examples. Consequently, there are ‘nnmerous examples that illustrate the theory and others that apply the theory to actual signal processing problems of current interest. Additionally, an abundance of homework problems have been included. They range fom simple applications of the theory to ‘extensions of the basic concepts. A solutions manual is available from the publisher. TD aid the reader, summary sections have been provided at the beginning of each chapter. Also, an overview of all the principal estimation approaches and the rationale fer choosing a particular estimator can be found in Chapter 14, Classical estimation Se TESTS Chapters 2 followed by Bayesian estimation in Chapters 10-13. This delineation will, hopefuly, help to clarify the basic differences between these two principal approaches. Finally, again in the interest of clarity, we present the estimation ‘Principles for scalar parameters first, followed by their vector extensions. This is because the matrix algebra required for the vector estimators can sometimes obscure the main concepts. "This book is an outgrowth of a one-semester graduate level course on estimation theory given at the University of Rhode Island. It includes somewhat more material than ean actually be covered in one semester. We typically cover most of Chapters 1-12, leaving the subjects of Kalman filtering and complex data/parameter extensions tothe student, ‘The necessary background that has been assumed is an exposure to the basic theory of digital signal processing, probability and random processes, and linearxii PREFACE. and matrix algebra. This book can also be used for self-study and so should be useful to the practicing engineer as well as the student. “The author would like to acknowledge the contributions of the many people who over the years have provided stimulating discussions of research problems, opportuni- ties to apply the results of that research, and support for conducting research. Thanks fare due to my colleagues L. Jackson, R. Kumaresan, L, Pakula, and D. Tufts of the University of Rhode Island, and L. Scharf of the University of Colorado. Exposure to practical problems, lading to new research directions, has been provided by H. Wood- eam of Sonetech, Bedford, New Hampshire, and by D. Mook, S. Lang, C. Myers, and. ‘D. Morgan of Lockheed-Sanders, Nashua, New Hampshire. The opportunity to apply ‘cetimation theory to sonar and the research support of J. Kelly of the Naval Under- Sea Warfare Center, Newport, Rhode Island, J. Salisbury of Analysis and Technology, Middletown, Rhode Island (formerly of the Naval Undersea Warfare Center), and D. Sheldon of the Naval Undersea Warfare Center, New London, Connecticut, are also greatly appreciated. Thanks are due to J. Sjogren of the Air Force Office of Scientific Ficccarch, whose continued support has allowed the author to investigate the field of Statistical estimation. A debt of gratitude is owed to all my current and former grad- Jate students. They have contributed to the final manuscript through many hours of peilagogical and research discussions as well as by their specific comments and ques- Fons’ In particular, P. Djurié of the State University of New York proofread much Of the manuseript, and V, Nagesha of the University of Rhode Island proofread the ‘manuscript and helped with the problem solutions. Steven M. Kay University of Rhode Island Kingston, RI 02881 See Chapter 1 Introduction 1.1 Estimation in Signal Processing Modern estimation theory can be found at the heart of many electronic signal processing [Systems designed to extract information. These systems include a 1, Radar 2, Sonar 3. Speech 4, Image analysis, 5, Biomedicine 6, Communications 7. Control 8, Seismology, and all share the common problem of needing to estimate the values of a group of pac ‘airaters, We briely describe the frst three of these systems. In radar we are interested Fr cete@bining the position of an aircraft, as for example, in airport surveillance radar [Skolnik 1980]. To determine the range F we transmit an electromagnetic pulse that is Cera a ae ee alc, where cho to be received by the antenna 7) seconds later, fe ihown in Figure Ta The range i determined by the equation 7» = 2R/c, where ¢ is the speed of electromagnetic propagation. Cleary, ifthe round trip delay ro can bbe measured, then so can the range. A typieal transmit pulse and received waveform areshown in Figure 1.1b. The received echo is decreased in amplitude due to propaga tion losses and henee may be obscured by environmental nose. Tes onset may also be perturbed by time delays introduced by the electronics of the receiver. Determination ST the round tip delay tan Therefore require more than just a means of detecting a jump in the power level at the receiver. It is important to note that e typical modern 12 CHAPTER 1. INTRODUCTION Radar procoug (a) Radar ‘Tanai ple (b) Transmit and received waveforms Figure 1.1. Radar system radar system will input the received continuous-time waveform into a digital computer by taking samples via an analog-to-digital convertor. Once the waveform has been sampled, the data compose a time series. (See also Examples 3.13 and 7.15 for a more detailed description of this problem and optimal estimation procedures.) Another common application is in sonar, in which we are also interested in the position of a target, such as a submarine [Knight et al. 1981, Burdic 1984) . A typical Passive sonar is shown in Figure 12a. The target radiates noise due to machiner ‘on board, propellor action, ete, This noise, which is actually the signal of interest, ropagates through the water and is received by an array of sensors. The sensor outputs 1. ESTIMATION IN SIGNAL PROCESSING 3 x SS — Sen bt (a) Passive sonar Sexsor Boutpat Taw (b) Received signals at array sensors Figure 1.2 Passive sonar system ‘are then transmitted to a tow ship for input toa digital computer, Because of the SUaiuoay Of the tensow latte to the arial angle Of the target signal, we fezeve the signals shown in Figure 1.2b, By measuring 7, the delay between sensors, we can deermine the bening 8 rom the expression = aco (!) (uy where c is the speed of sound in water and d is the distance between sensors (see Examples 3.15 and 7.17 for a more detailed description). Again, however, the received(CHAPTER 1. INTRODUCTION Vowel [a/ i Cee a a ee Lae neal seagate a) Time (2s) Figure 1.3 Examples of speech sounds ‘waveforms are not “clean” as shown in Figure 1.2b but ate embedded in noise, making the determination Of % more dificult. The value of J obtained from (1.1) is then only mainte ‘SnotheF application is in speech processing systems [Rabiner and Schafer 1978] 'A particularly important problem is speech recognition, which is the recognition of ‘speech by a machine (digital computer). The simplest example of this isin recognizing Thdividual speech sounds or phonemes. Phonemes are the vowels, consonants, ¢tc. oT The fundamental sounds of speech. As an example, the vowels 7a/ and /e/ are shown in Figure 1.3. Note that they are periodic Savers whose petid is called the pitch 1, ESTIMATION IN SIGNAL PROCESSING 5 ° 0 100 1800 200 200 Frequency (H2} 104 ° 0 ‘moO Frequency (Hs) Figure 1.4 LPC spectral modeling rinimizes some distance measure, Difficulties arise ifthe pitch of the speaker's vice anges Trom Ure te he oF she records the sounds (the training session) to the time Wrist ihe speech recognizer is used. This is a natural variability due to the nature of hhuman speech, In practice, attributes, other than the waveforms themselves, are used fomeasure distance. Attributes are chosen that ar less sustentible-to variation. For zample, the spectral envelope wil not change with pitch since the Fourier transform Fe perodie gnal ws sampled version of the Fourier transform of one period ofthe Signal. The period afects only the spacing between frequency samples, not the values ‘Toextract the spectral envelope we employ a model of speech called linear predictive ‘To recognize whether a sound is an /a/ oF an Je/ the following simple strategy might de employed. Have the person whose speech is to be recognized say each vowel three times and store The waveforms. To recognize the spoken vowel, compare it to the i oe Tea a an tha soot 0 the spoken vowel of the one thal Syecch sounds tn Figure 3 the power spectrum (magnitude-squared Fourier transform 0 models this). ‘The assumption of WGN is justified by the raat to formulate a mathematically tractable model so that closed form estimators can Ie tbund. Also, itis reasonable unless there is strong evidence to the contrary, such as formance of any estimator obtained will be highly correlated noise. Of course, the critically dependent on ee ee er We gaa only hope the estimator obtained ‘Frobust~in that sight changes in the SEP aa nt evel aft ts performance ofthe eee. More conservative approaches utilize robust statistical procedures [Huber 1981), Djtimation based on PDFs such as (1.3) is termed classical estimation in that the paremetersoficterest are assumed to be deterministic but unknown, In the Dow-Jones Paretge example we know a priori that the mean is somewhere around $000. It seems aera at enth reality, then, to choose an estimator of A that can result in values as Tea me 2000 or as high as 4000. We might be more willing to constrain the estimator reproduce values of A in the range [2800, 3200), To incorporate this pplor knowledge coerce assume that A is no longer deterministic but a random variable and assign it BBe SS antom over the 0 * Scsibly uniform over the (2800, 200) interval. Then, any subsequent estimator “jield value in this range. Such an approsch is termed Bayesian estimation. The Trarayneter we are attempting to estimate is then viewed as a realization of the random ‘aria aiid, the data are described by the jomnt PDF (x, 8) = p(xlP)P(0) schere p(@) is the prior PDF, summarizing our knowledge about 0 before any data ee ant Had) is a conditional PDF, summarising our knowledge provided the data x conditioned on knowin Feader should compare the notation Oe a a ore pixié) (a lamily of PDFs) and p(xlé) (a conditional PDF), as well {3s the Implied interpretations (see also Problem T. 1.4. ASSESSING ESTIMATOR PERFORMANCE 9 10+ 154 19. 004 os+ as 0 Stein 10 rao oe Roo evo a wo ets tc Figure 1.7 Realization of DC level in noise oO eT optimal estimator of Function of the data, asin (1.2). Note that an estimator ma Sen erg only tite ae foown An caiman ay be hough Se ee ia ‘The estimate of @ is Genaie oF oblained fora given realization of x. This distinction is analogous to fandom variable (which isa function defined on the sample space) and the value it takes se Although some authors distinguish between the wo by using capital and lowercase Tener, we will not do so. The meaning will, hopefully, be lear from the context. 13 Assessing Estimator Performance Case ih delat sown i Fgue 17. From» cua lespecia & appear Spee onpe yop Se pe rit apie mie Fe nee ert te tacoma faction) We cad model the dot se z{n] erp dnto ome sro mean ns proces, Based nthe data st (=f FRe iy ae Abe avege lvl ote (u(n] is zero mean), it would be reasonable to estimate A as it we cr by the sample mean of the data, Several questions come to mind: \ 1. How close will d be to 4? 2. Are there better estimators than the sample mean?10 CHAPTER 1. INTRODUCTION For the data set in Figure 1.7 it turns out that A = 0.9, which is close to the true value of A=. Another estimator might be A=2(0) Intuitively, we would not expect this estimator to perform as well since it does not Ineuitiveys al the data, There is no averaging to reduce the nose effects. However ane evga oot in Figure 1.7, A= 0.95, which is closer to the true value of A than soe eae ae muean estimate. Gan we conclude that A is a better estimator than A? ae aerene vie of course no. Because an estimator is a function of the data, which ae ae a jables, it too is a random variable, subject to many possible outcomes. are drat ais closer to the true value only means that for the given realization of Te at Showa in Figuse 17, the estimate A= 0.95 (or realization of A) is closer to rote oe valve than the estimate A = 0.9 (or realization of A). To assess performance a ao rec eatistically. One possibility would be to repeat the experiment that SESSA the dats and apply each estimator to every dataset, ‘Then, we could ask Ferrata ie erodes a better estimate inthe majority of the cases. Suppose we wae the experiment by fixing A= Land adding diferent noise realizations of win] to eae asa cuumble of realizations of zin]. Then, we determine the values of the two aerate ator each data set and finaly plot the histograms. (A histogram deseribes the ca once the estimator produces a given range of values and is an approximation snes PDE) For 100 realizations the histograms are shown in Figure L8. Tt should vor be evident that Ais a better estimator than A because the values obtained are ae radrated about the true value of A = 1. Hence, A will usually produce a value rae ee the teue one than A. The skeptic, however, might aguethat if we repeat the et tbe 1000 tunes instead, then the histogram of A will be more concentrated, To Sree ths notion, we cannot repeat the experiment 100 times, for surely the skeptic Cr acctsct his ot het conjecture for 10,000 experiments. ‘To prove that A is Tould (ee rod eotabish that the variance is less. The modeling assumptions that we aie we oy are that the w[nl's, in addition to being zero mean, are uncorrelated and ‘have equal variance o?. Then, we first show that ‘the mean of each estimator is the true value or Bia) = 2(#-h) Les = wh Bee A E(4) = E(2(0)) A so that on the average the estimators produce the true value. Second, the variances are 13. ASSESSING ESTIMATOR PERFORMANCE i i] L! i Sample mean value, 4 First sample value, A Figure 1.8 Histograms for sample mean and Bret sample estimator tes wd, wet) since the w)n)'s are uncorrelated and thus var(A) = var(z(0)) > var(A),12 CHAPTER 1, INTRODUCTION Furthermore, if we could assume that w[n] is Gaussian, we could also conclude that the probability of given magnitude error is less for A than for A (see Problem 2.7) is evetal important points |are illustrated by the previous example, which should Sa pt in min 1. An estimator is a random variable. As such, its performance can only be com Setely destibad stalistically oF by Tis PDF. soe 2. The use of computer simulations for assessing estimation performance, although quite valuable for gazing Tasight and motivating conjectures, is never conchsive, Jr ect, the true performance may be obtained to the desired degree of accuracy. At best he te cient number of experiments and/or ero coors the simulation SSE caployed erroneous results may be obsained (eee Appendix 7A for a Fae Siestmbor ot Monte Carlo computer tecbnives). Another theme that we will repeatedly encounter is the tradeoff between perfor- maice and computational complexity. AS in the previous example, even though A fas Letter performance, itaso requires more computation, We will see that optimal ‘cotimators can sometimes be difficult to implement, requiring « multidimensional opti: ‘Seaton or Tategration. In these situations, alternative estimators that are suboptimal Hee nih can be implemented on a digital computer, may be preferred. For any par Fiedler application, the user must determine whether the loss in performance is ofset by the reduced computational complexity of a suboptimal estimator. 1.4 Some Notes to the Reader - Cur philosophy in presenting theory of estimation isto provide the user with the So Ne ebaing-opUndl atimatar We have included resus aa re er nt wcll n pacice,omiting some portant theoretical ses ate om fo found im many. books on statistical eatimation theory which have aan ite toa ¢ more theoretical viewpoint (Cox and Hinkley 1974, Kendall and seen Lom, Rao 19TH, Zacks 1961|- As mentioned previously, our goal i to Sean oe inal cana, and we resort to a suboptimal one i the former cannot obtain an opti Soe af captors Te book follows ts ‘approach, so that optimal estimators are discussed first, followed by approximately SPPRoAET ator and Snaly suboptimal estimators. In Chapter 14 “road map" for SoA ettnator in presented along with a summary ofthe various extimators fining 6000 eitice The ven may wish to read this chapter fist 10 obtain an Tee tried to maximize insight by including many examples and minimising rong cahenataleeponion, although much of he tedious algebra and proofs have one tas appendices, ‘The DC level in noise described earlier will serve 36 2 oer ace aole introducing almost all the estimation approoches. Tt is hoped dana ena ender wil beable to develop his or her own intuition by building cade Sy aasimllated concepts, Also, where posible, the scalar estimator is [REFERENCES 7 presented first followed by the vector estimator. ‘This approach reduces the tendency Pirvector/matrix algebra to obscure the main ideas. Finally, classical estimation is Gescribed first, followed by Bayesian estimation, again in the interest of not obscuring the main issues. The estimators obtained using the two approaches, although similar in appearance, are fundamentally different. “The mathematical notation for all common symbols is summarized in Appendix 2. ‘The distinction between a continuous-time waveform and a discrete-time waveform or sequence is made through the symbolism 2(¢) for continuous-time and zn} for discrete- tine. Plots of z(n], however, appear continuous in time, the points having been cou- fected by straight lines for easier viewing. All vectors and matrices are boldface with al vectors being column vectors. All other symbolism is defined within the context of, the discussion. References Ber, GEP, GM. Jenkins, Time Series Analysis: Forecasting and Controk, Hokden-Day, San Francisco, 1870 Burdie, W'S. Underwater Acoustic System Analysis, Prentice-Hall, Englewood Cli, N.J., 1984 Ger, DIR, DV. Hinkley, Theoretical Statistics, Chapman and Hall, New York, 1974 Dibbous, TE. N.U. Ahied, J.C. MeMillan, DF. Liang, "Filtering of Discontinuous Processes “Arising in Macine Integrated Navigation,” ZEEE Trons, Aerosp. Electron. Syst, Vol. 24, pp. 85-100, 1988 Guus, K.G., Theory of Motion of Heavenly Bodies, Dover, New York, 1968. Haim 5. JM. Hovem, “Estimation of Sealar Ocean Wave Spectra by the Maximum Entropy ‘Method," IEEE J. Ocean Eng. Vol. 4, pp. 76-8, 1979. Huber, Pi, Robust Statistics, J. Wiley, New York, 1981 Jin, AK, Fundamentals of Digital Image Procesring, Prentice-Hall, Englewood Clif, N.., 1988 Statice, Jit, "Array Procesing in Exploration Seismology,” in Array Signal Processing, S. Haykin, ‘ds, Prentice-Hall, Englewood Clif, N.J., 1985. Kendall, Sir My A. Stuart, The Advanced Theory of Statistics, Vols. 1-3, Macmillan, New York, 1976-1979. Kaight, WS, R.G. Pridbam, SM. Kay, “Digital Signal Processing for Sonar,” Proc. IEEE, Vol 169, pp. 1451-1506, Nov. 1981. Proakie, 1G, Dugital Communtontions, MeGraw-Hil, New York, 1983, Ratner LL, Str, Dial Prong of Spek Siga, Penton Hal, ngewe Ci “1978 Reo, CR. Linear Statistical Inference and [ts Applications, J. Wiley, New York, 1973 ‘Schuster, A “On the Investigation of Hidden Petiodictiee with Application toa Supposed 26 Day Period of Meterological Phenomena," Terrestrial Magnetism, Vol. 3, pp. 13-41, March 1898, Stolnik, MLL, Jutrduction to Radar Systems, McGraw-Hill, New York, 1980, ‘Taylor, 8. Modeling Financial Time Series, J. Wile, New York, 1986 Widrow, B, Stearns, SD, Adaptive Signal Processing, Prentice-Hall, Englewood Cliff, NJ, 1985, Zacks, 8, Parametric Statistical Inference, Pergamon, New York, 1981