302

JOURNAL IEEE

ON SELECTED AREAS

IN COMMUNICATIONS, VOL. SAC-5, NO. 2, FEBRUARY 1987

Error Probability for Optimal and Suboptimal Quadratic Receivers in Rapid Rayleigh Fading Channels
MICHAEL J. BARRETT

Abstract-The exact performance of optimal and suboptimal quadratic receivers in a binary hypothesis test between jointly distributed zero mean complex Gaussian variates is derived. The probability of error is given as a function of the characteristic values of a generalized eigenvalue problem set up in terms of the covariance matrix of the received signal-plus-noise and, in the matrix of the quadratic form of the receiver. The results include the exact performance of various types of suboptimal receivers including those previously derived for the envelope matched filter for MFSK and the noncoherent DBPSK receiver in rapid Rayleigh fading, nonfrequency-selective channels. Also, the performance of near-optimal stationary process-long observation time [SPLOT] receiveis, energy detectors, and .other approximately optimal receivers may be calculated for noncoherent signaling in the same channel.

matched filter is less attractive. It is assumed that the received signal-plus-noise vector z defined by (2) is processed as a whole for each symbol,and that considerations of causality, end effects, etc., of the whitening filter are of little interest. The various quadratic receiver algorithms for binary reception in rapid Rayleighfading may be placed in decreasing order of complexity thus: A) optimal receivers, B) full rank approximations to the optimal receiver, C) reduced rank approximations to the optimal receiver, D) mismatched filter-squarer-integrator receivers, and E) matched filter forthe nonfading channel. The last of these is actually a linear receiver but may be put in the form of a quadratic receiver by squaring the log-likelihood and appealing to the monotonicity of the test. Algorithms of the classes A), B), and C) are known statistics receiversin that they assume a prior knowledge of the channel fading statistics. They can only be approximated, in practice, by estimating the unknown channel parameters and inserting the estimates in place of the true values in the algorithm. Algorithmsof the classes D) andE) are fixed and do not require estimators.

11. OPTIMUM RECEIVER I. INTRODUCTION Sampledbaseband.in-phaseand quadrature signals inarapid Thecharacteristicfunction ofaHermitianquadraticformin Rayleighfadingchannelaredistributedaccordingtothe-multicomplex normal variates was derived by Turin [ 11 and used by him variate complex normal density [15] which is also known as the to determine the error probability associated with binary a hypothcomplex Gaussian density esis test in diversity reception [2]. Following his method of solup(;) = exp [ - z T * ~ - z 1 , zEcN (lj tion, in a different set of circumstances, a general error formula for the binary error rate in a rapid Rayleigh fading channel is given where here. The term rapid Rayleigh fading is used to distinguish these z = [ z r j Z 22 . . . Z N - J T (2) results from those which are specialized to the slow fading limit [SFL]. However, the results here are applicable to slow fading as is an N-dimensional complex-valued vector whose real parts are well as fast fading,in which the decorrelation time To of the signal the in-phase samples from one symbol interval and whose imagirandom process is on the order of the symbol duration T. nary-valued parts are the corresponding quadrature samples. TypThe study of optimal and practical receivers for the demodulaically, the vector z refers to samples of the received signal-plustion or detection of Gaussian signals corrupted by additive Gauss- independent noise which is a complex Gaussian process. The maian noise has been carried out for over 25 years. Early discussions trix M is the covariance matrixof the sample vectors were given by Price [3], [4] and Middleton [5], and the problem M = E{UT*}. (3) was later revived and expanded upon by Kailath [6], Stratonovich and Sousulin [7], and Schweppe [8]. Kennedy [9] produced a small The covariance.matrix isan N X N Hermitian symmetric, generally volume on the subject. Bello and Nelin [lo] gave the binary error complex-valued matrix. It is also Toeplitz if the signal is a segment rates for incoherent and differentially coherent matched filters unof a stationary random process. The matrix M will be nonsingular der rapid fading conditions. and positive definite except at the extremes of slow fading and high Several communication texts have discussions of the noise in signal-to-noise,ratio, where all but one eigenvalue may go tozero. noise demodulation and detection problem. Pertinent chapters and Thecovariancematrixchangeswiththesymboltransmitted. sections may be found in Van Trees [l 11, Helstrom [12], Whalen Here we will consider such changes which do notaffect either the [13], and Hancock and Wintz [14]. determinant I M I or the trace tr ( M ) of the matrix. This will corMost of these authors pose the detection problem on the contin- respond to equal power signals on each hypothesis. Under this conuous time domain, where they deal with log-likelihoods on the form straint, the maximum aposteriori likelihood [MAP] receiver is that of nested integrals. Here we presume that the baseband in-phase which maximizes the exponent in (1). The test statistics or log likeand quadrature signals have each been sampled by an impulse sam- lihoods in a binary hypothesis test are then pler at a rate which is at least Nyquist for the two baseband filtered gi = -zT*M;z, i = 0, 1, (4) signals. Thus, in theory,noinformationislostinthesampling process. There are, in general, several samples per symbol duration and these are compared, for a given sample z, to determine the to achieve the Nyquist rate for typical noncoherent signaling forchannel decision. The sign of the differencein these statistics is all mats, MFSK, DPSK, etc. that is required to make the decision, so that one decides on The optimal receiver of Price and Kailath [4], [6] was interAg = zT* (ML - M 0 )z (5) preted by them and succeeding authors as an extension of the clas-

+/MI-

sical matched filter for deterministic signals, the so-called estimator correlator. With the sampled approach, this appeal to the

Manuscript received February 1 , 1986; revised September 30, 1986. The author is with the Department of Electrical Engineering, University of Lowell, Lowell, MA 01854. IEEE Log Number 8612179.

and chooses hypothesis zero whenAg is positive or one when it is negative. This leads to the minimum probability of error in the channel over all possible decision rules [ 161. Varioussuboptimalreceiversareusedwhichhavethesame structure as (5) but do not use the optimal statistic, that is, they are Hermitian quadratic forms in the data
Ag = zTQz

(6)

0733-8716/87/0200-0302$01 .OO 0 1987 IEEE

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS.

VOL.

SAC-5, NO. 2, FEBRUARY 1987

303

where Q is a Hermitian symmetric matrix. Among these are the envelope matchedfilter for binary signaling in NCFSK and the standard DBPSK receiver [17] when these are applied to the rapid Rayleigh fading channel. Other practical receivers, such as "energy detectors" and stationary process, long observation time [SPLOT] receivers [16] can also be posed as quadratic forms. The exact performance of any of these receivers may be obtained from the result given in Section 1 1 1 .

The envelopematched filter receiver forbinary NCFSK is a special case of (16) as is a standard noncoherent DBPSK receiver and an optimized version of this suggested by Wittwer [ 191. For this case (9) becomes

(ppT*- qqT*) wi = ~ - ' w ~ q ,i = 1, 2,

If both sides of (9) are premultiplied by M

. . ,N .
* *

(17)
(18)

M(ppT* - qqT*) wi = wiai,

i = 1 , 2,

,N ,

111. ERRORPROBABILITY The probability of error in using the text statistic (6) is derived in Appendix A. The result is

P,'

a;<O n = l n+i

cn

1 1 - u,/u~

(7)

where theset { a i , i = 1 , 2 , * , N } arethe eigenvalues ofqhe matrix MoQ. Here we assume that the "zero" hypothesis is active and that the "one" hypothesis was chosen. If the test is symmetric, the same error probability holds for each hypothesis; if it is not symmetric, the channel error probability may be obtained by averaging the P, on each hypothesis. Let us consider only the symmetric hypothesis test, the nonsymmetric case follows readily. The subscript on M will be dropped. The eigenvalues { ai } must satisfy

it becomes evident that there will be at most two nonzero eigenvalues because the matrix of (18) is the sum of two rank-one matrices: The error probability (7) will take three possible forms depending on the signs of the two nonzero eigenvalues,if both are negative, P, goes to one. If both are positive it goes to zero. Thus, in the case of practical interest, the two eigenvalues have opposite sign. Let the negative eigenvalue be a , , and the positive be a2. Then

MQwi = w i a i ,

i = 1, 2,

. ,N

(8)

where { wi } are the associated eigenvectors. If M is positive definite (and hence, nonsingular) which it is in nearly all cases of interest, (8) is equivalent to

1 (19) 1 - %/a'' where the denominator is guaranteed to be greater than one. The P, depends only on the ratio of the two nonzero eigenvalues. In another 'significant class of receiver, Q has a diagonal form. A discrete-time version of the stationary process, long observation time [SPLOT] receiver, discussed by Van Trees[ 1 11, falls into this category. The covariance matrix M is of the DFT coefficients of the received signal plus noise, rather than of the time samples, but the density (1) applies. Here (9) takes the form

P, =

@ v i= M-'wiai,

i = 1, 2,

* *

,N

(9)
(10).

@ v i =

M-'wiai,'

i = 1, 2,

,N

(20)

for which the eigenvalues are the solution to the equation

I Q - uM-' I = 0.

This type of eigenvalue problem is treated in Gantmacher [18], where it is shown that (10) has N real roots and that the eigenvectors may be chosen to satisfy

with Q diagonal, and perhaps with some zeros on the diagonal. Here a simple way to obtain the eigenvalues is to form the Hermitian matrix Q1'*MQ1/2, which has the same eigenvalues as MQ, and use a routine designed for Hermitian symmetric matrices. V. CONCLUSIONS The error formula (7) appears to be a very useful result. One may use this equation to study the performance for both optimal and mismatched fixed and adaptive receivers in the rapid Rayleigh fading channel. The results extend nicely to the SFL. Previous work in this channel has required the use of Monte Carlo routines to obtain the binary error rate [BER]. These random integration methods are not useful for generatingnew types of receiver structure or to evaluate these easily. It is also possible to use (7) in an approximation of the MFSK performance by assuming that the great majority of errors are adjacent tone errors. For instance, using such an approximation in an 8-ARY FSK channel, one obtains

wr"-'wk

tiik,

i , k = 1, 2,

. . ,N .

(11)

In effect, the inverse covariance matrix forms a metric for thevector space under which the eigenvectors are orthogonal. IV. OPTIMAL RECEIVER PERFORMANCE For the optimal [MAP] receiver
Q =

M;' - M 0'

(12)

so that, for the optimal performance (9) becomes

( M ; ' - M i 1 )wi = M o l w i ~ i , i = 1 , 2,
or

* *

,N

(13)

P,

14 Pb,

8-ARY FSK

(21 1

M ; ' w ~= ~ ; ' w ~ + ( aI )~,

i = 1, 2 ,

. . ,N .

(14)

It is not necessary to invert the two covariancematrices to compute these eigenvalues, as it is readily demonstrated that, because the eigenvalues are real valued and each covariance matrix is Hermitian, they also satisfy

where P, is the octal symbol error probability and Pb is the probability of binary error for the same channel, with the same frequency separation, etc. Thereis a 2Pb probability of adjacent channel error for each of the six internal tones, and a Pb probability of error for eachof the two end tones. APPENDIX A DERIVATION OF ERROR PROBABILITY The characteristic function
+ ( t ) = E{exp [ j t z T * e Z ] }

Mow,! = Mlwl (ai + l ) ,

i = 1, 2,

,N .

(15)

Where the eigenvectors { wl } are orthogonal under the metric induced by M I .

of the distribution of the test statistic

V. SUBOPTIMALRANK-TWO RECEIVERS Consider now a class of suboptimal receivers in which the mais given by [l] trix Q is in the form of the difference of two dyads or outer products
Q = ppT* - qqT*.
(16)

Ag
N

zT*@

+(t) =

n= 1

( 1 -$a,)-',

304

IEEEJOURNAL

ON SELECTEDAREASINCOMMUNICATIONS,

VOL. SAC-5, NO. 2, FEBRUARY 1987

j times the sum of the remaining UHP residues

P,

a ; < ~

1 II n = ~ 1 - a,/ai
n f i

(A.13)

REFERENCES
of Hermitianquadratic [l] G. L. Turin,Thecharacteristicfunction forms in complex normal variables, Biometrika, vol. 47, pp. 199201, June 1960. [2] -, On optimal diversity receiption, IRE Trans. Commun. Syst., VOI. CS-7, pp. 154-166, July 1961. [3] R. Price, The detection of signals perturbed by scatter and noise, IRE Trans. Inform. Theory, vol. IT-4, pp. 163-170, Sept. 1954. [4] -, Optimum detection of random signals in noise, with applicaI, IRE Trans. Inform. tionstoscatter-multipathcommunicators, Theory, vol. IT-4, pp. 125-135, Dec. 1956. Fig. 1. Contourintegral. [5] D. Middleton, On the detection of stochastic signal in additive normal noise, I, IRE Trans. Inform. Theory, vol.IT-3,pp.86-121, June 1957. [6] T. Kailath, Correlation detection of signals perturbed by a random where { an} are the eigenvaluesof the matrix MQ. The covariance channel, IRE Trans. Inform. Theory, vol. IT-6, pp. 361-366, June matrix of the received signal is M,whereas the matrix of the qua1960. dratic receiver is Q . Both M and Q are Hermitian matrices, and M [7] R. L. Stratonovich and Y. G. Sosulin, Optimal detection of a Maris positive definite. The matrixQ may be singular in general. Under kov process in noise, Eng. Cybern., vol. 6, pp. 7-19, Oct. 1964. these circumstances, the eigenvalues { a , 1 are real valued, but may [SI F. Schweppe, Evaluation of likelihood functions for Gaussian sigbe positive or negative [ 2 ] . nals, IEEE Trans. Inform. Theory, vol. IT-11, pp. 61-70, Jan. 1965. The probability of error is the integral of the density of the test [9]R. S. Kennedy, Fading Dispersive Communication Channels. New statistic Ag over the half-line Ag < 0 York: Wiley, 1969. [lo] P. A. Bello and B. D. Nelin, The influence of fading spectrum on 0 the binary error probabilities of incoherent and differentially coherent pe = -m P ( & ) d& (A.4) matched filter receivers, IRE Trans. Commun. Syst., vol. CS-8, p. 160, June 1962. m [Ill H. L. Van Trees, Detection Estimation and Modulation Theory, part =! + ( t ) exp [ -jtg] dr dAg (-4.5) 111. New York: Wiley, 1971. 2~ - m - m [12] C. W.Helstrom, Statistical Theory of Signal Detection. London, om rm I England: Pergamon, 1968, ch. 11. =+(t) exp [ - j t g ] dAg dr [13]A.D.Whalen, Detection of Signals in Noise. New York: Aca27r --m 0 demic, 1971, pp. 380-382. [14] J. C. Hancock and P. A . Wintz, Signal Detection -Theory. New York: McGraw-Hill, 1966, ch. 7 and 8. [15] R. A. Wooding, The multivariate distribution of complex normal variables, Biometrica, vol. 43, 1956. 1 1 ---( 1 - j t a n ) - ] ( j t ) - dt. (A.8) [16] H. L. Van Trees, Detection Estimation and Modulation Theory, vol. 2 2 7 F =I I. New York: Wiley, 1971. [17] M. Schwartz, W. Bennett, and S . Stein, Communication Systems and The remaining integralmay be solved by closing the path of Techniques. New York: McGraw-Hill, 1966. integration around the upper half-plane and using the residue theo[18] F. R. Gantmacher, ?e Theory of Matrices. NewYork: Chelsea, rem. The contour is illustrated in Fig. 1. The integrand vanishes 1977. by at least the order 1 / 1 z 1 for large I z I where the matrix M Q has [19] L. A. Wittwer, private communication, June 1979. at least one nonzero eigenvalue. Thus,

lo s

-0

[n

The integrand may be rearranged to the form

Adaptive Receiver for Digital Communication Over HF Channels

J. M. PERL,A.SHPIGEL,
AND

A. REICHMAN

for which the residues are Res { F ( z ) , o } = -j Res ( F ( z ) , - j a ; }

=
n=l

(A.ll) 1 +j . - q/an (A.12)

nfi

The pole at the origin contributes 27rj times ( -; ) of its residue to the principal value of the integrand when its contribution is evalManuscript received January 28, 1986; revised August 4, 1986. uated by an indented contour in the clockwise direction about the The authors are with the Communication Division, TADIRAN, Inc., origin as indicated in Fig. 1. The result is to cancel the leading P.O. Box 267, Holon 58 102, Israel. term of ( $ )in (A.9). The probability of error is then, from (A.9), IEEE Log Number 8612180. 0733-8716/87/0200-0304$01.OO 0 1987 IEEE

Abstract-The fading characteristics of the HF channel make communication over this channel difficult, even at moderate data rates. In this correspondence, a practical communication system is presented which has a near-optimal receiver for data detection and adaptive optimization of the receiver to channel parameters. The channel tracking is performed during the training sequence as well as during the detected data sequence. The receiver is fast converging, therefore suita-