A Soft Decision-Directed LMS Algorithm for Blind Equalization Steven J. Nowlan and Abstract—We propose a new adaptation algorithm for equal ers operating on very distorted channels. The algorithm fs based fn the iden of adjusting the equalizer tap gains to maximize the Wlihood that the equalizer outputs would be generated by A misture of two gauslans with known means. The familar Gecsion directed least mean square (LMS) algorithm is shown {o be an approximation to masiming the Wkeliiood tha the ‘equalizer outputs come from such an id source. The algorithm [s developed in the contest ofa binary PAM chanel and simula tions demonstrate thatthe new algorithm converges in channels Tor which the decision-directed LMS algorithm does not converse. |. twrRopuction HE system which we investigate in this paper is illu trated in Fig. 1. A sequence of data (a, is sent through 8 (nest) channel with unknown impulse response S. The data ‘ay take on one of a small number of discrete values (in the sequel we assume that ay may take on one of two values 42). The output of the unknown channel (7) is then passed through a linear transversal filter W’ whose impulse response is approximately S~?. The objective of HW is to cancel out most of the distorting effects of the channel S so that the sequence (yt) output by Wis a near copy of the original input sequence (a4). Finally, a zero-order (memoryless) nonlinear decision process (ZNL) examines each output ye and replaces it withthe closest value fom the set of disrete input values, producing the sequence ({.). We wish to consider the general case in which $ may be noncausa, so WY must introduce some delay into the system. AS a result, we desire in general that We © ay OF dy = ayy where N represents the (known) delay of W. Systems ofthe type just deserved occu fequently in digital ‘communications where data must be converted into analog form for transmission and then converted back into digital form at the recsiver (see [11], [14] for # general introduction to this problem). IV is generally filter with adjustable tap ‘weights and the problem we explore is how to adjust these tap weights so W’ becomes a good approximation of S~? I the sequence (as) is known, the classical approach is to use an LMS or siochastc gradient descent procedure [12] to Paper aproved ty te Er or Chanel Egzation ofthe IEEE Com rmunshonr Soc, Mamma reseed Apel 24, 10 vised May 3, Toot an Aap 12,191 Ts wort wae soppy rn fom be Ca aman ig Rah Ce Inst; Sun Deg, CA 931 “GLE: Hinon i wh the Depart! of Campa Scene, Uses of "ye ntiton a) fs 0 doe seguone fra vale, white wil ie oe Sent of hat segues. Geoffrey E. Hinton ee ee ree —_ TMS Fe 1k dg f sys of ee. inimize l{ye~ay—1)| im practical sations the sequence (a) i not Known. Instead, « wosep procedure i used to agus the equsliet 1) an inital setting phase in which the transmit sends 4 own initiation sequence and the reeciver performs LMS agjustment of the equalizer and 2) a Phase in which the reeiver uses the output ofthe ZNL (i) san estimate fr (a) in performing LMS adjustment of the quale. “The second step of the procedure is usually refered to a operating the equalizer In decisiondrected mode since Updates tothe tap of the cqulzcr are contlled by the decisions made by the ZN. Inthe classical decision-dtected LMS (DDLMS) algorithm [14], [13}. the ZNL i a simple threshold device. For the binary chanel we are considering, the output of this Z2NL- cam be represented asa ngn{ye) and the DDLMS algorithm an be regarded a8 minimizing Ely ~ 0 sgu(a.)). In this paper we derive different form of 2NL for operating the equalizer in decision-irected mode. The modified ZNL is derived by proposing that (y.) be modeled a the output of fan independent, identically distributed (iid) random process ‘with a pasian mixtore density. The ZL then comesponds {o the maximum liketibood estimate of dy given yr and the assumed model The next section of the paper derives the basic algorti, and Section MI presents some simulation results on a simulated binary PAM chanoel which demonstrate thatthe modified equalization algorithm can converge reliably ‘with initial ehannel eror ates considerably greater than those ‘Mlowed by the casseal DDLMS procedre u ‘We begin by introducing some mathematical notation, and then outline the basic method for adjusting the tap weights fof the equalizer via a stochastic gradient descent procedure DEVELOPMENT OF THE ALGORITHM Techs his expectation ead with pet 0 emsmbl ae geo he sg However pase to sme sana OF 5 SF cesses an tose tite verge or ensemble serge. the Spl cna onda may suet ll erpcton a eva 0-077493808.00 © 1953 IEEE‘We then develop our algorithm by introducing a particular functional to be optimized and relating this functional to the functional used in the classical DDLMS procedure. In the system depicted in figure 1, we assume that W represen an adjustable filter with 20V +1 taps and a delay of N, 50 where the superscript on the term wh indicates that the tap weights change over time. Define the vector w = (wy, sy) and the vector = (reyivs-+- trae}. Then we ‘may express (1) 38 w away here the superscript T' denotes the vector transpose ‘As mentioned in the introduction, the objective of adjusting the tap weighs in the equalize isto make yy ~ ay. The approach we shall use isto treat (ye) as a stochastic process and 10 minimize a functional Ey 710) = ELE) @ Following the classical approach (assuming stationary ergodic processes), we replace ensemble averages in (3) with time averages and we got the following stochastic gradient update procedure @ wees = we = eee, ® where and cis (fixed) stepsize. General convergence results for this lass of stochastic approximation algorithms may be found in {3}. To complete the algorithmic specification, we simply need to define Uy.) Since we do not have direct access 10 (ay) when updating ‘our tap weights, 1(y,) must coatain some assumptions about ‘ay. In the classical DDLMS algorithm for a binary PAM ‘channel with input values £0, W(y.) = (ye — a sen(s))? ‘ye ~ 0 sn(y.). Informally, the assumption in the algorithm i that if ye > 0 then a, = a, otherwise ‘We propose instead that we define Vy.) = —log fy(ye) coy ri 4 fae pnt) ee O Syn) isthe density function of a gaussian mixture with two. ‘components centered at +a.' p; and pp ate the proportions of the two gaussians in the mixtute, and o is their standard deviation. Inthe sequel, we will assume that p: — px = 0.5, 2p ie ten ema ns are (RY) an apg wer ig 2 Soft econ cor fectn fr dee ales of 2. (0 ‘eran ot hd Son) hich corresponds approximately to the assumption thatthe values a are equilly likely in the data sequence (a,). We assume for now o is also fixed ‘Minimizing ‘Y(ye) a8 defined in (S) corresponds to maxi- izing the (los) likelihood thatthe sequence (y.) is generated by an iid, source with density fy()- One can justify this ‘model for (yr) by assuming that (ay) is generated by an Lid, source with density f4(a,) = 0.56.4 + 0a and that the combined effect of 5 followed by Won (ax) i simply t0 add 0 mean gaussian noise (and a delay of N) to (ae). The added noise is composed of channel noise and an intersymbol interference term. Since the intersymbol interference term contains the contributions ftom a large number of random symbol, the characterization of the overall noise constribution as gaussian is not unreasonable. ‘To evaluate ¢, based on (5) we must deal with the sum of, exponentials in (6). One simple approximation i to ignae the smaller ofthe two exponential terms. Under this assumption oe oi ‘We can simply include the 1/0? term into our step size ¢ and \we find that this approximate expression fore, is identical to the expression fore; in the classical DDLMS algorithm. I we do not make any approximations, the expression for al ) ‘Comparing (8) tothe classical DDLMS erro, we see that we have replaced the siga function with a more complicated non- linearity. We refer to this particular non-linearity as a “soft” decision as compared to the “hard” decision represented by the sign function in the DDLMS algorithm. The choice of these terms is apparent from Fig. 2 where we have plotted (8) as a funetion of y for several values of 6. As o> 0 (8) becomes identical to the errr in the DDLMS algorithm. This error has a sharp discontinuity at y = 0, and as a increases this discontinuity is smoothed away. (ne) (n-a) n Nand dropped the superscript on uy for notational convenience. Let W (wtnosss-yw%,) denote the impulse response of an ideal inverse filter for S50 Dota = 6A I we replace W with W* in Fig, 1 we would have Lovins We can regard Was an approximation to the ideal equalizer W* and rewrite (11) as, m= Dowie + DO (woe up deee ean bg a ™ 43) as ‘where (nf) is convolutional noise sequence and we have used (13). The variance of each component in our mixture density is largely determined by the variance of the convolutional noise sequence (nf). From (14) i is apparent that nf contains contributions from a large number of transmitted symbols, $0 it is reasonable to assume thatthe properties of (nf) are not dramatically different for different symbol values (ve, samples representing the symbol and samples representing the symbol =a will have similar distributions for n°), This suggests that the variance of each component in the mixture density fy() is the same.* Equation (14) also suggests thatthe characteristics of (nf) will change as W changes, so the assumption that o? should be fixed does not seem reasonable. Simulation studies have shown that bette convergence may be obtained by adjusting in parallel with adjusting the equalizer tap weight [10} (7). [9]. The algorithm used to update is an incremental version Of the standard maximum likelitood estimator for mixtute components with tied variance [5], (8). The actual update rule (E+ 1) = no%(t) + (1 — WA +a)? + — AN a?) as) where : 06) Tene and ris a decay rate slightly less than 1 for discounting past data. The decay sate for the exponential averaging of past data js based on the degree of stationarity of the dita. A more detailed development of this expression may be found in [9] Finally, we note that the model for the data given by (0) may be generalized to allow a larger class of discrete signal distributions while stil allowing the distribution of the sequence (y.)t0 be modeled as a mixture of gaussians If we let A denote random process for which (a) isa realization, the pa of this Lid source may be writen inthe general form Sala) =F piba-n an This source wil e 0 mean and uncorrelated as long as Lem ‘An hid data source of the form (17) wil lead to an iid ‘ode! for (y,) with a Gaussian mitre density in which the mixing proportions, and means ate as specified in (17) andthe variance ofall components is the same. as) i “The discussion inthe previous section comparing the “hard” and “soft” decision noolineaities suggested thatthe difference between these two nonlinearities will be most apparent when channel is poorly equalized and there ae numerous incorrect decisions (i. cases where ay. and ye have opposite signs) Simulations reported elsewhere [10], [7] have shown thatthe SIMULATION RESULTS 0.5. Even inthis egion, the soft decision algorithm maintains a significant advantage (10% to 12% for DDLMS with >= 0.5 compared t0 20% 10 23% for the softlecision algorithm). IV, Coxctusion We have proposed a new algorithm for blind equalization ‘of very distorted channels. Th algorithm is based on the idea ‘of adjusting the equalizer ap gains to maximize the likelihood that the equalizer outputs come from an iid. source that is a mixture of two gaussians with known means. The approach is most similar to earlier work on blind deconvolution by Goalrey and Rocca [6] and blind equalization by Bellini (1) [2]. Our approach differs from these other approaches in that the algorithm is developed based on an assumed distribution for the equalizer output rather than assumptions about the Aistribution of signals at the chanael input. This leads to an algorithm with an adaptive non-linearity for estimating the ‘channel input given the equalizer output. This “soft” decision algorithm is compared to the classical decision-irected LMS algorithm on a binary PAM channel and simulations suggest that when operating in the bind mode the new algorithm can converge in channels with twice the intial bit error rate for ‘which the DDLMS algorithm is convergent. ACKNOWLEDGMENT ‘We thank ¥. Le Cun and J. Bridle for helpful discussions. REFERENCES (0) S. Bet, “tagang lectiques for iad equsinion” in IEE (GLORECOM Conf cy tp It 160, Dee 198 (21 $2 Belin sod FR FR Bearense, M Gounat, and G, Rust, “Robes Heniicaoe of * oonmiaimas pige spacer Bind alse! tea oer in"Gurcommrkatons” IEEE Tres, Automat Con vo NCS pp. aks 8. 16 TAC Bl he Try and Pace of Modem es New Is] ATP: Demir NM, Lai, and. Rain from lncompee ta vie he EM sige Soe 9 BS. pp. 1-8, 17 8 “Maxim tod ‘a Pre Rval Se(6) R Goaey and. Rover Zor meron meiner decovaiin” Geophys: Prospecting A. 28, 9p 19-228. 19S (7) GrE: Hime aed 5, Nowan The bottap Widow Tt re 4 lgserformaion agen” Neral Comput, 82 pp 35-362, is (9) G2). Metachin and KE Basar, Mazur Mol: Inference and Applian 0 Chstering_ New Yok: Mare Dobe, 1986 19] SU. Now, “Soft competitive adapta Netra rcwork lamin lgeritins based on fing sail matures” PD dose, ‘Sehnl of Comps Si. Carele Melon Uni Hitsbuah PA Age [00] SJ. Now ad 6. Hin, Main Hla dco dicted ‘pte equalization University of Toon, Tech Rep. RETR, Un] SUC owes, “Adapive cquliznion” Pr TEE, po Sept 1. Biro and ME Hott, “Adupve swing ccs” in IRE WESCON Camenton Ree 190, pp. 36-104 Widow, ML McC MG. Langan C Rha eS ‘onary and senor ering curate the EMS adapt {ee Proc IEE, vo pp HIST “1862 Aug 137, Udo ad SS aap Sigal crn Englewood Gf, Ni Prentice ae DS 1) 0) ne