Adaptive Echo Cancellation

The LMS Algorithm

By
Justin Allen
justin@utztech.com

Advisor
Andreas Spanias
Short History Of LMS In Echo
Control
•First proposed by Widrow and Hoff in 1960 [1].
•The LMS algorithm was proposed as an echo
cancellation algorithm in the later 1960’s [2,3].
•Advent of satellite communications brought the LMS
algorithm to commercial use.
•Today it is the standard by which other algorithms are
measured largely due to its simplicity.
Definition Of Terms
•LMS – Least Mean Squared
Adaptive algorithm in which the solution is found by minimizing
the mean error between the desired solution and the computed
observed data.
•xn– [xn, xn-1, …, xn-L+1]T
The excitation or “training” vector.
•sn
Echo plus near-end speech and background noise.
•hn – [h0,n, h1,n, …, hL-1,n]T
The filter coefficient vector, where hi,n is the ith tap weight at time n.
• en
The error signal or residual echo.
LMS Diagram
Echo Canceller
From
Far-End xn
Talker

hn Echoes

rn
To Far-
End -
Talker
en + sn
+ Near-End
Speech
Adaptive Transversal Filter
Echo Canceller
From
Far-End x xn-1 xn-2 xn-N+1
n
Talker
Z-1 Z-1 Z-1

h0,n h1,n h2,n hN-1,n

X X X X Echoes

+ + +
To Far-
End rn
-
Talker
en + sn
+ Near-End
Speech
LMS Algorithm[4]
First the training signal is filtered with the estimated
impulse response.
T (1)
r =x h
n n n −1
Then the error is calculated.
en = sn − rn (2)
Finally, the filter coefficients are updated.

h n = h n −1 + 2 µ x n en (3)
LMS Algorithm(2)
The signal names used in defining the
algorithm are the same as those used in the
diagram. The parameter in equation 3 is
called the step size. This parameter controls
the rate at which the algorithm converges. The
step size must be chosen carefully so that the
algorithm will be stable, yet still converge at a
reasonable rate.
LMS Algorithm(3)
The LMS algorithm is a stochastic gradient
algorithm. The algorithm converges by taking
steps along the error gradient until a minimum
is found. The exact minimum is not, but
instead the found minimum will randomly
jump around the true minimum. The amount
of jumping is dependent on the step size and
the SNR of the the input signal.
LMS Algorithm(4)
The LMS algorithm does not find the exact minimum
because an assumption is made in the coefficient
update equation. The true update equation is given by:

h n = h n −1 + 2 µΕ{x n en } (4)
Since the expectation cannot be done it is estimated by
using the instantaneous value as shown in equation 3.
This estimation greatly simplifies the LMS algorithm.
LMS Visualization: Filter Taps
• The following slide is a visualization of the LMS algorithm
adaptation. The blue line represents the actual impulse
response of the room. The green line is the adaptive filter.
As the time progresses (the video plays) the coefficients will
adapt and the green line will try to match the blue line.
• The length the of the filter is 20 taps (it is easier to
visualize with a short filter).
• There is no noise, no non-linearities, etc. This simulation
is of the ideal case. In the next section more simulation will
be presented with various real-world impediments.
LMS Visualization: Filter Taps
References
1. S.L. Gay and J. Benesty (Editors), Acoustic Signal Processing For
Telecommunication. Norwell, MA: Kluwer Academic Publishers, 2000.
2. A. B. Clark and R. C. Mathes, “Echo suppressors for long telephone circuits,”
Proc. AIEE, vol. 44, pp. 481-490, Apr. 1925.
3. S. B. Weinstein, “Echo cancellation in the telephone network,” IEEE Comm.
Soc. Mag., pp. 9-15, Jan. 1977.
4. S. Haykin, Adaptive Filter Theory. Fourth Edition, Prentice Hall, Upper
Saddle River, N.J., 2002.