Performance Analysis of Adaptive Noise

Cancellation by Using Algorithms

Student OF MS EE at FAST-NUCES,

Abstract— In statistical signal processing results will be presented here for noise cancellation.
adaptive filters plays a vital role. In Adaptive
filtering, filter coefficients are eventually
changes automatically according to some
adaptive algorithm in synchronization with
input signal by keeping error into account, to
adapt to changing signal characteristics. In
this research paper I have chosen noise
cancellation application of adaptive filers.
First of all we gave a brief overview of
adaptive filters. Then brief introduction
Figure 1: Flow Chart
about Least Mean Square (LMS), Normalized
Least Mean Square (NLMS) and Recursive II. ADAPTIVE FILTERS
Least Square (RLS) algorithm. LMS, NLMS According to an adaptive algorithm adaptive
and the RLS are implemented in MATLAB filter self-adjusts the filter coefficients. The
to remove noise from the information relationship between the input and output signals
carrying signal and present us helpful is iteratively modeled in adaptive filter. Figure
information as an output. Then we compared given below is a general adaptive filter
the results. Graphs are the results of consisting of a shift varying filter wn(z) and an
implementation of these algorithms; with the adaptive algorithm for updating the filter
help of these graphs performance of each coefficients wn(k).
algorithm is analyzed.

Keywords—Recursive Least Square (RLS),

Least Mean Square (LMS), Matrix LABoratory
(MATLAB) and Normalised least mean
square(NLMS).
I. INTRODUCTION
Adaptive filters are appropriate for the systems
where statistical parameters of system are
indefinite. Frequently known adaptation
algorithms are RLS and LMS, where RLS
algorithm has higher convergence speed than the
LMS algorithm, but at the expense of
computation complexity. LMS algorithm has Figure 2: Typical adaptive filter
less computational complexity than RLS
algorithm. Adaptive filters are capable of adaptation to an
After reading this research paper one can easily unknown setting. These filters are most
understand the theory behind the adaptive filters. frequently used because of their adaptability and
Possible Algorithms solutions and their performance low cost. Adaptive filters can operate in an
unknown environment and can track time
variations of input statistics. Undeniably,
adaptive filters are frequently and effectively
utilized over the years. Applications of Adaptive
filters can be classified as follows:
Identification, Prediction, Inverse modeling, and
noise cancelling.
Following characteristic are in common for all
the above mentioned applications:
Comparison of an input fed to an adaptive filter
is done with a required output which results an
error. This error than updates the weights
(coefficients of filter). This is done to lessen the
error so that we can reach the optimization.
Error in best possible case is zero but is always
some points greater than zero.
III. APPLICATIONS OF ADAPTIVE
FILTERS
To make the reader familiar with the overview
with the variety of adaptive filter’s applications
four main applications are listed below.
A. Identification or Modeling:
Figure 3 is illustrating the identification
problem. Here the same input x(n) is fed to the
adaptive filter as the system. y(n) is the adaptive
filter’s output. Output of adaptive filter is then
compared with the preferred response d(n).This
comparison generates something which is
known as error . Error is then used to amend the
w(n) weight for the reduction of error in order to
identify the system.
Figure 3: System Identification or Modeling
B. Inverse Modeling:
Figure 4 illustrates inverse modeling, the.
Inverse modeling is also known as de-
convolution. Its goal is to find out and follow the
inverse transfer function of the system. system
receives an input x(n) and its output u(n) is fed
to the adaptive filter .when a delay is given to
the input x(n) we get preferred output which is
d(n).Then we compare the filter output y(n) and
the desired response d(n). To update or correct
the weights of filter error is used.
Figure 4: Inverse Modeling
C. Prediction: The figure 5 is explaining the
logic of the predictor adaptive filter. This
practical usage gives the best estimate of a
arbitrary signal, this filter uses the previous
values of the arbitrary signal x(n), achieved as a
result of providing a delay to that signal fed to
the adaptive filter input and comparing its output
y(n), with the desired response d(n), that is the
actual random signal x(n). To adjust weights of
filter, when output of filter is used, then that
filter is said to be a predictor.
Figure 5: Predictor Adaptive Filter
D. Noise Cancellation:
Noise cancellation is the main focal point of this
research paper. The idea behind this application
is following
let d(n) is a preferred output tarnished by a
noise n2(n) and is said to be main signal.
Main signal = s(n) + n2(n)
Comparison of d(n) is then done with y(n) ,input
fed to adaptive filter is a reference signal n1(n).
This reference signal also produces noise which
tarnishes the main signal. e(n) is the system
output which is actually the comparison of
filter output and the preferred output .For best
state, this error will be equal to the original
signal .

Figure 6: Interference Cancellation
IV. ACTIVE NOISE CANCELLING
Adaptive noise cancellation is also known as
active noise cancellation. Active noise
cancellation can simply be referred as noise
cancellation class. Our main focus is to lessen
the noise intrusion or terminate the disturbance.
The approach chosen in the ANC algorithm, is
to try to mimic the original signal s(n).
Central goal is to use an active noise cancelling
algorithm to stop speech noise intrusion and to
coup with several type of tarnished signal.
Interference cancellation is explained in figure 6.
The intrusion signal for the reference sensor is a
noise and given in the system as a reference
signal. Primary sensor is finds out the preferred
signal. Adaptive filters produce a first response.
Preferred signal is then compared with that first
response which results an error. Error produced
is then used as the feedback of filter, correcting
weight of filter and the response of system .To
compute the best active noise canceller
algorithm diverse approaches can be used. Noise
canceller algorithms to a preferred usage with
diverse outputs for a large number of algorithm
can be designed. Depending on the specific
features pertaining in each Algorithm give its
unique applications having its own merits and
demerits.
V. LEAST MEAN SQUARE (LMS)
ALGORITHM
In LMS Algorithms we are concerned with
desired filter by finding the filter coefficients
that produce the least mean squares of the error
signal. Error signal is the difference between the
desired and the actual signal. Stochastic gradient
descent method is used in that algorithm.
The basic idea behind LMS filter is to approach
the most favorable filter weights (R-1 P), by
updating the filter weights in such a way to
converge to the most favorable filter weight.
Firstly, a small weight is assumed, most
commonly it is zero.
At each step gradient of the mean square error is
determined, the weights are updated
accordingly. If the MSE-gradient is positive, it
shows that error will increase positively if the
same weight is used repeatedly, so here is the
need to reduce the weights to be closer to the
required weight. In the same way, if the gradient
is negative reflects the need of increase in
weight till to get closer to the desired weight. So,
the basic weight update equation is:
wn+1 = wn - μΔε[n]
Here, ℇ is presenting the mean square error.
(-) sign is showing there is the need to change
the weight opposite to gradient slop,that is if
slope is positive reduce the weight,if negative
increase the weight.
A. LMS algorithm Summary
for pth order:
The LMS algorithm for a pth order algorithm
can be described as
P = filter order
μ = step size
Start with: ĥ (0) = 0
Computation: For n = 0, 1, 2...
X(n) = [x(n), x(n - 1), …, x(n – p + 1)]T
e(n) = d(n) – ĥH(n) X(n)
ĥ (n+1) = ĥ(n) + μ e*(n) X(n)
B. Convergence and stability in the mean of
LMS:
One of the drawback of LMS algorithm is that we
cannot obtain optimal weight in absolute sense due to
not using the exact values of the expectations, but
convergence to the mean is possible that is helpful in
achieving desired results. Here variation of weights
to optimal weight is possible to small amount.
Convergence to the mean can be misleading if the
variance is high with which weights are changing..
This problem arises when the value of step-size μ is
not selected correctly.
Thus, an upper bound on μ is needed which is
given as
0 < μ < 2/λmax
Where λmax is an autocorrelation matrix, its
eigen vales can never be negative. Algorithm’s
stability is dependent on its value, negative
value of λmax will make it unstable. If the value
of μ selected is very small then the algorithm
will converge very slowly. A large value of μ
will result into faster convergence but at the
expense of stability around the minimum value.
Maximum convergence speed can be achieved
by
2
𝜇=
𝜆𝑚𝑎𝑥 + 𝜆𝑚𝑖𝑛
Here in order to get convergence faster the value
of 𝜇 should be large which is achievable when
λmax is close to λmin, where λmin is the
smallest eigen value of R and also determining
the speed of convergence.
VI. NORMALIZED LEAST MEAN
SQUARE (NLMS) ALGORITHM
Learning rate μ assures the stability of the
algorithm and makes it more effective.
Sensitivity to the scaling of inputs makes pure
LMS less effective because it makes selection of
learning rate μ difficult which is a drawback.
The Normalized least mean squares (NLMS)
filter is a modification of the LMS algorithm. It
normalizes with the power of the input.
NLMS algorithm outline:
Parameters: P = filter order
μ = step size
Initialization: ĥ (0) = 0
Computation: For n = 0, 1, 2...
X(n) = [x(n), x(n - 1), …, x(n – p + 1)]T
e(n) = d(n) – ĥH (n) X(n)
ĥ(n+1) = ĥ(n) + μ e∗(𝑛) X(𝑛)/ XH(𝑛) X(𝑛)
Optimal Learning Rate:
If there is no intrusion [v(n) = 0], then the
optimal learning rate for the NLMS algorithm is
μopt = 1
And is free of the input X(n) and the real
(unfamiliar) impulse response h(n). In the
general case with intrusion v(n) does not equal
to 0, the most favorable learning rate is
μopt = 𝐸 [│𝑦 (𝑛) –y^(𝑛) │2]/𝐸 [│𝑒(𝑛)│2]
We assume that the signals v(n) and X(n) are
uncorrelated to each other.
VII. RECURSIVE LEAST SQUARE (RLS)
ALGORITHM
LMS may not provide fast rate of convergence
and small mean square error.
In case of LMS algorithm it is necessary to have
knowledge of autocorrelation of input signal and
cross-corelation between the input and desired
output.
Therefore, we consider error measures that do
not include from expectations and may be
included from the data.
ℰ(𝑛) = ∑𝑛𝑖=0 | 𝑒(𝑛)|2
No Statistical information about x(n) or d(n) is
required and may be evaluated directly from
x(n) and d(n).
In RLS algorithm we introduces forgetting
factor λ
𝑛
ℰ(𝑛) = ∑ λ𝑛−𝑖 | 𝑒(𝑛)|2
𝑖=0
RLS algorithm reduces a weighted linear least
squares error. The RLS algorithms gives
outstanding performance in non-stationary
situation.
VIII. MATLABSIMULATION RESULTS
A. LMS algorithm
Some alterations has been done to figure 6 and
summary for the LMS to attain the interference
cancelling.
The figure 7 shows the result of usage of LMS
algorithm to intrusion problem, where s(n) is the
input, the preferred signal x(n) = s(n) + n2(n) and
the error signal, which should be equal to the
input signal s(n). In figure 7, the step-size
parameter μ is se equal to 0.0002 and the length
of adaptive filter has kept 5. The input signal
s(n) is shown in blue. The input signal After the
noise corruption s(n) + n2(n) is shown in green
color, and the error signal e(n) is shown in red.
L = 5, μ = 0.0002
Figure 7: Output of noise cancellation using LMS
algorithm.
Analyzing Figure 7 the LMS we can see
algorithm has not done excellent performance,
not entirely removing the error signal e(n) hence
not resulting in the original signal s(n), entirely
free of the noise intrusion n1(n).
L = 5, μ = 0.0002
Figure 8- Mean-squared error of noise
cancellation using the LMS algorithm
Figure 8 is showing the mean-squared error for
the LMS algorithm for noise cancellation. This
error is not the error signal e(n) but the
difference between this signal and the input
signal s(n). even until 30000 iterations,
convergence has not occurred. Practically, the
algorithm should have a reasonable number of
iterations (a few seconds) for the error to reach a
value closer to its most favorable in the mean-
square sense.
B.NLMS ALGORITHM
L = 5, δ = 3.2, μ = 0.005
Figure 9- Output of noise cancellation using NLMS
algorithm.
Figures 9 and 10 show the outputs noise
cancellation using the NLMS algorithm. The
input signal s (n), is tarnished by the noise n1(n)
resulting in the tarnished signal
x (n) =s(n)+n2 (n). The error signal e (n) which is
thought to reproduce the input signal s (n) is
shown by color in red.
Looking at the figure 9, it can be seen that the
algorithm has done a good job, removing the
noise interfering the original signal.
Figure 10 illustrate the MSE which is difference
between the error signal and input signal. In this
figure algorithm has presented conversion after
round about 10000 iterations. After iterations
1000 number, the algorithm has shown some
variance but not at the expense of conversion.
L = 5, δ = 3.2, μ = 0.005
Figure 10- Mean-squared error of noise cancellation
using the NLMS algorithm
C. RLS ALGORITHM
Figures 11and 12 are showing output of noise
cancellation using RLS algorithm. Algorithm
has shown good job by removing noise and
proved it a good active noise canceller. The
original input signal s(n) is shown in blue color
line, the green line shows the same signal after
the noise corruption x(n)=s(n)+n2(n) and the red
line shows the error signal, which must be closer
to the original input signal s(n).
L = 5, λ = 1
Figure 11- Output of noise cancellation using RLS In a nutshell, this research paper gives a brief
algorithm. idea of our achievements in the field of adaptive
L = 5, λ = 1 filter usage for noise cancellation,
comprehensive investigation and evaluation of
LMS, NLMS and RLS algorithms for noise
cancellation.

X. REFERENCES

[1]. Diniz, P. S. R. Adaptive Filtering –

Algorithms and Practical
Implementation. Kluwer Academic
Publishers, 2008.
Figure 10- Mean-squared error of noise cancellation [2]. Falcão, Rafael Merredin Alves.
using the RLS algorithm "ADAPTIVE FILTERING
ALGORITHMS FOR NOISE
THE RLS ALGORITHM CANCELLATION." 2012.
[3]. Guan, X., X. Chen, and G. Wu. "QX-
LMS Adaptive Filters For System
Figures 11and 12 are also showing the cost
Identification." 2nd international
function of the error which is between the Congress on Image and Signal
original input signal represented by s(n) and the Processing, 17-19 Oct 2009: CISP '09,
error signal represented by e(n). By analyzing vol., no., pp.1-5,.
the figures it can be seen the algorithm has very [4]. Haykin, Simon S. Least-Mean-Square
fast convergence as compare to LMS AND Adaptive Filters. Wiley: Bernard
NLMS only after the first few thousand Widrow , ISBN 0-471-21570-8., 2003.
iterations, so worked very efficiently in [5]. [Haykin:, Simon. Adaptive Filter
removing noise. Theory. Prentice Hall,: ISBN 0-13-
048434-2., 2002.
[6]. Thenua, Raj Kumar, and S.K. Agarwal.
IX. RESULTS COMPARISON
"Simulation and Performance Analyasis
of Adaptive Filter in Noise
Analyzing above figures , we came to the Cancellation." International Journal of
conclusion that the LMS algorithm takes time Engineering Science and Technology,
and has a very slow convergence for the given 2010: Vol. 2(9),4373-4378.
number of iterations making noise cancellation [7]. Monson H. Hayes: Statistical Digital
less effective as compare to NLMS and RLS. Signal Processing and Modeling, Wiley,
Moreover in case of LMS even convergence 1996, ISBN 0-471-59431-8
occurs but it incorporates a high error value, [8]. Raj Kumar Thenua and S.K.
about 30dB per 20dB as compare to other AGARWAL” Simulation and
Performance Analyasis of Adaptive
algorithms. In a situation where we are
Filter in Noise Cancellation”
concerned about the convergence speed LMS International Journal of Engineering
algorithm is not favorable here we should select Science and Technology Vol. 2(9),
the NLMS or RLS algorithm depending upon 2010, 4373-4378. Functions
speed and other requirements. RLS algorithm (Universitext), New York: Springer,
works three times more efficiently and faster 1986
than the NLMS algorithm, So, in our research [9]. Bouchard, M.; Quednau, S.,
span the most efficient is the RLS algorithm "Multichannel recursive-least-square
having faster speed of convergence, lesser error algorithms and fast-transversal-filter
and efficient output. It has its own algorithms for active noise control and
sound reproduction systems," IEEE
computational cost higher than other two.
Transactions on Speech and Audio
Processing, vol.8, no.5, pp.606-618, Sep
2000.
[10]. Wallace and R.B. Goubran.
IX. CONCLUSIONS "Improved tracking adaptive noise
canceler for nonstationary
environments" IEEE Transactions on
Signal Processing, mar 1992: vol.40,
no.3, pp.700-703.