2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

A Modified IPNLMS Algorithm

Using System Sparseness
Genki Hirano and Tetsuya Shimamura
Graduate School of Science and Engineering, Saitama University
255 Shimo-Okubo,Sakura-ku, Saitama 338-8570, Japan
Email:{hirano, shima}@sie.ics.saitama-u.ac.jp

Abstract—Channels with sparse impulse response appear in echo

path, such as network echo and acoustic echo. A proportionate
adaptive algorithm has been proposed to accelerate convergence
speed than the normalized least mean square (NLMS) algorithm
in the sparse system. In general, the weight of the proportionate
term in improved proportionate NLMS (IPNLMS) algorithm
is set to a large value when the system is sparse. However,
the relationship between the weight of the proportional term
and the system sparseness is not clear. We reveal the rela-
tionship that the weight increases with an increase of system
sparseness by computer simulations. And we propose a variable
weight of the proportional term in IPNLMS algorithm using
the system sparseness. Computer simulations demonstrate that
the proposed algorithm provides faster convergence than the
conventional NLMS and IPNLMS algorithms on sparse systems.

Keywords - Adaptive filter; sparse system; NLMS; IPNLMS

I. I NTRODUCTION
The normalized least mean square (NLMS) algorithm is
widely used in adaptive signal processing applications such as
finite impulse response (FIR) estimator and equalizer[1][2].
It is well known that the NLMS algorithm is considerably
affected by the parameter called step size [3][4]. The NLMS
algorithm is simple but it is not appropriate to estimate a
sparse channel. Sparse channels mean that most of impulse
response individual values are zeros or nearly zeros. An exam-
ple of such channels is shown in Fig. 1. Such channels arise
in room acoustic echo [5], circuit echo path, and underwater
channels [6] etc. Sparse impulse response channel estimation
is applied to echo cancellation [7][8] such as acoustic echo
cancellation. It is inefficient for the NLMS algorithm to
estimate sparse channel impulse response because a fixed step
size parameter is assigned to each filter tap.
Proportionate adaptive algorithm has been proposed to im-
prove convergence speed when a channel has sparse impulse
response. In proportionate adaptive algorithm, step size is
assigned individually to each filter tap. Proportionate NLMS
(PNLMS) algorithm [7] is proposed using such an idea.
Therefore, PNLMS algorithm accelerates initial convergence
speed for sparse impulse response estimation. On the other
hand, PNLMS algorithm often behaves worse than NLMS
algorithm when the impulse response is not sparse. Improve
proportionate NLMS (IPNLMS) algorithm [9], one of propor-
tionate adaptive algorithms, assigns a combination of a fixed
Figure 1. An example of sparse channel impulse response.
and a variable step size parameter to each filter tap. IPNLMS
algorithm has characteristics of both NLMS and PNLMS by
the combination. IPNLMS shows better convergence speed
than PNLMS and NLMS algorithms on sparse and non-
sparse channels [10]. Experiential knowledge [6] is that the
parameter of IPNLMS is set to a large value when the system
is expected to be sparse. However, the relationship between
the parameter of IPNLMS and the system sparseness is not
clear [11]. At present, the determination of the parameter is
experiential and a constant value is used [9] [12].
In this paper, we reveal the relationship between the
parameter of IPNLMS and the system sparseness by computer
simulations. The simulation results provide a clue to select
the parameter of IPNLMS. Thereby, we propose a modified
IPNLMS which automatically controls the parameter value of
IPNLMS using the system sparseness, through the relation-
ship between them. Namely, the proposed algorithm controls
the step size of each tap based on the estimated system
impulse response at each iteration.
The organization of this paper is as follows. The PNLMS
and IPNLMS algorithms are introduced in Section II. In
Section III, we show the relationship between the parameter of
IPNLMS and the system sparseness, and derive the proposed
algorithm. The detail of simulations and the results are

978-1-4673-5082-2 ©2012 IEEE 876

 2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012

The diagonal elements of K(n) at PNLMS algorithm are

calculated as follows:

l(n) = max[δP , |w(1, n)|, . . . , |w(L, n)|], (7)

g(l, n) = max[ρl(n), |w(l, n)|], (8)
g(l, n)
k(l, n) = PL . (9)
i=1 g(i, n)

The parameters ρ and δP are small positive numbers for

Figure 2. System identification model using adaptive filter.
presented in Section IV. Conclusions are drawn in Section
V.
II. PNLMS AND IPNLMS ALGORITHMS
We assume the system shown in Fig. 2 throughout this
paper.
The unknown channel is linear and time invariant; it is
described by a discrete time FIR filter with a maximum lag
of L − 1 (i.e., filter length is L). The adaptive filter is used
to identify the unknown system. The input vector x(n) and
filter vector w(n) at the time of nth iteration are defined as
x(n) = [x(n), x(n − 1), . . . , x(n − L + 1)]T
and
w(n) = [w(1, n), w(2, n), . . . , w(L, n)]T
regularization, respectively. PNLMS shows faster initial con-
vergence speed than NLMS on sparse systems, but PNLMS
often becomes worse than NLMS for non-sparse systems.
This is because the components of adjustment step-size matrix
are very sensitive to the system impulse response to be
estimated. IPNLMS use not only filter coefficients but also
a fixed value depending on filter length to solve the problem.
The components of the adjustment matrix in IPNLMS are
described by
1−α |w(l, n)|
k(l, n) = + (1 + α) PL . (10)
2L 2 i=1 |w(i, n)| + ǫ
In (10), ǫ is used to prevent from dividing by zero. The
parameter α(−1 ≤ α < 1) combines the first term with the
second term(so called proportional term) on the right hand
side in (10). This means that α controls the weighting between
the NLMS and the PNLMS. When α = −1, IPNLMS is
equivalent to NLMS. In the case of a sparse system, α
(1)
is set to a large value so that IPNLMS can track rapidly
system sparseness. The weight of proportional term α affects
(2) significantly the convergence rate of IPNLMS.

where x(n) is the input signal and III. A MODIFIED IPNLMS ALGORITHM USING SYSTEM
w(1, n), w(2, n), . . . , w(L, n) are the adaptive filter SPARSENESS
coefficients. The filter output signal y(n) and error signal
e(n) are As described in Section II, the convergence rate of
IPNLMS is affected by the weight of proportional term
y(n) = wT (n)x(n), (3) according to system sparseness. The weight of proportional
e(n) = d(n) − y(n), (4) term should be a large value when the unknown system is
expected to be a sparse system. However, there is no distinct
respectively, where d(n) is the desired signal. The updating measure to decide the parameter. In this section, we show
equation of the proportionate adaptive algorithm is given by the relationship between system sparseness and optimal value
of the weight of proportional term. Then, we introduce a
K(n) = diag[k(1, n), k(2, n), . . . , k(L, n)], (5)
modified IPNLMS algorithm using the relationship.
µK(n)x(n)e(n)
w(n + 1) = w(n) + T (6) The problem is that we transform a system sparseness into
x (n)K(n)x(n) + δ
a numerical value in order to associate a degree of system
where µ (0 < µ < 2) is the fixed step size parameter, K(n) sparseness with optimal value of the weight. The following
is the diagonal matrix that regulates the fixed step size of equation is used to solve the problem:
the respective taps of the adaptive filter and δ is a positive  
constant used to avoid that the denominator in (6) results in L kwk1
S(w) = √ 1− √ (11)
zero values. The fixed step size µ governs significantly the L− L Lkwk2
convergence speed and the final error level. Therefore, by
where w is a system impulse response, L is the length of w,
individual adjustment in the step-size matrix, a proportionate
kwk1 and kwk2 are the L1 -norm and L2 -norm of w, which
adaptation is expected to improve the convergence rate. Em-
are defined as:
phasis of most of proportionate adaptive algorithms is placed L
X
on the calculation of components of the individual adjustment kwk1 = |wl | (12)
step-size matrix. l=1

877
 2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012
Figure 3. The relationship between system sparseness and optimal value of
β and an approximated curve of the relationship.
and
v
u L
uX
kwk2 = t wl2 (13)
l=1
respectively. S(w) means the system sparseness of w [13].
The range of S(w) is [0, 1]. Thus, it is convenient that
the range of α is transformed from [−1, 1) into [0, 1). The
transformation is achieved by using β = (α + 1)/2. In this
case, Eq. (10) is rewritten as
1−β β|w(l, n)|
k(l, n) = + PL . (14)
L i=1 |w(i, n)| + ǫ
We have investigated the relationship between the value of
the system sparseness and optimal value of β in convergence
speed by computer simulations. The specific conditions of the
computer simulations are explained in the next section. The
computer simulation result is shown in Fig. 3. Fig. 3 shows
the optimal value of β which provides the fastest convergence
of IPNLMS to the prescribed mean square error(MSE) levels
on sparseness characteristics of a system. Here, β = S 2 is
obtained as the approximated curve. The notable point is
the position of each point at each MSE in Fig. 3. When
the MSE is large (-20[dB]), the most of points are located
above the approximated curve. However, the most of points
which are prescribed as low MSE(-30[dB]) exist below the
approximated curve. From these observations, we propose that
the β (i.e. the weight of proportional term) is automatically
varied using the resulting relationship and tendency. The
determination method for the β is described now as
S 2 (w(n))
β(n) = γ (15)
S 2 (w(n)) + e2 (n)+δ
where γ and δ are positive constants, respectively. δ is set to
a small value to avoid dividing by zero. γ plays a role like
the weight of proportional term in IPNLMS. For the modified
878
Figure 4. Impulse response of unknown systems. Systems 1 and 2 have
sparseness characteristic S = 0.8656 commonly. The systems are considered
as typical sparse systems.
IPNLMS algorithm to be proposed in this section, (15) is used
with
1 − β(n) β(n)|w(l, n)|
k(l, n) = + PL (16)
L i=1 |w(i, n)| + ǫ
in proportionate adaptive algorithm.
IV. S IMULATIONS
We consider the situation that the unknown system is
identified by an adaptive filter as shown in Fig. 2. The
input signal is a white noise signal with standard normal
distribution N (0, 1). The noise v(n) is also white, and the
noise power is -40[dB]. Hence, the theoretical minimum of
error curve is -40[dB]. The fixed step-size µ = 1 is used in
all simulations. We use MSE as a measure of performance.
Number of independent trials of each simulation is 100 in
order to calculate the MSE.
The first simulation is that of finding the relationship
between the optimal value of β and the system sparseness.
The system is generated under the following conditions:
• The length of impulse response is 100.
• The coefficients of impulse response have only 0 or 1.
We change the ratio of the coefficient values being 0 or 1.
The simulation searches the optimal value of β in IPNLMS
for each impulse response. The number of impulse response
patterns is 100. Hence, the number of sparseness characteristic
patterns are also 100. The β is chosen from values from 0 to
1 by an increment of 0.05. However, β is set to 0.999 when β
is equivalent to 1, because of β ∈ [0, 1). The optimal value of
β is decided as that to achieve the fastest convergence to the
prescribed MSE. The relationship between the optimal value
of β and the sparseness characteristic is shown in Fig. 3.
The next simulation compares the proposed algorithm
 2012 IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS 2012) November 4-7, 2012
(a) Performance in typical sparse systems. The unknown system is switched (b) Performance in network echo path model. The unknown system is
between system 1 and system 2.
Figure 5.
with the conventional algorithms; the NLMS and IPNLMS
algorithms. One of the simulations uses the systems which
are shown in Fig. 4 as typical artificial sparse systems. In this
simulation, γ = 0.1, α = ±0.5 are used. Further simulation
uses network echo path impulse response defined in ITU-
T G.168 [14]. γ = 0.01 and α = ±0.5 are used in this
simulation. We use model 1 and model 5 in ITU-T G.168
here. For model 1 and model 5, several zero coefficients are
inserted to the beginning of impulse response to increase the
number of taps up to 100. The simulation result using the
typical sparse systems is shown in Fig. 5(a). Fig. 5(b) shows
the result of the echo path identification where the two echo
path models; model 1 and model 5, are switched.
The results in Fig. 5 indicate that the proposed algorithm
is superior to the conventional algorithms in sparse systems.
In particular, the proposed algorithm is faster for convergence
speed than the conventional algorithms in typical sparse
systems.
V. C ONCLUSION
In this paper, our two purposes have been achieved. One
of the purposes is to find the measure which provides a clue
to select the weight of proportional term in IPNLMS. The
other purpose is to propose a modified IPNLMS algorithm in
which the weight is automatically controlled. The relationship
between the weight of proportional term in IPNLMS and the
system sparseness has been shown. The proposed algorithm
is improved considering the relationship. Computer simula-
tions have demonstrated that the proposed algorithm provides
faster convergence than the conventional algorithms on sparse
systems.
R EFERENCES
[1] J. Homer, “Detection Guided NLMS Estimation of Sparsely
Parametrized Channels,” IEEE Trans. Circuits and Systems II:
Analog and Digital Signal Processing, vol. 47, no. 12, pp. 1437-1442,
2000.
[2] S. F. Cotter and B. D. Rao, “Sparse Channel Estimation via Matching
Pursuit With Application to Equalization,” IEEE Trans. Communications,
vol. 50, no. 3, pp. 374-377, 2002.
switched between model 1 and model 5 in ITU-T G.168.
Convergence comparison
[3] Y. Tsuda and T. Shimamura, “An Improved NLMS Algorithm for Chan-
nel Equalization,” in Proc. IEEE International Symposium on Circuits
and Systems, pp. v-353-v-356, 2002.
[4] S. Haykin, “Adaptive Filter Theory, Fourth Edition”, Prentice-Hall, 2002
[5] S. Makino, Y. Kaneda and N. Koizumi, “Exponentially Weighted Step-
size NLMS Adaptive Filter Based on the Statistics of a Room Impulse
Response,” IEEE Trans. Speech and Audio Processing, vol. 1, no. 1, pp.
101-108, 1993.
[6] K. Pelekanakis and M. Chitre, “Comparison of Sparse Adaptive Filters
for Underwater Acoustic Channel Equalization/Estimation,” in Proc.
IEEE International Conference on Communication Systems, pp. 395-
399, 2010.
[7] D. L. Duttweiler, “Proportionate Normalized Least-Mean-Squares Adap-
taion in Echo Cancelers,” IEEE Trans. Speech Audio Process., vol. 8,
no. 5, pp. 508-518, 2000.
[8] O. Hoshuyama, R. A. Goubran and A. Sugiyama, “A Generalized
Proportionate Variable Step-Size Algorithm for Fast Changing Acoustic
Environments,” in Proc. IEEE International Conference on Acoustics,
Speech, and Signal Processing, vol. 4, pp. iv-161-iv-164, 2004.
[9] J. Benesty and S. L. Gay, “An Improved PNLMS Algorithm,” in
Proc. IEEE International Conference on Acoustics, Speech, and Signal
Processing, pp. II-1881-II-1884, 2002.
[10] S. Sohn, J. Yun, J. Lee, H. Bae and H. Choi, “Convex Combination of
Subband Adaptive Filters for Sparse Impulse Response System,” in Proc.
IEEE 54th International Midwest Symposium on Circuits and Systems,
pp. 1-4, 2011.
[11] L. Liu, M. Fukumoto and S. Zhang, “A Variable Parameter Improved
Proportionate Normalized LMS Algorithm,” in Proc. IEEE Asia Pacific
Conference on Circuits and Systems, pp. 201-204, 2008.
[12] J. Benesty and Y. Huang, “Adaptive Signal Processing (Signals and
Communication Technology)”, Springer-Verlag, 2003
[13] Y. Huang, J. Benesty and J. Chen, “Acoustic MIMO Signal Processing
(Signal and Communication Technology).” Springer-Verlag, 2006
[14] ITU-T, “ITU-T Recommendation G.168: Digital Network Echo Can-
cellers”, 2009
879