1

SGN 2206 Adaptive Signal Processing

Lecturer:
Ioan Tabus
office: TF 414,
e-mail tabus@cs.tut.fi
Contents of the course:
Basic adaptive signal processing methods
Linear adaptive filters
Supervised training
Requirements:
Project work: Exercises and programs for algorithm implementation
Final examination
Lecture 1 2

Text book: Simon Haykin, Adaptive Filter Theory

Prentice Hall International, 2002

0 Background and preview 10

Kalman Filters
1 Stationary Processes and Models 11
Square Root Adaptive Filters
2 Wiener Filters 12
Order Recursive Adaptive Filters
3 Linear Prediction 13
Finite Precision Effects
4 Method of Steepest Descent 14
Tracking of Time Varying Systems
Adaptive Filters using Infinite-Duration
5 Least-Mean-Square Adaptive Filters 15
Impulse Response Structures
Normalized Least-Mean-Square Adaptive
6 16 Blind Deconvolution
Filters
7 Frequency-Domain Adaptive Filters 17 Back-Propagation Learning
8 Method of Least Squares Epilogue
9 Recursive Least-Squares Algorithm
Lecture 1 3

1. Introduction to Adaptive Filtering

1.1 Example: Adaptive noise cancelling

• Found in many applications:

Cancelling 50 Hz interference in electrocardiography (Widrow, 1975);
Reduction of acoustic noise in speech (cockpit of a military aircraft: 10-15 dB reduction);
• Two measured inputs, d(n) and v1 (n):
- d(n) comes from a primary sensor: d(n) = s(n) + v0 (n)
where s(n) is the information bearing signal;
v0 (n) is the corrupting noise:
- v1 (n) comes from a reference sensor:
• Hypothesis:
* The ideal signal s(n) is not correlated with the noise sources v0 (n) and v1 (n);

Es(n)v0 (n − k) = 0, Es(n)v1 (n − k) = 0, for all k

* The reference noise v1 (n) and the noise v0 (n) are correlated, with unknown crosscorrelation p(k),
Ev0 (n)v1 (n − k) = p(k)
Lecture 1 4
Lecture 1 5
Lecture 1 6

• Description of adaptive filtering operations, at any time instant, n:

* The reference noise v1 (n) is processed by an adaptive filter, with time varying parameters
w0 (n), w1 (n), . . . , wM −1 (n), to produce the output signal
M
∑ −1
y(n) = wk (n)v1 (n − k)
k=0
.
* The error signal is computed as e(n) = d(n) − y(n).
* The parameters of the filters are modified in an adaptive manner. For example, using the LMS
algorithm (the simplest adaptive algorithm)

wk (n + 1) = wk (n) + µv1 (n − k)e(n) (LM S)

where µ is the adaptation constant.

Lecture 1 7

• Rationale of the method:

* e(n) = d(n) − y(n) = s(n) + v0 (n) − y(n)
* Ee2 (n) = Es2 (n) + E(v0 (n) − y(n))2 (follows from hypothesis: Exercise)
* Ee2 (n) depends on the parameters w0 (n), w1 (n), . . . , wM −1 (n)
* The algorithm in equation (LMS) modifies w0 (n), w1 (n), . . . , wM −1 (n) such that Ee2 (n) is minimized
* Since Es2 (n) does not depend on the parameters {wk (n)}, the algorithm (LMS) minimizes E(v0 (n) −
y(n))2 , thus statistically v0 (n) will be close to y(n) and therefore e(n) ≈ s(n), (e(n) will be close to
s(n)).
* Sketch of proof for Equation (LMS)
· e2 (n) = (d(n) − y(n))2 = (d(n) − w0 v1 (n) − w1 v1 (n − 1) − . . . wM −1 v1 (n − M + 1))2
Error Surface

30

· The square error surface 25

20

e2 (n) = F (w0 , . . . , wM −1 )
15

is a paraboloid. 10
20

15 20
15
10
10
5
5
w 0 0
1 w0

2
· The gradient of square error is ∇wk e2 (n) = de (n)
dwk = −2e(n)v1 (n − k)
Lecture 1 8

· The method of gradient descent minimization: wk (n + 1) = wk (n) − µ∇wk e2 (n) = wk (n) + µv1 (n −
k)e(n)
* Checking for effectiveness of Equation (LMS) in reducing the errors
M
∑ −1
ε(n) = d(n) − wk (n + 1)v1 (n − k)
k=0
M
∑ −1
= d(n) − (wk (n) + µv1 (n − k)e(n))v1 (n − k)
k=0
M
∑ −1 M
∑ −1
= d(n) − wk (n)v1 (n − k) − e(n)µ v12 (n − k)
k=0 k=0
M∑ −1
= e(n) − e(n)µ v12 (n − k)
k=0
M −1
∑ 2
= e(n)(1 − µ v1 (n − k))
k=0

In order to reduce the error by using the new parameters, w(n + 1)

|ε(n)| < |e(n)|

2
0<µ < ∑M −1 2
k=0 v1 (n − k)
Lecture 1 9
Lecture 1 10
Lecture 1 11
Lecture 1 12
Lecture 1 13