Linear Prediction

Models
Correlation [1]

 The correlation is a second-order moment or statistic.

 The correlation of two signals is a measure of the similarity or the
dependency of their fluctuations in time or space.
Correlation [2]

 The correlation function and its Fourier transform, the power spectral density,
are extensively used in modelling and identification of patterns and structures
in a signal process.
 Correlators play a central role in signal processing and telecommunication
systems, including detectors, digital decoders, predictive coders, digital
equalisers, delay estimators, classifiers and signal restoration systems
Autocorrelation [1]

 Autocorrelation function of a random process 𝑋(𝑚), denoted by 𝑟𝑥𝑥 (𝑚1 , 𝑚2 )

 The autocorrelation function 𝑟𝑥𝑥 (𝑚1 , 𝑚2 ) is a measure of the self-similarity,

dependency, or the mutual relation, of the outcomes of the process 𝑋 at time
instants 𝑚1 and 𝑚2 . If the outcome of a random process at time 𝑚1 bears no
relation to that at time 𝑚2 then 𝑋(𝑚1 ) and 𝑋(𝑚2 ) are said to be independent or
uncorrelated and 𝑟𝑥𝑥 𝑚1 , 𝑚2 = 0
Autocorrelation [2]

 For a segment of N samples of signal 𝑥(𝑚), the autocorrelation function is

obtained as

 Autocorrelation of a signal can be obtained as the inverse Fourier transform of the

magnitude spectrum as
Autocorrelation of a
periodic signal:
estimation of period
 Autocorrelation can be used to
calculate the repetition period T of a
periodic signal such as the heartbeat
pulses

 (a) Heartbeat signal,

electrocardiograph ECG;
 (b) variation of period with time;
 (c) autocorrelation function of ECG.
Cross-Correlation

 The cross-correlation of two random processes x(m) and y(m) is defined as

 For wide-sense stationary processes, the cross-correlation function 𝑟𝑥𝑦 (𝑚1 , 𝑚2 )

depends only on the time difference 𝑚 = 𝑚1 − 𝑚2 :

 where 𝑘 = 𝑚1 − 𝑚2 is the cross-correlation lag

Ex : Time-delay estimation [1]

 Consider two signals 𝑦1 (𝑚) and 𝑦2 (𝑚), each composed of an information bearing
signal 𝑥(𝑚) and an additive noise, given by

 Where A is an amplitude factor and D is a time-delay variable. The cross-correlation

of the signals 𝑦1 (𝑚) and 𝑦2 (𝑚) yields
Ex : Time-delay estimation [2]

 Assuming that the signal and noise are

uncorrelated, we have 𝑟𝑦1𝑦2 (𝑘) =
𝐴𝑟𝑥𝑥 (𝑘 − 𝐷).
 The cross-correlation function has its
maximum at the lag D.
Linear Prediction Models

 Linear prediction (LP) models predicts/forecast the future values of a signal from a
linear combination of their past values
 A linear predictor model is an all-pole filter that models the resonances (poles) of
the spectral envelope of a signal or a system
 LP models are used in diverse areas of applications, such as data forecasting,
speech coding, video coding, speech recognition, model-based spectral analysis,
model-based signal interpolation, signal restoration, noise reduction, impulse
detection and change detection.
 Linear Prediction Coding [1]

 Two main motivations for the use of

predictors in signal processing
applications:
 To predict the trajectory of a signal. In the
frequency domain the trajectory prediction
is equivalent to the modelling of the
spectrum of the signal.
 To remove the predictable part of a signal
in order to avoid retransmitting ‘redundant’
parts of a signal that can be predicted at
the receiver and thereby save storage,
bandwidth, time and power.
Figure 8.1 Illustration of prediction of a sample x(m)
from P past samples.
 Linear Prediction Coding [2]
 𝑥(𝑚) is predicted as the product of the input vector [𝑥(𝑚 − 1), . . . , 𝑥(𝑚 − 𝑃)] and the
predictor coefficients [𝑎1 , . . . , 𝑎𝑃 ].
 The predictor coefficients are obtained from 𝒂 = 𝑹−1
𝑥𝑥 𝒓𝑥𝑥

Figure 8.2 Illustration of a linear predictor

Predictability, Information and
Bandwidth [1]
 The accuracy by which a
signal can be predicted from
its past samples depends on
the autocorrelation function,
or equivalently the
bandwidth and the power
spectrum, of the signal

Figure 8.3 The concentration or spread of

power in frequency indicates the predictable
or random character of a signal: (a) a
predictable signal; (b) a random signal.
Predictability, Information and
Bandwidth [2]
 in the time domain a predictable signal has a smooth and correlated
fluctuation, and in the frequency domain, the energy of a predictable signal is
concentrated in extremely narrow band(s) of frequencies.
 In contrast, the energy of an unpredictable signal, such as a white noise, is
spread over a wide band of frequencies.
 Most signals, such as speech, music and video signals, are partially predictable
and partially random. These signals can be modelled as the output of a linear
filter excited by an uncorrelated random input.
Applications of LP Model in Speech
Processing
 Speech is generated by inhaling air and then exhaling it through the glottis and the
vocal tract. The noise-like air, from the lung, is modulated and shaped by vibrations
of the glottal cords and the resonance of the vocal tract.

Figure 8.4 Asource-filter model of speech production. The filter

is usually modelled with a linear prediction model.
Applications of LP Model in Speech
Processing
 The source models the lung, and emits a random input excitation signal that is
filtered by a pitch filter.
 The pitch filter models the vibrations of the glottal cords, and generates a sequence
of quasi-periodic excitation pulses for voiced sounds
 The pitch filter model is also termed the ‘long-term predictor’ since it models the
correlation of each sample with the samples a pitch period away
 The vocal tract is modelled by a linear predictor model, which is also termed the
‘short-term predictor’, as it models the correlation of each sample with the few
(typically 8 to 20) preceding samples
Time-Domain Description of LP
Models
 A linear predictor model, Figures 8.1, 8.2 and 8.5, forecasts the amplitude of a signal at
time m, x(m), using a linearly weighted combination of P past samples [x(m−1), x(m−2),
. . . , x(m−P)] as :

 dimana variabel integer m adalah indeks waktu-diskrit, 𝑥ො (𝑚) adalah prediksi dari x (m), dan 𝑎𝑘
adalah koefisien prediktor. Implementasi diagram blok dari persamaan prediktor (8.1)
diilustrasikan pada Gambar 8.4
Time-Domain Description of LP
Models
For information-bearing signals, the prediction error
e(m) may be regarded as the information, or the
innovation (i.e. ‘new’), content of the sample x(m).
From Equation (8.2) a signal generated, or modelled,
by a linear predictor can be described by the
feedback Equation.

 The prediction error e(m), defined as the difference between the actual sample value x(m) and its
predicted value 𝑥ො (𝑚)
Illustration of a signal generated by
a linear predictive model
Figure 8.5 illustrates a linear predictor model of a signal x(m). In this model, the random input
excitation (i.e. the prediction error) is e(m)=Gu(m), where u(m) is a zero-mean, unit-variance random
signal, and G, a gain term, is the square root of the variance (i.e. power) of e(m):

where E[.] is an averaging,

or expectation, operator.

Figure 8.5 Illustration of a signal generated by a linear predictive model

 Frequency Response of LP Model
and its Poles
 The z-transform of LP Equation (8.3) shows that the LP model is an all-pole digital filter
with transfer functio.

 In Equation (8.5) it is assumed there are M complex pole pairs and N real poles with P =
N + 2M and 𝑟𝑘 dan 𝜑𝑘 are the radius and angle of the 𝑘 𝑡ℎ pole. The frequency response
of an LP model is given by
The pole–zero position and frequency
response of a linear predictor

Figure 8.6 The pole–zero position and frequency response of a linear predictor
Calculation of Linear Predictor
Coefficients

 is the autocorrelation matrix of the input vector

 is the autocorrelation vector
 is the predictor coefficient vector
Calculation of Linear Predictor
Coefficients