Professional Documents
Culture Documents
SSP 1 2 - Stochastic 2
SSP 1 2 - Stochastic 2
1 Introduction
2 Random processes
1 / 52
2 / 52 3 / 52
Ensemble and realisation Classification of processes
X (s, t ) represents a family or ensemble of time functions There are different types of random processes
Convenient short form x (t ) for specific waveform of the random process X (t ) One in particular interest: the discrete time random process
Each member time function is called a realisation We will in the following use the notation x (n) using index n in a discrete time sequence
Or, the notation x (t ) using time t in the time sequence (which still can be discrete)
A discrete time random process is a collection (or ensemble) of discrete-time signals.
Sloppy notation ahead: We will use x instead of X
4 / 52 5 / 52
Straightforward extension of the concept of random variables or the joint probability density function
Each random variable has a probability distribution
∂k
fx (α1 , . . . , αk , n1 , . . . , nk ) = Fx (α1 , . . . , αk , n1 , . . . , nk )
Fx (α, n) = Pr {x (n) ≤ α} ∂α1 . . . ∂αk
Sometimes it is sufficient to describe the random process with the first and second order
and a probability density function
distribution (or density)
dFx (α, n)
fx (α, n) =
dα
6 / 52 7 / 52
Stationarity Stationary processes
At a given time, the random process becomes a random variable First order distribution function for the random process
This random variable has statistical properties such as a probability density function, mean
Fx (α, n) = Pr {x (n) ≤ α}
value, variance, moments etc
The random process is called stationary to order one if
stationarity
If all the statistical properties does not change with time, the random process is said to be fx (α, n) = fx (α, n + k ) for all n, k
stationary
This implies that the expectation becomes constant
Stationarity is the same as statistical time-invariance
E {x (n)} = x̄ = constant
8 / 52 9 / 52
The random process is called stationary to order two if If the random process is second order stationary, we realise that we can write k = n2 − n1
which gives
fx (α1 , α2 , n1 , n2 ) = fx (α1 , α2 , n1 + k , n2 + k ) for all k , n1 , n2 Rxx (n1 , n2 ) = Rxx (k ) where k = n2 − n1
A second order stationary process is also first order stationary Written in time continuous notation
10 / 52 11 / 52
Wide sense stationary processes Time averages and ergodicity
A random process is said to be wide sense stationary (WSS) if
The time average of any quantiy is defined as
wide sense stationary Z T
1
The expected value (mean value) of the process is constant A{·} = lim {·}dt
T →∞ 2T −T
E {x (n)} = x̄ A denotes time average as opposed to E that denotes the ensemble average
The autocorrelation is only a function of lag Consider a random realisation x (t ) of the random process X (t )
The time average becomes x̄ = A{x (t )}
Rxx (n1 , n2 ) = Rxx (k )
Similarly, the time autocorrelation Rxx (τ ) = A{x (t )x (t + τ )}
The variance of the random process is finite When all sample functions in the ensemble is considered, we realise that x̄ and Rxx (τ )
becomes random variables
µ2 ≤ ∞
Ergodic processes stated loosely For Wide Sense Stationary (WSS) processes
If the time averages equal the corresponding statistical averages, the process is said to be Rxx (t , t + τ ) = Rxx (τ )
ergodic
Properties for the AC for WSS random processes:
What does this require? At least that the statistical properties does not change with time.
|Rxx (τ )| ≤ Rxx (0)
Rxx (−τ ) = Rxx (τ )
Rxx (0) = E {x 2 (t )}
14 / 52 15 / 52
In the computer: How to estimate the expectation In the computer: How to estimate the PDF
Recall Z ∞
The PDF can be estimated by the normalised histogram
E {x } = αfx (α)d α Why? Well, the histogram is defined as
−∞
How do we estimate the expected value from a random sequence? Histogram
In real life we do not have the probability density function. function N = hist( x(n), bins )
In real life, we never have all realisations, which means that we often have to assume for all x(n) values
ergodicity find the correct bin for x(n)
Count: N(bin) = N(bin) + 1
Given a sequence x (n) we approximate the ensemble average with the time average end
1 XN
E {x } ≈ A{x (n)} = x (n)
N n =1 The histogram gives directly the count of all different values. Normalise this, and we obtain
Why is this an approximate of the expectation? the probability that any value can occur (density).
What happened to the PDF? This multiplied with the hit number of all possible values gives, naturally, the count of all
values.
16 / 52 17 / 52
In the computer: How to estimate the autocorrelation In the computer: How to estimate the autocorrelation
Assuming a large time interval 2T and ergodicity (which implies that the random process is In matlab
WSS) Signal Processing Toolbox:
Correlation and Covariance
Rxx (t , t + τ ) = E {x (t )x (t + τ )} ≈ A{x (t )x (t + τ )}
Z t +T Similar tools in Python SciPy / NumPy.
1
= x (t )x (t + τ )dt ≈ Rxx (τ ) = Rxx (τ )
2T t −T
or in discrete notation
1 XN
Rxx (k ) ≈ A{x (n)x (n + k )} = x (n)x (n + k )
N n=1
18 / 52 19 / 52
What is the autocorrelation Cross correlation
Contains information about the history of the random process Assume two random processes x (t ) and y (t ).
The cross correlation function is defined as
We say they are jointly wide sense stationary if the cross correlation is only a function of time
difference and not absolute time
Rxy (t , t + τ ) = E {x (t )y (t + τ )} = Rxy (τ )
20 / 52 21 / 52
22 / 52 23 / 52
Complex numbers Complex vectors
Mathematical description: We define a complex number as Complex vectors:
z1
z = x + iy , z = x + jy z2
⃗z =
..
where x , y are real numbers and i 2 = −1 or j 2 = −1 .
zN
Convenient representation in wireless comms, radar, sonar
where each zn is complex
Complex conjugate: z ∗ = x − iy , in matlab: conj()
√ p ⃗z T = [z1 , z2 , . . . , zN ] is the transpose. transpose() or .’ in matlab
The absolute value or amplitude r = zz ∗ = x2 + y2
⃗z H = ⃗z T ∗ is the Hermitian transpose, the complex conjugate of the transpose
Euler’s formula
eix = cos x + i sin x ctranspose() or ’ in matlab
√
Note: Often common to use complex random variables ||⃗z || = ⃗z⃗z H is the Euclidean norm or 2-norm. norm() in matlab.
See wikipedia on complex numbers and the matlab manual Read the manual for correct usage. Note that matlab often have multiple usages.
24 / 52 25 / 52
30 / 52 31 / 52
The Rayleigh distribution - derivation of PDF 2 The Rayleigh distribution - derivation of PDF 3
Consider the joint probability that x1 < x ≤ x1 + dx and y1 < y ≤ y1 + dy Coordinate transform
−(x 2 + y 2 )
1 Pr {r1 < r ≤ r1 + dr , θ1 < θ ≤ θ1 + d θ}
Pr {x1 < x ≤ x1 + dx , y1 < y ≤ y1 + dy } = exp dxdy
2πσ 2 2σ 2
2
1 −r
= 2
exp rdrd θ
2πσ 2σ 2
Coordinate transform: dxdy = rdrd θ 2
r −r 1
= exp dr dθ
Pr {x1 < x ≤ x1 + dx , y1 < y ≤ y1 + dy } = Pr {r1 < r ≤ r1 + dr , θ1 < θ ≤ θ1 + d θ} σ 2 2σ 2 2π
−r 2
r 1
fr ,θ (r , θ) = exp
σ2 2σ 2 2π
32 / 52 33 / 52
and
1
fθ (θ) =
2π
Remembering (and checking) the property
Lord Rayleigh. Nobel Prize winner
Z in 1904 in Physics. wikipedia.org
fx (α)d α = 1
The PDF for the amplitude is the so-called Rayleigh distribution, and the PDF for the phase is
the uniform distribution with equal probability of any phase. More details can be found in any book on probability theory, wikipedia. This derivation is taken
from www.dsplog.com
34 / 52 35 / 52
Example: real data collected by a sonar Example: real data collected by a sonar
The HUGIN autonomous underwater vehicle Single channel timeseries from one ping
Wideband interferometric synthetic aperture sonar Consider the collected data a random process.
Transmitter that insonifies the seafloor with a LFM pulse Is the process stationary?
Array of receivers that collects the echoes from the seafloor How do we check for stationarity?
The signal scattered from the seafloor is considered to be random
The signal consists of a signal part and additive noise
36 / 52 37 / 52
38 / 52 39 / 52
Example: Estimating the PDF Example: Estimating the PDF
Question: Is the probability density function Gaussian? The sonar data are complex
(x − mx )2
1 x (t ) = xRe (t ) + jxIm (t ) = a(t )ej ϕ(t )
fx (x ) = p exp −
2πσx2 2σx2
The complex random sequence is considered as two independent random sequences (in a
where mx is the first order moment (the mean value), and σx2 = µ2 is the second order vector) with joint PDF.
central moment (or the variance) We can check the PDF of the real and imaginary part separately
Approach: Compare the theoretical PDF with the estimated PDF (normalised histogram) If xRe (t ) and xIm (t ) are statistically independent Gaussian random processes, the PDF of the
amplitude (or magnitude) q
a (t ) = 2 (t ) + x 2 (t )
xRe Im
should be a Rayleigh distribution, and the PDF of the phase
ϕ(t ) = tan−1 {xIm /xRe }
should be uniform.
40 / 52 41 / 52
42 / 52 43 / 52
Example: Estimating the PDF - Region 2 Example: Estimating the PDF - Conclusion
In region 2, the real and imaginary part fits a Gauss well
The phase is also uniform (all phase values are equally probable)
The magnitude also fits well a Rayleigh distribution
In region 1, this is not the case.
The histogram indicates that the PDF is heavy tailed.
This means that it is more likely to have spikes (large amplitude values) in the timeseries
than in a timeseries with Gaussian PDF.
This actually fits well the theory of acoustic scattering.
What can an estimate of the PDF be used to?
44 / 52 45 / 52
46 / 52 47 / 52
Example: Statistical dependence - Conclusion Original sonar image
The real and imaginary part of the signal is uncorrelated
The individual channels (receiver elements) are correlated
What physical phenomenon could cause this?
The channels are strongly correlated in region 2 (the noise region)
Why is this?
Any other way to detect the correlation between channels?
48 / 52 49 / 52
50 / 52 51 / 52
The project
The course is topic based with 7 different topics
Each topic has one week lectures and one week project work
Roy Edgar Hansen
Each project consists of a set of exercises in matlab or python
Each project delivery is in the form of a presentation
IN5340 / IN9340 Lecture 2
At least 4 projects must be delivered and approved Random variables, vectors and sequences
Each project will be worked through in the class
Each student must deliver his/her own presentation
The student must be prepared to present at the group session
Each student must present their project at least once
52 / 52