DC Stochastic Process

Digital Communication Unit-III
Stochastic Process
Ashok N Shinde
ashok.shinde0349@gmail.com
International Institute of Information Technology
Hinjawadi Pune
July 26, 2017
Ashok N Shinde DC Unit-III Stochastic Process 1/28

Introduction to Stochastic Process
Signals
Deterministic: can be reproduced exactly with repeated
measurements.

Signals
measurements.
s(t) = Acos(2πfc t + θ)

Signals
measurements.
where A,fc and θ are constant.

Signals
measurements.
Random: signal that is not repeatable in a predictable manner.

Signals
measurements.

Signals
measurements.
where A,fc and θ are variable.

Signals
measurements.
where A,fc and θ are variable.
Unwanted signals: Noise

Stochastic Process
Definition:
A stochastic process is a set of random variables indexed in time.
Mathematically:

Stochastic Process
Definition:
Mathematically:
Mathematically relationship between probability theory and
stochastic processes is as follows-

Stochastic Process
Definition:
Mathematically:
Sample point →Sample Function

Stochastic Process
Definition:
Mathematically:
Sample space →Ensemble

Stochastic Process
Definition:
Mathematically:
Random Variable→ Random Process

Stochastic Process
Definition:
Mathematically:
Sample point s is function of time :

Stochastic Process
Definition:
Mathematically:
X(s, t), −T ≤ t ≤ T

Stochastic Process
Definition:
Mathematically:
X(s, t), −T ≤ t ≤ T
Sample function denoted as:

Stochastic Process
Definition:
Mathematically:
X(s, t), −T ≤ t ≤ T
Sample function denoted as:
xj (t) = X(t, sj ), −T ≤ t ≤ T

Stochastic Process
A random process is defined as the ensemble(collection) of time

functions together with a probability rule

Stochastic Process
A random process is defined as the ensemble(collection) of time

functions together with a probability rule
|xj (t)|, j = 1, 2, . . . , n

Stochastic Process
Stochastic Process
Each sample point in S is associated with a sample function x(t)

Stochastic Process
Stochastic Process
X(t, s) is a random process

Stochastic Process
Stochastic Process
is an ensemble of all time functions together with a probability rule

Stochastic Process
Stochastic Process
X(tk , sj ) is a realization or sample function of the random process
{x1 (tk ), x2 (tk ), . . . , xn (tk ) = X(tk , s1 ), X(tk , s2 ), . . . , X(tk , sn )}

Stochastic Process
Stochastic Process
Probability rules assign probability to any meaningful event
associated with an observation An observation is a sample function
of the random process

Stochastic Process
Stochastic Process
A stochastic process X(t, s) is represented by time indexed ensemble

(family) of random variables {X(t, s)}

Stochastic Process
Stochastic Process

Represented compactly by : X(t)

Stochastic Process
Stochastic Process

Represented compactly by : X(t)
“A stochastic process X(t) is an ensemble of time functions, which,
together with a probability rule, assigns a probability to any
meaningful event associated with an observation of one of the
sample functions of the stochastic process”.

Stochastic Process:
Stationary Vs Non-Stationary Process
Stationary Process:

Stochastic Process:
Stationary Process:
If a process is divided into a number of time intervals exhibiting
same statistical properties, is called as Stationary.

Stochastic Process:
Stationary Process:
It is arises from a stable phenomenon that has evolved into a
steady-state mode of behavior.

Stochastic Process:
Stationary Process:
Non-Stationary Process:

Stochastic Process:
Stationary Process:
different statistical properties, is called as Non-Stationary.

Stochastic Process:
Stationary Process:
different statistical properties, is called as Non-Stationary.
It is arises from an unstable phenomenon.

Classes of Stochastic Process:
Strictly Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t = −∞ is said to be

Stationary in the strict sense, or strictly stationary if,


FX(t1 +τ ),X(t2 +τ ),...,X(tk +τ ) (x1 , x2 , . . . , xk ) =
FX(t1 ),X(t2 ),...,X(tk ) (x1 , x2 , . . . , xk ) Where,


X(t1 ), X(t2 ), . . . , X(tk ) denotes RVs obtained by sampling process
X(t) at t1 , t2 , . . . , tk respectively.


FX(t1 ),X(t2 ),...,X(tk ) (x1 , x2 , . . . , xk ) denotes Joint distribution
function of RVs.


function of RVs.
X(t1 + τ ), X(t2 + τ ), . . . , X(tk + τ ) denotes new RVs obtained by
sampling process X(t) at t1 + τ, t2 + τ, . . . , tk + τ respectively. Here
τ is fixed time shift.


function of RVs.
X(t1 + τ ), X(t2 + τ ), . . . , X(tk + τ ) denotes new RVs obtained by
sampling process X(t) at t1 + τ, t2 + τ, . . . , tk + τ respectively. Here
τ is fixed time shift.
FX(t1 +τ ),X(t2 +τ ),...,X(tk +τ ) (x1 , x2 , . . . , xk ) denotes Joint
distribution function of new RVs.

Properties of Strictly Stationary Process:


For k = 1, we have FX(t) (x) = FX(t+τ ) (x) = FX (x) for all t and τ .
First-order distribution function of a strictly stationary stochastic
process is independent of time t.


For k = 2, we have FX(t1 ),X(t2 ) (x1 , x2 ) = FX(0),X(t1 −t2 ) (x1 , x2 ) for
all t1 and t2 . Second-order distribution function of a strictly
stationary stochastic process depends only on the time difference
between the sampling instants and not on time sampled.


Weakly Stationary Process:


A stochastic process X(t) is said to be weakly stationary(Wide-Sense
Stationary) if its second-order moments satisfy:


The mean of the process X(t) is constant for all time t.


The autocorrelation function of the process X(t) depends solely on
the difference between any two times at which the process is sampled.


Summary of Random Processes


Wide-Sense Stationary processes


Strictly Stationary Processes


Ergodic Processes


Ergodic Processes
Non-Stationary processes
Mean, Correlation, and Covariance Functions of WSP
Mean:

Mean:
Mean of real-valued stochastic process X(t), is expectation of the
random variable obtained by sampling the process at some time t, as
shown by

Mean:
shown by
µX (t) = E[X(t)]

Mean:
shown by
µX (t) = E[X(t)]
R∞
µX (t) = −∞ xfX(t) (x)dx

Mean:
shown by
µX (t) = E[X(t)]
R∞
where fX(t) (x) is the first-order probability density function of the
process X(t).

Mean:
shown by
µX (t) = E[X(t)]
R∞
process X(t).
µX (t) = µX for weakly stationary process

Mean:
shown by
µX (t) = E[X(t)]
R∞
process X(t).
Correlation:

Mean:
shown by
µX (t) = E[X(t)]
R∞
process X(t).
Correlation:
Autocorrelation function of the stochastic process X(t) is product of
two random variables, X(t1 ) and X(t2 )

Mean:
shown by
µX (t) = E[X(t)]
R∞
process X(t).
Correlation:
MXX (t1 , t2 ) = E[X(t1 )X(t2 )]

Mean:
shown by
µX (t) = E[X(t)]
R∞
process X(t).
Correlation:
MXX (t1 , t2 ) = E[X(t
R ∞ R1∞ )X(t2 )]
MXX (t1 , t2 ) = −∞ −∞ x1 x2 fX(t1 ),X(x2 ) (x1 , x2 )dx1 dx2
where fX(t1 ),X(x2 ) (x1 , x2 ) is joint probability density function of the
process X(t) sampled at times t1 and t2 . MXX (t1 , t2 ) is a
second-order moment. It is depend only on time difference t1 − t2 so
that the process X(t) satisfies the second condition of weak
stationarity and reduces to.

Mean:
shown by
µX (t) = E[X(t)]
R∞
process X(t).
Correlation:
MXX (t1 , t2 ) = E[X(t
R ∞ R1∞ )X(t2 )]
MXX (t1 , t2 ) = −∞ −∞ x1 x2 fX(t1 ),X(x2 ) (x1 , x2 )dx1 dx2
where fX(t1 ),X(x2 ) (x1 , x2 ) is joint probability density function of the
process X(t) sampled at times t1 and t2 . MXX (t1 , t2 ) is a
second-order moment. It is depend only on time difference t1 − t2 so
that the process X(t) satisfies the second condition of weak
stationarity and reduces to.
MXX (t1 , t2 ) = E[X(t1 )X(t2 )] = RXX (t2 − t1 )

Properties of Autocorrelation Function

Autocorrelation function of a weakly stationary process X(t) can
also be represented as

RXX (τ ) = E[X(t + τ )X(t)]

RXX (τ ) = E[X(t + τ )X(t)]
where τ denotes a time shift; that is,t = t2 and τ = t1 − t2

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

RXX (τ ) = E[X(t + τ )X(t)]
Properties:
RXX (0) = E X 2 (t) (Mean-Square Value)


RXX (τ ) = E[X(t + τ )X(t)]
Properties:

RXX (τ ) = RXX (−τ ) also RXX (τ ) = E[X(t − τ )X(t)]
(Symmetry)

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
|RXX (τ )| ≤ RXX (0) (Bound)

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
R (τ )
ρXX (τ ) = RXX (0) (Normalization [−1, 1])
XX

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
R (τ )
XX
Covariance:

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
R (τ )
XX
Covariance:
Autocovariance function of a weakly stationary process X(t) is
defined by

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
R (τ )
XX
Covariance:
defined by
CXX (t1 , t2 ) = E[(X(t1 ) − µx )(X(t2 ) − µx )]

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
R (τ )
XX
Covariance:
defined by
CXX (t1 , t2 ) = E[(X(t1 ) − µx )(X(t2 ) − µx )]
CXX (t1 , t2 ) = RXX (t2 − t1 ) − µ2x

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
R (τ )
XX
Covariance:
defined by
CXX (t1 , t2 ) = E[(X(t1 ) − µx )(X(t2 ) − µx )]
CXX (t1 , t2 ) = RXX (t2 − t1 ) − µ2x
The autocovariance function of a weakly stationary process X(t)
depends only on the time difference (t2 − t1 )

RXX (τ ) = E[X(t + τ )X(t)]
Properties:

(Symmetry)
R (τ )
XX
Covariance:
defined by
CXX (t1 , t2 ) = E[(X(t1 ) − µx )(X(t2 ) − µx )]
CXX (t1 , t2 ) = RXX (t2 − t1 ) − µ2x
The autocovariance function of a weakly stationary process X(t)
depends only on the time difference (t2 − t1 )
The mean and autocorrelation function only provide a weak
description of the distribution of the stochastic process X(t).

Example Question:
InSem 2014, InSem 2015 (Sin), InSem 2016 (− π2 ≤ Θ ≤ π
2
)
Examples
Show that the Random Process X(t) = Acos(ωc t + Θ) is wide sense
stationary process, where Θ is RV uniformly distributed in range (0, 2π)
Answer:
The ensemble consist of sinusoids of constant amplitude A and constant
frequency ωc , but phase Θ is random.

Example Question:
2
)
Examples
Answer:
The phase is equally likely to any value in the range (0, 2π).

Example Question:
2
)
Examples
Answer:
Θ is RV uniformly distributed over the range (0, 2π).

Example Question:
2
)
Examples
Answer:
1
fΘ (θ) = , 0 ≤ θ ≤ 2π
2π

Example Question:
2
)
Examples
Answer:
1
fΘ (θ) = , 0 ≤ θ ≤ 2π
2π
= 0, elsewhere

Example Question:
2
)
Answer:
Because cos(ωc t + Θ) is function of RV Θ, Mean of Random Process
X(t) is

Example Question:
2
)
Answer:
X(t) is
X(t) = Acos(ωc t + Θ)

Example Question:
2
)
Answer:
X(t) is
= Acos(ωc t + Θ)

Example Question:
2
)
Answer:
X(t) is
= RAcos(ωc t + Θ)
2π
cos(ωc t + Θ) = 0 cos(ωc t + θ)fΘ (θ)dθ

Example Question:
2
)
Answer:
X(t) is
= RAcos(ωc t + Θ)
2π
1 R 2π
= cos(ωc t + θ)dθ
2π 0

Example Question:
2
)
Answer:
X(t) is
= RAcos(ωc t + Θ)
2π
1 R 2π
2π 0
=0

Example Question:
2
)
Answer:
X(t) is
= RAcos(ωc t + Θ)
2π
1 R 2π
2π 0
=0
X(t) =0

Example Question:
2
)
Answer:
X(t) is
= RAcos(ωc t + Θ)
2π
1 R 2π
2π 0
=0
X(t) =0
Thus the ensemble mean of sample function amplitude at any time
instant t is zero.

Example Question:
2
)
Answer:
X(t) is
= RAcos(ωc t + Θ)
2π
1 R 2π
2π 0
=0
X(t) =0
Thus the ensemble mean of sample function amplitude at any time
instant t is zero.
The Autocorrelation function RX X(t1 , t2 ) for this process can also be
determined as
RXX (t1 , t2 ) = E[X(t1 )X(t2 )]
= A2 cos(ωc t1 + Θ)cos(ωc t2 + Θ)

Example Question:
2
)
Answer:
Continue. . .

Example Question:
2
)
Answer:
Continue. . .

Example Question:
2
)
Answer:
Continue. . .
A2 h i
= cos(ωc (t2 − t1 )) + cos(ωc (t2 + t1 ) + 2Θ)
2

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
The term cos(ωc (t2 − t1 )) does not contain RV Hence,

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
cos(ωc (t2 − t1 )) = cos(ωc (t2 − t1 ))

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
cos(ωc (t2 − t1 )) = cos(ωc (t2 − t1 ))
The term cos(ωc (t2 + t1 ) + 2Θ) is a function of RV Θ, and it is

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
cos(ωc (t2 − t1 )) = cos(ωc (t2 − t1 ))
1 R 2π
cos(ωc (t2 + t1 ) + 2Θ) = cos(ωc (t2 + t1 ) + 2θ)dθ
2π 0

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
cos(ωc (t2 − t1 )) = cos(ωc (t2 − t1 ))
1 R 2π
2π 0
=0

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
cos(ωc (t2 − t1 )) = cos(ωc (t2 − t1 ))
1 R 2π
2π 0
=0
A2
RXX (t1 , t2 ) = cos(ωc (t2 − t1 )),
2

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
cos(ωc (t2 − t1 )) = cos(ωc (t2 − t1 ))
1 R 2π
2π 0
=0
A2
RXX (t1 , t2 ) = cos(ωc (t2 − t1 )),
2
2
A
RXX (τ ) = cos(ωc (τ )), · · · τ = t2 − t1
2

Example Question:
2
)
Answer:
Continue. . .
A2 h i
2
cos(ωc (t2 − t1 )) = cos(ωc (t2 − t1 ))
1 R 2π
2π 0
=0
A2
RXX (t1 , t2 ) = cos(ωc (t2 − t1 )),
2
2
A
RXX (τ ) = cos(ωc (τ )), · · · τ = t2 − t1
2
2
A
From X(t) = 0 and RXX (τ ) = cos(ωc (τ )) it is clear that X(t) is
2
Wide Sense Stationary Process

Time Vs Ensemble Average and Ergodic Process
Ensemble Average

Ensemble Average
Difficult to generate a number of realizations of a random process

Ensemble Average
Use time averages

Ensemble Average
Use time averages
1 R +T
Mean ⇒ µX (T ) = x(t)dt
2T −T

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
Autocorrelation ⇒ RXX (τ, T ) = x(t)x(t + τ )dt
2T −T

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
Ergodicity: A random process is called Ergodic if

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
it is ergodic in mean:

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
limT →∞ µX (T ) = µX

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
limT →∞ var [µX (T )] = 0

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
it is ergodic in autocorrelation:

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
limT →∞ RXX (τ, T ) = RXX (τ )

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
limT →∞ var [RXX (τ, T )] = 0

Ensemble Average
Use time averages
1 R +T
2T −T
1 R +T
2T −T
Ergodic Process
limT →∞ var [RXX (τ, T )] = 0
where µX and RXX (τ ) are the ensemble averages of the same
random process.

Transmission of a Weakly Stationary Process through a
LTI Filter
Linear Time Invariant Filter

LTI Filter

Suppose that a stochastic process X(t) is applied as input to a linear
time-invariant filter of impulse response h(t), producing a new
stochastic process Y (t) at the filter output.

LTI Filter


LTI Filter

It is difficult to describe the probability distribution of the output

stochastic process Y(t), even when the probability distribution of the
input stochastic process X(t) is completely specified

LTI Filter


For defining the mean and autocorrelation functions of the output
stochastic process Y (t) in terms of those of the input X(t),
assuming that X(t) is a weakly stationary process.

LTI Filter


For defining the mean and autocorrelation functions of the output
stochastic process Y (t) in terms of those of the input X(t),
assuming that X(t) is a weakly stationary process.
Transmission of a process through a linear time-invariant filter is
governed by the convolution integral

LTI Filter
Continue. . .

LTI Filter
Continue. . .
we may thus express the output stochastic process Y (t) in terms of
the input stochastic process X(t) as

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
where τ1 is local time. Hence, the mean of Y (t) is

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y (t)]

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
Provided that the expectation E [X(t)] is finite for all t and the filter
is stable.

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
is stable. R
+∞
µY (t) = −∞ h(τ1 )E [X(t − τ1 )] dτ1

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
is stable. R
+∞
µY (t) = −∞ h(τ1 )E [X(t − τ1 )] dτ1
R +∞
µY (t) = −∞ h(τ1 )µX (t − τ1 )dτ1

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
is stable. R
+∞
µY (t) = −∞ h(τ1 )E [X(t − τ1 )] dτ1
R +∞
µY (t) = −∞ h(τ1 )µX (t − τ1 )dτ1
When the input stochastic process X(t) is weakly stationary, the
mean µX (t) is a constant µX

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
is stable. R
+∞
µY (t) = −∞ h(τ1 )E [X(t − τ1 )] dτ1
R +∞
µY (t) = −∞ h(τ1 )µX (t − τ1 )dτ1
R +∞
µY = µX −∞ h(τ1 )dτ1

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
is stable. R
+∞
µY (t) = −∞ h(τ1 )E [X(t − τ1 )] dτ1
R +∞
µY (t) = −∞ h(τ1 )µX (t − τ1 )dτ1
R +∞
µY = µX H(0)

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
is stable. R
+∞
µY (t) = −∞ h(τ1 )E [X(t − τ1 )] dτ1
R +∞
µY (t) = −∞ h(τ1 )µX (t − τ1 )dτ1
R +∞
µY = µX H(0)
where H(0) is the zero-frequency response of the system

LTI Filter
Continue. . .
R +∞
Y (t) = −∞ h(τ1 )X(t − τ1 )dτ1
µY (t) = E [Y
hR (t)] i
+∞
µY (t) = E −∞ h(τ1 )X(t − τ1 ) dτ1
is stable. R
+∞
µY (t) = −∞ h(τ1 )E [X(t − τ1 )] dτ1
R +∞
µY (t) = −∞ h(τ1 )µX (t − τ1 )dτ1
R +∞
µY = µX H(0)
where H(0) is the zero-frequency response of the system
“The mean of the stochastic process Y(t) produced at the output of
a linear time-invariant filter in response to a weakly stationary
process X(t), acting as the input process, is equal to the mean of
X(t) multiplied by the zero-frequency response of the filter.”
LTI Filter
Autocorrelation

LTI Filter
Autocorrelation
Consider the autocorrelation function of the output stochastic
process Y (t)

LTI Filter
Autocorrelation
Consider the autocorrelation function of the output stochastic
process Y (t)
MY Y (t, u) = E [Y (t)Y (u)]

where t and u denote two values of the time at which the output
process Y (t) is sampled
Z +∞ Z +∞
.=E h(τ1 )X(t − τ1 )dτ1 h(τ2 )X(u − τ2 )dτ2
−∞ −∞
Here again, provided that the mean-square value E X 2 (t) is finite

for all t and the filter is stable
Z +∞ Z +∞
= h(τ1 ) dτ2 h(τ2 )E [X(t − τ1 )X(u − τ2 )] dτ1
−∞ −∞
Z +∞ Z +∞
= h(τ1 ) dτ2 h(τ2 )MXX (t − τ1 , u − τ2 ) dτ1
−∞ −∞
When the input X(t) is a weakly stationary process, the

autocorrelation function of X(t) is only a function of the difference
between the sampling times tτ1 and uτ2 . Thus, putting τ = ut

LTI Filter
Autocorrelation

LTI Filter
Autocorrelation

LTI Filter
Autocorrelation
Z +∞ Z +∞
RY Y (τ ) = h(τ1 )h(τ2 )RXX (τ + τ1 − τ2 )dτ1 dτ2
−∞ −∞
which depends only on the time difference τ

LTI Filter
Autocorrelation
Z +∞ Z +∞
−∞ −∞
.
“If the input to a stable linear time invariant filter is a weakly
stationary process, then the output of the filter is also a weakly
stationary process”.

LTI Filter
Autocorrelation
Z +∞ Z +∞
−∞ −∞
.
mean square value of the output process Y (t) is obtained by putting
τ = 0. We haveRY Y (0) = E Y 2 (t)


LTI Filter
Autocorrelation
Z +∞ Z +∞
−∞ −∞
.
τ = 0. We haveRY Y (0) = E Y 2 (t)

Z +∞ Z +∞
E Y 2 (t) =

h(τ1 )h(τ2 )RXX (τ1 − τ2 )dτ1 dτ2
−∞ −∞

LTI Filter
Autocorrelation
Z +∞ Z +∞
−∞ −∞
.
τ = 0. We haveRY Y (0) = E Y 2 (t)

Z +∞ Z +∞
E Y 2 (t) =

h(τ1 )h(τ2 )RXX (τ1 − τ2 )dτ1 dτ2
−∞ −∞
which, of course, is a constant.

Power Spectral Density of a Weakly Stationary Process
The impulse response of a linear time-invariant filter is equal to the
inverse Fourier transform of the frequency response of the filter

Using H(f ) to denote the frequency response of the filter, we may
thus write

Using H(f ) to denote the frequency response of the filter, we may
thus write
Z +∞
h(τ1 ) = H(f ) exp(j2πf τ1 )df
−∞
Z +∞ Z +∞
E Y 2 (t) =

h(τ1 )h(τ2 )RXX (τ1 − τ2 )dτ1 dτ2
−∞ −∞
Z +∞ Z +∞ Z +∞
= H(f ) exp(j2πf τ1 )df h(τ2 )RXX (τ1 − τ2 )dτ1 dτ2
−∞ −∞ −∞
Z +∞ Z +∞ Z +∞
= H(f ) h(τ2 )dτ2 RXX (τ1 − τ2 ) exp(j2πf τ1 )dτ1 df
−∞ −∞ −∞
by τ = τ1 − τ2
Z +∞ Z +∞ Z +∞
= H(f ) h(τ2 ) exp(j2πf τ2 )dτ2 RXX (τ ) exp(−j2πf τ )dτ
−∞ −∞ −∞
Z +∞ Z+∞
= |H(f )|2 RXX (τ ) exp(−j2πf τ )dτ df
−∞ −∞
as |H(f )|2 = H(f )H ∗ (f ) and square brackets represents fourier

transform of the autocorrelation function RXX of the input process
X(t) Ashok N Shinde DC Unit-III Stochastic Process 19/28
We may now define a new function for fourier transform of the

autocorrelation function.


The new function SXX (f ) is called the power spectral density, or
power spectrum, of the weakly stationary process X(t) and denoted
as


as
Z +∞
SXX (f ) = RXX (τ ) exp(−j2πf τ )dτ
−∞
Z +∞
E Y 2 (t) = |H(f )|2 SXX (f )df

−∞
which is the desired frequency-domain equivalent to the time-domain

relation


as
Z +∞
SXX (f ) = RXX (τ ) exp(−j2πf τ )dτ
−∞
Z +∞
E Y 2 (t) = |H(f )|2 SXX (f )df

−∞
which is the desired frequency-domain equivalent to the time-domain

relation
The mean-square value of the output of a stable linear time-invariant

filter in response to a weakly stationary process is equal to the
integral over all frequencies of the power spectral density of the input
process multiplied by the squared magnitude response of the filter.

Properties of the Power Spectral Density(PSD)
Properties of PSD

Properties of PSD
1 Zero Correlation among Frequency Components

Properties of PSD
The individual frequency components of the power spectral density
SXX (f ) of a weakly stationary process X(t) are uncorrelated with
each other.

Properties of PSD
each other.
No overlap, and therefore no correlation.

Properties of PSD
each other.

Properties of PSD
each other.
2 Zero-frequency Value of Power Spectral Density

Properties of PSD
each other.

The zero-frequency value of the power spectral density of a weakly
stationary process equals the total area under the graph of the
autocorrelation function

Properties of PSD
each other.

Z +∞
SXX (0) = RXX (τ )dτ
−∞

Properties of PSD
each other.

Z +∞
−∞
3 Mean-square Value of Stationary Process

Properties of PSD
each other.

Z +∞
−∞

The mean-square value of a weakly stationary process X(t) equals the
total area under the graph of the power spectral density of the process

Properties of PSD
each other.

Z +∞
−∞

The mean-square value of a weakly stationary process X(t) equals the
total area under the graph of the power spectral density of the process
Z +∞
E X 2 (t) =

SXX (f )df
−∞

Properties of PSD

Properties of PSD
4 Nonnegativeness of Power Spectral Density

Properties of PSD
The power spectral density of a stationary process X(t) is always
nonnegative.

Properties of PSD
nonnegative.

Properties of PSD
nonnegative.
SXX (f ) ≥ 0

Properties of PSD
nonnegative.
SXX (f ) ≥ 0
5 Symmetry

Properties of PSD
nonnegative.
SXX (f ) ≥ 0
5 Symmetry
The power spectral density of a real-valued weakly stationary process
is an even function of frequency

Properties of PSD
nonnegative.
SXX (f ) ≥ 0
5 Symmetry
SXX (f ) = SXX (−f )

Properties of PSD
nonnegative.
SXX (f ) ≥ 0
5 Symmetry
6 Normalization

Properties of PSD
nonnegative.
SXX (f ) ≥ 0
5 Symmetry
6 Normalization
The power spectral density, appropriately normalized, has the
properties associated with a probability density function in probability
theory

Properties of PSD
nonnegative.
SXX (f ) ≥ 0
5 Symmetry
6 Normalization
The power spectral density, appropriately normalized, has the
properties associated with a probability density function in probability
theory
SXX (f )
PXX (f ) = R ∞
−∞ SXX (f )df

The Gaussian Process
Stochastic process of interest is the Gaussian process which builds
on the Gaussian distribution.

It is the most frequently encountered random process in the study of
communication systems.

Gaussian Process

Gaussian Process
Let us suppose that we observe a stochastic process X(t) for an
interval that starts at time t = 0 and lasts until t = T

Gaussian Process
we weight the process X(t) by some function g(t) and then integrate
the product g(t)X(t) over the observation interval (0, T )

Gaussian Process
Z T
Y = g(t)X(t)dt
0

Gaussian Process
Z T
Y = g(t)X(t)dt
0
We refer to Y as a linear functional of X(t)

Gaussian Process
Z T
Y = g(t)X(t)dt
0
1 If the weighting function g(t) is such that the mean-square value of
the random variable Y is finite

Gaussian Process
Z T
Y = g(t)X(t)dt
0
2 And if the random variable Y is a Gaussian-distributed random
variable for every g(t) in this class of functions

Gaussian Process
Z T
Y = g(t)X(t)dt
0
Then the process X(t) is said to be a Gaussian process.

Gaussian Process
Z T
Y = g(t)X(t)dt
0
Then the process X(t) is said to be a Gaussian process.
A process X(t) is said to be a Gaussian process if every linear
functional of X(t) is a Gaussian random variable
Properties The Gaussian Process
Random variable Y has a Gaussian distribution if its probability
density function has the form
(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
where µ is the mean and σ 2 is the variance of the random variable Y

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
If a Gaussian process X(t) is applied to a stable linear filter, then the
stochastic process Y(t) developed at the output of the filter is also
Gaussian.

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
Gaussian.

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
Gaussian.

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
Gaussian.
2 Stationarity

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
Gaussian.
2 Stationarity
If a Gaussian process is weakly stationary, then the process is also
strictly stationary.

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
Gaussian.
2 Stationarity

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
Gaussian.
2 Stationarity
3 Independence

(y − µ)2

1
fY (y) = √ exp −
2πσ 2 2σ 2
1 Linear Filtering
Gaussian.
2 Stationarity
3 Independence
If the random variables X(t1 ), X(t2 ), . . . , X(tn ), obtained by
respectively sampling a Gaussian process X(t) at times t1 , t2 , . . . , tn ,
are uncorrelated, that is

E (X(tk ) − µX(tk ) )(X(ti ) − µX(ti ) ) = 0 i 6= k
then these random variables are statistically independent.
Noise
Unwanted signals that tend to disturb the transmission and

processing of signals in communication systems.

Noise

Sources of noise in a communication system

Noise

External

Noise

External
Atmospheric noise
Galactic noise
Man-made noise

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal
Noise that arises from the phenomenon of spontaneous fluctuations
of current flow that is experienced in all electrical circuits

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal
Shot noise - Discrete nature of current flow in electronic devices.
Thermal noise - Random motion of electrons in a conductor.

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal
Shot noise - Discrete nature of current flow in electronic devices.
Thermal noise - Random motion of electrons in a conductor.
The noise analysis of communication systems is based on a source of
noise called white-noise, which is to be discussed next.

White Noise
Adjective ’white’ is used in the sense that white light contains equal
amount of all frequencies within the visible band of electromagnetic
radiation.

White Noise
radiation.
White noise denoted by W (t), is a stationary process whose PSD
SW (f ) has constant value across all frequencies.

White Noise
radiation.
PSD of white noise is
N0
SW W (f ) = 2
for all f

White Noise
radiation.
N0
SW W (f ) = 2
for all f
Its auto-correlation function is
N0
RW W (τ ) = 2
δ(τ )

White Noise
radiation.
N0
SW W (f ) = 2
for all f
Its auto-correlation function is
N0
RW W (τ ) = 2
δ(τ )

References I
1 S. Haykin. Digital Communication, John Wiley & Sons, 2009.

2 A.B Carlson, P B Crully, J C Rutledge, Communication Systems,
Fourth Edition, McGraw Hill Publication.

Author Information
For further information please contact
Prof. Ashok N Shinde

Department of Electronics & Telecommunication Engineering
Hope Foundation’s
International Institute of Information Technology, (I 2 IT )
Hinjawadi, Pune 411 057
Phone - +91 20 22933441
www.isquareit.edu.in|ashoks@isquareit.edu.in

DC Stochastic Process

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DC Stochastic Process

Uploaded by

Copyright:

Available Formats

Digital Communication Unit-III

July 26, 2017

Ashok N Shinde DC Unit-III Stochastic Process 1/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

A random process is defined as the ensemble(collection) of time

Ashok N Shinde DC Unit-III Stochastic Process 4/28

A random process is defined as the ensemble(collection) of time

Ashok N Shinde DC Unit-III Stochastic Process 4/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

A stochastic process X(t, s) is represented by time indexed ensemble

Ashok N Shinde DC Unit-III Stochastic Process 5/28

A stochastic process X(t, s) is represented by time indexed ensemble

Ashok N Shinde DC Unit-III Stochastic Process 5/28

A stochastic process X(t, s) is represented by time indexed ensemble

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

Properties of Strictly Stationary Process:

Ashok N Shinde DC Unit-III Stochastic Process 8/28

Properties of Strictly Stationary Process:

Ashok N Shinde DC Unit-III Stochastic Process 8/28

Properties of Strictly Stationary Process:

Ashok N Shinde DC Unit-III Stochastic Process 8/28

Properties of Strictly Stationary Process:

Ashok N Shinde DC Unit-III Stochastic Process 8/28

Properties of Strictly Stationary Process:

Ashok N Shinde DC Unit-III Stochastic Process 8/28