Random Variables

Principles of Communications
Weiyao Lin, PhD

Associate Professor
Dept. of Electronic Engineering
Shanghai Jiao Tong University
Chapter 2: Signal, Random Process, and Spectra
Textbook: 2
2.1
1-2
2.6,
6 55.1
1-5
5.3
3
1
Signal and Noise in
Communication Systems
In communication systems,
y , the received waveform is usuallyy
categorized into the desired part containing the information and the
extraneous or undesired part. The desired part is called the signal, and
the undesired part is called noise.
noise
Noise is one of the most critical and fundamental concepts affecting
communication systems
The entire subject of communication systems is all about methods to
overcome the distorting or bad effects of noise
To do so, understanding random variables and random processes
becomes quite essential
Typical noise source
2009/2010 Meixia Tao @ SJTU 2

Topics to be Covered
2.1.Signals
2.2.Review of probability and random variables
2.3.Random Processes: basic concepts
2.4.Guassian and White Processes

What is Signal?
Any physical quantity that varies with time, space, or any
other independent variables is called a signal
In communication systems
systems, signals are used to transmit
information over a communication channel. Such signals
are called information-bearing signals

Classification of Signals
Signals can be characterized in several ways
Continuous-time signal vs. discrete-time signal
Continuous valued signal vs
vs. discrete-valued signal
o Continuous-time and continuous valued: analog signal (speech)
o Discrete-time and discrete valued: digital
g signal
g ((CD))
o Discrete-time and continuous valued: sampled signal
o Continuous-time and discrete valued: quantized signal

Deterministic signal
g vs. random signal
g

Energy and Power
Energy ∞ T /2
Ex = ∫
−∞
x(t ) 2 dt = lim
T →∞ ∫
−T /2
x(t ) 2 dt
T /2
1
T →∞ T ∫
Power
P Px = lim x (t ) 2
dt
−T /2
A signal is an energy signal if and only if Ex is finite

A signal is a power signal if and only if Px is finite
Physically realizable waveforms are of energy-type
Mathematical models are often of power-type

Probability
Let A be an event in a sample space S
The probability P(A) is a real number that
measures the
th likelihood
lik lih d off the
th eventt A
Axioms
A i off P
Probability
b bilit
1)
2)
3) Let A and B are two mutually exclusive events, i.e.
Then

Elementary Properties of Probability

When A and B are NOT mutually exclusive
If then

Conditional Probability
Consider two events A and B in a random experiment
p
The probability that event A will occur GIVEN that B has
occurred, P(A|B), is called the conditional probability
The probability that both A and B occur,
is called the joint probability
Joint
J i t and
d conditional
diti l probabilities
b biliti are related
l t dbby
P( AB ) = P( B ) P( A | B ) = P( A) P( B | A)
P( AB ) P( AB )
Alternatively, P ( A | B ) = P( B | A) =
P( B ) P( A)
Two events A and B are said statistically independent iff
then and
Law of Total Probability
Let be mutually exclusive events
with
Then for anyy event B we have

Bayes’ Theorem
Bayes
An extremely useful relationship for conditional
probabilities is Bayes’ theorem
Let are mutually exclusive events such
that and B is an arbitrary event with nonzero
probability. Then
This formula will be used to derive the structure of the

optimal receiver
Example
Consider a binary communication system
P(0) = 0.3 P01 = P(receive 1 | sent 0) = 0.01
P00 = P(receive 0 | sent 0) = 1- P01 = 0.09
P10 = P(receive 0 | sent 1) = 0.1
P(1) = 0.7
07
P11 = P(receive 1 | sent 1) = 1- P10 = 0.9
What is the probability that the output of this channel is 1?

Assuming that we have observed a 1 at the output
output, what is
the probability that the input to the channel was a 1?

Random Variables (r.v.)
A r.v. is a real
real-valued
valued function assigned to the events of the
sample space S. denoted by capital letters X, Y, etc
A r.v.
r v may be
Discrete-valued: range is finite (e.g. {0,1}), or countable
infinite (e.g.
(e g {1,2,3
{1 2 3 …})
})
Continuous-valued: range is uncountable infinite (e.g. )

The Cumulative distribution function (CDF), or simply the
probability distribution of a r.v. X, is
Δ
FX ( x ) = P ( X ≤ x )
Key properties of CDF

1. 0 ≤ FX(x) ≤ 1 with
2. FX(x) is a non-decreasing function of x
3. F (x1 < X ≤ x2) = FX(x2) − FX(x1)
16
Probability Density Function (PDF)
The PDF, of a r.v. X, is defined as
Δ
f X ( x ) = FX ( x ) FX ( x ) = ∫ f X ( y )dy
d x
or
dx −∞
Key properties of PDF

∞
1. p X ( x) ≥ 0 2. ∫
−∞
p X ( x)dx = 1
x2
3. P ( x1 < X ≤ x2 ) = PX ( x2 ) − PX ( x1 ) = ∫
x1
p X ( x)dx
Area = P ( x1 < X ≤ x2 )
FX ( x ) f X (x )
1 1
Area = f X ( x )dx
P( x1 < X ≤ x2 )
Xmin 0 x1 x2 Xmax x Xmin 0 x1 x2 x x+dx

d Xmax x
Joint Distribution
In many situation, one must consider TWO or more r.v.’s
r.v. s
⇒ joint distribution function
Consider 2 rr.v.
v ’ss X and Y,
Y joint distribution function is
defined as F ( x, y ) = P( X ≤ x, Y ≤ y )
XY
∂ 2 FXY ( x, y )
and joint PDF is f XY ( x, y ) =
∂x∂y
Key properties of joint distribution
∞ ∞
∫ ∫
−∞ −∞
p XY ( x, y )dxdy = 1
y2 x2
P( x1 < X ≤ x2 , y1 < Y ≤ y2 ) = ∫ ∫ p XY ( x, y )dxdy
y1 x1

Marginal distribution
∞ x
PX ( x) = P( X ≤ x, − ∞ < Y < ∞) = ∫ ∫ p XY (α , β )dαdβ
−∞ −∞
y ∞
PY ( y ) = ∫ ∫ p XY (α , β )dαdβ
−∞ −∞
Marginal density
∞
p X ( x) = ∫ p XY ( x, β )dβ
−∞
X and Y are said to be independent iff

PXY ( x, y ) = PX ( x) PY ( y )
p XY ( x, y ) = p X ( x) pY ( y )

Statistical Averages
Let X be a rr.v.
v Then X can either continuous or
discrete. For the moment, consider a discrete r.v.
which takes on the ppossible values x1, x2, …,, xM with
respective probabilities P1, P2, …, PM. Then the mean
or expected value of X is
M
m X = E [X ] = ∑ xi Pi
i =1
where “E[]” denotes the expectation operation

( t ti ti ll averaging)
(statistically i )

If X is continuous,
continuous then
mX = E [ X ] = ∫ xf X ( x )dx
∞
−∞
This is the first moment of the random variable X

Let g(
g(X)) be a function of X,, then
E [g ( X )] = ∫ g ( x) p X ( x)dx
∞
−∞
F
For the
th special (X) =xn, we obtain
i l case off g(X) the nth
bt i th
moment of X, that is
EX[ ]= ∫
n
∞
−∞
x n p X ( x )dx
Let n = 2,
2 we have the mean
mean-square
square value of X as
EX[ ]= ∫
2
∞
−∞
x 2 p X ( x )dx

n-th
n th Central moment is
[
E ( X − mX ) = ∫
n
−∞
](
∞
x − mx ) f X ( x )dx
n
The expected value of second central moment

((n=2)) is called variance
σ X2 = E [( X − mX )2 ]
[
= E X 2 − 2mX X + m2X ]
= E [X ] − m2 2
X
σX, square-root of the variance, is called the

standard deviation
deviation. It is the average distance from
the mean, a measure of the concentration of X
around the mean
Correlation
In considering multiple varaibles, the joint moments like
correlation and covariance between pairs of r.v.s are most
useful
Correlation of the two r.v. X and Y is defined as
RXY = E [ XY ] = ∫
∞ ∞
−∞ −∞ ∫ xyf XY ( x, y )dxdy
Correlation of X and Y is the mean of the product X and Y

Correlation of the two centered r.v. X-E[X] and Y-E[Y], is
called the covariance of X and Y
Cov XY = E [( X − E[ x])(Y − E[ y ])]
23
The covariance of X and Y normalized w r t σX σY is
w.r.t.
referred to the correlation coefficient of X and Y:
C ( XY )
Cov
ρ XY =
σ XσY
X and Y are uncorrelated iff their correlation coefficient is 0
ρ XY = 0
X and Y are orthogonal iff their correlation is 0
R XY = E[ XY ] = 0
If X and Y are independent,

p , then theyy are uncorrelated.
However, the converse is not true (The Gaussian case is
the only exception)
24
Some Useful Probability Distributions
Discrete Distribution
Binary distribution
Binomial distribution
Continuous Distribution
Uniform distribution
Gaussian distribution (most important one)
Rayleigh distribution (very important in mobile and
wireless
i l communications)
i ti )

Binary Distribution
Let X be a discrete random variable that has two
possible values, say X = 1 or X = 0. Distribution of X
can be described byypprobability
y mass function (p
(pmf))
This is frequently
y used to model binary
y data
Mean:
Variance

Binomial Distribution
Let , where are independent
binary r.v.s with
Then
Th
where
That is, the probability that Y = k is the probability that
k of the Xi are equal
q to 1 and n-k are equal
q to 0
Mean:
Variance:
V i
27
Example
Suppose that we transmit a 31-bit
31 bit long sequence with
error correction capability up to 3 bit errors
If the probability of a bit error is p = 0.001,
0 001 what is the
probability that this sequence is received in errors?
If no error correction is used, the error probability is

Uniform Distribution
The
Th pdf
df off uniform
if distribution
di t ib ti iis given
i b
by
Any example?

Gaussian Distribution
The Gaussian distribution, also called normal distribution,
is byy far the most important
p distribution in the statistical
analysis of communication systems
The PDF of a Gaussian r.v. is
1 ⎡ 1 2⎤
f X ( x) = exp⎢− 2 ( x − mX ) ⎥
2πσ X ⎣ 2σ X
2
⎦
A Gaussian r.v. is completely determined by its mean and

variance,
i and
dhhence usually
ll ddenoted
t d as
pX(x)
X ~ N (m X , σ X2 )
x
0 mX
The Q
Q-Function
Function
The Q-function is a standard form to express error
probabilities without
itho t a closed form
∞ 1 ⎛ u2 ⎞
Q( x ) = ∫ exp⎜⎜ − ⎟⎟du
x 2π ⎝ 2⎠
The Q-function is the area under the tail of a Gaussian p
pdf
with mean zero and variance one
Extremely important in error probability analysis!!!

31
More about Q-Function
Q Function
Q
Q-function
u c o iss monotonically
o o o ca y dec
decreasing
eas g
Some features
Craig’s alternative form of Q-function (IEEE MILCOM’91)
Upper
U b
bound
d
If we have a Gaussian variable X ~ N ( μ , σ 2 ) , then

⎛ x−μ ⎞
Pr( X > x) = Q ⎜ ⎟
⎝ σ ⎠
Joint Gaussian Random Variables
X1, X2, …, Xn are jointly Gaussian iff
x is a column vector
m is the vector of the means
C is the covariance matrix

Two-Variate
Two Variate Gaussian PDF
Given two rr.v.s:
v s: X1 and X2 that are joint Gaussian
Then

For uncorrelated X and Y
Y, ii.e.
e
X1 and X2 are also independent
If X1 and X2 are Gaussian and uncorrelated,

then they are independent.
Rayleigh Distribution
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 1 2 3 4 5 6
Rayleigh distributions are frequently used to model fading

for non
non-line
line of sight (NLOS) signal transmission
Very important for mobile and wireless communications

Sums of Random Variables
Consider a sequence of r.v.’s
r.v. s
Weak Law of Large Numbers
Let
A
And
d assume th
thatt Xi’s
’ are uncorrelated
l t d with
ith th
the same
mean and variance
Then
So what?
i.e. the average converges to the expected value

Central Limit Theorem
Let be a set of independent random
variables with common mean and common variance
Next let
Th
Then as , the
th distribution
di t ib ti off Y will
ill ttend
d ttowards
d a
Gaussian distribution
Key Conclusion: the sum of random variables is “Gaussian”
Thermal noise results from the random movement of many

electrons – it is well modeled by a Gaussian distribution.

Example

Random Process
A random process is the natural extension of random
variables when dealing with signals
Also referred to as stochastic process or random signal
Voices signals, TV signals, thermal noise generated by a

radio receiver are all examples of random signals
signals.

A random process can be described as
For each experiment n, there exists a time-function xn(t) ,
called a sample function or realization of the random
process
At any time instant t1, t2, …, the value of the random
process is a random variable X(t1), ) X(t2),
) …,
x1(t)
Outcome of 1st
experiment
i t
x2(t) Outcome of 2nd

Sample experiment
space S
46
xn(t) X(t1) X(t2) Outcome of nth
experiment
t1 t2
2009/2010 Meixia Tao @ SJTU
Statistics of Random Processes
By sampling the random process at any time
time, we
get a random variable
F
From this
thi view
i point,
i t we can thi think
k off a random
d
process as an infinite collection of random
variables
i bl specified
ifi d att titime t:
t {X(t1),
) X(t2),) …, X(tn)}
Thus,, a random process
p can be completely
p y
defined statistically as a collection of random
variables indexed byy time with p properties
p defined
by a joint PDF

A random process X(t) is described by its M-th
order statistics if for all and all
the joint pdf of {X(t1),
) X(t2),
) …, X(tn)} is given
This joint pdf is written as
f ( x1 , x2 ,K, xn ; t1 , t2 ,K, tn )
In order to completely specify a random process process,

one must given f (x1 , x2 ,K, xn ; t1 , t2 ,K, tn ) for all
possible values of {xi} {ti},
} and for all n n. This is
obviously quite difficult in general
48
First Order Statistics on Random
P
Processes
The first order statistics is simply the PDF of a
random variable at one particular time
f(x;t) = first order density of X(t)
F(x;t) = P(X(t) ≤x), first order distribution of X(t)
E [ X (t0 )] = E [ X (t = t0 )] = ∫ xf X ( x; t0 ) = X (t0 )
∞
Mean −∞
Variance [ 2
]
E X (t0 ) − X (t0 ) = σ X2 (t0 )

Second-Order Statistics on Random
Processes
Second-order statistics means the jjoint PDF of X(t
( 1) and
X(t2) for all choices t1 and t2.
Auto-correlation function: Let t1 = t, and t2 = t+τ,
RX (t;τ ) = E[ X (t ) X (t + τ )] = ∫
∞ ∞
−∞ −∞ ∫ x1 x2 f ( x1 , x2 ; t , t + τ )dx1dx2
The physical meaning of RX(t; τ) is a measure of the

relationship
p of the function X(t) ( τ)) (correlation
( ) and X(t+ (
within a process)
In general, the autocorrelation function is a function of both
t and τ.

Example
Given a stochastic process , where
is a random variable uniformly distributed from to
At each
h t,
t X(t) can b
be viewed
i d as a ffunction
ti off
The mean is
The auto-correlation is

Stationary Processes
A stochastic process is said to be stationary if for any n

and the following holds:
f X ( x1 , x2 ,L xn ; t1 , t2 ,L tn ) = f X ( x1 , x2 ,L xn ; t1 + τ , t2 + τ ,L tn + τ ) (1)
Therefore,
Te first-order statistics is independent of t
∞
mean E { X (t )} = ∫ xf X ( x)dx = mX ((2))
−∞
The second-order statistics only depends on the gap

between t1 and t2
∞ ∞
Autocorrelation RX (t1 , t2 ) =
function
∫ ∫
−∞ −∞ 1 2
x x f X ( x1 , x2 , t2 − t1 )dx1dx2
= RX (t2 − t1 ) = RX (τ ) , where τ = t2 − t1 (3)
Wide-Sense
Wide Sense Stationary
Our engineers often care about the first- and second-
order statistics only
A random process is said to be WSS when conditions
(2) and (3) hold
A random p process is said to be strictly
y stationary
y when
condition (1) holds
Example:
, where
Only depends on the time

difference
Thus, X(t) is WSS
Averages and Ergodic
Ensemble averaging
g g
Δ
X (t ) = E [X (t )] = ∫ xp( x; t )dx
∞
−∞
Δ
RX (t1 , t 2 ) = E [X (t1 ) X (t 2 )] = ∫
∞ ∞
−∞ −∞ ∫ x1 x2 p ( x1 , x2 ; t1 , t 2 )dx1dx2
Time averaging
g g Δ 1 T
T →∞ 2T ∫−T
< X (t ) > = lim x(t )dt
Δ 1 T
T →∞ 2T ∫−T
< X (t ) X (t − τ ) > = lim
li x(t ) x(t − τ )dt
In general, ensemble averages and time averages are not equal

A r.p. X(t) is said to be Ergodic if all time averages and
Ensemble averages are equal

Random Processes in the Frequency
D
Domain:
i Power
P Spectral
S t l Density
D it
Let
e X(t)
( ) de
denote
o e a random
a do p process
ocess a
and
d let
e de
denote
oea
sample function of this process
Truncate the signal
g by
y defining
g
in order to get an energy signal.

Performing a Fourier transform on , we get
According to Parseval theorem
∞ ∞
∫ T ∫ T
2
x 2
(t , n )dt = X ( f , n ) df
−∞ −∞
: energy
gy spectral
p density
y

Then the power spectral density is the average energy
spectral density per time unit, i.e.
Letting , we define the power spectral density for the

sample function: 2
X T ( f , n)
S X ( f , n) = lim
T →∞ T
If we take the ensemble average, the power spectral
density (PSD) of the random process is
(4)
Watts/Hz
The general definition of power spectral density

PSD of Stationary Process
Wiener-Khinchin
Wiener Khinchin theorem
For a stationary random process X(t), the PSD is equal to

the Fourier Transform of the autocorrelation function, i.e.,
∞
RX (τ ) = ∫ S X ( f ) exp( j 2πfτ )df
S X ( f ) ↔ RX (τ ) −∞
∞
S X ( f ) = ∫ RX (τ ) exp(− j 2πfτ )dτ
−∞
In general, is a measure of the relative power in the

random
d signal
i l at eachh ffrequency component
∞
RX (0) = ∫ S X ( f )df = total power
−∞

Gaussian Process
The importance of Gaussian processes in communication
systems is due to that thermal noise in electronic devices
can be closely modeled by a Gaussian process
Definition:
A random process X(t) is a Gaussian process if for all n
and all (t1, t2, …, tn), the random variables {X(t1),
X(t2),
( ), …,, X(tn)}
( )} have a joint
j Gaussian densityy function

Properties of Gaussian Processes
If a Gaussian random process is wide-sense
wide sense
stationary, then it is also stationary
Any sample point from a Gaussian random process is
a Gaussian random variable
If the
h input
i to a linear
li system iis a G
Gaussian
i random
d
process, then the output is also a Gaussian random
process.
process

Random Process Transmission Through
Li
Linear Systems
S t
Consider a linear system
X(t) Impulse Y(t)

response
p
h(t)
∞
Y (t ) = X (t ) * h(t ) = ∫ h(τ ) X (t − τ )dτ
−∞
The mean of the output random process Y(t)

Y (t ) = E [Y (t )] = ∫ h(τ ) E [X (t − τ )]dτ
∞
−∞
∞
= ∫ h(τ ) X (t − τ )dτ
−∞
If X(t) is WSS
∞
= X ∫ h(τ )dτ = X ⋅ H (0)
−∞
where H(0) is the zero-frequency response of the system

The autocorrelation of Y(t)
RY (t , u ) = E [Y (t )Y (u )]
= E ∫ h(τ 1 ) X (t − τ 1 )dτ 1 ∫ h(τ 2 ) X (u − τ 2 )dτ 2 ⎤

⎡ ∞ ∞
⎢⎣ −∞ −∞ ⎥⎦
= ∫ h(τ 1 )dτ 1 ∫ h(τ 2 ) E [X (t − τ 1 ) X (u − τ 2 )]dτ 2
∞
−∞
If X(t) is WSS
∞ ∞
RY (τ ) = ∫ ∫ h(τ 1 )h(τ 2 ) RX (τ − τ 1 + τ 2 )dτ 1dτ 2
−∞ −∞
If input is a WSS random process, the output

is also a WSS random process
Relation Among the Input
Input-Output
Output PSD
Autocorrelation of Y(t)
∞ ∞
RY (τ ) = ∫ ∫ h(τ 1 )h(τ 2 ) RX (τ − τ 1 + τ 2 )dτ 1dτ 2
−∞ −∞
∞ ∞
= ∫ h(τ 2 )dτ 2 ∫ h(τ 1 ) RX (τ + τ 2 − τ 1 )dτ 1
−∞ −∞
= ∫ h(τ 2 )[h(τ + τ 2 ) * RX (τ + τ 2 )]dτ 2

∞
−∞
= h(−τ ) * h(τ ) * RX (τ )
PSD of Y(t): SY ( f ) = H ( f ) 2 S X ( f )
X(t) Y(t)
h(t)
SX ( f ) SY ( f )
2
SY ( f ) = H ( f ) S X ( f ) Key
ey Results
esu ts

Noise
Noise is a critical component in the analysis of the performance
of communication receivers
Often assumed to be Gaussian and stationary
The mean is taken to be zero while the autocorrelation is usually
specified by the power spectral density
The noise is a white noise, when all frequency components
appear with equal power (white is used in white light for a similar
reason)
N
Sn ( f ) = N 0 2 Rn (τ ) = 0 δ (τ )
2 White noise is completely
Sn ( f ) Rn (τ )
uncorrelated!
N0 2 N0 2
0 f 0 τ
Bandlimited Noise
White noise Bandlimited white
Filter Bandwidth noise n(t)
Ｂ Hz
In most applications
N 0 = KT = 4.14 × 10−21
= −174 dBm/Hz
At what sampling rate to sample

the noise can we get uncorrelated
realizations?
Narrow-Band
Narrow Band Random Process
The bandwidth of the signal is limited to a narrow
band around a central frequency fc >> 0
S (ω )
− fc f c
Canonical form of a narrow band process

X (t ) = X I (t ) cos ( 2π f 0t ) − X Q (t ) sin ( 2π f 0t )
In-phase component Quadrature component

Narrow band Noise
Let n(t) be a zero-mean,
zero mean, stationary noise
n(t ) = nc (t ) cos ω0t − ns (t ) sin ω0t
Find
Fi d the t ti ti off nc(t)
th statistics ( ) and
d ns(t)
()
Result 1：
E {n(t )} = E {nc (t )} = E {ns (t )} = 0
Proof:
E [ n(t ) ] = E [ nc (t ) ] cos ω0t − E [ ns (t ) ] sin ω0t
Since n(t) is stationary

stationary，zero-mean,
zero-mean for any tt, we have
E [ n(t ) ] = 0
Thus： E {nc (t )} = E {ns (t )} = 0
Result 2：
⎧ S n ( f − f 0 ) + S n ( f + f 0 ), f ≤ B / 2
S nc ( f ) = S ns ( f ) = ⎨
⎩0 otherwise
h i
Proof HL ( f )
nc (t )
× Z (t)
1
1
-B/2 0 B/2 f
2cos ω0t
n(t )
−2sin ω0t
HL ( f )
ns (t )
× Z2 (t ) 1
-B/2 0 B/2 f

Result 3：for the same tt, nc(t) and ns(t) are
uncorrelated or independent
Rnc ns (0) = 0
Result 4：
E {n 2 (t )} = E {nc 2 (t )} = E {ns 2 (t )} = σ 2
Result 5：If n(t) is a Gaussian process, so are nc(t)

and ns(t)

Envelop and Phase
Angular representation of n(t)
n(t ) = R(t ) cos [ω0t + φ (t )]
R(t ) = nc 2 (t ) + ns 2 (t ) envelop
where ns (t )
φ (t ) = tan −1 [0 ≤ φ (t ) ≤ 2π ] phase
h
nc (t )
1 R(t )
≈
n(t ) B
0 t
1
≈
f0

Let n(t) be a zero-mean, stationary Gaussian process,
find the statistics of the envelop and phase
Result：
Envelop follows Rayleigh distribution while phase
follows uniform distribution
2π R ⎧ R2 ⎫
f ( R) = ∫ f ( R, φ )dφ = 2 exp ⎨− 2 ⎬ R≥0
0 σ ⎩ 2σ ⎭
P f？
Proof？ ∞ 1
f (φ ) = ∫ f ( R, φ ) dR = 0 ≤ φ ≤ 2π
0 2π
For
o thee sa
samee t,, the
eeenvelop
e op variable
a ab e R a
and
dpphase
ase
variable φ are independent (but not the two processes)

Homework 1
Textbook Chapter 2: 2.7(3)(4), 2.13(6)(13)(16)

Textbook Chapter 5: 5.5, 5.15, 5.22, 5.28, 5.44,
5.49
Due: in class on Sep. 28 (next Wednesday)

Random Variables

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Random Variables

Uploaded by

Copyright:

Available Formats

Principles of Communications

Weiyao Lin, PhD

Chapter 2: Signal, Random Process, and Spectra

Typical noise source

2009/2010 Meixia Tao @ SJTU 2

2.2.Review of probability and random variables

2.3.Random Processes: basic concepts

2.4.Guassian and White Processes

2009/2010 Meixia Tao @ SJTU 3

2009/2010 Meixia Tao @ SJTU 4

2009/2010 Meixia Tao @ SJTU 5

2009/2010 Meixia Tao @ SJTU 7

 A signal is an energy signal if and only if Ex is finite

2009/2010 Meixia Tao @ SJTU 8

2009/2010 Meixia Tao @ SJTU 9

 When A and B are NOT mutually exclusive

2009/2010 Meixia Tao @ SJTU 10

 Then for anyy event B we have

2009/2010 Meixia Tao @ SJTU 12

 This formula will be used to derive the structure of the

 What is the probability that the output of this channel is 1?

2009/2010 Meixia Tao @ SJTU 14

2009/2010 Meixia Tao @ SJTU 15

 Key properties of CDF

 Key properties of PDF

Xmin 0 x1 x2 Xmax x Xmin 0 x1 x2 x x+dx

2009/2010 Meixia Tao @ SJTU 18

 X and Y are said to be independent iff

2009/2010 Meixia Tao @ SJTU 19

where “E[]” denotes the expectation operation

2009/2010 Meixia Tao @ SJTU 20

 This is the first moment of the random variable X

2009/2010 Meixia Tao @ SJTU 21

 The expected value of second central moment

 σX, square-root of the variance, is called the

 Correlation of X and Y is the mean of the product X and Y

 If X and Y are independent,

2009/2010 Meixia Tao @ SJTU 25

2009/2010 Meixia Tao @ SJTU 26

 If no error correction is used, the error probability is

2009/2010 Meixia Tao @ SJTU 28

2009/2010 Meixia Tao @ SJTU 29

 A Gaussian r.v. is completely determined by its mean and

 Extremely important in error probability analysis!!!

 Craig’s alternative form of Q-function (IEEE MILCOM’91)

 If we have a Gaussian variable X ~ N ( μ , σ 2 ) , then

2009/2010 Meixia Tao @ SJTU 33

2009/2010 Meixia Tao @ SJTU 34

X1 and X2 are also independent

If X1 and X2 are Gaussian and uncorrelated,

 Rayleigh distributions are frequently used to model fading

2009/2010 Meixia Tao @ SJTU 36

i.e. the average converges to the expected value

2009/2010 Meixia Tao @ SJTU 37

Key Conclusion: the sum of random variables is “Gaussian”

 Thermal noise results from the random movement of many

2009/2010 Meixia Tao @ SJTU 38

2009/2010 Meixia Tao @ SJTU 39

 Also referred to as stochastic process or random signal

 Voices signals, TV signals, thermal noise generated by a

2009/2010 Meixia Tao @ SJTU 45

x2(t) Outcome of 2nd

2009/2010 Meixia Tao @ SJTU 47

 In order to completely specify a random process process,

A signal is an energy signal if and only if Ex is finite

When A and B are NOT mutually exclusive

Then for anyy event B we have

This formula will be used to derive the structure of the

What is the probability that the output of this channel is 1?

Key properties of CDF

Key properties of PDF

X and Y are said to be independent iff

This is the first moment of the random variable X

The expected value of second central moment

σX, square-root of the variance, is called the

Correlation of X and Y is the mean of the product X and Y

If X and Y are independent,

If no error correction is used, the error probability is

A Gaussian r.v. is completely determined by its mean and

Extremely important in error probability analysis!!!

Craig’s alternative form of Q-function (IEEE MILCOM’91)

If we have a Gaussian variable X ~ N ( μ , σ 2 ) , then

Rayleigh distributions are frequently used to model fading

Thermal noise results from the random movement of many

Also referred to as stochastic process or random signal

Voices signals, TV signals, thermal noise generated by a

In order to completely specify a random process process,

The physical meaning of RX(t; τ) is a measure of the

A stochastic process is said to be stationary if for any n

The second-order statistics only depends on the gap

In general, ensemble averages and time averages are not equal

Letting , we define the power spectral density for the

In general, is a measure of the relative power in the

The mean of the output random process Y(t)

Canonical form of a narrow band process

Since n(t) is stationary

Result 5：If n(t) is a Gaussian process, so are nc(t)

Textbook Chapter 2: 2.7(3)(4), 2.13(6)(13)(16)