Professional Documents
Culture Documents
Lecture02 Random Variables
Lecture02 Random Variables
Random Variables
Muhammad Shahzad
School of Electrical Engineering & Computer Science
National University of Sciences & Technology (NUST)
Pakistan
What will we cover in this lecture?
In this lecture, we will cover:
Definition of a Random Variable
2
Definition of a Random Variable
Random Variable [m-w.org]
: a variable that is itself a function of the result of a statistical
experiment in which each outcome has a definite
probability of occurrence
3
Definition of a Random Variable
A random variable is a mapping from an outcome s of a
random experiment to a real number
4
Definition of a Random Variable
Random
Sample Space X(s) Random
Experiment
Variable
head
0 1 R
tail
Sx
5
Definition of a Random Variable
X(s)
1 2 3 4 5 6 R
Random
Experiment
Sample Space, S Sx
Random Variable
6
Definition of a Random Variable
More than one outcomes can be mapped to the same real
number
X(s)
0 1
Random Sx
Experiment
Random Variable
Sample Space
7
Types of Random Variables
Discrete random variables: have a countable (finite or infinite)
image
Sx = {0, 1}
Sx = {, -3, -2, -1, 0, 1, 2, 3, }
Continuous random variables: have an uncountable image
Sx = (0, 1]
Sx = R
Mixed random variables: have an image which contains
continuous and discrete parts
Sx = {0} U (0, 1]
8
Discrete Random Variables
9
Probability Mass Function
The Probability Mass Function (pmf) or the discrete probability
density function provides the probability of a particular point in
the sample space of a discrete random variable (rv)
For a countable SX={a0, a1, , an}, the pmf is the set of
probabilities
pX(ak)
0 1 2 3 4 5 X
10
Properties of a PMF
P1:
P2:
pX(ak)
0 1 2 3 4 5 X
11
Cumulative Distribution Function (CDF) of Discrete
Random Variable
The Cumulative Distribution Function (CDF) for a discrete rv is
defined as:
pX(x) FX(x)
pmf
CDF
1 2 3 4 X 1 2 3 4 X
12
Cumulative Distribution Function (CDF) of Discrete
Random Variable
CDF can be used to find the probability of a range of values in a
rvs image:
pX(x) FX(x)
pmf
CDF
1 2 3 4 X 1 2 3 4 X
13
Properties of a CDF
F1:
F2:
pX(x) FX(x)
pmf
CDF
1 2 3 4 X 1 2 3 4 X
14
Properties of a CDF
F3:
F4:
pX(x) FX(x)
pmf
CDF
1 2 3 4 X 1 2 3 4 X
15
Expected Value of a Discrete Random
Variable
The expected value, expectation or mean of a discrete rv is the
average value of the random variable
16
Expected Value of a Discrete Random
Variable
What is the average value of the following random variable
whose image is SX={1, 6, 7, 9, 13}?
If your answer is 7.2 then you assumed that all of the values in the
rvs image have equal weights
17
Expected Value of a Discrete Random
Variable
Expectation assigns appropriate weights to each value in the rvs
image
Expectation is computed by using the probability of each point in
the rvs image as the weight of that point
Hence, the expectation of the random variable given below is:
pX(x)
0.4
0.3
0.2
0.1
1 6 7 9 13
18
Expected Value of a Discrete Random
Variable
Mathematically, the expected value of a discrete random
variable is:
19
Variance of a Random Variable
Variance of a rv is a measure of the amount of variation of a rv
around its mean
pX(x) qX(x)
E{X}=3.87 E{X}=5.2
0.5 0.4
0.4
0.2
0.1
0.033
1 6 7 9 13 X 1 6 7 9 13 X
20
Variance of a Random Variable
Intuitively, which of the following discrete rvs has a higher
variance?
qX(x) has a higher variance because it varies more around its mean
than pX(x)
pX(x) qX(x)
E{X}=3.87 E{X}=5.2
0.5 0.4
0.4
0.2
0.1
0.033
1 6 7 9 13 X 1 6 7 9 13 X
21
Variance of a Random Variable
Mathematically, the variance of a discrete rv is defined as:
22
Variance of a Random Variable
pX(x) qX(x)
E{X}=3.87 E{X}=5.2
0.5 0.4
0.4 var{X}=15.36
var{X}=9.91 0.2
0.1
0.033
1 6 7 9 13 X 1 6 7 9 13 X
23
Standard Deviation of a Random Variable
In many scenarios, we use the square root of the variance
called its standard deviation
sX = var {X }
24
Standard Deviation of a Random Variable
pX(x) qX(x)
E{X}=3.87 E{X}=5.2
0.5 0.4
0.4 X=3.92
X=3.14 0.2
0.1
0.033
1 6 7 9 13 X 1 6 7 9 13 X
25
Discrete Uniform Random Variable
A discrete uniform rv, D, has a finite image and all the elements
of the image have equal probabilities
Pr{D=k}
1/n
x1 x2 x3 xn
k
26
Bernoulli Random Variable
A Bernoulli Random Variable is defined on a single event A
This rv is based on an experiment called a Bernoulli trial
The experiment is performed and the event A either happens or does not
happen
Thus the sample space of a Bernoulli rv is binary
Bernoulli Random
Variable
Sample Space X(s)
Bernoulli Trial: Toss a
A=head
coin
Ac = Not head 0 1 R
= tail
Sx
27
Bernoulli Random Variable
A Bernoulli Random Variable is defined on a single event A
This rv is based on an experiment called the Bernoulli trial
The experiment is performed and the event A either happens or does not
happen
Thus the sample space of a Bernoulli rv is binary
Bernoulli Random
Variable
Pakistan cricket team
X(s)
plays Australia
A=Pak wins
R
Ac=Pak losses
0 1
Sample Space
Image Courtesies of http://www.tribuneindia.com and
images.google.com
28
Bernoulli Random Variable
A Bernoulli Random Variable is defined on a single event A
This rv is based on a the experiment called a Bernoulli trial
The experiment is performed and the event A either happens or does not
happen
Thus the sample space of a Bernoulli rv is binary
Of course the Pr{A} = 0 for this
experiment Bernoulli Random
Variable
Pakistan cricket team
X(s)
plays Australia
A=Pak wins
R
Ac=Pak losses
0 1
Sample Space
Image Courtesies of http://www.tribuneindia.com and
images.google.com 29
Bernoulli Random Variable
Typical examples of Bernoulli rvs in communication:
Transmit a bit over a wireless channel
Outcomes:
0 > bit is received error-free
1 > bit received is not received error-free => bit is received with errors
Transmit a packet over the Internet
Outcomes:
0 > packet is received
1 > packet is not received => packet is lost en-route due to congestion
30
Bernoulli Random Variable
Sample space of a Bernoulli rv, I, is binary
Both outcomes are mapped to real numbers,
Traditionally: I(A) = 1 and I(Ac) = 0 are used to represent a Bernoulli rvs
outcomes
The pmf of I is:
Pr{I = 1} = p
Pr{I = 0} = 1 p
Pr{I=k}
p
1-p
I
0 1
31
Bernoulli Random Variable
Example:
Consider the experiment of a fair coin toss. What is the expected
value and the variance of this Bernoulli rv?
Pr{I=k}
0.5
I
0 1
32
Discrete Random Variables
Example:
Consider the experiment of a fair coin toss. What is the expected
value and the variance of this Bernoulli rv?
Since the coin toss is fair: Pr{I = 1} = 0.5 and Pr{I = 0} = 0.5
E{I} = (1)x(0.5) + (0)x(0.5) = 0.5
var{I} = (10.5)2x(0.5) + (00.5)2x(0.5) = 0.25
Pr{I=k} I=0.5
I=0.5
0.5
I
0 1
33
Bernoulli Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error
0.1. What is the expected value and the variance of this Bernoulli
rv?
Pr{I=k}
0.9
0.1
I
0 1
34
Bernoulli Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error 0.1. What is the
expected value and the variance of this Bernoulli rv?
The event of interest here is a bit-error:
=> Pr{I = 1} = 0.9 and Pr{I = 0} = 0.1
E{I} = (1)x(0.9) + (0)x(0.1) = 0.9
var{I} = (10.9)2x(0.9) + (00.9)2x(0.1) = 0.09
Note that the variance of this pmf is smaller than the variance of the coin toss
pmf
Pr{I=k} I=0.3
0.9
I=0.9
0.1
I
0 1
35
Binomial Random Variable
Consider a collection of n independent Bernoulli trials
36
Binomial Random Variable
If Ij(A)=1 and Ij(Ac)=0, j=1,2,,n, are used to represent the
outcomes of the Bernoulli trials then the Binomial Random
Variable, X, is
37
Binomial Random Variable
If Ij(A)=1 and Ij(Ac)=0, j=1,2,,n, are used to represent the
outcomes of the Bernoulli trials then the Binomial Random
Variable, X, is
38
Binomial Random Variable
Pr{Ij(A)=1} = p and Pr{Ij(A)=0} = 1-p
39
Binomial Random Variable
The pmf of a Binomial rv X is:
40
Binomial Random Variable
Pr{X=k}
41
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Find the probability that a packet of n bits is received with one
or more errors?
42
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Find the probability that a packet of n bits is received with one
or more errors?
43
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Find the probability that a packet of n bits is received with one
or more errors?
44
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Find the probability that a packet of n bits is received with one
or more errors?
45
Connection Between Bernoulli and Binomial
RVs
S SI SX
A
Pr{I=1}=p
Pr{A}
Pr{I=0}=1-p
46
Normal Approximation of a Binomial RV
When n becomes large, computations using the binomial
formula become unmangeable
47
Geometric Random Variable
A Geometric Random Variable, M, is the number of Bernoulli
trials until the first occurrence of an event A
48
Geometric Random Variable
geometric pmf
OR
49
Geometric Random Variable
50
Connection between Geometric and Binomial
RVs
Can we find the probability of a Geometric random variable
using the Binomial random variable?
Geometric
Binomial
51
Connection between Geometric and Binomial
RVs
Can we find the probability of a Geometric random variable using
the Binomial random variable?
52
Connection between Geometric and Binomial
RVs
In k Bernoulli trials, there are k ways in which you can have 1
success and k-1 failures
53
Connection between Geometric and Binomial
RVs
For k=4, a Binomial random variable X jointly considers the
outcomes 0001, 0010, 0100, 1000
Pr{X = 1} = Pr{(0001) U (0010) U (0100) U (1000)}
Pr{X = 1} = Pr{0001} + Pr{0010} + Pr{0100} + Pr{1000}
Since the underlying Bernoulli trials are independent:
Pr{X = 1} = (k)Pr{one success in k trials}
54
Memoryless Property
It can be shown that the Geometric rv satisfies the memoryless
property
55
Memoryless Property
The memoryless property is satisfied when:
56
Memoryless Property
To prove that the Geometric rv satisfies the memoryless
property, we need to show that
57
Memoryless Property
Continued from last page
58
Memoryless Property
It can also be shown that the Geometric rv is the only discrete
random variable that satisfies the memoryless property
59
Poisson Random Variable
A Poisson Random Variable, N, is the number of occurrences or
arrivals of an event A in a time interval of fixed length
E.g.: The number of packets that arrive at a wireless access point per
second
Poisson arrivals
60
Poisson Random Variable
A good way of understanding Poisson rv is through the Binomial
rv
Let us divide the fixed Poisson time interval into n very small
sub-intervals of length t
t is so small that only one arrival is possible within each sub-interval
1 2 3 4 n
t
t Poisson arrivals
61
Poisson Random Variable
Now the experiment can be treated as Binomial rv where a
success corresponds to the presence of an arrival in a sub-
interval
1 2 3 4 n
t
t Poisson arrivals
62
Poisson Random Variable
Now the experiment can be treated as Binomial rv where a
success corresponds to the presence of an arrival in a sub-
interval
1 2 3 4 n
t
t Poisson arrivals
63
Poisson Random Variable
Now the experiment can be treated as Binomial rv with
parameter /n
1 2 3 4 n
t
t Poisson arrivals
64
Poisson Random Variable
Lets take the limit when n tends to infinity
1 2 3 4 n
t
t Poisson arrivals
65
Poisson Random Variable
Thus for large n, the Binomial pmf approaches the Poisson pmf
1 2 3 4 n
t
t Poisson arrivals
66
Poisson Random Variable
In general, the pmf of a Poisson random variable is
67
Poisson Random Variable
Pr{N=k}
k
Image courtesy of Wikipedia article on Poisson Distribution
68
End of Discrete Random Variables
At this point, we conclude our discussion on examples of
discrete random variables
69
Continuous Random Variables
70
Continuous Random Variables
Recall that a Continuous random variable has an uncountable
image
Sx = (0, 1]
Sx = R
71
Probability Density Function (pdf) of a
Continuous Random Variable
The Probability Density Function (pdf) of a continuous random
variable, fX(x), is a continuous function of the x Sx
Properties of pdf:
f1:
f2:
fX(x)
SX=(0, n]
0 n x
72
Probability Density Function (pdf) of a
Continuous Random Variable
f3:
fX(x)
0 n x
a b
73
Cumulative Distribution Function (CDF) of a
Continuous Random Variable
CDF of a continuous rv is computed by integrating the pdf
FX(x)
x
74
Properties of the CDF of a Continuous
Random Variable
F1:
F2:
F3:
FX(x)
75
Properties of the CDF of a Continuous
Random Variable
F4:
FX(x)
x
76
Properties of the CDF of a Continuous
Random Variable
F4:
FX(x)
a-/2 a a+/2 x
77
Properties of the CDF of a Continuous
Random Variable
F5:
FX(x)
FX(b)
Pr{a<x<b}
FX(a)
x
a b
78
Expected Value of a Continuous Random
Variable
The expected value of a continuous random variable is:
x x
79
Variance of a Continuous Random Variable
The variance of a continuous random variable is:
X > Y
fX(x) X fY(y) Y
x y
80
Exponential Random Variable
An exponential random variable measures the inter-arrival time
between two occurrences of an event
E.g., the inter-arrival time of packets arriving in a routers queue
Poisson arrivals
81
Exponential Random Variable
1 2 3 4 60 t
1 2 3 4 60 t
inter-arrival time =6
1 2 3 4 60 t
82
Exponential Random Variable
Exponential rv is the inter-arrival time between two Poisson
arrivals
1 2 3 4 5 6
Poisson arrivals
83
Exponential Random Variable
Exponential rv is the inter-arrival time between two Poisson
arrivals
1 2 3 4 5 6
Poisson arrivals
84
Exponential Random Variable
An exponential rv is the inter-arrival time between two Poisson
arrivals
Lets look at a sub-interval (0,t] of the Poisson window
On average, how many arrivals will take place in the (0,t]
subinterval?
t t t t t t
1 2 3 4 5 6
T
85
Exponential Random Variable
On average, how many arrivals will take place in the (0,t]
subinterval?
Arrivals in the (0,t] sub-interval = t/T
t t t t t t
1 2 3 4 5 6
T
86
Exponential Random Variable
t arrivals will take place in a (0,t] subinterval per window T
t t t t t t
1 2 3 4 5 6
87
Exponential Random Variable
Considering a window of T=1 unit time, the probability density
and cumulative distribution functions of an exponential rv are:
1 2 3 4 5 6
Poisson arrivals 88
Exponential Random Variable
fE (x)
x
Image courtesy of Wikipedia article on Exponential Distribution
89
Exponential Random Variable
Exponential rv is a limiting case of a geometric rv
90
Exponential Random Variable
Memoryless property
91
Erlang Distribution
Consider a system that has r different stages each of which is
exponentially distributed
When r phases of this system have identical exponential
distributions then the resulting probability distribution is called
an r-state Erlang
The density and distribution functions of an Erlang rv, R, are:
93
Gamma Distribution
95
Gamma Distribution
The gamma function has the following well-known and useful
properties:
96
Uniform Distribution
A uniform rv has a uniform distribution over an interval [a, b]
97
Uniform Distribution
fU (x)
1/(b-a)
a b x
98
Gaussian or Normal Distribution
Gaussian or Normal Distribution is by far the most famous
continuous distribution
99
Gaussian Distribution
The Gaussian rv is defined as:
100
Gaussian Distribution
fN (x)
102
Gaussian Distribution
Gaussian distribution does not deviate from its mean by more
than 3 standard deviations 99.7% of the times
103
Quiz#1
While rolling a die twice and counting their sum of outcomes,
What is the probability of getting 7 while one of the outcome is
atleast 2?
104