Probabilistic Methods for Electrical Engineers

Random Variables
Dr. Ali Hussein Muqaibel
Ver. 5.3

𝑆
{𝐻 𝐻 𝑇 }

{𝐻 𝑇 𝑇 }

{𝑇 𝑇 𝑇 }

Outcomes are random 0 1 2

(not numeric) Measurements will also be random
(numeric)

Dr. Ali Muqaibel 1

Outline
• Definition of Random Variable (RV)
• Conditions on Random Variables
• Types of RV
• Cumulative Probability Distribution Function (CDF)
• Probability Density Function (PDF)
• Gaussian Random Variable
• Other random variables:
• Discrete: Binomial, Poisson.
• Continuous: Uniform, Exponential, Rayleigh ….
• Conditional density & distribution

Dr. Ali Muqaibel 2

Definition of RV
𝑆
{𝐻 𝐻 𝑇 }

{𝐻 𝑇 𝑇 }

• We are usually interested not in the outcome {𝑇 𝑇 𝑇 }

itself, but rather in some measurement or

numerical attribute of the outcome. Outcomes are random 0 1 2
(not numeric) Measurements will also be random

• Example: (numeric)

• Example: Consider the experiment of tossing 𝑋(𝑠)

a coin 3 times and observing the number of 𝑆

“Heads” (which is a numeric value). Real line

𝑠 𝑋
𝑋
• A real random variable is a function that
𝑆
assigns a real number to each outcome 𝑆 is the range which is the
𝑆 is the domain subset of the real line
in the sample space.

Dr. Ali Muqaibel 3

Additional Random Variable Example
Example : The experiment of rolling a die and tossing a coin
Let be a random variable that maps the “head” outcome to the
positive number which correspond to the dots on the die, and
“tail” outcome map to the negative number that are equal in
magnitude to twice the number which appear on the die.
Tail

Head

Example: is the outcome of the wheel of chance.

Maps to the numbers from 0 to 144

• Not necessary that the sample space maps uniquely

or
Dr. Ali Muqaibel 4
 Conditions for a Function to be a Random Variable

RVs are functions that satisfies the followings:

1. RV can not be multivalued.
But the random variable function can map more than one point in into same
point on the real line.
2. of the probability of all elementary event that satisfies the
condition. (see example in next slide)
3. ,
This condition does not prevent from being or for some values of . It
only requires that the probability of the set of those be zero.

Dr. Ali Muqaibel 5

Example: Tossing a coin 3 times and observing the number of heads

Dr. Ali Muqaibel 6

Types of Random Variables
Discrete
• Flipping a coin 3 times and counting the number of heads.
• Selecting a number from the positive integers.
• Number of cars arriving at gas station .
Continuous
• Selecting a number between { 0 and 6 }

• Mixed RV
• Wheel of chance
The type is based on the event not the Sample Space. Example:
Wheel of chance
In this example is continuous and the event is discrete

Dr. Ali Muqaibel 7

Distribution Function
Example: Flipping a coin 3 times and counting the number of heads
We can describe the distribution of
the R.V using the probability

We will define two more distributions of the random variable which will help us finally to
calculate probability.

Dr. Ali Muqaibel 8

Random Variables:

Cumulative Distribution Function (CDF)

Probability Density Function (pdf)

Dr. Ali Hussein Muqaibel

Ver. 5.0
Probabilistic Methods for Electrical Engineers

Dr. Ali Muqaibel 9

Distribution Function
We define the cumulative probability distribution function (CDF)
𝐹 𝑥 =𝑃 𝑋≤𝑥 , where

In our flipping the coin 3 times and counting the number of heads

Dr. Ali Muqaibel 10

Example: Tossing 3 Coins
Example: Let with

Sketch the cumulative distribution function (CDF)

•

Practice Quiz 2 142

Dr. Ali Muqaibel 11
Distribution Function Properties 𝐹 𝑥
not accurate
1
7/8
1/8

3/8
1/2

3/8

1/8
1/8
𝑋
0 1 2 3

if

A right-continuous function

• From property 4: 𝐹 (𝑥) is a non decreasing function

• Properties 1,2,4, & 6 can be used as a test for the validity of distribution functions.

Dr. Ali Muqaibel 12

Consider the experiment of tossing the coin 3 times and observing the number of heads
The probabilities and the distribution function are shown below
𝐹 𝑥
1
7/8
1/8

3/8
1/2

3/8

1/8
The stair type distribution function can be written as
1/8
𝑋
0 1 2 3

discrete => stair steps, in general

𝑵

𝑭𝑿 𝒙 = 𝑷 𝑿 = 𝒙𝒊 𝒖(𝒙 − 𝒙𝒊 ) where 𝑁 can be infinite.

We may simply 𝑃 𝑋 = 𝑥 = 𝑃(𝑥 )
𝒊 𝟏

Dr. Ali Muqaibel 13

Density Function
Probability density function (pdf) is the derivative of the
distribution function 𝑋 as the 𝑋
• In our discrete R.V since

Dr. Ali Muqaibel 14

Additional Examples: CDF, pdf

Dr. Ali Muqaibel 15

Properties of Density Functions
1. , for all
 (Density is non-negative derivative of non-decreasing function)

 From (3),


• Properties (1) and (2) are used to prove if a certain function can be a
valid density function.
Dr. Ali Muqaibel 16
Additional Examples: Example 1
A random current is described by the sample space 𝑆 = {−4 ≤ 𝑖 ≤ 12}. A random variable is defined to by
−2 𝑖 ≤ −2 𝑥

𝑋 𝑖 = 𝑖 −2 < 𝑖 ≤ 1 6
1 1<𝑖≤4 4
6 4<𝑖
2
a. Show the mapping of the random variable. −4 −2 1

1 2 𝑖
b. What type of random variable is this? 4 6 12
−2

a. See figure
b. Mixed random variable

Dr. Ali Muqaibel 17

Example 2: PDF & CDF 𝑓 𝑥

The sample space of an experiment 1/4

is The random
variable is defined to be . 0 2 5 12
𝑋

Sketch the density and distribution

functions of the random variable . 𝐹 𝑥

Here there is one value of for each 1

element in
3/4

1/2

1/4
𝑋
0 2 5 12

Dr. Ali Muqaibel 18

 Example 3: Properties of pdf
Determine the real constant , for arbitrary real constants and , such that

is a valid density function (called Laplace density)

• The required tests are :
1. 𝑓 𝑥 ≥ 0, for all 𝑥
 (Density is non-negative derivative of non-decreasing function)

2. ∫ 𝑓 𝑥 𝑑𝑥 = 1

• From a sketch, is necessary for non-infinite area under

• For this assumed true
• We did change of variable
• Thus, we require

Dr. Ali Muqaibel 19

In class practice 2: review of random variables
The probability density function of a continuous
𝑓𝑋(𝑥)
random variable is shown in the figure below
a) Determine the value of .
b) Find an analytical expression for the cumulative
probability distribution function 𝑋 -1 +1
𝑥

c) Sketch 𝑋 vs. .
d) Find the value of 𝑋

e) Determine a numerical value for so that

Dr. Ali Muqaibel 20

Dr. Ali Hussein Muqaibel 1
mean=0,variance=0.2

Ver. 5.4
0.9 mean=0,variance=1
mean=0,variance=5
0.8 mean=-2,variance=0.5

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
-6 -4 -2 0 2 4 6

Probabilistic Methods for Engineers

Important Random Variables:

The Gaussian (Normal) Random Variable

• Importance
• Definition (pdf)
• CDF
• Normalized CDF
• Q- Approximation
Dr. Ali Muqaibel 21
Why Gaussian Random Variable?
• The Gaussian density is the most important
of all densities.

• It accurately describes many practical and

significant real-world quantities such as
noise, voltage, current, electrons, and
quantities which are results from many
small independent random effects.

• Under very general conditions the limiting

distribution of the average of any number
of independent, identically distributed
random variables is Gaussian (normal).

Dr. Ali Muqaibel 22

Defining the Gaussian (Normal) Random Variable
A random variable is called Gaussian if its density function has the form

where and are real constant representing the variance

and the mean. The “spread” about the point is related to .
1
𝑓 𝑥 = 𝑒
2𝜋𝜎

What is the value of the pdf at 1

𝑓 𝑥 = 𝑒
𝑥 = 𝑎 ,𝑥 = 𝑎 − 𝜎 ,𝑥 = 𝑎 + 𝜎 ? 2𝜋𝜎
1
𝑓 𝑥 = 𝑒
2𝜋𝜎
The notation 𝑋~𝑁 𝑎 , 𝜎
What is standard normal or unit normal distribution?
𝑁 0,1
23
Bolts & Nuts of the Gaussian Expression The red curve is the
standard normal
distribution 𝑁 0,1

A random variable is called Gaussian if its

density function has the form





𝑎

24
CDF for Gaussian R.V.
The distribution function is found from

The integral has no known closed-form solution

and must be evaluated by numerical or
approximation method (Tables, Matlab)

What is the value of the CDF at

Dr. Ali Muqaibel 25

However to evaluate numerically for a given

We need and 𝑋

Example: Let 𝑥
2 and 𝑥 , then

We then can construct the Table for various values of .

 Evaluate
Numerically
Uncountable infinite tables to

 Evaluate be constructed

Numerically
Dr. Ali Muqaibel 26
Problem!
• Finally we will get a Table for various values of .
• The table will only work for Gaussian distribution with 𝑥2 and 𝑥 .
• Since the combinations of 𝑥 and 𝑥2 are infinite (uncountable infinite)
• Uncountable infinite tables to be constructed
• Unpractical method

Dr. Ali Muqaibel 27

Normalized CDF
The general distribution function

can be found in terms of the normalized distribution

, ,

We make the variable change in

Dr. Ali Muqaibel 28

Tabulated
Normalized CDF

Dr. Ali Muqaibel 29

Example: Evaluation of Gaussian CDF
A Gaussian r.v. with . Evaluate the Prb.

Using the table

Dr. Ali Muqaibel 30

 Example 2:
Height of clouds above the ground is Gaussian r.v.
with .
Find .

Dr. Ali Muqaibel 31

 Q- Approximation
,

• There is no closed form solution for but it can be approximated

,
• of

Dr. Ali Muqaibel 32

Example: Q Approximation
•G , ,
• Find the

in the table
. .
• %
.
• MATLAB:

Dr. Ali Muqaibel 33

Important Distribution and Density
Examples (Random Variables)
Discrete: Binomial, Poisson
Continuous: Gaussian, Uniform,
Exponential, Rayleigh ….

𝑁 = 6, 𝑝 = 0.25
Dr. Ali Hussein Muqaibel
Ver. 5.0
Probabilistic Methods for Electrical Engineers
Dr. Ali Muqaibel 34
Binomial Distribution
Let , N = 1, 2,..., then the function

is called the binomial density function. 𝑁 = 6, 𝑝 = 0.25

is the binomial coefficient

The Binomial Cumulative distribution (CDF)

Dr. Ali Muqaibel 35

When to Apply Binomial Distribution?
The density can be applied to the
• Bernoulli trial experiment. (only two outcomes)
• Games of chance (win or lose).
• Detection problems in radar and sonar.
• It applies to many experiments that have only two possible outcomes
( , , {Target, No Target}) on any given trial ( ).
• It applies when you have trials of the experiment and you ask
what is the probability of -successes out of these trials.

Dr. Ali Muqaibel 36

Matlab Example: Binomial
• % Dr. Ali Muqaibel 0.4

• clear all 0.35

• close all 0.3

• % EE315 HW 3
0.25

N=6;
0.2

pdf
• 0.15

• P=0.25; 0.1

• x=0:N; 0.05

• y=binopdf(x,N,P) 0
0 1 2 3 4 5 6 7

• y1=cdf('bino',x,N,P) Power delivered by the Master Station

• figure (1)
• stem(x,y)
• xlabel('Power delivered by the Master Station') 1

• ylabel ('pdf')
0.9

0.8

• % axis ([-.1 0.9 0 0.6]) 0.7

• figure (2)
CDF 0.6

0.5

• stairs(x,y1) 0.4

•
0.3

0.2

• xlabel('Power delivered by the Master Station') 0.1

0 1 2 3 4 5
Power delivered by the Master Station
6 7

• ylabel ('CDF')
• % axis ([0 0.8 0 1.2]) The following figure illustrates the binomial density and
distribution functions for 𝑁 = 6 and 𝑝 = 0.25.
37
Dr. Ali Muqaibel
Example: Binomial Distribution
A canned juice company creates an “Under the Caps” promotion whereby a
customer can win an instant cash prize of SR 1 by looking under the bottle cap
with probability of ¼ . A customer buys eight bottles of juice from this company.
a) Find the pdf (or PMF) of the number of winning bottles, and denote as .
b) Compute the probability of wining more than SR 4.
a) The random variable is binomial with parameters and

where
b) The probability of wining more than SR 4 is

Dr. Ali Muqaibel 38

 𝑏=1

Poisson 𝑏=4
𝑏=10

𝑃𝑀𝐹
• As constant
• Then , Binomial (discrete time) Poisson (Continuous
time)
• The Poisson RV has a density and distribution 𝑥

𝐶𝐷𝐹

𝑏=1
where b > 0 is a real constant. 𝑏=4
𝑏=10

Dr. Ali Muqaibel 𝑥 39

When to Apply Poisson Distribution?
The Poisson RV applies to a wide variety of counting-type applications:
- The number of defective units in a production line.
- The number of telephone calls made during a period of time.
- The number of electrons emitted from a small section of a cathode in a given
time interval.

Counting type problems:

Time interval of interest
Average arrival rate
How many arrival

40
Dr. Ali Muqaibel
Example: Poisson Distribution
• At a gasoline station, cars arrive according to Poisson distribution at a rate of 50
per hour. There is 1 gasoline pump and it takes 1 min to obtain fuel.
• What is the probability that a waiting line will occur.
•
• car per min
 k
b
• FX ( x)  e b  u( x  k )
• k  0 k!

• Probability of waiting
• Probability of waiting in a line=20.32%

Dr. Ali Muqaibel 41

 Practice: Poisson Dist.
Packets arrive at an internet router at a rate of
and the router needs to serve
each packet. If the router can . Find the
probability that new incoming calls will not be served.

bk
f X ( x)  e 
b
 (x  k) The Poisson distribution rate of arrival 𝜆 = 20 , 𝑇 = 3 𝑠𝑒𝑐

k  0 k!
Let 𝑏 = 𝜆𝑇 = × × 3 sec = 1
𝑒 𝑏 𝑒
 k
b 𝑃 𝑋=𝑘 = =
FX ( x)  e  b 
k  0 k!
u( x  k ) 𝑘! 𝑘!
Prob. of blocked service is=𝑃 𝑋 > 4 = 1 − 𝑃 𝑋 ≤ 4 = 1 − ∑ 𝑃(𝑋 = 𝑘)
𝑃 𝑋 = 0 = 𝑒 = 0.368
𝑒 𝑒 𝑒
Ans. 𝑃 𝑋 = 1 = 𝑒 ,𝑃 𝑋 = 2 =
2
,𝑃 𝑋 = 3 =
6
,𝑃 𝑋 = 4 =
24
1 1 1 65
𝑃 𝑋 >4 = 1−𝑒 2+ + + = 1−𝑒 = 3.66 × 10
2 6 24 24

Dr. Ali Muqaibel 42

Uniform Distribution

where

• Practical Importance: quantization

Dr. Ali Muqaibel 43

Exponential Distribution
•

where
• is the average value
• Used to represent the fluctuation of signal
strength in “radar”… rain drop size

Dr. Ali Muqaibel 44

 Example: Exponential Distribution
• is the received by a radar from aircraft

• is the average amount of received power .

• What is the probability that the received power is above the average?

• Exponential with 𝑓 𝑥 = 𝑒 𝑥>𝑎

0 𝑥<𝑎

Dr. Ali Muqaibel 45

 Important in wireless applications,

Rayleigh and measurements errors

The Rayleigh density and distribution functions are

Dr. Ali Muqaibel 46

Median & Example for Rayleigh
• Median: It is the value of for which the probability is 0.5 that the
values of does not exceed the median.
• The value such that , this is called
the median.
• Find the median for Rayleigh distribution?
•
median≠ average
Find out the difference
•
•

Dr. Ali Muqaibel 47

Conditional Distribution and Density
Functions
• For two events and the conditional probability of event
given event had occurred was defined as

• We extend the concept of conditional probability to include

random variables
In conditional probability, the sample space is
reduced to the space given by the condition.

If given 𝐵, we just care about the probability of 𝐴

occurring inside of 𝐵.

Dr. Ali Muqaibel 48

Conditional Distribution

Let be a random variable and define the event

We define the conditional distribution function

Dr. Ali Muqaibel 49

 Properties of Conditional Distribution

if None Decreasing

Right continuous
(1) F (−∞|𝐵) = 0 (2) F (∞|𝐵) = 1
𝑝𝑟𝑜𝑜𝑓 𝐹 −∞ 𝐵 = P X ≤ −∞|𝐵 𝑃𝑟 𝑜 𝑜𝑓 𝐹 (∞|𝐵) = P X ≤ ∞|𝐵
𝑃 X ≤ −∞ ∩ 𝐵 0
= = =0 𝑃 X ≤ ∞∩𝐵 𝑃(𝐵)
𝑃(𝐵) 𝑃(𝐵) = = =1
𝑃(𝐵) 𝑃(𝐵)
Dr. Ali Muqaibel 50
Conditional Density Functions

Conditional Density Function of the random variable is the derivative

of the conditional distribution function

If contain step discontinuities as when X is discrete or mixed

(continues and discrete ) then will contain impulse functions.

Dr. Ali Muqaibel 51

Properties of Conditional Density

, for all

Dr. Ali Muqaibel 52

Example: Conditional density & distribution
• Two mutually exclusive events: 𝒙 Ball Color Small box Larger box Total
 “select the smaller box”, (𝐵 ) (𝐵𝟐 )

 “select the larger box”, 1 Red 5 80 85

 A random variable is defined to be 2 Green 35 60 95

 3 Blue 60 10 70
 Total 100 150 250


 + +
 + +

Dr. Ali Muqaibel 53

 Continue Example
• For comparison (non conditional) total
probability:
• 𝑃 𝑋=1
=𝑃 𝑋=1𝐵 𝑃 𝐵 +𝑃 𝑋 =1 𝐵 𝑃 𝐵
5 2 80 8
= + = 0.437
100 10 150 10

•
•
 𝑓 (𝑥) = 0.437𝛿(𝑥 − 1)+0.390𝛿(𝑥 − 2)+0.173𝛿(𝑥 − 3)
 𝐹 (𝑥) = 0.437𝑢(𝑥 − 1)+0.390𝑢(𝑥 − 2)+0.173𝑢(𝑥 − 3)

Dr. Ali Muqaibel 54

Range Conditioning
Next we define the event were is a real number
,
Case 1:

Dr. Ali Muqaibel 55

Case 2:

By combining the two expressions we get

Dr. Ali Muqaibel 56

Then the conditional distribution
is never smaller then the
ordinary distribution

Dr. Ali Muqaibel 57

Range Conditional Density Function 𝐹 (𝑥)
𝐹 𝑥 𝑋 ≤ 𝑏 = 𝐹 (𝑏)
𝑥<𝑏
1 𝑥≥𝑏
• The conditional density function derives from the derivative

• Similarly for the conditional density function

for

Dr. Ali Muqaibel 58

Example 1: Conditional Exponential R.V.
Let be a random variable with an exponential
probability density function given as

Find the probability

( )
f X (x | X  2)
• Since ∫

f X (x )

Dr. Ali Muqaibel 59

Continue Example Solution

Dr. Ali Muqaibel 60

Example2: Conditional Rayleigh
• 𝑃(𝑏𝑢𝑙𝑙’𝑠 𝑒𝑦𝑒 | 𝑙𝑎𝑛𝑑𝑖𝑛𝑔 𝑜𝑛 𝑡𝑎𝑟𝑔𝑒𝑡).

10𝑚

50𝑚

• P(bull’s eye | landing on target)=

Target
Bull's-eye
• Check end of chapter summary

Dr. Ali Muqaibel 61

Histogram & Converting Data to PDF
% Dr. Ali Muqaibel 0.45
Hist Matlab live demo:
close all 0.4 Model
KSDensity • Impact of number of points
0.35
clear all • Difference between histogram and pdf ;
0.3
Normalization
clc
0.25 • Fitting
n=10000; 0.2 • See also:
• http://www.math.uah.edu/stat
f=randn(1,n); 0.15
• http://mathworld.wolfram.com
[y,x]=hist(f,10); 0.1 • Check ‘pdf’, ‘cdf’ commands in Matlab
0.05
y=y/n/(x(2)-x(1));
0
xm=-1:0.01:10; -5 0
x
5 10

ym=pdf('norm',xm,0,1);
[yr,xr]=ksdensity(f);
plot(x,y,xm,ym,xr,yr,':');
legend
('Hist','Model','KSDensity')
xlabel('x')
Dr. Ali Muqaibel 62