MATH-361

Probability and Statistics

Lecture 17

Independence, CLT, Correlation,

Covariance

A/P Kamran Aziz Bhatti | Assistant Professor | Dept. of Electrical Engg. | NUST College of Electrical & Mechanical Engineering | Pakistan
Independence of
Random Variables
 RVs of Different Types

• Two RVs X and Y are independent if an event A defined in

terms of X is independent of any event B defined in terms of Y
𝑃 𝑋 𝑖𝑛 𝐴, 𝑌 𝑖𝑛 𝐵 = 𝑃 𝑋 𝑖𝑛 𝐴 𝑃 𝑌 𝑖𝑛 𝐵
• The Discrete random variables X and Y are independent if and
only if the joint pmf is equal to the product of the marginal
pmfs for all possible outcomes
• RVs X and Y are independent if and only if their joint cdf/pdf
is equal to the product of its marginal cdfs/pdfs
𝑓𝑋,𝑌 𝑥, 𝑦 = 𝑓𝑋 𝑥 𝑓𝑌 𝑦

3
Statistical Properties of
Multiple RVs
 Expected Value

• Expected value of the sum of RVs is equal to the sum of the

individual expectations (Even though they are not
independent)

5
 Correlation and Covariance

• The joint moments of two RVs X and Y summarize information

about their joint behavior.

• Setting either j or k equal to zero results in individual moments

• An important expectation is E[XY], which is termed as the
Correlation of X and Y
• If 𝐸 𝑋𝑌 = 0, X and Y are orthogonal.

6
 Correlation and Covariance

• The pairs of independent RVs have zero covariance.

7
 Correlation and Covariance

• The extreme values of 𝜌𝑋,𝑌 occur when X and Y are related

linearly.

8
 Correlation and Covariance

• VERY IMPORTANT POINTS:

• X and Y are said to be uncorrelated if 𝜌𝑋,𝑌 = 0

• X and Y are independent if 𝐶𝑂𝑉 𝑋, 𝑌 = 0
• If X and Y are independent then they are also uncorrelated
• It X and Y are uncorrelated, it is not necessary that they are
independent
• For jointly Gaussian RVs, X and Y are independent if and only if
they are uncorrelated.

9
Central Limit Theorem
 Central Limit Theorem

• Let 𝑋1 , 𝑋2 , … denote iid random variables with finite mean 𝜇

and finite variance 𝜎 2
• Let 𝑆𝑛 denote the sum of the first n random variables

• How to find the pdf of 𝑆𝑛 , when the distribution of 𝑋𝑗 ’s is

unknown???
• As n becomes large, the cdf of a properly normalized 𝑆𝑛
approaches that of a Gaussian RV.

11
 Central Limit Theorem

• Let 𝑆𝑛 be the sum of n iid RVs with finite mean 𝜇 and finite
variance 𝜎 2 , and let Z𝑛 be the zero-mean, unit variance RV
defined by,

Then

• This theorem explains why the Gaussian RV appears in so many

applications

12
Central Limit Theorem

13
Central Limit Theorem

14
Central Limit Theorem

15
Central Limit Theorem

16