EEF 271E

PROBABILITY AND STATISTICS

FALL 2022

Lecture 04
Independence
Dr. Ramazan Çağlar

Lecture Notes 04 Independence 1

 Lesson Overview

 Independence

 Independence of Multiple Events

 Combined Experiments or Sequences

of Experiments

 Special case: Sequences of independent

experiments

 Bernoulli Trial

 Binomial Probability Law

Lecture Notes 04 Independence 2

 INDEPENDENCE

In some cases, the conditional probability of B given that

A occurred might equal to the Probability of B.
P(A|B) = P(A)
P(B|A) = P(B)

In this special case, knowledge that the outcome of the

experiment is in event B does not affect the probability
that the outcome is in event A.

Lecture Notes 04 Independence 3

Definition:
Two events are independent if any one of the
following equivalent statements is true:

(1) P(A|B)=P(A)

(2) P(B|A)=P(B)

(3) P(AB)=P(A).P(B)

Lecture Notes 04 Independence 4

 Recall that
P(AB) = P(A|B).P(B) always

= P(B|A).P(A) true !

Then, if A and B are independent,

We must have
P(A|B) = P(A) and
P(B|A) = P(B)

since P(AB) = P(A).P(B)

Lecture Notes 04 Independence 5

Intuitively,

P(A|B) = P(A)

means that ‘’whether or not B occurs’’ plays no part

in determining the probability that A also occurs.

How do we conclude, in practice, that two events

A and B are independent?

Answer: Based on physical reasoning,

Empirical evidence etc.

Lecture Notes 04 Independence 6

Examples:
* Toss two coins, then whether or not heads
appears on one is independent of what
appears on the other coin.

* More generally, one can think of experiments

where the events of different trials are
independent of each other (and also
experiments where they are not).

Lecture Notes 04 Independence 7

Examples:
A desk drawer contains 8 pens, four of which
are blue, numbered 1 through 4 and the
remaining are red, numbered 5 through 8.

A pen is drawing at random.

 = { (B,1),(B,2),(B,3),(B,4),

(R,5),(R,6),(R,7),(R,8) }

Lecture Notes 04 Independence 8

Examples:
 = { (B,1),(B,2),(B,3),(B,4),

(R,5),(R,6),(R,7),(R,8) }

A = {(B,1),(B,2),(B,3),(B,4)} “blue pen”

B = { (B,2),(B,4),(R,6),(R,8) } “even numbered pen”

C = { (B,1),(B,2) } “number on pen is less than 3”

(a) Are A and B independent?

P(A)=4/8,P(B)=4/8, P(AB)=2/8=P(A).P(B)
YES
Lecture Notes 04 Independence 9
Examples:
(b) Are B and C independent?

P(B)=4/8 , P(C) = 2/8, P(BC) = 1/8 = P(B).P(C)

YES

(c) Are A and C independent?

P(A)=4/8 , P(C) = 2/8, P(BC) = 2/8 ≠ P(A).P(C)

N0
We see from (a) and (b) of this example
‘’independence’’ of events may not always be
intuitively apparent.
10
Lecture Notes 04 Independence
Example:

Suppose A and B are mutually exclusive.

Are they independent ?
(Answer: Always N0 ! )

A ∩ B = ∅ , so P A∩B = 0 ≠ P A P B
Unless P(A) = 0
or P(B) = 0

Lecture Notes 04 Independence 11

 Note:

A and B are mutually exclusive means

𝑃 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵

A and B are independent means

𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵

Independence 12
Lecture Notes 04
Example:

If A ⊂ B , are they independent ?

Want: P 𝐴∩𝐵 =𝑃 𝐴 = 𝑃 𝐴 𝑃 𝐵

true, only if P(B) = 1

( example, B = Ω )

Lecture Notes 04 Independence 13

 Independence of Multiple Events
Events A1, A2, …, An are said to be independent if
for k=2,3,…,n
𝑃 𝐴1 ∩ 𝐴2 ∩ ⋯ ∩ 𝐴𝑖𝑘 = 𝑃 𝐴1 𝑃 𝐴2 … 𝑃 𝐴𝑖𝑘
where 1 ≤ 𝑖1 ≤ 𝑖2 …≤ 𝑖𝑖𝑘 ≤ 𝑤
For all k-fold intersections, the problem of the
intersection is the product of the probabilities of k
sets.
For example, when n=3, we want
𝑃 𝐴1 ∩ 𝐴2 = 𝑃 𝐴1 𝑃 𝐴2
𝑃 𝐴1 ∩ 𝐴2 ∩ 𝐴3
𝑃 𝐴1 ∩ 𝐴3 = 𝑃 𝐴1 𝑃 𝐴3 ቮ
= 𝑃 𝐴1 𝑃 𝐴2 𝑃 𝐴3
𝑃 𝐴2 ∩ 𝐴3 = 𝑃 𝐴2 𝑃 𝐴3
Lecture Notes 04 Independence
14
 Example:

Expt: Toss a fair coin three times, and note dawn the
sequence of heads and tails.
Ω = {HHH, HHT, HTH, HTT, THH, TTH, THT, TTT}
If we assume that the events from the three coin
tosses are independent,
1
𝑃 {𝐻𝐻𝐻} = 𝑃 𝐻 𝑃 𝐻 𝑃 𝐻 =
8
Similarly, the other seven elementary events have a
𝟏
Probability of
𝟖

Lecture Notes 04 Independence 15

Combined Experiments or Sequences of Experiments
Suppose a random experiment consists of performing a
sequence of sub-experiments
E1, E2, E3, … , En
If wi denotes the outcome of experiment Ei, then the
outcome of the random experiments can be expressed as
(w1, w2, w3, … ,wn) (called “n tuple”
or “n vectors”)
Thus the sample space is
 =  1 x 2 x 3 x … x n | w1E1,
w2E2,…, wnEn, 
where i is the sample space of the ith experiment
x is the “ Cartesian product”
Lecture Notes 04 Independence 16
Events, as usual, are defined as subsets of Ω,
so their elements are n-tuples as well.
We can represent some of the events in
another way.

Let A1 ∊ F1 , A2 ∊ F2 , … , An ∊ Fn
where Fi denotes the events space of the ith
sub-experiment.

WARNING:
Not all events can be represented this way!!
That is, there are subsets of Ω that cannot
be expressed as A1xA2x … xAn for Ai∊ F
Lecture Notes 04 Independence 17
 Example:
Expt:
𝑪𝒐𝒊𝒏 𝒕𝒐𝒔𝒔 followed by 𝒓𝒐𝒍𝒍𝒊𝒏𝒈 𝒂 𝒅𝒊𝒆
E1 E2
Ω1 = { H, T } Ω2 = { 1, 2, 3, 4, 5, 6 }

𝐻, 1 , 𝐻, 2 , 𝐻, 3 , 𝐻, 4 , 𝐻, 5 , (𝐻, 6)
Ω1 x Ω2 =
𝑇, 1 , 𝑇, 2 , 𝑇, 3 , 𝑇, 4 , 𝑇, 5 , (𝑇, 6)

F1 = {{H}, {T}, {H,T}, ∅ }

F2 = { {1}, {2}, …, {1, 2, 3, 4, 5, 6}, ∅ }

Lecture Notes 04 Independence 18

The event
‘’heads followed by an even # on the die’’,
can be written as

{{H, 2}, {T,6}} = {H} x {2, 4, 6}

The event
{{H, 2}, {T,6}} cannot be expressed as A1xA2
for some A1 ⊂ Ω1 ,
A2 ⊂ Ω2

Lecture Notes 04 Independence 19

Special case: Sequences of independent experiments

Note that the word “independent” means that the

outcomes of the sub-experiments

E1, E2, E3, … ,En

do not influence each other. More precisely, for any

A1F1, A2F2, …, AnFn,

we have

P(A1 x A2 x … x An) = P(A1 ) x P(A2 ) x … x P(An)

Lecture Notes 04 Independence 20

 Example:

If the coin and die are fair,

and E1 & E2 are independent,
then

P( {H} x {2,4,6} ) = P ({H}) . P({2,4,6})

= (½). (3/6) = ¼

Lecture Notes 04 Independence 21

 Example:
A couple plans to have three children.
(a) What is the probability that the second child is a girl?
And,
(b) what is the probability that the second child is a girl
given that the first child is a girl?
(c) Is the event that the couple's third child is a girl
independent of the event that the couple's first two
children are girls?

Independence 22
Lecture Notes 04
Solution of the example:
Ω = { GGG, GGB, GBG, GBB, BGG, BGB, BBG, BBB }
#Ω = |Ω| = 8 : number of the elements of Ω
Let A = event first child is Girl.
A = { GGG, GGB, GBG, GBB}, #A = 4
#𝑨 𝟒 𝟏
𝑷 𝑨 = = = 𝑏𝑦 𝑐𝑙𝑎𝑠𝑠𝑖𝑐𝑎𝑙 𝑎𝑝𝑝𝑟𝑜𝑎𝑐ℎ
#𝜴 𝟖 𝟐
Let B = event second child is Girl.
B = { GGG, GGB, BGB, BGG}, #B = 4
#𝑩 𝟒 𝟏
𝑷 𝑩 = = = 𝑏𝑦 𝑐𝑙𝑎𝑠𝑠𝑖𝑐𝑎𝑙 𝑎𝑝𝑝𝑟𝑜𝑎𝑐ℎ
#𝜴 𝟖 𝟐
Let C = A∩B = event the couple's first two children are girls.
A∩B = { GGG, GGB}, #(A∩B) = 2
#(𝑨 ∩ 𝑩) 𝟐 𝟏
𝑷 𝑨∩𝑩 = = = 𝑏𝑦 𝑐𝑙𝑎𝑠𝑠𝑖𝑐𝑎𝑙 𝑎𝑝𝑝𝑟𝑜𝑎𝑐ℎ
#𝜴 𝟖 𝟒
Lecture Notes 04 Independence 23
Solution of the example (cont.):
Ω = { GGG, GGB, GBG, GBB, BGG, BGB, BBG,BBB }
A = {Event first child is Girl} P(A) = 0.5
B = {Event second child is Girl} P(B) = 0.5
(A∩B) =C = {Event the couple's first two children are girls}.
P(A∩B) = P(C) = 0.25
a) The probability that the second child is a Girl. P(B) = 0.5
b) The probability that the second child is a girl given that the first
child is a girl.
𝑷(𝑨 ∩ 𝑩) 𝟐/𝟖 𝟐 𝟏
𝑷 𝑩|𝑨 = = = =
𝑷(𝑨) 𝟒/𝟖 𝟒 𝟐
c) Is the event that the couple's third child is a girl independent of
the event that the couple's first two children are girls?
Let D = the event that the third child is a girl.
Lecture Notes 04 D = { GGG, GBG, BGG, BBG}, P(D) = 4/8 = 0.5 24
Solution of the example (cont.):
c) Let D = the event that the third child is a girl. P(D) = 0.5
Now, (C∩D) becomes the event that all three children are girls and
hence it looks like:
(C∩D) = {GGG}
Using the classical approach to assigning probability to the (C∩D):
#(𝑪 ∩ 𝑫) 𝟏
𝑷 𝑪∩𝑫 = = = 𝟎. 𝟏𝟐𝟓
#𝜴 𝟖
The condition of the independence is P(C∩D) = P(C).P(D)

Now, since: P(C) × P(D) = 2/8 ×4/8 = 8/64 = 1/8 = 0.125

P(C) × P(D) = P(C∩D)
we can conclude that events C and D are independent. That is, the
event that the couple's third child is a girl is independent of the event
that the first two children were girls.

Lecture Notes 04 Independence 25

 Three theorems for independence

Theorem: If A and B are independent events, then the

events A and Bc are also independent.
𝑃 𝐴 ∩ 𝐵 𝑐 = 𝑃 𝐴 𝑃 𝐵 𝑐 |𝐴 by multiplication rule
= 𝑃 𝐴 [1 − 𝑃 𝐵|𝐴 by Axiom of Probability
= 𝑃 𝐴 [1 − 𝑃 𝐵 ] by the rule of Independence
= 𝑃 𝐴 𝑃 𝐵𝑐 ⇒ A and Bc are independent
Theorem: If A and B are independent events, then the
events Ac and B are also independent.
It can be prove with same way as given above

Lecture Notes 04 Independence

26
 Three theorems for independence
Theorem: If A and B are independent events, then the
events Ac and Bc are also independent.
𝐴𝑐 ∩ 𝐵 𝑐 = 𝐴 ∪ 𝐵 𝑐
DeMorgan’s law
𝑃(𝐴𝑐 ∩ 𝐵 𝑐 ) = 𝑃[ 𝐴 ∪ 𝐵 𝑐 ]
= 1−𝑃 𝐴∪𝐵 prob. axiom
= 1− 𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵
= 1−𝑃 𝐴 −𝑃 𝐵 +𝑃 𝐴∩𝐵
= 1 − 𝑃 𝐴 − 𝑃 𝐵 + 𝑃 𝐴 𝑃(𝐵) independence
= 1−𝑃 𝐴 1 − 𝑃(𝐵) prob. axiom
= 𝑃 𝐴𝑐 𝑃(𝐵 𝑐 ) prob. axiom
⇒ Ac and Bc are independent
Lecture Notes 04 Independence
27
 Mutual Independence
Definition.
Three events A, B, and C are mutually independent if and
only if the following two conditions hold:
(1) The events are pairwise independent. That is,
• P(A∩B) = P(A) × P(B) and...
• P(A∩C) = P(A) × P(C) and...
• P(B∩C) = P(B) × P(C)
(2) P(A∩B∩C) = P(A) × P(B) × P(C)
The idea of mutual independence can be extended to
four or more events — each pair, triple, quartet, and so
on, must satisfy the above type of multiplication rule.
Lecture Notes 04 Independence
28
 Bernoulli Trial

 Some experiment is repeated , say n times.

 Repetitions are independent.

 Each time, we only care about whether a

certain event A has occurred or not. If A
occurs, a sub-experiment is deemed a
‘’success’’.

Lecture Notes 04 Independence

29
 Bernoulli Trial
Example: Toss a coin 10 times, {H} is a success.

Ω = {H, T} x {H, T} x … x {H, T}

Possible outcomes = 210

An event with 7
1. { H, H, H, T, T, T, H, H, H, H } success
2. { H, H, T, T, T, H, H, H, H, H } Events with 7
∙ ∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙ success
nCk∙ { H, H, T, H, T, H, H, H, T, H }
An event with 9 success
{ H, H, H, H, H, H, H, H, H, T }
∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙∙ Events with 9 success
{ T, H, H, H, H, H, H, H, H, H } n Ckis often called ‘’ n choose k’’
and shows the number of ways
etc. that an event with k success
Lecture Notes 04 Independence
30
 Bernoulli Trial
Generally, since we only care about whether A occurs
during each sub-experiment or not, the sample space is

Ω = {A, Ā} x {A, Ā} x … x {A, Ā}

Now, suppose P(A) = p
P(Ā) = q = 1 – p
Then for any elementary even, the probability is pkqn-k
where k is the number of ‘’success’’. WHY?

P({A, Ā,A,A,Ā,A,…,A,Ā}) = P(A)P(A)… P(A) P(Ā)P(Ā)…P(Ā)

say k successes k terms, (n-k) terms,

pk qn-k
Lecture Notes 04 Independence
31
 More general questions

 What is the probability that there are

exactly k successes in n trials?
 What is the probability that there are
at least k successes in n trials?
 What is the probability that there are
at most k successes in n trials?
etc.

Lecture Notes 04 Independence

32
Answers provided by the Binomial Probability Law
𝑛!
𝑃𝑛 𝑘 = 𝑝𝑘 1 − 𝑝 𝑛−𝑘
, 𝑘 = 0,1, … , 𝑛
𝑘! 𝑛 − 𝑘 !

where
𝑃𝑛 𝑘 is the probability that there are exactly k success.
k is # of success
n is total number of trials
𝑛! 𝑛 𝑛 𝑛
= 𝑜𝑟 𝐶
𝑛 𝑘 𝑜𝑟 𝐶𝑘 𝑜𝑟 𝐶𝑘
𝑘! 𝑛 − 𝑘 ! 𝑘
Combination: n choose k.
Lecture Notes 04 Independence
33
Sketch of proof (Counting Arguments!)

The event ‘’exactly’’ k success in n trials is the set

{ A, A, … A, Ā, …, Ā, A,…,A, A, Ā, …, Ā, Ā, A, … }
{Ā, A, Ā, A, … A, A, Ā, …, Ā, A,…, Ā, Ā, A, Ā, … }
{ A, A, … Ā, Ā, …, Ā, A,…, Ā, A, A, Ā, …, A, A, … }

Each has ‘’exactly’’

k occurrence of A
Lecture Notes 04 Independence
34
1. Show that there are
𝒏!
𝒌! 𝒏 − 𝒌 !
elements in this set (formed from the elements in
this set).
2. Each elementary event has Probability

𝒑𝒌 𝒒𝒏−𝒌
3. Thus, probability of ‘’exactly k success’’ is
𝒏!
𝑷 Exactly k succes = 𝒑𝒌 𝒒𝒏−𝒌
𝒌! 𝒏 − 𝒌 !

Lecture Notes 04 Independence 35

The binomial probability law helps answer the other
questions raised:
 P({at least k success})
𝑷 at least k succes = 𝑷 k succes + 𝑷 k+1 succes
+𝑷 k+2 succes + ⋯ + 𝑷 n succes
𝒏 𝒏

= ෍ 𝑷𝒏 𝑿 = 𝒊 = ෍ 𝑷𝒏 𝒊
𝒊=𝒌 𝒊=𝒌

 P({at most k success})

𝑷 at most k succes = 𝑷 0 succes + 𝑷 1 succes
+𝑷 2 succes + ⋯ + 𝑷 k succes
𝒌 𝒌

= ෍ 𝑷 𝒏 𝑿 = 𝒊 = ෍ 𝑷𝒏 𝒊
𝒊=𝟎 𝒊=𝟎
Independence
Lecture Notes 04 36
Example:
A pair of dice (let say a black dice and a white dice)
tossed eight (8) times, and number of points on each
die added up.
(a) What is the probability that ‘’7 occurs exactly 4
times’’ ?
P({ 7 occurs exactly 4 times }) = ?

Ω = { (i,j): i, j =1,2,3,4,5,6 } and |Ω| = 36.

ωi,j = (i,j) i: # on black die, j: # on white die
E = “ the two dices together give 7”
E = {(1,6) (2,5), (3,4), (4, 3), (5,2), (6,1)}
|E| = 6
Independence 37
Lecture Notes 04
Example:
(a) P({ 7 occurs exactly 4 times }) = ?
P({7 occurs during one toss}) =
= P({(1,6) (2,5), (3,4), (4, 3), (5,2), (6,1)})
= P(E)
|𝐸| 6 1
𝑃 𝐸 = = =
|Ω| 36 6
Thus, in our notation, p = 1/6 and q = 5/6
The answer we want is
4 4
8! 1 5
𝑃 Exactly 4 succes =
4! 8 − 4 ! 6 6
= 0.02605
Independence 38
Lecture Notes 04
Example:

(b) P({ 11 occurs twice}) = ?

P({ 11 occurs during one roll}) =
= P( { (6,5), (5,6) } )
= P(E)
2 1
𝑃 { (6,5), (5,6) } = → 𝑝=
36 18
So the answer we want is
𝑃 11 occurs 𝑡𝑤𝑖𝑐𝑒 = 𝑃8 2
2 6
8! 1 17
𝑃8 2 =
2! 8 − 2 ! 18 18
= 0.0613
Independence 39
Lecture Notes 04
Example:

(c) P({ 12 occurs more than once }) = ?

P({12 occurs more than once }) =
= 1- P({12 occurs 0 or 1 time(s) })
= 1- P8(0) – P8(1)
1 1 35
𝑃 { (6,6) } = → 𝑝= , 𝑞=
36 36 36
So the answer we want is
𝑃 12 occurs more than once =
1 𝑖 36−𝑖
8! 1 35
=1−෍
𝑖! 8 − 𝑖 ! 36 36
𝑖=0
= 0.0193
Independence 40
Lecture Notes 04