CLASS Statistics

03 Conditional Probability

Inclusion Exclusion Principal

|A UB UC | =

|A |+ | B |+ | C| Inclusion (+ve)
U U U
-|A B|-| A C|- |B C| Exclusion (-ve)
U U
+ |A B C | Inclusion (+ve)

A B

What is Probability

Number of ways it can happen

Probability of an event happening =
Total number of outcome

Probability is the measure of the Possibility

Impossible Unlikely Even Chance Likely Certain

0 1
1 in 6 Chance 4 in 5 Chance

Probability is always between 0 and 1

Types of Events

▪ Independent (each event is not affected by other events),

▪ Dependent (also called "Conditional", where an event is affected by
other events)
▪ Mutually Exclusive (events can't happen at the same time)

Mutually Exclusive vs Joint Probability

Hearts
Aces Kings Kings
A
A K 1 Q K
2  J
A A K K K K
3 6 8
A K 5 7 K
10 9

Aces and Kings are Mutually Exclusive

Hearts and Kings are not Mutually Exclusive
(can’t be both)
(can be both)

Dependent vs Independent
Number of Blue Marbles = 6
Number of Yellow Marbles = 9

Dependent Events Independent Events

The first marble is not replaced. The first marble is replaced.

X : Blue first X : Blue first

Y : Yellow second Y : Yellow second

P(X) x P(X/Y) = P(X and Y) P(X) x P(X/Y) = P(X and Y)

6 9 54 6 9 54
x = ~ 26% x = ~ 24%
15 14 210 15 15 225

Example Independent Probability

For each toss of a coin a Head has a probability of 0.5:

H
0.5

H H
1 1 1
0.5 x 0.5 = 0.25 (or x = )
2 2 4

H H H
1 1 1 1
0.5 x 0.5 x 0.5 = 0.125 (or x x = )
2 2 2 8

And so the chance of getting 3 Heads in a row is 0.125

 Dependent Probability

Probability of Given

P( A an d B) = P(A) x P(B|A)
Event A Event B

U
P (A B ) = P (A ) x P(B|A)
U
P (A B ) = P (B ) x P(A|B)

Conditional Probability

A B
Conditional Probability Formula

U
P(A B)
P(A|B)=
P(B)
Probability that A ocours given
that B has already occured

P(A)
P(B)
U
P(A B)

Example of Conditional Probability

2 in 4

P(Blue|Red)=2/4

2 in 5
1 in 4

P(Blue|Blue)=1/4

Posterior Probability

P ( A and B)
P(A|B)=
P ( B)
Posterior Probability Evidence or
Marginal Probability

Bayes Theorem

U
P (A B )
P(A|B) =
P (B )
U
P (B A )
P(B|A) =
P (A )

Likelihooh Prior

.
P (B|A) P(A)
Posterior P (A | B ) =
P(B)

More on Bayes Rule

P ( A | F ) . P (F ) P ( A | F) . P ( F)
P(F|A) = =
P (A ) P (A| F) P ( F) + P ( A | F ) P ( F )
C C

Where P(A) = P(A|F)P(F) + P(A| Fc)P(Fc)

Bayes rule with extra probability

Bayes Rule with Extra Conditions

P (M | A ). P (N |A ) . P ( A )
P(A|M,N) =
P (M ). P ( N)

Calculation of overall probability

0.5
Head Head, Head 0.5 x 0.5 = 0.25
0.5
Head
Tail Head, Tail 0.5 x 0.5 = 0.25
0.5
O
0.5
Head Tail, Head 0.5 x 0.5 = 0.25
Tail
0.5
Tail Tail, Tail 0.5 x 0.5 = 0.25
0.5

Total = 1.00
Difference between Probability and Odds

Prob ab i l i t y =

Odds =

Probability in terms of Odds

Odds(A) = P(A) / (1 – P(A))

P(A) = Odds(A) x (1 – P(A))

Also, 1 = 1 – P(A) 1/Odds(A) = 1/P(A) -1

Odds(A) P(A)

1/Odds(A) + 1 = 1/P(A)

(1 + Odd(A) )/Odds(A) = 1/P(A)

Thus, P(A) = Odds(A)/ (1 + P(A))

More on Dependent and Independent Probability

Independent Events

P( A and B ) = P (A ) x P(B)

Dependent Events

P( A and B ) = P (A ) x P(B|A)

Mutually Exclusive

P( A or B ) = P (A ) + P(B)

Mutually Inclusive

P( A or B ) = P (A ) + P(B) - P(A and B)