Lecture 3: Conditional Probability, Independence and Bayes

MA2224 Data Analysis

2024-01-29

Conditional Probability

Including Information

It most applications, dealing with an absolute probability is not the goal. You often want to know probability for
some subgroups or under different conditions.
For example, knowing the overall mortality rate is often not as useful as knowing the mortality rate for people over 60,
or for women or for people who bike the wrong way down the street.
Incorporating subgroups or other information into your probabilities is called conditioning on that subgroup so that
makes this topic conditional probability

Conditional Probability.
The conditional probability of event A given event or information B is written as

P (A | B) = P (A | Inf ormation)

Read as "The probability of A GIVEN B" or "The probability of A CONDITIONING ON B" or "The probability
of A KNOWING the information"
The information that is conditioned on becomes the new sample space.

Basic Examples

Try and answer the following conditional probability problems just by thinking about them with what you know
already.

1. You have 3 nickles and 5 quarters in your pocket, you randomly select 2, what is the probability the second one
is a quarter if the first one is a quarter?

There are 8 ∗ 7 = 56 total outcomes (if we order things) but we only need to use the given which is 5 ∗ 7 = 35
outcomes that start with a quarter. Of those 5 ∗ 4 = 20 are both quarters. P (2nd Q|1st Q) = 20/35 = 4/7.
Conditional Probability Thinking

2. Section A contains 3 seniors and 5 juniors, section B contains 6 seniors and 2 juniors. You randomly select a
student from section A and put it in section B without caring who they are. Then you randomly select a student
from section B. What is the probability it is a senior given the student moved was a senior?

There are 8 ∗ 9 = 72 ways this could go down total, but we only want the 3 ∗ 9 = 27 outcomes that start with a
senior from A based on the given information, then there will be 3 ∗ 7 = 21 that go senior senior. So 21/27 = 7/9

3. Two coins are flipped, what is the probability of both coins being heads if I tell you at least one coin is heads?

Initial sample space: {(HH), (HT ), (T H), (T T )}, Only 3 outcomes left based on the given: {(HH), (HT ), (T H)} so
P (HH|At least 1 D) = 1/3.

Conditional Probability Thinking

Conditional probability can be tricky to do and we will go over some formulas to help but getting the thought process
down a bit first would be a good idea.
Even if what you are working on is “gathering information” you can still think as if there is a sequence of events
(ordered sample spaces are always safe with probability just sometimes they have worse math). This basically creates
paths you can travel on that can be represented with a tree.
Imagine there are two events that you plan to observe, Event 1 can be A or B and Event 2 can be R or L.
The sample space will be SS = {AR, AL, BR, BL} but we have no way to break this down into an equally likely
sample space without knowing the story and even then it may not be possible.
Using a tree for visualization can be helpful.
Conditional Probability Formula

Concretely, Suppose I want to determine the probability a student is going to cheat on a quiz but I think it is connected
to their attendance. From past classes I have gathered the following information:

• 35% of students are absent (A) for more than half the lectures the rest are present (P)
• 50% of the students who are absent more than half the time cheat (Y)
• 5% of the students who are present more than half the time cheat (Y)

You can build this into a tree, I like to put the probabilities on the branches

Conditional Probability Formula

Conditional Probability Formula.

For two events A and B the Conditional Probability of A given B can be found as

P (AB) P (A ∩ B) P (Both)
P (A | B) = = =
P (B) P (B) P (Given)

With a little cross multiplication you get the more frequently used multiplication rule:

Multiplication Rule of Probability.

For two events A and B the probability of A and B happening is

P (A and B) = P (AB) = P (A ∩ B) = P (A)P (B|A) = P (B)P (A|B)

In some words:
P (One T hing)P (T he Other T hing | T he F irst T hing Happened)
Conditional Probability Formula Thinking

Conditional Probability Formula Thinking

In terms of the tree, this formula says that if you use the word AND then you find the path you need and multiply
the probabilities to get there.
So if we want the probability that a that student misses more than half the lectures AND cheats it will be:

P (AY ) = P (A)P ( Y | A) = 0.35(0.5) = 0.175

Which is just the product of the probabilities on that path.

Now if we want the probability that a student cheats (i.e. regardless of their attendance) then we just have to add up
the paths that lead to them cheating

P (Y ) = P (A)P (Y | A) + P (B)P (Y | B) = 0.35(0.5) + 0.65(0.05) = 0.2075

Which is just the sum of the path probabilities.

Example 1

Example 1

You text your ex once every 5 minutes until you get a response. Initially there is a 12% chance of a response but that
decreases 2% each time you send a text.

1. What is the probability you send exactly 3 texts?

P (IIR) = 0.88(0.90)(0.08) = 0.063

2. What is the probability you send at most 4 texts?

0.12 + 0.88(0.10) + 0.88(0.90)(0.08) + 0.88(0.9)(0.92)(0.06) = 0.315

3. What is the probability you get ignored more than 4 times?

P (Ignored > 4) = 1 − P (Ignored ≤ 4) =

1 − (0.12 + 0.88(0.10) + 0.88(0.90)(0.08) + 0.88(0.9)(0.92)(0.06) + 0.88(0.9)(0.92)(0.94)(0.04))

0.658

Example 2

Three guys are trapped in a house by zombies. They decide to try and escape one at a time in a random order. The
probability the first guy who runs away survives is 40% and if a guy gets away then that probability decreases by 15%
for the next guy. If that first guy doesnt survive then the probability the next guy escapes increases by 10%. This
pattern continues. Answer the following.

1. Write out the Sample Space for what happens to each guy, Survives (S) or Eaten (E).

SS = {EEE, EES, ESE, SEE, ESS, SES, SSE, SSS}

2. What is the probability all three survive? Use the multiplication rule.

0.4(0.25)(0.1) = 0.01

3. What is the probability none survive?

0.6(0.5)(0.4) = 0.12

4. What is the probability exactly one of them survives?

0.4(0.75)(0.65) + 0.6(0.5)(0.65) + 0.6(0.5)(0.6) = 0.57

Independence

Independence

Estimate the following.

1. P (Y ou f ail this class)

2. P (Y ou f ail this class | I wear black every class)
3. P (Y ou f ail this class | I send you daily hate email)

Parts 1 and 3 should have very different estimates, while part 2 should be the same as part 1 because sometimes
information is useless, as in (2), to what you want to know. When this is the case we say that the events are
Independent.

Independence.
Two events A and B are Independent if either of the following are true

P (A|B) = P (A) or P (B|A) = P (B)

Which means
A and B are Independent → P (AB) = P (A)P (B)

Independence is probably the most important probability idea we will need going forward.

Example 3

Three guys are trapped in a house by zombies in the sequel. They decide to try and escape all at once in different
directions. This makes their escape attempts independent and the probability each survives is 40%. Answer the
following.

1. What is the probability all three survive?

0.4(0.4)(0.4) = 0.064

2. What is the probability none survive?

0.6(0.6)(0.6) = 0.216

3. What is the probability exactly one of them survives?

3(0.4)(0.6)(0.6) = 0.432

4. If one of the guys chugs 6 red bulls before his escape attempt his probability of surviving increases to 80%. What
is the probability exactly one guy survives now.

0.8(0.6)(0.6) + 0.2(0.4)(0.6) + 0.2(0.6)(0.4) = 0.384

Sample Space Connection (KEY IDEA HERE)

Sample Space Connection (KEY IDEA HERE)

With the zombie questions, you are dealing with an identical sample space of 8 things that could happen. Each guy
could survive (S) or be eaten (E).

SS = {(EEE), (EES), (ESE), (ESS), (SSS), (SSE), (SES), (SEE)}

We saw this sample space in lecture 1, the only thing that changes is what probability weight we give to the elements
in the sample space. The following is what could happen from easy to hard:

• Equally Likely: Here this is identical to a 50-50 coin toss and all probabilities are 1/8 in the sample space.
• Independence with Equal Probabilities: Here identical groups of outcomes have the same probabilities so
you just need to
figure out the probability of one occurring then multiply by duplicates there are which you
can find using nx . Example 3 parts 1 - 3 are this situation
• Independence but Unequal Probabilities: Here you need to figure out all events individually then add
them but it is usually easy to do, because the independence just makes it simple multiplication and addition.
Example 3 part 4 is this situation
• Dependent Outcomes: Generally difficult to deal with as the dependency needs to be told to you and you
need to think carefully about each outcome’s probability. Example 2 is this situation.

So you really need to think about whether or not you need to focus on each individual sample space outcome or if you
can “group them”.

Example 4

Five people are independently trying to cheat during an exam. The probability of a cheater being caught is 15%.
What is the probability

1. At least one gets caught cheating

1 − P (N one Caught) = 1 − 0.855 = 0.556

2. Exactly 3 get caught cheating

From lecture 2 we know there are 53 = 10 ways to catch 3 of these 5 cheater. But the sample space is not equally

likely so we need to figure out the probability of each 10, however that probability is the same.

5
!
(0.153 )(0.852 ) = 10(0.153 )(0.852 ) = 0.024
3

With independence and equal probabilities the math is fairly simple.

Example 5

Example 5

Five people are independently trying to cheat during an exam. Three have a probability of 5% of being caught and
the other two have 30% probabilities of being caught. What is the probability

1. At least one gets caught cheating

1 − 0.953 (0.7)2 = 0.706

2. Exactly 3 get caught cheating

Still 10 outcomes to worry about but now those 10 are divided into 1 way where all the “good cheaters” get caught, 6
ways 2 of the good cheaters and 1 of the bad cheaters get caught, and 3 ways 2 of the bad cheaters get caught.

3 2 3
! ! !
0.05 (0.7) +
3 2
0.052 (0.3)(0.95)(0.7) + 0.05(0.3)2 (0.95)2 = 0.0152
2 1 1

Note that even with independence you can still need to think about each sample space outcome.
Bayes Theorem

Messin Around

Mathematically, the conditional probability formula is nothing special

P (AB)
P (A|B) =
P (B)

However if the events follow some order in time then there is a significant difference between

P (2nd Event and 1st Event)

P (2nd Event | 1st Event) =
P (1st Event)

and
P (2nd Event and 1st Event)
P (1st Event | 2nd Event) =
P (2nd Event)

Mathematically the second thing is just conditional probability, but philosophically it is a reversal of cause and effect
which angers people and gets a special result called Bayes Theorem.

What’s It?

Bayes Theorem.

P (2nd Event and 1st Event)

P (1st Event | 2nd Event) = =
P (2nd Event)
P (1st Event)P (2nd Event | 1st Event)
P (2nd Event)
Or more formally, for any event A and mutually exclusive partition (separation) of the sample space into parts
S1 , S2 , S3 , ..., Sk
P (Sk )P (A|Sk ) P (Sk )P (A|Sk )
P (Sk |A) = = Pk
P (A) i=1 P (Sk )P (Sk |A)

Don’t actually ever use the formula for this class. Just follow the examples using trees coming up. But some people
like formulas so there they are.
Example 1

Example 1

Suppose there are three attendance categories for this class: Always attends, sometimes attends and never attends.
and additionally you can either pass or fail the class. A “normal” probability question asks “what is the probability I
fail the class given I never attend?” A Bayes Theorem question on the other hand reverses the cause and effect chain
by asking “What is the probability a student never attended class given they failed the class?”. Bayes questions are
best done using a tree as we did before.

Now we need to add probabilities; from past semesters, suppose that 10% of students never attended and 70% of them
failed, 30% of students sometimes attended and 40% of them failed, and 15% of those who always attended failed. We
can fill out the tree as
Effect to Cause, Bayes Theorem

What would be the probability a student passes the class?

P (P ass) = 0.1(0.3) + 0.3(0.6) + 0.6(0.85)

P (P ass) = 0.72

What would be the probability a student fails the class?

P (F ail) = 0.1(0.7) + 0.3(0.4) + 0.6(0.15)

P (F ail) = 0.28

Effect to Cause, Bayes Theorem

Now if your friend tells you that they failed this class last semester you can calculate the probability that they never
showed up to class (you could probably ask them too). All you do is add up the paths that end on failure to get the
denominator (P (f ailed)) and the numerator is then the path that you want (“Never → Fail” in this case ).

P (N ever Attended |F ail) =

0.1(0.7)
0.1(0.7) + 0.3(0.4) + 0.6(0.15)
0.07
= P (N ever Attended |F ail) = 0.25
0.28
Example 2

Example 2

While waiting for a train during rush hour in NYC in August the car that you are standing in front of can be either
Packed, Normal or empty with 82% probability, 15% probability and 3% probability respectively. There is a 10%
chance of a terrible smell and 90% of no problem in packed cars; A 25% chance of terrible smell, 40% chance of broken
AC in normal cars and a 50-50 chance of terrible smell or broken AC in empty cars. Based on this information, answer
the following questions.

1. What is the probability that a car pulls up in front of you with a broken AC?
2. Suppose the car in front of you is empty, what is the probability it has a terrible smell inside?
3. Imagine you are blindfolded, kidnapped and taken on the subway. If the car you are on smells terrible, what is
the probability that it is empty?

1.
P (AC) = 0.82(0) + 0.15(0.4) + 0.03(0.5)
0.075

2.
P (Smell | Empty) = 0.5

3.
P (Empty | Smell) =
0.03(0.5)
0.82(0.1) + 0.15(0.25) + 0.03(0.5)
0.015
P (Empty | Smell) = = 0.112
0.1345