Math 32 Lecture 6: Bayes Rule (Concepts) Spring 2021

Bayes’ Rule combines the previous ideas as

P (A and B) P (B|A) · P (A)
P (A|B) = =
P (B) P (B|A) · P (A) + P (B|Ac ) · P (Ac )

1
Math 32 Lecture 6: Bayes Rule (Concepts) Spring 2021

1 Example 3: Spam Filtering

In 2002, Paul Graham used Bayes’ Rule as part of his algorithms to greatly
decrease false positive rates of unwanted e-mails (“spam”).1 Let H c be the
event that an e-mail is ”spam”. Let W be the event that an e-mail contains
a trigger word such as “watches”. Suppose that
• the probability that an e-mail contains that word given that it is spam
is 17%
• the probability that an e-mail contains that word given that it is not
spam is 9%
• the probability that a randomly selected e-mail message is spam is 80%
Find the probability that an e-mail message is spam, given that the trigger
word appears.

Translation to math notation:

• P (W |H c ) = 0.17
• P (W |H) = 0.09
• P (H c ) = 0.80
We want to compute P (H c |W ), so let us apply Bayes’ Rule,

c P (W |H c ) · P (H c )
P (H |W ) =
P (W |H c ) · P (H c ) + P (W |H) · P (H)
(0.17)(0.80)
=
(0.17)(0.80) + (0.09)(0.20)

≈ 0.8831
The probability that an e-mail message is spam given that the trigger word
appears is about 88.31 percent.

1 Source: http://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering

2
Math 32 Lecture 6: Bayes Rule (Concepts) Spring 2021

2 Example 4: DUI Checkpoint

A breath analyzer, used by the police to test whether drivers exceed the legal
limit set for the blood alcohol percentage while driving, is known to satisfy
P (A|B) = P (Ac |B c ) = x
where A is the event “breath analyzer indicates that legal limit is exceeded”
and B “driver’s blood alcohol percentage exceeds legal limit.” On Saturday
nights, about 5% of the drivers are known to exceed the limit.
1. Describe in words the meaning of P (B c |A)
2. Determine P (B c |A) if x = 0.95
3. How big should x be so that P (B|A) = 0.9?

2.1 Interpretation

P (B c |A) is the probability that the driver is sober given that the breath
analyzer claimed the driver was drunk (i.e. the probability of a false positive).

3
Math 32 Lecture 6: Bayes Rule (Concepts) Spring 2021

2.2 Example 4.1

Translation to math notation:

• Sensitivity: P (A|B) = 0.95
• Specificity: P (Ac |B c ) = 0.95
• Prior probability: P (B) = 0.05
Applying Bayes’ Rule,

c P (A|B c ) · P (B c )
P (B |A) =
P (A|B c ) · P (B c ) + P (A|B) · P (B)
We need to compute the complements,
• P (B c ) = 1 − P (B)
• P (A|B c ) = 1 − P (Ac |B c )

c P (A|B c ) · P (B c )
P (B |A) =
P (A|B c ) · P (B c ) + P (A|B) · P (B)
[1 − P (Ac |B c )] · [1 − P (B)]
=
[1 − P (Ac |B c )] · [1 − P (B)] + P (A|B) · P (B)
(1 − 0.95) · (1 − 0.05)
=
(1 − 0.95) · (1 − 0.05) + (0.95)(0.05)

= 0.5
The probability that the driver is sober given that the breath analyzer claimed
the driver was drunk (i.e. the probability of a false positive) is 50 percent!

4
Math 32 Lecture 6: Bayes Rule (Concepts) Spring 2021

2.3 Sensitivity and Specificity

What should the sensitivity and specificity of the blood alcohol test be so
that that the probability the driver is drunk given a positive test is 90 percent
(P (B|A) = 0.9)?

P (A|B) = P (Ac |B c ) = x
Using Bayes’ Rule,

P (A|B) · P (B)
P (B|A) =
P (A|B) · P (B) + P (A|B c ) · P (B c )
P (A|B) · P (B)
P (B|A) =
P (A|B) · P (B) + [1 − P (Ac |B c )] · [1 − P (B)])
0.05x
0.9 ≤
0.05x + (1 − 0.05)(1 − x)
The sensitivity and specificity each need to exceed x ≈ 0.9942 so that
P (B|A) ≥ 0.9.

5
Math 32 Lecture 6: Bayes Rule (Concepts) Spring 2021

3 Example 5: Disease Outbreak

Suppose that at UC Merced, there is a two percent chance that a freshman

has herpes at the end of the school year. Let H be the event of having the
virus, while C represents the event that the freshman is from the Cathedral
dorm. Among the herpes carriers, the probably of being a Cathedral resident
is 32%. Among those free of disease, the probably of being a Cathedral
resident is 13%. What is the probability that a freshman has herpes, given
that you know that he or she lived in the Cathedral dorm?

Translation to math notation,

• P (H) = 0.02
• P (C|H) = 0.32
• P (C|H c ) = 0.13
We want to compute P (H|C), so let us apply Bayes’ Rule,

P (C|H) · P (H)
P (H|C) =
P (C|H) · P (H) + P (C|H c ) · P (H c )
(0.32)(0.02)
=
(0.32)(0.02) + (0.13)(0.98)

≈ 0.0478
The probability that a freshman has herpes, given that you know that he or
she lived in the Cathedral dorm is about 4.78%.

6
Math 32 Lecture 6: Bayes Rule (Concepts) Spring 2021

4 Bayes’ Rule Extended

If we are conditioning B on an event A, where the latter can be partitioned

into several subsets,

A = {A1 , A2 , ..., Aj }
then the total probability is

P (B) = P (B|A1 ) · P (A1 ) + P (B|A2 ) · P (A2 ) + ... + P (B|An ) · P (An )

and Bayes Rule for computing the probability of Ai given B becomes

P (B|Ai ) · P (Ai )
P (Ai |B) =
P (B|A1 ) · P (A1 ) + P (B|A2 ) · P (A2 ) + ... + P (B|An ) · P (An )