Professional Documents
Culture Documents
IDS21 Bayes Theorem
IDS21 Bayes Theorem
IDS21 Bayes Theorem
Is it 90%?
Bayes theorem (or rule)
• Allows to combine, update
and invert conditional
probabilities.
• It has a wide range of
applications, including the
updating of beliefs.
• Philosophy: New evidence
(data/observations) don’t
speak for themselves. They
update prior beliefs.
• It provides an optimal way
to integrate current and
prior information.
Thomas Bayes (1701-1761)
Potential for confusion
• Simple probabilities are easy to invert.
• Particularly if the probability space is binary and the
events are mutually exclusive.
• You don’t need Bayes’ theorem for that.
• But this is not the case for conditional probabilities.
~A=1-A
A B
Revisiting conditional probability
A A B
U
B
P(-) = P(+) =
P(A) =
P(+) =
P(-) =
p(+∩ A) p(+∩ A)
p(+ | A) = p(A | +) =
p(A) p(+)
p(−∩ A) p(−∩ A)
p(− | A) = p(A | −) =
p(A) p(−)
Deriving Bayes rule
p(A ∩ B) p(A ∩ B)
p(A | B) = A A B B
U p(B | A) =
p(B) p(A)
p(A)∗ p(B | A)
p(A | B) =
p(B)
Some necessary terminology
Likelihood
p(A)∗ p(B | A)
p(A | B) =
p(B)
Posterior
Priors
What can we do with this?
• We know p(K|S) = 1
• We know p(S) = 0.01
• We know p(K) = 0.1
• p(S|K) = 0.01/0.1 = 0.1
• Reasonable to filter by these keywords?
The specific value of the priors matters:
A ~B A B B ~A
U U U
S (Sampling space)
• p(B)?
• p(B) = p(A B)Up(B ~A)
U U
p(A)∗ p(B | A)
p(A | B) =
p(B | A)∗ p(A) + p(B |¬A)∗ p(¬A)
What can we do with the explicit form?
This form is often used in diagnostics:
The prevalence of breast cancer in the population is
1%. A mammograph detects present breast cancer
99% of the time. If no breast cancer is present, the
false-positive rate is 10%.
After a test, the mammograph of a person is positive.
What is the probability that this person has breast
cancer?
This kind of inference has been well explored, e.g.
Hoffrage et al. (2000), Science.
Using the explicit form of Bayes theorem:
p(A) ∗ p(B | A)
p(A | B) =
p(A) ∗ p(B | A) + p(¬A) ∗ p(B |¬A)