A Simple Approach to Bayes

Theorem and its Application

Prepared By: Prof Farhan Hameed

Department of Statistics
Govt Islamia Graduate College
Civil Lines Lahore

1
 Bayes Theorem
If the events 𝐴1 , 𝐴2 … … … . . 𝐴𝑘 form a partition of sample space S, that is the events 𝐴𝑖 are
mutually exclusive and their union is S, and if B is any other event of S such that it can occur if
one of the 𝐴𝑖 occurs , then for any 𝑖.

𝑷(𝑨𝒊 )𝑷(𝑩⁄𝑨𝒊 )
𝑷(𝑨𝒊 ⁄𝑩) =
∑𝒌𝒊=𝟏 𝑷(𝑨𝒊 )𝑷(𝑩⁄𝑨𝒊 )
For 𝑖 = 1,2, … … . 𝑘.
This result is known as Bayes theorem. The original probabilities 𝑃(𝐴𝑖 ) are known as Priori
Probabilities and the conditional probabilities 𝑃(𝐴𝑖 ⁄𝐵 ) are called the Posteriori or Inverse
probabilities (because the probabilities are revised after additional information is obtained).

Example 1: In a bolt factory, machines A,B and C manufacture 25,35 and 40 percent of the total
output respectively. Of their output 5,4, and 2 percent respectively are defective bolts. A bolt is
selected at random and found to be defective. What is the probability that the bolt came from
machine A.

Solution.:

The prior probabilities (before the information that the bolt is defective) are

P(A)= 0.25, P(B)=0.35, P(C) =0.40

Let E be the event that bolt is defective (D). The conditional probabilities are

P(E/A)=0.5, P(E/B)=0.04, P(E/C)=0.02

The outcomes are as under in the tree diagram:-

2
Now, P(A/E) is a posteriori probability that the selected bolt came from machine A, therefore, by
Bayes’ theorem

𝑷(𝑨)𝐏(𝐄 ∕ 𝐀)
𝑷(𝑨⁄𝐄) =
𝑷(𝑨)𝐏(𝐄⁄𝐀) + 𝑷(𝑩)𝐏(𝐄⁄𝐁) + 𝑷(𝑪)𝐏(𝐄⁄𝐂)

(𝟎. 𝟐𝟓)(𝟎. 𝟎𝟓) 𝟎. 𝟎𝟏𝟐𝟓

𝑷(𝑨⁄𝐄) = = = 𝟎. 𝟑𝟔𝟐
(𝟎. 𝟐𝟓)(𝟎. 𝟎𝟓) + (𝟎. 𝟑𝟓)(𝟎. 𝟎𝟒) + (𝟎. 𝟒𝟎)(𝟎. 𝟎𝟐) 𝟎. 𝟎𝟑𝟒𝟓

3
Example 2: An urn contains four balls which are known to be either (i) all white or (ii) two white
and two black. A ball is drawn at random and found to be white. (a)What is the probability that all
the balls are white. (b) What is the probability that two white and two black.

Solution:

Let A1 be the hypothesis that all the balls are white (4 white balls).

Let A2 be the hypothesis that two white and two black balls.

The probabilities of selection of hypothesis will be:-

1
𝑃(𝐴1 ) = 𝑃(𝐴2 ) = = 0.5
2

Again, let B is that event that the selected ball is white. The conditional probabilities are;-

4 2
𝐶1 4 𝐶1 2 1
𝑃(𝐵⁄𝐴1 ) = 4 = = 1 , 𝑃(𝐵⁄𝐴2 ) = 4 = = = 0.5
𝐶1 4 𝐶1 4 2

(a) Now the posterior probability will be

𝑃(𝐴1 )P(B ∕ A1 )
𝑃(𝐴1 ⁄B) =
𝑃(𝐴1 )P(B⁄A1 ) + 𝑃(𝐴2 )P(B ∕ A2 )

(𝟎. 𝟓)(𝟏) 𝟎. 𝟓 𝟐
𝑷(𝑨𝟏 ⁄𝐄) = = =
(𝟎. 𝟓)(𝟏) + (𝟎. 𝟓)(𝟎. 𝟓) 𝟎. 𝟕𝟓 𝟑

(b) Now the posterior probability will be

𝑃(𝐴2 )P(B ∕ A2 )
𝑃(𝐴2 ⁄B) =
𝑃(𝐴1 )P(B⁄A1 ) + 𝑃(𝐴2 )P(B ∕ A2 )

(𝟎. 𝟓)(𝟎. 𝟓) 𝟎. 𝟐𝟓 𝟏
𝑷(𝑨𝟐 ⁄𝐄) = = =
(𝟎. 𝟓)(𝟏) + (𝟎. 𝟓)(𝟎. 𝟓) 𝟎. 𝟕𝟓 𝟑

4
Example 3: Three urns f same appearance have the following proportions of white and black balls

Urn A: 1 White, 2 black balls

Urn B: 2 White, 1 black balls

Urn C: 2 White, 2 black balls

One of the Urn is selected at random and a ball is drawn from it. It turns out to be white. What is
the probability it came from urn C.

Solution:

The probabilities of selection of Urns

1
𝑃(𝐴) = 𝑃(𝐵) = 𝑃(𝐶) =
3

Let E be the event that the ball is white, the conditional probabilities will be;-

1 2 2
𝐶1 1 𝐶1 2 𝐶1 2 1
𝑃(𝐸 ⁄𝐴) = 3 = , 𝑃(𝐸 ⁄𝐵) = 3 = , 𝑃(𝐸 ⁄𝐶 ) = 4 = =
𝐶1 3 𝐶1 3 𝐶1 4 2

Now the posterior probability will be

𝑃(𝐶)P(E⁄C)
𝑃(𝐶 ⁄E) =
𝑃(𝐴)P(E⁄A) + 𝑃(𝐵)P(E⁄B) + 𝑃(𝐶)P(E⁄C)

𝟏 𝟏 𝟏 𝟏
( )( ) 𝟏 𝟏𝟖 𝟏
𝑷(𝑪 ⁄𝐄) = 𝟑 𝟐 = 𝟔 = 𝟔 = × =
𝟏 𝟏 𝟏 𝟐 𝟏 𝟏 𝟏 𝟐 𝟏 𝟗 𝟔 𝟗 𝟑
(𝟑) (𝟑) + (𝟑) (𝟑) + (𝟑)(𝟐) 𝟗 + 𝟗 + 𝟔 𝟏𝟖

5
Example 4: There are three coins identical in appearance, one of which is ideal and the other two
1 2
biased with probabilities 3 abnd 3 respectively for a head. One coin is taken at random and tossed

twice. If a head appears both the times, what is the probability that the ideal coin was chosen.

Solution:-

Let A1 denotes event that ideal coin is selected

A2 denotes event that 2nd coin is selected

A3 denotes event that 2nd coin is selected

1
𝑃(𝐴1 ) = 𝑃(𝐴2 ) = 𝑃(𝐴3 ) =
3

Let E is the event that both times head appears when coin is tossed twice. The conditional
probabilities are

1 1 1 1 1 1 2 2 4
𝑃(𝐸 ⁄𝐴1 ) = . = , 𝑃(𝐸 ⁄𝐴2 ) = . = , 𝑃(𝐸 ⁄𝐴3 ) = . =
2 2 4 3 3 9 3 3 9

Now, the probability that ideal coin is selected

𝑃(𝐴1 )𝑃(𝐸 ⁄𝐴1 )

𝑃(𝐴1 ⁄𝐸 ) =
𝑃(𝐴1 )𝑃(𝐸 ⁄𝐴1 ) + 𝑃(𝐴2 )𝑃(𝐸 ⁄𝐴2 ) + 𝑃(𝐴3 )𝑃(𝐸 ⁄𝐴3 )

𝟏 𝟏 𝟏
(𝟑) (𝟒)
𝑷(𝑨𝟏 ⁄𝑬) = = 𝟏𝟐 = 𝟏 × 𝟏𝟎𝟖 = 𝟗
𝟏 𝟏 𝟏 𝟏 𝟏 𝟒 𝟐𝟗 𝟏𝟐 𝟐𝟗 𝟐𝟗
(𝟑) (𝟒) + (𝟑) (𝟗) + (𝟑) (𝟗) 𝟏𝟎𝟖

6
Example 5: In a Certain college, 4% of the men and 1% of the women are taller than 6 feet.
Furthermore, 60% of the students are women. Now, if a student is selected at random and is taller
than 6 feet , what is the probability that the student is a woman.

Solution:-

Let W denotes event woman is sleeted and M denotes the event man selected

P(W)=60%=0.6, P(M)=40%=0.4

Let E Event that person selected is taller than 6 feet, the conditional probabilities are:-

𝑃(𝐸 ⁄𝑊 ) = 1% = 0.01 , 𝑃(𝐸 ⁄𝑀) = 4% = 0.04

The probability that the person selected taller than six feet is woman

𝑃(𝑊)𝑃(𝐸 ⁄𝑊 )
𝑃(𝑊 ⁄𝐸 ) =
𝑃(𝑊)𝑃(𝐸 ⁄𝑊 ) + 𝑃(𝑀)𝑃(𝐸 ⁄𝑀)

(𝟎. 𝟔)(𝟎. 𝟎𝟏) 𝟎. 𝟎𝟎𝟔 𝟑

𝑷(𝑾⁄𝑬) = = =
(𝟎. 𝟔)(𝟎. 𝟎𝟏) + (𝟎. 𝟒)(𝟎. 𝟎𝟒) 𝟎. 𝟎𝟐𝟐 𝟏𝟏

Example 6: Three cooks A, B and C bake a special kind of cake and with the respective
probabilities 0.02, 003 and 0.05 it fails to rise. In the restaurant where they work, A bakes 50
percent of these cakes, B 30 percentage, and C 20 percent. What proportion of “failures” cause by
A.

Solution:- P(A)=0.5, P(B)=0.3, P(C) =0.2

Let E event the cake fails to rise. The conditional probabilities are (Failure of cake):-

𝑃(𝐸 ⁄𝐴) = 0.02 , 𝑃(𝐸 ⁄𝐵 ) = 0.03 , 𝑃(𝐸 ⁄𝐶 ) = 0.05

The probability that failure cause by cook A:-

𝑃(𝐴)P(E⁄A)
𝑃(𝐴⁄E) =
𝑃(𝐴)P(E⁄A) + 𝑃(𝐵)P(E⁄B) + 𝑃(𝐶)P(E⁄C)

7
 (𝟎. 𝟓)(𝟎. 𝟎𝟐) 𝟎. 𝟎𝟏𝟎 𝟏𝟎
𝑷(𝑨⁄𝑬) = = =
(𝟎. 𝟓)(𝟎. 𝟎𝟐) + (𝟎. 𝟑)(𝟎. 𝟎𝟑) + (𝟎. 𝟐)(𝟎. 𝟎𝟓) 𝟎. 𝟎𝟐𝟗 𝟐𝟗

Example 7: The stock of a warehouse consists of boxes of high, medium and low quality
lightbulbs in the respective proportion 1:2:2. The probabilities of bulbs of three types being
unsatisfactory are 0.0,0.1 and 0.2 respectively. If a box is chosen at random and two bulbs
in it are tested and found to be satisfactory, what is the probability that it contains (i) of
high (ii) Medium (iii) Low quality.

Solution:-

Let H denotes High quality lightbulbs

M denotes Medium quality lightbulbs

L denotes Low quality lightbulbs

The probabilities with proportions 1:2:2 are:-

1 2 2
𝑃 (𝐻 ) = = 0.2 , 𝑃(𝑀) = = 0.4 , 𝑃(𝐻 ) = = 0.4
5 5 5

Let S be the event that the bulb is satisfactory

𝑃(𝑆⁄𝐻 ) = (1 − 0.00)2 = 1 , 𝑃(𝑆⁄𝑀) = (1 − 0.1)2 = 0.81 , 𝑃(𝑆⁄𝐿) = (1 − 0.2)2 = 0.64

(a) The probability that bulbs are of High quality

𝑃(𝐻)P(S⁄H)
𝑃(𝐻 ⁄S) =
𝑃(𝐻)P(S⁄H) + 𝑃(𝑀)P(S⁄M) + 𝑃(𝐿)P(S⁄L)

(𝟎. 𝟐)(𝟏. 𝟎) 𝟎. 𝟐
𝑷 (𝑨 ⁄𝑬) = = = 𝟎. 𝟐𝟓𝟔
(𝟎. 𝟐)(𝟏. 𝟎) + (𝟎. 𝟒)(𝟎. 𝟖𝟏) + (𝟎. 𝟒)(𝟎. 𝟔𝟒) 𝟎. 𝟕𝟖

(b) The probability that bulbs are of Medium quality

𝑃(𝑀)P(S⁄M)
𝑃(𝑀⁄S) =
𝑃(𝐻)P(S⁄H) + 𝑃(𝑀)P(S⁄M) + 𝑃(𝐿)P(S⁄L)

8
 (𝟎. 𝟒)(𝟎. 𝟖𝟏) 𝟎. 𝟑𝟕𝟖
𝑷(𝑴⁄𝑺) = = = 𝟎. 𝟒𝟏𝟔
(𝟎. 𝟐)(𝟏. 𝟎) + (𝟎. 𝟒)(𝟎. 𝟖𝟏) + (𝟎. 𝟒)(𝟎. 𝟔𝟒) 𝟎. 𝟕𝟖

(c) The probability that bulbs are of Low quality

𝑃(𝐿)P(S⁄L)
𝑃(𝐿⁄S) =
𝑃(𝐻)P(S⁄H) + 𝑃(𝑀)P(S⁄M) + 𝑃(𝐿)P(S⁄L)

(𝟎. 𝟒)(𝟎. 𝟔𝟒) 𝟎. 𝟐𝟓𝟔

𝑷(𝑴⁄𝑺) = = = 𝟎. 𝟑𝟐𝟖
(𝟎. 𝟐)(𝟏. 𝟎) + (𝟎. 𝟒)(𝟎. 𝟖𝟏) + (𝟎. 𝟒)(𝟎. 𝟔𝟒) 𝟎. 𝟕𝟖

Example 8: A patient is thought to have one of the three diseases A1, A2 and A3 whose
1 1 1
probabilities under the given conditions are 2 , 6 and 3 respectively. A test is carried out to help

the diagnosis and it yields positive result with a probability of 0.1 for disease A1 , 0.2 for disease
A2 and 0.9 for disease A3. The test is conducted 5 times and the results are positive 4 times and
negative once. What is the probability of each disease after testing.

Solution:-

1 1 1
𝑃(𝐴1 ) = , 𝑃(𝐴2 ) = , 𝑃(𝐴3 ) =
2 6 3

The conditional probability for 𝐴1 will be obtained using binomial distribution with n=5 , p=0.1,
q=0.9 for 4 positive results out of 5.

5
𝑃(𝐸 ⁄𝐴1 ) = ( ) (0.1)4 (0.9) = 0.00045
4

The conditional probability for 𝐴2 will be obtained using binomial distribution with n=5 , p=0.2,
q=0.8 for 4 positive results out of 5.

5
𝑃(𝐸 ⁄𝐴2 ) = ( ) (0.2)4 (0.8) = 0.0064
4

9
The conditional probability for 𝐴3 will be obtained using binomial distribution with n=5 , p=0.9,
q=0.1 for 4 positive results out of 5.

5
𝑃(𝐸 ⁄𝐴3 ) = ( ) (0.9)4 (0.1) = 0.32805
4

The desired probability for 𝐴1

𝑃(𝐴1 )P(E⁄A1 )
𝑃(𝐴1 ⁄E) =
𝑃(𝐴1 )P(E⁄A1 ) + 𝑃(𝐴2 )P(E⁄A2 ) + 𝑃(𝐴3 )P(E⁄A3 )

𝟏
( ) (𝟎. 𝟎𝟎𝟎𝟒𝟓) 𝟎. 𝟎𝟎𝟎𝟐𝟑𝟓
𝑷(𝑨𝟏 ⁄𝑬) = 𝟐 = = 𝟎. 𝟎𝟎𝟐𝟎
𝟏 𝟏 𝟏 𝟎. 𝟏𝟏𝟎𝟔𝟒𝟐
( ) (𝟎. 𝟎𝟎𝟎𝟒𝟓) + ( ) (𝟎. 𝟎𝟎𝟔𝟒𝟎) + ( ) (𝟎. 𝟑𝟐𝟖𝟎𝟓)
𝟐 𝟔 𝟑

The desired probability for 𝐴2

𝑃(𝐴2 )P(E⁄A2 )
𝑃(𝐴2 ⁄E) =
𝑃(𝐴1 )P(E⁄A1 ) + 𝑃(𝐴2 )P(E⁄A2 ) + 𝑃(𝐴3 )P(E⁄A3 )

𝟏
(𝟔) (𝟎. 𝟎𝟎𝟔𝟒𝟎)
𝑷(𝑨𝟏 ⁄𝑬) = = 𝟎. 𝟎𝟎𝟗𝟔
𝟏 𝟏 𝟏
(𝟐) (𝟎. 𝟎𝟎𝟎𝟒𝟓) + (𝟔) (𝟎. 𝟎𝟎𝟔𝟒𝟎) + (𝟑) (𝟎. 𝟑𝟐𝟖𝟎𝟓)

The desired probability for 𝐴3

𝑃(𝐴3 )P(E⁄A3 )
𝑃(𝐴3 ⁄E) =
𝑃(𝐴1 )P(E⁄A1 ) + 𝑃(𝐴2 )P(E⁄A2 ) + 𝑃(𝐴3 )P(E⁄A3 )

𝟏
(𝟑) (𝟎. 𝟑𝟐𝟖𝟎𝟓)
𝑷(𝑨𝟑 ⁄𝑬) = = 𝟎. 𝟗𝟖𝟖𝟑
𝟏 𝟏 𝟏
(𝟐) (𝟎. 𝟎𝟎𝟎𝟒𝟓) + (𝟔) (𝟎. 𝟎𝟎𝟔𝟒𝟎) + (𝟑) (𝟎. 𝟑𝟐𝟖𝟎𝟓)

10