Chapter 4 Conditional probability and independence

CHAPTER 4

4. CONDITIONAL PROBABILTY AND INDEPENDENCE

Aims and objectives

The overall aspiration of this unit is to get readers accustomed to the notion of
conditional probability, multiplication theorem Baye’s theorem, total probability
theorem and independent event. Evaluation of different conditional probability and
independence is the core target of the unit.

At the end of the unit students will be able to:

 define conditional probability

 solve problems involving calculation of conditional probabilities
 state Bayes’ theorem and apply it in solving different problems
 discuss the meaning of independent events
 identify the condition(s) required for two events to be independent

Introduction

It is to be recalled that up to this moment we were talking about events that are
independent. Nevertheless, it seems to be somewhat ambitious to assume that events
are always independent. In the real world variables are known to be interrelated or
dependent to one another, and so do events. Accordingly, in most cases it seems rare
to find probabilities of events that do not involve conditional probabilities. Most
business problems call for Bayes’ theorem. A case in point can be the field of
operations research. The concept of Bayes’ theorem can be seen to be the reverse of
the idea of conditional probabilities.

4.1. CONDITIONAL PROBABILITY

Conditional probability refers to those probabilities that arise in the case of dependent
events. To introduce this important concept let us consider the following example.

Example: A box of 50 candles consists of 30 defective and 20 non-defective candles.

Suppose that we choose two candles from this box one after the other.

a) with replacement
b) without replacement

.“The more you read, the more you understand” 1

Chapter 4 Conditional probability and independence

Solution:

Define two events A and B as follows;

A=the first candle is defective

B=the second candle is defective
a) If we are choosing with replacement, P(A) =P(B)= 30/50 =3/5 because each time
the selected candle will be replaced back and hence the composition remains the
same. However, if we are choosing without replacement, the results are not quite
immediate.
b) P(B)= P(B/A)=29/49. Such types of probability are called conditional probabilities.

Example 2: Suppose a box contains 10 good items and 6 defective items. Choose 2
items at a random

a) With replacement
b) Without replacement

Exercise!

Definition (Conditional Probability): Let A and B be two events. The conditional

probability of B given that A has occurred is denoted by P (B/A) and is defined as

P( A  B)
P( B / A)  ; where P(A)  0
P( A)

P( A  B)
Similarly P( A / B)  ; where P(B)  0
P( B)

Remark: P(A/B) for fixed B satisfies the axioms of probability.

i. 0 P(A/B) 1
ii. P(S/B) =1; where S denotes the Sample Space.
n n
iii. P( A i /B )   P(Ai /B) if A i  Aj   for i  j
i 1 i 1

Example: We roll a pair of dice once and are given that the two faces are not the
same. Compute the probability that the sum is 7
Solution:
Let A= the two numbers that occur are different

B= the sum is 7

.“The more you read, the more you understand” 2

Chapter 4 Conditional probability and independence

1 2 3 4 5 6

1 (1,1) (2,1) (3,1) (4,1) (5,1) (6,1)

2 (1,2) (2,2) (3,2) (4,2) (5,2) (6,2)

3 (1,3) (2,3) (3,3) (4,3) (5,3) (6,3)

4 (1,4) (2,4) (3,4) (4,4) (5,4) (6,4)

5 (1,5) (2,5) (3,5) (4,5) (5,5) (6,5)

6 (1,6) (2,6) (3,6) (4,6) (5,6) (6,6)

From the table we can see that: P(A) = 30/36 =5/6 ,P(B)= 6/36 =1/6 , P(AB) =1/6

P(B/A) = P(AB)/P(A)= (1/6) /(5/6) = 1/5

Example 2: To study the proportion of smokers by sex from a population, a random

sample of 200 persons was taken. The following table shows the result

Non-smoker (A/) Smoker (A) Total

Female (B) 64 16 80

Male (B/) 42 78 120

Total 106 94 200

Find

a) The probability of getting a non-smoker given that the person selected is a

female.
b) The probability of getting a non-smoker given that the person selected is a
male.
c) The probability of getting a female given that the person selected is a smoker.
Solution:

.“The more you read, the more you understand” 3

Chapter 4 Conditional probability and independence

Let A: A selected person is smoker B: The selected person is female.

n( A  B)
P( A  B) n( S ) n( A  B) 64 4
a) P( A / B)     
P( B) n( B ) n( B ) 80 5
n( S )
n( A  B )
P( A  B ) n( S ) n( A  B ) 42 7
b) P( A / B )     
P( B ) n( B ) n( B ) 120 20
n( S )
n( B  A)
P( B  A) n( S ) n( B  A) 16 32
c) P( B / A)     
P( A) n ( A) n( A) 94 47
n( S )

4.2. MULTIPLICATION RULE OF PROBABILITY

Recall that

P( A  B)
P( A / B) 
P( B)
 P( A  B)  P( A / B) P( B)  (1)
P( B  A)
Similarly, P( B / A) 
P( A)
 P( B  A)  P( B / A) P( A)  (2)

Equations (1) and (2) are sometimes known as multiplication rule of probability. They
are used to find the probability of simultaneous occurrence of events.

Example:
Suppose that our sample space is the population of adults in a small town who have
completed the requirements for a college degree. They are categorized according to
sex and employment status as follows.

Employed(E) Unemployed Total

Male(M) 30 40 70
Female 20 70 90
Total 50 110 160

.“The more you read, the more you understand” 4

Chapter 4 Conditional probability and independence

A person is selected at random. What is the probability that the selected person is
a) a man given that he is employed
b) unemployed given that she is female
c) a male or a female
d) unemployed male
e) employed

Solution:

Let E: The person is employed  E/: The person is unemployed

M: The person is male  M/: The person is female

30
P( M  E ) 30 3
a) P( M / E )   160  
P( E ) 50 50 5
160

70
P( M   E ) 70 7
b) P( E  / M )   160  
P( M ) 90 90 9
160

c) P(M  M )  P(S )  1

n( E   M ) 40 1
d) P( E   M )   
n( S ) 160 4

n( E ) 50 5
e) P( E )   
n( S ) 160 16

4.2.2 BAYES’ THEOREM

Definition (Partition of Sample Space): We say that the events B1, B2, ….Bk represents
partition of the sample space S if

a) Bi  Bj =  i.e. if the events are mutually exclusive

k
b) B i S
i 1

c) P(Bi) > 0 i
Example: For tossing a die B1={1, 2} B2= {3, 4, 5} B3= {6} would represent the partition
of the sample space, while C1={1, 2, 3, 4} and C2= {4, 5, 6} would not because they have
intersection point.

.“The more you read, the more you understand” 5

Chapter 4 Conditional probability and independence

Let A be some event associated with S and let B1, B2, … Bk be partitions of S. Hence we
can write

A = (A  B1)  (A B2)  … (A  Bk)

Of course some of A  Bi may be empty. Note that since Bis are partitions and have
no intersection (mutually exclusive events) all sets (A  B1), (A B2), …,(A  Bk) are
pair wise mutually exclusive.

Hence P(A) = P(A  B1) + P(A B2) + … + P(A  Bk)

But P(A  Bi), each for each and every i, may be expressed as P(A/Bi) P(Bi) and we
obtain what is called the theorem of total probability.

Theorem (Total Probability Theorem)

If we have B1, B2, ….Bk to be partitions of S then for any event A

P(A) = P(A/B1) P(B1) + P(A/B2) P(B2) + …. + P(A/Bk) P(Bk)

k
=  P(A/B ) P(B )
i 1
i i

Example: Three machines A, B and C produce 50%, 30% and 20% of the total output
respectively. The percentages of defective output of these machines are 3%, 4% and
5% respectively. If an item is selected at random, what is the probability that the item
is defective?

Solution:

Let A: The selected item is defective

B1: Outputs produced by A P(B1) = 0.5 P(A/B1) = 0.03

B2: Outputs produced by B P(B2) = 0.3 P(A/B2) = 0.04

B3: Outputs produced by C P(B3) = 0.2 P(A/B3) = 0.05

P(A) = P(A/B1) P(B1) + P(A/B2) P(B2) + P(A/B3) P(B3)

= (0.5)(0.03)+(0.3)(0.04)+(0.2)(0.05)
= 0.037

.“The more you read, the more you understand” 6

Chapter 4 Conditional probability and independence

Theorem (Bayes’ Theorem)

Let B1, B2, …., Bk be a partition of the sample space S and let A be an event associated
P( A / Bi ) P( Bi )
with S. Then P( Bi / A)  k

 P( A / B ) P( B )
i 1
i i

Proof:
P( Bi  A) P( A / Bi ) P( Bi )
P( Bi / A)  
P( A) P( A)
k
But then from total probability theorem we have P(A)   P( A / Bi ) P( Bi ), thus
i 1

P( A / Bi ) P( Bi )
P( Bi / A)  k

 P( A / B ) P( B )
i 1
i i

Example: Consider the previous example. An item was selected and is found to be
defective. What is the probability that it is from B1?

Solution:

P(B1) = 0.5 P(A/B1) = 0.03

P(B2) = 0.3 P(A/B2) = 0.04

P(B3) = 0.2 P(A/B3) = 0.05

P( A / B1 ) P( B1 ) P( A / B1 ) P( B1 )
P( B1 / A)  
3
P( A / B1 ) P( B1 )  P( A / B2 ) P( B2 )  P( A / B3 ) P( B3 )
 P( A / B ) P( B )
i 1
i i

(0.03)(0.5) 15
 
0.037 37

4.2.INDEPENDENT EVENTS

Recall that for mutually exclusive events A  B =   P(A  B) = 0. On the other hand,
P( A  B) P( A)
if A B, P( B / A)   1
P( A) P( A)

Knowing the occurrence of one event gives us clue as to the probability of the
occurrence of the others. However, there are situations in which the occurrence or
non-occurrence of an event has no bearing on the probability of occurrence of the
other.

Example: Toss two fair dice and observe the faces up

.“The more you read, the more you understand” 7

Chapter 4 Conditional probability and independence

Solution

 1,1 1,2 1,3 1,4 1,5 1,6

2,1 2,2 2,3 2,4 2,5 2,6

3,1 3,2 3,3 3,4 3,5 3,6
S 
4,1 4,2 4,3 4,4 4,5 4,6
5,1 5,2 5,3 5,4 5,5 5,6
 
6,1 6,2 6,3 6,4 6,5 6,6
n(S) =36

Let A: the first die shows an even number

B: the second die shows an odd number

C: the two dice show both even or both odd numbers

P(A) = 18/36 =1/2 P(B) =18/36 =1/2 P(C) =18/36 =1/2

P(A  B) = 9/36 =1/4 =(1/2 )( 1/2) =1/4

P(A/B) =P(A  B)/P(B) = [P(A) *P(B)]/P(B) = P(A) =1/2

Definition (Independent events): Two events A and B are said to be independent iff
P(AB)=P(A) P(B)

Verbally speaking, A and B are independent provided knowledge of the occurrence

of A by no means influences the probability of occurrence of B.
Remarks
1) If A and B are mutually exclusive when do they become independent?
P(A  B) = P()  P(A) P(B) = 0

 A and B are independent and mutually exclusive iff A  B =  and either

P(A) =0 or P(B)=0
2) If A  B then A  B = A
 P(A  B) = P(A)

 A and B cannot be independent unless P(B)= 1

B=S

3)  and S are independent of any event

4) Three events A, B, and C are said to be mutually independent if and only if

.“The more you read, the more you understand” 8

Chapter 4 Conditional probability and independence

 P(A∩B)=P(A)P(B)
 P(B∩C)=P(B)P(C)
 P(A∩C)=P(A)P(C)
 P(A∩B∩C)=P(A)P(B)P(C)

.“The more you read, the more you understand” 9