Course Code: MMA 105

Course Title: INTRODUCTION TO PROBABILITY THEORY

Topic 1: INTRODUCTION TO PROBABILITY.

Introduction

In this topic, we introduce learners to the basic laws of probability.

Objectives

By the end of this topic, you should be able to:

1. Define probability and associated terms

2. Differentiate between the classical, axiomatic, relative frequency approaches to probability

3. Use laws of probability to answer real life questions

Learning Activities

Activity 1:
➢ Students to take note of the exercises provided within the text and at the end of the topic

Topic Resources

Students to take note of the reference text books provided in the course outline
 TOPIC 1: < PROBABILITY >

PROBABILITY
We begin by defining some important terms of probability.

TERMINOLOGIES
Definition 1.1: Probability of an event:
Probability is a measure of the likelihood that an event will happen and it is given on numerical scale from 0
to 1.
Definition 1.2: Deterministic Experiments:

➢ These are experiments which always produce the same results i.e. produces unique outcomes /
events.

➢ An experiment is deterministic given the conditions under which the experiment is carried and the
outcome is completely determined.

Definition 1.3: Trial:

➢ It’s the number of times an experiment is repeated.

Definition 1.4: Event:

➢ It’s the probable outcome of a trial.

Definition 1.5: Equally likely cases:

➢ Two or more outcomes of an experiment are said to be equally likely if one of them cannot be
expected to occur in preference to the other.

Definition 1.6: Mutually exclusive events:

➢ Two events are said to be mutually exclusive if the occurrence of either precludes the occurrence of
the other i.e. two exclusive events cannot occur simultaneously in the same trial. E.g. if a coin is
tossed once, the outcome is either a head or a tail but not both.

Definition 1.7: Exhaustive events:

➢ A set of events are said to be exhaustive when it includes all possible outcomes of a trial. It means
that, all possible events that can happen are included in the study of probability.
Page 2 of 13
 TOPIC 1: < PROBABILITY >

Definition 1.8: Favorable events:

➢ The number of cases favorable to an event. E.g. In throwing two dice, we have four number of ways
of getting a sum of five i.e. [1, 4], [4, 1], [2, 3] and [3, 2]

➢ The pair of observations [1, 4], [4, 1], [2, 3] and [3, 2] constitute favorable events.

Definition 1.9: Simple events:

➢ An event is said to be simple if it corresponds to a single possible outcome of an experiment. E.g. the
probability of drawing a green ball from a bag containing six red and ten green balls.

Definition 1.10: Compound events:

➢ An event is said to be compound if it corresponds to a joint occurrence of two or more simple

outcomes of an experiment. E.g. the probability of drawing a red, then a green ball from a bag
containing six red and ten green balls.

Definition 1.11: Independent events:

➢ Several events are said to be independent if the happening of one event is not affected by the
supplementary knowledge concerning the occurrence of the remaining events. Otherwise, the events
are said to be dependent!

Approaches to the Concept of probability

Approach to probability can either be objective or subjective.
Objective probability: is based on definite historical information, common experience (objective
evidence), or some rigorous analysis.
Subjective probability: is based on personal experience alone which becomes the basis of the probability
assignment
There are three main approaches to the concept of probability.
a) Classical approach

b) Axiomatic approach

c) Relative frequency approach.

Page 3 of 13
 TOPIC 1: < PROBABILITY >

Exercise 1.1:

Read and make notes on the three main approaches to the concept of probability

Axioms / Laws of Probability.

➢ Let S be the sample space and E be any random variable.

➢ Probability is a number that is assigned to each number of a collection of events from a random
experiment that satisfies the following properties.

i. P( E )  0
ii. P( S ) = 1
iii. If E1 , E 2 ,  , E n are pairwise disjoint events (mutually exclusive and exhaustive events), then
  
P  E n  =  P ( En ) .
 n =1  n =1
Theorem 1.1: The probability for an impossible event is zero. (The null event has probability 0) i.e
p( ) = 0 .

Proof:

If we consider a sequence of events E1 , E 2 ,  , E n where E1 = S , E i =  for i  1 , then the events are


mutually exclusive and S =  E i
i =1

  
P( S ) = P  E n  = P ( Ei ) by axiom 3
 n =1  i =1


= P ( E1 ) +  P ( E i )
i=2


= P ( S ) +  P ( )
i =2


  P ( ) = 0  P( ) = 0
i=2

Page 4 of 13
 TOPIC 1: < PROBABILITY >

Theorem 1.2: If Ac is the complimentary event of A, then P( Ac ) = 1 − P( A)  1

Proof:

A and Ac are mutually exclusive events such that

A  Ac = S

P( A  Ac ) = P(S ) = 1

P( A) + P( Ac ) = P(S ) = 1

P ( A c ) = 1 − P ( A) . Since P( A)  0 it follows that P ( A c )  1

Theorem 1.3: If B  A then P( B)  P( A) .

Exercise 1.2: Proof of Theorem 1.3: is left as an exercise!

Theorem 1.4: Addition Law of Probability. If A and B are any two events and are not disjoint, then
P( A  B) = P( A) + P( B) − P( A  B) .

Proof:

A B

c c

From the above Venn diagram,

Page 5 of 13
 TOPIC 1: < PROBABILITY >

A  B = A  ( Ac  B)

P( A  B) = P( A  ( Ac  B))

P( A  B) = P( A) + P( Ac  B)

P( A  B) = P( A) + P( B  Ac )

Adding and subtracting P( A  B) both sides we have:

P( A  B) + P( A  B) = P( A) + P( Ac  B) + P( A  B)

P( A  B) = P( A) + P( Ac  B) + P( A  B) − P( A  B)

P( A  B) = P( A) + P( B) − P( A  B)

Theorem 1.5: If A , B and C are any three events then

P( A  B  C ) = P( A) + P( B) + P(C ) − P( A  B) − P( B  C ) − P(C  A) + P( A  B  C ) .

Proof:

P( A  B  C ) = P ( A  B )  C  = P( A  B) + P(C ) − P[( A  B)  C ]

= P( A) + P( B) − P( A  B) + P(C ) − P ( A  C )  ( B  C ) 

= P ( A) + P ( B ) − P( A  B) + P(C ) − [ P( A  C ) + P ( B  C ) − P ( A  B  C )]

= P( A) + P( B) + P(C ) − P( A  B) − P( A  C ) − P( B  C ) + P( A  B  C ).

Theorem 1.6: Multiplication Theorem.

If two events A and B are independent and can happen simultaneously, the probability of their joint
occurrence P( A  B) = P( A)  P( B) .

Note: The above theorem can be extended to three or more events.

Theorem 1.7: If the events A and B are independent, then

i. Ac and B c are independent

Page 6 of 13
 TOPIC 1: < PROBABILITY >

ii. Ac and B are independent.

iii. A and B c are independent
Proof:

i. We know that ( A  B)c = Ac  Bc

P( Ac  Bc ) = P( A  B)c
= 1 − P( A  B)
= 1 − [ P( A) + P( B) − P( A  B)]
= 1 − P( A) − P( A) + P( A  B)
= 1 − P( A) − P( B) + P( A)  P( B)
= 1 − P( A) − P( B)[1 − P( A)]
= [1 − P( A)][1 − P( B)]
= P ( Ac )  P ( B c )
Therefore Ac and B c are independent

ii. The events A  B and Ac  B are mutually exclusive such that ( A  B)  ( Ac  B) = B

By addition theorem, P[( A  B)  ( Ac  B)] = P[ A  B] + P[ Ac  B] = P( B)

Then P( Ac  B) = P( B) − P( A  B)

= P( B) − P( A).P( B)

= P( B)[1 − P( A)]

= P( B).P( Ac )

Hence Ac and B are independent.

iii. Now
A = ( A  B)  ( A  B c )
P( A) = P( A  B) + P( A  Bc )
P( A  Bc ) = P( A) − P( A  B)
= P( A) − P( A).P( B)
= P( A)[1 − P( B)]
= P( A).P( Bc )
Thus A and B c are independent.

Page 7 of 13
 TOPIC 1: < PROBABILITY >

Example 1.1:

Events C and D are such that P(C ) = 19

30 , P( D) = 52 and P(C  D) = 54 . Find P(C  D) .

Solution:

P(C  D) = P(C ) + P( D) − P(C  D)

19 2 4 7
P(C  D) = P(C ) + P( D) − P(C  D) = + − =
30 5 5 30

Exercise 1.3:

Events A and B are such that P( A) = 0.3, P( B) = 0.4 and P( A  B) = 0.1 . Find

a) P( A  Bc )
b) P( Ac  Bc ) .

Exhaustive Events.

➢ Any two events A and B are such that between them they make up the whole of the possibility
space. Then A and B are said to be exhaustive events and P( A  B) = 1 .

➢ Any event A and its complimentary event Ac are both mutually exclusive and exhaustive.

Conditional Probability.

➢ If A and B are two events, not necessarily from the same experiment, then the conditional
probability that A occurs given that B has already occurred is written as P(A given B) or P( A | B) .

P( A  B)
➢ The conditional probability of A given B is written as P( A | B) = where P( B)  0 .
P( B)

P( A  B)
➢ Likewise, conditional probability of B given A is written as P( B | A) = where P( A)  0 .
P( A)

Remark:

P( A  A) P ( A)
1. P( A | A) = = =1
P( A) P( A)
Page 8 of 13
 TOPIC 1: < PROBABILITY >

Exercise 1.4:

P( A | B) P( A)
Show that =
P( B | A) P ( B )

Exercise 1.5:

If P( A)  P( B) , then P( A | B)  P( B | A)

Marginal Probability

A probability of only one event that takes place is called marginal probability.

Joint Probability

The probability of occurrence of both events A and B together denoted by P( A  B) is known as joint
probability of A and B .

Example 1.2:

If X and Y are two events such that P( X | Y ) = 0.4, P(Y ) = 0.25 and P( X ) = 0.2 , Find

i. P(Y | X )
ii. P( X  Y )
iii. P( X  Y )

Solution:

P(Y | X )  P( X ) = P( X | Y )  P(Y )
i. P(Y | X )  0.2 = 0.4  0.25.
 P(Y | X ) = 0.5
ii. P( X  Y ) = P( X | Y )  P(Y ) = 0.4  0.25 = 0.1
iii. P( X  Y ) = P( X ) + P(Y ) − P( X  Y ) = 0.2 + 0.25 − 0.1 = 0.35

Exercise 1.6:

When two dice are thrown (or a die is thrown twice), let A be the event that the sum of the points on the
faces is odd and B is the event that at least one number is 2. Find the probabilities of the following:

i. A
ii. B
Page 9 of 13
 TOPIC 1: < PROBABILITY >

iii. Ac
iv. Bc
v. A B
vi. A B
vii. Ac  B
viii. Ac  Bc
ix. Ac  Bc
x. B| A

Independent Events.

Definition 1.12:

➢ Let A and B be two events. If either of event A and B occur without being affected by the other, then
the two events are independent.

➢ If A and B are independent events, then

1. P( A  B) = P( A)  P( B)
P( A  B) P( A)  P( B)
2. P( A | B) = = = P( A)
P( B) P( B)
P( B  A) P ( A)  P( B)
3. P( B | A) = = = P( B)
P( A) P( A)

Example 1.3:

1 1
Events A and B are independent and P( A) = , P( A  B) = , find
3 12

i. P(B)
ii. P( A  B)

Solution:

P( A  B) = P( A).P( B)
i. 1
 P( B) =
4
1 1 1 1
ii. P( A  B) = P( A) + P( B) − P( A  B) = + − =
3 4 12 2
Exercise 1.7:

Page 10 of 13
 TOPIC 1: < PROBABILITY >

The events A and B are such that P( A | B) = 0.4, P( B | A) = 0.25, P( A  B) = 0.12 .

i. Calculate the value of P(B)
ii. Give a reason why A and B are not independent.
iii. Calculate the value of P( A  Bc )
Exercise 1.8:
The events A and B are independent and are such that P( A) = x , P( B) = x + 0.2 and P( A  B) = 0.15 .
i. Find the value of x . For that value of x , find
ii. P( A  B)
iii. P( A c | B c )

Note: If A and B are independent, then A c and B c are also independent.

BAYES THEOREM

Definition 1.13: If B1 , B2 ,..., Bn are mutually exclusive events with p ( Bi )  0 , i = 1,2,..., n of a random experiment,
then for any arbitrary event A of the sample space of the above experiment with p( A)  0 , we have

P ( Bi ) P ( A | Bi )
P ( Bi | A) = n
.
 P( B ) P( A | B )
i =1
i i

The above definition refers to the Baye’s Theorem.

Proof:

Let S be the sample space of the random experiment. The events B1 , B2 ,..., Bn being exhaustive (form a possibility
space/ disjoint and exhaustive).

S = B1  B2  ...  Bn

A = A  S = A  ( B1  B2  ...  Bn )

= ( A  B1 )  ( A  B2 )  ...  ( A  Bn ) (by Distributive law)

 P( A) = P ( A  B1 ) + P ( A  B2 ) + ... + P ( A  Bn )

= P( B1 ) P( A | B1 ) + P( B2 ) P( A | B2 ) + ... + P( Bn ) P( A | Bn )
n
=  P( B ) P( A | B )
i =1
i i

Now P( A  Bi ) = P( A) P( Bi | A)

Page 11 of 13
 TOPIC 1: < PROBABILITY >

P( A  Bi ) P( Bi ) P( A | Bi )
P ( Bi | A) = = n
 P( Bi ) P( A | Bi )
P( A)
i =1

P ( Bi ) P ( A | Bi )
P ( Bi | A) = n

 P( B ) P( A | B )
i =1
i i

Next, we look at examples to illustrate the application of Baye’s Theorem.

Example 1.4:
An urn I contains 3 white and 4 red balls and urn II contains 5 white and 6 red balls. One ball is drawn at
random from one of the urns and is found to be white. Find the probability that it was drawn from urn I.
Solution:
We need to get P(U I | w)

P(U I ) P( w | U I )
By Bayes Theorem, P(U I | w) =
P(U I ) P( w | U I ) + P(U II ) P( w | U II )

The two urns are equally likely to be selected.

Thus P(U I ) = P(U II ) = 1 .

2

P( w | U I ) = P (white ball from urn I) = 3

7

P( w | U II ) = P (white ball from urn II) = 5

11
1 3

2 7 33
P(U I | w) = =
 1 3   1 5  68
  +  
 2 7   2 11 
Example 1.5:
In a bolt factory, machines A, B and C manufacture 25%, 35% and 40% of the total. Of their output 5, 4 and
2 per cent are defective bolts. A bolt is drawn at random from the product and is found to be defective. What
is the probability that it was manufactured by machine B?
Solution:
P( A) = 0.25, P( B) = 0.35, P(C ) = 0.40

P( D | A) = 0.05 , P( D | B) = 0.04 and P( D | C ) = 0.02

Page 12 of 13
 TOPIC 1: < PROBABILITY >

P( B) P( D | B)
P( B | D) =
P( A) P( D | A) + P( B) P( D | B) + P(C ) P( D | C )

0.35  0.04
= = 0.41
(0.25  0.05) + (0.35  0.04) + (0.40  0.02)
Example 1.6:
The contents of three urns 1, 2 and 3 are as follows

Balls White black red

Urn I 1 2 3

Urn II 2 3 1

Urn III 3 1 2

An urn is chosen at random and from it two balls are drawn at random. The two balls are found to be one
white and one black. What is the probability that they are from the second urn?
Solution:
Let B1 , B2 , B3 denote the events that the urns I, II and III are chosen respectively.

Let A be the event that the two balls taken from the selected urn are white and black.

Then P( B1 ) = P( B2 ) = P ( B3 ) = 1 .
3
1
C1  2 C1 2 2
C1  3 C1 2 3
C1 1 C1 1
P( A | B1 ) = 6
= , P ( A | B 2 ) = 6
= , P ( A | B3 ) = 6
=
C2 15 C2 5 C2 15

1 2

P ( B2 ) P ( A | B2 ) 3 5 6
P( B2 | A) = 3 = = .
 1 2   1 2   1 1  11
i =1
P( Bi ) P( A | Bi )    +    +   
 3 15   3 5   3 5 
Exercise 1.9: A bag A contains 2 white and 3 red balls while a bag B contains 4 white and 5 red balls. One
ball is drawn at random from one of the bags and is found to be red. Find the probability that it was drawn
from bag B.…. Easy!!!!

Page 13 of 13