3.

Elementary Probability

1
 Elementary Probability
3.1 Deterministic and non-deterministic models
Statistics includes the process of finding out about patterns in the real world using
data. When solving statistical problems it is often helpful to make models of real
world situations based on observations of data, assumptions about the context,
and on theoretical probability.
• Statistical models are constructed to represent the
interrelationship between two or more variables.
• Depending on the types of experiments for which models are fitted,
they are classified as deterministic and non-deterministic models.

What is an experiment?
• Experiment is an act or process that can be done under certain
condition to obtain a well defined result(s).
• A result is usually called the outcome of an experiment.
• The possible results of an experiment may be one or more than one.
• Based on the number of possible results, we may classify an
experiment as deterministic and non-deterministic experiment. 2
 Elementary …

A) Deterministic experiment: has only one possible outcome which is certain or unique.
The result of such experiment is dependent on the condition under which it is performed. A
mathematical model to be defined on this experiment is called deterministic model.
Most of the experiments which may be conducted to verify science laws are examples of deterministic
experiments.
Eg., Newton’s second law: for a body whose mass m is constant, F = ma

B) Non-deterministic experiment: the outcome of a given trial cannot be known prior to

its conduct. We also call this experiment as unpredictable or probabilistic or stochastic or random
experiment.
• Usually the result of this experiment is subjected to chance and is possibly more than one.
These types of experiments are characterized by the following three properties:
I. The experiment is repeatable under identical conditions.
II. The outcome in any particular trial is a variable, i.e., it depends on some chance or random
mechanism.
III. If the experiment is repeated a large number of times, some regularity becomes apparent in the
outcomes obtained. This regularity enables us to set some mathematical form of equations called
non-deterministic model.
Definition: Non-deterministic model is a mathematical description of an uncertain situation.
Example: Tossing a coin: there are two possible outcomes; i.e., getting a head or tail. Though we know the
possible outcomes, we cannot for sure predict the outcome. 3
 3.2 Review of set theory

Definition and Types of Sets

• A set is a well-defined collection of objects.
• Sets are usually denoted by capital letters (A, B, C, etc.)
• In describing which objects are contained in the set A, three methods are available.
(a) List the members of A. Ex., A = {I, 2, 3, 4} describes the set consisting of the positive
integers I, 2, 3, and 4.
(b) Describe the set A in words. For example, we might say that A consists of all real
numbers between 0 and I, inclusive.
(c) To describe the above set we can simply write 𝐴 = 𝑥|0 ≤ 𝑥 ≤ 1 ; that is, A is the set
of all x's, where x is a real number between 0 and I, inclusive.

• The individual objects making up the collection of the set A are called members or elements of A.
Elements of sets are usually denoted by small letters (a, b, c, etc.)
• When "a" is a member of A we write a ∈ A and when "a" is not a member of A we write a ∉ A.
• When two sets A and B are considered, it may happen that being a member of A implies
being a member of B.
– In that case we say that A is a subset of B and we write A ⊆ B.
– A similar interpretation is given to B ⊆ A.
– And we say that two sets are the same (A=B).
– That is, two sets are equal if and only if they contain the same members. 4
 Review of …

Universal set (U): it is the collection of all objects under

consideration.
Examples:
The set of real numbers can be seen to be universal set of numbers.
 Consider an experiment of rolling a balanced die.

The set of all possible outcomes of this experiment can be considered

as the universal set.
U={1,2,3,4,5,6}

Empty set (φ or {} ): A set with no element is called empty set.

Examples:
 If the universal set is the set of positive integers, then getting a
negative integer is impossible.
 In the rolling a die experiment, facing nine dots is not possible.
5
 Review of …

Set operations
Union: Given two sets A and B, the union of A and B denoted by A∪ B is the set of all
elements, which belong to set A or B, or both.
A∪B ={ X: X ∈ A ∨ X∈ B}

Here the logical connective ∨ (OR) is used in its inclusive sense.

Note that A ∪ φ = A and A ∪ U = U

Intersection: For any two sets A and B the intersection of A and B is defined to be the set of
all elements that occur in both set A and also in set B. Symbolically, we write
A ∩ B = { x U | x  A and x  B }

Sets with no intersection are called disjoint sets. A and B are disjoint if A ∩ B = Φ. 6
 Review of …

Complement: The complement of a set, denoted by A', or A c

or Ā is the set of all
elements in the given universal set U that are not in A. That is A' = {x ∈ U | x ∉ A}.

Examples: U' = ∅ The complement of the universe is the empty set.

∅' = U The complement of an empty set is the universal set.

Properties of set operations

I. Commutative law
• i) A∪ B = B ∪ A
• ii) A ∩ B = B ∩ A
II. Associative law
• i) A∪ (B ∪ C) = (A ∪ B) ∪ C
• ii) A ∩ (B ∩ C) = (A ∩ B) ∩ C
III. Distributive law
• i) A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)
• ii) A ∩ (B ∪ C) = (A ∩B) ∪ (A ∩ C)

IV. Demorgan’s law

• (A∪ B) = A ∩ B and (A ∩B) = A ∪ B
c c c c c c 7
 3.3 and 3.4 Definition of some probability terms
Random experiments, sample space, events, finite sample spaces and equally likely outcomes
The concepts of sample space and events may be considered as the corner stones for the whole discussion in
probability. Our revision on universal set and subsets in set theory would help us to easily understand the
concepts of sample spaces and events; because they do have a perfect analogy.
1. Random Experiment: It is an experiment that can be repeated a number of times under similar conditions
and it is possible to enumerate the total number of outcomes without predicting an individual outcome.
• Statistical methods are used to evaluate information in uncertain situations and probability plays a key role in that
process. In personal and management decisions, we face uncertainty.
• The quantitative measure of uncertainty regarding one or more outcomes of a random experiment is called a probability.
A probability experiment is a chance process that leads to well-defined results called outcomes. Processes such as flipping a
coin, rolling a die, or drawing a card from a deck are called probability experiments.

2. An outcome is the result of a single trial of a probability experiment.

A sample space is the set of all possible outcomes of a probability experiment.

Some sample spaces for various probability experiments are shown below.
Experiment Sample space
Toss one coin H,T
Roll a die 1,2,3,4,5,6
Answer a true/false question True, False

Toss two coins H-H, T-T, H-T, T-H

8
 … some probability terms

Playing cards (Standard deck)

Flowers

Spades

Hearts

Diamonds

9
 … some probability terms
For the experiment of drawing a card from a standard deck, the following sets can be possible
sample spaces.
– S={Heart, Spades, Diamond, Flower}
– S={1s, 2s, 3s, … , Queens, Kings, Jacks}
– S={Black card, Red card}

As indicated in the above example, it is possible to have more than one sample space for a given
random experiment. Hence, we can say there are different types of sample space. These are:

• Finite sample space: if they have a finite number of elements.

Examples:
• Tossing a coin. S={Heads, Tails}
• Throwing a die. S={1, 2, 3, 4, 5, 6}
• Throwing a coin and a die. S={H1, H2, H3, H4, H5, H6, T1, T2, T3, T4, T5, T6}.

• Countable infinite sample space: if there is a one-to-one mapping between the elements of S and the
natural numbers.
Examples:
• S={0, 1, 2, 3, …}
• S={ … -3, -2, -1, 0, 1, 2, 3, …}
• S={p : p is prime} = {2, 3, 5, 7, 11, 13, 17, …}
• S={1/r : r is a positive natural number} = {1/2, 1/3, 1/4, 1/5, … }

10
 … some probability terms

• Uncountable sample space: if there are an infinite number of elements in S and there is
not a one-to-one mapping between the elements and the natural numbers.
Examples:
• S=(0, ∞)
• S=(–∞, ∞)
• S={(x, y): 0≤ x≤ 1; y>0}

3. An event consists of a set of outcomes of a probability experiment. An event can be one

outcome or more than one outcome. It is a subset of a sample space and
denoted by capital letters.

Examples:
• In the experiment of flipping a coin, we can define an event E1=Heads show up={H}
• In the experiment of rolling a die, we can define an event C2=The number is odd {1,3,5}
• In tossing a coin twice we can define an event D1=At least one head occurs={HH, TT, TH}

Notes:
• If S has n members, then there are exactly 2n subsets or events.
• Recall that any set is the subset of itself. Accordingly, the sample space S by itself is an
event called a sure event or certain event.
• An event occurs if any one of its elements turns out to be the outcome of the experiment.
• Empty set (φ) is called an impossible event.
• An event with one outcome is called elementary or a simple event.
• A compound event consists of two or more outcomes or simple events. 11
 … some probability terms

4. Mutually Exclusive events: Two events A and B are said to be mutually exclusive if they cannot occur
simultaneously (at the same time) or if A ∩ B = φ

Example: Experiment -Toss a coin twice. S= {HH, HT, TH, TT}

Let
A= Two heads {HH}
B= Two tails {TT}
C= At least one head occur {HH, HT, TH}

A ∩ B= φ ⇒ A and B are mutually exclusive events

B ∩ C= φ ⇒ B and C are mutually exclusive events
A ∩ C= {HH} ⇒ A and C are not mutually exclusive

5. Independent events: Two events are independent if the occurrence of one does not
affect the probability of the other occurring.

6. Dependent events: Two events are dependent if the first event affects the outcome or
occurrence of the second event in such a way that the probability is changed.

7. Equally likely events are events that have the same probability of occurring.
1
For example, when a single die is rolled, each outcome has a probability of , as
6
there are six outcomes.
12
 … some probability terms

8. Combination of events
a) Union: The event A∪ B occurs if either A or B or both occur.
b) Intersection: The event A ∩ B occurs if both A and B occur.
c) The complement of an event A is the set of outcomes in the sample space that are not included in the
outcomes of event A.

Exercise 1: A person is selected at random from a population of a given town.

Let us assign the events as follows:
A: the person is male
B: the person is under 30 years
C: the person speaks foreign language

Describe the following events symbolically.

a) A male under 30 years who does not speak foreign language.
– (A ∩ B) ∩ C′
b) A female who is either under 30 or speaks foreign language.
– A′ ∩ (B ∪ C)
c) A person who is either under 30 or female but not both.
– (B ∪ A′) ∩ (B ∩ A)′
d) Male who is either under 30 or speaks foreign language but not both.
– A ∩ [(B ∪ C) ∩ (B ∩ C) ′] 13
 … some probability terms

Exercise 2: Let A, B and C be three events associated with an experiment. Describe the following in symbols.
a) At least one of the events occur.
– AUBUC

b) Only event B occurs.

– B∩ A′∩C′

c) Exactly one of the events occur.

– (A∩ B′∩C′)∪(B ∩ A′∩C′)∪(C ∩ B′∩ A′)

d) Exactly two of the events occur.

– (A∩ B ∩C′)∪(B ∩ A′∩C)∪(C ∩ B′∩ A)

e) At least two of the events occur.

– ((A∩ B)∪C) ∪(A∩C)∪ B)∪(C ∩ B)∪ A)) ∪(A ∩ B∩C)

f) At most two of the events occur.

– A′∪ B′∪C′

g) None of the events occur.

– A′∩ B′∩C′

14
 3.5 Counting Techniques
Many problems in probability require a careful analysis of the outcomes of
a sequence of events.
 If the number of possible outcomes in an experiment is small, it
is relatively easy to list and count all possible events.
 When there are large numbers of possible outcomes, an
enumeration of every point may be extremely time consuming
or even practically impossible.

To overcome such problems, we can use the following

counting rules. These are:

fundamental counting rules

the permutation rule
the combination rule

15
 Counting ….
The fundamental Counting Rules
The fundamental counting rules can be classified as addition rule of counting and
multiplication rule of counting depending whether the events are happing simultaneously
or in sequence.

i) Addition rule of counting

In probability theory, it is important to understand the meaning of the
words “and” and “or”.

For example, if you were asked to find the probability of getting a queen
and a heart when you were drawing a single card from a standard deck,
you would be looking for the queen of hearts.
Here the word and means “at the same time”.

The word or has two meanings.

• For example, if you were asked to find the probability of selecting a
queen or a heart when one card is selected from a deck, you would be
looking for one of the 4 queens or one of the 13 hearts.
• In this case, the queen of hearts would be included in both cases and
counted twice. So there would be 4+13-1 = 16 possibilities.
16
 Counting …
• On the other hand, if you were asked to find the probability of
getting a queen or a king, you would be looking for one of the 4
queens or one of the 4 kings.
• In this case, there would be 4+4=8 possibilities.

• In the first case, both events can occur at the same time; we say
that this is an example of the inclusive or
.

• In the second case, both events cannot occur at the same time,
and we say that this is an example of the exclusive or.

• In other words, getting a queen or a king cannot occur at the

same time. Hence, they are mutually exclusive events.

17
 Counting …
Let E1 and E2 be any two events and event E describes the situation where
either event E1 or event E2 will occur.

The number of times event E will occur can be given by the expressions:

– n(E) = n(E1) + n(E2) (If the events are mutually exclusive)

– n(E) = n(E1) + n(E2) – n(E1∩E2) (for non-mutually exclusive events)

Where,
n(E) = Number of outcomes of event E
n(E1) = Number of outcomes of event E1
n(E2) = Number of outcomes of event E2

18
 Counting …
The multiplication rule of counting
In a sequence of n events in which the first one has k1
possibilities and a second event has k2 and the third has k3,
and so forth, the total number of possibilities of the
sequence will be

k1∙k2 k2∙k3∙∙∙kn

Note: In this case and means to multiply.

When determining the number of possibilities

of a sequence of events, you must know
whether repetitions are permissible.
19
 Counting …
Example:
The manager of a company wishes to make four-digit identification
cards for her employees. How many different cards can be made if
she uses the digits 1,2,3,4,5, and 6 and repetitions are permitted?
Soln
Since there are four 4 spaces to fill on each card and there are 6
choices for each space, the total number of cards that can be made
is 6x6x6x6=1296.
What if repetitions are not permitted?
The first digit can be chosen in 6 ways. But the second number can
be chosen in only 5 ways, since there are only five digits left, etc.
Thus, the solution is 6x5x4x3=360

In summary, if repetitions are permitted, then the numbers stay the

same going from left to right. If repetitions are not permitted,
then the numbers decrease by 1 for each place left to right.
20
 Counting …
 The other two rules that can be used to determine the total numbers of
possibilities of a sequence of events use factorial notation.
 The factorial notation uses the exclamation mark. 5!=5x4x3x2x1

Factorial formulas
For any counting n
• n! = n(n-1) (n-2)… 1
• 0! = 1
Permutation Rule
A permutation is an arrangement of n objects in a specific order
(meaning order is important).
The arrangement of n objects in a specific order using r objects at a
time is called a permutation of n objects taking r objects at a time. It
n!
is written as nPr and the formula is
P
n r  ( n  r )! 21
 Counting …
Example:
The director of the Ethiopian Television wishes to use 3 news
stories on an evening show. One story will be the lead story,
nd
one will be the 2 story and the last will be the closing story. If
the director has a total of 8 stories to choose from, how many
possible ways can the program be set up?
Soln:
Since order is important, the solution is

8! 8! 87 6 5!
P
8 3  ( 8  3 )!
 5!
 5!
 336
.
Hence, there would be 336 ways to set up the program.

22
 Counting …
A selection of distinct objects without regard to order is called a combination.
The difference between a permutation and a combination is that in
a combination, the order of arrangement of the objects is not
important; by contrast, order is important in permutation.
Combination Rule
The number of combinations of r objects selected from
n objects is denoted by nCr and is given by the formula

n!
C
n r  ( n  r )! r!
Note that Also
n! P
C
n r  ( n  r )! r!
 n r
r! C
n n  n!
( n  n )! n!
 n!
0!n!
1
23
 Counting …
Example:
In a club there are 7 women and 5 men. A committee of 3 women
and 2 men is to be chosen. How many different possibilities are
there?
Soln
• Here you must select 3 women from 7 women, which can be
done in 7C3 ways.
• Next 2 men must be selected from 5 men, which can be done in
5C2 ways.
• Finally, by the fundamental counting rule, the total number of
different ways is
7! 5! 7 6 5 4! 5 4 3!
C  C
7 3 5 2  ( 7  3 )! 3!
 ( 5  2 )! 2!
 4!3 2
 2 3!
 35  10  350

24
 3.6 Definitions of Probability
In any random experiment there is always uncertainty as to whether a particular
event will or will not occur. In every aspect of life there are many happenings and
one may want to determine the chance of occurrence of these happenings. This
initiated the need of probability theory.
• Probability is widely used in playing games of chance, such as card games or
lotteries. In addition being used in games of chance, probability theory is used in
several fields including physical sciences and engineering.
• Probability is the basis of inferential statistics. For example, predictions are
based on probability, and hypotheses are tested by using probability.

Approaches in defining probability

There are four different conceptual approaches to study probability theory.
These are:
• The classical approach
• The frequentist approach
• The subjective approach
• The axiomatic approach

25
 Approaches …

A) Classical Probability
Classical probability uses sample space to determine the numerical
probability that an event will happen. It is so named because it was the
first type of probability studied formally by mathematicians in the 17 th
th
and 18 centuries.
Formula for Classical Probability

Number of outcomes in E
• The probability of any event E is
Total number of outcomes in the sample space

• This probability is denoted by P( E )  n( E )

n( S ) and uses sample space S.

Example 1: A fair die is rolled once. What is the probability of getting

• a) Number 4?
• b) An odd number?
26
 Approaches …
B) The frequentist approach
In the frequentist approach the likelihood of outcomes are determined on
actual experience.

Definition: The probability of an event A is the proportion of outcomes

favourable to A in the long run when the experiment is repeated
under same condition.

𝑁𝐴
𝑃 𝐴 = lim
𝑁→∞ 𝑁

• Eg., If records show that 60 out of 100,000 bulbs produced are defective.
What is the probability of a newly produced bulb to be defective?
𝑁𝐴 60
𝑃 𝐴 = lim = = 0.0006
𝑁→∞ 𝑁 100,000

This probability is based on observation.

27
 Approaches …

C) Subjective Probability
Subjective probability uses a probability value based on an
educated guess or estimate, employing opinions and inexact
information. This guess is based on the person’s experience and
evaluation of a situation.

Definition: Subjective probability is the degree of believe assigned

to the occurrence of an event by a particular individual.

Example: A physician might say that, on the basis of her diagnosis,

there is a 30% chance the patient will need an
operation.

28
 Approaches …

D) Axiomatic Approach
Let E be a random experiment and S be a sample space associated
with E. With each event A defined on S, we can associate a real
number called the probability of A satisfying the following
properties called axioms of probability or postulates of probability.

1. 𝑃 𝐴 ≥ 0 (i.e., the probability of any event is non-negative real number)

2. 𝑃 𝑆 = 1 (Sure event)
3. If A and B are two mutually exclusive events, the probability that one or
the other occur equals the sum of the two probabilities.
That is , 𝑃 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃(𝐵)
4. If A1, A2, A3, …. form a sequence of pair wise mutually exclusive events
(meaning they satisfy 𝐴𝑖 ∩ 𝐴𝑗 = ∅ for all 𝑖 ≠ 𝑗 then
∞
∞
𝑷 𝑨𝟏 ∪ 𝑨𝟐 ∪ 𝑨𝟑 ⋯ = 𝑷 𝑨𝒊 = 𝑷(𝑨𝒊 )
𝒊=𝟏
𝒊=𝟏
29
 Approaches …
The axioms of probability or postulates of probability usually
referred as basic probability rules.
These rules are helpful in:
 solving probability problems
understanding the nature of probability
deciding if your answers to the problems are correct
Probability Rule 1
The probability of any event E is a number between and
including 0 and 1.

This is denoted by
0  P(E)  1
30
 Basic rules …
Probability Rule 2
If an event E cannot occur (i.e., the event contains no members in
the sample space), its probability is 0.
Example:
When a single die is rolled, find the probability of getting a 9.
Soln:
Since the sample space 1,2,3,4,5 and 6, it is impossible to get a 9.
0
Hence, the probability is P(9)  6  0
Probability Rule 3
If an event E is certain, then the probability of E is 1.
Example: When a single die is rolled, what is the probability of
getting a number less than 7.

Soln: Since all outcomes —1,2,3,4,5 and 6— are less than 7, the
probability is 6.
P(less than 7)  6
1
The event of getting a number less than 7 is certain.
31
 Basic rules …
Probability values range from 0 to 1. When the probability of an event is close to 0, its
occurrence is highly unlikely. When the probability of an event is near 0.5, there is
about a 50–50 chance that the event will occur; and when the probability of an event
close to 1, the event is highly likely to occur (i.e., certain).

Probability Rule 4
The sum of the probabilities of all outcomes in the sample
space is 1.
For example in the roll of a fair die, each outcome in the sample space has a
probability of 1/6. Hence, the sum of the probabilities of each outcome
= 1/6+1/6+1/61+1/6+1/6+1/6=6/6=1 32
 Addition rules …
The Addition Rules for Probability
The probability of two or more events (compound events) can be
determined by the addition rules.
Addition Rule 1
• When two events A and B are mutually exclusive, the probability
that A or B will occur is
P(A or B) = P(A)+P(B)
Examples: 1) Drawing a single card from a standard deck and getting a jack or an 8.
2) Rolling a single die and getting an odd or even number.
Addition Rule 2
• If A and B are not mutually exclusive, then
P(A or B) = P(A)+P(B)−P(A and B)
Examples: 3) Drawing a single card from a standard deck and getting a jack or a
diamond
4
52  13
52  1
52  16
52  4
13
4) Rolling a single die and getting a 4 or an even number
33
 Addition rules …

(a)mutually exclusive events (b) Non-mutually exclusive events

P(A or B) = P(A)+P(B) P(A or B ) = P(A ) + P(B ) – P(A and B )

The addition rules can be extended to three or more events.

• For three mutually exclusive events, A, B, C,
P(A∪B∪C) = P(A)+P(B)+P(C)

• For three events that are not mutually exclusive,

P(A∪B∪C) = P(A)+P(B)+P(C)−P(A∩B)−P(A∩C)−P(B∩C)−P(A∩B∩C) 34
 3.7 Derived theorems of probability

1. 𝑃 ∅ = 0 for any sample space S.

2. If A’ is the complement of event A, then P(A’) = 1 – P(A).
3. If A and B are any two events in S, then
P(A∪B) = P(A)+P(B)−P(A∩B)
4. If 𝐴 ⊆ 𝐵 then P(𝐴) ≤ 𝑃(𝐵)
5. For any two events say, A and B, the probability that exactly one
of the events A or B occurs but not both is :
P(A)+P(B)−2P(A∩B)

35
 Derived ….

Example
In a sample of 50 people, 21 had type O blood, 22 had type A
blood, 5 had type B blood, and 2 had type AB blood. Find the
following probabilities.

1. A person has type O blood.

2. A person has type A or type B blood.

3. A person has neither type A nor type O blood.

4. A person does not have type AB blood.

36
 Derived …

Type Frequency (f)

A 22
B 5
AB 2
O 21
Total=n 50
1. A person has type O blood.

2. A person has type A or type B blood.

3. A person has neither type A nor type O blood.

4. A person does not have type AB blood.

37
 Derived…
The counting rules can be combined with the probability rules to
solve many types of probability problems.

Exercise
A box contains 10 Compact Disks, 4 of which are defective. If 4 are
sold at random, find the following probabilities.

a) Exactly two are defective

b) None is defective
c) All are defective
d) At least one is defective

38