Elements of Probability

CMPE 343 Introduction to Probability and Statistics for

Computer Engineers

İnci M. Baytaş

Department of Computer Engineering

Bogazici University

Fall 2023

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 1
Fundamental Definitions

I We record the outcomes of events.

I Outcomes, either numerical or categorical, are called observations.
I Observations are subject to uncertainty.

I The process that generates the observations is referred to as an

experiment.
I Set of all possible outcomes of a statistical experiment is called
sample space, denoted by S.
I For a given experiment, we care about the occurrence of a certain
event, which is a subset of the sample space.
I An event can cover the entire sample space or contains no elements,
null set/event, ∅.

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 2
Fundamental Definitions

I Discrete sample space: S is countable.

I Continuous sample space: S is not countable.
0
I Complement, A of an event A with respect to S is the subset of all
elements of S that are not in A.
I Intersection of two events A and B, denoted by A ∩ B, is the event
containing all the elements common to A and B.
I When two events have no elements in common, they are called
mutually exclusive or disjoint, A ∩ B = ∅.
I Union of two events A ∪ B is the event containing all the elements
that belong to A or B or both.

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 3
Venn Diagrams

Figure: Events represented by various regions.

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 4
Exercise

(a) What are the elements of the following sample spaces?

I set of integers between 1 and 50 divisible by 8

I the set S = {x |2x − 4 ≥ 0 and x < 1}

(b) Describe the sample space S consisting of all points in the first
quadrant inside a circle of radius 3 with a center at the origin.
(c) An experiment involves tossing a pair of dice, one green and one red,
and recording the numbers that come up. If x equals the outcome on
the green die and y the outcome on the red die, describe the sample
space S,
I by listing the elements (x , y )
I by expressing the rule.

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 5
Multiplicative Rule

Multiplicative rule
If an operation can be performed in n1 ways, and if for each of these ways
a second operation can be performed in n2 ways, then the two operations
can be performed in n1 n2 ways.

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 6
Multiplicative Rule

Multiplicative rule
If an operation can be performed in n1 ways, and if for each of these ways
a second operation can be performed in n2 ways, then the two operations
can be performed in n1 n2 ways.

Examples:
I What is the size of the sample space of throwing a pair of dice once?

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 6
Multiplicative Rule

Multiplicative rule
If an operation can be performed in n1 ways, and if for each of these ways
a second operation can be performed in n2 ways, then the two operations
can be performed in n1 n2 ways.

Examples:
I What is the size of the sample space of throwing a pair of dice once?
I How many even 4-digit numbers can be formed from the digits
0, 1, 2, 5, 6, 9 if each digit can be used only once?

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 6
Probability of an Event

Probability
The likelihood of the occurrence of an event with a finite sample space
resulting from a statistical experiment is evaluated using a set of real
numbers, probabilities, ranging from 0 and 1.

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 7
Probability of an Event

Probability
The likelihood of the occurrence of an event with a finite sample space
resulting from a statistical experiment is evaluated using a set of real
numbers, probabilities, ranging from 0 and 1.

I To find the probability of an event A, sum all the probabilities

assigned to the sample points (elements) in A. This sum is called the
probability of A, P (A).

0 ≤ P (A) ≤ 1
P (∅) = 0
P (S) = 1

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 7
Probability of an Event

Mutually exclusive events

If A1 , A2 , A3 , · · · is a sequence of mutually exclusive events, then

P (A1 ∪ A2 ∪ A3 ∪ · · · ) = P (A1 ) + P (A2 ) + P (A3 ) + · · ·

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 8
Probability of an Event

Mutually exclusive events

If A1 , A2 , A3 , · · · is a sequence of mutually exclusive events, then

P (A1 ∪ A2 ∪ A3 ∪ · · · ) = P (A1 ) + P (A2 ) + P (A3 ) + · · ·

Example
I A die is loaded so that an even number is twice as likely to occur as
an odd number. If E is the event that a number less than 4 occurs on
a single toss of the die, find P (E ).

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 8
Probability of an Event

Equally likely events

If an experiment can result in any one of N different equally likely
outcomes, and if exactly n of these outcomes correspond to an event A,
then
n
P (A) =
N

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 9
Probability of an Event

Equally likely events

If an experiment can result in any one of N different equally likely
outcomes, and if exactly n of these outcomes correspond to an event A,
then
n
P (A) =
N
Example
I A statistics class for engineers consists of 25 industrial, 10
mechanical, 10 electrical, and 8 civil engineering students. If a person
is randomly selected by the instructor, find the probability that the
student is (a) an industrial major or (b) civil engineering or electrical
engineering major.

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 9
Additive Rule
Additive rule
If A and B are two events, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 10
Additive Rule
Additive rule
If A and B are two events, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

I If A and B are mutually exclusive, P (A ∪ B) = P (A) + P (B)

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 10
Additive Rule
Additive rule
If A and B are two events, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

I If A and B are mutually exclusive, P (A ∪ B) = P (A) + P (B)

I If collection of events A1 , A2 , · · · , An is a partition of sample space S,
P (A1 ∪ A2 ∪, · · · , ∪An ) = P (A1 ) + P (A2 ) + · · · + P (An ) = P (S) = 1

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 10
Additive Rule
Additive rule
If A and B are two events, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

I If A and B are mutually exclusive, P (A ∪ B) = P (A) + P (B)

I If collection of events A1 , A2 , · · · , An is a partition of sample space S,
P (A1 ∪ A2 ∪, · · · , ∪An ) = P (A1 ) + P (A2 ) + · · · + P (An ) = P (S) = 1

I For three events A, B, C ,

P (A ∪ B ∪ C ) =P (A) + P (B) + P (C )
− P (A ∩ B) − P (A ∩ C ) − P (B ∩ C )
+ P (A ∩ B ∩ C )
İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 10
Additive Rule

Examples
I What is the probability of getting a total of 7 or 11 when a pair of
fair dice is tossed?
I A fair coin is tossed 3 times. E1 corresponds to the event where we
see two heads and one tail. E2 corresponds to seeing the head at the
first two tosses and the tail at the third. Calculate P (E1 ) and P (E2 ).

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 11
Exercise
In a factory, there are day (7 am-3 pm), evening (3 pm-11 pm), and
graveyard shifts (11 pm-7 am). During last year, 300 accidents happened.

If an accident report is selected randomly from the 300 reports,

(a) what is the probability that the accident occurred on the graveyard
shift?

(b) what is the probability that the accident occurred due to human error?

(c) what is the probability that the accident occurred on either the
evening or the graveyard shifts?

(d) what is the probability that the accident occurred due to unsafe
conditions?
İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 12
Conditional Probability

Conditional probability
It is the probability of an event B occurring when it is known that some
event A has occurred,
P (A ∩ B)
P (B|A) = , provided P (A) > 0
P (A)

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 13
Conditional Probability

Conditional probability
It is the probability of an event B occurring when it is known that some
event A has occurred,
P (A ∩ B)
P (B|A) = , provided P (A) > 0
P (A)

Independent events
The occurrence of A has no impact on the odds of occurrence of B.

P (B|A) = P (B)

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 13
Product Rule
I If, in an experiment, the events A and B can both occur, then

P (A ∩ B) = P (A) P (B|A) , provided, P (A) > 0

Since P (A ∩ B) = P (B ∩ A) = P (A) P (B|A) = P (B) P (A|B)

I If two events are independent: P (A ∩ B) = P (A) P (B)
I If, in an experiment, the events A1 , A2 , · · · , Ak can occur, then

P (A1 ∩ A2 ∩ · · · ∩ · · · ∩ Ak ) = P (A1 ) P (A2 |A1 ) P (A3 |A1 ∩ A2 )

P (A4 |A1 ∩ A2 ∩ A3 ) · · · P (Ak |A1 ∩ A2 ∩ · · · ∩ Ak−1 )
I If the events A1 , A2 , · · · , Ak are independent
k
Y
P (A1 ∩ A2 ∩ · · · ∩ · · · ∩ Ak ) = P (Ai )
i=1

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 14
Exercise

1. One bag contains 4 white balls and 3 black balls, and a second bag
contains 3 white balls and 5 black balls. One ball is drawn from the
first bag and placed unseen in the second bag. What is the
probability that a ball now drawn from the second bag is black?

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 15
Exercise

1. One bag contains 4 white balls and 3 black balls, and a second bag
contains 3 white balls and 5 black balls. One ball is drawn from the
first bag and placed unseen in the second bag. What is the
probability that a ball now drawn from the second bag is black?

2. A: the river is polluted

B: a sample of water tested detects pollution
C: fishing is permitted  
0
Assume P (A) = 0.30, P (B|A) = 0.75, P B|A = 0.20,
 0
  0

P (C |A ∩ B) = 0.20, P C |A ∩ B = 0.15, P C |A ∩ B = 0.80,
 0 0

P C |A ∩ B = 0.90
 0

(a) P (A ∩ B ∩ C ) =?, (b) P B ∩ C =?, (c) P (C ) =?, (d) Find
the probability that the river is polluted given that fishing is
permitted and the sample tested did not detect pollution.
İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 15
Contingency Table

I Contains the frequency of occurrences of event combinations in a

large sample.
I Let us say in a course, there are female and male students from two
departments whose numbers are given below.

Gender
Department Male Female Total
CMPE 60 20 80
IE 30 10 40
Total 90 30 120

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 16
Bayes’ Rule
Theorem of total probability/rule of elimination
If the events B1 , B2 , · · · , Bk constitute a partition of the sample space S
such that P (Bi ) 6= 0 for I = 1, 2, · · · , k then for any event A of S,
k
X k
X
P (A) = P (Bi ∩ A) = P (Bi ) P (A|Bi )
i=1 i=1

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 17
Bayes’ Rule

Bayes’s rule
If the events B1 , B2 , · · · , Bk constitute a partition of the sample space S
such that P (Bi ) 6= 0 for I = 1, 2, · · · , k then for any event A in S such
that P (A) 6= 0,

P (Br ∩ A) P (Br ) P (A|Br )

P (Br |A) = Pk = Pk , r = 1, 2, · · · , k
i=1 P (Bi ∩ A) i=1 P (Bi ) P (A|Bi )

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 18
Exercise

In a certain assembly plant, three machines, B1 , B2 , and B3 , make

30%, 45%, and 25%, respectively, of the products. It is known from
experience that 2%, 3%, and 2% of the products made by each machine,
respectively, are defective. Now, suppose that a finished product is
randomly selected. What is the probability that it is defective?

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 19
Exercise

There is a 50-50 chance that the queen carries the hemophilia gene. Each
prince has a 50-50 chance of having hemophilia independently if she is a
carrier. The prince won’t have the disease if the queen is not a carrier.
Suppose the queen has had three princes without the disease. What is the
probability the queen is a carrier? If there is the fourth prince, what is the
probability that he will have hemophilia?

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 20
Exercise

A firm is accustomed to training operators who do specific tasks on a

production line. Those operators who attend the training course are known
to meet their product quotas 90% of the time. New operators who do not
take the training course only meet their quotas 65% of the time. 40% of
new operators attend the class. Given that a new operator meets his
production quota, what is the probability of participating in the program?

İnci M. Baytaş CMPE 343 Introduction to Probability and Statistics for Computer Engineers 21