Chapter 2

Probability

Lecture 1:

Random Experiment: an experiment whose outcome cannot be

predicted with certainty.
Sample Space: The set of all possible outcomes of a random
experiment. The sample space is denoted as S.

Example: toss a coin twice. Sample space: S={HH, HT, TH, TT}
Example: Machine life time S ={x: 0 ≤x<∞}

Event: An event is a subset of the sample space.

Events ≡ Sets

Set Operations
 union:
AUB = {x | x  A or x  B};
(either or both occur)
 intersection:

A∩B = {x | x  A and x  B};

(both must occur)
  complementation:

A  x : x  S , x  A
(the event did not occur)
 Difference:


A \ B  A  B  x : x  A and x  B 

 A  ( B  C )  ( A  B)  ( A  C )
 A  ( B  C )  ( A  B)  ( A  C )

Definition of Probability:
(1) Relative Frequency Probability
(2) Classical Probability

(1) Relative Frequency Probability

Example: Coin toss probability
Number of Number of Probability to get
tosses heads heads
4 1 0.25
100 56 0.56
1000 510 0.510
10000 4988 0.4988

Notice that for 10000 flips, the probability is close to 0.5

As the number of trials increases, the relative frequency fluctuates less
and gets closer to the theoretical probability 0.5

Law of Large Numbers: The true probability of an event (A) is

estimated by the relative frequency with which the event occurs in a
long series of trials.

(2) Classical Probability

This definition is for equally likely sample spaces
no. of outcomes in the event A n( A)
P( A)  
no. of outcomes in S n( S )
Example: A fair die is thrown, find the probability that the face on the
die is even?
A: be the event that the face is even
A = { 2, 4, 6}
n( A) 3 1
P( A)   
n( S ) 6 2
Assumptions of Probability
(1) P(A) ≥0, for any event A
(2) P(S) = 1
(3) 0<= P(A)<= 1
(4) For two events A and B with A∩B=ø (A and B are called disjoint or
mutually exclusive events)

P( A  B)  P( A)  P( B)
Rules of Probability:

1-
P( A)  1  P( A)
Proof:

Note: A A  S
P( A)  P( A)  P( S )
since A and A are disjoint , then
P( A)  P( A)  1

It follows:
P( A)  1  P( A)
Example: When throwing a fair die, P(not 6) = 1 – 1/6 = 5/6.

2- Monotonicity

If A  B, then P( A)  P( B)
3-
 P( A  B)  P( A)  P( A  B)
4- Addition Rule
P( A  B)  P( A)  P( B)  P( A  B) .
Proof: from Rule 3
P( A  B c )  P( A)  P( A  B) ........(1)
Note:

A  B  ( A  B)  B
P( A  B)  P( A  B)  P( B) .........(2)
because A  B and B are disjoint
From (1) and (2), it follows
P( A  B)  P( A)  P( B)  P( A  B)
Example: Consider the following grades obtained in M235 course. Of
200 students taking the course, 160 passed the midterm exam and 140
passed the final exam; 124 students passed both exams. After
reviewing the grades, the professor of the course decides to give a
passing grade to any student who passed at least one of the two exams.
That is, a passing grade will be given to any student who passed the
midterm, to any student who passed the final, and to any student who
passed both exams. What is the probability of receiving a passing grade
in this course?

Solution
Let A= event of passing the midterm exam, and
B= event of passing the final exam, then
A∩B= event of passing both exams
The given relative frequency information leads to the following event
probabilities:
P(A)=160 / 200 = 0.80
P(B)= 140 / 200 = 0.70
P(A∩B) = 124 / 200 = 0.62
Need P(AUB)?
P( A  B)  P( A)  P( B)  P( A  B)
P(AUB) = 0.8 + 0.7 – 0.62 = 0.88

Counting Techniques

Count the number of ways that a phenomenon can occur?

(1) Multiplication Rule
n outcomes of experiment 1
m outcomes of experiment 2
then there are n×m outcomes of the two experiment.
Example: you have 3 shirts and 4 pants.
That means 3×4=12 different outfits.
 The Multiplication Principle. If there are:
 n1 outcomes of experiment 1

 n2 outcomes of experiment 2

 ...

 nm outcomes of experiment m

then there are n1 × n2 × ... × nm outcomes of the m experiments

Example : A test consists of 5 multiple-choice questions. Each

question has 4 possible answers. There are
4×4×4×4×4 =45=1024
possible answers

Example: How many possible license plates could be stamped if each

license plate were required to have exactly 3 letters and 4 numbers?

Solution.

the Multiplication Principle tells us that there are:

Again, that is:

(26 × 26 × 26) × (10 × 10 × 10 × 10)
or 175,760,000 possible license plates.
 (2) Permutations
Example 1: How many 3 digit numbers can you make using
the digits 1, 2 and 3 ?

Method (1) Using a tree diagram.

We can make 6 numbers using 3 digits

Method (2) counting:

look at the tree diagram above.
We have 3 choices for the first digit, 2 choices for the second
digit and 1 choice for the third digit.
Using the multiplication rule, we can say:
The total number of 3-digit numbers is given by
3*2*1=6
There is a special notation for the product 3 * 2 * 1 = 3! and it
is read 3 factorial.

A permutation is an ordered arrangement of n different objects.

The number of permutations of n different elements is n!, where
 n! n  (n  1)  (n  2)      2 1
Example: In how many ways can 7 different books be arranged on a
shelf?

Solution:
We have seven positions that we can fill with seven books. There are 7
possible books for the first position, 6 possible books for the second
position, five possible books for the third position, and so on. The
Multiplication Principle tells us therefore that the books can be
arranged in:
7 × 6 × 5 × 4 × 3 × 2 × 1= 5,040 ways.

Example: If four people A, B, C, and D are randomly arranged in a

line, how many different arrangements such that A and B are next to
each other?
Solution:

2(3!)
Result: The number of permutations of n distinct objects taken r at a
time is
n!
P 
n

(n  r )!
r

Example: How many 3 letter words can we make with the letters in
the word MATH?
Solution: There are 4 letters in the word MATH and making 3 letter
words is similar to arranging these 3 letters and order is important
The number of words is given by
 4!
P  4
 24
(4  3)!
3

Example: 4 names are drawn from among the 24 members of a club

for the offices of president, vice president, treasurer, and secretary.
How many different ways can be done?
Solution 1
24(23)(22) (21)=255024
Solution 2
24!
P  24
 24(23)(22)(21)  255024
(24  4)!
4

Lecture 2:
(3) Combinations
If you are drawing k elements from a set of n elements without regard
to the order of the k elements, the number of different results
(combinations) is

n n!

k 
  k!( n  k )!
 
It is often called "n choose k

Combination Rules:
𝑛 𝑛 𝑛
( ) = 1, ( )=( )
𝑛 𝑘 𝑛−𝑘

Example: In how many ways you can pick a team of 3 people from a
group of 10?10∗9∗8∗7!
3!7!
10∗9∗8
= 3∗2∗1
Solution:
10  10!
    120
 3  3!(7)!
Example: There are 5 boys and 4 girls in a class. A committee of 5 is
to be selected such that there are 3 boys and 2 girls in the committee.
In how many ways the committee can be selected?
Solution:
Number of ways to select 3 boys out of 5 is (53)
Number of ways to select two girls out of 4 is (42),
Then number of ways to select the committee is

 5  4  5! 4!
     10  6  60
 3  2  3!2! 2!2!
So, in Mathematics we use more precise language:
If the order doesn't matter, it is a Combination.
If the order does matter it is a Permutation.

Multinomial Coefficients:
Theorem: The number of ways in which a set of n distinct objects can
be portioned into k subsets with n1 objects in the first subset, n2 in the
second subset, …, nk in the k-th subset is
 n  n
  
 n1 , n2 ,..., nk  n1!.n2!...nk !
Example: An engineering office has 10 engineers, if the office policy
is to have 5 engineers for sketching , 2 of the engineers working full
time at the office, and 3 on reserve at the office. How many different
divisions of the 10 engineers into 3 groups are possible?
Solution-1
 10  10!
    2520
 5,2,3  5!.2!.3!

Solution-2
10  5  3 
     2520
 5  2  3 

Probability Using Counting Techniques

(Combinatorial Probability):

Remember !
Classical Probability
This definition is for equally likely sample spaces
no. of outcomes in the event A n( A)
P( A)  
no. of outcomes in S n( S )
Example: the digits 1, 2, 3 are permuted randomly, what is the
probability that 1 and 2 are next to each other?

Method 1- By inspection. Using a tree diagram.

 4 permutations out of 6 permutations the digits 1 and 2 are next to
each other.

Now, let A the event that the digits 1 and 2 next to each other, then the
number of outcomes in A is 4.
n( A) 4 2
P( A)   
n( S ) 6 3
Method-2: Using counting techniques
n( A) 2(2!) 2
P( A)   
n( S ) 3! 3
Example: If four people A, B, C, and D are randomly arranged in a
line, what is the probability that A and B are next to each other?
Solution:
2(3! )
4!
Example: A committee of 5 is to be chosen from a group of 6 men
and 4 women, the selection is made randomly, what is the probability
that the committee will consist of 3 men and 2 women?
Solution:
Note in the committee the order of the members is not important, so
Number of ways to choose the men is (63), the number of ways to
choose 2 women is (42), and number of ways to choose the committee
is (10
5
), so the
(6)(4 )
P(choosing the committee) = 3 2
10
( )
5

Example: If the numbers 1, 2, 3, … , 10 are permuted randomly.

What is the probability that at least one of the numbers 1 and 2 are in
its proper place- that is, that 1 is first or 2 is second or both?

Solution:
Let A be the event 1 is first and B the event 2 is second. Need
P( A  B) ?
P( A  B)  P( A)  P(B)  P( A  B)
9! 9! 8!
P( A  B)   
10! 10! 10!

Conditional Probability
Definition: The conditional probability of the A given the B denoted
by P(A|B) and defined by
P( A  B)
P( A | B)  , P( B)  0
P( B)
The Venn Diagram below illustrates P(A), P(B), and P(A and B).
Example: A research study investigating the relationship between
smoking and heart disease in a sample of 1000 men provided the
following table
Smoker Nonsmoker Totals
(S)
Heart Disease (H) 100 80 180
No Heart Disease 200 620 820
Totals 300 700 1000

-The probability that a selected person is smoker and has heart disease
is
100/1000 = 0.1
-The probability that a selected person has a heart disease if he is
smoker is
P( H  S ) 100 / 1000
P( H | S )    1/ 3
P( S ) 300 / 1000
Lecture 3:

Three Rules Involves Conditional Probability

(1) Multiplication Rule of Probability
(2) Law of Total Probability (LTP)
(3) Bayes Theorem

1- Multiplication Rule:
 P(A1∩A2) = P(A1)P(A2|A1)
 P(A1∩A2∩A3) = P(A1)P(A2|A1)P(A3|A1∩A2)

Example: The soil in certain locality is likely to settle when subject

to a concentrated load; from experience, the estimated probability
that one of the two footings of a rigid structure show in the figure
below will settle is 0.1 while the probability that a footing will
settle given that the other one has settled is 0.7.

Figure: Rigid frame with settlement problem

Determine the probability of

(a) Settlement;
(b) Differential settlement

Solution:
For (a), if we refer to the settlement of the two footings as A and
B, then the possible events regarding settlement of the two footings
are:
1- both A and B settle (A∩B)
2- A does not settle but B settles (𝐴̅ ∩ 𝐵)
3- A settles while B does not settle (𝐴 ∩ 𝐵̅)

4- either A nor B settles (𝐴̅ ∩ 𝐵̅). Thus

P(A⋃B)= P(A)+P(B) –P(A∩B)

 = 0.1 + 0.1 – P(A|B)P(B)=0.1 + 0.1-(0.7*0.1)=0.13

For (b), the event of differential settlement is a compound event,

that is, event 𝐴̅ ∩ 𝐵 and event 𝐴 ∩ 𝐵̅. Both these events are mutually
exclusive. Therefore, the required probability is
P((𝐴̅ ∩ 𝐵)⋃(𝐴 ∩ 𝐵̅))= P(𝐴̅ ∩ 𝐵) + P(𝐴 ∩ 𝐵̅)
= P(𝐴̅|𝐵)𝑃(𝐵) + P(𝐵̅|𝐴)𝑃(𝐴)
= [1-P(A|B)]P(B) + [1- P(B|A)] P(A)
= [1 -0.7]*0.1 + [1- 0.7]* 0.1 = 0.06

(2) Law of Total Probability (LTP)

A1 B S
A2 A3 …. An
.

Suppose A1 ,A2,…,An are mutually exclusive events form a partition

of the sample space S, then for any event B if its occurrence is
accompanied with the Ai’s, then
n
P( B)   P( B | Ai ) P( Ai )
i 1
(3) Bayes Theorem
If the events A1 , A2,…, An form a partition of the sample space S,
then for any event B and Ai is given by
P( B | Ai ) P( Ai )
P( Ai | B)  n

 P( B | A ) P( A )
i 1
i i

Example: A manufacturing firm receives 65% of its parts from one

supplier and 35% from a second supplier. The quality of the
 purchased parts varies with the supplier. The following table shows
the percentages of good and defective parts from the two suppliers

Supplier Good Parts (%) Defective Parts (%)

1 98 2
2 95 5
(a) What is the probability that the manufacturing firm had a
machine breakdown due to defective part?

Solution: Let A1: the event that a part comes from the first supplier,
P(A1 )= 0.65
A2: the event that a part comes from the second supplier,
P(A2)=0.35
G: the event that the part is good
D: the event that the part is defective
P(G|A1) = 0.98, P(D|A1) = 0.02, P(G|A2) = 0.95, P(D|A2) = 0.05
Need P(D) ?
Using the Law of Total Probability
P(D) = P(D|A1)P(A1)+P(D|A2)P(A2)
P(D)=0.02(0.65)+0.05(0.35)=0.0305

(b) Suppose that the manufacturing firm has just had a machine
breakdown due to defective part, and wants to determine the
probability that the part come from the first supplier
Solution: Need P(A1|D) ?
Using Bayes Theorem
P( D | A1 ) P( A1 ) 0.02(0.65)
P( A1 | D)    0.426
P( D | A1 ) P( A1 )  P( D | A1 ) P( A1 ) 0.02(0.65)  0.05(0.35)

Example: A computer center has three printers A1, A2, and A3 , which
print at different speeds. Programs are routed to the first available
printer. The probability that a program is routed to printers A1, A2, and
A3 are 0.6, 0.3 and 0.1 respectively. Occasionally a printer will jam
and destroy a printout. The probability that printers A1, A2, and A3 will
jam are 0.01, 0.05 and 0.04, respectively. Your program is destroyed
when a printer jams. What is the probability that printer A1 is
involved? printer A2 is involved? printer A3 is involved?
Solution: Let
A1: event programs routed printer A1, P(A1) = 0.6
A2: event programs routed printer A2, P(A2) = 0.3
A3: event programs routed printer A3, P(A3) = 0.1
J: event printer will jam and destroy the printout
P(J| A1) = 0.01, P(J|A2) = 0.05, P(J|A3) = 0.04
Need P(A1|J), P(A2|J), and P(A3|J) ?

P( J | A1 ) P( A1 )
P( A1 | J ) 
P( J | A1 ) P( A1 )  P( J | A2 ) P( A2 )  P( J | A3 ) P( A3 )
0.01  0.6
P( A1 | J )   0.24
0.01  0.6  0.05  0.3  0.04  0.1

P( J | A2 ) P( A2 )
P( A2 | J ) 
P( J | A1 ) P( A1 )  P( J | A2 ) P( A2 )  P( J | A3 ) P( A3 )
0.05  0.3
P( A2 | J )   0.6
0.01 0.6  0.05  0.3  0.04  0.1
P( J | A3 ) P( A3 )
P( A3 | J ) 
P( J | A1 ) P( A1 )  P( J | A2 ) P( A2 )  P( J | A3 ) P( A3 )
0.04  0.1
P( A3 | J )   0.16
0.01  0.6  0.05  0.3  0.04  0.1
 Independence of Events
If it happens that P(A|B) = P(A), then the information that B has
occur does nothing to change our view of the probability that A has
occurred. The events are called independent in this case.
Note:
P( A  B)
P( A | B)  , P( B)  0
P( B) ,
This implies P(A∩B) = P(B)P(A|B)
P(A∩B) = P(A)P(B) by independence

Definition: Events A and B are called independent if

P(A∩B) = P(A)P(B).
Result: If events A and B are independent, then
c
(a) A and B are independent
c
(b) A and B are independent
c c
(c) A and B are independent

Definition: Three events A ,B, and C are independent if the

following hold:
P(A∩B) = P(A)P(B),
P(A∩C) = P(A)P(C),
P(B∩C) = P(B)P(C), and
P(A∩B∩C) = P(A)P(B)P(C).

Example: Toss a coin 3 times

S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
Let A : event of getting 2 or more heads, then
A = {HHH, HHT, HTH, THH}
so P(A) = 4 /8 = ½

Let B : event first toss came up head

B = {HHH, HHT, HTH, HTT}
Then P(B) = 4/8 = ½
Is A and B are independent?
A∩B = {HHH, HHT, HTH}
So P(A∩B)= 3/8
3 1 1 1
P( A  B)   P( A)  P( B)   
Note 8 2 2 4
Then the events A and B are not independent

Example: Toss coin n times. What is the probability of getting all

heads?
Solution: Let, A1 : the event that the result of the first toss is head.
A2 : the event that the result of the second toss is head. …, An : the
event that the result of the n-th toss is head
P(A1) = P(A2) = … = P(An) = 1/2
n
1
P( A1  A2  ...  An )  P( A1 ).P( A2 )...P( An )   
2
by independence.

Lecture 4:

Example: A box contains 6 blue balls and 4 white balls. Two balls
are drawn, one after the other. Are the events B1 = “first ball is
blue” and W2 = “second ball is white”. Are B1 and W2 independent?
Solution:
Without replacement: P(W2|B1) = 4/9
P(W2) = P(B1)P(W2|B1)+P(W1)P(W2|W1) =
6 4 4 3 4
   
10 9 10 9 10
 4 4

Since 9 10 , B1 and W2 are not independent.
4
With replacement: P(W2|B1) = 10
P(W2) = P(B1)P(W2|B1)+P(W1)P(W2|W1) =
6 4 4 4 4
   
10 10 10 10 10
So, B1 and W2 are independent.

Example: An engineering system has two components (A and B)

that function independently. Suppose that the probability that
component A function is 0.9, and the probability that component B
function is 0.8. Find the probability that the system function in the
following cases?
Case (1): The components are connected in series, that is, the
system works only when both components function
(Series System)
P(A)= 0.9 P(B)=0.8

A B
P(A∩B) = P(A). P(B) = 0.9(0.8) = 0.72

Case (2): The components are connected in parallel, that is, the
system works if either components functions.
(Parallel System)
P(A)=0.9
A

P(B)=0.8
Solution:
P( A  B)  P( A)  P( B)  P( A  B)
P( A  B)  P( A)  P( B)  P( A) P( B) by independence
P( A  B)  0.9  0.8 - 0.9(0.8)  0.98

or
P( A  B)  1  P( A  B)
Using De Morgan’s Laws
A B  A B
then
P ( A  B )  1  P ( A  B)
P( A  B)  1  P( A)  P( B)  1  0.1(0.2)  0.98

Communication Systems:
Components connected in series

Components connected in parallel

Example:
Consider the following communication system:
P(A)=0.9 P(B)=0.8
 A B

1 2

C D

P(C)=0.85 P(D)=0.95
What is the probability that the system function?
P((A∩B)⋃(C∩D)) = P(A∩B)+P(C∩D) - P(A∩B∩C∩D)
= 0.9(0.8)+0.85(0.95)-0.9(0.8)(0.85)(0.95)
= 0.9461

Example: A and B play 12 games of chess of which 6 are won by A, 4 won by

B, and 2 end in a tie. They agree to play a tournament consisting of 3 games.
Find the probability that
(a) A wins the three games, (b) two games end in a tie (c) B wins
at least one game.
Solution:
Let A1, A2, A3, denote the events “A wins” in 1st, 2nd, and 3rd games respectively, B1, B2, B3,
denote the events “B wins” in 1st, 2nd, and 3rd games respectively, and T1, T2, T3, denote
the events “there is tie” in 1st, 2nd, and 3rd games respectively.
P(A wins any one game)= 6/12=1/2, P(B wins any one game) = 4/12 = 1/3, and
P(any one game ends in a tie) = 2/12 = 1/6.

(a) P(A wins all three games)= P(A1 ∩ A2 ∩ A3) = P(A1) P(A2) P(A3) = (1/2)(1/2)(1/2)=1/8
since all the games are independent of each other.

(b) P(2 games end in a tie) = P( 1st and 2nd or 1st and 3rd or 2nd and 3rd games end in a
tie) = P(T1∩ T2 ∩Tc3) + P(T1 ∩Tc2 ∩T3) + (Tc1∩ T2 ∩T3)
= P(T1) P(T2) P(Tc3) + P(T1) P(Tc2)P(T3) + P(Tc1)P(T2)P(T3)
=(1/6)(1/6)(5/6)+(1/6)(5/6)(1/6) + (5/6)(1/6)(1/6)=5/72.

(c) P(B wins at least one game)=1 – P(B wins no game)=1 – P(Bc1 ∩ Bc2 ∩ Bc3)
= 1 – P(Bc1) P(Bc2) P(Bc3)=1 –(8/12)(8/12)(8/12)=19/27