Applied Statistics

Lecture (3)
Basic Probability Concepts

Dr. Mohamed Gad Lecture 3 – Page 1

 Random Experiments
◼ Random Experiment

◼ Sample Space

Dr. Mohamed Gad Lecture 3 – Page 2

 Sample Space
◼ Deck of Cards
◼ 52 cards in a deck of cards: {c1, …, c52}
◼ Value: {K, Q, J, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1}
◼ spades (♠), hearts (♥), diamonds (♦) and clubs (♣) : {S, H, D,
C}
◼ Red and black cards: {R, B}

◼ Throwing a die (dice)

◼ Value: {1, 2, 3, 4, 5, and 6}
◼ Value odd or even: {O, E}

Dr. Mohamed Gad Lecture 3 – Page 3

 Events

◼ Simple event
◼ Outcome from a sample space with one characteristic
◼ e.g.: A red card from a deck of cards
◼ e.g.: An odd number from a dice throw
◼ Joint event
◼ Involves two outcomes simultaneously
◼ e.g.: A King that is also red from a deck of cards
◼ e.g.: An even number less than 3 from a dice throw
Dr. Mohamed Gad Lecture 3 – Page 4
 Visualizing Events
◼ Venn Diagram
◼ Sample space S
◼ Events A and B

Dr. Mohamed Gad Lecture 3 – Page 5

 Special Events

◼ Impossible event
e.g.: getting “7” from throwing one die
drawing a card “0” from a deck of cards

◼ Complement of event
◼ Complement of event A: all events not in A
◼ Denoted as A’
◼ For example
◼ A: all defective bulbs in a box
◼ A’: all non-defective bulbs in the box

Dr. Mohamed Gad Lecture 3 – Page 6

 Special Events
◼ Complement of an event
◼ Sample space S
◼ Event A A A’
◼ Complement of A → A’ S
◼ eg.: Throwing dice
◼ A = even number = {2, 4, 6}
◼ A’ = odd number = {1, 3, 5}
◼ B = number < 3 = {1, 2}
◼ B’ = number  3 = {3, 4, 5, 6}

Dr. Mohamed Gad Lecture 3 – Page 7

 Special Events

Union: A B
(A or B)
A B
S

Intersection:
A B A B
(A and B)
S

• Mutually Exclusive Events

A B =  = {} A B
S
A and B cannot occur together
Dr. Mohamed Gad Lecture 3 – Page 8
 Special Events
(continued)
◼ Collectively exhaustive events
◼ One of the events must occur
◼ The set of events covers the whole sample space
◼ e.g.: -- A: all the aces; B: all the black cards; C: all the
diamonds; D: all the hearts
◼ Events A, B, C and D are collectively
exhaustive
◼ Events B, C and D are also collectively

exhaustive A

C D
B ♦ ♥

Dr. Mohamed Gad Lecture 3 – Page 9

 Probability

◼ Probability is a numerical 1 Certain

measure of the likelihood S
that an event will occur
◼ Value is between 0 and 1
.5
◼ Sum of the probabilities of
all mutually exclusive and
collectively exhaustive events

is equal to 1
0 Impossible
Dr. Mohamed Gad Lecture 3 – Page 10
 Computing Probabilities
◼ The probability of an event E:

number of event outcomes

P( E ) =
total number of possible outcomes in the sample space
X
=
T e.g. P( ) = 2/36
(There are 2 ways to get one 6 and the other 4)

◼ Each of the outcomes in the sample space is equally

likely to occur (i.e., has the same probability to
occur)
Dr. Mohamed Gad Lecture 3 – Page 11
 Computing probabilities
◼ If an event can occur (happen) in h different
ways out of a total number of n possible
ways, all of which are equally likely, then the
probability of the event is h/n
Example: What is the probability of drawing a
king from a set of 52 playing cards?
Solution:
◼ h = 4 kings in the deck of cards

◼ n = 52

◼ n/h = 4/52 = 1/13

Dr. Mohamed Gad Lecture 3 – Page 12

 Computing Joint Probability

◼ The probability of a joint event, A and B:

P (A and B ) = P (A  B )
number of outcomes in A and B at the same time
=
total number of possible outcomes in sample space

e.g. P (Red Card and King)

2 Red Kings 1
= =
52 Total Number of Cards 26
Dr. Mohamed Gad Lecture 3 – Page 13
 Computing Compound (or
Multiple) Probability
◼ Probability of a compound event, A or B:
P( A or B) = P( A  B)
number of outcomes from either A or B or both
=
total number of outcomes in sample space
E.g. P (Red Card or King)
4 Kings + 26 Red Cards - 2 Red Kings
=
52 total number of cards
28 7
= =
52 13
Dr. Mohamed Gad Lecture 3 – Page 14
 Compound Probability
(Addition Rule)
P(A1 or B1 ) = P(A1) + P(B1) - P(A1 and B1)
P(A1  B1 ) = P(A1) + P(B1) - P(A1  B1)
For Mutually Exclusive Events (Events that cannot
occur at the same time):
P(A or B) = P(A) + P(B)

Dr. Mohamed Gad Lecture 3 – Page 15

 Computing probabilities

◼ Complement of an Event, P(A’)

A A ' =S
P (A ) + P (A ') − P (A  A ') = P (S )
A and A ' are mutually exclusive
P (A  A ') = P ( ) = 0
P (A ) + P (A ') = P (S ) = 1
P (A ') = 1 − P (A )

Dr. Mohamed Gad Lecture 3 – Page 16

 Conditional Probability

◼ The probability of event A given that event B

has occurred:
P( A and B)
P( A | B) =
P( B)
E.g.
P (Red Card given that it is a King)
2 Red Kings 2 / 52 1
= = =
4 Kings 4 / 52 2
Dr. Mohamed Gad Lecture 3 – Page 17
 Conditional Probability and
Statistical Independence

◼ Conditional probability:
P( A and B)
P( A | B) =
P( B)
◼ Multiplication rule:

P( A and B) = P( A | B) P( B)
= P( B | A) P( A)
Dr. Mohamed Gad Lecture 3 – Page 18
 Conditional Probability and
Statistical Independence
(continued)

◼ Events A and B are independent when the

probability of one event, A, is not affected by
another event, B
◼ Events A and B are independent if
P( A | B) = P( A)
or P( B | A) = P( B)
or P( A and B) = P( A) P( B)

Dr. Mohamed Gad Lecture 3 – Page 19

 Example 1
◼ A water treatment plant may fail for two reasons: inadequacy of
materials (event A) or mechanical failure (event B).
If P(A) = 2 P(B), P(A | B) = 0.8 and the probability of failure of
the treatment plant equals 0.001 , what is the probability that a
mechanical failure occurs? What is the probability of a failure
due to inadequate materials?

The probability of failure P (A U B) may be written as:

P (A U B) = P (A) + P (B) – P (A ∩ B)
= 2 P (B) + P (B) – P (B) P (A | B)
= 3 P (B) – 0.8P (B)
0.001 = 3 P (B) – 0.8P (B)
→ P (B) = 0.00045
→ P (A) = 0.0009

Dr. Mohamed Gad Lecture 3 – Page 20

 Example 2
◼ Each of two pumps Q1 and Q2 may not operate. We may observe
one of the outcomes of the following set: {(ƒ, ƒ), (ƒ, o), (o, ƒ), (o, o)}.
The notation (ƒ, o) means that pump Q1 fails and pump Q2 operates.
The sample space has four elements. From experience it is known
that:
◼ P ((ƒ, ƒ)) = 0.1, P ((ƒ, o )) = 0.2 A B
◼ P ((o, ƒ)) = 0.3, P ((o, o )) = 0.4
(o,f) (o,o) (f,o)
◼ Let
0.3 0.4 0.2
◼ A: Q1 operates
◼ B: Q2 operates
◼ C: at least one of the pumps operates (f,f) 0.1
◼ Compute P(A), P(B), P(A ∩ B), P(C)

Dr. Mohamed Gad Lecture 3 – Page 21

 Example 2 (Cont.)
A B
◼ P ( A ) = 0.7
◼ P ( B ) = 0.6 (o,f) (o,o) (f,o)
◼ P ( A ∩ B ) = P (( o , o )) = 0.4 0.3 0.4 0.2

◼ P(C)=(AUB) (f,f) 0.1

=P(A)+P(B)–P(A∩B)
= 0.9

Dr. Mohamed Gad Lecture 3 – Page 22

 Total Probability
◼ Total Probability Rule (multiple events)

E1 E2 Ek

B
Dr. Mohamed Gad Lecture 3 – Page 23
 Example 3
◼ The demand of a water supply system can be low (event L) ,
moderate (event M) or high (event H) with known probabilities
P(L) = 0.1, P(M) = 0.7 and P(H) = 0.2. Failure of the system (event F)
can only occur if a certain pump fails to function. From past
experience the following conditional probabilities are known:
P(F | L) = 0.05, P(F | M) = 0.10 and P(F | H) = 0.25. What is the
probability of a pump failure P(F).

First we observe that the events L, M and H are clearly mutually

exclusive and that their union is just the sure event S. Hence
P ( F ) = P (( L ∩ F ) U ( M ∩ F ) U ( H ∩ F ))
= P (L ∩ F) + P (M ∩ F) + P (H ∩ F)
Using the definition of conditional probability, we get
P ( F ) = P (L) P (F | L) + P (M) P (F | M) + P (H) P (F | H)
= 0.1  0.05 + 0.7  0.10 + 0.2  0.25
= 0.125
Dr. Mohamed Gad Lecture 3 – Page 24
 Bayes’s Theorem

◼ B1, B2, …, Bk are mutually exclusive and

collectively exhaustive
◼ Suppose it is known that event A has
occurred
◼ What is the (conditional) probability that
event Bi has occurred?
B1 B2 Bk

Dr. Mohamed Gad Lecture 3 – Page 25

 Bayes’s Theorem
P ( B i and A )
P (Bi | A ) =
P (A )
P (A | Bi ) P (Bi )
=
P ( A | B1 ) P ( B1 ) + • • • + P ( A | B k ) P ( B k )
B1 B2 Bk

Dr. Mohamed Gad Lecture 3 – Page 26

 Example 3(b)
◼ In example 3, If a failure occurs, what is the
probability that it occurred during moderate demand?

P (F | M ) P (M )
P (M / F ) =
P (F | L )P (L ) + P (F | M )P (M ) + P (F | H )P (H )

P (F | M ) P (M ) (0.1)(0.7) 0.07
= = = = 0.28
P (F ) 0.125 0.25

Dr. Mohamed Gad Lecture 3 – Page 27

 Example 4
Fifty percent of graduating engineers in a certain year worked as
water resources (WR) managers. Out of those who worked as WR
managers, 40% had a Masters degree. Ten percent of those
graduating engineers who did not work as WR managers had a
Masters degree. What is the probability that a randomly selected
engineer who has a Masters degree is a WR manager?

P (WR ) = 0.5, P (M /WR ) = 0.4, P (M /WR ) = 0.1, P (WR / M ) = ?

P (M |WR ) P (WR )
P (WR / M ) =
P (M |WR )P (WR ) + P (M |WR )P (WR )
(0.4)(0.5) 0.2
= = = 0.8
(0.4)(0.5) + (0.1)(0.5) 0.25
Dr. Mohamed Gad Lecture 3 – Page 28