Probability Rules

What is the probability (likelihood)

an event will occur?

Take a chance on me
Probability model
A model is a simplification of something in the
real world.
A probability model is a mathematical
representation of a random phenomenon.
It is defined by
● its sample space and events
● beliefs about the probabilities associated
with each event. 2
Sample space
A sample space (denoted S) of a random process is the set
of all possible non-overlapping outcomes.
Example 1: What is the sample space of rolling a six-sided
die?
S = {1, 2, 3, 4, 5, 6}
Example 2: What is the sample space of tossing two
coins?
S = {HH, HT, TT, TH}
3
Probability of an event
An event is any subset of the sample
space. Events are usually designated
by uppercase letters, like A, B, C, etc.

P(A) denotes the probability of a

specific event A occurring.
4
Probability model - Example
Suppose we flip a fair coin and observe the side
that faces up. Let
H = a heads comes up.
T = a tails comes up.
The sample space is {H, T}.
Because we expect both side to face up with equal
probability, we model this as
P(H) = ½
P(T) = ½ Note: P(H) + P(T) = 1 5
Probability of an event
The probability of an event A, P(A), is the
number between 0 and 1, inclusive, that
describes the proportion of times the outcome
would occur in a very long series of repetitions.

That is, 0 ≤ P(A) ≤ 1 for every event A.

Also, Σpi = 1
6
Equally likely case
If the sample space consists of N possible and
equally likely outcomes, then the probability of
any event A is

P(A) = (number of elements in A) / N

7
Equally likely case
If we roll a pair of dice one time, what is the
probability the sum of the two is 7?
Sample space has 6*6 = 36 elements
Let C = event that sum is 7
= {(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)}
Number of elements in C = 6
P(C) = 6/36
8
Odds of an event
Odds = P(A)/P(A’)

For example, “the odds are 3 to 1 against a

horse” means that the horse has a 25%
probability of winning.

9
Rules of complementary events
The probability that an event occurs and the probability that
it does not occur always add to 1 or 100%. The probability
that an event does not occur is 1 minus the probability that
the event does occur.
Example 1: When rolling a die, the probability of not getting
a 1 is ⅚, or 83.3%.

Example 2: If an event occurs in, say, 70% of all trials, it

fails to occur in the other 30%.
10
Complements: The probability of “at least one”

To find the probability of at least one of

something, calculate the probability of none
and then subtract that result from 1.
That is, P(at least one) = 1 – P(none)

11
Complements: The probability of “at least one”

The complement of getting at least one item of

a particular type is that you get no items of that
type.
Example: When tossing 3 coins, what is the
probability of getting at least one tail?

12
Complements: The probability of “at least one”

The complement of getting at least one item of

a particular type is that you get no items of that
type.
Example: When tossing 3 coins, what is the
probability of getting at least one tail?
Answer: P(at least one tail) = 1 - P(no tails)
= 1 - (1/2)3 = 1 - 1/8 = 7/8 or 87.5%.
13
Rules of complementary events
P(A) + P(Ac) = 1

P(Ac) = 1 - P(A)

P(A) = 1 - P(Ac)

14
Recap: Basic rules of probability
● A probability of any event is a number between
0 and 1, i.e., 0 ≤ P(A) ≤ 1
* The probability of an event that never
occurs is 0, e.g., P(∅)=0.
* The probability of an event that is certain to
occur is 1.
● All possible outcomes together must have
probability 1, i.e, P(Ω) = 1 where Ω is the
universal set, Σp = 1. 15
● The probability that an event does not occur is 1
minus the probability that the event does occur.
● If all outcomes in a sample space are equally
likely, the probability that event A occurs is
P(A) = number of outcomes corresponding to A
total number of possible outcomes in S
● The Law of Large Numbers says that the more
times we repeat a random process, the average
results will approach the expected value or true
relative (theoretical) frequency.
16
Basic set theory
Ø and { } refer to the empty set.

Ω is the universal set, i.e., the set that contains

all possible elements. The set of outcomes
must be
● mutually exclusive,
● collectively exhaustive, and
● at the “right” level of granularity.
17
Basic set theory with Venn diagrams
A U B refers to the union of set A and B. That
is, all elements in A or B or both.
x ∈ AUB ⇔ x ∈ A or x ∈ B or both

18
Basic set theory with Venn diagrams
A ∩ B refers to the intersection of A and B.
That is, all elements in both A and B.
x ∈ A ∩ B ⇔ x ∈ A and x ∈ B

19
Basic set theory with Venn diagrams
A＇or Ac or A is the complement of A. That is,
all elements in Ω that are not in A.
x ∈ A＇⇔ x ∉ A

20
Basic set theory with Venn diagrams
A ⊂ B means A is a subset of B. That is, every
element of A is also in B.
A ⊂ B: x ∈ A ⇒ x ∈ B

P(A) ≤ P(B)

P(A∩B)=P(A)

21
Basic set theory with Venn diagrams
Disjoint sets (mutually exclusive sets) have no
common elements, i.e., A ∩ B = Ø.

P(AUB) = P(A)+P(B)

22
Intersection (A ∩ B)
The event A ∩ B (“A and B”) consists of all
those outcomes in both A and B.
The probability that events A and B both occur,
sometimes called the joint probability, is the
probability of the intersection of A and B,
denoted P(A ∩ B).

23
Venn diagrams
Suppose 35% of the population reads USA
Today, 20% read the NY Times and 5% read
both.

24
Intersection (A ∩ B)
Suppose that we draw one card from a deck. Let
A = draw a face card
B = draw a heart
What is the probability that the card drawn is a face card
and a heart?
A ∩ B = draw a face card and a heart (J❤ or Q❤ or K❤).
P(face card and heart) = P(A ∩ B) = 3/52 since three cards
are both face cards and hearts.
25
Venn diagram
In a Venn diagram, the area inside the rectangle represents
the sample space and each circle represents an event.

To find probabilities, we
can count outcomes in
the appropriate region.

26
Mutually exclusive (disjoint) sets
If events A and B cannot happen together or
have no outcomes in common, we say that they
are mutually exclusive or disjoint and that
P(A ∩ B) = 0 and P(A|B) = 0.
Example 1: Tossing a single coin and getting heads and
tails.

Example 2: A and Ac are disjoint. It is impossible for an

event and its complement to occur at the same time. 27
Two-way table
When two variables are categorical, the raw
data can be summarized in a two-way table,
also called a contingency table, that gives
counts of observations for each combination of
values of the two categorical variables.

28
Two-way table: Example
Here is a two-way table classifying a sample of HS
students by gender and whether they right- or left-handed.
Two variables: Gender (column) and handedness (row).
Gender

Male Female Total

Right 52 36 88
Dominant
hand? Left 8 4 12

Total 60 40 100 29
 Gender

Male Female Total

Right 52 36 88
Dominant
hand? Left 8 4 12

Total 60 40 100

Suppose a student is selected at random. Find the probability that:

1. P(Male ∩ Left-handed) = 8/100 = 0.08 or 8%
2. P(Male) = 60/100 = 0.60 or 60%
3. P(Male’) = 1-60/100= 40/100 Male’ is the complement of Male
4. P(Left-handed) = 12/100 = 0.12 or 12%
5. P(Male U Left-handed) = (52+8+4)/100 = 0.64 or 64%
6. P(Male | Left-handed) = 8/12 = 0.67 or 67%
7. P(Left-handed | Male) = 8/60 = 0.13 or 13% 30
Joint distributions
We can use the counts found in a two-way table to
calculate each cell’s percentage of the total. The collection
of these proportions is called the joint relative frequency
distribution of the two categorical variables.
The percent of
left-handed males is Gender
P(Male ∩ Left-handed) =
52/100 = 0.52 or 52% Male Female Total

Dominant Right 52% 36% 88%

hand?
Left 8% 4% 12%

Total 60% 40% 100% 31

 Marginal distributions
When we want to examine one of our categorical variables
by itself, we can look at its marginal distribution. These are
the row and column totals for a two-way table: one
for each categorical variable. Joint and marginal distributions can
be expressed in either probabilities
or percentages.

Marginal distribution by gender Marginal distribution by gender

Male Female Male Female

Proportion .60 .40 Percent 60% 40%

P(Male) = 0.60 32
Marginal distributions
When we want to examine one of our categorical variables
by itself, we can look at its marginal distribution. These are
the row and column totals for a two-way table: one
for each categorical variable.

Marginal distribution by handedness

Probability or Percent

Right 0.88 or 88%

Left 0.12 or 12%

33
Union (A U B)
The event A U B (“A or B”) consists of all
outcomes in event A, event B or both.
Example: Suppose we draw one card from a well-shuffled
deck. Let
A = draw a face card
B = draw a heart
Then, A U B = draw a face card or a heart.

34
General addition rule
What is P(A U B)?

Be careful. We
don’t want to count
an outcome more
than once.

35
General addition rule
P(A U B) = P(A) + P(B) - P(A∩B).

P(A∩BC) P(AC∩B)
P(A∩B)

P(AC∩BC)
36
Notation
P(A and B) = P(A ∩ B)
= P(event A occurs in the first trial
and event B occurs in the second
trial)

37
General addition rule - example
The probability of a Coppell S = Owns a skateboard
teenager owning a skateboard B = Owns a bicycle
is 0.40, of owning a bicycle is P(S) = 0.40
0.80 and of owning both is P(B) = 0.80
0.30. If a Coppell teenager is P(S ∩ B) = 0.30
chosen at random, what is the
probability that the teenager P(SUB) = P(S) + P(B) - P(A∩B)
owns a skateboard or a = 0.40 + 0.80 - 0.30
bicycle? = 0.90

Make a two-way table or Venn diagram with this data. 38

Addition rule for disjoint sets
If events A and B are mutually exclusive
(disjoint), then P(A ∩ B) = 0 and the general
addition rule simplifies to P(A U B) = P(A) + P(B).

39
Addition rule for disjoint sets
Suppose we draw one card from a well-shuffled deck. What is
the probability that it is a heart or a black card?
Let
P(B) = P(draw a heart) = 13/52
P(C) = P(draw a black card) = 26/52
Since a card cannot be both a heart and black, P(B∩C) = 0.
Therefore,
P(heart or black card) = P(B U C) = P(B) + P(C) - P(B∩C)
= 13/52 + 26/52 - 0
= 39/52
40
Recap for disjoint sets
If A and B are disjoint (mutually exclusive), then
Given mutually exclusive
P(A ∩ B) = 0 events, finding the
probability of at least one
of them occurring is
P(A U B) = P(A) + P(B) accomplished by adding
their probabilities.

41
Recap: Basic rules of probability
● To find the probability that an event A OR
another event B happens, add their probabilities
and subtract the probability that both happen.
P(A U B) = P(A) + P(B) - P(A∩B).
● Mutually exclusive events (also called “disjoint”)
cannot both happen at the same time.
If A and B are disjoint, then P(A∩B) = 0 and
P(A∪B) = P(A) + P(B).
42
Basic craps play (pass line bet)

43
Basic craps play (pass line bet)
A player bets $1 and then rolls two six-sided dice. If the total on the up-faces is 7 or 11, the player
wins $1 and gets the original $1 back. If the sum is 2, 3, or 12, the player loses the $1 that was bet.
Any other total on the first roll becomes “THE POINT” and the game continues. The player keeps
rolling until the point comes up again (a win) or a sum of 7 is rolled (a loss).

44
Sum 2 3 4 5 6 7 8 9 10 11 12
Prob 1/36 2/36

Combinations that make each sum If you roll Probability Percent

1 2 3 4 5 6 Win on first roll 6 +2 =8 2 .2222

(7 or 11) 36 36 36 9

1 2 3 4 5 6 7 Win with 4 3 *3 = 1
36 9 36

2 3 4 5 6 7 8 Win with 5 4 *4 = 2
36 10 45
3 4 5 6 7 8 9 Win with 6

4 5 6 7 8 9 10 Win with 8

Win with 9
5 6 7 8 9 10 11
Win with 10
6 7 8 9 10 11 12
Total probability of winning =
45
Basic craps play (pass line bet)

46
Basic craps play (pass line bet)

So the player has a 49.29% chance of gaining

$1 and a 50.71% chance of losing $1. In the
long run, the player would expect to gain
(0.4929)($1) + (0.5071)(–$1) = –$0.0142, that
is, to lose just over 1 cent for every dollar bet.
47
Appendix: Set properties
(AC)C = A
A ∩ AC = ∅ (A and AC are disjoint)
A∪Ω=Ω
A∩Ω=A
A∪B=B∪A
(A ∪ B) ∪ C = A ∪ (B ∪ C)
A ∩ (B ∩ C) = (A ∩ B) ∩ C
A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)
{A ⊂ B and B ⊂ A} ⇒ A = B
48