Statistics Tutorial: Working With Probability

The probability of an event refers to the likelihood that the event will occur.
How to Interpret Probability
Mathematically, the probability that an event will occur is expressed as a number between 0 and
1. Notationally, the probability of event A is represented by P(A).
• If P(A) equals zero, event A will almost definitely not occur.
• If P(A) is close to zero, there is only a small chance that event A will occur.
• If P(A) equals 0.5, there is a 50-50 chance that event A will occur.
• If P(A) is close to one, there is a strong chance that event A will occur.
• If P(A) equals one, event A will almost definitely occur.
The sum of all possible outcomes in a statistical experiment is equal to one. This means, for
example, that if an experiment can have three possible outcomes (A, B, and C), then P(A) + P(B)
+ P(C) = 1.
How to Compute Probability: Equally Likely Outcomes
Sometimes, a statistical experiment can have n possible outcomes, each of which is equally
likely. Suppose a subset of r outcomes are classified as "successful" outcomes.
The probability that the experiment results in a successful outcome (S) is:
P(S) = ( Number of successful outcomes ) / ( Total number of equally likely outcomes ) = r / n
Consider the following experiment. An urn has 10 marbles. Two marbles are red, three are green,
and five are blue. If an experimenter randomly selects 1 marble from the urn, what is the
probability that it will be green?
In this experiment, there are 10 equally likely outcomes, three of which are green marbles.
Therefore, the probability of choosing a green marble is 3/10 or 0.30.
How to Compute Probability: Law of Large Numbers
One can also think about the probability of an event in terms of its long-run relative frequency.
The relative frequency of an event is the number of times an event occurs, divided by the total
number of trials.
P(A) = ( Frequency of Event A ) / ( Number of Trials )

For example, a merchant notices one day that 5 out of 50 visitors to her store make a purchase.
The next day, 20 out of 50 visitors make a purchase. The two relative frequencies (5/50 or 0.10
and 20/50 or 0.40) differ. However, summing results over many visitors, she might find that the
probability that a visitor makes a purchase gets closer and closer 0.20.
The scatterplot (above right) shows the relative frequency as the number of trials (in this case,
the number of visitors) increases. Over many trials, the relative frequency converges toward a
stable value (0.20), which can be interpreted as the probability that a visitor to the store will
make a purchase.
The idea that the relative frequency of an event will converge on the probability of the event, as
the number of trials increases, is called the law of large numbers.
Test Your Understanding of This Lesson
Problem
A coin is tossed three times. What is the probability that it lands on heads exactly one time?
(A) 0.125
(B) 0.250
(C) 0.333
(D) 0.375
(E) 0.500
Solution
The correct answer is (D). If you toss a coin three times, there are a total of eight possible
outcomes. They are: HHH, HHT, HTH, THH, HTT, THT, TTH, and TTT. Of the eight possible
outcomes, three have exactly one head. They are: HTT, THT, and TTH. Therefore, the
probability that three flips of a coin will produce exactly one head is 3/8 or 0.375.

Statistics Tutorial: Rules of Probability

Often, we want to compute the probability of an event from the known probabilities of other
events. This lesson covers some important rules that simplify those computations.
Definitions and Notation
Before discussing the rules of probability, we state the following definitions:
• Two events are mutually exclusive or disjoint if they cannot occur at the same time.

• The probability that Event A occurs, given that Event B has occurred, is called a
conditional probability. The conditional probability of Event A, given Event B, is
denoted by the symbol P(A|B).

• The complement of an event is the event not occuring. The probability that Event A will
not occur is denoted by P(A').

• The probability that Events A and B both occur is the probability of the intersection of A
and B. The probability of the intersection of Events A and B is denoted by P(A ∩ B). If
Events A and B are mutually exclusive, P(A ∩ B) = 0.

• The probability that Events A or B occur is the probability of the union of A and B. The
probability of the union of Events A and B is denoted by P(A ∪ B) .
 • If the occurence of Event A changes the probability of Event B, then Events A and B are
dependent. On the other hand, if the occurence of Event A does not change the
probability of Event B, then Events A and B are independent.
Probability Calculator
Use the Probability Calculator to compute the probability of an event from the known
probabilities of other events. The Probability Calculator is free and easy to use. It can be found
under the Stat Tools tab, which appears in the header of every Stat Trek web page.
Probability Calculator

Rule of Subtraction
In a previous lesson, we learned two important properties of probability:
• The probability of an event ranges from 0 to 1.
• The sum of probabilities of all possible events equals 1.
The rule of subtraction follows directly from these properties.
Rule of Subtraction The probability that event A will occur is equal to 1 minus the probability
that event A will not occur.
P(A) = 1 - P(A')
Suppose, for example, the probability that Bill will graduate from college is 0.80. What is the
probability that Bill will not graduate from college? Based on the rule of subtraction, the
probability that Bill will not graduate is 1.00 - 0.80 or 0.20.
Rule of Multiplication
The rule of multiplication applies to the situation when we want to know the probability of the
intersection of two events; that is, we want to know the probability that two events (Event A and
Event B) both occur.
Rule of Multiplication The probability that Events A and B both occur is equal to the
probability that Event A occurs times the probability that Event B occurs, given that A has
occurred.
P(A ∩ B) = P(A) P(B|A)
Example
An urn contains 6 red marbles and 4 black marbles. Two marbles are drawn without replacement
from the urn. What is the probability that both of the marbles are black?
Solution: Let A = the event that the first marble is black; and let B = the event that the second
marble is black. We know the following:
• In the beginning, there are 10 marbles in the urn, 4 of which are black. Therefore, P(A) =
4/10.
• After the first selection, there are 9 marbles in the urn, 3 of which are black. Therefore,
P(B|A) = 3/9.
Therefore, based on the rule of multiplication:
P(A ∩ B) = P(A) P(B|A)
P(A ∩ B) = (4/10)*(3/9) = 12/90 = 2/15

Rule of Addition
The rule of addition applies to the following situation. We have two events, and we want to
know the probability that either event occurs.
Rule of Addition The probability that Event A or Event B occurs is equal to the probability that
Event A occurs plus the probability that Event B occurs minus the probability that both Events A
and B occur.
P(A ∪ B) = P(A) + P(B) - P(A ∩ B))

Note: Invoking the fact that P(A ∩ B) = P( A )P( B | A ), the Addition Rule can also be
expressed as
P(A ∪ B) = P(A) + P(B) - P(A)P( B | A )
Example
A student goes to the library. The probability that she checks out (a) a work of fiction is 0.40, (b)
a work of non-fiction is 0.30, , and (c) both fiction and non-fiction is 0.20. What is the
probability that the student checks out a work of fiction, non-fiction, or both?
Solution: Let F = the event that the student checks out fiction; and let N = the event that the
student checks out non-fiction. Then, based on the rule of addition:
P(F ∪ N) = P(F) + P(N) - P(F ∩ N)
P(F ∪ N) = 0.40 + 0.30 - 0.20 = 0.50
Test Your Understanding of This Lesson
Problem 1
An urn contains 6 red marbles and 4 black marbles. Two marbles are drawn with replacement
from the urn. What is the probability that both of the marbles are black?
(A) 0.16
(B) 0.32
(C) 0.36
(D) 0.40
(E) 0.60
Solution
The correct answer is A. Let A = the event that the first marble is black; and let B = the event
that the second marble is black. We know the following:
• In the beginning, there are 10 marbles in the urn, 4 of which are black. Therefore, P(A) =
4/10.
• After the first selection, we replace the selected marble; so there are still 10 marbles in
the urn, 4 of which are black. Therefore, P(B|A) = 4/10.
Therefore, based on the rule of multiplication:
P(A ∩ B) = P(A) P(B|A)
P(A ∩ B) = (4/10)*(4/10) = 16/100 = 0.16

Problem 2
A card is drawn randomly from a deck of ordinary playing cards. You win $10 if the card is a
spade or an ace. What is the probability that you will win the game?
(A) 1/13
(B) 13/52
(C) 4/13
(D) 17/52
(E) None of the above.
Solution
The correct answer is C. Let S = the event that the card is a spade; and let A = the event that the
card is an ace. We know the following:
• There are 52 cards in the deck.
• There are 13 spades, so P(S) = 13/52.
• There are 4 aces, so P(A) = 4/52.
• There is 1 ace that is also a spade, so P(S ∩ A) = 1/52.
Therefore, based on the rule of addition:
P(S ∪ A) = P(S) + P(A) - P(S ∩ A)
P(S ∪ A) = 13/52 + 4/52 - 1/52 = 16/52 = 4/13

THE RELATIVE FREQUENCY INTERPRETATION OF

PROBABILITY
We are interested in learning about the probability of some event in some process. For example,
our process could be rolling two dice, and we are interested in the probability in the event that
the sum of the numbers on the dice is equal to 6.
Suppose that we can perform this process repeatedly under similar conditions. In our example,
suppose that we can roll the two dice many times, where we are careful to roll the dice in the
same manner each time.
I did this dice experiment 50 times. Each time I recorded the sum of the two dice and got the
following outcomes:
4 10 6 7 5 10 4 6 5 6 11 11 3 3 6
7 10 10 4 4 7 8 8 7 7 4 10 11 3 8
6 10 9 4 8 4 3 8 7 3 7 5 4 11 9
5 5 5 8 5
To approximate the probability that the sum is equal to 6, I count the number of 6's in my
experiments (5) and divide by the total number of experiments (50). That is, the probability of
observing a 6 is roughly the relative frequency of 6's.
 # of 6's
PROBABILITY (SUM IS 6) is approximately -----------
# of tosses

5
= ---- = .1
50

In general, the probability of an event can be approximated by the relative frequency , or

proportion of times that the event occurs.
# of times event occurs
PROBABILITY (EVENT) is approximately -----------------------
# of experiments
Comments about this definition of probability:
1. The observed relative frequency is just an approximation to the true probability of an
event. However, if we were able to perform our process more and more times, the relative
frequency will eventually approach the actual probability. We could demonstrate this for
the dice example. If we tossed the two dice 100 times, 200 times, 300 times, and so on,
we would observe that the proportion of 6's would eventually settle down to the true
probability of .139.
Click here for a demonstration of this idea using computer dice rolling.
2. This interpretation of probability rests on the important assumption that our process or
experiment can be repeated many times under similar circumstances. In the case where
this assumption is inappropriate, the subjective interpretation of probability is useful.
MEASURING PROBABILITIES USING A
CALIBRATION EXPERIMENT
Probabilities are generally hard to measure. It is easy to measure probabilities of events that are
extremely rare or events that are extremely likely to occur. For example, your probability that the
moon is made of green cheese (a rare event) is probably close to 0 and your probability that the
sun will rise tomorrow (a sure event) is likely 1. But consider your probability for the event
"There will be a white Christmas this year". You can remember years in the past where there was
snow on the ground on Christmas. Also you can recall past years with no snow on the ground. So
the probability of this event is greater than 0 and less than 1. But how do you obtain the exact
probability?
To measure someone's height we need a measuring instrument such as a ruler. Similarly, we
need a measuring device for probabilities. This measuring device that we use is called a
calibration experiment . This is an experiment which is simple enough so that probabilities of
outcomes are easy to specify. In addition, these stated probabilities are objective; you and I
would assign the same probabilities to outcomes of this experiment.
The calibration experiment that we use is called a chips-in-bowl experiment. Suppose we have a
bowl with a certain number of red chips and white chips. We draw one chip from the bowl at
random and we're interested in
Probability(red chip is drawn)
This probability depends on the number of chips in the bowl. If, for example, the bowl contains 1
red chip and 9 white chips, then the probability of choosing a red is 1 out of 10 or 1/10 = .1. If
the bowl contains 3 red and 7 chips, then the probability of red is 3/10 = .3. If the bowl contains
only red chips (say 10 red and 0 white), then the probability of red is 1. At the other extreme, the
probability of red in a bowl with 0 red and 5 white is 0/5 = 0.
Let's return to our event "There will be a white Christmas this year". To help assess its
probability, we compare two bets -- one with our event and the second with the event "draw a red
chip" from the calibration experiment. This is best illustrated by example. Consider the following
two bets:
• BET 1: You get $100 if there is a white Christmas and nothing if there is not a white
Christmas.
• BET 2: You get $100 if you draw red in a bowl of 5 red and 5 white and nothing
otherwise.
Which bet do you prefer? If you prefer BET 1, then you think that your event of a white
Christmas is more likely than the event of drawing red in a bowl with 5 red, 5 white. Since the
probability of a red is 5/10 = .5, this means that your probability of a white Christmas exceeds .5.
If you prefer BET 2, then by similar logic, your probability of a white Christmas is smaller than .
5.
Say you prefer BET 1 and you know that your probability is larger than .5, or between .5 and 1.
To get a better estimate at your probability, you make another comparison of bets, where the
second bet has a different number of red and white chips.
Next you compare the two bets
• BET 1: You get $100 if there is a white Christmas and nothing if there is not a white
Christmas.
• BET 2: You get $100 if you draw red in a bowl of 7 red and 3 white and nothing
otherwise.
Suppose that you prefer BET 2. Since the probability of red in a bowl of 7 red and 3 white is
7/10 = .7, this means that your probability of a white Christmas must be smaller than .7.
We've now made two judgements between bets. The first judgement told us that our probability
of white Christmas was greater than .5 and the second judgement told us that our probability was
smaller than .7. What is our probability? We don't know the exact value yet, but we know that it
must fall between .5 and.7. We can represent our probability by an interval of values on a
number line.
OUR PROBABILITY OF WHITE CHRISTMAS LIES IN HERE:

*********
+----+----+----+----+----+----+----+----+----+----+
0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1
What if we wanted to get a more accurate estimate at our probability? We need to make more
comparison between bets. For example, we could compare two bets, where the first bet used our
event and the second used the event "draw red" from a bowl of chips with 6 red and 4 white.
After a number of these comparisons, we can get a pretty accurate estimate at our probability.

PROBABILITIES OF "OR" AND "NOT" EVENTS

Sometimes we are interested in computing probabilities of more complicated
events. Here we introduce two basic probability rules. The first rule is useful for
finding the probability of one event or another event. The second rule tells us how
to compute the probability that an event does not occur.

The addition rule (for computing probabilities of "or" events)

We will illustrate this rule with an example. Suppose Ohio has a two-digit lottery
game and the winning number will be chosen at random from all possible two-digit
numbers
{00, 01, 02, 03, ..., 97, 98, 99}.
There are 100 possible winning numbers and since each has the same chance of being chosen,
we assign a probability of 1/100 = .01 to each number.
Suppose we want to find the probability that the winning number has the same two digits or the
winning number is between 89 and 96 inclusive. If these two events ("same two digits" and
"between 89 and 96") are nonoverlapping, then we can find the probability of "same two digits"
or "between 89 and 96" by adding:
Prob("same two digits" or "between 89 and 96") = Prob("same two digits")
+Prob("between 89 and 96")
Are these two events nonoverlapping? Nonoverlapping means that it is impossible for the two
events to occur at the same time. Here "same two digits" means the winning number is from the
set {00, 11, 22, 33, 44, 55, 66, 77, 88, 99}. "Between 89 and 96" means that the number is in the
set {89, 90, 91, 92, 93, 94, 95, 96}. Note that these two sets have nothing in common; in other
words, it is impossible for the winning number to have the same two digits and be between 89
and 96. So we can add the probabilities to find the probability of the "or" event. The probability
of "same two digits" is 10/100 and the probability of "between 89 and 96" is 8/100. Therefore the
probability of interest is
Prob("same two digits" or "between 89 and 96") = 10/100 + 8/100 = 18/100 = .18
What if we wanted to find the probability of "same two digits" or "an even second digit"? Here
we can't use this addition rule, since these two events are overlapping. It is possible for the
winning to have the same two digits and have an even second digit -- the number 44 (and other
numbers) is in both events. So this rule cannot be used in this case.
This rule is also applicable in the case where you want to find the probability of a collection of
different outcomes. Suppose you toss a coin five times and you wish to find the probability that
the number of heads is 2 or fewer. You can think of the event "2 or fewer heads" as an "or"
event:
{2 or fewer heads} = {0 heads} or {1 head} or {2 heads}
By definition, the three outcomes {0 heads}, {1 heads} and {2 heads}, since you can only
observe at most one of these outcomes when you toss the coin three times. So the addition rule
can be used and
Prob(2 or fewer heads) = Prob(0 heads) + Prob(1 head) + Prob(2 heads)

The complement rule (for computing probabilities of "not" events)

Let's return to our lottery example. What if you're interested in the probability that
the winning number does not have the same two digits. The rule for "not" events is
called the complement rule:
Probability("not" an event) = 1 - Probability(event)
In this case, we can write
Probability(not same digits) = 1 - Probability(same digits)
We have already found the probability that the winning number has the same two digits, so the
probability of interest is
Probability(not same digits) = 1 - 10/100 = 90/100
The complement rule is especially useful in the case where it hard to compute the probability of
an event, but it is relatively easy to compute the probability of "not" the event. For example,
suppose we wish to compute the probability of tossing at least one head in 10 tosses of a coin. In
this case, it would make sense to first perform the easier computation, the probability of "not at
least one head" or "no heads". Then we apply the complement rule to find the probability of the
event of interest.
Probability(at least one head) = 1 - Probability(no heads)