Chapter Five – Randomness and Probability

At the beginning of this course, we mentioned that decisions are constantly being made where the
information on which those decisions are based and the outcomes of these decisions are usually not
known with certainty. One set of measures used which relate to uncertainties are measures of
variability contained in our data sets. This will become more obvious and more important in our
subsequent statistics course where sample results will be used to make inferences concerning some
aspect of a population of interest which will then be used to make our decisions.

In chapter five, we will analyse uncertainty using probabilities. To do this, we first need to determine all
possible mutually exclusive outcomes of some random phenomenon, experiment or process, and their
probabilities of occurring. This ‘list’ of all possible outcomes is defined as the sample space, S.

From this sample space, we can combine outcomes and create a smaller number of events, A, B, … . If
necessary, additional events can be created by combining two or more of these individual events. In any
case, not only do we need to know all possible outcomes, events, and/or necessary combinations of
events, we need to know their probabilities of occurring. To create these probabilities, there are rules
that may need to be applied. For the majority of this chapter, we will be developing and using rules
when we combine only two events.

Before we develop and discuss these rules, several definitions and symbols must be mentioned:

 sample space, S – set of all possible mutually exclusive outcomes of a random phenomenon (or
random experiment, or random process)

ex. When randomly selecting a card from a deck of 52 playing cards, the sample space can
expressed as:

S = {A, … , K, A, … , K, A, … , K, A, … , K}

 A, B, … - events consisting of subsets of the sample space

ex. A = { A, … , K}

B = {A, A, A, A}

 P(A) – the probability that event A occurs

13 1
=
ex. P(A) = 52 4

4 1
=
P(B) = 52 13

 P(AC) – the probability that event A does not occur

39 3
=
ex. P(A ) = 52 4
C
 48 12
=
P(BC) = 52 13

 P(A and B) – the probability that both events A and B occur

1
ex. P(A and B) = P(A) = 52

 P(A or B) – the probability that either A or B or both occur

16
ex. P(A or B) = P(A, … , K, A, A, A) = 52

 P(A│B) – the probability that A occurs given (or if) that event B occurs

1
ex. P(A│B) = P(A│A, A, A, A) = 4

1
P(B│A) = P(A│A, … , K) = 13

 mutually exclusive events – the occurrence of one event does not allow the occurrence of other
events

ex. If A = red ace and B = black ace, A and B are mutually exclusive since a card cannot be
both a red ace and a black ace

Therefore, events A and B are mutually exclusive

 independent events – the occurrence of one event does not affect the occurrence of other
events

ex. If A = { A, … , K} and B = {A, A, A, A}, A and B are independent events
because the occurrence of B does not affect the occurrence of A because

1 1
P(A) = and P(A│B) = 4
4

That is, the probability of A does not depend on whether B occurs or not.

All of the above probabilities are based on each outcome in the sample space having the same
probability of occurring (or each card has the same chance of being selected).

Rule: If the sample space consists of n equally likely outcomes and if A consists of k of these outcomes,

k
 P(A) = n

Other basic rules whether the outcomes in the sample space are equally likely to occur or not:

Rule: an event occurs if any outcome in that event occurs

Rule: P(S) = 1
Rule: 0 ≤ P(A) ≤ 1
(If event A cannot occur, P(A) = 0. If event must occur, P(A) = 1. The more likely that event A
occurs, the closer P(A) is to 1)

Rule: P(A) = 1 – P(AC)

More rules:

P ( A and B )
Rule: P(A│B) = P (B )

 if A and B are mutually exclusive events, P(A│B) = 0

 if A and B are independent events, P(A│B) = P(A)

P ( A and B )
Rule: P(B│A) = P( A )

 if A and B are mutually exclusive events, P(B│A) = 0

 if A and B are independent events, P(B│A) = P(B)

Rule: P(A and B) = P(B)P(A│B) = P(A)P(B│A)

 if A and B are mutually exclusive events, P(A and B) = 0

 if A and B are independent events, P(A and B) = P(A)P(B)

Rule: P(A or B) = P(A) + P(B) – P(A and B)

 if A and B are mutually exclusive events, P(A or B) = P(A) + P(B)

 if A and B are independent events, P(A or B) = P(A) + P(B) – P(A)P(B)

Note: An accompanying file with Venn Diagrams illustrates some of the above rules.

Examples

1. Given the following probabilities:

P( A )=. 35 P(B )=. 40 P( A and B)=.25

calculate: P(AC) P(A or B) P(A│B) P(B│A) P(AC and BC)

P(AC) = 1 – P(A) = 1 - .35 = .65

P(A or B) = P(A) +P(B) – P(A and B) = .35 + .40 - .25 = .50

P ( A and B ) . 25
= =
P(A│B) = P (B ) . 40 .625
 P ( A and B) .25
= ≈¿ ¿
P(B│A) = P( A ) .35 .714

P(AC and BC) = P(AC)P(BC│AC), but this information is not given to us.

One way to find this probability is to create a contingency table in which all the probabilities for
this random process are determined. To do this we begin with the following table which, at
first, contains all the probabilities that we were given.

A AC total

B .25 .40

BC

total .35

We these 3 probabilities, we are able to complete this table:

A AC total

B .25 .15 .40

BC .10 .50 .60

total .35 .65 1.00

Thus, P(AC and BC) = .50

Note: A basic contingency table (2 characteristics, 2 categories each) gives us the following
probabilities:

A AC total

B P(A and B) P(AC and B) P(B)

BC P(A and BC) P(AC and BC) P(BC)

total P(A) P(AC) 1.00

2. Given the following probabilities:

P( A)=.7 P(B| A )=.45 P( A or B)=.80

Calculate: P( A and B) P(B ) P( A| B)

P(A and B) = P(A)P(B│A) = (.7)(.45) = .315

P(B) = ? → P(A or B) = P(A) + P(B) – P(A and B) → P(B) = P(A or B) – P(A) + P(A and B)
 P(B) = .80 - .70 + .315 = .415

P ( A and B ) .315
= ≈¿ ¿
P(A│B) = P (B ) .415 .759

3. Given the following probabilities:

P( A )=. 60 P(B )=. 70 P( A and B )=. 42

 Are A and B mutually exclusive events? Why or why not?

 Are A and B independent events? Why or why not?

A and B are not mutually exclusive because P(A and B) ≠ 0

A and B are independent because P(A and B) = P(A)P(B) (.42 = (.6)(.7))

4. A large conference dealing with health care was recently held in Detroit. Of those participating
in this conference, 20% were Canadians and 80% were Americans. Of those participating
Canadians, 70% preferred Canada’s universal health care over the American system of health
care. Of those participating Americans, 40% indicated that they would prefer Canada’s universal
health care over the system of health care that they currently have. If one of these participants
in this conference was randomly selected for further questioning, calculate the probability that
this participant:
 was a Canadian and preferred universal health care
 was American and preferred universal health care
 preferred universal health care
 was a Canadian or preferred universal health care
 was a Canadian given that he/she preferred universal health care
 was an American given that he/she preferred universal health care
 was a Canadian and did not prefer universal health care
 was and American and did not prefer universal health care

This example is a bit more difficult than examples 1 and 2 because we first need to convert
statements and questions asked into symbols and expressions.

Let C = Canadian (CC = American)

U = preferred universal health care (UC = did not prefer universal health care)

Therefore,

P(C) = .20 P(CC) = .80 P(U│C) = .70 P(U│CC) = .40

 was a Canadian and preferred universal health care

P(U and C) = P(C)P(U│C) = (.20)(.70) = .14

 was American and preferred universal health care

P(U and CC) = P(CC)P(U│CC) = (.80)(.40) = .32

 The remaining probabilities can easily be calculated with the following contingency table (with
red type probabilities completing the table and the black type probabilities already known).

C CC total

U .14 .32 .46

UC .06 .48 .54

total .20 .80 1.00

 preferred universal health care

P(U) = .46

 was a Canadian or preferred universal health care

P(U or C) = P(U) + P(C) – P(U and C) = .46 + .20 - .14 = .52

 was a Canadian given that he/she preferred universal health care

P (U andC ) .14
= ≈¿ ¿
P(C│U) = P (U ) .46 .304

 was an American given that he/she preferred universal health care

P(U and Cc ) .32

P(C c|U )= = ≈¿ ¿
P(U ) .46 .696

 was a Canadian and did not prefer universal health care

P(C and Uc) = .06

 was and American and did not prefer universal health care

P(Cc and Uc) = .48

1
In the ‘deck of cards’ example, how did we determine the initial probabilities of 52 for each specific card
being selected? We knew how the process worked in that we assumed that the cards were randomly
shuffled and that the individual selecting the card had no idea where each specific card was located in
the deck. This resulted in our conclusion that each specific card was equally likely to be selected. The
type of probability used to determine our probabilities can be labelled theoretical probability. But
suppose we had no idea of how the process worked except that we had outcomes for a very large
number of previous repetitions or trials of this process and we observed the number of times, f i, that
each outcome occurred in these n trials. Using empirical probability, we estimated the probability of
fi
each outcome, Oi, occurring to be n . Lastly, suppose that we didn’t know how some random process
worked and we didn’t have a large number of previous trials on which to estimate our probabilities. In
this case, we would assess the likelihood of the various outcomes occurring using personal probability.
But, no matter how we determined our initial probabilities, the above stated rules apply.

A couple of times above we have used contingency tables to determine probabilities, and, we have used
the following example and its contingency table to determine probabilities when the contingency table
contained numbers of elements falling into various categories and when each element had the same
chance of being selected.

Example

A survey, using a random sample of 400 customers, was recently conducted among customers who
made a purchase of a HD TV at a large chain of electronics stores. Each customer was classified
according to age (young (less than 35 years old), middle (between 35 and 55 years old), and, old (over 55
years old) and the size of TV purchased (small (no larger than 40 inches), medium (more than 40 inches
but less than 50 inches), and, large (at least 50 inches)). The number of customers falling into the
various categories are indicated in the following table.
Age small medium large total
young 15 90 45 150
middle 40 105 35 180
old 30 25 15 70
total 85 220 95 400
If one of these customers were to be randomly selected to win a 7-day cruise in the Caribbean, what is
the probability that this customer:
150
 would be young? = 400

95
 would have purchased a large TV? = 400

25
 would be old and purchased a medium TV? = 400

180+85−40 225
=
 would be middle aged or purchased a small TV? = 400 400

85+220 305
=
 would have purchase a small or medium TV? =400 400
 30
 would be old given that this customer purchased a small TV? =85

45
 would have purchased a large TV given that this customer was young? =150

If two customers were randomly selected, what is the probability that both were young and purchased a
small TV?

(= 15400 )(14399 )
Notes:

1. Up to now, we have calculated probabilities of ‘simple’ events occurring (e.g., P(A)), or, some
combination of 2 events occurring (e.g. P(A and B)) using probability rules. When we have a
random process where we want to determine probabilities for a combination of 3 or more
events (e.g. P(A or B or C)), this may become quite complicated. But, some of our previous rules
can be expanded to accommodate these types of processes.

 If A, B, C, … are mutually exclusive events,

P(A or B or C or … ) = P(A) + P(B) + P(C) + …

 If A, B, C, … are independent events,

P(A and B and C and … ) = P(A)P(B)P(C) …

 If A, B, C, are not independent events,

P(A and B and C and … ) = P(A)P(B│A)P(C│A,B) …

2. When we have a random process with more than two events, we can use a probability tree as a
tool from calculating any probabilities about this process. This tool is even more useful if
conditional probabilities (i.e., P(A│B), etc.) are involved. A probability tree consists basically of
sets of branches, with the first set of branches representing all possible outcomes of a specific
aspect of the process. Branches then build out from this first set until all aspects of the process
are taken into account. The following example illustrates the use of a probability tree.
Example

PLM, Inc. is submitting bids on 3 government contracts. It believes that it has a 40% chance of
winning contract A. If it wins contract A, it believes that it has a 30% chance of winning contract
B. If it doesn’t win contract A, it believes that it has a 50% chance of winning contract B. If it
wins contract B, it believes that it has a 60% of winning contract C. If it doesn’t win contract B, it
believes that it has a 20% chance of winning contract C. Assuming that its beliefs are correct,
the tree for this example looks like:
prob.
B .60 C ABC .072
.30
.40 CC ABCC .048
A
.40 BC .20 C ABCC .056
.70
.80 CC ABCCC .224

.60 .50 B .60 C ACBC .180

AC
.40 CC ACBCC .120
.50
BC .20 C ACBCC .060

.80 CC ACBCCC .240

If its beliefs are correct, use this probability tree to determine the probability that:

 it will win contract C

P(C) = .072 + .056 + .180 + .060 = .368

 it will win exactly 2 contracts

P(wins 2 contracts) = .048 + .056 + .180 = .284

 it will win contract C if it wins contract A

P ( A and C ) . 072+. 056 .128

= = =
P(C│A) = P( A ) . 40 . 40 .320
 it will win contract A and/or contract B

P(A or B) = .400 + .180 + .120 = .700

 it would have won contract B if it won contract C

P (B and C) . 072+.180 . 252

= = ≈¿ ¿
P(B│C) = P(C ) . 368 . 368 .685
Observations:

1. When creating a new set of branches, the sum of the probabilities for this new set must always
equal one.
2. The final result of the probability tree is a set of all mutually exclusive events of a random
process along with their probabilities summing to 1.
3. We were initially given the probability of winning contract C given PLM has won contract B.
And, you were asked to determine the probability of winning contract B given that PLM won
contract C. That is, the conditional probability was reversed from P(C│B) to the probability,
P(B│C). This chapter discusses Bayes’ Rule which is used to calculate these types of probabilities
but this rule is unnecessary if one knows how to use probability trees.

Closing remarks:

1. The text discusses three interesting situations in which the resulting probabilities defy common
sense. Two of them will be reviewed here
a. The birthday example was discussed in class in which the probability of at least two
individuals in a room of so many people having the same birthdate (month and day) is
much larger than common sense would lead us to believe. For example, the probability
of at least two people out of 30 having the same birthdate is approximately .70.
b. The ‘Let’s Make a Deal’ example in which there are 3 curtains behind which there are
prizes. Two curtains have insignificant prizes behind them and one has an automobile.
The contestant picks one of the curtains. Common sense would be correct and tell us
that the probability of the curtain you chose contained the car is 1/3. Then the host
purposely opens one of the remaining two curtains that he knows does not contain the
card. Common sense would tell us that the probability of the contestant now winning
the car has increased to 1/2. But now, common sense fails us. Using the following
probability tree (where W represents the contestant winning the car and E represents
that the host opened a current that does not contain the car),

prob

W 1 E W and E 1/3

1/3 0

EC W and EC 0

2/3

WC 1 E WC and E 2/3

EC WC and EC 0

P (W and E ) 1/3 1 /3
= = =
P(W│E) = P (E ) 1/3+2/3 1 1/3
 From a logical stand point, the initial probability of 1/3 should not have changed
because no new information was provided to the contestant if he/she knew that a
curtain not containing the car would be opened.

So, if the contestant didn’t choose the curtain with the car, and, since there are only two
curtains left, the probability that the other unopened curtain containing the car must be
2/3. Therefore, the contestant should switch to the other curtain if given the chance.

2. The text discusses the law of large numbers which essentially states that if a random process is
repeated a large number of times with each repetition being independent of the others, the
relative frequencies of each of the outcomes occurring approach the true probabilities of these
outcomes occurring.

3. It also mentions the law of averages which really doesn’t exist. Basically it states that if some
event hasn’t occurred for quite a while, its chance of occurring in the near future increases. For
example, the law of averages would conclude that if a fair coin is flipped 10 times and each flip
results in a head, there is a greater chance that a tail will appear in the next flip. This conclusion
is incorrect as long as the flips are independent of one another.

4. The text closes with some warnings:

c. The sum of the probabilities for all mutually exclusive events (i.e. disjoint events) must
add up to 1.
d. Don’t add probabilities for events that are not mutually exclusive.
e. Don’t multiply probabilities for events that are not independent.
f. Don’t confuse independent events and mutually exclusive events.