Basic Probability Concepts

Key Topics

 Basic probability concepts

 Sample spaces and events, simple probability, joint
probability
 Conditional probability
 Statistical independence, marginal probability
 Bayes’ Theorem

2
 Sample Spaces

 Collection of all possible outcomes

 e.g.: All six faces of a die:

 e.g.: All 52 cards in a deck:

3
 Events

 Simple event
 Outcome from a sample space with one
characteristic
 e.g.: A red card from a deck of cards
 Joint event
 Involves two outcomes simultaneously
 e.g.: An ace that is also red from a deck of
cards

4
 Visualizing Events

 Contingency Tables
Ace Not Ace Total
Black 2 24 26
Red 2 24 26
Total 4 48 52

 Tree Diagrams Ace

Red
Full Cards Not an Ace
Deck Black Ace
of Cards Cards Not an Ace 5
 Special Events
Null Event

 Impossible event ♣♣
e.g.: Club & diamond on one card
draw
 Complement of event
 For event A, all events not in A
 Denoted as A’
 e.g.: A: queen of diamonds
A’: all cards in a deck that are
not queen of diamonds

6
 Special Events
(continued)
 Mutually exclusive events
 Two events cannot occur together
 e.g.: -- A: queen of diamonds; B: queen of clubs
 Events A and B are mutually exclusive
 Collectively exhaustive events
 One of the events must occur
 The set of events covers the whole sample space
 e.g.: -- A: all the aces; B: all the black cards; C: all the
diamonds; D: all the hearts
 Events A, B, C and D are collectively
exhaustive
 Events B, C and D are also collectively

exhaustive 7
 Probability

 Probability is the numerical 1 Certain

measure of the likelihood
that an event will occur
 Value is between 0 and 1
.5
 Sum of the probabilities of
all mutually exclusive and
collective exhaustive events
is 1
0 Impossible
8
 Computing Probabilities

 The probability of an event E:

number of event outcomes
P( E ) =
total number of possible outcomes in the sample space
X For example, the probability of drawing an ace
=
T from a standard deck of cards is: P(Ace) = 4/52

 Each of the outcomes in the sample space is

equally likely to occur

9
 Computing Joint Probability

 The probability of a joint event, A and B:

P(A and B) = P(A ∩ B)
number of outcomes from both A and B
=
total number of possible outcomes in sample space
If we draw a card from a standard deck or cards, what
is the probability it will be a red ace (red and ace)?
E.g. P(Red Card and Ace)
2 Red Aces 1
=
52 Total Number of Cards 26 10
 Joint Probability Using
Contingency Table

Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)
A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probability Marginal (Simple) Probability

11
 Marginal and Joint probabilities with
a Contingency Table
Joint Probability of a A Deck of 52 Cards
Red Ace is 2/52 or 1/26

Not an Total
Ace
Ace
Red 2 24 26
Black 2 24 26
Total 4 48 52
Marginal (Simple) Probability
of an Ace is 4/52
Sample Space
12
 Computing Compound
Probability
 Probability of a compound event, A or B:
P(A or B) = P(A) + P(B) - P(A and B)

For Mutually Exclusive Events: P(A or B) = P(A) + P(B)

E.g.P (Red Card or Ace)

4 Aces + 26 Red Cards - 2 Red Aces
=
52 total number of cards
28 7
= =
52 13
13
 Compound Probability with a
Contingency Table
Not an Total
Ace
Ace
Red 2 24 26
Black 2 24 26
Total 4 48 52

P(A or B) = P(A) + P(B) - P(A and B)

E.g. P (Red Card or Ace)
4 Aces + 26 Red Cards - 2 Red Aces
=
52 total number of cards
28 7
= =
52 13 Chap 4-14
 Computing Conditional
Probability

 The probability of event A given that event B

has occurred:
P( A and B)
P( A | B) =
P( B)

E.g. Suppose we draw a red card – what is the probability it is

an Ace?
P(Ace given Red Card) = P (Ace and Red Card) / P (Red Card)

15
 Conditional Probability Using
Contingency Table
Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52

Revised Sample Space

P(Ace and Red) 2 / 52 2
P(Ace | =
Red) = =
P(Red) 26 / 52 26
16
 Conditional Probability and
Statistical Independence

 Conditional probability:
P( A and B)
P( A | B) =
P( B)
 Multiplication rule:

P( A and B) = P( A | B) P( B)
= P( B | A) P( A)
17
 Conditional Probability and
Statistical Independence
(continued)

 Events A and B are independent if

P ( A | B) = P( A)
or P( B | A) = P( B)
or P( A and B) = P( A) P( B)
 Events A and B are independent when the
probability of one event, A, is not affected by
another event, B

18
 Bayes’ Theorem
P ( A | Bi ) P ( Bi )
P ( Bi | A ) =
P ( A | B1 ) P ( B1 ) + • • • + P ( A | Bk ) P ( Bk )
P ( Bi and A )
=
P ( A) Adding up
the parts
Same of A in all
Event the B’s

19
 Bayes’ Theorem
Using Contingency Table
A study of borrowers that had taken an educational loan found
that only fifty percent had repaid their loans. Out of those who
repaid, 40% had a college degree. Ten percent of those who
defaulted had a college degree. What is the probability that a
borrower who has a college degree will repay the loan?
Solution: A contingency table helps solve these types of problems. Let’s use the
information provided above to set up a contingency table.
Since fifty percent of borrowers repaid their loans, we know that the remaining
fifty percent of borrowers defaulted on their loans.
Of those who repaid, 40% had a college degree – thus, percentage of borrowers
who repaid and have a college degree in the sample is .5*.4=.2, or 20%.
Ten percent of those who defaulted had a college degree – thus, percentage of
borrowers who defaulted and have a college degree in the sample is .5*.1=.05.
This information is organized using a contingency table on the next slide
20
 Bayes’ Theorem
Using Contingency Table
(continued)
Loan Not
Repaid Repaid Total
Note: Whether or not the
borrower repaid the loan College
forms one dimension of the Degree .2 .05 .25
contingency table. The
No College
other dimension is whether
Degree
.3 .45 .75
or not the borrower has a
college degree. Total .5 .5 1.0

What is the Using the contingency table, the

probability that a probability that a borrower with a
borrower who has a college degree will repay the loan is
college degree will .2/.25, or 80%.
repay the loan?

21
 Medical Diagnosis problem
The probability a person has a certain disease is 3%. Medical
diagnostics tests are available to determine whether the person actually
has the disease. If the disease is present, the probability the diagnostic
will give a positive result (correctly identify the disease) is 90%. If the
disease is absent, the probability of a positive test result is 2%.
Suppose a person is administered the test and the result is positive.
What is the probability this person has the disease?
Solution: Since probability of a person having the disease is 3%, we know that
the chance this person does not have the disease is 97%.
For a person with disease, the chance of a positive test result is 90% -- thus, the
probability that a person has the disease and tests positive is .03X.9=.027
If the disease is absent, the chance of a positive test result is 2% -- thus, the
probability that a person has the disease and tests positive is .97X.02=.0194
This information is organized using a contingency table (next slide)
22
 Using Contingency Table for the Medical
Diagnosis problem
Has No
Note: Whether or not the Disease Disease Total
patient has the disease Test
forms one dimension of the Positive .027 .0194 .0464
contingency table. The
Test
other dimension is whether
Negative
.003 .9506 .9536
the test is positive or
negative. Total .03 .97 1.0

What is the Using the contingency table, the

probability that a probability that a person who tests
person who tests positive has the disease is
positive has the .027/.0464, or 58.2%.
disease?

23
 Summary
 Discussed basic probability concepts
 Sample spaces and events, simple (marginal)
probability, and joint probability
 Defined conditional probability
 Statistical independence
 Discussed Bayes’ theorem

24