CIS 2033:

Computational Probability and Statistics.

Temple University, Fall 2014

Chapter 3: Conditional probability and independence

Instructor: Yuhong Guo

Fall 2014, Temple Univ.

3.1 Conditional Probability

Conditional Probability: the probability that an event
will occur, given that another event has occurred that
changes the likelihood of the event

Example:

Event L is person was born in a long month, and event R is

person was born in a month with the letter R in it.
We ask someones birthday month,
 Then P(L)=7/12, P(R) = 8/12
 If we know in the experiment, L occurs, then P(R) =?
it is affected by the given fact whether or not L has occurred
The probability that R occurs, given that L has occurred is
P(R|L) = 4/7

Fall 2014, Temple Univ.

3.1 Conditional Probability

Definition: The conditional probability of A given C is:

P(A|C) = P(A C) / P(C) , provided P(C) > 0.

Note unconditional probability is with respect to the

sample space, conditional probability is with respect to the
condition.
P(A) = P(A|) = P(A ) / P()

If event A includes m of the n equally probable elements

in C, then P(A|C) = m/n

Fall 2014, Temple Univ.

3.1 Multiplication Rule

The multiplication rule: For any events A and C:
P(A C) = P(A|C)P(C)

Example: (assume non-leap year) :
B2: two arbitrarily chosen persons have different birthdays.
The probability: P(B2) = 1 1/365
B3: three arbitrarily chosen persons have different birthdays.
P(B3) = ?
B3 can be seen as A3 B2 with event A3--the third person has a
birthday that does not coincide with that of one of the first two
persons, so
P(B3) = P(A3 B2) = P(A3|B2)P(B2)
= (1 2/365) (1 1/365)

Fall 2014, Temple Univ.

3.1 Multiplication Rule

.similarly, for event Bn a group of n arbitrarily
chosen people all have different birthdays

Fall 2014, Temple Univ.

3.1 Multiplication Rule

Fall 2014, Temple Univ.

3.1 Multiplication Rule

It matters how one conditions
P(A C) = P(A|C) P(C)
= P(C|A) P(A)
One form may be easier or more useful to calculate than the
other in a certain scenario.

Example:
In the previous example, we can have P(B3) = P(A3 B2)
= P(B2|A3)P(A3). But this is not very helpful for the task.

Fall 2014, Temple Univ.

3.3 Total Probability and Bayes Rule

The Law of Total Probability
Suppose C1, C2, ,Cm are disjoint events such that
C1 U C2 U U Cm = . The probability of an arbitrary
event A can be expressed as:
P(A) = P(A|C1)P(C1)
+ P(A|C2)P(C2)
+
+ P(A|Cm)P(Cm)

Fall 2014, Temple Univ.

3.3 Total Probability and Bayes Rule

Fall 2014, Temple Univ.

3.3 Total Probability and Bayes Rule

Bayes Rule:

P(C | A) =

P( A | C) P(C)
P( A)

Suppose the events C1, C2, Cm are disjoint and

C1 U C2 U U Cm = . The conditional probability
of Ci, given an arbitrary event A, can be expressed as:
P(Ci | A) =

P( A | Ci) P(Ci)
P( A)

P(Ci | A) =

P( A | Ci) P(Ci)
P( A | C1) P(C1) + P( A | C 2) P(C 2) + ... + P( A | Cm) P(Cm)

or

Fall 2014, Temple Univ.

3.4 Independence

Definition. An event A is called independent of B if

P(A|B) = P(A)

That is to say: A is independent of B if the probability of A

occurring is not changed by whether or not B occurs.

Equivalent Statements:

A independent of B P(A B) = P(A)P(B)

A independent of B B independent of A
A independent of B Ac independent of B

Fall 2014, Temple Univ.

3.4 Independence

Tests for Independence
To show that A and B are independent it suffices to prove
just one of the following:

P(A|B) = P(A)

P(B|A) = P(B)

P(A B) = P(A)P(B)
where A may be replaced by Ac and B replaced by Bc, or both.

If one of these statements holds, all of them are true. If two

events are not independent, they are called dependent.

Fall 2014, Temple Univ.

3.4 Independence

Independence of Two or More Events
Events A1, A2, , Am are called independent if:
P(A1 A2 Am) = P(A1)P(A2) P(Am)

This statement holds true if any event or events is/are

replaced by their complement throughout the equation.

Fall 2014, Temple Univ.

3.4 Independence

Is independence for three events A, B, and C
the same as:
A and B are independent; B and C are independent;
and A and C are independent ?

Example:
A fair coin is flipped twice independently. Use A for
"heads on the first flip", B for "heads on the second flip",
and C for "the two flips have the same outcome".
Then the three events are pairwise independent, but
dependent altogether.