AMA2104 Probability and Engineering Statistics

1 Probability

Dr Bob He
Review: the Set Notations
Concept
Set: a collection of objects satisfying:
1. Rigorous membership: whether an object is in one set is well defined.
2. Uniqueness: all the objects in a set are distinct.
3. No order: there is not any order among the objects in a set.

Examples:
a A= {All the students in the class AMA2104 in Fall 2019}
a B = { v'2, 34, 1r}
5x -3 = 0}={3, - ½}
a C ={x : 2x2 -

a D={x:-l.5<x::;3}=(-1.5,3]
Counterexamples: The following examples are NOT sets
a {All the nice people} (ambiguous membership: Am I nice? Always
so?)
a {1,3,3, 4} (The number "3" appears twice.)
1
Set Relations
a x ES: x is an element of the set S.
a x (j. S: x is not an element of the set S.
a A is a subset of B if and only if every element of A is also an
element of B. This subset relation is denoted by AC B, or B::) A.
a A= B if and only if Ac B and B c A

2
Set Intersection

Set Intersection B
An B: intersection of A and B.
That is
AnB = {x: x EA and x EB}.

Example AnB
{v2,1r} n {1r,5} = {1r},
Figure: Venn diagram for the set
[-1,3] n (o,5] = (o,3].
intersection.

0: empty set (a set with no element).

For example, {1, 2} n {3, 4} = 0.

3
Set Union
Recall that N = {1, 2, ...} is the set of all the positive integers.
Set Union
AUB: the union of two sets A
and B. That is, A B

A U B = { x:xE A or xEB}.

Example AUB
Let A be the set of odd positive
Figure: Venn diagram for the set
integers and B be the set of union.
even positive integers, then
AUB=N.

relative complement "\"

B\A={xEB:xiA}.

For example [O, 2]\N = [O, 1) U (1, 2). 4

Experiments

Experiment Outcomes
Toss a coin Head, Tail
Roll a die 1,2,3,4,5,6
Play a football game win, lose, tie

Definition (Experiment)
An experiment is defined to be any process which randomly generates
well defined outcomes.

0 First, each single repetition of the experiment generates one and

only one of the possible outcome.
G Second, the outcome is random, and different repetitions may
generate different outcomes.

5
Sample space

Definition
The sample space is defined as the set of all possible experimental
outcomes. Any one particular experimental outcome is referred to as a
sample and is an element of the sample space.

Definition
An event is a set of zero, one, or more of the possible outcomes of an
experiment.

Usually the event is denoted by a capital letters such as A, B, C, etc.

Example: Rolling a die
• Sample space: {1, 2, 3, 4, 5, 6}
• In a specific experiment, one has a sample, for example, "x=3".
• Examples of events: {1}, "faces with odd numbers"={l, 3, 5}, "all
the faces"={l, 2, 3, 4, 5, 6}.
• We have: x (/. {l}, x E {1, 3, 5}, and x E {1, 2, 3, 4, 5, 6}.
6
The definition of Probability

Definition
Probability is a numerical measure of the likelihood that a specific event
will occur.

Probability is a function of event, not a function of sample! Let A be an

event. The probability of event A is denoted by P(A).

Two axioms of Probability

• 0 � P(A) � 1. When O = P(A), the event A cannot happen; when
P(A) = 1, the event will happen for sure.
• Let S be the sample space, then P(S) = 1.

7
Addition Rule

P(A or B) = P(A U B) = P(A) + P(B) - P(A n B)

The following venn diagrams illustrate the idea of proof.

AUB
s s AnB

s s

8
Definition (Mutually Exclusive)
Two events are said to be mutually exclusive if when one of them
occurs, the other will not occur for sure. Therefore,

P(AnB) = 0

s
00
Venn diagram: the events A and B are mutually exclusive.

Definition (Collectively Exhaustive)

A collection of events A 1 , A 2 , . . . , An is said to be collectively
exhaustive if their union is the whole sample space. (Therefore at least
one of them will occur).

9
Definition
The complement of an event A, denoted by A, or Ac , or A', is defined
to be the event consisting of all sample points that are not in A.

s
Venn diagram: the event A and its complement A. We see that they are
mutually exclusive, and collectively exhaustive.

For any event A, we have P(A) = 1 - P(A), and A= A.

10
Theorem (De Morgan's laws)
Let A and B be two events, then A U B =AnB and A nB=AUB
AnB

s ahaded part: Au B s ahaded part: An B

s ahaded part: A s ahaded part: B

11
Conditional Probability and Independence

Definition (Conditional Probability)

Let A and B be two events and P(B) > 0. The conditional probability
of A given B is defined as

Theorem (Multiplication Rule)

Let A and B be two events with P(A) > 0. Then

P(A n B) = P(A)P(BIA).

Definition (Independence)
Two events A and B are independent if P(A n B) = P(A)P(B).

12
The difference/ relation between independent" and
II

11
mutually exclusive"

a A and B are independent P(A n B) = P(A) x P(B).

<=}

a A and B are mutually exclusive <=} An B = (/J (remember that

P((JJ) = 0, so now P(A n B) = 0).
a If A and B are both mutually exclusive and independent, then
0 = P(A n B) = P(A) x P(B), therefore P(A) = 0 or P(B) = 0 or
P(A) = P(B) = 0.
a If P(A) > 0 and P(B) > 0, then whenever A and B are mutually
exclusive, they are not independent; whenever they are independent,
they are not mutually exclusive.
a If P(A) = 0, then A is independent of any event. Why?
a If P(A) = 1, then A is independent of any event. Why?

13
Law of Total Probability
Theorem (Law of total probability)
Assume that events B 1 , B2 , . . . , B n form a partition of the sample space
{i.e., they are collectively exhaustive, and any pair of them are mutually
exclusive), and P(Bi) >0 for i = 1, ... , n. Then for any event A,

We use a special case n = 3 to demonstrate the idea of the proof.

Ba

P(A) = P(B1 n A)+ P(B2 n A)+ P(B3 n A)

= P(B1)P(AIB1) + P(B2)P(AIB2) + P(B3)P(AIB3). 14
Bayes' Theorem
Theorem (Bayes' Theorem)
Suppose B1 , · · · , Bn form a partition of the sample space S (mutually
exclusive and collectively exhaustive), then for any k = 1, · · · n and any
event A,

P(Bk n A) P(Bk)P(AIBk)
P(BklA) = = P(B1)P(AIB1) + · · · + P(Bn)P(AIBn)
P(A)

Example: cancer diagnostic testing

Let C denote the event that a patient has a cancer. Let B denote the
event that the test reports "positive" for the patient. Some laboratory
studies show that P(C) = 0.001, P(BIC) = 0.99, and P(BIC) = 0.01.
We apply the Bayes' Theorem to give

P(C)P(BIC)
P( GIB) =
P(C)P(BIC) + P(C)P(BIC)
0.001 X 0.99
= 0.090 l 6393·
0.999 X 0.01 + 0.001 X 0.99 15
Example: the Monty Hall problem
Suppose you're on a game show, and you're given the choice of three
doors: Behind one door is a car; behind the others, goats. You pick a
door, say No. 1, and the host, who knows what's behind the doors, opens
another door, say No. 3, which has a goat. He then says to you, "Do you
want to pick door No. 2?" Is it to your advantage to switch your choice?

1 2

·7 ·7 •

prob leM and pi.cture froM wi.ki.pedi.a. org

16
Suppose that Tom will switch, John will not. Suppose that they attend
the game independently. Denote:
a T1: The car is behind Tom's first selected door;
a Tw : Tom wins the car;
a J1 : The car is behind John's first selected door;
a Jw: John wins the car;

17