Week 2

Lecture 2: Basic Concepts of Probability Theory I
Muhammad Ryan Sanjaya
Universitas Gadjah Mada
February 21, 2020
Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 2 February 21, 2020 1 / 47
1 Introduction
Motivation
Random Events and Probability
Counting Techniques and Calculation of Probabilities
The conditional probability, independence, and Bayes’ rule
Motivation
In the last lecture, we discussed how we need a model to help us

make (important) decisions.
Most of the time, we deal with incomplete information:
I using representative sample for a certain population
Probability theory allows us to learn mathematical models to make
predictions
1 Introduction
Motivation
Experiment
Experiment is a process whose outcome is not known in advance but

is random.
I The definition of experiment has a wider context than lab or field
experiment
We are going to assume that the experiment can be repated any
number of times under identical conditions:
I Each repetition is a trial
I Think of coin tossing as an experiment.
Experiment satisfies three conditions:
I set of all possible outcomes are known in advance in each trial
I in any particular trial, it is not known which outcome will happen
I the experiment can be repeated under identical conditions.
Definitions
Definition
The sample space associated with an experiment is the set consisting of all
possible outcomes. A sample space is also referred to as a probability
space. It is denoted by S.
Definitions
Definition
An event A is a subset of outcomes in S, A ⊂ S. Event A occurs if the
outcome of the experiment is in A.
Definitions
Definition
The null subset φ of S is called an impossible event.
Definitions
Definition
The event A ∪ B consists of all outcomes that are in A or in B or in both
Definitions
Definition
The event A ∩ B consists of all outcomes that are in A and B.
Definitions
Definition
The event AC (the complement of A in S) consists of all outcomes not in
A, but in S.
Definitions
Definition
Two events A and B are said to be mutually exclusive or disjoint of
A ∩ B = φ.
Example
Suppose we roll a (balanced) die:

I What is the sample space?
I Provide an example of a possible event
Classical Definition of Probability
Definition
If there are n equally likely possibilities, of which one must occur, and m of
these are regarded as favourable to an event, or as success, then the
probability of the event or a success is given by m/n. The probability of
any event A in the sample space S is denoted by P (A).
Frequency Definition of Probability
Definition
The probability of an outcome (event) is the proportion of times the
outcome (event) would occur in a long run of repeated experiments.
Example
Suppose we toss two coins and assume fair coins:

I What is the sample space?
I Provide an example of a possible event
Let A be the event that at least one of the coins shows up heads.
I Find P (A)
Frequency Definition of Probability
The frequency definition of probability is not applicable to every event.

Recall that the frequency definition of probability requires repetitions
in identical experiments.
Thus, in 1933, A. N. Kolmogorov developed axioms for the probability
theory.
Axiomatic definition of probability
Definition
Let S be a sample space of an experiment. Probability P (·) is a real-valued
function that assigns to each event A in the sample space S a number
P (A), called the probability of A, with the following conditions satisfied:
it is nonnegative, P (A) ≥ 0
it is unity for a certain event. That is, P (S) = 1
it is additive over the union of an infinite number of pairwise disjoint
events, that is, if A1 , A2 , . . . form a sequence of pairwise mutually
exclusive events (Ai ∩ Aj = φ, i 6= j) in S, then:
P∞
P (∪∞ i=1 Ai ) = i=1 P (Ai )
Example
A die is loaded (not all outcomes are equally likely) such that the
probability of number i shows up is Ki, i = 1, 2, . . . , 6, and K is a
constant. Find:
I the value of K
I the probability that a number greater than 3 shows up
Basic Properties of Probability
For two events A and B in S:

I P AC = 1 − P (A), where AC is the complement of the set A in S.
I If A ⊂ B, then P (A) ≤ P (B)
I P (A ∪ B) = P (A) + P (B) − P (A ∩ B).
F if A ∩ B = φ, then P (A ∪ B) = P (A) + P (B)
Example
In a Statistics II class, the Sintesis says that 30% of the students were
in the top 10% of their class and that 40% are economics students, of
whom 15% were in the top 10% of their class. What is the probability
that a student selected randomly from this class either was in the top
10% of their class or an economics student?
1 Introduction
Motivation
Motivation
We went through examples in which the sample space is quite small.

For large sample space, counting the number of outcomes for a
specific event could be cumbersome.
Thus, we need to learn counting techniques and apply it to probability
computations.
Multiplication principle
Definition
If the experiments A1 , A2 , . . . , Am contain respectively: n1 , n2 , . . . , nm
outcomes, such that for each possible outcomes of A1 there are n2 possible
outcomes for A2 , and so on, then there are a total of n1 , n2 , . . . , nm
possible outcomes for the composite experiments A1 , A2 , . . . , Am .
Example
There are 100 economics students in FEB UGM. In how many

different ways Himiespa can choose its president and vice president?
Sampling with replacement and the objects are ordered
Random sample of size k is taken from a total of n objects and the

objects are being ordered.
Then, there are nk possible ways of selecting k−tuples.
Example:
I A die is rolled four times
Sampling without replacement and the objects are ordered
Let n! be:
n! = n (n − 1) (n − 2) . . . (2) (1)
If r objects are chosen from a set of n distinct objects without
replacement, an ordered arrangement is called a permutation.
Example:
I what is the number of permutation for three letters ABCD?
Permutation of n object taken m at a time
Theorem
The number of permutations of m objects selected from a collection of n
distincts objects is:
n!
n Pm =
(n − m)!
= n (n − 1) (n − 2) . . . (n − m + 1)
Example
How many distinct three-digit numbers can be formed using digits 1,

2, 5, 7, 9 if no digit can be repeated?
Sampling without replacement and the objects are not
ordered
In permutation, the order in which each object is selected becomes

important.
When the orders of arrangement is not important, the arrangement is
called combination
Combinations of n object taken m at a time
Theorem
The number of ways in which m objects can be selected (without
replacement) from a collection of n distinct objects is:
!
n n!
=
m m! (n − m!)
n (n − 1) (n − 2) . . . (n − m + 1)
=
m!
where m = 0, 1, 2, . . . , n.
Example
Suppose that you have 9 balls numbered from 1 to 9. How many

different ways can you choose 2 even numbers and 3 odd numbers?
Example
In an urn containing 9 balls, there are three red balls and 6 blue balls.
We select three balls at random.
I What is the probability that exactly one red ball is selected?
I What is the probability that at most one red ball is selected?
I What is the probability that at least one red ball is selected?
Example
Suppose that there are 100 economics students and 200 business
students. The BEM board consists of 10 members. What is the
probability that a chosen 10 will have 3 economics students and 7
business students?
1 Introduction
Motivation
Motivation
Your knowledge about the probability of an event maybe influenced

by partial information about the event.
For example:
I Arsenal winning a game
I Probability of raining
Conditional Probability
The conditional probability of an event A, given that an event B has

occurred, denoted by P (A | B), is equal to:
P (A ∩ B)
P (A | B) =
P (B)
given that P (B) > 0.
Example
Suppose that you are tossing two dies in sequence. What is the
probability that you see a sum of 8 given that the first die gives you a
face value of 3?
Some Properties of Conditional Probability
If E2 ⊂ E1 , then P (E2 | A) ≤ P (E1 | A)

If P (E | A) = 1 − P E C | A
P (E1 ∪ E2 | A) = P (E1 | A) + P (E2 | A) − P (E1 ∩ E2 | A)
Multiplication law:
P (A ∩ B) = P (B) P (A | B) = P (A) P (B | A)
Example
Suppose that you have basket of 30 green and red apples, of which 10
are green apples. If two fruits are randomly picked in sequence, what
is the probability that both fruits are red apples?
Two independent events
Definition
Two events A and B with P (A) 6= 0 and P (B) 6= 0 are said to be
independent if
P (A | B) = P (A)
or
P (B | A) = P (B) .
Consequently, for two independent events A and B
P(A ∩ B) = P(A)P(B)
Example
Suppose that we toss two fair dices. Let E1 denote the event that the
sum of the dice is 6 and E2 denote the event that the first die equals
4. Are E1 and E2 independent events?
Law of Total Probability
Theorem
Assume S = A1 ∪ A2 ∪ . . . ∪ An , where P (Ai ) > 0, i = 1, 2, . . . , n and
Ai ∩ Aj = φ (null set) for i 6= j. Then, for any event B:
n
X
P (B) = P (Ai ) P (B | Ai ) .
i=1
Example
Assume that a noisy channel transmit 0s 60% of time and 1s 40% of

the time. At the receiver, there is a 1% chance of obtaining any
particular symbol distorted (transmit 0, but receiver shows 1, vice
versa). What is the probability of receiving a 1, irrespective of which
symbol is transmitted?
Bayes’ Rule
Theorem
Assume S = A1 ∪ A2 ∪ . . . ∪ An , where P (Ai ) > 0, i = 1, 2, . . . , n and
Ai ∩ Aj = φ for i 6= j. Then for any event B, with P (B) > 0:
P (Aj ) P (B | Aj )
P (Aj | B) = Pn .
i=1 P (Ai ) P (B | Ai )
Example
A a math for economist I class contained 55% male and 45% female
students. At the end of the semester, 3% of males and 10% of
females got an “A/B or above” grade. If one student from this class
is randomly selected and observed to have an “A/B or above” grade,
what is the probability that this is a male student?
Steps to apply Bayes’ Rule
List all the probabilities including conditional probabilities given in the

problem:
P (A1 ) , . . . , P (An )
P (D | A1 ) , P (D | A2 ) , . . . , P (D | A3 ) .
Write the numerator as the product, P (A1 ) P (D | A1 )
Using the probability rule, find the denominator probability in the
Bayes’ rule
The desired probability is the ratio between numerator and
denominator.

Week 2

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Week 2

Uploaded by

Copyright:

Available Formats

Lecture 2: Basic Concepts of Probability Theory I

Muhammad Ryan Sanjaya

Universitas Gadjah Mada

February 21, 2020

In the last lecture, we discussed how we need a model to help us

Experiment is a process whose outcome is not known in advance but

Suppose we roll a (balanced) die:

Suppose we toss two coins and assume fair coins:

The frequency definition of probability is not applicable to every event.

For two events A and B in S:

We went through examples in which the sample space is quite small.

There are 100 economics students in FEB UGM. In how many

Random sample of size k is taken from a total of n objects and the

How many distinct three-digit numbers can be formed using digits 1,

In permutation, the order in which each object is selected becomes

Suppose that you have 9 balls numbered from 1 to 9. How many

Your knowledge about the probability of an event maybe influenced

The conditional probability of an event A, given that an event B has

given that P (B) > 0.

If E2 ⊂ E1 , then P (E2 | A) ≤ P (E1 | A)

Consequently, for two independent events A and B

Assume that a noisy channel transmit 0s 60% of time and 1s 40% of

List all the probabilities including conditional probabilities given in the

You might also like