Advanced Statistics

Probability

Economics
University of Manchester
Probability

Events and probabilities

Lecture focuses on CONDITIONAL probabilities

Relative frequency interpretation of probability:

Experiment (e.g. roll a die) has an outcome (eg,
obtain a 4)
Repeat experiment a very large number of times
Compute proportion of times you observe particular
outcome (proportion times 4 obtained)
The probability of the specific outcome equals the
computed proportion
Probabilities must be in the range [0, 1]
Often also expressed in % terms
UNION of events

EF

E
F

E = 1st yr. BA Econ Students

F = 1st yr. Students taking ECON10072
E  F = 1st yr. students either on BA Econ or
taking ECON10072
INTERSECTION of events
EF

E = 1st yr. BA Econ Students

F = 1st yr. Students taking ECON10072
E  F = 1st yr. BAEcon students taking ECON10072
The COMPLEMENT of an event

E = Complement of E : all possible sample outcomes,

except those in E
Addition rule of probability

evaluate probabilities of
unions
+ +

area ( E  F ) = area ( E ) + area ( F ) − area ( E  F )


Pr( E  F ) = Pr( E ) + Pr( F ) − Pr( E  F )
Example:
Smoking (A) and heart disease (D)
Can use Venn Diagrams to illustrate probabilities
50% of people surveyed
were SMOKERS
30%
20% of those surveyed SMOKED
20% and
had experienced HEART DISEASE
10%
40% of those surveyed were
40% NON-SMOKERS and had NOT
experienced HEART DISEASE
Conditional Probability
The probability of an event, E, occurring GIVEN
that an event, F, has done so is the
CONDITIONAL PROBABILITY of E given F and is
denoted Pr(E|F)

Involves a revision of our probability assessment

in the light of known information
contents insurance premiums given location
heart disease given you smoke …
Smoking (A) and Heart Disease (D)

Consider Pr(D|A)
The conditioning event, A,
defines a new RESTRICTED
SAMPLE SPACE.
30%
2/5=20/50 of those who
smoke, have experienced
20% heart disease

10% Pr ( D | A) = 0.4  Pr ( D ) = 0.3

1
Pr (D | A ) = = 0.2  Pr ( D | A)
5
Multiplication rule of probability

Pr( E  F ) conditional
Pr( E F ) = probability
Pr( F )

Pr( E  F ) = Pr( E F )  Pr( F )
multiplication rule
Independence
INDEPENDENCE is an important special case

Independence of events E and F implies Pr(E|F) = Pr(E)

knowledge of F does not alter probability of E
thus, with independence:

Pr( E  F )
Pr( E | F ) = Pr( E ) =
Pr( F )

• Consequently, independent events E and F satisfy:

Pr ( E  F ) = Pr ( E )  Pr ( F )
Multiplication rule of probability
again

Pr( E ) = Pr( E  F )
+ EF
Pr( E  F )
EF

Pr( E  F ) = Pr( E F )  Pr( F )

Pr ( E ) = Pr ( E | F ) Pr ( F ) + Pr ( E | F ) Pr ( F )
Bayes’ Theorem I

Consider a sample space three events: A, B and C.

A, B and C are MUTUALLY EXCLUSIVE and cover all
possible outcomes of an experiment.
Which of A, B or C has occurred we don’t know … all
we observe is E.
Bayes’ Theorem II

BAYES’ THEOREM is used to obtain Pr(A|E),

or Pr(B|E), or Pr(C|E); e.g., the probability that A has
occurred given that we only observe E.
We start with the usual definition of CONDITIONAL
PROBABILITY ...
Bayes’ Theorem III

Pr ( A  E ) Pr (E | A) Pr ( A)
Pr ( A | E ) = =
Pr (E ) Pr (E )
Rewrite the numerator using the multiplication rule of
probability, and then the denominator ...
Bayes’ Theorem IV

Pr ( E | A) Pr ( A)
Pr ( A| E ) =
Pr ( A  E ) + Pr ( B  E ) + Pr ( C  E )
... using the addition rule of probability; finally ...
Bayes’ Theorem V

Pr ( E | A) Pr ( A)
Pr ( A| E ) =
Pr ( E | A) Pr ( A) + Pr ( E | B) Pr ( B) + Pr ( E | C) Pr ( C)

... the multiplication rule of probability again!

THE RESULT IS BAYES’ THEOREM
Smoking and Heart Disease
Of people who have heart disease
(D), what proportion smoke (A)?
2
Pr ( A | D ) =
3
30%
Or, using Bayes’ Theorem directly
20%
Pr( A  D)
Pr( A | D ) =
Pr(D)
10%
Pr(D | A) Pr( A)
=
Pr(D | A) Pr( A) + Pr(D | A )Pr(A )
(2 / 5)  (1 / 2)
=
(2 / 5)  (1 / 2) + (1 / 5)  (1 / 2)
= 2/3