Binomial Distribution &

Bayes Theorem

Questions
What is a probability?
What is the probability of obtaining 2
heads in 4 coin tosses? What is the
probability of obtaining 2 or more
heads in 4 coin tosses?
Give an concrete illustration of p(D|H)
and p(H|D). Why might these be
different?

Probability of Binary Events

Probability of success = p
p(success) = p
Probability of failure = q
p(failure) = q
p+q = 1
q = 1-p
Probability long run relative
frequency

Permutations & Combinations

1 Suppose we flip a coin 2 times

HH
HT
TH
TT
Sample space shows 4 possible outcomes or
sequences. Each sequence is a permutation.
Order matters.
There are 2 ways to get a total of one heads (HT
and TH). These are combinations. Order does
NOT matter.

Perm & Comb 2

HH, HT, TH, TT
Suppose our interest is Heads. If the coin is
fair, p(Heads) = .5; q = 1-p = .5.
The probability of any permutation for 2
trials is = p*p, or p*q, or q*p, or q*q. All
permutations are equally probable.
The probability of exactly 1 head in any
order is 2/4 = .5 = HT+TH/
(HH+HT+TH+TT) [what is probability of at
least 1 head?]

Perm & Comb 3

3 flips
HHH,
HHT, HTH, THH
HTT, THT, TTH
TTT
All permutations equally likely = p*p*p
= .53 = .125 = 1/8.
p(1 Head) = 3/8

Perm & Comb 4

Factorials: N!
4! = 4*3*2*1
3! = 3*2*1
N
N!
Combinations: NCr
r r!( N r )!
The number of ways of selecting r combinations of N objects,
regardless of order. Say 2 heads from 5 trials.

5
5!
5 x 4 x3 x 2 x1

5 x 2 10
2 2!(5 2)! (2 x1)(3 x 2 x1)

Binomial Distribution 1
N r N r
p ( x r ; N , p ) p q , r 1,2,...N
r

Is a binomial distribution with

parameters N and p. N is the number of
trials, p is the probability of success.
Suppose we flip a fair coin 5 times; p =
q = .5

Binomial 2
5
4
3
2
1
0

N r N r
p ( x r ; N , p ) p q , r 1,2,...N
r
5 5
p ( X 5; N 5, p .5) p .55
5
5 4 1
p ( X 4; N 5, p .5) p q 5 p 4 q
4
5 3 2
p ( X 3; N 5, p .5) p q 10 p 3 q 2
3
5 2 3
p ( X 2; N 5, p .5) p q 10 p 2 q 3
2
5 1 4
p ( X 1; N 5, p .5) p q 5 p1q 4
1
5 0 5
p ( X 0; N 5, p .5) p q q 5
0

.03125
.15625
.3125
.3125
.15625
.03125

Binomial 3
Flip coins and compare observed to
expected frequencies

Binomial 4
Find expected frequencies for number
of 1s from a 6-sided die in five rolls.

Binomial 5

Relative Frequency (Number of Combos)

When p is .5, as N increases, the

binomial approximates the Normal.
Binomial N = 10 p = .5
300

200

100

0
0

4
6
Number Heads

10

Review
What is a probability?
What is the probability of obtaining 2
heads in 4 coin tosses? What is the
probability of obtaining 2 or more
heads in 4 coin tosses?

Bayes Theorem (1)

Bayesian statistics are about the revision of belief. Bayesian
statisticians look into statistically optimal ways of
combining new information with old beliefs.
Prior probability personal belief or data. Input.
Likelihood likelihood of data given hypothesis.
Posterior probability probability of hypothesis given data.
Scientists are interested in substantive hypotheses, e.g.,
does Nicorette help people stop smoking. The p level
that comes from the study is the probability of the sample
data given the hypothesis, not the probability of the
hypothesis given the data. That is

p (hypothesis | data ) p (data | hypothesis )

Bayes Theorem (2)

Bayes theorem is old and mathematically correct. But its
use is controversial. Suppose you have a hunch about the
null (H0) and the alternative (H1) that specifies the
probability of each before you do a study. The probabilities
p(H0) and p(H1) are priors. The likelihoods are p(y| H0) and
p(y| H1). Standard p values. The posterior is given by:
p( y | H 0 ) p( H 0 )
p ( H 0| y )
p( y | H 0 ) p( H 0 ) p( y | H1 ) p( H1 )
p(H1|y)=1-p(H0|y)

Bayes Theorem (3)

Suppose before a study is done that the two hypotheses are
H0: p =.80 and H1: p=.40 for the proportion of male grad
students. Before the study, we figure that the probability is
.75 that H0 is true and .25 That H1 is true. We grab 10 grad
students at random and find that 6 of 10 are male.
p ( H 0 ) .75; p ( H 1 ) .25
Binomial applies.
p (.6 | H 0 ) p (.6 | p .8) .088
p (.6 | H1 ) p (.6 | p .4) .111

p( y | H 0 ) p( H 0 )
p ( H 0| y )
p( y | H 0 ) p( H 0 ) p( y | H1 ) p( H1 )
p (.6 | y )

(.088)(.75)
.704
(.088)(.75) (.111)(.25)

Bayes Theorem (4)

Bayes theorem says we should revise our belief of the
probability that H0 is true from .75 to .70 based on new
data. Small change here, but can be quite large depending
on data and prior.
Problems with choice of prior. Handled by
empirical data or by flat priors. There are
Bayesian applications to more complicated
situations (e.g., means and correlations). Not
used much in psychology yet except in metaanalysis (empricial Bayes estimates) and
judgment studies (Taxis, etc). Rules for
exchangeability (admissible data) need to be
worked out.

Review

Give an concrete illustration of

p(D|H) and p(H|D). Why might
these be different?