120 BAYESIAN INFERENCE FOR DISCRETE RANDOM VARIABLES

weights, are what counts. If there is a formula for the prior, any part of it
that does not contain the parameter can be absorbed into the constant. This
may make calculations simpler for us!
Multiplying the likelihood by a constant does not change the result of Bayes’
theorem. The prior likelihood values would also be multiplied by the same
constant, which would cancel out in the posterior probabilities. The likelihood
can be considered the weights given to the possible values by the data. Again,
it is the relative weights that are important, not the actual weights. If there
is a formula for the likelihood, any part that does not contain the parameter
can be absorbed into the constant, simplifying our calculations!

EXAMPLE 6.2 (continued)

We used a prior that gave each value equal prior probability. In this
example there are three possible values, so each has a prior probability
equal to 13 . Let us multiply each of the 3 prior probabilities by the constant
3 to give prior weights equal to 1. This will simplify our calculations. The
observations are binomial (n = 4; ), and y = 3 was observed. The formula
for the binomial likelihood is
4 3
f (yj ) = (1 )1 :
3
The binomial coe cient 43 does not contain the parameter, so it is a
constant over the likelihood column. To simplify our calculations, we will
absorb it into the constant and use only the part of the likelihood that
contains the parameter. In Table 6.15 we see that this gives us the same
result we obtained before.

6.4 Bayes’ Theorem for Poisson with Discrete Prior

We will see how to apply Bayes’ theorem when the observation comes from a
Poisson( ) distribution and we have a discrete prior distribution over a few
discrete possible values for . Y jmu is the number of counts of a process that
is occurring randomly through time at a constant rate. The possible values of
the parameter are 1 ; : : : ; I . We decide on the prior probability distribution,
g( i ) for i = 1; : : : ; I. These give our belief weight for each possible value
before we have looked at the data. In Section 6.2 we learned that we do not
have to use the full range of possible observations. Instead, we set up a table
only using the reduced universe column, i.e., the value that was observed.

Setting up the Table for Bayes’ Theorem on Poisson with Discrete Prior
Set up table with columns for parameter value, prior, likelihood, prior
likelihood, and posterior.
 MAIN POINTS 121

Put the parameter value, prior, and likelihood in their respective columns.
The likelihood values are Poisson( ) probabilities evaluated at the ob-
served value of y. They can be found in Table B.5 in Appendix B, or
evaluated from the Poisson formula.

Multiply each element in the prior column by the corresponding element

in the likelihood column, and enter them in the prior likelihood column.

Divide each prior likelihood by the sum of the prior likelihood column
and put them in the posterior column.

EXAMPLE 6.3

Let Y j be Poisson( ). Suppose that we believe there are only four pos-
sible values for , 1,1.5,2, and 2.5. Suppose we consider that the two
middle values, 1.5 and 2, are twice as likely as the two end values 1 and
2.5. Suppose y = 2 was observed. Plug the value y = 2 into formula
y
e
f (yj ) =
y!

to give the likelihood. Alternatively, we could nd the values for the like-
lihood from Table B.5 in Appendix B. The results are shown in the Table
6.16. Note: We could use the proportional prior and the proportional
likelihood and we would get the same results for the posterior.

Table 6.16 The simpli ed table for nding posterior distribution given Y = 2
prior likelihood prior likelihood posterior
1 1:02 e 1:0
:0307
1.0 6 2!
= :1839 .0307 :2473
= :124
1 1:52 e 1:5
:0837
1.5 3 2!
= :2510 .0837 :2473
= :338
1 2:02 e 2:0
:0902
2.0 3 2!
= :2707 .0902 :2473
= :365
1 2:52 e 2:5
:0428
2.5 6 2!
= :2565 .0428 :2473
= :173
marginal P (Y = 2) .2473 1.000

Main Points

The Bayesian universe has two dimensions. The vertical dimension is the
parameter space and is unobservable. The horizontal dimension is the
sample space and we observe which value occurs.

The reduced universe is the column for the observed value.

122 BAYESIAN INFERENCE FOR DISCRETE RANDOM VARIABLES

For discrete prior and discrete observation, the posterior probabilities are
found by multiplying the prior likelihood and then dividing by their
sum.

When our data arrives in batches we can use the posterior from the rst
batch as the prior for the second batch. This is equivalent to combining
both batches and using Bayes’ theorem only once, using our initial prior.

Multiplying the prior by a constant does not change the result. Only
relative weights are important.

Multiplying the likelihood by a constant does not change the result.

This means we can absorb any part of formula that does not contain the
parameter into the constant. This greatly simpli es calculations.

Exercises

6.1. There is an urn containing 9 balls, which can be either green or red. The
number of red balls in the urn is not known. One ball is drawn at random
from the urn, and its color is observed.

(a) What is the Bayesian universe of the experiment.

(b) Let X be the number of red balls in the urn. Assume that all possible
values of X from 0 to 9 are equally likely. Let Y1 = 1 if the rst ball
drawn is red, and Y1 = 0 otherwise. Fill in the joint probability table
for X and Y1 given below:
X prior Y1 = 0 Y1 = 1

(c) Find the marginal distribution of Y1 and put it in the table.

(d) Suppose a red ball was drawn. What is the reduced Bayesian universe?
(e) Calculate the posterior probability distribution of X.
(f) Find the posterior distribution of X by lling in the simpli ed table: