Subject: Bayesian Revision

Author: Gavin Lawrence

Recipients: XXXXXXX
nd
Date: 22 November, 2000

During work on Metronet a number of issues arose that were perceived to be in

conflict with F+A’s cost analyses. These can be summarised as:

a. Estimation:
a. Under estimation of supplier costs
b. Under estimation of delays
c. Underestimation of cost-over-runs
d. Overestimation of probabilities of unlikely events
e. Under estimation of very large impact £ values

b. Perceived risk increases related to (and excluding probability):

a. Proximity of responsibility to decision-maker
b. Time to impact
c. Size of impact
d. Biases from diverse sources ranging from entrepreneurial
optimism to question framing.

This discussion paper looks at one of the problems areas – estimates in conflict with
the views of the quantity surveyors. However, the other areas, mentioned above,
have all been subject to extensive academic study and I will try, subject to time and
motivation, to write something about them for them.

If you have time please send me comments on any of these issues as well as about
Bayesian Revision.

Bayes Theorem to represent the updating of knowledge

To tackle the problem of estimation error, one possible solution is the use of
‘Bayesian Revision’ to bridge the gap between the differing perspectives in
determining the necessary contingency funding.

A simple example of this might be:

The National Audit Office report that concludes that all major government
sourced civil/engineering projects have cost over-runs of 40% on average.

A consortium of companies assess that the impact of cost over-runs from all
sources will only amount to 15% at worst on this government project.

The QSs state that in their experience of similar projects the cost-over-runs
have been between –5% to 29% and averaging 20%.

This paper endeavours to summarise the general mathematical approach to

incorporating different perspectives such as this into the risk modelling process. (If I
have made a mistake let me know, and if you know an easier way of multiplying two
distributions please call me on 0776 346 5063).
Bayesian Revision

Bayesian revision simply involves multiplying two distributions together to achieve a

third distribution.
Prior Belief  Likelihood  Posterior Belief
The basic formula is widely used to represent the learning process in the Decision
Sciences field
Assumption 1. The only important statistics of any distribution are the mean and the
standard deviation. We can cheat here because most @Risk models
have loads of distributions and the outcome is a normal distribution
(Central Limit Theorem).
1. Basic method of multiplying two distributions using ‘Triple Notation’:
This method was this method was taught me in university and is supposed to
make Bayesian manipulation of the normal density easier instead of the
standard notation.
N ( , 2) means that the variable x is normally distributed with the mean
and variance 2.
The Normal distribution is represented by:
x~ N ( , u )
2
p = 1/u where u = 
Thus the distribution can be represented as:
{x ,  ; p} = exp ( - ½ p ( x -  ) 2 )
Therefore, when multiplying two normal distributions – the prior and the
sample:
{x , a ; p} x {x , b ; q} = {x , ( pa + qb) / (p+q) ; p+q} x { a, b: pq/(p + q)

2. The general example 1

A sample of n values from N (2 
with a sample mean = m
distributed as N (2/n 
The prior has the distribution N ( , p2 )
The prior precision is 1/p2=p
2
The sample precision is 1/ =P
By Bayes Theorem:
m ) = { m ; np } x { ; P }
= {  n.p P n.p P ) ; n.p + P} x { m, n.p.P) / (n.p + P)}
3. A specific example (if you have some sample data!)

Sample of n=9 values from N ( , 4)

Sample mean of m = 20

Prior distribution N (25 , 10 )

i.e.  =25 and u = 10

Let the sample precision p=1/4

Let the prior precision P = 1/10

By Bayes Theorem:

{  n.m.p P.n.p P ) ; n.p + P} x { m, n.p.P) / (np + P)}

{  9 x 1/4 x 20 1/10 x 259 x 1/4 1/10 ) ; 9 x 1/4 + 1/10} x { 20, 259 x 1/4
x 1/10) / (9 x 1/4 + 1/10)}

{  452.259 /4  /10 ) ; ( 2.25 + 0.1); 2.25 + 0.1} x { 20, 252.25 x

1 1

0.1) / (2.25 + 0.1)}

{  47.252.35 ) ; 2.35} x { 20, 252.25 x 0.1) / (2.25 + 0.1)} {20, 25, .225
/ 2.35}

{ ; 2.35} x { 20, 25, 2.35} x {20, 25, .0.9}

Therefore  ~ N ( 20.21, 1 / 2.35)

 = N (20.21, 0.43)

4. Confidence Interval

To calculate 95% confidence interval for  :

(n.p x / (n.p + P) +/- 1.96  (1/(n.p + p)

= {(9 x ¼ x 20.21) / 9 x ¼ + 1/10)} +/- 1.96 ((9 x ¼) + 1/10)

= (45.4725/2.35) +/- 1.96 x  (1 / 2.35 )

= 19.35 +/- 1.96 X 0.6523

= 19.35 +/- 1.279

= {18.92, 21.50} i.e. 20.21 +/- 1.29

a. Bayes and his Theorem
(for those who have forgotten it !!)

Thomas Bayes lived and died in relative obscurity, but

200 years later his method of statistical analysis is
enjoying a new vogue.

Bayes was the son of a non-conformist clergyman, and he went into the
ministry, ending his life at the Presbyterian Chapel in Tunbridge Wells,
Kent.

Born in 1702, he was educated privately, and in his lifetime gained a

reputation as a mathematician. Bayes was elected a Fellow of The Royal
Society in 1742, but he published little.

But after his death in 1761, Richard Price, a friend, found among his
papers “Essay towards solving a problem in the doctrine of chances”,
work produced as a result of a curious hobby – he used to while away his
time in the local court trying to assess whether a defendant was guilty on
the basis of his appearance and how the subsequent evidence altered
this perception. It was published by the Royal Society in 1763 and
became the basis of a statistical technique, now called Bayesian
estimation, for calculating how true a proposition is likely to be. Unlike
classical statistics, it enables prior judgement to be factored into the
equation.

His conclusions fell into disuse earlier this century, and Bayesians have
always felt despised by orthodox statisticians because of their guesswork.
However, now they are growing in numbers and confidence. Bayesian
methods have been used in court cases, in analysing the results of drug
trials and in improving the service of banks.

The importance of Bayes Theorem lies in:

That it is an excellent model of the human learning process encapsulated

in the general formula –

Prior belief x likelihood -= posterior belief

Put simply this means that whatever our initial knowledge state the
addition of new information will change our perceptions in some way to a
new altered state – our posterior belief. In risk management terms we can
use the Bayesian approach to produce a risk weighting of new
information using the precision (reciprocal of the standard deviation) to
derive a new perspective from impact of diverse data sources.

(There is a significant area of research addressing why people don’t

actually conform to the Bayesian learning model).
b. Deriving Bayes Theorem
( a really boring bit )

C
A B

P ( A  B) = P ( A ) x P ( B ) = P ( B ) x P ( A )

Now including the conditional notation “ | “ which means ‘given that’ or ‘conditional
on’ and “, C “ where C represents the state of the world and need not be included in
the formula because it is implicit.

P ( B  A, C) = P ( A | B , C ) x P ( B | A , C ) = P ( B | A , C ) x P ( A | B , C)

P ( B | A) = P ( B  A ) / P ( A )

 P ( B | A ) = P ( A | B ) x P (B) / P ( A )
7. The general example 2

Here is another quite simple way of doing the calculation that is given in Statistics: A
Bayesian Perspective by D. A. Berry. (It doesn’t need sample sizes but it does
require that you have some idea of the variance attached to your ‘beliefs’ – where
might it come from and how might it be elicited from respondents?) It is in fact just a
weighted averaging process using the reciprocal of the variance i.e. the precision.
2
Prior precision: c0 = 1 / h 0 = 0

Sample Precision: c = n / h2

Posterior Precision: c1 =c0 + c = c

Posterior mean: m1 = m0 x c0 / c1 +  x c / c1= m0 x 0 / c +  x x c / c

Posterior Standard Deviation: h1 = 1/ c1 = 1 / c= h / n

When population standard deviation is unknown inflate standard deviation as follows,

where s = sample size:
2
h = s ( 1 + 20 / n )

(I have had some problems with this method when there are more than
three sources of new data – the variance and the mean don’t change as
expected; I have yet to work out what is going on.

What happens when the new data increases the uncertainty e.g. In a Court
case there is some evidence that is we believe to be compelling, the
Council for the plaintiff will present it forcefully so that the judge and jury
increase their belief in it, whereas the council for the defence will do the
reverse. In addition the quality of the respective QCs in their presentation,
of which we have had previous experience, will affect the perception of the
evidence by virtue of their performance.

Clearly, there is a requirement for further exploration of Bayes Theorem in

areas of value judgement.)

8. The specific example 2

I notice my bathroom scales are a bit variable –

my prior belief is m0 = 174 and h0 = 4.69

I weigh myself 10 times:

 = 176 and s=  10 = 3.16

Correct sample standard deviation for small sample size:

2
h = s ( 1 + 20 / n ) =  10 ( 1 + 20 / 100) = 3.79

Prior precision: =0.0455

Sample precision: = 0.694
Posterior precision: = 0.740
Posterior mean: = 175.9
Posterior standard deviation: = 1.163
8. A ‘generalised’ influence diagram of the risk process at Metronet

In the influence diagram below I have tried to capture the general risk/business
process, however, for the purpose of entertainment I have included two
hypothesised authoritative risk research studies that may influence the way the
Lenders’ Technical Advisors (LTA) due diligence team view Metronet’s risk
assessment.

a. In Report 1, a study of 15 government capital projects states that costs

exceeded quotes by 40%.

b. In Report 2, a study of 30 railway projects, in all areas, stated that cost

over-runs amounted to 25%.

c. The consultant has worked on 6 rail projects and has observed that cost
over-runs amounted to 23% on average.

d. Assume that Metronet have allocated 10% contingency for all cost over-
runs.

In the attached spreadsheet the calculation is made for the Bayesian revision.