Accident Analysis and Prevention 39 (2007) 546–555

Empirical Bayes before–after safety studies: Lessons learned from two

decades of experience and future directions
Bhagwant Persaud ∗ , Craig Lyon
Ryerson University, Department of Civil Engineering, 350 Victoria Street, Toronto, Ontario M5B2K3, Canada
Received 7 May 2006; received in revised form 28 August 2006; accepted 21 September 2006

Abstract
The empirical Bayes (EB) methodology has been applied for over 20 years now in conducting statistically defendable before–after studies of the
safety effect of treatments applied to roadway sites. The appeal of the methodology is that it corrects for regression to the mean and traffic volume
and other changes not due to the measure. There is, therefore, a natural tendency to put a stamp of approval on any study that uses this methodology,
and to assume that the results can then be used in specifying crash modification factors for use in developing treatments for hazardous locations, or
in designing new roads using tools such as the interactive highway safety design model (IHSDM). At the other extreme are skeptics who suggest
that the increased sophistication and data needs of the EB methodology are not worth the effort since alternative, less complex methods can produce
equally valid results. The primary objective of this paper is to capitalize on experience gained from two decades of conducting EB studies around
the world to illustrate that the EB methodology, if properly undertaken, produces results that could be substantially different and less biased than
those from more conventional types of studies. A secondary objective is to emphasize that caution is needed in assessing the validity of studies
undertaken with the EB methodology and in using these results for providing crash modification factors. To this end, a number of issues that
are critical to the proper conduct and interpretation of EB evaluations are raised and illustrated based on lessons learned from recent experience
with these studies. These include: amalgamating the effects on different crash types; the specification of the reference/comparison groups; and
accounting for traffic volume changes. Current and future directions, including the improvements offered by a full Bayes approach, are discussed.
© 2006 Elsevier Ltd. All rights reserved.

Keywords: Safety countermeasures; Road safety; Empirical Bayes; Bayes; Safety evaluation; Before–after studies

1. Introduction
There is an undisputed need to evaluate the safety effect of
roadway improvements that may impact accident frequency.
What seems to be still in dispute is whether or not it is the
worth the effort of using sophisticated methodology such as
the empirical Bayes (EB) procedure (Hauer, 1997) for conduct-
ing observational before–after studies. This is because (a) the
relative complexity of the methodology requires analysts with
considerable training and experience, (b) the data needs can be
quite extensive, and (c) the result of (a) and (b) is that the per-
sonnel and financial resource needs can be prohibitive.
The EB methodology has been developed to account for
regression-to-the-mean effects that arise when sites with ran-
domly high short-term accident counts are selected for treatment
and experience a reduction in accidents subsequently when
∗ Corresponding author. Tel.: +1 416 979 5345; fax: +1 416 979 5122.
E-mail address: bpersaud@ryerson.ca (B. Persaud).
these counts regress towards their true long-term mean. While
compelling evidence of the existence of the regression-to-the-
mean phenomenon has been presented (Hauer, 1997; Hauer and
Persaud, 1983) there is a certain amount of skepticism about the
need for the EB methodology, doubts that are fuelled by a belief
that road safety improvements are implemented for a variety of
reasons and that a randomly high accident count is not among the
key selection criteria. Numerical data from Norway, presented
by Elvik (2004), appears to support this belief in part in that both
treated and untreated sites over several years were equally likely
to have above or below normal accident rates.
Further skepticism about the need for the EB methodology
arises from the belief that when many years of pre-treatment data
are used to select entities for treatment or in an evaluation, and
these entities have high accident counts, there will be little or
no regression-to-the-mean. While there is some validity to this
belief, it is difficult to establish how many years of pre-treatment
data are required or how high accident counts need to be for
regression-to-the-mean to be virtually non-existent. Evidence
in Hauer and Persaud (1983) suggests that there is fairly large

0001-4575/$ – see front matter © 2006 Elsevier Ltd. All rights reserved.
doi:10.1016/j.aap.2006.09.009
 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555
regression to the mean for two-lane rural road segments even for
a 5-year “before” periods, illustrating that a long before period
will not eliminate regression-to-the-mean, especially when the
average annual accident count is relatively small. This tends to be
the case for many types of treated entities such as short two-lane
rural road segments, low volume stop controlled intersections
and rail-highway crossings.
The more conventional alternatives to the EB method, involv-
ing a simple before–after comparison of accident counts or rates,
with or without a comparison or control group, are appeal-
ing in that they are relatively easy to apply. These alternative
methods, however, are fraught with difficulties, which are well
documented. The “best of the rest” involves a process in which
sites are selected for possible treatment on the basis of their
safety record and then randomly allocated to either a treatment
or a control group—a classical experimental design. This would
create similar accident frequency distributions in the two groups,
allowing for regression-to-the-mean effects to be controlled for.
In practice, this method of project selection is problematic since
there may be moral and liability issues if some sites that end
up in the control group are more worthy of treatment than some
in the treatment group. In addition, this method will not control
for changes in safety resulting from changes in traffic volume
at the treatment sites that might result from the treatment itself.
Measures such as left turn treatments at intersections are known
to have such effects.
To avoid these issues in using a control group, a quasi-
experimental design is commonly used in which an untreated
“comparison” group of sites similar to the treated ones is selected
separately from the treatment site selection process. A com-
parison group can account for unrelated effects such as time
and travel trends but will not account for regression-to-the-
mean unless sites are precisely matched on the basis of accident
occurrence in addition to all the factors that affect accident occur-
rence. There are immense practical difficulties of achieving this
ideal as illustrated in Pendleton (1996). In addition, the neces-
sary assumption that the comparison group is unaffected by the
treatment is difficult to test and can be an unreasonable one in
some situations. And this method, like the classical experimen-
tal design, will not control for changes in safety resulting from
changes in traffic volume at the treatment sites that might result
from the treatment itself. Most fundamentally, the comparison
group needs to be similar to the treatment group in all of the
possible factors that could influence safety. A paper by Scopatz
(1998) points to the difficulties of fulfilling this need by examin-
ing the result from Hingson et al. (1996) that lowering legal BAC
limits to 0.08% resulted in a 16% reduction in the probability that
a fatally injured driver would have a BAC above that level. The
treatment group consisted of States that passed a lower legal
BAC law while the comparison states retained a 0.10% BAC
legal limit. Scopatz showed that if logically valid but different
comparison states are chosen the results change dramatically,
and in most cases are in fact consistent with a conclusion of “no
effect”.
The empirical Bayes (EB) method (Hauer, 1997) can over-
come the limitations of conventional methods by accounting not
only for regression-to-the-mean effects, but also for traffic vol-
547
ume changes and for time trends in accident occurrence due to
changes over time in factors such as weather, accident report-
ing practices and driving habits. However, there are a number
of difficulties which, if not properly resolved, will render this
methodology just as invalid as the conventional methods, result-
ing in a misuse of precious resources and a general lack of faith
in the method. It is important to recognize and address these
issues since it is natural for those involved in safety manage-
ment to give a stamp of approval to results from an EB study
just because they claim to have been produced by such a statis-
tically sound methodology.
Given the two extremes in beliefs on the EB methodology –
blind faith and skepticism – it seems worthwhile and timely to
address the concerns in both camps by consolidating the lessons
learned in conducting EB evaluations over the past 20 years or
so since the first applications of this methodology. This need is
the motivation for this paper. First, the basics of EB evaluation
are reviewed. This is followed by three substantive sections,
one that presents evidence supporting the need for and valid-
ity of the EB approach, one that uses the results from several
published before–after studies to compare estimates of safety
effect obtained by the EB and the naı̈ve methods, and one that
addresses issues in EB evaluations that need to be considered in
assessing the validity of EB studies.
2. Basics of empirical Bayes evaluation
In the empirical Bayes evaluation of the effect of a treatment,
the change in safety for a given crash type at a treated intersection
is given by
B − A, (1)
where B is the expected number of crashes that would have
occurred in the “after” period without the treatment and A is
the number of reported crashes in the after period. Because of
changes in safety that may result from changes in traffic volume,
from regression-to-the-mean, and from trends in crash reporting
and other factors, the count of crashes before a treatment by itself
is not a good estimate of B (Hauer, 1997)—a reality that has
now gained common acceptance. Instead, B is estimated from an
empirical Bayes (EB) procedure (Hauer, 1997) in which a safety
performance function (SPF) is used to first estimate the number
of crashes that would be expected in each year of the “before”
period at locations with traffic volumes and other characteristics
similar to a treatment site being analyzed. The sum of these
annual SPF estimates (P) is then combined with the count of
crashes (x) in the before period at the treatment site to obtain
an estimate of the expected number of crashes (m) before the
treatment. This estimate of m is
m = w1 (x) + w2 (P). (2)
The weights w1 and w2 are estimated as
P
w1 = , (3)
P + 1/k
548 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555

Column 9: EB
w2 = , (4)
k(P + 1/k)

5.0
−4.8
7.4
1.1
−2.5
3.5
−8.6
−0.5
−10.5
18.6
−13.2
−9.0
5.5
where k is the dispersion parameter of the negative binomial dis-
tribution that is assumed for the crash counts used in estimating
Percent difference

the SPF. The value of k is estimated from the SPF calibration

Column 8: CG

process with the use of a maximum likelihood procedure.

A factor is then applied to m from Eq. (2) to account for
Increase

−14.6
−14.0
−22.3
−17.1
−22.9

−23.6
−19.8
−9.7 the length of the after period as well as differences in traffic

−8.6
17.6
4.3

1.3
volumes and general trends in crash risk due to factors such as
weather, reporting practices and the other safety countermea-
Column 7: EB

sures between the before and after periods. This factor is the
sum of the annual SPF predictions for the after period divided
by P, the sum of these predictions for the before period. The
0.20
0.42
0.67
0.92
1.20
1.43
1.75
2.00
2.37
2.64
3.02
3.33
4.73
result, after applying this factor, is an estimate of B. The proce-
accidents/site per year

dure also produces an estimate of the variance of B, the expected

number of crashes that would have occurred in the after period
Column 6: CG
in 1997–1999

without the treatment.

Estimate of

The estimate of B is then summed over all road sections in a

0.00
0.34
0.69
1.03
1.37
1.72
2.06
2.40
2.75
3.09
3.43
3.78
5.46

treatment group of interest (to obtain Bsum ) and compared with

the count of crashes during the after period in that group (Asum ).
The variance of B is also summed over all sections in the group
Using data for 1669 California rural stop controlled intersections to illustrate regression to the mean and the validity of the EB method
Column 5: observed

of interest.
The index of safety effectiveness (θ) is estimated as
percent change

Asum /Bsum
Increase

θ= . (5)
−12.0
−11.4
−20.0
−14.6
−20.6

−21.3
−17.4

1 + [Var(Bsum )/Bsum
2 ]
−7.0

−5.9
21.2
7.4

4.3

The standard deviation of θ is given by

accidents/site per year

Stddev(θ)
 0.5
θ 2 {[Var(Asum )/Asum 2 ]+[Var(Bsum )/Bsum 2 ]}
in 1997–1999

= 2
. (6)
Column 4:

[1 + Var(Bsum )/Bsum 2 ]
0.21
0.40
0.72
0.93
1.17
1.48
1.60
1.99
2.12
3.13
2.62
3.03
4.99

The percent change in crashes is in fact 100(1 − θ); thus a

value of θ = 0.7 with a standard deviation of 0.12 indicates a
accidents/site per year

30% reduction in crashes with a standard deviation of 12%.

3. Evidence supporting the need for and validity of the

in 1994–1996
Column 3:

empirical Bayes approach

0.33
0.67
1.00
1.33
1.67
2.00
2.33
2.67
3.00
3.33
3.67
5.30

Evidence presented in Hauer (1997), Hauer and Persaud

0

(1983) and Persaud and Hauer (1984) based on a large number

of diverse datasets has demonstrated that that regression-to-the-
Column 2: accidents/site

mean effects can be quite substantial and that the EB method

is a very effective tool for accounting for this effect. In keeping
in 1994–1996 (x)

with the objectives of this paper it is still useful to present some

additional evidence and discussion.
Data in Table 1 pertain to accident counts at 1669 rural,
> = 12

4-legged stop controlled intersections in California. These inter-

10
11
0
1
2
3
4
5
6
7
8
9

sections averaged 0.81 accidents per year during 1994–1996

and 0.83 accidents per year in 1997–1999. Intersections are
with (x) accidents

grouped into rows based on the count of accidents in 1994–1996,

Column 1: sites

in 1994–1996

shown in column 2. As column 5 shows, those intersections in

groups which in 1994–1996 had more than the average number
Table 1

of accidents in this period experienced a reduction in accidents in

584
348
203
144
103
63
56
31
28
31
21
11
46

1997–1999 in all except one case. And intersections with fewer

 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555
accidents than the average experienced considerable increases.
These changes are due to regression-to-the-mean since these
intersections were largely unaltered during the 6-year period
from 1994 to 1999, according to information in the Highway
Safety Information System (HSIS) (Griffith and Council, 2000)
from which these data were extracted. For the numbers in col-
umn 6, a comparison ratio of (0.83/0.81), which is based on all
1669 sites in the dataset, was applied to the numbers in column
3 as might be done in a “simple before–after with comparison
group” (CG) method. It is seen that the regression-to-the-mean
effects still remain substantial because the comparison ratio is
so small in this case.
The 1994–1996 accident count data were used, along with
safety performance functions estimated from the entire database,
to develop EB estimates for each group of intersections, using the
methodology detailed earlier. These estimates are shown in col-
umn 7. As mentioned, these are estimates of what safety would
be in a subsequent after period at a treated site had treatment not
been applied. Since treatment was not in fact applied to these
intersections, the actual “after” period (1997–1999) count rep-
resents in this case “what safety would be in a subsequent after
period at a treated site had treatment not been applied”. Thus the
validity of the EB estimates of the 1997–1999 frequencies in the
absence of treatment (column 7) can be assessed by comparing
them to the actual accident counts in 1997–1999 (column 4) and
contrasting them to estimates that would have been obtained by
the CG method.
The comparison of the estimates in columns 6 and 7 with
the actual accidents per year in column 4 is reflected in the per-
centage differences shown in columns 8 and 9. This comparison
reveals that the EB estimates are closer to the 1997–1999 counts
than are the estimates from the CG method. Most importantly,
it is seen from the numbers in columns 8 that the CG method
systematically overestimates these counts for sites with above
average accident frequencies. By contrast, as seen in column
9, the EB method appears to be unbiased in that it sometimes
overestimates and sometimes underestimates the frequencies for
these sites.
The superiority of the EB method is illustrated vividly in
Fig. 1, which was constructed by first subtracting the numbers in
columns 6 and 7 from those in column 4, for each row in Table 1,
to find the mean residuals for the CG and EB predictions when
compared to the observed accident frequency in 1997–1999.
Fig. 1. Cumulative mean residuals based on 1994–1996 accident count group-
ings in Table 1.
549
The cumulative residuals for each were then plotted against the
1994–1996 accident count (column 2).
Fig. 1 graphically confirms what is shown in Table 1, in that
the EB predictions not only estimate efficiently, as evidenced by
the closeness of the plotted line to 0.0 on the Y-axis, but also
produce unbiased estimates since the plot oscillates about the
X-axis. By contrast, the CG predictions show substantial bias
in consistently under-predicting when the 1994–1996 count is
less than about 2.0 (when the line is rising) and overpredicting
when the count is higher than 2.0 (i.e., when the line is falling).
The upshot is that regression-to-the-mean does exist and cannot
usually be accounted for by the CG method.
4. Comparing before–after study results obtained by
EB and traditional methods
Having provided further evidence and discussion to support
the validity of, and need for the EB methodology, it remains to
see if the EB results from actual before–after studies are mate-
rially different from those that would have been obtained with
more traditional methods. To do so, we have summarized in
Table 2 the results of a number of EB studies that one or both
of the authors have been involved in over the years. These are
compared to results that would have been obtained with a naı̈ve
before–after analysis.
The two sets of analysis essentially differ in the values of
“expected after without treatment” shown in the second and third
columns. For the conventional (naı̈ve) method this value (in the
second column) was obtained by multiplying the “before” period
count for each site by the ratio of the “after” period length to the
“before” period length. Where AADT information was available
for both the before and after periods, as was the case for round-
abouts, centre-line rumble strips and red light cameras, the ratio
of the after period to before period traffic volume was applied
as an additional factor. This ratio assumes a linear relationship
between crashes and traffic volumes, an assumption generally
found to be invalid. In no case were the naı̈ve estimates cor-
rected for time trend in accident frequencies using a comparison
group, although this might have been possible if this was the
primary study method. For red light cameras, centre-line rum-
ble strips, and conversion from signal to all-way stop, this trend
was known, based on available reference group data, to be an
increasing one, so the naı̈ve estimates shown are likely to be on
the low side for these three treatments.
For the EB analysis, the estimates of “expected after with-
out treatment” in the third column were obtained as detailed
in Eqs. (1)–(6) presented earlier. These estimates, as noted, cor-
rect for differences in length and underlying accident experience
between the before and after periods. They also correct for traffic
volume changes between the before and after period where such
changes are known. (For rail crossing protection upgrades, con-
versions from signal to all-way stop, and for some roundabout
conversions, it had to be assumed that the before and after period
traffic volumes were the same, because of the unavailability of
data for both periods.)
Most importantly, the EB estimates correct for regression-
to-the-mean, which is perhaps the main reason for the often
 550
Table 2
Comparison of EB and naı̈ve study results
Column 1: accident type Expected after without treatment Column 5: accident Column 6: naı̈ve Column 7: EB Naive/EB reduction
count after accident reduction accident reduction
Column 2: naı̈ve Column 3: EB Column 4: % diff. Column 8: number Column 9: %

Converting 222 intersections from two- to all-way stop (Persaud et al., 1984)
All 1329 1079 18.8% 616 713 (54%) 463 (43%) 1.54 1.26
Injury 313 226 27.8% 60 253 (81%) 166 (73%) 1.52 1.11

B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555

Right-angle 726 558 23.1% 126 600 (83%) 432 (77%) 1.39 1.08
Rear-end 151 123 18.5% 101 50 (33%) 22 (18%) 2.27 1.83
Pedestrian 139 123 11.5% 75 64 (46%) 48 (39%) 1.33 1.18
Installing gates at 934 rail crossings with flashers (Hauer and Persaud, 1987)
All 286 208 37.9% 114 172 (60%) 94 (45%) 1.83 1.33
Installing gates at 1037 rail crossings with crossbucks (Hauer and Persaud, 1987)
All 239 162 32.2% 50 189 (79%) 112 (69%) 1.69 1.14
Installing flashers at 891 rail crossings with crossbucks (Hauer and Persaud, 1987)
All 165 101 38.8% 49 116 (70%) 52 (51%) 2.23 1.37
Change 189 intersections from signal to all-way stop (Persaud et al., 1997)
Angle/turn 636 625 1.7% 476 160 (25%) 149 (24%) 1.07 1.04
Rear-end 128 144 −12% 102 26 (20%) 42 (29%) 0.62 0.69
All 1056 1063 −0.7% 809 247 (23%) 254 (24%) 0.97 0.96
Centre-line rumble strips on 211 miles of two-lane roads (Persaud et al., 2004)
All 1961 2030 −3.5% 1777 184 (9%) 253 (12%) 0.73 0.75
Injury 769 749 2.6% 647 122 (16%) 102 (14%) 1.20 1.14
Roundabouts at 23 US intersections (Persaud et al., 2001)
All 553 455 17.7% 275 278 (50%) 180 (40%) 1.54 1.25
Injury 84 58 31.0% 12 72 (86%) 46 (79%) 1.57 1.09
Red light cameras at 132 intersections in 7 jurisdictions (Persaud et al., 2005)
Angle all 1580 1541 2.4% 1163 417 (26%) 378 (25%) 1.10 1.04
Angle injury 912 896 1.8% 634 278 (30%) 262 (29%) 1.06 1.03
Rear-end all 2399 2531 −5.5% 2896 −497 (−21%) −365 (−14%) 1.36 1.50
Rear-end inj. 944 984 −4.2% 1008 −64 (−7%) −24 (−2.4%) 2.67 1.92
Left turn priority treatment at 35 traffic signals (Lyon et al., 2005)
Inj. left turn (LT) 215 180 16.3% 152 63 (29%) 28 (16%) 2.25 1.81
Inj. LT side impact 157 165 −5.1% 135 22 (14%) 30 (18%) 0.73 0.78
 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555
substantial difference between them and the naı̈ve estimates.
This difference, which is shown as percentages in the fourth
column of Table 2, is of interest by itself in that it is largest for
two sets of measures—conversion from two-way to all-way stop
control and upgrading protection at rail-highway crossings. In
both cases, there was a known tendency to quickly apply the
treatment in response to the occurrence of one or more recent
accidents—the classical situation for regression-to-the-mean. At
the other extreme, are the three cases for which this difference
between the EB and naı̈ve estimates is not substantial—red light
cameras, centre-line rumble strips, and conversion from signal
to all-way stop. This is not to say that there was little or no
regression-to-the-mean. In fact, these are the same three cases
noted above for which the naı̈ve estimates of “expected after
without treatment” are likely to be on the low side because they
did not correct for a significant increasing trend in accident fre-
quency.
The most significant conclusion from the range of values
in column 4 is that it seems almost impossible to estimate the
amount of regression-to-the-mean that was present in reported
studies that used the naı̈ve method.
Another key revelation from Table 2 is in the difference
between the naı̈ve reduction in column 6 (the difference between
columns 2 and 5) and the EB reductions in column 7 (the differ-
ence between columns 3 and 5). In terms of % reduction, it is
seen from the numbers in column 9 that there is a relatively small
difference in many cases between the naı̈ve and EB estimates of
safety effect, which may support a belief that the EB analysis
may not be worth the effort. However, if we estimate safety effect
in terms of the actual reduction in accident frequency rather than
the % reduction, which is what really matters, we see substantial
differences between the EB and naı̈ve estimates, as evidenced
by the ratios in column 8. On this basis, it must be concluded that
EB analysis does in fact make a substantial enough difference.
5. Issues that may affect the validity of EB results
As suggested earlier, one should not use blind faith in assess-
ing the validity of EB studies. This is because there are a number
of tricky issues in conducting these studies and, if these are not
properly addressed, the results of an EB study can be just as
invalid as those from conventional studies. Below, three such
issues are focused on. It should be noted in passing that there
are in fact other issues that are not addressed and that aspects
of these issues apply also to the more conventional evaluation
Table 3
Economic evaluation of the safety effect of red light cameras (from Council et al., 2005)
EB estimate of crashes expected in the after period without RLC
Count of crashes observed in the after period (370 site years)
% change in crashes (standard error) [negative is decrease]
Estimate of the change in crashes [negative is decrease]
Crash cost change [negative is decrease] ($)
% change in crash cost [negative is decrease]
551
methods. These other issues relate to the assessment of poten-
tial crash migration or spillover effects to non-treatment sites,
the assessment of how site characteristics may impact on the
effect of a treatment, the difficulties caused when treatments are
applied in combination, as is typically the case, and the com-
plications that arise when treatment effects vary over time. The
three issues that are addressed are the differential effects for dif-
ferent crash types, the specification of the reference groups, and
the consideration of the effects of traffic volume changes.
5.1. Issue 1: differential effects for different crash types
Most treatments affect various accident impact and severity
types differently. Therefore, in assessing the overall impact of
a treatment it is necessary to somehow amalgamate these dif-
ferential impacts. This is especially critical when a measure has
positive impacts on some accident types and negative impacts
on others. Examples of such measures are conversion to traf-
fic signal control and installation of red light cameras (RLC),
both of which are known to increase rear-end crashes but to
reduce the more severe right-angle crashes. Some recent results
from Persaud et al. (2005) and Council et al. (2005) for an
FHWA study emphasize the importance of properly weighting
these effects in arriving at an overall impact. Until then other
researchers would merely report these effects separately with-
out attempting to derive a net safety benefit of RLC programs,
information that is crucial to the continuation of such programs.
The FHWA research involved an EB study that confirmed the
conventional belief that RLCs increase rear-end and decrease
right-angle crashes. This was followed by an examination of
the economic costs of these changes, based on a consideration
of specially derived rear-end and right-angle unit crash costs
for various severity levels for urban signalized intersections
(Zaloshnja et al., 2004), in order to establish the aggregate effects
of the RLC programs evaluated. These results, shown in Table 3,
suggest that costs of the right-angle crashes saved clearly out-
weigh the cost of the increased rear-end crashes, even though
the net savings in crashes was not substantial.
5.2. Issue 2: specification of reference groups
In the EB methodology, safety performance functions need
to be calibrated for each of the before and after periods and
desirably for each year of these periods. A reference group of
“similar” entities to the treated ones is used for this purpose.
Right-angle Rear-end
1542 2521
1163 2896
−24.6 (2.9) 14.9 (3.0)
−379 375
−18,497,977 5,875,156
−27.7 8.5
552 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555
Typically, multipliers are estimated for each period or each year
of each period. Three primary considerations arise in specifying
this reference group.
First, it almost goes without saying that the reference group
must be representative of the treated entities. That is, the ref-
erence group must be similar to the treated group in terms of
geometric design, traffic volumes, vehicle fleet, and so on. Where
the reference group is also used to account for time trends, a test
of comparability should be applied to evaluate its suitability in
this respect. In essence, this test of comparability compares a
time series of target accident counts for a treatment group and
a candidate reference group. If a candidate reference group is a
good one, then the yearly trends in accident frequencies track
each other well over time. Hauer (1997) proposes calculating a
sequence of sample odds ratios using 1 year of “before” data and
the following year as the “after” data, starting with years 1 and
2 and incrementally increasing by 1 year. From this sequence
of ratios, the sample mean and standard error is determined. If
this sample mean is not sufficiently close to 1.0 then the can-
didate reference group is unsuitable. As an example, consider
the data from one of the cities used to study the effects of red
light cameras that was referenced earlier (Persaud et al., 2005).
The mean and standard errors of the sequence of sample odds
ratios for total accidents were estimated to be 1.045 and 0.150,
respectively. If we selected a 95% confidence interval the odds
ratio is estimated to be between 0.751 and 1.339, suggesting that
there is not sufficient evidence to conclude that the odds ratio is
in fact not 1.0 and, therefore, that the candidate reference group
cannot be rejected on this basis.
The second consideration arises when all or most of the enti-
ties that would form a potential reference group are treated.
In this case, there is no natural reference group. However,
regression-to-the-mean is unlikely in evaluating the compos-
ite treatment effect over all sites, since all or almost all sites
are selected for treatment and not just those with a high acci-
dent frequency. Nevertheless, an EB evaluation is still preferred
because it can be informative to look separately at the disag-
gregate treatment effects for sites with high and low accident
counts. In doing an EB study in this case, the “before” period
data for the treatment group can be used to develop the SPF for
use in the EB methodology. Factors for the “after” period at the
treatment sites could not be obtained this way since the after
period SPF is required to estimate what would happen without
the treatment. In this case, another entity set for the jurisdiction
is used to derive a trend between the after and before period, in
effect a ratio of the “after” period SPF multiplier to the “before”
period multiplier. This trend factor is then applied to the SPF
based on the “before” period data. This procedure was followed
for a recent study of raised pavement markers (RPMs), which
were installed non-selectively for four-lane freeways in Mis-
souri and Pennsylvania and for two-lane roadways in Illinois
and New Jersey. For these, the reference group information used
for calibrating SPFs comprised the before period data at all the
identified locations with RPMs. This meant that data available
for calibrating the SPFs would be non-existent for the period
after non-selective installation was complete, and could be lean
toward the end of the installation period. To calibrate the SPFs
for these later years, a comparison group of sites that consisted of
as yet untreated locations, or locations on which RPMs had been
installed prior to the beginning of the study period, was identified
where possible, to account for time trends between the SPF cal-
ibration period and the rest of the analysis period. For example,
for four-lane freeways in Missouri and Pennsylvania, the com-
parison group consisted of a sample of multilane (non-freeway)
roadways.
The third issue is that in some cases the treatment may affect
the logical reference group. Red light camera programs are a
classical example, but there is evidence of this effect for other
measures, such as traffic calming, all-way stop installation, and
raised pavement markers. A good example of how this issue can
affect study results is an evaluation of raised pavement mark-
ers by Orth-Rodgers Associates Inc. (1998) who estimated the
effects on nighttime crashes at 91 interstate highway locations
in Pennsylvania for “before” and “after” periods of 1–3 years.
Daytime crashes at the same sites were used as a comparison
group. The authors found an insignificant 1.2% decrease in all
night time crashes, but suspected that the lack of an “expected”
positive effect might have been due to the fact that there was a
reduction in the daytime crashes (due to the rumbling effect of
RPMs) that was used for the comparison group.
In the case of red light cameras (RLC), the actual hope is
that there would be a general deterrent or spillover effect at all
signalized intersections, not just those with cameras, especially
if the public does not know where the cameras are. Ignoring
spillover effects to intersections without RLCs, will lead to an
underestimation of RLC benefits, more so if sites with these
effects are used as a comparison group. To resolve this issue in
the recent EB evaluation of RLCs by Persaud et al. (2005), the
effects of regression-to-the-mean and changes in traffic volume
were explicitly accounted for using safety performance func-
tions (SPFs) relating crashes of different types and severities to
traffic flow and other relevant factors for each jurisdiction based
on signalized intersections without RLCs. Annual SPF multipli-
ers were calibrated to account for the temporal effects on safety
of variation in weather, demography, crash reporting and so on.
This is common practice in applying the EB methodology out-
lined in Eqs. (1)–(6). However, because of the possibility of
spillover effects to neighboring signalized intersections, it was
decided to estimate annual multipliers for the period after the
first RLC installation from the trend in annual multipliers of
SPFs calibrated for a comparison group of unsignalized inter-
sections in the jurisdiction of interest. The assumption here, of
course, is that this comparison group is unaffected by the treat-
ment.
To illustrate, consider the information in Table 4 for one of the
cities that supplied data for the study. Using the SPFs calibrated
with the reference data collected for unsignalized and signalized
intersections, annual multipliers were estimated for each year.
The year of the first camera installation was 1996. In case there
was a spillover effect at the non-treated signalized intersections,
the annual multipliers estimated from the signalized reference
group were applied only for the years 1992–1995.
Yearly multipliers for the years 1996–2002 were then esti-
mated as follows:
 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555 553

Table 4
Development of yearly trend factors when a treatment may affect logical reference sites
Reference group Unsignalized Adjusted unsignalized Signalized Adjusted signalized

1992 1.34 1.61

1993 1.20 1.52
1994 1.17 1.44
1995 0.99 1.15
Avg. 1992–1995 1.18 1.43
1996 0.84 0.71 0.89 1.02
1997 0.78 0.66 0.88 0.95
1998 0.93 0.79 0.94 1.13
1999 0.78 0.66 1.01 0.95
2000 1.23 1.05 1.09 1.50
2001 0.84 0.71 1.21 1.02
2002 1.96 0.82 1.09 1.17

• Step 1. For both the unsignalized and signalized reference this results in an even smaller increase in accidents, that using
groups the average yearly multiplier between 1992 and 1995 SPFs and the EB methodology to account for this change is, in
was determined. effect, overkill. However, changes in traffic volume at treatment
• Step 2. The unsignalized reference group yearly multipliers sites can in fact be much larger, and can go in either direction,
for the years 1996–2002 were divided by the average from because many treatments by themselves cause such an increase.
1992 to 1995. These results are shown in the column labeled Intersection treatments such as left turn accommodation, traffic
“Adjusted Unsignalized”. signal installation, red light cameras and conversion to round-
• Step 3. Yearly multipliers for the signalized reference group abouts are well known to affect traffic volumes. The data in
for 1996–2002 were then estimated by multiplying the aver- Table 5 illustrates this phenomenon for roundabout conversions
age from 1992 to 1995 by the adjusted unsignalized factor for evaluated by Persaud et al. (2001). It appears that many round-
each year. abouts, by alleviating a congestion problem, actually re-attracted
traffic that had gone elsewhere to avoid the congestion.
The data in Table 4 show that with the exception of 2001, the If the objective of the evaluation is to assess a treatment
yearly multipliers are being increased. The mean of the unad- programme in a specific jurisdiction, then these traffic volume
justed and adjusted factors from 1996 to 2002 are 1.02 and 1.11, changes, and the resulting safety changes at the treated sites
respectively, indicating a possible spillover effect, and justifying and elsewhere, are part of the effect that is being evaluated.
the approach used. However, a more common objective has been to estimate the
effect of a treatment to derive an accident modification factor for
5.3. Issue 3: properly accounting for traffic volumes widespread application; in that case, it is of interest to estimate
changes
Table 5
Volume changes on roadways will occur and, typically, Illustrating traffic volume changes for intersections converted to roundabout
AADTs increase over time, of the order of 2–4% per annum. AADT before AADT after % change
These changes, by themselves, will cause accident frequencies
7185 9840 37.0
to increase, usually by less than the increase in traffic because of 7650 8500 11.1
the non-linear relationship between accidents and traffic volume, 7654 9293 21.4
which typically has a decreasing slope. Therefore, in evaluating 11934 12205 2.3
treatments, benefits would be underestimated if these changes 12627 15990 26.6
are not properly accounted for. In a naı̈ve evaluation, the propor- 12627 11010 −12.8
13272 26691 101.1
tional effect on the expected accident frequency after treatment 13300 16900 27.1
is assumed to be the same as the proportional increase in traffic 13972 15500 10.9
volume. This assumption is incorrect, because of the non-linear, 15153 17825 17.6
decreasing slope relationship between accidents and traffic vol- 15300 17000 11.1
ume. This would serve to further exaggerate the benefit of the 15300 17000 11.1
15345 17220 12.2
treatment, if regression-to-the-mean is already present. The EB 15600 18450 18.3
method properly accounts for the effect of the increase in traffic 18000 20000 11.1
volume by using safety performance functions to represent the 18475 27525 49.0
actual relationship, linear or otherwise, between accidents and 18795 31476 67.5
traffic volumes. 18942 30418 60.6
22030 31525 43.1
It can be argued that since the 2–4% increase in traffic is so 27000 30000 11.1
small and within the realm of an insignificant change, and since
554 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555
Table 6
Roundabout evaluations with and without accounting for AADT changes
Accident type Expected after without treatment
Naı̈ve EB
All (with AADT change) 553 455
All (without AADT change) 436 354
Injury (with AADT change) 84 58
Injury (without AADT change) 68 48
the effect of the treatment had there been no change in traffic
volume. This second objective is the focus of this discussion.
To further emphasize the point in the previous paragraph, the
roundabout conversion data were re-analyzed with the assump-
tion that, for all 23 conversions in the study, traffic volumes
were the same for the before and after periods. The results of
this reanalysis are compared in Table 6 with the original results
in Table 2 for which actual changes in traffic volumes were avail-
able for most of the intersections and were accounted for. Quite
clearly, not accounting for AADT changes makes a substantial
difference.
6. Lessons learned and current and future directions
There are two principal messages in the paper. The first is that,
based on evidence from actual studies and empirical data, the EB
methodology, if properly undertaken, does produce results that
are substantially different, and more valid, than those produced
by more traditional methods. It is therefore worth the invest-
ment in data collection and analysis, and in training analysts,
to undertake such evaluations. On the other hand, quick and
dirty conventional evaluations, often done as a compromise of
convenience, will produce questionable results, and should gen-
erally be avoided. The second message is a caution against blind
faith in assessing the validity of studies undertaken with the EB
methodology and in using these results for deriving crash modi-
fication factors. To this end, a number of issues that are critical to
the proper conduct of EB evaluations were raised and illustrated
based on recent experience.
Current and future research seems to be in the areas of
improving SPFs and, more fundamentally, to explore whether or
not the increased sophistication in these is worth the considerable
effort in collecting the required data to develop the best possible
SPFs. Related to this, is the estimation of the negative binomial
dispersion parameter which, as shown in Eqs. (3) and (4), are
crucial to the EB methodology. There are current efforts (Miaou
and Lord, 2003) that recognize that this dispersion parameter is
not constant for a given SPF, as was assumed in all EB studies
done to date; these efforts are aimed at modelling this param-
eter as function of the SPF variables. A useful complement to
this research should be an examination of whether this increased
accuracy matters materially.
Finally, there are several researchers who are currently
exploring full Bayes (FB) modeling (Miaou and Lord, 2003;
Pawlovich et al., 2006) for evaluating safety treatments. There
are a number of attractive characteristics of the full Bayes
Recorded after Apparent reduction Actual reduction
275 278 (50%) 180 (40%)
275 161 (37%) 79 (22%)
12 72 (86%) 46 (79%)
12 56 (82%) 36 (75%)
approach. The modeling framework allows for the specification
of quite complex model forms not easily handled in conven-
tional generalized linear modeling approaches., For example,
models including both multiplicative and additive terms (to rep-
resent point hazards such as driveways), can be estimated with
relative ease. The small sample properties of FB models should
also allow the estimation of valid models with smaller sample
sizes. This may be particularly valuable for evaluating treat-
ments based on relatively rare accident types such as those
involving pedestrians. Full Bayes modeling also provides the
ability to include prior knowledge on the values of the coeffi-
cients in the modeling along with the data collected. Perhaps
most advantageous is the ability to consider spatial correlation
between sites in the model formulation. Spatial correlation con-
siders the effect of one location’s proximity to other locations
on the expected accident frequency. For before–after studies,
spatial correlation will likely be an issue where the treated and
comparison sites are close together. Considering this spatial cor-
relation allows the inclusion of sites geographically close to each
other. A recent study of county-level injury and fatal crashes in
Pennsylvania published in Accident Analysis and Prevention
(Agüero-Valverde and Jovanis, 2006) found spatial correlation
to be significant. If exposure over time is not known then the
comparison group selected should in fact be as close in prox-
imity as possible to the treated site since the exposure is more
likely to be similar than if the comparison sites were farther
away. The disadvantage of the full Bayes approach, at least for
the time being, is that the methodology is quite complex, and
may require a very high level of statistical training, especially
since it does not lend itself readily to implementation in a black
box. Once the FB research gets far enough along, it will be of
interest to do an extensive comparison of the EB and FB results.
Acknowledgements
This paper is based on research conducted by one or
both authors over the years, often in collaboration with other
researchers. The contributions of the other researchers, espe-
cially Dr. Ezra Hauer, who literally wrote the book on the
empirical Bayes methodology, are gratefully acknowledged, as
are those by various persons in a number of agencies that pro-
vided data and guidance. Sponsors of these research projects
deserve special credit for their past and on-going support that
has likely saved lives. These include the Natural Sciences and
Engineering Research Council of Canada, the Federal Highway
Administration, the National Cooperative Highway Research
 B. Persaud, C. Lyon / Accident Analysis and Prevention 39 (2007) 546–555
Program, the Insurance institute for Highway Safety, Transport
Canada, and the Network Centres of Excellence on the Automo-
bile and the 21st Century (Auto21). Versions of this paper were
presented at the Road Safety on Four Continents Conference
held in Warsaw, Poland in October 2005 and at the Transporta-
tion Research Board 2006 Annual Meeting.
References
Agüero-Valverde, J., Jovanis, P., 2006. Spatial analysis of fatal and injury crashes
in Pennsylvania. Anal. Prev. 38 (3), 615–618.
Council, F., Persaud, B., Lyon, C., Eccles, K., Griffith, M., Zaloshnja, E., Miller,
T., 2005. Guidance for implementing red light camera programs based on
an economic analysis of safety benefits. Transport. Res. Rec. 1922, 38–43.
Elvik, R., 2004. To what extent is there bias by selection? Selection for road
safety treatment in Norway. Transport. Res. Rec. 1897, 200–205.
Griffith, M.S., Council, F.M., 1999. The Highway Safety Information Sys-
tem: United States Department of Transportation multi-state safety analysis
database. In: Proceedings of the Traffic Safety on Two Continents Confer-
ence. Malmo, Sweden.
Hauer, E., Persaud, B.N., 1987. How to estimate the safety of rail highway grade
crossings and the safety effect of warning devices. Transport. Res. Rec. 1114,
131–140.
Hauer, E., Persaud, B.N., 1983. A common bias in before and after comparisons
and its elimination. Transport. Res. Rec. 905, 164–174.
Hauer, E., 1997. Observational Before–after Studies in Road Safety: Estimating
the Effect of Highway and Traffic Engineering Measures on Road Safety.
Pergamon Press/Elsevier Science Ltd., Oxford/UK.
Hingson, R., Hereen, T., Winter, M., 1996. Lowering State legal blood alcohol
limits to 0.08%: the effect on fatal motor vehicle crashes. Am. J. Public
Health 86 (9), 1297–1299.
Lyon, C., Persaud, B., Haq, A., Kodama, S., 2005. Development of safety
performance functions for signalized intersections in a large urban area
and application to evaluation of left turn priority treatment. Transportation
Research Record. J. Transport. Res. Board 1908, 165–171.
555
Miaou, S.-P., Lord D., 2003. Modeling traffic crash-flow relationships for inter-
sections: dispersion parameter, functional form, and Bayes versus empirical
Bayes methods. Transport. Res. Rec. 1840: 31–40. TRB, National Research
Council, Washington, DC.
Orth-Rodgers Associates Inc., 1998. Safety and congestion management
research and advanced technology applications—final report. Technical
Assistance to the RPM Task Force, Research Work Order Number 1, Orth-
Rodgers Associates Inc., Philadelphia, PA.
Pendleton, O., 1996. Evaluation of accident analysis methodology. Federal High-
way Administration Report FHWA-RD-96-039.
Persaud, B., Council, F., Lyon, C., Eccles, K., Griffith, M., 2005. Multi-
jurisdictional safety evaluation of red light cameras. Transport. Res. Rec.
1922, 29–37.
Persaud, B., Retting, R., Lyon, C., 2004. Crash reductions following installation
of centre-line rumble strips. Accident Anal. Prev. 36, 1073–1079.
Persaud, B., Retting, R., Garder, P., Lord, D., 2001. Safety effect of roundabout
conversions in the U.S.: empirical Bayes observational before–after study.
Transport. Res. Rec. 1757, 1–8.
Persaud, B.N., Hauer, E., Retting, R., Vallurapalli, R., Mucsi, K., 1997. Crash
reductions following traffic signal removal in Philadelphia. Accident Anal.
Prev. 29, 803–810.
Persaud, B.N., Hauer, E., Lovell, J., 1984. The Safety Effect of Conversion to
All Way Stop Control in Philadelphia. University of Toronto, Department of
Civil Engineering Publication No. 84, p. 14.
Persaud, B.N., Hauer, E., 1984. A comparison of two methods for de-biasing
before and after accident studies. Transport. Res. Rec. 975.
Scopatz, R., 1998. Methodological study of between-states comparisons, with
particular application to.08% BAC law evaluation. In: Presented at the Trans-
portation Research Board 77th Annual Meeting, Washington, DC.
Pawlovich, M., Li, W., Carriquiry, A., Welch, T., 2006. Iowa’s experience
with road diet measures: use of Bayesian approach to assess impacts
on crash frequencies and crash rates. Transport. Res. Rec. 1953, 163–
171.
Zaloshnja, E., Miller, T., Council, F., Persaud, B., 2004. Comprehensive and
human capital crash costs by maximum police-reported injury severity within
selected crash types. In: Proceedings of the Annual Meeting. American Asso-
ciation for Automotive Medicine, Key Biscayne, FL.