Accident Analysis and Prevention 38 (2006) 579–589

Temporal transferability and updating of zonal

level accident prediction models
Alireza Hadayeghi a,∗ , Amer S. Shalaby b,1 , Bhagwant N. Persaud c,2 , Carl Cheung c,3
a Department of Civil Engineering, University of Toronto and Synectics Transportation Consultants Inc.,
36 Hiscott Street, St. Catharines, Ont., Canada L2R 1C8
b University of Toronto, 35 St. George Street, Toronto, Ont., Canada M5S 1A4
c Ryerson University, 350 Victoria Street, Toronto, Ont., Canada M5B 2K3

Received 3 March 2005; received in revised form 22 September 2005; accepted 5 December 2005

Abstract
This paper examines the temporal transferability of the zonal accident prediction models by using appropriate evaluation measures of predictive
performance to assess whether the relationship between the dependent and independent variables holds reasonably well across time. The two
temporal contexts are the years 1996 and 2001, with updated 1996 models being used to predict 2001 accidents in each traffic zone of the City of
Toronto. The paper examines alternative updating methods for temporal transfer by imagining that only a sample of 2001 data is available. The
sensitivity of the performance of the updated models to the 2001 sample size is explored. The updating procedures examined include the Bayesian
updating approach and the application of calibration factors to the 1996 models. Models calibrated for the 2001 samples were also explored,
but were found to be inadequate. The results show that the models are not transferable in a strict statistical sense. However, relative measures
of transferability indicate that the transferred models yield useful information in the application context. Also, it is concluded that the updated
accident models using the calibration factors produce better results for predicting the number of accidents in the year 2001 than using the Bayesian
approach.
© 2005 Elsevier Ltd. All rights reserved.

Keywords: Temporal transferability; Model calibration; Bayesian approach; Safety planning models

1. Introduction tions of alternative transport planning options that are considered

Accident prediction models are used to predict the num-
ber of accidents at intersections, on arterial road sections, on
highways or on transportation networks. Of late, there is con-
siderable interest in transportation planning applications, for
which these models (usually developed at the planning zone
level) are aimed at establishing a relationship between accident
frequency and a number of explanatory variables such as traf-
fic volume, road network characteristics and socioeconomic and
demographic features. It is important for safety planning models
to enable transportation planners to assess the safety implica-
∗ Corresponding author. Tel.: +1 905 704 0763; fax: +1 905 682 4495.
E-mail addresses: ahadayeghi@synectics-inc.net (A. Hadayeghi),
amer@ecf.utoronto.ca (A.S. Shalaby), bpersaud@ryerson.ca (B.N. Persaud),
c22cheun@ryerson.ca (C. Cheung).
1 Tel.: +1 416 978 5907; fax: +1 416 978 5054.
2 Tel.: +1 416 979 5000x6464; fax: +1 416 979 5122.
3 Tel.: +1 416 979 5000; fax: +1 416 979 5122.
for medium to long-range implementation, in a way similar to
how road designers make use of tools such as the Interactive
Highway Safety Design Model (IHSDM)4 to assess the safety
implications of design decisions. For this reason and others, it
is desirable to obtain reasonably accurate safety planning mod-
els that are transferable across time and have good predictive
performance.
In general, the importance of accident prediction models
makes it crucial that they be properly calibrated (Persaud et al.,
2002). The quantity and quality of the data used to calibrate acci-
dent prediction models is very important to the success or failure
of these models. Models calibrated with limited data will not
perform adequately in terms of predictive accuracy. Therefore,
having a reasonable dataset is very important in the development
of accident prediction models. Moreover, the specification of an
4 Interactive Highway Safety Design Model (IHSDM), Federal Highway
Administration, US Department of Transportation.

0001-4575/$ – see front matter © 2005 Elsevier Ltd. All rights reserved.
doi:10.1016/j.aap.2005.12.003
580 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589
appropriate functional model form for calibration of accident
prediction models is a complex task. It gets even more com-
plicated for safety planning models because of the paucity of
these.
Data for transportation planning variables are usually
obtained from surveys held periodically (for example, the Trans-
portation Tomorrow Survey is conducted once every 5 years in
the Greater Toronto Area). These surveys usually cover a com-
prehensive area and are therefore expensive in terms of the time
and cost associated with data collection and analysis. As such,
once a model is well specified to capture the decision process in
one context, it is cost effective to transfer such a model to other
contexts that have limited calibration data as long as the basic
nature of the decision-making process remains the same (Badoe
and Miller, 1995a).
There are two aspects to model transferability—temporal and
spatial. The first, results from application of a model estimated
at one point of time for prediction at another point of time for the
same spatial environment. This occurs whenever estimated mod-
els are applied in a forecasting framework. The second aspect
– spatial transferability – involves the application of a model
estimated in a specific geographic region for prediction in a dif-
ferent one for the same time period. Of course, there can be a
combination of temporal and spatial transfers.
Several research efforts have been conducted to address the
temporal and spatial transferability of transportation planning
models. Most of this research was on trip generation models and
disaggregate and aggregate mode choice models (Ben-Akiva,
1981; Koppelman and Wilmot, 1982; Atherton and Ben-Akiva,
1976; Gunn et al., 1985; Gunn, 2001; Badoe and Miller, 1995b;
Ben-Akiva and Bolduc, 1987; Parody, 1977).
However, no similar efforts have been directed at safety plan-
ning models. This is likely in reflection of the reality that few
safety planning models have been calibrated to date. A litera-
ture search indicated that only few studies have been undertaken
to evaluate and quantify traffic safety impacts in the planning
process (Hadayeghi et al., 2003; Lovegrove and Sayed, 2005;
Ladron de Guevara et al., 2004; NCHRP 8-44, in press).
Of most relevance to the current effort was some work
by Oh et al. (2003) that validated intersection accident pre-
diction models intended for use in the IHSDM from two
perspectives—internal validity and external validity. Internal
model validity examines the strength of the original models in
explaining the underlying phenomenon. External validity exam-
ines the ability of the models to predict crashes over space and
time. The authors recommended that the validation over time
could suffer from omitted variables that influence safety, poorly
measured and surrogate variables, and model functional forms.
On the other hand, the authors argue that internal validity and
validation over space could suffer from omitted variables, site
selection and countermeasure selection bias, poorly measured
and surrogate variables, and misspecification of model func-
tional form.
This paper has two basic objectives. First, it examines the
temporal transferability of the accident prediction models by
using appropriate evaluation measures of predictive perfor-
mance to assess whether the relationship between the depen-
dent and independent variables holds reasonably well across
time.
The two temporal contexts are the years 1996 and 2001, with
models estimated from 1996 data being used to predict 2001
accidents. A series of accident prediction models based on 1996
database were calibrated in the previous study (Hadayeghi et al.,
2003) to estimate the number of accidents in planning zones in
the City of Toronto as a function of planning variables. It should
be stressed that the spatial transferability of accident prediction
models is outside the scope of this paper. However, some insights
on this issue have been provided in some other papers (Persaud
et al., 2002).
The second objective of the paper is to compare alternative
updating methods of temporal transfer and to estimate effects of
the size of the application context data on model transferability.
The updating procedures examined are the Bayesian updating
and updating based on recalibration factors. Models calibrated
for the 2001 samples were also explored, but were found to be
inadequate, so were not tested in the 2001 application context.
In addition, this study investigates the transfer bias caused by
sample size. This issue was not addressed in previous studies
which focused mainly on using only one method and one sam-
ple size. This is important since it is believed a large portion of
differences in coefficients and predictions is caused by random
variation due to selection of data. Also, the goodness of fit mea-
sures resulting from updated models are compared to the models
estimated using the entire dataset in the application context. A
key contribution of this paper is the variety in the validation
methods and measures used, tested and compared.
2. Data
The data used in this study are comprised of accident, socioe-
conomic, demographic and road network characteristics, and
traffic flow for the City of Toronto’s 463 traffic zones for 1996
and 481 traffic zones for 2001. The zone definitions for 1996 and
2001 are not identical. For the purpose of this study, the 2001
zone definition is adopted for both 1996 and 2001.
The data for this study were obtained from three sources.
The accident data for 1996 and 2001, the first dataset, was pro-
vided electronically by the Traffic Data Centre (TDC) of the
City of Toronto. The accident data are geocoded, which allows
the grouping of accidents by traffic zone using GIS tools. The
relevant information for each accident record includes sever-
ity (fatal or non-fatal injury and property damage only (PDO))
and period of day. The latter allows for separate models to be
developed for the morning peak period (6:00–9:00 am) to which
traffic intensity data pertained.
The second dataset, consisting of socioeconomic and demo-
graphic data, was obtained from the 1996 and 2001 Transporta-
tion Tomorrow Survey (TTS), a 1-day travel survey conducted
in the Greater Toronto and surrounding areas (GTA) on approxi-
mately 5% of its population. The survey is conducted to provide
detailed demographic and travel information of GTA house-
hold members to local, regional, provincial road authorities
and to transit operating agencies. The traffic demand data were
obtained from the results of a traffic assignment of the morning
 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589 581

peak-period vehicle trips to the road network using the EMME/2
(1998) software package to estimate vehicle kilometers travelled
(VKT) in each zone. This approximation is necessary because
cordon counts, the only available source of traffic demand data
collected consistently across the city, are made at a limited
number of locations and therefore cannot be used directly and
sufficiently to provide traffic flow estimates for each zone in the
city. It is worth mentioning that it is a common practice in build-
ing travel demand and planning models to use traffic assignment
model outputs.
In addition to the above data, 1996 GIS road files were
obtained from the Integrated Land Use, Transport and Envi-
ronment (ILUTE) group at the University of Toronto, and the
corresponding 2001 database, CanMap Street Files, was attained
from University of Toronto Map Library.
3. Comparison of 1996 and 2001 conditions
Table 1 presents the magnitude of changes in various char-
acteristics in the City of Toronto between 1996 and 2001. The
number of households grew by 3.68%, from 908,499 in 1996
to 943,252 in 2001. The population of the City of Toronto grew
by 2.7% in the 5-year period, while the total population of the
Greater Toronto Area (GTA) increased by 9.3%. The difference
in percent growth between the GTA and City of Toronto is indica-
tive of the reality that more people lived in the suburbs in 2001
as a result of urban sprawl. During this period, the total employ-
ment in the City of Toronto increased by more than 7%.
Total vehicle kilometers travelled (VKT) per person
increased by 15%, from 0.75 in 1996 to 0.88 in 2001. How-
ever, there was little change in total major road kilometers in
the same period, suggesting that traffic conditions should have
worsened from 1996 to 2001. This observation is confirmed by
the average volume to capacity (V/C) ratios in Table 1, which
increased by almost 30% from 1996 to 2001. The percent aver-
age changes in zonal traffic volume from 1996 to 2001 in the
City of Toronto, based on the 1996 zonal definitions, are pre-
sented in Fig. 1 . As seen, most of the zones experienced an
increase in traffic volume in the 5-year period.
Table 1 also shows that the total number of accidents
increased by 27% from 1996 to 2001, an average annual increase
of 5.41%. On the other hand, the number of severe (i.e., fatal plus
injury) accidents increased only by a total of 5.64% in the 5-year
period.
Fig. 2 presents the zonal percentage change in total number
of accidents for the City of Toronto. As can be seen, most of
the zones outside of downtown have experienced a higher per-
centage change in total number of accidents than that for the
downtown zones.
4. Model specification
A series of macro-level, zonal accident prediction models that
estimate the number of zonal accidents in the City of Toronto for
1996 was developed in the work described in Hadayeghi et al.
(2003). A generalized linear modelling approach was employed,
in which negative binomial regression models were developed
separately for total accidents and for severe (fatal and non fatal
injury) accidents as a function of socioeconomic/demographic,
traffic demand and network data variables. A total of 22 possi-
ble explanatory variables describing zonal characteristics were
explored. The definition of variables was based on (a) previous
research for similar studies and (b) an iterative process in which
various definitions were tried and the ones that gave the best
results were ultimately selected.
The following model form was used
n


E(A) = a0 VKTb0 exp bi Xi (1)
i=1

Table 1
Change between 1996 and 2001 data in the City of Toronto
Attribute for independent variables 1996 2001 Difference Percent change (%)

(a) Changes in urban structure, travel behavior and socioeconomic and demographic
Vehicle kilometer travelled (VKT) 1728775 2089257 360482 17.25
Average volume capacity (V/C) 0.22 0.31 0.09 30.18
Average posted speed 42.08 42.48 0.40 0.95
Total population 2305600 2368700 63100 2.66
Number of households 908499 943252 34753 3.68
Number of part-time employees 181224 187890 6666 3.55
Number of full-time employees 875225 943373 68148 7.22
Total number of employees 1056449 1131263 74814 6.61
Total minor road kilometer 4172.66 4406.24 233.58 5.30
Total major road kilometer 859.83 900.73 4.90 0.54
Total number of intersections 15498 16649 1151 6.91

Attribute for dependent variables 1996 2001 Difference Percent change (%)

(b) Change in Number of Accidents

Total accidents 53286 73043 19757 27.1
Severe (fatal and injury) accidents 15818 16764 946 5.64
Total accidents, morning peak period 5317 8691 3374 38.82
Severe accidents, morning peak period 1664 1971 307 15.58
582 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589

Fig. 1. Percent change in average traffic volume from 1996 to 2001 in the City of Toronto based on 1996 zonal definitions.

Fig. 2. Percent change in total number of accidents from 1996 to 2001 in the City of Toronto based on 1996 zonal definitions.
 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589 583

Table 2
Results of negative binomial regression for models I and II
Parameter Yearly total accident model (Model I) Yearly severe accident model (Model II)

1996 Model 2001 Model 1996 Model 2001 Model

Intercept (log of α0 )
Log of VKT
Total major road in kilometer
Number of households (× 10−3 )
Speed
Volume/capacity (V/C)
Intersection density
Goodness of fit measures
Mean Pearson χ2 /d.f.
Pearson R2
α 0.1941
4.1767 (0.2957, 199.47)
0.2656 (0.0347, 58.76)
0.1175 (0.0186, 40.05)
0.0864 (0.015, 33.27)
−0.0431 (0.0071, 36.61)
−1.3678 (0.2766, 24.45)
0.0052 (0.0012, 19.87)
1.0313
37.48%
2.8065 (0.3572, 61.75)
0.2845 (0.0540, 27.81)
0.1960 (0.0197, 93.98)
0.0582 (0.0156, 13.89)
−0.0124 (0.0065, 3.64)
−0.6472 (0.3112, 4.33)
0.0024 (0.0012, 3.98)
1.1559
37.24%
0.2355
2.6618 (0.3239, 67.55)
0.2415 (0.039, 38.41)
0.1225 (0.0202, 36.77)
0.0858 (0.0164, 27.38)
−0.0312 (0.0078, 16.05)
−1.2232 (0.3042, 16.18)
0.0038 (0.0013, 9.14)
1.0346
31.90%
0.2092
1.9593 (0.3736, 27.50)
0.1739 (0.0561, 9.60)
0.2423 (0.0208, 136.31)
0.0650 (0.0162, 16.03)
−0.0096 (0.0067, 2.07)
−0.3963 (0.3193, 1.54)
0.0013 (0.0012, 1.03)
1.0110
43.06%
0.2296

Note: Refer to Eq. (1). The dependent variable is the natural logarithm of yearly total accidents. Parameter estimate (S.E., χ2 ).

where E(A) is the predicted accident frequency per zone per year,
VKT the zonal vehicle kilometers travelled during peak hours
(6:00–9:00 am), Xi the other explanatory variables, a0 , b0 and bi
is the model parameters
The above model form is consistent with the non-linear
relationship between accidents and traffic intensity (measured
in this case by VKT) suggested by several other researchers.
Separate models are calibrated for total and severe (fatal plus
non-fatal injury) accidents per year and, for each, models were
estimated for the entire day and for the morning peak period
(6:00–9:00 am) to which the VKT and other traffic variable
estimates from EMME/2 pertain. The final model parameters
and corresponding χ2 and goodness-of-fit criteria are shown in
Tables 2 and 3. This Table is taken from Hadayeghi et al. (2003)
which discusses the modelling approach and related issues.
The models developed in this study for both 1996 and 2001
include major arterial and minor road data in each zone, but
exclude freeway data since the presence of a freeway in a zone
was found after exploratory analysis to be generally unrelated
to characteristics of the zone.
5. Measures of model transferability
The transferability of a model relates to the application of
a model estimated in one context based on adequate data to
another context with limited data. Model transferability is based
on the idea that the estimated model from a previous study in
a different context may provide useful information for estimat-
ing parameters appropriate for the application context. “Local”
or “estimation” context refers to the context for which the ade-
quate dataset was used in the model estimation. The “transfer”
or “application” context refers to the context for which predic-
tions are required, and to which the model from the local context
is transferred for application.
Transportation planning models are not perfectly transferable
between contexts since the models do not capture the behavior
of individuals completely and the data available for the esti-
mation of the unknown parameters are not error free due to a
variety of sources such as sampling errors (Lerman, 1981). As
Lerman (1981) and Ben-Akiva (1981) discuss, it is appropriate
to describe the measure of transfer effectiveness on a continuous

Table 3
Results of negative binomial regression for models III and IV
Parameter Yearly total accident model for Yearly severe accident model for
morning period (Model III) morning period (Model IV)
1996 Model 2001 Model 1996 Model 2001 Model

Intercept (log of α0 )
Log of VKT
Total major road in kilometer
Number of households (× 10−3 )
Number of employements (× 10−3 )
Speed
Volume/capacity (V/C)
Intersection density
Total minor road in kilometer
Goodness of fit measures
Mean Pearson χ2 /d.f.
Pearson R2
α 0.1586
1.7851 (0.2909, 37.66) 0.4328 (0.3127, 1.92) 0.2079 (0.0403, 2.7) 1.0527 (0.4562, 5.32)
0.2225 (0.0430, 26.73) 0.3268 (0.0574, 32.47) 0.2171 (0.0606, 12.85) 0.3163 (0.0803, 15.53)
0.1171 (0.0201, 34.07) 0.2127 (0.0202, 111.39) 0.1316 (0.0264, 24.8) 0.2528 (0.0267, 89.51)
– – 0.0892 (0.0220, 16.43) 0.0701 (0.0216, 10.51)
0.0692 (0.0162, 18.18) 0.0533 (0.0155, 11.80) – –
−0.0324 (0.0073, 19.59) −0.0139 (0.0063, 4.91) −0.0213 (0.0102, 4.34) −0.0141 (0.0087, 2.61)
−1.1156 (0.3296, 11.46) −0.7236 (0.3234, 5.01) −0.9074 (0.4500, 4.07) −0.8137 (0.4412, 3.40)
– – – –
0.0108 (0.0051, 4.52) 0.0006 (0.0049, 0.02) – –
1.0805 1.0802 1.0791 0.9659
36.56% 48.46% 31.57% 42.17%
0.1959 0.1961 0.2290

Note: Refer to Eq. (1). The dependent variable is the natural logarithm of yearly total accidents. Parameter estimate (S.E., χ2 ).
584 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589
scale rather to describe transferability as a dichotomous property.
Transferability is a property that may exist to a greater or lesser
degree depending on the formation of the model and the char-
acteristics of the estimation and application contexts (Lerman,
1981; Ben-Akiva, 1981).
Measures of model transferability were used to assess the
effectiveness of models when they are transferred for use in an
application context (2001 in this study) that is different from
their calibration context (1996). The indices and test-statistics
are all based on log-likelihood measure. These measures, defined
below, are largely similar to those used by Koppelman and Rose
(1983) and Badoe (1994).
5.1. Nested likelihood ratio test (LR)
This measure indicates the statistical similarity of the esti-
mation and application model coefficients. In this test, the null
hypothesis is that the application and estimation context can
be presented by a single model. Basically this test examines
whether the model parameters have remained stable over time
and is given by
LR = −2[LLi∪j (θ̂i∪j ) − LLi (θ̂i ) − LLj (θ̂j )] (2)
The above equation is χ2 distributed with degrees of freedom
equal to the number of model parameters. In the event of rejec-
tion of the null hypothesis, an asymptotic t-test can be performed
to test which parameter(s) were responsible for the rejection. For
this, the t-statistic is given as
 Harwood et al. (2000) in essence obtained a calibration factor
t = (θ̂ki − θ̂kj )/ (var(θ̂ki ) + var(θ̂kj )) (3)
where θ̂ik is the estimate of parameter k in context I and var(θ̂ik )
is the variance of the estimate of parameter k in context i.
5.2. Transfer index (TI)
Transfer index (TI) is a relative measure that indicates how
well a transferred model performs in predicting the application
dataset relative to a locally estimated model in the application
context and of similar specification. For this particular research,
this index compares performance of the transferred and appli-
cation context models relative to the performance of a 2001
“constant” model that has no independent variables. The index
has an upper bound, 1.0, which is attained when the transferred
model performs as well as an estimated model on the applica-
tion data. It has no lower bound. Negative values imply that the
transferred model is worse than the local constant model. It is
expressed as
TI = [LLj (θ̂i ) − LLj (ĉ)]/[LLj (θ̂j ) − LLj (ĉ)] (4)
where LLj (θ̂i ) is the log-likelihood value in context j using
model parameters estimated from year i, LLj (θ̂j ) the log-
likelihood value given by application context model j and LLj (ĉ)
is the log-likelihood value given by constant model estimated in
application context j.
6. Updating 1996 models
This section provides an approach for updating local mod-
els into an application context in a situation when only limited
data are available in the application context for calibrating a
meaningful model. The updating technique is helpful in apply-
ing an “old” estimated model to an application context to predict
present conditions with minimum data requirements, conse-
quently cutting the cost of data collection and time. Also, it
seems desirable, when transferring a model, to update it by
making adjustments based on locally available information. To
illustrate such a situation, and to investigate the feasibility of this
approach, random subsets of zonal information were selected
from the 2001 database. To explore the impact of sample size
on updating performance, 10 samples of each of 10, 20, 25, and
35 zones and their corresponding information were randomly
selected from the 2001 dataset. However, before using updat-
ing procedures, a set of models were estimated using different
sample sizes. These models were used to examine whether the
coefficients in the estimation and application contexts are similar
when limited data are available.
6.1. Updating using a calibration factor
The updating procedure was recommended by Harwood et
al., 2000 and a recent paper (Oh et al., 2003), for application
in the Interactive Highway Safety Design Model (IHSDM) for
transferring accident prediction models calibrated for one geo-
graphic region to apply to another region. In this procedure,
as the total number of accidents observed in a sample from one
jurisdiction divided by the sum of the predicted number of acci-
dents in the sample using the model from another jurisdiction.
The calibration factor was calculated as follows:
n

n


calibration factor = Yi / Ŷi (5)
i=1 i=1
where Yi is the number of observed accidents in zone i in 2001 for
a given sample size and Ŷi is the predicted number of accidents
in zone i by applying values of the 2001 independent variables
(for the same sample size of 2001) to the 1996 model.
Then, the calibration factors estimated for each sample size
were simply multiplied by the base models (1996 models) in
order to “transfer” the 1996 models for application to 2001 con-
ditions.
6.2. Bayesian updating
Bayesian updating, which was introduced by Atherton and
Ben-Akiva (1976), is another approach that is adopted for
updating the 1996 models using the 2001 sample data. This
methodology combines sample information with prior infor-
mation in order to achieve more accurate updated information.
Atherton and Ben-Akiva (1976) assumed both prior and pos-
terior (updated) distributions of the parameters to be normally
distributed. The updated parameters are expressed as
 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589 585

βupdated = [(β96 /σ96

2
) + (β01-sample /σ01-sample
2 2
)]/[(1/σ96 ) A comparison of MSPE and MSE shows potential over or
under fitting of the models to the estimation data. A MSPE that
+ (1/σ01-sample
2
)] (6) is higher than MSE may indicate that models may have been
over fit to the estimation data and that some of the observed
0.5 relationships may have not been correctly shown in the model.
σupdated = 1/[(1/σ96
2
) + (1/σ01-sample
2
)] (7)
where βupdated is the 2001 updated coefficient, β96 the coefficient 7. Results and discussion
estimated from 1996 model, β01-sample the coefficient estimated
from 2001 sample model, σ updated the standard deviation of 2001 7.1. Transferability results
updated coefficient, σ 96 the standard deviation of coefficient esti-
mated from 1996 model and σ 01-sample is the standard deviation The dependent variable of each developed model is the num-
of coefficient estimated from 2001 sample model. ber of zonal accidents per year. The regression parameters were
estimated based on the maximum likelihood method for neg-
6.3. Measures of goodness of fit ative binomial (NB) regression and using PROCGENMOD in
SAS statistical software (SAS Institute Inc., 1999).
Measures of goodness of fit are used to assess the perfor-
mance of the updated models relative to the 1996 models. These 7.1.1. Yearly model for total accidents
measures are similar to those used by Oh et al. (2003). Table 2 presents the parameter estimates and corresponding
χ2 ’s for the 1996 model and the 2001 model for total yearly
6.3.1. Mean prediction bias (MPB) accidents (Model I). A comparison of the parameter estimates in
This measure describes the magnitude and direction of the Table 2 indicates that the speed becomes insignificant at the 5%
average bias in the subject model. The smaller the absolute value level in the 2001 model. The parameter estimates of each variable
of average prediction bias is, the better the model predicts the have identical signs in both models. The dispersion parameters
observed data. Positive values of MPB indicate that the model (α) are in the range of 0.19 and 0.23 for the 1996 model and the
over-predicts the observed accident data, and vice versa: 2001 model, respectively, which indicates that the data are quite
n overdispersed relative to the Poisson distribution and that the


MPB = (Ŷi − Yi )/n (8) negative binomial error structure is therefore justified. The good-
i=1
ness of fit measures for each model, also presented in Table 2,
show the mean Pearson χ2 ratios to be close to 1, an indica-
where Ŷi is the predicted number of accidents at zone i, Yi the tion that the NB regression model fits the data very well. The
observed number of accidents at zone i and n is the data sample Pearson R2 values for the 1996 model and the 2001 model are
size. 37.48% and 37.24%, respectively, which indicate that the devel-
oped models explain the variation in the dependent variable to
6.3.2. Mean absolute deviation (MAD) a reasonable degree. In addition, a test was done to examine
This measure provides a measure of the average mis- whether keeping the dispersion parameter in the 2001 model the
prediction of the model. A value close to zero suggests that the same as the one in the 1996 model would affect the 2001 model
model on average predicts the observed data well. It is defined as performance considerably. It was found that the model parame-
n

ters and corresponding parameters’ signs were almost the same
MAD = |(Ŷi − Yi )|/n (9) as those of Model I. The total predicted number of accidents
i=1 from this model for the whole study area is different by less than
1% from the total from Model I, which is quite insignificant in
6.3.3. Mean squared prediction error and mean squared comparison. Therefore, no further investigation was conducted.
error Results of tests of parameter equality, which were done by
The mean squared prediction error (MSPE) is the sum of the nested likelihood ratio test, are shown in Table 4. These
squared differences between observed and predicted accidents indicate that 1996 and 2001 model parameters are not statisti-
divided by the sample size. It is defined as cally equal. Therefore, the null hypothesis can be rejected. The
n

errors in the modelling calibration procedures are most likely to
2
MSPE = (Ŷi − Yi ) /n (10) affect transferability of the explanatory variables and the esti-
i=1 mated parameters. To investigate whether the rejection of the
null hypothesis is because of an individual coefficient or sub-set
The mean squared error (MSE) is the sum of squared differences
of coefficients, the asymptotic t-test is used to test equality of
between observed and predicted accidents divided by the sample
1996 and 2001 variable coefficient estimates. Part (b) of Table 4
size minus the number of model parameters. It is defined as
presents the results of this test for the yearly model for total
n

2 accidents. As it can be seen, only two explanatory variables
MSE = (Ŷi − Yi ) /(n − p) (11) of VKT and number of households cannot be rejected at the
i=1 5% significance level under the null hypothesis of equality in
where p is the number of model parameters. the two contexts. Theoretically, it can be concluded that the
586 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589

Table 4
Results from transferring 1996 Models to 2001 dataset
Transfer measures Model I Model II Model III Model IV

(a) Absolute and relative transfer measures

Nested likelihood ratio 128.6 52.4 177.4 30.84
LL01 (θ 96 )a (× 105 ) 3.087 0.460 0.179 0.0116
Transfer index −0.966 0.528 −0.881 0.640

Parameter Model I Model II Model III Model IV

(b) Statistical test of parameter equality in two context

Log of VKT −0.29 0.99 −1.45 −0.99
Total major road in kilometer −2.71 −4.13 −3.35 −3.23
Number of households (× 10−3 ) 1.30 0.90 – 0.62
Number of employements (× 10−3 ) – – 0.71 –
Speed −3.19 −2.10 −0.06 −0.54
Volume/capacity (V/C) −1.73 −1.88 −1.21 −0.15
Intersection density 1.65 1.41 – –
Total minor road in kilometer – – 1.44 –
a LL01 (θ 96 ) is the 2001 context log-likelihood using θ 96 from year 1996.

roadway parameters such as major road kilometers and posted
speed did not remain stable over time. The value of transfer
index, which describes the degree to which the log likelihood
of the transferred model exceeds the reference model relative to
improvement provided by a model developed in the application
context, is found to be −0.966. The negative sign implies that
the transferred model is worse than the local reference model.
7.1.2. Yearly model for severe accidents
The model parameters and corresponding χ2 values for the
yearly model of severe accidents (Model II) are shown in Table 2.
All the coefficients are close in magnitude and share the same
sign in the both the 1996 and 2001 models. The intersection
density and V/C variables are insignificant at the 5% level in the
2001 model but are significant in the 1996 model.
The dispersion parameters (α) of 0.2092, and 0.2226 for the
1996 model and the 2001 model, respectively, again confirm the
appropriateness of using negative binomial as opposed to Pois-
son regression. The mean Pearson χ2 ratios for the two models
are within the acceptable range of 0.8–1.2, another indication
that the choice of the negative binomial error structure appears
to be appropriate. The Pearson R2 values for the 1996 model
and the 2001 model are 31.90% and 43.3%, respectively, which
indicate that the developed models explain the variation in the
dependent variable to a reasonable degree.
Results of the test of parameter equality in Table 4 indicate
that the 1996 and 2001 model parameters are not statistically
equal. V/C, speed, and major road kilometers are the parameters
most responsible for the statistical inequality, as shown in part
(b) of Table 4. The value of transfer index of 0.528, suggests that
the transferred model provides some portion of the information
obtained from the 2001 model, irrespective of model specifica-
tion.
7.1.3. Morning peak period accident models
For the morning peak period models, accidents between 6:00
and 9:00 am were extracted from the dataset by running a query
in Microsoft Access. Table 3 shows the parameter estimates for
the total accident models (Model III) and severe accident models
(Model IV) for the morning peak period. For both Models III
and IV, the signs of all variables and constant terms are the same
for the 1996 and 2001 models. The coefficients in the 2001
models are largely different from those of the 1996 models, in
particular, those with low levels of significance such as total
minor road kilometers in the 2001 model. For both Models III
and IV, the coefficient of the total major road kilometers variable
for the 2001 models is three times as large as that for the 1996
models.
The goodness of fit statistics show mean Pearson χ2 values
close to one, again indicating that the models fit the data reason-
ably well. The dispersion parameters (α) for all the models are
significantly different from zero, again confirming the appropri-
ateness of negative binomial regression. The Pearson R2 values
are reasonable for all the models.
The statistical tests of equality of model parameters for the
Models III and IV reject the null hypothesis of equality, indicat-
ing that some or all model parameters have not remained stable
over time. Both Models III and IV exhibit statistically signifi-
cant equality at the 5% significance level in all the parameters,
except major road kilometers. The values of Transfer index are
found to be −0.881 and 0.640 for Models III and IV, respec-
tively, suggest that the transferred model provides some portion
of the information obtained from the 2001 model, irrespective of
model specification. However, the negative value, as explained
previously, implies that the transferred model is worse than the
local reference model.
7.2. Updating results
As expected, none of the models calibrated with limited data
performed reasonably in terms of the magnitude of the coeffi-
cient parameters or their level of significance in the application
context regardless of the sample size. The magnitude of coeffi-
cient parameters for some of the models were the same as the
1996 models but either the parameters were insignificant at the
5% confidence level or the goodness of fit measures were not
Table 5
Goodness of fit measures for predicting 2001 accidents from 1996, 2001 and updated models

A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589

Model types GOF 2001 1996 Updating with sample of Updating with sample of Updating with sample of Updating with sample of
Model Model 10 zones in 2001 20 zones in 2001 25 zones in 2001 35 zones in 2001
Recalibrated Bayesian update Recalibrated Bayesian update Recalibrated Bayesian update Recalibrated Bayesian update
1996 model 1996 model 1996 model 1996 model

Model I MPB 1.17 −37.80 −10.57 (27.39) −30.05 (39.78) −6.87 (15.49) −28.04 (18.50) 1.31 (17.59) −34.15 (26.86) −2.42 (21.94) −40.64 (23.72)
MAD 59.20 65.17 71.62 (4.86) 61.95 (16.97) 69.73 (2.61) 58.85 (16.14) 71.51 (5.38) 65.19 (18.26) 71.33 (5.61) 65.77 (16.39)
MSE 8067
MSPE 9294 9504 (1690) 8993 (5823) 8956 (294) 9685 (3506) 9066 (688) 8392 (4600) 9205 (673) 8762 (4463)
Model II MPB 2.31 −0.51 −0.66 (8.59) 7.49 (20.33) 0.54 (3.05) −2.69 (5.41) 0.62 (2.12) 0.53 (1.80) −1.17 (4.71) 3.21 (6.00)
MAD 14.89 15.36 17.68 (2.38) 21.45 (13.33) 16.97 (0.56) 15.69 (2.14) 16.78 (0.46) 13.99 (1.87) 16.85 (0.73) 16.58 (2.58)
MSE 449
MSPE 419 545 (123) 808 (902) 482 (12) 445 (127) 476 (4) 513 (90) 494 (25) 441 (180)
Model III MPB 2.28 −6.10 −1.42 (3.56) −6.09 (3.52) −0.53 (1.77) −6.54 (1.78) 1.16 (1.90) −8.24 (2.38) 0.26 (2.51) −5.87 (3.13)
MAD 8.11 8.51 8.47 (0.77) 7.49 (3.21) 8.24 (0.36) 8.74 (1.63) 8.62 (0.63) 10.16 (1.57) 8.48 (0.70) 8.21 (2.10)
MSE 132
MSPE 168 152 (12) 138 (155) 140 (4) 143 (50) 142 (9) 278 (149) 143 (11) 178 (149)
Model IV MPB 0.39 −0.52 0.23 (1.11) −0.76 (1.08) −0.04 (0.85) −0.28 (0.69) 0.10 (0.66) −0.48 (1.06) −0.25 (0.58) −0.29 (0.79)
MAD 2.39 2.34 2.65 (0.16) 2.66 (0.62) 2.54 (0.18) 2.32 (0.75) 2.53 (0.13) 2.50 (0.38) 2.47 (0.10) 2.23 (0.39)
MSPE 10.15
MSE 10.90 12.58 (1.09) 12.89 (7.06) 12.02 (0.65) 11.16 (7.35) 11.72 (0.42) 10.46 (3.98) 11.73 (0.46) 9.22 (3.79)

Note: MPB = mean prediction bias, MAD = mean absolute deviation, MSPE = mean squared prediction error, MSE = mean squared error (S.D.).

587
588 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589
satisfactory. This was expected since the sample sizes selected
for this study were too small (less than 8% of the total data) to
result in a meaningful conclusion. Therefore, it was concluded
that the updating procedure is the best approach when limited
data are available and no further investigation was conducted
regarding developing models with limited data.
Table 5 presents the relative goodness of fit measures for pre-
dicting the number of accidents in 2001 using the 1996 models,
the 2001 models, models that apply a calibration factor to the
1996 model and the Bayesian approach. For the latter two, vari-
ous sample sizes are used to estimate the “updated” models that
are applied.
As expected, the measures indicate that 2001 model fits the
2001 data better than other models. The real value of these results
is in the comparison of the performance of the “updated” models
among themselves for different sample sizes and with the 1996
model. This comparison shows that in general the recalibrated
“updated” models perform better than the updated Bayesian
models and the1996 models. However, no clear-cut conclusions
can be drawn regarding the performance of updated models in
respect to the sample size used for estimating them.
8. Conclusions and summary
This paper examined the temporal transferability of zonal
accident prediction models for the City of Toronto. In so doing,
model calibration and related transferability issues were also
addressed. The coefficient estimates for the 2001 models are
consistent with the 1996 models. Most of the coefficient param-
eters and constant terms are significant for the 1996 and the 2001
models across all four accident type models (Models I–IV).
The results of nested likelihood ratio test indicate that the
model parameters from two urban contexts are not statistically
equal and have not remained stable over time. Therefore, from
a theoretical perspective, the transfer of macro-level accident
prediction models is undesirable. However, relative measures of
transferability indicate that the transferred models yield useful
information in the application context. Transfer index values
show that transferred models capture at least 52% of the variance
provided by estimating all the parameters in the 2001 data. The
results of the t-test for all the models indicate that the VKT
and socioeconomic and demographic parameters are stable over
time.
In the absence of a complete dataset for calibrating the 2001
model, it looks like the “updated” models, which are developed
by applying calibration factors to the 1996 models, perform
reasonably and better than the 1996 models and the Bayesian
updated models regardless of the sample size used for esti-
mating these factors. The Bayesian approach has not yielded
good results in general. The possible problem for this method
is that in this method the coefficients in model transferring are
heavily affected by the estimation context data since the num-
ber of observations is larger in this context. Also, this method
assumes that the prior and posterior (updated) distributions of
the parameters are normally distributed, which may not be a
good assumption.
This research looked only at temporal transferability of zonal
accident prediction models. A natural next step for further
research would be the investigation of spatial transferability and
the combination of temporal and spatial transferability. That
research should aim to identify the conditions that make two
study areas “similar enough” to justify transferring a model from
one to the other.
Acknowledgment
The authors acknowledge the Natural Sciences and Engi-
neering Research Council of Canada (NSERC) for its support
of some of the fundamental research on which this work is based.
References
Atherton, T.J., Ben-Akiva, M.E., 1976. Transferability and updating of dis-
aggregation travel demand models. Transport. Res. Rec. 610, 12–18.
Badoe, D.A., 1994. An investigation into the long range transferability of
work-trip discrete mode choice models. PhD Dissertation. Department of
Civil Engineering, University of Toronto, Ont., Canada.
Badoe, D.A., Miller, E.J., 1995a. Analysis of the temporal transferability of
disaggregate work trip mode choice models. Transport. Res. Rec. 1493,
1–11.
Badoe, D.A., Miller, E.J., 1995b. Comparison of alternative methods for
updating disaggregated logit mode choice models. Transport. Res. Rec.
1493, 90–100.
Ben-Akiva, M., 1981. Issues in transferring and updating travel-behaviour
models. In: Stopher, P.R., et al. (Eds.), New Horizons in Travel Behaviour.
Lexington Books, Lexington, MA, pp. 665–686.
Ben-Akiva, M., Bolduc, D., 1987. Approaches to model transferability and
updating: the combined transfer estimator. Transport. Res. Rec. 1139,
1–7.
EMME/2, 1998. User’s Guide Manual Release 9. INRO Consultants Inc.
Gunn, H.F., 2001. Spatial and temporal transferability of relationships
between travel demand, trip cost and travel time. Transport. Res. Part
E: Logist. Transport. Rev. 37 (2/3), 163–189.
Gunn, H.F., Ben-Akiva, M.E., Bradley, M.A., 1985. Tests of the scaling
approach to transferring disaggregate travel demand models. Transport.
Res. Rec. 1037, 21–30.
Hadayeghi, A., Shalaby, A.S., Persaud, B., 2003. Macro-level accident pre-
diction models for evaluating safety of urban transportation systems.
Transport. Res. Rec. 1840, 87–95.
Harwood, D.W., Council, M., Hauer, E., Hughes, W. E., Vogt, A., 2000.
Prediction of the expected safety performance of rural two-lane highways.
Report FHWA-RD-99-207, FHWA, Department of Transportation.
Koppelman, F.S., Rose, G., 1983. Geographic transfer of travel choice mod-
els: evaluation and procedure. In: Optimization and Discrete Choice in
Urban Systems: Proceedings of the International Symposium on New
Directions in Urban Systems Modeling, pp. 272–309.
Koppelman, F.S., Wilmot, C.G., 1982. Transferability analysis of dissagregate
choice models. Transport. Res. Rec. 895, 18–24.
Ladron de Guevara, F., Washington, S.P., Oh, J., 2004. Forecasting crashes at
the planning level: simultaneous negative binomial crash model applied
in Tucson, Arizona. Transport. Res. Rec. 1897, 191–199.
Lerman, S.R., 1981. A comment on interspatial, intraspatial, and temporal
transferability. In: Stopher, P.R., et al. (Eds.), New Horizons in Travel
Behaviour. Lexington Books, Lexington, MA, pp. 628–632.
Lovegrove, G.R., Sayed, T., 2005. Macro-level collision prediction models
for evaluating neighbourhood traffic safety. In: Proceedings of the Sixth
Transportation Specialty Conference, Canadian Society of Civil Engineer-
ing, Toronto, Canada.
Oh, J., Lyon, C., Washington, S., Persaud, B., Bared, J., 2003. Validation
of FHWA crash models for rural intersection lessons learned. Transport.
Res. Rec. 1840, 41–49.
 A. Hadayeghi et al. / Accident Analysis and Prevention 38 (2006) 579–589 589

Parody, T.E., 1977. An analysis of disaggregate mode choice models in pre-
diction. Transport. Res. Rec. 649, 23–31.
National Cooperative Highway Research Program (NCHRP), in press. Incor-
porating Safety into Long-range Transportation Planning, Project 8-44.
National Cooperative Highway Research Program.
Persaud, B., Lord, D., Palmisano, J., 2002. Calibration and transferability of
accident prediction models for urban intersections. Transport. Res. Rec.
1784, 57–64.
SAS Institute Inc., 1999. SAS/STAT® Users Guide, Version 8. SAS Institute
Inc., Cary, NC.