Simulation Modelling Practice and Theory 106 (2021) 102172

Contents lists available at ScienceDirect

Simulation Modelling Practice and Theory

journal homepage: www.elsevier.com/locate/simpat

An extreme value theory based approach for calibration of

T
microsimulation models for safety analysis
Yanyong Guoa,b,c, , Tarek Sayedd, Lai Zhenge, Mohamed Essad
⁎

a
School of Transportation, Southeast University, Nanjing, Jiangsu, China
b
Jiangsu Key Laboratory of Urban ITS, Nanjing, Jiangsu, China
c
Jiangsu Province Collaborative Innovation Center of Modern Urban Traffic Technologies, Nanjing, Jiangsu, China
d
Department of Civil Engineering, University of British Columbia, 6250 Applied Science Lane, Vancouver, BC, Canada V6T 1Z4
e
School of Transportation Science and Engineering, Harbin Institute of Technology, 202 Haihe Road, Nangang District, Harbin, Heilongjiang, 150090,
China

ARTICLE INFO ABSTRACT

Keywords: There is an increasing interest in using microsimulation models for traffic safety evaluations and
Microsimulation traffic models several calibration procedures were proposed for the simulation models to obtain a high corre­
Simulated traffic conflicts lation between the simulated and field-measured conflicts. However, since the ultimate goal of
Extreme value theory safety studies is to reduce crashes, the simulation model needs to be calibrated to ensure an
Automated computer vision
accurate prediction of crashes. The objective of this study is to propose an innovative calibration
approach for microsimulation models using the extreme value theory (EVT) approach. The EVT
approach was shown to connect non-crash events to crashes, providing reasonable crash esti­
mates from traffic conflicts. The goal of the proposed calibration process is to produce an extreme
value distribution of simulated conflicts that matches the one of field-measured conflicts. A
Genetic algorithm is utilized to obtain VISSIM model parameters that can achieve this goal.
Traffic video data collected from two approaches at a signalized intersection in Surrey, Canada
were used as case studies. Automated traffic conflicts analysis techniques were used to extract
field-measured conflicts. Simulated conflicts were extracted from vehicle trajectories from
VISSIM using the surrogate safety assessment model (SSAM) tool. EVT models were developed to
estimate the generalized Pareto (GP) distributions of both field-measured conflicts and simulated
conflicts of different scenarios. The calibration results show considerable matching of the esti­
mated GP distributions between simulated conflicts and field-measured conflicts. As such, the
proposed calibration procedure for microsimulation model is recommended as a promising ap­
proach in simulation-based safety evaluation of signalized intersections.

1. Introduction

Traffic safety analysis has traditionally relied on historical crash data which has several shortcomings. First, relying on crash data
is a reactive approach that requires a prolonged period for collecting crash data to conduct a reliable safety analysis. Other short­
comings include the limited availability, unreliability and poor quality of collision data [1]. Therefore, there is growing interest in
using traffic safety analysis techniques that rely on surrogate safety measures such as traffic conflicts. Traffic conflicts occur more

Corresponding author.
⁎

E-mail addresses: guoyanyong@seu.edu.cn (Y. Guo), tsayed@civil.ubc.ca (T. Sayed), zhenglai@hit.edu.cn (L. Zheng),
m.a.essa@civil.ubc.ca (M. Essa).

https://doi.org/10.1016/j.simpat.2020.102172
Received 26 February 2020; Received in revised form 25 July 2020; Accepted 29 August 2020
Available online 05 September 2020
1569-190X/ © 2020 Elsevier B.V. All rights reserved.
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

frequently than collisions, can be clearly observed, and are of marginal social cost. Moreover, the use of traffic conflicts is a proactive
approach and provides a much better understanding of collision contributing factors [2]. The increased use of traffic conflicts in
safety analysis has been aided by advances in video-based automated computer vision techniques which were shown to accurately
detect and evaluate the severity of traffic conflicts [3].
Traffic simulation has also emerged as a popular approach for road safety analysis in recent years [4-7]. The use of simulation
models in traffic safety analysis provides an easy way to collect simulated conflicts data instantly without actual field observation [8].
More importantly, the use of simulation models for road safety analysis enables the evaluation of the safety impact to changes in the
design and/or traffic control that have not been implemented, and to evaluate unconventional traffic designs [9]. However, recent
studies evaluating the validity of using simulation models for safety analysis have raised several concerns [10]. These concerns relate
to the difficulty of representing unsafe road user interactions and near misses given the existence of rules aiming at collision
avoidance. Furthermore, the validity of simulated traffic conflicts is challenged by whether they can produce reasonable and reliable
safety estimates beyond what can be expected from using exposure based models [10].
To improve the ability of simulation models to evaluate safety, a proper calibration of simulation models is of considerable
importance. Previous studies have proposed calibration procedures for simulation models with the aim of obtaining a high correlation
between the simulated and field-measured conflicts [8,10-13]. However, a high correlation between the number of simulated and
field-measured conflicts for a specific threshold of conflict indicator (e.g., Time-to-collision) does not guarantee a proper matching
between the simulated and field-measured conflict severity distributions. Moreover, since the ultimate goal of safety studies is to
reduce crashes, the simulation model needs to be calibrated to ensure an accurate estimation of crashes. Extreme value theory (EVT)
provides a statistical tool that can enable the extrapolation from simulated traffic conflicts to extreme events such as crashes. As such,
the combined use of traffic simulation and the EVT approach can lead to a more realistic traffic safety analysis [14,15].
This study proposes an innovative calibration method of microsimulation models based on extreme value theory. The calibration
procedure includes two steps. The first-step is to calibrate the arrival type and average vehicle delay time to match the real traffic
conditions. The second-step is to calibrate VISSIM parameters to produce an extreme value distribution of simulated conflicts that
matches the one of field-measured conflicts. Thirty-six hours of video data at a signalized intersection were used in this study.
Automated traffic conflicts analysis using computer vision techniques is used to extract the field-measured conflicts. Simulated
conflicts were estimated from VISSIM trajectories using the SSAM tool.
Different from existing safety simulation studies which focused on obtaining a good correlation between simulated and observed
conflicts, the contribution of this study is to ensure the extreme value distribution of simulated conflicts matches that of field-
measured conflicts through the calibration of simulation model. This should provide calibration of estimated crash risk from the
simulation models. As well, the relationship between crashes estimated from simulation models and actual crashes is enhanced.

2. Previous work

2.1. Traffic simulation in road safety analysis

Previous studies have extensively investigated the development, calibration and validation of various simulation models such as
VISSIM, PARAMICS, and AIMSUN for use in road safety analysis. The SSAM tool is commonly used to extract traffic conflicts and
several conflicts indicators from simulated vehicle trajectories. Gettman et al. [16] examined the relationship between observed
crashes and simulated conflicts that extracted from VISSIM and SSAM using eighty-three signalized intersections. Cunto and Sac­
comanno [17] developed a four-step systematic procedure to calibrate and validate simulated vehicle safety performance at signa­
lized intersections. The safety performance for individual vehicles was estimated using VISSIM, and was calibrated and validated
using the real vehicle tracking data from the NGSIM program [17]. Dijkstra et al. [18] explored the relationship between simulated
conflicts from PARAMICS and field crashes from 569 intersections using generalized linear regression models. Vasconcelos et al. [19]
compared the number of simulated conflicts estimated by SSAM and the number of injury crashes predicted by analytical models at
three hypothetical intersections. The results from these studies generally indicated that statistically significant correlations can be
found between the simulated conflicts and the observed crashes.
Fan et al. [12] and Huang et al. [13] proposed a procedure of calibrating headway and conflict related parameters in VISSIM
models for freeway merge areas and signalized intersections. Essa and Sayed [10,11] presented a calibration procedure of VISSIM/
PARAMICS models to improve the relationship between actual conflicts and simulated conflicts. Guo et al. [8] conducted a cali­
bration process for VISSIM models to increase the correlation between simulated and field-measured conflicts at signalized inter­
sections in a left-hand traffic environment. Results from these studies showed that the correlation between the simulated and real-
world conflicts could be significantly improved through proper model calibration. A recent study by Wang et al. [7] developed three
calibration strategies for VISSIM models including non-calibration, semi-calibration of headway, and full-calibration of both headway
and conflicts count. Estimated annual crash frequency from the simulated and real conflicts using the EVT were assessed and
compared with observed crashes. Results showed that full-calibration strategy could give a better estimate of crashes than the other
two strategies. However, the results were not validated. As the full-calibration strategy aimed to obtain reasonable results of conflict
counts instead of enhancing the relationship between estimated and observed crashes, the link between the estimated crashes from
simulated conflict and the calibration strategy was not clear.
The review of previous studies show that using default parameters in simulation models could produce unreliable traffic conflict
estimates. Proper calibration of the simulation models can minimize the discrepancy between simulated conflicts and observed
conflicts. However, these studies were mainly targeted at obtaining a good correlation between simulated and observed conflicts. To

2
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

the best of the authors’ knowledge, no studies have attempted to directly calibrate simulation model parameters to enhance the
relationship between crashes estimated from simulation models and actual crashes. In this study, the extreme value distribution of
simulated conflicts that can predict crashes is used in the calibration process. The extreme value distribution is obtained from traffic
conflicts of various severity levels and therefore there is no need to identify specific conflict thresholds in the calibration process.

2.2. The EVT approach in traffic safety analysis

There has been recent considerable increase in the use of the EVT approach in traffic safety evaluation. The main concept in using
the EVT approach in safety analysis is to estimate rarely occurring crashes from various surrogate safety measures that are more
frequently observed. Campbell et al. [20] were among the first to apply the EVT approach in road safety analysis. The EVT approach
was further used to relate non-crash events to crashes in early seminal studies [14,21,22]. Songchitruksa and Tarko [14] applied the
EVT approach with a generalized extreme value (GEV) distribution to estimate right-angle crashes at signalized intersections. They
found a reasonable relationship between the estimated crashes and observed crashes. Subsequently, Tarko [22] utilized a generalized
Pareto (GP) distribution in an EVT approach to estimate the risk of run-off-the-road crashes based on simulated road departure
events. These studies demonstrated several advantages of the EVT approach including abandoning the assumption of fixed crash-to-
surrogate ratio and not requiring crash data in the model estimation process [14,22].
Using the EVT approach, Gordon et al. [23] estimated road departure related crashes based on the surrogate measure of time to
road edge crossing. Results showed that the estimated crashes were comparable to the observed crashes. Zheng et al. [15] compared
the block maxima (BM) and peak over threshold (POT) methods in the EVT approach. Results showed that the POT approach
outperformed the BM approach in terms of data utilization, estimated accuracy and reliability. Farah and Azevedo [24] used the EVT
approach to estimate the probability of head-on-collisions in passing maneuvers. They found that the BM approach yielded more
stable results compared to the POT approach, which is inconsistent with the results from Zheng et al. [15]. Åsljung et al. [25] applied
the EVT approach in validating the safety of autonomous vehicles. Two surrogate safety measures including TTC and Brake Threat
Number (BTN) were employed in the study. Tarko [26] estimated the probability of a crash given a traffic conflict using a Lomax
distribution and confirmed that traffic conflicts defined by sufficiently small threshold separation of indictors allow unbiased crash
estimates. Zheng et al. [27] and Zheng and Sayed [28] employed the EVT approach in before-after safety studies and concluded that
this approach is able to provide reliable and sound evaluation results. Zheng and Sayed [29] also compared four different traffic
conflict indicators in terms of crash estimation using the POT approach, and they showed that the POT approach was a reliable tool
for traffic conflict based crash estimation.
The literature review shows that previous studies focused on applying the EVT approach to link various surrogate safety measures
to crashes. The studies developed EVT models and tested the reliability of using the EVT approaches in road safety analysis [30-33].
In general, both univariate and bivariate EVT models have been shown as promising tools for road safety evaluation.

3. Study location and data collection

The intersection of 72nd Avenue and 128th street in the City of Surrey, British Columbia, Canada was selected for this study. The
traffic control on the four approaches of the intersection are protected-permissive left turns. Traffic videos were recorded using four
cameras installed to cover the entire functional area of two approaches located on the 72nd avenue corridor. A total of 36 h (9 h × 2
approaches × 2 scenes) were used for this study in order to consider peak and off-peak hours. The video data obtained from the
eastbound approach (scenes 1 and 2) were used as a calibration dataset, while the video-data obtained from the westbound approach
(scenes 3 and 4) were used as a validation dataset. Before recording started, the two cameras at each approach were synchronized so
that the data extracted from different scenes could be matched. Fig. 1 illustrates the location and the camera positions.
Detailed traffic data were collected automatically or semi-automatically by a self-developed computer program from video re­
cordings. The traffic data includes: (1) the signal timing (i.e. cycle length, green time, red time, and yellow time); (2) the traffic
volume and traffic composition for all movements in the intersection; (3) the number of vehicles arriving during green time; and (4)
the average travel time, average delay time, and desired speed. The desired speed is the speed selected by a vehicle if not hindered by
other vehicles or traffic control devices [34]. About 1000 vehicles traveling during the green time were selected to estimate the
desired speed. In addition, the average delay time per hour for each approach was determined by comparing the travel time of each
vehicle with its ideal travel time. The ideal travel time was calculated by using the predefined segment length divided by the average
desired speed.

4. Methodology

4.1. Framework

The calibration procedure is illustrated in Fig. 2. The traffic operations including arrival type and average vehicle delay are firstly
calibrated to match the real traffic condition as shown in Essa and Sayed [10]. Subsequently, the parameters in VISSIM model were
calibrated to ensure that the extreme value distribution of simulated conflicts matches that of field-measured conflicts. The gen­
eralized Pareto (GP) distribution was used to develop the extreme value model.
Several emerging techniques are employed in the calibration process including computer vision techniques, Genetic Algorithms
(GA), traffic simulation, and the EVT approach. Automated traffic conflicts analysis based on computer vision techniques are used to

3
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

Fig. 1. Study location and camera positions.

Fig. 2. Simulation model calibration procedure.

extract vehicle trajectories and identify field-measured conflicts. Simulated conflicts are estimated from VISSIM trajectories using the
SSAM tool. A Genetic Algorithm is utilized to update the values of parameters in VISSIM model based on the fitness value. The EVT
approach is applied to estimate the GP distributions and safety measures of both simulated conflicts and field-measured conflicts.

4.2. Automated traffic conflict analysis

The automated traffic conflicts analysis system was used to extract traffic conflicts from the video recordings [35,36]. The system
is based on the computer vision techniques and has been used in many safety analysis and data collection studies [9,37,38]. An
outline of the analysis procedure is shown in Fig. 3.
The procedure starts with camera calibration which aims at creating a mapping between the three-dimensional real word and the

4
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

Fig. 3. Schematic diagram of the procedure of automated traffic conflict analysis.

two-dimensional image space [39]. Then, feature tracking is conducted in which distinguishable features on the moving objects are
tracked using the Kanade-Lucas-Tomasi (KLT) feature tracker algorithm [35]. Features that move at similar speed and movement
pattern, and satisfy predefined spatial proximity are grouped to create coherent objects representing vehicles, and trajectories of
these vehicles are recorded. Subsequently, prototypes describing the motion patterns of the moving vehicles are generated. The
trajectory of a moving object is matched with more than one prototype with a probability weighting determined using a Longest
Common Sub Sequence (LCSS) algorithm [40]. The prototypes provide a set of predicted future positions with associated probabilities
of occurrence. Finally, conflicts between vehicles are determined by evaluating if any of these future positions coincide spatially and
temporally with other vehicles.
It is noted that rear-end conflicts are the majority conflicts at the intersection approaches. Due to the small number of the other
conflict types, they were not included in this study. Using the automated traffic conflict analysis system, rear-end conflicts at the
eastbound and westbound approaches were identified for each hour. The severity of conflicts is evaluated using the time to collision
(TTC) conflict indicator. For any two consecutive vehicles on a collision course, the TTC values are calculated continuously frame by
frame. The minimum TTC is used to represent the overall severity of the interaction. Finally, rear-end conflicts with minimum TTC
less than 3 s are extracted and kept for further analysis.

4.3. Traffic simulation model development

VISSIM was used to develop the simulation model due to its flexibility in simulating traffic conditions and the ease of collecting
conflict data. The geometry of the intersection (number of lanes, lane widths, and turning radius) was specified and all links and
connectors were set up with their real dimensions. Subsequently, the fundamental parameters were defined in the VISSIM simulation
model for each hour, including traffic volume, traffic composition, and the calibrated desired speed curve. The priority rules, conflict
areas, and reduced speed areas were then added in VISSIM model to more accurately reflect actual vehicle movements.
The final step was to define signal control strategy based on the actual signal timing. Actual signal settings were set up in the
VISSIM simulation model for each hour using the Ring Barrier Controller (RBC) [41], in which each phase has a minimum green time,
maximum green time, yellow time, and all red time. To reproduce the protected-permissive left-turn phase, a detector was defined at

5
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

each left-turn lane. In each signal cycle, if the detector is occupied, a protected left turn phase will be called by RBC, otherwise the left
turn will be permissive. Once all the settings in the VISSIM simulation model were defined, the simulation model was run and
inspected visually to make sure that there were no abnormal movements of the moving vehicles.

4.4. Extreme value theory and safety measures

Generally, there are two techniques to sample extreme values, one is block maxima (BM) and the other is peak over threshold
(POT). The POT approach is used in this study since it has been shown to be more efficient and lead to less biased estimates in
previous studies [15,26].
The POT approach takes observations that exceed a high enough threshold as extremes. Let X1, X2, …, Xn be a sequence of
independent and random variables with common distribution function F. The distribution function of exceedances X over a threshold
u is Fu(x) =1-Pr(X-u | X > u). For high enough threshold u, the distribution function of y = (X-u) is approximately a Generalized
Pareto (GP) distribution [42]
1/
y
G (y ) = 1 1+
(1)
y
defined on {y: x > u and 1 + > 0 }, where σ is the scale parameter; -∞ < ξ < ∞ is the shape parameter.
When the GP distribution is fitted based on TTC values of traffic conflicts, safety implications such as the risk of crash can be
obtained [14,15]. According to the definition of TTC, a smaller TTC value means a higher risk of a traffic event ending up as a crash,
and TTC ≤ 0 indicates that a crash occurs. Therefore, the risk of a target event (e.g., a crash, or a severe conflict) can be calculated
from the tail region of fitted GP distribution of negated TTC, and that is
1/
( u)
R = Pr{z }=1 G( ) = 1 +
(2)
where R is the risk of target event; δ is a threshold value that defines the target event, and z is the threshold exceedance.
Two safety measures, i.e. the annual crash frequency (ACF) and annual extreme-serious conflict frequency (AECF), are used in the
calibration. Traffic conflicts are usually obtained from a short observation period t, which is assumed to be representative for a longer
period T, say T = 1 year, then the estimated annual event number can be calculated as
T
AENT = R
t (3)
where AENT is the annual target event number. When δ=0, AENT corresponds to ACF, and when δ is a specific small value AENT
corresponds to AECF.

5. Simulation model calibration

5.1. Traffic operations calibration

To ensure the simulated average vehicle delay similar to the actual average delay, the arrival type which is a primary control
factor for delay was calibrated to match the field conditions. The arrival type is determined by the platoon ratio which can be
calculated as follows [43]
g
Pr = P ×
C (4)
where Pr is the platoon ratio; P is the proportion of vehicles arriving during the green time; g is the effective green time (sec); and C is
the cycle length (sec).
Using the field-measured signal timing and the proportion of vehicles arriving during green time, the platoon ratio and arrival
type were calibrated. More details on the calibration of arrival type can be found in [10]. After the calibration, trajectory files were
processed in SSAM to estimate the approaching rear-end simulated conflicts within the interested area.

5.2. EVT-based traffic safety calibration

After calibrating the traffic operation of VISSIM model, VISSIM parameters are further calibrated from the safety perspective. The
purpose of this calibration is to ensure that the extreme value distribution of simulated conflicts matches that of field-measured
conflicts through calibrating VISSIM parameters. This step starts with a sensitivity analysis, which focuses on identifying the im­
portant parameters in VISSIM that significantly affect the simulated conflicts. Eight parameters were found significant including: CC0
(standstill distance), CC1 (headway time), CC3 (threshold for entering ‘following’), CC4 and CC5 (negative and positive following
thresholds), DD (desired deceleration), RF (safety distance reduction factor for lane change), SRF (reduction factor for safety distance
close to stop line), and SUPD (start upstream of stop line). More details about the sensitivity analysis process is described in Guo et al.
[8] and Essa and Sayed [10].
After the significant parameters in VISSIM model were determined, a Genetic Algorithm (GA) was applied to calibrate the selected

6
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

parameters [44]. The optimization goal is to estimate the best values of these parameters, where the extreme value distribution of the
simulated conflicts based on these VISSIM parameters can be matched well with that of the field-measured conflicts. The GP dis­
tributions of both simulated conflicts and the field-measured conflicts were estimated. In order to cover a wide space for each
parameter, an original generation which contained 90 individuals was generated randomly using the Latin hypercube sampling (LHS)
method [45]. Each individual was considered as a chromosome that represented a combination of the eight parameters with different
values. As such, an approximate range for each parameter was obtained based on the best ten individuals.
Subsequently, the first generation in GA that contains 10 individuals was produced using the LHS method with the parameters’
ranges from the original generation. Simulation runs for each individual were performed and the simulated conflicts were estimated
by SSAM. Based on the simulated conflicts, a fitness value was assigned to each individual. In this study, the fitness value was defined
as the area difference between the estimated GP probability density distribution curves of simulated conflicts and field-measured
conflicts. Specifically, the fitness value can be calculated as follows
0
F (i ) = × A1 + (1 ) × A2 = × gsim gfil dy + (1 ) × gsim gfil dy
0 (5)

where F(i) is the fitness value for individual i; A1 is the area difference between the two estimated GP curves with negative TTC less than 0
(i.e., non-crash area); A2 is the area difference between the two estimated GP curves with the negative TTC greater than 0 (i.e., crash area);
gsim and gfil are probability density functions of GP distributions for simulated and field-measured conflicts, respectively, which can be
estimated using (1); α is a weight coefficient that ranges from 0 to 1. A sensitivity analysis was conducted to determine α, and the result
showed the best value of α with 0.3 in this study. A smaller F(i) indicates a better matching between the two GP distributions.
Three basic operators of the GA, i.e. reproduction, crossover, and mutation, were used for producing the next generation. The
same procedure was conducted until the fitness value could not be further improved. After 11 generations, the optimum solution of
the values for the eight parameters achieved convergence. In total, 540 simulation runs (1 generation × 10 individuals × 9 h + 10
generations × 5 individuals × 9 h) were performed. Accordingly, 540 SSAM runs were conducted to extract the conflicts from the
simulated trajectories.

6. Results

6.1. Calibration results

The data obtained from the eastbound approach was used as the calibration dataset. The results of the traffic operation calibration
are shown in Fig. 4. The results showed that the simulated platoon ratios are almost the same as the field platoon ratios and the
simulated average vehicle delay are closer to the field average delay after the calibration. The mean absolute percentage error
(MAPE) of the average delay decreased from 30.5% to 12% after the calibration.
After the calibration of traffic operation in VISSIM model, the EVT-based traffic safety calibration procedure was performed. The
EVT models were developed for field-measured conflicts as the basis to calculate fitness values in this calibration step. The first and
foremost step to estimate the GP distribution is determining the conflict indicator threshold. This involves considering the bias-
variance trade-off [46]. In this study, the mean residual life plot and threshold stability plots for scale and shape parameters were
employed to identify an appropriate threshold. Through the two methods, a range of thresholds, where the mean residual life plot is
linear and the modified scale and shape estimates appear to be stable, can be obtained. The highest threshold in the range is selected
as a compromise between bias and variance. Fig. 5 shows an example of the threshold selection for field-measured traffic conflicts.
The EVT models were also developed for simulated conflicts of three scenarios: default (S1), after the traffic operation calibration
(S2), and after the EVT-based traffic safety calibration (S3). The estimated EVT model parameters are shown in Table 1. The results
indicate that the GP parameters (u, σ, ξ) of simulated conflicts are closer to the GP parameters estimated from real conflicts after the
traffic operation calibration and were further enhanced after EVT-based traffic safety calibration. Fig. 6 shows the GP distributions
based on the estimated EVT model parameters in Table 1. The figure shows a considerable difference between the estimated GP
distribution curve of the default scenario (S1) and the GP distribution curve of the real conflicts. After the traffic operation cali­
bration, the GP distribution curve of simulated conflicts (S2) moves closer to that of the real conflicts, but still with a significant
difference. In contrast, the GP distribution curve of simulated conflicts after the EVT-based traffic safety calibration (S3) is almost the
same as that of the real conflicts.
As mentioned earlier, the safety measures (i.e., ACF and AECF) can be estimated based on the right tail of the GP distribution. As
such, the tails of the GP distributions of real conflicts and simulated conflicts were examined. As shown in Fig. 6, the right tails of GP
distributions of real conflicts and simulated conflicts after the EVT-based traffic safety calibration (S3) end up to 0 after crossing the
point of negative TTC=0. In contrast, the right tails of GP distributions of simulated conflicts for both default scenario (S1) and after
the traffic operation calibration (S2) end up to 0 before crossing the point of negative TTC=0, which means zero crash would be
expected within the observation period.
Furthermore, the safety measures were calculated based on (2) and (3). In this study, various thresholds δ were used as shown in
Table 2. In order to quantify the uncertainty, the confidence intervals (95% C.I.) of safety estimates were calculated. Table 2 shows
the estimated safety measures for different scenarios. The results show that ACFs from the real conflicts and S3 scenario are 1.14 and
0.02 respectively, indicating that crashes occur at the approach of the intersection. However, the ACFs for the S1 and S2 scenarios are
zero. AECFs with different thresholds δ from the simulated conflicts for the S1 and S2 scenarios are also significantly lower than that

7
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

Fig. 4. Traffic operation calibration results (calibration data) (data from [10]).

from the real conflicts. After the EVT-based traffic safety calibration (S3), AECFs from the simulated conflicts achieve a better
consistency with that from the real conflicts. Table 3 shows the MAPE values for AECFs with different thresholds. The average MAPE
value decreased from 64.17% in default scenario (S1) to 58.77% after the traffic operation calibration (S2), and then decreased to
27.45% after the EVT-based traffic safety calibration (S3). As such, the EVT-based traffic safety calibration is shown to have a higher
impact on improving the estimation of safety measures than the traffic operation calibration.
The results show that the safety measures are significantly underestimated from the simulated conflicts of default scenario and
after traffic operation calibration, while the estimation of safety measures from simulated conflicts after the EVT-based traffic safety
calibration are more reasonable and much closer to that from the real conflicts. Overall, the results indicate the usefulness of the
proposed calibration process in obtaining reasonable safety estimates in terms of estimated collisions and extreme-serious conflicts.

6.2. Validation results

The data obtained from the westbound approach was used as the calibration dataset. After calibrating the traffic operation of

8
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

Fig. 5. Graphical diagnostics of threshold selection for field-measured traffic conflicts.

Table 1
Estimation results of EVT models for different scenarios (calibration data).
Scenarios Threshold Scale parameter (σ) Shape parameter (ξ)
μ Mean Standard error Mean Standard error

Real −1.95 0.949 0.101 −0.456 0.077

S1* −1.50 0.360 0.043 −0.203 0.078
S2** −1.60 0.539 0.026 −0.384 0.024
S3*** −1.95 0.896 0.046 −0.455 0.029

Notes: * default scenario; ** after the traffic operation calibration; *** after the EVT-based traffic safety calibration.

VISSIM model, the eight calibrated parameters from the calibration dataset were then applied in the VISSIM model and trajectory
files were exported and processed in SSAM to extract the simulated conflicts for the validation analysis. The EVT models were
developed to test the validity and reliability of the calibrated parameters. The estimated GP parameters are shown in Table 4 which
show a similar trend as that of the calibration results. Specifically, the results show that the GP parameters of simulated conflicts are
closer to the GP parameters estimated from real conflicts after the traffic operation calibration (S2) and were further improved after
transferring the calibrated VISSIM parameters (S3). As shown in Fig. 7, there is an apparent discrepancy between the GP distribution
curve of the default scenario (S1) and the GP distribution curve of the real conflicts. The GP distribution curve of simulated conflicts
(S2 and S3) gets closer to that of the real conflicts after each step of calibration.

9
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

Fig. 6. Estimated GP distributions for different scenarios (calibration data).

Table 2
Estimates of safety measures for different scenarios (calibration data).
Scenarios ACF AECF1 (δ = −0.2) AECF2 (δ = −0.4) AECF3 (δ = −0.6) AECF4 (δ = −0.8) AECF5 (δ = −1.0)
Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI

Real 1.14 0 29.07 8.66 0 46.01 24.37 0 68.62 49.10 3.19 96.75 83.42 26.30 132.32 127.80 67.05 174.99
S1* 0 0 6.33 0.73 0 12.46 4.12 0 23.85 14.85 0.06 44.07 41.01 7.36 79.69 95.17 44.28 139.61
S2** 0 0 0.01 0 0 2.02 3.23 0 11.40 19.03 6.81 33.66 54.17 34.64 73.41 114.02 91.10 135.03
S3*** 0.02 0 5.63 3.85 0 16.19 16.15 2.75 33.96 38.17 17.87 59.74 70.71 46.02 94.32 114.36 88.18 138.27

Notes: * default scenario; ** after the traffic operation calibration; *** after the EVT-based traffic safety calibration.

Table 3
MAPE values for AECFs with different thresholds (calibration data).
δ Default (%) 1st-calibration (%) 2nd-calibration (%)

−0.2 91.62 99.00 55.54

−0.4 83.09 86.76 33.73
−0.6 69.76 61.23 22.25
−0.8 50.84 35.06 15.24
−1.0 25.53 10.79 10.52
Average 64.17 58.77 27.45

Table 4
Estimation results of EVT models for different scenarios (validation data).
Scenarios Threshold Scale parameter (σ) Shape parameter (ξ)
μ Mean Standard error Mean Standard error

Real −1.75 0.971 0.084 −0.613 0.064

S1* −1.50 0.289 0.052 −0.142 0.119
S2** −1.60 0.523 0.039 −0.365 0.051
S3*** −1.70 0.905 0.048 −0.612 0.035

Notes: * default scenario; ** after the traffic operation calibration; *** after transferring the calibrated VISSIM parameters.

The safety measures were also calculated for different scenarios for the validation dataset, as shown in Table 5. Similar to the
calibration results, the validation results show that the safety measures are significantly underestimated from the simulated conflicts
of the default scenario (S1) and after the traffic operation calibration (S2). In contrast, safety measures estimated from the simulated
conflicts after transferring the calibrated VISSIM parameters (S3) are similar to results from the real conflicts. Overall, the agreement
in both estimated EVT models and estimated safety measures between simulated conflicts and real conflicts confirms the validity and

10
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

Fig. 7. Estimated GP distributions for different scenarios (validation data).

reliability of the calibration process.

7. Discussion

Previous studies have mainly focused on obtaining a high correlation between simulated conflicts and observed conflicts through
calibrating the traffic simulation models [7,8,10–13]. Instead, this study aims at ensuring the extreme value distribution from the
simulated conflicts match that from the field-measured conflicts, which is more sound in traffic safety evaluations because crashes are
deemed to be the ultimate criteria used in safety analysis. The correlation-based calibration and EVT distribution-based calibration
are compared. Specifically, the calibration results in this study were compared with a previous study [10]. In this study, eight VISSIM
parameters were calibrated to ensure the extreme value distributions of simulated conflicts match that of the field-observed conflicts.
Essa and Sayed [10] calibrated a VISSIM model using the same signalized intersection with the aim of obtaining a high correlation
between simulated conflicts and field-measured conflicts. The calibrated simulation model parameters as well as the default values
are shown in Table 6. The table shows a difference in the calibrated parameters between this study and Essa and Sayed [10] as a
result of the different calibration objectives.
It should note that only simulated conflicts after calibrating the VISSIM parameters from these two studies are used for com­
parison. The EVT models were developed for the simulated conflicts in Essa and Sayed [10]. Fig. 8 shows the estimated GP dis­
tributions of the compared two studies. The figures clearly show that the estimated GP distributions of simulated conflicts for both
calibration dataset and validation dataset in Essa and Sayed [10] are far away from the GP distributions of real conflicts, whereas the
estimated GP distributions of simulated conflicts in this study are closer to the GP distributions of real conflicts. The results indicate
that a good matching of the extreme value distributions between simulated conflicts and field-measured conflicts cannot be achieved
through calibrating the simulation model parameters with the aim of enhancing the correlation between simulated conflicts and field-
measured conflicts.
To assess the ability of safety estimation, safety measures from the two studies were also compared as shown in Table 7. It can be
seen that the estimated safety measures from simulated conflicts in Essa and Sayed [10] are significantly lower than that from the real
conflicts, whereas the estimated safety measures in this study are similar with that from the real conflicts. The difference in estimated
safety measures are reasonable and can be supported by the calibrated parameters in Table 6. Take CC0 (i.e. headway time) as an
example, the CC0 are 0.5 s and 1.3 s respectively, indicating that the calibrated headway time in this study is lower than that in Essa
and Sayed [10]. The smaller headway time could lead to more severe traffic conflicts. Another example is the SRF. The calibrated
value of 0.6 in this study means that safety distance close to the stop line reduces by 40%, whereas the corresponding safety distance
reduces by 25% in Essa and Sayed [10]. Therefore, the shorter safety distance results in more serious rear-end conflicts at the
intersection approach. In addition, the average MAPE for estimated safety measures for the calibration data and validation data of
Essa and Sayed [10] are 75.24% and 61.17% respectively, which are much higher than that in this study (i.e. 27.45% and 28.80%
respectively). The results suggest that the calibrated simulation model that enhances the correlation between field-measure conflicts
and simulated conflicts cannot ensure reasonable safety measures estimations.
Overall, the comparison results indicate that calibrated simulation model with the objective of enhancing the correlation between

11
 Y. Guo, et al.

Table 5
Estimates of safety measures for different scenarios (validation data).
Scenarios ACF AECF 1 (δ = −0.2) AECF 2 (δ = −0.4) AECF 3 (δ = −0.6) AECF 4 (δ = −0.8) AECF 5 (δ = −1.0)
Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI

12
Real 0 0 14.07 0.87 0 35.13 21.37 0 66.05 58.74 7.53 105.69 109.14 53.28 153.96 170.80 118.90 210.65
S1* 0.04 0 9.53 0.39 0 15.42 2.08 0 25.33 8.09 0 42.74 25.24 0 72.86 67.22 8.29 125.75
S2** 0 0 1.68 0.02 0 6.95 3.36 0 19.40 18.35 1.11 43.29 51.94 21.14 82.67 110.18 72.77 142.60
S3*** 0 0 0.27 0 0 13.05 15.18 0 40.87 52.30 22.41 81.34 104.85 72.90 133.36 170.49 141.07 195.88

Notes: * default scenario; ** after the traffic operation calibration; *** after transferring the calibrated VISSIM parameters.
Simulation Modelling Practice and Theory 106 (2021) 102172
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

Table 6
Calibrated parameters in VISSIM.
CC0 CC1 CC3 CC4&CC5 DD RF* SRF⁎⁎ SUPD

This study 2.75 m 0.5 s −11.5 ± 0.3 −3 m/s2 0.8 0.6 120 m
Essa and Sayed [10] 2.5 m 1.3 s – ± 0.25 −2.8 m/s2 – 0.75 110 m
Default in VISSIM 1.5 m 0.9 s −8 ± 0.35 −2.8 m/s2 0.6 0.6 100 m

– not calibrated;* Safety distance reduction factor for lane change;** Reduction factor for safety distance close to the stop line.

Fig. 8. Estimated GP distributions for the compared studies.

13
 Y. Guo, et al.

Table 7
Estimates of safety measures for the compared study.
Calibration data Validation data
Threshold Simulated conflicts from Essa Simulated conflicts in this study Real conflicts Simulated conflicts from Essa and Simulated conflicts in this study Real conflicts
and Sayed [10] Sayed [10]
Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI

ACF (δ = 0) 0.01 0 1.06 0.02 0 5.63 1.14 0 29.07 0.18 0 5.03 0 0 0.27 0 0 14.07

14
AECF 1 (δ = −0.2) 0.59 0 3.72 3.85 0 16.19 8.66 0 46.01 1.76 0 11.48 0 0 13.05 0.87 0 35.13
AECF 2 (δ = −0.4) 3.52 0.33 9.82 16.15 2.75 33.96 24.37 0 68.62 7.73 0.12 24.21 15.18 0 40.87 21.37 0 66.05
AECF 3 (δ = −0.6) 11.58 4 21.57 38.17 17.87 59.74 49.1 3.19 96.75 23.31 4.95 47.28 52.3 22.41 81.34 58.74 7.53 105.69
AECF 4 (δ = −0.8) 28.24 15.78 41.93 70.71 46.02 94.32 83.42 26.3 132.32 56.21 25.92 86.81 104.85 72.9 133.36 109.14 53.28 153.96
AECF 5 (δ = −1.0) 57.67 40.87 74.46 114.36 88.18 138.27 127.8 67.05 174.99 116.93 78.09 150.67 170.49 141.1 195.88 170.8 118.9 210.65
Average MAPE 75.24% 27.45% 61.17% 28.8%
Simulation Modelling Practice and Theory 106 (2021) 102172
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

field-measured conflicts and simulated conflicts cannot be applied in traffic safety analysis due to its limitation in yielding sound
estimated safety measures estimates. Moreover, the comparison results highlight the need for calibration of the microsimulation
models in this study for using them in safety analysis of signalized intersections.

8. Conclusions

This study proposed an innovative calibration approach for microsimulation models to enhance the models’ ability to estimate
crashes from simulated traffic conflicts at signalized intersections. Automated traffic conflicts analysis based on computer vision
techniques was used to extract field-measure traffic conflicts and their severities at two approaches of a signalized intersection.
Simulated conflicts were collected by extracting the vehicles trajectories from VISSIM and processing the trajectories in SSAM. An
EVT-based calibration procedure of the VISSIM simulation model was proposed to enhance the matching of extreme value dis­
tributions from the simulated conflicts and field-measured conflicts. The first step is to ensure that the simulation model matches the
real traffic condition in terms of arrival type and average vehicle delay. The second step is to calibrate the VISSIM parameters using
genetic algorithms to allow the extreme value distribution (i.e., GP distribution) of simulated conflicts to match the extreme value
distribution of field-measured conflicts. The calibrated VISSIM parameters from one approach were validated by transferring them
directly to the VISSIM model of the other approach. In addition, the calibration results were compared with a previous study [10]
regarding calibrated VISSIM parameters, estimated GP distributions, and estimated safety measures.
The results show that improved matching of estimated GP distributions between the field-measured conflicts and simulated
conflicts is achieved by the proposed calibration approach for VISSIM simulation model. Moreover, the proposed calibration ap­
proach can enable the estimated safety measures from the simulated conflicts to match well with these from the field-measured
conflicts. Regarding the calibration procedure, the traffic operation calibration has a significant impact on improving the estimation
of safety measures, which is reflected by the decreased MAPE value. The EVT-based traffic safety calibration can further significantly
enhance the improvement and has a higher impact compared to the traffic operation calibration. Furthermore, the safety measures
are significantly underestimated for the simulated conflicts from the VISSIM model without calibration or with only the traffic
operation calibration. The estimated safety measures from the simulated conflicts after the EVT-based traffic safety calibration are
more consistent with those from the real conflicts. The comparison results also show that the calibration approach in this study
outperforms the one in Essa and Sayed [10] in terms of their safety prediction as reflected in estimated crash frequency.
There are several limitations of this study. The data used in this study is only from two approaches at one intersection focusing on
the rear-end conflicts, more work is required with a larger data set to generalize the results to other types of intersections and
conflicts types. Theoretically, different types of conflicts can be affected by different parameters in the VISSIM model. The impact of
model parameters on different conflict types should be investigated in future studies. Because of limited data availability, the esti­
mated ACF from both simulated conflicts and field-measured conflicts were not compared with observed crashes. Collecting crash
data could be helpful to validate the results of the EVT-based calibration. Lastly, only the TTC measure was used in the EVT models in
this study. A combination of different conflicts indicators in the EVT approach in the simulation model calibration can also be an
important future research area [30-32].

References

[1] T. Sayed, K. Ismail, M. Zaki, J. Autey, Feasibility of computer vision-based safety evaluations: case study of a signalized right-turn safety treatment, Transp. Res.
Rec., J. Transp. Res. Board 2280 (1) (Jan. 2012) 18–27.
[2] T. Sayed, S. Zein, Traffic conflict standards for intersections, Transp. Plan. Technol. 22 (4) (Mar. 1999) 309–323.
[3] J. Autey, T. Sayed, M.H. Zaki, Safety evaluation of right-turn smart channels using automated traffic conflict analysis, Accid. Anal. Prev. 45 (Mar. 2012)
120–130.
[4] T. Fu, L. Miranda-Moreno, N. Saunier, A novel framework to evaluate pedestrian safety at non-signalized locations, Accid. Anal. Prev. 111 (Feb. 2018) 23–33.
[5] C. Katrakazas, M. Quddus, W. Chen, A simulation study of predicting real-time conflict-prone traffic conditions, IEEE Trans. Intell. Transp. Syst. 19 (10) (Oct.
2018) 3196–3207.
[6] Y. Zhang, et al., Force-driven traffic simulation for a future connected autonomous vehicle-enabled smart transportation system, IEEE Trans. Intell. Transp. Syst.
19 (7) (Jul. 2018) 2221–2233.
[7] C. Wang, et al., A combined use of microscopic traffic simulation and extreme value methods for traffic safety evaluation, Transp. Res. C, Emerg. Technol. 90
(May. 2018) 281–291.
[8] Y. Guo, et al., A comparison between simulated and field-measured conflicts for safety assessment of signalized intersections in Australia, Transp. Res. C, Emerg.
Technol. 101 (Apr. 2019) 96–110.
[9] Y. Guo, et al., Safety evaluation of unconventional outside left-turn lane using automated traffic conflict techniques, Can. J. Civil Eng. 43 (7) (May. 2016)
631–642.
[10] M. Essa, T. Sayed, Simulated traffic conflicts: do they accurately represent field-measured conflicts? Transp. Res. Rec., J. Transp. Res. Board 2514 (Jan. 2015)
48–57.
[11] M. Essa, T. Sayed, Transferability of calibrated microsimulation model parameters for safety assessment using simulated conflicts, Accid. Anal. Prev. 84 (Nov.
2015) 41–53.
[12] R. Fan, et al., Using VISSIM simulation model and Surrogate Safety Assessment Model for estimating field measured traffic conflicts at freeway merge areas, IET
Intell. Transp. Syst. 7 (1) (Mar. 2013) 68–77.
[13] F. Huang, et al., Identifying if VISSIM simulation model and SSAM provide reasonable estimates for field measured traffic conflicts at signalized intersections,
Accid. Anal. Prev. 50 (Jan. 2013) 1014–1024.
[14] P. Songchitruksa, A. Tarko, The extreme value theory approach to safety estimation, Accid. Anal. Prev. 38 (Jul. 2006) 811–822.
[15] L. Zheng, K. Ismail, X. Meng, Freeway safety estimation using extreme value theory approaches: a comparative study, Accid. Anal. Prev. 62 (Jan. 2014) 32–41.
[16] D. Gettman, et al., “Surrogate safety assessment model and validation: final report,” FHWA, McLean, VA, USA, Tech. Rep. FHWA-HRT-08-051, Jun. 2008.
[17] F. Cunto, F. Saccomanno, Calibration and validation of simulated vehicle safety performance at signalized intersections, Accid. Anal. Prev 40 (3) (May. 2008)
1171–1179.

15
Y. Guo, et al. Simulation Modelling Practice and Theory 106 (2021) 102172

[18] A. Dijkstra, et al., Do calculated conflicts in microsimulation model predict number of crashes? Transp. Res. Rec., J. Transp. Res. Board 2147 (Jan. 2010)
105–112.
[19] L. Vasconcelos, et al., Validation of the surrogate safety assessment model for assessment of intersection safety, Transp. Res. Rec., J. Transp. Res. Board 2432
(Jan. 2014) 1–9.
[20] K. Campbell, H. Joksch, and P. Green, “A bridging analysis for estimating the benefits of active safety technologies,” Univ. of MI, Transp. Res. Inst., Ann Arbor,
MI, USA, Tech. Rep. UMTRI-96-18, Apr. 1996.
[21] P. Songchitruksa, Innovative non-crash-based safety estimation: an extreme value theory approach, Lyles Sch. Civ. Eng., Purdue Univ., West Lafayette, IN, USA,
2004.
[22] A. Tarko, Use of crash surrogates and exceedance statistics to estimate road safety, Accid. Anal. Prev. 45 (Mar. 2012) 230–240.
[23] T. Gordon et al., “A multivariate analysis of crash and naturalistic driving data in relation to highway factors,” Transp. Res. Board, Washington, D.C., USA, Tech.
Rep. S2-S01C-RW-1, Apr. 2013.
[24] H. Farah, C.L. Azevedo, Safety analysis of passing maneuvers using extreme value theory, IATSS Res. 41 (1) (Apr. 2017) 12–21.
[25] D. Åsljung, J. Nilsson, J. Fredriksson, Using extreme value theory for vehicle level safety validation and implications for autonomous vehicles, IEEE Trans. Intell.
Veh. 2 (4) (Dec. 2017) 288–297.
[26] A. Tarko, Estimating the expected number of crashes with traffic conflicts and the Lomax Distribution - a theoretical and numerical exploration, Accid. Anal.
Prev. 113 (Apr. 2018) 63–73.
[27] L. Zheng, T. Sayed, A. Tageldin, Before-after safety analysis using extreme value theory: a case of left-turn bay extension, Accid. Anal. Prev. 121 (Dec. 2018)
258–267.
[28] L. Zheng, T. Sayed, Application of extreme value theory for before-after road safety analysis, Transp. Res. Rec., J. Transp. Res. Board 2673 (4) (Apr. 2019)
1001–1010.
[29] L. Zheng, T. Sayed, Comparison of traffic conflict indicators for crash estimation using peak over threshold approach, Transp. Res. Rec., J. Transp. Res. Board
2673 (5) (Apr. 2019) 493–502.
[30] L. Zheng, et al., Bivariate extreme value modeling for road safety estimation, Accid. Anal. Prev. 120 (Nov. 2018) 83–91.
[31] C. Wang, C. Xu, Y. Dai, A crash prediction method based on bivariate extreme value theory and video-based vehicle trajectory data, Accid. Anal. Prev. 123 (Feb.
2019) 365–373.
[32] L. Zheng, T. Sayed, M. Essa, Validating the bivariate extreme value modeling approach for road safety estimation with different traffic conflict indicators, Accid.
Anal. Prev. 123 (Feb. 2019) 314–323.
[33] L. Zheng, T. Sayed, From univariate to bivariate extreme value models: approaches to integrate traffic conflict indicators for crash estimation, Transp. Res. C,
Emerg. Technol. 103 (Jun. 2019) 211–225.
[34] VISSIM 5.30-05 User Manual, PTV AG, Karlsruhe, Baden-Württemberg, Germany, 2011.
[35] N. Saunier, T. Sayed, A feature-based tracking algorithm for vehicles in intersections, Proc. 3rd CRV, Quebec, Canada, 2006, pp. 1–7.
[36] N. Saunier, T. Sayed, C. Lim, Probabilistic collision prediction for vision-based automated road safety analysis, Proc. IEEE ITSC, Seattle, WA, USA, 2007, pp.
872–878.
[37] A. Tageldin, T. Sayed, K. Ismail, Evaluating the safety and operational impacts of left-turn bay extension at signalized intersections using automated video
analysis, Accid. Anal. Prev. 120 (Nov. 2018) 13–27.
[38] Y. Guo, T. Sayed, M.H. Zaki, Evaluating the safety impacts of powered two wheelers on a shared roadway in China using automated video analysis, J. Transp. Saf.
Secur. 11 (4) (2019) 414–429.
[39] K. Ismail, Sayed T, N. Saunier, A methodology for precise camera calibration for data collection applications in urban traffic scenes, Can. J. Civil Eng. 40 (1) (Jan.
2013) 57–67.
[40] N. Saunier, T. Sayed, K. Ismail, Large-scale automated analysis of vehicle interactions and collisions, Transp. Res. Rec., J. Transp. Res. Board 2147 (Jan. 2010)
42–50.
[41] Ring Barrier Controller User Manual, PTV America, Portland, OR, USA, 2010.
[42] S. Coles, An Introduction to Statistical Modeling of Extreme Values, Springer-Verlag, UK, 2001.
[43] Highway Capacity Manual, Transp. Res. Board, Washington, D.C., USA, 2010.
[44] D. Goldberg, J. Holland, Genetic algorithms and machine learning, Mach. Learn. 3 (2) (1988) 95–99.
[45] R. Iman, J. Davenport, and D. Zeigler, (1980). “Latin hypercube sampling (program user's guide),” Sandia Labs, Albuquerque, NM, USA, Tech. Rep. SAND-79-
1473, Jan. 1980.
[46] Y. Guo, L. Zheng, T. Sayed, A comparison of collision-ased and conflict-based safety evaluation of left-turn bay extension, Transportmetrica A 16 (3) (2020)
676–694.

16