Adaptive Video-Based Algorithm for Accident

detection on Highways
Boutheina Maaloul∗† , Abdelmalik Taleb-Ahmed∗ , Smail Niar∗ , Naim Harb† and Carlos Valderrama†
∗ University of Valenciennes and Hainaut Cambresis, Valenciennes-France
Email: surname.name@univ-valenciennes.fr
† University of Mons, Mons-Belgium

Email: Surname.NAME@umons.ac.be

Abstract—For the past few decades, automatic accident be used as evidence for further analysis. Consequently, de-
detection, especially using video analysis, has become a veloping a video based algorithm that can be embedded in a
very important subject. It is important not only for traffic System on Chip (SoC) connected to CCTV can accelerate the
management but also, for Intelligent Transportation Systems
(ITS) through its contribution to avoid the escalation of accidents information diffusion to infrastructure.
especially on highways. In this paper a novel vision-based road In this paper, we present a novel algorithm for automatic
accident detection algorithm on highways and expressways accident recognition on highways. It consists of a traffic
is proposed. This algorithm is based on an adaptive traffic motion modeling using Farneback Optical flow [5] for pix-
motion flow modeling technique, using Farneback Optical els velocity vectors estimation and a heuristic approach for
Flow for motions detection and a statistic heuristic method
for accident detection. The algorithm was applied on a set adaptive thresholds setting. First, estimated pixels velocity
of collected videos of traffic and accidents on highways. The vectors are filtered and classified, then a model of traffic
results prove the efficiency and practicability of the proposed motion field is established. Thresholds are then set from
algorithm using only 240 frames for traffic motion modeling. few numbers of frames. Typically 240 frames are used for
This method avoids to utilization of a large database while more accurate detection. Thus, the changes of traffic density,
adequate and common accidents videos benchmarks do not exist.
such as congestion, increasing traffic rate, etc., are taken
Keywords– Accident detection; Abnormal behavior detection; into consideration. Our approach is aimed to be used as
computer vision; Intelligent Transportation Systems; ITS; video a pre-processing technique and may correspond to the first
processing; video surveillance; Farneback optical flow. stage of more complex algorithm for abnormal behavior on
road. Our method can be easily implemented on hardware
I. I NTRODUCTION without recurring to complex deep learning algorithms, while
achieving significant interesting results.
Automatic video surveillance systems have been developed The paper is organized as follows: Section 2 shows a brief
to detect and analyze abnormal behaviors or situations of risk, presentation of existing video-based accident detection algo-
reducing human monitoring of activities captured by cameras. rithms. Section 3 details our approach while Section 4 presents
One of the applications of video surveillance is traffic moni- experimental results. In the last section of the paper, we
toring such as traffic flow detection, cars license plate recog- conclude and we present some perspectives.
nition, vehicle tracking, and vehicle motion analysis[1][2].
In particular, motion analysis aims to detect abnormal traffic
behavior, accidents, and sudden events, especially in the case
of emergency and disaster management. Road accidents can
cause serious injuries leading to lifelong disabilities and even
death. Each additional rescue minute can mean the difference
between life and death as revealed by The Golden hour [3].
Therefore, providing rapid assistance to injuries is mandatory.
Moreover, if not addressed promptly, accidents may cause
traffic jams and eventually accident escalation, especially
on highways, where accidents are most likely to occur. A
statistic study in France shows that the number of accidents in
highways has increased by 25% in 2015 [4], while the number
of accidents in urban region remains the same. Surveillance
cameras or CCTV, covering large areas such as wide roads,
provide detailed information on the accident causes, a direct
communication infrastructure and video storage resources to Fig. 1: General steps for video based accident detection
978-1-5386-3166-9/17/$31.00 c 2017 IEEE algorithms
 Ref Motion detection
[7],[8] BS + Vehicle tracking
[9] GMM + Mean Shift Algorithm
[10] BS, KLT Tracker
[11] BS, shadow filtering, vehicle tracking
[13] Lukas kanade Optical flow
[15] Lukas kanade Optical Flow
[16] Farneback OF, thermal diffusion, enthalpy Model
[17] Background subtraction, extrating line positions, tacking
[18] OF-SIFT
Ours Farneback optical flow
TABLE I: A comparison study of the different video based approaches for accident detection
II. R ELATED WORK
Video-based road accident detection approaches are mainly
done using three steps: 1) Motion detection, 2) Feature Extrac-
tion and then 3) Features Analysis and Accident Recognition
as shown in figure 1 [6].
The most frequently used algorithms for motion detection
are Background Subtraction (BS) and Optical Flow (OF).
Features related to traffic motion and vehicles trajectories
can be extracted using different processing techniques such
as tracking. Finally, analysis of these traffic features can
differentiate normal motion versus abnormal.
For the past few decades, researchers presented different
techniques to increase the detection rate of road accidents.
Some approaches use tracking algorithms to extract vehicles
parameters, such as speed, acceleration, orientation, position
and dimension. These features are compared to a predefined
thresholds to detect an accident. In [7] and [8] vehicle tracking
is performed by looking for the same vehicle in the next
frame using its area, then accident is detected when vehicles
parameters exceed the predefined thresholds. The features
extracted are velocity, acceleration, area and the orientation
per vehicle. Zu et al.[9], however, uses the Gaussian Mixture
Model (GMM) for motion detection then the Mean Shift
algorithm for tracking to extract vehicles features (velocity,
acceleration and orientation).
The accident recognition is also performed by comparing
to static thresholds. In some other approaches, vehicles trajec-
tories are extracted, clustered and compared to a pre-learned
models of normal traffic trajectories. For instance, Akos and
al [10] and Lee and Oh [11] use trajectory sudden change
for accident detection. They used BS and the continuous
Hidden Markov Model (C-HMM) in [10] and Calogero-Moser
system in [11] for motion detection and features extraction and
analysis.
Unfortunately, these approaches were based on tracking
algorithms which are very sensitive to change in luminosity,
presence of shadow and occlusion. This may cause the loss of
the target and hence, may give unsatisfactory results. For this
reason researchers propose to consider the road scene as one
entity to be analyzed instead of decomposing it and analyzing
the motion of each vehicle seperately. They focus on particles
(pixels) motion, so a traffic motion field is extracted using
Dense OF [5] or Sparse OF [12].
Features extracted Accident recognition Scenario
Velocity, acceleration, orientation, area Static threshold Intersections
Velocity, acceleration, orientation Static threshold Highway
Vehicle trajectory clustered C-HMM Intersections
Vehicle lane (trajectory) Calogero-Moser system Highway
HFG Logistic regression Multiple scenarios
Histogram of words Topic Model Multiple scenarios
Traffic model SPH Multiple scenarios
Time and space model diagram SVM Multiple scenarios
BOF for feature coding Extream learning machine Highway
Velocity and oriention: motion model Dynamic Thresholds Highway
Sadek et al.[13] proposed a framework for accident recog-
nition, based on a new algorithm named Histogram of Flow
Gradient (HFG) for motion detection and feature extraction.
HFG is similar to Histogram of Oriented Gradient (HOG) [14]
running on Lukas Kanade OF estimated vectors. A map for
all motions is created and logistic regression is applied to
detect accidents. However, they use OF vectors without noise
filtering, thus eliminating perturbed and uncoherent vectors.
Ahmadi et al. in [15] exploit the Topic Model technique.
In this technique a model of normal traffic motion is used
based on Lukas Kanade OF vectors indexed in a document of
words. Each word represents velocities in a specific range of
orientation. Abnormal traffic is detected when the computed
words are different from the model.
On the other hand, Ullah et al. [16] used Farneback OF
for motion detection then applied Thermal Diffusion to find
the coherent motion field. They used the Enthalpy Model
to filter particles and model the traffic motion which is the
feature in these case. Then they used Smoothed Particle Hy-
drodynamics SPH for accident recognition. In another related
work, Ren et al.[17] proposes to extract moving objects using
background subtraction then extract road segments and lines
positions. Then a traffic tracking and time and space diagram
is generated. A Support Vector Machine (SVM) was used for
classification and accident recognition.
Chen et al.[18] extract OF and Scale Invariant Feature
Transform (SIFT) features. Then they use Bag of Feature
(BOF) for features encoding and finally they use extreme
learning machine classifier for accident detection. All related
sited works are summarized in Table.1. However comparison
in terms of efficiency and reliability study remains not
possible. The main reasons is databases and metrics used to
measure the accuracy of these approaches are not the same.
We also added our approach in Table.1 using different
algorithms for accident recognition based on different
extracted features. Our work uses the Farneback OF for
motion detection to extract the velocity and the orientation
of pixels. Using this features and a heuristic approach for
adaptive thresholds setting for accident recognition. The
aim of our approach is to achieve a significant accuracy
considering the change of luminosity and traffic density
using adaptive threshold. On the other hand, our algorithm is
theoretically implementable on hardware architecture.
 270
225 315

Optical flow and Computing 180 360

0
noise filtering Model

135 45
90

Frame1 SVA(1) SVB(1)

.
.
.

Frame N SVA(N) SVB(N)

Optical flow and

noise filtering SVA(i) SVB(i)

THA THB

Comparison

Alarm

Fig. 2: Flowchart of the proposed approach for accident detection on highways

computed using equations (1) and (2) respectively. Following,

III. P ROPOSED APPROACH
In this section, we will elaborate our proposed approach for
accident detection on highways. At first, we consider a traffic
motion scene in a motorway as normal motion flow when
the velocity vector of each moving vehicle (vehicle speed)
has a constant or gradual magnitude, and have an orientation
suitable to the shape of the road. Consequently, an accident
including one or more vehicles could be detected when a
sudden and sharp change in their velocity vectors occurs. This
may concerns either their magnitude and/or orientation.
It is important to note that velocity and orientation pa-
rameters depend on the position and the calibration of the
camera. Hence, detecting an accident needs first the extraction
of normal motion flow parameters from different traffic scenes
captured by the same camera to create a normal traffic model.
This model is composed of a number of thresholds that we
update at every frame with normal motion flow in the video.
The initial thresholds are calculated based on the first N frames
of the video. The N frames value is explained in the following
section.
The normal traffic model is created by analyzing the traffic
motion field. This is done using the Farneback OF which
estimates the next position of each moving pixel (particle)
between two given frames in a predefined Region Of Interest
(ROI). In our case, ROI is limited to the highway road
segment. Given a moving pixel P with coordinate (xi , yi ) in
f ramei , OF computes its coordinate in the next f ramei+1
(xi+1 , yi+1 ). The velocity and the orientation of P can be
we first filter the noise and then we create our model before
finally comparing to the normal traffic model to detect an
accident.
→
− p
k V k = (xi+1 − xi )2 + (yi+1 − yi )2 (1)
(yi+1 − yi ) 180
θ = tan−1 . (2)
(xi+1 − xi ) π
A. Velocities classification and noise filtering
After observation, we distinguish two categories of velocity
vectors: vectors generated by video distortion (dust, wind,
etc) and vectors related to the vehicles motion. Since OF is
sensitive to change of luminosity and noise, distorted vectors
could appear. These vectors typically have small magnitudes
so that a threshold ε has been set for filtering. Every vector
with a smaller magnitude than ε is filtered. The threshold ε is
calculated by performing OF on a fixed number of successive
frames on which no traffic is detected. In out case, we fixed
the number of frames to calculate ε to 10. ε is computed once,
and depends on the camera location and calibration.
Remaining velocity vectors, vectors related to the vehicles
motion, belong to the following groups: vectors with orienta-
tion corresponding to the normal motion flow and distracting
vectors with other orientations. These orientations can be
caused by the motion of the wheels, the presence on the road of
shadows, the variation in the light reflected on the car’s body,
 Fig. 3: Traffic velocity variation: SVA (on the left) and SVB (on the right)
etc. It is due to these vectors that the optical flow precision is
reduced.
B. Normal motion flow modeling
Our normal motion flow is presented in Figure.2 as a
flowchart of the proposed approach in case of one way road.
The first step of motion detection consists on applying OF
for each f ramei , followed by noise filtering. Next step is
feature extraction: for each f ramei we compute the sum of the
vectors with orientation range corresponding to the direction
of traffic [θ1 , θ2 ], SVA (which are presented in green color
in Figure.2) and the sum of the remaining distracting vectors
(presented in yellow color in Figure.2) SVB. SV Ai and SV Bi
present a sub-model of traffic motion of f ramei . The range
of orientation [θ1 , θ2 ] corresponding to normal motion flow
is already known as the position of the camera is known.
Figure.3 shows an example of the sum of the vectors
velocities SVA and SVB variation as a function of frames,
for an example of normal traffic sequence. in the Figure.3 we
show also some different frames and the values of SVA and
SVB indicated in the two curves. We notice that, the more
the vehicles are close to the camera, the more the velocities
magnitude are important.
We assume that the variation of SVA and SVB follows
the binomial parameter. So, we compute the average of the
different SVA and SVB and their respective standard deviation
σ1 and σ2 for N frames. These N samples are saved as a
list LN of N sub-models. The thresholds THA and THB are
the normal traffic model and are computed as formulated in
equations (3) and (4). λ is a tunable constant fixed to acquire
higher precision for the model. λ and N are the parameters
that will be experimentally determined and fixed later.
1 N
T HA = Σ (SV A)i + λσ1,i (3)
N 1
1 of the number of Correct Positive Results(CPR) divided by
T HB = ΣN (SV B)i + λσ2,i (4)
N 1
C. Adaptive threshold computation and accident detection
After N processed frames, we start the detection step
by comparing every new sub-model extracted from the
following frame to the normal traffic model thresholds. In
case of exceeding values, an accident is reported. The alert is
performed by a simple alert message. On the other hand, if
the sub-model presents a normal traffic flow, threshold will
be updated using the late sub-model as a new sample in LN
and thus deleting the first saved sub-model sample. In this
case, car line change, traffic density, and luminosity change
from day to night and night to day are not detected as an
accident thanks to our adaptive threshold. This, on the other
hand does not affect the precision of our system since these
changes are progressive while accidents cause brutal changes.
IV. E XPERIMENTAL RESULTS
The algorithms is developed on Visual C++ 2010 express,
using the OpenCV library running on windows 7 machine
with Intel i7 processor. Experimentation was performed on
40 different videos collected from YouTube as samples of
different quality, orientation and environmental conditions,
with normal traffic on highways and others with accidents.
This helps establishing parameters needed to compute traffic
models and building a calibration method (THA, THB). These
parameters are N: the number of frames for modeling, and λ:
the constant that defines the thresholds.
To obtain the adequate parameters giving the most accurate
results, the algorithm was tested on videos using different
combination of values of N and λ. The accuracy of obtained
results is measured using the F1 score formulated in equation
(5) where p is the precision given by the result of the division
the number of All Positive Results (APR) and r is the recall
Fig. 4: Variation of p, r and F1 score for different values of
N and λ

given by result of the division of CPR by the Effective Positive

Results (EPR) that should have been returned.

p.r
F1 score = 2. (5)
p+r
CP R
p= (6)
AP R
CP R
r= (7)
EP R
We present the results of the test of the different combination
of N and λ as curves in Figure.4. The horizontal axis presents Fig. 5: Examples of detected accident scene with the proposed
the number of frames. The first vertical axis presents the approach. Accidents are annotated by rectangles
values of p and r, The second vertical axis presents the values
of F1 score. The black curves presents the values of the p,
r and F1 score as a function of N for λ equal to 1 and the Array) implementation of different algorithms of Optical flow
green ones presents the variation of the values of the p, r and such as [19][20]and [21]. The heuristic part of our algorithm
F1 score as a function of N for λ equal to 2. Results show can be performed on a processor such as ARM.
that for N equal to 240 and λ equal to 1, we achieve the
V. C ONCLUSIONS
highest value of F1 score equal to 0,77. While for N equal to
60 and λ equal to 1 we achieve the lowest value of F1 sore In this paper we proposed a new approach for road acci-
equal to 0,19. Figure.5 shows annotated accidents detected dents detection on highways. Our approach is based on OF
by our approach in some traffic accident videos. estimation and heuristic computation for adaptive thresholds.
Experimentation proved the efficiency and practicability of the
Since there is no a proper benchmark of video accidents, proposed algorithm using only 240 frames for traffic motion
comparing to stae of the art results is not valuable. In addi- modeling. We achieved a precision of 0.68, a recall of 0.91
tion, learning techniques and tests become more complicated. and an F1 score of 0.77.
Hence, the motivation of creating an approach based on a In our future work, we will improve the algorithm to
few and sufficient number of frames to model the traffic detect accidents in other scenarios such as in intersections
flow and set adaptive thresholds. In addition, the advantage and in tunnel. Next, we will focus on the implementation
of using a reduced amount of data makes the approach more of our algorithm on a hardware heterogeneous architecture
appropriate for further hardware implementation. For instance, making surveillance camera automatically capable of detecting
some existing works present FPGA (Field-Programmable Gate abnormal behaviors on road.
 We think that a specific hardware design implementation is [18] Y. Chen, Y. Yu, and T. Li, “A vision based traffic accident detection
mandatory to accelerate the algorithm ensuring real time pro- method using extreme learning machine,” in 2016 International Confer-
ence on Advanced Robotics and Mechatronics (ICARM), Aug 2016, pp.
cessing and accident detection alerts. In the realized measure- 567–572.
ments, the dense optical flow alone performed 1 frame/second [19] K. Seyid, A. Richaud, R. Capoccia, and Y. Leblebici, “Fpga based
for a video resolution of 1280x720 running an ARM cortex- hardware implementation of real-time optical flow calculation,” IEEE
Transactions on Circuits and Systems for Video Technology, vol. PP,
A9 processor. For real time applications like accident detection no. 99, pp. 1–1, 2016.
and a 30 frame/sec camera, this delay is not acceptable. [20] J. Monson, M. Wirthlin, and B. L. Hutchings, “Implementing high-
performance, low-power fpga-based optical flow accelerators in c,”
R EFERENCES in 2013 IEEE 24th International Conference on Application-Specific
Systems, Architectures and Processors, June 2013, pp. 363–369.
[1] N. Buch, S. A. Velastin, and J. Orwell, “A review of computer vision [21] R. Rustam, N. H. Hamid, and F. A. Hussin, “Fpga-based hardware
techniques for the analysis of urban traffic,” IEEE Transactions on In- implementation of optical flow constraint equation of horn and schunck,”
telligent Transportation Systems, vol. 12, no. 3, pp. 920–939, September in 2012 4th International Conference on Intelligent and Advanced
2001. Systems (ICIAS2012), vol. 2, June 2012, pp. 790–794.
[2] S. Sivaraman and M. M. Trivedi, “Looking at vehicles on the road:
A survey of vision-based vehicle detection, tracking, and behavior
analysis,” IEEE Transactions on Intelligent Transportation Systems,
vol. 14, no. 4, pp. 1773–1795, Dec 2013.
[3] E. B. Lerner, MS, EMT-P, R. M. Moscati, and MD, “The golden
hour: Scientific fact or medical ”urban legend”?” Official Journal of the
Society for Academic Emergency Medecine, vol. 8, no. 3, pp. 758–760,
July 2001.
[4] “National observatory on road safety road accidentality 2015 - estimates
as of january 26, 2016.” [Online]. Available: http://www.securite-
routiere.gouv.fr/
[5] G. Farnebäck, Two-Frame Motion Estimation Based on Polynomial
Expansion. Berlin, Heidelberg: Springer Berlin Heidelberg, 2003, pp.
363–370.
[6] B.Maaloul, A. Ahmed, S.Niar, C.Valderrama, F. Derraz, and N. Harb:,
“Vision-based traffic accident detection techniques opportunities and
challenges,” in International Conference on Advanced Communication
Systems and Signal Processing, November 2015.
[7] Y. K. Ki and D. Y. Lee, “A traffic accident recording and reporting
model at intersections,” IEEE Transactions on Intelligent Transportation
Systems, vol. 8, no. 2, pp. 188–194, June 2007.
[8] “Okcars: Oklahoma collision analysis and response system.” [Online].
Available: http://www.trb.org/
[9] Z. hui, X. yaohua, M. lu, and F. Jiansheng, “Vision-based real-time traffic
accident detection,” in Intelligent Control and Automation (WCICA),
2014 11th World Congress on, June 2014, pp. 1035–1038.
[10] O. Akooz and M. E. Karsligil, “Severity detection of traffic accidents at
intersections based on vehicle motion analysis and multiphase linear
regression,” in Intelligent Transportation Systems (ITSC), 2010 13th
International IEEE Conference on, September 2010, pp. 474–479.
[11] I. J. Lee, “An accident detection system on highway through cctv
with calogero-moser system,” in 2012 18th Asia-Pacific Conference on
Communications (APCC), October 2012, pp. 522–525.
[12] B. D. Lucas and T. Kanade, “An iterative image registration
technique with an application to stereo vision,” in Proceedings
of the 7th International Joint Conference on Artificial Intelligence
- Volume 2, ser. IJCAI’81. San Francisco, CA, USA: Morgan
Kaufmann Publishers Inc., 1981, pp. 674–679. [Online]. Available:
http://dl.acm.org/citation.cfm?id=1623264.1623280
[13] S. Sadek, A. Al-Hamadi, B. Michaelisy, and U. Sayed, “Real-time
automatic traffic accident recognition using hfg,” in Pattern Recognition
(ICPR), 2010 20th International Conference on, August 2010, pp. 3348–
3351.
[14] N. Dalal and B. Triggs, “Histograms of oriented gradients for human
detection,” in 2005 IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR’05), vol. 1, June 2005, pp. 886–
893 vol. 1.
[15] P. Ahmadi, M. Tabandeh, and I. Gholampour, “Abnormal event detection
and localisation in traffic videos based on group sparse topical coding,”
IET Image Processing, vol. 10, no. 3, pp. 235–246, October 2016.
[16] H. Ullah, M. Ullah, H. Afridi, N. Conci, and F. G. B. D. Natale, “Traffic
accident detection through a hydrodynamic lens,” in Image Processing
(ICIP), 2015 IEEE International Conference on, Septembre 2015, pp.
2470–2474.
[17] J. Ren, Y. Chen, L. Xin, J. Shi, B. Li, and Y. Liu, “Detecting and
positioning of traffic incidents via video-based analysis of traffic states
in a road segment,” IET Intelligent Transport Systems, vol. 10, no. 6,
pp. 428–437, 2016.