Vehicle Monitoring on Highways

Siddharth Das Joshin Rexy Dr. Vishal Satpute

Electronics & Communication Electronics & Communication Electronics & Communication
VNIT Nagpur VNIT Nagpur VNIT Nagpur
Nagpur, India Nagpur, India Nagpur, India
hiiamsid518@students.vnit.ac.in hiiamsid518@students.vnit.ac.in hiiamsid518@students.vnit.ac.in

Abstract— It is a well-known fact that are predicted to become increasingly complicated,

perception of road (lane) plays a vital role in
assisting drivers by using software that can convey
them the possible risks in their surroundings. The
strategy of this work is to prevent car accidents on
highways mainly due to rash driving that consists of
sudden lane departure or not keeping up with the
lane in case of curved roads. Hence, we propose a
method to reduce these crashes by mounting a
camera on the vehicle which will continuously keep
track of the view in front of vehicle and deliver
certain relevant warnings to the driver in case of
any rash driving or fatal situations. Also, it can be
helpful for self-driving cars by providing a wide
range of possibilities to learn by software and move
vehicle on its own in the lane.
We employ image processing techniques to
detect and solve some of the most prevalent
roadway concerns. The problem statement is
broken down into basic building blocks to better
understand the comprehensive list of potential
techniques within this framework. It comes with a
decision model that will provide the driver the
required cautions.
eventually achieving total freedom.[1]
The main problem we face while developing such
systems is the perception problem. [3]
The perception problem includes mainly two
elements:
a) Road-lane perception 
b) Obstacle detection   
Road-Lane Line detection is an important concept
used to define the path taken by “self-driving
cars” in order to avoid getting into the wrong lane.
Our system nee In urban, rural, and highways,
identifying the size of the road, the number and
position of lanes, merging, splitting, and ending
lanes and roads are all part of road and lane
knowledge.[2][4]

Keywords—Image processing, warning system,

road, lane detection, driver aid.
I. INTRODUCTION (HEADING 1)
In today's, era motor vehicles in emerging nations
have progressively increased during the last
decade so do the traffic accidents. It is vital to
build technology that can avoid such unanticipated
incidents. Nowadays Vehicles are rapidly
incorporating advanced driver aid systems, which
alert drivers in risky circumstances or actively
participate in driving. However, maintaining a
constant watch on drivers to eliminate all potential
threats involves a number of difficult difficulties.
Human actions are, in reality, difficult to
understand, anticipate, and manage using present
technology. During the next decade, such systems
 II. LITERATURE SURVEY Orientation Consistency Ratio for finding the 
dominant edges[28]. Peyman et al. in his paper
suggested a method where he used only four
Gabor filters and suggested a novel fast voting
scheme for real-time requirements [29,30]. In[31]
new background is computed using  temporal
median function  using  previously sampled
frames.A. Elgammal, D. Harwood, and L.S. Davis
in their paper ‘‘Non-parametric model for
background subtraction’’ [32].With five cameras
and thirteen lidars ,Huang, A.S., Moore, D.,
Antone, M. et al[33] suggested a methodology
which later incorporated into a closed-loop
controlled to successfully steer an autonomous
vehicle.

In the DARPA Grand Challenge robot race,

Hendrik Dahlkamp demonstrated a system for
recognising drivable surfaces in tough unpaved
and offroad terrain conditions[34].Sebastian
Thrun, Mike Montemerlo, Andrei Aron in their
paper “Probabilistic Terrain Analysis For High-
Speed Desert Driving” suggested a PTA
(probabilistic
terrain analysis) algorithm for terrain
classification with a “fast-moving robot
platform”[35].

III. APPROACH

A. Distortion Correction
When a camera captures a 3D world image in
2D it is obvious that it will have many
imperfections. An image taken from camera has
many distortions in it. Our first step will be to clear
the image of any such distortions. There are
generally two kinds of distortions in an image
namely barrel and pincushion distortion.

Du, Xinxin, and Kok Tan in their paper discussed

edge-based detection techniques which can be
applied to detect the road[24].In[25] author used
Region-based methods to look for texture , color
attributes in the image to segment the road region
Curved lenses are used in real cameras to create
from the background[25] .One of the region based
images, and light rays bend a bit too much or too
method where segmentation was performed in the little at the lens' edges. This produces an effect that
seeded region[26].In 2004  Rasmussen, C. distorts picture edges, making lines or objects look
presented a algorithm which is used to estimate more or less curved than they are. Hence, this
the dominant orientations by bank of Gabor results in radial distortion.
wavelet[27].The author Kong et al. in their paper
proposed a road segmentation approach using Also, if camera is not aligned properly parallel to
the image, some image points may seem to be
closer/farther than they actually are. This is termed
as tangential distortion.
We can rectify this error by calculating some
constants for the camera k1, k2, k3 using OpenCV.
Formula for radial distortion correction.
Hence, we get 2D top view of the road which
becomes easier for the further processing.
C.
Formula for tangential distortion correction.
B. Perspective Transform (Bird’s View)
Human eyes have the tendency to make things
nearer look bigger and that of far away objects
comparatively smaller. This is simple definition of
perspective.
To align the image appropriately, Perspective
Transform is a really handy function. After
applying Perspective Transformation to the
picture, it transforms it in a linear way. Here we
transform the image of road as seen by the camera
on the vehicle to an image such that it appears as
the picture it taken from above parallel to the
ground. In this way we convert a 3D image into a
2D image which is a lot easier to process for our
further applications.
D.
Fig- Original image of highway
We take a simple fact into consideration here that
the lanes are always parallel and in the bird’s view
transformation they must comply the same. So, we
can draw two imaginary straight lines over the
lanes until they meet and the coordinates of the
lines will give us the points for the required
transformation.
Fig- Image after perspective transform
Edge detection & Thresholding
Next step will be to perform the edge
detection on the image. It is helpful to find out
the lane lines on the road as it comprises of
sharp lines on the road and can be easily
identified in edge detection process. For this
we will use Canny edge detection algorithm. It
uses Sobel filter which can find out the vertical
as well as the horizontal edges in an image.
Fig-Sobel operator for vertical & horizontal edges resp.
Using this filter, we find out the x and y
component of the edge detection output.
Finally, we add both images with an ideal
thresholding value to get the final result for our
Saturation binary.
Slope Threshold
By applying saturation binary, we see that we got
the lane lines but with that we also get unwanted
noise which may be due to the imperfections in
road texture. As Canny edge detection picks up all
the edges these unwanted lines also pop up in the
edge detection. So, to tackle this problem we apply
Slope threshold which takes into account only a
limited range of edges which come in the bracket of
specified angles. The direction is given by inverse
tangent of the y gradient divided by x gradient.
i.e. arctan (Sobel-y/Sobel-x)
Using this we can remove unwanted noise that
comes with the edge detection. Again, applying an
ideal value of threshold will result into an image
shown below
 loses much information about white lanes if the
road is cemented one)

Fig-Combined Threshold.

E. Gamma correction
Fig-Original image.
The quality of digital photos may be
compromised due to the limits of image-capturing
technologies or the presence of a non-ideal
environment. Despite significant advances in
imaging technology, users' expectations for clean
and relaxing sights are not always met.[1]
To to suitably improve the contrast of the
picture, an adaptive gamma correction is proposed,
with the parameters of AGC being modified
dynamically depending on the image information.

Fig-Problems in gray image.

As we know the fact that H and S channels of

HSL model stay fairly consistent in shadow or
excessive brightness, so for this problem we
explore the overlying issue with a new color model
HSL (hue, saturation, lightness) to try and process
our image. Here we discover that the saturation
component is quite useful as it stores the
Fig-Original Image.
information of lane lines pretty well in harsh
conditions!
Equations for conversion from RGB to S channel
is given below. Here
Vmax is max(R,G,B)
Vmin is min(R,G,B)

Fig-After Gamma correction

F. Trying out HSL Colour Model

Till now our thresholding works perfectly fine,
but it also has some shortcomings. When we
encounter harsh weather conditions or a cemented
road with almost matches the white color of the
lane line the edge detection fails to differentiate the
lane lines from the image. (As we have to covert
image to gray component for edge detection it

Fig-S channel threshold.
The S channel reliably detects lane lines of
various colors and henceforth we will be also
calculating the threshold corresponding to this
processed image so as to improve our pipeline.
G. Finding lane lines in thresholded image
A histogram is plotted corresponding to the
obtained output to find out the x-coordinate of
the lane line. Histogram is plotted according to
white pixels in the corresponding x axis.(white
pixels here represent the lane lines). Here two
x-coordinate having most number of white
pixels will be chosen as the start of the left and
right lane where y=0.
Fig-Peaks showing position of lanes.
The second technique relies on knowledge of
the previous frames, as well as the position of lane
lines inside them. By looking around the final
frames fitting polynomial, additional lane lines are
discovered. These new lane line pixels are fitted
with a polynomial.
This approach is less computationally intensive,
which aids in the speeding up of the video data
lane detection process.
The function was designed to keep the lane lines
fairly straight and their distance from one other
from increasing too much. When the road edge
was recognized as the left lane line in the video,
this happened on occasion. When the improper
line was recognized, the lane line distances were
watched, and the acceptable distance was
determined to avoid this. The difference in the first
polynomial coefficient was detected and a suitably
modest limit was set to sanity check for similar
curvature to assure parallel lane lines.
H. Calculate polynomial & adding line to image
We need to overlap our lane lines which we
identified earlier. For this purpose we calculate a
second degree equation for the lane lines (second
order as the roads can also be curved). The
calculated polynomial from cv2.polyfit function is
then superimposed on the image of road.
I. Checking for best fit
A best fit of lane mask is calculated using back
propagation having the previous and new values of
the lane matrix. New position is calculated by
combining both using weights so that the previous
value is predominant to maintain the stability and
continuity.
J. Curved Road detection
We can create important information for a self-
driving automobile using polynomials, such as the
curvature of the road.
Here y is the function having the second-degree
equation of the lane line. Solving the differential
gives the value of radius of curvature. We generate
warning signals if the radius of curvature of the
lane crosses a set limit.
K. Vehicle Distance from Center
We've calculated the locations of the left and
right lanes assuming the camera is about in the
middle of the front glass. As a result, we could
simply compare the bottom positions of the left
and right lanes to the center of the picture frame.
We might also use prior information of the
distance between the left and right lanes (for
example, 3.7 m in the United States) to determine
the distance in actual size.
IV. RESULTS
According to previous section, we analyze the
pipeline in different conditions and roads and
extend the implementation using video processing.
Low lighting, extreme curves and uneven roads
presented a difficulty in processing but still we
have a fair detection due to the methods used to
detect lanes in such harsh conditions. We have
good results on straight road images because we
specialize the mask for each left and right lane in
the road, convert the color space to HSL, and use
an additional erosion mask. We also have good
results on curved roads because we specialize the
mask for each left and right lane in the road,
convert the color space to HSL, and use the
additional erosion mask. The output of our code
can be seen below which masks the lane area and
issues necessary warnings if the vehicle is too far
from center or when it encounters a curved road.
Fig-Output.
Fig- Position Caution and Curve warning.
V. DISCUSSION & FUTURE WORK
Driver assistance systems must have extremely
low mistake rates in order to be helpful. The false
alarm rate for a warning system like LDW should
be very low, as excessive rates upset drivers and
lead to system rejection. The exact number of false
alerts that drivers are willing to accept is still under
investigation [1]. In addition, there are a variety of
conditions that road and lane detection must deal
with, and the look of the lane and road might alter
considerably, as shown in the accompanying
diagram.
The current solution is solely based on a
monocular camera and computer vision
techniques, and it includes a hard-coded
perspective transform as well as several manually
set lane-line filtering parameters. This may be
sufficient for good-condition road driving, such as
highway driving during the day with obvious lane
markings, but it may fail in a variety of difficult
conditions. We could choose one of two ways in
the future:
- When possible, use more sensing
modalities than just the monocular camera,
such as stereo cameras, lidar, and radar.
- Machine learning techniques should be
used.
REFERENCES
[1] . López, A., Serrat, J., Canero, C., Lumbreras, F., Graf, T.: Robust
lane markings detection and road geometry computation. Int. J.
Autom. Technol. 11(3), 395---407 (2010)
[2] . Sivaraman, S., Trivedi, M.M.: Integrated lane and vehicle detection,
localization, and tracking: a synergistic approach. Intell. Transp. Syst.
IEEE Trans. 14(2), 906---917 (2013)
[3] . Tapia-Espinoza, R., Torres-Torriti, M.: A comparison of gradient
versus color and texture analysis for lane detection and tracking. In:
Robotics symposium (LARS), 2009 6th Latin American, IEEE, pp.
1---6 (2009)
[4] . Borkar, A., Hayes, M., Smith, M.T.: A novel lane detection system
with efficient ground truth generation. Intell. Transp. Syst. IEEE
Trans. 13(1), 365---374 (2012)
[5] McCall, J., Trivedi, M.: Video-based lane estimation and tracking for
driver assistance: survey, system, and evaluation. IEEE Trans. Intell.
Transp. Syst. 7, 20–37 (2006)
[6] Labayrade, R., Douret, J., Laneurit, J., Chapuis, R.: A reliable and
robust lane detection system based on the parallel use of three
algorithms for driving safety assistance. IEICE Trans. Inf. Syst. E
89D, 2092–2100 (2006)
[7] Cheng, H., Jeng, B., Tseng, P., Fan, K.: Lane detection with moving
vehicles in the traffic scenes. IEEE Trans. Intell. Transp. Syst. 7, 571–
582 (2006)
[8] Batavia, P.H.: Driver-adaptive lane departure warning systems. CMU-
RI-TR-99-25 (1999)
[9] Hofmann, U., Rieder, A., Dickmanns, E.: Radar and vision data
fusion for hybrid adaptive cruise control on highways. Mach. Vis.
Appl. 14(1), 42–49 (2003)
[10] Wu, S., Chiang, H., Perng, J., Chen, C., Wu, B., Lee, T.: The
heterogeneous systems integration design and implementation for lane
keeping on a vehicle. IEEE Trans. Intell. Transp. Syst. 9, 246– 263
(2008)
[11] Gao, T., Aghajan, H.: Self lane assignment using egocentric smart
mobile camera for intelligent GPS navigation. In: Workshop on
Egocentric Vision, pp. 57–62 (2009)
[12] Jiang, Y., Gao, F., Xu, G.: Computer vision-based multiple-lane
detection on straight road and in a curve. In: Image Analysis and
Signal Processing, pp. 114–117 (2010).
[13] Huang, A.S., Moore, D., Antone, M., Olson, E., Teller, S.: Finding
multiple lanes in urban road networks with vision and LIDAR. Auton.
Robots 26, 103–122 (2009)
[14] Lipski, C., Scholz, B., Berger, K., Linz, C., Stich, T., Magnor, M.: A
fast and robust approach to lane marking detection and lane tracking.
In: Southwest Symposium on Image Analysis and Interpretation, pp.
57–60 (2008)
[15] . Kornhauser, A.L., et al.: DARPA Urban Challenge Princeton
University Technical Paper (2007). http://www.stanford.edu/~jmayer/
papers/darpa07.pdf
[16] Urmson C., et al.: Autonomous driving in urban environments: boss
and the urban challenge. J. Field Robot. 25(8), 425–466 (2008)
[17] (2008) 23. Bacha, A. et al.: Odin: team victor Tango entry in the
DARPA urban challenge. J. Field Robot. 25(8), 467–492 (2008)
[18]  Rasmussen, C., Korah, T.: On-vehicle and aerial texture analysis for
vision-based desert road following. In: CVPR Workshop on Machine
Vision for Intelligent Vehicles, vol. III, p. 66 (2005)
[19] Kong, H., Audibert, J., Ponce, J.: Vanishing point detection for road
detection. In: IEEE Conference on Computer Vision and Pattern
Recognition, pp. 96–103 (2009)
[20] Broggi, A., Cattani, S.: An agent based evolutionary approach to path
detection for off-road vehicle guidance. Pattern Recognit. Lett. 27,
1164–1173 (2006)
[21]  Alon, Y., Ferencz, A., Shashua, A.: Off-road path following using
region classification and geometric projection constraints. In: IEEE
Conference on Computer Vision and Pattern Recognition, vol. I, pp.
689–696 (2006)
[22] Nefian, A., Bradski, G.: Detection of drivable corridors for off-road
autonomous navigation. In: International Conference on Image
Processing, pp. 3025–3028 (2006)
[23]  Katramados, I., Crumpler, S., Breckon, T.: Real-time traversable
surface detection by colour space fusion and temporal analysis. In:
Computer Vision Systems, pp. 265–274 (2009)
[24] Du, Xinxin, and Kok Tan. ”Vision- Based Approach towards Lane
Line Detection and Vehicle Localization.” Machine Vision and
Applications 27.2 (2016): 175-91.
[25]  Qin, H., Zain, J. M., Ma, X., & Hai, T. (2010, August). Scene
segmentation based on seeded region growing for foreground
detection. In 2010 Sixth International Conference on Natural
Computation (Vol. 7, pp. 3619-3623). IEEE.
[26] Adams, R., Bischof, L.: Seeded region growing. IEEE Transactions
on Pattern Analysis and Machine Intelligence 16(6), 641647.
[27] Rasmussen, C. (2004). Grouping dominant orientations for
illstructured road following. Proceedings of the IEEE Computer
Society Conference on Computer Vision and Pattern Recognition, 1,
I470-I477.
[28] Hui Kong, Audibert, and Ponce. (2010). General Road Detection
From a Single Image. Image Processing, IEEE Transactions on, 19(8),
2211- 2220
[29] Moghadam, P., Starzyk, J., and Wijesoma, W. (2012). Fast
VanishingPoint Detection in Unstructured Environments. Image
Processing, IEEE Transactions on, 21(1), 425-430
[30]  Moghadam, P., & Dong, J. F. (2012, October). Road direction
detection based on vanishing-point tracking. In 2012 IEEE/RSJ
International Conference on Intelligent Robots and Systems (pp.
1553-1560). IEEE
[31] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Statistical and
knowledge-based moving object detection in traffic scene,” in
Pmceedzngs of IEEE Int’l Conference on Intellzgent Transportatzon
Systems, Oct. 2000, pp 27-32.
[32] A. Elgammal, D. Harwood, and L.S. Davis, “Non-parametric model
for background subtraction,” in Proceedings of IEEE ICCVYY
FRAME-RATE Workshop, 1999. 
[33] Huang, A.S., Moore, D., Antone, M. et al. Finding multiple lanes in
urban road networks with vision and lidar. Auton Robot 26, 103–122
(2009). 
[34] http://www.roboticsproceedings.org/rss02/p05.pdf
[35] http://www.roboticsproceedings.org/rss02/p21.pdf
[36]