Rolling shutter compensation for vehicle to vehicle
positioning using CMOS sensor camera
Trong-Hop Do
School of Electronic Engineering
Soongsil University
Seoul, South Korea
Abstract—This paper presents a visible light communication
(VLC) based vehicle to vehicle (V2V) positioning system using
CMOS sensor camera. In this system, a camera is attached on
the estimating vehicle to capture the image of two LED tail lamps
of the target vehicle. The position of the target vehicle relative to
the estimating vehicle is obtained through geometric relationship
between the LEDs in the real word and the captured image. In
existing systems, the rolling shutter mechanism of CMOS sensor
when combining with the movement of vehicles would create
image artifacts which brings about positioning errors. In this
paper, a compensation method is proposed to eliminate the effect
of these rolling shutter artifacts to achieve a high positioning
accuracy even when vehicles are moving at high speeds.
Index Terms—VLC, vehicle, positioning, rolling shutter
I. I NTRODUCTION
Vehicle to vehicle positioning plays a vital role in an
intelligent transport system (ITS), which is an application
meant to improve the safety, mobility, and efficiency of trans-
portation. Regarding vehicle to vehicle positioning, there are
many techniques based on cellular network, global positioning
system (GPS), infrared, LIDAR, computer vision, and VLC
using camera can be used. Among these techniques, VLC
based V2V positioning have several advantages and thus has
gained a lot of attentions in recent years [1-3]. The first
advantage is the cost effectiveness of the VLC based system.
The two major components of a VLC system are camera
and LEDs. Nowadays, most cars are equipped with dashboard
cameras. LEDs have been also increasingly used for cars.
Therefore, the cost for the VLC system would be very low
considering the availability of cameras and LEDs in vehicles.
The second advantage is about accuracy. While the positioning
accuracy provided by VLC system is not comparable to that of
the expensive LIDAR system, it is still much better than what
infrared, GPS, or cellular network-based system can provide.
The third advantage is that VLC can fulfil not only positioning
but also many other functions including warning light, car
black box, and especially vehicle to vehicle communication,
which is another vital component of ITS behind positioning.
In a VLC based vehicle to vehicle positioning system, a
camera is attached on the interior windshield of the estimating
to capture images of two LED tail lamps of the target vehicle.
The captured image is then processed to determine the coordi-
nate of LEDs on the imaged. Given that the distance between
the two LEDs on the vehicle and other intrinsic parameters
Myungsik Yoo
School of Electronic Engineering
Soongsil University
Seoul, South Korea
of the cameras are known, the position of the target vehicle
relative to the estimating vehicle can be determined using
existing camera geometry equations. The problem is that the
rolling shutter mechanism of CMOS sensor, which is currently
the type of sensor used in almost all cameras, when combining
with the movement of vehicles would introduce image artifacts
which might create severe positioning errors. In this paper,
a compensation method is proposed to eliminate the errors
caused by the rolling shutter artifacts to provide high vehicle to
vehicle positioning accuracy even when the vehicle is moving
at high speeds.
II. S YSTEM ARCHITECTURE AND THE ROLLING SHUTTER
ARTIFACT
The system architecture of a VLC based V2V positioning
system is depicted in Fig. 1. The image of two LED tail lamps
of the target vehicle is captured by a camera and processed
by a CPU on the estimating vehicle to determine the relative
position between the two vehicles.
Fig. 1: VLC based V2V positioning system
The rolling shutter mechanism of CMOS sensor is explained
through Fig. 2. In CMOS sensor, the scene is captured and
read row by row in the sensor. The time required for the
whole sensor to be read is called the readout time. Because of
this mechanism, image artifact would appear if the camera is
moving while capturing the image. Assuming that the camera
is moving towards the LED tail lights, the LEDs would appear
in the captured image as if they are closer to the camera. After
the LEDs are captured in the image, the two vehicles are still
moving. Therefore, the position of vehicle determined through
the image with artifact would contain errors.
III. P ROPOSED POSITIONING ALGORITHM
The camera geometric of pinhole camera model [4] is used
to determine the position of the target vehicle. In this model,
 1m from each other. Both LED lights are round panels with the
diameter of 10cm. The camera in the estimating vehicle has
the lens focal length of 35mm, the sensor physical size of 36
× 24mm, the resolution of 3600 × 2400 pixels, the readout
time of 0.1 second, and the frame rate of 36fps. The LED
image is simulated using pinhole camera model. A gaussian
noise is applied to the simulated image to replicate the blur
effect. Then, the simulated image is processed to determine
the image coordinates of the two LEDs. Then the proposed
positioning algorithm is used to determine the position of the
target vehicle relative to the estimating vehicle.

Fig. 2: Rolling shutter mechanism and image artifact

(x1 , y1 ) and (x2 , y2 ) are the image coordinates of the two

LEDs, (X1 , Y1 , Z1 ) and (X2 , Y2 , Z2 ) are the unknown world
coordinates of the two LEDs, which can be considered the
position of the target vehicle. A set of collinearity equations
can be derived from the camera model:

R11 X1 +R12 Y1 −R13 Z1

 x1 = −f R 31 X1 +R32 Y1 −R33 Z1
 y = −f R21
 X1 +R22 Y1 −R23 Z1
1 R31 X1 +R32 Y1 −R33 Z1
R11 X2 +R12 Y2 −R13 Z2 , (1)

 x2 = −f R31 X2 +R32 Y2 −R33 Z2 Fig. 3: Rolling shutter mechanism and image artifact
 y = −f R21 X2 +R22 Y2 −R23 Z2

2 R31 X2 +R32 Y2 −R33 Z2
The simulation result is shown in Fig. 3. Without compen-
where f is the focal length of the lens and R is the camera sation, the positioning error greatly increases as the vehicle
rotation matrix. In Eq. (1), the lens focal length is assumed speed increases. With compensation, the positioning error is
to be known. Assuming that the pose of the camera can be maintained at almost the same value as the vehicle speed
obtained through inertial sensor, the camera rotation matrix increases.
can be obtained. The LED image coordinates (x1 , y1 ) and
x2 , y2 are obtained through image processing. Given that the V. C ONCLUSION
relative position between the two LEDs in the target vehicle This paper present a VLC based V2V positioning system
is known, the three variables X2 , Y2 , and Z2 in the third and using CMOS sensor camera. Because of the image artifact
fourth equations in Eq. (1) can be represented in terms of X1 , caused by the rolling shutter mechanism used in CMOS sensor,
Y1 , and Z1 . Therefore, Eq. (1) can be transformed into set of the positioning result suffers from high error when the vehicle
linear equations which can be solved using linear least square is moving at high speed. In this paper, a compensation method
method. is proposed to eliminate the error caused by the rolling shutter
To compensate for the rolling shutter artifact, the speeds image artifact and thus high positioning accuracy can be
of the two vehicles are obtained. The speed of the estimating obtained when the vehicle is moving at high speeds.
vehicle is obtained using speedometer. The speed of the target
vehicle can be obtained through a sequence of positioning ACKNOWLEDGMENT
over a short period of time. From the speed of vehicles and This research was supported by the Basic Science Research
the readout time of the sensor, the change in the position of Program through the National Research Foundation of Korea
the camera while taking the image of LEDs can be estimated. (NRF), funded by the Ministry of Education, Science and
The change in the position of camera is then translated into Technology (NRF-2018R1A2B6004371)
the change in the position of LEDs X1 , Y1 , and Z1 and thus
the rolling shutter artifact are eliminated. R EFERENCES
[1] Yoshino, M., Haruyama, S. and Nakagawa, M., 2008, January. High-
IV. S IMULATION accuracy positioning system using visible LED lights and image sensor.
In the simulation, the estimating and target vehicles are In Radio and Wireless Symposium, 2008 IEEE (pp. 439-442). IEEE.
[2] Roberts, R., Gopalakrishnan, P. and Rathi, S., 2010, December. Visible
assumed to move in the street with the speed ranging from 0 to light positioning: Automotive use case. In Vehicular Networking Con-
30m/s. Two LED tail lights in the target vehicles are placed at ference (VNC), 2010 IEEE (pp. 309-314). IEEE.
[3] Armstrong, J., Sekercioglu, Y. and Neild, A., 2013. Visible light posi-
tioning: a roadmap for international standardization. IEEE Communica-
tions Magazine, 51(12), pp.68-73.
[4] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision,
second ed. Cambridge University Press, Cambridge CB2 8RU, UK