2016 IEEE Intelligent Vehicles Symposium (IV)
Gothenburg, Sweden, June 19-22, 2016
Continuous Extrinsic Online Calibration for Stereo Cameras
Georg R. Mueller Hans-Joachim Wuensche
University of the Bundeswehr Munich
Abstract— Accurate stereo camera calibration is crucial for
3D reconstruction from stereo images. In this paper, we propose
an algorithm for continuous online recalibration of all extrinsic
parameters of a stereo camera, which is rigidly mounted on an
autonomous vehicle. The algorithm estimates the six degrees-
of-freedom (6-DoF) of the transformation from the vehicle
coordinate system to the coordinate system of the stereo camera
and at the same time the relative 6-DoF transformation between
the two camera sensors. Salient points in the environment that
are observed by both cameras are tracked over time in 3D space.
An Unscented Kalman Filter (UKF) is applied to recursively
estimate the extrinsic stereo camera calibration and the 3D
position of all observed points. The projections of the points
and the measured vehicle motion, which is estimated using
an inertial measurement unit (IMU), are given as input. The
observability of the stereo camera calibration states is analyzed
to identify critical vehicle motion sequences. Results with real
world data show that the algorithm is capable of continuously
estimating the stereo camera calibrations in spite of large initial
errors and varying extrinsic parameters.
I. I NTRODUCTION
Stereo vision provides colored, dense 3D information in
the overlapping field of view of two camera sensors. The ac-
curacy of the 3D reconstruction from stereo images depends
crucially on the intrinsic (camera internal) and extrinsic
parameters, which define the transformation between the two
camera sensors and between the stereo camera coordinate
system and a common robot coordinate system.
Conventional stereo camera calibration is typically carried
out offline by observing calibration patterns of a known
spatial geometry from different viewing angles [1]. Hereby
the relative transformation between the two camera sensors,
as well as camera intrinsics, can be calculated. Intrinsic
parameters, however, usually remain constant for cameras
with fixed focal lengths. Extrinsic camera parameters, in
contrast, can be easily influenced by temperature variations
or mechanical stress, which degrades the quality of 3D
reconstruction. This deviation is also called ’drift’, which
cannot be eliminated, thus more frequent maintenance is
required for offline recalibration.
In recent years, a lot of research has been conducted
on online stereo calibration algorithms, which aim at de-
termining the parameters without the need of specialized
calibration patterns during the operation of a robot. Many
algorithms focus only on estimating the 6-DoF relative
transformation between the cameras, without considering the
All authors are with the Institute for Autonomous Systems Technology
(TAS), University of the Bundeswehr Munich, 85577 Neubiberg, Germany,
Contact author email: {georg.mueller,jw}@unibw.de
978-1-5090-1821-5/16/$31.00 ©2016 IEEE
6-DoF transformation from a robot to the stereo camera
coordinate system.
Based on epipolar geometry the fundamental matrix is
calculable from point correspondences in the two images of a
stereo camera system. The fundamental matrix relates points
in one camera image to epipolar lines in the other image.
This relation is also called the epipolar constraint. Given
the intrinsic calibration of both cameras, the essential matrix
can be obtained. From this matrix the relative transformation
between the cameras can be estimated e.g. with a linear
method [2] or via singular value decomposition [3]. The al-
gorithm of Hansen et al. [4] calculates the relative orientation
between the sensors of a stereo camera system given a fixed
baseline by minimizing the error of the epipolar constraint.
The obtained estimates are then further refined by a Kalman
filter. In [5] the epipolar constraint is directly integrated into
an Implicit Kalman filter as a measurement to estimate the
relative transformation of two individually rotating cameras,
which are mounted on a humanoid robot head. It is shown
that the Kalman filter can robustly estimate the relative
transformation despite large initial errors. A major drawback
of the epipolar constraint is that the obtained translation
vector between the cameras can only be determined up to
an unknown scale factor.
Another well known method for estimating the relative
transformation between multiple camera images is called
bundle adjustment. Given an initial guess for the camera
parameters, bundle adjustment has proven to be able to
calculate camera calibration with high precision [6]. For a
multidimensional optimization problem, bundle adjustment
finds the set of camera parameters, minimizing the repro-
jection error between the measurements in each frame and
the predicted measurements of the estimated 3D position
of the observed points [7]. From point correspondences
in a set of images, a 3D reconstruction of the observed
scene is obtained. Several modifications and improvements
have been developed on bundle adjustment, exploiting the
sparsity of the bundle adjustment problem [8]. However,
estimating relative transformations between camera images
with bundle adjustment still remains time consuming as
these algorithms seek to refine camera calibration from
numerous camera images in a batch optimization process.
Based on monocular visual SLAM and bundle adjustment,
Carrera et al. [9] calibrate the relative transformation between
multiple cameras on a robot platform up to scale. Heng
et al. [10] extended this algorithm to deduce metric scale
from odometry data and additionally determine the extrinsic
calibration from a vehicle coordinate system to one of the
966
 ycaml
yveh
Hcam
veh
l
x
zcaml caml
xveh Hcamr ycamr
caml
zveh
xcamr
zcamr
Fig. 1: The extrinsic stereo camera calibration parameters can
be expressed by two homogeneous transformation matrices
Hcam camr
veh and Hcaml .
l
cameras. Both algorithms [9], [10] can calibrate cameras with
non-overlapping field of views, but are not performed online.
Dang et al. [6] introduce a real-time algorithm that con-
tinuously estimates the relative transformation between two
sensors of a stereo system, as well as the vehicle motion,
using an Implicit Iterated Extended Kalman Filter (IEKF).
Three different geometric constraints are suggested as obser-
vation models for the IEKF. These are derived from bundle
adjustment, trifocal and epipolar constraints. Dang et al.
conclude that the constraint based on bundle adjustment
itself delivers the most accurate results. However, bundle
adjustment is prone to inaccuracies in dynamic environments.
In these situations the epipolar constraint stabilizes the esti-
mation process as no assumption on the observed 3D scene
is needed. The algorithm, however, does not calibrate the
significant transformation from a vehicle to the coordinate
system of the stereo camera due to the lack of precise IMU
measurements. This transformation is crucial for autonomous
vehicles, e.g. to enable perception for obstacle avoidance.
In this paper, we contribute an algorithm for continuous
extrinsic online calibration of a stereo camera. The proposed
algorithm estimates both, the 6-DoF relative transformation
between two camera sensors, and the 6-DoF transformation
from a vehicle coordinate system (’veh’) to the stereo camera
coordinate system, which we define to be fixed to the coor-
dinate system of the left camera (’caml ’). These calibration
parameters can be expressed by two homogeneous trans-
formation matrices (HTM), Hcam camr
veh and Hcaml as visualized
l
in Fig. 1.
This paper is structured as follows: In Section II the
algorithm is presented in detail and an observability analysis
of the stereo camera calibration states is performed. In the
following Section III the algorithm’s performance is evalu-
ated on real-world data and compared to offline calibrated
results. Finally we draw a conclusion and give insight into
further work in Section IV.
II. A LGORITHM FOR C ONTINUOUS E XTRINSIC S TEREO
C AMERA O NLINE C ALIBRATION
An Unscented Kalman Filter (UKF) is applied to re-
cursively estimate the extrinsic stereo camera calibration
parameters and to keep track of changes. The state vector
967
of the filter basically consists of 12 parameters which can be
separated into xcaml and xcamr :
>
xcaml = [xcaml , ycaml , zcaml , Φcaml , Θcaml , Ψcaml ] (1)
>
xcamr = [xcamr , ycamr , zcamr , Φcamr , Θcamr , Ψcamr ] (2)
The states in eq. (1) describe the transformation from the
vehicle coordinate system to the stereo camera coordinate
system, which is defined to be identical with the coordinate
system of the left camera. The further states in eq. (2) specify
the relative transformation between the left and right camera
coordinate system.
The states are described by the cartesian coordinates x, y,
z and the angles roll Φ, pitch Θ and yaw Ψ. The algorithm re-
quires both cameras to be calibrated intrinsically beforehand.
We assume that the intrinsic calibration remains constant
over time, which is viable for cameras with fixed focal length
lenses. An initial guess of the extrinsic parameters can be
obtained from CAD-drawings or manual measurements and
is provided to the UKF as an initial value.
The UKF estimates the extrinsic stereo camera calibration
by tracking points in 3D space which can be observed in
both cameras while the vehicle moves. Each observed point
is described w.r.t. the left camera coordinate system in 3D
cartesian coordinates:
>
xp = [xp , yp , zp ] (3)
Since all observed points are assumed to be static, their
position is predictable with the vehicle’s inverse motion,
which is estimated using an IMU. The 3D position of these
points, however, can only be determined based on the current
stereo camera calibration. Similar to a bundle adjustment
approach, the UKF seeks to estimate an optimal stereo
camera calibration, as well as each points’ 3D position to
minimize the reprojection error. The point states are added
to the filter’s state vector:
x = x> > > >
 
caml , xcamr , xp1 , . . . , xpN (4)
The state vector thus increases or decreases dynamically
depending on the amount of points observed.
A. Initialization
New points are initialized for tracking from correspon-
dences between the left and right camera. These features
can be obtained from any feature detection algorithm e.g.
SIFT [11]. Brute force matching of all detected features in
the left and right camera images can result in ambiguous
invalid matches. These can significantly impact the esti-
mation process. Invalid matches thus have to be detected
and removed, which is usually done via fundamental matrix
estimation from correspondences and RANSAC filtering.
This can be a very time consuming task. Therefore a different
method for matching is applied by using the current estimate
of the stereo camera calibration. Based on the relative cali-
bration estimate between the camera sensors, a fundamental
matrix F can be calculated:
F =K−>
r · [t]x R · K−1
l , (5)
whereas t is the translation vector and R the rotation matrix P caml
which can be calculated from xcamr . The expression [t]x is the
skew-symmetric representation of the translation vector. Kl
Time step
and Kr are the camera matrices of left and right camera,
which are obtained from the intrinsic calibration of both
cameras.
For any detected feature in the left camera image k+1
>
xl = [ul , vl ] a corresponding epipolar line lr in the right
Hcam l Hcam
caml
r
camera can be determined by: veh

lr =F · xl , (6)
whereas homogeneous coordinates are denoted by (·). Just
those features in the right camera that lie on or are at the
proximity of the corresponding epipolar line are regarded
as possible matches for the given feature in the left camera.
This distance criterion can be adjusted dynamically based on
the filter’s error covariance. Only this reduced list of features
are considered as possible matches and compared with the
given feature in the left camera. The best matching feature
is only accepted as a valid match if the descriptor distance
is below a certain threshold, which depends on the applied
feature extraction algorithm. If the match is valid, the feature
from the right camera will not be considered for matching
anymore to avoid multiple matches to a single feature.
Correspondences are often detected at salient points in
local image regions. This can bias the optimization and thus
limit the performance of the calibration process. Therefore
the obtained matches are sampled in the camera image to
ensure equal distribution. Matches in similar image regions
in the left and right camera image are clustered together.
Just the best matching correspondence from each cluster is
retained for tracking, assuming that this is the most reliable
match. This sampling step also reduces computational costs
as less features with consistent information have to be
tracked and filtered.
For all remaining correspondences initial 3D positions
w.r.t. the left camera are calculated by stereo triangulation.
The 3D position of each detected correspondence is deter-
mined from the intersection of two rays which are obtained
from back-projection of the matched feature in each camera.
Especially at the beginning of the estimation process the
stereo camera calibration is only a rough estimate. Thus rays
from both cameras will not intersect in 3D space. In this
situation a 3D point is calculated that has minimal distance
between both rays. Should this point lie behind the left
camera, the detected correspondence is considered invalid
and will be neglected.
For optimal initialization of new points, a Scaled Un-
scented Transform (SUT) [12] is applied. It projects the
12 × 12 state covariance matrix of the current stereo camera
calibration estimate and the estimated 2 × 2 measurement
noise of each camera into 3 × 3 euclidean space of the left
camera coordinate system based on the stereo triangulation
measurement model. From the weighted sum of the sigma
points, the initial 3D state as well as a state covariance can
be obtained. If the estimated 3D position is ahead of the
vehicle, the point’s 3D state and covariance is added to the
k
Hveh
caml


Hveh
veh k+1|k
Fig. 2: Overview of the continuous extrinsic stereo camera
calibration algorithm. The vehicle is shown in two time steps
k and k + 1. Red indicates parameters that are estimated by
the UKF in each time step. Green lines visualize the filter’s
measurements in the left and right camera. The filter predicts
the 3D position of all observed points into time step k + 1
based on the measured vehicle’s motion, displayed in orange.
filter. Moreover, the matched descriptors from both cameras
are stored to enable tracking of the point in further frames.
B. Prediction Step
In the filter’s prediction step, the states of the stereo
camera calibration and of all observed points are predicted
from time step k to the next time step k + 1 based on the
vehicle motion.
1) Prediction of stereo camera calibration states xcaml and
xcamr : It is assumed that the stereo cameras are rigidly fixed
to the vehicle and that the stereo camera calibration is only
affected by drift. The stereo camera calibration states (xcaml
and xcamr ) therefore remain constant during prediction.
2) Prediction of tracked points xp : Besides the stereo
camera calibration, all observed points are also assumed to be
static in the scene. As their states are described w.r.t. the left
camera coordinate system, they have to be predicted based on
the measured vehicle’s motion. This motion from time step k
to k + 1 is described in the vehicle

coordinate system (’veh’)
and expressed as a HTM Hveh veh k+1|k . The states of all points
are first transformed into the vehicle coordinate system to
incorporate vehicle motion.
Each state is described in homo-
geneous coordinates P caml k to apply the transformation as
shown in eq. (7). The current estimated transformation be-
tween the left camera and the vehicle

coordinate system can
be expressed by a HTM Hveh caml k . In time step k+1 the point
is transformed into the the left camera coordinate system
l?
again with the predicted left camera calibration Hcam


veh k+1
.

968
Since the camera calibration is assumed to be constant, the δy1 P caml
δx1
predicted camera calibration in timestep
l?

k + 1 is equal to
the calibration in time step k: Hcam veh k+1
= (Hcaml
veh )k . The
δx2
complete formula for prediction of each observed point is
shown in eq. (8) and visualized in Fig. 2. δy2
> δΨ
  h i
= x>


P caml p k ,1 (7) δΨ
  k
? l?
= Hcam · Hveh



P caml veh k+1 veh k+1|k
k+1
xcaml
· Hveh



caml k · P caml k (8) ycaml Ψ

3) Process Noise: Process noise is added in the prediction zcaml

step to account for inaccuracies in the prediction. It is
assumed that the stereo camera calibration states, as well Fig. 3: Errors in the vehicle motion measurement δΨ falsi-
as each observed point state, are independently affected by fying the prediction of points in xl and yl .
white Gaussian noise. (Q)k is thus a diagonal matrix:

(Qcaml ,camr )k 0 ... 0
 angular measurement noise of the vehicle motion is equally
 0 (Qp,1 )k 0 0  distributed in all three directions:
(Q)k =   (9)
 
.. .. . . .. σIMU,Φ = σIMU,Θ = σIMU,Ψ = 0.02◦ (15)
 . . . . 
0 0 0 (Qp,N )k This noise covariance can easily be transformed into the
stereo camera coordinate system independently of the current
The discrete standard deviations (σ. . . ,. . . )k are based on a
stereo camera calibration state. In Fig. 3 the angular errors for
sampling rate of 10 Hz. The index k is omitted for better
a planar scene with yaw measurement errors are illustrated.
readability.
The figure visualizes that Ψ (yaw) angular measurement
For the stereo camera calibration parameters the corre-
errors in the camera coordinate system impact the prediction
sponding 12 × 12 noise covariance matrix (Qcaml ,camr )k is
of an observed point in x- and y-direction, but have no
assumed with minimum variances, based on the standard
influence in the z-direction. In eq. (16)–(24) formulas for
deviations given in eq. (10)–(13). These account for drift
calculation of the process noise covariance elements in each
in the parameters between each time step.
axis for different angular errors are shown.
σx,caml = σy,caml = σz,caml = 0.05 mm (10) zp q
Φp = arctan( ) ryz = yp2 + zp2 (16)
◦ yp
σΦ,caml = σΘ,caml = σΨ,caml = 0.005 (11)
zp q
σx,camr = σy,camr = σz,camr = 0.01 mm (12) Θp = arctan( ) rxz = x2p + zp2 (17)
◦ xp
σΦ,camr = σΘ,camr = σΨ,camr = 0.0005 (13) xp q
Ψp = arctan( ) rxy = x2p + yp2 (18)
For each tracked point, an individual 3 × 3 process noise yp
covariance matrix (Qp,n )k is determined. This is done by sin (Φp + σIMU,Φ ) − sin (Φp − σIMU,Φ )
σΦ,z = ryz (19)
considering two different errors: The first accounts for inac- 2
curacies in the 3D state estimation due to incorrect stereo cos (Φp + σIMU,Φ ) − cos (Φp − σIMU,Φ )
σΦ,y = ryz (20)
camera calibration. In each direction equal Gaussian noise is 2
assumed: sin (Θp + σIMU,Θ ) − sin (Θp − σIMU,Θ )
σΘ,z = rxz (21)
2
σp,x = σp,y = σp,z = 0.2 m (14) cos (Θp + σIMU,Θ ) − cos (Θp − σIMU,Θ )
σΘ,x = rxz (22)
This step adds significant noise to the error covariance 2
of each observed point. The UKF therefore rather optimizes sin (Ψp + σIMU,Ψ ) − sin (Ψp − σIMU,Ψ )
σΨ,x = rxy (23)
the 3D positions of points than the stereo camera calibra- 2
tion, which enforces smoothness in estimation of the stereo cos (Ψp + σIMU,Ψ ) − cos (Ψp − σIMU,Ψ )
σΨ,y = rxy (24)
camera calibration parameters. 2
The second error, distorting the prediction of points occurs The process noise covariances for a filtered point from
due to inaccuracies in the measurement of the vehicle’s both error models are combined to a diagonal 3 × 3 noise
motion. The motion observation is affected by angular and covariance matrix (Qp,n )k with the following diagonal ele-
translational errors. Translational errors are often within a ments:
few millimeters. These are not considered in this paper since q11 = σΘ,x 2 + σΨ,x 2 + σp,x 2 (25)
they have only minimum influence on the prediction step. In 2 2 2
q22 = σΦ,y + σΨ,y + σp,y (26)
contrast, angular observation errors can have a significant
impact on remote points. In this paper, it is assumed that the q33 = σΦ,z 2 + σΘ,z 2 + σp,z 2 (27)

969
 The individual process noise covariance matrices are then
integrated into the (12 + 3N ) × (12 + 3N ) matrix (Q)k . The
complete a priori error covariance can then be calculated by
the UKF as following:
Ψ
X2N h ih i>
? (i) ?(i) ? ?(i) ?
(P )k+1 = w x −x x −x +(Q)k (28)
i=0
C. Correction Step
The estimated stereo camera calibration as well as the 3D
positions of all observed points are optimized in the filter’s
correction step. Measurements of tracked points are predicted Fig. 4: The stereo camera system mounted on a rotatable
into the platform inside MuCAR-4.
 left ?and
 right camera image based on the predicted
state P caml and compared with real measurements.
k+1
In contrast to the filter’s prediction step, measurements III. E VALUATION
can be predicted independently of the current left camera The algorithm has been evaluated on an autonomous
calibration estimate, as each point is already described w.r.t. vehicle platform called MuCAR-4, the Munich Cognitive
the left camera. The state of each predicted point is projected Autonomous Robot Car 4th Generation. This vehicle is a
into the left camera sensor by the intrinsic camera calibration Volkswagen Tiguan which is equipped with a high end IMU
matrix Kcaml , as shown in eq. (29). for measuring the vehicle motion and a stereo vision system,
For
predicting measurements in the right camera comprising of two color cameras with similar focal lengths,
pcamr ? k+1 , the state of the predicted point in the right mounted behind the windshield on a rotatable platform as
camera coordinate system is needed. This can be obtained shown in Fig. 4. Both cameras are hardware-triggered with
by transforming the predicted point from the left to
the right a frequency of 10 Hz and are calibrated offline beforehand
r?
camera coordinate system via the HTM Hcam caml k+1 . This as reference. The algorithm runs in real-time on a Dual
HTM can be calculated from (xcamr ? )k+1 . The measurement Intel Xeon E5-2667 processor clocked at 2.9 GHz on-board
is then predicted with eq. (30). MuCAR-4.
 ?
 The proposed algorithm is initialized with inaccurate start
pcaml ? k+1 = Kcaml · P caml


(29) values of the stereo calibration extrinsics. For the extrinsics
k+1
in xcaml , an initial error of 0.5 m in all directions and 10◦
 ?

r?
pcamr ? k+1 = Kcamr · Hcam



caml k+1 · P caml (30)
k+1 around all axes is given. For the relative transformation a
Similar to [13], we perform local feature matching only major base length error in yr -direction of 0.1 m is added.
at the proximity of these predicted image positions. This im- While running the algorithm, the vehicle drives off-road
proves matching reliability and reduces computational costs. over a meadow. The observed environment is static. Fig. 5
A measurement is considered valid, if a correspondence and Fig. 6 visualize the performance of the algorithm for es-
has been detected in both camera images. For these valid timating the stereo camera calibration. It shows that the filter
matches the points’ stored feature descriptors are updated can cope with large initial errors. The parameter zl converges
with the recently detected descriptors. If matching fails, the last. As noted in Section II-D, substantial angular rotations
observed point is deleted from the system state and the are required around two axes to observe this parameter.
corresponding covariance elements are removed from the In frame 774 the stereo camera platform is rotated around
filter’s state covariance. the Ψ axis by 10◦ . The filter is capable of detecting this rapid
change in the stereo camera calibration parameters and con-
D. Observability verges to a valid calibration. In Table I the root-mean-square
For recursive state estimation it is required that all states error (RMSE) between the online estimated calibration and
are observable. Therefore the system equations were tested the offline determined calibration is shown. The RMSE is
with the Kalman criterion for observability [14]. This re- evaluated after the algorithm converged, starting from frame
vealed that tracking three individual points in two motion 1000 to 1150. The table also visualizes the filter’s final state
sequences is already sufficient to observe all extrinsic stereo error 1σ at frame 1150.
camera calibration states. For observability of all states it is
required that the vehicle performs at least a linear motion IV. C ONCLUSION
with superimposed rotations around two axis of the vehicle In this paper an algorithm has been presented which
coordinate system. These rotations are required as e.g. for continuously estimates the 12-DoF extrinsic calibration of a
planar motion the height of the stereo system zl cannot be stereo camera by observing static points in the environment.
observed without further assumptions. Providing independent It has been shown that the algorithm can cope with large
angular rotations in two different dimensions in consecutive initial errors. For observability of all calibration parameters,
frames is a difficult task in urban scenarios, but it can be substantial angular rotation around two axis between consec-
achieved in off-road scenarios. utive frames is required. This occurs only seldom in on-road

970
 1.0 1.4
1.4 State RMSE 1σ State RMSE 1σ
0.8 1.2 1.2 xl 0.0608 m 0.0185 m xr 0.0044 m 0.0020 m
0.6 1.0 1.0 yl 0.0382 m 0.0206 m yr 0.0048 m 0.0013 m
0.4 0.8 0.8 zl 0.0132 m 0.0223 m zr 0.0037 m 0.0011 m
Φl 0.2995◦ 0.0025◦ Φr 0.0236◦ 0.0100◦
xl in meter

yl in meter

zl in meter
0.2 0.6 0.6
Θl 0.7061◦ 0.0011◦ Θr 0.0174◦ 0.0067◦
0 0.4
0.4
Ψl 0.2844◦ 0.0011◦ Ψr 0.0193◦ 0.0076◦
-0.2 0.2 0.2
(a) Estimation of the extrinsic cali- (b) Estimation of the extrinsic cali-
-0.4 0 0
bration from ’veh’ to ’caml ’ (xcaml ). bration ’caml ’ to ’camr ’ (xcamr ).
-0.6 -0.2 -0.2
200 400 600 800 1000
Frame
200 400 600 800 1000
Frame
200 400 600 800 1000
Frame
TABLE I: Estimation of the extrinsic stereo camera cali-
bration. The RMSE between online and offline determined
calibration is shown (from frame 1000 to frame 1150). In
10 14 12
addition the filter’s state error 1σ in frame 1150 is displayed.
8 12 10

6 8
scenarios. For these situations we plan to integrate a further
10
measurement model, assuming a planar scene directly ahead
Ψl in degree
Θl in degree
Φl in degree

4 8 6
of the vehicle. From this the height of the stereo camera
2 6 4
system can be continuously determined without the need of
0 4 2
roll or pitch motion.
-2 2 0
In a further improvement the filter’s update step will
200 400 600 800 1000
Frame
200 400 600 800 1000
Frame
200 400 600 800 1000
Frame be modified to freeze certain system states, that cannot be
observed well, due to the vehicle’s motion. These critical
Fig. 5: Estimation of the extrinsic calibration from ’veh’ to motion sequences can be determined from evaluation of a
’caml ’ (xcaml ). The filter’s calibration estimate is visualized Fisher information matrix. This can smoothen the estimation
in black. The corresponding 1σ error region is shown in gray. of the stereo camera calibration and improve the precision.
The yellow line shows the offline determined calibration.
The vertical red line indicates the frame 774, in which the R EFERENCES
platform of the stereo camera system is rotated. [1] Z. Zhang, “A flexible new technique for camera calibration,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 22,
no. 11, pp. 1330–1334, 2000.
[2] H. Longuet-Higgins, “A computer algorithm for reconstructing a scene
0.06
0.10 from two projections,” Nature, vol. 293, pp. 133–135, 1981.
0.04 -0.06 [3] R. I. Hartley, “An investigation of the essential matrix,” GE CRD,
0.08
-0.08 Schenectady, NY, Tech. Rep, 1995.
0.02
0.06 [4] P. Hansen, H. Alismail, P. Rander, and B. Browning, “Online continu-
xr in meter

yr in meter

zr in meter

-0.10
0.0
0.04
ous stereo extrinsic parameter estimation,” in Conference on Computer
-0.02 -0.12 Vision and Pattern Recognition (CVPR). IEEE, 2012.
0.02 [5] N. Moutinho, R. Ferreira, J. Gaspar, A. Bernardino, and J. Santos-
-0.14
-0.04
0
Victor, “Markerless online stereo calibration for a humanoid robot,”
-0.06 -0.16 in International Conferences on Development and Learning and
-0.02 Epigenetic Robotics (ICDL-Epirob). IEEE, 2014, pp. 454–460.
-0.08 -0.18
[6] T. Dang, C. Hoffmann, and C. Stiller, “Continuous stereo self-
200 400 600 800 1000 200 400 600 800 1000 200 400 600 800 1000
Frame Frame Frame calibration by camera parameter tracking,” IEEE Transactions on
Image Processing, vol. 18, no. 7, pp. 1536–1550, 2009.
[7] B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon,
1.2
“Bundle adjustment—a modern synthesis,” in Vision Algorithms: The-
1.0 -1.8 ory and Practice. Springer, 2000, pp. 298–372.
1.0 [8] K. Konolige, “Sparse sparse bundle adjustment,” in Proceedings of the
0.8 -2.0 British Machine Vision Conference. BMVA Press, 2010, pp. 102.1–
0.8 102.11.
0.6 -2.2
[9] G. Carrera, A. Angeli, and A. J. Davison, “Slam-based automatic ex-
Ψr in degree
Θr in degree
Φr in degree

0.4
0.6 trinsic calibration of a multi-camera rig,” in International Conference
-2.4
on Robotics and Automation (ICRA). IEEE, 2011.
0.4
0.2 -2.6
[10] L. Heng, B. Li, and M. Pollefeys, “Camodocal: Automatic intrinsic
0.2
and extrinsic calibration of a rig with multiple generic cameras and
0 -2.8 odometry,” in International Conference on Intelligent Robots and
200 400 600 800 1000 200 400 600 800 1000 200 400 600 800 1000 Systems (IROS). IEEE, 2013.
Frame Frame Frame [11] D. G. Lowe, “Object recognition from local scale-invariant features,”
in The Proceedings of the seventh International Conference on Com-
Fig. 6: Estimation of the extrinsic calibration from ’caml ’ to puter Vision, vol. 2. IEEE, 1999, pp. 1150–1157.
’camr ’ (xcamr ). The filter’s calibration estimate is visualized [12] S. J. Julier, “The scaled unscented transformation,” in American
in black. The corresponding 1σ error region is shown in gray. Control Conference. IEEE, 2002.
[13] S. Schneider, G. R. Mueller, J. Kallwies, and H.-J. Wuensche, “RAS:
The yellow line shows the offline determined calibration. Recursive Automotive Stereo,” in Intelligent Vehicles Symposium.
The vertical red line indicates the frame 774, in which the IEEE, 2014, pp. 1282–1287.
platform of the stereo camera system is rotated. [14] R. Kalman, “On the general theory of control systems,” IRE Transac-
tions on Automatic Control, vol. 4, no. 3, pp. 110–110, 1959.

971