Camera Calibration

(Trucco, Chapter 6)

• What is the goal of camera calibration?

- To produce an estimate of the extrinsic and intrinsic camera parameters.

• Procedure

- Given the correspondences between a set of point features in the world (X w , Y w , Z w )

and their projections in an image (x im , y im ), compute the intrinsic and extrinsic cam-
era parameters.
pixel frame P
x im
yim

Yc
Xc y
Camera Yw
Frame optical axis x Xw
Zc ox ,oy World
Frame
center of Zw
perspective projection image principal point
plane
frame

• Establishing the correspondences

- Calibration methods rely on one or more images of a calibration pattern:

(1) a 3D object of known geometry.

(2) it is located in a known position in space.

(3) it is generating image features which can be located accurately.

 -2-

- Consider the above calibration pattern:

* it consists of two orthogonal grids.

* equally spaced black squares drawn on white, perpendicular planes.

* assume that the world reference frame is centered at the lower left corner of the
left grid, with axes parallel to the three directions identified by the calibration
pattern.

* given the size of the planes, their angle, the number of squares etc. (all known
by construction), the coordinates of each vertex can be computed in the world
reference frame using trigonometry.

* the projection of the vertices on the image can be found by intersecting the
edge lines of the corresponding square sides (or through corner detection).
 -3-

• Methods

(1) Direct parameter calibration.

Direct recovery of the intrinsic and extrinsic camera parameters.

(2) Camera parameters through the projection matrix

 m 11 m 12 m 13 m 14 
M = M in M ex =  m 21 m 22 m 23 m 24 
 
 m 31 m 32 m 33 m 34 

(2.1) Estimate the elements of the projection matrix.

(2.2) Compute the intrinsic/extrinsic as closed-form functions of the entries of

the projection matrix.
 -4-

Method 1: Direct Parameter Calibration

- We assume that the world reference frame is known (e.g., the origin is the middle
lower corner of the calibration pattern).

• Review of basic equations

- From world coordinates to camera coordinates (note that we have changed the order
of rotation/translation):

P c = R(P w − T ) or P c = RP w − RT or P c = RP w − T ′

- In the rest of this discussion, I will replace T ′ with T :

 X c   r 11 r 12 r 13   X w   T x 
Y  = r r 22 r 23   Y w  +  T y 
 c   21    
 Z c   r 31 r 32 r 33   Z w   T z 

- From camera coordinates to pixel coordinates:

f Xc
x im = − x/s x + o x = − + ox
s x Zc
f Yc
y im = − y/s y + o y = − + oy
s y Zc

- Relating world coordinates to pixel coordinates:

r 11 X w + r 12Y w + r 13 Z w + T x
x im − o x = − f /s x
r 31 X w + r 32Y w + r 33 Z w + T z
r 21 X w + r 22Y w + r 23 Z w + T y
y im − o y = − f /s y
r 31 X w + r 32Y w + r 33 Z w + T z
 -5-

• Independent intrinsic parameters

- The five intrinsic parameters f , s x , s y , o x , o y are not independent.

- We can define the following four independent parameters:

f x = f /s x , the focal length in horizontal pixels

= s y /s x (or = f x / f y ), aspect ratio
(o x , o y ), image center coordinates

• Method 1: main steps

(1) Assuming that o x and o y are known, estimate all the remaining parameters.

(2) Estimate o x and o y

• Step 1: estimate f x , , R, and T

- To simplify notation, consider (x im − o x , y im − o y ) = (x, y)

r 11 X w + r 12Y w + r 13 Z w + T x
x = − fx
r 31 X w + r 32Y w + r 33 Z w + T z
r 21 X w + r 22Y w + r 23 Z w + T y
y = − fy
r 31 X w + r 32Y w + r 33 Z w + T z

- Using the fact that the above two equations have the same denominator, we get the
following equation:

x f y (r 21 X w + r 22Y w + r 23 Z w + T y ) = y f x (r 11 X w + r 12Y w + r 13 Z w + T x )

Problem Statement
Assuming that o x and o y are known, compute f x , , R, and T from N correspond-
ing pairs of points (X iw , Y iw , Z iw ), (x i , y i ), i = 1, . . . , N .
 -6-

Derive a system of equations

- Each pair of corresponding points leads to an equation:

x i f y (r 21 X iw + r 22Y iw + r 23 Z iw + T y ) = y i f x (r 11 X iw + r 12Y iw + r 13 Z iw + T x )

- Rewrite the above equation as follows (i.e., divide by f y ):

x i X iw v 1 + x i Y iw v 2 + x i Z iw v 3 + x i v 4 − y i X iw v 5 − y i Y iw v 6 − y i Z iw v 7 − y i v 8 = 0

where
v1 = r 21 v5 =
r 11
v2 = r 22 v6 =
r 12
v3 = r 23 v7 =
r 13
v4 = Ty v8 =
Tx

- N corresponding points lead to a homogeneous system of N equations with 8

unknowns:
Av = 0 where:

 x1 X1
w
x 1Y 1w x 1 Z 1w x1 −y 1 X 1w −y 1Y 1w −y 1 Z 1w −y 1 
 x 2 X 2w x 2Y 2w x 2 Z 2w x2 −y 2 X 2w −y 2Y 2w −y 2 Z 2w −y 2 
A= 
 ... ... ... ... ... ... ... ... 
 x N X Nw x N Y Nw x N Z Nw xN −y N X Nw −y N Y Nw −y N Z Nw −y N 

Solving the system

- It can be shown that if N ≥ 7, then A has rank 7.

- If A = UDV T , we have discussed in class that the system has a nontrivial solu-
tion v which is proportional to the column of V corresponding to the smallest sin-
gular value of A (i.e., the last column of V which we denote as v):

v = v ( is the scale factor) or v = v ( = 1/ )

    

- Using the components of v and v:

(v 1 , v 2 , v 3 , v 4 , v 5 , v 6 , v 7 , v 8 ) = (r 21 , r 22 , r 23 , T y , r 11 , r 12 , r 13 , T x )





 -7-

Determine  and | |

v 21 + v 22 + v 23 = √
√ 
2 (r 2 + r 2 + r 2 ) = |
21


22 23  |
2
(r 21 + r 22
2
+ r 23
2
= 1)

√ v 25 + v 26 + v 27 = √
 
2 2 (r 2 + r 2 + r 2 ) =


11 12


13  | | 

2
(r 11 + r 12
2
+ r 13
2
= 1 and >0) 

Determine r 21 , r 22 , r 23 , r 11 , r 12 , r 13 , T y , T x

- We can determine the above parameters, up to an unknown common sign.

r 21 = 1/|  | v1 r 11 = 1/  |  | v5
r 22 = 1/|  | v2 r 12 = 1/  |  | v6
r 23 = 1/|  | v3 r 13 = 1/  |  | v7
T y = 1/|  | v4 T x = 1/ |
 | v8

Determine r 31 , r 32 , r 33

- Can be estimated as the cross product of R1 and R2 :

R3 = R1 x R2

- The sign of R3 is already fixed (the entries of R3 remain unchanged if the signs
of all the entries of R1 and R2 are reversed).

Ensuring the orthogonality of R

- The computation of R does not take into account explicitly the orthogonality
constraints.
T
- The estimate R̂ of R cannot be expected to be orthogonal (e.g., R̂ R̂ = I ).

- We can "enforce" the orthogonality on R̂ by using its SVD: R̂ = UDV T

- Replace D with I , e.g., R̂′ = UIV T ( R̂′ R̂′T = I )

 -8-

Determine the sign of 

- Consider the following equations again:

r 11 X w + r 12Y w + r 13 Z w + T x Xc
x = − f /s x = − f /s x
r 31 X w + r 32Y w + r 33 Z w + T z Zc
r 21 X w + r 22Y w + r 23 Z w + T y Yc
y = − f /s y = − f /s y
r 31 X w + r 32 r w + r 33 Z w + T z Zc

- If Z c >0, then x and r 11 X w + r 12Y w + r 13 Z w + T x must have opposite signs (it is

sufficient to check the sign for one of the points).

if x(r 11 X w + r 12Y w + r 13 Z w + T x ) > 0, then

reverse the signs of r 11 , r 12 , r 13 , and T x
else
no further action is required

- Similarly, if Z c >0, then y and r 21 X w + r 22Y w + r 23 Z w + T x must have opposite

signs (it is sufficient to check the sign for one of the points).

if y(r 21 X w + r 22Y w + r 23 Z w + T y ) > 0, then

reverse the signs of r 21 , r 22 , r 23 , and T y
else
no further action is required
 -9-

Determine T z and f x :

- Consider the equation:

r 11 X w + r 12Y w + r 13 Z w + T x
x = − f /s x
r 31 X w + r 32Y w + r 33 Z w + T z

- Let’s rewrite it in the form:

x(r 31 X w + r 32Y w + r 33 Z w + T z ) = − f /s x (r 11 X w + r 12Y w + r 13 Z w + T x )

- We can obtain T z and f x by solving a system of equations like the above, writ-
ten for N points:
 Tz 
A  = b where
 fx 

 x1 (r 11 X 1w + r 12Y 1w + r 13 Z 1w + T x )   −x 1 (r 31 X 1 + r 32Y 1 + r 33 Z 1 + T x ) 
w w w

 x2 (r 11 X 2w + r 12Y 2w + r 13 Z 2w + T x )   −x 2 (r 31 X 2w + r 32Y 2w + r 33 Z 2w + T x ) 
A= b= 
 ... ...   ... 
 xN (r 11 X Nw + r 12Y Nw + r 13 Z Nw + T x )   −x N (r 31 X Nw + r 32Y Nw + r 33 Z Nw + T x ) 

- Using SVD, the (least-squares) solution is:

 Tz  −1 T
  = ( A A) A b
T

 fx 

Determine f y :

- From f x = f /s x and f y = f /s y we have:

fy = fx/ 
 -10-

• Step 2: estimate o x and o y

- The computation of o x and o y will be based on the following theorem:

Orthocenter Theorem: Let T be the triangle on the image plane defined by the three
vanishing points of three mutually orthogonal sets of parallel lines in space. The
image center (o x , o y ) is the orthocenter of T .

- We can use the same calibration pattern to compute three vanishing points (use three
pairs of parallel lines defined by the sides of the planes).

Note 1: it is important that the calibration pattern is imaged from a viewpoint guaran-
teeing that none of the three mutually orthogonal directions will be near parallel to
the image plane !

Note 2: to improve the accuracy of the image center computation, it is a good idea to
estimate the center using several views of the calibration pattern and average the
results.
 -11-

Method 2: Camera parameters through the projection matrix

• Review of basic equations

 Xw   Xw   m m 14  
Xw 
 xh   Yw   Y w   11
m 12 m 13
 Yw 
y = M M  =M   =  m 21 m 22 m 23 m 24   
 h in ex
 Z w   Z w   m 31
  Zw 
w  1   1 
m 32 m 33 m 34 
 1 
x h m 11 X w + m 12Y w + m 13 Z w + m 14
x= =
w m 31 X w + m 32Y w + m 33 Z w + m 34
y h m 21 X w + m 22Y w + m 23 Z w + m 24
y= =
w m 31 X w + m 32Y w + m 33 Z w + m 34

(Note: I have replaced x im with x and y im with y for simplicity)

• Step 1: solve for mij s

- The matrix M has 11 independent entries (e.g., divide every entry by m 11 ).

- We would need at least N =6 world-image point correspondences to solve for the

entries of M.

m 11 X iw + m 12Y iw + m 13 Z iw + m 14 − m 31 x i X iw − m 32 x i Y iw − m 33 x i Z iw + m 34 = 0

m 21 X iw + m 22Y iw + m 23 Z iw + m 24 − m 31 y i X iw − m 32 y i Y iw − m 33 y i Z iw + m 34 = 0

- These equations will lead to a homogeneous system of equations:

Am = 0 where

 X1
w
Y 1w Z 1w 1 0 0 0 0 −x 1 X 1w −x 1Y 1w −x 1 Z 1w −x 1 
 0 0 0 0 X 1w Y 1w Z 1w 1 −y 1 X 1w −y 1Y 1w −y 1 Z 1w −y 1 
 Xw Y 2w Z 2w 1 0 0 0 0 −x 2 X 2w −x 2Y 2w −x 2 Z 2w −x 2 
 2 
A= 0 0 0 0 X 2w Y 2w Z 2w 1 −y 2 X 2w −y 2Y 2w −y 2 Z 2w −y 2 
 ... ... ... ... ... ... ... ... ... ... ... ... 
 w 
 XN Y Nw Z Nw 1 0 0 0 0 −x N X Nw −x N Y Nw −x N Z Nw −x N 
 0 0 0 0 X Nw Y Nw Z Nw 1 −y N X Nw −y N Y Nw −y N Z Nw −y N 
 -12-

- It can be shown that A has rank 11 (for N ≥ 11).

- If A = UDV T , the system has a nontrivial solution m which is proportional to the

column of V corresponding to the smallest singular value of A (i.e., the last column of
V denoted here as m):

m = m ( is the scale factor) or m = m ( = 1/ )

  

• Step 2: find the intrinsic/extrinsic parameters using mij s

- The full expression for M is as follows:

 − f x r 11 + o x r 31 − f x r 12 + o x r 32 − f x r 13 + o x r 33 − f xT x + oxTz 
M =  − f y r 21 + o y r 31 − f y r 22 + o y r 32 − f y r 23 + o y r 33 − f yT y + oyTz 
 
 r 31 r 32 r 33 Tz 

- Let’s define the following vectors:

q 1 = (m 11 , m 12 , m 13 )T

q 2 = (m 21 , m 22 , m 23 )T

q 3 = (m 31 , m 32 , m 33 )T

q 4 = (m 14 , m 24 , m 34 )T

- The solutions are as follows (see book for details):

o x = qT1 q 3 o y = qT2 q 3

fx = √

qT1 q 1 − o2x fy = √

qT2 q 2 − o2y

- The rest parameters are easily computed ....

Question: how would you estimate the accuracy of a calibration algorithm?

 -13-

• Some comments

- The precision of calibration depends on how accurately the world and image points
are located.

- Studying how localization errors "propagate" to the estimates of the camera parame-
ters is very important.

- Although the two methods described here should produce the same results (at least
theoretically), we usually obtain different solutions due to different error propaga-
tions.

- Method 2 is simpler and should be preferred if we do not need to compute the

intrinsic/extrinsic camera parameters explicitly.