Camera models Synonyms

A camera model describes a projection from P 3 to P 2 .
The camera center is also called
The most basic camera model is the pinhole camera where a 3D projection center, perspective
world point X is projected onto a 2D image point x in the image center, optical center, or focus.
plane through a camera center C.
The image plane is also called
In the physical camera, a mirror image is formed behind the focal plane.
camera center. However, often a virtual image plane is depicted in The distance between the cam-
front of the camera center.
era center and the image plane is
The points X, C and (the 3D point) x are collinear, i.e. lie on the called focal distance, principal dis-
same line in space. tance, camera constant, or film-
focus-distance.
Y plane is called the principal plane.
Y
The ray through the camera
center orthogonal to the image
plane is called principal axis,
principal ray, camera axis or
central ray.
The intersection between the
principal ray and the image plane
is called principal point or foot
point.
The plane through the camera
center parallel with the image

X X
y X X
x y
x
C x
Z x
p
C Z
principal axis p
camera
centre image plane principal axis
camera
centre image plane
– p. 1 – p. 2

Central projection Central projection in homogenous coordinates

If the camera center is at the origin and the image plane is the The corresponding expression in homogenous coordinates may
plane Z = f the world coordinate (X, Y, Z)> is mapped to the point be written as
(f X/Z, f Y /Z, f )> in space or (f X/Z, f Y /Z) in the image plane, i.e.    
X     X
 Y  fX f 0
(X, Y, Z)> 7→ (f X/Z, f Y /Z)>   Y 
 
 7→  f Y  =  f 0  
    
 Z   Z  .
 
Z 1 0
Y
1 1
Y
X X
x P X
y
x
x
fY/Z The matrix P is called the camera matrix and maps the world point
C Z
p C X onto the image point x.
principal axis p Z
camera
centre image plane f In more compact form P may be written as

P = diag(f, f, 1)[I | 0],

where diag(f, f, 1) is a diagonal matrix and I is the identity matrix.

– p. 3 – p. 4
 The principal point The camera calibration matrix
If the principal point is not at the origin If we write 2 3
of the image coordinate system, the map- ycam
f px
ping becomes K=4
6
f py 5
7

py p 1
(X, Y, Z)> 7→ (f X/Z + px , f Y /Z + py )> , y
x cam
the projection may be written as
where (px , py )> are the image coordinates x px x = K[I | 0]Xcam .
for the principal point.
In homogenous coordinates The matrix K is called the camera calibration matrix. The notation Xcam
emphasizes that the world coordinates are with respect to the camera, i.e.
    the camera center is at the origin, the X and Y axes coincide and the
X     X
f X + Zpx f px 0  principal axis coincides with the Z axis.
 Y  Y 
 7→  f Y + Zpy  =  f py 0 
      
Ycam
Z Z
 
   
Z 1 0
1 1
Zcam
C

Xcam
– p. 5 – p. 6

The camera position and orientation Internal and external parameters

Usually the camera coordinate system does not coincide with the world The projection equation
coordinate system. The two coordinate systems are related via a rotation x = KR[I | − C̃]X
and a translation.
that describes the general projection for a pinhole camera has 9 degrees of freedom: 3
If the point X̃ denotes an cartesian 3D point in world coordinates and X̃cam
in K (the elements f, px , py ), 3 in R (rotation angles) and 3 for C̃.
denotes the same point in camera coordinates, then their relation is given by
The elements of K describes properties internal to the camera while the parameters of
X̃cam = R(X̃ − C̃), R and C̃ describe the relation between the camera and the world.
The parameters are therefore called one of
where C̃ is the camera center in world coordinates and R is a 3 × 3 rotation K R, C̃
matrix describing the orientation of the camera in space. internal parameters external parameters
In homogenous coordinates internal orientation external orientation
intrinsic parameters extrinsic parameters
2 3
X sensor model platform model
" # " #
R −RC̃ 6 Y 7 R −RC̃
6 7 Sometimes it is practical not to write the camera center explicitly but instead describe
Xcam = 6 7= X,
0 1 4 Z 5 0 1
the world-to-camera transformation as X̃cam = RX̃ + t with camera matrix
1
P = K[R | t],
or
x = KR[I | − C̃]X. where t = −RC̃.
– p. 7 – p. 8
 Aspect ratio Skew
If we have different scale in the x and y directions, i.e. the pixels are not For an even more general camera model we can add a skew parameter s to
square, we have to include that deformation into the equation. describe any non-orthogonality between the image axis. Then the camera
matrix becomes
Let mx and my be the number of pixels per unit in the x and y direction of the 2 3
image. Then the camera calibration matrix becomes αx s x0
K=4 αy y0 5 .
6 7
2 32 3
mx f px 1
K=4 my f py 5
6 76 7
A camera matrix
54
1 1
P = KR[I | − C̃] = [KR | − KRC̃]
2 3 2 3
mx f mx px αx x0 where the camera calibration matrix K is in the form above is called a finite
=4 my f my py 5=4 αy y0 5 ,
6 7 6 7
projective camera.
1 1
A finite projective camera has 11 degrees of freedom, the same as a 3 × 4
homogenous matrix.
where αx = f mx and αy = f my is the focal length in pixels in the x and y
For a finite projective camera P the left 3 × 3 block KR is always non-singular.
directions and x̃0 = (x0 , y0 )> = (mx px , my py )> is the principal point in
pixels. If we remove this restriction we get a general projective camera described by
a 3 × 4 matrix of rank 3.
A camera with unknown aspect ratio has 10 degrees of freedom.

– p. 9 – p. 10

The general projective camera The camera center

A general projective camera P maps world points X onto image The camera matrix P has a 1-dimensional null-space. Let the vector C be a basis for
points x according to the null-space, i.e. PC = 0. We will see that C represents the camera center in
homogenous coordinates.
x = PX,
Study the line through C and an arbitrary point A in space. Points on this line may be
where the camera matrix P is a 3 × 4 homogenous matrix of rank 3. described by
X(λ) = λA + (1 − λ)C, −∞ < λ < ∞.
A general projective camera may be blocked as
Under the mapping x = PX points on the line are mapped to
P = [M | p4 ],
x = PX(λ) = λPA + (1 − λ) PC = λPA, −∞ < λ < ∞.
|{z}
where M is a 3 × 3 matrix. We will see that P is a finite camera iff M =0
is non-singular
This means that all points on the line are mapped to the same point PA. Thus, the line
has to be a ray through the camera center. Since A was arbitrary, C must be a
homogenous representation of the camera center.
For finite cameras this is given from the fact that C = (C̃> , 1)> is a null-vector to
P = KR[I | − C̃].
Another line of reasoning concludes that the projection PC = (0, 0, 0)> is undefined
and the camera center is the only point with an undefined projection.

– p. 11 – p. 12
 The camera column vectors The camera row vectors
Let pi , i = 1, . . . , 4 be the columns of P. Then p1 , p2 , p3 are the Let Pi> , i = 1, . . . , 3 be the rows of P.
mappings of the axis directions in the world coordinate system. The principal plane of the camera is the plane through
E.g. the X axis D = (1, 0, 0, 0)> is mapped to the camera center parallel with the image plane. It
consists of all points X that are mapped to the line at
  infinity in the image plane, i.e. PX = (x, y, 0)> .
1
h i 0  A point is on the principal plane iff P3> X = 0. Thus
PD = p1 p2 p3 p4  = p1 .
 
the vector P3 represents the principal plane. P3 y x
0

Especially C is in the principal plane since PC = 0.
 
0
All points X on the plane P2 satisfy P2> X = 0 and principal plane

are therefore mapped to PX = (x, 0, w)> on the

The column vector p4 is the mapping of the world origin (0, 0, 0, 1)> .
image x axis. Since PC = 0 the plane P2 the plane
may be defined by the camera center and the image x
axis. y x
Z P2
1
p
3
Similarly the plane P may be defined by the camera
C O center and the image y axis.
p Y
2
The vectors P1 and P2 depend on the internal coordi-
p
1
nate system of the camera. P3 does not.
X – p. 13 – p. 14

The principal point and the principal axis The direction of the principal axis
The principal axis is the line through the camera center C In theory all points X outside the principal plane are projected to a finite point
orthogonal to the principal plane P3 . in the image plane according to x = PX. In reality only points in front of the
camera.
The axis intersects the image plane in the principal point.
Let P = [M | p4 ]. We know that m3 points in the direction of the principal axis.
The normal of the principal plane P3 = (p31 , p32 , p33 , p34 )> has However, the sign of P is undefined so the question is if m3 or −m3 points in
direction P̂3 = (p31 , p32 , p33 , 0)> . the positive direction.
Study the projection
The projection of this point gives the principal point PP̂3 .
If P = [M | p4 ] the principal point is calculated as x = Pcam Xcam = K[I | 0]Xcam ,

x0 = Mm3 , where Xcam is a 3D point in camera coordinates. Note that the vector
v = det(M)m3 = (0, 0, 1)> points forward in the direction of the principal axis,
where m3> is the third row of M. independently of the scaling of Pcam since

Pcam → kPcam ⇒ v → k 4 v

with unchanged sign.

In world coordinates the camera matrix is P = kK[R | − Rp4 ], where M = kKR.
Since det(R) = +1 the direction of v = det(M)m3 is unchanged.
– p. 15 – p. 16
Forward and backward projection of points Point depth
A world point X is forward projected (mapped) onto an image point Study a camera P = [M | p4 ] that projects a point X = (X, Y, Z, 1)> = (X̃> , 1)> in
according to x = PX. R3 on the image point x = w(x, y, 1)> = PX.

Points at infinity (vanishing points) D = (d> , 0)> are mapped to Let C = (C̃> , 1)> be the camera center. Then

x = PD = [M | p4 ]D = Md w = P3> X = P3> (X − C)

since PC = 0 for the camera center C.

and are only affected by the left 3 × 3 block M of P.
Since
An image point is backward projected as a ray in space. We know P3> (X − C) = m3> (X̃ − C̃),
two points on this ray: The camera center C and the point P+ x,
where m3 is the direction of the principal axis,
where P+ is the psuedo inverse of P. The psuedo inverse of P is
the matrix P+ = P> (PP> )−1 such that PP+ = I. w = m3> (X̃ − C̃)
+
The point P x is on the ray since it is projected onto x, since
may be interpreted as the inner product between the direction of the principal axis and
P(P+ x) = Ix = x. the ray through the camera center and X.
The ray can thus be describes as the line

X(λ) = P+ x + λC.
– p. 17 – p. 18

Point depth Factorizing the camera matrix

If the camera matrix has been normalized such that det(M) > 0 and km3 k = 1 the m3 Given a camera matrix P the camera center is given from the nullspace
is a unit vector in the positive principal axis direction. Then w may be interpreted as PC = 0.
the depth of the point X from the camera center C in the direction of the principal axis
For a finite camera
X
P = [M | − MC̃] = K[R | − RC̃]
with det(M) > 0, K and R may be calculated by the RQ factorization of M.
m3
C
The RQ factorization of a matrix A gives an upper triangular matrix K (“R”)
X . m3
and an orthogonal matrix R (“Q”) such that KR = A.

For an unnormalized camera matrix, the depth of a point X = (X, Y, Z, T )> with The matrix R describes the orientation and K the calibration of the camera.
respect to a camera P = [M | p4 ] may instead be calculated as If K is chosen to have positive diagonal elements the factorization is unique.

sign(det(M))w If the calibration matrix K is scaled such that K33 = 1 it may be interpreted as
depth(X; P) = ,
T km3 k 2 3
αx s x0
where K=4 αy y0 5 .
6 7
P(X, Y, Z, T )> = w(x, y, 1)> . 1
If P is a finite camera, then all points satisfying

sign(det(M))w/T > 0
– p. 19 – p. 20
 Affine cameras
An affine camera P = [M | p4 ] is an infinite camera, i.e. M has rank 2.
An affine camera P has last row P3> = (0, 0, 0, 1)> .
Points at infinity in P 3 are mapped onto points at infinity in P 2 .
The camera center C is on the plane at infinity.
The principal plane is the point at infinity.
The principal point is undefined.
The canonical form [I | 0] is replaced by the parallel projection matrix
2 3
1 0 0 0
4 0 1 0 0 5.
6 7

0 0 0 1

The camera caibration matrix K is replaced by

" #
K2×2 0
.
0 1
– p. 21