Robert Collins

CSE486, Penn State

Lecture 13:
Camera Projection II
Reading: T&V Section 2.4
Robert Collins
CSE486, Penn State
Recall: Imaging Geometry

W
Object of Interest
in World Coordinate
System (U,V,W)
V
U
Robert Collins
CSE486, Penn State
Imaging Geometry

Camera Coordinate
Y System (X,Y,Z).

X Z
• Z is optic axis
f • Image plane located f units
out along optic axis
• f is called focal length
Robert Collins
CSE486, Penn State
Imaging Geometry

W
Y y
X Z x

V
U

Forward Projection onto image plane.

3D (X,Y,Z) projected to 2D (x,y)
Robert Collins
CSE486, Penn State
Imaging Geometry

W
Y y
X Z x

V
u U

v Our image gets digitized

into pixel coordinates (u,v)
Robert Collins
CSE486, Penn State
Imaging Geometry
Camera Image (film) World
Coordinates Coordinates W
Coordinates
Y y
X Z x

V
u U

v Pixel
Coordinates
Robert Collins
CSE486, Penn State
Forward Projection

World Camera Film Pixel

Coords Coords Coords Coords
U X x u
V Y y v
W Z

We want a mathematical model to describe

how 3D World points get projected into 2D
Pixel coordinates.

Our goal: describe this sequence of

transformations by a big matrix equation!
Robert Collins
CSE486, Penn State
Intrinsic Camera Parameters

World Camera Film Pixel

Coords Coords Coords Coords
U X x u
V Y y v
W Z

Affine Transformation
Robert Collins
CSE486, Penn State
Intrinsic parameters

• Describes coordinate transformation

between film coordinates (projected image)
and pixel array
• Film cameras: scanning/digitization
• CCD cameras: grid of photosensors

still in T&V section 2.4

Robert Collins
CSE486, Penn State
Intrinsic parameters (offsets)
film plane
pixel array
(projected image)
ox (0,0) u (col)

oy x v (row)
(0,0)

X Y
u  f  ox v  f  oy
Z Z
ox and oy called image center or principle point
Robert Collins
CSE486, Penn State
Intrinsic parameters
sometimes one or more coordinate axes are flipped (e.g. T&V section 2.4)

film plane pixel array

ox (0,0) u (col)

oy y
v (row)
x
(0,0)

X Y
u  f  ox v  f  oy
Z Z
Robert Collins
CSE486, Penn State
Intrinsic parameters (scales)
sampling determines how many rows/cols in the image

film scanning
resolution
pixel array

C cols x R rows
CCD
analog

resample
Robert Collins
CSE486, Penn State
Effective Scales: sx and sy

1 X 1 Y
u s f  ox v  f  oy
x Z sy Z

Note, since we have different scale factors in x and y,

we don’t necessarily have square pixels!

Aspect ratio is sy / sx

O.Camps, PSU
Robert Collins
CSE486, Penn State
Perspective projection matrix
Adding the intrinsic parameters into the
perspective projection matrix:
X 
 x'  f / s x 0 ox 0  
 y '   0  Y
   f / sy oy 0  
Z 
 z '   0 0 1 0  
1
To verify:
x’
u 1 X Y
1
z’ u s f  ox v  f  oy
y’ x Z sy Z
v
z’
O.Camps, PSU
Robert Collins
CSE486, Penn State
Note:
Sometimes, the image and the camera coordinate systems
have opposite orientations: [the book does it this way]

X X 
f  ( u  o x ) s x  x'  f / s x 0  ox 0  
Z  y '   0   Y
    f / sy  oy 0
Y Z 
f  ( v  o y )s y  z '   0 0 1 0  
Z 1
Robert Collins
CSE486, Penn State
Note 2
In general, I like to think of the conversion as
a separate 2D affine transformation from film
coords (x,y) to pixel coordinates (u,v):
X 
u’ a11 a12 xa' 13f 0 0 0  
    Y
v’ a21 a22 ya' 23  0
   f 0 0  
w’ Z 
0 0  z 1'   0 0 1 0  
1
Maff Mproj

u = Mint PC = Maff Mproj PC

Robert Collins
CSE486, Penn State
Huh?

Did he just say it was “a fine” transformation?

No, it was “affine” transformation, a type of

2D to 2D mapping defined by 6 parameters.

More on this in a moment...

Robert Collins
CSE486, Penn State
Summary : Forward Projection
World Camera Film Pixel
Coords Coords Coords Coords
U X x u
V Mext Y Mproj Maff
y v
W Z

U Mext X Mint u
V Y v
W Z

U u
M
V m11 m12 m13 m14 v
W m21 m22 m23 m24
m31 m31 m33 m34
Robert Collins
CSE486, Penn State
Summary: Projection Equation
Film plane Perspective
World to camera
to pixels projection

Maff Mproj Mext

Mint

M
Robert Collins
CSE486, Penn State

Lecture 13/14:
Intro to Image Mappings
Robert Collins
CSE486, Penn State
Image Mappings
Overview

from R.Szeliski
Robert Collins
CSE486, Penn State
Geometric Image Mappings
Geometric
image transformation transformed image

(x,y)
(x’,y’)

x’ = f(x, y, {parameters})
y’ = g(x, y, {parameters})
Robert Collins
CSE486, Penn State
Linear Transformations
(Can be written as matrices)
Geometric
image transformation transformed image

(x,y)
(x’,y’)

x’ x
y’ = M(params) y
1 1
Robert Collins
CSE486, Penn State
Translation
y y’
transform
1

ty

0
0 1 x tx x’

equations matrix form

Robert Collins
CSE486, Penn State
Scale
y y’
transform S

0 0
0 1 x 0 S x’

equations matrix form

Robert Collins
CSE486, Penn State
Rotation
y y’
transform
1


)
0
0 1 x x’

equations matrix form

Robert Collins
CSE486, Penn State
Euclidean (Rigid)
y y’
transform 

)
1
ty

0
0 1 x tx x’

equations matrix form

Robert Collins
CSE486, Penn State
Partitioned Matrices

http://planetmath.org/encyclopedia/PartitionedMatrix.html
Robert Collins
CSE486, Penn State
Partitioned Matrices

2x1 2x2 2x1 2x1

1x1 1x2 1x1 1x1 matrix form

equation form
Robert Collins

Another Example (from last time)

CSE486, Penn State

X r11 r12 r13 tx U

Y r21 r22 r23 ty V
Z r31 r32 r33 tz W
1 0 0 0 1 1
3x1 3x3 3x1 3x1
PC R T PW
1x1 = 1x3 1x1 1x1
1 0 1 1

PC = R P W + T
Robert Collins
CSE486, Penn State
Similarity (scaled Euclidean)
y y’ S
transform 

)
1
ty

0
0 1 x tx x’

equations matrix form

Robert Collins
CSE486, Penn State
Affine
y y’
transform
1

0
0 1 x x’

equations matrix form

Robert Collins
CSE486, Penn State
Projective
y y’
transform
1

0
0 1 x x’
Note!

equations matrix form

Robert Collins

Summary of 2D Transformations
CSE486, Penn State
Robert Collins

Summary of 2D Transformations
CSE486, Penn State

Euclidean
Robert Collins

Summary of 2D Transformations
CSE486, Penn State

Similarity
Robert Collins

Summary of 2D Transformations
CSE486, Penn State

Affine
Robert Collins

Summary of 2D Transformations
CSE486, Penn State

Projective
Robert Collins
CSE486, Penn State
Summary of 2D Transformations

from R.Szeliski