Intro to Robotics, Class Handout –Lecture 19, Camera calibration

Name:

Camera calibration. (First, a Review)

𝑥$ 𝑢 𝑘𝑥 $ = 𝑢
𝑘 "𝑦 $ ( = *𝑣 ., 𝑘𝑦 = 𝑣
𝑧$ 𝜆 𝑘𝑧 = 𝜆

Image plane coordinates (𝑢, 𝑣)

𝑥 𝑦
𝑢=𝜆 , 𝑣=𝜆
𝑧 𝑧

Coordinates of principal point are ((𝑜3 , 𝑜$ ). Find using

Pixel coordinates are (𝑟, 𝑐).
𝑢 𝑣
− = (𝑟 − 𝑜3 ) , − = (𝑐 − 𝑜$ )
𝑠8 𝑠9

If we know position and orientation of camera frame relative to world:

𝑥 : = 𝑅$: 𝑥 $ + 𝑂$:
If we know 𝑥 : , but want 𝑥 $
𝑥 $ = 𝑅:$
(𝑥 : − 𝑂$: )
Define the extrinsic camera parameters: 𝑅 = 𝑅: $
, 𝑇 = −𝑅: $ :
𝑂$
Rewrite using parameters: 𝑥 $ = 𝑅𝑥 : + 𝑇

Camera orientation typically is a pan about the world z and a tilt about the camera x-axis.
𝑅 = 𝑅?,@ 𝑅8,A

𝜆𝑥 𝜆𝑦
𝑟=− + 𝑜3 , 𝑐=− + 𝑜$
𝑠8 𝑧 𝑠9 𝑧
Unknowns are 𝜆, 𝑠8 , 𝑜3 , 𝑠9 , 𝑜$
C C
Just need to know 𝑓8 = DE
and 𝑓9 = DF
.

Acquire a data set of the form {𝑟H , 𝑐H , 𝑥H: , 𝑦H: , 𝑧H: } where (𝑟H , 𝑐H ) are image pixel coordinates
that correspond to world coordinates 𝑥H , 𝑦H , 𝑧H

𝑟JJ 𝑟JK 𝑟JL 𝑇8

𝑅 = "𝑟KJ 𝑟KK 𝑟KL ( , 𝑇 = M𝑇9 N
𝑟LJ 𝑟LK 𝑟LL 𝑇?

𝑥$ = 𝑟JJ 𝑥 + 𝑟JK 𝑦 + 𝑟JL 𝑧 + 𝑇8

𝑦$ = 𝑟KJ 𝑥 + 𝑟KK 𝑦 + 𝑟KL 𝑧 + 𝑇9
𝑧$ = 𝑟LJ 𝑥 + 𝑟LK 𝑦 + 𝑟LL 𝑧 + 𝑇?

Page 1 of 4
 𝑥$ = 𝑟JJ 𝑥 : + 𝑟JK 𝑦 : + 𝑟JL 𝑧 : + 𝑇8
𝑦$ = 𝑟KJ 𝑥 : + 𝑟KK 𝑦 : + 𝑟KL 𝑧 : + 𝑇9
𝑧$ = 𝑟LJ 𝑥 : + 𝑟LK 𝑦 : + 𝑟LL 𝑧 : + 𝑇?

𝜆 𝑟JJ 𝑥 + 𝑟JK 𝑦 + 𝑟JL 𝑧 + 𝑇8 𝜆 𝑟KJ 𝑥 + 𝑟KK 𝑦 + 𝑟KL 𝑧 + 𝑇9

𝑟 − 𝑜3 = − , 𝑐 − 𝑜$ = −
𝑠8 𝑟LJ 𝑥 + 𝑟LK 𝑦 + 𝑟LL 𝑧 + 𝑇? 𝑠9 𝑟LJ 𝑥 : + 𝑟LK 𝑦 : + 𝑟LL 𝑧 : + 𝑇?

Define
𝑟O = , 𝑐 O = 𝑐 − 𝑜$

𝑟JJ 𝑥 + 𝑟JK 𝑦 + 𝑟JL 𝑧 + 𝑇8 𝑟KJ 𝑥 : + 𝑟KK 𝑦 : + 𝑟KL 𝑧 : + 𝑇9

𝑟′ = −𝑓8 , 𝑐′ = −𝑓9
𝑟LJ 𝑥 : + 𝑟LK 𝑦 : + 𝑟LL 𝑧 : + 𝑇? 𝑟LJ 𝑥 + 𝑟LK 𝑦 + 𝑟LL 𝑧 + 𝑇?

Solve for 𝑧 $ and set the two equations equal.

𝑟H ′𝑓9 Q𝑟KJ 𝑥 : + 𝑟KK 𝑦 : + 𝑟KL 𝑧 : + 𝑇9 R = 𝑐H ′𝑓8 (𝑟JJ 𝑥H + 𝑟JK 𝑦H + 𝑟JL 𝑧H − 𝑇8 ),

Divide by 𝑓9 and define a new value as the ratio: 𝛼 =/𝑓9

𝑟HO 𝑟KJ 𝑥H: + 𝑟HO 𝑟KK 𝑦H: + 𝑟HO 𝑟KL 𝑧H: + 𝑟HO 𝑇9 − −𝛼𝑐H ′𝑟JJ 𝑥H − 𝛼𝑐H ′𝑟JK 𝑦H − 𝛼𝑐H ′𝑟JL 𝑧H − 𝛼𝑐H ′𝑇8

Write as matrix with the unknowns as 𝑥: 𝐴𝑥 = 0

𝑟KJ
⎡ 𝑟KK ⎤
𝑟JO 𝑥J: 𝑟JO 𝑦J: 𝑟JO 𝑧J: 𝑟JO −𝑐JO 𝑥J: −𝑐JO 𝑦J: −𝑐JO 𝑧J: −𝑐JO ⎢ 𝑟KL ⎥
⎢ 𝑇 ⎥
𝑟O𝑥 : 𝑟KO 𝑦K: 𝑟KO 𝑧K: 𝑟KO −𝑐KO 𝑥K: −𝑐KO 𝑦K: −𝑐KO 𝑧K: −𝑐KO 9
𝐴=W K K Z , 𝑥 = ⎢𝛼𝑟 ⎥
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⎢ JJ ⎥
𝑟YO 𝑥Y: 𝑟YO 𝑦Y: 𝑟YO 𝑧Y: 𝑟YO −𝑐YO 𝑥Y: −𝑐YO 𝑦Y: −𝑐YO 𝑧Y −𝑐YO ⎢𝛼𝑟JK ⎥
⎢𝛼𝑟JL ⎥
⎣ 𝛼𝑇8 ⎦

Solution is 𝑥̅ = [𝑥̅J 𝑥̅ K 𝑥̅ L 𝑥̅ c 𝑥̅ d 𝑥̅ e 𝑥̅ f 𝑥̅ g ]i , but this is just

Page 2 of 4
 [𝑥̅J 𝑥̅ K 𝑥̅ L 𝑥̅ c 𝑥̅ d 𝑥̅ e 𝑥̅ f 𝑥̅ g ]i = 𝑘[𝑟KJ 𝑟KK 𝑟KL 𝑇9 𝛼𝑟JJ 𝛼𝑟JK 𝛼𝑟JL 𝛼𝑇8 ]i

𝑟JJ 𝑟JK 𝑟JL 𝑇8

𝑅 = "𝑟KJ 𝑟KK 𝑟KL ( , 𝑇 = M𝑇9 N
𝑟LJ 𝑟LK 𝑟LL 𝑇?

We know that 𝑅 is a

Solve for |𝑘|: k𝑥̅JK + 𝑥̅ KK + 𝑥̅ LK = k𝑘 K (𝑟KJ

K K
+ 𝑟KK K)
+ 𝑟KL = |𝑘|

Solve for 𝛼: k𝑥̅ dK + 𝑥̅ eK + 𝑥̅ fK = k𝛼 K 𝑘 K (𝑟JJ

K K
+ 𝑟JK K)
+ 𝑟JL = 𝛼|𝑘| (by def. 𝛼 > 0)

Solve for sign of 𝑘: If a point is visible, it must have a 𝑧 $ > 0 and scaling factor 𝑓8 > 0.

8m J J
Therefore, since 𝑟 O = −𝑓8 ? m, 𝑟 O 8 m == −𝑓8 ? m < 0

Choose 𝑘 such that: Sign[𝑟 O ] ∙ Sign[(𝑟JJ 𝑥 : + 𝑟JK 𝑦 : + 𝑟JL 𝑧 : + 𝑇8 )] < 0

Solve for third row of 𝑅: [𝑟LJ 𝑟LK 𝑟LL ]i = [𝑟JJ 𝑟JK 𝑟JL ]i × [𝑟KJ 𝑟KK 𝑟KL ]i

3 8 v w3 9 v w3 ? vwi
Solve for 𝑇? and 𝑓8 : redo the matrix equations for 𝑟 O = −𝑓8 3uu 8 v w3ux 9 v w3uy ? v wiE
yu yx yy z

𝑟HO (𝑟LJ 𝑥J: + 𝑟LK 𝑦J: + 𝑟LL 𝑧J: ) 𝑟HO (𝑟JJ 𝑥J: + 𝑟JK 𝑦J: + 𝑟JL 𝑧J: + 𝑇8 )
1
𝑟 O (𝑟 𝑥 : + 𝑟LK 𝑦K: + 𝑟LL 𝑧K: ) 𝑟HO (𝑟JJ 𝑥K: + 𝑟JK 𝑦K: + 𝑟JL 𝑧K: + 𝑇8 )
𝑇
𝐵𝒚 = 0 ⇒ 𝐵 = W H LJ K Z, 𝒚 = " ? (
⋮ ⋮ ⋮ 𝑓8
𝑟YO (𝑟LJ 𝑥Y: + 𝑟LK 𝑦Y: + 𝑟LL 𝑧Y: ) 𝑟YO (𝑟JJ 𝑥Y: + 𝑟JK 𝑦Y: + 𝑟JL 𝑧Y: + 𝑇8 )

Solution is 𝑦• = [𝑦•J 𝑦•K 𝑦•L ]i , but this is just a scalar multiple of the true solution

Page 3 of 4
Intro to Robotics, Class Worksheet – Preparation for Lecture 19

1. Suppose that a circle lies in a plane parallel to the image plane. By drawing a picture,
show that the perspective projection of the circle is a circle in the image plane.
Determine the circle’s radius in the image plane if 𝜆 = 5, the circle is 20 units away, and
the actual circle has radius 24.

2. The three vanishing points for parallel lines a, parallel lines b, and parallel lines c are
𝑉‚ , 𝑉ƒ and 𝑉$ . The vanishing points define a triangle, and the three altitudes ℎ‚ , ℎƒ and ℎ$
intersect at the orthocenter of this triangle. For this special case, where the three direction
vectors for line sets a, b, and c are mutually orthogonal, what is the significance of this
point?

The orthocenter of the triangle 𝑉‚ , 𝑉ƒ , 𝑉$ is (𝑜, 𝑜, 𝜆) it lies on the camera’s z-axis.

Page 4 of 4