Video 1.

1
Jianbo Shi
Property of University of Pennsylvania, Jianbo Shi
1
Kodak moments

Property of University of Pennsylvania, Jianbo Shi

2
 Capturing Moment

“Rituals Enhance Consumption,” Kathleen Vohs,

Psychological Science 2013
Property of University of Pennsylvania, Jianbo Shi
3
Moments: selfie

Reid Wiseman and Alexander Gerst

with a Nikon D2X and a Nikkor
10.5mm fisheye lens, 4
Property of University of Pennsylvania, Jianbo Shi
 DATE:1920

Side view of
photographers
posing together for a
photograph on the
roof of Marceau's
Studio, while Joseph
Byron holds one
side of the camera
with his right hand
and Ben Falk holds
the other side with
his left hand.

Property of University of Pennsylvania, Jianbo Shi

5
stereoscope was invented by Sir
Charles Wheatstone in 1838.

Property of University of Pennsylvania, Jianbo Shi

6
 3D Shape perception
1) Depth: the distance of the surface from the observer
2) Surface orientation: the slant and tilt of the surface with
respect to observers’ sight

Property of University of Pennsylvania, Jianbo Shi

7
 Depth ambiguity
Inverse problem: multiple solution exists

Property of University of Pennsylvania, Jianbo Shi

8
 Pictorial cues for 3D shape
Perspective projection gives us the relative position to horizon,
therefore we can deduce its physical size.

Shading also reveal shape using illumination model

9
Property of University of Pennsylvania, Jianbo Shi
9
Stereo Vision

Property of University of Pennsylvania, Jianbo Shi

10
Property of University of Pennsylvania, Jianbo Shi
11
Property of University of Pennsylvania, Jianbo Shi
12
 Camera Design

What is a camera?
Photon collecting machine

Property of University of Pennsylvania, Jianbo Shi

13
 How do we see the world?

• Let’s design a camera

– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
Property of University of Pennsylvania, Jianbo Shi
14
Slide by Steve Seitz
 Pinhole camera

• Add a barrier to block off most of the rays

– This reduces blurring
– The opening known as the aperture
– How does this transform the image?

Property of University of Pennsylvania, Jianbo Shi

15
Slide by Steve Seitz
 Pinhole camera model

• Pinhole model:
– Captures pencil of rays – all rays through a single point
– The point is called Center of Projection (COP)
– The image is formed on the Image Plane
– Effective focal length f is distance from COP to Image Plane

Property of University of Pennsylvania, Jianbo Shi

16
Slide by Steve Seitz
Camera with lense

Property of University of Pennsylvania, Jianbo Shi

17
Property of University of Pennsylvania, Jianbo Shi
18
 Video 1.2
Jianbo Shi
Property of University of Pennsylvania, Jianbo Shi
19
Property of University of Pennsylvania, Jianbo Shi
20
Property of University of Pennsylvania, Jianbo Shi
21
 In a computer…
an image is a 2 dimensional table of numbers, a 2D matrix

j
121 121 118 111 ... 21
134 136 137 132 ... 23 I(i,j) is the sensor value at
i 133 131 136 136 ... 25 location y = i, x = j
136 145 148 151 ... 34
137 140 147 149 ... 54 I(2,1) = 134
... ... ... ... ... ... I(3,4) = 136
231 233 243 244 ... 179

Any 2D matrix can be seen as an image

Property of University of Pennsylvania, Jianbo Shi
22
 Camera Design

What is a camera?
Dimension Reduction Machine

Property of University of Pennsylvania, Jianbo Shi

23
Dimensionality Reduction Machine (3D to 2D)

3D world 2D image

Point of observation

• What have we lost?

– Angles
– Distances (lengths)
Property of University of Pennsylvania, Jianbo Shi
24
Figures © Stephen E. Palmer, 2002
Funny things happen…

Property of University of Pennsylvania, Jianbo Shi

25
3D to 2D mapping

Property of University of Pennsylvania, Jianbo Shi

26
 J. Barozzi’s
Perspectograph

Property of University of Pennsylvania, Jianbo Shi

27
 Lens

Pixel

CCD sensor

3D object

Property of University of Pennsylvania, Jianbo Shi

28
3D Point Projection (Metric Space)

3D point (X,Y, Z )

(−uccd, −vccd )

fm : Focal length in meter

X Y
(X,Y, Z ) → (uccd,vccd ) = (fm ,fm )
Z Z
2D projection onto CCD plane
(−uimg, −vimg )

Property of University of Pennsylvania, Jianbo Shi

29
3D Point Projection (Metric Space)

Projection plane 3D point (X,Y, Z )

(uccd,vccd )

(uimg,vimg )
(−uccd, −vccd ) fm

Focal length in meter

X Y
(X,Y, Z ) → (uccd,vccd ) = (fm ,fm )
Z Z
2D projection onto CCD plane
(−uimg, −vimg )

Property of University of Pennsylvania, Jianbo Shi

30
3D Point Projection (Metric Space)

Projection plane 3D point (X,Y, Z )

(uccd,vccd )

(uimg,vimg )
fm

Focal length in meter

X Y
(X,Y, Z ) → (uccd,vccd ) = (fm ,fm )
Z Z
2D projection onto CCD plane

Property of University of Pennsylvania, Jianbo Shi

31
 Video 1.3
Jianbo Shi
Property of University of Pennsylvania, Jianbo Shi
32
3D Point Projection (Pixel Space)

Projection plane 3D point (X,Y, Z )

(uimg ,vimg )

(uimg,vimg )
fm

Focal length in meter

wimg X himg Y
(X,Y, Z ) → (uimg ,vimg ) = (fm ,fm )
wccd Z hccd Z

Property of University of Pennsylvania, Jianbo Shi

33
 Exercise
What f to make the height of Eifel tower appear 960 pixel distance?

960 pix

1280
324 m pix

fm ?

1500 m

Property of University of Pennsylvania, Jianbo Shi

34
 Exercise
What f to make the height of Eifel tower appear 960 pixel distance?

960 pix

21.8 mm
1280
324 m pix
ccd size

fm ?

1500 m

Property of University of Pennsylvania, Jianbo Shi

35
 Exercise
What f to make the height of Eifel tower appear 960 pixel distance?

960 pix

21.8 mm
1280
324 m pix
ccd size

Y himg Y
fm ? yimg = f = fm
1500 m Z hccd Z

Property of University of Pennsylvania, Jianbo Shi

36
 Exercise
What f to make the height of Eifel tower appear 960 pixel distance?

960 pix

21.8 mm
1280
324 m pix
ccd size

Y himg Y
fm ? yimg = f = fm
1500 m Z hccd Z

1280 324
960 = fm → fm = 0.0757m
0.0218 1500
Property of University of Pennsylvania, Jianbo Shi
37
 Exercise
What f to make the height of Eifel tower appear 960 pixel distance?

960 pix

21.8 mm
1280
324 m pix
ccd size

fm = 50 mm

Z?

Property of University of Pennsylvania, Jianbo Shi

38
 Exercise
What f to make the height of Eifel tower appear 960 pixel distance?

960 pix

21.8 mm
1280
324 m pix
ccd size

Y himg Y
fm = 50 mm
yimg = f = fm
Z? Z hccd Z

Property of University of Pennsylvania, Jianbo Shi

39
 Exercise
What f to make the height of Eifel tower appear 960 pixel distance?

960 pix

21.8 mm
1280
324 m pix
ccd size

Y himg Y
fm = 50 mm
yimg = f = fm
Z? Z hccd Z

1280 324
960 = 0.05 → Z = 990.826m
0.0218 Z 40
Property of University of Pennsylvania, Jianbo Shi
 Exercise
What Zp to make the height of Eifel tower appear twice of the person?

he
hp

324 m

1.7 m
fm = 50 mm

Z p?
1500

Property of University of Pennsylvania, Jianbo Shi

41
 Exercise
What Zp to make the height of Eifel tower appear twice of the person?

he
hp

324 m

s.t.
1.7 m Ye Yp
he = f hp = f hp =
he
fm = 50 mm Ze Zp 2

Yp Ye
Z p?
f =f →
1500 Zp 2Ze

Property of University of Pennsylvania, Jianbo Shi

42
 Where Was I?

324 m

0.9 m
670 pix
fm = 50 mm

Z1 ?
Z2 ?
Y himg Y1 himg Y1 1280 0.9
250 y1 = f = fm → Z1 = fm = 0.05 = 8.03m
pix Z hccd Z1 hccd y1 0.0218 250

Circa 1984

Property of University of Pennsylvania, Jianbo Shi

43
 Where Was I?

324 m

0.9 m
670 pix
fm = 50 mm

Z1
Z2

Y h Y h Y 1280 0.9
y1 = f = fm img 1 → Z1 = fm img 1 = 0.05 = 8.03m
250 Z hccd Z1 hccd y1 0.0218 250
pix
Y2 himg Y2 himg Y2 1280 324
Circa 1984 y2 = f = fm → Z2 = fm = 0.05 = 1079m
Z2 hccd Z2 hccd y2 0.0218 670

Property of University of Pennsylvania, Jianbo Shi

44
 Where Was I?
Y2 h Y h Y 1280 324
y2 = f = fm img 2 → Z2 = fm img 2 = 0.05 = 1079m
Z2 hccd Z2 hccd y2 0.0218 670

670 pix

200m
400m
600m
250 800m
pix 1000m
Circa 1984

Property of University of Pennsylvania, Jianbo Shi

45
Where Was I?

800 m

Property of University of Pennsylvania, Jianbo Shi

46
 Focal Length

Shorter focal length, larger field of view!

Property of University of Pennsylvania, Jianbo Shi

47
 Focal Length

Property of University of Pennsylvania, Jianbo Shi

48
 Focal Length

Property of University of Pennsylvania, Jianbo Shi

49
Focal Length

Property of University of Pennsylvania, Jianbo Shi

50
 Dolly(Vertigo
Dolly Zoom ZoomEffect)
(Jaws 1975)

Property of University of Pennsylvania, Jianbo Shi

51
 Dolly Zoom
Given focal length (fm=100mm),
what Z100 to make the height of the person remain the same as fm=50mm?

hp Hp

f100 = 100 mm

Z 100 ?
Z 50 =157.41m

Property of University of Pennsylvania, Jianbo Shi

52
 Dolly Zoom
Given focal length (fm=100mm),
what Z100 to make the height of the person remain the same as fm=50mm?

hp Hp

Y Y
h50 = f50 h100 = f100
f100 = 100 mm Z50 Z100

Z 100 ? s.t. h100 = h50

Z 50 =157.41m

Property of University of Pennsylvania, Jianbo Shi

53
 Dolly Zoom
Given focal length (fm=100mm),
what Z100 to make the height of the person remain the same as fm=50mm?

hp Hp

Y Y
h50 = f50 h100 = f100
f100 = 100 mm Z50 Z100

Z 100 ? s.t. h100 = h50

Z 50 =157.41m
f100 100
Z100 = Z50 Z100 = 157.41= 314.8m
f50 50
Property of University of Pennsylvania, Jianbo Shi
54