Multimedia and Computer Graphics

Lecture 4 – Digital Video

Representation
 Color and Visual System
 Color refers to how we
perceive a narrow band of
electromagnetic energy
 source, object, observer

 Visual system transforms

light energy into sensory
experience of sight
 Human Visual System
 Eyes, optic nerve, parts of the brain
 Transforms electromagnetic energy

 Image Formation
 cornea, sclera, pupil,
iris, lens, retina, fovea
 Transduction
 retina, rods, and cones
 Processing
 optic nerve, brain
 Tri-stimulus Theory
 3 types of cones (6 to 7 million of them)
 Red = L cones, Green = M cones, Blue = S cones
 Ratio differentiates for each person
 E.g., Red (64%), Green (32%), rest S cones
 E.g., L(75.8%), M(20%), rest S cones
 E.g., L(50.6%), M(44.2%), rest S cones
 Source of information:
 See ‘cone cell’ in Wikipedia
 www.colorbasics.com/tristimulus/index.php
 Each type most responsive to a narrow band
 red and green absorb most energy, blue the least
 Light stimulates each set of cones differently, and the ratios produce
sensation of color
 Color Perception (Color Theory)
 Hue (color degree)
Hue Scale
 distinguishes named colors, e.g.,
RGB
 dominant wavelength of the light

 Saturation

Original
 Perceived intensity of a specific color
 how far color is from a gray of equal
intensity
 Brightness (lightness)

Saturation
 perceived intensity

lightness
Source: Wikipedia
 Visual Perception: Resolution and Brightness
 Spatial Resolution (depends on: )
 Image size
 Viewing distance
 Brightness
 Perception of brightness is higher than
perception of color
 Different perception of primary
colors
 Relative brightness:

green:red:blue=59%:30%:11%
 B/W vs. Color

Source: wikipedia
Visual Perception: Temporal
Resolution
 Effects caused by inertia of human eye
 Perception of 16 frames/second as continuous
sequence
 Special Effect: Flicker
Temporal Resolution
 Flicker
 Perceived if frame rate or refresh rate of
screen too low (<50Hz)
 Especially in large bright areas
 Higher refresh rate requires:
 Higher scanning frequency
 Higher bandwidth
Visual Perception Influence
 Viewing distance
 Display ratio (width/height – 4/3 for
conventional TV)
 Number of details still visible
 Intensity (luminance)
Television History
 1927, Hoover made a
speech in Washington
while viewers in NY
could see, hear him

 AT&T Bell Labs had the

first “television”
 18 fps, 2 x 3 inch
screen, 2500 pixels
 Television Concepts
 Production (capture)
 2D array of light energy to
electrical signals
 signals must adhere to known, structured formats
 Representation and Transmission
 popular formats include NTSC,
PAL, SECAM
 Re-construction
 CRT technology and raster scanning
 display issues (refresh rates, temporal resolution; frame rate)
 relies on principles of human visual system
Video Representations
 Composite
 NTSC - 6MHz (4.2MHz video), 29.97 fps
 PAL - 6-8MHz (4.2-6MHz video), 25 fps
 Component
 Maintain separate signals for color
 Color spaces
 RGB, YUV, YCRCB, YIQ
CS 414 - Spring 2009
CS 414 - Spring 2009
CS 414 - Spring 2009
CS 414 - Spring 2009
CS 414 - Spring 2009
 Color Coding: YUV
 PAL video standard
Y is luminance
 UV are chrominance

 YUV from RGB

U-V plane at Y=0.5
Y = .299R + .587G + .114B
U = 0.492 (B - Y)
V = 0.877 (R - Y)

Source: wikipedia
CS 414 - Spring 2009
CS 414 - Spring 2009
CS 414 - Spring 2009
CS 414 - Spring 2009
CS 414 - Spring 2009
CS 414 - Spring 2009
YCrCb
 Subset of YUV that scales
and shifts the chrominance
values into range 0..1

Y = 0.299R + 0.587G + .114B

Cr = ((B-Y)/2) + 0.5
Cb = ((R-Y)/1.6) + 0.5
 YIQ
 NTSC standard

 YIQ from RGB

Y = .299R + .587G + .114B
I = .74 (R - Y) - .27 (B - Y) YIQ with Y=0.5
Q = 0.48 (R - Y) + 0.41 (B -
Y)

Source: wikipedia
NTSC Video
 525 scan lines per frame; 29.97 fps
 33.37 msec/frame (1 second / 29.97 frames)
 scan line lasts 63.6 usec (33.37 msec / 525)
 aspect ratio of 4/3, gives 700 horizontal pixels

 20 lines reserved for control information at the

beginning of each field
 so only 485 lines of visible data
NTSC Video
 Interlaced scan lines divide each frame
into 2 fields, each of which is 262.5 lines
 phosphors in early TVs did not maintain
luminance long enough (caused flicker)
 scanning also interlaced; can cause visual
artifacts for high motion scenes
HDTV
 Digital Television Broadcast (DTB) System
 Twice as many horizontal and vertical
columns and lines as traditional TV
 Resolutions:
 1920x1080 (1080p) – Standard HDTV
 Frame rate: options 50 or 60 frames per
second
 Pixel Aspect Ratio

• Computer Graphics parameter

• Mathematical ratio describing horizontal length of a pixel to its vertical height
• Used mainly in digital video editing software to properly scale and render video
Into a new format

Source: wikipedia
 Pixel aspect rati
Resolution o
Pixels Aspect Ratio Video Format Description
(WxH)
(Standard 16:9)
1024×768 786,432 16:9 4:3 720p/XGA Used on HDTV

Used on
1280×720 921,600 16:9 1:1 (square) 720p/WXGA
Digital television,
720p/WXGA -
Approx. 1:1 HDTV Used on LCD
1366×768 1,049,088 16:9
(square) standard HDTV displays
format
Used on PDP
1024×1080 1,105,920 16:9 15:8 1080p
displays HDTV
Used on PDP
1280×1080 1,382,400 16:9 3:2 1080p
HDTV displays
1080p - HDTV Used on all types
1920×1080 2,073,600 16:9 1:1 (square) standard of HDTV
format technologies

3840x2160 8,294,400 16:9 1:1 (square) 2160p Quad HDTV,

HDTV
 Interlaced and/or progressive formats
 ConventionalTCs – use interlaced formats
 Computer displays (LCDs) – use progressive
scanning
 MPEG-2 compressed streams
 In Europe (Germany) – MPEG-4
compressed streams
 Aspect Ratio and Refresh Rate
 Aspect ratio
 Conventional TV is 4:3
(1.33)
 HDTV is 16:9 (2.11)
 Cinema uses 1.85:1 or
2.35:1
 Frame Rate
 NTSC is 60Hz interlaced
(actually 59.94Hz)
 PAL/SECAM is 50Hz interlaced
 Cinema is 24Hz non-interlaced

Source: wikipedia
SMPTE Time Codes
 Society of Motion Picture and Television Engineers defines time
codes for video
 HH:MM:SS:FF
 01:12:59:16 represents number of pictures corresponding to
1hour, 12 minutes, 59 seconds, 16 frames
 If we consider 30 fps, then 59 seconds represent 59*30

frames, 12 minutes represent 12*60*30 frames and 1 hour

represents 1*60*60*30 frames.
 For NTSC, SMPTE uses a 30 drop frame code
 increment as if using 30 fps, when really NTSC has only 29.97fps
 defines rules to remove the difference error
Take Home Exercise
 Given a SMPTE time stamp, convert it
back to the original frame number
 e.g., 00:01:00:10