Image Processing : Basics, Challenges & Perspectives

Ecole Centrale de Paris; Mai 12 Juin 12 2006

Outline
Digital Image Processing Image Generation Image Perception Image Acquisition Color Images

A. Digital Image Processing

Digital image: f(x,y) More than human perception (or EM spectrum):
Ultrasound, electron-microscopy, computergenerated images

Related fields: computer vision, image analysis / understanding Better characterization:

Low-level: both input and output are images
Noise reduction, contrast enhancement, image sharpening

Mid-level: input - image, output extracted attributes

Segmentation, classification

High level: making sense of the images

Vision-related tasks Lilac flower petals

A bit of history
1960s: computers powerful enough + space program 1964: pictures from the Moon (JPL, Pasadena, CA) Other fields:
Remote Earth resources observation Medical Image processing: 1970s CAT / CT (Hounsfield, Cormack 1979 Nobel Prize) Aerial and satellite images Archeology Physics Machine perception: inspection, product assembly, character recognition, ...etc

B. Image generation
Electromagnetic (EM) energy spectrum Acoustic energy Ultrasonic energy Electric energy (e.g.: electron microscope)

Electromagnetic energy spectrum

http://www.desy.de/pr-info/desyhome/gfx/presse/fotos/hasylab/150dpi/EM-Spektrum_en.jpg

Gamma ray imaging

Nuclear Medicine
Radioactive isotope injected to emit gamma rays while decaying gamma ray detectors e.g.: Bone scans, PET scans

Bone scan

Astronomy
Cygnus Loop, nuclear reactors (natural radiation!)

http://www.physics.ubc.ca/~mirg/home/tutorial/pics/bone_scan.jpg http://www.doemedicalsciences.org/pubs/sc0033/images2/DOESC0033sc_10_0002.jpg

PET scan

X-ray Imaging
Medical diagnostics, astronomy Examples:
X-ray: X-ray source and film sensitive to x-ray energy; intensity depends on the absorption passing through the subject Contrast enhancement radiography
Angiography: X-ray contrast medium injected with catheter through blood vessels

Computerized axial tomography (CAT)

3D capability by assembling slices
http://www.xray.hmc.psu.edu/rci/contents_1.html http://www.gemedicalsystems.com/rad/xr/education/index.html http://zoot.radiology.wisc.edu/~block/Med_Gallery/ia_dsa.html http://www.ddc.musc.edu/ddc_pro/pro_development/case_studies/images/case003a.jpg

Imaging in the Ultraviolet Band

Lithography, industrial inspection, microscopy lasers, biology, astronomy, ... Fluorescence microscopy
Study materials that can be made to fluoresce (primary / secondary fluorescence)

Cultured epithelial cells triple stained for the nucleus (blue), microtubules (green), and actin (red). Images were acquired with a 20X objective (left) and a 100X objective (right) http://www.itg.uiuc.edu/technology/atlas/microscopy/images/100x_fluor.jpg

Imaging in the Visible and Infrared Bands

Light microscopy, astronomy, remote sensing, industry, law enforcement, ... Examples:
Monitoring environmental conditions Weather observation Visual inspection: e.g.: detecting anomalies Licence plate reading, ....

LANDSET: Multi-spectral Images

Band 1

Band 2

Band 3

Band 4

Band 5

Band 6

Band 7

True color image: band 1 displayed in blue, band 2 displayed in green, and band 3 displayed in red https://zulu.ssc.nasa.gov/mrsid/tutorial/Landsat%20Tutorial-V1.html

Microwave Band Imaging

Radar: any region at any time
Provides own illumination and takes snapshot Antenna and computers to recorded the image Result: Microwave energy reflected back

Imaging in radio band

Medicine, astronomy Magnetic Resonance Imaging (MRI)
Magnet -- Radio waves in short pulses Images in any plane!
General Electric 3 Tesla Imager

http://www.gemedicalsystems.com/rad/mri/images/med/3t/

Others...
Acoustic imaging
Geology (mineral and oil exploration), industry, medicine (obstetrics) Ultrasound images
high frequency sound pulses through body they hit tissue boundaries reflected waves picked up by probe display distances from probe to tissue / organ boundaries

http://www.alabamababy.com/images/ http://www.pregnancycheck.com/img/pregnancy-ultrasound-17-weeks.jpg

Others...
electron microscopy
Use focused beam of electrons (instead of light) to image specimens; capable of very high magnification (10 000x or more!)

synthetic imaging (generated by computers)

Fractals (iterative reproduction of a basic pattern by some mathematics rule) 3D modeling: medical training, special effects, ...

C. Human Visual Perception

Eye anatomy
Lens: absorbs ~8% of visible light spectrum Retina: light receptors spread over
Cones: around fovea, ~67million / eye; highly sensitive to color; own nerve connections Rods: 75-150 million; whole retinal surface; general picture of the FOV; sensitive to low levels of illumination

Fovea: circular indentation in the retina (square sensor array)

http://www.photo.net/photo/edscott/humaneye.gif

Image Formation in the Eye

Flexible lens
Flattened for distant objects; thickened for near objects

Retinal image is reflected mostly in the area of the fovea Relative excitation of light receptors radiant energy into electrical pulses

Brightness Adaptation & Discrimination

Light intensity levels to which we can adapt: 10^10 Not a simultaneous operation! Brightness adaptation level: given sensitivity level for a set of conditions Light intensity changes can be detected too Perceived brightness does not merely depends on intensity; human visual system overshoots / undershoots around the boundary of regions of different intensities Simultaneous contrast

Mach band

Optical illusions

Count the black dots!

http://refwd.ourmemorybox.com/images/rbCXmF7H03.jpg

D. Light and EM spectrum

1666.Newton: light through a glass prism continuous spectrum of colors [violet,, red] visible light: small portion of EM spectrum EM description: wavelength (), frequency (), or energy = c/
c = 2.998*10^8m/s (speed of light)

E = h*
h: Plancks const.

.43 m --- .79 m

Image Sensing & Acquisition

Illumination energy
Single sensor
Photodiodes, laser source

digital images

(Circular) sensor strip

Flat bed scanners; CAT; PET

Sensor array
Complete image can be obtained by focusing the energy pattern onto its surface

Image Formation
f : ( x, y )
2

Proportional to radiated E from physical source [0, ] Incident illumination; reflected illumination

f ( x, y ) = i ( x, y ) r ( x, y )

Image Sampling, Quantization

Continuous sensed data Example (on board) digital form

Sample in spatial coordinates and amplitude sampling & quantization

Sampling: depends on sensor arrangement

Single sensing Sensing strip

Representing Digital Images

A 2D image f(x,y) of size: MxN (0,0) y [0,N-1]

x [0,M-1]

pixel

Digitization: M, N, L (# discrete gray levels) L = 2^k dynamic range; ~contrast To store an image: b = MxNxk bits

Spatial and Grayscale Resolution

Sampling spatial resolution
Smallest discernible detail

Gray-scale resolution 8, 16 bit images...

Aliasing
Shannon sampling theorem
For band limited functions, if they are sampled at the rate equal or greater than twice of its highest frequency; original function can be recovered completely from its samples Under-sampled aliasing Sampling rate; number of samples taken per unit distance

To avoid aliasing: reduce its high frequency components by blurring prior to sampling

Zooming and Shrinking

Over- and under-sampling applied to digital images NN-interpolation; bilinear interpolation

Pixel Relationships
Adjacency relationships
(0,0) y

4-adjacent 8-adjacent

Definitions
Digital path & length Connected Connected component: set of pixels.... Connected set: set with only one connected component Region Boundary Edge: local concept; graylevel discontinuity
(0,0) y

Distance functions
Distance function: D( p, q ) 0 (" = "iff p = q )

Examples
Euclidean City-block

D ( p, q ) = D ( q, p ) D ( p, z ) D ( p, q ) + D ( q, z )

( x s ) 2 + ( y t ) 2 D ( p, q ) =

D ( p, q ) = x s + y t

Chessboard

D( p, q ) = max ( x s , y t )

E. Color Image Processing

Humans can discern thousands of color shades vs. two dozens shades of gray Areas
Full-color Pseudo-color

Sunlight beam through prism continuous spectrum of colors (violet, blue, green, yellow, orange, red) Color we see: nature of light reflected from the object

Chromatic light: EM spectrum ~400-700 m Cones are responsible for color vision: 6-7 million R(65%)G(33%)B(2%) sensitivity primary colors: RGB (adding) secondary colors: magenta, cyan, yellow Color differentiation: brightness (chromatic notion), hue (dominant wavelength), saturation (relative purity / amount of white light mixed with hue) = brightness and chromaticity

e.g.: green: reflects light with wavelengths in 500-570 m and absorbing all the rest

Color Models
Specification of colors according to some standards For digital image processing
RGB color monitors / video cameras CMY; CMYK color printing HSI closest to human perception (separate gray-scale and color info)

RGB
Useful for color generation Color cube -- Cartesian system
Corners: primary and secondary colors Grayscale: BW diagonal All values normalized in [0,1]

Pixel depth: number of bits used to represent each pixel in the RGB space
e.g.: full-color image: 24-bit RGB color image

Grayscale values

Safe RGB/Web Colors

subset of 216 colors that can be reproduced faithfully and independently of hardware often, hexadecimal representation is used

CMY (CMYK) Color Models

For color printing Secondary colors of light [C M Y] = [1 1 1] [R G B] CMYK: in order to have true black instead of muddy black produced from CMY combination

HSI
Color description; closest to human perception Hue pure color Saturation amount of dilution by white light Brightness subjective measure; ~intensity

RGB HSI (below) HSI RGB (see Book)

if B G H = ( 360 ) if B > G S = 1 I= 3 [ min( R, G, B)] ( R + G + B)

1 ( R G ) + ( R _ B ) 1 2 = cos 1/ 2 ( R G )2 + ( R B )( G B )

1 ( R + G + B) 3

End
TP1 with Martin