Digital Image Processing

(UST 2007 Fall)

Sang Chul Ahn asc@imrc.kist.re.kr

Digital Image Fundamentals

Elements of Visual Perception

Although the digital image processing field is built on a foundation of mathematical and probabilistic formulations, human intuition and analysis play a central role in the choice of techniques Basic understanding of human visual perception is important The mechanics and parameters related to how images are formed in the eye The physical limitations of human vision in terms of factors that also are used in digital image processing The factors how human and electronic imaging compare in terms of resolution and ability to adapt to changes in illumination

Structure of the Human Eye

Nearly a sphere with an average diameter of 2cm 3 membranes enclose the eye
Cornea() & Sclera() Choroid()
ciliary body, iris diaphragm

Retina ()

Structure of the Human Eye

Cornea: transparent sclera: opaque Choroid
contains blood vessels heavily pigmented and reduce the amount of light entering the eye and backscatter within the eye At anterior, Ciliary body & iris diaphragm Iris: control the amount of light, 2~8mm Lens:
60~70% water, 6% fat, and protein, slightly yellow, absorbs approximately 8% of the visible light spectrum

Structure of the Human Eye

Retina
Image is focused on retina Two classes of receptors: cones and rods Cones
6~7million Located primarily in the fovea Color sense One nerve for one cone: resolve fine detail Cone vision called photopic or bright-light vision

Rods
75~105million Distributed over the retinal surface One nerve for several rods Sensitive to low levels of illumination Rod vision called scotopic or dim-light vision

Structure of the Human Eye

Blind spot Receptor distribution Fovea
Circular indentation of about 1.5mm in diameter Density of cones is approximately 150,000 cones/mm2 1.5mm x 1.5mm square 337,000 cones

Image Formation
The shape of lens is controlled by tension of the ciliary body Lens flattened for distant objects The distance between the center of the lens and the retina(focal length) varies 17mm~14mm

Bright Adaptation
Bright adaptation range is on the order of 1010 Subjective brightness is a logarithmic function of the light intensity The range of photopic vision is about 106 The visual system cannot operate over the range simultaneously It is done by changes in its overall sensitivity Bright adaptation

Discrimination
Weber ratio
Ic/I, where Ic is the increment of illumination discriminable 50% of the time Small value represents good brightness discrimination

poor by rods good by cones

Discrimination
Perceived brightness is not a simple function of intensity The visual system tends to undershoot or overshoot around the boundary of regions of different intensities Mach bands Although the intensity of the stripes is constant, we actually perceive a brightness pattern that is strongly scalloped

Discrimination
A regions perceived brightness does not depend simply on its intensity simultaneous constrast All the center squares have exactly the same intensity. However, they appear to the eye to become darker as the background gets lighter

Optical illusion
The eye fills in nonexisting information or wrongly perceives geometrical properties of objects

Electromagnetic Spectrum
In 1666, Sir Isaac Newton discovered that when a beam of sunlight is passed through a glass prism, the emerging beam of light is not white but consists instead of a continuous spectrum of colors ranging from violet to red

Physics of Light and EM Spectrum

Frequency( or f ) vs. wavelength() f=c/ c: 2.998x108 m/s (the speed of light) Energy (unit: electron-volt) h: Plancks constant E = h Energy is proportional to frequency Higher-frequency electromagnetic phenomena carry more energy per photon Gamma rays are dangerous to living organisms

Physics of Light and EM Spectrum

Light that is void of color is called achromatic or monochromatic light The only attribute of such light is its intensity, or amount Chromatic light spans the electomagnetic energy spectrum from approximately 0.43 to 0.79um Three basic quantities are used to describe the quality of a chromatic light source: radiance, luminance, and brightness Radiance(unit: W) is the total amount of energy that flows from the light source Luminance(unit: lm, lumens) gives a measure of the amount of energy an observer perceives from a light source
ex) Light emitted from a source operating in the far infrared region of the spectrum could have high radiance, but an observer would hardly perceive it(zero luminance)

Brightness is a subjective descriptor of light perception that is practically impossible to measure. It embodies the achromatic notion of intensity

Physics of Light and EM Spectrum

Near-infrared: the part of the infrared band close to the visible spectrum Far-infrared: the part of the infrared band close to the microwave band It is important to note that the wavelength of an electromagnetic wave required to see an object must be of the same size as or smaller than the object. For example, a water molecule has a diameter on the order of 10-10 m. Thus, to study molecules, we would need a source capable of emitting in the far ultraviolet or soft X-ray region

Image Sensors

Image Sensing

CCD vs. CMOS

CCD (charge-coupled device) and CMOS (complimentary metal-oxide semiconductor)

A CCD is like a threedecker sandwich. The bottom layer contains the photosites. Above them is a layer of colored filters that determines which color each site records. Finally, the top layer contains microlenses that gather light.Courtesy of Fujifilm.

CCD vs. CMOS

In a CCD device, the charge is actually transported across the chip and read at one corner of the array. An analog-to-digital converter turns each pixel's value into a digital value. In most CMOS devices, there are several transistors at each pixel that amplify and move the charge using more traditional wires. The CMOS approach is more flexible because each pixel can be read individually. CCDs use a special manufacturing process to create the ability to transport charge across the chip without distortion. This process leads to very high-quality sensors in terms of fidelity and light sensitivity. CMOS chips, on the other hand, use traditional manufacturing processes to create the chip. CMOS sensors, traditionally, are more susceptible to noise. Because each pixel on a CMOS sensor has several transistors located next to it, the light sensitivity of a CMOS chip tends to be lower. Many of the photons hitting the chip hit the transistors instead of the photodiode.

CCD vs. CMOS

http://www.axis.com/edu/axis/index.htm

http://www.pictureline.com/newsletter/2004/september/pixels.html

Simple Image Model

Monochrome(Gray) Image : f(x,y)
f(x,y) = intensity value at coordinates (x,y) ; f(x,y) is energy 0 < f(x,y) <

Light Transport
Simple image formation
f(x,y) = i(x,y)r(x,y) 0 < i(x,y) < 0 < r(x,y) < 1 ; illumination ; reflectance

In real situation
Lmin l (=f(x,y)) Lmax

Sampling & Quantization

Image sampling
Digitization of spatial coordinates (x,y)

Quantization
Amplitude digitization

The quality of a digital image is determined to a large degree by the number of samples and discrete gray levels used in sampling and quantization f(x,y) f(0,0) f(1,0) f(M-1,0) f(0,1) f(1,1) f(M-1,1) : Continuous image f(0,N-1) f(1,N-1) f(M-1,N-1)
Digital image

Sampling & Quantization

Sampling & Quantization

What is Digital Image?

Digital image : x,y,f(x,y), three values are all discretized Pixel : Image elements, Picture elements, Pels
x Origin (0,0)

Pixel y

What is Digital Image?

Digital Color Image
R(x,y), G(x,y), B(x,y), three values are assigned at the same pixel location HSI, YIQ, CMY representation We can use more than 3 values for a pixel such as CMYK representation

Comparison
f(x,y) : 2D still image f(x,y,z) : 3D object f(x,y,t) : Video or Image Sequence f(x,y,z,t) : moving 3D object

What is Digital Image?

Meaning
brightness(luminance) or color of an object
TV camera, scanner

absorption characteristics of objects (especially bodies)

X-ray imaging, Ultrasonic imaging, CT

distance between objects and measuring instrument

sonar imaging, radar imaging, range camera

temperature of an object
IR(infrared) camera

What is Digital Image?

Spatial Resolution (x,y)
Spatial resolution is the smallest discernible detail in an image A line pair : a line and its adjacent space A widely used definition of resolution is the smallest number of discernible line pairs per unit distance ex) 100 line pairs/mm But, unit distance or unit area is omitted in most cases

Spatial Resolution

Spatial Resolution

cf) Full HD

Screen resolution [wikipedia]

What is Digital Image?

Gray-level Resolution
Gray-level resolution is the smallest discernible change in gray level (but, highly subjective!) Due to hardware considerations, we only consider quantization level Usually an integer power of 2. The most common level is 28=256 However, we can find some systems that can digitize the gray levels of an image with 10 to 12 bits of accuracy.

Gray-level Resolution

Image displayed in 256, 128, 64, 32, 16, 8, 4, 2 gray-levels

Storage
For MxN image with L(=2k) discrete gray level
The number, b, of bits required to store the image is b = MNk ex1) 1024x1024x8bit = 1Mbytes

Isopreference curves
Relation between subjective image quality and resolution Tested by images with low/medium/high detail Result
A few gray levels may be needed for high detailed image Perceived quality in the other two image categories remained the same in some intervals in which the spatial resolution was increased, but the number of gray levels actually decreased

Aliasing
Shannon sampling theorem
if the function is sampled at a rate equal to or greater than twice its highest frequency, it is possible to recover completely the original function from its samples if the function is undersampled, then a phenomenon called aliasing corrupts the sampled image The corruption is in the form of additional frequency components being introduced into the sampled function. These are called aliased frequencies Nyquist freq. = 0.5 x sampling rate

Aliasing
Except for a special case, it is impossible to satisfy the sampling theorem in practice The principal approach for reducing the aliasing effects on an image is to reduce its high-frequency components by blurring the image prior to sampling However, aliasing is always present in a sampled image The effect of aliased frequencies can be seen under the right conditions in the form of so-called Moir patterns

Aliasing in Images
When we view a digital photograph, the reconstruction (interpolation) is performed by a display or printer device, and by our eyes and our brain. Typical aliasing in images can be seen in the form of Jaggies The checkers should become smaller as the distance from the viewer increases. However, the checkers become larger or irregularly shaped when their distance from the viewer becomes too great Anti-aliasing is the technique of minimizing the distortion artifacts known as aliasing when representing a high-resolution signal at a lower resolution

(a)Aliased (b),(c)Anti-aliasing is applied

Neighbors of a Pixel
N4(p) : 4-neighbors of pixel p(y,x)
{ p(y+1,x), p(y-1,x), p(y,x+1), p(y,x-1)}

ND(p) : diagonal neighbors of pixel p(y,x)

{ p(y+1,x+1), p(y-1,x-1), p(y-1,x+1), p(y+1,x-1)}

N8(p) : 8-neighbors of pixel p(y,x)

N8(p) = N4(p) ND(p)

Some of the neighbors of pixel p lie outside the digital image if the pixel p is on the border of the image

N4
p(x,y )

N8 ND

Adjacency
V : set of gray-level values to define adjacency
ex) V={1} ; binary image V={32,33,,63,64} ; gray image

4-adjacency : Two pixels p, q with values from V are 4adjacency if q is in the set N4(p) 8-adjacency : Two pixels p, q with values from V are 8adjacency if q is in the set N8(p) m-adjacency(mixed adjacency) : Two pixels p and q with values from V are m-adjacency if
i) q is in N4(p), or ii) q is in ND(p) and the set N4(p) N4(q) has no pixels whose values are from V

Adjacency

8-adjacency

m- adjacency

Path
A (digital) path(or curve) from pixel p at (x,y) to pixel q at (s,t) is a sequence of distinct pixels with coordinates (x0,y0), (x1,y1), ,, (xn,yn) where (x0,y0) =(x,y), (xn,yn)=(s,t), and pixel (xi,yi) and (xi-1,yi-1) are adjacent for 1 i n n is the length of the path If (x0,y0) =(xn,yn), the path is a closed path The path can be defined 4-,8-,m-paths depending on adjacency type

Connectivity
Let S be a subset of pixels in an image. Two pixels p and q are said to be connected in S if there exists a path between them consisting entirely of pixels in S For any pixel p in S, the set of pixels that are connected to it in S is called a connected component of S. If it only has one connected component, then set S is called a connected set. Let R be a subset of pixels in an image. We call R a region of the image if R is a connected set The boundary of a region R is the set of pixels in the region that have one or more neighbors that are not in R

Distance Measures
Let pixels be p=p(x,y), q=q(s,t), z=z(u,v) D() is a distance function or metric if
(a) D(p,q) 0 (D(p,q)=0 iff p=q) (b) D(p,q) = D(q,p) (c) D(p,z) D(p,q) + D(q,z)

Distance Measures
Euclidean distance
De(p,q) = [(x-s)2 + (y-t)2]1/2

D4 distance (city-block distance)

D4(p,q) = |x-s| + |y-t|

2 2 1 2 2 1 0 1 2 2 1 2 2

Distance Measures
D8 distance (chessboard distance)
D8(p,q) = max(|x-s|, |y-t|)

2 2 2 2 2

2 1 1 1 2

2 1 0 1 2

2 1 1 1 2

2 2 2 2 2

Arithmetic/Logic Operations
Arithmetic operation
Addition: p+q Subtraction: p-q Multiplication: pxq Division: p q

Logic Operation
AND: p AND q (p. q) OR: p OR q (p + q) COMPLEMENT: NOT q ( q )

Logic Operations

A-B

A XOR B NOT(A) A AND B A OR B

2D Image transformation
Translation x x = y y Scaling x = y Rotation x = y + tx ty x y sin cos x y

Sx 0 0 Sy cos -sin

Order of application is important !!

Origin

2D Image transformation

Original

30deg. rotation y axis translation +100

Scaling(Sx=2, Sy=2) 30deg. rot. y axis trans. +100 y axis trans. +100 30deg. rot.