SGM4: STUDY GUIDE FOR MODULE 4

MODULE 4- BASIC CONCEPTS OF IMAGE

FORMATION
SUBTOPIC 1- DIGITAL IMAGE FORMATION AND
DISPLAY

IMAGES

• Images are a description of how a parameter

varies over a surface

• Standard visual images result from light

intensity variations across a two-dimensional
plane.

• Images can be formed from:

• temperature (e.g., of an integrated

circuit)

• blood velocity in a patient's artery

• x-ray emission (e.g., from a distant

galaxy)

• ground motion during an earthquake

DIGITAL IMAGE STRUCTURE

• Pixel – (contraction of the phrase: picture IMAGE SIZE AND RESOLUTION

element) individual samples in an image
• A "typical" digital image is composed of about
• Images have their information encoded in the 500 rows by 500 columns
spatial domain
• 256 gray levels (quantization levels) is common
• Features in images are represented by
• single byte per pixel
edges, not sinusoids!
• brightness step size of 1/256 (0.39%) is
• For grayscale images, each pixel is a single
smaller than the eye can perceive
number:
• Value of each pixel are defined by sampling
• 0 is black, 255 is white, intermediate
aperture and sampling grid
values are shades of gray
• The region of the continuous image that
contributes to the pixel value is called
the sampling aperture.
 • The spacing in between samples is
referred to as the sampling grid.

BIT DEPTHS OF COMMON IMAGE TYPES

FOCUSING BY LENS

• The lens assembly focuses light pattern onto the

imaging sensor.
IMAGE COORDINATES

• Each lens system has two adjustable

parameters: focus and iris diameter

• Adjustable iris is necessary because the range of

light intensities in environment is much larger
than can be directly handled by light sensors

CAMERA AND HUMAN EYE • Adjustable distance (focus) is necessary for

proper mapping of the information onto the
Both camera and human eye are based on two major imaging sensor
components:
HUMAN EYE’S RETINA
• a lens assembly, and
Two types of cells:
• an imaging sensor
1.) rods - specialized in sensing luminance and operating
with low light

2.) cones- (red sensitive, green sensitive, and blue

sensitive) specialized in distinguishing color, but can only
operate when a reasonable amount of light is present.
SPECTRAL RESPONSE OF THE EYE

COLOR IMAGES

COMPOSITE VIDEO SIGNAL

OPERATION OF THE CCD

• The imaging sensor transforms the pattern of

light into a video signal

• The most commonly used camera imaging

sensor is the charge coupled device or CCD (an
integrated circuit that replaced most vacuum
tube cameras in the 1980s)
OTHER IMAGE ACQUISITION AND DISPLAY
• Line Scanning (e.g., fax machines, airport x-ray
baggage scanners)

• Point-by-point scanning (e.g., scanning probe

microscope)

• Halftone display

• Each pixel is formed from many

• Temporal sampling is done during integration individual dots, with each dot being
period; completely black or completely white

• Pattern of light striking the CCD is transferred BRIGHTNESS AND CONTRAST

into a pattern of charge within the CCD “wells”
• Brightness - overall lightness or darkness of the
• Three phase readout- a Every third electrode is image
connected together;
• Contrast - difference in brightness between
• Accumulated electrons in each “well” are pushed objects or regions
to the output amplifier
• Example of intensity modifying functions:
 GRAYSCALE TRANSFORM

EXAMPLE (TEST IMAGE)

HISTOGRAM EQUALIZATION FOR GRAYSCALE

TRANSFORMATION

EXAMPLE (BRIGHTNESS ADJUSTMENT)

EXAMPLE (CONTRAST ADJUSTMENT)

THREE VERY DIFFERENT IMAGES WITH • Warping takes the original image (a two-
IDENTICAL HISTOGRAMS dimensional array) and generates a warped image
(another two-dimensional array)

• Transferring each pixel from the

original to the warped image

• The row and column that the pixel comes from

in the original image is called the comes-from
address
INVERTING IMAGES
SUBPIXEL INTERPOLATION

Example:

• Inputs: row = 10, column = 20

• Outputs: comes-from row = 20.2, comes-from

column = 14.5.

IMAGE BINARIZATION

REFERENCES

• Digital Signal Processing: A Practical Guide for

Engineers and Scientists (by S. Smith)

• Principles of Digital Image Processing

(by W. Burger and M. Burge)

• Digital Image Processing Using MATLAB

IMAGE WARPING (by R. Gonzalez, R. Woods and S. Eddins)

• ECE ELECTIVE 2 (ADVANCED DIGITAL

SIGNAL PROCESSING) Notes (Ronald
Pascual, Ph. D.)

SUBTOPIC 2- BASIC IMAGE PROCESSING

TECHNIQUES

LINEAR IMAGE PROCESSING

Based on two conventional DSP techniques:

1. Convolution

2. Fourier Analysis

Linear filtering applications:

 • sharpening edges of objects

• reducing random noise

• correcting for unequal illumination

Techniques for reducing execution time:

• convolution by separability

• FFT convolution

LINEAR IMAGE FILTERING (BY CONVOLUTION)

• Linear image (or two-dimensional) filtering has

the same concept as with one-dimension

• The filtered image is the 2D convolution of the

image filter kernel / 2D impulse response (or
point spread function or PSF) and the input
image
IMAGE CONVOLUTION

Steps performed at each image position (u, v):

1. The filter matrix H is moved over the original

image I such that its origin H(0, 0) coincides
with the current image position (u, v)
HUMAN EYE’S PSF (AN EXAMPLE)
2. All filter coefficients H(i, j) are multiplied with
the corresponding image element I(u+i, v+j),
and the results are added

3. Finally, the resulting sum is stored at the

current position in the new image I(u, v)

IMAGE CONVOLUTION (AN ILLUSTRATION)

MACH BANDS

• An optical illusion produced by the human eye.

• Near the edge, the

• overshoot makes the dark region look darker,

and the light region look lighter.

• This produces dark and light bands that run

parallel to the edge.
 • Strategies to speed up calculation:

IMAGE FILTERING • Use a very small PSF (often only 3×3)

• Use a filter kernel (PSF) that is

separable

• Use FFT convolution

3×3 EDGE MODIFICATION

FILTER PSFS

1. Smoothing filters (e.g., pillbox, Gaussian,

square)

• Same as lowpass filters for one-

dimensional signals

• An image convolved with these PSFs

will appear blurry and have less defined
edges, but will be lower in random noise

• Edge enhancement filter

• Same as highpass filters for one-

dimensional signals

• Formed by taking the negative of a SMOOTHING FILTERS

smoothing filter, and adding a delta
function in the center

COMMON POINT SPREAD FUNCTIONS

PROBLEM WITH IMAGE CONVOLUTION

• Large number of calculations

• A 512×512 pixel image convolved with a

64×64 pixel PSF will involve more than
a billion multiplications and additions!

• (i.e., 64×64×512×512 )
 0 1 2 1 0 Answer: Yes. But there is only one! (i.e., the Gaussian
1 3 5 3 1 PSF)
1  
H  2 5 9 5 2
57  
1 3 5 3 1
 0 1 2 1 0 

CONVOLUTION BY SEPARABILITY

• PSF is separable if it can be broken into two one-

dimensional signals: a vertical and a horizontal
projection

SEPARABLE PSFS

CONVOLUTION BY SEPARABILITY

Convolution with a separable filter kernel :

1. convolve each row in the image with the

horizontal projection

2. convolve each column of the intermediate image

with the vertical projection

NOTE:

The convolution of an N×N image with an M×M filter

kernel requires a time proportional to N2M2;

The convolution by separability only requires a time

proportional to N2M.

SEPARABLE PSFS

Question: Is there a PSF that is circularly symmetric and

separable?
ILLUMINATION FLATTENING

IMAGE (2-D) FOURIER TRANSFORM

To calculate the Fourier transform of an image:

1. Take the one-dimensional FFT of each of the

rows in the given image

2. Next, take the one-dimensional FFT of each of

the columns in the intermediate data produced CORRELATION-BASED PATTERN RECOGNITION
from step (1). (AN EXAMPLE)
Note: The algorithm can also be carried out by
transforming the columns first, and then transforming the
rows

FOURIER IMAGE ANALYSIS

CORRELATION-BASED PATTERN RECOGNITION

(AN EXAMPLE)

• Development of a correlation filter kernel

FFT IMAGE CONVOLUTION surrounding pixels with a smaller value (at the
same time widens all the dark structures in the
image)

MINIMUM FILTER

REFERENCES

• Digital Signal Processing: A Practical Guide for

Engineers and Scientists (by S. Smith)
• Original signal (top) and result after filtering
• Principles of Digital Image Processing (by W. (bottom)
Burger and M. Burge)

• ECE ELECTIVE 2 (ADVANCED DIGITAL

SIGNAL PROCESSING) Notes (Ronald
Pascual, Ph. D.)

SUBTOPIC 3- IMAGE PROCESSING (PART 2:

NONLINEAR & SPECIAL IMAGE PROCESSING
TECHNIQUES)

NONLINEAR FILTERS

Linear filters for smoothing noise:

• Disadvantage: All image structures are

also blurred MAXIMUM FILTER
Nonlinear Filters (can provide better solution) • Original signal (top) and result after filtering
(bottom)
• Minimum and Maximum filters

• Median filters

• Weighted Median filters

MINIMUM & MAXIMUM FILTERS

• Compute the result at some image position (u, v)

from the pixels inside the moving region Ru,v

• where R denotes the filter region (set of filter

coordinates)

• For the minimum filter, any single white pixel

within the filter region is replaced by one of its
MEDIAN FILTER MORPHOLOGICAL IMAGE PROCESSING

• The median filter replaces every image pixel by • Techniques used on binary images for altering
the median of the pixels in the corresponding image structures such as form, shape or size
filter region R
• Basic Operations:

• Erosion (“shrinking”)

• Dilation (‘growing”)

• Opening

• Closing

EROSION AND DILATION

• Effects of a 3 × 3 pixel median filter on two-

dimensional image structures

WEIGHTED MEDIAN FILTER

• Assigns individual weights to the positions in the

filter region

• Distribution of weights is specified by a weight

matrix W, with W(i, j) ∈ N DEFINITIONS OF “NEIGHBORHOOD” ON A
RECTANGULAR PIXEL GRID
• Each pixel value I(u + i, v + j) is inserted W(i, j)
times into the extended pixel vector
 CONNECTED VS. UNCONNECTED NEIGHBORS
OPENING AND CLOSING
• Opening  an erosion followed by a dilation
• Closing  dilation followed by an erosion
COMPUTED TOMOGRAPHY (CT)
BINARY SKELETONIZATION (“THINNING”)
• The skeletonized version contains ridges only a
single pixel wide (Skeletonization simplifies the
image by removing redundant pixels)
• Rule one: The pixel under consideration must
presently be black.
• Rule two: At least one of the pixel's close
neighbors must be white.
• Rule three: The pixel must have more than one
black neighbor.
• Rule four: A pixel cannot be removed if it
results in its neighbors being disconnected.
• Tomography means “a picture of a plane."
• Basic problem in imaging with x-rays:
• a two-dimensional image is obtained of a
three-dimensional object (thus, structures
can overlap in the final image)
• CT revolutionized the medical x-ray field with
its ability to visualize the anatomic structure of
the body
• This CT slice is of a human abdomen, at the
level of the navel. Many organs are visible, such
as the (L) Liver, (K) Kidney, (A) Aorta, (S)
Spine, and (C) Cyst covering the right kidney.
CT can visualize internal anatomy far better
than conventional medical x-rays.
• A simple CT system passes a narrow beam of x-
rays through the body from source to detector.
The source and detector are then translated to
obtain a complete view. The remaining views are
obtained by rotating the source and detector in
about 1o increments, and repeating the
translation process.

CT RECONSTRUCTION ALGORITHMS
• CT acquires a set of views and then reconstructs
the corresponding image. Each sample in a view
is equal to the sum of the image values along the
ray that points to that sample.
• Possible Solution: solving many (overdetermined)
simultaneous linear equations
JPEG (TRANSFORM COMPRESSION)
• Backprojection reconstructs an image by taking
each view and smearing it along the path it was
originally acquired. The resulting image is a
blurry version of the correct image.
• Filtered backprojection reconstructs an image
by filtering each view before backprojection.
This removes the blurring seen in simple
backprojection.
• The Fourier Slice Theorem describes the
relationship between an image and its views in
the frequency domain. In the spatial domain,
each view is found by integrating the image
along rays at a particular angle. In the
frequency domain, the spectrum of each view is
a one-dimensional "slice” of the two-
dimensional image spectrum.
• JPEG (named after its origin, the Joint
Photographers Experts Group) is a lossy image
compression method
• Based on the following steps:
• Breaking the image into 8×8 pixel
groups
• Applying Discrete Cosine Transform
(DCT) to each 64-pixel group
• Compressing each 8×8 spectrum by:
• reducing the number of bits
• eliminating some of the
frequency components
• JPEG image division.
 • JPEG transform compression starts by breaking
the image into 8×8 groups, each containing 64
pixels

• JPEG serial conversion. A serpentine pattern

used to convert the 8×8 DCT spectrum into a
linear sequence of 64 values. This places all of
the high frequency components together, where
the large number of zeros can be efficiently
compressed with run-length encoding.
DCT basis functions:

• Just as the Fourier transform uses sine and

cosine waves to represent a signal, the DCT only
uses cosine waves.

• All 8×8 spectral components are real (instead of

complex).

• The low frequencies reside in the upper-left

corner of the spectrum, while the high
frequencies are in the lower-right.

DCT basis functions:

• DCT calculates the spectrum by correlating the

8×8 pixel group with each

• of the basis functions

 IMAGE COMPRESSION USING SVD

IMAGE COMPRESSION USING SINGULAR VALUE

DECOMPOSITION (SVD)

REFERENCES

• Digital Signal Processing: A Practical Guide for

Engineers and Scientists (by S. Smith)

• Principles of Digital Image Processing (by W.

Burger and M. Burge)

• ECE ELECTIVE 2 (ADVANCED DIGITAL

SIGNAL PROCESSING) Notes (Ronald
Pascual, Ph. D.)