IP Full

Image Processing & Pattern recognition
Department of CSE
Prof. Debasis Chaudhuri
DC
1
Syllabus
Program: B. Tech in CSE (BCS) Year, Semester: 3rd yr., 7th Sem.
Course Title: Image Processing Subject code: TIU-UCS-E40#
Contact Hours/week: 3-0-0 (L-I-P) Credit: Theory – 3
Course Content
Module-1: Digital Image Fundaments, Morphological Image
Processing & Image Transforms
Module-2: Image Enhancement
Module-3: Image Segmentation
Module-4: Representation and description
Module-5: Clustering and Classification
Module-6: Real life data applications
DC
2
Requirements
• Prerequisites
– Data Structures and Algorithms
– Linear Algebra and Statistics
– Experience with Programming Language
• Text
– Computer Vision by Shapiro & Stockman, Prentice Hall
– R. C. Gonzalez, W. E. Woods, “Digital Image Processing”,
Pearson-Prentice Hall, Eds. – 3, 2008
– Pattern Classification and Scene Analysis by Duda and Hart,
Wiley-Interscience Publication, New York
– A. K. Jain, “Fundamentals of Digital Image Processing”, Prentice Hall,
Addison-Wesley, 1989
– W. K. Pratt, “Digital Image Processing”, PIKS inside, Wiley, Newyork,
3rd Eds, 2001
– M. Petrou, P. Bosdogianni, “Image Processing- The Fundamentals”,
Wiley, 1999
– Research papers in journals
DC
3
What is Digital Image?
• A digital image is a representation of a two-
dimensional image as a finite set of digital values,
called picture elements or pixels
DC
4
• Pixel values typically represent gray levels,
colors, heights, opacities etc.
• Remember digitization implies that a digital
image is an approximation of a real scene
1 pixel
DC
5
• Common image formats include:
– 1 sample per point (B&W or Grayscale)
– 3 samples per point (Red, Green, and Blue)
– 4 samples per point (Red, Green, Blue, and “Alpha”,
a.k.a. Opacity)
• For most of this course we will focus on grey-

scale images
DC
6
• Digital image processing focuses on three

major tasks
– Improvement of pictorial information for
human interpretation
– Image processing for autonomous machine
application
– Processing of image data for storage,
transmission and representation
• Some argument about where image processing ends

and fields such as image analysis and computer
vision start DC
7
•Human perception
•Employ methods able to enhance pictorial
information for human interpretation and
analysis such as:
•Noise filtering
•Content enhancement
• - Contrast enhancement
• - Deblurring
•Remote sensing
DC
8
Image Sharpening
(a) Original Image (b) After sharpening

DC
9
Removing noise
(a) Original Image (b) After removing noise
DC
10
Deblurring
(a) Original Image (b) After removing the blur
DC
11
History of Digital Image Processing
•Early 1920s: One of the first applications of

digital imaging was in the news-
paper industry
– The Bart lane cable picture
transmission service Early digital image
– Images were transferred by submarine cable

between London and New York
– Pictures were coded for cable transfer and
reconstructed at the receiving end on a
telegraph printer
DC
12
•Mid to late 1920s: Improvements to the

Bart lane system resulted in higher quality
images
– New reproduction
processes based
on photographic
techniques
– Increased number
of tones in Improved
digital image Early 15 tone digital
reproduced images image
DC
13
•1960s: Improvements in computing

technology and the onset of the space race
led to a surge of work in digital image
processing
– 1964: Computers used to
improve the quality of
images of the moon taken
by the Ranger 7 probe
– Such techniques were used
in other space missions A picture of the moon taken
including the Apollo landings by the Ranger 7 probe
minutes before landing DC
14
•1970s: Digital image processing begins to

be used in medical applications
– 1979: Sir Godfrey N.
Hounsfield & Prof. Allan M.
Cormack share the Nobel
Prize in medicine for the
invention of tomography,
the technology behind
Computerised Axial Typical head slice CAT
Tomography (CAT) scans image
DC
15
•1980s - Today: The use of digital image

processing techniques has exploded and
they are now used for all kinds of tasks in all
kinds of areas
– Image enhancement/restoration
– Artistic effects
– Medical visualisation
– Industrial inspection
– Law enforcement
– Human computer interfaces DC
16
Examples: Image Enhancement
•One of the most common uses of DIP techniques:
improve quality, remove noise etc
DC
17
Examples: The Hubble Telescope
•Launched in 1990 the Hubble
telescope can take images of
very distant objects
•However, an incorrect mirror
made many of Hubble’s
images useless
•Image processing
techniques were
used to fix this
DC
18
Examples: Artistic effects
•Artistic effects are
used to make
images more visually
appealing, to add
special effects and to
make composite
images
DC
19
Examples: Medical Image Processing
•Take slice from MRI scan of canine heart,
and find boundaries between types of tissue
– Image with gray levels representing tissue
density
– Use a suitable filter to highlight edges
Original MRI Image of a Dog Edge Detection Image

Heart DC
20
Examples: GIS
•Geographic Information Systems
– Digital image processing techniques are used
extensively to manipulate satellite imagery
– Terrain classification
– Meteorology
DC
21
Examples: GIS
•Night-Time Lights of
the World data set
– Global inventory of
human settlement
– Not hard to imagine
the kind of analysis
that might be done
using this data
DC
22
Examples: Industrial Inspection
•Human operators are

expensive, slow and
unreliable
•Make machines do the
job instead
•Industrial vision
systems
are used in all kinds of
industries
DC
23
Examples: Law Enforcement
•Image processing
techniques are used
extensively by law
enforcers
– Number plate
recognition for speed
cameras/automated
toll systems
– Fingerprint recognition
– Enhancement of CCTV
images
DC
24
What is Computer Vision?
Computer Vision : Recovers useful

information about a scene from its two-
dimensional projections.
Vision = Geometry + Measurement +

Interpretation
Image Recognition:
Image Interpretation:
DC
25
Relationships to other fields
Image Processing : process an image to enhance

certain aspects.
Computer Graphics : Pictorial synthesis of
real/imaginary objects from models.
Pattern Recognition: classification of numerical and
symbolic data
Neural networks: Computational methods using neuron
model
Psychophysics: studies in human vision. Many
researchers in computer vision are more interested in
preparing computational models of human vision than in
designing machine vision systems.
DC
26
Applications of computer vision and
type of processing
(a) Medical Image may be (b) Satellite Image may

processed by a computer be processed by a
vision system to assist in computer vision system
diagnosis for target/ mineral
extraction DC
27
Applications of computer vision and
type of processing
Face Recognition Industrial Surface Inspection Surveillance
Measure Tool Bar code reading OCR Pattern Matching

DC
28
Levels in Computer Vision
Low level : Image to image
- filtering, noise removal, etc.
Intermediate level: image to sub-symbolic

- segmentation, line finding etc.
High level: sub-symbolic to symbolic

- matching, scene labeling, object
recognition etc.
DC
29
Origin
x f ( x, y )
DC
30
Fundamental steps
Representation
And Description
Segmentation
Preprocessing
Knowledge
Recognition
Base
And
Image Interpretation
Acquisition
Problem Result
Domain
DC
31
Key Stages in Digital Image
Processing
Image Morphological
Restoration Processing
Image
Segmentation
Enhancement
Image Object
Acquisition Recognition
Representation
Problem Domain
& Description
Colour Image Image
Processing Compression
DC
32
Processing: Image Acquisition
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
Processing Compression DC
33
Processing: Image Enhancement
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
34
Processing: Image Restoration
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain & Description
Colour Image Image
35
Processing: Morphological Processing
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
36
Processing: Segmentation
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
37
Processing: Object Recognition
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
38
Key Stages in Digital Image Processing:
Representation & Description
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
39
Processing: Image Compression
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
40
Processing: Colour Image Processing
Image Morphological
Image
Segmentation
Enhancement
Image Object
Representation
Problem Domain
& Description
Colour Image Image
41
Element of digital image processing
systems
 The basic operations performed in a digital image processing
systems include (1) acquisition, (2) storage, (3) processing, (4)
communication and (5) display.
Storage
 Optical disks
 Tape
 Videotape
 Mag disks
Image acquisition
equipment Processing Unit Display Unit
 Video  TV Monitors
 Computer
 Scanner  Printers
 Workstation
 Camera  Slide projectors
Communication Channel
Basic fundamental elements of image processing DC
42
Imaging Basics
DC
43
Pretest: Matrices
 a11 a12 . . . a1n 
a a22 . . . a2 n 
 21 
 .
A
. . . . . 

A  [aij ]mn
 . . . . . . 
 . . . . . . 
 
am1 am 2 . . . amn  mn
Row Matrix, Column Matrix, Square Matrix, Diagonal Matrix
A  1 4 3 7 9 1 
 4 A  aij  mn ; m  n
  m n ; m  n
A  aij
  aii  0, aij  a ji  0
A   3
 
7 
9 DC
44
Pretest: Matrices
  mn ; m  n
A  aij
Identity Matrix:
aii  1, aij  a ji  0
  m n ; m  n
A  aij
Symmetric Matrix:
aij  a ji i, j
Skew-symmetric Matrix:
  m n ; m  n
A  aij
aii  0 and aij   a ji i, j
0  4 0 0
Sparse Matrix: Most elements are 0
0 0  3 0
A 
0 0 0 0
 
0 2 0 0
DC
45
Pretest: Matrices
A  [ aij ]mn and B  [bij ]mn A  B  [cij ]mn
Addition:
where cij  aij  bij
1 3 0  0 2 1 1 5 1 
A   4 0 3 , B   2 3 1  A  B  6 3 4 
     
6 1 2 1 2 0 7 3 2
A  [aij ]mn and B  [bij ]mn A  B  [cij ]mn
Subtraction:
where cij  aij  bij
1 3 0  0 2 1  1 1  1
A  4 0 3, B  2 3 1 A  B  2  3 2 
     
6 1 2 1 2 0 5  1 2 
A  [aij ]mn 1 3 0   2 6 0
Scalar
and kA  [ kaij ]mn A  4 0 3, k  2, kA   8 0 6
Multiplier:     DC
6 1 2 12 2 4 46
Pretest: Matrices
A  [ aij ]mn and B  [bij ]n p AB  [cij ]m p
Multiplication: n
where cij   aik bkj , i  1,2,..., m; j  1,2,..., p
k 1
Transpose Matrix: A  [aij ]mn , AT  [a ji ]n m

+ve Definite Matrix: X T AX  0, X  0, X is a column vector
Strictly +ve Definite : X T AX  0, X  0, X is a column vector
-ve Definite Matrix: X T AX  0, X  0, X is a column vector
Strictly -ve Definite : X T AX  0, X  0, X is a column vector
Inverse of a Matrix : AA1  I DC
47
Pretest
2 5 3
7 2 1
4 3 9
The values are 1 2 2 3 3 4 5 7 9
Mean : 4
Median : 3
Varience : 6
Standard Deviation : 2.4495
DC
48
Image Formation
• Geometry of image formation

– Points in the scene
– Projection plane
• Physics of light
– Light sources
– Surface properties
DC
49
Image Formation
DC
50
Image Geometry and perspective
projection
y
x Object Point
f (x,y,z)
r
z
x r
(x,y) z
f r'
y 
z r
Also
x' y' r'
 
x y r
f f
Therefore, x' x, y' y DC
z z 51
Imaging Geometry (Basic Transformations)
X *  X  X 0 , Y *  Y  Y0 , Z *  Z  Z 0
Translation:
( X , Y , Z ) : Old point, ( X *,Y *, Z *) : New point
and ( X 0 , Y0 , Z 0 ) : Displacement
 X * 1 0 0 X 0  X 
 Y *  0 1 0 Y0   Y 
 
 Z *  0 0 1
 
Z0   Z  V *  TV
    
 1  0 0 0 1  1 
 X * 1 0 0 X0 X 
Y * 0 1 0 Y0  Y 
V*    T   V  
 Z * 0 0 1 Z0  Z 
     
 1  0 0 0 1  1
DC
52
Imaging Geometry (Basic Transformations)
S x 0 0 0
Scaling: 0 Sy 0 0
S 
The rotation angle
0 0 Sz 0
  is measured
0 0 0 1 clockwise when
looking at the 
Z  cos sin  0 0 origin from a point
Rotation:  sin  cos 0 0 on the +Z axis.
 R  
 0 0 1

0
Affects only the
values of X and Y
 
 0 0 0 1 coordinates
 Y

X
1 0 0 0 cos  0  sin  0
0 cos  sin  0  0 1 0 0
R    R   
0  sin  cos  0  sin  0 cos  0
    DC
0 0 0 1  0 0 0 1 53
Coordinate Systems
• Model coordinates
– Own coordinate
• Absolute coordinates
– World coordinates
• Camera coordinates
– Measured w. r. t. the camera
DC
54
Simple Image Model
Image refers to a 2 - D light intensity function f ( x, y ) and 0  f ( x, y )  
f ( x, y )  i ( x, y ) r ( x, y ) where 0  i ( x, y )   and 0  r ( x, y )  1
We call the intensity of a monochrome image f at (x, y) the gray level (L ) of
the image at that point. Then Lmin  L  Lmax
Sampling and Quantization

Digitization of the spatial co-ordinates (x, y) is called image sampling, and
amplitude digitization is called gray-level quantization.
 f ( 0 ,0 ) f ( 0,1)  f ( 0, M  1) 
 f (1,0 ) f (1,1)  f (1, M  1) 
f ( x, y )   
     
 
 f ( N  1,0 ) f ( N  1,1)  f ( N  1, M  1)  DC
55
Sampling & Quantization
The digitization process requires decisions about values for N, M, and the
number of discrete gray levels allowed for each pixel. Common practice in
digital image processing is to let these quantities be integer powers of two; that
is
N  2n , M  2k G  2m
The number, b, of bits required storing a digitized image:
b N M m If M = N b N2 m
For example, a 128128 image with 64 gray levels requires 98,304 bits of storage.
DC
56
Quantization
6 bpp 8 bpp
DC
57
Sampling
128 X 128 64 X 64 32 X 32
DC
58
Image Properties
• An array of pixels
• Pixel
– 1 bit per pixel (bpp): binary
– 8 bpp : grayscale
– 24 bpp : red, green and blue
– Satellite images have many more bpp
DC
59
Image Coordinates
• Many systems
– (a) Origin at top left
– (b) Origin at bottom left
– (c) Origin at the center
• Useful for different situations
– (a) most common
– (c) windows processing
DC
60
Image Coordinate Systems
DC
61
Levels of processing
• Point level
– thresholding
• Local
– averaging
• Global
– histogramming
• Object
– segmentation
DC
62
Binary Image Analysis
DC
1
Motivation
• Need less space

• Simpler computational algorithms
• Adequate for many applications
– Counting
– OCR
– Foreground-background separation
– Shape Analysis
– Target detection
DC
2
Binary Images
• Foreground : pixels that belong to objects

of interest
• Background : Everything else
• Separation is important
• Simpler in binary images
• OCR :
– Scan in gray scale
– Convert to binary
DC
3
Examples
DC
4
Binary Image
• Pixels are 0 or 1
– 1 foreground
– 0 background
• Assume mxn image
– Indexed from 0 .. m-1 for rows
– and 0 …. n-1 for columns
– I[i,j] : refers to individual pixels
DC
5
Neighborhood
How many neighbors does a pixel have?
NW N NE i-1, j-1 i-1, j i-1, j+1

W P E i, j-1 i, j i, j+1
SW S SE i+1, j-1 i+1, j i+1, j+1
DC
6
Neighborhood
• 4 neighborhood j
i
N i-1, j
W P E i, j-1 i, j i, j+1
S i+1, j
A pixel p at coordinate (i, j ) has horizontal and vertical neighbours as

N 4 ( P ) : {(i  1, j ), (i  1, j ), (i, j  1), (i, j  1)}
DC
7
Neighborhood
j
• Diagonal neighbor
i
NW NE i-1, j-1 i-1, j+1

P i, j
SW SE i+1, j-1 i+1, j+1
Four diagonal neighbours as

N D ( P) : {(i  1, j  1), (i  1, j  1), (i  1, j  1), (i  1, j  1)}
DC
8
Neighborhood
j
• 8 neighborhood
i
NW N NE i-1, j-1 i-1, j i-1, j+1

W P E i, j-1 i, j i, j+1
SW S SE i+1, j-1 i+1, j i+1, j+1
All together are called 8 - neighbors of p

N8 ( p ) : {N 4 ( p )  N D ( p )}
DC
9
Basic Relationships Between Pixels
Connectivity:
• Connectivity between pixels is important
• Pixels should be neighboring pixels and satisfy a specified criterion of

similarity
• All neighboring pixels are not connected unless satisfy the certain condition
Let V  {1} (Binary image) or V  {15,16,17,..., 25} (Gray value image)

4  Connectivity : Two pixels p and q with values from V are 4 - connected
if q is in the set N 4 ( p).
8  Connectivity : Two pixels p and q with values from V are 8 - connected
if q is in the set N8 ( p ).
m  Connectivity : (mixed connectivity) : Two pixels p and q with values
from V are m - connected if
(i) q is in N 4(p), or (ii) q is in N D(p) and the set N 4(p)  N 4(q) is empty DC
10
• Mixed connectivity is a modification of 8-connectivity and is introduced to
eliminate the multiple path connections that often arise when 8-connectivity
is used.
• Example
0 1 1 0 1 1 0 1 1
0 1 0 0 1 0 0 1 0
0 0 1 0 0 1 0 0 1
(a) (b) (c)
DC
11
Adjacent : A pixel p is adjacent to a pixel q if they are connected.

Two image subsets S1 and S 2 are adjacent if some pixel in S1 is adjacent
to some pixel in S 2 .
Path : A path from pixel p with coordinates ( x, y ) to pixel q with

coordinate ( s, t ) is a sequence of distinct pixels with coordinates
( x0 , y0 ), ( x1, y1 ),..., ( xn , yn ) where ( x0 , y0 )  ( x, y ) and ( xn , yn )  ( s, t ),
( xi , yi ) is adjacent to ( xi 1, yi 1 ), 1  i  n, and n is the length of the path.
DC
12
Relations and Equivalence
Binary relation : A binary relation R on a set A is a set of pairs of
elements from A. If the pair (a,b) is in R, denoted aRb as which
mean " a is related to b".
Example : A  { p1, p2 , p3 , p4 } arranged as
p1 p2
p3
p4
R is the set of pairs of points from that are 4 - connected; that is,
R  {( p1, p2 ), ( p2 , p1 ), ( p1, p3 ), ( p3 , p1 )}
DC
13
Relations and Equivalence
A binary relation R over set A is said to be

(i) Re flexive if for each a in A, aRa ;
(ii) Symmetric if for each a and b in A,
aRb implies bRa ; and
(iii) Transitive if for a, b and c in A, aRb and
bRc implies aRc. A relation satisfying these three
properties is called as equivalanc e relation .
DC
14
Arithmetic and Logical Operation
NOT AND
A NOT (A) A B (A) AND (B)
OR XOR
A B (A) OR (B) A B (A) XOR (B)
NOT
AND
A B [NOT(A)]
DC
AND (B) 15
Applying Masks
• Basic concept
• Based on convolution
• Variety of tasks can be framed in terms of
mask operations
• Mask = a set of pixels
– Have associated weights
– Generally square
– Has an origin (usually the center pixel)
– Generally 2D or 1D
DC
16
Example of Masks
1 1 1 -1 0 1 -1 1
1 1 1 -1 0 1
1 1 1 -1 0 1
1
1 2 1
1
2 4 2
1
1 2 1
1
1
DC
17
Mask Operation
• Overlay the origin of the mask at a pixel

• Pair wise multiply
– Pixel intensity of the image
– Weight of the mask at the pixel
• Sum the terms
• Defines the value at that pixel
• Normalize
– Divide by the size of the mask
• Produces a new image
DC
18
Boundary Conditions
• What if the mask extends to outside the

image?
• Many possibilities
– Zero
– Mirror image
DC
19
Example
-1 0 1
-1 0 1
9 9 9 9 1 1 1
-1 0 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
DC
20
Example -1 0 1
Mask -1 0 1
-1 0 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1 0 0 -24 -24 0
9 9 9 9 1 1 1 0 0 -24 -24 0
9 9 9 9 1 1 1 0 0 -24 -24 0
9 9 9 9 1 1 1 0 0 -24 -24 0
Image 0 0 -24 -24 0
Result DC
21
Example
40 40 80 80 80 1 2 1
40 40 80 80 80 2 4 2
40 40 80 80 80 1 2 1
(b) 33 mask
40 40 80 80 80
40 40 80 80 80
(a) Original gray-tone image
Here we are considering the mirror image: row and

column values are added outside the image
DC
22
Example
640 800 1120 1280 1280
40 50 70 80 80
640 800 1120 1280 1280
40 50 70 80 80
640 800 1120 1280 1280 40 50 70 80 80
40 50 70 80 80
640 800 1120 1280 1280 40 50 70 80 80
(d) Normalized result after
division by the sum of the
640 800 1120 1280 1280 weights in the mask (b)
(c) Result to applying the mask to the

image
DC
23
Counting Objects
• Number of foreground objects

• Counting number of holes
– Equivalent problem
– Swap the role of foreground and background
• Assumption
– Objects are 4-connected
– Foreground pixels are 1’s
– No interior holes
DC
24
Counting Objects
• External Corners • Internal Corners

• Three 0’s and one 1 • Three 1’s and one 0
0 0 0 0 1 1 1 1
0 1 1 0 1 0 0 1
1 0 0 1 0 1 1 0
0 0 0 0 1 1 1 1
DC
25
Counting Objects
• External corners=4 • External corners=8

• Internal corners=0 • Internal corners=4
• Number of objects =(External-Internal)/4
DC
26
Counting Objects
• Take the 4 external masks and 4 internal

masks
• Put them at each pixel in the image
• Count the number of E’s and I’s
– The number of pixels where they match
• Determine the number of objects based on
E and I
DC
27
Counting Objects
DC
28
Connected Components
• Given
– A binary image B
– A pair of pixels B[r,c]  B[r ' ,c' ]  v where v is either 0 or 1
• Pixel (r,c) is connected to (r ' ,c' ) if

– there is a sequence of pixels (r,c)  (r0 ,c0 ), (r1, c1 ),  , (rn ,cn )  (r',c')
– B[ri ,ci ]  v
– (ri ,ci ) and (r i 1, ci 1 ) are adjacent
(r0 ,c0 ), (r1, c1 ),  , (rn ,cn )  (r',c') is a connected path from (r,c) to (r',c')
DC
29
• A connected component is a set of pixels

with the same intensity value such that
each pair of pixels in the set is connected
• Use 4 or 8 neighborhood
• Use complementary connectedness for
foreground and background
• Usually foreground is chosen to be the 1
pixels
DC
30
DC
31
Connected Components Labeling
• A connected component labeling of a

binary image B is a labeled image in which
the value of each pixel is the label of its
connected component
• Background label is 0
• Each component has unique label
• Consecutive labels starting with 1
DC
32
Classical Algorithm
• Take a template and go over the image

• 2 pass algorithm
– Make an initial labeling
– Make a final labeling
Top
Left Current
DC
33
Classical Algorithm
Top Top
0 0
Left Current Left Current
0 1 1 1
Top Top
1 1
Left Current Left Current
0 1 1 1
DC
34
DC
35
Label Collision
1 2 • Labelcollision is a very common
1 2 occurrence – example U-shaped object
1 2 • Label collision pixels are stored in a

separate data structure – an equivalence
1 1 1 ? table.
• In the second pass , whole imaged is
scanned and pixels are re-labeled using
the equivalence table information (for
example, with lowest value in a
equivalence class).
DC
36
Classical Algorithm
First Pass
currentlabel=1
for (i=0; i<nrows; i++)
for(j=0; j<ncols; j++)
if (Image[i,j]==0) break;
if ((top==0) && (left==1) label (current) = label(left);
if ((top==1) && (left==0) label (current) = label(top);
if ((top==0) && (left==0) {
label (current) = currentlabel;
currentlabel++;
}
if ((top==1) && (left==1) {
label (current) = label(left);
equivelent(label(left), label(top));
}
DC
37
Classical Algorithm
Second Pass
for (i=0; i<nrows; i++)
for(j=0; j<ncols; j++)
if (Image[i,j]==0) break;
currentlabel=label(i,j);
label(i,j)=equivalentclass(currentlabel);
DC
38
Classical Algorithm
DC
39
Recursive Scanning Method
 Begin by selecting a pixel with label 1 (water) and label it as “marked.”
 The procedure simultaneously looks for all other pixels in its 3 × 3

neighborhood and marks them if they are also water pixels. This is a
variation of the region growing method, but growing and marking take
place in all directions within a 3 × 3 window simultaneously.
 Once the pixel is “marked,” it will not be considered in the search

procedure.
 Therefore, this method is very fast. Unlike many other growing

methods, this method is independent of a starting point, and there is no
requirement to select suitable properties for including points in the
various regions during the growing process, except to check for water
pixels. The procedure terminates when all the water pixels are
“marked.”
 D. Chaudhuri and A. Samal, "An automatic bridge detection technique for

multispectral image”, IEEE Trans on Geoscience and Remote Sensing, 46(9),
DC
2720-2727, 2008.
40
Pseudocode of Recursive Scanning Method
void srch_a(int i,int j,int dummy)
{
if ((aa[i][j]!=ZERO)&&(aa[i][j]!=dummy))
{
/* aa[i][j] is an object pixel not already taken care of */
aa[i][j] = dummy;
/* give it the value of dummy and then recursively search its
neighbours */
srch_a(i-1,j-1,dummy);
srch_a(i-1,j,dummy);
srch_a(i-1,j+1,dummy);
srch_a(i,j-1,dummy);
srch_a(i,j+1,dummy);
srch_a(i+1,j-1,dummy);
srch_a(i+1,j,dummy);
srch_a(i+1,j+1,dummy);
counta++; /* counta is the component size */
}
} DC
41
Distance Measures
2
2 1 2
2 1 0 1 2
2 1 2
2 DC
42
Distance Measures
2 2 2 2 2
2 1 1 1 2
2 1 0 1 2
2 1 1 1 2
2 2 2 2 2
D. Chaudhuri, C. A. Murthy and B. B. Chaudhuri, "A modified metric to compute

distance", Pattern Recognition, 25(7), pp. 667-677, 1992.
DC
43
Gray Scale Images
DC
1
Histogram
 Map the frequency of intensities in an image

 Table of frequencies
 Graphical representation
 Shows first order properties of an image

E.g. Bright/dark
 Modality
 Number of peaks of a histogram
 Bimodal, unimodal, etc.
DC
2
Histogram
Image Histogram
DC
3
Thresholding
• Common way to convert a gray scale

image to a binary image
• Binary Thresholding
– Map the intensities in the image into 0 or 1
• Multilevel Thresholding
– Map the intensities in the image into a finite
number (usually small) of intensities
• Often guided by the histogram
DC
4
Thresholding
DC
5
Thresholding
Threshold = 180 DC
6
Image Enhancement
 Image enhancement is basically the improving the
perception of information in images for human
viewers and providing better input for the further
image processing techniques.
 Techniques:
 Spatial domain
 Frequency domain
DC
7
Image Enhancement
• Improve the “quality” of the image

• May be for human consumption or
automated processing
• Emphasize certain aspects of the image
– Boundary of an object
• De-emphasize certain aspects of the
image
– Noise
DC
8
Image Enhancement
• Image to Image transformation

• Many techniques
• Choice depends on the goal at hand
DC
9
Image Enhancement
DC
10
Image Enhancement
DC
11
Point Processing Enhancement
DC
12
DC
13
DC
14
Examples
Show better detail
DC
15
Examples
Show boundary of an object
DC
16
Image Enhancement
• Image enhancement: Improve detectability

of important image details or objects by
man or machine
– Noise reduction, smoothing, contrast
stretching, edge enhancement
• Image restoration: Restore a degraded
image to an ideal condition
– Inversion of the degradation process
– Model of processes and sensors
DC
17
Image Enhancement
Image Enhancement
Spatial Domain Technique Frequency Domain Technique
Radiometric enhancement Geometric enhancement

Technique Technique
Histogram Modification Rule Smoothing Filter
Histogram Stretching Sharpening Filter

Histogram Equalization
Derivatives Filter
Linear Contrast/Logarithmic
Contrast/Exponential
Contrast DC
18
Histogram Modification
Histogram modification performs a

function similar to gray level mapping, but
works by considering histogram’s shape
and spread
Gray level histogram of an image is the
distribution of the gray levels in an image
Examination of the histogram is one of the
most useful tools for image enhancement,
as it makes easy to see the modifications
that may improve an image
DC
19
The histogram can be modified by a

mapping function, which will stretch,
shrink (compress), or slide the histogram
Histogram stretching and histogram
shrinking are forms of gray scale
modification, sometimes referred to as
histogram scaling
DC
20
hi (x) hi ( x) and y  f ( x) are known. Find ho ( y )
hi ( x)x  ho ( y )y. Now as x, y  0
dx 1 d ( f 1 ( y ))
ho ( y )  hi ( x)  hi ( f ( y ))
dy dy
x x
x  x
x
ho ( y )
y
y  y
y  f (x)
y y
DC
21
Histogram Stretch
Histogram Shrink
Histogram Slide DC
22
Histogram stretching
• The mapping function equation is as

follows:
where: I(r,c)MAX is the largest gray level

value in the image I(r,c), I(r,c)MIN is the
smallest gray level value in I(r,c) and
Max and Min correspond to the maximum
and minimum gray level values possible
(for an 8-bit image these are 0 and 255)
DC
23
Histogram stretching
• This equation will take an image and
stretch the histogram across the entire
gray level range, which has the effect of
increasing the contrast of a low contrast
image
• If most of the pixel values in an image fall
within a small range, it is useful to allow a
small percentage of the pixel values to be
clipped at the low and high end of the
range (for an 8-bit image this means
truncating at 0 and 255) DC
24
Histogram Stretching
b) Histogram of image (a)

a) Low-contrast image
d) Histogram of image after stretch

c) Image (a) after histogram stretch DC
25
Histogram Stretching with Clipping
b) Histogram of
a) Original image original image
c) Image after
histogram stretching
with out clipping
e) Image after histogram

d) Histogram of stretching with f) Histogram of
image (c) clipping 1% of the image (e)
values at the high
and low ends DC
26
Histogram shrink
The mapping function equation is as
follows:
where I(r,c)MAX is the largest gray level

value in the image I(r,c), I(r,c)MIN is the
smallest gray level value in I(r,c) and
ShrinkMax and ShrinkMin correspond to the
maximum and minimum desired in the
compressed histogram
DC
27
Histogram shrink
• Decreases image contrast by compressing
the gray levels
• However this method may not be useful as
an image enhancement tool, but it is used
in an image sharpening algorithm (unsharp
masking) as a part of an enhancement
technique
DC
28
Histogram shrink
a) Original image b) Histogram of image
c) Image after shrinking the histogram d) Histogram of image (c)

to the range [75,175] DC
29
Histogram slide
• Used to make an image either darker or
lighter, but retain the relationship between
gray level values
• Accomplished by simply adding or
subtracting a fixed number from all of the
gray level values, as follows:
where the OFFSET value is the amount to

slide the histogram
DC
30
Histogram slide
• In this equation we assume that any

values slide past the minimum and
maximum values will be clipped to the
respective minimum or maximum
• A positive OFFSET value will increase the
overall brightness, while a negative
OFFSET will create a darker image
DC
31
Histogram slide
b) Histogram of
a) Resultant image from sliding the image (a)
histogram down by 50
c) Resultant image from

sliding the d) Histogram of image (c)
histogram up by 50 DC
32
• Another enhancement technique to obtain

a uniform histogram for the output image
• Requirements
– Output image should have all available gray
levels
– Approximately same number of pixels in each
level
• Generally work, but not always
DC
33
Algorithm :
Step 1 : For a N  M image of G gray levels, create an array H of
length G. Initialized with 0 values i.e. H [i ]  0 for i  0,1,.., G.
Step 2 : From image histogram scan every pixel and increment the
relevant number of H - if pixel p has intensity g p , perform
H[g p ]  H[g p ]  1
Step 3 : Form the cumulative image histogram H c , H c [0]  H [0]
H c [ p ]  H c [ p  1]  H [ p ], p  1,2,..., G  1
 G 1 
Step 4 : Set T [ g ]  round  H c [ g ]
 NM 
step 5 : Rescan the image and write an output image with gray
levels g q , setting g q  T [ g p ].
DC
34
• Consists of four steps:
1. Find the running sum of the histogram
values
2. Normalize the values from step (1) by
dividing by the total number of pixels
3. Multiply the values from step (2) by the
maximum gray level value and round
4. Map the gray level values to the results
from step (3) using a one-to-one
correspondence
DC
35
Example:
3-bits per pixel image – range is 0 to 7.
Given the following histogram:
Number of Pixels
Gray Level Value (Histogram values)
0 10
1 8
2 9
3 2
4 14
5 1
6 5
7 2
DC
36
1) Create a running sum of the histogram values.
This means the first value is 10, the second is
10+8=18, next 10+8+9=27, and so on. Here we
get 10, 18, 27, 29, 43, 44, 49, 51
2) Normalize by dividing by the total number of
pixels. The total number of pixels is:
10+8+9+2+14+1+5+2 = 51 (note this is the last
number from step 1), so we get: 10/51, 18/51,
27/51, 29/51, 43/51, 44/51, 49/51, 51/51
3) Multiply these values by the maximum gray
level values, in this case 7, and then round the
result to the closest integer. After this is done we
obtain: 1, 2, 4, 4, 6, 6, 7, 7
DC
37
4) Map the original values to the results from step
3 by a one-to-one correspondence. This is done
as follows:
Original Gray Histogram
Level Value Equalized Values
0 1
1 2
2 4
3 4
4 6
5 6
6 7
7 7 DC
38
Histogram Equalization Examples
1.
Input image Resultant image after histogram

equalization
DC
39
Histogram Equalization Examples
2.
Input image Resultant image after

histogram equalization
Note: As can be seen histogram equalization provides
similar results regardless of the input image
DC
40
Gray-Level Mapping
• Change the quality of an image by simply

a change of intensities
– E.g. region labeled image
• Map the intensities to new values
• Need a mapping function
• Stretch the gray level values in “dark”
regions
DC
41
Linear Contrast Enhancement
y
255
0 x
Bmin Bmax
y  f ( x)  ax  b. Now it is passing through ( Bmin ,0) and ( Bmax ,255)

x  Bmin
then y   255
Bmax  Bmin
DC
42
Example: Enhancement with
different transform functions
0.5
Original image f ( x)  x
DC
43
Geometric Enhancement
Techniques
DC
44
Geometric Enhancement Techniques
+ + + + + y
+ + + + +
+ + + + +
Pixel (i, j ) x ( x, y )
+ + + + + Image
+ + + + +
Template/M
ask
For an M by N pixel sized template, the response for image

M N
pixel (i, j ) is r (i, j )   f (i  m, j  n)t (m, n), where f (i, j )
m 1 n 1
is the pixel brightness value of the image and t (m, n) is the template
entry at that location.
DC
45
Local Enhancement
g ( x, y )  A( x, y )[ f ( x, y )  m( x, y )]  m( x, y )
M
where A( x, y )  k ; 0  k 1
 ( x, y )
Here m( x, y ) and  ( x, y ) are mean gray value and standard

deviation in a neighborhood centered at ( x, y ), M is the global
mean of f ( x, y ) and k is a constant
DC
46
Mean Value Smoothing
M N
1
r (i, j ) 
MN
 f (i  m, j  n)
m 1 n 1
To recover high frequency components
1 M N
Let  (i, j)  
MN m 1 n 1
f (i  m, j  n) then
 (i, j ) if f (i, j )   (i, j )  T

r (i, j )  
 f (i, j ) otherwise
where T is the prespecified threshold.
DC
47
Image Averaging
• A noisy image:
g ( x , y )  f ( x, y )  n ( x, y )
• Averaging K different noisy images:
M
1
g ( x, y ) 
K
 g ( x, y )
i 1
i
DC
48
Image Averaging
• As K increases, the variability of the
pixel values at each location decreases.
– This means that g(x,y) approaches f(x,y)

as the number of noisy images used in the
averaging process increases.
• Registering (aligned) of the images is

necessary to avoid blurring in the output
image.
DC
49
Image Enhancement in the spatial domain
a b
c d
e f
a) Image of Galaxy Pair

NGC 3314, b) Image
corrupted by additive
Gaussian noise with
mean and a standard
deviation of 64 gray
levels, (c)-(f) Results of
averaging K=8, 16, 64
and 128 noisy images.
Original image courtesy
NASA DC
50
Noise Reduction
• Salt-and-pepper noise is a common form

• Single black pixel in bright regions
• Single bright pixel in dark regions
• Binary thresholding results in this kind of
noise
• Errors introduced during the imaging
process
DC
51
Noise Reduction
• Use masking operations to remove noise

• If the mask matches the pattern then
replace with a second mask
• Only two sets of masks are needed
DC
52
Mean Filter
+ + + + + j
+ + + + +
i 1 1 1
+ + + + +
+ + + + + 1 1 1
+ + + + + 1 1 1
Image
Template/Mask
DC
53
Mean Filter
DC
54
Mean Filter
DC
55
Noise Reduction
Before After remove of After remove of

black spot white spot
DC
56
Noise Removal
Before After
DC
57
General Templates
8-neighbor case
4-neighbor case
DC
58
Removal of Small Components
• Connected components (segmentation)

often results in many small components
• Some are noise
• Many others are too small to be
meaningful objects
• Removing them reduces the
computational overhead on the high level
processing
DC
59
Removal of Small Components
Before After
DC
60
Smoothing
• Effect of random noise can often be

minimized by using an averaging process
i m j m
  In[r  i, c  j ]
i  m j  m
Out[r , c] 
(2m  1)(2m  1)
• Smoothing an image by equally weighting

a rectangular neighborhood is called using
a box filter DC
61
Smoothing
• Better to use a weighting function that is based
on the distance from the center
• Gaussian filter is a common method
d2
1  2
g ( x, y )  e 2
2 
d  ( x  xc ) 2  ( y  yc ) 2
• Gaussian noise of mean 0 and standard

deviation of σ=1
DC
62
Directional Smoothing
DC
63
Median Filtering
• Mean filtering produces good results when

it is used over a homogeneous region with
zero mean noise
• When the neighborhood straddles
boundary of two regions, the result blurs
the boundary between the regions
• Median filter is a better choice in that case
DC
64
Median Filtering
• Replace the pixel by a median of the

neighborhood
Before After DC
65
Median Filtering
• Works well for salt and pepper noise also

• More difficult to implement in hardware
than mean filtering
– Need to obtain a partial sort
– Modify quick sort to obtain the median
element
DC
66
Median Filtering
Before After
DC
67
Median Filter
a) Image with added salt-and-pepper b) After median filtering

noise, with a 3x3
the probability for salt = probability window, all the noise
for pepper = 0.10 is not removed
DC
68
Median Filter
c) After median filtering with a 5x5 window, all the noise is

removed, but the image is blurry acquiring the “painted”
effect
DC
69
High-Pass Filtering
High-pass = Original – Low-pass
High-Boost Filtering
High-boost = A  Original – Low-pass

= (A-1)  Original + Original – Low-pass
= (A-1)  Original + High-pass
DC
70
Frequency Domain Technique
DC
71
Position invariant operator: h x, y  Image: f x, y 
g ( x, y )  f ( x, y )  h ( x, y )
Fourier transform of g ( x, y ), h( x, y ) and f ( x, y )
are G (u , v), H (u , v) and F (u , v)
The convolution theorem :
G (u , v)  H (u , v) F (u, v)
Then take inverse
g ( x, y )  G 1H (u , v) F (u , v)
DC
72
DC
73
Frequency Domain: Lowpass filtering
DC
74
Ideal filter:
DC
75
H(u,v)
v
1
u v
D(u,v)
u D0
DC
76
Butterworth lowpass filter:
DC
77
Butterworth lowpass filter:
DC
78
Frequency Domain: High pass filtering
Ideal filter:
DC
79
Ideal filter:
H(u,v)
v 1
u
v
D(u,v)
u D0
DC
80
Ideal filter:
DC
81
Butterworth high pass filter:
DC
82
Butterworth high pass filter:
DC
83
Frequency Domain: Band pass filtering
 Band-pass filters reduce signal frequencies outside of a
range (band) of interest.
 In image analysis, they can be used to denoise images

while at the same time reducing low-frequency objects
such a uneven illumination.
 Band-pass filters can be used to find image features such

as blobs and edges.
 One method for applying band-pass filters to images is to

subtract an image blurred with a Gaussian kernel from a
less-blurred image.
DC
84
Frequency Domain: Band pass filtering
This example shows two applications of the Difference of
Gaussians approach for band-pass filtering.
DC
85
Two-pass combined method
Published in Defence Science Journal, 2007
ORIGINAL IMAGE
FREQUENCY
DOMAIN SPATIAL DOMAIN
LINE PATTERN REMOVAL

TECHNIQUE
IMAGE ENHANCEMENT
ENHANCED IMAGE
DC
86
Example
(a) Original Passive millimeter (b) After Frequency Domain

wave sensor image
DC
87
Example
(c) After Image Domain (d) After Fusion
(e) Final result
DC
88
Homomorphic Filtering
DC
89
DC
90
DC
91
DC
92
Adaptive Gaussian Filter:
DC
93
(a) (b) (c)

(d) (e) (f)
Human and building thermal IR image: (a) original image, (b) adaptive
histogram equalisation image, (c) Gaussian filter image by 5×5 window
with σ = 1.5, (d) proposed adaptive Gaussian method without
homomorphic filter, (e) homomorphic filter image without adaptive
Gaussian method, and (f) proposed method. DC
94
Garden thermal IR image: (a) original image and (b)

proposed image.
DC
95
Moving tank thermal IR image: (a) original image at

Pokhran and (b) processed image by proposed
algorithm.
DC
96
Tower thermal IR image: (a) original image at

Mussoorie and (b) processed image by proposed
algorithm.
DC
97
Color
Image Enhancement
DC
98
Enhancement of Color Images
Gray scale transforms and histogram

modification techniques can be applied
by treating a color image as three gray
images.
Care must be taken in how this is done

to avoid color shifts.
DC
99
Enhancement of Color Images
Histogram modification can be performed
on color images, but doing it on each color
band separately can create relative color
changes
The relative color can be retained by

applying the gray scale modification
technique to one of the color bands, and
then using the ratios from the original
image to find the other values
DC
100
Histogram Equalization of Color
Images
a) Original poor b) Histogram equalization

contrast image based on
the red color band
DC
101
Images (contd.)
c) Histogram equalization d) Histogram

based on the green color equalization based on
band the blue color band
Note: In this case the red band gives the best results
This will depend on the image and the desired result DC
102
Images
Typically the most important color band is
selected, and this choice is very much
application-specific and will not always
provide us with the desired result.
Often, we really want to apply the gray

scale modification method to the image
brightness only, even with color images
DC
103
Images
 Histogram modification on color images can be
performed in the following ways:
– Retain the RGB ratios and perform the modification

on one band only, then use the ratios to get the other
two bands’ values, or
– Perform a color transform, such as HSL, do the

modification on the lightness (brightness band), then
do the inverse color transform.
DC
104
Image Transformation
DC
1
Mathematical Transformation
• Why
– To obtain a further information from the signal that
is not readily available in the raw signal.
• Raw Signal
– Normally the time-domain signal
• Processed Signal
– A signal that has been "transformed" by any of the
available mathematical transformations
• Fourier Transformation
– The most popular transformation
DC
2
Frequency Analysis
• Frequency Spectrum
– Be basically the frequency components (spectral
components) of that signal
– Show what frequencies exists in the signal
• Fourier Transform (FT)

– One way to find the frequency content
– Tells how much of each frequency exists in a
signal
DC
3
Stationarity of Signal
• Stationary Signal
– Signals with frequency content unchanged
in time
– All frequency components exist at all times
• Non-stationary Signal
– Frequency changes in time
– One example: the “Chirp Signal”
DC
4
Stationarity of Signal
2 Hz + 10 Hz + 20Hz 3 6 0 0
2 5 0 0
Magnitude
Magnitude
1 4 0 0
Stationary 0 3 0 0
-1 2 0 0
-2 1 0 0
-3 0
0 0 .2 0 .4 0 .6 0 .8 1 0 5 1 0 1 5 2 0 2 5
Time Frequency (Hz)
0.0-0.4: 2 Hz + 1 2 5 0
Magnitude
0.4-0.7: 10 Hz +
0 .8
0 .6 2 0 0
0.7-1.0: 20Hz
Magnitude
0 .4
0 .2 1 5 0
Non- 0
Stationary -0 .2
-0 .4
1 0 0
-0 .6 5 0
-0 .8
-1 0
0 0 .5 1 0 5 1 0 1 5 2 0 2 5
Time Frequency (Hz)
DC
5
Chirp Signals
Frequency: 2 Hz to 20 Hz Different in Time Domain Frequency: 20 Hz to 2 Hz
1 150 1 150
0.8 0.8
0.6 0.6
Magnitude
0.4 Magnitude 0.4
Magnitude
Magnitude
100 100
0.2 0.2
0 0
-0.2 -0.2
50 50
-0.4 -0.4
-0.6 -0.6
-0.8 -0.8
-1 0 -1 0
0 0.5 1 0 5 10 15 20 25 0 0.5 1 0 5 10 15 20 25
Time Frequency (Hz) Time Frequency (Hz)
Same in Frequency Domain
At what time the frequency components occur?

FT can not tell!
DC
6
Fourier Transform
Fourier
f(x)
Transform
Inverse Fourier
f(x)
Transform
DC
7
Fourier Transform
DC
8
2-D Fourier Transform
DC
9
Discrete Fourier Transform
DC
10
(1)
(2)
DC
11
(3)
(4)
(5)
(6)
DC
12
DC
13
Result Fourier Transform
Original image Magnitude image
DC
14
Result Fourier Transform
Logarithmic image Phase image
DC
15
Properties of Fourier Transform
Row transform Column transform
Multiplication by N Multiplication by N
DC
16
DC
17
DC
18
DC
19
DC
20
Graphical
illustration of
convolution.
The shaded
areas indicate
regions where
the product is
not zero
DC
21
DC
22
DC
23
DC
24
DC
25
Morphology
DC
1
Morphology
• Study of shape (form and structure)

• Mathematical morphology was designed
for a set of points
• Treat the object as binary
• A shape = a set of points
• Use shape-based approach
DC
2
Structuring Elements
• Morphology operations involve

– Binary image
– Structuring element
• Structuring Element
– Binary image
– Can be any arbitrary binary image in general
– Usually much smaller
DC
3
Common Structuring Elements
DC
4
Operations
• Intersection
A  B  { p | p  A and p  B}
• Union
A  B  { p | p  A or p  B}
• Complement
A  { p | p   and p  A}
Ω is a universal binary image (all 1s)
DC
5
Translation
• A : binary image
• p : pixel
• Translation of A by p is an image
• Shifts the origin of A to p
Ap  {a  p | a  A}
DC
6
Binary Dilation
A  (a1 , a2 ,..., a N )  E N and B  (b1 , b2 ,..., bN )  E N then Dilation is defined as
A  B   c  E N | c  a  b for some a  A and b  B
  
    
    
B
  
A A B
Here A  {(0,1), (1,1), (2,1), (2,2), (3,0)}, B  {(0,0), (0,1)}
Then A  B  {(0,1), (1,1), (2,1), (2,2), (3,0), (0,2), (1,2), (2,2), (2,3), (3,1)}
DC
7
Properties of Dilation
1. A  B  B  A (Commutative)
2. ( A  B )  C  A  ( B  C ) (Associative)
3. A  B    Ab
bB
4. ( A) x  B  ( A  B ) x
5. [ A  B1  ...  ( Bm ) x  ...  Bn ]  ( A  B1  ...  Bn ) x
6. ( A) x  ( B )  x  A  B
7. A  B  A  D  B  D
8. ( A  B )  C  ( A  C )  ( B  C )
9. A  ( B  C )  ( A  B )  ( A  C )
10. ( A  B)  C  ( A  C )  ( B  C ) DC
8
Examples
(b) Structuring
element
(a) Original (c) Dilation
A B DC
9
Dilation
DC
10
Binary Erosion
A  (a1 , a2 ,..., a N )  E N and B  (b1 , b2 ,..., bN )  E N then Erosion is defined as
AB   x  E N | x  b  A for every b  B
            


B


A AB
Here A  {(1,0), (1,1), (1,2), (1,3), (1,4), (1,5), (2,1), (3,1), (4,1), (5,1)},
B  {(0,0), (0,1)} Then AB  {(1,0), (1,1), (1,2), (1,3), (1,4)} DC
11
Properties of Erosion
1. AB    Ab
bB
2. Ax B  ( AB) x (Translation invariance of erosion)

3. ABx  ( AB)  x (Translation invariance of erosion)
4. If A  B then AC  BC (Erosion is increasing)
5. If A  B then CA  CB
6. ( A  B)C  ( AC )  ( BC )
7. ( AC )  ( BC )  ( A  B )C
DC
12
Erosion
(b) Structuring
element
(a) Original (c) Erosion
DC
13
Erosion and Dilation
DC
14
Complement of A and Reflection of B
A B' DC
15
'
AB
DC
16
A B '
DC
17
Opening
• Erosion followed by dilation

A  B  ( AB)  B
• Remove all pixels in a region too small to
contain the structuring element
• Disc shaped probe
– Eliminate all convex regions and isolated
points
DC
18
Example
(b) Structuring
element
(a) Original (c) Opening
A B DC
19
Example
Opening operation: Left: Initial erosion. Right:

Succeeding dilation. Boundary of the original A is
shown as a bold line DC
20
Example
(b)
(a) (c)
A noisy binary image. (a) Original noisy image of

size 128128. (b) Structuring element of size 51.
(c) Image after opening operation.
DC
21
Closing
• Opposite of opening
• Dilation followed by erosion
A  B  ( A  B)B
• Fill holes smaller than the structuring
element
DC
22
Example
(b) Structuring
element
(a) Original (c) Closing
A B DC
23
Example
Closing Operation. Left: Initial dilation. Right:

Succeeding erosion. The boundary of the original
figure A is shown as bold line.
DC
24
Example
(a) (b) (c)
(a) Original image: 512512 16-bit gray-scale medical

image. (b) Thresholded at 1070 to produce the binary
image. (c) Opening with DISK(13) structuring element
and closing with a DISK(2)
DC
25
Example
DC
26
HIT and MISS Transformation
Let A is a set and B is a structuring element, which is composed of X
and background. The Hit and Miss transformation of A by B is
A * B  ( AX )  [ A(W  X )] where X is a set and W is a small window.
We can generalize, A * B  ( AB1 )  ( AB2 )
Example: Objective is to find the location of one of the shapes, say X
A  X Y  Z
 
Origin
Z
 W W-X
(b) Local
 background of X
Y with respect to W
X
(a)
DC
27
 AX 
A

(c) (d)
AW  X 
 AX   A[W  X ]
 
(e) (f)
DC
28
Boundary Extraction
 ( A)  A   AB 
Origin
(b) B
(a) A
(c) AB (d)   A

DC
29
Region Filling
Let A denotes a set containing a subset whose elements are
8-connected boundary points. Region filling algorithm is as
follows:
Step 1: Assume all non-boundary points are labeled 0. Assign
a value 1 to start the procedure.
Step 2: Fill the region with 1’s as X k  ( X k 1  B)  A, k  1,2,..
where X 0  p and B is the symmetric structuring element.
Step 3: Iteration stop, if X k  X k 1
Step 4: The set union of X k and A contains the filled set and
its boundary
DC
30
Region Filling
Example
Origin
(b) B
(a) A (a) A
DC
31
(d) X 0 (e) X1 (f) X2
(g) X (h) X 7 (i) X7  A DC

6
32
Gray scale dilation and erosion
Let F , K subsets of E N 1 and f : F  E , k : K  E. The gray
scale dilation of f by k is denoted by f  k , f  k : F  K  E
can be computed by  f  k ( x)  max f ( x  z )  k ( z ) | z  K , x  z  F 
Let F , K subsets of E N 1 and f : F  E , k : K  E. The gray

scale erosion of f by k is denoted by fk , fk : FK  E
can be computed by  fk ( x)  min f ( x  z )  k ( z ) | z  K 
DC
33
Road Detection from Remote Sensing
Images (<2m)
DC
34
.
.
Road Detection from Remote Sensing Images (<2m)
Enhancement technique: For each pixel (x, y) in the image f
1 5
1. Compute the mean gray value,  d ( x, y )   | f d ( x, y )[i ]  S d [i ] |, d  1,..., 4
5 i 1
5
1
2. Compute the standard deviation,  (x, y) 
d 
5 i1
( fd (x, y)[i]  Sd[i]  d (x, y)[i])2 , d  1,...,4
3. Find the direction dmin for which  d , d  1,..., 4 is minimum.

4. Perform gray level erosion at (x,y) using by the structuring element along the direction dmin.
5. Apply the contrast stretching with a range of [0, 255] to derive an intermediate image g ( x, y )
For each pixel (x, y) in the image g
1 5
6. Compute the mean gray value,  d ( x, y )   | g d ( x, y )[i ]  S d [i ] |, d  1,..., 4
5 i 1
15
7. Compute the standard deviation, d (x, y)  (gd (x, y)[i]  Sd [i]  d (x, y)[i])2 , d 1,...,4
5 i1
8. Find the direction dmin for which  d , d  1,..., 4 is minimum.

9. Perform gray level dilation at (x,y) using by the structuring element along the direction dmin.
10. Apply the contrast stretching with a range of [0, 255] to derive an intermediate image h( x, y)
11. Repeat the above operations in reverse order (i.e. dilation followed by erosion). DC
35
Road Detection from Remote Sensing Images (<2m)
Directional Morphological Enhancement Technique:
(a) (b)
(a) Original
Image
(b) Enhanced
Image
(a) (b)
DC
36
Build-Up area Analysis: Enhancement
Fig. IKONOS satellite panchromatic image of ground

resolution 1m×1m. (a) Original image, (b) enhanced
image
DC
37
Shape Features
DC
1
Shape Features
• Organization of points in space
• Not related to any intensity or color features
• Common Features
– Centroid
– Area
– Perimeter
– Radius
– Orientation
– Major and Minor Axes
– Elongatedness
– Aspect Ratio
– Bounding box
– Moment
– Compactness
– Complexity of object (No. of perimeter pixels/No. of object pixels) DC
2
Centroid
N
1
x
N
x
i 1
i
N
1
y
N
y
i 1
i
where N is the total number of points

in the region R and  xi , yi   R
DC
3
Area
• Number of pixels in a component
• Gives an indication of size
• Easy to compute
A 1
( x , y )R
where R is a given region
DC
4
Perimeter
• Boundary points of the object
• Length of the perimeter is the number of
pixels on the boundary of the object
• Length of the perimeter is an important
measure of the shape
• First order of approximation
– Number of pixels on the boundary
• More accurate approximation
– Horizontal and vertical moves : 1 unit
– Diagonal move: 1.4 unit DC
5
Circularity
• Degree of similarity with a circle
2
P
Circularity 
A
• Based on continuous planer shapes
• Make adjustments for digital shapes
DC
6
Circularity
R
Circularity 
R
 R :Mean radial distance
 r :Standard deviation of radial distance
• Radial distance computed as the distance
between the centroid and a boundary point
DC
7
Circularity
1 k 1
 R   ( xi , yi )  ( x, y )
k i 0
  ( x , y )  ( x, y )   
1
1 k 1 2
 2
r  i i R 
k i 0 
( x, y ) is the centroid of the region

DC
8
Region Properties
DC
9
Compactness
DC
10
Compactness
Calculate the "compactness" of each of the following
'islands' using the above definition
DC
11
Compactness
Solution: The results for area ¸ perimeter for the six shapes are shown
below:
Shape A B C D E F
Area 12 28.26 3 16 16 4
Perimeter 16 18.84 8 16 32 8
(cm)
Area ¸ 0.75 1.5 0.375 1 0.5 0.5
Perimeter
Use your results to explain why Area ¸ Perimeter is not a suitable

definition for "compactness."
Solution:
· While at first the measure area ¸ perimeter might seem
reasonable, it is not dimensionless, so it will give different results for
enlargements of shapes.
· Thus in the examples D and F are both squares, but they give
different measurements of compactness. Similarly with A and C. DC
12
Compactness
Invent your own measure of "compactness".
Put the shapes A to F in order of "compact-ness" using your measure.
Discuss whether or not your measure is better than 'Area ¸ Perimeter.'
Solution:
A better measure of compactness might therefore be Area ¸ Perimeter2.
This is dimensionless and does give measures in accord with intuition:
Staircase A B C D E F
Area  Perimeter2 0.047 0.080 0.047 0.063 0.016 0.063
Thus, in order of "compactness", we have that

B (most compact), D and F (tie), A and C (tie), E (least compact).
DC
13
Compactness
Adapt your measure so that it ranges from 0 to 1.
A perfectly compact shape should have a measure of '1,' while a long, thin,
shape should have a measure near to 0.
A
Compactness Ratio:
R 2
A = area
R = radius of the smallest circle which will surround the shape
Other Definition for Compactness
2
(Re gion _ border _ length)
Compactness 
Area
DC
14
Compactness
Proposed Method: Published in DSJ, 2008
Center
(a) Object
(b) Fitted Circle
Matching_Area
(c) Matching area of object

and fitted circle DC
15
Compactness
Non_Matching_area_due_to_fitted_circle
New _ Measure _ of _ Compactness

Total _ Non _ Matching _ area

Matching _ area
Non_Matching_area_due_to_Object #[ A  ( A  B)]  [ B  ( A  B )]

# ( A  B)
# ( A  B )  ( A  B ) 

# A  B
DC
16
Result of Proposed definition of general
measure for rectangle, circle and ellipse
(a) (b) (c) (d)
(a) Binary image Objects of Degree of Fitness

Fig. 7(a)
Rectangularity Circularity Ellipticity
(b) Fitted rectangle
Object 1 1.378815 3.835594 0.615690
(c) Fitted circle Object2 0.000000 0.207373 0.219076
Object 3 1.288053 3.766476 0.897333
(d) Fitted ellipse
Object 4 0.366713 1.229934 0.049481
Object 5 0.274922 0.0318950 0.032764
Object 6 0.000000 3.661320 0.677681
Object 7 0.442777 0.696997 0.074589
Object 8 0.338803 0.747632 0.064988 DC
17
Euler’s Number
Topology: is the study of properties of a figure that are unaffected by any
deformation.
(a) A region with two holes

(b) Same region by rotation
and stretching transformation
If a topological descriptor is defined by the number of holes in the region,

this property will not be affected by a stretching or rotation transformation.
DC
18
Euler’s Number
Connected Component: A connected component of a set is a
subset of maximal size such that any two of its points can be
joined by a connected curve lying entirely within the subset.
Euler Number E  C  H
C - Connected component
H Number of holes
A region with three

connected component
For Letter A: C = 1 and H = 1 so E = 0. For Letter B: C = 1 and H = 2

so E = -1 DC
19
Moments
• Used to indicate the shape of the objects
• Describe the distribution of points in shape
• Many moments can be defined
• Only low order moments are useful
DC
20
Moments
M pq   x p q
y f ( x , y ) dx dy
x y
where
1 if ( x, y ) is in the object 
f ( x, y )   
0 otherwise 
DC
21
Moments
• Defines moments of order p+q
• Changes with location of the object
• Discrete case
M pq   x y f ( x, y )
p q
x y
DC
22
Moments
M 00   x 0 y 0 f ( x, y )   f ( x, y )  Area of the object
x y x y
M 10   xf ( x, y )  Sum of all x - coordinate s

x y
M 01   yf ( x, y )  Sum of all y - coordinate s

x y
M 10 M 01
x y
M 00 M 00
DC
23
Central Moments
• Moments change with the location of the object
• Not useful for comparing shapes
 pq   ( x  x ) p ( y  y ) q f ( x, y )
x y
10   ( x  x )1 ( y  y )0 f ( x, y )
x y
  xf ( x, y )  x  f ( x, y ) 00   f ( x, y )  Area
x y x y x y
M10 10 01

 M10  ( M 00 )  0 x  0 and y  0
M 00 00 00
01  0
DC
24
Central Moments
2 2
M10 M 01 M10 M 01
11  M11  , 20  M 20  , 02  M 02 
M 00 M 00 M 00
2
30  M 30  3x M 20  2 x M10
12  M12  2 yM11  x M 02  2 y 2 M10
 21  M 21  2 x M11  yM 20  2 x 2 M 01
2
03  M 03  3 yM 02  2 y M 01
DC
25
Central Moments
• Centroid forms the origin
1 1 2M 11
• Orientation of on object is given by   tan ( )
2 M 20  M 02
Alternate Method: (D.Chaudhuri and A. Samal in Pattern
Recognition,Vol. 40, pp. 1981-1989, 2007)
( xib , yib ), i  1,2,..., n  Border points of the object.  Angle of the major
axis with horizontal axis. ( xc , yc )  Centriod of the object. Then equation of the
line passing through centroid and makes an angle  be tan  x  y  yc  tan  xc  0.
Now the perpendicular distance from the border points
pi  ( xib  xc ) sin   ( yib  yc ) cos , i  1,2,.., n Sum square perpendicular distance
2

P   xib  xc sin   yib  yc cos 
n
i 1
2 xib  xc  yib  yc 
n
P
Now for minimum  0  tan 2  i 1
 x  x   y  y  
 n
b 2 b 2
i c i c
i 1 DC
26
Moments
• Row moments
1
 pp ( x  x ) f ( x, y)
p
Ax
• Column moments
1
 pp ( x  y) p f ( x, y)
Ay
• Eccentricity (elongatness)
1
(M 20  M 02  4M11
) 2
e
M 00
DC
27
Elongatedness
Elongatedness is a ratio between the length and width of the region bounding
rectangle. This is the rectangle of minimum area that bounds the shape.
Alternate Method:
• Elongatedness can be evaluated as a ratio of the region area and the square
of its thickness.
• Maximum region thickness (holes must be filled if present) can be

determined as the number of erosion steps that may be applied before the
region totally disappears.
• d is the number of erosion steps then
area
elongatedness  2
2d DC
28
Elongatedness
Another Method: (D. Chaudhuri, IEEE Trans. on Systems, Man and Cybernetics,
Vol. 27, No. 5, pp. 871-877, 1997)
• Find the centroid and border points of the object.
• Find the minimum from the centroid and border points.
• Find the maximum distance from centroid and border points.
• Find the difference of those two distances.
• If difference is more than tolerance then it is elongated.
DC
29
Bounding Box
DC
30
Bounding Box
• Gives a rough idea of the location of the
object
• Smallest rectangle to enclose the shape
• Extremal points
– Leftmost, rightmost, topmost and bottommost
– Lie on the bounding box boundary
– May have up to 8 extremal points
– Occur in opposite pairs
DC
31
Bounding Box
• Each point of opposite extremal point pairs
define an axis
• Properties of axes are useful
– Lengths
– Orientations
• Need to make corrections to the lengths
– Error due to sampling
DC
32
Bounding Rectangle
Orientation of on object is given by
1 2 M 11
  tan 1 ( )
2 M 20  M 02
Defining  ( x, y )  x cos  y sin   ( x, y )   x sin   y cos
search for the minimum and maximum of  and  over all
boundary points ( x, y ). The values of  min ,  max ,  min ,  max
then define the bounding rectangle. l1  ( max   min )
and l2  (  max   min ) are its length and width.
b
Bounding rectangle can not
represent elogatedness DC
33
Bounding Rectangle
New Proposed Method: (D.Chaudhuri and A. Samal in Pattern
  
Recognition,Vol. 40, pp. 1981-1989, 2007) n
2 xib  xc yib  yc
i 1
tan 2 
 2  yib  yc 2 
n
  xib  xc
• Find Centroid and orientation of the object. i 1
• Find Major and Minor axes of the object.

• Find upper and bottom sides border points of the object with respect to major axis.
• Find left and right sides border points of the object with respect to minor axis.
• Find the furthest points with respect to major axis from both sides.
• Find the furthest points with respect to minor axis from both sides.
• Find the lines parallel to the major axis for furthest points with respect to major axis.
• Find the lines parallel to the minor axis for furthest points with respect to minor axis.
• Find the points of intersection of these lines which are mutually intersect pair wise.
• Draw the rectangle through these points.
DC
34
Bounding Rectangle
(a) Example (b) Centroid and board of (c) Major and Minor
object axes
(d) Upper and lower furthest (e) Four vertices of bounding

points rectangle
DC
35
Bounding Rectangle
(a) Binary image (b) Fitted bounding

rectangle image
DC
36
Bounding Rectangle
(a) (b)
(a) PAN Image (IRS-

1C)
(b) Segmented image
(c) Segmented area
(c)
(d) filtered and filling
image
(d) Fitting bounding
rectangle
DC
37
Best-fitted Rectangle
Bisection method for finding best-fitted rectangle based on area minimization
(Published in Machine Vision and Applications, Springer, 23(6), 2011.)
1. First find the minimum bounding rectangle (by our previous method), which
we call Upper Estimated Rectangle (UPER)
2. Find Under Estimated Rectangle (UNER) by least square method.
3. Find the optimal rectangle by bisection method using corresponding vertices
of UPER and UNER.
H G
L K
D C
A B
I J
E F
Example of under, intermediate and

upper estimated rectangles with center
and diagonals
DC
38
Optimal Rectangle Fit
A lg orithm : Optimal fit rectangle
Step 1 : Find the area of the object/region, obj _ area.
Step 2 : Find the vertices (up _ tlx , up _ tl y ), (up _ trx , up _ try ), (up _ blx , up _ bl y )
and (up _ brx , up _ bry ) of UPER and find the length (a) and width (b) of UPER.
Find the area of UPER, up _ area  ab
Step 3 : Find the vertices (un _ tl x , un _ tl y ), (un _ trx , un _ try ), (un _ blx,un _ bl y )
and (un _ brx , un _ bry ) of UNER and find the length (c) and width (d ) of UNER.
Find the area of UNER, un _ area  cd .
Step 4 : Initially,let the parameter old _ area  un _ area.
Step 5 : Find the vertices of the intermediate rectangle by bisection method between
the corresponding vertices of UPER and UNER, say (mid _ tlx , mid _ tl y ),
mid _ trx , mid _ try , mid _ blx , mid _ bl y  and mid _ brx , mid _ bry .
Step 6 : Find the length (e), width ( f ) and the area mid _ area  ef of the
intermediate rectangle in the current iteration. DC
39
Step 7 : If mid _ area  obj _ area then replace the corresponding vertices of
UPER by the vertices of intermediate rectangle in the current iteration. i.e.
(up _ tl x , up _ tl y )  (mid _ tl x , mid _ tl y ), (up _ trx , up _ try )  (mid _ trx , mid _ try ),
(up _ bl x , up _ bl y )  (mid _ bl x , mid _ bl y ), (up _ brx , up _ bry )  (mid _ brx , mid _ bry )
Also replace the area of UPER as the area of intermediate rectangle. So intermediate
rectangle become UPER and up _ area  mid _ area.
Step 8 : If mid _ area  obj _ area then replace the corresponding vertices of
UNER by the vertices of intermediate rectangle in the current iteration. i.e.
(un _ tl x , un _ tl y )  (mid _ tl x , mid _ tl y ), (un _ trx , un _ try )  (mid _ trx , mid _ try ),
(un _ bl x , un _ bl y )  (mid _ bl x , mid _ bl y ), (un _ brx , un _ bry )  (mid _ brx , mid _ bry )
Also replace the area of UNER as the area of intermediate rectangle. So intermediate
rectangle become UNER and un _ area  mid _ area.
Step 9 : If old _ area  mid _ area  0 then old _ area  mid _ area and go to Step 5.
Step 10 : If old _ area  mid _ area  0 then corresponding intermediate rectangle
will be the optimal rectangle and STOP. DC
40
(a) Binary image (b) Fitted rectangles
DC
41
(a) Binary noisy image (b) Rectangles fitted image
DC
42
(a) PAN image (b) Segmented binary image
DC
43
(c) Area filtered (d) UPER and UNDER

and filling image fitted image
DC
44
(e) Optimal rectangle

fitted image
DC
45
Circular Fit
• In the literature various algorithms are available. Journal: Pattern
Recognition, Pattern Recognition Letters, IEEE Trans. on System, Man and
Cybernetics, IEEE Trans. on Image Processing etc.
One proposed Method: (Published in Defence Science Journal, Sep 2008)
Assume that the area of the object and fitting circle area are same.
Let A be the area of the object and r be the radius of the fitting circle.
A
Therefore, A  r  r 
2

1 n 1 n
The centroid is xc   xi and yc   yi , where n is the number of
n i 1 n i 1
border points of the object and  xi , yi  be the position of those border
points of the object.
DC
46
Circular Fit
Published in CVGIP
xi , yi  be the border points of the object, i  1,2,.., n. Let R
 
be the raidus and x, y be the center of the fitting circle. Now
 
find R and x, y ?
     
n 2
Define error function e( R, x, y )   R 2   xi  x  yi  y

2 2
i 1
     
n 2
e( R, x, y )
  R 2  xi  x  yi  y
2 2
J ( R , x, y ) 
 2
i 1
Now J ( R, x, y ) should be minimized.

J J J
 0,  0,  0 gives the solution of R, x, y
R x y
DC
47
Circular Fit
Other proposed Method: (Published in Pattern Recognition Letters, 20010)
1 n 1 n
1. Compute centroid x   xi ; y   yi ; where ( xi , yi ) are the
n i 1 n i 1
border points of the object
2. Equation of the circle with center (0,0) will be x 2  y 2  r 2
 
n
3. Total error for all border points will be E   xi 2  yi 2  r 2
i 1
4.
E
r
1 n 2

 0 gives r   xi  yi 2
2
n i 1

DC
48
Circular Fit
(a) Binary image (b) Fitted circle
DC
49
Circular Fit
(a) PAN image (b) Segmented

image
DC
50
Circular Fit
(c) Filtered (d) Circle fitted

image image
DC
51
Elliptic Fit
Proposed Method: (Published in Defence Science Journal, Sep 2008)
xi yi  are n border points of the object. Let A be the area of the object.
  1 n 1 n 
Centroid x, y    xi ,  yi . Let  be angle of major axis i.e.
 n i 1 n i 1 
  
n
2 xi  x yi  y
orientation of the object. tan 2  i 1
and
 x  x    y 
n n
2 2
i i y
i 1 i 1
n
   
2
  
n n
 xi  x   yi  y  cos 2  2 xi  x yi  y sin 2  0
2
 i 1 i 1  i 1
Now the area of the fitted ellipse is ab, where a and b are major
and minor axes. Now the equation of the ellipse is
x2 y2
2
 2  1, with center at (0,0). We assume that A  ab. Now
a b
A
b  . Putting the value b in equation of ellipse we get
a
 2 x 2b 4  A2b 2  A2 y 2  0. Find b 2 and correspondingly find a. DC
52
Elliptic Fit
Other proposed Method: (Published in Pattern Recognition Letters, 2010)
1 n 1 n
1. Find centroid of the object : x   xi ; y   yi
n i 1 n i 1
where ( xi , yi ) are n border points of the object
n
2  xi  x  yi  y 
i 1
2. Find orientation of the object : tan 2 
  x  x  
n
i
2
  yi  y 2
i 1
x2 y 2
3. Equation of the ellipse origin at (0,0) : 2  2  1
a b
 xi n 2 2 2
yi 
4. Total error for all border points : E   1  2  2 

i 1  a b 
DC
53
Elliptic Fit
S S
5.  0 and  0 gives
a b
2
n n  n
2 2
 x i  yi 4 4
   x i y i 
i 1 i 1  i 1 
a2  n n n n
 xi 2
 y i   xi yi
4 2 2
 i
y 2
i 1 i 1 i 1 i 1
2
n n  n
2 2
 x i  yi 4 4
   x i y i 
i 1 i 1  i 1 
b2  n n n n
 xi 4
 yi   xi yi
2 2 2
 i
x 2
i 1 i 1 i 1 i 1
DC
54
Elliptic Fit
(a) Binary image (b) Fitted ellipse
DC
55
Elliptic Fit
(a) Segmented (b) Fitted ellipse

image
DC
56
Edge Detection
DC
1
Importance
• Boundaries of objects are important

in perception
• Boundaries define objects
• Difference between object and the
surroundings
• Intensity is the key
DC
2
Edge Basics
• Intensity discontinuity in the image

• Attributes
– Location (x,y)
– Magnitude
– Orientation
DC
3
Gradients
• Intensity discontinuity in the image
• Attributes
– Location (x,y)
– Magnitude
– Orientation
 f 
 G x   x 
G[ f ( x, y )]      f 
G y   
 y 
DC
4
Gradients
• Two properties are associated with the

gradient
– Vector G[f(x,y)] points in the direction
of maximum rate of change of f(x,y)
– Magnitude of the gradient gives the
maximum rate of change in the
direction of G
– Magnitude of the gradient is
independent of the direction of the
edge (isotropic)
DC
5
Edge Operators
• Gradient Edge Operators
f f
1  2 
x y
Magnitude of the edge :   

2
1
2
2
1 2
Orientation of the edge : tan ( )
1
DC
6
Edge Operators
• Discrete Case
f f ( x  x, y )  f ( x, y )
1   lim
x x 0 x
f f ( x, y  y )  f ( x, y )
2   lim
y y 0 y
1  f ( x  1, y )  f ( x, y )
 2  f ( x, y  1)  f ( x, y )
DC
7
Edge Operators
Three techniques for detecting edges:
• By subtracting a smoothed image from its original.
• By calculating spatial derivatives.
• By using edge detecting template operator.
j
Pixel values of a sub-area of an image
i
w1 w2 w3 i -1, j -1 i -1, j i -1, j +1
Z1 Z2 Z3
w4 w5 w6 i , j -1 i,j i , j +1
Z4 Z5 Z6
w7 w8 w9 i +1, j -1 i +1, j i +1, j +1
Z7 Z8 Z9
Template weights 8-neighbors of (i , j ) pixel
9
Image
Response : Z   wi Z i
i 1
DC
8
Edge Operators
The gradient of f ( x, y ) at co - ordinate ( x, y ) is defined as the vector
2 2
f / x   f   f 
f    and f      
f / y   x   y 
The magnitude can be approximated at point Z 5 in a number of ways :
f  Z 5  Z8 2  Z 5  Z 6 2 Also f  Z5  Z8  Z5  Z 6
Roberts Derivative Operator:

f  Z 5  Z 9  Z 6  Z 8  f (i, j )  f (i  1, j  1)  f (i, j  1)  f (i  1, j )
Roberts Template Operator:
1 0 0 1
0 -1 -1 0
DC
9
Edge Operators
• Gradient is estimated in eight direction for a 3  3 mask.
• The convolution result of greatest magnitude indicates the gradient
direction.
Sobel Derivative Operator:
f  ( Z 7  2 Z 8  Z 9 )  ( Z 1  2 Z 2  Z 3 )  ( Z 3  2 Z 6  Z 9 )  ( Z 1  2 Z 4  Z 7 )
Sobel Template Operator:
1 2 1 -1 -2 -1 1 0 -1 -1 0 1
0 0 0 0 0 0 2 0 -2 -2 0 2
-1 -2 -1 1 2 1 1 0 -1 -1 0 1
0 1 2 0 -1 -2 2 1 0 -2 -1 0
-1 0 1 1 0 -1 1 0 -1 -1 0 1
-2 -1 0 2 1 0 0 -1 -2 0 1 2
DC
10
Edge Operators
Prewitt Derivative Operator:
f  ( Z 7  Z 8  Z 9 )  ( Z 1  Z 2  Z 3 )  ( Z 3  Z 6  Z 9 )  ( Z 1  Z 4  Z 7 )
Prewitt Template Operator:
1 1 1 -1 -1 -1 1 0 -1 -1 0 1
0 0 0 0 0 0 1 0 -1 -1 0 1
-1 -1 -1 1 1 1 1 0 -1 -1 0 1
0 1 1 0 -1 -1 1 1 0 -1 -1 0
-1 0 1 1 0 -1 1 0 -1 -1 0 1
-1 -1 0 1 1 0 0 -1 -1 0 1 1
DC
11
Edge Operators
Robinson Template Operator:
1 1 1 -1 -1 -1 1 1 -1 -1 1 1
1 -2 1 1 -2 1 1 -2 -1 -1 -2 1
-1 -1 -1 1 1 1 1 1 -1 -1 1 1
1 1 1 1 -1 -1 1 1 1 -1 -1 1
-1 -2 1 1 -2 -1 1 -2 -1 -1 -2 1
-1 -1 1 1 1 1 1 -1 -1 1 1 1
DC
12
Edge Operators
Kirsch Template Operator:
3 3 3 -5 -5 -5 3 3 -5 -5 3 3
3 0 3 3 0 3 3 0 -5 -5 0 3
-5 -5 -5 3 3 3 3 3 -5 -5 3 3
3 3 3 3 -5 -5 3 3 3 -5 -5 3
-5 0 3 3 0 -5 3 0 -5 -5 0 3
-5 -5 3 3 3 3 3 -5 -5 3 3 3
DC
13
Edge Operators
Frei and Chen Template Operator:
1 2 1 1 0 -1
0 0 0 2 0 -2
-1 -2 -1 1 0 -1
0 -1 2 2 -1 0
1 0 -1 -1 0 1
-2 1 0 0 1 -2
DC
14
Laplacian Edge Operator
• Mathematical approximation
• Gives only magnitude
• No orientation
 f  f 2 2
L ( x, y )  2  2
x y
L(i, j )  4 f (i, j )  [ f (i, j  1)  f (i, j  1)  f (i  1, j )  f (i  1, j )]
DC
15
Laplacian Edge Operator
-1 0 -1 -1 -1 1
0 4 0 -1 8 -1
-1 0 -1 -1 -1 1
4-Laplacian Template 8-Laplacian Template

operator operator
DC
16
Finding Boundaries
DC
17
Problem in Binary Image Edge
Operation
(a) Two circular shape objects (b) Edges of these circles are
one pixel apart from each other connected after edge operation
Best solution is morphological edge operator to overcome

the problem
DC
18
Problem in Binary Image Edge Operation
Our proposed method: (published in Pattern Recognition, 2007)
(a) (b) (c) (d)

(a) Upper left, (b) Upper right, (c) Lower left, (d) Lower right
quadrant positions of 22-sub-template within 33-template
We assume that the center pixel in case is a 1 and the other pixels
in the 22-sub-template may appear in any combinations of 0 and
1. Four possible types of configurations are possible in these sub-
templates.
Type 1: All three-pixel values are 0.
Type 2: Two of them are 0 and one pixel value is 1.
Type 3: Two of them are 1 and one pixel value is 0.
Type 4: All three-pixel values are 1. DC
19
Fig. (1) – (28): Different 22-sub-templates within 33-

DC
template
20
Figure shows an example of an object with several holes. Clearly the

points P and Q shown in figure are edge points. Such edge pixels are
often hard to detect. However, these points are easily identified using
templates shown in Fig. (17) and (26) respectively.
DC
21
Region of interest
Region of interest
(A) Original Binary Image (B) Edge Image by Roberts

Operator
DC
22
(C) Edge Image by Sobel (D) Edge Image by proposed

Operator Operator
DC
23
Edge thresholding
• Too many edges

– Many are for small changes
• Remove “less significant” edges
– Thresholding
• Approaches
– Guess: A certain percentage of the max
• 10% is a good guess
– Percentage: Keep only a certain percent of
the total number of edges
DC
24
Notion of edge
Intensity function Sharp Change
The profile that an edge (transition from dark to
light) is modeled as a smooth, rather than as an
abrupt, change of gray value
1st Derivative Peak

The 1st derivative of the gray level profile is +ve at
the leading edge of a transition, -ve at the trailing
edge and zero in areas of constant gray level
2nd Derivative Zero Crossing

The 2nd derivative is +ve when transition
associate with dark side of the edge, -ve when
transition associate with light side and zero in
areas of constant gray level
DC
25
Gaussian Mask
We know the equation of Gaussian function is :
DC
26
Gaussian Mask
5x5 Gaussian kernel DC

27
Gaussian Mask
Now , we can see that all the values are very small . If we
normalize all the values by dividing all the values by 0.003 , we
will get the following table :
5x5 Gaussian Kernel after Normalization

DC
28
Gaussian Mask
Note: Image quality depends on Kernel size
The Original Image Wasp Gaussian Kernel 3×3

Weight 9.25
DC
29
Gaussian Mask
Wasp Gaussian Kernel 7×7 Wasp Gaussian Kernel 9×9

Weight 9.25 Weight 9.25
DC
30
Gaussian Mask
Wasp Gaussian Kernel 17×17 Wasp Gaussian Kernel 19×19

Weight 9.25 Weight 9.25
Note: σ is the parameter of the function. The Filter values

will be changed if we change the value of σ
DC
31
Marr-Hildreth Operator
• Sobel, Prewitt, Kirsch etc were based on
convolution in very small neighborhoods and
worked well only for specific images
• Disadvantage of these edge detectors is their
dependence on the size of the object and
sensitivity to noise
• New approach required
- Marr-Hildreth Operator
- Canny Operator
DC
32
• Based on the zero-crossing of the second
derivative
• Agree with the fact that edge corresponds to
an abrupt change in the image function
• First derivative of the image should have an
extremum at edge position in the image
• Second derivative should be zero at the same
position
• Much easier to find a zero-crossing position
than an extremum
DC
33
• Intensity changes occur at different scales

at different points in the image
• Edge Detection : Differential operator (1st
or 2nd)
• Tuned to act in any scale
– Blurry edges
– Sharp edges
DC
34
• Blur the image

Convolution with a Gaussian
• Compute the second derivative
• Threshold the image
+ve: White; -ve: Black
• Find the zero crossings
transition from black to white
DC
35
Convolution
Continuous Case
h ( x, y )  f ( x , y ) * g ( x , y )
 
h ( x, y )    f ( x' , y ' ) g ( x  x' , y  y ' )dx' dy '
 
Discrete Case
n m
h(i, j )   f (k , l )g (i  k , j  l )
k 1 l 1
DC
36
Convolution
h(i, j )  Ap1  Bp2  Cp3  Dp4  Ep5  Fp6  Gp7  Hp8  Ip9
DC
37
Marr Hildreth Operator
Step 1 : I  G * I
'
x 2  y2
-
G ( x, y )  e 2 2
Step 2 : I   I'' 2 '
  2 2
  2 2
2
x y
I   I   (G * I )  ( G ) * I
'' 2 ' 2 2
DC
38
Marr Hildreth Operator
r2 r2
 2 1  2
Let r  x  y so G (r )  e
2 2 2 2
Now G(r )   re 2
 2
r2
1 r  2  2
and the second derivative G(r )  2  2  1e 2
  
Now returning to the original co - ordinates x, y and introducing
a normalizing multiplicative coefficient c, we get a convolution
mask of a LoG operator :
x2  y 2
 x  y 
2 2 2
 
h( x, y )  c e 2 2
, where c normalizes the sum of
  4

mask elements to zero.
DC
39
G 2
One Dimensional Two Dimensional DC

40
Laplacian of Gaussian Mask
17X17 Mask
DC
41
Laplacian of Gaussian Mask
5X5 Mask DC
42
Blurring
Image =8 =4

DC
43
Zero Crossing
Image After Convolution
+ve: white
-ve: black
After Thresholding Zero Crossings DC

44
Zero Crossing
Image After Convolution
+ve: white
-ve: black
After Thresholding Zero Crossings DC

45
Canny Edge Detection
DC
46
1. Smooth the image with a Gaussian filter to reduce noise and unwanted
details and textures g (m, n)  G (m, n) * f (m, n) where
1  m2  n2 
G  exp  2 
2   2 
2. Compute gradient of g (m, n) using any of the opertors (Roberts, Sobel) of
get M (m, n)  g m2 (m, n)  g n2 (m, n) and  (m, n)  tan 1[ g n (m, n) / g m (m, n)]
 M (m, n) if M (m, n)  T
3. Threshold M : M T (m, n)  
0 otherwise
where T is so chosen that all edge elements are kept while most of the
noise is supressed
DC
47
4. Supress non - maximum pixels in the edges in M T obtained above to thin
the edge ridges. To do so, check to see whether each non - zero M T (m, n) is
greater than its two neighbors along the gradient direction  (m, n). If so, keep
M T (m, n) unchanged, otherwise, set it to 0
5. Threshold the previous result by two different thresholds T1 and T2 (where

T1  T2 ) to obtain two binary images
6. Link edges segments in T2 to form continuous edges. To do so, trace each

segment in T2 to its end and then search its neighbors in T1 to find any edge
segments in T1 to bridge the gap untill reaching another edge segment in T2
DC
48
Non-Max Suppression:
 Consider the three edge points. Assume point (x,y) as the
point having the largest gradient of edge.
 Check for the edge points in the direction perpendicular to
the edge and verify if their gradient is less than (x,y).
 If the values are less than (x,y) gradient, then we can
suppress those non-maxima points along the curve.
DC
49
Hysteresis Thresholding:
If the gradient at a pixel is :
– Above “High” declare it as an ‘edge pixel.’
– Below, “Low” declares it as a ‘non-edge pixel.’
– Between “low” and “high.”
•Consider its neighbors iteratively then declare it an “edge
pixel” if it’s connected to an “edge pixel” or via pixels
between “low” and “high.”
DC
50
DC
51
(a) (b)
(c)
(d)
(a) Noisy image, (b) edge by

Sobel, (c) edge by Prewitt, (d)
edge by Canny DC
52
DC
53
Problems with Edge Detection
• Arbitrary thresholding
• Thick edges
– Use Edge thinning
• Broken edges
– Use some parametric methods
• False edges
– Filtering
DC
54
Segmentation
DC
1
Segmentation
• Partition an image into a set of regions

– Gray scale or color image
• Partition into meaningful areas
– Crop, urban, forests (satellite image)
– Eye, hair, skin (face images)
• Partition covers the image
– Meaningful to look at some regions in some
applications
• Defining regions is equivalent to defining the
boundary of regions
DC
2
Goals
• Regions can then be used for analysis

– Each region is a meaningful object
– Each region is part of an object
• Change of representation
– Pixels to regions
– Higher level units
DC
3
Bottom Up Approaches
• Start with pixels and group into regions

• No special domain knowledge
• General purpose
• Results are not always good
– Difficult problem
• Use of domain knowledge may be
necessary for good results
DC
4
Histogram Based methods
• Many segmentation algorithms make

multiple passes of the image
• Histogram based methods make only one
pass
• Very little computation
• Compute the histogram of the image
• Use the histogram to determine the
thresholds for the different regions
DC
5
Histogram Based Methods
• Typical gray scale image is multimodal

• Need to use automatic thresholding
techniques
– Seek to find the valleys in the histogram
Distinct Modes Overlapped Modes DC

6
Histogram Based Methods
Single Threshold Technique Multiple Threshold Technique
T T1 T2
1 for f (i, j )  T  1 for 0  f (i, j )  T1

g (i, j )  
0 for f (i, j )  T  2 for T  f (i, j )  T
 1 2
g (i, j )   
 N for T  f (i, j )  T
 N 1 N
0 Otherwise DC
7
Iterative Threshold Selection
 Select an initial estimate of the threshold, T .

- Good choice is the average intensity of the image
 Partition the image into two groups R1 and R2 by T .
 Calculate the mean gray values 1 and  2 of R1 and R2
1
 Select a new threshold : T  1   2 
2
 Repeat steps 2 - 4 until 1 and  2 are not changed in
successive iterations.
DC
8
Automatic thresholding
• Describe the histogram as a probability

function
{(r , c) | Image(r , c)  i}
P(i ) 
RC
• If the histogram is bimodal, the goal is to

find the threshold that separate the two
modes
DC
9
• Each threshold (T) determines

– Variance of the group of values less than T
– Variance of the group of values greater than T
• Best threshold
– Minimize the weighted sum of the within-
group variances
– Weight: Probability of the respective group
DC
10
First Method
The mixture probability density function is given by
P( x )  P1 p1 ( x )  P2 p 2 ( x )
For Gaussian case
P1   ( x  1 ) 2  P2   (x  2 )2 
P( x )  exp   exp  
2 1  2 1 2 2  2 2
2 2
 
P1 and P2 are the a priori probabilities of the two levels
P1  P2  1
Let us assume that the background represents as dark region and
object represent as bright region. So in this case 1<2 and we may
define a threshold, T, such that all pixels with gray levels below T are
considered background and the opposite as object DC
11
The probability of classifying an object point as a background point is
T
E1 (T )  p

2 ( x )dx
The probability of classifying an background point as a object point is


E 2 (T )   p1 ( x )dx
T
So, the overall probability of error is given by

E (T )  P2 E1 (T )  P1E2 (T )
Find T for which this error is minimum. So dE(T)/dT = 0 gives
P1 p1 (T )  P2 p 2 (T )
DC
12
Now putting
1   (T   1 ) 2 
p1 (T )  exp   and
2 1  2 1
2

1   (T   2 ) 2 
p 2 (T )  exp  
2 2  2 2
2

Taking log we get
AT  BT  C  0
2
B  2(  1 2   2 1 )
2 2
A 1  2
2 2
where
  2 P1 
C   1  2   2 1  2 1  2
2 2 2 2 2 2
ln 
  1 P2 
DC
13
If the variance are equal, a single threshold is sufficient:
1   2  2
 P2 
T  ln 
2 1   2  P1 
If the prior probabilities P1 = P2
1   2
T
2
DC
14
Otsu’s Method:
Let [1,2,.....,L], L gray levels in the image. The number of pixels at level i is
denoted by ni and the total number of pixels by N =n1+n2+......+nL.
ni L
pi  , pi  0  pi  1
N i 1
Now suppose there are two classes C0 and C1. By a threshold at level k C0
contains pixels with levels [1,.......,k] and C1 contains pixels with levels
[k+1,.......,L]. Then the probability of class occurrence and the class mean
levels, respectively, are given by,
k
 0  P (C0 )   pi   (k ),
i 1
L
 1  P (C1 )   pi  1   (k )
i  k 1 DC
15
k k
 (k )
 0   iP (i | C0 )   ipi /  0  ,
i 1 i 1  (k )
L  T   (k )
L
 1   iP(i | C1 )   ip i  1 
i  k 1 i  k 1 1   (k )
k L
p, T   ipi
k
where  (k )  i  ( k )   ip i and
i 1 i 1 i 1
We can easily verify the following relation for any choice of k:
 0  0   1 1   T ,  0   1  1
DC
16
In order to evaluate the goodness of the threshold (at level k),
we shall introduce the following discriminant criterion
measures (or measure of class separatibility) used in the
discriminant analysis:
B 2
T B2 2
  2 ,   2 ,  2
w w T
where  w 2   0 0 2  1 12 ,
L
T 2
 (i   T ) pi
2
and
i 1
 B   0 (  0   T )   1 (  1   T )   0 1 (  1   0 )
2 2 2 2
DC
17
The class variances are
k k
 0   (i   0 ) P ( i | C 0 )   (i   0 ) 2 p i  0
2 2
i 1 i 1
L L
1 
2
 (i   1 ) P (i | C 1 ) 
2
 (i  1 ) 2 pi  1
i  k 1 i  k 1
 is the simplest measure with respect to k.

[  T  ( k )   ( k )] 2
B 2

 (k )[1   (k )]
The optimal threshold k* is,
 B ( k *)  max  B ( k )
2 2
1 k  L DC
18
Fig: (a) Original ENVISAT SAR image. (b) Segmentation of original image using single threshold. (c)
Segmentation of original image using the proposed segmentation technique. (d) Features enhanced
image. (e) Segmentation of enhanced image using a single threshold. (f) Segmentation of enhanced D C
image using proposed technique. 19
(a) Original RADARSAT image. (b) Features enhanced image. (c) segmented image using
proposed algorithm. (d) Thinned image. (e) Image after branch removal. (f) Final linked image. DC
20
Ohlander’s Approach
• Recursive algorithm
• Compute the modes of the histogram for
the whole image
– Derive the clusters
• Compute the histogram for the clusters
– Derive the sub-clusters if needed
• Histogram has multiple modes
• Use a mask to determine the region of
interest
DC
21
Region Based Approach
Regions:
• Each region should be uniform, and
homogeneous with respect to some
characteristic
– Color, gray scale, texture etc
• Region interiors should be simple without many
small holes
• Adjacent regions should have significantly
different values/characteristics
• Boundaries of the regions should be smooth
• Regions should cover the image
DC
22
Region Based Segmentation
 The objective of region based segmentation is a partition an
image into regions
Basic Formulation:
 Image segmentation partitions the set R into the subsets Ri, i

= 1, 2, …, N, having the following properties:
N
1.  Ri  R,
i 1
2. Ri is a connected region, i  1,2,..., N

3. Ri  R j   for all i and j , i  j
4. P( Ri )  TRUE for i  1,2,..., N and
5. P ( R i  R j )  FALSE for i  j DC
23
Region Growing by Pixel Aggregation
0 0 5 6 7 A A B B B
1 1 5 8 7 A A B B B
0 1 6 7 7 A A B B B
2 0 7 6 6 A A B B B
0 1 5 6 5 A A B B B
(a) Original Image (b) At T = 3
A A A A A
A A A A A
A A A A A
A A A A A
A A A A A
(c) At T = 8 DC
24
Region Growing Segmentation
Region Growing:
 Region growing is a procedure that groups pixels

or subregions into larger regions.
 The simplest of these approaches is pixel

aggregation, which starts with a set of “seed”
points and from these grows regions by appending
to each seed points those neighboring pixels that
have similar properties (such as gray level, texture,
color, shape, entropies, fractal dimensions etc.).
 Region growing based techniques are better than

the edge-based techniques in noisy images where
edges are difficult to detect. DC
25
 Region growing is a technique that starts with a
set of seed pixels and expands them into regions
by adding neighboring pixels that satisfy a
similarity measure.
 The main challenges of region growing are

choosing the appropriate seeds, defining the
similarity measure, and controlling the region
size and shape.
DC
26
Select the initial
seed point
Neighbors are 8-
Select next
neighbors of seed
seed pixel
pixel
Satisfying pixels
Compare neighbors to
are assigned to
the seed pixel with
the region and are
homogeneity criterion
the new neighbors
Neighbor pixels True

satisfy the
homogeneity
False condition?
DC
27
DC
28
DC
29
DC
30
Region-Oriented Segmentation
DC
31
Region-Oriented Segmentation
DC
32
Double Thresholding for Region
Growing
• Select two thresholds T1 and T2.
• Partition the image into three regions based on
these threshold values.
- R1 containing all pixels with gray values
below T1. R2 containing all pixels with gray
values between T1 and T2. R3 containing all
pixels with gray values above T2.
• Visit each pixel assigned to region R2. If the pixel
has a neighbor in region R1, then reassign the
pixel to region R1.
• Repeat step 3 until no pixels are reassigned
• Reassign any pixels left in region R2 to region R3 DC
33
Region Splitting and Merging
R
R21 R22
R1 R R2R
23 24 R1 R2 R3 R4
R3 R4
R21 R22 R23 R24
(a) Partitioned Image (b) Corresponding Quad tree
For region R, P ( R )  FALSE then split the region R

If P ( R )  TRUE then not to split
Merge two regions Ri and Rk if P ( Ri  Rk )  TRUE
Stop when no further merging or splitting is possible DC
34
Region Splitting:
 Region growing starts from a set of seed points.
 An alternative is to start with the whole image as

a single region and subdivide the regions that do
not satisfy a condition of homogeneity.
Region Merging:
 Region merging is the opposite of region splitting.
 Start with small regions (e.g. 2x2 or 4x4 regions)

and merge the regions that have similar
characteristics (such as gray level, variance).
 Typically, splitting and merging approaches are

used iteratively. DC
35
Edge Based Segmentation
Find the edges
Selection of appropriate threshold to

remove non-significant edges
Edge linking techniques can be employed

to bridge short gaps of broken boundary
DC
36
 Edge-based segmentation is a popular image

processing technique that identifies the edges of
various objects in a given image.
 It helps locate features of associated objects in

the image using the information from the edges.
DC
37
 Edge detecting an image significantly reduces the
amount of data and filters out useless information, while
preserving the important structural properties in an
image.
 Edge-based segmentations rely on edges found in an

image by edge detecting operators.
 But the image resulting from edge detection cannot be

used as a segmentation result.
 The final aim is to reach at least a partial segmentation.
DC
38
DC
39
Edge Relaxation:
 Borders resulting from the previous method are strongly

affected by image noise, often with important parts
missing.
 Considering edge properties in the context of their
mutual neighbors can increase the quality of the
resulting image.
 For example, A context example is a weak edge
sandwiched between two strong edges. It is highly
probable that this inter-positioned weak edge should be
a part of a resulting boundary. On the other hand, if a
borderless edge—even one that is strong—is placed by
itself without any supporting context, it is probably not a
component of any boundary.
DC
40
Duda and Hart Curve Fitting Edge Linking Method
End points are connected by straight lines
Find the point of greatest departure from the
straight line
If the separation of this point is too large, the
point becomes an anchor point for two straight
line segments.
Repeat until data points are well fitted by line
segments B
A
D
DC
41
Example
C
A
B
(a) First Step (b) Second Step
(c) Third Step

DC
42
Vector-based edge Linking Method (Published in IEEE-
ICPR 1996) – D. Chaudhuri
 Find end pixels of each component and the direction near

the end pixels. Let A and B be two end points and corresponding
directions near the end be A and B. Also let direction from A to B be C.
 Join A and B by a straight line if and only if
- Angle between A and  B is small
- Angle between A and C is small
- Angle between C and  B is small
- Distance between A and B is small
DC
43
Boundary Detection
Goal of intermediate level processing is to

identify regions which correspond to
objects or parts of objects
Approach
Segmentation : Find the pixels (inside) which
belong to an object
Boundary Detection : Find the boundary of

the objects
DC
44
Boundary Detection
 Approach 1
First find the regions using segmentation methods
Derive the boundary by boundary following algorithms
(chain code)
 Approach 2
Directly find the regions from the intensity image
(Canny edge detector and linker)
 Approach 3
Link edges to find the boundary points of regions
DC
45
Fitting Models to Segments
Least Square Method
• For determining the parameters of the best
mathematical model fitting observed data
• Data might be obtained using one of the region or

boundary segmentation methods
• Model fit and parameters determined means test

for acceptability of the fitted model with given data
• Good fit means detected of an object with some

specified shape
DC
46
Fitting Models to Segments
Least Squares Error Criteria:
The measure of how well a model y  f ( x) fits a set of n observations
n
 xi , yi , i  1,..., n is LSE    f ( xi )  yi 2 . The best model y  f ( x)
i 1
is the model with the parameters minimizing the criteria.
Root-Mean-Square Error (RMSE):

RMSE is the average difference of observations from the model :
1/ 2
 
RMSE    f ( xi )  y j  / n 
n
2
 j 1 
Max-Error Criteria:
n
MAXE  max f ( xi )  yi . NOTE : this measure depends only on the
i 1
worst fit point, but RMS depends on the fit of all of the points. DC
47
Fitting a Straight Line
y y Circular fit is
unbiased
Model is y  c1 x  c0
Linear fit is
Error is e j  y  y j
biased
0 x
0 x
DC
48
Fitting a Straight Line
n
LSE  E (c1 ,c 0 )   (c1 xi  c0  yi ) 2 : Two parameters (c0 , c1 )
i 1
E E
Best fit means error will be minimum i.e.  0 and 0
c0 c1
E  n  n
Now  0    xi c1  nc0   yi  0
c 0  i 1  i 1
E  n 2  n  n
and  0    xi c1    xi c0   xi yi  0
c1  i 1   i 1  i 1
By solving we get
n n n n n n n
 i  yi   xi  xi yi
x 2
n xi yi   xi  yi
c0  i 1 i 1 i 1 i 1
2
and c1  i 1 i 1 i 1
2
n
 n
 n
 
n
n xi2    xi  n xi2    xi 
i 1  i 1  i 1  i 1  DC
49
Split-and-merge technique
Result
(a) Blood image (b) Segmented image
DC
50
Split-and-merge technique
Result
(a) IKONOS image (b) Segmented image

DC
51
Bi-Entropy Segmentation
DC
52
DC
53
DC
54
a. QuickBird image, b
histogram, c by Liscano and
Wong, d Chaudhuri and
Agarwal, e Arifin and Asano, f K-
means algorithm, g Otsu
method, h Kapur et al., i the
proposed algorithm, j K. Vennila
DC
et al. and k Suresh Manic et al 55
a Original CT scan image, b

Liscano and Wong algorithm, c
Chaudhuri and Agarwal
algorithm, d Arifin and Asano, e
K-means, f Otsu method, g Kapur
et al., h the proposed algorithm,
i Vennila et al. and j Suresh DC
Manic et al . 56
Pattern Recognition
DC
1
Pattern Recognition
• Techniques are complex

• Overlap of pattern recognition, computer
vision, machine learning
• Focus is on approaches that have worked
well in computer vision
• Large body of theoretical literature
DC
2
Pattern Recognition
 Applications
 Automatic inspection
 Speech recognition
 Document processing (character recognition)
 Medical diagnostics
 Remote Sensing
 Computer vision
 Personal identity verification, etc
DC
3
Model for Classification
Class : A set of objects with some

important common properties
Class label
Do not have to be identical
Classification: the process of assigning
the class label to an object
Classifier: A device/algorithm that takes
an object as its input and produces the
label as the output
DC
4
PR Model
 Reject class: A generic class of object that can’t

be placed in any known classes
 Sensor/Transducer: Some device to sense the
physical object and output the label and other
description in digital form
 Computer Vision : Sensors that produce 2D
array of pixels are of most interest
 Other sensors used in Pattern Recognition
DC
5
PR General Model
Measurement Feature
Space Space
M FD Decision
Space
DC
6
Feature Extractor
Extracts relevant information from the

sensed data
Usually done in software
Which features are important?
Difficult to know sometimes
Research & Development
Local vs. global
Many image features
DC
7
Feature Extraction
Binary (Shape)
Feature Extraction
Segmentation Gray Level & Entropy Based

Statistical Feature Feature Extraction
Extraction
Preprocessing Fractal Dimension Based
Textural Feature Feature Extraction
Extraction
Single Channel Textural Feature
Extraction for each
channel
Multi Channel
Preprocessing
Sensor for each channel
Color Feature
Problem Extraction
Domain DC
8
Shape Features
• Organization of points in space
• Not related to any intensity or color features
• Common Features
– Centroid
– Area
– Perimeter
– Radius
– Orientation
– Major and Minor Axes
– Elongatedness
– Aspect Ratio
– Bounding box
– Moment
– Compactness
– Complexity of object (No. of perimeter pixels/No. of object pixels) DC
9
Gray level & Statistical Features
• Common Features
– Maximum gray value of object
– Minimum gray value of object
– Mean gray value of object
– Standard deviation of object gray value
– Mean gray level in background
– Difference of mean gray values of object and background
– Contrast ratio
– Bright pixel ratio (No. of pixels within 10% of max
gray/No. of object pixels)
DC
10
Texture Based Features
What is Texture?
• No formal definition of texture exists
• Visual texture is something that everyone
understands, but it is difficult to define
- “Despite its importance in image data, a formal approach or precise
definition of texture does not exist.” (Haralick, 1979)
- “Texture is something composed of closely interwoven elements.”

(Webster’s dictionary)
• Texture is “repeating patterns of local variations in

image intensity which are too fine to be distinguished
as separate objects at the observed resolution” (Jain,
kasturi and Schunck, 1995) DC
11
What is Texture?
• No formal definition of texture exists, intuitively this
descriptor provides measures of properties such as
smoothness, coarseness and regularity
Example: Smooth and Coarse textures
DC
12
Why should we study Texture?
• Texture analysis has been used in various
application:
- Industrial Inspection: Automated inspection problems, applications
include defect detection in images of textiles and automated inspection of
carpet wear and automobile paints
Fig: Texture pairs with identical second order statistics. The bottom halves of
the images consist of texture tokens that are different from the ones in the
top half. (a) Humans cannot perceive the two regions without careful
scrutiny, (b) The two different regions are immediately discriminable by
humans. DC
13
Applications
DC
14
Applications
Example images used in paint inspection. (a) A non-defective

paint which has a smooth texture, (b) A defective paint which
has a mottled look.
DC
15
Applications
Medical Image Analysis
The processing of the ultrasound images of the heart using textural features.
(a) A 128  432 ultrasound image from the left ventricle of the heart. (b) The
fractal dimension used as the texture feature. (c) Image segmentation from a k-
means clustering algorithm. The white region is cluster 2 which corresponds to
the blood. DC
16
Applications
Document Processing
Locating bar code in a newspaper image. (a) A

scanned image of a newspaper that contains a bar
code. (b) The two-class segmentation using texture
features. The bar code region in the image has a DC
distinct texture. 17
Applications
Document Processing
Text/graphics separation using texture information. (a) An

image of a newspaper. (b) The three class segmentation
obtained by texture feature based segmentation algorithm
(c) The regions identified as text.
DC
18
Applications
Remote Sensing
(a) (b)
(a) And (b) are examples of aerial images using SAR sensors
- Distinguishing forests from urban area

- Changes of water body
- Target detection and shape analysis
DC
19
Applications
DC
20
Modeling Texture
• Structural Approaches
– Organize texture as a repeated pattern of a primitive
element (texel). Deal with the arrangement of image
primitives, such as the description of texture based on
regularly spaced parallel lines.
• Statistical Approaches
– View texture as a quantitative measure of the
arrangement of the intensity. Characterized of textures as
smooth, coarse, grainy and so on.
• Syntactic Approaches
– Analyze texture as a grammar of primitive elements and
their organization
DC
21
Statistical Approaches
Gray Level Co-occurrence Matrices (GLCM)
The GLCM is a tabulation of how often different combinations of

pixel brightness values occur in an image. Let P(i, j; d , ) denote
the transition probability from gray level i to gray level j defined by
# {((k , l ), (m, n)  ( Lx  Ly )  ( Lx  Ly ) : (k , l )(m, n)
  , (k , l )  (m, n)  d , I (k , l )  i, I (m, n)  j}
P(i, j; d , ) 
N ( d , )
M (d , )  P(i, j; d , )L L is a GLCM.
DC
22
Order of Texture
 First order texture measures are statistics

calculated from image, like variance, and do not
consider pixel relationships.
 Second order means they consider the
relationship between groups of two pixels in the
image.
 Third and higher order textures considering the
relationships among three or more pixels.
Theoretically possible but not easy to implement.
DC
23
Framework for the GLCM
0 0 1 1
Example: Test image gray levels
0 0 1 1
0 2 2 2
2 2 3 3
Framework matrix:
Neighbor pixel value  0 1 2 3

Reference pixel value 
0 0,0 0,1 0,2 0,3
1 1,0 1,1 1,2 1,3
2 2,0 2,1 2,2 2,3
3 3,0 3,1 3,2 3,3
DC
24
Fill in the matrix framework for the east spatial relationship:
1350 (NW) 900 (N) 450 (NE)
2 2 1 0
0 2 0 0 00 (W) 00 (E)
0 0 3 1
0 0 0 1 450 (SW) 1350 (SE)
900 (S)
• Square: the reference and neighbor pixels range

values are same, so the values at top and side are
same
• Same number of rows and columns as the
quantization level of image. So 256  256 square
matrix for 8-bit data
• Symmetrical around the diagonal DC
25
How to make the matrix symmetrical?
2 0 0 0
Take the transpose of the east 2 2 0 0
matrix to get west matrix
1 0 3 0
Add east and west matrices to 0 0 1 1
get symmetry matrix West
Matrix
4 2 1 0
2 4 0 0
1 0 6 1
0 0 1 2
DC
26
Expressing the GLCM as a probability
Vi , j
Normalization equation : Pi , j  N 1
where Vi , j is the
V
i , j 0
i, j
number of occurance of gray values i and j in a particular

direction. N is the total number of gray value in the image.
.166 .083 .042 0

Normalized
(Horizontal) GLCM: .083 .166 0 0
.042 0 .250 .042
0 0 .042 .083
DC
27
Texture Parameters
L 1 L 1 L 1 L 1
Let  x   i  Pi, j; d ,   x 2   i   x 2  Pi, j; d , 
i 0 j 0 i 0 j 0
L 1 L 1
 y   j  Pi, j; d ,  L 1 L 1
j 0 i 0  y 2    j   y 2  Pi, j; d , 
j 0 i 0
1. Con(d , )  Contrast
L -1 L 1 2
  i  j  Pi, j; d , 
i  0 j 0
2. E (d , )  Energy
L -1 L 1 2
  Pi, j; d , 
i 0 j 0
DC
28
Texture Parameters
3. H (d , )  Entropy
L -1 L 1
 -  P i, j; d , logP i, j; d , 
i  0 j 0
4. C d ,   Correlation
L 1 L 1
 i    j   Pi, j; d , 
i 0 j 0
x y

 x y
5. Ld ,   Local Homogeneity
L -1 L -1
1
  Pi, j; d ,  It should be
j  0 1  i - j
2
i 0
noted that most
6. SE  Sum Entopy  - p sum k  log psum k  cases d = 1 and 
k
= 00, 450, 900 and
where psum k    Pi, j; d ,  with i  j  k
i j
1350.
DC
29
Texture Segmentation
• Use the texture energy measures to
– Segment
– Cluster
DC
30
Classifier
DC
31
Classifier
• Number of classes : m
• Size of the feature vector : d
• K : Knowledge about the class
• Classification
– Using nearest mean
– Maximum a posteriori probability
– Feed forward artificial neural network
DC
32
Pattern Matching (by Minimum
Distance Classifier)
M number of class : Wi , i  1,..., M and N i is
the number of pattern vector from class Wi
d i ( X )  X  mi , i  1,..., M
1
where mi 
Ni
 X , i  1,2,..., M
X Wi
Euclidean distance : d i ( X )   X  mi   X  mi 
2 T
 X X  2 m X  m mi
T T
i
T
i
2
 
Classification : X  Wi if d i ( X )  d j ( X ) for j  i
2
DC
33
Pattern Matching (by Correlation)
y
N
x Origin t
K
J (s, t)
M s
W(x-s, y-t)
f(x, y)
DC
34
Pattern Matching (by Correlation)
Correlation between f ( x, y ) and w( x, y ) is
C ( s, t )   f ( x, y ) w( x  s, y  t ),
x y
s  0,..., M  1 and t  0,..., N  1

 Maximum value indicate best match
Correlation Coefficient :
 [ f ( x, y)  f ( x, y)][w( x  s, y  t )  w]
 ( s, t )  x y
1/ 2
 
 [ f ( x, y )  f ( x, y )]  [ w( x  s, y  t )  w] 
2 2
 x y x y 
DC
35
Nearest Mean Classifier
• Summarize the sample data using a mean vector
(use centroid)
ni
1
xi 
ni
x
j 1
i, j
xi , j is the jth sample feature vector of class i
• Given an unknown sample (feature vector) goal is

to assign to the class it is nearest to mean
DC
36
• f : computes the distance between the feature

vector and the centroid
• K : knowledge of training samples
DC
37
• Put in the reject class if it not close to any

of the classes
• Simple, fast
• Works in many problems
– Each cluster is compact
– The clusters are far apart
DC
38
DC
39
• Problems
– When the structure of
the classes is complex
– Class is separated
(multimodal)
– Mean vector is not
representative
– Separate the class (2)
• Difficult in higher
dimensions
– Problems may remain
DC
40
Clustering
DC
41
Pattern and pattern Class
 Pattern: A pattern is a quantitative or structural

description of an object or some other entity in
an image. A pattern is an arrangement of
descriptors or features.
 Pattern Class: A pattern Class is a family of

patterns that share some common properties.
DC
42
Clustering
• Given a set of patterns the goal is to group them.

• Patterns in a cluster are similar to others in the
same cluster
• Patterns in a cluster are different from patterns
in the other clusters
• Each cluster represents an abstract concept
• A form of unsupervised learning
– No a priori information about the groups
• Prototype: Centroid of the patterns
DC
43
Clustering
XXXXXX
XXXXX
XXX
XXX
Intra Cluster
XXXXXXX
XXXXXXXXX XXX
XX
DC
44
What is Cluster?
Everiit documents some of the definitions of a cluster as follows:
• A cluster is a set of entities which are alike and entities from different
clusters are not alike.
• A cluster is an aggregation of points in the test space such that the

distance between any two points in the cluster is less than the distance
between any point in the cluster and any point not on it.
• Clusters may be described as connected regions of a multi-

dimensional space containing a relatively high density of points,
separated from other such regions by a region containing a relatively
low density of points.
DC
45
Applications of Clustering
 Document Processing
 Man-Machine interfaces
 Remote Sensing data analysis
 Bio-medical data analysis

 Data Mining
 In detection errors in data
 In grouping user queries in database
 Several other areas
DC
46
Clustering
Given a set of patterns   x1 ,..., xn , the ith cluster
C
X i  , i  1..., C ; C  n,  X i  , and no X i   . If
i 1
in addition X i  X j   , for all i and j , i  j , then we

have a hard partition.
There are 2 n 1  1 partitions of n patterns. The number

of partitions of n patterns into K blocks is
1 K K i  K 
  
1   i   n
K ! i 1 i
DC
47
Clustering
• Entails multiple steps

– Pattern generation
– Similarity formulation
– Cluster generation
– Data abstraction
– Cluster validation
DC
48
Clustering
• Hard data and hard clustering

– Treat data as crisp numbers
– Treat the clusters as hard (fixed boundaries)
– K-means (ISODATA)
• Hard data and fuzzy clustering

– Treat data as crisp numbers
– Treat the clusters as soft (fuzzy boundaries)
– Fuzzy C-means
DC
49
Similarity Measures (Metric)
1/ 2
 2
n
Euclidean Distance : d E  X , Y     xi  yi  
 i 1 
n
City - Block Distance : d C  X , Y    xi  yi
i 1
n
Chessboard Distance : d M  X , Y   max xi  yi
i 1
n
1
New Distance : d N  X , Y   xi XY  yi XY   xi  yi
 n  2  i 1,i i XY
n 
 2 
Published in Pattern Recognition 1992, Vol. 25 DC
50
Decision Theoretic Methods
X   x1 , x2 ,..., xn   N - Dimensional pattern vector.
T
Wi , i  1,2,..., M  Pattern Classes

d i ( X ), i  1,..., M  Decision Functions
Property : if a pattern X  Wi then d i ( X )  d j ( X )

j  1,..., M , j  i. The decision boundary separting
class Wi from W j is given by values of X for which
d ij ( X )  d i ( X )  d j ( X )  0
DC
51
Tree of Classification
CLASSIFICATION
NON-EXCLUSIVE EXCLUSIVE (NON-

(OVERLAPPING) OVERLAPPING)
INTRINSIC EXTRINSIC
(UNSUPERVISED) (SUPERVISED)
NON-HIERARCHICAL
HIERARCHICAL (PARTITIONAL)
DC
52
Clustering by Density Estimation
and Mode Seeking
 Densely populated regions are usually viewed as
clusters.
 By finding highest density point called mode.
 The nearest point of each mode can be considered
as a cluster center.
 Partition is obtained by assigning each pattern to
its closest cluster center.
DC
53
Density Estimation
Kn
fn ( X ) 
nVn
Kn is the number in a region Vn and n is the total number of
patterns.
Note:
 In the Parzen window approach Vn is a function of n.
 In nearest Neighbor approach Kn is a function of n.
 For convergence Vn is inversely proportional to n in both
the methods and Kn is usually taken as proportional to n.
DC
54
Parzen Density Estimator
1 n
 x  xi 
f n ( x) 
nhn
 K   where x, xi  , i  1,..., n be a sequence
i 1  hn 
of independent and identically distributed random variables.
{hn } is a sequence of positive constant satisfying hn  0 and
nhn   as n   and K ( y ) is a function on  such that
Sup K ( y )  ,
y 
 K ( y) dy  
lim y K ( y )  0
y   K(y)dy  1

Normally, Parzen window is consider as

hn  n  where 0    1 DC
55
Data Driven Procedure for Density
Estimator
Published in Pattern Recognition, Vol. 26, 1996
(a) Densely Populated Pattern (b) Sparsely Populated Pattern
Both objects are same number of patterns

Question: whether previous method for selection hn is
OK? DC
56
Data Driven Procedure for Density
Estimator
1 n
 x  xi 

f n ( x)  q
nhn
K 
 h   where x , xi   q
, i  1,..., n be a sequence
i 1  n 
of independent and identically distributed random vectors.
1/ q
 ln 
hn    ; q is the dimension and ln is the length of the MST
n
 1
if x j  xij  1 j  1,2,..., q
K(X )   2 q
0 otherwise
DC
57
Hierarchical Clustering
 Begin with Proximity matrix
 Graphical representation is the main feature
HIERARCHICAL
AGGLOMERATIVE DIVISIVE
DC
58
Agglomerative Method
Algorithm:
 Compute similarity matrix and initialize each
pattern in a cluster.
 Find closest pair of clusters and merge them.

Update the similarity matrix to reflect the merge.
 If all the patterns are in one cluster, stop.

Otherwise go to Step 2.
DC
59
Agglomerative Method
Algorithm: Single-link Algorithm

In the Single - link algorithm, the distance between two
clusters C1 and C2 is the minimum of the distances d ( x, y ),
where x  C1 and y  C2
Algorithm: Complete-link Algorithm

In the Complete - link algorithm, the distance between two
clusters C1 and C2 is the maximum of the distances d ( x, y ),
where x  C1 and y  C2
DC
60
Dendrogram for Single-link Algorithm
F G
F2
B
A C D E
F1
(C,D)
(D,F)
Similarity
(A,C)
A B C D E F G DC
61
Dendrogram for Complete-link Algorithm
F G
F2
B
A C D E
F1
(A,G)
(D,G)
Similarity
(A,B)
A B C D E F G DC
62
Divisive Method
 Reverse of Agglomerative method.
 Hierarchical algorithms generate a nested

sequence of partitions of the data at different levels
of hierarchy.
 In divisive methods, if two patterns are assigned to
different clusters at any level l, then they cannot
belong to the same cluster at subsequent levels.
 In agglomerative methods, if two patterns belong to

the same cluster at any level l, then they cannot be
separated at the subsequent levels.
DC
63
Partitional Method
 Partitional clustering algorithms generate a

hard or a soft partition of the data.
 The most popular partitional clustering

algorithm is the K-means algorithm
DC
64
Mathematical Construction
Suppose S  X 1 ,..., X n    q be given set and has some partition
into K clusters C1 ,..., C K such that cluster C K has nK patterns and
K
each pattern is in exactly one cluster. So  ni  n
i 1
nK
1
Mean vector is X ( K ) 
nK
 i i is the ith pattern
X (K )
i 1
, X (K )
belonging to cluster C K . The squared error for the cluster
  X 
nK T
C K is e   X
2
K i
(K )
X (K )
i
(K )
 X i( K )
i 1
K
So the squared error for the clustering is Ei2   ei2
i 1 DC
65
Partitional Method (K-Means)
Basic Steps:
1. Find the seed point of each K cluster.
2. Find a similarity measure.
3. Cluster the data on the basis of seed points and
similarity measure.
4. Re-compute the cluster centers.
5. Check for convergence. If converge then stop

otherwise go to step 3.
DC
66
Cluster Center or Seed Point
Astrahan Method:
1. Compute density for each data unit.
2. Order the data units by density.
3. Choose the highest density point as the first
seed point.
4. Subsequent seed points are chosen in order of
decreasing density subject to the condition that
each new seed point is at least a minimum
distance from all previously chosen points.
DC
67
Ball and Hall Method:
1. Overall mean vector of data set is considered
as the first seed point.
2. Subsequent seed points are selected by

examining the data units in their input sequence
and accepting any data unit which is at least some
specified distance from all previously chosen
points.
DC
68
Our Method: (Problem of choosing 1 out of n0 data
units) (Published in IEEE Trans. SMC, Vol. 24, 1994)
1. Compute density for each datum.
2. Rearrange the mi in decreasing order. Let L be the ordered
list. Let i1.
3. Choose the datum X that tops the list L as the ith seed
point. If i = K go to step 6.
4. Count the number of data in the current set S. If the number
is less than n0 –1 then go to step 6. Else, from the current set
S find the n0 –1 nearest neighbors of the data X. Delete X and
theses n0 –1 neighbors from L and S to get the list of L and S
for next iteration.
5. Make i i +1 and go to step 3.
6. Stop DC
69
K-Means Algorithm
1. Select an initial partition of n objects into K
clusters.
2. Assign each pattern to its closest center.
3. Re-compute new cluster centers.
4. If convergence criterion is satisfies, then stop;
else, go to step 2.
 There are several variations of the basic

algorithm.
 Seed point detection and initial partition
generation is very important; different initial
partitions may lead to different final clusterings. DC
70
Crisp Data & Hard Boundaries
Let the k clusters be C  {C1 , C2 , Ck }
Let c j be the center of cluster C j
Euclidean distance between a pattern xi  X and C j

d
  ( xi , p c j , p ) 2
2
d 2 ( xi , c j )  xi  c j
p 1
Assign pattern xi to a cluster C j according to the rule :
xi  C j iff d 2 ( xi , c j )  d 2 ( xi , c p ), p  j , p  k , j  k .
DC
71
Crisp Data & Hard Boundaries
Update the cluster center for each cluster
x
xiC j
i, p
c j, p  ,1  p  d
Cj
Objective function to be minimized
k
J(X,C)    d 2 ( xi , c j )
j 1 xiC j
DC
72
The KMeans Algorithm( X , k ,  )
X : Given set of patterns
k : Number of predetermined clusters
 : Termination parameter
K-Means Algorithm
Initialize prototypes C  {c1 , c2 ,  , ck }
Jm  0
Repeat
c previous  c; J mprevious  J m
for each x j  X
for each c j  C
2
compute d  x j  c j
2
i, j
for each x j  X , update its cluster membership

update each prototype;
compute J m , the objetive function
until J m  J mprevious   DC
73
Clustering
K=6 K=5 DC
74
Seed-based Clustering Technique
Forgy’s K-Means Method:

1. Go to step 2 if beginning with a set of seed points;
go to step 3 if beginning with a partition of the data
units.
2. Form clusters with the nearest seed points.

3. Compute new seed points as the centroids of the
clusters of data units.
4. Alternate steps 2 and 3 until the process
converges.
DC
75
Seed-based Clustering Technique
MacQueen’s K-Means Method:

1. Take first K data units in the data set as clusters
as one member each.
2. Assign m-K data units to the nearest centroid

cluster. Re-compute the centriods.
3. Take the existing cluster centroids as fixed seed

points and make one more pass as step 2.
DC
76
Split-and-Merge Technique
MacQueen’s K-Means Method with Coarsening and
Refining Parameters:
1. Choose values for three parameters K, C, R.

2. Take the first K data units as initial clusters.
3. Find pair-wise distances of K data. If pair distance < C then
merge the pairs. Continue until all pair distance > C.
4. Assign remaining m – K data to nearest cluster center.
Again calculate the centroid of each cluster. Find pair-wise
distances of these new centroids. Merge the two cluster if the
distance < C and continue. If the distance > R then take as
new cluster.
5. Reallocate each data unit to its nearest seed point.
DC
77
ISODATA Clustering Technique:
What is ISODATA?
Iterative Self Organizing Data Analysis Technique
 ISODATA is a method of unsupervised classification
 Don’t need to know the number of clusters
 Algorithm splits and merges clusters
 User defines threshold values for parameters
 Computer runs algorithm through many iterations until

threshold is reached DC
78
How ISODATA works:
 Cluster centers are randomly placed and pixels are

assigned based on the shortest distance to center
method
 The standard deviation within each cluster, and the

distance between cluster centers is calculated
 Clusters are split if one or more standard deviation

is greater than the user-defined threshold
 Clusters are merged if the distance between them is

less than the user-defined threshold
DC
79
How ISODATA works:
 A second iteration is performed with the new cluster
centres
 Further iterations are performed until:
 The average inter-center distance falls below the

user-defined threshold
 The average change in the inter-center distance

between iterations is less than a threshold, or
 The maximum number of iterations is reached
DC
80
How ISODATA works:
Along the way…
 Clusters associated with fewer than the user-

specified minimum number of pixels are eliminated.
 Isolated pixels are either put back in the pool for

reclassification, or ignored as “unclassifiable”.
DC
81
Parameters
 Desired number of clusters

 Starting number of clusters
 Maximum number of clusters
 Minimum number if pixels per cluster
 Exclusion Distance
 Closeness criterion
 Elongation criterion
 Maximum number if iterations
 Maximum number of clusters that can be merged
at one time
 Absolute value of inter-cluster centre distance
 Relative decline in inter-cluster centre distance
DC
82
When choosing parameters:
 Experiment!
 Final configuration depends largely on starting

configuration
 Can store starting centre coordinates to minimize

changes
DC
83
Output
Original Landsat Clustered Image

Image (bands 3,4,5)
DC
84
Advantages/Disadvantages of
ISODATA
Advantages of ISODATA
 Don’t need to know much about the data

beforehand
 Little user effort required
 ISODATA is very effective at identifying spectral
clusters in data
Disadvantages of ISODATA
 May be time consuming if data is very

unstructured
 Algorithm can spiral out of control leaving only
one class
DC
85
Strip-based Clustering Technique: (published in Pattern
Recognition Letters, Vol. 13, 1992)
3
2
4
5 1
Center Pixel
6 8
7
DC
86
Strip-based Clustering Technique:
 Find the strips of finite width at different directions
around the centroid of the data.
 Find smallest density strip and if it’s percentage
satisfied the splitting restriction then split.
 Sparsely populated regions need not always be
present at centroid of the data. So apply modified K-
means (K = 2) method with restriction imposed.
 Merging should be occurring in the border region of
two clusters. Find the merging circle set and
calculate the density for each cluster within circle. If
it is satisfied the merging restriction then merge. DC
87
DC
88
Remotely sensed image Resulting two

corresponding to infrared clusters mapped in
hand image domain.
DC
89
Multi-seed Clustering Technique
(Published in IEEE Trans. on SMC, Vol. 27, 1997 and Int. Journal
of System Science, Vol. 26, 1995)
(a) Original data (b) By K-means

DC
90
(d) MST for merging

(c) Multi seed formation
and splitting the data
(e) Final result by multi-seed

DC
91
Multi-seed Clustering Technique:
 Find the maximum density point (one seed point).
• Find the border points and find max and min

distances from the seed point and border points. If the
difference between two distances is greater than T
then data is elongated.
• Collect all those points whose distances from the
seed point less than equal to minimum distance. From
rest of the data find maximum density point as second
seed and repeat this step. Let us find K number of
seed points.
DC
92
 Apply K-means algorithm.
 For merging draw MST of those seed points. The

seed points, which are connected by MST, those
two clusters are looking for merging if they satisfy
the merging criteria.
DC
93
(a) Original (b) Seed formation

pattern and splitted
regions DC
94
(c) Final clustered

image by merging
DC
95
Fuzzy Clustering Technique
Well separated Overlapping

clusters clusters
DC
96
Fuzzy Clustering Technique
• Crisp (ordinary) Cluster
Membership grade for pattern xi in a particular cluster is 1 if

the pattern belongs to the cluster and 0 if it does not.
• Fuzzy Cluster
Pattern xi has a grade of membership,  q ( xi )  0, or degree of

belonging to the qth cluster, where   q ( xi )  1. The larger
q
 q ( xi ), the more confidence exists that xi belongs to cluster q.
DC
97
Fuzzy Clustering Algorithm
 Fuzzy C-means clustering – Each data point belongs to each
cluster to a degree that is specified by a fuzzy membership
grade.
 Choice of the initial seed points.
 Choose the membership values i.e. assigns every data point a

membership value for each cluster.
 By iteratively updating the cluster centres and membership value of

each data point.
 The algorithm iteratively moves the cluster centres to the optimal

location within the data set.
 The iteration is based on the minimization an objective function that

represents the distance from any given data point to a cluster centre
weighted by the data point membership grade. DC
98
Fuzzy C-Means (FCM)
 Here m – is the fuzzy partition matrix exponent for controlling

the degree of fuzzy overlap, with m > 1.

 is the degree of membership of the jth data point in the ith
cluster. For a given data point, the sum of the membership
DC
values for all clusters in one. 99
FCM Algorithm
Step 1: Set the initial cluster centers. By default, the initial

cluster centers are random.
Step 2: Randomly initialize the cluster membership values μij.
Step 3: Calculate the cluster centers.
Step 4: Compute the distance matrix for each data point by
, .
DC
100
FCM Algorithm Continue…
Step 5: Update the membership values for each data point
Step 6: Calculate the objective function
Step 7: repeat steps 3 – 6 unit one of the following conditions

is satisfy.
 improves by less than a specified minimum
threshold.
 The algorithm performs the specified maximum number

of iterations.
DC
101
Remote
Sensing
DC
1
Orbit
An orbit definition is a path one object in space uses to move
around another object in space. The path is predictable
because it repeats regularly. Objects in space include
satellites, planets, moons, space stations, and more.
Orbits can have varying

speeds due to elliptical
orbits or constant speeds in
the case of circular orbits.
DC
2
Orbit
Two types orbit: 1. Synchronous orbit and 2. Geostationary orbit
 Synchronous orbit is a type of orbit in which the period of
time it takes a satellite to orbit a body is equal to the body's
rotational speed and is orbiting in the same direction as its
rotation.
 Geostationary orbit is a common
kind of circular synchronous orbit.
Geostationary orbit involves a satellite
orbiting above the Earth’s
equator while its orbital period is equal
to Earth's rotational period or day.
Satellites following a geostationary
orbit will usually orbit above the
Earth's equator at ~36,000 kilometers
and follow a circular orbit, maintaining
this distance. DC
3
Sun-Synchronous Satellite
 Sun-synchronous
satellite is an orbit a
satellite takes around
the Earth where the
movement and orbit
of the satellite always
appear the same
from the viewpoint of
the Sun. The satellite
essentially follows an
orbital path around
the boundary
between the
nighttime and
daytime sides of
Earth. DC
4
Geo-stationary Satellite
 Satellites in geostationary orbit
(GEO) circle Earth above the
equator from west to east following
Earth’s rotation – taking 23 hours 56
minutes and 4 seconds – by
travelling at exactly the same rate
as Earth. This makes satellites in
GEO appear to be ‘stationary’ over
a fixed position. In order to perfectly
match Earth’s rotation, the speed of
GEO satellites should be about 3
km per second at an altitude of 35
786 km. This is much farther from
Earth’s surface compared to many
satellites.
DC
5
Remote Sensing
Remote sensing is the process of acquiring data/information
about objects/substances not in direct contact with the sensor,
by gathering its inputs using electromagnetic radiation or
acoustical waves that emanate from the targets of interest. An
aerial photograph is a common example of a remotely sensed
product.
Types of remote sensing: Active
Passive
DC
6
Applications of Remote Sensing
• Meteorology - Study of atmospheric temperature, pressure, water
vapor, and wind velocity.
• Oceanography - Measuring sea surface temperature, mapping
ocean currents, and wave energy spectra and depth sounding of
coastal and ocean depths.
• Glaciology- Measuring ice cap volumes, ice stream velocity, and sea
ice
distribution.
• Geology- Identification of rock type, mapping faults and structure.
• Geodesy- Measuring the figure of the Earth and its gravity field.
• Topography and cartography - Improving digital elevation models.
• Agriculture - Monitoring the biomass of land vegetation.
• Forest- monitoring the health of crops, mapping soil moisture.
• Botany- forecasting crop yields.
• Hydrology- Assessing water resources from snow, rainfall and
underground aquifers.
DC
7
Applications of Remote Sensing
• Disaster warning and assessment - Monitoring of floods and
landslides, monitoring volcanic activity, assessing damage zones from
natural disasters.
• Planning applications - Mapping ecological zones, monitoring
deforestation, monitoring urban land use.
• Oil and mineral exploration- Locating natural oil seeps and slicks,
mapping geological structures, monitoring oil field subsidence.
• Military- developing precise maps for planning, monitoring military
infrastructure, monitoring ship and troop movements.
• Urban- determining the status of a growing crop.
• Climate- the effects of climate change on glaciers and Arctic and
Antarctic
Regions.
• Sea- Monitoring the extent of flooding.
• Rock- Recognizing rock types.
• Space program- is the backbone of the space program.
• Seismology: as a premonition.
DC
8
Principles and process of
remote sensing
1. Energy Source or Illumination
(A)
2. Radiation and the
Atmosphere (B)
3. Interaction with the Target (C)
4. Recording of Energy
(Electromagnetic
Radiation) by the Sensor (D)
6. Transmission, Reception,
and Processing (E)
7. Interpretation and Analysis
(F)
8. Application (G)
DC
9
DC
10
Spectral Ranges Commonly
Used in Remote Sensing
• Gamma Rays <0.30 nm: This range is completely absorbed by the upper
atmosphere and not available for remote sensing.
• X-Rays 0.03-30.0 nm: This range is completely absorbed by the
atmosphere and not employed in remote sensing.
• Ultraviolet: 0.03-0.40 μm: Hard UV 0.03-0.3 μm: This range is completely
absorbed by the atmosphere and not employed in remote sensing.
Photographic UV 0.30-0.40 μm: This range is not absorbed by the
atmosphere and detectable with film and photo detectors but with severe
atmospheric scattering.
• Visible Light: This narrow band of electromagnetic radiation extends from
about 400 nm (violet) to about 700 nm (red). It’s Available for remote
sensing the Earth, can be imaged with photographic film. Violet: 400-430
nm, Indigo: 430-450 nm, Blue: 450-500 nm: water increasingly absorbs
EM radiation at longer wavelengths, band 1 provides the best data for
mapping depth detail of water covered areas. It is also used for soil,
vegetation discrimination, forest mapping.
DC
11
Spectral Ranges Commonly
Used in Remote Sensing
• Green: 500-570 nm: The spectrum corresponds to the chlorophyll
absorption of healthy vegetation and is useful for mapping detail such as
depth or sediment in water bodies. Cultural features such as roads and
buildings also show up well in this band.
• Yellow: 570-590 nm, Orange: 590 - 610 nm
• Red: 610-700 nm: Chlorophyll absorbs these wavelengths in healthy
vegetation. Hence, this band is useful for distinguishing plant species, as well
as soil and geologic boundaries.
• Infrared: 0.7-300 μm: This region is sensitive to plant water content, which
is a useful measure in studies of vegetation health. This band is also used for
distinguishing clouds, snow, and ice. NIR: 0.7-1.5 μm, SWIR: 1.5-3 μm,
MWIR: 3-8 μm, LWIR: 8-15 μm, FIR:>than 15 μm.
• Microwaves (Radar) 1 mm-1 m: Microwaves can penetrate clouds, fog,
and rain. Radar is the active form of microwave remote sensing. Radar
images are acquired at various wavelength bands.
• Radio and TV Waves: 10 cm-10 km: The longest-wavelength portion of the
electromagnetic spectrum. DC
12
Spectral Ranges Commonly Used in
Remote Sensing
• Major ranges utilized for earth resources sensing are between
0.4 and 12 m (visible/infrared range).
• Between 30 to 300 mm for microwave range (SAR).
• Microwave wavelengths are commonly used as frequency

rather than wavelength. 30 to 300 mm corresponds to
frequencies between 1 GHz to 10 GHz.
DC
13
Characteristics
• Acquired from aircraft or spacecraft platforms
• Available in digital format
• Scanning Micro-densitometer is used for digital conversion
• Major characteristic of the image is the wavelength band it
represents.
• Passive sensor: some images depends on the solar
radiation in ultraviolet, visible and near-to-middle infrared
range wavelengths.
• Others are measurements of the spatial distribution of
energy emitted by earth surface, so called Thermal infrared
wavelength range.
• Active sensor: Microwave band of wavelengths.
DC
14
Characteristics
Resolution in satellite images:
• Spatial Resolution: Described by pixel size. It is defined in
terms of the Instantaneous Field Of View (IFOV) of the sensing
element.
• Spectral Resolution: Number and width of spectral bands of
electromagnetic energy detectable by the sensor.
• Temporal resolution: Measure of the repeat cycle or frequency
with which a senor revisits the same part of the earth’s surface.
• Radiometric Resolution: It is ability of the sensor to sense
small changes in reflected energy from the target. More the
number of quantization levels by which the reflected energy is
represented, higher the radiometric resolution.
DC
15
Characteristics
• Spatial and spectral resolutions determine the data volume
provided by sensor
• Example: Landsat Thematic Mapper has 7 wavelength bands
with 8-bit radiometric resolution. 6 of which have 30m spatial
resolution and one of which has 120m spatial resolution
(thermal band, long wavelength, long aperture is required to
collect sufficient signal energy to maintain the radiometric
resolution).
• 185 Km  185 Km image represents 2.37 million pixels in
thermal band and 38 million pixels for other 6 bands.
• At 8 bits per pixel a complete 7 band image is composed of
1.848 GB.
DC
16
Characteristics of Landsat TM Satellite
•LANDSAT-1 launched 1972 and lasted
until 1978.
•LANDSAT-2 launched 1975
•Three more satellites were launched in
1978, 1982, and 1984 (LANDSAT-3, 4,
and 5 respectively).
•LANDSAT-6 was launched on October
1993 but the satellite failed to obtain
orbit.
•LANDSAT-7 launched in 1999
•Only 7 and 5 are still working
DC
17
Characteristics of Landsat TM Satellite
No. of Wavelength Name of Spatial Spectral

Band (Spectral the resolutio Resolutio
s bands) (m) bands n (m) n (bits)
1 0.45-0.52 Blue 3030 8
2 0.52-0.60 Green 3030 8
3 0.63-0.69 Red 3030 8
4 0.76-0.90 Near IR 3030 8
5 1.55-1.75 Mid IR 3030 8
6 2.08-2.35 Mid IR 3030 8
7 10.4-12.5 Thermal 120120 8
DC
18
LANDSAT TM: applications
Band Nominal Spectral applications
location
1 Blue Water body penetration, soil-water
discrimination, forest type mapping, cultural
feature ID
2 Green Green reflectance peak of veg, for veg ID and
assessment of vigor, cultural feature ID
3 Red Chlorophyll absorption region, plant species
differentiation, cultural feature ID
4 Near infra red Veg types, vigor and biomass content,
delineating water bodies, soil moisture
assessment
5 mid infra red (1.55- Veg moisture, soil moisture, diff of soil from
1.75 mm) clouds
6 Thermal infra red Veg stress analysis, soil moisture, thermal
mapping
7 mid infra red(2.08- Discriminating mineral and rock types, veg
2.35 mm) moisture DC
19
LANDSAT TM
•An example: August 14, 1999 (left) and October 17, 1999
(right) images of the Salt Lake City area
• differences in color due to growing season
DC
20
SPOT Satellite
•Launched by France
• Stands for Satellite Pour
l'Observation de la Terre
•Operated by the French
Space Agency, Centre
National d'Etudes Spatiales
(CNES).
DC
21
Characteristics of SPOT Satellite
Altitude 832 Km
SPOT
Orbit Sun synchronous, 10:30 AM

nominal equator crossing
Repeat Cycle 26 days

Multi-spectral Mode Panchromatic Mode
Bands 0.50-0.59 m (Green)
0.61-0.68 m (Red)
0.79-0.89 m (NIR) Bands 0.57-0.73 m
Spatial 20 m  20 m Spatial 10 m  10 m
Resolution Resolution
Spectral 8 bit Spectral 8 bit
Resolution Resolution DC
22
SPOT Satellite
 Mosaic false color tiles of Australia
DC
23
IKONOS data
 High resolution satellite developed
by Space Imaging, launched 1999
 Has sun-synchronous orbit and
crosses equator at 10:30 AM
 Ground track repeats every 11
days
 Extremely flexible: it can quickly
stabilize itself after pointing at a
new target, allowing it to follow
wavy features.
 The entire spacecraft moves, not
just the sensor
 11 km by 11 km image size, but
user specified strips and mosaics
can be ordered DC
24
IKONOS data
 IKONOS collects panchromatic band (.45 to
.90 mm) at 1 m resolution
 Collects four multispectral bands at 4 m
resolution
 Bands include blue (.45 to .52 mm) , green
(.51 to .60 mm) , red (.63 to .70 mm), near IR
(.76 to .85 mm)
 Radiometric resolution is 11 bits, or 2048
values
DC
25
IKONOS data
 Here is 1m IKONOS view of suburbs, near winter Olympics
DC
26
IKONOS data
 1m IKONOS view of Dubai
DC
27
IKONOS data
 1m IKONOS pan image of Rome
DC
28
IRS-1C/1D Satellite
 The first satellite, IRS 1C was launched on 28 December
1995 on a Russian Molniya – M (Blok - 2BL) rocket. It
operated until it was decommissioned on 21 September
2007 after serving for 11 years and 8 months.
 The second satellite, IRS 1D followed on 29 September

1997 on an indigenous PSLV – G (2) rocket, although
the orbit was lower than intended. Its mission was
completed during January 2010 after serving for 12 years
and 3 months.
DC
29
Characteristics of IRS-1C Satellite
Altitude 860 Km
IRS-1C
Orbit Sun synchronous, 10:30 AM

nominal equator crossing
Revisit Cycle 16 days

Multi-spectral Mode Panchromatic Mode
Bands 0.52-0.59 m (Green)
0.62-0.68 m (Red) Bands 0.50-0.75 m
0.77-0.86 m (NIR)
Spatial 23.5 m  23.5 m Spatial 5.8 m  5.8 m
Resolution Resolution
Spectral 8 bit Spectral 8 bit
Resolution Resolution DC
30
IRS-1C Satellite Images
Multi-spectral Panchromatic
Image Image
DC
31
CARTOSAT Series Satellite
 CARTOSAT–1 is the first Indian Remote Sensing Satellite

capable of providing in-orbit stereo images. The images
were used for Cartographic applications meeting the
global requirements. Cameras of this satellite have a
resolution of 2.5m (can distinguish a small car).
 The CARTOSAT–1 provided stereo pairs required for

generating Digital Elevation Models, Ortho Image
products, and Value added products for various
applications of Geographical Information System (GIS).
DC
32
CARTOSAT – 1 Satellite
DC
33
CARTOSAT – 1 Satellite
Launch date 5 May 2005
Launch vehicle PSLV- C6
Orbit 618 km Polar Sun
Synchronous
Payloads PAN FORE, PAN - AFT
Spatial Resolution PAN-F: 2.5 m x 2.78 m
PAN-A: 2.22 m x 2.23 m
Orbit Period 97 min
Number of Orbits Per day 14
Local time of equator 10:30 am
crossing
Repetivity 126 days
Revisit 5 days DC
34
CARTOSAT - 1 Satellite
DC
35
CARTOSAT - 3 Satellite
DC
36
Microwave Remote Sensing Image Data
Specular Reflector: appear Diffuse Reflector: scatter the

more Dark in the image incident energy in all directions
including towards remote
sensing platform, appear light in
image
Corner Reflector: Resulting from the right angle formed

between a vertical structure such as building and a
horizontal plane as the surface of the earth DC
37
Characteristics of Seasat SAR
Altitude 800 Km
Wavelength 0.233 m
Incidence angle 20o
Swath width 100 Km
Range resolution 25 m
Azimuth resolution 25 m
Spectral resolution 5 bit
DC
38
Seasat SAR Image
DC
39
Band Combinations
3,2,1
4,3,2
5,4,3
DC
40
Feature space image
• A graphical
representation of
the pixels by
plotting 2 bands
Band 4
vs. each other
• For a 6-band
Landsat image,
there are 15 Band 3
feature space
images DC
41
Remote Sensing
 What is remote sensing image processing?
Remote Sensing image processing refers to the techniques

and methods used to analyze and manipulate images
obtained from remote sensing satellites/aircraft or sensors. It
involves the extraction of valuable information from these
images to understand and monitor various phenomena on
Earth’s surface.
 What are the main steps involved in remote sensing

image processing?
 Pre-processing
 Enhancement
 Classification
 Interpretation DC
42
Remote Sensing
 What are some commonly used techniques for remote
sensing image enhancement?
 Histogram equalization
 Contrast stretching
 Spatial filtering
 Image fusion
 How is remote sensing classification performed?
 Supervised
 Unsupervised
DC
43
Remote Sensing
 What are the applications of remote sensing image
processing?
 Agriculture for crop monitoring and yield estimation.

 Environmental monitoring for tracking changes in land
cover and detecting natural disasters
 Urban planning for mapping and analyzing urban growth
 Forestry for assessing forest health and estimating biomass
 Geology – Identification mineral deposits and geological
structures
 Crucial role in mapping and monitoring climate change
impacts and supporting disaster management efforts
 Reference scene generation for missile guidance system
 Target detection for Defence applications
 Image Intelligent Environment for Battelfield Application
DC
44
Characteristics
• Acquired from aircraft or spacecraft platforms
• Available in digital format
• Scanning Micro-densitometer is used for digital conversion
• Major characteristic of the image is the wavelength band it
represents
• Passive sensor: some images depends on the solar
radiation in ultraviolet, visible and near-to-middle infrared
range wavelengths
• Others are measurements of the spatial distribution of
energy emitted by earth surface, so called Thermal infrared
wavelength range
• Active Sensor: microwave band of wavelengths DC
45
Characteristics
• Different kinds of resolution
• Spatial Resolution: Described by pixel size
• Spectral Resolution: Number of binary digits, or bits,

represent the range of available brightness values
• Thermal Resolution: Isolation of two bodies due to

temperature difference.
• Time Resolution: Capture a particular image in a

particular time
DC
46
Spatial Resolution
 Spatial resolution is the detail in pixels of an image. High
spatial resolution means more detail and a smaller grid cell
size. Whereas, lower spatial resolution means less detail
and larger pixel size.
 Overall, spatial resolution describes the quality of an

image and how detailed objects are in an image. If the
grid cells are smaller, this means the spatial resolution has
more detail with more pixels. DC
47
Spectral Resolution
 Spectral resolution describes the amount of spectral detail
in a band. High spectral resolution means its bands are
more narrow. Whereas low spectral resolution has broader
bands covering more of the spectrum.
 For example, a multispectral image breaks light into 4 to

36 bands. Then, it assigns those bands names such as red,
green, blue, and near-infrared. Each band may span 0.05
um in the electromagnetic spectrum.
 Similarly, hyperspectral imaging captures a spectrum of
light. But it divides the light into hundreds of narrow spectral
bands. For hyperspectral images, spectral resolution is very
DC
high. 48
Characteristics
• Spatial and spectral resolutions determine the data volume
provided by sensor
• Example: Landsat Thematic Mapper has 7 wavelength bands

with 8-bit radiometric resolution. 6 of which have 30m spatial
resolution and one of which has 120m spatial resolution
(thermal band, long wavelength, long aperture is required to
collect sufficient signal energy to maintain the radiometric
resolution).
• 185 Km  185 Km image represents 2.37 million pixels in

thermal band and 38 million pixels for other 6 bands.
• At 8 bits per pixel a complete 7 band image is composed of
1.848 Gbit
DC
49
Spectral Ranges Commonly Used in
Remote Sensing
• Major ranges utilized for earth resources sensing are
between 0.4 and 12 m (visible/infrared range)
• Between 30 to 300 mm for microwave range (SAR)
• Microwave wavelengths are commonly used as

frequency rather than wavelength. 30 to 300 mm
corresponds to frequencies between 1 GHz to 10 GHz
DC
50
Spectral Reflectance of Earth
60 Surface Material
3
%
50
Reflectance
40
30
20
2
10
1
0
0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6
Wavelength m
Green
Blue
Red
Near infrared Middle infrared

DC
1: Water, 2: Vegetation and 3: Soil 51
Basic Image Processing for Remote
sensing Images
 Image Correction: Geometric correction by
interpolation technique
 Image Registration: by affine transformation
 Radiometric Enhancement: Histogram

modification, Histogram equalization, Contrast
stretching etc
 Geometric Enhancement: Different kinds of
filtering for removal of noise and enhancement
DC
52
Classification of Remote Sensing Data
Classification
Supervised Unsupervised
(Clustering)
Distribution Free Statistical Techniques No extensive prior knowledge

required Unknown, but distinct,
Based on spectral classes are generated
Euclidean classifier
probability Limited control over classes
K-nearest neighbour
distribution models, and identities, No detailed
Minimum distance
which may be information
Decision Tree
parametric
or nonparametric
(Bay’s Classifier)
DC
53
Each color
represents a
different
“cluster”
pixels that
may
correspond to
the land
cover classes
you are
interested in
DC
54
DC
55
Image Classification
• Why classify?
• Make sense of a landscape
– Place landscape into categories
(classes)
• Forest, Agriculture, Water, etc
• Classification scheme = structure
of classes
– Depends on needs of users
DC
56
What is a Classified Image
• Image has been processed to put each pixel into

a category
• Result is a vegetation map, land use map, or
other map grouping related features
• Categories are defined by the intended use of
the map
• Can be few or many categories, depending on
the purpose of the map and available resources
DC
57
Supervised Classification Technique: essential steps

 Decide set of ground cover types into which the images
are to be segmented
 Extract training sample
 Extract parameters from the training set
 Select appropriate classification algorithm
 Classify the imagery into m class
 Statistically evaluate the classification accuracy
DC
58
Training Areas
DC
59
Parallelepiped Classification Algorithm:
Number of spectral bands is n. mck is mean value
of the training data obtained for class c in band k
out of M possible classes. Sd ck is the standard
deviation of the training data class c of band k .
Let Gijk be the gray value of the (i, j ) th pixel of
band k . Now Gijk in class c iff
mck  Sd ck  Gijk  mck  Sd ck
where c  1,2,..., M and k  1,2,..., n DC
60
Minimum distance to Mean Classification Algorithm
Number of spectral bands is n. mck is mean value
of the training data obtained for class c in band k
out of M possible classes. Suppose X  ( x11 ,..., x1n )
is the unknown pixel to be classified. The Euclidean
distance between X and M c is

d ( X, M c )   x11  mc1    x12  mc 2   ...   x1n  mcn 
2 2

2 1/ 2
Classification is performed on the basis of

x  wi if d ( x, mi )  d ( x, m j ) j  i
where wi , i  1,2,..., M be the spectral classes for an image DC
61
Bayes’ Theorem
P(A|B) = probability of A given that B is true.
P(B|A)P(A)
P(A|B) =
P(B)
In practice we are most interested in dealing with events e
and data D.
e = “I have a cold”
D = “runny nose,” “watery eyes,” “coughing”
P(D|e)P(e)
P(e|D)=
P(D)
So Bayes’ theorem is “diagnostic”.
DC
62
Bayes’ Theorem
D = Data, e = some event
P(D|e)P(e)
P(e|D) =
P(D)
 P(e) is called the prior probability of e. Its what we know
(or think we know) about e with no other evidence.
 P(D|e) is the conditional probability of D given that e
happened, or just the likelihood of D. This can often be
measured or computed precisely – it follows from your
model assumptions.
 P(e|D) is the posterior probability of e given D. It’s the
answer we want, or the way we choose a best answer.
 You can see that the posterior is heavily colored by the
prior, so Bayes’ has a GIGO liability. e.g. its not used to
test hypotheses DC
63
Bayesian Classifier
• Use the knowledge of probability distributions

• Classifies objects into the class to which it is
most likely to belong to based on observed
features
• Has been used extensively and extended to
many applications
DC
64
Bayesian Classifier
Simple Example
• Classifying Cars into two classes of a scene
– Red
– Black
• Inspection using IR camera
• Use only one feature (intensity)
• Training :
– Use many samples from the two classes
– Measure the feature values
DC
65
Bayesian Classifier
Example:
• Assumptions in the decision rule

– Both classes are equally likely
– Penalties are equal (false alarm vs. false
dismissal)
• Optimal decision?
• Move the threshold to left or right
– The total error rate increases
DC
66
Bayesian Classifier
Example:
• False dismissal rate = ½ the overlapped

area to the right of t
• False alarm rate = ½ the overlapped area
to the left of t
• Why half?
– Both classes are equally likely
– Area under each curve should be 0.5 (not 1)
DC
67
Bayesian Classifier
Example
 : Class of red car
 2 : Class of black
x : Observed feature from scene (scalar)
p(x |  ) :Probability of the feature value of a car  x
given that the car is red
p(x |  2 ) :Probability of the feature value of a car  x

given that the car is black
DC
68
Bayes Rule (Basic Concepts)
Class conditional distribution : p ( x |  i ) for each class  i
a priori probability : P( i ) for each class  i
unconditional distribution : p( x)
Bayes Rule :
P(  i | x) p( x)  p ( x |  i ) P( i )
p ( x |  i ) P( i )
P(  i | x) 
p( x)
p( x |  i ) P( i )
 m
 p( x | ) P( )
i 1
i i DC
69
Maximum Likelihood Classification Algorithm
Baye’s Classification:
wi , i  1,..., M classes. The probabilty P ( wi / X )

gives the likelihood that the correct class is wi
for a pixel at position X . Classification is
performed according to
x  wi if P ( wi / x)  P( w j / x) j  i
DC
70
Maximum Likelihood Classification Algorithm
P( X / wi ) is the posterior probability i.e. for X in
multi - spectral space a set of probabilities can be
computed that give relative likelihood that pixel belongs
to each available class. Baye's theorem
P( wi / x)  P ( x / wi ) P( wi ) / P( X)
P( wi ) is a priori probability and P( X) is probability density
x  wi if P( x / wi ) P( wi )  P ( x / w j ) P( w j ) j  i
g i ( x)  ln[ P( x / wi ) P( wi )]  ln P( x / wi )  ln P( wi )
x  wi if g i ( x)  g j ( x) j  i DC
71
Multivariate Normal Class Models:
 1
1 / 2 
P( x / wi )  (2 ) N / 2
i exp ( x  mi )  i ( x  mi )
1
 2 
where mi and  i are the mean vector and covariance
matrix of the data in class wi .
1 1
 1
g i ( x)  ln P( wi )  ln  i  ( x  mi )  i ( x  mi )
2 2
Equal probable case
 1
g i ( x)   ln  i  ( x  mi )  i ( x  mi )
DC
72
Classification of Remote Sensing Data (Multi-seed
Technique) (Published in Proc. CODIS 2004 and IEEE Trans. GRS, Sep
2008)
Processing Steps:
 Training Sample Selection
 Test for homogeneity
 If heterogeneous then form heterogeneous sample as
the union of homogeneous regions
 Statistical parameters extraction from each
homogeneous region
 Minimum Distance Logical Based Classifier
 Majority must be granted logic
 Minimum of minimum distance granted logic
DC
73
Plot of the vegetation, concrete and water classes training data DC

74
Single Seed Classification Result

DC
75
Multi Seed Classification Result

DC
76
Original Multi-spectral Image of IRS-1C Satellite

DC
77
Single Seed Classification Result

DC
78
Parallelepiped Classification Result

DC
79
Multi-Seed Classification Result

DC
80
Accuracy Assessment
Precision and Recall: Problem Statement:
One fine morning, Jack got a phone call. It was

a stranger on the line. Jack, still sipping his
freshly made morning coffee, was hardly in a
position to understand what was coming for
him. The stranger said, “Congratulations Jack!
You have won a lottery of $10 Million! I just
need you to provide me your bank account
details, and the money will be deposited in
your bank account right way…”
DC
81
Precision and Recall
Type I (False Positive) and Type II Errors (False Negative)
 Let me try to explain the complexity here. Assuming Jack
is a normal guy.
 He would think of this as a joke, or maybe, a scam to get

his bank details, and hence will deny to provide any
information.
 However, this decision is based on his assumption that

the call was a joke.
 If he is right, he will save the money in his bank account.
 But, if he is wrong, this decision would cost him a million

dollars! DC
82
Let’s talk in statistical terms for a bit:
 According to me, the null hypothesis in this case is that this

call is a fraud.
 As a matter of fact, if Jack would have believed the stranger

and provided his bank details, and the call was in fact a
fraud, he would have committed a type I error, also known
as a false positive.
 On the other hand, had he ignored the stranger’s request,

but later found out that he actually had won the lottery and
the call was not a joke, he would have committed a Type II
error, or a false negative.
DC
83
DC
84
DC
85
 Precision means the percentage of

your results which are relevant.
 Recall refers to the percentage of total

relevant results correctly classified by
your algorithm.
DC
86
Accuracy Assessment
DC
87
Accuracy Assessment
DC
88
Accuracy Assessment
DC
89
Accuracy Assessment
DC
90
Accuracy Assessment
DC
91

IP Full

Uploaded by

Copyright:

Available Formats

You might also like

IP Full

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IP Full

Uploaded by

Copyright:

Available Formats

Image Processing & Pattern recognition

Prof. Debasis Chaudhuri

• For most of this course we will focus on grey-

• Digital image processing focuses on three

• Some argument about where image processing ends

(a) Original Image (b) After sharpening

(a) Original Image (b) After removing noise

(a) Original Image (b) After removing the blur

•Early 1920s: One of the first applications of

– Images were transferred by submarine cable

•Mid to late 1920s: Improvements to the

•1960s: Improvements in computing

•1970s: Digital image processing begins to

•1980s - Today: The use of digital image

Original MRI Image of a Dog Edge Detection Image

•Human operators are

Computer Vision : Recovers useful

Vision = Geometry + Measurement +

Image Processing : process an image to enhance

(a) Medical Image may be (b) Satellite Image may

Face Recognition Industrial Surface Inspection Surveillance

Measure Tool Bar code reading OCR Pattern Matching

Intermediate level: image to sub-symbolic

High level: sub-symbolic to symbolic

Transpose Matrix: A  [aij ]mn , AT  [a ji ]n m

• Geometry of image formation

Sampling and Quantization

• Need less space

• Foreground : pixels that belong to objects

NW N NE i-1, j-1 i-1, j i-1, j+1

A pixel p at coordinate (i, j ) has horizontal and vertical neighbours as

NW NE i-1, j-1 i-1, j+1

Four diagonal neighbours as

NW N NE i-1, j-1 i-1, j i-1, j+1

All together are called 8 - neighbors of p

• Pixels should be neighboring pixels and satisfy a specified criterion of

Let V  {1} (Binary image) or V  {15,16,17,..., 25} (Gray value image)

(a) (b) (c)

Adjacent : A pixel p is adjacent to a pixel q if they are connected.

Path : A path from pixel p with coordinates ( x, y ) to pixel q with

A binary relation R over set A is said to be

A NOT (A) A B (A) AND (B)

A B (A) OR (B) A B (A) XOR (B)

• Overlay the origin of the mask at a pixel

• What if the mask extends to outside the

Here we are considering the mirror image: row and

(c) Result to applying the mask to the

• Number of foreground objects

• External Corners • Internal Corners

• External corners=4 • External corners=8

• Number of objects =(External-Internal)/4

• Take the 4 external masks and 4 internal

• Pixel (r,c) is connected to (r ' ,c' ) if

• A connected component is a set of pixels

• A connected component labeling of a

• Take a template and go over the image

1 2 • Label collision pixels are stored in a

 The procedure simultaneously looks for all other pixels in its 3 × 3