T3: Segmentation and Feature Extraction: Computer Vision

Master Course
Computer Vision

T3: Segmentation and Feature Extraction
Computer Vision Stages
Scene
Image
acquisition
Digital
Image
Processing
Segmentation
Descriptors
and
Feature
Extraction
Recognition
and
Interpretation
Detection,
Classification,
Recognition
Object
Models
Introduction: Examples of Feature Extraction
Original Image Segmentation Feature Extraction
Index
- Contour detection

- Region based segmentation

- Connectivity analysis and labeling

- Basic edge and region feature extraction

Index
- Contour detection

- Region based segmentation

- Connectivity analysis and labeling

- Basic edge and region feature extraction

Contour Detection: High Pass Filter
High pass filter and Laplacian:
!
2
f !
"
2
f
" x
2
#
"
2
f
" y
2
In a 8 neighborhood the Laplacian is expressed as follows,
!
2
f ! $ f % x#1, y&# f % x' 1, y &# f % x , y# 1&# f % x , y' 1& (' 4f % x , y&
We can also do this operation by using one of the following masks,
Contour Detection: Gradient Technique
Gradient technique:
y y
f(x,y) f(x,y+1)
f(x+1,y)
x
f(x,y) f(x,y+1)
f(x+1,y)
x
f(x+1,y+1)
The gradient of an image is defined as:
! F!
"
G
x
G
y
#
!
"
$ f
$ x
$ f
$ y
#
!
%
f % x&1, y'( f % x , y '
f % x , y&1'( f % x , y'
'

where the magnitude is: ! f ! mag% ! F '!
)
G
x
2
&G
y
2
* G
x
& G
y
and the orientation % ! ' is: !% x , y'! tag

( 1
%
G
y
G
x
'
Biased technique Non biased technique (Roberts)
Gradient technique using convolution:
Robertss mask
Prewitts mask
Sobels mask
Vertical border
detectors
Horizontal border
detectors
g ! x , y "#
$
s# % a
a
$
t # % b
b
w! s , t "! f ! x& s , y&t "
Examples of contour detection:
Contour Detection: Canny detector
Canny detector: Canny's aim was to discover the optimal edge detection algorithm.
In this situation, an "optimal" edge detector means:

(1) good detection the algorithm should mark as many real edges in the image as
possible

(2) good localization edges marked should be as close as possible to the edge in
the real image; and

(3) minimal response a given edge in the image should only be marked once, and
where possible, image noise should not create false edges.

Stages of the Canny's algorithm:
1- Noise reduction : The image is convolved with a Gausssian filter ! in the example is used a
5x5 Gaussian with !" 1. 4#
2- Gradient computation: Canny algorithm uses four filters to detect horizontal, vertical
and diagonal edges in the blurred image. A contour detector is used and their gradient
! magnitud and orientation # are computed,
! f " mag! ! F #"
$
G
x
2
%G
y
2
; "! x , y#" tag
& 1
!
G
y
G
x
#
The computed orientation is is rounded to one of four angles representing vertical, horizontal
and the two diagonals ! 0, 45, 90 and 135 degrees for example # .
3- Non maxima supression: a search is then carried out to determine if the gradient magnitude
assumes a local maximum in the gradient direction. The point will be considered to be on the
edge of ! 0, 45, 90 or 135 # , if its gradient magnitude is greater than the magnitudes on one
of these directions . The result is a binary image of the set of assigned edges .
4- Tracing edges through the image and hysteresis thresholding: They are used two image
gradients low and high to mark the pixels that belongs or not to an edge .
The image after a 5x5 Gaussian mask
has been passed across each pixel.
A binary edge map, derived from the
Sobel operator, with a threshold of 80.
The edges are colored to indicate the
edge direction: yellow for 90 degrees,
green for 45 degrees, blue for 0
degrees and red for 135 degrees.
The same binary map shown on the
left after non-maxima suppression.
The edges are still colored to indicate
direction..
Original image Canny detector on the original image
Region Based Segmentation: Thresholding
Thresholding: The aim is to divide the image in two regions, the background is
labeled with 0s and foreground is labeled with 1s. The resulting image is
denominated binary image.

Note: There are many variations of thresholding; optimum global thresholding
(Otsus method), multiple thresholding, variable thresholding, etc.
Gray level
frequency
Gray level
T (threshold)
g ! x , y "#
$
1 si f ! x , y"%T
0 si f ! x,y"& T
Region Based Segmentation: Thresholding
Thresholding:
Original image
Histogram
Binarized image
Region Based Segmentation: Region Growing
Region growing: The technique segments the regions using a seed and an
image property (for example if the difference between the gray levels is greater
than a threshold)
g ! x , y "#
$
a
k
si f ! x% r , y% s"& h
a
k
! x , y" 'T
Nothing is changed
where r , s# 0,1
0 0 5 6 7
1 1 5 8 7
0 1 6 7 7
2 0 7 6 6
0 1 5 6 5
a a b b b
a a b b b
a a b b b
a a b b b
a a b b b
a a a a a
a a a a a
a a a a a
a a a a a
a a a a a
Original image T ! 3 T! 5
The seeds are in the pixels " 2,1# y " 2,3#
Region Based Segmentation: Region Growing
Example:
Region Based Segmentation: Splitting and Merge
Splitting and merge: Consist on dividing the image in 4 regions, and if any of the
regions does not maintain an uniform property (for example the same gray level),
then the region is split in 4 additional regions. This process stops when all the
regions individually accomplish with the same property. Once the splitting process
is finished, then the split regions are merged using a similar criteria.
Algorithm:
Divide in 4 disjoint regions any region R where P(R
i
) = False
Join the adjacent region R
j
and R
k
for which P(R
j
R
k
) = True
Stop when no regions can be divided or joined
Region Based Segmentation: Splitting and Merge
Example:
Region Based Segmentation: Watershed
Watersheds: The idea is to find the watersheds lines, the dam boundaries that
prevent the flooding when the water reaches the top of one of the dams. These
ideas are explained in the figures.
Dam construction ! watershed lines " :
Let M
1
and M
2
denote the sets of coordinates of points in two regional minima .
Then let the set of coordinates of points in the catchement basin associated with
these two minima at stage n-1 of flooding be denoted by C
n# 1
! M
1
" and C
n# 1
! M
2
".
Let C $ n-1% denote the union of these two sets. Finally q is connected component
that encompasses the earlier two components, which indicates that the water
between the two catchement basins has merged at flooding step n. Note that the
two components from step n# 1 can be extracted from q by performing the simple
AND operation q&C$ n# 1%.
1- Do the dilation subject to two conditions:
! 1" the dilation has to be constrained to q! this means that the center of the
structuring element can be located only at points in q during dilation " and
! 2" the dilation cannot be performed on points that would cause the sets being
dilated to merge ! become a single connected component " .
2- The dam boundaries are the points that accomplish both conditions ! see figure "
Dam construction:
Example:
Note: The watersheds let to oversegmentation, which means that sometimes the
result are many more regions that expected. An approach is the use of markers,
where a marker is a connected component belonging to an image.
Region Based Segmentation: K-means
K-means: aims to partition the n observations into k sets (k ! n)
S = {S
1
, S
2
, !, S
k
} so as to minimize the within-cluster sum of squares

where
i
is the mean of points in S
i
.

The most common algorithm uses an interactive refinement technique
arg min
S
!
i" 1
k
!
x
j
S
i
x
j
#
i
2
Region Based Segmentation: K-means
Given an initial set of k means m
1
1
,! ,m
k
1
" see below#, the algorithm proceeds by alternating between two steps:
Step 1 Assigment step : Assign each observation to the cluster with the closest mean
S
i
t
$
%
x
p
: x
p
& m
i
t
' x
p
& m
j
t
1' j ' k
(
where each x
p
goes into exactly one S
i
t
, even if it could go in two of them
Step 2 Update step : Calculate the new means to be the centroid of the observations in the cluster
m
i
t ) 1
$
1
S
i
t
*
x
j
S
i
t
x
j
The algorithm is deemed to have converged when the assigments no longer change .
1) k initial "means" (in this
case k=3) are randomly
generated within the data
domain (shown in color).
2) k clusters are created
by associating every
observation with the
nearest mean. The
partitions here represent
the Voronoi diagram
generated by the means.
3) The centroid of
each of the k clusters
becomes the new
mean.
4) Steps 2 and 3 are
repeated until
convergence has
been reached.
Region Based Segmentation: Mean-shift
Mean-shift: it is a procedure for locating the maxima of a density function given
discrete data sampled from that function.

It is useful for detecting the modes of
this density.

Mean-shift: it is a procedure for locating the maxima of a density function given
discrete data sampled from that function.

It is useful for detecting the modes of
this density.

This is an iterative method, and we start with an initial estimate x. Let a kernel
function be given K! x
i
" x #. This function determines the weight of nearby points
for re-estimation of the mean . Typically we use the Gaussian kernel on the
distance to the current estimate,
K ! x
i
" x#$ e
c x
i
" x
2
The weighted mean of the density in the window determined by K is
m! x#$
%
x
i
N! x #
K ! x
i
" x # x
i
%
x
i
N! x #
K ! x
i
" x #
where N ! x# is the neigborhood of x , a set of points for which K ! x #& 0.
The mean-shift algorithm now sets x'm! x#, and repeats its estimation
until m! x # converges .
Mean-shift: Examples
Connectivity Analysis and Labeling
Connectivity analysis and labeling: Once an image is binarized, then we
have to find the image regions and give a label to each region. The method
follows a two stage procedure. In the first one, all the pixels are labeled
following the procedure from top to bottom and from left to right. In the second
stage, the miss labeled pixels are corrected. Example:
Binarized Image
Contour
Label Scanning
Labels Scanning
Labels Scanning
Labels Scanning
Scanning
0 1 0 0 0 0 0 0
0 1 0 0 1 0 0 0
0 1 0 0 1 0 0 0
0 1 0 0 1 0 1 0
0 0 1 1 1 0 1 0
0 0 0 0 0 1 1 1
0 0 1 1 0 0 0 0
0 0 0 1 0 0 0 0
!
0 4 0 0 0 0 0 0
0 1 0 0 1 0 0 0
0 1 0 0 1 0 0 0
0 1 0 0 1 0 1 0
0 0 1 1 1 0 1 0
0 0 0 0 0 1 1 1
0 0 1 1 0 0 0 0
0 0 0 1 0 0 0 0
!
0 4 0 0 0 0 0 0
0 4 0 0 5 0 0 0
0 4 0 0 5 0 0 0
0 4 0 0 5 0 6 0
0 0 1 1 1 0 1 0
0 0 0 0 0 1 1 1
0 0 1 1 0 0 0 0
0 0 0 1 0 0 0 0
!
0 4 0 0 0 0 0 0
0 4 0 0 5 0 0 0
0 4 0 0 5 0 0 0
0 4 0 0 5 0 6 0
0 0 4 4 ? 0 6 0
0 0 0 0 0 6 6 6
0 0 7 7 0 0 0 0
0 0 0 7 0 0 0 0
!
0 4 0 0 0 0 0 0
0 4 0 0 4 0 0 0
0 4 0 0 4 0 0 0
0 4 0 0 4 0 4 0
0 0 4 4 4 0 4 0
0 0 0 0 0 4 4 4
0 0 7 7 0 0 0 0
0 0 0 7 0 0 0 0
Example:

The connectivity result are two regions one with the label 4 and the
other one with the label 7.

The region with the label 4 has the following pixels: (0,1),(1,1),(2,1),
(3,1),(4,2),(4,3),(4,4),(3,4),(2,4),(1,4),(5,5),(5,6),(5,7),(4,6),(3,6)

The region with label 7 has the following pixels: (6,2),(6,3),(7,3)
Contour Extraction
Contour extraction: Once an image is binarized, we can extract the pixels of
the object contour and obtain the perimeter.

Algorithm:
Step 1: Scan the image to look for the first 1. Put pixel c=(x
c
,y
c
) at the same
position of the first pixel with value 1. Put pixel d at the position d=(xc,yc-1)
Scanning
First 1 pixel
Contour Extraction

Algorithm:
c
,y
c
) at the same
Scanning
c=(x
c
,y
c
)
d=(xc,yc-1)
Contour Extraction

Algorithm:
c
,y
c
) at the same
Scanning
c=(x
c
,y
c
)
d=(xc,yc-1)
Contour Extraction

Algorithm:
c
,y
c
) at the same
Step 2: Change the value of c to c=3 and d to d=2.
c=3
d=2
Contour Extraction

Algorithm:
Step 3: With center on c and starting from d, turn clockwise and assign the
label e
k
to the first pixel which value is 1, 4 or 3. Then:
If c is c=3, e
k
=4 and e
h
=2 for any h < k, then change the value 3 to 4
and 2 to 0 and STOP (the algorithm has arrived to the first contour pixel)
Otherwise, change c to c=4 (if its value was c=1). Then take e
k
as the
new c (c= e
k
) and e
k-1
as the new d (d= e
k-1
). Finally return to Step 2.
Contour Extraction

Algorithm:
label e
k
If c is c=3, e
k
=4 and e
h
k
as the
new c (c= e
k
) and e
k-1
as the new d (d= e
k-1
Contour Extraction

Algorithm:
label e
k
If c is c=3, e
k
=4 and e
h
k
as the
new c (c= e
k
) and e
k-1
as the new d (d= e
k-1
Contour Extraction

Algorithm:
label e
k
If c is c=3, e
k
=4 and e
h
k
as the
new c (c= e
k
) and e
k-1
as the new d (d= e
k-1
Contour Extraction

Algorithm:
c
,y
c
) at the same

Step 2: Change the value of c to c=3 and d to d=2.

label e
k

If c is c=3, e
k
=4 and e
h
=2 for any h < k, then change the value
3 to 4 and 2 to 0 and STOP (the algorithm has arrived to the first
contour pixel)
k
as
the new c (c= e
k
) and e
k-1
as the new d (d= e
k-1
). Finally return to
Step 2.
Contour Extraction

0 0 0 0 0 0
0 1 1 0 0 0
0 1 1 0 0 0
0 0 0 1 0 0
0 0 0 0 0 0
0 0 0 0 0 0
!
0 2 0 0 0 0
0 3 1 0 0 0
0 1 1 0 0 0
0 0 0 1 0 0
0 0 0 0 0 0
0 0 0 0 0 0
!
0 2 0 0 0 0
0 3 4 0 0 0
0 1 1 0 0 0
0 0 0 1 0 0
0 0 0 0 0 0
0 0 0 0 0 0
!
0 2 0 0 0 0
0 3 4 0 0 0
0 1 4 0 0 0
0 0 0 1 0 0
0 0 0 0 0 0
0 0 0 0 0 0
!
0 2 0 0 0 0
0 3 4 0 0 0
0 1 4 0 0 0
0 0 0 4 0 0
0 0 0 0 0 0
0 0 0 0 0 0
!
0 2 0 0 0 0
0 3 4 0 0 0
0 1 4 0 0 0
0 0 0 4 0 0
0 0 0 0 0 0
0 0 0 0 0 0
!
0 2 0 0 0 0
0 3 4 0 0 0
0 4 4 0 0 0
0 0 0 4 0 0
0 0 0 0 0 0
0 0 0 0 0 0
!
0 0 0 0 0 0
0 4 4 0 0 0
0 4 4 0 0 0
0 0 0 4 0 0
0 0 0 0 0 0
0 0 0 0 0 0
Final contour " x,y# coordinates: " 1,1# ," 1,2# ," 2,2# ," 3,3# ," 2,2# , " 2,1#
Perimeter : 6
Contour
Basic Feature Extraction
Basic features:
Rectangle which includes the labeled pixels: Diagonal coordinates

Rectangle area of the region: Total number of pixels of the rectangle

Region area: Total number of pixels of the region

Perimeter: Number of pixels from the initial pixel to the final pixel

Compacity: C = P
2
/ A where (P is the perimeter and A is the area)

Excentricity: e = ((M
20
M
02
)
2
+ 4 M
11
) / A (where M
ij
is the moment ij of
second order)

Euler number: E = (Number of connected regions) (Number of holes)
Basic Feature Extraction: Second Order Moments
Features extracted from second order moments of an image
Area: M
00
!
"
x! 0
N
"
y! 0
N
f # x , y$
Coordinates of the geometric area
M
01
!
"
x! 0
N
"
y! 0
N
f # x , y$ x
M
00
M
10
!
"
x! 0
N
"
y! 0
N
f # x , y$ y
M
00
Magnitude and orientation:
G!
1
2
# M
02
% M
20
$&
'
# M
20
( M
02
$
2
( 4# M
11
$
2
y !!
1
2
%tag
( 1
#
2M
11
M
20
( M
02
$
with
M
02
!
"
x! 0
N
"
y! 0
N
f # x , y $ y
2
M
00
( M
10
2
; M
20
!
"
x! 0
N
"
y! 0
N
f # x , y$ x
2
M
00
( M
01
2

M
11
!
"
x! 0
N
"
y! 0
N
f # x , y$ xy
M
00
( M
10
M
01

T3: Segmentation and Feature Extraction: Computer Vision

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

T3: Segmentation and Feature Extraction: Computer Vision

Uploaded by

Copyright:

Available Formats

Master Course

and the orientation % ! ' is: !% x , y'! tag

You might also like