Image Segmentation

• Segmentation is an image analysis task that subdivides an

image into disjoint regions of interest for further analysis. It is
usually the first step in image analysis.
• The disjoint regions usually correspond to the different objects of
interest in the image.

Given Image Segmented Image

• It is an important and usually difficult step in image processing.

• Accurate segmentation determines the accuracy of subsequent
image analysis steps.
 Segmentation

Discontinuity Grayvalue
Detection Similarity

Detection of: • Thresholding

• Isolated points • Region growing
• Lines • Region splitting
• Edges and merging
 Detection of Discontinuities
• This is usually accomplished by applying a suitable mask to
the image.
w1 w2 w3 3×3 mask
D = w4 w5 w6
w7 w8 w9

f1 f2 f3
f4 f5 f6 → R
f7 f8 f9

9
f5 → R = wi f i
i=1

• The mask output or response at each pixel is computed by

centering centering the mask on the pixel location.
• When the mask is centered at a point on the image boundary, the
mask response or output is computed using suitable
boundary condition. Usually, the mask is truncated.
 Point Detection
• This is used to detect isolated spots in an image.
• The graylevel of an isolated point will be very different from
its neighbors.
• It can be accomplished using the 3×3 mask:
following

−1 −1 −1
−1 8 −1
−1 −1 −1

• The output of the mask operation is usually thresholded.

• We say that an isolated point has been detected if
8 f5 − ( f1 f2 + f3 + f4 + f6 + f7 + f8 + f9 ) > T
+

for some pre-specified non-negative threshold T.

Orig. Image T = 0.25 T = 0.35 T = 0.5

 Detection of lines
• This is used to detect lines in an image.
• It can be done using the following four masks:
−1 −1 −1
D0 = 2 2 2 Detects horizontal lines
−1 −1 −1
−1 −1 2
D45 = −1 2 −1 Detects 45 lines
2 −1 −1
−1 2 −1
D90 = −1 2 −1 Detects vertical lines
−1 2 −1
2 −1 −1
D135 = −1 2 −1 Detects135 lines
−1 −1 2

• Let R0 , R45 , R90 , and R135 , respectively be the response to

masks
D0 , D45 , D90 , and D135 , respectively. At a given pixel (m,n), if
R135 is the maximum among {R0 , R45 , R90 , R135 }, we say that a
135 line is most likely passing through that pixel.
R0 = max{R0 ,R45 , R90 ,R135 } R45 = max{R0 , R45 , R90 , R135 }

R90 = max{R0 , R45 , R90 , R135 R135 = max{R0 , R45 , R90 , R135
} }
 Edge Detection
• Isolated points and thin lines do not occur frequently in
most practical applications.
• For image segmentation, we are mostly interested in detecting the
boundary between two regions with relatively distinct gray-
level properties.
• We assume that the regions in question are sufficiently
homogeneous so that the transition between two regions can
be determined on the basis of gray-level discontinuities alone.
• An edge in an image may be defined as a discontinuity or abrupt
change in gray level.

Intensit Intensit

Distanc Distanc
Ideal Step Edge Ideal Roof Edge

Intensit Intensit

Distanc Distanc

Combination of Step Ideal Spike Edge

and Roof Edges
• These are ideal situations that do not frequently occur in practice.
Also, in two dimensions edges may occur at any orientation.
• Edges may not be represented by perfect discontinuities.
Therefore, the task of edge detection is much more difficult than
what it looks like.
• A useful mathematical tool for developing edge detectors is
the first and second derivative operators.

• From the example above it is clear that the magnitude of the first
derivative can be used to detect the presence of an edge in
an image.
• The sign of the second derivative can be used to determine
whether an edge pixel lies on the dark or light side of an edge.
• The zero crossings of the second derivative provide a powerful
way of locating edges in an image.
• We would like to have small-sized masks in order to detect
fine variation in graylevel distribution (i.e., micro-edges).
• On the other hand, we would like to employ large-sized masks in
order to detect coarse variation in graylevel distribution (i.e.,
macro-edges) and filter-out noise and other irregularities.
• We therefore need to find a mask size, which is a
compromise between these two opposing requirements, or
determine edge content by using different mask sizes.
• Most common differentiation operator is the gradient.
∂f ( x ,
y)
∇f (x, y) = ∂x
∂f ( x,
y)
∂y
• The magnitude of the gradient is:
2 2 1/ 2
∂f ( x, ∂f ( x, y )
∇f (x, y) = y) +
∂y
∂x

• The direction of the gradient is given by :

∂f
∠∇f (x, y) = tan
−1 ∂y
∂f
∂x

• In practice, we use discrete approximations of the partial

∂f ∂f
derivatives and , which are implemented using the masks:
∂x ∂y
 1
Dh = 1{[−1 1 0]+ [0 −1 1]}= [−1 0 1]
2 2
−1 0 −1
1 1
Dv = 1 + −1 = 0
2 2
0 1 1

• The gradient can then be computed as follows:

Dh ∇f (m,n)
(•)2 + (•)2
f (m,n)

Dv −1 • ∠∇f (m,n)
tan
•

• Other discrete approximations to the gradient (more precisely, the

appropriate partial derivatives) have been proposed
(Roberts, Prewitt).
• Because derivatives enhance noise, the previous operators may
not give good results if the input image is very noisy.
• One way to combat the effect of noise is by applying a smoothing
mask. The Sobel edge detector combines this smoothing
operation along with the derivative operation give the following
masks:
• Since the gradient edge detection methodology depends only
on the relative magnitudes within an image, scalar
multiplication by
 factors such as 1/2 or 1/8 play no essential role. The same is
true for the signs of the mask entries. Therefore, masks like
−1 0 1 1 0 −1 −1 0 1
1
−2 0 2 2 0 − −2 0 2
8
−1 0 1 2 −1 0 1
1 0 −1

correspond to the same detector, namely the Sobel edge detector.

• However when the exact magnitude is important, the proper
scalar multiplication factor should be used.
• All masks considered so far have entries that add up to zero.
This is typical of any derivative mask.
 Example

Usual gradient
mask

Negative mag. of Thresholded Thresholded

Gradient output output for noisy
(noise-free image) (noise-free image) image

obel edge
etector

Negative mag. of Thresholded Thresholded

Gradient output output for noisy
(noise-free image) (noise-free image) image
 The Laplacian Edge Detector
• In many applications, it is of particular interest to construct
derivative operators, which are isotropic (rotation invariant).
This means that rotating the image f and applying the operator
gives the same result as applying the operator on f and then
rotating the result.
• We want the operator to be isotropic because we want to equally
sharpen edges which run in any direction.
• This can be accomplished by another detector, which is based on
the Laplacian of a two-dimensional function.
• The Laplacian of a two-dimensional f (x, y)is given by
function

2 ∂2 f ∂2 f
∇ f (x, y) = 2 + 2
∂x ∂y

• The Laplacian is rotation invariant!

• A discrete approximation can be obtained as:
∇2 f (m, n) ≈4 f (m, n) −( f (m −1, n) f (m +1,n) + f (m, n −1+ f (m, n +1))
+

• This can be implemented using the following mask:

0 −1 0
−1 4 −1
0 −1 0

• Other approximations of the Laplacian can be obtained by

means of the masks:
 −1 −1 −1 1 −2 1
−1 8 −1 or −2 4 −2
−1 −1 −1 1 −2 1

• The Laplacian operator suffers from the following drawbacks:

It is a second derivative operator and is therefore
extremely sensitive to noise.
Laplacian produces double edges and cannot detect
edge direction.
• Laplacian normally plays a secondary role in edge detection.
• It is particularly useful when the graylevel transition at the edge
is not abrupt but gradual.

Bacteria image Laplacian of image Thresholded

output