Image Segmentation

Dr. A S Jalal
 Key Stages in Digital Image Processing

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Image Aquisition

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Image Enhancement

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Image Restoration

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Morphological Processing

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Segmentation

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Object Recognition

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Representation & Description

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Image Compression

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
 Key Stages in Digital Image Processing:
Colour Image Processing

Image Morphological
Restoration Processing

Image
Segmentation
Enhancement

Image Object
Acquisition Recognition

Representation &
Problem Domain
Description
Colour Image Image
Processing Compression
Image Segmentation
Image enhancement & restoration

Digital Image
Digital Image (of improved
quality)

Image Processing
Techniques
(enhancement &
restoration)
Image segmentation

Attributes
Digital Image extracted from
that image

Image Analysis
Techniques
(segmentation)
 The Segmentation Problem
 Segmentation attempts to partition the pixels of an
image into groups that strongly correlate with the
objects in an image
 Typically the first step in any automated computer
vision application
Image segmentation: Example

Original Image Region Segmented Image

Image segmentation: Example …
The goals of segmentation
 Separate image into coherent “objects”
image human segmentation
Image segmentation

 Autonomous segmentation is the

most important but

 One of the most difficult tasks in

image processing

 Success of segmentation determines

success of image analysis
Introduction
 The purpose of image segmentation is to
partition an image into meaningful regions with
respect to a particular application
 The segmentation is based on measurements

taken from the image and might be greylevel,

colour, texture, depth or motion
 Usually image segmentation is an initial and

vital step in a series of processes aimed at

overall image understanding
Principal approaches
 Segmentation algorithms generally are based on
one of 2 basis properties of intensity values

 Discontinuity (abrupt changes )

to partition an image based on abrupt changes
in intensity (such as edges)

 Similarity (homogeneity)
to partition an image into regions that are
similar according to a set of predefined criteria.
 Input image

Result of edge detection Result of region

technique extraction technique
Segmentation : Fundamentals
 Let v be the spatial domain on which the image
is defined. Image segmentation divides v into n
regions, Ri (i = 1 to n) such that
 Segmentation based
on
Detection of homogeneity

Pixel based Region based

approach approach
Pixel – based Approach

Regions in an image

Different objects Parts of an object

in the scene
Pixel – based Approach ...

Different objects or
parts of the same object have
different characteristics

So

Feature values recorded at pixels belonging to

different regions should be different
Basic Idea of Image Segmentation

All the image segmentation methods assume

that:
1. the intensity values are different in
different regions, and,
2. within each region, which represents the
corresponding object in a scene, the intensity
values are similar.
Basic Idea of Image Segmentation ...

Feature Mapping
values at Feature
every space
pixel

Distinct pixel clusters corresponding to different regions

will be formed
&
The image is segmented !!!
Thresholding

 Gray level thresholding is the simplest segmentation

process.
 Correct thresholding leads to better segmentation.
 Many objects or image regions are characterized by
constant reflectivity or light absorption of their
surface.
 Thresholding is computationally inexpensive and fast.
 Thresholding can easily be done in real time
Thresholding ...

Thresholding as a Transformation Function

T = T[x, y, p(x,y), f(x,y)]
where
 f(x,y) is the gray level of the point (x,y)
 p(x,y) denotes some local property of the point
(e.g. the average gray level of a neighbourhood
centred at (x,y)
Thresholding ...
 Global Thresholding
 If T is only a function of f(x,y), then the threshold is
called global threshold.
 Apply the same threshold to the whole image
 Local Thresholding
 If T is a function of both f(x,y) & p(x,y), the threshold is
called local.
 The threshold depends on local property
 Dynamic (Adaptive) Thresholding
 The threshold depends on spatial coordinates (x,y)
Thresholding …
 Thresholding is the transformation of an input
image f to an output (segmented) binary image g
1 if f ( x, y )  T
g ( x, y )  
0 if f ( x, y )  T
where T is the threshold

g(x,y) = 1 for image elements of the objects &

g(x,y) = 0 for image elements of the background
From a grayscale image, thresholding can be used to
create binary images
Thresholding: Example
 Imagine a poker playing robot that needs to
visually interpret the cards in its hand

Original Image Thresholded Image

 But Be Careful
 If you get the threshold wrong the results can be
disastrous

Threshold Too High Threshold Too Low

 Thresholding: Example ...

Greyscale image "Boat" Its grey level histogram

Binary regions for T = 26 Binary regions for T = 235 Binary regions for T = 235
 Selection of threshold is not a trivial task !!!!!!

Any inappropriate threshold will incur significant

& non-acceptable error in classification

Two problems to be addressed in finding the

threshold:
I. Choice of features/properties to achieve the
desired segmentation
II. Selection of optimum threshold that would
incur the least classification error
Choice of Feature

Segment the image into

two regions –
the disk & the background

What threshold values will

serve the purpose?
Gray levels
Any value between 128 &
Disk : 255 255 (both inclusive)
Background : 127
 Gray levels

Disk : 25% pixels with graylevel 127

50% pixels with graylevel 191
25% pixels with graylevel 255
(all pixels are spatially uniformly distributed)
Background : 127

What now ??
Both background & disk contain pixels with gray level 127

Apply mean filter to the image

The pixels in the central disk will have graylevels
around 191 while that of background will still be 127.
Threshold can be any value between 128 & 191
Thresholding …
 Suppose that the gray-level histogram
corresponds to an image, f(x,y), composed of
dark objects in a light background, in such a way
that object and background pixels have gray
levels grouped into two dominant modes.
 One obvious way to extract the objects from the
background is to select a threshold „T‟ that
separates these modes.
 Then any point (x,y) for which f(x,y) > T is called
an object point, otherwise, the point is called a
background point.
Basic Global Thresholding
 Based on the histogram of an image
 Partition the image histogram using a single
global threshold
 The success of this technique very strongly

depends on how well the histogram can be

partitioned
Basic Global Thresholding Algorithm

 The basic global threshold, T, is calculated as

follows:
1. Select an initial estimate for T (typically the
average grey level in the image)
2. Segment the image using T to produce two
groups of pixels:
G1 consisting of pixels with grey levels >T and
G2 consisting pixels with grey levels ≤ T
1. Compute the average grey levels of pixels in
G1 to give μ1 and G2 to give μ2
Basic Global Thresholding Algorithm …

4. Compute a new threshold value:

1   2
T
2
5. Repeat steps 2 – 4 until the difference in T in
successive iterations is less than a predefined
limit T∞
 This algorithm works very well for finding thresholds
when the histogram is suitable
Thresholding : Example
Thresholding : Example …
Problems With Single Value Thresholding

 Let‟s say we want to

isolate the contents
of the bottles.
 Think about what the
histogram for this
image would look like.
 What would happen if we
used a single threshold value?
Problems With Single Value Thresholding

 Single value thresholding only works for bimodal

histograms
 Images with other kinds of histograms need more
than a single threshold
Problems With Single Value Thresholding …

 Uneven illumination can really upset a single

valued thresholding scheme
Basic Adaptive Thresholding
 An approach to handling situations in which single
value thresholding will not work is to divide an
image into sub images and threshold these
individually
 Since the threshold for each pixel depends on its
location within an image this technique is said to
adaptive
Basic Adaptive Thresholding Example

 The image below shows an example of using adaptive

thresholding with the image shown previously

 As can be seen success is mixed

 But, we can further subdivide the troublesome sub
images for more success
What is a Region?
 Basic definition :- A group of connected
pixels with similar properties.

 Important in interpreting an image because

they may correspond to objects in a scene.

 For correct interpretation, image must be

partitioned into regions that correspond to
objects or parts of an object.
 Region – based Approach
Let R represent the entire image region. Segmentation partitions R
into n subregions, R1, R2, ..., Rn, such that:
a) n
a) Every pixel must be in a
R i R
i 1 region
b) Ri is a connected region, i  1, 2, ..., n .
b) Points in a region must be
connected.
c) Ri  R j   for all i and j, i  j
c) Regions must be disjoint.
d) P( Ri )  TRUE for i  1,2,..., n. d) All pixels in a region satisfy
specific properties.
e) P(Ri  R j )  FALSE for i  j. e) Different regions have
different properties.
P(Ri) is a logical predicate property defined over the points in set Ri
 Region – based Approach

The fundamental drawback of histogram-based

region detection is that histograms provide no
spatial information (only the distribution of gray
levels)

Region-growing approaches exploit the important

fact that pixels which are close together have
similar gray values
Region Growing
 Start from a seed, and let it grow
(include similar neighborhood)
 Once a pixel is accepted as a member of the
current region, the nearest neighbours of this new
pixel are examined.
 The process goes on recursively until no more
pixels are accepted.
 All pixels in the current region are marked with a
unique label.
 Then another seed pixel is picked up & the
procedure is repeated.
Region Growing …

1.Choose the seed pixels (1 for every segment)

2.Check the neighboring pixels and add them to

the region if they are similar to the seed

3.Repeat step 2 for each of the newly added

pixels; stop if no more pixels can be added
Region Growing …
 For region growing we need a rule describing a
growth mechanism and a rule checking the
homogeneity of the regions after each growth step.
 The growth mechanism – at each stage k and for
each region Ri(k), i = 1,…,N, we check if there are
unclassified pixels in the 8-neighbourhood of each
pixel of the region border.
 Before assigning such a pixel x to a region Ri (k),we
check if the region homogeneity:
P(Ri(k) U {x}) = TRUE , is valid
Region Growing: Example
Region Growing: Example
Region Growing: Example
Region Growing: Problems
 Large execution time

 The selection of the property to be

satisfied

 The selection of the seed points

Region Splitting
1. Split starts from the assumption that the entire image
is homogeneous
2. If this is not true (by the homogeneity criterion), the
image is split into four sub images
3. This splitting procedure is repeated recursively until
we split the image into homogeneous regions
Region Splitting ...
 We try to satisfy the homogeneity property over
a rectangular region.
 If the gray levels (or the feature being measured)
present in the region do not satisfy the property,
we divide the region into four equal quadrants.
 If the property is satisfied, we leave the region as
it is.
 This is done recursively until all the regions satisfy
the property.
Region Splitting ...
 If the original image is square N x N, having
dimensions that are powers of 2(N = 2n):
 All regions produced but the splitting algorithm
are squares having dimensions M x M , where
M is a power of 2 as well (M=2m,M<= n).
 Since the procedure is recursive, it produces an
image representation that can be described by
a tree whose nodes have four sons each
 Such a tree is called a Quadtree.
Region Splitting ...

Since the procedure is recursive, it produces an image

representation that can be described by a tree whose
nodes have four childs each
Quadtree
R0 R1
R0

R3
R2 R1

R00 R01 R02 R04

Region Splitting and Merging

Split advantage:
Created regions are adjacent and homogenous

Split dis-advantage:
Over splitting since no merge is performed

Improvement::

Split & merge

 Region Splitting and Merging ...

1. Split (& merge) starts from the assumption that the

entire image is homogeneous

2. If this is not true (by the homogeneity criterion), the

image is split into four sub images
3. This split is repeated until no further splitting is possible
4. Merging phase: If 2 adjacent regions are homogenous,
they are merged
5. Repeat step 4 until no further merging is possible
Region Splitting and Merging
 If a region R is inhomogeneous
(P(R)= False) then R is split into four sub
regions
 If two adjacent regions Ri, Rj are homogeneous

(P(Ri U Rj) = TRUE), they are merged

 The algorithm stops when no further splitting or
merging is possible
Region growing: Example
Region Split: Example
Region Split and Merge: Example
Any Questions ?