BLOBs

Lecture on the image part (5)

Automatic Perception 9

Thomas Moeslund Computer Vision and Media Technology lab. Aalborg University tbm@cvmt.dk

Fundamental Steps in Computer Vision

So far: Preprocessing Segmentation

Point 1: 22,33 Point 2: 24, 39 ..

Segmentation

Representation and description

Today BLOBs Segmentation Representation Feature extraction Representation

Actor sitting

Preprocessing

Knowledge base
Problem domain Image acquisition

Recognition and Interpretation

2

Result

Agenda
BLOBs
Finding the pixels for each object

Extracting features
Calculate features
For example size and shape

Feature vector = [2,1,,3]

. . . . . .

Feature vector = [4,7,,0]

3

BLOB what is it?

Blob versus BLOB BLOB = Binary Large Object Also know as a Particle

. . .
4

What can BLOBs be used for?

Isolate the different objects (BLOBs)
Represented as a list of pixels

Representing each object (BLOB) by a set of features (=characteristics)

Calculated from the list of pixels For example the position and size of an object

Why calculate features for a BLOB?

When more objects are present after segmentation, how will you figure out which is the one of interest? (very relevant for your projects!) Solution:
Make a model of the object you are looking for Calculate its characteristics (features) Calculate the characteristics (features) of each BLOB in the image Compare the features of each BLOB with the features of the model The best match defines the object
6

Isolating a BLOB
What we want:
For each object in the image, a list with its pixels

How do we get that?

Connected component analysis (Region growing)

Define: connectivity
Who are my neighbors? 4-connected 8-connected
7

Connected component analysis

Binary image (0,1) Seed point: where do we start? Grassfire concept
Delete (burn) the pixels we visit Visit all CONNECTED (4 or 8) neighbors 4 3 1

Connected component analysis

4 3 x 1 2 4 3 x 1 x 2 4 x 3 x 1 x 2

Connected component analysis

4 3 x x x 1 2 x x 4 x 3 x 1 2 x x x 4 x 3 x 1 2

10

Connected component analysis

x x 4 x 3 x 1 2 x 4 3 x 1 x x 2 x x 4 x 3 x 1 x 2 x x

Recursive algorithm (show: blob, TH, analyze (set: outlines))

11

Extracting features

12

Agenda: Extracting features

What is the purpose? What kind of features can be extracted and how? Feature matching
Relating Features and Pattern Recognition

13

Why feature extracting?

Generate some entities (numbers) that can be used in pattern recognition
For example size and shape

From image operations to mathematical operations

Input: a list of pixel positions Output: Feature vector First step: remove too small, too big, and border BLOBs (show: AuPbSn40, TH, analyze (set: outlines, size, border))

Feature vector = [2,1,,3]

. . . . . . . . . .

Feature vector = [4,7,,0]

14

Features

15

Features
One BLOB

Area (number of pixels)

Used to remove noise (small/big objects)

Number of holes in the object Holes area Total area = area + holes area Perimeter = length of contour Bounding box
Upper left corner Height and width of bounding box
16

Features
Center of mass (xm,ym)

xm

Compactness

iobject

ym

iobject

Perimeter 2 Minimum for a disc Area Area

Minimum for a rectangle

Perimeter 2 area

Circularity Longest distance (Ferets diameter) Orientation of the object

Orientation of Ferets diameter

Width Height

17

Features
Degree of elongation

Shapes
Bounding box ratio: Height / Width
Says something about the elongation

How well does the object fit a rectangle

Can be derived from: Area and Perimeter

How well does the object fit an ellipse

Can be derived from: Area and Perimeter

Many other features: see the notes for today and be creative!
18

Feature Matching

19

Feature Matching
But what if more BLOB have the same number of TRUEs? Solution: Measure the DISTANCE from each BLOB to the model and pick the BLOB with the shortest distance, i.e. the BLOB most similar to the model
A much better method as it is based on statistics!

How do we define a distance?

20

Feature Matching
Solution: Measure the DISTANCE from the BLOB to each of the models and pick the model with the shortest distance, i.e. the model most similar to the BLOB How do we define a distance?

21

Feature Matching
Distance in feature-space Feature 1: Area Feature 2 Feature 2: Circularity 2 dimensional feature space BLOB: Model types:

MATCH!

Feature 1
22

Feature Matching: Distance

D(i) ( M f 1 Bi , f 1 ) ( M f 2 Bi , f 2 )
2 2 # features j 1

( M fj Bi , fj ) 2

Feature 2

Euclidean distance

Feature 1

23

Feature Matching
Problem:
Area is measured in 1000s and circularity [0,1] That means that the area will dominate the distance measure completely! Solutions:
Use ratios: So W/H is a better feature than W or H Normalize all feature to the same interval, e.g., [0,1] Let some of the features be TRUE/FALSE and others Euclidean distances
24

Feature Matching
What if we trust some features more than others?
For example, with small objects the perimeter might be uncertain

Solution: Weigthing
D(i) ( M f 1 Bi , f 1 ) ( M f 2 Bi , f 2 )
2 2 # features j 1

( M fj Bi , fj ) 2
# features j 1

D(i) W1 ( M f 1 Bi , f 1 ) W2 ( M f 2 Bi , f 2 )
2 2

W j ( M fj Bi , fj ) 2

25

What to remember
BLOB = Binary Large OBject
Find a list of pixels for each object

Connectivity: 4 versus 8 connected Connected component analysis Features (=characteristics) of an object

Many exist Many more can be derived

Feature matching
TRUE/FALSE Euclidean distance Normalize Learn the features of the model
26

Exercises
Questions to the lecture? What was good about the lecture and what could have been better? Discuss the PE-questions In what order will the pixels be marked if a
4-connected kernel is used? 8-connected kernel is used?

Calculate the perimeter of this object using different distance measures

Euclidean distance, city-block (manhatten) distance, chessboard distance Find out what they are by searching the internet

Play around in ImageJ with the BLOB analysis (Analyze) perhaps you can combine it with morphology Find the approximate centers of all dots in the image: dots-lines 27

Exercises
If youre using images in you project:
Which features could you imagine using?

Imagine you are using the Area of a BLOB as a feature in your project. Find a way to normalize the feature matching, so the result always is in the interval: [0,1] (where 0 means a very poor match and 1 means a perfect match) What features can be used to classify the different letters in the image: alphabeth? Draw the signature for a box
What is the problem with this signature compared with the signature for the circle? How can this problem be handled?

How is region growing different from connected component analysis?

28