Biometric System Laboratory

University of Bologna - ITALY

http://biolab.csr.unibo.it

Using synthetic data to

evaluate the quality of
minutiae extraction algorithms
Raffaele Cappelli
 Outline

• How to test semantic conformance to minutiae standard formats?

• SFinGe: a synthetic fingerprint generator

– Generating images with ground truth minutiae

• Experiments
– The impact of fingerprint quality on feature extraction errors
– Analysis of minutiae placement errors
– Correction of minutiae placement to improve interoperability

Using synthetic data to evaluate the quality of minutiae extraction algorithms

1
 Semantic conformance to minutiae standard formats?
• Recent interoperability projects (MINEX, MTIT) showed that:
– Syntactic conformance is not enough for interoperability Minutiae from two
– Some template generators are semantically non-conformant ISO-compliant
• Such non-conformance degrades interoperability extractors
– Extraction algorithm standardization should embed testing
Different type,
• The INCITS 378 and the ISO/IEC 19794-2 standards contain position and direction
definitions of minutiae points and rules for their placement
– Revisions of the two standards are being proposed with refined
guidance on minutia placement
• How to test semantic conformance to data format standards?
– i.e., is a given record an accurate representation of the true
fingerprint minutiae?
– A lot of data (with ground truth minutiae!) is needed Different
– How to ensure the test data is error free (correct ground truth)? position Different position
and direction
– How to deal with data protection / privacy issues?
• Does the fingerprint quality affect the semantic conformance
of feature extraction algorithms?

Using synthetic data to evaluate the quality of minutiae extraction algorithms

2
 SFinGe – A Synthetic Fingerprint Generator
• SFinGe (the Italian for Sphinx, pron. sphin-ge)
– A software able to synthetically (randomly) generate large databases of
realistic fingerprint images
– Used (and validated!) in FVC2000, FVC2002, FVC2004, and FVC2006
– Many parameters can be adjusted to control the fingerprint generation and
simulate different acquisition devices/conditions:
• skin conditions, finger placement, skin deformation, noise, scanner background...
– Several research groups and companies are currently using SFinGe to:
• compare different fingerprint matching algorithms
• train pattern recognition techniques that require large learning-sets (e.g. neural
network, PCA,...)
• easily generate a large number of “virtual users” to develop and test
medium/large-scale fingerprint-based systems

• References:
– [Book Chapter] R. Cappelli,”Synthetic fingerprint generation”, in D. Maltoni,
D. Maio, A.K. Jain and S. Prabhakar, Handbook of Fingerprint Recognition,
Springer (New York), 2003.

Using synthetic data to evaluate the quality of minutiae extraction algorithms

3
 SFinGe - Examples

Real Synthetic Real Synthetic

Real Synthetic

Using synthetic data to evaluate the quality of minutiae extraction algorithms

4
Typical feature extraction steps (from a real fingerprint)

Fingerprint
Fingerprint
image
image

Directional
Directional Fingerprint
Fingerprint Density
Density
map
map shape
shape map
map

Singularities Ridge
Ridge
Singularities
pattern
pattern

Minutiae
Minutiae

Using synthetic data to evaluate the quality of minutiae extraction algorithms

5
 How SFinGe works (1)

Singularities
Singularities

Directional
Directional Fingerprint
Fingerprint Density
Density
map
map shape
shape map
map

Ridge
Ridge
pattern
pattern

Fingerprint
Fingerprint
image
image

Using synthetic data to evaluate the quality of minutiae extraction algorithms

6
 How SFinGe works (2)
Shape Class and Average Singularities
parameters singularities density

Fingerprint Directional map Density map

shape model model model
Fingerprint Directional
Density
shape map
map
Ridge pattern Initial
generation seeds

Level
Contact Erosion
Master-fingerprint region Dilation
Position

Fingerprint
image Background Translation Noising & Skin deforma-
generator rotation rendering tion model
Background type Noise Deformation
dx,dy,θ probability parameters
and noise level
Using synthetic data to evaluate the quality of minutiae extraction algorithms
7
The “master fingerprint”: a “perfect” binary ridge pattern

Iteratively
Iterativelycreated
createdby
byapplying
applyingGabor
Gabor Realistic
Realisticminutiae
minutiaeappear
appearatatrandom
random
filters
filterswhose
whoseorientation
orientationand
andfrequency
frequency positions
positions
are
arelocally
locallyadjusted
adjustedaccording
accordingtotothe
the
directional and density maps.
directional and density maps.

SfinGe “master
fingerprints” are well-
suited for applying the
precise minutiae
extraction procedures
that are being proposed
as ANSI and ISO
standards.

Using synthetic data to evaluate the quality of minutiae extraction algorithms

8
 Automatic generation of the ground-truth
Master-fingerprint

Fingerprint image
SFinGe impression-
generation steps

Standard-
Master-fingerprint minutiae

compliant

Ground-truth minutiae
feature
extraction

Using synthetic data to evaluate the quality of minutiae extraction algorithms

9
 Some advantages

• Automatic generation of large fingerprint databases with ground-truth

minutiae
– Features can be extracted by applying the standard procedures easily and
without ambiguities (extraction occurs on a binary image without noise)
• The main fingerprint characteristics can be controlled
– e.g. Fingerprint class, ridge line density, finger placement, skin distortion,
fingerprint quality, ...
– Datasets to test the impact of a given parameter (e.g. fingerprint quality)
can be easily generated
• The ground truth is always unique and sound, even when the quality of
the final image is very low

Using synthetic data to evaluate the quality of minutiae extraction algorithms

10
 Evaluating minutiae extractors quality
• Datasets:
– Dataset 0: images generated by SFinGe with no noise (binary images –
maximum quality)
– Datasets 1,2,3,4,5: obtained from a larger dataset generated by SFinGe (with
background and noise), by partitioning the images according to the NIST
Fingerprint Quality Index

0 1 2 3 4 5
– Minutiae ground-truth generated according to the ISO/IEC 19794-2 standard
• Algorithms
– 5 feature extractors able to save ISO/IEC 19794-2 records have been tested
Using synthetic data to evaluate the quality of minutiae extraction algorithms
11
 Experiment 1: fingerprint quality vs. extraction errors
7

• The minutiae template created by each algorithm is 6

compared to the ground truth 5

• The “corresponding” minutiae are determined using

4
Algorithm 1
3 Algorithm 2
the Hungarian algorithm to maximize the number of 2
Algorithm 3
Algorithm 4
Algorithm 5
matched minutiae (assignment problem) 1

• Interesting statistics can be gathered: 0

Binary 0 NFIQ 1 NFIQ 2 NFIQ 3 NFIQ 4 NFIQ 5

– False and missed minutiae, Type errors, Position errors, Average position error
Direction errors (in pixels)
200,0% 50,0% 40
Algorithm 1
180,0% Algorithm 2 45,0% 35 Algorithm 1
Algorithm 2
160,0% Algorithm 3 40,0%
30 Algorithm 3
Algorithm 4 Algorithm 4
140,0% 35,0%
Algorithm 5 Algorithm 5
120,0% 30,0% 25

100,0% 25,0% Algorithm 1 20

80,0% 20,0% Algorithm 2
Algorithm 3 15
60,0% 15,0% Algorithm 4
10
40,0% 10,0% Algorithm 5

20,0% 5,0% 5

0,0% 0,0% 0
Binary 0 NFIQ 1 NFIQ 2 NFIQ 3 NFIQ 4 NFIQ 5 Binary 0 NFIQ 1 NFIQ 2 NFIQ 3 NFIQ 4 NFIQ 5 Binary 0 NFIQ 1 NFIQ 2 NFIQ 3 NFIQ 4 NFIQ 5

False+Missed minutiae Type errors Average direction error

(ridge ending/bifurcation) (in degrees)
Using synthetic data to evaluate the quality of minutiae extraction algorithms
12
 Experiment 2: Minutiae density
Intensity I[x,y] is proportional to the estimated likelihood
that a minutia will generated by SFinGe at (x,y).

The density function has been estimated from 200,000

minutiae templates generated by SFinGe.

I[x,y] is proportional to the estimated

likelihood that a minutia will be found at
(x,y) by the feature extraction algorithm.

The density function has been estimated

from 1,000 fingerprints [288x384 pixels]
in dataset 0.

Ground truth Algorithm 5

2
3
1
4
Using synthetic data to evaluate the quality of minutiae extraction algorithms
13
 Experiment 3: Analysis of minutiae placement
Intensity I[x,y] is proportional to the estimated likelihood that a minutia
will be found by a feature extraction algorithm at position (x,y) with
respect to the ground truth minutia direction (denoted by the arrow).

Algorithm 1 Algorithm 2 Algorithm 3 Algorithm 4 Algorithm 5

Ridge
Bifurcations

Ridge
Endings

Dataset used: 1 (NFIQ=1)

Using synthetic data to evaluate the quality of minutiae extraction algorithms

14
 Experiment 4: Minutiae placement correction
Alg. 4 Alg. 5
Offset added to each minutia along the
minutiae direction reported by the algorithms Ridge
Bifurcations
Minutiae
Alg. 4 Alg. 5
detected as
Bifurcations -1 pixel +1 pixel
Ridge
Endings -4 pixels -1 pixel
Endings

Experiment on FVC2006 DB2 (1680 fingerprints)

A real (non-synthetic) fingerprint database
-12%
Without correction With correction
FNMR at FNMR at -38%
Alg. 4 Alg. 5 Alg. 4 Alg. 5
FMR=0.1% FMR=0.1%
Alg. 4 0.108 1.212 Alg. 4 0.108 0.747 +1%
Alg. 5 0.801 1.537 Alg. 5 0.703 1.558

Using synthetic data to evaluate the quality of minutiae extraction algorithms

15
 Software tools that are being developed at BioLab

New version of SFinGe

• Generates fingerprints
(500/1000ppi) with ground
truth minutiae
• Distributes the generation
among multiple computers

Minutiae Template Analyzer

• Compares ground truth with
extracted minutiae
• Generates statistics

Using synthetic data to evaluate the quality of minutiae extraction algorithms

16
 Conclusions
• Syntactic conformance to a standard minutiae format is not enough for
interoperability: semantic conformance is needed
• SFinGe can be used to generate large fingerprint databases with
ground-truth minutiae
– The minutiae can be extracted by applying the standard procedures easily
and without ambiguities (extraction occurs on a binary image without
noise)
– Datasets to test the impact of a given parameter (e.g. fingerprint quality)
can be easily generated
• From the preliminary experimental results:
– Fingerprint quality heavily affects the quality of the extracted minutiae
(false/missed minutiae, placement errors, direction errors, type errors)
• Some algorithms are more affected by low quality than others
– The distribution of minutiae position errors with respect to the ground truth:
• Clearly reveals cases of semantic non-conformance
• Depends on the minutiae type and is characteristic of each extraction algorithm
• Its analysis may be useful to improve interoperability

Using synthetic data to evaluate the quality of minutiae extraction algorithms

17
Biometric System Laboratory
University of Bologna - ITALY
http://biolab.csr.unibo.it

Thank you

Raffaele Cappelli
cappelli@csr.unibo.it