DIS - 19989-2 - Criteria and Methodology For Security Evaluation of Biometric Systems

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 19989-2
ISO/IEC JTC 1/SC 27 Secretariat: DIN

Voting begins on: Voting terminates on:
2019-12-30 2020-03-23
Information security — Criteria and methodology for

security evaluation of biometric systems —
Part 2:
Biometric recognition performance
ICS: 35.030
THIS DOCUMENT IS A DRAFT CIRCULATED

FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL, This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 19989-2:2019(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
Licensed to: Plessis, Patrice M.
TO SUBMIT, WITH THEIR COMMENTS,
Downloaded:
NOTIFICATION OF ANY RELEVANT PATENT 2020-03-09
RIGHTS OF WHICH THEY ARE AWARE
SingleAND TO licence only, copying
user and networking prohibited
PROVIDE SUPPORTING DOCUMENTATION. © ISO/IEC 2019
ISO/IEC DIS 19989-2:2019(E)

COPYRIGHT PROTECTED DOCUMENT

© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland Licensed to: Plessis, Patrice M.
Downloaded: 2020-03-09
Single user licence only, copying and networking prohibited
ii © ISO/IEC 2019 – All rights reserved
ISO/IEC DIS 19989-2:2019(E)

Contents Page
Foreword......................................................................................................................................................................................................................................... iv
Introduction...................................................................................................................................................................................................................................v
1 Scope.................................................................................................................................................................................................................................. 1
2 Normative references....................................................................................................................................................................................... 1
3 Terms and definitions...................................................................................................................................................................................... 1
4 Symbols (and abbreviated terms)....................................................................................................................................................12
5 Supplementary activities to ISO/IEC 18045 on ATE tests........................................................................................13
5.1 General......................................................................................................................................................................................................... 13
5.1.1 Guidance............................................................................................................................................................................... 13
5.1.2 Remarks for performance evaluation.......................................................................................................... 14
5.1.3 Identification of the type of performance evaluation.................................................................... 14
5.1.4 Biometric recognition error rates................................................................................................................... 15
5.2 Planning the evaluation................................................................................................................................................................. 19
5.2.1 Overview............................................................................................................................................................................... 19
5.2.2 Estimation of test sizes............................................................................................................................................. 20
5.2.3 Test documentation..................................................................................................................................................... 20
5.3 Data collection....................................................................................................................................................................................... 21
5.3.1 Choice of test data or acquiring test crew and capture device.............................................. 21
5.3.2 Performing test............................................................................................................................................................... 22
5.4 Analyses...................................................................................................................................................................................................... 23
5.5 Specific requirements on assurance components.................................................................................................. 23
5.5.1 Overview............................................................................................................................................................................... 23
5.5.2 Specific requirements on ATE_IND.1............................................................................................................ 23
5.5.3 Specific requirements on ATE_IND.2............................................................................................................ 24
5.6 Reviewing and assessing developer tests...................................................................................................................... 24
5.6.1 Review of developer tests...................................................................................................................................... 24
5.6.2 Repeating a test subset............................................................................................................................................. 25
5.7 Conducting independent testing........................................................................................................................................... 25
5.7.1 Overview............................................................................................................................................................................... 25
5.7.2 Identification of the type of performance evaluation.................................................................... 27
6 Supplementary activities to ISO/IEC 18045 on Vulnerability Assessment (AVA)...........................27
6.1 General aspects..................................................................................................................................................................................... 27
6.2 TOE for testing....................................................................................................................................................................................... 28
6.3 Potential vulnerabilities................................................................................................................................................................ 28
6.4 Rating attack potential................................................................................................................................................................... 29
Annex A (informative) Examples of attack potential computation for AVA activities.....................................30
Annex B (informative) Examples for ATE activities............................................................................................................................36
Annex C (informative) Example of Developer’s performance test document and its
assessment strategy........................................................................................................................................................................................38
Bibliography.............................................................................................................................................................................................................................. 42

© ISO/IEC 2019 – All rights reserved iii
ISO/IEC DIS 19989-2:2019(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work. In the field of information technology, ISO and IEC have established a joint technical committee,
ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting.
Publication as an International Standard requires approval by at least 75 % of the national bodies
casting a vote.
Attention is drawn to the possibility that some of the elements of this document may be the subject of
patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
ISO/IEC 19989-2 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittee SC 27, Information security, cybersecurity and privacy protection.
A list of all parts in the ISO/IEC 19989 series, published under the general title Information security —
Criteria and methodology for security evaluation of biometric systems, can be found on the ISO website.

iv © ISO/IEC 2019 – All rights reserved
ISO/IEC DIS 19989-2:2019(E)

Introduction
Biometric systems, could be vulnerable to presentation attacks where attackers attempt to subvert
the system security policy by presenting their natural biometric characteristics or artefacts holding
copied or faked characteristics. Presentation attacks can occur during enrolment or identification/
verification events. Techniques designed to detect presentation artefacts are generally different from
those to counter attacks where natural characteristics are used. Defence against presentation attacks
with natural characteristics typically relies on the ability of a biometric system to discriminate
between genuine enrolees and attackers based on the differences between their natural biometric
characteristics. This ability is characterised by the biometric recognition performance of the system.
Biometric recognition performance and presentation attack detection have a bearing on the security
of biometric systems. Hence the evaluation of these aspects of performance from a security viewpoint
will become important considerations for the procurement of biometric products and systems.
Biometric products and systems share many of the properties of other IT products and systems which
are amenable to security evaluation using the ISO/IEC 15408 series and ISO/IEC 18045 in the standard
way. However, biometric systems embody certain functionality that needs specialised evaluation
criteria and methodology which is not addressed by the ISO/IEC 15408 series and ISO/IEC 18045.
Mainly these relate to the evaluation of biometric recognition and presentation attack detection. These
are the functions addressed in ISO/IEC 19989 (all parts).
ISO/IEC 19792 describes these biometric-specific aspects and specifies principles to be considered
during the security evaluation of biometric systems. However, it does not specify the concrete criteria
and methodology that are needed for security evaluation based on the ISO/IEC 15408 series.
ISO/IEC 19989 series provide a bridge between the evaluation principles for biometric products
and systems defined in ISO/IEC 19792 and the criteria and methodology requirements for security
evaluation based on the ISO/IEC 15408 series. The ISO/IEC 19989 series supplements the ISO/IEC 15408
series and ISO/IEC 18045 by providing extended security functional requirements together with
assurance activities related to these requirements. The extensions to the requirements and assurance
activities found in the ISO/IEC 15408 series and ISO/IEC 18045 relate to the evaluation of biometric
recognition and presentation attack detection which are particular to biometric systems.
ISO/IEC 19989-1 consists of the introduction of the general framework for the security evaluation
of biometric systems, including extended security functional components, and supplementary
methodology, which is additional evaluation activities for the evaluator. The detailed recommendations
are developed for biometric recognition performance aspects in this document and for presentation
attack detection aspects in ISO/IEC 19989-3.
This document describes supplements to the evaluation methodology for biometric recognition
performance evaluation for the security evaluation of biometric products. It supplements the
ISO/IEC 15408 series, ISO/IEC 18045 and ISO/IEC 19989-1. It builds on the general considerations
described in ISO/IEC 19792 and the biometric performance testing methodology described in
ISO/IEC 19795-1 by providing additional guidance to an evaluator.
In this document the terms “data subject” is used while "user" is used in ISO/IEC 19989-1, in order to be
consistent with biometric vocabulary, as biometric experts are supposed to be the main readers of this
document.

© ISO/IEC 2019 – All rights reserved v
DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 19989-2:2019(E)
Information security — Criteria and methodology for

security evaluation of biometric systems —
Part 2:
Biometric recognition performance
1 Scope
For security evaluation of biometric verification systems and biometric identification systems, this
document is dedicated to the security evaluation of biometric recognition performance applying the
ISO/IEC 15408 series. It provides:
— guidance and requirements to the developer and the evaluator for the supplementary activities on
biometric recognition performance specified in ISO/IEC 19989-1.
The following item is outside the scope of this document:
— the evaluation of presentation attack detection techniques except for presentation from impostor
attempts under the policy of the intended use following the TOE guidance documentation.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.
ISO/IEC 15408-3:2008, Information technology — Security techniques — Evaluation criteria for IT
security — Part 3: Security assurance components
ISO/IEC 18045:2008, Information technology — Security techniques — Methodology for IT security
evaluation
ISO/IEC 19792:2009, Information technology — Security techniques — Security evaluation of biometrics
ISO/IEC 19795-1, Information technology —Biometric performance testing and reporting —Part 1:
Principles and framework
ISO/IEC 19795-2, Information technology — Biometric performance testing and reporting — Part 2:
Testing methodologies for technology and scenario evaluation
ISO/IEC 19989-1:2020, Information Technology — Security techniques — Criteria and methodology for
security evaluation of biometric systems – Part 1: framework
3 Terms and definitions

For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminological databases for use in standardization at the following addresses:
— IEC Electropedia: available at http://w ww.electropedia.org/
— ISO Online browsing platform: available at http://w ww.iso.org/obp

© ISO/IEC 2019 – All rights reserved 1
ISO/IEC DIS 19989-2:2019(E)

3.1
assurance
grounds for confidence that a TOE meets the SFRs
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.4]
3.2
attack potential
measure of the effort to be expended in attacking a TOE, expressed in terms of an attacker's expertise,
resources and motivation
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.5]
3.3
authentication
<security> act of verifying the claimed identity of an entity
Note 1 to entry: authentication: term and definition standardized by ISO/IEC [ISO/IEC 2382-8:1998].
Note 2 to entry: 08.01.11 (2382)
[SOURCE: ISO-IEC-2382-8: 1998 ]

[SOURCE: ISO/IEC 2382:2015, 1. 2126251]
3.4
biometric (adjective)
of or having to do with biometrics
EXAMPLE 1 Incorrect usage #1: ICAO resolved that face is the biometric most suited to the practicalities
of travel documents.
EXAMPLE 2 Correct usage #1: ICAO resolved that face recognition is the biometric mode most suited to the
practicalities of travel documents.
EXAMPLE 3 Incorrect usage #2: The biometric recorded in my passport is a facial image.
EXAMPLE 4 Correct usage #2: T he biometric characteristic recorded in my passport is a facial image.
Note 1 to entry: The use of biometric as a noun, to mean for example, biometric characteristic, is deprecated.
Note 2 to entry: Since the late 19th century, the designations ‘biometrics’ and ‘biometry’ have been used with
the general meaning of counting, measuring and statistical analysis of any kind of data in the biological sciences
including the relevant medical sciences.
[SOURCE: ISO/IEC 2382-37:2017, 3.1.1]

3.5
biometric capture
obtain and record, in a retrievable form, signal(s) of biometric characteristic(s) directly from
individual(s), or from representation(s) of biometric characteristic(s)
Note 1 to entry: Representation’ is used in the natural language sense, e.g. a photograph.
Note 2 to entry: Retrievable’ refers to the record and not the original signal.
Note 3 to entry: A signal can be generated by the biometric characteristic or generated elsewhere and affected by
the biometric characteristic. For example, face illuminated by incident light.
[SOURCE: ISO/IEC 2382-37:2017, 3.6.3]

2 © ISO/IEC 2019 – All rights reserved
ISO/IEC DIS 19989-2:2019(E)

3.6
biometric capture device
device that collects a signal from a biometric characteristic and converts it to a captured biometric sample
Note 1 to entry: A signal can be generated by the biometric characteristic or generated elsewhere and affected by
the biometric characteristic, for example, face illuminated by incident light.
Note 2 to entry: A biometric capture device can be any piece of hardware (and supporting software and firmware).
Note 3 to entry: A biometric capture device may comprise components such as an illumination source, one or
more biometric sensors, etc.
[SOURCE: ISO/IEC 2382-37:2017, 3.4.1]

Note 4 to entry: In this document, the expression "capture device" is sometimes used instead of this term.
3.7
biometric characteristic
biological and behavioural characteristic of an individual from which distinguishing, repeatable
biometric features can be extracted for the purpose of biometric recognition
EXAMPLE Examples of biometric characteristics are: Galton ridge structure, face topography, facial skin
texture, hand topography, finger topography, iris structure, vein structure of the hand, ridge structure of the
palm, retinal pattern, handwritten signature dynamics, etc.
[SOURCE: ISO/IEC 2382-37:2017, 3.1.2]

3.8
biometric enrolee
biometric data subject whose biometric data is held in a biometric enrolment database
[SOURCE: ISO/IEC 2382-37:2017, 3.7.6]
Note 1 to entry: In this document, the short expression “enrolee” is used instead of this term
3.9
biometric enrolment
DEPRECATED: registration
act of creating and storing a biometric enrolment data record in accordance with an enrolment policy
Note 1 to entry: Registration has a different meaning in the signal processing community and its use is therefore
deprecated in biometrics in favour of enrolment.
Note 2 to entry: Enrolment in a biometric system might, in some cases, not involve storage of biometric data, for
example, when biometric data from an enrolee cannot be acquired.
[SOURCE: ISO/IEC 2382-37:2017, 3.5.3]

Note 3 to entry: In this document, the short expression “enrolment” is used instead of this term.
3.10
biometric enrolment database
database of biometric enrolment data record(s)
Note 1 to entry: A database of biometric data not attributable to biometric data subjects is a biometric database,
but not a biometric enrolment database, e.g. a database utilised in the training of a Universal Background Model.
Note 2 to entry: The biometric enrolment database may or may not contain the biometric reference database.
Separation of the databases may be required due to security, privacy, legislation, architecture, performance, etc.
Note 3 to entry: A single biometric reference (e.g. a fingerprint on a storage card) may be considered as a
biometric enrolment database in some transactions.
[SOURCE: ISO/IEC 2382-37:2017, 3.3.9]

ISO/IEC DIS 19989-2:2019(E)

3.11
biometric feature
numbers or labels extracted from biometric samples and used for comparison
Note 1 to entry: Biometric features are the output of a completed biometric feature extraction.
Note 2 to entry: The use of this term should be consistent with its use by the pattern recognition and mathematics
communities.
Note 3 to entry: A biometric feature set can also be considered a processed biometric sample.
Note 4 to entry: Biometric features may be extracted from an intermediate biometric sample.
Note 5 to entry: Filters applied to biometric samples are not themselves biometric features, however the output
of the filter applied to these samples may be. Therefore, for example, eigenfaces are not biometric features.
[SOURCE: ISO/IEC 2382-37:2017, 3.3.11]

Note 6 to entry: In this document, the short expression “feature” is often used instead of this term.
3.12
biometric identification
process of searching against a biometric enrolment database to find and return the biometric reference
identifier(s) attributable to a single individual
Note 1 to entry: Use of the term “authentication” as a substitute for biometric identification is deprecated.
[SOURCE: ISO/IEC 2382-37:2017, 3.8.2]

3.13
biometric impostor
subversive biometric capture subject who performs a biometric impostor attack
Note 1 to entry: COED defines impostor as person who assumes a false identity in order to deceive or defraud.
Note 2 to entry: COED defines impersonate as pretend to be (another person) for entertainment or fraud.
[SOURCE: ISO/IEC 2382-37:2017, 3.7.13]

Note 3 to entry: In this document, the expression "impostor" is sometimes used instead of the full term.
3.14
biometric presentation
interaction of the biometric capture subject and the biometric capture subsystem to obtain a signal
from a biometric characteristic
Note 1 to entry: The biometric capture subject may not be aware that a signal from a biometric characteristic is
being captured.
[SOURCE: ISO/IEC 2382-37:2017, 3.6.7]

Note 2 to entry: In this document, the short expression “presentation” is often used instead of this term
3.15
biometric recognition
biometricsautomated recognition of individuals based on their biological and behavioural
characteristics
Note 1 to entry: In the field of biometrics (as defined in this document), “Individual” is restricted in scope to refer
only to humans.
Note 2 to entry: The general meaning of biometrics encompasses counting, measuring and statistical analysis of
any kind of data in the biological sciences including the relevant medical sciences.
ISO/IEC DIS 19989-2:2019(E)

Note 3 to entry: Biometric recognition encompasses biometric verification and biometric identification (3.8.2) .
Note 4 to entry: Automated recognition implies that a machine based system is used for the recognition either for
the full process or assisted by a human being.
Note 5 to entry: Behavioural and biological characteristics cannot be completely separated which is why
the definition uses ‘and’ instead of ‘and/or’. For example, a fingerprint image results from the biological
characteristics of the finger ridge patterns and the behavioural act of presenting the finger.
Note 6 to entry: Use of ‘authentication’ as a synonym for “biometric verification or biometric identification” is

deprecated; the term biometric recognition is preferred.
[SOURCE: ISO/IEC 2382-37:2017, 3.1.3]

3.16
biometric reference
one or more stored biometric samples, biometric templates or biometric models attributed to a
biometric data subject and used as the object of biometric comparison
EXAMPLE Face image stored digitally on a passport, fingerprint minutiae template on a National ID card or
Gaussian Mixture Model for speaker recognition, in a database.
Note 1 to entry: A biometric reference may be created with implicit or explicit use of auxiliary data, such as
Universal Background Models.
Note 2 to entry: The subject/object labelling in a comparison might be arbitrary. In some comparisons, a biometric
reference might be used as the subject of the comparison with other biometric references or incoming samples
and input to an algorithm for biometric comparison. For example, in a duplicate enrolment check a biometric
reference will be used as the subject for comparison against all other biometric references in the database.
[SOURCE: ISO/IEC 2382-37:2017, 3.3.16]

3.17
biometric sample
analog or digital representation of biometric characteristics prior to biometric feature extraction
EXAMPLE A record containing the image of a finger is a biometric sample.
[SOURCE: ISO/IEC 2382-37:2017, 3.3.21]

3.18
biometric system
system for the purpose of the biometric recognition of individuals based on their behavioural and
biological characteristics
Note 1 to entry: A biometric system will contain both biometric and non-biometric components.
[SOURCE: ISO/IEC 2382-37:2017, 3.2.3]

3.19
biometric verification
process of confirming a biometric claim (3.6.4) through biometric comparison
Note 1 to entry: Use of the term “authentication” as a substitute for biometric verification is deprecated.
[SOURCE: ISO/IEC 2382-37:2017, 3.8.3]

3.20
bona fide presentation
interaction of the biometric capture subject and the biometric data capture subsystem in the fashion
intended by the policy of the biometric system
Note 1 to entry: Bona fide is analogous to normal or routine, when referring to a bona fide presentation.
ISO/IEC DIS 19989-2:2019(E)

Note 2 to entry: Bona fide presentations can include those in which the user has a low level of training or
skill. Bona fide presentations encompass the totality of good-faith presentations to a biometric data capture
subsystem.
[SOURCE: ISO/IEC 30107-3: 2017, 3.1.2]

3.21
bona fide presentation classification error rate
BPCER
proportion of bona fide presentations incorrectly classified as presentation attacks in a specific
scenario
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.2]
3.22
check
generate a verdict by a simple comparison
Note 1 to entry: Evaluator expertise is not required. The statement that uses this verb describes what is mapped.
[SOURCE: ISO/IEC 18045: 2008, 3.3]

3.23
class
set of ISO/IEC 15408 families that share a common focus
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.9]
3.24
comparison
estimation, calculation or measurement of similarity or dissimilarity between biometric probe(s)
[SOURCE: ISO/IEC 2382-37:2017, 3.5.7]
3.25
component
smallest selectable set of elements on which requirements may be based
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.12]
3.26
confirm
declare that something has been reviewed in detail with an independent determination of sufficiency
Note 1 to entry: The level of rigour required depends on the nature of the subject matter. This term is only applied
to evaluator actions.
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.14]

3.27
describe
provide specific details of an entity
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.21]
3.28
detection error trade-off curve
DET curve
modified ROC curve which plots error rates on both axes (false positives on the x-axis and false
negatives on the y-axis)
Note 1 to entry: An example set of DET curves is shown in ISO/IEC 19795-1:2006, 10.6.2, Figure 3.
ISO/IEC DIS 19989-2:2019(E)

[SOURCE: ISO/IEC 19795-1:2006, 4.7.1]

3.29
determine
affirm a particular conclusion based on independent analysis with the objective of reaching a particular
conclusion
Note 1 to entry: The usage of this term implies a truly independent analysis, usually in the absence of any previous
analysis having been performed. Compare with the terms “confirm” or “verify” which imply that an analysis has
already been performed which needs to be reviewed
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.22]

3.30
developer
organisation responsible for the development of the TOE
[SOURCE: ISO/IEC 15408-1: 2009, 3. 4.15]
3.31
development
product life-cycle phase which is concerned with generating the implementation representation of the
TOE
Note 1 to entry: Throughout the ALC: Life-cycle support requirements, development and related terms (developer,
develop) are meant in the more general sense to comprise development and production.
[SOURCE: ISO/IEC 15408-1: 2009, 3. 4.16]

3.32
enrol
create and store a biometric enrolment data record in accordance with the biometric enrolment policy
[SOURCE: ISO/IEC 2382-37:2017, 3.5.8]
3.33
ensure
guarantee a strong causal relationship between an action and its consequences
Note 1 to entry: When this term is preceded by the word “help” it indicates that the consequence is not fully
certain, on the basis of that action alone.
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.25]

3.34
evaluation
assessment of a PP, an ST or a TOE, against defined criteria
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.26]
3.35
examine
generate a verdict by analysis using evaluator expertise
Note 1 to entry: The statement that uses this verb identifies what is analysed and the properties for which it is
analysed.
[SOURCE: ISO/IEC 18045: 2008, 3.7]

ISO/IEC DIS 19989-2:2019(E)

3.36
false accept rate
FARproportion of verification transactions with wrongful claims of identity that are incorrectly
confirmed
[SOURCE: ISO/IEC 19795-1:2006, 4.6.6]
3.37
failure-to-acquire rate
FTAR
proportion of a specified set of biometric acquisition processes that were failures to acquire
Note 1 to entry: The results of the biometric acquisition processes may be biometric probes or biometric
references.
Note 2 to entry: The experimenter specifies which biometric probe (or biometric reference) acquisitions are in
the set, as well as the criteria for deeming a biometric acquisition process has failed.
Note 3 to entry: The proportion is the number of processes that failed divided by the total number of biometric
acquisition processes within the specified set.
[SOURCE: ISO/IEC 2382-37:2017, 3.9.4]

3.38
failure-to-enrol rate
FTER
proportion of a specified set of biometric enrolment transactions that resulted in a failure to enrol
Note 1 to entry: Basing the denominator on the number of biometric enrolment transactions may result in a
higher value than basing it on the number of biometric capture subjects.
Note 2 to entry: If the FTER is to measure solely transactions that fail to complete due to quality of the submitted
biometric data the denominator should not include transactions that fail due to non-biometric reasons (i.e. lack of
eligibility due to age or citizenship).
[SOURCE: ISO/IEC 2382-37:2017, 3.9.7]

3.39
false match rate
FMR
proportion of the completed biometric non-mated comparison trials (3.9.2) that result in a false match
Note 1 to entry: The value computed for the false match rate will depend on thresholds, and other parameters of
the comparison process, and the protocol defining the biometric non-mated comparison trials.
Note 2 to entry: Comparisons between:
— identical twins;
— different, but related biometric characteristics from the same individual, such as left and right hand
topography will need proper consideration (see ISO/IEC 19795-1).
Note 3 to entry: “Completed” refers to the computational processes required to make a comparison decision, i.e.
failures to decide are excluded.
[SOURCE: ISO/IEC 2382-37:2017, 3.9.9]

3.40
false-negative identification-error rate
FNIR
proportion of identification transactions by users enrolled in the system in which the user’s correct
identifier is not among those returned
[SOURCE: ISO/IEC 19795-1:2006, 4.6.8]
ISO/IEC DIS 19989-2:2019(E)

3.41
false non-match rate
FNMR
proportion of the completed biometric mated comparison trials that result in a false non-match
Note 1 to entry: The value computed for the false non-match rate will depend on thresholds, and other parameters
of the comparison process, and the protocol defining the biometric mated comparison trials.
Note 2 to entry: “Completed” refers to the computational processes required to make a comparison decision, i.e.
failures to decide are excluded.
[SOURCE: ISO/IEC 2382-37:2017, 3.9.11]

3.42
false-positive identification-error rate
FPIR
proportion of identification transactions by users not enrolled in the system, where an identifier is
returned
[SOURCE: ISO/IEC 19795-1:2006, 4.6.9]
3.43
false reject rate
FRR
proportion of verification transactions with truthful claims of identity that are incorrectly denied
[SOURCE: ISO/IEC 19795-1:2006, 4.6.5]
3.44
guidance documentation
documentation that describes the delivery, preparation, operation, management and/or use of the TOE
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.36]
3.45
identity
representation uniquely identifying entities (e.g. a user, a process or a disk) within the context of the TOE
Note 1 to entry: An example of such a representation is a string. For a human user, the representation can be the
full or abbreviated name or a (still unique) pseudonym.
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.37]

3.46
impostor attack presentation match rate
IAPMR
<full-system evaluation of a verification system> proportion of impostor attack presentations using the
same PAI species in which the target reference is matched
[SOURCE: ISO/IEC 30107-3: 2017, 3.2.6]
3.47
interaction
general communication-based activity between entities
[SOURCE: ISO/IEC 15408-1: 2009, 3.2.16]
3.48
interface
means of interaction with a component or module general communication-based activity between
entities
[SOURCE: ISO/IEC 15408-1:
Licensed2009, 3.2.17]
to: Plessis, Patrice M.
ISO/IEC DIS 19989-2:2019(E)

3.49
match (noun)
comparison decision stating that the biometric probe(s) and the biometric reference are from the
same source
Note 1 to entry: Historically, the word match has been used as a verb to indicate the act of comparison and
decision making. As ‘match’ is the decision coming out of the comparison process, its use as a verb is deprecated
in favour of compare.
[SOURCE: ISO/IEC 2382-37:2017, 3.3.31]

3.50
methodology
system of principles, procedures and processes applied to IT security evaluations
[SOURCE: ISO/IEC 18045: 2008, 3.9]
3.51
object
passive entity in the TOE, that contains or receives information, and upon which subjects perform
operations
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.45]
3.52
operation
usage phase of the TOE including “normal usage”, administration and maintenance of the TOE after
delivery and preparation
[SOURCE: ISO/IEC 15408-1: 2009, 3.3.2]
Note 1 to entry: This term is related to the AGD class.
3.53
operational environment
environment in which the TOE is operated
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.48]
3.54
operating point
setting of a biometric system to operate at a fixed decision threshold
Note 1 to entry: an operating point can be directly represented by a decision threshold or can be represented by
a predefined configuration parameter.
3.55
potential vulnerability
suspected, but not confirmed, weakness
Note 1 to entry: Suspicion is by virtue of a postulated attack path to violate the SFRs.
[SOURCE: ISO/IEC 15408-1: 2009, 3.5.5]

3.56
presentation attack
presentation to the biometric data capture subsystem with the goal of interfering with the operation of
the biometric system
Note 1 to entry: Presentation attack can be implemented through a number of methods, e.g. artefact, mutilations,
replay, etc.
Note 2 to entry: Presentation attacks may

Licensed to: have a number
Plessis, of goals, e.g. impersonation or not being recognized.
Patrice M.
ISO/IEC DIS 19989-2:2019(E)

Note 3 to entry: Biometric systems may not be able to differentiate between biometric presentation attacks with
the goal of interfering with the systems operation and non-conformant presentations.
[SOURCE: ISO/IEC 30107-1: 2016, 3.5]

3.57
presentation attack detection
PAD
automated determination of a presentation attack
Note 1 to entry: PAD cannot infer the subject’s intent. In fact it may be impossible to derive that difference from
the data capture process or acquired sample.
[SOURCE: ISO/IEC 30107-1: 2016, 3.6]

3.58
presentation attack instrument
PAI
biometric characteristic or object used in a presentation attack
Note 1 to entry: The set of PAI includes artefacts but would also include lifeless biometric characteristics (i.e.
stemming from dead bodies) or altered biometric characteristics (e.g. altered fingerprints) that are used in
an attack.
[SOURCE: ISO/IEC 30107-1: 2016, 3.7]

3.59
Protection Profile
implementation-independent statement of security needs for a TOE type
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.52]
3.60
receiver operating characteristic curve
ROC curve
plot of the rate of false positives (i.e. impostor attempts accepted) on the x-axis against the corresponding
rate of true positives (i.e. genuine attempts accepted) on the y-axis plotted parametrically as a function
of the decision threshold
[SOURCE: ISO/IEC 19795-1:2006, 4.7.2]
3.61
report
include evaluation results and supporting material in the Evaluation Technical Report or an Observation
Report[SOURCE: ISO/IEC 18045: 2008, 3.14]
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.62]
3.62
Security Target
implementation-dependent statement of security needs for a specific identified TOE
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.63]
3.63
target of evaluation
set of software, firmware and/or hardware possibly accompanied by guidance
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.70]

ISO/IEC DIS 19989-2:2019(E)

3.64
threshold (noun)
numerical value (or set of values) at which a decision boundary exists
[SOURCE: ISO/IEC 2382-37:2017, 3.3.36]
3.65
TOE security functionality
combined functionality of all hardware, software, and firmware of a TOE that must be relied upon for
the correct enforcement of the SFRs
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.74]
3.66
verify
confirm a biometric claim through biometric comparisons
Note 1 to entry: It is understood that, in general, biometric claims can neither be proven nor be refuted with
certainty.
[SOURCE: ISO/IEC 15408-1: 2009, 3.1.84]

3.67
vulnerability
weakness in the TOE that can be used to violate the SFRs in some environment
[SOURCE: ISO/IEC 15408-1: 2009, 3.5.5]
4 Symbols (and abbreviated terms)

ATE
Assurance class Tests
AVA
Assurance class Vulnerability Assessment
FAR
False Accept Rate
FRR
False Reject Rate
FMR
False Match Rate
FNIR
False-Negative Identification-error Rate
FNMR
False Non-Match rate
FPIR
False-Positive Identification-error Rate
FTAR
Failure To Acquire Rate
FTER
Failure To Enrol Rate
IT
Information Technology

ISO/IEC DIS 19989-2:2019(E)

PP
Protection Profile
ST
Security Target
TOE
Target Of Evaluation
TSF
TOE Security Functionality
5 Supplementary activities to ISO/IEC 18045 on ATE tests
5.1 General
5.1.1 Guidance
Clause 5 introduces guidance and supplements to evaluation activities from ISO/IEC 19989-1:20xx,
Clause 13.
The organization of Clause 5 is as follows: Subclause 5.1 to 5.4 are applicable to both ATE_IND and
ATE_FUN. Subclause 5.5 introduces the specific aspects resulting from ISO/IEC 19989-1. Subclause 5.6
is specific to ATE_FUN and ATE_IND.2 part related to functional and developer tests. Subclause 5.7 is
specific to independent testing.
Biometric systems employ technology and functionality that require special considerations when
conducting security evaluation, including security evaluation based on the ISO/IEC 15408 series. One of
these is the non-deterministic nature of biometric decisions, i.e. match; non-match; and other decisions,
and the consequent possibility of decision errors (e.g. false match, false non-match) which may have
security implications for biometric systems.
A test of the security relevant biometric recognition error rates is an important aspect of every security
evaluation of a biometric system. Further, the requirements in ISO/IEC 19989-1 ensure that also the
developer of the biometric system under evaluation shall test the error rates of the system under the
policy of the intended use following the TOE guidance document.
NOTE 1 In this document, the intended use following the TOE guidance document covers both genuine and
imposter attempts, as long as the usage is consistent with the guidance.
Clause 5 contains guidance and additional requirements for an evaluator for the evaluation and review
of the developer tests as well as for planning, conducting and reporting independent testing of the error
rates of the biometric system. Clause 5 may also be used by the developer of a biometric system to be
informed about the requirements.
NOTE 2 In this document, the evaluator is considered competent under the framework of evaluations under
ISO/IEC 15408 (all parts) and in particular[7].
The evaluator should, as a default principle, follow the recommendations introduced thereafter (e.g.
on error rates, maximum values, developer testing methodology, etc.). If the evaluator judges they
are not appropriately chosen with respect to the TOE and the application, he or she shall justify other
choices in the evaluation report. The technology specific aspects in Clause 5 have been developed
under consideration of the requirements in ISO/IEC 19795-1. Both scenario and technology tests,
which definitions are reminded in 5.1.3, are concerned in Clause 5; the evaluator shall choose the most
appropriate test case with respect to the TOE.
In addition to the requirements and recommendations provided in Clause 5, the evaluator shall
also follow the requirements for the assurance components selected by the TOE for the ATE class in
ISO/IEC 15408-3 and shall follow the requirements of the corresponding activities in ISO/IEC 18045.
ISO/IEC DIS 19989-2:2019(E)

The configuration of the TOE may have an effect on the biometric recognition performance; hence the
evaluator shall ensure that the TOE configuration for testing complies with the requirements specified
in the ST or PP. In particular when the TOE includes PAD functionality, the evaluator shall check that
the PAD functionality is enabled and correctly configured while conducting the biometric recognition
performance testing. If both biometric recognition performance and PAD are evaluated for the TOE,
BPCER (Bona fide Presentation Classification Error Rate) as defined in ISO/IEC 30107-3 should be
calculated in biometric recognition performance testing, by additionally observing the output of
the PAD subsystem, and written as supplementary information in the document for the ATE_FUN
activity of the PAD evaluation. Similarly IAPMR (Impostor Attack Presentation Match Rate) as defined
in ISO/IEC 30107-3 may be retrieved from the ATE_FUN activity of the PAD evaluation and taken in
account, as it is related to biometric performance.
NOTE 3 The information on IAPMR from PAD evaluation is not used by the evaluator for estimating FMR/
FAR, as those metrics are not directly related. Nevertheless, IAPMR can be a useful information for the evaluator
to understand specific behaviours of the recognition algorithms (and it could also be useful on AVA to identify
potential weaknesses).
NOTE 4 ATE is focused on validating the performance of the TOE by testing under the policy of the intended
use following the TOE guidance document. It therefore encompasses bona fide presentation attempts (as opposed
to presentation attacks considered in ISO/IEC 19989-3) for both mated-comparison trials and non-mated
comparison trials (a.k.a. imposter attempts). In both trials the evaluator can assume use of the TOE following the
intended use. All other kinds of presentations are considered in ISO/IEC 19989-3.
5.1.2 Remarks for performance evaluation
The relationship between the two error rates FAR/FRR can best be illustrated using a Detection Error
Trade-off (DET) curve showing the dependency between the two biometric error rates as the decision
threshold is varied over its working range - see ISO/IEC 19795-1 for more information. DET curves can
be useful to compare the recognition performance of biometric systems and to track improvements in a
biometric system over its development. It is however common that the developer of a biometric system
decides to submit to evaluation the system at only one or a very limited set of decision thresholds.
The developer shall specify the operating point or points of the TOE in the Security Target, in order to
ensure that customer of the TOE is informed about the evaluated configuration.
The evaluator shall ensure that the relevant security settings (including at least the operating points) of
the TOE used during performance testing are set, and that the evaluation is performed, in accordance
with the values stated in the security target.
Further it should be considered that a security evaluation following the ISO/IEC 15408 series is focused
on IT-security. Therefore the security relevant biometric recognition error rates of a biometric system
shall be assessed in the context of an evaluation. However, because security can be achieved at the
expense of usability, usability-related error rates should also be evaluated. Guidance on identifying
relevant error rates is given in 5.1.4.3.
The remaining subclauses of Clause 5 provide more detailed information for the evaluator regarding
the review of developer tests, repeating a test subset as outlined in ATE_IND.2 and regarding the
independent test as required by ATE_IND.1.
5.1.3 Identification of the type of performance evaluation
According to ISO/IEC 19795-1, three basic types of evaluation for the performance rates of a biometric
system can be distinguished:
— Technology evaluation: off-line evaluation of one or more algorithms for the same biometric modality
using a pre-existing or specially collected corpus of samples.
— Scenario evaluation: evaluation in which the end-to-end system performance is determined in a
prototype or simulated application using live biometric presentations made by a test crew recruited
for the test.
ISO/IEC DIS 19989-2:2019(E)

— Operational evaluation: evaluation in which the performance of a complete biometric system is

determined in its operational environment with a specific target population.
The first step for the evaluator is to identify the correct type of the evaluation for the biometric system
under evaluation. The type of evaluation shall be determined by the composition of the TOE and what is
specified in the security target. The composition of the TOE shall be capable of supporting the specified
type(s) of evaluation.
NOTE As evaluations usually refer to an instance of a biometric product rather than to a concrete instance
of an installation of a biometric system, the operational test of the security relevant biometric recognition error
rates is often not considered. Consequently, most of the content in Clause 5 refers specifically only to technology
and scenario cases.
The type of evaluation to be performed depends on the definition of the TOE and the scope in the
security target. The evaluator shall verify that the developer testing is appropriate to the type of
evaluation.
ISO/IEC 19795-1 further distinguishes between online and offline tests. In online tests the enrolment
or comparison process is executed at the time of image or signal submission while those phases of
testing are kept separately in offline tests. Technology evaluations are carried out using offline
processing of biometric data. Due to requirements regarding the repeatability and reproducibility
that apply to evaluations, pure online tests (in which the images or signals will directly be discarded)
should not be used.
5.1.4 Biometric recognition error rates
5.1.4.1 Metrics for biometric verification
The class ATE refers in this document to the tests that shall be performed to assess the performance of
the biometric system under the policy of the intended use following the TOE guidance document. In the
case of a verification biometric system the intended use may be defined as follows: ``a data subject tries
to be recognized by the system as a legitimate enrolled data subject related to a claimed identity''.
In this scenario the system may anticipate two cases that shall be distinguished: biometric mated
comparison trial (a.k.a. as genuine), and non-mated comparison trial (a.k.a as impostor). According to
these two cases the following decision error rates shall be reported:
- For an algorithm evaluation the FMR and FNMR;
- For a system evaluation FAR and FRR.
The difference between algorithm error rates (FMR and FNMR) and the system error rates (FAR and
FRR) is that the latter depends on the permitted number of verification attempts and may also include
other type of errors such as the Failure to Acquire and Failure to Enrol.
The FAR (respectively FMR) and FRR (respectively FNMR) error rates of a biometric system are
inversely related, the trade-off between the two being determined by the verification decision threshold
setting for the system.
Note that other error rate testing that includes the sample acquisition stage may also result in
differences in the results for single attempts and transactions, for example: Failure To Enrol Rate, FTER,
and Failure To Acquire Rate, FTAR.
5.1.4.2 Metrics for biometric identification
In an identification scenario a subject provides a biometric sample without making an explicit claim
of identity. The biometric system identifies the subject by biometric comparison of the biometric
identification sample with the biometric references of all enrolled subjects until a match is found (or not)
based on identification decision criteria defined for the system. This is known as 1:many comparison.
Depending on the criteria, zero or more matches may be found and reported by the system. When more
ISO/IEC DIS 19989-2:2019(E)

than one match is reported, the matching identities may be ranked according to the corresponding
comparison scores.
In the case of an identification biometric system the intended usemay be defined as follows:
— If positive identification scenario is considered, "a data subject tries to be identified by the system
as a legitimate enrolled data subject";
— If negative identification scenario is considered, "a data subject tries not to be identified by the
system as a legitimate enrolled data subject".
As with the verification scenario, the class ATE activity refers in this document to the tests that shall be
performed to assess the performance of the biometric identification system (TOE) under its intended
use. Performance testing shall include:
— If positive identification scenario is considered, performance testing for the case of bona fide
presentations, i.e. where members of a test crew (or test data) comprising legitimate enrolled data
subjects attempt to be identified by the system as themselves, and performance testing for the case
of impostor presentations where members of a test crew (or test data) who are not enrolled in the
system attempt to be falsely identified by the system as legitimate enrolees using presentations of
their natural biometric characteristics.
— If negative identification scenario is considered, performance testing for the case of non-enrollees-
related presentations where members of a test crew (or test data) who are not enrolled in the
system attempt not to be identified by the system as enrollees, and performance testing for the
case of enrollees-related presentations, i.e. where members of a test crew (or test data) comprising
enrolled data subjects attempt to be falsely not identified by the system, using presentations of their
natural biometric characteristics.
Contingent on the system functions, the main metrics that should be assessed are:
— If positive identification scenario is considered true-positive identification rate, false-positive
identification-error rate (FPIR), and identification rank.
— If negative identification scenario is considered, true-negative identification rate, false-negative
identification error rate (FNIR).
NOTE For any scenario, further biometric system error rates exist that may become relevant for specific
scenarios. ISO/IEC 19795-1 gives a complete overview over all possibly relevant biometric recognition error rates.
From these types of error rates the evaluator shall decide which are relevant for a particular evaluation
(cf. 5.1.4.3).
5.1.4.3 Identification of relevant error rates
There is no comprehensive and single answer to the question of which error rates are relevant for a
particular biometric system under a specific evaluation. The evaluator should consider a number of
aspects when determining relevant error rates as described below.
The following paragraphs provide an overview of the most important aspects to be taken into account
in order to answer this question. The evaluator shall also take in account the claimed error rates
(following 7.3, 7.4 or 7.5 in ISO/IEC 19989-1:20xx).
The primary error rates of interest are those that are security relevant. The error rates that are
security relevant depend on the purpose of the biometric recognition. For verification, the primary
security metric for scenario testing is FAR (FMR for technology testing). For positive identification, the
primary security metric for scenario testing is FPIR (FMR for technology testing). Other parameters
may influence the security relevant biometric recognition error rates, like for instance the use of a
retry counter.

ISO/IEC DIS 19989-2:2019(E)

If both biometric recognition performance and PAD are evaluated for the TOE, BPCER is an additional
relevant error rate. It relates to the performance and usability of a system as it is a measure of the rate at
which a PAD subsystem determines that a bona fide presentation is a presentation attack when it is not.
In addition to the requirements coming from ISO/IEC 19989-3, the evaluator should observe the output
of the PAD subsystem during the performance testing to measure the false classifications of bona fide
presentation as presentation attack. Similarly the evaluator may consider IAPMR as a relevant metric,
if provided from the ATE_FUN activity of the PAD evaluation, as it relates to simultaneously succeed to
pass PAD and biometric recognition.
During an evaluation, the security relevant biometric recognition error rates shall be assessed in depth.
The other related error rates should be assessed.
In order to identify all relevant error rates the evaluator shall consider all error rates that are defined
in ISO/IEC 19795-1, and answer two questions for each rate:
— Is the error rate security relevant for the TOE?
— What is their relevance to the application?
Only if both questions have been positively answered, the error rate should be taken into account for
the evaluation.
It should be noted that some of the error rates of biometric systems depend on other error rates via the
setting for the decision threshold of the system. An example is the inverse relationship between False
Accept Rate and False Reject Rate. Thus, all error rates that are correlated to the security relevant
biometric recognition error rates shall also be reported. This requirement aims to ensure that usability
of the TOE can also be estimated from the evaluation report.
The evaluator shall check that the result of this analysis is consistent with the information that the
developer provided in the Security Target.
The following table introduces a synthesis of main errors and their impact on security and other
functionality of the TOE, in the case of scenario testing.
Table 1 — Effect of Biometric Errors on Application Functions

Application Error Application Function and Impact
Metric Enrolment Verification Identification
FTER - Inability to enrol
Provision and se- Provision of ex-
data subjects curity of exception ception handling
handling proce- procedures
- Provision of
dures for data sub-
exception handling
jects who cannot be
procedures
enrolled
FAR N/A Impostor may be N/A
wrongly verified
as legitimate data
subject
BPCER* Inability to enrol Legitimate data Legitimate data
*if PAD functionality data subjects subject may fail to subject may fail to
pass the PAD check pass the PAD check
and thus to be and thus be wrong-
verified ly not identified in
candidate list
FRR N/A Legitimate data N/A
subject may fail to
be verified

ISO/IEC DIS 19989-2:2019(E)

Table 1 (continued)
Application Error Application Function and Impact
Metric Enrolment Verification Identification
FPIR N/A N/A Impostor may be
wrongly identified
as a legitimate data
subject candidate
FNIR N/A N/A Legitimate data
subject may be
wrongly not identi-
fied in candidate list
An example of identifying relevant error rates for a specific use case is discussed in Annex B.
5.1.4.4 Determining maximum values for error rates
There is no absolute and simple answer to the question of what is the maximum permissible value
of an error rate as it is influenced by many factors. Subclause 5.1.4.4 provides guidance to the
evaluator for determining which values are consistent with the TOE. See also Annex B for a typical
example. The evaluator shall also take in account the claimed error rates (following 7.3, 7.4 or 7.5 in
ISO/IEC 19989-1:20xx). In order to ensure consistency with the application context, Table 2 taken from
ISO/IEC 29115:2013 may be used by the evaluator to identify the required level of assurance on the
basis of an assessment of the severity of the possible impact of authentication errors. Determination of
what constitutes minimum, moderate, substantial, and high risk depends on the risk criteria for each of
the possible consequences.
Table 2 — Potential impact of authentication errors at each level of assurance

Possible impact Level of assurance
Low medium high very high
Inconvenience, distress, or damage to standing or minimum moderate substantial high
reputation
Financial loss or agency liability minimum moderate substantial high
Harm to the organization, its programs or public N/A minimum moderate high
interest
Unauthorized release of sensitive information N/A moderate substantial high
Personal safety N/A N/A minimum to substantial to
moderate high
Civil or criminal violations N/A minimum substantial high
In addition, for the case of a biometric verification system used as the sole authentication factor,
[ISO/IEC TR 29156:2015, 6.4] may also be used as a guidance by the evalualor to decide the potential
target for the security relevant error rates (e.g. FAR, FMR). ISO/IEC TR 29156 suggests to select the
following FAR values, that can be used by the evaluator in case of scenario testing for setting the FAR or
for deriving other error rates:
— a FAR of less than 0,000 1 % for high assurance;
— a FAR of less than 0,01 % for medium assurance;
— a FAR of less than 1 % for basic assurance.
If the biometric verification system is used in conjunction with other (e.g. knowledge-based)
authentication factors, higher FAR values may be acceptable.
If no specific level has been defined, the evaluator shall at least follow existing guidelines, such as
FRONTEX guidelines.[5] ForLicensed
instance, the evaluator should evaluate the system at a threshold that
to: Plessis, Patrice M.
ISO/IEC DIS 19989-2:2019(E)

corresponds to a security level in terms of the False Accept Rate (FAR) of at least 0,001 (0,1 per cent). At
this configuration the FRR should not be higher than 0,05 (5 per cent).
If the ST claims a conformance to the PP that defines the maximum values for error rates, the TOE’s
error rates shall be below the maximum.
An example of determining maximum values for a specific use case in discussed in Annex B.
5.2 Planning the evaluation
5.2.1 Overview
Planning for biometric recognition performance testing should include two important considerations.
Firstly to ensure that the test design models as closely as possible the real world scenario of the intended
use of the TOE. Secondly to ensure that the test results are statistically significant in the context of the
intended use of the TOE.
Such statistical significance is basically defined by the test data on which tests are run and on the actual
access attempts that are performed with the test data available. Results should be reported according
to a mean error rate value and a confidence interval.
Subclause 5.2 encompasses the recommendations for evaluator to plan and execute testing. If developer
testing is considered adequate and valid by the evaluator, the evalutor may limit himself to simpler and
reduced testing activities; this choice shall be justified into the evaluation report.
A comprehensive test plan shall be devised and documented. The test plan shall satisfy the following
objectives:
— The test setup shall match the intended operation of the biometric system as closely as possible.
— The test plan shall exactly identify the relevant steps to be taken during testing. Specifically, when
a deep interaction with a test crew is required (e.g. in a scenario test) the test plan shall clearly
describe the flow of the test.
— The test plan shall include a very detailed description of the test data that the evaluation will be
performed with, this includes for instance: number of data subjects, number of samples per data
subject, number of acquisition sessions involved, environment and external conditions of the
acquisition (e.g., background, illumination, pose).
— The test plan shall include a very clear protocol on how the test data is used or how test data subjects
should interact with the TOE in the capture process.
— The test plan shall include whether collected data is needed for re-training of the system. If that is
the case, then the collected data should be separated in training and test data and the purpose of
each dataset shall be described
— The test dataset shall contain several biometric samples per data subject / biometric instance (e.g.
from each finger) and samples from different data subjects to generate both mated comparison
trials and non-mated comparison trials
— The test plan shall specify a pre-verification phase, including capture (if applicable) and feature
extraction, to establish a list of samples to be compared together with the information whether the
comparison pair corresponds to a genuine test or an impostor test. For each, the evaluator shall
plan to measure the time for the suboperations, the number of errors to acquire (if applicable),
and number of errors to extract the samples. The list of samples to be compared shall include both
genuine comparison pairs and impostor comparison pairs, and the number of pairs shall be set
accordingly to the range of error rates (e.g. FAR and FRR for scenario testing of a verification system)
values to be checked by the evaluator.
— The test plan shall specify a comparison phase, to compute all comparisons from the list prepared in
pre-verification phase. For
Licensed to: each comparison,
Plessis, Patrice M. the evaluation plan shall include measure of the time
ISO/IEC DIS 19989-2:2019(E)

for the operation, the number of errors to compare, store the output score when possible, or the
decision. Based on the output score and the threshold specified by the TOE (if available), or based
on the decision, the number of false accepts and the number of false rejects shall be measured.
— The test plan shall define the metrics to be reported. In addition to the values measured during
the operations, the evaluator shall identify the test metrics to be reported and how to derive them
from the different errors observed by the evaluator. For scenario testing of a biometric verification
system, they shall include FTA (if applicable), FTE, FAR and FRR. If comparison scores are available,
they may include computation of DET curve in order to embed a DET curve into the report.
— The test plan shall also include a complete specification of the computed comparison scores and
how the performance metrics are deducted in order to support statistical analyis and to ensure full
reproductibility.
— The test plan shall include the report preparation, that shall report computed values for the different
metrics identified and a synthesis of the results.
Testing shall be made in accordance with ISO/IEC 19795-2. Thus the test plan shall be defined
accordingly.
This test plan allows, in the case of a biometric verification system, the test of the FMR (respectively the
FAR) and the FNMR (respectively the FRR) for the biometric system under test. Similarly, for a biometric
identification system, it allows to test the FMR (respectively FPIR) and the FNMR (respectively FNIR).
5.2.2 Estimation of test sizes
Acquiring and handling the test data is one of the most challenging and most expensive tasks in each
test of a biometric system. According to ISO/IEC 19795-1, the test data shall be as large as practically
possible. Note that, in order to obtain statistically reliable results, there is a minimum size of the test
data. This depends on various factors:
— The size of the error rate to be measured (the smaller the error rate the larger the necessary test
data numbers).
— The required confidence interval.
NOTE The size of the test data set is not the only factor impacting the level of confidence on the error rate
to be measured. The dependencies between the different comparison pairs also impact the level of confidence.
Full cross comparison is possible, but then no more full independencies is granted and the impact on level of
confidence would be taken into account.
As the required size of the test data also depends on the results of the test themselves the required test
size is usually only estimated roughly in the context of test planning.
The purpose of independent testing at ATE_IND.2 is to seek to validate, even if with limited confidence,
the developer claimed error rates. This may be done with a test size much smaller than the size of
developer’s test data.
The number of test data subjects and test transactions needed to ensure statistical significance of a
measured security relevant biometric recognition error rate should be determined from the permitted
maximum value of the error rate using either the “Rule of 3” or the "Rule of 30" [ISO/IEC 19795-1].
For biometrics on mobile devices, practical guidance to conduct the performance testing is provided in
ISO/IEC 21879. The evaluator should refer such document to estimate the appropriate test sizes.
5.2.3 Test documentation
It is essential to plan the complete documentation for the test before starting any other activities.
As introduced in 5.1.1, the evaluator should, as a default principle, follow the recommendations
introduced thereafter (e.g. on error rates, maximum values, developer testing methodology, etc.). The
ISO/IEC DIS 19989-2:2019(E)

evaluator may decide from a consideration of the developer test plan and the conduct of developer
testing that the developer testing is adequate for the purpose of the evaluation and the evaluator does
not need to repeat entirely the developer testing. Otherwise, the evaluator should follow his own test
plan and the documentation shall follow the same principles as for any other test in the context of
security evaluation following the ISO/IEC 15408 series and ISO/IEC 18045. In addition, the following
aspects shall be addressed specifically:
— The exact type of performance evaluation should be carefully described.
— The relevant error rates shall be identified and their maximum acceptable values shall be identified
and justified.
— Special characteristics of the test data shall be documented.
— The size and characteristics of the test data should be described with special attention to number of
sessions involved in its acquisition, data subjects and samples per data subject.
— It should be clearly described how are the different parameters of the system trained/set (if there
are any).
— It should be clearly described how the different score sets (mated and non-mated comparison trials)
are computed.
5.3 Data collection
5.3.1 Choice of test data or acquiring test crew and capture device
Subclause 5.3.1 applies to both scenario and technology testing. An important consideration is the
source of the biometric sample data and the associated test crew. This may have been acquired in
various ways, e.g.:
1) live capture: the test data are captured from a test crew specifically for the TOE evaluation (it could
be required by specific setting of the TOE, e.g., some particular illumination or background setup in
the case of a face recognition system);
2) re-use of pre-existing database, either captured by the developer and/or the evaluator on a previous
similar biometric systems, or obtained from acquired data from third parties (such as the multiple
public or private biometric databases available today for benchmarking purpose).
In both case the ground truth shall be known i.e. which samples in the acquired data are true matches
and which are not.
The most desirable case would be option 1), in which a different database is acquired for each
evaluation. However, this is also the most time and resource consuming solution and a final decision
should be adopted on a case by case basis. For instance, for a technology or a scenario evaluation the
second case could be sufficient if no specific contextual or external features have to be met (e.g., specific
acquisition sensor). If the evaluator choses to reuse an already existing data, he or she should ensure
that this dataset is sequestered and was not accessible by the developer. The main disadvantage of
reusing already existing data, that the developer may had access to, is that it could be used to tune their
system. The evaluator shall mitigate it by using more sequestered data.
For the acquisition of a new database or test crew, several important factors shall be taken into account
in order to obtain results as precise as possible. Among such factors some ideal characteristics that
should be met by a biometric evaluation database are highlighted below:
— The choice of the test data (live capture process or re-use of pre-existing data) shall be under the
sole control of the evaluator. As the quality of the test data is essential for the results of the tests, it
is important that the evaluator has a detailed knowledge of the acquisition process.
— A question that may often arise is whether test data that has been acquired by the developer
beforehand can be re-used
Licensed duringPatrice
to: Plessis, an independent
M. evaluation. While the final decision on the re-
ISO/IEC DIS 19989-2:2019(E)

use of test data is the decision of the evaluator, this guide encourages the re-use of test data within
certain limits. Specifically a test shall never be based completely on test data that has been acquired
by the developer beforehand. Instead, the evaluator shall acquire a small subset of test data and
replace it in the original set of test data before using it.
— If the biometric system is designed to work with a specific data subject profile (e.g., men, Asian, over
65 years of age, right-handed) the data subjects in the database/crew should be as close as possible
to that profile.
— If the biometric system is not designed to work with one specific sensor, it is better to capture the
same individuals with different acquisition devices so that the final evaluation is more general and
also to be able to obtain interoperability results (i.e., comparison results between enrolled and test
biometric samples captured with different devices).
— The data set should be organized to enable an explicit identification of each sample. However, to
ensure personal data protection, the identifier of a sample shall not contain any information related
to the real identity of the data subject
— A sufficiently large number of individuals should be enrolled in the database in order to obtain
statistically significant results. Such number will depend on the maximum error rates allowed for
the system, the lower the error rates the larger the number of required data subjects in order to
achieve reliable results.
— Also, the different samples of the same data subject should not be captured consecutively but leaving
enough time between them in order to simulate the intra-subject variability of the biometric traits.
Ideally, the database should be acquired in different sessions separated several weeks among them.
— If relevant, other metadata related to the data subjects could be also acquired. This may include
for instance the gender, the age, the use of visual aids (e.g., glasses), or the handedness. These
metadata can help to further tune the performance evaluation or to reuse the biometric data in
future evaluations.
— Biometric data acquisition is prone to various errors such as: missing samples, invalid samples, low
quality samples and ground truth errors where samples are incorrectly assigned to subject IDs.
Some of these errors can be the result of human fallibility and may be reduced by the application of
automated data acquisition and recording techniques.
— Biometric data is Personally Identifiable Information (PII). As such, national privacy legislations
shall be considered and followed during the acquisition process. During the acquisition the evaluator
should comply with, at least, the data protection laws of his working country. This usually includes
informing the acquired data subjects of the use that will be given to their data and obtaining from
them a signed consent form for the acquisition of those data.
5.3.2 Performing test
Subclause 5.2.2, 5.2.3 and 5.3.1 defined all the initial steps that should be performed and documented
in the test plan prior to the evaluation, that is: relevant error rates and their maximum values, type
of evaluation, database and evaluation protocol associated to it, acquisition of data. Once all those
steps have been covered, the evaluation should be run according to the predesigned plan. During the
evaluation, several aspects shall be documented such as:
— Any significant deviations from the original test plan.
— Time required to perform each experiment considered in the evaluation. Other temporal information,
that may be recorded depending on the evaluation, is the system response time for every access
attempt.
Once the tests are performed the results should be reported using standard metrics (cf. 5.1.4.3 and
ISO/IEC 19795-1).

ISO/IEC DIS 19989-2:2019(E)

5.4 Analyses
The analysis of the results should not be restricted to a mere certification of the error rates but should
also include a more in-depth investigation of the error instances and whether those errors disclose some
security problem of the system, for instance a significant variability in performance for certain data
subject profiles (e.g. men vs women). This type of analysis can help to identify potential problematic
working scenarios for the system.
It should be noted that - strictly speaking - such an analysis would rather belong into the area of the
AVA class than the ATE class as it would open the path to a potential vulnerability of the TOE.
These possible deviations from the average expected performance of the system should be reported in
the final documentation so that it is clear under which circumstances the system behaves as expected
(under the maximum allowed error rates) and in which scenarios the error rates may increase.
NOTE A test of the performance of a biometric system in the course of a security evaluation focuses on
security relevant biometric recognition error rates such as FAR. Other error rates e.g. FRR are relevant to other
factors such as usability. Error rates that are security relevant (such as the FAR) may with a dependant error
rates (in this case the FRR) via the threshold of the system.
The dependant error rates shall be evaluated and reported.
5.5 Specific requirements on assurance components
5.5.1 Overview
As described in 19989-1, the following elements of assurance components require the evaluator to
conduct their own test of the TOE:
— ATE_IND.1.2E: The evaluator shall test a subset of the TSF as appropriate to confirm that the TOE
operates as specified.
— ATE_IND.2.3E: The evaluator shall execute a sample of tests and document them to verify the
developer test results.
All evaluations of biometric TOEs shall include the assurance component ATE_IND.2 as supplemented
by ISO/IEC 19989-1.
Subclause 5.2.2 and 5.2.3 contain dedicated guidance regarding the evaluation of those two components.
5.5.2 Specific requirements on ATE_IND.1
The evaluator will derive a subset of the TSF to be tested independently in accordance with the guidance
in ISO/IEC 18045. As the biometric performance is an essential part of the TOE, the evaluator shall in
any case ensure that this part of the TSF falls into the subset.
Table 3 — Specific requirements on ATE_IND.1 from ISO/IEC 19989-1:20xx

ATE_IND.1-3 The evaluator shall also devise independent testing for performance eval-
uation setting up a test crew or a test dataset.
ATE_IND.1-4 The evaluator shall produce test documentation for performance evalua-
tion which satisfies the relevant requirements from ISO/IEC 19795.
The evaluator shall explain any deviation from the test procedures spec-
ified in ISO/IEC 19795 and also shall describe any potential effects and
implications for the test results in the test documentation.
ATE_IND.1-5 The evaluator shall conduct testing using test crew which the evaluator
arranged or test data which the evaluator possesses.
ATE_IND.1-6 The evaluator shall record information of test crew or test data as speci-
fied in ISO/IEC 19795.
ISO/IEC DIS 19989-2:2019(E)

Table 3 (continued)
ATE_IND.1-8 The evaluator shall also report in the ETR the evaluator testing effort on
biometric recognition performance in terms of test size, time spent, and
also dataset characteristics.
The conducted tests shall follow the requirements and recommendations listed in 5.2, 5.3 and 5.4.
5.5.3 Specific requirements on ATE_IND.2
Biometric testing is complex, time consuming and expensive. Specifically the acquisition of sufficient
test data is a challenge for every test. It is therefore an essential question whether the test data that
the developer used can be completely re-used when repeating the test from the developer’s test
documentation in the context of ATE_IND.2. Guidance on this subject is provided in 5.7.
Table 4 — Specific requirements on ATE_IND.2 from ISO/IEC 19989-1:20xx

ATE_IND.2-6 The evaluator shall also devise independent testing for performance eval-
uation setting up a test crew or a test dataset.
ATE_IND.2-7 The evaluator shall produce test documentation for performance evalua-
tion which satisfies the relevant requirements from ISO/IEC 19795.
The evaluator shall explain any deviation from the test procedures spec-
ified in ISO/IEC 19795 and also shall describe any potential effects and
implications for the test results in the test documentation.
ATE_IND.2-8 The evaluator shall conduct testing using test crew which the evaluator
arranged or test data which the evaluator possesses.
ATE_IND.2-9 The evaluator shall record information of test crew or test data as speci-
fied in ISO/IEC 19795.
ATE_IND.2-11 The evaluator shall also report in the ETR the evaluator testing effort on
biometric recognition performance in terms of test size, time spent, and
also dataset characteristics.
The conducted tests shall follow the requirements and recommendations listed in 5.2, 5.3 and 5.4.
5.6 Reviewing and assessing developer tests
5.6.1 Review of developer tests
In the course of the evaluation activities around ATE_FUN.1 the evaluator shall evaluate the test
documentation and results that are provided by the developer.
The developer should follow the requirements from ISO/IEC 19795-1 and ISO/IEC 19795-2 for their
performance test. Any deviation from the requirements of the standard shall be justified in the test plan.
The full set of information about the test shall be handed over to the evaluator during the evaluation.
Only this way it can be ensured that the evaluator gets a complete overview over all details of the test.
In this context, the developer shall provide the evaluator with complete access to the test equipment
and test data used for developer testing.
The evaluator shall assess the developer test documentation to confirm:
— The developer test plan, the conduct of the developer testing and the test documentation conform to
the requirements stated in ISO/IEC 19795-1 (see Annex C for an example);
— Any deviations from the above requirements are justified;
— The test results show that the security relevant biometric recognition error rates are in accordance
with the claims in the TOE security target.
ISO/IEC DIS 19989-2:2019(E)

5.6.2 Repeating a test subset
The requirements of ATE_IND.2 mandate the evaluator to repeat a subset of the developer tests. Tests
of the security relevant biometric recognition error rates shall form part of the subset to be repeated.
When repeating a test of the security relevant biometric recognition error rates of the biometric system
it often comes to the question whether it is sufficient to simply repeat the test of the developer. This is of
specific relevance if the developer managed to separate the test data acquisition from the actual test of
the biometric algorithm. In those cases the evaluator could decide to simply repeat the test.
The possibilities for the evaluator also depend on the type of biometric test followed by the developer:
— In the case of a technology test, the evaluator has a potential choice of using the same biometric data
used by the developer or obtaining a new source of biometric data.
— In the case of a scenario test, the evaluator might be able to use the same test crew as the developer;
otherwise a new test crew would have to be recruited. In case the evaluator judges that the repetition
of a subset of developer tests can be restricted to a technology test, he or she may reuse biometric
data obtained from the developer test crew (if available), or he or she may use a new source of
biometric data.
When possible, the evaluator should take benefit of the efforts the developer spent for test data
acquisition by re-using this data when executing testing. On this basis, when possible the evaluator
is encouraged to follow this strategy: Re-use the test data of the developer when executing a subset
of the developer tests. In order to avoid a pure repetition of the test using exactly the same data the
evaluator shall consider to replace a subset of the test data by their own data (that should be acquired
by the evaluator following 5.3.1). It falls into the responsibility of the evaluator to decide about the size
of this subset. They shall consider the overall quality of the test data of the developer and the quality of
the acquisition process (based on its documentation). Instead of replacing a subset, the evaluators can
supplement the developer data by their own data. Technically it is essential that the exchanged subset
of the data or the supplemental dataset is large enough to ensure that the developer could not tune their
algorithm based on the database.
NOTE Biometric data is Personally Identifiable Information (PII). As such, national privacy legislations will
have to be considered and followed during the testing process, even while repeating developer’s test with some
existing test data.
5.7 Conducting independent testing
5.7.1 Overview
ATE_IND.1 (as well as ATE_IND.2 and ATE_IND.3) requires the evaluator to conduct their own test of
the security relevant biometric recognition error rates of the TOE. Subclause 5.7 provides the evaluator
with the corresponding guidance. Figure 1 summarizes the different steps that shall be performed by
an evaluator when planning, conducting and reporting independent testing of the security relevant
biometric recognition error rates of a biometric system.
As introduced in 5.1.1, the evaluator should, as a default principle, follow the recommendations
introduced in Clause 5 (e.g. on error rates, maximum values, developer testing methodology, etc.). If the
evaluator judges they are not appropriately chosen with respect to the TOE and the application, he or
she shall justify other choices in the evaluation report.

ISO/IEC DIS 19989-2:2019(E)

Figure 1 — Processes for independent testing
The process can be divided into the following steps that will be later described in more detail:
— Identify the relevant type of performance evaluation: Various kinds of test approaches are available
starting from a technology performance test of a biometric algorithm to an evaluation of the
operational performance test of the biometric system. The correct test approach highly depends on
the functionality provided by the TOE.
— Identify the relevant error rates: As a security evaluation focuses on the security relevant
biometric recognition error rates only, not all error rates of the biometric system are relevant. The
identification of the relevant error rates is performed based on the type of the biometric system and
its application case as defined in the Security Target.
— Plan test execution: The actual test execution shall be planned in advance.
— Estimate test size: Collecting test data takes a significant amount of the effort of the overall test. It is
essential to develop an idea about the amount of test data that is required before starting the actual
process of test data acquisition.
— Plan documentation: It is essential to plan the required documentation for the test in advance of the
test itself.
— Acquire test data: Test data comprises suitable biometric samples which shall be obtained from
live presentations by members of a test crew or using a pre-existing corpus of biometric samples,
depending on the type of test (scenario or technology) and the test methodology. In both cases the
test data shall include so called ground truth, that is knowledge of the source of each sample (e.g.
the subject ID) such that with the corpus both mated-comparison trials and non-mated comparison
trials can be executed. To ensure the quality of the test results the evaluator shall utilize test data
that is not known to the developer of the TOE.
— Perform Test: The test shall be carried
Licensed out Patrice
to: Plessis, under M.
the sole control and responsibility of the evaluator.
ISO/IEC DIS 19989-2:2019(E)

— Evaluate Results: Test results shall be evaluated and reported using standard metrics.
The evaluator should follow the requirements from ISO/IEC 19795-1 for their performance tests. Any
deviation from the requirements of the standard shall be justified in the test plan and documented in
the test report.
In the case of technology testing using a pre-existing corpus of biometric samples, the evaluator
shall provide justification that the corpus satisfies the needs of the test and is independent from the
developer test data.
5.7.2 Identification of the type of performance evaluation
The type of evaluation to be performed mainly depends on the definition of the TOE and type of the
developer’s performance evaluation. The evaluator should verify that the developer’s type of evaluation
is appropriate, if this is the case, the evaluator should follow the same; if not appropriate, the evaluator
is free to conduct the independent test following another type. If the same type of evaluation is not
followed by evaluator, then the evaluator should justify the motivation in the testing report. If the TOE
is made of software functions only, this is likely that the developer will decide to perform a technology
evaluation and evaluator should, if appropriate, perform the same technology evaluation. If the TOE
is a complete biometric system, in particular with a sensor, it will more likely be tested in a scenario
evaluation by the developer and thus evaluator should perform a scenario evaluation. Note that if a
TOE does not provide the comparison and decision functions, its biometric recognition performance
cannot be tested. It should also be noted that scenario testing may include offline testing of the
software algorithms for the purpose of cross-comparison testing for the measurement of performance
achievement rates and error rates and for the determination of DET plots.
6 Supplementary activities to ISO/IEC 18045 on Vulnerability Assessment (AVA)
6.1 General aspects

Clause 6 introduces guidance and supplements to evaluation activities from ISO/IEC 19989-1:20xx,
Clause 14 in regard to vulnerabilities which result from biometric recognition errors that may occur
when presenting non-mated natural biometric characteristics or when injecting or altering non-
mated biometric characteristics after the data capture subsystem. Vulnerabilities associated with
presentation attack detection errors for biometric presentation attacks using PAIs are addressed in
ISO/IEC 19989-3.
The purpose of clause 6 is to introduce the context and provide general guidance to evaluators. Each
vulnerability being a specific case, the examples in Annex A are there to provide some concrete guidance
to the evaluator. However, the evaluator shall devise his/her own strategy based on his/her expertise
and the TOE instance under evaluation.
In addition to the requirements and recommendations provided in Clause 6, the evaluator shall
also follow the requirements for the assurance components selected by the TOE for the AVA class in
ISO/IEC 15408-3 and shall follow the requirements of the corresponding activities in ISO/IEC 18045.
The objective of the evaluation task is to search for vulnerabilities and to demonstrate that the TOE is
resistant to a certain and predefined attack potential.
NOTE 1 If the TOE includes PAD functionality, the evaluator assesses the vulnerabilities in the context of the
whole system to take in account the potential interactions between the subcomponents.
NOTE 2 Referring to the threats a) - j) listed in [ISO/IEC 19989-1:20xx, 5.1], the threats and evaluation
activities addressed in ISO/IEC 19989-2 for the vulnerability assessment of biometric verification or identification
systems mainly relates to the potential exploitation of “a) Performance limitations”, “h) Hostile environment”, “i)
Procedural vulnerabilities around the enrolment process”, for attacking the system, together wih the possible
impacts of “j) Leakage and alteration of biometric data”, “e) Similarity due to blood relationship”, “f) Special
biometric characteristics”, and “g) Synthesised wolf biometric samples” that can help the evaluator to identify
specific weaknesses and to mount more efficient attack.
ISO/IEC DIS 19989-2:2019(E)

The approach taken here is a “counter example” demonstration: for the evaluation of a TOE at a targeted
level of security resistance, i.e. at a given targeted minimum value of attack potential for vulnerabilities
found on the TOE, if one attack with a potential strictly lower than the targeted level is found, then it
is demonstrated that the TOE is not resistant to this level. So, the evaluation will focus on finding one
attack, applicable to the TOE in its usage context and with a rating lower than the targeted level.
The definition of a successful attack shall be done regarding the security definitions of the ST (Security
Target) and should generate a failure in the security objectives of the TOE (access to forbidden data,
unauthorized operation, etc.).
The knowledge used by the evaluator to define an attack and an attack path is all the available
knowledge, including a specific background in the area (to be accredited a laboratory has to
demonstrate its competence in the area), publicly available knowledge (web, dedicated conferences,
etc.) but also its knowledge of the TOE gained from other evaluation tasks (for example knowledge of
the implementation if targeting AVA_VAN.3 or higher).
The objective of an evaluation test is to determine whether the TOE is vulnerable to a specific attack
at the targeted attack potential. Preparatory to testing the evaluator should perform a preliminary
attack potential rating based on the attack specifications (what the attack is, how the attack testing will
be conducted and how the results will be interpreted). The preliminary attack rating may be used to
prioritise the testing (the lower the rating, the higher the priority). The evaluator may test attack paths
for which the preliminary rating is above the target attack potential.
Once the tests list is established, the evaluator executes the corresponding attacks and, in case of success
performs a final rating. This rating is then used to validate or refute the resistance level of the TOE.
NOTE 3 The developer often expects some more information from the evaluation: what are all the weaknesses
of my product? What are all the attacks to counter in a certified product? This information is an added value for
the evaluation report. However, it is not strictly required by the security evaluation under the ISO/IEC 15408
series and ISO/IEC 18045 where a single successful attack can stop the evaluation process, and there is no
guarantee that, in case of successful attack, all the possible attacks have been tested and that the TOE is resistant
to all the other possible attacks.
6.2 TOE for testing

The developer shall provide the TOE for testing and the TOE shall be suitable for testing.
The exact definition of the TOE delivered shall be done by reference to the guides: any option,
configuration, parametrisation referenced in the subjects or administration guide should be available
to the evaluator. In particular, a biometric system should allow the evaluator to enrol specific people.
In addition, when extra equipment is required to use the TOE, the developer shall make it available to
the evaluator (for example, if the TOE is defined as a biometric capture device/sensor, hardware and/or
software for connection to a computer and acquisition, processing and exploitation of data).
In some cases, emulators or simulators exist and are used by the developer to validate part of the TOE
(for example the comparison process and its validation over a large database of images). These should
be made available to the evaluator.
6.3 Potential vulnerabilities

The vulnerabilities that an evaluator should analyse shall at least take in account those introduced in
[ISO/IEC 19989-1:20xx, 5.1].
Additionaly the evaluator shall consider the combination of those vulnerabilities with other IT-related
vulnerabilities. For instance, when considering the possibility to mount a hill-climbing attack, the

ISO/IEC DIS 19989-2:2019(E)

evaluator shall not restrict the feedback loop to the score reading, but shall also estimate the possibility
to deduce information related to the score (e.g. based on execution timing or side-channel leakage).
NOTE As highlighted in ISO/IEC 19792:2009, Clause 6, the evaluator also considers the whole system and
interactions between subcomponents, as a vulnerability of a subcomponent may be compensated by another
subcomponent.
The evaluator shall follow the guidance in ISO/IEC 19792:2009, Clause 8, to assess the potential
vulnerabilities that may be combined with the biometric recognition performance limitations.
6.4 Rating attack potential

For the AVA (vulnerability assessment) class, the rating (i.e. rating of the different factors, calculation of
attack potential and comparison with the targeted level) shall be done according to 19989-1, as well as
the consideration of the attack potential (cf. ISO/IEC 19989-1:20xx, Annex D).
Rating examples related to biometric recognition performances vulnerabilities are provided in Annex A.

ISO/IEC DIS 19989-2:2019(E)

Annex A
(informative)
Examples of attack potential computation for AVA activities
A.1 General
Annex A provides several examples that include various systems that could be evaluated (access control
device for a building, an office, etc., access control to a personal device, etc.) and the “classical” attacks
that could be applied. The attack potential computations are made according to ISO/IEC 19989-1:20xx,
Clause A.3.
A.2 Hill climbing

EXAMPLE Consider a fingerprint based system operating in an uncontrolled environment” (for example,
protecting the access to an item of equipment). Assume that there is a way to easily connect a computer just
before the comparison process, enabling a program to inject probe samples in a suitable format for comparison
and that the comparison score is available (for example, through a debug connection). The attack will consist in
injecting probe samples (for example starting with random locations of minutia) and to optimize the location
using the comparison score in a so called "hill climbing" attack.
This scenario of attack may correspond to two different objectives: One could be related to privacy
leakage (to learn information on the enrolled data), a second one could be related to forging
authentication (i.e. an attacker seeking to achieve authentication against a specific enrolee).
It is considered that:
Elapsed time: The identification phase corresponds to find the right interface to the system (connecting
a computer to the targeted signals, enabling the presentation of built templates) and to get or write
the optimization software for the template generation. It is considered as easy in this example. The
exploitation phase is just running the program to get access. 2 weeks for identification and 1 day for
exploitation are realistic.
Expertise: Even if the attack method is known and published, setting up the right connections,
exploiting specific signals and adapting an optimization software is considered to require an Expert
level for identification and a Proficient level for exploitation.
Knowledge of the TOE: A deep knowledge of the TOE is required (template formats, internal protocols,
etc.). So, a Sensitive knowledge is required.
Window of opportunity (Access to the TOE): The TOE operates in anuncontrolled environment
(rated as easy for exploitation phase) and it is considered as easy to buy system (rated as easy for
identification phase).
Window of opportunity (Access to biometric characteristics): It is rated as Immediate as the attack
does not require the availability of real data (synthetic templates).
Equipment: A specialized equipment is required (computer, connection to the system, and specialized
mostly because of template generation, optimization software) for identification. For exploitation, as
the software is available the equipment is rated standard.
With the above assumptions, the summary of the corresponding attack potential calculation is provided
in Table A.1.

ISO/IEC DIS 19989-2:2019(E)

Table A.1 — Calculation of attack potential for scenario from Clause A.2

Window of opportunity Total
Knowledge of Access to biom- 16
Elapsed Time Expertise Equipment
TOE Access to TOE etric character-
istics
Id Exp Id Exp Id Exp Id Exp Id Exp Id Exp Id Exp
2 0 4 4 4 - 0 0 - 0 2 0 12 4
The rating for the attack is Enhanced-Basic.

If the attack can be performed successfully and no other successful attack with a lower rating is found,
the resistance of the TOE is Basic.
The system is compatible with the AVA_VAN.2 component.
A.3 Combined attack (getting comparison scores in an indirect way)

EXAMPLE Assume the same system as the previous example, except that the comparison score signal cannot
directly be accessed. However, an indirect observation of the comparison process, for example through power
consumption or processing time, can give an information of the comparison score (this method is widely used in
smart-card evaluations and is known as Side Channel Attacks).
As in the previous example, the identification phase corresponds to find the right interface to the
system (Acquiring internal signals – consumption, time –, connecting a computer to the targeted
signals, enabling the presentation of built templates) and to get or write the optimization software for
the template generation. The exploitation phase corresponds to implement the signal acquisition and
processing to the real TOE and run the optimization program to get access.
Elapsed time: More than 1 month for identification and 1 day (less than a week) for exploitation are
realistic.
Expertise: Multiple expertise is required (electronics, signal acquisition and processing, biometrics –
template format and generation –, optimization) for the identification phase. Even if scripted, an Expert
level is required for exploitation (software for template generation and optimization is written and has
just to be used, but physical instrumentation of the TOE has to be done).
Knowledge of the TOE: A deep knowledge of the TOE is required (template formats, internal protocols,
etc.). Sensitive knowledge is required.
Window of opportunity (Access to the TOE): The TOE operates in a fully uncontrolled environment
Equipment: Multiple specialized equipment is required for the identification phase (signal acquisition
and processing, computer, connection to the system, template generation, optimization software). For
exploitation, as the software is considered as written, specialized equipment is enough.
in Table A.2.

ISO/IEC DIS 19989-2:2019(E)


istics
8 0 8 8 4 - 0 0 - 0 4 4 24 12
The rating for the attack is High.

the resistance of the TOE is Moderate.
A.4 Inverse biometrics attack

EXAMPLE Assume the system is an iris based system operating in an uncontrolled environment (for example,
protecting the access to a device or an equipment). There is an easy way to connect a computer just before the
feature extractor, enabling a program to inject synthetic samples and that there is a signal corresponding to the
comparison score (for example, through a debug connection). The attack will consist of injecting reconstructed
synthetic samples and optimizing them using the comparison score.
Note the main difference between this inverse biometrics attack and the hill-climbing attack described
in A.2 is the entry point of the attack, which is before the comparator for hill-climbing (i.e., after the
feature extractor), and before the feature extractor for the inverse biometrics attack. The inverse
biometrics attack requires therefore more expertise in identification, as a realistic input to the feature
extractor needs to be generated in each iteration of the attack (this is why the attack is called inverse
biometrics), so the resulting rating is higher here compared to the hill-climbing attack in A.2.
The identification phase corresponds to find to right interface to the system (connecting a computer to
the targeted signals, enabling the presentation of synthetic samples) and to get or write the optimization
software for the synthetic sample generation. It is considered as easy in this example. The exploitation
phase is just running the program to get access.
Elapsed time: 2 weeks for identification and 1 day for exploitation are realistic.
Expertise: Even if the attack method is known and published, setting up the right connections,
exploiting specific signals and adapting an optimization software is considered to require an Expert
level for identification and a Proficient level for exploitation.
Knowledge of the TOE: A restricted level of knowledge of the TOE is required.
Window of opportunity (Access to the TOE): The TOE operates in a fully uncontrolled environment
Equipment: A specialized equipment is required (computer, connection to the system, synthetic samples
generation, optimization software) in identification. For exploitation, the software being considered as
available, the rating is Standard.
in Table A.3.

ISO/IEC DIS 19989-2:2019(E)


istics
2 0 4 4 2 - 0 0 - 0 2 0 10 4
The rating for the attack is Enhanced-Basic.

A.5 Dictionary attack

EXAMPLE Assume the system is an iris, a two-irises or a face based system operating in an uncontrolled
environment (for example, protecting the access to a device or an equipment). There is an easy way to connect a
computer just before the feature extractor. However, now it is not necessary to access to the comparison score.
The attack will consist in sending, by injection after the data capture subsystem, real biometric samples to the
system until one is accepted.
The identification phase corresponds to find to right interface to the system (connecting a computer to
the targeted signals, enabling the presentation of biometric images). This is considered as easy in this
example. The exploitation phase is just inputting images to get access.
Elapsed time: 2 weeks for identification and exploitation are realistic.
Expertise: A proficient attacker should be able to locate the input of the feature extractor using some
specialized equipment, and thus insert the sample images into the system.
Knowledge of the TOE: A restricted knowledge of the TOE is required.
Window of opportunity (Access to the TOE): The TOE operates in an uncontrolled environment
Window of opportunity (Access to biometric characteristics)
The dictionary attack is conducted using real samples. Getting a large face database for the attack is
very easy (rated as Immediate), but getting a large enough iris database may be very difficult, and
even more challenging for two irises. The rating in exploitation for Access to Biometric Characteristics
reflects both facts. On the other hand, note that although it may be difficult to get a large enough iris
database, once obtained it may be used for attacking different system. The level Easy reflects that fact.
For two-irises, for a similar reason, level Moderate is selected.
Equipment: A specialized equipment is required (computer, connection to the system, data bases).
in Table A.4 for the case of face, Table A.5 for the case of one iris, and Table A.6 for the two-irises case.

ISO/IEC DIS 19989-2:2019(E)

Table A.4 — Calculation of attack potential for scenario from Clause A.5 (face)
istics
2 4 2 0 2 - 0 0 - 0 2 4 8 8
Table A.5 — Calculation of attack potential for scenario from Clause A.5 (iris)
istics
2 4 2 0 2 - 0 0 - 2 2 4 8 10
Table A.6 — Calculation of attack potential for scenario from Clause A.5 (two irises)
istics
2 4 2 0 2 - 0 0 - 4 2 4 8 12
The rating for the attack is Enhanced-Basic (both face and iris) and Moderate (for the two-irises case).
the resistance of the TOE is Basic (both face and iris) and Enhanced-Basis (two-irises).
The system is compatible with the AVA_VAN.2 (both face and iris) component and AVA_VAN.3 component
(two-irises).
NOTE The overall quotation of the attack would in principle depend on the FAR of the system, in particular
for the elapsed time and for the equipment needed during exploitation. If it takes 1 second per attempt and if one
needs 10 000 tries for FAR 10-3, it may take less than 2 weeks. One may moreover need a very large dataset for
exploitation phase. For instance for iris if operating point is FAR 10-6, the need of a dataset of more than 1 million
of iris may be considered as a specialized equipment. For two-irises, with a FAR below 10-8 for instance, the size of
the dataset will justify to increase the equipment to bespoke, thus increasing the overall total for the attack to 26.
A.6 Wolf Attack

EXAMPLE Assume the system is a biometric system for a given modality, operating in an uncontrolled
environment (for example, protecting the access to a device or an equipment). Assume that it is a purely software-
based system, for which it is thus straightforward to connect a computer just before the feature extractor.
Assume also that there is a flaw in the comparison algorithm such that there is a way to construct an image (not
necessarily from natural biometric characteristics) for which the false acceptance rate is significantly higher
than any randomly drawn biometric data. This corresponds to a system with a high rate of successful attack. The
attack will consist in finding the particular (or one of) image(s) that leads to a high chance of being accepted.
The identification phase corresponds to find the weakness in the matching algorithm and therefore to
generate an image which exploits this flaw. The exploitation phase is just inputting the image to get access.
Elapsed time: More than one month for identification might be needed while exploitation is immediate.
ISO/IEC DIS 19989-2:2019(E)

Expertise: An expert attacker should be needed to find the flaw in the algorithm. A layman will be able
to input the image once generated.
Knowledge of the TOE: Sensitive knowledge of the TOE is required to learn the detail of the matching
algorithm.
Window of opportunity (Access to the TOE): The TOE operates in an uncontrolled environment
Window of opportunity (Access to biometric characteristics): It is rated as Immediate as no access is
needed for the attack.
Equipment: A specific equipment may be required to generate an image, acceptable for the feature
extractor.
in Table A.7.

istics
8 0 4 0 4 - 0 0 - 0 2 0 18 0
The rating for the attack is: Enhanced-Basic


ISO/IEC DIS 19989-2:2019(E)

Annex B
(informative)
Examples for ATE activities
B.1 General
Annex B contains example of applying requirements and recommandations from Clause 5 for ATE.
B.2 Example on identifying relevant errors

EXAMPLE Letthe TOE be an Automated Border Control System, considering only face verification
technology, i.e., a fully automatic face verification kiosk as discussed in [1]. This is therefore an operational
scenario representing an automated border control point (BCP) in an airport.
For analyzing which biometric recognition errors are security relevant for an automatic border control
system, a comprehensive list of possible errors (performance measures in the standard terminology)
is reported in the standard ISO/IEC 19795-1. In Table B.1 all the errors reported in that standard are
analyzed one by one.
Table B.1 — Example on identifying relevant errors for an Automated Border Control system
Performance Measure Comment security relevant / minor impact
FTER The system uses pre-enrolment of No / No
the data subjects conducted out of
the scope of the TOE when the data
subjects receive their electronic
national IDs or electronic pass-
ports (eMRTD)
FTAR The application is automated border No / Yes
control, where detailed knowledge
of FTAR or FMR is not necessary re-
garding security evaluation. System
operation is only evaluated in the
general terms provided by FAR and
its corresponding FRR (see Sect. 3.2
for clarification between these 4
error rates). FTAR plays a role in the
context (the higher FTAR the higher
FRR) but it is of no relevance for the
evaluation
FNMR idem (in this case the higher FNMR No / Yes
the higher FRR, therefore FNMR
plays a role but it is not relevant for
the evaluation)
FMR idem (in this case the higher FMR No / Yes
the higher FAR, therefore FMR plays
an important role, but the evalua-
tion is focused in FAR, not in FNMR)

ISO/IEC DIS 19989-2:2019(E)

Table B.1 (continued)

Performance Measure Comment security relevant / minor impact
FRR This error rate plays an important No / Yes
role, as it determines the through-
put of the system. False Rejections
will delay significantly the process
(either because the data subjects
try again, or because an exception
is launched for manual inspection).
Anyway, the relevant error rate here
is only FAR, but FRR should be also
reported to demonstrate that the
system is usable
FAR This is the most important and rele- Yes / Yes
vant error rate, as the main purpose
of the system is to avoid granting
access to illicit data subjects
identification rate n/a (the system is in verification No / No
mode)
FNIR idem No / No
FPIR Idem No / No
pre-selection alg. Idem No / No
pre-selection error Idem No / No
penetration rate Idem No / No
identification rank Idem No / No
As a result, the only identified error rate that shall be taken into account in the evaluation is FAR. In
addition, FRR shall be reported to demonstrate that the system being evaluated is usable.
B.3 Example on determining maximum permissible values for relevant error rates
EXAMPLE Similar to B.2 let the TOE be an Automated Border Control (ABC) system, considering only face
verification technology using pre-enrolled faces stored in electronic passports (eMRTD). This is therefore an
operational scenario representing a border control point in an airport.
The evaluator may now investigate the state-of-the-art and in particular reports on pilots conducted by
independent evaluation bodies or research groups, in addition to observing the context factors involved in
the operation of the system being evaluated. As mentioned above, there is no fixed formula for computing
the desired maximum error rates, but the knowledge from previous works can be a valuable help.
In the specific example here, based on an ABC kiosk, there is the report,[1] very comprehensive in terms
of factors considered (which can be consulted there), which studies in-depth the same system and
operational environment being evaluated here (Test Case number 9 in that report). From the experience
reported based on the Entry-Exit-System-pilot at Schipol Airport and Frankfurt Airport (in which more
than 6 000 travellers participated), and fixing the ABC kiosk to FAR = 0,1 %, then the observed FRR
was around 25 %. The maximum values for error rates considered in this example may thus be fixed
as slightly greater the one observed in that report, for instance., FAR = 0,2 % and for information FRR
= 30 %. The evaluator should also estimate whether these values are acceptable for the security of the
applications that would rely on the TOE.

ISO/IEC DIS 19989-2:2019(E)

Annex C
(informative)
Example of Developer’s performance test document and its

assessment strategy
C.1 General
Annex C is provided as an example for the developer to prepare its performance test document and for
the evaluator to assess it. It is inspired from [6].
The developer may create the test document to report the result of performance testing (e.g. FRR/FAR
or FNMR/FMR).
The evaluator may examine the test document following the assessment strategy described in this
Annex to verify that the developer’s performance test was done in an objective and repeatable manner
to check the trustworthiness of the measured error rates.
The guideline defined in this Annex have been created based on following international standards.
ISO/IEC 19795-1:2006 Biometric performance testing and reporting - Part 1: Principles and framework
ISO/IEC 19795-2:2007 Biometric performance testing and reporting - Part 2: Testing methodologies for
technology and scenario evaluation.
C.2 Providing the test document

The developer may provide the test document for evaluations following this document. The remaining
sublcauses of Annex C define the content of the test document.
C.3 Summary of contents

Table C.1 shows items that may be reported in the test document. Name or structure of test document
doesn’t need to follow the table. However, all items in table are in interest for the test document. Also,
if some items are not included in the test document, the developer may provide a rationale for such
exclusion to the evaluator.
Table C.1 — Reporting items

Subclause Item
C.4.2 Overview of the performance testing (includ-
ing test results)
C.4.3 Target application and influential factors
C.4.4 Test subject selection
C.4.5 Test instructions and training
C.4.6 Test subject management
C.4.7 Test procedure

ISO/IEC DIS 19989-2:2019(E)

C.4 Reporting items description

C.4.1 General
Clause C.4 describes each item in Table C.1 in detail. All items are created based on
ISO/IEC 19795-1 and ISO/IEC 19795-2 however some of them are modified to adjust to the evaluations
under ISO/IEC 15408 (all parts).
C.4.2 Overview of the performance testing
C.4.2.1 General principle
The developer may report following general information about the performance testing.
C.4.2.2 Performance test configuration
The test document may report the following information, in consistence with the information in the ST,
to uniquely identify the test configuration of the performance testing.
— TOE reference: Information that uniquely identify the TOE. Modification to the TOE for performance
testing, if any (e.g. The TOE is modified to export biometric data for off-line testing). The rationale
that such modification doesn’t affect the TOE performance. For example, the developer may claim
that the performance is not affected because modified code isn’t executed during mobile biometric
verification or the developer may run regression test to verify that modification doesn’t change the
result of verification (e.g. similarity score).
— TOE configuration: Any configurable parameters or setting of the TOE that may affect the
performance. Value of each parameter set for the testing. For example, if threshold (e.g. decision
threshold and image quality threshold) is configurable by users, value of threshold set for the
testing.
— Performance test tools: Information that uniquely identify all testing tools (e.g. SDK) used for the
performance testing.
C.4.2.3 Result of the performance testing
The test document may report the following items to provide the result of testing.
— Test period and location: Timeline for the performance testing (samples or templates may be
collected over multiple sessions) and location of testing.
— Result of testing for each modality reported separately.
— Definition of genuine and imposter transaction: If FAR/FRR is concerned, a clear definition of
what constitutes the transaction based on the worst case senario in order to follow the same rule
consistently throughout the performance testing.
— Number of test subjects, templates and samples: the following numbers used for calculating FMR/
FNMR or FAR/FRR (this Annex assumes that at least the FMR or FAR is measured through offline
testing, i.e. cross-comparison, to achieve the maximum number of attempts or transactions. FNMR
or FRR may be measured through online or offline testing).
a) Test subject: Number of test subjects who participated in the testing
b) Enrolment templates: Number of enrolment templates used for testing. All test subjects may not
generate the templates successfully and total number of templates may be less than (number of test
subjects) × (number of body parts of a test subject).
c) Samples: Number of samples collected for each body part and total number of samples collected
from all test subjects. Allto:
Licensed test subjects
Plessis, may
Patrice M. not generate the samples successfully and total number
ISO/IEC DIS 19989-2:2019(E)

of samples may be less than (number of test subjects) × (number of body parts of a test subject) ×
(number of samples collected for each body part).
— Result of testing: Error rates measured by the performance testing. If FAR and FRR are concerned,
number of genuine and imposter transaction. If FMR and FNMR are concerned, number of genuine
and imposter attempts.
C.4.3 Target application and influential factors

Test document may specify a target application modeled in the test, such as mobile biometric verification
in an indoor office environment with a habituated crew.
Test document may also report influential factors that may influence performance, measures to control
such factors and under what factors the performance testing was conducted.
Influential factors, consistent with the target application, can be determined by referring appropriate
documents (e.g. ISO/IEC 19795-3) or referring the product datasheet (e.g. operating temperature).
The following factors are examples of controlling factors for finger/hand vein verification. The
developer may define these factors properly, for example, based on ISO/IEC 19795-3. Any information
that are useful in the context of the used biometric modality may be considered by the developer to
determine the factors.
All influential factors may be controlled appropriately because different error rates may be measured
under different influential factors.
— Test subject demographics: age distribution ratio by arbitrary age groups (e.g., 1, 5, 10 years); gender
distribution; distribution ratio by ethnic origin. Category of ethnic origin may be arbitrarily defined
by developer.
— Posture and positioning: posture of test subject or positioning of his/her hand/finger (e.g.
Orientation of hand/finger in relation to the sensor or distance to the sensor), consistent with the
TOE operational guidance or automated feedback provided by the TOE.
— Indoor or outdoor environment in which testing is to be conducted. In case of outdoor environment,
other factors affecting the performance (e.g. environmental illumination)
— Temperature: range of temperature at which the testing is to be conducted (e.g. “Testing was
conducted in an air-conditioned environment where temperature was kept between X and Y
degrees”)
— Time interval (e.g. minimum, maximum and average time) between enrolment and verification.
— Habituation: The degree to which the test subject is familiarized with the TOE (e.g. frequency of use
of the TOE)
— Template adaptation: how much template adaptation occur prior to measure the FNMR or FRR if the
TOE is able to adapt the templates over time with the aim to reduce the false rejection rates.
C.4.4 Test subject selection

Selection method of test subjects may be reported (e.g. gather test subjects from developer’s employees
or recruit them from public). Demographics of test subjects may follow the target application.
C.4.5 Test instructions and training

Instructions and training given to the test subjects may be reported. The same instructions and training
may be given to the all test subjects.
— Test information and general test instructions given to test subject prior or after biometric data
collection, in consistence to automated guidance or feedback given by the TOE or instructions
ISO/IEC DIS 19989-2:2019(E)

described in the TOE operational guidance. Testing may not be adjusted to the TOE specification
that is not described in the TOE operational guidance.
— Confirmation of habituation: method for how to confirm the level of subject habituation prior to
biometric data collection. If the habituation was confirmed through training, method to ensure
the consistency of training among test subjects and the tools used for training (e.g. developer can
prepare the script for training in advance and apply it to all test subjects to ensure the consistency).
C.4.6 Test subject management

Following information about test subject management may be reported. Proper management is
necessary to avoid human errors that may occur during the testing.
— Management processes: Biometric data can be corrupted by human error during the collection
process (e.g. using a middle finger when the index finger is required). Report the test subject
management processes to avoid such errors and management processes to cover the following
processes.
a) method of initial test subject registration
b) method of ensuring test subject uniqueness
c) amount and type of personal data collected
d) method of avoiding data collection errors (e.g. Use of data collection software minimizing the
amount of data requiring keyboard entry)
C.4.7 Test procedure

A test protocol for the testing may be reported. The following items may be covered.
— Type of attempt or transaction: whether the attempt or transaction is executed online or offline.
Online means that enrolment and verification is executed at the time of image submission. Offline
means that enrolment and verification is executed separately from image submission.
— Test flow: details of flow of genuine and imposter attempt or transaction to measure the error rates,
with the same flow applied to all test subjects. The developer may maintain a log file in which each
interaction with the TOE is recorded. The log may include all test attempts, preprative or practice
attempts, set-up procedure (e.g. setting a threshold) and maintenance activities (e.g. cleaning a sensor).
Such a log file can be very useful to make sure the testing was conducted following the test flow.
— Sample exclusion criteria: criteria for sample exclusion. Test operator may not manually discard
nor use an automated mechanism to discard collected samples unless the samples conform to
documented exclusion criteria. Report the number of excluded samples. If transactions are failed
because of such excluded samples, report the number of such failed transactions. These failed
transactions may be counted as failed transactions to calculate the error rates.
— Advice or remedial action: Advices or remedial actions to test subjects who fail to complete
transactions or sample collections. Such advices or remedial actions may be limited to the minimum
amount necessary with respect to the target application. The same advices or remedial actions may
be given to test subject at the same condition.

ISO/IEC DIS 19989-2:2019(E)

Bibliography
[1] Smart borders pilot project: Report on the technical conclusions of the pilot. Technical report,
eu-LISA, December 2015. doi:10.2857/086263, http://w ww.eulisa.europa.eu/Publications/
Reports/Smart%20Borders%20-%20Technical%20Report.pdf
[2] ISO/IEC 2382-37:2017, Information technology — Vocabulary — Part 37: Biometrics
[3] ISO/IEC/TR 29156:2015, Information technology — Guidance for specifying performance
requirements to meet security and usability needs in applications using biometrics
[4] ISO/IEC 29115:2013, Information technology — Security techniques — Entity authentication
assurance framework
[5] Best Practice Operational Guidelines for Automated Border Control (ABC) Systems,
FRONTEX, September 2015. DOI 10.2819/39041, https://f rontex.europa.eu/assets/Publications/
Research/Best_Practice_Operational_Guidelines_ ABC.pdf
[6] Evaluation Activities for collaborative Protection Profile for Mobile biometric enrolment
and verification - for unlocking the device – cPP, Supporting Document Mandatory Technical
Document, Version 0.2 17-AUG-2018, https://g ithub.com/biometricITC/cPP-biometrics
[7] ISO/IEC 15408-1:2009, Information technology — Security techniques — Evaluation criteria for IT
security — Part 1: Introduction and general model


DIS - 19989-2 - Criteria and Methodology For Security Evaluation of Biometric Systems

Uploaded by

Copyright:

Available Formats

You might also like

DIS - 19989-2 - Criteria and Methodology For Security Evaluation of Biometric Systems

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DIS - 19989-2 - Criteria and Methodology For Security Evaluation of Biometric Systems

Uploaded by

Copyright:

Available Formats

DRAFT INTERNATIONAL STANDARD

ISO/IEC DIS 19989-2

ISO/IEC JTC 1/SC 27 Secretariat: DIN

Information security — Criteria and methodology for

THIS DOCUMENT IS A DRAFT CIRCULATED

COPYRIGHT PROTECTED DOCUMENT

Licensed to: Plessis, Patrice M.

Licensed to: Plessis, Patrice M.

Licensed to: Plessis, Patrice M.

Information security — Criteria and methodology for

3 Terms and definitions

Licensed to: Plessis, Patrice M.

Note 2 to entry: 08.01.11 (2382)

[SOURCE: ISO-IEC-2382-8: 1998 ]

[SOURCE: ISO/IEC 2382-37:2017, 3.1.1]

[SOURCE: ISO/IEC 2382-37:2017, 3.6.3]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 2382-37:2017, 3.4.1]

[SOURCE: ISO/IEC 2382-37:2017, 3.1.2]

[SOURCE: ISO/IEC 2382-37:2017, 3.5.3]

[SOURCE: ISO/IEC 2382-37:2017, 3.3.9]

Note 4 to entry: Biometric features may be extracted from an intermediate biometric sample.

[SOURCE: ISO/IEC 2382-37:2017, 3.3.11]

[SOURCE: ISO/IEC 2382-37:2017, 3.8.2]

[SOURCE: ISO/IEC 2382-37:2017, 3.7.13]

[SOURCE: ISO/IEC 2382-37:2017, 3.6.7]

Note 6 to entry: Use of ‘authentication’ as a synonym for “biometric verification or biometric identification” is

[SOURCE: ISO/IEC 2382-37:2017, 3.1.3]

[SOURCE: ISO/IEC 2382-37:2017, 3.3.16]

[SOURCE: ISO/IEC 2382-37:2017, 3.3.21]

[SOURCE: ISO/IEC 2382-37:2017, 3.2.3]

[SOURCE: ISO/IEC 2382-37:2017, 3.8.3]

[SOURCE: ISO/IEC 30107-3: 2017, 3.1.2]

[SOURCE: ISO/IEC 18045: 2008, 3.3]

[SOURCE: ISO/IEC 15408-1: 2009, 3.1.14]

[SOURCE: ISO/IEC 19795-1:2006, 4.7.1]

[SOURCE: ISO/IEC 15408-1: 2009, 3.1.22]

[SOURCE: ISO/IEC 15408-1: 2009, 3. 4.16]

[SOURCE: ISO/IEC 15408-1: 2009, 3.1.25]

[SOURCE: ISO/IEC 18045: 2008, 3.7]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 2382-37:2017, 3.9.4]

[SOURCE: ISO/IEC 2382-37:2017, 3.9.7]

Note 2 to entry: Comparisons between:

[SOURCE: ISO/IEC 2382-37:2017, 3.9.9]

[SOURCE: ISO/IEC 2382-37:2017, 3.9.11]

[SOURCE: ISO/IEC 15408-1: 2009, 3.1.37]

[SOURCE: ISO/IEC 2382-37:2017, 3.3.31]

[SOURCE: ISO/IEC 15408-1: 2009, 3.5.5]

Note 2 to entry: Presentation attacks may

[SOURCE: ISO/IEC 30107-1: 2016, 3.5]

[SOURCE: ISO/IEC 30107-1: 2016, 3.6]

[SOURCE: ISO/IEC 30107-1: 2016, 3.7]

Licensed to: Plessis, Patrice M.

[SOURCE: ISO/IEC 15408-1: 2009, 3.1.84]

4 Symbols (and abbreviated terms)

Licensed to: Plessis, Patrice M.

5 Supplementary activities to ISO/IEC 18045 on ATE tests