BS ISO/IEC 25059:2023

Software engineering - Systems and software

Quality Requirements and Evaluation
(SQuaRE) - Quality model for AI systems

bsi.
BS ISO/IEC 25059:2023 B RITISH STANDARD

National foreword

This British Standard is the UK implementation of ISO/ IEC 25059:2023.

The UK participation in its preparation was entrusted to Technical

Committee A RT/1, Artificial Intelligence.

A list of organizations represented on this committee can be obtained on

request to its committee manager.

Contractual and legal considerations

This publication has been prepared in good faith, however no

representation, warranty, assurance or undertaking (express or
implied) is or will be made, and no responsibility or liability is or will be
accepted by B S I in relation to the adequacy, accuracy, completeness or
reasonableness of this publication. All and any such responsibility and
liability is expressly disclaimed to the full extent permitted by the law.

This publication is provided as is, and is to be used at the

recipient's own risk.

The recipient is advised to consider seeking professional guidance with

respect to its use of this publication.

This publication is not intended to constitute a contract. Users are

responsible for its correct application.

©The British Standards Institution 2023

Published by BSI Standards Limited 2023

ISBN 978 0 539 14032 3

ICS 35.080

Compliance with a British Standard cannot confer immunity from

legal obligations.

This British Standard was published under the authority of the

Standards Policy and Strategy Committee on 31 July 2023.

Amendments/corrigenda issued since publication

Date Text affected

 BS ISO/IEC 25059:2023

INTERNATIONAL ISO
STANDAR D 25059

First edition
2023-06-28

Software engineering - Systems and

software Quality Requirements and
Evaluation (SQuaRE) - Quality model
for AI systems

- -
Reference number
- -
-
- �
-
-
- ISO 2 5 0 5 9:202 3(E)
-
�ISOJ
-
-
-
-
-�-
- -

- - ©ISO 2023
BS ISO/IEC 25059:2023
ISO 25059:2023(E)

COPYRIGHT PROTECTED DOCUMENT

© ISO 2023, Published in Switzerland
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized otherwise in any form
or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior
written permission. Permission can be requested from either ISO at the address below or ISO's member body in the country of
the requester.
ISO copyright office
Ch. de Blandonnet 8 • CP 401
CH-1214 Vernier, Geneva, Switzerland
Tel. +41 22 749 01 11
Fax +41 22 749 09 47
copyright@iso.org
www.iso.org

ii ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Contents Page

Foreword ........................................................................................................................................................................................................................................ iv

Introduction..................................................................................................................................................................................................................................v

1 Scope ................................................................................................................................................................................................................................. 1

2 Normative references ...................................................................................................................................................................................... 1

3 Terms and definitions ..................................................................................................................................................................................... 1

3.1 General 1
...........................................................................................................................................................................................................

3.2 Product quality . . . . . 2

... ............ ........................ ............ ............ ...................................................................................................................

3.3 Quality in use ............................................................................................................................................................................................ 3

4 Abbreviated terms .............................................................................................................................................................................................. 3

5 Product quality model .................................................................................................................................................................................... 3

5.1 General ........................................................................................................................................................................................................... 3
5.2 User controllability 4
..............................................................................................................................................................................

5.3 Functional adaptability 4

....................................................................................................................................................................

5.4 Functional correctness . . . . . 4

........... ........................ ............ ............ .................................................. ...................................................

5.5 Robustness . . . . . . . 4
............. ........................ ......................... ........................ ......................... ........................ ......................... ..........................

5.6 Transparency . . . . . . . . . 5
........................................................ ............ ............ ........... ............ ............ ........................ ............ ............ ................

5.7 Intervenability 5
.........................................................................................................................................................................................

6 Quality in use model ......................................................................................................................................................................................... 6

6.1 General . . . . . . .
.......... ............ ........................ ............ ............ ............................................................... .................................................. ............. 6
6.2 Societal and ethical risk mitigation . . . .
................................ ........................ ......................... ........................ .......................... 6
6.3 Transparency ............................................................................................................................................................................................ 7
Annex A ( informative) SQuaRE . . . . . .
........................ ......................... ........................ ......................... ........................ ......................... .......................... 8

Annex B ( informative) How a risk-based approach relates to a quality-based approach and

quality models ..................................................................................................................................................................................................... 10

Annex C (informative) Performance .................................................................................................................................................................. 13

Bibliography ............................................................................................................................................................................................................................. 14

©ISO 2023 - All rights reserved iii

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Foreword

ISO (the International Organization for Standardization) and !EC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or !EC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and !EC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and !EC, also take part
in the work.

The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/ IEC Directives, Part 1. In particular, the different approval criteria needed
for the different types of document should be noted. This document was drafted in accordance
with the editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or www.iec.
ch/members experts/refdocs).

ISO and !EC draw attention to the possibility that the implementation of this document may involve the
use of (a) patent (s). ISO and !EC take no position concerning the evidence, validity or applicability of
any claimed patent rights in respect thereof. As of the date of publication of this document, ISO and !EC
had not received notice of (a) patent (s) which may be required to implement this document. However,
implementers are cautioned that this may not represent the latest information, which may be obtained
from the patent database available at www.iso.org/patents and https://patents.iec.ch. ISO and !EC shall
not be held responsible for identifying any or all such patent rights.

Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.

For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.
org/iso/foreword.html. In the !EC, see www.iec.ch/understanding-standards.

This document was prepared by Joint Technical Committee ISO/ IEC JTC 1, Information technology,
Subcommittee SC 42, Artificial intelligence.

Any feedback or questions on this document should be d irected to the user's national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and www.iec.
ch/national-committees.

iv ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Introduction

High-quality software products and computer systems are crucial to stakeholders. Quality models,
quality requirements, quality measurement, and quality evaluation are standardized within the
International Standards on SQuaRE, see Annex A for further information.

AI systems require additional properties and characteristics of systems to be considered, and

stakeholders have varied needs. AI systems have different properties and characteristics. For example,
AI systems can:

replace human decision-making;

be based on noisy, or incomplete data;

be probabilistic;

adapt during operation.

According to ISO/IEC TR 24028,[2_] trustworthiness has been understood and treated as both an ongoing
organizational process as well as a non-functional requirement specifying emergent properties of a
system - that is, a set of inherent characteristics with their attributes - within the context of quality
of use as indicated in ISO/IEC 25010.

ISO/ IEC T R 24028 discusses the applicability to AI systems of that have been developed for conventional
software. According to ISO/IEC TR 24028, does not suff iciently address the data-driven unpredictable
nature of AI systems. While considering the existing body of work, ISO/IEC T R 24028 identifies the
need for developing new International Standards for AI systems that can go beyond the characteristics
and requirements of conventional software development.

ISO/ IEC TR 24028 contains a related discussion on different approaches to testing and evaluation
of AI systems. It states that for testing of an AI system, modified versions of existing software and
hardware verification and validation techniques are needed. It identifies several conceptual differences
between many AI systems and conventional systems and concludes that "the ability of the [AI] system
to achieve the planned and desired result ... may not always be measurable by conventional approaches
to software testing". Testing of A I systems is addressed in ISO/IEC TR 29119-11:2020.L3.l

This document outlines an application-specific A I system extension to the SQuaRE quality model
specified in I SO/IEC 25010.

AI systems perform tasks. One or more tasks can be defined for an AI system. Quality requirements can
be specified for the evaluation of task fulfilment.

The quality model is considered from two perspectives, product quality as described in Clause 5 and
quality in use in Clause 6. The relevance of these terms is explained, and links to other standardization
deliverables (e.g. the ISO/ IEC 24029 series[.4][.SJ) are h ighlighted.

ISO/ IEC 25012:2008[.6.] contains a model for data quality that is complementary to the model defined in
this document. ISO/IEC 25012:2008 is being extended for AI systems by the ISO/IEC 5259 series.[2]

©ISO 2023 - All rights reserved v

This page deliberately left blank
 BS ISO/IEC 25059:2023

INTERNATIONAL STANDARD ISO/IEC 25059:2023

Software engineering - Systems and software Quality

Requirements and Evaluation (SQuaRE) - Quality model
for AI systems

1 Scope

This document outlines a quality model for AI systems and is an application-specific extension to
the standards on SQuaRE. The characteristics and sub-characteristics detailed in the model provide
consistent terminology for specifying, measuring and evaluating A I system quality. The characteristics
and sub-characteristics detailed in the model also provide a set of quality characteristics against which
stated quality requirements can be compared for completeness.

2 Normative references
The following documents are referred to in the text in such a way that some or all of their content
constitutes requirements of this document. For dated references, only the edition cited applies. For
undated references, the latest edition of the referenced document (including any amendments) applies.

ISO/ IEC 25010:2011, Systems and software engineering - Systems and software Quality Requirements
and Evaluation (SQuaRE) - System and software quality models

ISO/ IEC 22989:2022, Information technology - Artificial intelligence - Artificial intelligence concepts
and terminology

ISO/ IEC 2 3053:2022, Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML)

3 Terms and definitions

For the purposes of this document, the terms and definitions given in ISO/IEC 22989:2022,
ISO/IEC 2 3053:2022 and the following apply.

ISO and IEC maintain terminology databases for use in standardization at the following addresses:

ISO Online browsing platform: available at https://www.iso.org/obp

IEC Electropedia: available at https:/Jwww.electropedia.org/

3.1 General

3.1.1
measure,noun
variable to which a value is assigned as the result of measurement

Note 1 to entry: The term "measures" is used to refer collectively to base measures, derived measures,
and indicators.

[SOURCE: ISO/IEC/IEEE 15939:2017, 3.15]

3.1.2
measure,verb
make a measurement

[SOURCE: I SO/IEC 25010:2011, 4.4.6]

©ISO 2023 - All rights reserved 1

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

3.1.3
software quality measure
measure of internal software quality, external software quality or software quality in use

Note 1 to entry: Internal measure of software quality, external measure of software quality or software quality
in use measure are described in the quality model in ISO/IEC 2 5 0 10.

[SOURCE: ISO/IEC 25040:2011, 4.61]

3.1.4
risk treatment measure
protective measure
action or means to eliminate hazards or reduce risks

[SOURCE: ISO/IEC Guide 51:2014, 3.13, modified - change reduction to treatment.]

3.1.5
transparency
degree to which appropriate information about the AI system is communicated to relevant stakeholders

Note 1 to entry: Appropriate information for AI system transparency can include aspects such as features,
components, procedures, measures, design goals, design choices and assumptions.

3.2 Product quality

3.2.1
user controllability
degree to which a user can appropriately intervene in an AI system's functioning in a timely manner

3.2.2
functional adaptability
degree to which an AI system can accurately acquire information from data, or the result of previous
actions, and use that information in future predictions

3.2.3
functional correctness
degree to which a product or system provides the correct results with the needed degree of precision

Note 1 to entry: AI systems, and particularly those using machine learning models, do not usually provide
functional correctness in all observed circumstances.

[SOURCE: ISO/IEC 25010:2011, 4.2.1.2, modified - Note to entry added.]

3.2.4
intervenability
degree to which an operator can intervene in an AI system's functioning in a timely manner to prevent
harm or hazard

3.2.5
robustness
degree to which an AI system can maintain its level of functional correctness under any circumstances

2 ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

3.3 Quality in use

3.3.1
societal and ethical risk mitigation
degree to which an AI system mitigates potential risk to society

Note 1 to entry: Societal and ethical risk mitigation includes accountability, fairness, transparency and
explainability, professional responsibility, promotion of human value, privacy, human control of technology,
community involvement and development, respect for the rule of law, respect for international norms of
behaviour and labour practices.

4 Abbreviated terms

AI artificial intelligence

ML machine learning

5 Product quality model

5.1 General

An AI system product quality model is detailed in Figure 1. The model is based on a modif ied version of
a general system model provided in ISO/ IEC 25010. New and modified sub-characteristics are identified
using a lettered footnote. Some of the sub-characteristics have different meanings or contexts as
compared to the ISO/ IEC 25010 model. The modif ications, additions and differences are described in
this clause. The unmodified original characteristics are part of the A I system product model and shall
be interpreted in accordance with ISO/IEC 25010.

AI system product
quality

I
I I I I I I I I
Functional Performance
Compatibility Usability Reliability Security Maintainability Portability
suitability efficiency

Functional Time behaviour Co-existence �ppropriateness Maturity Confidentiality Modularity Installability

completeness recognisability
Resource Interoperability Availability Integrity Reusability Replaceability
Functional utilisation Learnabiility
correctness m Fault tolerance Non-repudation Analysability Adaptability
Capacity Operability
Recoverability Accountability Modifiability
Functional
User error
appropriatenes
protection Robustnessa Authenticity Testability
Functional
User interface ntervenability a
adaptability a
aesthetics

Accessibility

User
contrallabilitya

Transparency a

a New sub-characteristics.
m Modified sub-characteristics.

Figure 1 - AI system product quality model

©ISO 2023 - All rights reserved 3

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Each of these modified or new sub-characteristics are listed in the remainder of this clause.

5.2 User controllability

User controllability is a new sub-characteristic of usability. User controllability is a property of an AI

system such that a human or another external agent can intervene in its functioning in a timely manner.
Enhanced controllability is helpful if unexpected behaviour cannot be completely avoided and that can
lead to negative consequences.

User controllability is related to controllability, which is described in ISO/ IEC 22989:2022, 5.12.

5.3 Functional adaptability

Functional adaptability is a new sub-characteristic of functional suitability. Functional adaptability of

an AI system is the ability of the system to adapt itself to a changing dynamic environment it is deployed
in. AI systems can learn from new training data, production data and the results of previous actions
taken by the system. The concept of functional adaptability subsumes that of continuous learning, as
defined in ISO/ IEC 22989:2022, 5.11.9.2.

Continuous learning is not a mandatory requirement for functional adaptability. For example, a
system that switches classification models based on events in its environment can also be considered
functionally adaptive.

Functional adaptability in AI systems is unlike other quality characteristics as there are system specific
consequences that cannot be interpreted using a straight-line linear scale (e.g. bad to good). Generally,
higher functional adaptability can result in improvements for the outcomes enacted by A I systems.

For some systems, high functional adaptability can cause additional unhelpful outcomes to become
more likely based on the system's previous choices. Weightings of a decision path with relatively
high uncertainty, reinforced based on previous A I system decisions, can result in higher likelihood of
unintended negative outcomes. In this fashion, functional adaptability can reinforce negative human
cognitive biases.

While conventional algorithms usually produce the same result for the same set of inputs, AI systems,
due to continuous learning, can exhibit different behaviour and therefore can produce different results.

5.4 Functional correctness

Functional correctness exists in ISO/IEC 25010. The AI system product quality model amends the
description since A I systems, and particularly probabilistic ML methods, do not usually provide
functional correctness because a certain error rate is expected in their outputs. Therefore, it is
necessary to measure correctness and incorrectness carefully. Numerous measurements exist for these
purposes in the context of ML methods and examples of these as applicable to a classification model can
be found in ISO/IEC TS 4213.[11]

Additionally, there can be a trade-off between characteristics such as performance efficiency,[11]

robustness[Ll] and functional correctness.

Annex C provides further information about why functional correctness is preferred to other terms
such as the more general performance to describe the correctness of the model.

5.5 Robustness

Robustness is a new sub-characteristic of reliability. It is used to describe the ability of a system to

maintain its level of functional correctness (see Annex C for discussion on the term performance) under
any circumstances including:

the presence of unseen, biased, adversarial or invalid data inputs;

external interference;

4 ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

environmental conditions encompassing generalization, resilience, reliability;

attributes related to the proper operation of the system as intended by its developers.

The proper operation of a system is important for the security of the system and safety of its
stakeholders in a given environment or context. Information about functional safety in the context of A I
systems can b e found in ISO/ IEC T R 5469:-1). [14]

Robustness is discussed in ISO/IEC T R 24028:2020, 10.7,[2.l and methods for assessment are described
in ISO/ IEC T R 24029-1[1] and defined in ISO/ IEC 24029-2. LSJ

5.6 Transparency

Transparency is sub-characteristic of usability in the product quality model and a sub-characteristic of

satisfaction in the quality in use model.

It relates to the degree to which appropriate information about the AI system is communicated to
stakeholders.

Transparency of AI systems can help potential users of AI systems to choose a system to fit their
requirements, improving stakeholders' knowledge about the applicability and the limitations of an A I
system, and assisting with the explainability of AI systems.

The transparency information can include a description of an AI system functionality, the system's
decomposition, interfaces, ML models used, training data, verification and validation data, performance
benchmarks, logs and the management practices of an organization responsible for the system.

Transparent AI systems document, log or display their internal processes using introspection tools and
data files. The flow of data can be trackable at each step, with applied decisions, exceptions and rules
documented. Log output can track processes in the pipeline as they permute data, as well as system level
calls. Errors are logged explicitly, particularly in transform steps. Highly transparent A I systems can be
built of well-documented subcomponents whose interfaces are explicitly described. The transparency
of AI systems eases investigations of system malfunctions.

A system with low transparency has internal workings which are difficult to inspect
externally. Unavailability of detailed processing records can impair testability and societal and ethical
impact assessment and risk treatment.

Ultimately, transparency of A I systems contributes to establishing of trust, accountability

and communication among stakeholders. Some aspects of transparency are discussed in
ISO/IEC T R 24028:2020, 10.2.LZ.l

5. 7 lntervenability

The extent of intervenability can be determined depending on the scenarios where the A I system can
be used. The key to intervenability is to enable state observation and transition from an unsafe state
to a safe state. Operability is the degree to which an A I system has attributes that enable operation and
control, which emphasizes the importance of an AI system's user interface. Compared to operability,
intervenability is more fundamental from a quality perspective and is intended to prevent an AI system
from doing harm or hazard.

Intervenability is related to controllability, which is described in ISO/IEC 22989:2022, 5.15.5.

1) Under preparation. Stage at the time of publication ISO/IEC CD TR 5469 :2023.

©ISO 2023 - All rights reserved 5

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

6 Quality in use model

6.1 General

An AI system quality in use model is detailed in Figure 2. The model is based on a modified version of a
general quality in use model provided in ISO/IEC 25010. New sub-characteristics are identified using a
lettered footnote. Some of the sub-characteristics have different meanings or contexts as compared to
the ISO/IEC 25010 model. The additions and differences are described in this clause. The unmodified
characteristics are part of the quality in use model and shall be interpreted as defined in ISO/ IEC 25010.

AI system
quality in use

I I I I
Freedom Context
Effectiveness Efficiency Satisfaction
from risk coverage

Effectiveness Efficiency Usefulness Economic risk Context

mitigation completeness
Trust
Health and Flexibility
Pleasure safety
risk mitigation
Comfort
Environment
Transparency a risk mitigation

Societal and
ethical risk
mitigation a

a New sub-characteristics.

Figure 2 - AI system quality in use model

6.2 Societal and ethical risk mitigation

Societal and ethical risk mitigation is a new sub-characteristic of freedom from risk.

ISO/ IEC T R 24368:2022,[15] explores this topic and outlines the following themes:

accountability;

fairness and non-discrimination;

transparency and explainability;

professional responsibility;

promotion of human values;

privacy;

6 ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

safety and security;

human control of technology;

community involvement and development;

human-centred design;

respect for the rule of law;

respect for international norms of behaviour;

environmental sustainability;

labour practices.

These themes should be considered along with the information in ISO/IEC T R 24368, with regard to the
desirable characteristics of systems in order to mitigate societal and ethical risk.

Safety and security as well as sustainable environments are included within the existing SQuaRE
characteristics health and safety risk mitigation, and environment risk mitigation, and therefore are
not outlined in this document.

Annex 8 describes how a risk-based approach relates to a quality-based approach and quality models
and, through examples, demonstrates how the two can be used in support of each other.

B ias, including in the context of fairness and non-discrimination, is described in ISO/IEC T R 24027.[l.6.]

6.3 Transparency

Transparency is also a sub-characteristic of satisfaction in the product quality model. See 5.6 for further
information.

©ISO 2023 - All rights reserved 7

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Annex A
(informative)
SQuaRE

A.1 SQuaRE divisions

Figure A.1 (adapted from ISO/IEC 25000:2014[1]) illustrates the organization of the SQuaRE
International Standards, further called divisions.

Quality Model Division

ISO/IEC 2 501n

Quality Quality
Quality Management
Requirements Evaluation
Division
Division Division
ISO/IEC 2 500n
ISO/IEC 2 303n ISO/IEC 2 304n

Quality Measurement
Division
ISO/IEC 2 502n

Extension Division ISO/IEC 2 5050 to ISO/IEC 2 5099

Figure A.1 - Organization of SQuaRE International Standards

The divisions within SQuaRE are:

ISO/IEC 2500n -Quality Management Division. The International Standards that form
this division define all common models, terms and definitions further referred to by all other
International Standards relating to SQuaRE. The division also provides requirements and guidance
for a supporting function that is responsible for the management of the requirements, specification
and evaluation of software product quality.

ISO/IEC 2501n - Quality Model Division. The International Standards that form this division
present detailed quality models for computer systems and software products, quality in use and
data. Practical guidance on the use of the quality models is also provided.

ISO/IEC 2502n - Quality Measurement Division. The International Standards that form this
division include a software product quality measurement reference model, mathematical definitions
of software quality measures, and practical guidance for their application. Examples are given
of internal and external measures for software quality and measures for quality in use. Quality
Measure Elements ( QME) forming foundations for these measures are defined and presented.

ISO/IEC 2503n - Quality Requirements Division. The International Standards that form this
division help specify quality requirements based on quality models and software quality measures.
These quality requirements can be used in the process of quality requirements elicitation for a
software product to be developed or as input for an evaluation process.

ISO/IEC 2504n Quality Evaluation Division. The International Standards that form this division
-

provide requirements, recommendations and guidelines for software product evaluation, whether

8 ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

performed by evaluators, acquirers or developers. The support for documenting a measure as an

Evaluation Module is also present.

ISO/IEC 25050toISO/IEC 25099- SQuaRE Extension Division. These International Standards

currently include requirements for quality of Commercial Off-The-Shelf software and Common
Industry Formats for usability reports, as well as for this document.

©ISO 2023 - All rights reserved 9

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

AnnexB
(informative)
How a risk-based approach relates to a quality-based approach
and quality models

B.1 General

R isk-based and quality-based approaches use two complementary methodologies to contribute to the
specif ication of systems requirements, developing systems in accordance with the requirements and
evaluating system functionality. In most cases a combined approach is necessary to cover all aspects of
a specific system behav iour. The applicability and exact use of each methodology relies on many factors,
including, but not limited to the maturity of technologies in use, the role of the system in the value
chain, and the field of application. Such factors are taken into consideration when specifying product
or regulatory requirements or a framework for the development and evaluation of a certain system in
order to achieve practical results and appropriately manage customer or user expectations.

Using a quality model is most effective in situations where quantifiable or clearly qualifiable system­
specific measures are established for all characteristics as well as for many sub-characteristics. For
mature technologies in a context of a specific application field, the values of the measures can be
specif ied as a part of the system requirements.

For new technologies or new applications, using a partial quality model can still be effective in high
negative risk situations. In such cases, only the measure values for high negative risk sub-characteristics
are specified and evaluated.

For new or evolving technologies, software quality measures are often subject to active research and
collaboration between academia and industry.

A risk-based approach is better suited for situations where quantifiable measures are not established
for many characteristics or important sub-characteristics. In addition, and by definition, a risk-based
approach is uniquely suited to deal with unexpected events.

B.2 Relationship with other International Standards

ISO 31000[19] defines a risk management framework and process applicable to any kind of organization
and any activ ity, including decision-making at all levels. ISO/ IEC 2 3894[.18.] is based on ISO 31000; it
customizes the risk management framework and process to managing risks to organizations and
projects dealing with Al.

ISO/IEC 25010 defines two quality models. A product quality model contains characteristics and sub­
characteristics of a software or a computer system. A quality in use model contains characteristics and
sub-characteristics describing the interaction of a system w ith its env ironment. This can be viewed as
a two-layered approach with the product quality properties providing the foundation for many of the
properties demonstrated during the use of the product.

"Freedom from risk" is identified as one of the quality characteristics under the quality in use model.
Its sub-characteristics are called "risk mitigations" and correspond to common high-level societal
concerns about economy, health, safety, environment, and ethics. In comparison, according to the risk­
based approach, the essential societal properties related to economy, health, safety, environment, and
ethics are called "objectives". Furthermore, in a risk-based approach, objectives are not limited to
societal considerations. They include all relevant engineering considerations, for example, those related
to system security, reliability, and transparency. Subsequently, for each identified objective, potential
risks are identified and mitigated by applying selected risk treatment measures or "controls". Therefore,

10 ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

each relevant property (societal, system, etc.) can be expressed both in terms of characteristics and in
terms of objectives.

B.3 Comparison of approaches

The main challenge of specifying or evaluating systems using a quality-based approach alone is that
quality sub-characteristics are often not specific enough and do not directly correspond to accepted
qualifying or quantifying measures.[.12.]

Looking at "fairness" as an AI system quality model sub-characteristic, being "fair" can carry drastically
different meanings across different application areas and systems with different purpose.[1.6.] No
single fairness measure exists to measure a system "fairness". Moreover, the availability and maturity
of fairness measures depends on the system type and the technology used. For example, fairness
measures for classification systems are better understood than fairness measures for reinforcement
learning systems.

The risk-based approach allows the specification and assessment of objectives, including those for
which no direct measures are available, by shifting the task to a set of new objectives. These objectives
do not necessarily have corresponding measures. In a risk-based approach, potential negative effects,
sometimes referred to as repercussions or consequences, are identified and quantified or qualified. As a
part of risk treatment, risk sources leading to negative repercussions are examined, and risk treatment
measures to reduce (or eliminate) the risk sources are selected. An iterative process of selection and
implementation of risk treatment measures and the assessment of the residual risk continues until the
risk is reduced to an acceptable level (as specified in the system requirements).

In comparison to the two defined quality models, risk management is a multi-layered top-down
approach. This means that risk treatment measures (or "controls") at one layer selected by a risk
management process become the objectives of a risk management process at a layer below. In turn,
these new objectives correspond to a set of different sub-characteristics, for which software quality
measures or their values can be available or not.

Assuming that "fairness" was stated as one of the A I system requirements, as a result of risk assessment,
the following potential risks sources were identified:

lack of experience in the application field;

biased training data.

The selected mitigation risk treatment measures (or "controls") included:

a functional specification review by experts in the application field;

reducing selection bias in the training data using appropriate data quality methods.

The next layer of objectives is:

implementing the experts' review recommendations;

keeping the data bias at an acceptable level as specified using selected statistical measures of
data quality.

The risk-based approach can also assist with developing new software quality measures. It can be used
to assess the behaviour or the value of a new measure under various conditions and risk levels as a
function of existing "proxy" measures.

Suppose that in the process of requirements specification for an A I classification system used for
people hiring, Demographic Parity and Equality of Opportunity[16] have been identified as suitable
measures for judging a system's fairness. Nevertheless, currently no sufficient experience exists to
determine the required measure's values. In such a situation, the risk-based approach will still allow
the implementation of the A I system by identifying the appropriate risk mitigation risk treatment
measures and assessing their results.

©ISO 2023 - All rights reserved 11

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Subsequently, experience from the system deployment and the collected data will help to find the
correlation between the levels of risk and the optimal measure values. This knowledge can then be
used for the specification of similar systems in the future.

In conclusion, using a risk-based approach increases flexibility in specifying requirements. System

characteristics and sub-characteristics in the quality-based approach correspond to high-level objectives
in the risk-based approach. Each system property can be expressed both in terms of characteristics and
in terms of objectives. Risk-based approach allows the specif ication, implementation, and verification of
system behaviour for objectives, including those for which software quality measures are not available
or their values are not specified. In addition, the risk-based approach can assist with establishing
suitable values for new measures.

12 ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Annexe
(informative)
Performance

The term performance is used differently in the field of software and systems engineering, compared
to the field of artificial intelligence.LZ.. O. l

Within the field of software and systems engineering, performance typically means how fast a certain
piece of software executes and how efficient it is. These aspects fall under the existing SQuaRE
characteristic performance efficiency.

Within the field of artificial intelligence, performance typically means how well a certain AI system
performs the intended tasks. Performance of intended tasks of an AI system can be measured by
determining suitable evaluation measures that are relevant to the type of AI system. For example,
ISO/ IEC TS 4213[11] specifies methodologies for assessing ML classification performance.

©ISO 2023 - All rights reserved 13

BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

Bibliography

[1] ISO/ IEC 25000:2014, Systems and software engineering - Systems and software Quality
Requirements and Evaluation (SQuaRE) - Guide to SQuaRE

[2] ISO/IEC/TR 24028:2020, Information technology - Artificial intelligence - Over view of

trustworthiness in artificial intelligence

[ 3] ISO/ IEC/TR 29119-11:2020, Software and systems engineering - Software testing - Part 1 1 :
Guidelines o n the testing o f AI-based systems

[4] ISO/ IEC/TR 24029-1, Artificial Intelligence (AI) - Assessment of the robustness of neural networks
- Part 1: Over view

[5] ISO/ IEC 24029-2, Artificial intelligence (AI) - Assessment of the robustness of neural networks -
Part 2: Methodology for the use of formal methods

[6] ISO/ IEC 25012:2008, Software engineering - Software product Quality Requirements and
Evaluation (SQuaRE) - Data quality model

[7] ISO/IEC 5259 (series), Information technology - Artificial intelligence - Data quality for analytics
and machine learning (ML)

[8] ISO/ IEC/ IEEE 15939, Systems and software engineering - Measurement process

[9] ISO/ IEC 25040:2011, Systems and software engineering - Systems and software Quality
Requirements and Evaluation (SQuaRE) - Evaluation process

[10] ISO/ IEC Guide 51, Safety aspects - Guidelines for their inclusion in standards

[11] ISO/ IEC/TS 4213, Information technology - Artificial intelligence - Assessment of machine
learning classification performance

[12] PREUVENEE R S. D., TSINGE NOPOULOS, I., JOOSEN, W., 2020, Resource Usage and Performance
Trade-offs for Machine Learning Models in Smart Environments.Sensors (Basel), vol. 20, no. 4,
p. 1176, doi: 10. 3390/s20041176

[13] ZHA NG. H., YU, Y., J IAO, J., XING, E.P., GHAOUI, L.E., JOR DA N, M.I., Theoretically Principled Trade­
off between Robustness and Accuracy, 2019. Available from: https://arxiv.org/abs/1901.08573v3

[14] ISO/ IEC T R 5469:-2), Information Technology - Artificial intelligence - Functional safety
and AI systems

[15] ISO/ IEC TR 24368, Information Technology - Artificial intelligence - Over view of ethical and
societal concerns

[16] ISO/ IEC/TR 24027, Information technology - Artificial intelligence (AI) - Bias in AI systems and
AI aided decision making

[17] ISO 31000, Risk management - Guidelines

[18] ISO/ IEC 23894, Information technology - Artificial intelligence - Guidance on risk management

[19] WAGNER S. Software Product Quality Control. Heidelberg: Springer Berlin, 201 3, DOI
10.1007/978-3-642-38571-1 2. Available from:3)

2) Under preparation. Stage at the time of publication ISO/!EC CD TR 5469 :2023.

3) https: //link.springer.com/book/10.1007/9 78-3-642-38571-1

14 ©ISO 2023 - All rights reserved

 BS ISO/IEC 25059:2023
ISO/IEC 25059:2023

[20] LENARDUZZI V., LOMIO F., MORESCHINI S., TAI B I D., TAM BURRI D. A. Software Quality for AI:
Where We Are Now?. Software Quality: Future Perspectives on Software Engineering Quality,
2021, vol. 404,

©ISO 2023 - All rights reserved 15

NO COPYING WITHOUT BSI PERMISSION EXC EPT AS PERMITIED BY COPYRIGHT LAW

B ritish Sta n d a rds I nstitution (BSI)

BSI is the national body responsi ble for prepa ri ng B ritish Sta ndards and other
standards-related publ ications, i nformation and services .
BSI is i ncorporated by Roya l C harter. B ritish Sta ndards and other standardization
prod ucts are publ ished by BSI Standards Lim ited .

About us Reprod ucing extracts

We bring together business, industry, government, consumers, innovators For permission to reproduce content from BSI publications contact the BSI
and others to shape their combined experience and expertise into standards Copyright and Licensing team.
-based solutions.

The knowledge embodied in our standards has been carefully assembled in

Subscriptions
a dependable format and refined through our open consultation process. Our range of subscription services are designed to make using standards
Organizations of a l l sizes and across all sectors choose standards to help easier for you. For further i nformation on our subscription products go to bsigroup.
them achieve their goals. com/subscriptions.

I nformation on standards
We can provide you with the knowledge that your organ ization needs
to succeed. Find out more about British Standards by visiting our website at
bsigroup.com/standards or contacting our C ustomer Services team or
Knowledge Centre.
With British Standards Online (BSOL) you'll have i nstant access to over 55,000
British and adopted European and international standards from your desktop.
It's available 24/7 and is refreshed daily so you'll always be up to date.
You can keep in touch with standards developments and receive substantial
discounts on the purchase price of standards, both in single copy and subscription
format, by becoming a BSI Subscribing Member.

PLUS is an updating service exclusive to BSI Subscribing Members. You will

Buying sta ndards
automatically receive the latest hard copy of your standards when they're
You can b u y a n d download P D F versions o f B S I publ ications, including British and
revised or replaced.
adopted European and international standards, through our website at bsigroup.
To find out more about becoming a BSI Subscribing Member and the benefits
com/shop, where hard copies can also be purchased.
of membership, please visit bsigroup.com/shop.
If you need international and foreign standards from other Standards Development
With a Multi-User Network Licence (MUNL) you are able to host standards
Organizations, hard copies can be ordered from our C ustomer Services team.
publications on your intranet. Licences can cover as few or as many users as you
wish. With u pdates supplied as soon as they're available, you can be sure your
Copyright in BSI publications
documentation is current. For further information, email cservices@bsigroup.com.
All the content in BSI publications, including British Standards, is the property
of a n d copyrighted by BSI or some person or entity that owns copyright in the
Revisions
information used (such as the i nternational standardization bodies) a n d has
Our British Standards and other publications a r e updated b y amendment or revision.
formally licensed such i nformation to BSI for commercial publication and use.
We continually improve the quality of our products and services to benefit your
Save for the provisions below, you may not transfer, share or disseminate any
business. If you find a n inaccuracy or ambiguity within a British Standard or other
portion of the standard to any other person. You may not adapt, distribute,
BSI publication please inform the Knowledge Centre.
commercially exploit or publicly display the standard or any portion thereof in any
manner whatsoever without BSl's prior written consent.
Useful Contacts
Storing and using sta ndards Customer Services

Standards purchased in soft copy format: Tel: +44 345 086 9001
Email: cservices@bsigroup.com
• A British Standard purchased in soft copy format is licensed to a sole named
user for personal or internal company use on ly.
Subscriptions
• The standard may be stored on more than one device provided that it is Tel: +44 345 086 9001
accessible by the sole named user only and that only one copy is accessed at
Email: subscriptions@bsigroup.com
any one time.

• A single paper copy may be printed for personal or internal company use only. Knowledge Centre
Tel: +44 20 8996 7004
Standards purchased in hard copy format:
Email: knowledgecentre@bsigroup.com
• A British Standard purchased in hard copy format is for personal or internal
company use only. Copyright & Licensing
Tel: +44 20 8996 7070
• It may not be further reproduced - in any format - to create an additional copy.
Email: copyright@bsigroup.com
This includes scanning of the document.

If you need more than one copy of the document, or if you wish to share the
BSI Gro u p Headquarters
document on an internal network, you can save money by choosing a subscription
product (see 'Subscriptions').
389 Chiswick High Road London W4 4AL U K

bsi.