Module 4-6-Training For AI Ethics - Bangkok & Indonesia Feb Mar 2023 - Final

TERMS OF USE
This is a copyrighted IEEE Document.

You may access, download, print and/or retain one (1) copy of this document for your individual use only, including job-
related functions.
Except as allowed by the copyright laws of the United States of America or applicable international treaties, or as explicitly
allowed in these Terms of Use, you may not further copy, prepare, and/or distribute copies of the Document, nor
significant portions of the Document, in any form, without prior written permission from IEEE.
Requests for permission to reprint this Document, in whole or in part, or requests for a license to reproduce and/or
distribute this Document, in any form, must be submitted via email to the Standards Licensing and Contracts ,
( https://standards.ieee.org/ipr/copyright-permissions-form.html ) or in writing to:
IEEE-SA Licensing and Contracts

445 Hoes Lane
Piscataway, NJ 08854 USA
© Copyrighted to IEEE
AGENDA
• How Standards support Critical and Emerging Technology, especially Artificial Intelligence (AI)
• Global Standardization on AI
• Gender Responsive Standard & How to develop
IEEE – OVER 135 YEARS OF INNOVATION
From the telegraph…to autonomous vehicles and AI, IEEE has

been at the center of ground-breaking innovation for over 135
years
4IR
3IR
2IR
3
ADVANCING
TECHNOLOGY
FOR HUMANITY
ABOUT IEEE
▪ Inspiring a global community of innovation
▪ Where forward-thinking professionals collaborate to:
▪ Discover what’s next in tech innovation
▪ Build technical communities
▪ Shape and share research
▪ Engage in Humanitarian activities
▪ Create global standards
4
RAISING THE WORLD’S STANDARDS WHAT STANDARDS DO FOR INDUSTRY & SOCIETY
Ensure system Continue to
interoperability enhance technology
while enabling product by motivating participation,
differentiation collaboration & competition
from a diverse set of companies
Create new Improve

markets GLOBAL reliability
and expand addressable STANDARDS and become a vehicle
markets of existing to address regulatory
products & technology requirements
Allow for better

Reduce market risk
cost efficiency especially in areas that require
through economies of scale large investments
& multi-vendor sourcing
RAISING
THE WORLD’S
STANDARDS
ABOUT IEEE SA
Developing market relevant open standards and solutions:
▪ Advancing global technologies and technology platforms
▪ Promoting innovation
▪ Protecting public safety, health & wellbeing
▪ Contributing to a sustainable future
AI LANDSCAPE
Digital transformation seems to be at the forefront
of every business’s organizational strategy.
It is defined as “the adoption of digital technology
to transform services or businesses, through
replacing non-digital or manual processes with
digital processes or replacing older digital
technology with newer digital technology”.
-ServicePath Website based on research from Accenture, Deloitte, and Mckinsey
https://servicepath.co/2021/04/the-future-of-digital-transformation-accenture-deloitte-and-mckinsey/#:~:text=Digital%20transformation%20should%20be%20at,digital%20technology%20with%20newer%20digital
THE USE OF AI SYSTEMS IS ONLY GROWING ….
• Worldwide revenues for the artificial intelligence (AI) market,
including software, hardware, and services, are forecast to grow
year over year in 2021 to ~ $1,800B by 2030
• There is a conscious increase by services providers to applying AI to
solve industry- and domain-specific problems for clients
• ~80M US Adults use an AI assistant in their cars at least once a
Market Growth
month
• 50+% of companies monitor AI-created bias
• Regional and National strategies have only increased investments
into horizontal and vertical-oriented AI systems – coordination
across University systems and research bodies is key.
“…meeting the needs of the present without
compromising the ability of future generations to
meet their own needs.”
- UN/IPCC definition of Sustainability
REAL, UNPREDICTABLE CHALLENGES
Consequences that are a matter of livelihood, life, & death…
Algorithmic Transparency AI Accountability AI City Governance AI Org. Governance
Copyright Issues
Algorithmic Equity Imbalance Algorithmic Bias Algorithmic Cheating
…and existing approaches have not raised to

Algorithmic Communications properly address these real challenges.
AND THESE ARE NOT ONE OFF EXAMPLES…
DIGITAL TRANSFORMATION JOURNEY - ELEMENTS FOR SUCCESS
Today’s challenges… …create tomorrow’s opportunities
 Opportunities left on the table due to  Demonstrative commitment to trustworthy and

existing infrastructure’s inability to realize transformative relationships with stakeholders
longer-term outcomes  Integrated strategic planning
 the absence of systematic planning  Increased agility with organizational alignment and
 budget allocation and capital investment empowered staff
to support tech advancements  Market opportunities and pursuits
 Staff skills development – concepts,  Business process efficiencies realized
applications, and execution
 Improved customer engagements and outcomes
NOTED AI RISKS
• Given the broad reach and impact of AI
technologies and associated markets,
the need for regulatory compliance
presents a natural risk concern.
• Socio-technical elements make up 33%
of the categories associated with AI
Market Perspectives
Risk per the survey (Explainability,

Privacy, Equity and Fairness) showing
the significance of the category on
organizational considerations.
• Organizational reputation is identified
https://www.mckinsey.com/business-functions/quantumblack/our-insights/global-survey-the-state-of-ai-in-2021
as a relevant risk that organizations
would like to mitigate concerning their
AI implementations.
… INCREASING MARKET DRIVERS THAT DEMAND ONGOING,
DEMONSTRATED TRUSTWORTHY ENGAGEMENTS AT MULTIPLE LEVELS &
TOUCHPOINTS…
• Soft or socio-technical attributes are difficult to demonstrate compliance to the marketplace
without accepted and proven certification schemes from recognized entities
• AI Ethics expertise available to large companies is not necessarily available to small and mid-
size enterprises; increasing the competitive gaps and challenging resource prioritization to
earn compliance where necessary
Market Considerations
• Research has shown bias in AI-based systems producing detrimental results to people of color,
gender, and more. The negative brand perception and impact can lead to loss of revenues and
business. Further, the trusted relationship with an Entity’s stakeholders are not easily repaired.
• Draft regional regulations seek to encourage socio-technical considerations to be
implemented in situations where the risk is unacceptable or high-risk. Yet, there is no evident
test or standardized approach that Entity’s can rely upon immediately without undertaking an
additional risk of further testing fees or AI solutions being deployed with a risk of having to
recall.
• Cities and Governments tasked with procuring technologies with embedded or hosted AI
systems require the means to properly vet the ethical AI concerns and considerations to
ensure citizens and residents are treated consistently in the most trust-worthy manner
possible.
“Our industry has started to embrace and apply key “We want to design healthy relationships with our users. The
technologies such as AI and robotics, which could potentially potential of AI is wrapped up in its longevity as a solution—
disrupt the whole energy value chain. We should be thoughtful meaning everything we design must address current and
about delivering this change in the right way, and avoiding future needs for users. To truly understand those needs, we
unintended consequences, before the technology becomes need an inclusive and ethical approach to the entire
widespread.” process. Globally, we are starting to see the repercussions that
come when companies do not prioritize AI ethics in their
solutions. We want to make sure that ethical practices are
“Our core values drive everything we do. Trust,
ingrained on our teams so they can then be embedded into
customer success, innovation, and equality
the products themselves.”
mean that developing AI our customers and
society can trust is critical to our mission.”
“…. a values-driven company who puts our responsibilities toward kids and parents at the
center of our decision making. We aim to apply the same rigor to our implementation of
technical experiences/ solutions as we do to production, quality, and safety.”
“We see opportunities for AI and automation to be embedded into our day-today technology design and operational processes, so it’s
vital that our people are aware of and understand AI ethics in the context of their work. This will enable us to proactively
manage potential impacts on employees, customers, and society.”
“We have an ethical foundation which we continuously work to strengthen and expand. In the past this has included adopting a
progressive code of conduct, developing and adhering to global human rights principles, and assessing our supply chain to enable
sustainable development including pursuing conflict-free minerals. As we become more data and AI-centric, we see the need to
expand our principles in such a way to mitigate any future harm our hardware, software, or data science contributions could
make in the world.”
Source: EAD for Business
OBSERVED TECHNIQUES & APPROACHES TOWARDS TRUSTWORTHY AI
Increased Hiring of Roles Dedicated to AI Ethics
• Chief AI Ethics Officer
• Data Ethicists
• Responsible AI Ethicist
Growth in Internal Orientations efforts

• Developing AI Review Boards
• Internal training programs, values, and frameworks
• Updating and integrating into their risk profiles and AI harms reductions additional perspectives and identifying
audit mechanisms
• Monitoring or directly engaging in the policy development and feedback
Market Activities
• Integrating criteria as an aspect of procurement, pricing strategies, and/or investment strategies

• Understanding implications to offerings and internal systems
Forging a path to demonstrate Commitment externally

• Evaluating externally facing mechanisms to demonstrate compliance or forward consideration
of stakeholder basis
• Engaging in proof of concepts and pilots
• Externally communicating efforts dedicated to AI Ethics
• Issuing or contributing to external research
Principles & Frameworks
Tools
Standards
Certification
AI Literacy
Market Basis
Community
AI Governance Platforms
EAD for Business:
AI Ethics Readiness Framework
TECHNOLOGY ETHICS
• Algorithmic Bias & Fairness

• Transparency &
Explainability
• Accountability
• Privacy
• Safety & Security
• Responsible Governance
Credit: Berkmann Klein Center
“Salesforce’s core values drive everything we do. Trust, customer success, innovation, and
equality mean that developing AI our customers and society can trust is critical to our
mission.”
—Kathy Baxter, Salesforce
“We have an ethical foundation which we continuously work to strengthen and expand. In the past this has
included adopting a progressive code of conduct, developing and adhering to global human rights
principles, and assessing our supply chain to enable sustainable development including pursuing conflict-
free minerals. As we become more data and AI-centric, we see the need to expand our principles in such a
way to mitigate any future harm our hardware, software, or data science contributions could make in the
world.” — Anna Bethke, Intel
“We see opportunities for AI and automation to be embedded into our day-today technology design and
operational processes, so it’s vital that our people are aware of and understand AI ethics in the context of
their work. This will enable us to proactively manage potential impacts on employees, customers, and
society.”
Source: EAD for Business —Amy Oding, Vodafone
“Our industry has started to embrace and apply key technologies such as AI and robotics,
which could potentially disrupt the whole energy value chain. We should be thoughtful about
delivering this change in the right way, and avoiding unintended consequences, before
—Bob Flint, BP
the technology becomes widespread.”
“We want to design healthy relationships with our users. The potential of AI is wrapped up in
its longevity as a solution—meaning everything we design must address current and future
needs for users. To truly understand those needs, we need an inclusive and ethical approach
to the entire process. Globally, we are starting to see the repercussions that come when
companies do not prioritize AI ethics in their solutions. We want to make sure that ethical
practices are ingrained on our teams so they can then be embedded into the products
themselves.”
—Milena Pribić, IBM
“The LEGO Group is a values-driven company who puts our responsibilities toward kids and
parents at the center of our decision making. We aim to apply the same rigor to our
implementation of technical experiences/ solutions as we do to production, quality, and
safety.”
—Sam Coates, LEGO, Creative Play Lab
Source: EAD for Business
AS INVESTMENTS IN AI INNOVATION & USE INCREASE, RISK MITIGATION HAS NOT MATCHED…
Source: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2022-and-a-half-decade-in-review
A Deloitte global survey of 500 government leaders in 2021
found that 92 percent of respondents at the federal level,
95 percent at the state level, and 84 percent at the local
level are of the view that AI is “mission critical” over the
next five years.[4] At least half of those surveyed listed gaps
in skills as a crucial reason for the inability of governments
to utilize AI applications in the most effective manner.
Source: https://www.orfonline.org/research/g20-ai-national-strategies-global-ambitions/
© Copyrighted to IEEE 35
FURTHER EMPHASIZING THE NEED FOR POLICIES EMERGING ENCOURAGING GREATER
TRUSTWORTHY IMPLEMENTATION
Several policy developments with broader

ripple effects on AI Systems…
Europe
AI Act
Digital Markets Act
Digital Services Act
Data Governance Act
US Algorithmic Accountability Act
…building upon a number of principles that

have emerged over the past 5+ years,
including but not limited to:
Ethically Aligned Design, 1st Edition
G20/OECD AI Principles & Recommendations
HLEG AI – Ethics Guidelines for Trustworthy AI
Asilomar AI Principles
NIST AI RMF
AI POLICY PERSPECTIVES: DIFFERENT APPROACHES REFLECTED IN NATIONAL AI STRATEGIES
Citation: TY - JOUR, AU - Paunov, Caroline ,AU - Planes-Satorra, Sandra, AU - Ravelli, Greta, PY - 2019/10/17, SP - , T1 - REVIEW OF https://www.researchgate.net/figure/Timeline-of-national-adoption-of-AI-strategies_fig1_336686040
NATIONAL POLICY INITIATIVES IN SUPPORT OF DIGITAL AND AI-DRIVEN INNOVATION, DO - 10.1787/15491174-en, ER - Source: https://www.plattform-lernende-systeme.de/ai-strategies.html
GLOBAL AI STANDARDIZATION
EXAMPLE: POLICY & REGULATORY PURSUIT OF
MANAGING TRUSTWORTHY AI
IMPLEMENTATIONS WITH THE DRAFT EU AI ACT
PROPOSAL FOR EU HARMONIZED RULES ON AI
“Mandatory requirements applicable to the design and

development of certain AI systems before they are placed
on the market that will be further operationalised through
harmonised technical standards”
High Risk - horizontal mandatory requirements and

conformity assessment
Limited Risk - Transparency obligations
Minimal Risk - Voluntary codes of conduct
Risk
Management
EU AI Act: High Risk System Requirements
Accuracy,
Data
Robustness &
Governance
Cybersecurity
There are instruments to achieve

Presumption of conformity with
High Risk AI requirements (Standards,
Systems common specifications,
Requirements conformity assessments,
Human Technical
Oversight Documentation Certificates and EU declarations
of conformity)
Transparency &
Provision of
Record Keeping
Information to
Users
EU AI Act - High Risk AI Systems
Broad Definition of AI Risk Based Approach
Systems (ML + Logic (Fundamental Human
Based+ Statistical Rights, Human Mental
Models) and Physical Safety)
Policies Impacting AI System
European AI Act Elements
Compliance
Categorization
Market Intended AI
Requirements
System Use
(Unacceptable, High,
Not-High, Minimal)
Internal Assessments,
Value Chain
Conformity Assessment
Responsibilities and
(ex-ante) & Monitoring
Obligations
(Post-market)
Source: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, AI Act Extraterritorial scope

(Selling into EU market;
High-risk AI list power is
Users of located in EU;
delegated
Providers/Users where
AI output used in EU)
EU AI Act - High Risk AI Systems
Transparency &
Technical Accuracy, Robustness
Risk Management Data Governance Record Keeping Provision of Human Oversight
Documentation & Cybersecurity
Information to Users
Policies Impacting AI System
European AI Act Elements
Source: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, AI Act
DIFFERENT WAYS TO APPROACH (RISK VS. IMPACT)
Canadian Impact
EU AI Act AI Bill of Rights
Assessment Act
STANDARDIZATION
INFLUENCED THE HUMAN FOCUSED AI / TECHNICAL S LANDSCAPE
EAD, Version 1 EAD, Version 2 EAD, Edition 1 BKCenter Chart

December, 2016 December 2017 March 2019 January, 2020
2015 2016 2017 2018 2019 2020 2021
AI ETHICS P7000 SERIES ASILOMAR AI ETHICS GUIDELINES IEEE 2863 IEEE 7000-2021
INITIATIVE OCEANIS Recommended IEEE Standard Model
LAUNCHES PRINCIPLES EU High Level
IEEE IEEE Practice for Process for
IEEE Standards Working FLI Institute Experts Group
Work Begins July 2018 Organizational Addressing Ethical
Groups January 2017 April 2019 Governance of
October 2015 Inspired by EAD Concerns During
Artificial Intelligence
June 2016 Inspired by EAD System Design
AI PRINCIPLES
February 2020 LAUNCHED 2021
OECD
May 2019
IEEE 7010-2020
Recommended
Practice for Assessing
EVERYDAY ETHICS FOR AI the Impact of
IBM Autonomous and
© Copyrighted to IEEE October 2019 Intelligent Systems on
Human Well-Being
LAUNCHED 2020
• European Commission
Collaborations and Engagements
• Council on Europe
• OECD
• UNESCO
• National initiatives
• Cities
• Standardization organizations
• Industry associations
• NGOs, Consumers..
• Global Initiative
• Council for Extended Intelligence (CXI)
• Open Community for Ethics in
Autonomous and Intelligent Systems (OCEANIS)
• Digital Inclusion and Trust in Agency
• …
ETHICALLY ALIGNED DESIGN
Embargoed Version
AIS ETHICS IN ACTION SINCE 2015
ETHICALLY
ALIGNEDDESIGN
First Edition
A Vision for Prioritizing Human Well-being
with Autonomous and Intelligent Systems
• First version released in 2016. • EAD,v2 used by OECD for their AI principles.
• Latest version released March, 2019. • EAD, v2 used by IBM for “Everyday Ethics for AI.”
• Received over 500 pages of feedback. • EAD, v2 used by FLI for their AI Principles.
• Written by more than 700 global thought leaders. • EAD, v2 used by UNICEF for their Children’s Data principles.
• Over 300 pages long – not just a list of AI Principles. • EAD, v2 used by UNESCO for their AI Principles.
• Features hundreds of evergreen, pragmatic recommendations.
• EAD in all versions mentioned in more than three dozen academic journals, AI
• Inspired IEEE’s AIS Ethics Certification work. Principles and media since 2016
IEEE SA IMPACT STANDARDS
• IEEE 7000-2021™ - Standard for Model Process for Addressing • IEEE 7010-2020™ - IEEE Recommended Practice for Assessing the
Ethical Concerns During System Design Impact of Autonomous and Intelligent Systems on Human Well-being
• IEEE 7001-2021™ - Standards for Transparency of Autonomous • IEEE P7010.1™ - IEEE Recommended Practice for Environmental Social
Systems Governance (ESG) and Social Development Goal (SDG) Action
Implementation and Advancing Corporate Social Responsibility
• IEEE 7002-2022™ - Standard for Data Privacy Process
• IEEE P7011™ - Standard for the Process of Identifying and Rating the
• IEEE P7003™ - Standard for Algorithmic Bias Considerations Trustworthiness of News Sources
• IEEE P7004™ - Standard for Child and Student Data Governance • IEEE P7012™ - Standard for Machine Readable Personal Privacy Terms
• IEEE P7004.1™ - Recommended Practices for Virtual Classroom • IEEE P7014™ - Standard for Ethical considerations in Emulated
Security, Privacy and Data Governance Empathy in Autonomous and Intelligent Systems
STANDARDS
• IEEE 7005-2021™ - Standard for Transparent Employer Data • IEEE P7015™ - , Standard for Data and Artificial Intelligence (AI)
Governance Literacy, Skills, and Readiness
• IEEE 7007-2021™ - Ontological Standard for Ethically Driven • IEEE P7016™ - Standard for Ethically Aligned Design and Operation of
Robotics and Automation Systems Metaverse Systems
• IEEE P7008™ - Standard for Ethically Driven Nudging for Robotic, • IEEE 2089-2021™ - IEEE Standard for an Age Appropriate Digital
Intelligent and Autonomous Systems Services Framework Based on the 5Rights Principles for Children
• IEEE P7009™ - Standard for Fail-Safe Design of Autonomous and • IEEE P2863™-Recommended Practice for Organizational Governance
Semi-Autonomous Systems of Artificial Intelligence
Red font indicates approved standard. © Copyrighted to IEEE

https://standards.ieee.org/initiatives/artificial-intelligence-systems/standards.html
DEEP DIVE EXAMPLE: IEEE SA SOCIO-TECHNICAL
STANDARDS
VALUES ORIENTED, METRICS-DRIVEN TECHNOLOGY AT THE OUTSET OF DESIGN
Ethical Concerns & Systems Design
IEEE 7000TM offers concrete processes, with activities, tasks and

mandatory outcomes that ensure an ethically aligned design
Value-based Engineering with IEEE 7000TM Processes

IEEE 7000-2021
VALUES ELICITATION = DESIGN REQUIREMENTS
IEEE 7000-2021
OVERVIEW
IEEE 7000TM
1 What it delivers
Who is it for Development Stage
4 Ethical Requirements
Risk-based design
2 Value Based System Design Overview
5 A Use Case: IEEE 7000 & YOMA
Concept of Operations Stage

3 Context Exploration
Value Elicitation
IEEE 7000TM – WHAT IT DELIVERS
First global standard of its kind that:

• Provides activities and tasks for a human-centric and ethically
Concept Exploration Stage Development Stage
founded value elicitation (going beyond lists of principles)
• Provides the processes for engineers to translate stakeholder Concept of

Ethical Values Ethical
Operations & Ethical Risk-
ethical expectations (principles) into system requirements Context
Elicitation & Requirements
Based Design
and design (practices) Prioritization Definition
Exploration Process
Process Process
Process
• Provides a systematic and transparent approach to address
regulatory and socio-technical best practice in the design of
Transparency Management Process
autonomous intelligent systems
IEEE 7000TM – WHO IS IT FOR
IEEE 7000 is for all who participate in the design of

Autonomous Intelligent Systems (AIS)
• With a specific known context or use cases.
• Where the value of enhancements and modifications of

existing legacy systems can be improved
• Less appropriate for generic product, service and system or

indefinite context of us
IEEE 7000TM – VALUE-BASED SYSTEM DESIGN
Two Stages / Four Steps

1 2
1. Assessing the eco-system of the System of Interest (SOI) and Concept Exploration Stage Development Stage
determining its value to our well-being
1 2 3 4
Concept of
2. Defining the overarching features of the eco-system that will Context
Based Design
prevent harmful outcomes and foster the positive values Exploration
Prioritization Definition
Process
Process Process
Process
STAGE ONE - CONCEPT EXPLORATION
Systems that interact with humans always exist in and

have a context
• It’s based on the SOI use, it’s stakeholders and its ethical
benefit and harm
Concept of
Importantly the context is from the user’s perspective Context
Based Design
Exploration Process
Process Process
• This requires participation of users in the exploration process Process
STEP ONE - CONCEPT OF OPERATIONS
What will be the life-cycle of the SOI that is being

considered
• What is the organization’s intent of the SOI
• How is the intended SOI expected to operate Concept of

Context Based Design
• What is the ethical environment in which it will live Exploration
Process
Process Process
Process
Exploration is Transparency Management Process
• Future focused
• Impacted stakeholder based
• At scale impact
STEP TWO - EXPLORING ETHICAL VALUES
What are…
• The values that may influence the requirements and the
design of the SOI Concept Exploration Stage Development Stage
• The decisions made by the extended design team that will
foster positive social values and avoid negative values Concept of
Exploration Process
Value Lead Process
Process Process
• Net new role within the design team

• Responsible for the elicitation process
• Values are ranked, not traded off of one another
ETHICAL VALUES ELICITATION & PRIORITIZATION
Based on ethical theories and relevant

cultural/spiritual traditions in which a system is
launched
• How would these values be actualized within, for and by the
user’s physical and psychological existence
• Prioritization is multi-dimensional comprising a values’
impact, propagation, integrity and sustainability
ELICITING VALUES – QUESTIONS TO ASK
Three fundamental ethical theories provide the core questions the Value lead asks for the
project team to determine the values the SOI is to have designed into it
Utilitarian ethics
• “What benefits or harms would arise when the SOI is deployed?“
• How can the SOI foster positive benefits or promote negative harms
Virtue Ethics
• “What are the effects of the system on the character of the User?”
• How could the SOI or its features nurture positive characteristics or promote vices
Duty ethics
• “What are the personal principles of the User which can be undermined or fostered by the
SOI?”
• How the day-to-day personal values that could be supported or undermined
• Duty ethics is also important for prioritizing values identified
STAGE TWO - DEVELOPMENT STAGE
Ethical Value Requirements (EVR) is risk based to

protect and preserve the core values within the SOI.
• What are the requirements to promote the values elicited and Concept Exploration Stage Development Stage
to prevent the harms identified
Concept of
Ethical Risk-Based Design is to realize ethical values and Exploration
Process
Process Process
required functionality in the system or software design Process
• Scope of design includes eco-system requirements

STEP THREE - ETHICAL VALUE REQUIREMENTS
EVRs are for the eco-system wherein the SOI will

operate
• Focus is to describe risk treatment options that promote the Concept Exploration Stage Development Stage
core values identified
Concept of
• Described in socio-technical statements, use cases, scenarios, Operations &
Ethical Risk-
functional features or other forms Elicitation & Requirements
Exploration Process
Process Process
Process
STEP FOUR - ETHICAL RISK-BASED DESIGN
It is a design activity fundamental to the realization of

value-based system requirements and the relevant
risk treatment options Concept Exploration Stage Development Stage
• System design treatments are identified for value-based Concept of
system requirements and prioritized in response to identified Ethical Values Ethical
risks Context
Based Design
Exploration Process
Process Process
• Trade-off between realization of EVR and other functions and Process
performance requirements
• Includes functional as well as operational, procedural,
organizational, or structural
TRANSPARENCY MANAGEMENT
It Transparency regards communication of technical

decision and system functions to stakeholders
• Stakeholders are internal – those who have influence, and
external – those who are impacted, the users
Concept of
• Communications is to be explainable and meets need of the Context
Based Design
stakeholders receiving the communications Exploration Process
Process Process
Process
• Accountability of the SOI including roles and responsibilities
IEEE 7000TM AND SYSTEM DEVELOPMENT
Fits with any system development process
IEEE 7001TM - IEEE STANDARD FOR TRANSPARENCY OF AUTONOMOUS SYSTEMS
Measurable, testable levels of transparency, so that autonomous systems can be

objectively assessed, and levels of compliance determined, are described in this
standard.
Purpose:
The purpose of this standard is to set out measurable, testable levels of transparency
for autonomous systems. The general principle behind this standard is that it should
always be possible to understand why and how the system behaved the way it did.
Transparency is one of the eight General Principles set out in IEEE Ethically Aligned
Design [B21], stated as “The basis of a particular autonomous and intelligent system
decision should always be discoverable.”
IEEE 7002TM - IEEE STANDARD FOR DATA PRIVACY PROCESS
The requirements for a systems/software engineering process for privacy-oriented

considerations regarding products, services, and systems utilizing employee, customer,
or other external user's personal data are defined by this standard. Organizations and
projects that are developing and deploying products, systems, processes, and
applications that involve personal information are candidate users of the IEEE Std
7002™ standard. Specific procedures, diagrams, and checklists are provided for users of
the IEEE Std 7002 standard to perform conformity assessments on their specific privacy
practices. Privacy impact assessments (PIAs) are described as a tool for both identifying
where privacy controls and measures are needed and for confirming they are in place.
IEEE 7005TM - IEEE STANDARD FOR TRANSPARENT EMPLOYER DATA GOVERNANCE
Specific methodologies to help employers in accessing, collecting, storing, utilizing,

sharing, and destroying employee data are described in this standard. Specific metrics
and conformance criteria regarding these types of uses from trusted global partners
and how third parties and employers can meet them are provided in this standard.
Certification processes, success criteria, and execution procedures are not within the
scope of this standard.
IEEE 7007TM - IEEE ONTOLOGICAL STANDARD FOR ETHICALLY DRIVEN ROBOTICS AND
AUTOMATION SYSTEMS
A set of ontologies with different abstraction levels that contain concepts, definitions,
axioms, and use cases that assist in the development of ethically driven methodologies
for the design of robots and automation systems is established by this standard. It
focuses on the robotics and automation domain without considering any particular
applications and can be used in multiple ways, for instance, during the development of
robotics and automation systems as a guideline or as a reference “taxonomy” to enable
clear and precise communication among members from different communities that
include robotics and automation, ethics, and correlated areas. Users of this standard
need to have a minimal knowledge of formal logics to understand the axiomatization
expressed in Common Logic Interchange Format.
IEEE 7010-2020: Societal Impact Assessment Framework
7010-2020 - IEEE
Recommended Practice for
Assessing the Impact of
Autonomous and Intelligent
Systems on Human Well-Being
• 7010-2020 helps AI Systems creators and policymakers to ask the critical question:
What are we optimizing our AI Systems to do as Key Performance Indicators of
societal success?
IEEE 7010-2020
• 7010 provides an Impact Assessment methodology at the outset of design requiring

users to identify globally validated Indicators focusing on issues of human wellbeing
and environmental sustainability in complement to existing financial and growth
metrics which are often the only KPIs considered for AIS development and use.
• 7010 is designed to complement IEEE P7000 so Indicator identification and
measure can be utilized in any application of systems engineering.
• Global Indicators like the OECD’s Better Life Index or the UN SDGs provide data all
AIS designers can use to more holistically consider issues of harm, risk, and
innovation than when utilizing KPIs of exponential profit or growth in isolation.
• What we measure matters. For AIS to benefit all humans and society, it must take
human wellbeing and environmental sustainability into full account in design,
policy, and use to move beyond status quo ”success.”
https://ieeexplore.ieee.org/document/9084219
IEEE 2089TM - IEEE STANDARD FOR AN AGE APPROPRIATE DIGITAL SERVICES
FRAMEWORK BASED ON THE 5RIGHTS PRINCIPLES FOR CHILDREN
This standard is the first in a family of standards focused on the 5Rights principles and
establishes a set of processes for developing age appropriate digital services for
situations where users are children. The framework centers around the following key
areas as follows:
a) Recognition that the user is a child
b) Consideration for the capacity of and upholds the rights of children
c) Offers terms appropriate to children
d) Presents information in an age appropriate way
e) Offers a level of validation for service design decisions
IEEE 2089TM - IEEE STANDARD FOR AN AGE APPROPRIATE DIGITAL SERVICES
FRAMEWORK BASED ON THE 5RIGHTS PRINCIPLES FOR CHILDREN
IEEE P2863 Recommended Practice for Organizational
Governance of Artificial Intelligence aims to specify governance
criteria such as safety, transparency, accountability,
AI SYSTEMS GOVERNANCE /
responsibility and minimizing bias, and process steps for

effective implementation, performance auditing, training and
compliance in the development or use of artificial intelligence
within organizations.
PROCUREMENT
IEEE P3119 Standard for the Procurement of Artificial

Intelligence and Automated Decision Systems aims to address
the needs of government workers, policymakers, and
technologists to make meaningful and accountable choices that
are transparent about the socio-technical considerations and
impact of AI products, services, and or systems on the public.
IEEE P7015 - DATA AND AI LITERACY, SKILLS, AND READINESS
To coordinate global data and AI literacy-building efforts, this
standard establishes an operational framework and associated
capabilities for designing policy interventions, tracking their
progress, and empirically evaluating their outcomes. The standard
includes a common set of definitions, language, and understanding
of data and AI literacy, skills, and readiness.
AI Ethics Literacy
IEEE SA AIE Training

• Online training
• Blended learning
• Assessor Workshops
• Onsite training
PRINCIPLES TO PRACTICE+
A SYSTEMATIC APPROACH
Principles
RAISING THE WORLD’S
Standards Market Acknowledgement Ecosystem
RAISING THE WORLD’S STANDARDS
STANDARDS
• Training
• Assessor
• Audit
• Registry
• Strategic
Partners
• DataPort
76 © Copyrighted to IEEE
11
DEEP DIVE: AIE SYSTEM CERTIFICATION
Step 1- Information about the system (ConOps Demo)
Five Steps to IEEE AI Ethics Assessment of ECS
The system and its context explained to a panel of five

IEEE experts, including:
■ The background and goals of the project

■ The system’s architecture and interfaces
■ The machine learning component
■ The data used for model training
■ The effects of the new system on people and
processes in the organization
Step 2- Ethics Profiling & Impact Assessment
Five Steps to IEEE AI Ethics Assessment of ECS
Based on the information an ethics profiling and

impact assessment according to the IEEE AI Ethics
Certification framework was conducted.
■ For relevant ethical values such as transparency, dignity

and (avoidance of) discrimination, the expert panel
rated the likelihood & impact of the ECS to undermine
that ethical value.
■ The results of this analysis were used to determine the

most relevant of the four IEEE AI Ethics Certification
criteria sets for the application.
■ Furthermore, the overall low-impact (Baseline) class of

the system resulting from the risk/reward assessment
meant that only a subset of the accountability criteria
set needed to be addressed.
Five Steps to IEEE AI Ethics Assessment of ECS Step 3- The Selection of Pertinent Ethics Suite & Criteria
Based on the information an ethics profiling and
impact assessment according to the IEEE AI Ethics
Certification framework was conducted.
These criteria range from rather technical
aspects such as;
■ Error analysis
■ Hyperparameter tuning
■ Mitigation of false positives
to more governance-related aspects

concerning the organization, such as;
■ Adopting a layered approach
■ Avoidance of inaction
■ Delay and indifference
■ Human authority and autonomy
Five Steps to IEEE AI Ethics Assessment of ECS Step 4 - The Case for Ethics document
For each of the criteria, client provided evidence in
the form of
■ Technical documentation
■ system architecture and software implementation details
■ process and role definitions
■ organigrams, etc.
giving full detail for the respective criterion.

A Case for Ethics document was compiled, using a
structure and template provided by IEEE SA, where
client claim that they believe our system and its use
are ethically unproblematic, providing general
information about the system, its background,
scope, etc., as well as all the evidence for the 43
accountability criteria.
This Case for Ethics was then submitted for
assessment.
Five Steps to IEEE AI Ethics Assessment of ECS Step 5 – Assessment Report
Finally, an Assessment & Ethics Progressions Report
was delivered back to the client by IEEE SA. This
included
■ Specific feedback for each of the criteria from
the expert panel members, indicating to what
degree the respective criterion was considered
fulfilled
■ What could be done to further improve in the
respective areas
■ An overall confirmation that the submitted Case for
Ethics justifies recognition and certification through
the IEEE AI Ethics Certification program for client’s
ECS
Source: https://beyondstandards.ieee.org/the-ieee-certifaied-framework-for-ai-ethics-applied-to-the-city-of-vienna/
IEEE CertifAIEd - Five Steps to IEEE AI Ethics Assessment of ECS
#2.Ethics Profiling & #4. The Case for

•Expert panel reviews Impact Assessment •Technical aspects (Error analysis, Ethics document • Outlines degree of criterion
contextualized system and scope Hyperparameter tuning, Mitigation fulfillment, opportunities to
of false positives) and governance- improve and overall confirmation
•Ethics profiling and impact related aspects concerning the • Full evidence related to IEEE of commitment.
assessment according to the IEEE organization (Adopting a layered CertifAIEd criteria provided
AI Ethics Certification framework approach, Avoidance of inaction, through technical documentation,
system architecture and software
Delay and indifference, Human
authority and autonomy) implementation details, process
and role definitions,, organigrams,
etc.
#1. Info about the #3. The Selection of
Pertinent Ethics Suite #5. Assessment
system (ConOps
& Criteria Report
Demo)
EXAMPLES IN ACTION
In action – City of Vienna
Problem set/ rationale

The City of Vienna sought to advance its Digital Humanism Strategy
in a practical and demonstrative manner to reflect the impact of its
outcomes.
What they did about it
Identified a specific use case to evaluate
Engaged with the IEEE CertifAIEd team to understand the scope and
assessment requirements
Result
Real World Example
City took a milestone step toward advancing its Digital Humanism

strategy through the process.
City of Vienna’s dedicated resource has a clearer perspective on how
to practically consider AI Ethics and certification and can cross-
pollinate learned expertise with other divisions.
And where do they go from here?
Additional use cases continue to emerge to explore earning of a
similar designation.
Learn More at:

https://www.businesswire.com/news/home/20211115005200/en/City-of-Vienna-Earns-IEEE-AI-Ethics-Certification-Mark-Reinforcing-Commitment-to-Digital-Humanism-Strategy
https://www.ots.at/presseaussendung/OTS_20211115_OTS0071/foto-hankeweineltkarachalios-wien-machts-vor-kuenstliche-intelligenz-in-der-verwaltung-ethisch-zertifiziert
IN THE NEWS
Problem set/ rationale
 AI Ethics requires all parts of the organization to have a base level of AI
literacy and alignment. From procurement to
 Second, senior leaders often don’t have the requisite knowledge for
spotting ethical flaws in their organization’s AI, putting the company at
risk, both reputationally and legally
 Third, an AI ethical risk program requires knowledgeable data scientists
and engineers
What they did about it
Real World Example
 Formulated a company AI Governance statement aimed at the

development, provision, and use of trustworthy AI
• Group to train more people in AI literacy, strengthen the development of a
framework for maintaining the quality of AI systems
• “Established the AI Center of Excellence Project Team, which is charged
with leading group-wide promotion of the governance required for AI's “While AI adds to convenience, problems from its misuse or
implementation in society” unintended operation point to the need for ethical
• “ developing frameworks to maintain the quality of its AI systems, guidelines and governance covering its utilization. In Japan,
including AI Quality Assurance Guidelines and quality cards that visualize
quality assurance from the customer perspective” the government has formulated multiple guidelines*, and
legislatures overseas are increasingly taking action,
Learn More at:
https://finance.yahoo.com/news/toshiba-group-announces-ai-governance-020000133.html?guccounter=1
exemplified by the EU's AI Act. Addressing AI governance is
now a must for companies that use AI.”
GENDER RESPONSIVE STANDARDIZATION
For reflection:
● What barrier(s) (they can be digital, education, economic, work

opportunities, support, etc) did you face (and overcome) in your career
that could have prevented you getting to where they are today?
● What was key in enabling you to overcome those barriers?
● Have you ever knowingly experienced discrimination by algorithm?
● Have you ever knowingly benefited from interventions made to address
social barriers or discrimination?
Governments are not immune from Algorithmic Bias
Amplification of Miscarriages of Justice
● Key items from the paper
1. Establish a whole society view and mapping of the broader goals we seek to
achieve;
2. Generate an understanding of AI Ethics Principles and how to position

gender equality within them;
3. Reflect on possible approaches for operationalizing AI and Gender Equality

Principles;
4. Identify and develop a funded multi-stakeholder action plan and coalition as

a critical next step.
● Key reflections and cross-learnings for AI from the paper
1. Highlights the importance of a holistic understanding to policy making.
2. It is not sufficient to have “gender blind” policies - Policies are incorrectly
considered neutral by policy makers where they ignore the differential impacts
they have on different genders and socioeconomic and cultural groups. Based on
such false assumptions, the policies are less effective and/or have unintended
effects. Gender awareness and sensitive not homogenous and equal
approaches. Herein lies the difference between equality and equity.
3. Recognises the important role that energy, energy infrastructure, connectivity
and digital infrastructure as well as access to the internet play with regard to any
government being able to fulfil on its National AI strategies.
4. The availability and use of energy and domestic appliances in the home tends to
impact women more than men, and can be a barrier to paid for work, education
and training outside the home.
5. Energy, energy infrastructure, digital infrastructure, connectivity and digital
access through devices and broadband availability presents a dependency
potentially on private sector to act where these are not nationalised industry.
According to a report commissioned by the European Parliament’s
Committee on Women’s Rights and Gender Equality:
“Energy policies throughout the EU Member States appear to be gender

blind and implementation appears not to adopt gender approaches which is
exacerbated by the lack of gender-disaggregated data”
Ref: Women, Gender Equality and the Energy Transition in the EU (europa.eu)
Dated May 2019
The DIITA report shows that energy policies which neglect gender issues and
also fail to take into account other social characteristics, such as age, marital
status, and ethnicity, which also influence energy access, and can
disproportionately affect one gender over another.
● Key reflections from the paper
● Gender stereotypes can be embedded in avatars
● AR/XR presents new challenges to privacy
● Information shared by users can be used by providers to infer information about
gender.
● The availability of the sensed user data, coupled with real-time local and edge
computing processing and sensor fusion, introduces and amplifies not only risks
to anonymity and privacy, but also identity.
● Behaviour, including hand - eye tracked activity and, internal mental states are
predicted or inferred or linked by AI.
● Inferences and insight from AI are being used to classify and profile gender or
sexual orientation or sexual preferences!
● Such inferences based on unverified or inaccurate data can falsely identify a
woman as a man (false positive) or incorrectly not identify a person as a woman
at all (false negative).
● Key Takeaways for Gender Based AI Ethics Policy Making
● Awareness - increase awareness of the potential for AI to discriminate and cause
unfair outcomes, and understanding about how standards and an outcomes
based approach can help identify these AI ethics risks.
● Intelligence - Upskill your government departments, legislators, regulators and
businesses to AI ethics issues, to apply an outcomes-based approach to policy
making and by engaging new and emerging AI applications in ways which include
multiple stakeholders such as in a Regulatory Sandbox
● Expertise
a. Consider becoming the first South East Asian nation to be IEEE CertifAIEd
b. Mandate the use of AI Ethics standards in government (procurement)
contracts
c. Promote AI Ethics standards use amongst businesses to increase
international competitiveness
d. Mandate compliance with certain ethics guidelines, international AI ethics
guidelines © Copyrighted to IEEE
● Remember: IEEE are here to help you on your AI Ethics journey
ADDITIONAL RESOURCES TO SUPPORT YOU ON
YOUR JOURNEY
IEEE GET AI Ethics & Governance Standards Program
Purpose: List of Standards*:
IEEE SA, together with some strategic partners makes AIE & 2089-20212089-2021 - IEEE Standard for an Age
Governance standards available at no cost to help raise awareness Appropriate Digital Services Framework Based on the
5Rights Principles for Children
and literacy around AI ethics and help AI developers incorporate
7000-20217000-2021 - IEEE Standard Model Process for
human-centric design principles into their product roadmaps and Addressing Ethical Concerns during System Design
organizational and governance structures, supporting more trustworthy 7001-20217001-2021 - IEEE Standard for Transparency of
AI systems. Autonomous Systems
7002-20227002-2022 - IEEE Standard for Data Privacy
Terms of Use: Process
7005-20217005-2021 - IEEE Standard for Transparent
-Access, download, print and/or retain one (1) copy of each Document
Employer Data Governance
in this Program 7007-20217007-2021 - IEEE Ontological Standard for
for individual use only, including job-related functions. Ethically Driven Robotics and Automation Systems
-Not to further copy, prepare, and/or distribute copies of the Document, 7010-20207010-2020 - IEEE Recommended Practice for
nor significant portions of the Document, in any form, without prior Assessing the Impact of Autonomous and Intelligent
written permission from IEEE. Systems on Human Well-Being
-Contact us at AIE-GET@ieee.org for further information
*More to be added as they are approved.
AIE READINESS FRAMEWORK
AI LITERACY THROUGH THE IEEE AIE FRAMEWORK
Awareness Intelligence Expertise
A
•
•
•
Introduction to AI Ethics
AI Standards: Roadmap for Ethical and
Responsible Digital Environments
Artificial Intelligence (AI) Assurance Artificial
Intelligence and Ethics in Design: Responsible
•
•
•
•
•
•
I
Certification
Transparency
Accountability
Algorithmic Bias
Privacy
Enterprise Governance
•
•
•
•
E+
AI Ethics CertifAIEd Assessor course
CertifAIEd examination
CertifAIEd formal review
CertifAIEd designation
• Custom Workshops &
Training to meet the
needs of your
organization
Innovation • Outcomes & Risk-based Certification

• Responsible Innovation in the Age of Artificial • AI Standards: Organizational Transparency
Intelligence • AI Standards: Roadmap for Ethical and Responsible
• Artificial Intelligence and Ethics in Design Digital Environments
• Human Emotion in Devices and Technology • AI Standards: Best Practices for Ethical Systems
• The Basis for No Bias • AI Standards: Configuring Systems for Privacy
• The Economic Advantage of Ethical Design For • AI Standards: System Design Considerations for
Business Data Privacy
• Values by Design in the Algorithmic Era
TRUSTWORTHY AI PORTFOLIO
• Available for public access and
improved understanding
AIS ETHICS IN ACTION SINCE 2015
• Help organizations differentiate and

build competencies and
methodologies to achieve the
envisioned trustworthy organization.
Ethically Trusted Data Ethically Aligned Use Case for
Aligned Design and AIS for Design for Ethical Evaluation
– Version II Financial Business and Verification of
Services - IEEE Contact Tracing
Finance Apps (and
Playbook – V1 Technology
(CTA/CTT)
Extended Reality Ontological Specifications for Ethical Algorithmic Bias, Privacy,

Seven published technical (XR) Ethics Reports Transparency, Accountability
Standards in the GET
Program for AI Ethics and
Governance Standards
IEEE CertifAIEd
• Affirms an organization’s commitment to upholding human values, dignity,

and well-being, and to respecting, protecting and preserving fundamental
human rights.
• Conveys an organization’s capability to fulfill applicable transparency,

accountability, reduction of algorithmic bias and privacy requirements
stipulated in the appropriate criteria to foster trust and facilitate the adoption
and use of AI products and services.
https://engagestandards.ieee.org/ieee • Enhances confidence in public and private entities that wish to realize the
benefits of AI ethics certification in the absence of or as a complement to
certifaied.html broadly accepted and enforced regulations for AI, while mitigating risks,
liabilities and adverse impacts on their reputation and market share.
• The demand for trustworthy systems will continue to grow for the foreseeable
future. Addressing Trustworthy AI system implementations continues to be of
interest for all market stakeholders
• There are a number of entities interested in orienting themselves to effectively

shift from Principles to Practice, but a clear pathway is not evident for
everyone at this time – and a fundamental AI Literacy gap must be
addressed.
Key Takeaways
• The IEEE SA’s adaptative and customizable approach to support your digital
transformation journey with its standards, training and CertifAIEd adaptive risk
certification methodology offers a path forward
Thank You
twitter.com/ieeesa
445 Hoes Lane, Piscataway, NJ 08854 USA
+1 732 981 0060
certifaied@ieee.org or AIE-GET@ieee.org

Module 4-6-Training For AI Ethics - Bangkok & Indonesia Feb Mar 2023 - Final

Uploaded by

Copyright:

Available Formats

You might also like

Module 4-6-Training For AI Ethics - Bangkok & Indonesia Feb Mar 2023 - Final

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Module 4-6-Training For AI Ethics - Bangkok & Indonesia Feb Mar 2023 - Final

Uploaded by

Copyright:

Available Formats

TERMS OF USE

This is a copyrighted IEEE Document.

IEEE-SA Licensing and Contracts

From the telegraph…to autonomous vehicles and AI, IEEE has

Create new Improve

Allow for better

- UN/IPCC definition of Sustainability

Consequences that are a matter of livelihood, life, & death…

Algorithmic Transparency AI Accountability AI City Governance AI Org. Governance

…and existing approaches have not raised to

Today’s challenges… …create tomorrow’s opportunities

 Opportunities left on the table due to  Demonstrative commitment to trustworthy and

Risk per the survey (Explainability,

Growth in Internal Orientations efforts

• Integrating criteria as an aspect of procurement, pricing strategies, and/or investment strategies

Forging a path to demonstrate Commitment externally

• Algorithmic Bias & Fairness

Credit: Berkmann Klein Center

—Kathy Baxter, Salesforce

—Sam Coates, LEGO, Creative Play Lab

Source: EAD for Business

Several policy developments with broader

US Algorithmic Accountability Act

…building upon a number of principles that

“Mandatory requirements applicable to the design and

High Risk - horizontal mandatory requirements and

There are instruments to achieve

Source: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, AI Act Extraterritorial scope

Source: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, AI Act

EAD, Version 1 EAD, Version 2 EAD, Edition 1 BKCenter Chart

2015 2016 2017 2018 2019 2020 2021

Red font indicates approved standard. © Copyrighted to IEEE

IEEE 7000TM offers concrete processes, with activities, tasks and

Value-based Engineering with IEEE 7000TM Processes

2 Value Based System Design Overview

5 A Use Case: IEEE 7000 & YOMA

Concept of Operations Stage

First global standard of its kind that:

• Provides the processes for engineers to translate stakeholder Concept of

IEEE 7000 is for all who participate in the design of

• Where the value of enhancements and modifications of

• Less appropriate for generic product, service and system or

Two Stages / Four Steps

Transparency Management Process

Systems that interact with humans always exist in and

Transparency Management Process

What will be the life-cycle of the SOI that is being

• How is the intended SOI expected to operate Concept of

Exploration is Transparency Management Process

• Impacted stakeholder based

• Net new role within the design team

Based on ethical theories and relevant

Ethical Value Requirements (EVR) is risk based to

• Scope of design includes eco-system requirements

EVRs are for the eco-system wherein the SOI will

Transparency Management Process

It is a design activity fundamental to the realization of

It Transparency regards communication of technical